Disclaimer: I understand the advantages of moving to ESM and support that people do so but, I'm not a fan of just moving everything to new tech while breaking tech that was already working. Users are also at blame for not checking release notes but it's on both parties but finding solutions is a part of our work so, this.
tldr;There's huge number of bundlers, each with their own implementation of the esm spec and so we either need to support each spec or at least find common ground to support most bundler setups. (though most bundlers are now on par with the official spec, this post will go through things you can do to reduce friction in older setups)
Creating ESM and CJS packages that work in most environments. I wasn't even aware of this being an issue until I started writing my own packages.
The problem is based on how bundlers handle these files and different ES syntax that are available today.
Most user(developer) setups involve some form of configuration to decide what ES syntax they can write in their code.
Example.
const array = [1, 2, 3]
const shallowCloned = [...array]
// this might not work for you if the ES syntax your transpiler supports doesn't
// have spread syntax support.
So, as a package maintainer, we have to be sure of what version of ES syntax we plan to support and compile/transpile our code so the user setups can handle them.
I wish it ended at that, but it doesn't. As we have to keep moving forward with
the standards so a new standard emerged a while back which was already being
used by browsers for a while and its called EcmaScript Modules (ESM). ESM is
basically a way of treating modules as asynchronous sources of javascript code /
behaviour.
Which makes it possible to use cached modules from a remote source (at least in browsers). The advantages towards having it in node environments would add a more unified language standard and reduce the need for bundlers as the esm package could be just shipped as is.
The support for this spec was added behind an experimental flag in earlier versions of Node 12.
A few package maintainers, decided to move to ESM right away and their newer
package versions(major version, no breaks for existing users, unless they do
npm install pkg@latest) would break setups that weren't respecting the ESM
standards.
Users(developers) would just go npm install <package-name> and that installs
the latest version and half of them never read docs so they had no idea what was
going on.
Oh, you should've seen the amount of cannot use esm in common js issues that
were raised by devs during this phase.
Now, bundlers that added support for esm as patches handled the issue pretty well. But bundlers that were undergoing changes in arch and API took a little longer to get it working and most people who reported these compat issues were on these bundlers.
"Where's the problem? You're just ranting Reaper". More like giving context, but this is where the problem is, the diverse nature of setups in the javascript world is what's responsible for the existence on this problem. How we mitigate it, is up next.
"Workarounds", cause these are not concrete solutions.
If you wish to support both sides of the party, CJS and ESM. The points mentioned here might help you both as a maintainer and as a library user but there's certain behaviours that I didn't spent much time researching on. Whatever mentioned here is based on my personal work with these kind of packages and in some cases browsing the codebases of bundlers where an issue for this was raised.
For the maintainer Name the files as .mjs and ship the package. This will
trigger errors on the user(developer)'s bundler and they can add support for
.mjs accordingly. You can also keep the extension as .js and instead change
the type field in package.json to be "module"
{
// @filename: package.json
"type": "module"
}
For the user Most of your bundlers come with configuration to handle such cases. What you are looking for is a way to add support for custom extensions and transpile them as normal JS/ES syntax and transpile as needed.
NOTE: If the library is using very specific ESM syntax like
import x from 'node:fs' , you might need to see if your bundler supports
handling protocol based imports. If not, well, talk to the maintainer, and
figure out if they wish to help you with it or if they even have that plan in
the scope, if not, you might wanna look for an alternative replacement for the
library or look for an older version of the lib.
This is where I'll be standing till the ecosystem has stabilised (at least for me), which is a decision I've taken mainly due to the bundlers and setups used by major frameworks. In my case this would be (React, React Native) and a lot of other system level CLI libraries, cause I work with these the most.
Like everyone else rooting for ESM, I'm also waiting for the point in time where I won't have to use a transpiler anymore, sounds like an amazing place to be, but I'm not the only developers and not everyone is on the same page and people want CJS to stay for longer so we're going to work for both sides.
Before actually writing your package, you need to decide where is library going to be used?
Write it like you already did, babel will take care of it. Don't have to deal with ESM and CJS for now.
Write it in ESM, it already works in all major browsers but, maybe add an IIFE/UMD version just in case. It's not that hard to generate these from existing code.
Well, this is where the fun is , isn't it?
Let's see the number of bundlers we have that should be able to work with our package.
There's a few more actually, but 8 sounds like a nice number to stop at,
daunting enough already. You can add Typescript's tsc to the list, as you can
kinda use it to generate a single file.
Cool, getting to the fun part.
Starting with package.json
The entry point of your package decides what most bundlers see and this is what they look for, when trying to figure out what should be allowed to import and what should be ignored.
This segregation helps with handling private dependencies or internal code that you don't want exposed.
This can also be done by having a single index.js file exporting modules that
should be exposed, which is the easier way out.
But, when you work with packages that might need multiple entries, you'll have to configure a few things.
A good example of this is jotai and zustand which have imports like the following
import {} from 'jotai'
import {} from 'jotai/utils'
import create from 'zustand'
import {} from 'zustand/middleware'
This gives the user a clean import and makes it obvious as to what's being used and from where.
How do we achieve this?
This is how the package.json for something like this would look like if all
you were writing for was ESM.
// @filename: package.json
{
"exports": {
".": "index.js",
"middleware": "middleware.js"
}
}
We aren't working with just ESM so let's compile a few CJS versions using whatever bundler and adding them in the entry points as well.
// @filename: package.json
{
"exports": {
".": {
"import": "index.js",
"require": "index.cjs"
},
"middleware": {
"import": "middleware.js",
"require": "middleware.cjs"
}
}
}
This is what Node's spec specifies for conditional imports. We basically are asking the bundlers to make sure that they import the right file when working with our package.
Just doing this should solve the issue in your user's code editor because they all use Typescript's LSP engine and this satisfies the conditions for that to work.
Later versions of Node 12 need path specific exports, so we'll have to change the exports a bit.
// @filename: package.json
{
"exports": {
".": {
"import": "./index.js",
"require": "./index.cjs"
},
"middleware": {
"import": "./middleware.js",
"require": "./middleware.cjs"
}
}
}
The change being, adding path specfiers for the files references. This should still work with the LSP engine but the bundlers need a little more configuration so let's fix that
Webpack 4 uses the module field to find the esm files so add module to
exports
// @filename: package.json
{
"exports": {
".": {
"import": "./index.js",
"module": "./index.js",
"require": "./index.cjs"
},
"middleware": {
"import": "./middleware.js",
"module": "./middleware.js",
"require": "./middleware.cjs"
}
}
}
If working with single entry file, you add the module field to the top level of
your package.json
// @filename: package.json
{
"name": "pkg",
"module": "./index.js",
"main": "./index.cjs"
}
In certain cases you might have to tell babel-loader to consider .mjs as a
.js file. You can find this online on "how to configure webpack 4 for .mjs
files"
I work with react native a lot and most of my packages start as a utility for one of my work related apps and then made as a generic package for all platforms.
If you can keep the library limited to 1 entry file, then you don't have to write the exports section at all. You can do something like this and this should work in both webpack and metro, no issues.
// @filename: package.json
{
"main": "./dist/index.js",
"module": "./dist/index.mjs",
"types": "./dist/index.d.ts"
}
Multi entry packages, don't worry. I'm still here.
You will setup everything according to Webpack 4 support and add 1 additional
field and one extra export. Also, if you used .cjs like me for commonjs files,
you'll have to let the user know that they need to tell metro to consider .cjs
as a valid format
// @filename: package.json
{
"exports": {
// as stupid as it looks, it's needed for metro or it'll complain that it can't import cause it wasn't exported
"./package.json": "./package.json",
".": {
"import": "./index.js",
"module": "./index.js",
"require": "./index.cjs",
"default": "./index.cjs",
// you can also add types if you wish to, bundlers might not, but the ts engine does so it should work in most cases.
"types": "./index.d.ts"
},
"middleware": {
"import": "./middleware.js",
"module": "./middleware.js",
"require": "./middleware.cjs",
"default": "./middleware.cjs"
}
}
}
Metro configuration if you used .cjs, same goes if you used .mjs for esm
// @filename: metro.config.js
module.exports = {
resolver: {
sourceExts: ['.mjs', '.cjs', '.js'], // <= the user will have to add this
},
}
That's all the information you need to support most of them since esbuild,
rollup, parcel, and sandpack handle the generic exports spec pretty well, so
just releasing mjs and cjs files for them just works out of the box
But there's always people who have a setup on something like node10 and if I
was maintaining something like Jotai, i wouldn't want to leave them hanging so
there's other steps that were taken in libs like these and you can read about
that on
How Jotai handles package entries.
These points are just to mitigate the issues when you are using or writing packages and I wish there were better ways to do things but this is the closest you can get to it right now.
I'd very much like for the majority of the ecosystem to get compatible with esm without having to configure things in each setup like I do right now.
I think, the newer versions of metro should already start taking in the .mjs
and .cjs as normal.
Also, if you are reading this and are still using webpack 4, please, upgrade to webpack 5, please!