Creating an NPM-Only Build Step for JavaScript — the Easy Way

Nowadays it is impossible to write frontend code without a build tool. We work with TypeScript and Sass to bundle and optimize our code. Gulp was my choice for the last three years. But lately I have been growing frustrated with the overhead it brings. Doing what you want can be very hard, especially if what you need does not exist in a ready-made recipe. And if you try to write your own plugins, streams and buffers can scare you off.

For my open source JavaScript packages I always used node modules to build the code. This made me wonder, could I do the same for my website? It turns out, you can, and others had this idea as well.

Advantages of NPM as a Build Tool

The biggest advantage for me is that you have one less dependency chain. You don’t need to download and learn a build tool. With npm, which you already use, you use simple CLI commands or Node.js scripts.

Using modules in the CLI instead of via the build tool wrapper removes a layer of complications. You don’t need to wait for somebody to update or fix the wrapper. If the module can do it, so can you. Compared to the main module, a wrapper for a specific build tool is more likely to be unmaintained.

Most scripts start out as Node.js scripts and afterwards might be adapted for your build tool. Using Node.js assures you the best choice of tools and quickest update and fix rates.

The Downsides of NPM

npm scripts are not quite as plug-and-play as Gulp or Grunt. Tutorials for the typical use cases make the entrance into a build tool easy. Only when you have more advanced requirements will it become a hassle.

As of now there are far fewer tutorials and recipes on npm build steps compared to other build tools.

Some people are saying npm scripts are slower because Gulp uses streams. However, as Cory House suggested, the command line always had streaming.

• The pipe (|) streams one commands output to the next commands input
• The redirection (>) operator redirects the output to a file

And to be honest, on any normal projects, a couple of milliseconds won’t matter all that much.

Learn More About Node.js on Built In's Expert Contributors NetworkNode.js App Security: Let No One Through the (Digital) Gates

Let’s Build an NPM-only Script

All our work will be within package.json. We will accomplish the following:

1. Run a node server that watches for file changes
2. Compile Sass to CSS and revision the files
3. Compile TypeScript to JavaScript and revision files
4. Move some node_modules into the public folder
5. Watch for changes in *.scss and *.ts files and recompile

Running a Node Server

I am currently using supervisor for development because it is dead simple. You could easily replace it with ts-node, forever, nodemon or any other server.

With the package installed we can add the script to our package.json. The -w flag defines the directories app,resources/templates that should be watches. In my case, any changes to my Node server files or my handlebars templates will trigger a server restart. The --extensions flag allows you to specify which file types should be watched. The last argument is the main app file that should be run by the server.

// package.json
"scripts": {
	"supervisor": "supervisor -w app,resources/templates --extensions node,js,hbs app.js"
}

Building Our CSS

We want to compile Sass into CSS, remove old compile files, and revision the new files. If we change our Sass files we want them to be recompiled. We will split the steps into individual scripts and compose them together. This makes it easier to read and think about.

Compiling Sass Files

All we need to install is the node-sass package. In my case, the app.scss file that loads all other files is in resources/css/. The converted app.css file is saved to public/css/ as well as the source map. Check out the github project page to understand all flags and options.

// package.json
"scripts": {
	"supervisor": "supervisor -w app,resources/templates --extensions node,js,hbs app.js"
}

Removing Old Files

While it is possible to check which files changed and only update those, I opted for the simple option. With the speed at which Sass compiles, there is no reason to invest any time in micro optimizations. This means we can remove all CSS files with a simple rm .

// package.json"scripts": {
  …,
  "css:clean": "rm -f public/css/*"

Revisioning Files

I actually did not find any module that does revisioning the way I wanted it. Luckily with Node.js it was easy to write the Node-file-rev module myself. After installing it, you provide the file(s) to the script and it creates the revisioned file as well as the manifest. You can specify the manifest directory and name with the --manifest flag. The --root flag allows you to specify the root directory to remove in the manifest (e.g. public/css/app.css → css/app.css).

// package.json

"scripts": {

…,

 "css:rev": "node-file-rev public/css/app.css --manifest=public/rev-manifest.json --root=public/"

Learn More JavaScript on Built In’s Expert Contributors NetworkJavaScript Call Stacks: An Introduction

Composing CSS Scripts Together

A new build:css script combines the clean, compile and rev script using the && operator. With a && the next command will only execute if the first command exits with 0, meaning it was successful.

The build:css:watch script executes build:css after every update of a file in the specified folder. We are installing onchange for this. The -i flag makes onchange run the script once when it is started (without any change).

When adding the two scripts you will have the following five scripts for your CSS workflow.

// package.json

"scripts": {

 …,

     "css:clean": "rm -f public/css/*",

     "css:compile": "node-sass --source-map public/css/app.css.map --output-style compressed -o public/css/ resources/css/app.scss",

 "css:rev": "node-file-rev public/css/app.css --manifest=public/rev-manifest.json --root=public/",

 "build:css": "npm run css:clean && npm run css:compile && npm run css:rev",

 "build:css:watch": "onchange -i 'resources/css/*.scss' 'resources/css/*/*.scss' -- npm run build:css"

Building Our JavaScript

To compile our TypeScript files we need to replicate the same logic as we had for our Sass. We also want to move some files from node_modules into public/js.

Compiling TypeScript files

I am using rollup, but you could use webpack or any other tool you need to convert your files. There is definitely a CLI version available. Just add another script to the package.json named js:compile and run the needed CLI call, for rollup it is rollup --config.

The --config flag tells rollup to use the rollup.config.js file so you don’t have to specify everything in the CLI. Explaining my config would go too far for this article. In short I’m using rollup-plugin-typescript and rollup-plugin-uglify-es to convert and uglify my TypeScript.

Read More About Software Development on Built In’s Expert Contributors NetworkFront-end vs. Back-end Development: Which Should You Prioritize?

Removing Old Files and Moving Files

We use the same rm script to remove old JavaScript files.

The files we want to move are stored in a variable in a config section of the package.json called moveFilesJs. You have to use a space-separated list because arrays are not supported. In your script you can reference the files by using $npm_package_config_moveFilesJs. With this in place we can run a simple copy cp command.

// package.json"config": {
	"moveFilesJs": "node_modules/@webcomponents/webcomponentsjs/bundles/webcomponents-sd-ce.js node_modules/@webcomponents/webcomponentsjs/bundles/webcomponents-sd-ce.js.map node_modules/fetch-inject/dist/fetch-inject.min.js"
},
"scripts": {
  …,
  "js:clean": "rm -f public/js/*",
  "js:move": "cp $npm_package_config_moveFilesJs public/js/"

Revisioning Files

For the revisioning we can use the same Node-file-rev module and replace the path. Instead of defining a single file, we define a “glob” by using public/js/*.js. This will revision all JavaScript files in the folder.

// package.json

"scripts": {

 …,

 "js:rev": "node-file-rev public/js/*.js --manifest=public/rev-manifest.json --root=public/"

Composing JS Scripts Together

Like with the CSS, a build:js script combines the clean, compile, rev and move script using the && operator.

The build:js:watch script executes build:js when a file changes.

// package.json"scripts": {
  …,
  "build:js:watch": "onchange -i 'resources/js/*.ts' 'resources/ts/*/*.ts' -- npm run build:js",
  "build:js": "npm run js:clean && npm run js:compile && npm run js:rev && npm run js:move",
  "js:clean": "rm -f public/js/*",
  "js:move": "cp $npm_package_config_moveFilesJs public/js/",
  "js:compile": "rollup --config",
  "js:rev": "node-file-rev public/js/*.js --manifest=public/rev-manifest.json --root=public/"

Composing It All Together

Finally we can combine both build scripts into a single one called build. By combining build and supervisor into the start script we can run everything with npm start.

Because both scripts keep running we install the ttab module to start each in its own terminal tab.

The -t flag allows you to specify the name for the tab.

// package.json

"scripts": {

 …,

 "build": "npm run build:js:watch & npm run build:css:watch",

 "start": "node_modules/.bin/ttab -t 'Node Server' 'npm run supervisor' & node_modules/.bin/ttab -t 'Building assets' 'npm run build'"

}

Replacing a build tool like Gulp with npm scripts is not as hard as it seems. While it can be a bit more inconvenient, it makes you less dependent on plugin authors. Scripts are less connected which lets you replace one part without touching the rest.

You can always write a Node or bash script to do what you need and execute it via npm.

In the end it comes down to preferences and values. I value ease of maintenance and fewer dependencies over the convenience of a task runner.

Read More From Lukas Oppermann on Built In’s Expert Contributors NetworkThe Gestalt Principle of Proximity for Designers, Explained