4 Ways to Minimize your Dependencies in Node.js

We all know the joke about how node_modules is the heaviest object in the universe.

For example, a project that uses only fastify, knex, typescript, and uuid generates an 83MB node_modules folder! That’s huge! And those four packages are far from a complete set for a relatively small back-end. A more realistic size for node_modules is north of 100MB, in some cases reaching 1GB.

In this post, we’ll explore four methods to minimize your code’s dependencies, resulting in faster CI/CD execution and safer code.

But first, let’s touch on some of the problems with heavy node_modules.

The Issues with Heavy node_modules in Node.js

Heavy node_modules can cause slower CI/CD pipelines, as dependencies are usually installed during those pipelines, and require network calls to a registry (whether npm or your mirror of it). This affects your development experience.

Moreover, package creep can introduce serious issues like security vulnerabilities, as you don’t own the code that resides inside the packages.

Let’s jump into how to minimize the dependencies in your Node.js code.

Method One: Check the Age of Node.js Packages

Imagine a situation where you have a function that looks like this:

function findSomethingByIds(ids: number | number[]) {
  ...
}

This function’s purpose is to either find a single entity or a list of entities by their id. It’s common practice with different repository patterns.

However, let’s say you want to know whether you’ve requested a single entity or a list of entities. A quick npm search leads us to npm/isarray, an insanely popular package with over 62 million weekly downloads! You know the drill now:

npm i isarray

Don’t forget:

npm i -D @types/isarray

But wait! Before you do that, have you noticed that this package is three years old? And if you do a better search, you will discover that isArray is now a part of the JavaScript core, can be invoked using Array.isArray, and is perfectly supported in Node.js version 10 and up.

Read more about Array.isArray in Mozilla MDN.

Package age is a good first indicator. Many old (2+ years) packages might have security vulnerabilities or be outdated. Some of the outdated packages are also merged into the official JavaScript spec.

Another example is the trim package. It has more than 4 million downloads, and while it’s not that old (only 1), a native solution exists in JavaScript core: String.prototype.trim().

So always try to find a native solution first, as JavaScript evolves fast and its standard library is always expanding. Don’t be fooled by the weekly downloads counter in npm.

Method Two: Using ‘One-liner’ Node.js Packages

Many ‘simple’ packages are actually what I call ‘one-liners’. A one-liner is a package that contains very few lines of code.

If we continue with our isArray example from above, by examining the content of index.js we can see that it’s a simple one-line function:

var toString = {}.toString;

module.exports = Array.isArray || function (arr) {
  return toString.call(arr) === '[object Array]';
};

Source: GitHub

I always recommend you look at the source code of packages, at least simple ones because they can teach you a lot. Looking at the above source code, we discover one important thing: that Array.isArray exists (the same conclusion we came to with method one).

However, even if we did not have a native is-array method, the entire function is a simple if statement. And instead of requiring this package as our dependency, we can simply write the code as part of our project. It’s worth stopping for a minute and discussing the pros and cons of such an approach.

Using Npm Vs. Writing Your Own Code

Npm is a public registry, and the code contributed to npm comes from people worldwide. It is amazing that we have such a big repository of free and open-source code. However, this also comes with some disadvantages.

Npm, unfortunately, is known to be at high risk of attack. Packages might get compromised, whether by third parties or by developers themselves.

We all remember the scandal around left-pad and the recent rise in crypto-mining and crypto-stealing code that resides inside popular npm packages.

While it’s impossible to audit every single package we install, we can lower the attack vector by using fewer packages in our projects. One great way to reduce the amount of packages we depend on is to write trivial packages ourselves.

On the other hand, by writing some code ourselves, we deprive ourselves of the community’s sheer knowledge. Multiple people maintain popular packages, so they can quickly react to new vulnerabilities (for example, by monitoring the GitHub issues page of their package). This is something you, as a sole developer or part of a small organization, might lack the resources to do.

So the next time you are eager to install a package, check its source code. It might be a one-liner you can write instead of introducing a potential attack risk and slowing your CI/CD pipeline.

But don’t go too far with this method: you don’t want to reinvent the wheel or entirely deprive yourself of the community’s support.

Method Three: Extracting Sub-packages with lodash

Imagine we have the following object interface:

interface SomeObject {
  foo?: {
    bar?: {
      baz?: string;
    }
  }
}

And we have a function that accepts an object with that interface.

function getBazOrDefault(obj: SomeObject, defaultValue: string)

As you’ve guessed from the function’s name, it will give us the value of baz, or defaultValue, if the path to baz is undefined. Here is one implementation of that function:

function getBazOrDefault(obj: SomeObject, defaultValue: string) {
  if(!obj.foo || !obj.foo.bar || !obj.foo.bar.baz) return defaultValue;
  return obj.foo.bar.baz;
}

Ugly, right? It will get more ugly if you need to deal with arrays. Luckily, we can use the popular lodash library!

Install lodash, and the code becomes nice and easy to read:

import _ from "lodash";

function getBazOrDefault(obj: SomeObject, defaultValue: string) {
  return _.get(obj, "foo.bar.baz", defaultValue);
}

Neat!

However, if we install lodash:

npm i lodash

And its type definitions (because we use TypeScript):

npm i -D @types/lodash

We will introduce 8.4MB of dependencies to our node_modules. That’s a lot!

❯ du -sh node_modules
8.4M    node_modules

We have some experience now, so let’s put it to practice!

• Package age — lodash is relatively old - it was published a year ago. I couldn’t find any JavaScript core functionality to get a nested value from an object by a string key. Let’s move on.

• One-liner package — lodash is not a one-liner. It has tens of files and lots of tests. Even looking at the functionality of _.get, it’s not exactly a one-liner. While it’s a simple two-line function, it has an internal dependency on ./internals/baseGet.js, which in turn depends on ./castPath.js and ./toKey.js - and each depends on more files! It’s too much to be a candidate for extracting standalone code.

So it seems we are stuck with lodash then. But wait! There is another trick I want to show you! Sub-package extraction. If we read the lodash readme carefully, we will notice that every single functionality lodash provides is extracted into its own sub-package, including the _.get function!

Instead of installing the entire lodash, we can install lodash.get for just the _.get function (of course, don’t forget the type definitions). This reduces the node_modules size from 8.4MB to 3.6MB. It’s still a large folder, but a 57% reduction from the original size! I’ll happily take such a percentage reduction.

❯ npm i lodash.get
❯ npm i -D @types/lodash.get
❯ du -sh node_modules
3.6M    node_modules

The getBazOrDefault code will look like this:

import get from "lodash.get";

function getBazOrDefault(obj: SomeObject, defaultValue: string) {
  return get(obj, "foo.bar.baz", defaultValue);
}

In conclusion, be aware that many packages, especially collections of utilities like lodash, can be published to npm as individual packages. Instead of installing the entire 312 different methods from lodash, we can install the ones we actually need, reducing the weight of our node_modules.

Method Four: Do It Yourself

Last but not least is the DIY method.

Let’s say we need a method to capitalize the first letter of each word in a given string. We can run an npm search for the term capitalize and be presented with 266 different packages (not all relevant).

We can spend some time looking for a relevant, relatively maintained package and add it as a dependency to our project. Or we can write it ourselves!

Why, you ask? Software engineering is the art of problem-solving. To become a great software engineer, you need to be able to solve software engineering problems.

If you only know how to use third-party packages and copy code from Stack Overflow, you will be a software engineer, but not a great one. And you do want to be great, don’t you? Then let’s write our own capitalize function!

Let’s start with a signature:

function capitalize(str: string)

First, let’s identify words from our string. We can do that by splitting the string with whitespace:

const words = str.split(" ");

Then we need to capitalize the first letter for each word:

for(let i = 0; i < words.length; ++i) {
  words[i] = words[i].charAt(0).toUpperCase() + words[i].substring(1);
}

And lastly, we need to join the words back to the string:

const capitalizedString = words.join(" ");

Here’s the entire function:

function capitalize(str: string) {
  const words = str.split(" ");
  for(let i = 0; i < words.length; ++i) {
  	words[i] = words[i].charAt(0).toUpperCase() + words[i].substring(1);
	}
  return words.join(" ");
}

If you run this with some example strings:

console.log(capitalize("hello world")) // Hello World
console.log(capitalize("how are you?")) // How Are You?

Great success!

Note - this function is for demonstration purposes only. It does not consider scenarios like quoted words - for example, hello "world" will be capitalized incorrectly. However, this is a great opportunity to up your problem-solving skills! Go on and figure out how to capitalize quoted words as well!

As with the second method, you’ll want to avoid ‘reinventing the wheel’ here. On the one hand, you don’t want to write the same functionality over and over. On the other hand, you don’t want to introduce a dependency for every small functionality you need.

Always analyze and evaluate each solution. Many packages do way more than you need, so it’s better to implement the code yourself. However, there are good quality packages for common utils like slugify, which is well-maintained and provides basic functionality you probably wouldn’t want to implement yourself in a production environment.

Wrap Up: Decrease your Dependencies in Node.js

In this post, we ran through four methods to analyze packages and shrink your node_modules:

• Checking the age of packages
• Using on-liner packages
• Extracting sub-packages
• Doing it yourself

These methods should make your development process faster, reducing the amount of packages you need to pull every time you run your CI/CD pipeline.

All in all, the npm ecosystem is great! It is the biggest package registry as of today. But installing packages shouldn’t be a panacea. Our project is not only the code we write — it’s also the packages it consists of. Knowing what goes into our node_modules makes our projects more resilient and makes us better developers.

Until next time, happy coding!

This post was initially published in AppSignal Blog