javascriptrecipeJavaScript Heap Out Of Memory Error

I've encountered this scary-looking error on multiple occasions when my application ran out of memory.

In this short article, I'll first give you a solution that you can use to fix this issue quickly.

After that, in case you want to know more about it, I'll explain what the causes of this error can be and how to prevent it from happening in the future.

How to solve the "Heap out of memory" error in JavaScript

The fastest way of solving this issue is by increasing the memory limit of Node. Starting from Node.js v8, you can set the limit in MB with the --max-old-space-size flag like this:

node --max-old-space-size=4096 index.js

4096 translates to 4 GB of memory. You can set the limit to whatever you like, but make sure you don't use all the available memory, or otherwise, your system might crash.

As an alternative to this, you can also set the flag in an environment variable like this:

NODE_OPTIONS="--max-old-space-size=4096" node index.js

Change the Node.js memory limits of your environment

If you want to change the memory limits of Node.js for your entire environment, you need to set the following variable in your environment's configuration file. (.bashrc, .bash_profile, sometimes .zshrc, etc.)

Add this line to your configuration file:

export NODE_OPTIONS=--max_old_space_size=4096

"npm install" heap out of memory

If you run into this issue when installing a package with npm or yarn, you can bypass the memory limit temporarily by installing the package as follows:

node --max-old-space-size=4096 $(which npm) install -g nextawesomelib

What does this error even mean?

By default, Node.js has memory limits that prevent the program from consuming too much memory and crashing the whole system. The results differ depending on the version and the system architecture of your system (32bit or 64bit).

The memory limits of different Node.js versions

There are no official numbers regarding those limits, but with a small program I wrote, I was able to get them for different Node.js versions on a 64bit architecture.

You can test this yourself by creating an index.js file with the following code and running it with the Node.js version you want to know the limit for.

const array = [];
while (true) {
  // This makes the array bigger on each iteration
  array.push(new Array(10000000));

  const memory = process.memoryUsage();
  console.log((memory.heapUsed / 1024 / 1024 / 1024).toFixed(4), 'GB');
}

How to prevent Node.js from running out of memory

Increasing the memory limit is a quick fix to the problem, which in some cases is enough. Other times though, you may not have more memory available on your system, or the increased limit only fixes the problem temporarily.

In any case, you might want to get to the root of the issue and find out what caused it in the first place.

Here are three alternative solutions to reduce memory consumption that I've used or come across at some point.

Split data processing into smaller chunks

There are cases where you need to process large sets of data. For example, you wrote an importer that takes data from a CSV file, sanitizes it, and finally adds it to your database (this is also known as ETL: Extract - Transform - Load).

If your application keeps running out of memory while trying to import those big sets of data, an alternative solution is to split the data into smaller chunks.

# split -l numberoflines filename
split -l 1000000 users.csv

Avoid memory leaks

Your app keeps running out of memory, and increasing the memory limit only buys you time? Chances are you have some memory leaks in your application.

In my previous article, I explained how JavaScript's memory management works and how you can prevent some of the most common memory leaks.

To summarize it, most of the memory leaks can be traced back to not removing all references to objects that you don't need anymore. This can happen when you forget to remove intervals or timers, or you make excessive use of global variables.

Profiling

Profiling helps you to detect memory leaks. On the frontend, you can profile the memory usage in the Chrome DevTools under the Memory tab.

In Node.js, you can also Chrome for debugging the memory usage, starting from v6.3.0.

First, you need to run your app in inspection mode:

node --inspect index.js

Then open chrome://inspect in Chrome and click on Open dedicated DevTools for Node.

A new debugging window will open, which automatically connects to your Node.js application.

Spawn processes and restart them once they run out of memory

Let's say you are running your program on a machine that has very limited memory, like on a Raspberry Pi, for example.

This article explains how to spawn new processes once they run out of memory.

In this approach, we'd have one master process and multiple workers. As soon as a worker detects that it is running out of memory, the master will kill the process and use another worker to handle the main logic.

I found this an interesting way of handling the problem, but I haven't encountered yet a use-case myself where this was necessary.

Conclusion

In this article, I tried to give you all the information at hand that you need to solve the "JavaScript Heap Out Of Memory" error.

If you want to learn more about how JavaScript works, I have recently written these two articles on memory management and the event loop:

• JavaScript Event Loop And Call Stack Explained

  Heap, call stack, and event loop are all terms that I'd hear and read from time to time but which I never fully understood. In this article, I explain what they mean and what happens to our code when an engine executes it.

• JavaScript's Memory Management Explained

  Here I dive deeper into how JavaScript's memory management works, explaining what the heap and the stack are used for and how garbage collection works.

I hope you liked this article. If you did, make sure to subscribe to my email newsletter, so you get notified when I publish another one!