Как найти утечку памяти node js

Method 1 –
Node allows us to manually trigger Garbage Collection, and it is the first thing that we should do when trying confirm a memory leak. This can be accomplished by running Node with –expose-gc flag (i.e. node –expose-gc index.js). Once node is running in that mode, you can programmatically trigger a Garbage Collection at any time by calling global.gc() from your program.

You can also check the amount of memory used by your process by calling process.memoryUsage().heapUsed.

By manually triggering garbage collection and checking the heap used, you can determine if you in fact observing a memory leak in your program. Memory growth becomes even more evident as you do this test.

Method 2 –

3 Heap Dumps Method

Node’s –inspect flag landed in node version 6. This feature lets you debug and inspect your node process from within Chrome’s DevTools.

Simply start the application passing the –inspect flag:

$ node --inspect index.js

Follow the URL to the inspector and navigate to the memory tab.

DevTools Memory Tab

Here’s what we will do:

Hit the application with autocannon -c 1 -d 60 http://localhost:PORT
[ https://www.npmjs.com/package/autocannon ]
Take a heap snapshot after roughly 10 seconds, and again after 30 seconds.

Heap Snapshot Comparison
The comparison view shows us what has happened between snapshots. You can see a huge number of objects have been created or not being collected by the GC.

Please read these amazing articles to learn more.
They have been of great help to find & fix a leak :-

https://www.alexkras.com/simple-guide-to-finding-a-javascript-memory-leak-in-node-js/

https://www.nearform.com/blog/self-detect-memory-leak-node/

https://addyosmani.com/blog/taming-the-unicorn-easing-javascript-memory-profiling-in-devtools/

In this article, you’ll learn how to understand and debug the memory usage of a Node.js application and use monitoring tools to get a complete insight into what is happening with the heap memory and garbage collection. Here’s what you’ll get by the end of this tutorial.

Memory leaks often go unnoticed. This is why I suggest using a tool to keep track of historical data of garbage collection cycles and to notify you if the heap memory usage starts spiking uncontrollably.

You can restart the app and make it all magically go away. But, you ultimately want to understand what happened and how to stop it from recurring.

That’s what I want to teach you today. Let’s jump in!

What Are Memory Leaks in Node.js?

Long story short, it’s when your Node.js app’s CPU and memory usage increases over time for no apparent reason.

In simple terms, a Node.js memory leak is an orphan block of memory on the Heap that is no longer used by your app because it has not been released by the garbage collector.

It’s a useless block of memory. These blocks can grow over time and lead to your app crashing because it runs out of memory.

Let me tell you what all of this means, under the hood.

What Is Garbage Collection in Node.js?

If a piece of memory segment is not referenced from anywhere, it can be released. However, the garbage collector can have a hard time keeping track of every piece of memory. That’s how you get the strange increase in app load even if it doesn’t make any sense.

Node.js uses Chrome’s V8 engine to run JavaScript. Within V8, memory is categorized into Stack and Heap memory.

• Stack: Stores static data, method and function frames, primitive values, and pointers to stored objects. The stack is managed by the operating system.
• Heap: Stores objects. Because everything in JavaScript is an object this means all dynamic data like arrays, closures, etc. The heap is the biggest block of memory and it’s where Garbage Collection (GC) happens.

Garbage collection frees up memory in the Heap used by objects that are no longer referenced from the Stack, either directly or indirectly. The goal is to create free space for creating new objects. Garbage collection is generational. Objects in the Heap are grouped by age and cleared at different stages.

There are two stages and two algorithms used for garbage collection.

New Space and Old Space

The heap has two main stages.

• New Space: Where new allocations happen. Garbage collection is quick. Has a size of between 1 and 8 MBs. Objects in the New Space are called the Young Generation.
• Old Space: Where objects that survived the collector in the New Space are promoted to. Objects in the Old Space are called the Old Generation. Allocation is fast, however, garbage collection is expensive and infrequent.

Young Generation and Old Generation

Roughly a fifth of the Young Generation survives garbage collection and gets moved to the Old Generation. Garbage collection of Old Generation objects will only start when the memory gets exhausted. In that case V8 uses two algorithms for garbage collection.

Scavenge and Mark-Sweep collection

• Scavenge collection: Fast and runs on the Young Generation.
• Mark-Sweep collection: Slower and runs on the Old Generation.

Why is garbage collection expensive?

Sadly, V8 stops the program execution while garbage collection is in progress. This can causes increased latency. It’s obvious you do not want garbage collection cycles to take up significant time and block the main thread of execution.

Woah, that was a lot to soak in. But, now once you know all of this, let’s move on to figuring out what causes memory leaks and how to fix them.

What Causes Them: Common Node.js Memory Leaks

Some Node.js memory leaks are caused by common issues. These can be circular object references that are caused by a multitude of reasons.

Let me tell you about the most common causes for memory leaks.

Global variables

Global variables in Node.js are the variables referenced by the root node, which is global. It’s the equivalent of window for JavaScript running in the browser.

Global variables are never garbage collected throughout the lifetime of an app. They occupy memory as long as the app is running. Here’s the kicker: this applies to any object referenced by a global variable, and all their children, as well. Having a large graph of objects referenced from the root can lead to a memory leak in Node.js applications.

Let me start a trend of saying, “please, don’t do this.”

Multiple references

If you reference the same object from multiple objects, it can lead to a memory leak if one of the references is garbage collected while the other one is left dangling.

Closures

Closures memorize their surrounding context. When a closure holds a reference to a large object in heap, it keeps the object in memory as long as the closure is in use.

This implies easily ending up in situations where a closure holding such a reference can be improperly used leading to a memory leak.

Timers & Events

The use of setTimeout, setInterval, Observers, and event listeners can cause memory leaks when heavy object references are kept in their callbacks without proper handling.

How to Avoid Memory Leaks in Node.js Applications: Prevention Best Practices

Now that you have a better understanding of what causes memory leaks in Node.js applications, let me explain how to avoid them and a few best practices to use to make sure the memory is used efficiently.

Reduce Use of Global Variables

Since global variables are never garbage collected, it’s best to ensure you don’t overuse them. Here are a few ways you can make sure this does not happen.

Avoid Accidental Globals

When you assign a value to an undeclared variable, JavaScript automatically “hoists” it. This means it’s assigned to the global scope. Very bad!

This could be the result of a typo and could lead to a memory leak. Another way could be when assigning a variable to this within a function in the global scope.

// This will be hoisted as a global variable
function hello() {
  foo = "Message";
}

// This will also become a global variable as global functions have
// global `this` as the contextual `this` in non strict mode
function hello() {
  this.foo = "Message";
}

To avoid issues like this, always write JavaScript in strict mode using the ‘use strict’; annotation at the top of your JS file.

When you use ES modules or transpilers like TypeScript or Babel, you don’t need it as it’s automatically enabled.

In Node.js, you can enable strict mode globally by passing the –use_strict flag when running the node command.

"use strict";

// This will not be hoisted as a global variable
function hello() {
  foo = "Message"; // will throw runtime error
}

// This will not become global variable as global functions
// have their own `this` in strict mode
function hello() {
  this.foo = "Message";
}

When you use arrow functions, you also need to be mindful not to create accidental globals, and unfortunately, strict mode will not help with this. You can use the no-invalid-this rule from ESLint to avoid such cases.

If you are not using ESLint, just make sure not to assign to this from global arrow functions.

// This will also become a global variable as arrow functions
// do not have a contextual `this` and instead use a lexical `this`
const hello = () => {
    this.foo = 'Message";
}

Finally, keep in mind not to bind global this to any functions using the bind or call method, as it will defeat the purpose of using strict mode.

Use Global Scope Sparingly

You should always avoid using the global scope whenever you can, including global variables. Here are some best practices you should follow in this regard:

• Use local scope inside functions, as it will be garbage collected. This frees up memory. If you have to use a global variable due to some constraints, set the value to null when it’s no longer needed. This means you can garbage collect it “manually”.
• Use global variables only for constants, cache, and reusable singletons.
• Never use global variables for the convenience of avoiding passing values around. For sharing data between functions and classes, pass the values around as parameters or object attributes.
• Never store big objects in the global scope. If you have to store them, set the values to null when they’re not needed.
• When you’re using objects as cache, set a handler to clean them up once in a while and don’t let them grow indefinitely.

Use Stack Memory Effectively

Accessing the stack memory is more efficient and performant than accessing the heap. It’s illogical to only use static values as in the real world you use objects and dynamic data.

However, there are few things you can do to make this process more memory friendly.

First, avoid heap object references from stack variables when possible. Secondly, delete unused variables. Third, destructure objects and use only the fields you need from an object or array rather than passing around entire objects or arrays to functions, closures, timers, and event handlers.

If you follow these simple suggestions you’ll avoid keeping references to objects inside closures. The fields you pass are primitives, which will be kept in the stack. Here’s an example below.

function outer() {
    const obj = {
        foo: 1,
        bar: "hello",
    };

    const closure = () {
        const { foo } = obj;
        myFunc(foo);
    }
}

function myFunc(foo) {}

Use Heap Memory Effectively

You have to use the heap in every Node.js application you build. It’s up to you to use it efficiently.

Passing object references is more expensive regarding heap usage than copying objects. You should pass a reference only if the object is absolutely massive. But, this won’t happen often at all in your day-to-day Node.js development.

You can use the object spread syntax (…) or Object.assign to copy an object. This paradigm is aligned with immutability in functional programming where you are constantly creating new objects while making sure functions are as “pure” as possible.

This also means you should avoid creating huge object trees. If you can’t avoid it, try to keep them short-lived in the local scope.

This is also what you should do with variables in general, make them short-lived.

And while you are at it, make sure you monitor your heap size. I’ll explain how to do this with Sematext a bit further down in this article.

Properly Using Closures, Timers, and Event Handlers

Closures, timers, and event handlers can often create memory leaks in Node.js applications.

Let’s look at a piece of code from the Meteor team explaining a closure that leads to a memory leak.

It leads to a memory leak as the longStr variable is never collected and keeps growing in memory. The details are explained in this blog post.

var theThing = null;
var replaceThing = function () {
  var originalThing = theThing;
  var unused = function () {
    if (originalThing) console.log("hi");
  };
  theThing = {
    longStr: new Array(1000000).join("*"),
    someMethod: function () {
      console.log(someMessage);
    },
  };
};
setInterval(replaceThing, 1000);

The code above creates multiple closures, and those closures hold on to object references. The memory leak, in this case, can be fixed by nullifying originalThing at the end of the replaceThing function.

You avoid issues like this by creating copies of the object and using the immutable objects approach I mentioned above.

The same logic applies to timers as well. Remember to pass copies of objects and avoid mutations. Clear the timers when you don’t need them anymore by using the clearTimeout and clearInterval methods.

The same goes for event listeners and observers. Clear them once they’re done doing what you want them to. Don’t leave event listeners running forever, especially if they are going to hold on to any object reference from the parent scope.

Node.js Memory Leak Detectors

Fixing a memory leak is not as straightforward as it may seem. You’ll need to check your codebase to find any issues with your global scope, closures, heap memory, or any other pain points I outlined above.

A quick way to fix Node.js memory leaks in the short term is to restart the app. Make sure to do this first and then dedicate the time to seek out the root cause of the memory leak.

Here are a few tools to help you detect memory leaks.

Memwatch

It’s been 9 years since memwatch was published on npm, but you can still use it to detect memory leaks.

This module is useful because it can emit leak events if it sees the heap grow over 5 consecutive garbage collections.

Heapdump

Heapdump is another great tool for taking snapshots of the heap memory. You can then later inspect them in the Chrome Developer Tools.

node-inspector

Sadly memwatch and heapdump can’t connect to a running app. However, node-inspector can! It lets you connect to a running app by running the node-debug command. This command will load Node Inspector in your default browser.

V8 Inspector & Chrome Dev Tools

You can use Dev Tools in Chrome to inspect Node.js apps. How? They both run the same V8 engine which contains the inspector used by the Dev Tools.

Here’s an example with a sample Express app. Its only purpose is to display all the requests that it has ever received.

const express = require('express')
const app = express()
const port = 3000
const requestLogs = [];

app.get('/', (req, res) => {
  requestLogs.push({ url: req.url, date: new Date() });
  res.status(200).send(JSON.stringify(requestLogs));
})

app.listen(port, () => {
  console.log(`Sample app listening on port ${port}.`)
})

In order to expose the inspector, let’s run Node.js with the –inspect flag.

node --inspect index.js

Debugger listening on ws://127.0.0.1:9229/7fc22153-836d-4ed2-8090-a84a842a199e
For help, see: <https://nodejs.org/en/docs/inspector>
Sample app listening on port 3000.

Open up Chrome and go to chrome://inspect. Voila! A full-featured debugger for your Node.js application. Here you can take snapshots of the memory usage.

Watching Memory Allocation in Real Time

A more optimized method to measure the memory allocation is to view it live instead of taking multiple snapshots.

You use the Allocation instrumentation on timeline option in this case. Select that radio button and check the Record stack traces of allocations checkbox. This will start a live recording of the memory usage.

For this use case, I used loadtest to run 1000 requests against the sample Express app with a concurrency of 10.

For the first few requests, you can see a spike in memory allocation. But it’s obvious that most memory is allocated to the arrays, closure, and objects.

Monitoring Tools

Using debugging tools to watch memory allocation and find memory leaks is one thing, but tracking it in real-time all the time is another cup of tea.

You need proper monitoring tools to give you historical data a time dimension where you can track and gain real insight into how your app is behaving.

Here are some tools to look into:

• Sematext Monitoring
• PM2
• Clinic.js
• Prometheus
• Express Status Monitor

If you want to gain more insight into the many options to detect memory leaks you can take a look at another article I wrote a while back: Node.js monitoring tools.

Now, let’s take this to the next level and watch for Node.js memory leaks in real-time with a proper monitoring tool.

Debugging Example: How to Find Leaks with Sematext

Sematext Monitoring enables you to monitor your entire infrastructure stack, from top to bottom, with one tool. You can map out, and monitor, your whole infrastructure in real-time, down to below a granularity of below a minute.

You also get real-time insight into the performance of your Node.js app. This includes garbage collection, worker threads, HTTP request and response latency, event loop latency, CPU and memory usage, and much more.

You’re ready to set up your monitoring!

Start by creating a Node.js App in Sematext Cloud.

Follow the instructions in the UI to install the required Agents that will collect infra and Node.js metrics.

Create a .env file at the root of your project. Add this snippet.

MONITORING_TOKEN=b2b0b02b-xxxx-xxxx-xxxx-11f89c2ccc64

Edit the index.js of your app to require the sematext-agent-express. Make sure to include it at the very top of the index.js file.

// Load env vars
require('dotenv').config()

// require stMonitor agent
const { stMonitor } = require('sematext-agent-express')

// Start monitoring metrics
stMonitor.start()

You’ll see the dashboards get populated with Node.js metrics within a few minutes.

By using the garbage collection and memory charts, you can get a complete insight into what is happening in your app.

If the Garbage Collection and Heap Memory keep rising without releasing constant amounts of memory consistently, you most definitely have an issue. A quick solution would be to restart your app. But in the long run, this won’t do. You need to apply all the steps I outlined above to reduce the chance of memory leaks.

A great option when using a tool like Sematext is to create alerts when your memory reaches a certain threshold.

Here’s how you do it!

Click the Alert Rules on the left-hand side nav and create a new Alert Rule with the memory as a metric and the amount of memory you want to set to the threshold.

Want to learn more? Check out this quick video to see exactly what Sematext Monitoring is, and how it can help you.

Conclusion

This sure was a roller-coaster ride of emotions. You learned how to understand and debug the memory usage of a Node.js application and use monitoring tools such as Sematext Monitoring to get a complete insight into what is happening with the heap memory and garbage collection. Now you know how to detect Node.js memory leaks so they never go unnoticed. Hope this helps you build more reliable and performant Node.js apps, ultimately saving you time on debugging issues and your sanity. If you’re looking for the right solution for your use case, give Sematext a try! There’s a 14-day free trial available for you to test all its features.

Hope you liked reading this article, leave a comment and feel free to share your thoughts on Twitter!

Start Free Trial

---

I once drove an Audi with a V8 twin-turbo engine inside, and its performance was incredible. I was driving at around 140MPH on IL-80 highway near Chicago at 3AM when there was nobody on the road. Ever since then, the term “V8” has become associated with high performance to me.

Node.js is a platform built on Chrome’s V8 JavaScript engine for easy building of fast and scalable network applications.

Although Audi’s V8 is very powerful, you are still limited with the capacity of your gas tank. The same goes for Google’s V8 – the JavaScript engine behind Node.js. Its performance is incredible and there are many reasons why Node.js works well for many use cases, but you’re always limited by the heap size. When you need to process more requests in your Node.js application you have two choices: either scale vertically or scale horizontally. Horizontal scaling means you have to run more concurrent application instances. When done right, you end up being able to serve more requests. Vertical scaling means that you have to improve your application’s memory usage and performance or increase resources available for your application instance.

Debugging Memory Leaks in Node.js Applications

Recently I was asked to work on a Node.js application for one of my Toptal clients to fix a memory leak issue. The application, an API server, was intended to be able to process hundreds of thousands of requests every minute. The original application occupied almost 600MB of RAM and therefore we decided to take the hot API endpoints and reimplement them. Overhead becomes very pricey when you need to serve many requests.

For the new API we chose restify with native MongoDB driver and Kue for background jobs. Sounds like a very lightweight stack, right? Not quite. During peak load a new application instance could consume up to 270MB of RAM. Therefore my dream of having two application instances per 1X Heroku Dyno vanished.

Node.js Memory Leak Debugging Arsenal

Memwatch

If you search for “how to find leak in node” the first tool you’d probably find is memwatch. The original package was abandoned a long time ago and is no longer maintained. However you can easily find newer versions of it in GitHub’s fork list for the repository. This module is useful because it can emit leak events if it sees the heap grow over 5 consecutive garbage collections.

Heapdump

Great tool which allows Node.js developers to take heap snapshot and inspect them later with Chrome Developer Tools.

Node-inspector

Even a more useful alternative to heapdump, because it allows you to connect to a running application, take heap dump and even debug and recompile it on the fly.

Taking “node-inspector” for a Spin

Unfortunately, you will not be able to connect to production applications that are running on Heroku, because it does not allow signals to be sent to running processes. However, Heroku is not the only hosting platform.

To experience node-inspector in action, we will write a simple Node.js application using restify and put a little source of memory leak within it. All experiments here are made with Node.js v0.12.7, which has been compiled against V8 v3.28.71.19.

var restify = require('restify');

var server = restify.createServer();

var tasks = [];

server.pre(function(req, res, next) {
  tasks.push(function() {
    return req.headers;
  });

  // Synchronously get user from session, maybe jwt token
  req.user = {
    id: 1,
    username: 'Leaky Master',
  };

  return next();
});

server.get('/', function(req, res, next) {
  res.send('Hi ' + req.user.username);
  return next();
});

server.listen(3000, function() {
  console.log('%s listening at %s', server.name, server.url);
});

The application here is very simple and has a very obvious leak. The array tasks would grow over application lifetime causing it to slow down and eventually crash. The problem is that we are not only leaking closure but entire request objects as well.

GC in V8 employs stop-the-world strategy, therefore it means more objects you have in memory the longer it will take to collect garbage. On log below you can clearly see that in the beginning of the application life it would take an average of 20ms to collect the garbage, but few hundred thousand requests later it takes around 230ms. People who are trying to access our application would have to wait 230ms longer now because of GC. Also you can see that GC is invoked every few seconds which means that every few seconds users would experience problems accessing our application. And delay will grow up until application crashes.

[28093]     7644 ms: Mark-sweep 10.9 (48.5) -> 10.9 (48.5) MB, 25.0 ms [HeapObjectsMap::UpdateHeapObjectsMap] [GC in old space requested].

[28093]     7717 ms: Mark-sweep 10.9 (48.5) -> 10.9 (48.5) MB, 18.0 ms [HeapObjectsMap::UpdateHeapObjectsMap] [GC in old space requested].

[28093]     7866 ms: Mark-sweep 11.0 (48.5) -> 10.9 (48.5) MB, 23.2 ms [HeapObjectsMap::UpdateHeapObjectsMap] [GC in old space requested].

[28093]     8001 ms: Mark-sweep 11.0 (48.5) -> 10.9 (48.5) MB, 18.4 ms [HeapObjectsMap::UpdateHeapObjectsMap] [GC in old space requested].

...

[28093]   633891 ms: Mark-sweep 235.7 (290.5) -> 235.7 (290.5) MB, 357.3 ms [HeapObjectsMap::UpdateHeapObjectsMap] [GC in old space requested].

[28093]   635672 ms: Mark-sweep 235.7 (290.5) -> 235.7 (290.5) MB, 331.5 ms [HeapObjectsMap::UpdateHeapObjectsMap] [GC in old space requested].

[28093]   637508 ms: Mark-sweep 235.7 (290.5) -> 235.7 (290.5) MB, 357.2 ms [HeapObjectsMap::UpdateHeapObjectsMap] [GC in old space requested].

These log lines are printed when a Node.js application is started with the –trace_gc flag:

node --trace_gc app.js

Let us assume that we have already started our Node.js application with this flag. Before connecting the application with node-inspector, we need to send it the SIGUSR1 signal to the running process. If you run Node.js in cluster, make sure you connect to one of the slave processes.

kill -SIGUSR1 $pid # Replace $pid with the actual process ID

By doing this, we are making the Node.js application (V8 to be precise) enter debugging mode. In this mode, the application automatically opens the port 5858 with V8 Debugging Protocol.

Our next step is to run node-inspector which will connect to the debugging interface of the running application and open another web interface on port 8080.

$ node-inspector
Node Inspector v0.12.2
Visit http://127.0.0.1:8080/?ws=127.0.0.1:8080&port=5858 to start debugging.

In case the application is running on production and you have a firewall in place, we can tunnel remote port 8080 to localhost:

ssh -L 8080:localhost:8080 admin@example.com

Now you could open your Chrome web browser and get full access to Chrome Development Tools attached to your remote production application. Unfortunately, Chrome Developer Tools will not work in other browsers.

Let’s Find a Leak!

Memory leaks in V8 are not real memory leaks as we know them from C/C++ applications. In JavaScript variables do not disappear into the void, they just get “forgotten”. Our goal is to find these forgotten variables and remind them that Dobby is free.

Inside Chrome Developer Tools we have access to multiple profilers. We are particularly interested in Record Heap Allocations which runs and takes multiple heap snapshots over time. This gives us a clear peek into which objects are leaking.

Start recording heap allocations and let’s simulate 50 concurrent users on our home page using Apache Benchmark.

ab -c 50 -n 1000000 -k http://example.com/

Before taking new snapshots, V8 would perform mark-sweep garbage collection, so we definitely know that there is no old garbage in the snapshot.

Fixing the Leak on the Fly

After collecting heap allocation snapshots over a period of 3 minutes we end up with something like the following:

We can clearly see that there are some gigantic arrays, a lot of IncomingMessage, ReadableState, ServerResponse and Domain objects as well in heap. Let’s try to analyze the source of the leak.

Upon selecting heap diff on chart from 20s to 40s, we will only see objects which were added after 20s from when you started the profiler. This way you could exclude all normal data.

Keeping note of how many objects of each type are in the system, we expand the filter from 20s to 1min. We can see that the arrays, already quite gigantic, keeps growing. Under “(array)” we can see that there are a lot of objects “(object properties)” with equal distance. Those objects are the source of our memory leak.

Also we can see that “(closure)” objects grow rapidly as well.

It might be handy to look at the strings as well. Under the strings list there are a lot of “Hi Leaky Master” phrases. Those might give us some clue too.

In our case we know that the string ”Hi Leaky Master” could only be assembled under the “GET /” route.

If you open retainers path you will see this string is somehow referenced via req, then there is context created and all this added to some giant array of closures.

So at this point we know that we have some kind of gigantic array of closures. Let’s actually go and give a name to all our closures at real-time under sources tab.

After we are done editing the code, we can hit CTRL+S to save and recompile code on the fly!

Now let’s record another Heap Allocations Snapshot and see which closures are occupying the memory.

It’s clear that SomeKindOfClojure() is our villain. Now we can see that SomeKindOfClojure() closures are being added to some array named tasks in the global space.

It’s easy to see that this array is just useless. We can comment it out. But how do we free memory the memory already occupied? Very easy, we just assign an empty array to tasks and with the next request it will be overridden and memory will be freed after next GC event.

Dobby is free!

Life of Garbage in V8

Well, V8 JS does not have memory leaks, only forgotten variables.

V8 heap is divided into several different spaces:

• New Space: This space is relatively small and has a size of between 1MB and 8MB. Most of the objects are allocated here.
• Old Pointer Space: Has objects which may have pointers to other objects. If object survives long enough in New Space it gets promoted to Old Pointer Space.
• Old Data Space: Contains only raw data like strings, boxed numbers and arrays of unboxed doubles. Objects that have survived GC in the New Space for long enough are moved here as well.
• Large Object Space: Objects which are too big to fit in other spaces are created in this space. Each object has it’s own mmap‘ed region in memory
• Code space: Contains assembly code generated by the JIT compiler.
• Cell space, property cell space, map space: This space contains Cells, PropertyCells, and Maps. This is used to simplify garbage collection.

Each space is composed of pages. A page is a region of memory allocated from the operating system with mmap. Each page is always 1MB in size except for pages in large object space.

V8 has two built in garbage collection mechanisms: Scavenge, Mark-Sweep and Mark-Compact.

Scavenge is a very fast garbage collection technique and operates with objects in New Space. Scavenge is the implementation of Cheney’s Algorithm. The idea is very simple, New Space is divided in two equal semi-spaces: To-Space and From-Space. Scavenge GC occurs when To-Space is full. It simply swaps To and From spaces and copy all live objects to To-Space or promote them to one of the old spaces if they survived two scavenges, and is then entirely erased from the space. Scavenges are very fast however they have the overhead of keeping double sized heap and constantly copying objects in memory. The reason to use scavenges is because most objects die young.

Mark-Sweep & Mark-Compact is another type of garbage collector used in V8. The other name is full garbage collector. It marks all live nodes, then sweeps all dead nodes and defragments memory.

GC Performance and Debugging Tips

While for web applications high performance might not be such a big problem, you will still want to avoid leaks at all costs. During the mark phase in full GC the application is actually paused until garbage collection is completed. This means the more objects you have in the heap, the longer it will take to perform GC and the longer users will have to wait.

Always give names to closures and functions

It’s much easier to inspect stack traces and heaps when all your closures and functions have names.

db.query('GIVE THEM ALL', function GiveThemAllAName(error, data) {
    ...
})

Avoid large objects in hot functions

Ideally you want to avoid large objects inside of hot functions so that all data is fit into New Space. All CPU and memory bound operations should be executed in background. Also avoid deoptimization triggers for hot functions, optimized hot function uses less memory than non-optimized ones.

Hot functions should be optimized

Hot functions that run faster but also consume less memory cause GC to run less often. V8 provides some helpful debugging tools to spot non-optimized functions or deoptimized functions.

Avoid polymorphism for IC’s in hot functions

Inline Caches (IC) are used to speed up execution of some chunks of code, either by caching object property access obj.key or some simple function.

function x(a, b) {
  return a + b;
}

x(1, 2); // monomorphic
x(1, “string”); // polymorphic, level 2
x(3.14, 1); // polymorphic, level 3

When x(a,b) is run for the first time, V8 creates a monomorphic IC. When you call x a second time, V8 erases the old IC and creates a new polymorphic IC which supports both types of operands integer and string. When you call IC the third time, V8 repeats the same procedure and creates another polymorphic IC of level 3.

However, there is a limitation. After IC level reaches 5 (could be changed with –max_inlining_levels flag) the function becomes megamorphic and is no longer considered optimizable.

It’s intuitively understandable that monomorphic functions run the fastest and also have a smaller memory footprint.

Don’t add large files to memory

This one is obvious and well known. If you have large files to process, for example a large CSV file, read it line-by-line and process in little chunks instead of loading the entire file to memory. There are rather rare cases where a single line of csv would be larger than 1mb, thus allowing you to fit it in New Space.

Do not block main server thread

If you have some hot API which takes some time to process, such as an API to resize images, move it to a separate thread or turn it into a background job. CPU intensive operations would block main thread forcing all other customers to wait and keep sending requests. Unprocessed request data would stack in memory, thus forcing full GC to take longer time to finish.

Do not create unnecessary data

I once had a weird experience with restify. If you send a few hundred thousand requests to an invalid URL then the application memory would rapidly grow on up to hundred megabytes until a full GC kicks in a few seconds later, which is when everything would go back to normal. Turns out that for each invalid URL, restify generates a new error object which includes long stack traces. This forced newly created objects to be allocated in Large Object Space rather than in New Space.

Having access to such data could be very helpful during development, but obviously not required on production. Therefore the rule is simple – do not generate data unless you certainly need it.

Know your tools

Last, but certainly not the least, is to know your tools. There are various debuggers, leak cathers, and usage graphs generators. All those tools can help you make your software faster and more efficient.

Conclusion

Understanding how V8’s garbage collection and code optimizer works is a key to application performance. V8 compiles JavaScript to native assembly and in some cases well written code could achieve performance comparable with GCC compiled applications.

And in case you are wondering, the new API application for my Toptal client, although there is room for improvement, is working very well!

Joyent recently released a new version of Node.js which uses one of the latest versions of V8. Some applications written for Node.js v0.12.x may not be compatible with the new v4.x release. However, applications will experience tremendous performance and memory usage improvement within the new version of Node.js.

Further Reading on the Toptal Engineering Blog:

• The 10 Most Common JavaScript Issues Developers Face
• Benchmarking A Node.js Promise
• Is It Time to Use Node 8?
• Software Reengineering: From Spaghetti to Clean Design
• Eliminating the Garbage Collector: The RAII Way