What is cluster?

• Cluster is a feature in nodejs that lets you create child processes
• It uses (shares) the same port.
• The spawned child worker processes has its own memory/event loop/v8 instance.
• It can be used to load balance your application
• There is a master process and the spawned child (worker) processes.
  • The master one creates/manages the child processes
  • the child (worker) processes handle the actual request
• when an incoming request comes in, the master process listens to it and distributes it (in round-robin) to a child worker process.

It is often recommended to spawn as many child processes as you have CPU cores.

Example of using cluster with an express server

Example usage of cluster with express...

const cluster = require('cluster');
const express = require('express');
const numCpuCores = require("os").cpus().length;

const app = express();

if (cluster.isMaster) {
    // master process
    
  // spawn child processes, one for each cpu core
  for (let i = 0; i < numCpuCores; i++) {
    cluster.fork();
  }

  cluster.on('exit', (worker) => {
    console.log(`closing down pid: ${worker.process.pid}`);
    // you could run cluster.fork() to spawn another one...
  });
} else {
  // worker process
    
  // create server on port 3000 (same for all processes)
  app.listen(3000, () => {
    console.log(`Server started on http://localhost:3000, process id ${process.pid}`);
  });

  app.get('/', (_req, res) => {
    res.send(`Example response from ${process.pid}`);
  });
}

How the isMaster property works

The cluster.isMaster is basically a wrapper for this code:

// pseudo code
cluster.isMaster = process.env.NODE_UNIQUE_ID === undefined 

How to manage a cluster (with PM2)

The example above shows how you can manage your own cluster. But it is a very simplified example.

I'd recommend using PM2 ('process manager 2'). It is a much better way to load balance, and it uses cluster under the hood.

Example usage of pm2

Install it:

npm install pm2

Set up a basic server like this (note: we're not importing in pm2 here)

const express = require("express");
const app = express();
const port = 3000;
 
app.get(`/`, (req, res) => {
  res.send(`Example response...`);
});
 
 
app.listen(port, () => {
  console.log(`App started localhost:${port}`);
});

Then use pm2 to spawn it...

pm2 start index.js -i 0
# stop it with pm2 stop index.js

This passes in -i 0, which is for number of workers. If it is set to 0, then PM2 will use the number of CPU cores

PM2 is really nice, really powerful - lots of config options.

Worker threads

Use worker threads to execute JS in parallel.

It has been available in nodejs since Node v10.5, and stable since node v12 (LTS). You used to have to run node with node --experimental-worker your-script.js flag before Node v11.7.

Of course normally nodejs is single threaded. We can use promises (and callbacks) to have a nice async/await system, but at its core really the JS code we write is single threaded (but there might be background operations running in parallel when dealing with i/o, like disk or database).

Example of using worker_threads

Spawn worker threads and post messages.

const {
  Worker, isMainThread, parentPort, workerData,
} = require('node:worker_threads');

if (isMainThread) {
  module.exports = function parseJSAsync(script) {
    return new Promise((resolve, reject) => {
      const worker = new Worker(__filename, {
        workerData: script,
      });
      worker.on('message', resolve);
      worker.on('error', reject);
      worker.on('exit', (code) => {
        if (code !== 0)
          reject(new Error(`Worker stopped with exit code ${code}`));
      });
    });
  };
} else {
  const { parse } = require('some-js-parsing-library');
  const script = workerData;
  // can do slow things here, that block (are synchronous) 
  // and it won't block the main thread
  parentPort.postMessage(parse(script));
}