Unblocking the Loop: A Guide to Node.js Concurrency
Node.js is famous for being single-threaded. This architecture is its greatest strength for I/O-bound tasks, allowing it to handle thousands of concurrent connections with minimal overhead. However, this same architecture becomes a bottleneck when faced with CPU-intensive tasks.
The Problem: One Thread to Rule Them All
Imagine a restaurant with only one waiter (the Event Loop). This waiter is incredibly fast at taking orders and delivering food (I/O operations). But if a customer asks the waiter to stand at their table and calculate the square root of a billion (CPU operation), the waiter effectively stops working. No other customers get served. The restaurant freezes.
Solution 1: Worker Threads (Parallel JavaScript)
Introduced in Node.js v10.5.0, Worker Threads allow you to execute JavaScript in parallel threads. Unlike the main thread, Workers can block without affecting the rest of the application.
✔️ Use Case: Workers
- Video encoding/decoding
- Parsing large JSON files
- Complex mathematical calculations (Primes, Fibonacci)
- Image processing/resizing
ℹ️ Shared Memory
Workers run in the same process ID. They can share memory using SharedArrayBuffer, making data transfer extremely fast compared to child processes.
Solution 2: Child Processes (OS Level Execution)
Sometimes you don't need to run JavaScript; you need to run an external program. The child_process module allows Node.js to spawn new OS processes. This is like the waiter hiring a separate delivery service to handle a specific job.
// Example: Using spawn vs exec
const { spawn, exec } = require('child_process');
// exec: Buffer output (good for small output)
exec('git status', (err, stdout) => console.log(stdout));
// spawn: Stream output (good for large data)
const ls = spawn('ls', ['-lh', '/usr']);
ls.stdout.on('data', chunk => console.log(chunk));
Key Distinction: Use Worker Threads when you need to perform heavy JavaScript logic within your application. Use Child Processes when you need to execute an external binary (like Python, Git, or ImageMagick) or ensure complete process isolation.