Introduction

Streams are Node.js’s built‑in abstraction for handling data that isn’t available all at once—such as files, network sockets, or HTTP request bodies. Unlike reading an entire file into memory, streams process data chunk by chunk, enabling efficient I/O and lower memory usage. This tutorial explains the four stream types, demonstrates common patterns, and shows how to handle backpressure and errors properly.

Step 1 – Understand the Stream Types

• Readable: emits data you can .pipe() out (e.g., file reads, HTTP responses).
• Writable: accepts data you .write() in (e.g., file writes, HTTP requests).
• Duplex: both readable and writable (e.g., TCP sockets).
• Transform: duplex that modifies data along the way (e.g., zlib.createGzip()).

Step 2 – Reading a File as a Stream

Instead of fs.readFile(), use fs.createReadStream() to handle large files without exhausting memory:

const fs = require('fs');

const readStream = fs.createReadStream('large.txt', { encoding: 'utf8' });

readStream.on('data', chunk => {
  console.log('Received', chunk.length, 'bytes');
});

readStream.on('end', () => console.log('File fully read'));

chunk is a Buffer (or string if encoding is specified). Node reads the file in 64 KB blocks by default.

Step 3 – Piping a Readable into a Writable

The .pipe() method connects streams and manages flow automatically:

const writeStream = fs.createWriteStream('copy.txt');
readStream.pipe(writeStream);

Node handles backpressure—pausing the source when the destination’s internal buffer is full and resuming when drained.

Step 4 – Transform Streams (Gzip Example)

Transform streams modify data mid-flight. Compress a file with Gzip:

const zlib = require('zlib');
const gzip = zlib.createGzip();

fs.createReadStream('large.txt')
  .pipe(gzip)
  .pipe(fs.createWriteStream('large.txt.gz'));

Step 5 – Handling Backpressure Manually

Sometimes you need explicit control. Check the return value of write() and wait for the drain event:

const writable = fs.createWriteStream('output.txt');

function writeLots() {
  let ok = true;
  let i = 0;
  while (i < 1e6 && ok) {
    ok = writable.write(`Line ${i++}\n`);
  }
  if (i < 1e6) {
    writable.once('drain', writeLots);
  }
}
writeLots();

drain signals that the buffer has flushed and writing can continue without data loss.

Step 6 – Robust Error Handling

Always attach error listeners to prevent uncaught exceptions:

readStream.on('error', console.error);
writeStream.on('error', console.error);

You can also use pipeline() from stream/promises to automatically handle cleanup on error.

Conclusion

Node.js streams enable efficient, non-blocking data handling—crucial for building scalable applications. By mastering the four stream types, piping, backpressure, and error handling, you can process large datasets, handle HTTP uploads/downloads, and chain complex transformations without overloading memory.