Programmatic video generation in Node.js

Node is a reasonable place to put a video-generation pipeline. The same project that runs your Next.js app can drive headless Chromium, capture frames, and shell out to an encoder. The pieces are all npm install-able. The hard part isn't getting Node to do it; it's structuring the pipeline so the output is reproducible.

The components

A working pipeline has four:

1. A renderer process. Usually puppeteer-core or playwright driving headless Chromium.
2. An HTML scene with a render(t) contract — see HTML to MP4 programmatically.
3. A frame capture loop. For each frame index, set t, wait for paint, screenshot.
4. An encoder. mp4-muxer (pure JS), h264-mp4-encoder (WASM), or shell out to ffmpeg. All three work; pick by deployment target.

A skeleton

import puppeteer from "puppeteer-core";
import { Muxer, ArrayBufferTarget } from "mp4-muxer";

async function renderScene(url: string, durationSec: number, fps = 30) {
  const browser = await puppeteer.launch({ headless: "new" });
  const page = await browser.newPage();
  await page.setViewport({ width: 1920, height: 1080, deviceScaleFactor: 1 });
  await page.goto(url, { waitUntil: "networkidle0" });
  await page.evaluate(() => document.fonts.ready);

  const total = Math.round(durationSec * fps);
  const muxer = new Muxer({
    target: new ArrayBufferTarget(),
    video: { codec: "avc", width: 1920, height: 1080 },
    fastStart: "in-memory",
  });

  for (let i = 0; i < total; i++) {
    const t = i / fps;
    await page.evaluate((time) => window.dispatchEvent(new CustomEvent("hf-seek", { detail: { time } })), t);
    await page.evaluate(() => new Promise((r) => requestAnimationFrame(() => requestAnimationFrame(r))));
    const buf = await page.screenshot({ type: "png", omitBackground: false });
    muxer.addVideoChunk(toEncodedChunk(buf, i, fps));
  }

  muxer.finalize();
  await browser.close();
  return muxer.target.buffer;
}

The toEncodedChunk step is the part most tutorials hand-wave. For a production pipeline, route the PNGs through an h.264 encoder (@ffmpeg/ffmpeg, a native binding, or a CDP-based VideoEncoder).

Watch out for

Concurrency. Don't fan out frames within a single page; the DOM is one shared state. Fan out across multiple browser contexts. Empirically, 4-8 contexts per CPU core is the sweet spot for 1080p.

Memory. PNGs from page.screenshot are 6-8 MB at 1080p. A 60-second 30fps render is 1800 frames × 7 MB = 12 GB if you hold them all. Stream them through the muxer, don't accumulate.

Fonts. document.fonts.ready is necessary but not sufficient — variable fonts can shift their axis values after first paint. Either inline fonts as data URLs or wait an extra 100ms after fonts.ready for variable axes to settle.

Locale. toLocaleString reads the OS locale. A CI runner and a developer machine produce different numbers ("1,234" vs "1.234"). Force the locale at the top of every scene: Intl.NumberFormat("en-US", ...).

Where to deploy

For low-volume, on-demand rendering (one video per user request), a single Node + Chromium process per box is fine. For batch (thousands of videos), put Chromium in a container, ship the container to a queue worker, and parallelize.

For one-off long videos (10+ minutes), parallelism matters more than throughput. Split into chunks, render in parallel, concatenate.

The detailed CI-render story is in deterministic video rendering in CI. The headless-Chromium tuning is in headless Chrome video rendering.

A working summary

Node + headless Chromium + a deterministic HTML scene + an encoder is the smallest viable video-generation stack. It fits in a single repo, runs in CI, scales out per worker, and produces MP4s that are byte-identical between runs. None of the pieces are exotic; the discipline is in keeping wall-clock time out of the rendering loop.