One of the themes of this blog has been writing programs that are small enough so they can be easily distributed, without needing the user to download libraries to use it first. Sometimes you want that, but little shell script replacement scripts sometimes don’t.

For example, if I’m distributing a small Node.js script to teammates I want to just give them a .js file, not a .js file plus a package.json file then have them run npm install before they can use my script.

This is why I’ve enjoyed using Deno. But, with recent changes with Deno, maybe it’s not a good, long term, dependency.

Thus, my question: can I make plain old Node.js auto-download packages for me

Today I - well, mostly Cursor - came up with a solution named npx-node.

First, a sample script

We specify our dependent packages via the shebang line, using some abilities of npx.

npx is really meant to quickly run a command from a remote (or local) module, downloading it if required. Here we use it outside of its intended use case: -p is downloading packages, but we’re NOT going to actually run them. Instead we run node.

#!/usr/bin/env -S npx -y -p chalk@4 -p axios npx-node

const chalk = require('chalk');
const axios = require('axios');

console.log(chalk.blue('Both chalk and axios were auto-installed by npx!'));

Why npx-node, or: when it gets complicated

It turns out that npx -p downloads packages somewhere in ~/.npm/_npx/(random hex number). Because it’s really meant to run downloaded applications it then sets the PATH variable to pick up the downloaded executables. Node module resolution happens as normal, as the binary exists in the standard node_modules structure, and things execute like any other Node.js project.

Except here, what we want to execute lives far outside the downloaded node_modules.

npx-node:

#!/bin/sh
# Drop-in for `node` after `npx -p`. Modern npx puts those packages on PATH
# but not NODE_PATH, so require() cannot see them. This repairs that and execs node.
#
#   #!/usr/bin/env -S npx -y -p chalk@4 -p axios npx-node
#
# When your shebang runs, npx does two things:
#
#  1. Installs modules you specify with -p parameters under something like
#     ~/.npm/_npx/85b45fc17e66a366/node_modules
#  2. Prepends the bin folder of that cache to PATH
#
#  So PATH starts with:
#
#  /Users/rwilcox/.npm/_npx/85b45fc17e66a366/node_modules/.bin
#
#  That is enough to run CLI tools (cowsay, tsx). require('foobar')
#  does not look at PATH. It looks at NODE_PATH and node_modules next
#  to the script. npx never sets NODE_PATH, and it does not export
#  “here is my cache.” The only breadcrumb it leaves is that .bin
#  entry on PATH.
#
# If you do not want to use node, but instead ts-node or tsx
# have your shebang line set NODE_BIN (-S NODE_BIN=tsx)

NODE_BIN=${NODE_BIN:-node}
npx_modules=
old_ifs=$IFS
IFS=:
for entry in $PATH; do
  case $entry in
    */.npm/_npx/*/node_modules/.bin)
      npx_modules=${entry%/.bin}
      break
      ;;
  esac
done
IFS=$old_ifs

if [ -n "$npx_modules" ]; then
  if [ -n "$NODE_PATH" ]; then
    NODE_PATH="$npx_modules:$NODE_PATH"
  else
    NODE_PATH="$npx_modules"
  fi
  export NODE_PATH
fi

exec "$NODE_BIN" "$@"

Using npx-node

It’s easy:

1. Put it somewhere on $PATH
2. Add your packages in your npx call

If you want to execute Typescript, not regular Javascript, use a shebang line like:

#!/usr/bin/env -S NODE_BIN=tsx npx -y -p chalk@4 -p axios -p tsx npx-node