Intro

In a long standing blog patterns start to emerge. I’m interested in what I call ”Systems Level Programming“.

By “Systems Level Programming” I mean:

• Ideally OS independent (native Bash is usually out)
• Easily Deployable (so anything you have to npm install or bundle install is out)
• Ideally avoids - or offers an upgrade out of - relying on external tools. (Maybe jq isn’t installed on your CI setup, or you don’t want to care about the differences between MacOS and Alpine find. Really want to avoid these problems.)

Ideally I also get a fast iteration time and easy refactoring: if my small script turns into a large script, will it turn into a mess or will the language grow with my needs?

What Bun brings to the table

The Javascript Runtime ‘Bun’ seems interesting for these type of tools.

Bun has interesting characteristics for this level of work:

1. Typescript support built in. Unlike Node.js, natively, I don’t need another tool transpiling my code.
2. SQLite support built in. My recent duckdb play has me more and more convinced that having a simple SQL database ready at hand is a good idea.
3. A cross platform shell API
4. I can generate a Single File Executable. Usually I feel Node.js programs are out: I don’t want to make users npm install, especially not if that runs the risk of conflaiting my tool’s required packages with their dependencies. Generating a single file executable fixes that problem.

Let’s try it for that systems programming use case: see how it interacts with just shell commands, then prototype out how much we can grow from there.

What goes good with buns? Shells

The shell API in Bun is interesting in its simplicity:

import { $ } from "bun";

await $`ls`

When executing this program the output of the given command - ls in this case - will be printed to stdout.

Depending on your use case this might be very useful: maybe I am a fancy wrapper over ls, configuring the command the way I want to call it, and want the result to be shown.

If I am doing one command out of many and don’t want this behavoir, I can call .quiet()

import { $ } from "bun"
await $`ls`.quiet()

If we want to interact with the results - the script’s output - in Javascript land, there’s a few methods we can use, the simpliest being .text().

const response = await $`ls`.text()
console.log(response)

First script: how many files in a directory

The object generated by $`COMMAND` is a Shell Promise, and has a few useful methods on it, including .json() and .lines() and of course .text() as already seen.

.lines() is an async generator function and, as such, is mildly awkward to use.

Create a helper function to make it easier.

async function getLines(shellResponse): [string] {
    const output = []
    // lines is an async iterator so mildly hard to handle
    for await (let line of shellResponse.lines()) {
        output.push(line.trim())
        // ends in a newline. Trim to avoid that
    }

    return output
}

Now:

const output = await getLines($`ls`)
console.log(output.length)

We turned our shell output lines into an array of Javascript lines, then counted the length of that in Javascript land

But wait, ls is a Unix thing…

Bun has implemented cross-platform versions of common Unix utilities, like ls or pwd or basename.

Using those cross-platform versions, let’s get the full paths to the files in our folder:

const pwd = await $`pwd`.text()
const output = await getLines($`ls`)
const fileNames = output.map( (entry) => `${pwd}/${entry}`)
console.log(fileNames)

This gives us a full paths in a cross-platform way!

There’s an open Github issue with all the commands they want to add to the Bun shell, for this cross-platform support. Bash scripts I write tend to lean into third party tools pretty quickly, and any script I share with coworkers I’d want to clean up: replace some shell script hackery with Javascript tools where I can.

Towards that end: we’ve seen how to get data out of Bun, how do we put data into a shell pipeline?

Feeding data into the shell

We can take Javascript objects (well, Buffer and native type Arrays, plus Response objects) and send them into shell land, with piping.

Here’s an example straight from the docs, but I kinda like it

const response = new Response("hello i am a response body");
const result = await $`cat < ${response} | wc -w`.text();

Note that piping is supported by the shell library, even (apparently) on systems that don’t support piping natively (Windows).

We’re done coding! Time to package this for distribution!

With this shell module I can quickly iterate with Unix tools I have on my machine, then - when situations grow - I can add abilities onto the script. Maybe even replace hacky solutions (calling curl and piping to jq, for example) with more Javascript native solutions (fetch and lenses).

Regardless how hacky the solution is, I want an easy way to distribute my work: one that doesn’t include “Now, do a bun install“.

Thankfully, Bun can create standalone applications!

bun build --compile myentrypoint.ts --outfile=my_app

Bun even supports cross compilation!

For me, targeting MacOS, this generated a 55MB-ish binary (I did pull in one dependency, Ramba).

Conclusion

The Bun shell looks like an interesting way to build easily distributed, cross-platform, tools for development work: allowing quick pipelines to be built up, then layered on top of (or replaced) with Javascript tools.