Playing with GToolkit’s Lepiter Notebook and its support for Node.js cells

One of the components of GToolkit is Lepiter, a knowledge base excutable notebook. Each notebook page has many cells, and these cells can be lines of text, blocks of code, pictures, interfaces to Jenkins, Wardley Maps, (now) GraphQL queries, and many other things.

One of the more interesting things is the ability to call certain non Smalltalk languages: right now Python and Javascript via Node.js.

Executable blocks help you explore whatever you might be investigating, or for creating live documentation.

Today I started playing with the abilities of the Node.js cells. The Lepiter documentation does an OK, but not great, job explaining how Lepiter lets you execute Node code in cells in a page.

Here’s some more info on it (and a fix…)

Setup

Per the built-in documentation you need to set the location of Nodejs (if it’s not already on the path), and where the shim project is.

So create a few Pharo cells in your notebook…

"Set the location of the nodejs executable"
thisSnippet database properties jsLinkSettings
	nodejsPath: ('/opt/local/bin/node' asFileReference).


JSLinkPharoNodejsProcess nodejsPath: '/opt/local/bin/node' asFileReference.

If GToolkit can’t find your Node install it will throw an error with some nice commented text about how to set it up, except that’s wrong. Took me a minute to figure out the correct way…

"Set the JavaScript runtime directory"
thisSnippet database properties jsLinkSettings
	directory: '{dbParentDirectory}/js' asFileReference

The JSLink magic uses an Express app as a shim between the Pharo world and the Node.js world. You can not use an arbitrary Node app and use this like a fancy runtime workbench: this has a bunch of special code in it.

You could, of course, create seperate shim apps for various needs, if you have vasty different experiments you don’t want to contaminate each other… but I can’t just point Lepiter into one of my own Node projects and expect it to work

You can see this by inspecting the package.json for the JSlink application we’ve set

JSLinkApplication uniqueInstance settings workingDirectory / 'package.json'

Done with setup, let’s see it in action

Javascript, in a cell, in the notebook:

1 + 1

Result is 2. Neat

Diggin in past the built-in documentation…

That’s mostly as far as the docs go, but I want to dive deeper…

So can I define a Smalltalk variable in my notebook and use it in some Javascript code?

Smalltalk, in a cell, in the notebook:

myName := 'Ryan'

Javascript, in a cell, in the notebook:

  myName + " Wilcox"

The result is “Ryan Wilcox”. Neat!

Does it work the other way? Defining variables in my Javascript code and grabbing them from Pharo? Yes, except they have to be vars, not let or const (we WANT globally scoped variables!!!)

Javascript, in a cell, in the notebook:

  var myVar = "hello"
  myVar

Smalltalk, in a cell, in the notebook:

myVar , ' a variable from Javascript that is now in Pharo'

If we inspected this cell we would see “hello a variable from Javascript that is now in Pharo”

Note: the last line in the cell needs to be an operation, not a declaration. For example, when we declared myVar the last line gave us the value of the variable. You don’t have to do that, but the last line returning something is important because that will be the inspected value of the cell in the notebook.

But sometimes we’re executing cells just for the side effects, which is certainly evil and not best practice but sometimes ya gotta do what you gotta do.

More Advanced Stuff

GToolkit / Lepiter lets us do some advanced stuff, with some limits (just not ES modules - standard imports). Yes await works as you expect, which is useful because Pharo expects an answer, not a promise

Let’s see it in action:

const fs = require('fs/promises')
const res = await fs.stat("/Users/rwilcox/somefile.txt")
res.ctime

When inspected this gives us the creation time. Thanks to await/async, even though this code technically has a promise in it we don’t have to care.

(Interesting fact: the Node shim app automatically wraps our code in an async function, thus how this works at all)

Can we define our own functions? A Javascript cell

function myNewThing() {
  return "hello"
}

""

Can I use it elsewhere, like a seperate Javascript cell in my notebook?

  myNewThing()

ERROR. Nope, can’t do that.

What if we get evil?

var myNewFunctionVar = () => {
  return "hello"
}
""

Now, in a seperate cell:

  myNewFunctionVar()

Result: Success!

I’m actually not ure how much is this serialized function going back and forth, through Lepiter, or if the Node scripts are what retains state. But this is interesting

Conclusion

I was hoping the Lepiter notebook would let me interact with my own Node programs, calling methods directly from a notebook interface. While I can’t do that, these pieces would probably let me build up tools in Javascript if I need them for analysis. (For example, libraries only availible in JS).

The ability of both GToolkit and Node to support new Javascript features makes this powerful and useful for more than just toy programs. I’m excited for what this means about my own analysis work, and this is great work by the GToolkit team!