Wilcox Development Solutions Blog

Intermediate Functional Programming Patterns in Javascript

June 05, 2023

Intro

Functional programming means different things to different people. Personally I’ve come to the - somewhat incremental but somewhat not - kinds of functional programming:

  1. Declarative data transformations: someArray.map( (x) => x+ 1): much better than a for loop, especially as you add primitives like filter or select into the mix. Build on the provided blocks!

  2. Traversal systems. Why write all that code to get to the deeply nested part of the data, you can write a descriptive path?

  3. Declarative program logic: Pattern matching, expressions that return results (let res = if (...)), etc

  4. Lenses: even better traversal systems!

  5. Result/Option types: avoid that nasty if statement all together with someResult.getOrDefault(‘foo’).

  6. Mutable state is bad, mmmkay?

  7. Monads

  8. ??? type theory??? Functors and effects, etc etc

  9. Passing around global, read only, state.

There’s many, many, blog articles about the first type or stage of functional programming: using functional tools like collect, filter, select, reduce and (if you have to) forEach to avoid boilerplate and off by one errors.

We’re going to skip right over talking about those techniques.

I’ve been doing weird science on the second two: what’s the state of the art for object traversal in Javascript in 2023, and how can we use some of these declarative program logic techniques in our Javascript?

Let’s see!

Deep Selection / Traversal Systems

Complex and deeply nestled objects are an annoying source of errors and complicated code. Let’s take the following object, for example:

const theUser = {
  name: "Ryan",
  github: "https://github.com/rwilcox",
  pets: [
    {
       name: "Leo",
       kind: "cat",
       gender: "male"
    },
    {
       name: "Robin",
       kind: "cat",
       gender: "male"
    },
    {
       name: "Cosmo",
       kind: "dog",
       gender: "male"
    }
  ]
}

How do I make a list of every pet’s name? Let’s use the ECMAScript 2020 null coalescing (”??”) operator to make this work even if the user has no pets:

const petNames = []
for (let pet of (theUser.pets ?? [])) {
  petNames.push(pet.name)
}

This is not great, even for this relatively simple example. It gets worse if the pets are two or three levels deeper in the object!

If we had an object, not an array, we could use ECMAScript 2020’s other null coalescing operator: theUser.pets?.cosmo?.name would return us Cosmo’s name or undefined (and not potentially a crash if you don’t have a pet named Cosmo).

But, alas, we have an array of objects here, which we want to traverse - kind of awkward.

We need a strategy for traversal of our object graph, allowing us to select deeply nestled objects: ideally by declaratively describing where, not how to get the data we want.

Groovy (my writings on it) has a neat idea of GPath expressions: an expression path we can use to reference deeply nestled properties, including through arrays.

With GPaths we could solve our problem easily: theUser.pets.*.name

Can we reproduce this in Javascript?

I’ve previously written extensions on top of lodash to mimic this syntax, but I’ve recently discovered object-scan:

import { objectScan } from ‘object-scan’

let petsNames = objectScan([[‘pets’, ‘[*]’, ‘name’]], { rtn: ‘value’ })(theUser)

Gets us [ ‘Leo’, ‘Robin’, ‘Cosmo’]

Object Scan can give us not just values that match, but keys that match or object paths too.

Interesting note: Object Scan it doesn’t seem to work for ES6 getter methods. With this rtn value syntax it doesn’t let us traverse into the value and return multiple fields, but object-scan is flexible enough you should be able to do it, or just stop a level up and map it yourself.)

This is still relatively ugly, compared to the elegant Groovy GPath we had above, however it’s a relatively simple matter now of adding this as easy data accessor methods to a class (keeping in mind that objects couple data with operations that work, get or set that state!)

class Person {
  constructor() {
    this.pets = {}
    this.getPetNames = objectScan([[‘pets’, ‘[*]’, ‘name’]], { rtn: ‘value’ }).bind(null, this)
    // ^^ remap this into the first parameter of this function. Look ma, little boilerplate!
  }

  /// get any attribute on pet, in case you want more than name
  getPetAttribute(attrName) {
    Return objectScan([[‘pets’, ‘[*]’, attrName]], { rtn: ‘value’ })(this)
  }
}

Use it like so: const p = new Person(); p.getPetnames().

Deep Traversal of JSON objects

Objects serialized over JSON - either via REST or even GraphQL - sometimes return large, deeply nestled structures. However, these may be hard to work with when you really only care about certain data bits returned and don’t want to care about the structure of the returned documented.

(While GraphQL is awesome for letting you tell the query what data to return, it’s not as good if you want to tell the server the shape of the object to return. So we get a result back and transform it to an object more suitable to the rest of our program.

Let’s see this in action with GraphQL and the Star Wars GraphQL API. I want to create a table of every Star Wars planet, with population, that has appeared in the films.

I can do that easily with the following GraphQL query against the Star Wars API:

query ExampleQuery {
  allFilms {
    films {
      planetConnection {
        planets {
          name
          population
        }
      }
    }
  }
}

The awesome: query and I’m able to get all the information I need!

Now comes the kinda bad: while I’m interested in just planets and populations, a bunch of other irrelevant structure comes along for the ride:

{
  "data": {
    "allFilms": {
      "films": [
        {
          "planetConnection": {
            "planets": [
              {
                "name": "Fobar",
                "population": 200000
              },
              {
                "name": "Bazbar",
                "population": 2000000000
              },
              {
                "name": "Jarvar IV",
                "population": 1000
              }
            ]
          }
        },
        {
          "planetConnection": {
            "planets": [
              {
                "name": "Zero",
                "population": null
              },
              {
                "name": "Nope",
                "population": null
              }
            ]
          }
        }
      ]
    }
  }
}

I can structure my query, but can’t structure my output: I want a list of planets and populations, but I get query specific object hierarchy too. (Sorted by movie, which is relevant to my query, but irrelevant to what I want to display: a list of planets.

We need to traverse and transform this data: preferably in a declarative way!

In the past I’ve used node-json-transform for this task. Let’s give it a shot:


const { transform } = require("node-json-transform")

const swjson = ...

const res = transform(swjson, {item: 'data.allFilms.films'})

That line of code returns us a list of planetCompositionObjects, as we see below:

  [
	{
      planetConnection: {
	        planets: [
			        { name: 'Zero', population: null },
			        { name: 'Nope', population: null }
	        ]
	 }
	 }
]

We tell node-json-transform that our item is inside the path data.allFilms.films, which does cut a bit of the object hierarchy out of our GraphQL response.

At this point it’s important to note that the item object you’re constructing is the shape of the output object you want. In some in of my projects I have something like the following:

transform(someObj, {item:
	name: ‘response.name’,
	currentId: ‘response.current.id’,
	hash: ‘response.current.lastAction.id.uuid’
	})

effectivly remapping that complex and deeply nestled object into something flatter and easier to use. Even better? If an parts of the object traversal evaluate to null then the result is null: I don’t have to check or account for the null!

In this example, our result isn’t quite right: we want to flatten that list of planets quite a bit: for each item in the films array we want to iterate into its planetConnection array and list the planets - ideally with minimal code!

In addition to the static list of strings we can run operations on found/created field names. In fact these operations can be recursive: calling additional node-transform-json operations!!

const res = transform(json_object,
   {item: {planets: 'data.allFilms.films'},
    operate: [
		{ run: (x) => transform(x, {item: 'planetConnection.planets'}),
		  on: 'planets'
	    }
		]})

The on keyword is interesting here: given names of your output object it will run the operations specified on it. Thus, since we named our output object planets we use that name (even though that’s not what the API response calls it!)

Returns us:

{
  planets: [
      [
		  { name: 'Fobar', population: 200000 },
		  { name: 'Bazbar', population: 2000000000 },
		  { name: 'Jarvar IV', population: 1000 }
	  ],
	  [
		  { name: 'Zero', population: null },
		  { name: 'Nope', population: null }
	  ]
	]
}

We’re getting much closer: we have an object with planets, but to the consumer’s perspective they are oddly grouped: I want only one list of planets!

We can get very clever with operation here, embedding a run operation inside a run operation to get our results.


const output = []
const res = transform(json_object, {item: {planets: 'data.allFilms.films'},
      operate: [
      {
		  run: (x) => {
			  transform(x,
				  {
					  item: {pz: 'planetConnection.planets'},
					  operate: [
						  {
							  run: (y) => output.push(...y),
							  on: 'pz'
						  }
					  ]
				  }
			  )
			  return output
		  },
		  on: 'planets'
      }
	  ]
  }
)

Here we have a transform operation called inside an operate function. What this inner transform function does is, for planetConnections.planets renames that pz, shoves the contents of pz (aka the planets array) into some temporary state we have laying around, then returns that instead of the result of the transform.

It’s a bit ugly, but shows that the contents of the run keywords don’t have to be transform statements: they can be whatever JS you want.

But we have our result:


  planets: [
    { name: 'Fobar', population: 200000 },
    { name: 'Bazbar', population: 2000000000 },
    { name: 'Jarvar IV', population: 1000 },
    { name: 'Zero', population: null },
    { name: 'Nope', population: null }
  ]
}

For our simple experiments here this seems like only marginally a win, but with deeply nestled objects it’s an excellent way to get the parts of the object you need closer to where you need to access them, in a declarative way!

Doing this task with Object Scan

If we can only focus on one thing from this data response - planet names, objectScan can do this just as well (and with slightly more understandable code)

objectScan([['data', 'allFilms', 'films', '[*]', 'planetConnection', 'planets', '[*]']], {rtn: 'value'})(json_object)

This returns us a list of planets and populations, flat like we want it.

However, it gives us no options to return other bits of data: we might want all the planets, and the names of the onscreen starships, which we then act upon later. With node-data-transform this is another


transform(json_object, {item: {
	planets: 'data.allFilms.films',
	starships: 'data.allFilms.films.starships'
    }})

Or perhaps you want planets, starships, and space stations, all in one big object for later use: all from one query (thanks GraphQL) and in an easy to use single object, with one pass through the complex object (thanks node-data-transform!!

It doesn’t have to be complex objects: in a recent script I was just pulling simple data fields into easier places (user.name.first.string became first_name in my output document, for example).

Declarative Program Logic

Earlier in the year I read a blog entry that proposed a simple construct for Javascript programs:


async function run( x => await x() )`

This bit of ES6 syntax mirrors - in ES5 - what would be called an Immediately Invoked Function Expression (simplest possible IIFE included here):


(function() {

})()

In fact, Coffeescript has something like this, called the do keyword. As far as I can see this keyword isn’t documented anywhere, but here’s an example:


do () ->
  "foo"

// Javascript output from the compiler
(function() {
  return "foo";
})();

The do keyword is mentioned and used heavily in the early parts of Coffeescript Ristretto.

The advantage of run, do / an IIFE, in this case, is that it gives us two advantages:

  1. a single statement we can use in places when the language only does expect a single statement.
  2. allows us to control variable scope (this blog entry ignores this ability of IIFEs)
  3. A function, unlike a keyword in Javascript, can return a value.

That last one is intriguing.

Let’s look at a practical use case:

The modern practice in Javascript / Typescript is to prefer const variables: this means the variable can only be assigned once (but the data or object can be internally mutated).

In complex code sometimes I struggle with this restriction if the variable needs to be assigned one thing in one condition, and another value in another condition:


if (true) {
  const h = "hi"
} else {
  const h = "bye"
}

console.log(`h is ${h}`)

Except h is only valid inside the expression if or else blocks, so the console.log line will fail.

Sometimes this restriction forces me to rethink my code, and sometimes it results in more elegant ways of doing things. However, sometimes it means I fall back to a ternary operator (which can return a value):

const h = true == true ? "hi" : "bye"

But the thing about ternary operator: there’s only one statement allowed in either of those slots in the expression.

Turns out, we can fix that with our run function:


const h = true == true ? run( async () => {
  console.log("hello world")
  return "hi"
  })
: run( async () => {
  console.log("huh?")
  return "bye"
 })

This works because we do only have one statement: it just happens to be a statement in which we’ve hidden a bunch of other statements.

This is either the ugliest or most elegant thing ever, I’m not sure.

Refactoring it a bit to put the if statement inside the run function may make it more elegant, and the result is still the same:


const h = run( () => {
  if (true) {
    return "hi"
  } else {
    return "bye"
  }
})

Or, to expand that back:

const h = (function () {
  if (true) {
    return "hi"
  } else {
    return "bye"
  }
})()

We assign h to be the result of running (the function object)… in which we return “hi” or “bye”, depending.

Reproducing Lisp’s cond statement

Lisps have a cond statement that is half way between modern pattern matching and a more powerful select statement.

Here’s the documentation for cond, in my favorite Lisp, Racket.

JavaScript’s select statement only matches if the value in the case branch equals the value at the start of the select statement. Which is mildly limiting if you want to write a series of complex expressions.

Pattern matching lets us take the value at the start of the pattern match and compare it to a wide variety of tests (no just value equality).

cond is a collection of (compactly written!) if statements that are executed until one finally returns true.

With this definition my PatternMatcher for Smalltalk is probably more a “cond but for Smalltalk” than pattern matching, but that’s from another article (Functional Programming Patterns in Smalltalk)

We could reproduce a cond function here in Javascript-land:

/**
  @description a cond statement, but for Javascript.
  @param conditionArray an array where each element of the array is a tuple: the first item a Boolean (the condition), and the second item a function object which will be called if the condition is true.

*/
async function cond(conditionArray) {
    for (let current of conditionArray ) {
        const [comp, _do] = current

        if (comp) {
            return await _do()
        }
    }
}

and using it


const res = cond([
    [(((a ?? 43) == 42) && b === "goodbye"),       async () => console.log("a == 42 and b === goodbye")],
    [(((new Date()).getFullYear() == 2023) && a),  async () => console.log("it’s 2023 and a is truthy")],
])

In this example you see a bunch of conditions that may lead to the first scenario:

  • a (or its default, 43) == 42 AND the variable b === goodbye: do this thing
  • it’s 2023 and a exists at all: do this other thing

We’ve mostly made a table of this code - in fact, you could say we’ve declared a table of this code, vs a series of awkward if statements.

In scenarios of increasing complexity this could be very useful. Let’s list semi-related conditions that, never-the-less, if we find a match we should stop evaluating other potential matches:

  • if a is undefined, or null, do one thing
  • if a > 10 and < 21 do another
  • if a > 23 and it’s 1990, do this other thing
  • if a > 23, and it’s 2023, do something
  • true <— a catch all default case, do a boogie

(Replace this with your favorite bit of complex domain logic in your app).

Conclusion

While other languages inspire some of these tools, JavaScript’s multi-paradigm abilities give us the opportunity to go a bit further that the commonly used tools: declarative object traversal and replacement techniques for some of JavaScript’s less functionally enlightened operators!