The quest for request makers

As a backend developer I’m often looking to make some one-off requests to test new API endpoints, or to explore some returned data. In the past I’ve used curl directly, or Postman. Postman turning into an enterprise collaboration tool is fine, (or even a cli runner) but in general its gotten to feel too much for simple cases.

I want something simple, reusable, that I can easily commit into Git, that I can pass parameters into easily, and that I can use with other tools I have laying around (let’s be honest: jq).

Meanwhile, my use of Emacs as my day to day driver continues, and I’d like to send requests from it, ideally without demanding that others on my team use the editor ”Exclusively used by Middle Age Computer Scientists”.

In my travels I found two things: hurl and - you guessed it - Org Mode’s Babel package.

NOTE: I’m an org newbie, but some of this stuff isn’t well documented on the web in general, thus my efforts in documenting it here. Excuse any obvious Org mistakes.

Hurl

Hurl is a cross-platform abstraction CLI over CURL. (In fact, my favorite kind of abstraction layer: you can output curl commands from your Hurl configuration!)

Hurl can do more than just sending single requests, it can play a part in your test pipelines, but we’ll focus on the single request cases.

Org Mode Babel

Org Mode is the massively popular, text based, knowledge repository system. Because its built on Emacs it’s also extensible. One of the excellent things you can do with your files in Orgmode is add blocks of executable source code in any language (this is Org Babel). So if you want your document to execute some Ruby and spit out the result, cool, awesome, go for it.

Hurl has an org bable mode in hurl-mode

Hurl primer

Let’s have a quick overview on putting together a request with Hurl. With Hurl you create a file with the hurl request format (quickly documented below) and tell Hurl to execute that request.

Simple Hurl examples

GET https://swapi.dev/api/people/1/
# this is a comment

HTTP/2 200

This is valid HURL and returns us information about everyone’s favorite moisture farmer.

The HURL Request format

The format for making requests in Hurl is sometimes not explicit, but here’s the snippet I use:

GET https://example.com
# GET/POST/ETC URL

# START HEADER SECTION
Content-Type: application/json
# END HEADER SECTION

# START SECTIONS: [Options], [Cookies], etc
[Options]
very-verbose: true
variable: a-variable=some-value
# END SECTIONS

# START BODY
{
}
# END BODY
# RESPOSE CODE ASSERTION below! vvv
HTTP 200
# START [Captures]/[Assert] BLOCK

# END  [Captures]/[Assert] BLOCK

There are two interesting parts of this format.

The first is that Hurl allows you to specify variables, which are later referenced. via the {{a-variable}} syntax.

The second is that assertions or capture groups require a response code assertion. It’s easy to trip up here.

Org mode primer

Org Mode is its own markup language, most of which we’ll skip here: org babel lets you put snippets of any supported language into your org mode document and execute it. Let’s see a simple shell command:

#+BEGIN_SRC shell
ls
#+END_SRC

If you execute this in an org mode file you’ll get a directory listing right there in your org document.

Org Babel can do some interesting - and not well documented - stuff.

Output declarations

Maybe you don’t want your command’s output to go to your org document, but to a file instead.

#+BEGIN_SRC shell :results output file :file /tmp/ls.txt
ls
#+END_SRC

the :results output file :file directs the output to the specified file (it’ll even add a little link to said file in your org mode document).

Dependencies

You can have one org babel block depend on another like so:

#+NAME: generatefile
#+BEGIN_SRC shell :results output file :file /tmp/ls.txt
ls
#+END_SRC

#+BEGIN_SRC shell :var forceDependency=generatefile
cat /tmp/ls.txt
#+END_SRC

In the second block the forceDependency variable references the generatefile named block. If you run the second block it will automatically run the first block.

Hurl in org mode

Usin Hurl in Org Mode lets us make use of those tricks above!

Usage of hurl in Org Babel looks like this:

#+BEGIN_SRC hurl
GET https://swapi.dev/api/people/1/

HTTP/2 200
#+END_SRC

Very easy, same hurl syntax as above.

Org Variables Used in Hurl

Hurl itself lets you pass in variables. Let’s say we have two different environments for our API: test.swapi.dev and swapi.dev. We want to make it easy to send requests to whatever one we’re working on at the time.

#+BEGIN_SRC hurl :var hostname="https://swapi.dev"
GET {{hostname}}/api/people/1
#+END_SRC

Org Babel variables are passed into Hurl and referenced via Hurl’s variable syntax.

Neat.

Iin some cases - I ran into this especially when passing in JWT tokens - the way Hurl Mode passes variables to Hurl may be problematic. For those cases I create and assign a Hurl variable in the Options block in the Hurl config.

jq

We can easily integrate JQ into our org mode documents, with jq-mode.

Let’s use Hurl + JQ in our org mode file to return the name of our future moisture farmer.

#+NAME: getPeople
#+BEGIN_SRC hurl :results output file :file /tmp/people_one.json
GET https://swapi.dev/api/people/1/
HTTP/2 200
#+END_SRC

#+RESULTS: getPeople
[[file:/tmp/people_one.json]]

#+NAME: peopleName
#+begin_src jq :in-file /tmp/people_one.json :var forceDependency=getPeople
.name
#+end_src

We have hurl writing the results to a file, then jq mode picking up that file (using dependencies to make sure it’s always generated), then executing a jq command to get the name.

All we need to do is execute the peopleName block: it will execute the hurl command then run the result through jq.

Conclusion

Hurl gives us an excellent abstraction for making HTTP(S) requests on the CLI. Org Babel makes this even better by giving us an excellent Emacs friendly way to execute those commands at arbitrary times: while still being able to share your request library with colleagues.