An Architecture Notebook

Sometimes changes in a codebase are straight forward, sometimes they require a bit of research and investigation before some kind of architecture or design document is created.

GToolkit is often where my architecture notebook is created, but lately the power of Org Mode compels me - sometimes providing better integration with the tools I need.

This month I’ve been comparing (and timing) API endpoints, validating and understanding database schemas, writing entire proof of concept programs in Org Mode, and drawing diagrams using PlantUML: all from my Org Mode document, with my plain text architecture thoughts.

I keep these Org Mode documents in a folder next to the appropriate codebase: I’ve long ago set a Git global ignore to ignore anything in a _PERSONAL folder. I may have a few org files in each codebase: one file per complicated feature, and a file with just hurl requests.

This blog entry has my collection of tips of how to create such a thing with Org Mode.

Basic Emacs Tips: Helpful File Variables with Org

Even after 5 years of using Emacs I’m still learning how to use this editor. I’m finding more and more variables I need to specify in my org mode documents Preferences about your Org Mode content can be kept in the file variables section of a file

# -*- org-image-actual-width: nil; org-imenu-depth: 5 -*-

However, it seems some variables can not be set in this (slightly normal) Emacs way, but must be set in a special Org Mode way.

For example, to turn on inline images in Org Mode - (spoilers), add this anywhere in the buffer.

#+STARTUP: inlineimages

See Org Mode Documentation on In-Buffer Settings for more on these Org Mode specific settings.

Org Mode and Org Babel

Org Mode’s Babel set of packages give us the ability to execute programming blocks right in an Org Mode document.

But, as a refresher in case you didn’t read my blog entry on Org Mode And Hurl:

#+BEGIN_SRC sh
ls
#+END_SRC

In an executable block of code: in an Org Mode document I can C-c C-c and execute that line of (in this case) shell, seeing the results directly below the code block.

The shebang variable

While there’s a huge number of languages built into Org Babel, there is (on Unix!) a very easy way to call any language: use the shebang variable

Let’s use Racket here, a language so different we can’t possibly get confused:

#+BEGIN_SRC sh :shebang #!/Applications/Racket_v8.9/bin/racket
#lang racket/base
(+ 1 1)
#+END_SRC

Tweaking your output type

Org Mode will by default try to put the results of your evaluation into a table, but that may not be right (and may be very slow for large data sets).

#+begin_src sh :results output raw

or

#begin_src sh :results output verbatim

Verbatim kind of quotes the results, where raw does not. (However, raw mode means M-x org-babel-remove-result won’t work either)

#+begin_src sh :results output raw
echo "hi"
#+end_src

#+RESULTS:
hi

#+begin_src sh :results verbatim
echo "hi"
#+end_src

#+RESULTS:
: hi

Returning Org Babel Blocks from Your Org Babel Blocks

You may want your block to return an org mode block itself. (I find helps in some cases when I’m outputting a lot of data, like JSON API responses)

:results output code is your friend here.

If you want the babel block to be marked as a particular language, json for example: :results output code :wrap src json.

will return your results (with proper syntax coloring!) as such:

#+RESULTS:
#+begin_src json

NOTE: If you are overriding the shebang option, especially to make sh actually execute another language (as we did above) you want to use :results output, not :results value, as that tries to call the codeblock as a function (apparently)… which may try to create some very invalid code.

NOTE #2: This works great with Hurl-Mode: outputting a JSON block gives us syntax highlighting in that little block.

Org Mode And Deno

Deno

I haven’t talked about Deno before, which is my mistake. Deno has three interesting things to offer the world:

1. A Rust library for executing Javascript/Typescript (see my previous writing on that library)
2. From day one: ways to pull in modules without using Node.js’s package.json file
3. You explicitly allow operations like file and network access.

Item two here makes it easier to distribute Deno scripts: where a Node.js program would likely need the source script and package.json to pull in external dependencies, Deno will do that dynamically at script runtime. This ability also making it excellent for quick one-off script creation.

(Deno 2.3 allows one to opt-into a more traditionally node_modules paradigm, which is more appealing to me if I was running server applications.)

A sample Deno app

import * as R from "npm:ramda"

R.forEach( (i) => console.log(i), [1, 2, 3, 4])

The first line here will download ramda from the NPM registry, then run the program!

#+BEGIN_SRC bash :shebang "#!//Users/ryan.wilcox/bin/deno run --allow-net --allow-import --allow-write" :results output

Org Babel and Deno Together

Although Org Mode doesn’t explicitly support Deno we can use the shebang variable to call it, as we did with our Racket script

#+BEGIN_SRC sh :shebang "#!/user/bin/env deno --allow-net --allow-import" :results output
import * as R from "npm:ramda"

R.forEach( (i) => console.log(i), [1, 2, 3, 4])

#+END_SRC

In my exploration work I was building up a small Deno program iteratively: I made one request to an API endpoint, looked at the results, realized for each API result I would need to call another API endpoint, so wrote some code to do that, building up to my final program. All being able to execute my program quickly, and I was able to jump to other parts of my analysis (some hurl requests) in the same document.

Org Mode and PlantUML

This is pretty straightforward, but looks awesome: plantuml-mode.

#+begin_src plantuml  :file /tmp/foo.png
@startuml
class Animal {
  +String name
}

class Dog {
  +bark()
}


Animal <|-- Dog
@enduml
#+end_src

This will output an org link to the created file on disk. You can render that file on disk with M-x org-toggle-inline-images OR by setting #+STARTUP: inlineimages.

Thanks very much to the handy M-x plantuml-download-jar for installing PlantUML on my computer, without me having to manually set something. (Or trying: I did end up manually having to set org-plantuml-jar-path for some reason).

Just Normal Image Things

You could import a normal image too, something created outside of Org Mode.

Embed an image

#+ATTR_ORG: :width 700px
[[file:related_file.png]]

Here we tell Org that our image is 700px wide.

However, Org has some settings we may need to turn off first.

By default, Org mode displays inline images according to their actual width, but no wider than fill-column characters.

(Documentation)

To fix this we have org-image-actual-width:

When set to nil, try to get the width from an ‘#+ATTR.*’ keyword and fall back on the original width or org-image-max-width if none is found.

Which, in fact, you saw set as a file global in the first section of this blog entry.

# -*- org-image-actual-width: nil; org-imenu-depth: 5 -*-

With org-image-actual-width set you may still need to set org-image-max-width: either setting it to a max value or itself to nil (which does not limit the width).

A good default for you might be # -*- org-image-actual-width: nil; org-image-max-width: nil -*- (use the actual width, to the MAX) or something like # -*- org-image-actual-width: nil; org-image-max-width: 800 -*- (800px is not too big, while being big enough).

Org Mode and Databases

I know GToolkit has fantastic support for databases (in particular Postgres). Compared to that environment executing SQL in Org Babel felt a bit awkward, but in previous gigs I was making GToolkit’s database support sing so there’s a high bar.

Simple select queries are where Org Babel’s default output-to-table support shines: I got data results that looked like tables, easy to read.

well of course you can execute data SQL, but you can also DESCRIBE a table

#+BEGIN_SRC sql :engine mysql
  DESCRIBE `some_table`;
#+END_SRC

I do like all my tools in my editor, and if I’m building up an architecture notebook I want all the pieces of my analysis to be in the same place.

Conclusion

In creating an architecture notebook or design document, I want to execute real results from various running systems: executing queries against real APIs, or executing real code. From there I make my conclusions about how the system’s current design informs the desired design. Theoretically Org Mode’s ability to export as different file formats could let me easily share these documents with others.

I mostly create the org documents one per ticket, with an additional document named requests.org, filled with hurl requests.

For all that stuff, and staying within my editor, I’m thankful for the Org Mode and Org Babel people!

I do wish GToolkit had some of the abilities around easy interaction with external systems: being able to make a request using hurl or render something using PlantUML. GToolkit’s graphing abilities are awesome, but they align around using Smalltalk, not a higher level DSL, to make graphs. (Now, for this extra effort you get zoomable, scalable vector based graphics in an interactive UI. Sometimes I want speed of graph creation, not fancy graph display.)