NOTE: This started as an OCaml type-system thing and ended up as a library design and documentation copy UX thing. So caveat emptor.
I’ve found that some of the best displays of mastery of OCaml are also some of the worst examples from a usability perspective.
To make the discussion more concrete, let’s talk about cmdliner, the command line toolkit by Daniel Bünzli.
Cmdliner is one of those libraries that expect you to know what you’re doing. It gives you a flexible type-level toolkit to describe type-safe routing from a string of command line arguments (like
ls -hal ./*) into actual code.
This is not any different than an HTTP Router library, really, and it is just a function from a string to some status code.
It is also very thoroughly documented, and on a recent thread in the OCaml forums it came out as a popular choice! The thread was “What are some libraries you almost always use?”.
And yet, I have the absolute worst times trying to figure out what to do next, and I always rely on copy-pasta from prior cli’s that I already have working to get things right.
I can only praise Daniel’s command of the OCaml language and type system and for their commitment to building powerful and stable libraries with thorough docs. The OCaml ecosystem is better for it, and as a small part of it, I thank you for your work.
But even Daniel has publicly stated Cmdliner makes him slightly nauseous these days 🥲
So what gives me such a hard time using it? I’ve been thinking about this and I’d like to split the issue into a few slices:
- UX of the Documentation 📚
- DX of an API 🧑🏽💻
Let’s dive in.
UX of the Documentation 📚
I think that as the years went by, I’ve become less interested in sitting down and muddling through walls of text without a clear indication of whether what I need is in there or not. If some text doesn’t communicate to me quickly that the odds of it answering my questions are high, then my brain just skips it.
It’s a sort of foraging behavior, where there’s a curve describing the amount of time it’ll take you to get the maximum amount of resources. After that peak, you’re better off foraging somewhere else. The more you stay, the less and less you’ll get. If we get used to getting what we’re looking for faster, our tolerance and patience seem to go down the drain.
This reminds me of how often squirrels 🐿
switch trees when looking for food. I read about this in The Distracted Mind (Gazzale, Rosen, MIT Press) and it caught my attention. It’s called the Marginal Value Theorem, and as you can see in the graph, the amount of time spent in one particular tree (document) can’t be longer than the amount of time it’ll take you to find a new tree (document) where you could find what you’re looking for. Otherwise, you’re better off looking for food on the next tree!
Perhaps it’s sort of like Diminishing Returns in Economics?
Cmdliner docs seem like a rather tall tree that I know I need to get to the top of…at some point…but there’s so many other trees that are closer to jump to. And they all have exactly what I want right now.
These other trees are random notes that I’ve taken, comments in my own code, other pieces of code that I’ve used or that I know use cmdliner as well.
To really showcase what I mean, I’ve written bsdoc, cactus, oak, servus, twitchboard, reuniverse, and more recently Caramel. All of them use similar cmdliner patterns because it was easier to go back to something I knew worked than it was to go through the Cmdliner documentation to figure out how to write one from scratch. Think of all the things I could’ve learned by now!
I need to say this: the docs are thorough. If you want to learn the tool, the docs will teach you how. But they expect a level of commitment from you that is uncommon from other ecosystems these days. Something about the writing style perhaps?
Recently I read Strategic Writing for UX and I learned a bunch of things about writing copy for humans. The sort of mental process we go through, how an entire brand can be defined by the choice of punctuation that we use, and how to communicate effectively via well-established patterns of interaction.
So when I looked into writing the manual for Caramel, I realized that Developer Docs try to:
- Attract developers by presenting sweet trade-offs
- Help them pick our library instead of other ones
- Onboard developers and get them using our lib
- Help them deepen their knowledge of the lib
- Provide support when things go wrong
- and ultimately turn them into advocates that’ll spread the word and bring in more developers to make the library better
If this sounds a lot like what a Product would do, then you guessed right: that book above is about how to write copy to help people have an excellent time with digital services, and there’s a crazy amount of parallels between onboarding anyone to your new bus ticket app, and getting a developer to successfully use a library.
This wheel comes from the book’s first chapter on trying to help people meet their goals. It fits so damn well with the needs of library docs.
So how does Cmdliner wage in here? The docs have a Basics section, they have some examples, and then you’re left with interface file documentation. Here’s the types, here’s the functions, and here’s a paragraph for each one of them.
There’s no clear structure, and there’s certainly not an inviting narrative to help me go from “What is this” to “I’d like to try it” to actually achieving my objective.
In other words, this is not written with the goals of the users in mind.
Something something Umberto Eco model reader
I think I’ll write more about this in the future as I’m currently reading more about making copy more accessible and directed as well. Maybe we’ll do a breakdown comparison of Rust vs Elixir vs Python docs? I don’t know.
Let me know what you think would be interesting to explore in this space 🙌🏽
DX of the API 🧑🏽💻
Now once you’ve decided to actually try Cmdliner, you have to download it, add it to your project, and use it.
Oh god the using it. Let’s dig right into it.
Cmdliner has what’s known as an applicative interface. This comes from McBride and Petterson’s paper “Applicative Programming with Effects”.
The gist of an applicative is that it’s more powerful than a functor, but less powerful than a monad. Adit Bhargava put it more eloquently in their blog post “Functions, Applicatives, and Monads in Pictures”:
If this means nothing to you, don’t worry. This is the entry-level knowledge required to actually understand how to use cmdliner. It’s pretty damn high if you ask me.
What it does give you in return is the type-level machinery to make absolutely certain that your application will parse all parameters into the right types, and enforce other semantic constraints.
- a flag like
-v can be repeated up to N times
- a param like
--output-dir=./out has to be an existing and reachable dir in the file system
- or a param like
--target=erlang has to be exactly one of a few options (like
ocaml or what have you)
It also gives you help pages in various formats, which is always great to have generated for you.
So what’s the developer experience really like here?
First, you create a
Term is kind of like a description of your program. It says “these are the arguments you want to parse” and “this is the function that should run”. So there’ll be
Arg values that represent our arguments too.
For example, here’s
(* our echo function just prints stuff out *)
let echo x = print_string x
(* this is our argument *)
let msg = Arg.(value & pos 0 string "" & info )
(* this is our program *)
let echo_t = Term.(const echo $ msg)
let () = Term.eval (echo_t, Term.info "echo") |> Term.exit
You can tell from this small snippet that Cmdliner is fairly compact, but what does the
$ mean here?
& is a composition operator, so
msg is really a function that we are defining by stitching together other smaller functions. This is transparent to us.
$ is our applicative operator that will use the result of the argument parser to call our
echo function and print out whatever you’ve written.
So to top the requirement of understanding applicatives, now you also need to get familiar with custom operators, and you need to be comfortable with the declarative style of “computation as values” as well.
Requirements to use it start to get pretty high if you ask me!
A few years ago, a friend introduced me to the Laws of UX. These are useful principles, heuristics, and cognitive biases that can help you build user experiences that are effective. I’ve built a few interfaces myself and relying on these “laws” has almost always helped make them more intuitive.
There’s a couple of principles in there that are interesting to consider when applying them to this API design. Let’s have a look:
Jakob’s Law says that users spend most of their time using other libraries, so users will prefer yours if it works the same as all the other libraries they already know.
Hick’s Law says the time it takes to make a decision increases with the number and complexity of choices.
Miller’s Law says that folks will be able to keep 7±2 things in their working memory. So between 5 and 9, ish.
Alright, I’d love to write about more of them but I think we can quickly do these 3 and have the point made still.
From Jakob’s Law, cmdliner isn’t doing very well. It uses a form of interface that is actually not that common even in OCaml libraries itself (except perhaps other work from Bünzli). If you’re coming into OCaml, and you didn’t come from Haskell, chances are you have never used an Applicative Interface before.
On Hick’s Law, cmdliner aims to be great by allowing you a lot of flexibility when composing the terms and arguments. This only works because of the type-system, and would otherwise be rather messy and painful in an untyped language. But this also means you have so many choices for how to build your terms and arguments that there isn’t an obvious path!
Lastly, by Miller’s Law, I think cmdliner asks me to keep quite a lot of things in my head at the same time:
- the meaning of the custom operators
- the inferred type of the function (until the compiler complains or compiles)
- the positional relevance of the arguments to the argument functions
- the thing I actually want to do with the command line
Which doesn’t leave a lot of room to think of the time, the current sprint, whether my colleagues will be comfortable writing this too, and other super important things like coffee and memes.
Wrapping up, you must by now think that I hate this library.
But I actually really enjoy using Cmdliner. At the peak, you feel like you’re starting to grasp it and by the end, the thing compiles and it Just Works ™️.
Goddammit Leandro you’ve been convincing me it’s such a terrible experience just to tell me you like this!?!? 🤬
Well, what can I say? The Peak-End Rule is real!
But we could absolutely make this significantly better for most people with docs that are written to help them achieve their goals and APIs that are more predictable and less demanding.
That’s all for today folks, thanks for reading and send me an email back telling me what you thought of this issue. I’d love to start having some discussions on Twitter as well around it so feel free to quote-tweet or tag.
Have a great weekend and see you next week!
PS: the Peak-End Rule says the Peak and the End of an experience dominate how good the experience is deemed.