Code is Law

If you thought this article was about Ethereum or Smart Contracts, I’m very sorry. This article is about engineering practices, and about my thinking on — and application of — process.

An example, to start

On my very first day at my new job, I tried to bring the bootcamp and BBL culture into Arcblock. Two weeks later, our bootcamp site looked like this:

Please forgive this bare-bones UI for looking like an orphan nobody loves — we simply haven’t had the energy to make it prettier. My hero, Mr. Chen Daoming, says: simple, but not simplistic. Beneath this interface lies a fairly complex automated pipeline.

To write new slides, an engineer on the team only needs to create a markdown file in our bootcamp repo and follow a certain standard, and suitable slides get generated. For instance:

Again, don’t worry about the plain UI — there will be bread, and everything will be fine. What you should care about is this: the difference between a confidential page and a non-confidential page is nothing more than a predefined CSS class, .confidential.

Once you’ve finished writing the slides, make create-pr (more on this later) automatically bumps the repo’s version and opens a pull request.

At that point, the industrious Travis gets to work: it builds the slides once, confirms nothing is broken, and gives a green light. Then, after the reviewer reviews and merges the PR, the industrious Travis shows up again — it tags the current version, creates a GitHub release, and deploys the slides to an S3 bucket hosting the static website. That’s the picture you saw at the beginning — note the version number in the title.

For the engineers at Arcblock, the UX of this whole pipeline is clear, simple, and just part of day-to-day work. There is no extra, intrusive chore to perform — and wherever things can be simplified for you, the system simplifies them: even chores like manually bumping the version number and creating a pull request take a single command.

If you’re only changing a bit of text and find even that too cumbersome, you can inline-edit (note the label in the top-right corner of the image below):

Then you can edit and create the PR without ever touching the command line:

Once the PR is merged, the site updates itself automatically.

Why call it Code is Law?

The longer I work, the more strongly I feel that automation and ease of use in process are vitally important. Everyone knows security best practices; everyone understands the importance of key rotation. But nobody actually does it — why? Because it’s too cumbersome, there are too many rules to remember, and the process strangles itself. Yet a good process often has already been refined to the point where nothing more can be cut — apply powder and it’s too white, apply rouge and it’s too red. At that point, the only thing we can rely on is code. Through code we can crystallize process; through code we can keep process complete while making its UX as simple and unobtrusive as possible.

At Arcblock, the example above is merely a drop in the ocean. Many of the ways we work are, to the greatest extent possible, a practice of — and an optimization of — Code is Law. Our development process has the following requirements:

• Every change must have a pre-commit hook — code style and static checks, compilation, unit tests, and automatic doc generation must all pass before you can commit
• Every change must go through a pull request; it must be up to date, pass CI, and have at least one review before it can be merged
• Every PR must be squash-merged — no rebase, no merge commits
• Every PR must bump the version; after merging to master there must be a corresponding changelog entry and a GitHub release
• For compiled languages, the build output must be attached to the corresponding GitHub release
• For static websites, the generated output must be placed in the corresponding S3 bucket and be directly accessible

Doing all this by hand would be enormously tedious and would make engineers miserable — naturally, smart engineers would find a way to automate it. But a further problem emerges: the automation mechanics may differ from repo to repo, so working across repos carries learning costs and context-switching costs.

So we asked ourselves a question: can we build a system that makes all repos look as alike as possible and share the same capabilities?

From a product-design perspective, this engineer-facing product should be a CLI, and it should interact like this to help engineers generate a new repo:

$ arcli create:repo
? Repo name: arc-awesome-repo
? Please write concise description: Awesome repo in arcblock
? Choose a template: (Use arrow keys or type to search)
❯ elixir
  general
  nodejs
  python
  web

Why an interactive CLI rather than one with flags? Because we don’t want to add to the engineer’s memory burden — even an engineer using it for the first time can work it out without friction. Of course, if it’s being called from another program, we also support piping the required data as JSON directly into arcli create:repo, in keeping with UNIX culture.

The auto-generated repo has the following layout, with all these files and directories created automatically:

If you look at what make can do inside the generated repo:

$ make <tab>
watch           dep             precommit
all             deploy          release
browse-pr       doc             run
build           init            test
bump-version    install         travis
clean           lint            travis-deploy
create-pr       post-build      travis-init
delete-release  pre-build

You’ll be surprised to find that many foundational tools are ready to use out of the box. You can use make create-pr to open a pull request, make browse-pr to open the current repo’s pull requests in a browser, make bump-version, and so on. By this point, your first Travis build should already have finished and be all green — and you haven’t written a single line of real code yet, but the whole world is already wired up and rolling forward.

Take make create-pr on its own: when a PR is created, an engineer needs to do three things — bump the software version, commit, write the changelog, and go to GitHub to create the PR. make create-pr does all three from the command line:

$ git commit -a -m "my awesome commit"
$ make create-pr
Bump version...
-e  Current version: 0.2.0
-e  Latest commit hash: cca0f8c
-ne  Enter a version number [0.3.0]:
[feature/test 53e617d] bump version
 4 files changed, 9 insertions(+), 3 deletions(-)
remote: Resolving deltas: 100% (4/4), completed with 3 local objects.
To github.com:ArcBlock/arc-test.git
 * [new branch]      feature/test -> feature/test
https://github.com/ArcBlock/arc-test/pull/1

If your project needs prerequisite software, put it under init; other users can download the repo and just run make init to produce a fully working development environment, instead of reading page after page of “how to install”.

If the project needs to be deployed by Travis, put that under travis-deploy, and .travis.yml will reference that make target.

Any engineer who has learned how to use one repo can use every other repo exactly the same way — truly learn once, run everywhere.

For engineers, the convoluted process above boils down to a few commands:

$ arcli init:repo # First time only; afterwards it calls `make init` in the current repo
$ arcli start:story # Pick a story to work on - automatically runs git checkout -b feature/<story-id>/<title>
$ git commit
$ arcli create:pr # Generate a PR, along with a commit that bumps the version and updates changelog.md

This makes the process dramatically more usable.

Using code to implement process has the following benefits:

• reuse: an implementation of a process can be reused elsewhere
• compose: processes can be composed together
• compile: a process can be transformed from one structure into another
• configure: processes are configurable — can be lax or strict, can be cute or commanding
• extend: processes can be inherited and extended
• debug: processes can be debugged with ease
• test: processes can be tested
• version: processes have their own version numbers
• document: processes have documentation living right alongside the code

And of course, the greatest benefit of code: rapid iteration. The moment I spot a problem, I can fix it and send a pull request, drawing my colleagues into the discussion. A few days ago we finished laying out the interview process, and after yesterday’s discussion, some PRs went out:

Now stop for a moment and think carefully about how terrifying this iteration speed really is.

Think about it again.

We often say we shouldn’t let process ossify, that everyone should be able to participate and everyone should be able to modify it. But when your company’s processes are sheets of paper taped to the wall, PDFs sitting in inboxes, how exactly do you participate? How do you modify them? File feedback with HR? Which HR? Which HR do I know who even owns this? If I want to change it, how do I change it? What is the process for changing the process? With all these questions in front of you, you give up. But a GitHub repo just sits there: as long as you know markdown, you can make the change in minutes; a few minutes after merging, whatever should build into a PDF builds into a PDF, whatever should go on the intranet goes on the intranet — you don’t have to care. Your work gets near-instant feedback — and, just like on a blockchain, your name is written permanently into this revision.

Everyone talks about lean startup; everyone knows the build–measure–learn feedback loop — “among all martial arts, nothing beats speed” is really saying that this loop has to iterate fast. But you can’t make that loop fast by talk alone; it takes good mechanisms, it takes code.

Take the interview process again. Yesterday I interviewed a data engineer; she gave us some suggestions about our code test. I thought it over afterwards and found them quite sensible, so I floated this proposal in the clubhouse (not yet implemented):

$ arcli create:code-test
Please choose a role: Data Engineer
Please choose a test: General
Please set expiration (1-7 days, default 7): 7
Please put the github id for the candidate: tyrchen

Code test is generated for candidate tyrchen. Temporal link (expire in 7 days): https://....
Github repo is generated for candidate tyrchen. It will be merged into arcblock/arc-archived-code-tests upon expiration.

Basically, when we want to send a candidate a code test, we pick the relevant role and test, generate the GitHub repo, generate a temporary access URL for the test, and then notify the candidate via email or WeChat. This way the test instructions can be updated at any time, with no back-and-forth verbal explanation over email or WeChat — and verbal explanation is conversation-based: it distorts easily, and it invites improvisation that can lead the candidate in the wrong direction.

arcli is itself an embodiment of our Code is Law philosophy. One of our principles is that internal tools should be unified and easy to use. The corresponding requirements are:

• The CLI’s interface must be consistent
• CLIs must be easy to discover
• CLIs must be interactive and self-documenting, so they’re easy to pick up

So I spent a bit of time writing a self-registering CLI framework with commander and inquirer (don’t let the grand name intimidate you — the whole core is under 100 lines of code), plus a CLI template that can auto-generate the entire layout of a new CLI, where you then just fill in the action to execute:

$ arcli create:arcli
? name of the cli (use ":" to differentiate action and object. e.g. create:repo): hello:world
? Please write concise description: hello world
Creating target index.js with file index.js.tpl.
Creating target cli.js with file cli.js.tpl.
Regenerateing /Users/tchen/projects/arcblock/arcli/src/cli/hello/index.js
Regenerateing /Users/tchen/projects/arcblock/arcli/src/cli/index.js
Code generated at: /Users/tchen/projects/arcblock/arcli/src/cli/hello/world

This way, when an engineer needs to build some automation tool, she can create it quickly and easily, focusing on the logic rather than on how the CLI is registered, how it’s discovered, how to set and choose parameters, and so on.

Internally, we’re also constantly discussing how to make these tools and processes more humane, so the experience becomes a peak experience:

We know that if a company talks endlessly about being user-centric and giving users love and care, while its internal processes are hostile to people at every turn and not user-centric at all, how can employees possibly love and care for users? If a company pounds its chest saying every employee is a leader, yet never gives those leaders the right tools to exercise their ownership, how is that any different from saying nothing? In Harvard’s Positive Psychology course, Tal uses vivid examples to show that people don’t follow what you say, but follow what you do.

So at Arcblock, Code is Law. We express process in code, we record process with GitHub, we compile and deploy process with Travis, and we evolve process the way open-source collaboration does. And with all of this, we hope not only engineers benefit but the whole company does; not only engineers can send PRs — people in other functions will also be taught the simple use of markdown and GitHub, so they can use and update it without friction.

Just think: if one day the company’s processes change — say all PPT templates must switch to a new UI, all docs must switch to a new logo — what happens to all your historical documents?

At Arcblock, it’s simply a matter of changing the stylesheet in a few places, opening a PR, and merging.

Code is Law!