Embracing Constraints

Lately, thanks to work, I’ve been using dynamodb in my code. Amazon’s dynamodb is a key-value database; back in ’07 Amazon published a famous paper on dynamodb that gave rise to a number of open-source projects, such as cassandra and riak. dynamodb’s advantages are plain to see, for example:

• scalability: thanks to its hash-key-based sharding/partitioning (each hash-key belongs to a single partition) and AWS’s infrastructure, capacity is practically unlimited
• performance: excellent and constant
• auto data-replication: each piece of data is guaranteed multiple writes
• provisioned throughput: set the read/write capacity you need and maximize the performance/cost ratio
• plug and play: no complex setup, straight to work, an extremely simple API
• and so on

But the choices it made for the sake of partitioning and performance make its drawbacks equally glaring. Take querying: if you’re used to MySQL or another RDBMS, or a NoSQL DBMS like MongoDB that supports rich queries, then looking at dynamodb it seems like nothing but a stupid monster. Say a table needs to find the top 10 rows in reverse chronological order by updated_at. In any other database that’s one line of code; in dynamodb it’s an insurmountable wall. A dynamodb table or index must have a hash-key, and the hash-key is what does the partitioning, so it only supports equal-to queries (you can think of hash-keys as stateless with respect to one another, utterly unaware of each other’s existence). You can’t compare them, you can’t sort by them. It does offer a separate range-key that supports numeric comparison (gte/lte/gt/lt/eq/neq) or string comparison (begins-with) — but a range-key is only valid within the scope of a single hash-key. Which is maddening: to support such a simple query there are only three approaches:

• full table scan. Prohibitively expensive.
• Set up an index whose hash-key is a field that is always 1 and whose range-key is updated_at. This forces dynamodb to collapse that index down to a single partition — a hack.
• Switch to another database.

With the hash-key restriction, the first time you use dynamodb the constraints are so severe that you feel boxed in on every side, sick to your stomach. After a month of use and a good deal of best-practices reading, I finally worked out the trick of it:

Constraints are not a bad thing; they force you to think more clearly about your system’s design, and thus to make better choices.

Take that query above. First ask in what scenario it’s used, whether it’s truly necessary, and whether some other technology — a cache, say (e.g. redis) — could implement it. If it really is unavoidable and genuinely important, then consider reimplementing it on another database.

Often, when we build systems, we start out with no deep appreciation of the system’s latent constraints. We build along, discover a constraint, patch the design, and carry on. The patches pile up until at last we have Ptolemy’s geocentric model of the heavens: ever more elaborate structures that exist only to accommodate ever more “exceptions,” until the whole thing becomes a bloated freak riddled with corner cases, and we have no choice but to tear it down and start over.

Every product or tool involves trade-offs. The CAP theorem tells us that in a distributed system you can have only two of Consistency, Availability, and Partition tolerance. Some products sacrifice Partition tolerance (MySQL, for instance); others sacrifice Consistency (dynamodb). Whatever is sacrificed is precisely their constraint. Only by truly grasping a product’s inherent constraints can you think properly about product design and tool selection.

When we use MySQL, creating an index on a field we need to query is almost reflexive, because in MySQL creating an index is roughly “free.” But in dynamodb an extra index demands extra quota — which is to say, extra money. That “not free” constraint makes us design dynamodb fields and indexes with great care. We pack several queryable fields into a single range-key so the index can be reused, or we cut unnecessary queries (drop a feature) — and the system ends up simpler and clearer for it.

After my article “Writing Decent REST APIs” came out last month, many people said REST is too rigid — why not use the more flexible JSON-RPC? True enough: at the API layer REST imposes more constraints than JSON-RPC. Everything is a resource; everything is stateless; and sometimes it’s genuinely hard to express the API you need. JSON-RPC is right there at your fingertips, so very convenient. What people don’t realize is that REST’s constraints may be exactly what pulls you out of the mire later, whereas freewheeling JSON-RPC can leave your code spaghetti, tangled on itself. I’ve seen RPC code where the causal chains were so convoluted and so interdependent that one RPC depended on the execution state of other RPCs (note: not their results). Writing code this freely may be very efficient at first, but it grows harder and harder as you go: to call one API you must consult several documents and carefully ensure the call order of several RPC calls — ultimately not worth the cost. I once built an app with meteor (a reactive framework on nodejs). RPC code poured freely from my fingertips and features materialized overnight — but in the face of that flexibility I had thought too little and prepared too poorly (a matter of my own skill, perhaps). Afterwards I spent a great deal of effort making up for it — and the way I made up for it was, amusingly, to voluntarily pile on restrictions.

The contrast between JSON-RPC and REST is a bit like the contrast between Client/Server architecture and Browser/Server architecture. The structurally flexible, highly controllable C/S architecture ultimately lost to the rule-riddled B/S architecture; application software built atop TCP, free to do whatever it pleased, lost to application software built atop stateless, “single-action,” “weakly expressive” HTTP. In both cases, too few constraints and too much room led people to make a mess of things.

Having written all this, I am not saying dynamodb is superior to MySQL, REST superior to JSON-RPC, B/S superior to C/S, or HTTP superior to TCP. Each has its fitting use case; it is never either/or. What I am saying is that explicit constraints make you think more and decide better. MySQL indexes are not free of cost; APIs designed with JSON-RPC are best kept stateless; C/S software, or applications carrying data over TCP, should define a protocol between the two sides. It’s just that these tools don’t lay their latent constraints out in the open.

Put another way, a constraint is a form of isolation, a form of protocol.

Look at the evolution of programming languages. Early machine and assembly language could access memory at will (in real mode), and code could jump anywhere it pleased. Then came high-level languages, where control statements (if, for, and the like) and functions isolated the vast majority of jumps, so they could only go to specific places — even goto and setjmp/longjmp saw their flexibility sharply curtailed. Later, clear-thinking people launched a campaign to eradicate goto, with the result that today any software company, when reviewing code, will nitpick goto to death, while setjmp/longjmp is strictly confined to low-level systems code. The two lines of development that followed — OOP and FP — imposed yet more constraints on how code is written, each from a different direction. For example: in most functional languages, variables can only be bound, not modified. The price is that even the for loop everyone takes for granted becomes hard to write intuitively; the gain is extra benefits in concurrency, debugging, undo/redo, and more footnote:[See my earlier article “The Enduring Allure of Immutability”]. This is the isolating effect of constraints.

As for the protocol effect of constraints, allow me to quote someone else. I have quoted the following passage several times in different articles, but I cannot resist quoting it again — this is Bezos, roughly, laying down the law to Amazon’s technical teams in ’02 — and looking at AWS’s enormous success today, one cannot help but admire the deep understanding and foresight about software constraints held by this CEO who didn’t write code and didn’t much understand technology:

1. All teams will henceforth expose their data and functionality through service interfaces.

2. Teams must communicate with each other through these interfaces.

3. There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network.

4. It doesn’t matter what technology they use. HTTP, Corba, Pubsub, custom protocols – doesn’t matter.

5. All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions.

6. Anyone who doesn’t do this will be fired.

7. Thank you; have a nice day!

On the surface, these terrifying constraints from Bezos bound the teams’ hands and lowered their efficiency (grabbing your lib and making a function call is a matter of minutes; agreeing on a protocol, writing interfaces, calling them, and handling every sort of error case takes days at least, sometimes weeks). But these constraints became the necessary guarantee of Amazon’s enormous success in SOA (Service Oriented Architecture). Savored carefully, they have very much the feel of HTTP versus TCP, REST versus JSON-RPC. Set the rules clearly in advance, and you avoid the Mario predicament: eating a mushroom in level one and growing big, only to find that level two requires you to be small to get through.