Reactive Programming

No new essay this week — instead, I’m sharing the talk I gave to my team today.

Today’s topic is reactive programming — or, more precisely, spreadsheet-like programming. Saying “reactive” easily gets confused with FRP (functional reactive programming), whereas today’s protagonists — hoplon / meteor / mobx — have essentially nothing to do with Elm / ReactiveX.

We all know that the spreadsheet (and please, don’t hear “spreadsheet” and think only of Excel — the world’s spreadsheets are not just Excel; there’s the departed Lotus, the half-alive Numbers, and the reigning Google Sheets) is an ancient yet enormously powerful development tool. Time and again, a task that a spreadsheet can knock out in a few minutes, handed to a programmer, won’t be done in days, will certainly be less extensible, may not even perform better, and will break with bugs at the drop of a hat. Don’t believe me? Here’s a problem for you: periodically pull data (sessions, DAU, etc.) out of Google Analytics, compare this week against last week, this month against last month, this year against last year, visualize it all in charts, and turn it into a dashboard. Those of you in the room can estimate for yourselves: how many engineer-hours would that take?

With Google Sheets, this takes about ten minutes.

The power of the spreadsheet lies in this: you can define a set of inputs (source data), then define formulas that depend on that data at any number of levels (derived data), all the way up to the result you need (data or charts). An intermediate step can depend on any number of other data — whether source data or derived data — and the spreadsheet automatically records and computes those dependencies for you. When the source data changes, the change ripples outward, like a stone dropped into a lake, into every derived datum it touches. You can have hundreds or thousands of source data, tens of thousands of derived data, and the spreadsheet will still handle every consequence of a data change efficiently, quickly, and correctly.

In fact, when we write code day to day, we are often doing exactly this: deriving all kinds of data from raw data and presenting all kinds of results. Take the classic TodoMVC. The source data is the individual todo items; the derived data is the number of todo items, the number of incomplete todo items, and the todo items visualized on the page. A little TodoMVC app is trivially easy to build with a spreadsheet — a few formulas and you’re done — whereas with the endless parade of front-end frameworks, you can’t do it well in under several hundred lines of code. The reason: the spreadsheet manages dependencies automatically, while almost every programming language and framework requires the programmer to handle data dependencies by hand.

The spreadsheet is a product of the 1980s. More than thirty years have passed; the software world has been turned upside down; new languages and new concepts keep pouring forth. So why has no good library, framework, or even language emerged that works like a spreadsheet — where you simply declare which data is source data and which is derived, and the underlying architecture manages the dependencies between data, so that when source data changes, the change propagates automatically to all derived data, correctly and efficiently?

The first answer I know of from the industry is hoplon, a framework that shone brightly at Clojure/West two years ago. If you’re interested, see this video: https://www.youtube.com/watch?v=wVXjExRiFy0. hoplon uses Clojure’s metaprogramming capabilities to introduce the spreadsheet concepts of cell and formula. You can use cell to define source data and cell= to define derived data; hoplon handles the dependency tree for you and recomputes the relevant derived data when the source changes.

hoplon’s concepts are excellent, but Clojure is, after all, still a niche language. So although I excitedly wrote several small programs after discovering it, I never used it in production.

hoplon may be unfamiliar to many, but meteor is probably known to most of you. Although meteor never connects its reactive ideas to the spreadsheet, its internal implementation is similar: meteor has the concepts of reactive source and reactive computation. Meteor’s session variables and database collections are reactive sources; template helpers are reactive computations. Once you grasp this, meteor holds no particular mystery.

Another rising star in the JavaScript world, mobx, stands on the shoulders of its predecessors and does spreadsheet-like data flow even better. The company behind mobx apparently developed the library out of frustration while building a front-end drawing tool. We know that drawing tools like Visio contain so many complex states that touching one hair moves the whole body: say you have a node A connected to N other nodes. When you drag A, the positions of all those connections must shift accordingly, which may in turn affect the positions and states of the other N nodes, and so on to even more nodes. If you can’t automatically handle this propagation of data change and must do it by hand, the cost is enormous and the results are poor.

To automatically handle the propagation of state changes, mobx defines these concepts (compare them yourself with meteor / hoplon):

• observable state: source data. Don’t confuse it with ReactiveX’s Rx.Observable.
• computed state: derived data, analogous to a formula in a spreadsheet. Must be side-effect free.
• reaction: similar to computed state, but may have side effects.
• action: the various external events (which cause observable state to change).

mobx’s approach to state propagation is admirable — feel free to look through my slides and think about why it’s made so complex and what the benefits are (especially when the dependency tree is very complex). As a result, its performance is excellent.

Another advantage of mobx is its excellent React support — it’s practically tailor-made. Think about how much boilerplate flux makes you write just to keep state seamlessly synced between data store and view. With mobx, you only need to decorate the source data in your data store with the observable decorator (yes, decorator is ES7 syntax, but don’t we have the great Babel?), decorate a few helper functions that produce derived data with the computed decorator, and then use the observer decorator when defining your React components. That’s basically it. I believe that as mobx develops and more people recognize that letting the system automatically propagate state beats handling it by hand, mobx will overtake redux and claim an important place in the React ecosystem. Just picture it: React (view) + mobx (state management) + RxJS (event streaming) — how beautiful a picture is that?

Enough talk — let’s look at the pictures rather than the prose.