Red: Neither Red Nor Expert, But Sexy

red lang released 0.6.3 a couple of days ago. OSX GUI support has finally landed.

The news barely caused a stir even on Hacker News. After a couple hundred upvotes the post quietly sank out of sight, never to be seen again.

I first heard of red around 2015. There was an interview with Nenad Rakocevic on Medium at the time, discussing red’s philosophy, its inheritance from rebol, and its wildly ambitious roadmap. In that piece, red 1.0 was projected to arrive within about a year — that is, 2016. Yet here we are in 2017, and 0.6.3 has only just limped in.

Why does a language that everyone reading this has probably never heard of command my attention? Nenad Rakocevic is one reason — I’ll dig into that fellow another time. The other is that red is different from all those languages you know by heart. Very different. Like Lü Yan standing tall and willowy in the middle of a lineup of Victoria’s Secret Angels.

Let’s see just how seductive red really is.

• homoiconic. Readers from the lisp camp will know the cachet of this word. In plain terms: code is data, data is code. Each contains the other.
• macro / DSL. red is lisp’s blood relative, so anything lisp can get away with, red can too.
• A genuinely formidable parser (PEG). Essentially inherited from rebol’s famous parse (another topic for another day).

So is red just another clojure or racket? Read on.

• low memory footprint, GC & cross-compilation. The style suddenly shifts — now it’s walking the golang path. Yes, red is a compiled language. No VM: it compiles straight to a native binary for the target platform, with C-level performance (unoptimized output runs at about 1/4 the speed of the same C code compiled with O2). At compile time it can cross-compile directly to target OSes like windows / linux / osx and target CPUs like x86 / arm. And golang’s cross-compilation is nowhere near as convenient as red’s.

• small target binary. Again like golang, except a golang hello world weighs several megabytes while red’s is a few hundred K.
• actors, parallel collections. Concurrency doesn’t go through CSP, but actors plus goroutines deliver concurrency support at the same level.

Doesn’t it feel like lisp stuffed inside golang? If you don’t have a firm grasp of either golang or lisp, here’s an analogy: it’s like Alice from Resident Evil transforming into Consort Wan at Emperor Yongzheng’s family banquet and performing the Dance of the Startled Swan.

And that’s still not all:

• Cross platform native GUI. Well now — a language that looks like a backend player is brazenly going after the frontend’s rice bowl. And not just that: it wants to cover osx, windows, android, and iOS, and unify them all. We all know that anyone who tries to unify the UI layer usually ends up walking the Yue Buqun road — something has to be cut off. What electron cut off was the lightness of a native app; what cordova cut off was performance. red, like react native, cuts off a bit of flexibility — but it goes about it in earnest, carefully implementing DSL-to-native-code compilation on each platform. [facepalm]

• rich data types (50+). So many times we’ll finally be free of those “universal data structures” for expressing things like colors and IP addresses (yes, string, come here, pat pat).

• single file toolchain. This one is outrageous — the entire compiler is a single binary, and it’s just over a megabyte. Unscientific. Deeply unscientific. Look at what it can do:

You’re telling me a tool with a UI lib, cross-platform compilation, and the ability to AOT (ahead of time compile), JIT (just in time compile) and interpret code is only a little over 1 MB? That is just too damn unscientific.

• highly expressiveness. Writing its code without having read the docs isn’t easy, but reading (or guessing at) its code without much background knowledge isn’t hard either. Two quick examples:

The code is so simple and clear it doesn’t look like a heavyweight (I know the racket / clojure fans will object).

The style keeps shifting: as if, no sooner had the graceful Dance of the Startled Swan ended, Alice launched into Gangnam Style.

Space is limited, so let’s leave red’s features there. Now let’s look at its ambition:

From the lowest-level hardware and device drivers all the way up to high-and-mighty applications and DSLs, it wants a hand in everything. Nenad proudly defines it as: a full stack language.

full stack language? Hold on — are you sure you’re not talking about javascript? [facepalm]

We live in an age both fortunate and unfortunate. In ancient times there weren’t so many languages, and certainly not so much division of labor. A programmer was a hacker who covered the whole battlefield, a craftsman. Today, software development has been carved up into domains so fine it beggars belief: frontend and backend split like the Northern and Southern Dynasties, and then the frontend fractured again into Eastern and Western Wei. A while back I was interviewing someone who introduced himself with enormous pride as working on “the backend of backends.” I asked for details and nearly choked. The man had merely built a few service-ish things with node, and already felt he’d been promoted to Kshatriya, eager to distance himself from the Vaishyas writing APIs.

The consequence of this fine-grained division of labor is that programmers have devolved from hackers and artists into interchangeable screws — the end-to-end generalist is increasingly rare, nobody feels remarkable anymore, nobody feels like King of the World. Writing code has gone from something cool and fun to a trade you do for a living. The backend builds its muscle, the frontend applies its rouge, and that’s that. If the backend starts plucking its eyebrows, or the frontend starts flashing its V-lines and abs, the order collapses — that’s transgression.

Nenad says no. He doesn’t believe that today’s multilingual, high-division-of-labor army is the only way to build software, or even the best way. He wants to invent a language with enough expressive power (good to write, good to read), enough performance (so it doesn’t become a toy), enough breadth of use cases (front and back, usable anywhere), and enough fun (to recover the joy of programming) — red is his answer. Though five years have passed since 2012 and red has yet to fully realize his ambition, the features enumerated above are dazzling enough. And don’t forget: most of the time he has been fighting alone. To this day the repo has only single-digit core contributors — github lists 37 contributors, but the overwhelming majority are soy-sauce passersby whose diffs run under a hundred lines.

Let’s look at a few of red’s contrarian traits.

In most languages, the runtime / toolchain / libraries have grown unbelievably complex — python, for instance, has thousands of files and dozens of CLIs. red is one file, 1.2M.

In most languages, installing and running means following the docs step by step like performing a religious rite; some stubborn specimens won’t even start without a particular directory layout and config files. With red you download it, chmod +x, and run. Compiler, interpreter, class libraries, cross-compile toolchain — all inside that tiny 1.2M world.

In most languages, even the language’s own father isn’t quite sure what the compiled target code will look like. Yet Nenad, in a feat that is frankly bizarre and slightly unhinged, hand-writes arm/ia32 assembly and machine code for red (see: https://github.com/red/red/blob/master/system/targets/ARM.r). His skill speaks for itself. Take this stack unwind when throwing an exception:

Whether you kneel or not, this programmer kneels. If you compare with racket’s implementation — racket’s lower layers are written in C, and exceptions go straight to the OS libraries (e.g. mach/exception.h) — that’s exactly why, though both descend from lisp, red’s performance dares to keep pace with C while racket can only weep silently.

Two more words: red’s approach to generating ELF executables is equally against the current. It couldn’t care less about “vast and garbage” libraries like libelf, and it disdains the road every normal language takes through LLVM. Instead it builds ELF from nothing, section by section — that isn’t compiling, that’s straightforwardly text processing, except the output “happens” to be an executable ELF binary.

In most languages, tending your own plot of land is enough. But red aims high. Look at the diagram:

The host interface accounts for everything from OS APIs down to hardware, aiming at native support on every platform. At the same time red doesn’t forget who it is: it builds bridges to mainstream languages to enhance interoperability — meaning that under java you can drool happily while calling red’s parse to serve as a rule engine.

Many people reading this will sneer: what’s the use of all that power if it’s a palace consort left forgotten in the deep chambers? True enough. For all these years red has scarcely stirred a ripple in the mainstream programming-language market — it has a whiff of the walking dead about it. Yet to hold on as it has is no small feat. To my mind, Nenad is like David dueling the giant Goliath — or, more tragically still, Sisyphus pushing the boulder uphill: though the power he challenges is so invincible, he steps forward to challenge it anyway.

As everyone knows by heart:

“Princess, in this world eight or nine things out of ten don’t go as we wish. Since it has come to this, there’s no forcing it.”

“Then I insist on forcing it.”

0.6.3 is out. 1.0 still seems far away — but not all that far. For now, just try it; if it interests you, give it a proper play. I hope you rediscover the joy of programming!