Rails, Django, Phoenix: You're All Wrong

Writing a title like this is bound to earn me some curses.

These three famous MVC (or MTV) frameworks correspond to Ruby, Python, and Elixir respectively. Calling them the pillars of those languages is no exaggeration. Many people come drawn by a framework’s fame and only then learn the language. Rails is the quintessential case — once wildly popular (and still a first-tier star today): many Rails engineers initially knew only Rails, and only after writing Rails did they run into the language’s shortcomings and go back to learn Ruby. Phoenix and Elixir are much the same story.

In Django and Phoenix you can see many traces of Rails. Rails was the star framework engineers adored back in the flickr / delicious era. Afterwards came a wave of imitators and learners in other languages — Django wasn’t the first, and Phoenix won’t be the last. Through these frameworks, an engineer can rapidly scaffold a web project, bind it to a database (usually an RDBMS), generate models, controllers, and views, and within a few days a deployable “website” has been assembled.

Is developer efficiency high? Very. Is code efficiency high? Rails and Django draw plenty of criticism, but Phoenix is extremely fast, crushing other frameworks in the benchmarks.

Is the architecture excellent? It seems excellent too — if you or I had to write a web framework from scratch, we’d never match their quality.

So where are they wrong?

They are wrong in that while they give web app developers the hope that “anyone can build a web app,” they also imprison many programmers’ thinking inside that one small MVC box.

Suppose we want to build a MOOC application. Users can browse courses, enroll in courses, bookmark courses, and during the course they can rate it, take notes, interact with others, and so on.

Let’s look at how a Rails programmer typically starts building the backend:

• Design the database schema: User / Content / Bookmark / Review / …
• rails new mooc
• rails generate model
• Map the database design onto Rails models
• rails generate controller
• Write the controller and view for each page
• (If there’s time) write tests

Following the framework’s line of thinking, we unconsciously make a set of assumptions:

• All state is stored in one or several databases
• If a database has no adapter for the framework, we can’t use it
• Data is tightly coupled — for instance, there is an enroll table between User and Content serving as the record of which course a user signed up for
• A controller can pull data across multiple models and hand it to a view to render

With these assumptions we can assemble an application very quickly, but at the cost of high coupling.

Some will wonder: wasn’t decoupling the whole point of the MVC pattern? Why does coupling get worse instead? The classic MVC layering is a vertical decoupling: data flows in an orderly way, each layer minds only its own business — “render unto God what is God’s, and unto Caesar what is Caesar’s” — with no need to care how other layers are implemented. But it cannot prevent horizontal coupling, such as model-to-model coupling, or a controller coupled to several models. Yet web frameworks, deliberately or not, actively encourage this coupling. Worse still, they bake it into the data layer, so that later on, whether you try to decouple at the code level or the data level, you’re in for a world of pain.

Before Rails appeared, we knew there was also a business logic layer. After Rails appeared, in everyone’s practice that business layer was inexplicably folded into the model layer, some of its functions drifted off into controllers, and it simply vanished. But is it really quite right for the business layer to be kneaded into a web framework like this?

The web layer that Rails and its kin represent is not the whole of the business. What if one day we need to expose an API to third parties? What if the web logic gets overhauled beyond recognition? What if the company is suddenly acquired and user accounts must be merged into the acquirer’s system — what if we don’t get to keep a user table of our own at all?

Back to our MOOC example. Course management, scheduling, enrollment, and so on are all business-layer concerns. A user enrolling in a course should, at the business layer, be expressed as: {:enroll, uid, cid} -> true/false, not as the tangled logic sitting in controllers and models. And displaying all the courses a user is subscribed to should be expressed as: {:show, uid} -> [a list of courses].

None of this has anything to do with models. The User model shouldn’t even be able to see the Content model, nor the enroll table that links them.

This is horizontal decoupling. Each of us is a black-box service: the user service maintains and presents personal user information, the auth service verifies identity, the content service manages course content, the content enroll service handles everything related to enrollment, and so on. As shown below:

We can even group these services by nature — some are core services, some social, some communication. Each has clearly defined interfaces; the auth service, for example, provides:

• Username/password verification: {user, pass} -> {:ok, access_token, refresh_token} or :error
• Token verification: access_token -> {:ok, token_info} or :error
• Token refresh: refresh_token -> {:ok, access_token} or :error
• Change password: {refresh_token, old_pass, new_pass} -> {:ok, new_refresh_token}

The auth service stores only user/password-related information. Only the auth service itself knows that information; not even the user service has permission to access it.

At first, this decoupling costs extra work, but as the system grows you’ll find that this design pays off enormously in extensibility and reusability. Adding a new service doesn’t disturb existing ones; we can even write a brand-new, upgraded replacement for an existing service, route part of the traffic to it, and once it tests well, delete the old service outright.

Another benefit is a return to the proper path of being business-centric. To put it bluntly, frameworks like Rails easily lead people astray down a web-frontend-centric road. “Frontend” here means the backend’s frontend. We should, driven by requirements, first model the business and get each service built properly, and only then use Rails and friends to craft the frontends. We may need a user-facing frontend, an admin-facing frontend, and each independent service may need its own admin frontend; we’ll also want analytics frontends, user-behavior analysis frontends, and so on. None of these frontends really have models, because data storage is solved inside each service.

In this way we break the assumptions above: data becomes loosely coupled, each service owns its own independent data, and that data is assembled only when needed.

As for these individual services — call them microservices, call them applications, whatever you like. As long as they are independent enough and can move with the needs of the moment, that’s fine.