Musings on Indirection

In computing, the line I hold in highest regard is: “All problems in computer science can be solved by another level of indirection.”

This is not hard to grasp. Let me offer a few examples:

• The operating system is an abstraction between bare metal and applications. Think back to the era before operating systems — how much hardware detail did an application developer have to understand?
• Virtual memory is an abstraction between physical memory and applications. In the days of real addresses, applications were scrawny and impoverished, racking their brains to save every byte of memory, and one careless slip could bring the entire system crashing down. With virtual memory, the tables turned: applications suddenly became rich and handsome, and they enjoyed all manner of protection — someone else crashing was basically none of my concern.
• The JVM (or VMs of any language — the Python interpreter, the Erlang BEAM, and so on) is an abstraction between the operating system and applications. People who write software most desire to “write once, run everywhere.” Add this layer, and cross-platform is no longer a problem.
• The virtual machine is an abstraction between the host OS and the guest OS. It directly gave rise to the era of virtualization and cloud computing.

Such abstractions are visible everywhere in software architecture. From something as large as the Hardware Abstraction Layer (HAL) down to an API or an interface you write yourself, all follow the same idea: isolate caller from callee, hide the details, and increase flexibility.

Abstractions abound in daily life too, and the phone charger is the most typical one. Input current and voltage differ across the world, and the sockets are even more wildly varied — no phone could possibly be designed to fit every outlet. So we insert an adapter between the two: the phone charger.

But abstraction is not merely about hiding details and making a system more flexible. The shipping container separates cargo from the vehicle that carries it, dramatically improving transport efficiency and enhancing safety along the way.

Back to software development — let’s see how abstraction improves efficiency. Take a classic MVC application: I want to read data from the database, load it into the model, and render it into the view. If I insert an abstraction between the caller (the model) and the callee (the database), so that the caller doesn’t know where the data comes from and merely fetches it through a specific interface, then I can stash the data in a cache and only hit the database when necessary — greatly improving the application’s performance.

The principle is dead simple, and seemingly everyone understands it, yet doing it is not so simple. Knowing which part of a system needs surgery to add a layer of abstraction is a matter of experience and wisdom; knowing how to add that abstraction is a matter of ability. Nowadays people take containers and chargers for granted, regarding them as simple concepts even a child understands — but if I asked you to find the next “container” or “charger,” could you find it?

To sum up, the benefits of abstraction:

• Hides details
• Increases flexibility
• Enhances safety
• Improves efficiency