Why SOA Matters So Much

SOA is short for Service Oriented Architecture—I hope you have a rough idea of what it means. In the previous post, ‘Measurement,’ I quoted a memo from Bezos that completely converted Amazon’s software architecture to SOA. It was so pivotal that it may, in some sense, have changed Amazon’s fate (think AWS).

SOA is a way of thinking, not a collection of ‘bygone’ technologies listed on Wikipedia (XML/HTTP/SOAP/WSDL/UDDI). Ten years ago SOAP/WSDL/UDDI were all the rage; today almost no one pays attention to them (if you have waded through the murky waters of online travel, you may know that booking.com still offers APIs wrapped in SOAP). Technologies can decline, but ideas do not.

The essence of SOA is strict loose coupling: everyone communicates according to a contract (a service interface). As Bezos put it, “There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network”.

No shared memory, no back doors, and no direct access to another service’s data are all easy to understand, because they break encapsulation and create an internal dependency. When a service’s internal state changes, the effects of that change spread to everything that depends on the service. Anyone who builds software knows this is one of the biggest headaches; all kinds of mysterious bugs tend to arise in exactly this situation.

But why not package the service as a library, expose a standard API, and let callers link it into their own services? As long as the API doesn’t change, no matter how the library churns internally, it won’t affect the places that depend on it, right?

Consider a function: given the path to an image, retrieve its EXIF information. Implementing this in Python is trivial:

import exifread
import urllib
def get_exif(path):
    if path.startswith('http'):
        path, _ = urllib.urlretrieve(path)
    f = open(path, 'rb')
    return exifread.process_file(f, details=False)

Of course, this implementation is not perfect, and it is not HTTP-friendly (the entire file is downloaded, written to disk, and then read back), but it is basically usable.

If we hand it to others as a package, there doesn’t seem to be much of a problem. Let’s do that for now.

Later, as the system expanded, a new feature X chose to be implemented in Golang. Unfortunately, X also needed to read EXIF information from images. Golang cannot directly call Python code, and if we insist on static linking, we face two choices:

1. Reimplement the same service in Golang.

2. Rewrite the core EXIF-reading code in C, then create separate Python and Golang bindings.

3. is obviously bad, because it violates the DRY (Don’t Repeat Yourself) principle. 2) is theoretically feasible, but it is digging a pit for yourself—what originally required maintaining a few lines of code now becomes three sets of code in three languages.

This is one of the many potential pitfalls of static linking.

If we instead make EXIF retrieval a service, how should we do it? Here is one approach (I deliberately did not make it an HTTP service):

import exifread
import urllib
import zmq

ENDPOINT = "tcp://*:5555"

def main_loop():
    ctx = zmq.Context()
    sock = ctx.socket(zmq.REP)
    sock.bind(ENDPOINT)
    print("Listen to %s" % ENDPOINT)

    while True:
        cmd, path = sock.recv_string().split()[:2]
        print("Got request: %s %s" % (cmd, path))
        if cmd == 'EXIF':
            if path.startswith('http'):
                path, _ = urllib.urlretrieve(path)
            f = open(path, 'rb')
            tags = exifread.process_file(f, details=False)
            tags = dict(map(lambda (k,v): (k, str(v)), tags.iteritems()))
            print("Sending response: %s" % tags)
            sock.send_json(tags)

if __name__ == '__main__':
    main_loop()

The contract between the exif service and its callers is:

1. The caller sends CMD PATH to port 5555 (for example: EXIF http://domain.com/img.jpg).

2. The exif service returns EXIF information in JSON format.

Accessing this service from Golang is very simple:

package main

import (
    zmq "github.com/pebbe/zmq4"
    "log"
)

func main() {
    sock, err := zmq.NewSocket(zmq.REQ)
    if err != nil {
        log.Printf("err create: %v", err);
        return
    }
    defer sock.Close()
    err = sock.Connect("tcp://127.0.0.1:5555")
    if err != nil {
        log.Printf("err connect: %v", err);
        return
    }

    _, err = sock.Send("EXIF https://farm8.staticflickr.com/7163/6602016347_24803ae230_o.jpg", zmq.DONTWAIT)
    if err != nil {
        log.Printf("err send: %v", err);
        return
    }

    msg, err := sock.Recv(zmq.Flag(0))
    if err != nil {
        log.Printf("err recv: %v", err);
        return
    }

    log.Printf("exif: %v\n", msg);
}

Besides decoupling the service from the language used by its callers, this approach has many other benefits:

1. The service itself is highly focused and does not get mixed up with messy logic, making problems easier to locate.

2. The service itself is easy to measure; you can check its performance and latency at any time. Measurement is a prerequisite for optimization. Once you have good measurement in place, optimization is only a matter of time—you will always find a way (you can even rewrite it in another language).

3. It is easy to scale out.

4. You can easily add ACLs and set priorities and thresholds for callers.

One day we discover that this service can be used not only internally but also exposed to third parties to generate revenue. We only need to add another service caller and publish the retrieved data through an HTTP service.

A few more words on the importance of ‘decoupling the service from the language used by its callers.’

From a software engineering perspective, this helps projects finish quickly and with high quality. We know that every language (and its libraries) has strengths and weaknesses. Using C instead of Python in a situation that calls for a glue language, or Ruby instead of Golang in a situation that demands high performance and high concurrency, can only produce half the result with twice the effort.

From a software engineer’s perspective, they do not have to sink deep into the swamp of an existing system, fully understand the ins and outs of the whole system and the tortuous history of the code before they can start doing the work they should do. Ideally, a new feature is 80% decoupled from old code, with only 20% coupled through interfaces. If someone already steeped in the same industry needs at least three months of training and study before they can begin working on a system, then something is definitely wrong with the system architecture.

Some people believe SOA cannot be applied to scenarios with very high performance requirements. That is a misconception. Every system has a realistic performance target; it is not a matter of ‘the higher the better’ without any goal. If that were the case, setting aside the services provided by the operating system, writing all code directly in assembly could theoretically squeeze the maximum capability out of the hardware. But except for a very few projects, no one does that. With a performance target to refer to, all a service needs to do is meet that target. The choice of hardware, development language, concurrency model, various tools, and even the message-passing scheme can all help achieve the agreed performance target. Using SOA actually encourages this kind of diverse choice, and its loose structure makes aggressive optimization possible. If the system is tightly coupled, even if you have measurement in place to perceive the critical path affecting performance, ‘pulling one hair moves the whole body,’ making it harder to deal with.

Finally, the SOA mindset is actually of great benefit to a software engineer’s career. You do not have to be ‘locked in’ to a particular language, platform, or framework used by a particular company; instead, you can exercise ‘free judgment’ according to the industry’s ‘best practices.’ A company will always have its day of collapse; a technology will always have its moment of decline. Only change and ideas remain constant.