Alan Kay, Roy Fielding, and Growable Systems that Last for Decades

Mike Amundsen, Director of Architecture - API Academy, CA Technologies
<mca@amundsen.com>

Tip

This is a late draft of an article published by DevX in 2015.

Introduction

While doing some background research on distributed software recently, I rediscovered some of the history of two major themes that affect many developers today: the Object-Oriented programming style (OOP [1]) and Representational State Transfer architectural style (REST [2]). Even though the two concepts seem to focus on very different aspects of designing and implementing systems (local coding vs. distributed co-ordination), I was pleasantly surprised to find that the two people responsible for these important paradigms — Alan Kay [3] for OOP and Roy Fielding [4] for REST — both talk about their ideas in rather simliar ways. Both speak in terms of long-term viability as a key goal of their work and even used some of the same language to describe and clarify implementation details of their ideas.

Alan Kay’s OOP came out of his work at the Xerox PARC [5] facility in the 1970s. Fielding’s REST model was developed a generation later in the 1990s at UC Irvine [6]. One of Kay’s goals was to make it easier to create large-scale systems that could adapt over time without the need to be shut down and restarted. He called these "growable systems." Fielding says that designing systems that are 1) evovable, 2) can be modified at runtime, and 3) can "last for decades" is one if the primary motivators for his architectural constraint model.

One of the key concepts that link the two men together is an emphasis on a message-oriented approach to communicating between components. What I find very interesting is that many developers today use what they think are OOP and REST principles to create applications that work over the Web but few of them actually leverage the message-oriented model that both Kay and Fielding say is so important.

I thought it would be interesting to explore the history of the two ideas and note the similarities. I also think it is important for software developers and architects to acknowledge the common message-passing features of OOP and REST and consider ways to make it easier to employ these ideas in product applications today.

Alan Kay on Message-Passing

When Alan Kay first coined the phrase "object-oriented" he was trying to convey the idea that coding applications should be a process of a) identifying the objects that best express a problem domain and then; b) representing both the object and its processing in a concise and consistent manner. He envisioned passing simple messages between objects and allowing the objects to use code hidden within the object to operate on those messages. Unfortunately, that’s not quite how Object-Oriented Programming (OOP) progressed.

In an email as early at 1998, Kay made public his regrets in using the words object-oriented to name is concept. He complained that too many had focused on the object and missed the importance of the message in his design. And he openly worried that this would lead to systems that wouldn’t scale and would result in far too much coupling between components.

"I’m sorry that I long ago coined the term "objects" for this topic because it gets many people to focus on the lesser idea. The big idea is "messaging" - that is what the kernel of Smalltalk/Squeak is all about."[7]
— Alan Kay 1998

In subsequent years Kay continued to chide the software community on missing this fundamental point. He often complained that too many OOP systems failed to implement his message-orientation and opted instead to expose internal properties directly. He captured the basic problem in a phrase he often repeated:

"Lots of so called object oriented languages have ‘setters’ and when you have an setter on an object you turned it back into a data structure." [8]
— Alan Kay 2011

The Message Model of the WWW

The basic concepts of the World Wide Web (WWW) are also rooted in the notion of passing messages. Initially, the entire web was a set of linked documents. In the 1989 paper that outlined the WWW "Information Management: A Proposal," [9] each document was described in a single response message. Requests were, at first, simple messages with an instruction and an address:

GET /telephone-list.html

This message was then interpreted by the receiver which retreived the requested data and returned that data in the form of another message:

200 OK

content-type: text/html
content-length: XX

<html>
....
</html>

As the Web grew more sophisticated, both the request and response messages expanded to include a simple name/value pair to carry metadata and additional formats for expressing request variables and/or parameters to pass when writing data to the server. By the time the HTTP protocol that underpins the entire WWW was standardized in 1996 (in RFC1945 [10]), all the components of the WWW that we know today were in place.

*** REQUEST ***
GET / HTTP/1.1
Host: api.example.com
Accept: */*
Authorization: Basic XXX
Accept-Encoding: gzip, deflate

*** RESPONSE ***
HTTP/1.1 200 OK
Vary: Authorization,Accept
Transfer-Encoding: chunked
Etag: "1q2w3e4r5t6y"
Content-Type: application/vnd.collection+json; charset=utf-8
Date: Wed, 01 Dec 2014 14:59:30 GMT
Allow: GET,OPTIONS,HEAD
Cache-Control: public,max-age=30
...

Fielding on HTTP and REST

Back in the mid-90s, as the WWW was first developing (literally), a graduate student at Univeristy of California at Irvine (UCI) started working in his spare time on the library that would eventually be the catalyst for understanding the principles at the core of the World Wide Web. That student’s name was Roy T. Fielding and his spare-time coding project was the first version of libwww-perl or LWP [11]. At the time, it was only the second WWW library available (the first was libwww written in Objective-C by Tim Berners-Lee).[12]

One of the interesting side-bars in the story of the first libraries for the WWW is that Fielding’s library was — at first — known as the HTTP Object Model. But that name didn’t last for long since, as Fielding tells it.

"REST was originally referred to as the "HTTP object model," but that name would often lead to misinterpretation of it as the implementation model of an HTTP server. The name "Representational State Transfer" is intended to evoke an image of how a well-designed Web application behaves: a network of web pages (a virtual state-machine), where the user progresses through the application by selecting links (state transitions), resulting in the next page (representing the next state of the application) being transferred to the user and rendered for their use." [13]
— Rou Fielding 2013

In language that is rather simliar to Kay’s, Fielding describes a "well-designed Web application" as one that relies on a virtual state machine updated by executing state transitions. Fielding used the the message-passing paradigm to help codify the essentials of in his REST style description.

Distributing Objects Misses the Mark

While the roots of the WWW are steeped in the idea of enabling state transitions via message-passing, that’s not how most developers approach the task of ‘programming the Web.’ Instead, as evidenced by the popularity of the MVC (Model-View-Controller), MVVM (Model-View-View-Model) and related patterns.

It’s common for Web developers to define an object model (including hierarchies) and then require both the client apps and the server-side services share the exact same object model. This is, in effect, a barrier to entry for web developers. It’s as if the web service developers area saying: "Ok, want to use my service? Then first build your app to match the object model this service uses, then we’ll talk."

Many of the implementations for the Web today implement a kind of distributed object style interface. These objects are given URLs as identifiers (e.g. http://example.com/users/mike) and the responses from these URLs look like simple objects:

** REQUEST **
GET http://example.com/users/mike
...

** RESPONSE **
200 OK HTTP/1.1
Content-Type: application/json
Content-Length: XX

{ "user" :
  { "fullname" : "Mike Amundsen",
    "email" : "mike@example.com",
    "phone" : "123-456-7890",
    "address" :
    { "street" : "123 Main St",
      "city" : "Byteville",
      "state" : "MD",
      "zip" : "12345"
    }
  }
}

In fact, these are not actually objects; they’re data structures. It’s not very common to ship responses that include both data (see above) and executable code (as object-oriented programming uses). These kinds of responses start to look very much like the data structures Alan Kay complains about.

Passing Messages Scales the Web

This ‘Object Model First’ style is not the idea that Tim Berners-Lee and Robert Cailliau had in mind when they described the WWW in their 1989 proposal to CERN:

HyperText is a way to link and access information of various kinds as a web of nodes in which the user can browse at will. Potentially, HyperText provides a single user-interface to many large classes of stored information such as reports, notes, data-bases, computer documentation and on-line systems help." [14]
— Tim Berners-Lee and Robert Cailliau 1989

The WWW as originally proposed made it possible for clients and servers to interact without first writing custom code. That’s because the original WWW was based on an agreed-upon format (HTML, TXT, etc.) over an agreed-upon protocol (HTTP). What was not agreed-upon ahead of time was the information passed between client and server — that would always be encapsulated within the message format.

And that is the pattern Fielding found in the ‘libwww` code Berners-Lee wrote when creating the first WWW servers and clients. Not only does this `Message-First’ style closely match the ideas of Berners-Lee and Cailliau, it is one of the features of the WWW that allows it to scale so well. By limiting the amount of agreement needed for clients and server to interact, you are maintaining a low barrier of entry for new clients and servers.

In the WWW-style, the responses contain more than just data, they also contain hints and instructions on how the client application can manipulate the data using additional messages passed to the server. For example, a WWW-style response to a request for the a user identified by the URL http://example.com/users/mike might look like this:

** REQUEST **
GET http://example.com/users/mike
...

** RESPONSE **
200 OK HTTP/1.1
Content-Type: application/vnd.collection+json
Content-Length: XX

{ "collection" :
  {
    "href" : "http://example.com/users/",

    "items" :
     [
      { "href: "http://example.com/users/mike",
        "data" :
        [
          {"name" : "fullname", "value" : "Mike Amundsen"},
          {"name" : "email", "value" : "mike@example.com"},
          {"name" : "phone", "value" : "123-456-7890"},
          {"name" : "street", "value" : "123 Main St."},
          {"name" : "city", "value" : "Byteville"},
          {"name" : "state", "value" : "MD"},
          {"name" : "zip", "value" : "12345"}
        ],
        "links" :
        [
          {"rel" : "edit", "href" : "http://example.com/users/mike;edit"},
          {"rel" : "friends", "href" : "http://example.com/users/friends"}
        ]
      }
    ],

    "template" :
    {
      "data" :
      [
        {"name" : "username", "value" : ""},
        {"name" : "email", "value" : ""},
        {"name" : "phone", "value" : ""}
      ]
    }
  }
}

In this second example, you can easily find the instuctions for crafting new messages (the "template") and hints on other available data (the "links" collection). You can also see that, like the format most commonly used on the Web today (HTML), this response provides no hints about the object model hidden behind the server interface. This is what the message-passing approach looks like on the Web.

Another advantage of using this message-oriented style of implementation is that it simplifies the process of establishing agreement up front. Clients and servers do not need to agree on the object model, they just need to agree on the message format (called a media-type in the standards world) and the various field and actions words used in the domain. Fielding’s way of explaining it is:

"[I]t is far easier to standardize representation [formats] and relation type [names] than it is to standardize objects and object-specific interfaces." [15]

— Roy Fielding 2008

‘OK, I Get the Message!’

So, it turns out that both Alan Kay and Roy Fielding were thinking along the same lines. Kay wanted to promote the creation of large systems that could safely grow over time without having to be stopped, rewritten, and redeployed. And his way of supporting that was through the message-passing paradigm at the code level. He designed a coding approach that hid functionality and exposed messages.

"The key in making great and growable systems is much more to design how its modules communicate rather than what their internal properties and behaviors should be." [16]
— Alan Kay 1998

And Roy Fielidng was describing essentially the same thing. But instead of focusing on the components themselves, Fielding was creating a software style that focused on the connectors — the way information was shared between the components. And this style (called ‘REST’) relied on message-passing, too.

"REST is software design on the scale of decades: every detail is intended to promote software longevity and independent evolution." [17]
— Roy Fielding 2008

So why do many developers today continue ignore this message-passing model and instead use ‘getters’ and ‘setters’ in their source code and pass data structures over the web? There are a handful of reasons. First, it is rare that developers will get training that covers the history of Fielding and Kay and what they originally intended. Second, many of the developer tools are designed to promote getter/setter patterns in source code and distributed objects across the network. Finally, most testing tools focus on exposed objects and properties since they are easier to validate than details within a message.

With all these hurdles, it can be tough for a developer team to maintain focus on what Kay and Fielding said so long ago. We can, however, make it better for all developers when we provide in-depth training, look for tooling that supports these proven paradigms, and provide QA environments that make it easier to test and debug distributed software that scales — all through more reliance on message-passing patterns.

And that can get us all closer to building "growable systems" that can "last for decades."

1. http://en.wikipedia.org/wiki/Object-oriented_programming

2. http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm

3. http://en.wikipedia.org/wiki/Alan_Kay

4. http://roy.gbiv.com/

5. http://www.parc.com/about/

6. http://isr.uci.edu/architecture/

7. http://lists.squeakfoundation.org/pipermail/squeak-dev/1998-October/017019.html

8. https://www.tele-task.de/archive/lecture/overview/5819/

9. http://www.w3.org/History/1989/proposal.html

10. http://tools.ietf.org/html/rfc1945

11. http://en.wikipedia.org/wiki/Library_for_WWW_in_Perl