Upon designing the Basar platform

The last blog entry about designing an asynchronous ecommerce platform dates back to March 2013 which is ... quite a while. As things slightly changed a bit since than (I switched departments, projects where abandonded and so on) I was unfortunately too busy to work on that topic intensively. With some fresh new ideas, encouragement gained from personal talks and some interesting hints on technology retrieved from talks at this year JavaZone, I will get into more detailed work during the upcoming weeks.

Actually, there are two things, I would like to talk about in this blog input: (1) project name change and (2) design decisions.

The first topic is short and has a short explanation as well: I changed the projects name from iServices to Basar as it - from my perspective - expresses the intention of the platform much better than the former name. But names are just sound and smoke, therefore read on for some architectural facts whereas you can find the conceptual aspects in another blog post.

For several reasons I always found the idea of having loosely coupled services formed into a larger integrated system very intriguing. Amongst others these are my arguments:

1. clear separation of concerns, even across teams
2. only a small number of global design requirements along with a larger responsibility mapped to the teams
3. perfect match of technologies applied inside components to solve local problems - no need to swing the hammer
4. easy exchangeability of features
5. easy extensibility

For some more - very good - reasons see the video of James Lewis about "Micro-Services - Java, the UNIX way held at JavaZone 2013.

But where does loosely coupling start and where does it come to an end? Is it just like throwing some guys into the ring which have never met before or is there a common foundation they all adhere to like common frameworks or concepts? .... as always: it depends on the context. Or to say it with the words of Harold Samuel: location, location, location

In this case, the desired goal to provide a mixture of SaaS and PaaS (as a combined product) along with a heavy need to decouple the single building blocks, I decided to go for the communication as common foundation. Rather than on a technical level I choose the conceptual/business level to define the common base. Therefore all services belong to service types which in turn have a defined communication protocol. If a new component wishes to add some recommendation engine features it must adhere with the requirements defined for reco engines inside the platform ... or ... for any good reason, extend the platform and provide a version 2 of the recommendation engine service type.

What we have so far is a set of textual service representations, the desire to leave components in a certain type of isolation and provide the chance to extend the platform if required. Now comes the technology challenge: what paradigm to follow and which framework to use.

As I have stated in an earlier post the platform will definitely be build on an asynchronous foundation. I believe that asynchronous information processing offers the opportunity to leverage the existing hardware at its best - well, you can still mess things up but achieving the opposite is made easy through the paradigm itself.

Another decision made on the path of technology challenge is to let the asynchronous communication be carried out through message passing along with the strong requirement that particular context knowledge must be represented through the message state only. What does that mean, does the platform not have any clue about which reality it represents? No, that's wrong. The building blocks/components have a state but that state is independent from any special context represented inside a message. The message is used to aggregate knowledge from the component which in turn is incorporated into the context of the message but the component does not hold any specific state for any specific message.

Take a recommendation engine for example. It knows about customers, products and which products to offer to which customer. When a customer message requesting a recommendation comes in, it knows - depending on the message state - which data to respond. The component has its own state but is not dependent on the message nor does it keep anything that is bound with the message. However, the message might initiate a state change but that in turn is global and visible to all upcoming messages.

To draw a picture on how I see the platform imagine a large set of chaotically ordered neurons where each neuron may be connected with an arbitrary number of other neurons. A message put into this mesh is ping-ponged between different neurons, each carrying out the task requested until - if requested - the response finally is spilled out to the calling party. Along its path it may have an impact on data or state provided through different neurons, like changing the set of recommendable products as the message told everyone that the last pair of Levi's just left the stock.

Now over to the final stept: which framework to use?

Over the last two years I became fascinated by the Akka framework and its simplicity towards implementation and application management. As it provides me with all required building blocks I decided to go after it and use it as foundation ground.

Now that I have defined the paradigm to follow, the communication methodology to use and the framework to apply: how does this may fit with the idea to have a loosely coupled system as the messages needs to be passed around ... somehow ... and the natural answer would be to use an Enterprise Service Bus? The latter would couple things again more tightly, may become a bottleneck AND is so old skool, thus I need to find another approach.

The challenge was to bring together message passing and loosely coupling along with distributed components. Somehow it is like mashing up parts of Amazons two-pizza rule together with message passing and loosely coupled components into a new approach.

The idea is to have components that accept certain messages and produce defined output - both known to the entire platform and defined on service specification level. These components register themselve on a central instance which keeps track on which service type is implemented by which service component. Additionally it knows in which cases a specific component accepts incoming messages of a certain type. If a service wishes to communicate with another service it looks up its actor reference from the central phonebook, checks which messages it accepts (eg. customer age must be 25) and starts communicating.

Having the core foundation layed out the upcoming posts will address different problems - technical as well as conceptual ones - that needs to be solved to have a consistent architecture.