It’s All About Messages -I-

double-black-hole-quasar

In my previous post I talked about a architectural concept I had defined for my trading platform. Looking backward from now, it was a good decision to go for a service oriented architecture. But how do all the communication is done in a distributed system? Lets talk about this, and try to find some answers. Messaging is one of the most fundamental topics in computer science and its so important. I guest one post wont be enough to cover all important aspects, but lets see where we land… To get a preview about upcoming topics, please have look here: Content++.

Lets start with flying over some basics. I won’t go to much in detail cause there are tons of great resources about messaging. Messaging is one of the fundamental and most complex topics you can find in computer science and if you ask me, together with algorithms its one of the most important. So lets quickly clarify some terms to get a similar idea what we are talking about. First we should define what is a message.

What is a message?

A message is a information “segment” which is transferred from Alice => Bob. But how knows “Bob” whats inside the segment? To take care “Bob” got the right interpretation about whats inside he segment, Alice and Bob must use the same “protocol“. The protocol defines how the message segment is structured. There are many protocols out there and it depends on the use case which protocol to chose. The use case is grouped into a “layer” which describes a protocol is suited for. To get a overview about it, please have a look here: Internet Protocol Suite.

Typically a message segment contains a “header” and data “section“. The parameters in this sections are protocol dependent. That means the protocol tells you how much and what kind of information the header and data section contains.

So lets try to summarize it a little for our needs: A message is a information segment containing header and data sections which are interpreted by a protocol. So Alice and Bob can communicate via network.

Bild1.pngpic.1: example of TCP message segment with header section and data section

The protocol helps you to keep control about all your messages and its important to use the right protocol depending what you want to achieve and your priorities. Beside the layer group, there are two categories of protocols “connection-oriented” and “connection-less“. In short, if your priority is reliability take a connection-oriented protocol, if minimal latency is you focus no matter if all messages will arrive take connection-less protocol. An example for each category is TCP and UDP.

Serialization

OK, I hope you are still with me because now I want to get a little more explicit. A typical message we have to deal with would be a price quote. This quote is just a number, lets say type of decimal. But if we don’t want to send or receive random numbers without knowing the instrument they belong to, we need to give quotes attributes before sent. Here is the point where serialization/de-serialization comes into the message soup. In short, serialization brings the all properties of an object to a format that can be send as a message. On the other side from the wire the message is de-serialized to a copy of the origin object. By doing this we can send a quote with price, instrument, date, … properties and reconstruct anything on the other side. In case this is not quite clear now, please let me show you an example what serialization is doing in practice.

For instance, lets say we want to send a new price quote trough the wire starting with define a class PriceQuote containing some properties. Thanks to .Net we can use the build in WCF serializer stuff for our convenience. The only thing to do is to give the class of the object you want to send the attribute DataContract. All members we want to send get the DataMember attribute. That’s it!

    [DataContract]
    public class PriceQuote
    {
        [DataMember]
        public string Instrument { get; set; }
        [DataMember]
        public decimal Price { get; set; }
        [DataMember]
        public DateTime Date { get; set; }
    }

This allows us to apply the DataContract-Serializer against any instance of the class PriceQuote, to conserve the object with members and their values. So lets put some values into the fields:

    class Program
    {
        static void Main(string[] args)
        {
            var newPriceQuote = new PriceQuote
            {
                Instrument = "EUR/USD",
                Date = DateTime.UtcNow,
                Price = 1.23456M
            };

            var serializer = new DataContractSerializer(newPriceQuote.GetType());
            var streamToRam = new MemoryStream();
            serializer.WriteObject(streamToRam, newPriceQuote);
        }
    }

Finally we serialized the object newPriceQuote and write it into memory for any further use, e.g. sending through the wire as a message. On the other side the received message need to be de-serialized to reconstruct the object and do further processing.There are 3 common types of serialization format’s, XML, JSON, binary, I want to mention here. XML and JSON are read able for normal humans but binary format, hmm, as you may know: “There Are Only 10 Kinds Of People. Those Who Understand Binary And Those Who Don’t”.-)

Sending Messages

To send the serialized message, we just need to put the serialized object into WCF OperationContract which is part of a WCF ServiceContract. To see this more in detail, please have a look over here: Link. If you don’t like the hidden concept of WCF and want the pure socket stuff, you should play around with this: Link.

Finally, for messaging we have the message, a protocol and a serializer to do all the stuff. The most common protocol I found during my research is TCP. Of course, it fits perfectly to my demands. Another option would be UDP but if you don’t want to play with HFT were latency matters only, forget about it. If you go with UDP you pay a high price in terms of reliability. My choice was to start with TCP protocol.

So, that’s for now about messaging basics. In my next post I want to talk about different message libraries and frameworks. For doing this I will go back to “Technologies Screening” topic. If you have some comments don’t hesitate to contact me. To get a preview about upcoming topics, please have a look here: Content++.

 

CZVfD4pWcAEWTWj

 

quantocracy-badge-130

Advertisements

2 thoughts on “It’s All About Messages -I-

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s