Written by Martin Führlinger, Software program Engineer Backend
ALL ABOUT MESSAGE BUS
Welcome to the sixth publish on this message bus associated weblog publish collection. I extremely advocate studying the others too, to get the total context. These are:
Now, I need to present extra insights into our documentation round messages and the way we automated this.
Message Documentation
Documentation is difficult. Regardless of who you ask, documentation is at all times onerous to create and most essential hold updated. As at all times, we began with documenting our messages in our firm wiki’s data area. Each time new messages have been added or present ones modified, somebody had so as to add or replace them. The message subjects have been listed in a matrix, containing the producer, the subjects, the shoppers, and a few further notes. It appeared like this:
This course of was sluggish and error-prone. So we wished to enhance this.
The best way to Automate Documentation
Throughout considered one of our Day of New Concepts (DONIs), some colleagues and myself began to work on a small service to routinely doc the present messages in our system. We got here up with a reasonably easy service, named MessageDoc, simply listening to all attainable messages, so to all routing keys (find details about routing here), by listening to `#.#`.
Each time we obtain a message, we parse it and retailer its info. This implies we retailer a doc in a small MongoDB containing the messages meta information, just like the producer, the message itself and a few extra attributes. That is accomplished per matter. The stream of receiving and storing could be seen within the following diagram.
To see how this appears to be like like in our database, right here is an instance:
{
"_id" : "pattern.up to date.altered",
"producer" : "samples",
"description" : "",
"shoppers" : [
<list of consumers>
],
"entities" : [ {
"type" : "run_session",
"updated_at" : ISODate("2020-11-11T14:56:02.304Z"),
"example" : <complete message payload>,
} ],
"model" : 4124,
"created_at" : ISODate("2019-11-28T11:15:26.105Z"),
"updated_at" : ISODate("2020-11-12T06:00:12.950Z")
}
As we don’t need to retailer each single message, we solely replace the message’s content material as soon as a day. We typically have polymorphic messages. This implies we’re sending various kinds of entities inside the identical message sort. As an example, a “sample_message” could be of sort “run_session”, “sleep_session” or others. We retailer just one instance per entity sort.
We all know who the producer of a message is, as it’s a part of the message payload (see how we defined our message content). Each time a doc is written to mongodb, we additionally replace the shoppers asynchronously. We achieved this by querying the Rabbitmq API configuration by way of a HTTP request (utilizing the all-configuration path), parsing it and updating all shoppers of all our saved messages. Particulars about “shoppers” could be discovered within the first blog post on this collection.
Constructing a easy Internet-Interface
To have the ability to entry the saved info with out querying the database instantly, we constructed a easy internet interface on prime of this. Principally, the overview appears to be like just like the earlier matrix within the wiki. It exhibits all subjects grouped collectively by the producer.
When accessing a subject, the subject particulars are proven, with all of the shoppers once more for this particular matter. This additionally means the shoppers of a subject could be completely different within the particulars view, as e.g.”samples.created.accomplished” messages are consumed by different providers than “samples.up to date.altered”. This matter element view additionally permits so as to add some description to the subject.
As talked about above, we retailer an instance message for each matter and each entity sort as soon as a day. These could be proven too by increasing the corresponding entity hyperlink under.
Making Guide Message Documentation Out of date
With this service in place, we have been in a position to deprecate the message documentation within the wiki. Even when we delete information saved within the message doc service, it’s routinely recreating all the knowledge. The one factor which might be misplaced are the extra descriptions per matter which have been added manually. This straightforward service, with lower than 1000 traces of code total, made guide documentation of messages and its content material out of date, and our life simpler.
***
