Event-Driven Architecture in the Media Magic Platform

The Media Magic Platform is moving in the direction of an event-driven architecture. There are many scenarios within the platform which are fundamentally event-driven in nature.

For instance, when a user creates a deployment, which could potentially take several minutes for certain models. We may wish to send them an email notification, or we may wish to trigger an update to the UI. Other examples could be more common to architectures in genera. For example, when a user has registered, we may wish to send them an email to verify their email address. All of these scenarios are 'events' that have occured within the platform.

There may be several different microservices interested when a specific event happens. Handling this synchronously would mean handling several gRPC requests/responses, which asides from requiring a lot of code, is wasteful when we don't neccessarily care about the response.

We can deploy a 'fire-and-forget' approach to these situations. A deployment.created event may trigger a whole host of reactions across the platform, from sending emails, to updating analytics. But, analytics and emails aren't a concern for the deployment service. Therefor, events maintain a greater degree of separation between our services and their responsibilities.

What do we use?

Every service has a NATS_ADDR environment variable set: https://github.com/pingponglabs/mediamagic-platform/blob/f3e9b4748468b8515efc6d9d083ce96c03fcabf2/libs/infrastructure-components/container/container.go#L63

We have a NATS cluster deployed into the nats namespace, within the platform's K8s cluster.

How do we use it?

We have a handy library we can use for doing basic publish/subscribe operations: https://github.com/pingponglabs/mediamagic-platform/tree/main/libs/pubsub

This library has two modes of operation. Publish/Subscribe and PublishEvent/SubscribeEvent.

Publish and Subscribe, both take a generic []byte argument as the event data. Whereas, PublishEvent and SubscribeEvent, take a specific Event type.

If you want to publish an event which tracks the amount of times something has happened across the platform, then used PublishEvent. This has fields specific for tracking these types of events.

If you want to publish a more generic event, for example, a user has registered, then use Publish. This is more flexible.