Service Infrastructure

A note on infrastructure as code at Media Magic

The Media Magic platform uses Pulumi as its primary IAC framework. Some of the older components are using Terraform, but those are being phased out in favour of pulumi.

Shared infrastructure, for example, infrastructure not belonging to a specific service live in infrastructure/new. This includes things such as ingress rules, and platform wide features.

Creating infrastructure for microservices

Each service should have an infrastructure directory in the root of the service.

To create a new infrastructure project, run $ pulumi new. Use the service name, for example payment-service as the project name and use dev as the stage for now (we will create a prod stage once we're creating a prod/staging set-up).

Select kubernetes-go as the type.

You should see a main.go file created with some example code. We can remove this sample code.

We have created some useful abstractions to save us time setting up new services. These can be found in the package libs/infrastructure-components. Take a look through that repo, and take a look at the packages and the functions available. These ensure consistent naming conventions etc.

The easiest route to get something up and running is using the scaffold package. This will take a yaml file with some basic configuration options, and turn them into infrastructure as code.

Your main.go file will look like:

package main

import "github.com/pingponglabs/mediamagic-platform/libs/infrastructure-components/scaffold"

func main() {
	scaffold.Run("./service.yaml")
}

How easy is that?

Now to take a look at the configuration options:

- dockerpath: "../"
  name: "deployment-service"
  namespace: "mediamagic-dev"
  project: "mediamagic-core-ewanv"
  type: "grpc"
  env:
    MEDIA_GRPC_ADDR: "upload-service:9000"
  command:
    - "./service"
    - "service"
  ports:
    grpc: 9000

These are the minimally required options. As you can see, we define a service name, a namespace (which is always mediamagic-dev for the time being), we can define our environment variables and the run command for that service.

We can also define our ports. Which in turn, get turned into the port configuration for the pod, the service and the are automatically set as environment variables as well.

For example, grpc: 9000, will get set as an environment variable GRPC_ADDR: 9000. So bare that in mind when you write your connecting code.