Serverless Camunda Terraform Recipe using Cloud Run and Cloud SQL
Camunda is an excellent piece of infrastructure to host business logic. Business logic is dynamic, so it makes sense to express it as configuration and provide tooling for business users to visualize what is going on. Camunda ticks these boxes, but its deployment is a bit old fashioned, requiring dedicated machines and the headaches that implies. I thought, wouldn't it be cool if we could run Camunda as a serverless process?
Persistence with Cloud SQL
Camunda requires a relational database. I used Google's managed Cloud SQL. Hardly any ops required! I put Camunda in its own database namespace, so we can reuse the SQL service for other applications. I also provisioned a unique user, so we can track Camunda's access uniquely.Embedded content: https://gist.github.com/tomlarkworthy/9e8d745b83907177d01c4e54f16146f9?file=cloudsql.tf
Customizing the Camunda Docker Image With Cloud Build
Camanda very helpfully supplies a basic docker image, but we need to customize it. We will typically need to do this for two reasons. First, we need to supply a custom database driver to auto handle Google Cloud's certificate rotations. Second, we need to load our Camanda runtime customization. For now, I just show you how to load the custom driver, but the approach extends to the second case that you will also probably need to do.
First, we mirror the public docker image into our projects private image repository. We don't want our assets on the general internet! Plus, it's much faster for Cloud Build to work with images hosted on Google Container Registry.Embedded content: https://gist.github.com/tomlarkworthy/9e8d745b83907177d01c4e54f16146f9?file=docker.tf
Next, in a Dockerfile, we start with the mirrored images, install wget, delete the old postgres driver and download Google's driver into the right place. There is also some bullshit with linux USER accounts to get apk working properly.Embedded content: https://gist.github.com/tomlarkworthy/9e8d745b83907177d01c4e54f16146f9?file=Dockerfile
How did I know where to install the driver? I shelled into a container to explore the filesystem with the command
docker run -it camunda/camunda-bpm-platform:7.12.0 /bin/sh
To build the container I use Cloud Build invoked at the command line through gcloud. We can express this with a null_resource.Embedded content: https://gist.github.com/tomlarkworthy/9e8d745b83907177d01c4e54f16146f9?file=build.tf
That's a bit ugly, but for a real deployment I would build cloud automation in a CD/CI pipeline. So this step would not normally be in Terraform, so customize this step to suit your needs. The important thing is that we have offloaded the image building to a cloud hosted process, and that process will place a custom image in our private image repository.
Hosting Camunda Workers on Cloud Run
We use Cloud Run to host the Camunda service. This will spin up docker processes on demand, based on the customized docker image we built earlier. We need the service exposed on the public internet to access it, and, even though Camunda has authentication, I feel there is risk associated. So we provision a locked down service account, so the Camunda process only has access to Cloud SQL.
By default, Cloud Run does not allow public access to its end points, so we first create a public cloud run invoker policy and bind that to our Camunda service to expose it. Then we create a custom service account with just Cloud SQL client permissions.Embedded content: https://gist.github.com/tomlarkworthy/9e8d745b83907177d01c4e54f16146f9?file=security.tf
Finally we run the service. We have to pass the database credentials in through env parameters. We have to customize the database URL and threadpool, to prevent too many instances of Camunda starting so we stay within the connection limits of our tiny Cloud SQL test instance. After running
terraform apply, it should be up! Check the cloud console to find the domain. You can play with mine at https://camunda-flxotk3pnq-ew.a.run.app/camunda-welcome/index.html.
Stop Writing Business Logic, Start Hosting Business Workflows
Camunda ships with an REST API. You can use these workers as a service in a larger deployment. I myself have big plans to make delivery of custom business automation simpler. Stay tuned! The full code is on Github
- Tom LarkworthySenior Cloud Architect