Build and Push Docker Images to a Private Registry from Azure Devops Pipeline

How did we automate a WSO2 Deployment, CI/CD with Azure Devops — part 02

automate wso2 deployments with azure devops cicd pipelines — part 02

This is the 2nd post of a series of posts about Automating a WSO2 deployment using Azure Devops pipelines. Links to all the posts can be found in the bottom of this article.

In this post Im going to talk about how to write pipelines in Azure Devops for a WSO2 deployment.

Build and Push Docker Images

As I mentioned in my previous post, we had repositories for each wso2 product to build docker images. We could have used docker images from wso2 official docker registry and use those images as base images and create our own images. Customer preferred the other way which is to download the latest wum zip pack and built docker images on our own from the scratch. In this way we had the full control over the underlined OS, JDK etc of the docker image.

We used wso2 official docker source repositories and customised them.

API Manager — https://github.com/wso2/docker-apim/tree/2.6.x/dockerfiles/alpine

Enteprise Integrator — https://github.com/wso2/docker-ei/tree/6.4.x/dockerfiles/alpine

Below is the directory/file structure we had in our docker-wso2am repository

docker-wso2am : $tree
.
├── Dockerfile
├── README.md
├── azure-pipeline.yml
├── files
│ ├── keystores
│ └── lib
│ └── sqljdbc42.jar
└── init.sh

Stage : Get WUM Updated pack

We achieved this in single step and 4 tasks.

  1. A cache task to cache this wum update, therefore pulling updates from the WSO2 wum server will be done only when we have changed the wum timestamp. Otherwise it will use the same wum updated pack in every pipeline run. This way we can reduce the pipeline execution time and we are not wasting network bandwidth.
Cache task’s classic configuration

2. CommandLine task to get wum installation.

3. CommandLine task to pull wum updates from wso2 central update server.

4. PublishPipelineArtifact task to publish the wum updated zip pack to Azure artifact repository.

Publish Pipeline Artifact task’s Classic configuration

Please find below the pipeline code of the Stage for Getting wum updated zip pack publish it to artifact repository.

Stage : Get WUM Updated WSO2 pack

Stage : Build Docker image

Special case in build stage, we have added a condition to check the branch name. In that way we can have the same content for pipeline.yaml in all the branches yet execute stages selectively depending on the branch.

We have added another condition to depend on the WUM update stage i.e this build stage will be executed only if previous(WUM Update stage) has been executed.

We have different stages for each Dev, Test and Prod with conditions to check the repository branch which this pipeline is being executed.

We achieved this using several tasks

  1. DownloadPipelineArtifact task to download the wum updated zip pack from internal artifact repository published in the previous stage.
Download Pipeline Artifact task’s classic configuration

If you have any custom jar files need to be burned in to docker image (probably into dropins or lib directory) you can have the same Download pipeline artifact to retrieve a built artefact from another pipeline.

eg: We had a custom component that needed to be added to dropins directory of API Manager. We had the source code in a separate repo and this repository had its own pipeline and whenever a commit it pushed, the pipeline will be executed and the built artefact(.jar) is pushed to the Azure Artefact repository. We can retrieve artefacts not only from current pipeline, but also from any other pipelines too. We had added this in each Build_Docker stage.

Classic UI configuration for retrieving artifact from another build

2. DownloadSecureFile tasks to download and copy java keystores(.jks files)

Download secure file task’s Classic configuration

3. Docker task to build docker image and push to Azure internal docker registry. We tag the built docker image with three different tag names at each build. BuildId, WUM timestamp and with the tag ‘latest’.

Docker task Classic configuration

Please find the pipeline segment for Building and pushing Docker image with wum updated wso2 pack.

Stage : Push Docker image to Openshift registry

We need to push the newly built docker image to openshift. For each wso2 product/component we have added a trigger in the deployment such that when there is a new image pushed to Openshift Registry, there will be a rolling out update to the deployment in that environment.

Another beauty in Azure devops is you can have agents on the cloud as well as on premise to run these jobs. For pulling docker image from Azure internal docker registry and push to Openshift Internal docker registry is done by a on premise agent. This on premise agent has access to both Azure registry and Openshift Cluster.

push docker images to openshift docker registry

Thats it folks , I hope it was helpful :)

Links to other posts in this series…

Part 01

Part 02

Part 03

Part 04