Build and Push Docker Images to a Private Registry from Azure Devops Pipeline
How did we automate a WSO2 Deployment, CI/CD with Azure Devops — part 02
This is the 2nd post of a series of posts about Automating a WSO2 deployment using Azure Devops pipelines. Links to all the posts can be found in the bottom of this article.
In this post Im going to talk about how to write pipelines in Azure Devops for a WSO2 deployment.
Build and Push Docker Images
As I mentioned in my previous post, we had repositories for each wso2 product to build docker images. We could have used docker images from wso2 official docker registry and use those images as base images and create our own images. Customer preferred the other way which is to download the latest wum zip pack and built docker images on our own from the scratch. In this way we had the full control over the underlined OS, JDK etc of the docker image.
We used wso2 official docker source repositories and customised them.
API Manager — https://github.com/wso2/docker-apim/tree/2.6.x/dockerfiles/alpine
Enteprise Integrator — https://github.com/wso2/docker-ei/tree/6.4.x/dockerfiles/alpine
Below is the directory/file structure we had in our docker-wso2am repository
docker-wso2am : $tree
.
├── Dockerfile
├── README.md
├── azure-pipeline.yml
├── files
│ ├── keystores
│ └── lib
│ └── sqljdbc42.jar
└── init.shStage : Get WUM Updated pack
We achieved this in single step and 4 tasks.
- A cache task to cache this wum update, therefore pulling updates from the WSO2 wum server will be done only when we have changed the wum timestamp. Otherwise it will use the same wum updated pack in every pipeline run. This way we can reduce the pipeline execution time and we are not wasting network bandwidth.
2. CommandLine task to get wum installation.
3. CommandLine task to pull wum updates from wso2 central update server.
4. PublishPipelineArtifact task to publish the wum updated zip pack to Azure artifact repository.
Please find below the pipeline code of the Stage for Getting wum updated zip pack publish it to artifact repository.
Stage : Build Docker image
Special case in build stage, we have added a condition to check the branch name. In that way we can have the same content for pipeline.yaml in all the branches yet execute stages selectively depending on the branch.
We have added another condition to depend on the WUM update stage i.e this build stage will be executed only if previous(WUM Update stage) has been executed.
We have different stages for each Dev, Test and Prod with conditions to check the repository branch which this pipeline is being executed.
We achieved this using several tasks
- DownloadPipelineArtifact task to download the wum updated zip pack from internal artifact repository published in the previous stage.
If you have any custom jar files need to be burned in to docker image (probably into dropins or lib directory) you can have the same Download pipeline artifact to retrieve a built artefact from another pipeline.
eg: We had a custom component that needed to be added to dropins directory of API Manager. We had the source code in a separate repo and this repository had its own pipeline and whenever a commit it pushed, the pipeline will be executed and the built artefact(.jar) is pushed to the Azure Artefact repository. We can retrieve artefacts not only from current pipeline, but also from any other pipelines too. We had added this in each Build_Docker stage.
2. DownloadSecureFile tasks to download and copy java keystores(.jks files)
3. Docker task to build docker image and push to Azure internal docker registry. We tag the built docker image with three different tag names at each build. BuildId, WUM timestamp and with the tag ‘latest’.
Please find the pipeline segment for Building and pushing Docker image with wum updated wso2 pack.
Stage : Push Docker image to Openshift registry
We need to push the newly built docker image to openshift. For each wso2 product/component we have added a trigger in the deployment such that when there is a new image pushed to Openshift Registry, there will be a rolling out update to the deployment in that environment.
Another beauty in Azure devops is you can have agents on the cloud as well as on premise to run these jobs. For pulling docker image from Azure internal docker registry and push to Openshift Internal docker registry is done by a on premise agent. This on premise agent has access to both Azure registry and Openshift Cluster.
Thats it folks , I hope it was helpful :)
Links to other posts in this series…
Part 01
Part 02
Part 03
Part 04
