Mistral logo

Create GIT Tags In The Form Of Semantic Build Numbers Using AWS CodeBuild & CodePipeline

18.06.2018 - READING TIME: 5 minutes

Create GIT Tags In The Form Of Semantic Build Numbers Using AWS CodeBuild & CodePipeline

Authors: Srđan Milovanović and Amar Bašić

More about semantic versioning can be found here. 

 

For quite some time now we’ve been using semantic build numbers to tag successful builds in tools like Jenkins and TeamCity in order to have the usual overview of all builds.

Some of the clients we’ve been working with asked us to use tools like GitLab CI and AWS CodeBuild to implement CI and CD processes, but the semantic build number feature wasn’t and still isn’t supported for some reason. We’d been searching for any kind of solution that could support it, but without much success.

Then, on the latest project we’ve been working on, we had a real need for numbered builds in order to track our releases. That’s where we got an idea to create it ourselves and completely automate the process of tagging.

Since a tag relies on a specific branch, we wanted to create a tag with the following pattern:

branchName-majorVersion.minorVersion.PatchVersion

For example:

develop-2.1.0 

Here’s how we managed to do it:

We knew that we needed to create a Tag on Git repository and it can be accomplished using standard Git CLI to list and create Tags.

Therefore, we created a new AWS CodeBuild project, used AWS’s Base Ubuntu 14.04 version runtime and created the following buildspec.yml file:

version: 0.2
env:
variables:
BRANCH_NAME: “develop”
MAJOR_VERSION: “1”
MINOR_VERSION: “0”
GIT_REPO: “https://git-codecommit.us-east-1.amazonaws.com/v1/repos/SomeProjectName"
parameter-store:
GIT_USER: “AS-GIT-USER-NAME”
GIT_PASS: “AS-GIT-PASSWORD”
phases:
install:
commands:
- apt-get update -y && apt-get install -y git
pre_build:
commands:
- echo “machine git-codecommit.us-east-1.amazonaws.com” >> ~/.netrc
- echo “login $GIT_USER” >> ~/.netrc
- echo “password $GIT_PASS” >> ~/.netrc
- git clone -b $BRANCH_NAME $GIT_REPO
build:
commands:
- cd SomeProjectName
- bash git_taging.sh


The explanation of the buildspec.yml function file:

We created six environment variables; four “regular” environment variables, and two (GIT_USER and GIT_PASS) mapped on the EC2 Parameter Store variables.

BRANCH_NAME - Branch name on which we’re creating our new tag and from which we’ll clone our repository.

MAJOR_VERSION - Major version for our build number. If and when needed, it has to be updated manually in our buildspec.yml file. When major version is changed, script restarts counters to zero for minor and patch version numbers.

MINOR_VERSION - Minor version for our build number. If and when needed, it has to be updated manually in our buildspec.yml file. When minor version is changed, script restarts counter to zero for patch version numbers.

GIT_REPO - This is an environment variable which stores our git repository https URL.

Additionally, we added two more environment variables that store values of our git username and git password in EC2 parameter store. We did it this way for security reasons and not to hold these credentials in a clear text format.

GIT_USER - Environment variable mapped on AS-GIT-USER-NAME EC2 Parameter Store variable.

GIT_PASS - Environment variable mapped on AS-GIT-PASSWORD EC2 Parameter Store variable.

 

Since we’re using Ubuntu 14.04 base image, we needed to install git in order to be able to log in, clone our repository, create and push a new tag.

We accomplished git installation with the following command in buildspec.yml file:

apt-get update -y && apt-get install -y git

 

After installing git, we had to store our credentials and repository provider URL in .netrc file in a user’s home directory in order to be able to automatically login to Git repository without any user interaction. The following three commands inserted these parameters into the previously mentioned file:

echo “machine git-codecommit.us-east-1.amazonaws.com” >> ~/.netrc
echo “login $GIT_USER” >> ~/.netrc
echo “password $GIT_PASS” >> ~/.netrc

 

The next step was to clone our Git repository into ephemeral AWS CodeBuild image:

git clone -b $BRANCH_NAME $GIT_REPO

Just to emphasize; in order to create a tag, git commands must be executed from the Git repository containing .git folder, which is why we entered into SomeProject directory before executing the bash script for tag creation.

cd SomeProjectName
bash CICD/git_taging.sh

 

Besides the fully automated Git login, repository cloning and creation of the new tag, this whole procedure relies on the bash script written by my colleague Amar Bašić. The script auto increments build number and /or resets counters to zeros if major or minor version is changed on each segment on the right side of the changed segment (already explained with the environment variables above).


Below is the script that does all the whole magic, and the explanation of how it does what it does:

#!/bin/bash
major_max=1;
minor_max=0;
patch_max=0;
branch_name="${BRANCH_NAME}"
$(git fetch --tags)
last_tag=$(git describe --abbrev=0 --tags)
if [[ $last_tag ]]; then
version=$(echo $last_tag | grep -o '[^-]*$')
major=$(echo $version | cut -d. -f1)
minor=$(echo $version | cut -d. -f2)
patch=$(echo $version | cut -d. -f3)
if [ "$major_max" -lt "$major" ]; then
let major_max=$major
fi
if [ "$minor_max" -lt "$minor" ]; then
let minor_max=$minor
fi
if [ "$patch_max" -lt "$patch" ]; then
let patch_max=$patch
fi
echo 'Latest version:' $major_max'.'$minor_max'.'$patch_max
let patch_max=($patch_max+1)
fi
if [ "$major_max" -ne "${MAJOR_VERSION}" ] || [ "$minor_max" -ne "${MINOR_VERSION}" ]; then
major_max="${MAJOR_VERSION}"
minor_max="${MINOR_VERSION}"
patch_max=0
fi
echo 'Switching to new version:' $major_max'.'$minor_max'.'$patch_max
$(git tag -a $branch_name-$major_max.$minor_max.$patch_max $branch_name -m "Version $major_max.$minor_max.$patch_max")
echo 'Push tag to remote'
$(git push origin $branch_name-$major_max.$minor_max.$patch_max $branch_name)


After cloning the repository, we needed to fetch all the tags related to the current branch.

$(git fetch — tags)

The next thing we wanted to know is what was the last tag we created:

last_tag=$(git describe — abbrev=0 — tags)

Executed command, for example, should return the following result as our latest tag: develop-1.0.5.


What we needed to do next is to extract those numbers from our tag so we could later use it to compare with the values of previously mentioned environment variables in our buildspec.yml file, and then update these values for our new tag. This can be done using the following commands:

version=$(echo $last_tag | grep -o ‘[^-]*$’)
major=$(echo $version | cut -d. -f1)
minor=$(echo $version | cut -d. -f2)
patch=$(echo $version | cut -d. -f3)


Furthermore, we needed to check if these values were greater than the initial version 
1.0.0 and to increment our patch segment with+1. We assumed that our initial version was 1.0.0 and we set major_max, minor_max and patch_max values with these values.

if [ “$major_max” -lt “$major” ]; then
let major_max=$major
fi
if [ “$minor_max” -lt “$minor” ]; then
let minor_max=$minor
fi
if [ “$patch_max” -lt “$patch” ]; then
let patch_max=$patch
fi
echo ‘Latest version:’ $major_max’.’$minor_max’.’$patch_max
let patch_max=($patch_max+1)


Also our environment variables (MAJOR_VERSION and MINOR_VERSION) had to be checked. If they were different from the values in the latest tag (for example our current values are 
major_max=1, minor_max=0 and patch_max=5), we would have needed to reset counters on each segment of our semantic build number on the right side of the changed value.

if [ “$major_max” -ne “${MAJOR_VERSION}” ] || [ “$minor_max” -ne “${MINOR_VERSION}” ]; then
major_max=”${MAJOR_VERSION}”
minor_max=”${MINOR_VERSION}”
patch_max=0
fi


Finally, when we set our new build version, we needed to create a new annotated tag and then push it back to the repository.

$(git tag -a $branch_name-$major_max.$minor_max.$patch_max $branch_name -m “Version $major_max.$minor_max.$patch_max”)
$(git push origin $branch_name-$major_max.$minor_max.$patch_max $branch_name)

And, that’s it :)


This method enables you to manage your environment branches with automatic tagging and setting environment variables. We hope you will find it useful, efficient and time-saving, as we did!

Happy Git Tagging in form of semantic build numbers!