Times Higher Education

CI/CD pipelines with preview environments

TechAmigos have gone over and above to help transform our Developer experience from consultation and advice to rebuilding our CI/CD pipeline

Robin Mofakham, Director-Engineering

Outcomes Delivered

  • A reusable capability to optionally create ephimeral environments  for feature testing when a PR is raised

  • Tear down feature environments when PR is closed/merged 

  • Improved developer experience with deployment feedback in Github.

The Challenge

THE faced a software challenge where they wanted to introduce a process of verification for branches, before they are ultimately deploy for production. As THE regularly roll-out features for a number of B2C and B2B projects, they wanted to find a way to do this with speed, whilst adhering to a QA process.

Some additional challenges have included:

  • Optimising resource management for CI/CD pipelines, with tear-down strategies and cost controls.

  • Establishing ground-work to enable deploys, which may have complex integrations.

  • Establishing which back-end with dependencies, should be tested against.

  • Accommodating a wide range of configurations for flexibility

  • Improving logging and monitoring

TA (Tech Amigos) have provided tools to facilitate a feature deployments process to address challenges. This is based around the idea of standardising and enriching a CI process and emplacing a verification process when pull requests (PRs) for feature branches are raised.

     Technologies

    • Github
    • Kubernetes
    • AWS – R53, ALB, EKS
    • Terraform
    • Harness

    The Solution

    The premise of the Feature Deployments process, is that coded features can be verified, before they are integrated with mainline code. In the broad sense, the process encapsulates managing risk and enabling velocity, where it becomes easier to test and deploy individual features safely, over a short space of time. The process helps enable a mindset that the main branch for a project should always provide a deployable artefact for production.

    An intermediary Staging environment provides a temporary sandbox for deployments. This is compared to a more traditional approach where a collection of features may meet final verification as part of a ‘feature toggling’ strategy in a production environment. This is where features may be deployed to a pool of end-users for testing, as part of a beta, which can take time to provide feedback.

    TA’s Feature Deployments process for THE enables stakeholders to:

    • Check changes branches with approved PRs can continuously provide a deployable and functional software for production.

    • Run QA/PO tests by default earlier and address when tests fail.

    • Run end ‘build and deploy’ phase earlier in the delivery life cycle.

    Deep-dive into Feature Deployments with Kubernetes

    When a PR is raised for a feature branch, it is allocated a namespace within Kubernetes and is deployed to a Staging environment within AWS.

    With regard to THE, they are deployed to B2C or B2B Kubernetes clusters, within EKS (Elastic Kubernetes Service), dependent on the application they relate to.

    The clusters are hosted within different VPCs, in separate AWS accounts. The clusters also host a Staging namespace, which provision app dependencies as per the Production environment.

    Establishing namespaces per a feature within Kubernetes clusters, serves a clean and simple mechanism for querying and tearing down resources.

     

    THE provides trusted performance data on universities across the globe and ranks them in a league table