Skip to main content

Release Process

This document describes how release and deployment processes are handled in Archipelo and what to consider when making decision about releasing/deploying.

Note Make sure to familiarize with the documentation about our software environments before.

Dictionary​

What is a deployment​

A deployment is a process of pushing a specific snapshot of the product to another environment, and it doesn't necessarily need to be a release to production. Depending on the need, we can deploy new version of the product to dev environment (e.g. to test some specific feature), QA (e.g. to check integration of different components) or to production (to share our latest updates or fixes with the users).

What is a release​

A release happens when a specific version of the product goes live to the public users. In theory, a release is a deployment to production environment.

Deployment Procedure​

Warning The deployment procedure is not in a final state. More precisely, it will never be, since it's an always improving process. This section describes its current state, but keep in mind that we're in a phase of performing many changes to this process.

Deployment procedure varies depending on the environment which is going to be promoted with the latest version of the application. There are several goals of having a well-defined deployment procedure that we need to follow:

  • clear top-down view of the whole process,
  • easy to execute steps with solutions to potential issues,
  • high quality of the releasing version of the product,
  • certainty that specific tests were triggered and passed,
  • easy to maintain optimal way of the deployment,
  • known team members taking part in the process.

Deploying to dev environment​

Currently, there is no automated way of releasing to dev environment.

On each push to your remote developer branch, tests are performed and binaries are built but not pushed anywhere at the moment. If you need the images in your own Container Registry for your dev environment, you need to push them manually from your machine.

Note Soon, we want to have a way to deploy to shared dev environment using GitHub Actions.

To deploy spider or web to the dev environment, you need to have your environment configured and run the following commands:

Deploy spider to dev environment​

gcloud run deploy arspider-<workspace_name> --image gcr.io/archipelo-dev/arspider-<account_name>:latest --region europe-west1 --platform managed

Deploy web to dev environment​

gcloud run deploy arweb-<workspace_name> --image gcr.io/archipelo-dev/arweb-<account_name>:latest --region europe-west1 --platform managed

The names of the services are arweb/arspider followed by a workspace_name, e.g. arweb-rafal1.

The --image parameter specifies the image that is built in the dev environment, and it contains your GitHub account_name. This will usually be something like gcr.io/archipelo-dev/ar[web|spider]-[account_name]:latest, if you want to deploy the version that was built recently.

Deploying to QA environment​

Deployments to QA environment are done automatically and are triggered every time a Pull Request (PR) is merged into main branch of the Archipelo/top repository.

The resulting images are then tagged with the sha1 of the commit and pushed to the Container Registry on the QA project. The latest tag is also applied to the pushed imaged. The Cloud Run services on QA are then updated to run with the newly pushed images.

Note QA version of the WebApp is available here.

Releasing to production environment​

Note Production environment is sometimes called prod in short.

When to release​

There are no strict rules defined when to release the code, but we prefer to make the decision based on data. That's why usually the decision whether to release to production or not is made on Tuesday, when we have more data about the quality of our products from Mondays' Product Quality Review (PQR) done by the QA Team.

We aim to release more often with smaller batches of code. This simplifies detecting potential issues and mitigates the risk of any kind of failure.

How to prepare the release​

Everyone is allowed to request a new release. The action needs to be preceded by asking others on #release Slack channel. If a request is explicitly approved by at least 1 QRB member, release can be triggered. Without explicit approval, the release is halted. It is a good practice to notify others at least 30 minutes before doing a release, to give them some time to respond.

To create new release draft for production, go to our Archipelo/top repository page, click on Releases on the right side and use Draft a new release button on the top, or use this link directly.

How release gets approved​

Warning It is required for our E2E tests on QA to pass before releasing to production! You can find them as one of the workflows in our GitHub Actions.

Each release request to production needs to be approved by at least 1 person from the Quality Review Board (QRB). Current composition of the board is (sorted alphabetically):

  • Dom Sagolla
  • Edmundas Ramanauskas
  • Mateusz Nojek
  • Michał Rychlik
  • Pavel Radzkov
  • Paweł Róg
  • Rafał Kuć

Warning Make sure that the build in GitHub Actions is passing on main branch (see the build badge on the top of the README file in the root of the repository).

Also, please check if QA environment is working properly.

Filling the details of the release​

Each release draft needs to have:

  • a tag (a.k.a. version number) that follows the semantic versioning standard preceded with a v character, e.g. v0.5.9.
  • a title which equals the tag
  • release notes which can be generated automatically by using Generate release notes button. It is recommended to adjust automatically generated release notes manually.

Warning Before the first public release, we follow a custom versioning approach with a version template 0.X.Y, where X is a major version, Y is a minor version, and there is no patch version number at the end.

How to trigger the release​

When release draft is ready, click on Publish release button which will trigger the GitHub AR-Deploy workflow and initiate jobs that deploy our APIs (arspider and arweb) to production environment. On the repository's actions page , you should see your deployment as a job named after the release title. From the moment when the job finished successfully, a new versions of the API and Spider are running on production.

Warning At the moment, only the Cloud Run services are updated through a GitHub release process. To release a new version of the Chrome extension, you need to manually send a new version to the store.

Post-release actions​

Each release needs to be verified in production, especially the changes introduced in the release. It is a responsibility of the person making the release. Because of the fact we do not have automated tests on production, we need to conduct the testing manually.

Slack notifications are being sent automatically to #release channel with the deployment status for the specific components along with the full release notes.

What is released​

Considerations At the moment, we are using the monorepo approach to managing the code, which means that all the code lives in a single repository. Each engineer should remember to keep that in mind when developing and releasing the code to both QA and production environments. The changes in the backend code must be compatible with the currently running production code - if you are doing code that includes breaking changes, you need to release each component at the same time.

There are different components in our repository that can be released:

  • Backend
    • web API
    • Data Storage
  • Frontend
    • Web Application (a.k.a WebApp)
    • Chrome Extension
    • VS Code Extension
  • Data Pipelines (spider service)

Backend, data pipelines and WebApp are published on production with each release.

Chrome Extension needs some manual steps to trigger its release.

VS Code extension is currently not released publicly. More about releasing frontend components in section below.

Frontend components​

WebApp is pushed to production with each release.

OneBox - Chrome Extension release process is separate from the process written here. Whole step-by-step instructions can be found in a chrome_extension_release.md file.

VS Code extension is not currently released publicly. Only an artifact is produced during our build process. It will be soon automatically uploaded to Google Cloud storage for easier access.

Note It's possible to create a pre-release, which will deploy only the frontend part of the product to QA. More about it here.

Improvements to release process​

Current process is not ideal, and we are aware of it. We aim to improve it regularly. The changes need to be small and implemented gradually. Each change should be documented here in this document. When in transition phase between different approaches, they should be properly explained to avoid any confusion.

TODO​

  • Release Schedule
  • Quality Gates
  • GitHub Actions (checks, required checks etc.)