Mozilla VPN Release Process#

Mozilla VPN is shipped in two parts, the client and addons.

The client is shipped on Windows, Mac and Linux, as well as Android and iOS. Ensure you are in the #releaseduty channel in Matrix.

Find the Release Promotion Action#

Many of the steps below use a release promotion action, these all share the same initial steps:

  1. Navigate to the Decision task of the push in Taskcluster. Ensure it is the one for the github-push event, it will be called Decision (push).

  2. Navigate to the task group.

  3. From the three dot menu in the bottom right, select Release Promotion.

Windows and Mac Clients#

Promote a Build#

You’ll receive a ping in the #releaseduty channel in Matrix asking to “promote a build” or “push a build to candidates”. The request should include a link to the commit they would like promoted.

  1. Find the Release Promotion Action.

  2. Change the release_promotion_flavor to promote-client.

  3. Trigger the action. This will run beetmover tasks that uploads the signed builds to the candidates directory on archive.mozilla.org. The builds will be under the path with the following pattern: pub/vpn/candidates/<version>-candidates/build<buildId>/

  4. Update Balrog, making sure to use the FirefoxVPN: release-cdntest channel.

  5. Reply in-thread that the candidate builds are ready for testing.

Ship a Build#

You’ll receive a ping in the #releaseduty channel in Matrix asking to ship or release a build. The request should include a link to the commit they would like promoted.

Note

As of this writing, the ship phase doesn’t actually depend on the promote phase as the build signing tasks run on-push. So for now the action can simply be triggered from the github-push Decision task the same as the promote phase. This will change in the future.

  1. Find the Release Promotion Action.

  2. Change the release_promotion_flavor to ship-client.

  3. Trigger the action. This will run beetmover tasks that copy the signed builds from candidates, over to the releases directory on archive.mozilla.org.

  4. Update Balrog, making sure to use the FirefoxVPN: release channel.

  5. Reply in-thread in the #releaseduty channel that the release has been shipped on Windows and Mac.

Update Balrog#

Once the builds are on archive.mozilla.org, you’ll need to update the FirefoxVPN: release-cdntest channel on Balrog to point at them.

  1. Create the release. It’s usually easiest to copy/paste the previous release.

    1. Set the release name to FirefoxVPN-<version>.

    2. Set the channel. Use FirefoxVPN: release-cdntest for the candidate builds. Use FirefoxVPN: release for the actual release.

    3. Update the URLs.

    4. Update the sha512 hashes, you’ll need to download the binaries and compute them manually. Make sure the files are gunzip’ed first (downloading via browser is safe way to ensure this).

    5. Update the version in the name and version fields.

  2. Update the default rule to point to the new release.

Required Releases#

From time to time, you may be asked to create a “required” release. This means that the client will force users to update to the newer version (normally they would have the option to ignore the update prompt). Required releases should be created in addition to the release you created in the previous step.

  1. Create the required release by copy/pasting the previous release blob except:

    1. Append -Required to the release name (both in Balrog and in the release blob).

    2. Change the required field to true.

  2. Update or create the appropriate rules to point to this release. It will likely be the case that only certain platforms or version numbers will want to use this release, so be sure to work with the VPN team to determine what exactly is desired.

Linux Client#

The Linux client is shipped via launchpad, credentials are in Releng SOPS under ubuntu-store.yml.

Note

For now we are still using builds generated by Github Actions, but will hopefully be switching to the Taskcluster builds shortly.

  1. Ensure that publish is disabled in the edit section of launchpad. This will ensure we don’t accidentally ship the builds that are uploaded immediately.

  2. Find the Sources.zip artifact from Github Actions. Ideally this will be linked by someone from VPN but if not:

    1. Click the Actions tab in Github.

    2. Click PPA Automated Releases workflow on the left.

    3. Under the branch filter, type in the version tag (e.g v2.10.0).

    4. Click the task and download the Sources.zip at the bottom.

  3. Extract the zip to a temporary directory:

    $ unzip -d vpn_release Sources.zip
$ cd vpn_release

  4. Run a docker container to perform the signing:

    export RELENG_GPG_KEYCHAIN=/path-to/releng-secrets-global/keys/mozillavpn-launchpad-gpg
docker run --rm -it -v $(pwd):/packages -v $RELENG_GPG_KEYCHAIN:/keychain --entrypoint /bin/bash ubuntu:latest -i

  5. When inside the container, run the following commands:

    apt update && DEBIAN_FRONTEND=noninteractive apt install -y devscripts dput rsync
rsync -av /keychain/ /keychain2/
export GNUPGHOME=/keychain2
cd /packages
# have the passphrase at hand in keys/mozillavpn-launchpad-gpg.passphrase
# for the next command for signing
debsign -k Release --re-sign *.dsc *.buildinfo *.changes
dput ppa:mozillacorp/mozillavpn mozillavpn*.changes

    An email will be sent by lauchpad to release+ubuntu-store@mozilla.com.

    If dput complains about the package had already been uploaded, then delete all files *.ppa.upload and try again.

    If the build fails for some reason (i.e.: GPG key rotated), and the server complains the package has already been uploaded, then dput can be run with -f to force the upload/override.

  6. The build should eventually show up on launchpad. You can also watch the state of the builds here.

  7. Once the builds are successful, open the edit section of launchpad and check the Publish box. Click Save.

  8. After the releases have been shipped and verified, go back and disable Publish again.

Android and iOS Clients#

Releng is not involved with the mobile release process.

Addons#

Addons are shipped independently from the main client, they are akin to Firefox’s “system addons”. They live in the main repo and are not to be confused with web extensions (they use a custom format to VPN).

Promote Addons#

  1. Find the Release Promotion Action, using the lastest push from main (unless otherwise specified).

  2. Change the release_promotion_flavor to promote-addons.

  3. Trigger the action. This will trigger beetmover tasks that will upload the addons plus the manifest to the addons candidates directory. The path will have a pattern like: pub/vpn/addons/candidates/<buildId>

  4. Provide the link in a thread to the original request in #releaseduty.

Ship Addons#

  1. Find the Release Promotion Action, using the same push as in the promote phase.

  2. Change the release_promotion_flavor to ship-addons.

  3. Trigger the action. This will trigger beetmover tasks that will upload the addons plus the manifest to the addons releases directory. The files will be uploaded to two locations:

    1. pub/vpn/addons/releases/<buildId>

    2. pub/vpn/addons/releases/latest

  4. Provide the link in a thread to the original request in #releaseduty.