How Tasks are Triggered

How are tasks triggered in Taskcluster?

  • You can create arbitrary tasks through, e.g. the create task API/UI.

  • Or you can encode one or more tasks in .taskcluster.yml

  • If you pass a certain threshold of complexity, you probably want your .taskcluster.yml to schedule a decision task, which will create a task graph via taskcluster-taskgraph.

… But what generates the task(s) in .taskcluster.yml?

[Pulse ->] hooks -> build-decision

We have a python module named build-decision that knows how to trigger decision tasks for hg.m.o pushes and github+hg.m.o cron tasks. This is bundled in the build-decision docker image.

Once we build that image in automation, Releng chooses a given build-decision image to use and uploads it to docker-hub. We then update the pinned image in the hg push hook template and the cron hook template; the next time we run ci-admin, ci-admin updates the various on-push (e.g.) and cron (e.g.) hooks.

hg.m.o on-push

When we push a change to hg.m.o, we send a pulse message. The mozilla-central hg-push hook listens to exchange/hgpushes/v2 with mozilla-central, creates a task using the build-decision image, and sets the PULSE_MESSAGE env var to the contents of the pulse message. This task then creates the decision task, which in turn creates a task graph as appropriate.

We can replicate this later, using fxci, via the fxci replay-hg-push subcommand.

Cron

We have three types of cron hooks:

Time-based cron hooks

When we enable either gecko-cron (firefox desktop or thunderbird only), or taskgraph-cron for a project, and when there is a .cron.yml at the top of the repository, then we create a cron task every 15 minutes to see if anything should run. This cron task runs build-decision, which knows how to generate a cron decision task.

For example, the mozilla-central cron hook has a schedule of 0 0,15,30,45 * * * *, and runs the cron command entry point in the build-decision docker image.

Manually triggered cron hooks

Sometimes we want to be able to trigger some of these cron tasks outside of the regularly scheduled times. (Sometimes we even have 0 regularly scheduled times for specific cron tasks, and we only trigger them manually.)

To enable this, we specify the project.cron.targets, like for mozilla-central here; in cron.targets we enable the desktop nightlies to be triggered off-cycle, as well as l10n-bumper and others. The scriptworker-canary cron hook accepts input (allow-input: true).

So when we run ci-admin, we generate things like the mozilla-central l10n-bumper cron hook, which has no schedule, and we have the '--force-run=l10n-bumper' option specified in the cron command. We can trigger these hooks manually.

The mozilla-central scriptworker-canary cron hook similarly has no schedule, and has the '--force-run=scriptworker-canary' option. Because we allow for input, we set the HOOK_PAYLOAD env var to the payload that we send to the hook.

Pulse-triggered cron hooks

Sometimes we want to be able to trigger some of these cron hooks via some external, non-time-based, non-manually-triggered signal. In this case, we can add a pulse binding to a given cron hook.

In the android-l10n-tooling project, we have the update-l10n cron.target where we specify bindings as follows:

- exchange: exchange/taskcluster-github/v1/push
  routing_key_pattern: primary.mozilla-mobile.fenix
- exchange: exchange/taskcluster-github/v1/push
  routing_key_pattern: primary.mozilla-mobile.android-components
- exchange: exchange/taskcluster-github/v1/push
  routing_key_pattern: primary.mozilla-mobile.firefox-tv
- exchange: exchange/taskcluster-github/v1/push
  routing_key_pattern: primary.mozilla-lockwise.lockwise-android

And indeed, the update-l10n hook generated by ci-admin has no schedule; it does have those pulse bindings, it includes the cron option '--force-run=update-l10n', and is also manually triggerable.

The github push pulse event appears to be sent by taskcluster-github, so we trigger this hook whenever there is a github push to these repos, but we can also trigger the hook manually if there’s an issue with the pulse-driven pipeline.

Through action hooks

As noted in Actions are broken after modifying .taskcluster.yml, we embed the contents of .taskcluster.yml into action hooks, which are named by .tc.yml hash. This avoids having an intermediate task:

hook -> action task -> result

rather than:

hook -> build-decision -> action task -> result

Some of the rationale and debate are surfaced in bug 1463522 and bug 1415868 comment 77. We may want to revisit this debate, but until then, our action hooks directly create an action task.

However, these are different than other hooks, in that action tasks are run against a previously-run decision task, and rely on the decision task’s artifacts, especially actions.json.

Through taskcluster-github

Taskcluster-github listens to Github events, directly parses .taskcluster.yml, and creates decision tasks.

We may want to revisit whether we want the app to do this, or if we want an intermediate build-decision task in between.

Side note: taskcluster_yml_repo

In Releng-RFC 36 we are trying to enable standard build/test workflows without having to land custom code in a given repository.

One feature we already support in ci-configuration is taskcluster_yml_repo. Before we combined the ci-configuration and ci-admin repos, ci-configuration specified ci-admin as its taskcluster_yml_project. We refer to this in the ciadmin.generate.hg_pushes.make_hook function; the build-decision hg-push cli supports that; the build_decision.hg_push.build_decision function then specifies the taskcluster_yml_repo’s .taskcluster.yml as the URL to use to render the decision task.

This is pretty great: it’s already supported; if you download the .taskcluster.yml from another repo, you can also clone it and use its taskcluster/ directory; and if we didn’t mind multiple template repos, we could create a taskcluster_yml_repo for every build/test workflow we want to support in a generic way.

However. There are caveats:

  • taskcluster_yml_repo and taskcluster_yml_project are only supported in hg.m.o projects. If we want to support these fully across all types of projects, we need to support all of the above ways of triggering decision/action/cron tasks, or the unsupported way won’t know how to find the appropriate .taskcluster.yml file and will fail. We only support the hg-push, build-decision workflow with taskcluster_yml_repo currently.

  • Once we get that working, we have to figure out how we refer to arbitrary repos in the shared .taskcluster.yml and possibly taskcluster.ci.config. This may be simple: we were able to use ${repoUrl} and ${push.revision} in the shared ci-admin .tc.yml, but we also listed it under our taskcluster.ci.config.taskgraph.repositories. Perhaps this isn’t a worry for level 1 repos: xpi-template doesn’t list each downstream repo, but we needed to in xpi-manifest.

Depending on the answers to the above, taskcluster_yml_repo could be a good stopgap solution or a stepping stone on the way to Releng-RFC 36.