# Workflow Elements

Complete Listing of all Workflow Elements used by Rig

In this section, we describe the various workflow elements and how to use them.

TIP

As of now, Rig is a thin abstraction over GitLabs configuration files. We have identified great potential for generalizations and hope to support more platforms in the future. Rig does not (yet) implement all sanity checks required for valid configurations, e.g. ensuring the correct Docker image for the release property.
Consult the GitLab CI/CD Documentation (opens new window) if you are having trouble implementing a pipeline.

# Job

We call a step or unit in a CI/CD workflow a job. Jobs can be executed either in parallel or in series. Since jobs are connected with their successors (which depend on their predecessors), a CI/CD workflow is a directed, acyclic graph (DAG).

Rig will resolve the dependencies between jobs automatically and generate the proper configuration to pass output (artifacts) from one job to another from the edges in the workflow model.

Since a job is a step in the build process of the application, it must have a defined script. Dependencies between jobs are configured by connecting a job with its sucessor.

Jobs have a number of important configuration options:

Name: The name of the job, as displayed on the canvas and used as job name for GitLab.
Script: The main script to be executed when the job runs. Can be multiple lines long.
Before Script: Script executed before the main script. Use to set up everything for the job.
After script: Scrupt executed after the main script, but before artifacts and cache are created. Use to clean up the environment.

# Job properties

The following properties can also be configured on jobs. They can also be provided by other model elements. When a property is provided both directly from a job and a model element, the property on the job takes precedence.

Script Argument: Used to define additional variables to be used inside the scripts. Script arguments can take their values from either variables or parameters. See Parameterization for details.
Image: Name of the Docker container image to run the job on. See Image.
Service: Name of a Docker container image to be available as service. Can be specified multiple times. See Service.
Except: Prevents execution of the job. Can be a regular expression matching a git ref (commit hash, tag name or branch name) or event. Can be specified mltiple times. See Except & Only / Ref.
Coverage: Regular expression to extract test coverage from the executed scripts output and display a percentage.
Allow Failure: Whether or not this job is allowed to fail. Defaults to false. Use for optional jobs.
Interruptible: TBD
Parallel: TBD
Retry: Number of times the job is tried again if it fails. Defaults to 0.
Timeout: Specify a timeout after which the job is cancelled. Useful to save runner minutes.
Resource Group: TBD
Tag: TBD

# Target

In order to reduce the complexity of pipelines and to make re-use of jobs easier, we have introduced the concept of build targets to our workflow description.

As described in Parameterization, targets can be used to parameterize other workflow elements (e.g. jobs).

Targets can also be used to define default values for all jobs leading to them, for example the build environment or conditions that enable or disable jobs. For example, the workflow for this documentation defines the node:14.16.0 image directly on the target. This way, it is sufficient to define the image only once.

The following workflow elements can be set directly on targets:

Image
Service
Except & Only
Rules

# Precedence

If a workflow element is defined both on a target and a job, the element defined on the job takes precedence.

If the elements are mergeable, e.g. the various conditions, they are merged.

# Artifacts

Besides jobs and targets, artifacts are arguably the most important workflow elements. Artifacts serve as the output of one job and can be passed to another job for further processing.

In order to pass artifacts from one job to another, the jobs need to be connected with an edge in the workflow model.

WARNING

Artifacts are not passed transitively. You need to explicitly add edges from one job to another to pass artifacts along, even if you have a long chain of jobs. If a job C depends on B, which depends on A, then artifacts from A are not passed to C automatically. C needs to declare a dependency on A explicitly.

Name: The name of the artifact (tarball)
Path: File or directory that should be included. You can use wildcards for filenames (*) and directories (*). Can be specified multiple times.
Exclude: Exclude files or directories. You can use wildcards for filenames (*) and directories (*). Can be specified multiple times.
Expose As: The name under which the artifact is exposed when viewed in a merge request.
Expire In: The time after which the artifact expires. Use this to avoid wasting storage space by setting reasonable limits on the availability of your artifact. The artifact produced in the last run is always retained until a newer run occurs, but this property can be used to retire old artifacts. Ues 'chronic_duration' (opens new window) syntax.

# Cache

In many situations, it can be advantageous to cache files between different runs of a pipeline.

For example, it is often advantageous to cache the local maven repository (the .m2 directory), so that maven only needs to fetch the same dependency once. Otherwise, dependencies have to be fetched for every run from the remote repositories.

WARNING

In order to have reproducible builds, you should only cache files that do not change and can be re-acquired in exactly the same form if the cache is invalidated. This is true for maven dependencies, especially those in Maven Central, as artifacts cannot be changed once uploaded there.

# Release

TBD

# Image

Many jobs in a CI/CD workflow require special tooling to be available, for example, Java & Maven, C# & Nuget, Node.js & NPM or many of the other programming languages and their tooling.

In order to make sure your job runs in an environment where all needed tools are available, you can specify a Docker container image (and an entrypoint) that should be used to execute your job on.

Name: This property is mandatory and specifies the container image to run.
Entrypoint: You cn override the image entrypoint with this property and run a custom command on the container to initialize it.

# Services

Sometimes, it is necessary to have additional services available, e.g. databases in order to run integration tests. You can specify additional Docker images that should be available while your job runs with this workflow element.

In addition to the properties available on an Image element, the following properties are available:

Command: You can specify one or more additional commands to run on the container
Alias: You can specify a custom name for the container in order to make it available under that name

# Environment

When deploying your application, you might want to be able to monitor the environment in which you have deployed it. This workflow element makes your environment available in your GitLab Repo under "Deployments". From there, you can also stop the environment.

Name: The display name of the environment, e.g. "Staging".
URL: The URL under which the environment is available
Action: The action to take with the environment. You can either start, prepare or stop the given environment.

WARNING

Rig does not currently support running a job upon stopping the environment. Support for the on_stop property is still under investigation and planned for a future release.

# Retry

Sometimes, jobs can fail. With this property, you can specify when and how often a job is allowed to be retried.

When: Choose the situation in which the job can be retried, e.g. runner failure, script failure and more. Default is always.
Max: Maximum number of retries

# Except & Only

The Except and Only elements control when a job is executed. As the name suggests, the "Except" element is used to prevent execution when the conditions in this element are met, and allows execution otherwise. The "Only" elements works exactly in the opposite manner, it doesn't allow any execution unless the conditions are met.

Both elements are configured the same way, thus they are described together here. They can also both be defined, in which case a job is executed only if it passes the test of the "Only" alement and isn't prevented by the "Except" element.

All elements have the following properties:

Ref

The Ref property has multiple uses.

It can be used to pattern-match an expression against a git ref, e.g. a commit hash, a branch name or tag name. You can use regular expressions (RE2, no backreferences) to match arbitrary values.
It can be used to specify certain events. We provide a small grammar with auto-completion, but the allowed values are api, branches, chat, external, external_pull_request, merge_requests,pipelines, pushes, schedules, tags, triggers, web (more info (opens new window))
The "Ref" property can be specified multiple times.
The default value of this property is branches and tags

Change

The Change property is used to watch for changes in files or directories.

It can only be used with the Ref branches, external_pull_request and merge_request
You can use wildcards for file names (*) and directories (**).

Variables

The Variables property is used to control execution of the job depending on CI/CD Variables. We provide a small grammar that helps with writing correct variable expressions .

WARNING

If you prevent the execution of a prerequisite job, the subsequent job will try to start, but the pipeline will fail at runtime. GitLab cannot detect this, and Rig currently can't, either.

We are working on replacing this with a much better modeling experience that is able to reason about these conditions and prevent misconfigurations.

# Rules

The rules element is another element used to control execution of a job. It cannot be used in conjunction with "Except" or "Only".

TBD

← Parameterization Examples →