An opinionated CI based on OpenShift Pipelines / Tekton on Pipelines as Code

Overview

Mon, 01 Jan 0001 00:00:00 +0000

Pipelines-as-Code - Installation #

Pipelines-as-Code support different installation method to Git provider platforms (i.e: GitHub, Bitbucket and so on)

The preferred method to use Pipelines-as-Code is configured with a GitHub Application.

Install Pipelines-as-Code infrastructure #

To get started with Pipelines-as-Code, you need to

Install Pipelines-as-Code.
- If you run on OpenShift Pipelines (from 1.7.x) Pipelines-as-Code should be automatically installed
- If you want to do a manual install you can follow the installation guide.
Configure your Git Provider (eg: a GitHub Application) to access Pipelines as Code.

Git Provider Setup #

After installing Pipelines-as-Code you are now ready to configure your Git provider. Choose your preferred install method, if you don’t have any preferences the preferred install method is the GitHub Application method.

Pipelines-as-Code Flows

Mon, 01 Jan 0001 00:00:00 +0000

Diagram of a Pull/Merge Request Flow #

Repository CR

Mon, 01 Jan 0001 00:00:00 +0000

Repository CR #

The Repository CR serves the following purposes:

Informing Pipelines-as-Code that an event from a specific URL needs to be handled.
Specifying the namespace where the PipelineRuns will be executed.
Referencing an API secret, username, or API URL if necessary for Git provider platforms that require it (e.g., when using webhooks instead of the GitHub application).
Providing the last PipelineRun statuses for the repository (5 by default).
Letting you declare custom parameters within the PipelineRun that can be expanded based on certain filters.

To configure Pipelines-as-Code, a Repository CR must be created within the user’s namespace, for example project-repository, where their CI will run.

Concurrency Flow

Mon, 01 Jan 0001 00:00:00 +0000

Concurrency Flow #

graph TD A1[Controller] --> B1(Validate & Process Event) B1 --> C1{Is concurrency defined?} C1 -->|Not Defined| D1[Create PipelineRun with state='started'] C1 -->|Defined| E1[Create PipelineRun with pending status and state='queued'] Z[Pipelines-as-Code] A[Watcher] --> B(PipelineRun Reconciler) B --> C{Check state} C --> |completed| F(Return, nothing to do!) C --> |queued| D(Create Queue for Repository) C --> |started| E{Is PipelineRun Done?} D --> O(Add PipelineRun in the queue) O --> P{If PipelineRuns running < concurrency_limit} P --> |Yes| Q(Start the top most PipelineRun in the Queue) Q --> P P --> |No| R[Return and wait for your turn] E --> |Yes| G(Report Status to provider) E --> |No| H(Requeue Request) H --> B G --> I(Update status in Repository) I --> J(Update state to 'completed') J --> K{Check if concurrency was defined?} K --> |Yes| L(Remove PipelineRun from Queue) L --> M(Start the next PipelineRun from Queue) M --> N[Done!] K --> |No| N

Getting started guide.

Mon, 01 Jan 0001 00:00:00 +0000

Getting started with Pipelines-as-Code #

This guide will walk you through the process of getting started with Pipelines-as-Code.

This will start with the installation of Pipelines-as-Code on your cluster, then the creation of a GitHub Application, the creation of a Repository CR to specify which repository you want to use with Pipelines-as-Code, and finally, we are going to create a simple Pull Request to test that configuration and see how the Pipelines-as-Code flow looks like.

Installation Through Operator

Mon, 01 Jan 0001 00:00:00 +0000

Installation Through Operator #

The easiest way to install Pipelines-as-Code on OpenShift is with the Red Hat OpenShift Pipelines Operator.

On the OpenShift Pipelines Operator, the default namespace is openshift-pipelines.

Note:

When Pipelines-as-Code is installed through the Tekton Operator the configurations of Pipelines-as-Code is controlled by TektonConfig Custom Resource. That means Tekton Operator will revert the configurations changes done directly on pipeline-as-code configmap or OpenShiftPipelinesAsCode custom resource.

The default configurations for Pipelines-as-Code in TektonConfig looks like below

Manual Installation

Mon, 01 Jan 0001 00:00:00 +0000

Installation #

Operator Install #

Follow Operator Installation to install Pipelines As Code on OpenShift.

Manual Install #

Prerequisite #

Before installing Pipelines As Code, please verify tektoncd/pipeline is installed. You can install the latest released version using the following command

  kubectl apply --filename https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml

If you are not installing the most recent version, ensure that you have Tekton Pipeline installed and running at a version that is higher than v0.44.0.

If you want to do a manual installation of the stable release of Pipelines-as-Code on your OpenShift cluster you can apply the template with kubectl :

Resolver

Mon, 01 Jan 0001 00:00:00 +0000

Pipelines-as-Code resolver #

The Pipelines-as-Code resolver ensures that the PipelineRun you are running doesn’t conflict with others.

Pipelines-as-Code parses any files ending with a .yaml or .yml suffix in the .tekton directory and subdirectory at the root of your repository. It will automatically attempt to detect any Tekton resources like Pipeline, PipelineRun, Task.

When detecting a PipelineRun it will try to resolve it as a single PipelineRun with an embedded PipelineSpec of the referenced Task or Pipeline. This embedding ensures that all dependencies required for execution are contained within a single PipelineRun at the time of execution on the cluster.

Authoring PipelineRun

Mon, 01 Jan 0001 00:00:00 +0000

Authoring PipelineRuns in `.tekton/` directory #

Pipelines-as-Code will always try to be as close to the Tekton template as possible. Usually, you will write your template and save them with a .yaml extension, and Pipelines-as-Code will run them.
The .tekton directory must be at the top level of the repo. You can reference YAML files in other repos using remote URLs (see Remote HTTP URLs for more information), but PipelineRuns will only be triggered by events in the repository containing the .tekton directory.

Matching a PipelineRun

Mon, 01 Jan 0001 00:00:00 +0000

Matching a PipelineRun to a Git provider Event #

A PipelineRun can be matched to a Git provider event by using specific annotations in the PipelineRun metadata.

For example, when you have these as metadata in your PipelineRun:

metadata:
  name: pipeline-pr-main
annotations:
  pipelinesascode.tekton.dev/on-target-branch: "[main]"
  pipelinesascode.tekton.dev/on-event: "[pull_request]"

Pipelines-as-Code will match the PipelineRun pipeline-pr-main if the Git provider events target the branch main and it’s coming from a [pull_request]

Multiple target branches can be specified, separated by commas, e.g.:

Settings

Mon, 01 Jan 0001 00:00:00 +0000

Pipelines-as-Code configuration settings #

There is a few things you can configure through the config map pipelines-as-code in the pipelines-as-code namespace.

application-name

The name of the application for example when showing the results of the pipelinerun. If you’re using the GitHub App you will need to customize the label on the github app setting as well. . Default to Pipelines-as-Code CI
secret-auto-create

Whether to auto create a secret with the token generated through the GitHub application to be used with private repositories. This feature is enabled by default.

Custom certificates

Mon, 01 Jan 0001 00:00:00 +0000

Custom certificates #

If you need to configure Pipelines-as-Code with a Git repository that requires a privately signed or custom certificate to access, then you will need to expose the certificate to Pipelines-as-Code.

OpenShift #

If you have installed Pipelines-as-Code through the OpenShift Pipelines operator, then you will need to add your custom certificate to the cluster via the Proxy object. The operator will expose the certificate in all OpenShift Pipelines components and workloads, including Pipelines-as-Code.

Global Repository Settings

Mon, 01 Jan 0001 00:00:00 +0000

Global repository settings is a Technology Preview feature only. Technology Preview features are not currently supported and might not be functionally complete. We do not recommend using them in production. These features provide early access to an upcoming Pipelines-as-Code features, enabling you to test functionality and provide feedback during the development process.

Pipelines-as-Code Global Repository Settings #

Pipelines-as-Code lets you have a global repository for settings of all your local repositories. This enables you to define settings that will be applied to all local repositories on your cluster.

Running the PipelineRun

Mon, 01 Jan 0001 00:00:00 +0000

Running the PipelineRun #

Pipelines-as-Code (PAC) can be used to run pipelines on events such as pushes or pull requests. When an event occurs, PAC will try to match it to any PipelineRuns located in the .tekton directory of your repository that are annotated with the appropriate event type.

The PipelineRuns definitions are fetched from the .tekton directory at the root of your repository from where the event comes from, this is unless you have configured the provenance from the default branch on your Repository CR.

For example, if a PipelineRun has this annotation:

GitOps Commands

Mon, 01 Jan 0001 00:00:00 +0000

GitOps Commands #

Pipelines-as-Code supports the concept of GitOps commands, which allow users to issue special commands in a comment on a Pull Request or a pushed commit to control Pipelines-as-Code.

The advantage of using a GitOps command is that it provides a journal of all the executions of your pipeline directly on your Pull Request, near your code.

GitOps Commands on Pull Requests #

For example, when you are on a Pull Request, you may want to restart all your PipelineRuns. To do so, you can add a comment on your Pull Request starting with /retest, and all PipelineRuns attached to that Pull Request will be restarted.

PipelineRun status

Mon, 01 Jan 0001 00:00:00 +0000

Status #

GitHub apps #

After the PipelineRun has finished, its status will be shown in the GitHub Check tabs, along with a concise overview of the status the name and the duration of each task in the pipeline. If the task has a displayName it will use it as the description of the task or otherwise just the task name.

If any step fails, a small portion of the log from that step will also be included in the output.

Private Repositories

Mon, 01 Jan 0001 00:00:00 +0000

Private repositories #

Pipelines-as-Code allows the use of private repositories by creating or updating a secret in the target namespace. This secret contains the user token required for the git-clone task to clone private repositories.

Whenever Pipelines-as-Code creates a new PipelineRun in the target namespace, it also creates a secret with a specific name format:

pac-gitauth-REPOSITORY_OWNER-REPOSITORY_NAME-RANDOM_STRING

This secret contains a Git Config file named .gitconfig and a Git credentials file named .git-credentials. These files configure the base HTTPS URL of the git provider (such as https://github.com) using the token obtained from the GitHub application or from a secret attached to the repository CR on git provider when using the webhook method.

PipelineRuns Cleanup

Mon, 01 Jan 0001 00:00:00 +0000

PipelineRuns Cleanups #

There can be many PipelineRuns into a user namespace and Pipelines-as-Code has the ability to only keep a certain amount of PipelineRuns and cleaning the old ones.

When your PipelineRun has this annotation :

pipelinesascode.tekton.dev/max-keep-runs: "maxNumber"

Pipelines-as-Code sees this and will start cleaning up right after one of the PipelineRun finishes to a successful execution keeping only the last maxNumber of PipelineRuns.

It will skip the Running or Pending PipelineRuns but will not skip the PipelineRuns with Unknown status.

GitHub Apps

Mon, 01 Jan 0001 00:00:00 +0000

Create a Pipelines-as-Code GitHub App #

The GitHub App install is different from the other install methods since it acts as the integration point with OpenShift Pipelines and brings the Git workflow into Tekton pipelines. You only need one GitHub App for every user on the cluster usually setup by the admin.

You need the webhook of the GitHub App to point to your Pipelines-as-Code Controller route or ingress endpoint which would listen to GitHub events.

GitHub Webhook

Mon, 01 Jan 0001 00:00:00 +0000

Use Pipelines-as-Code with GitHub Webhook #

If you are not able to create a GitHub application you can use Pipelines-as-Code with GitHub Webhook on your repository.

Using Pipelines-as-Code through GitHub webhook does not give you access to the GitHub CheckRun API, therefore the status of the tasks will be added as a Comment on the PullRequest and not through the Checks Tab.

gitops comment (ie: /retest /ok-to-test) with GitHub webhook is not supported. If you need to restart the CI you will need to generate a new commit. You can make it quick with this command line snippet (adjust branchname to the name of the branch) :

GitLab

Mon, 01 Jan 0001 00:00:00 +0000

Use Pipelines-as-Code with GitLab Webhook #

Pipelines-As-Code supports on GitLab through a webhook.

Follow the pipelines-as-code installation according to your Kubernetes cluster.

Create GitLab Personal Access Token #

Follow this guide to generate a personal token as the manager of the Org or the Project:

https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html

Note: You can create a token scoped only to the project. Since the token needs to be able to have api access to the forked repository from where the MR come from, it will fail to do it with a project scoped token. We try to fallback nicely by showing the status of the pipeline directly as comment of the Merge Request.

Bitbucket Cloud

Mon, 01 Jan 0001 00:00:00 +0000

Use Pipelines-as-Code with Bitbucket Cloud #

Pipelines-As-Code supports on Bitbucket Cloud through a webhook.

Follow the Pipelines-As-Code installation according to your Kubernetes cluster.

Create Bitbucket Cloud App Password #

Follow this guide to create an app password:

https://support.atlassian.com/bitbucket-cloud/docs/app-passwords/

Check these boxes to add the permissions to the token:

Account: Email, Read
Workspace membership: Read, Write
Projects: Read, Write
Issues: Read, Write
Pull requests: Read, Write

NOTE: If you are going to configure webhook through CLI, you must also add additional permission

Bitbucket Data Center

Mon, 01 Jan 0001 00:00:00 +0000

Install Pipelines-As-Code on Bitbucket Data Center #

Pipelines-As-Code has a full support of Bitbucket Data Center.

After following the installation:

You will have to generate a personal token as the manager of the Project, follow the steps here:

https://confluence.atlassian.com/bitbucketserver/personal-access-tokens-939515499.html

The token will need to have the PROJECT_ADMIN and REPOSITORY_ADMIN permissions.

Note that the token needs to be able to have access to the forked repository in pull requests, or it would not be able to process and access the pull request.

Metrics

Mon, 01 Jan 0001 00:00:00 +0000

Metrics Overview #

The metrics for pipelines-as-code can be accessed through the pipelines-as-code-watcher service on port 9090.

pipelines-as-code supports various exporters, such as Prometheus, Google Stackdriver, and more. You can configure these exporters by referring to the observability configuration.

Name	Type	Description
`pipelines_as_code_git_provider_api_request_count`	Counter	Number of API requests submitted to git providers
`pipelines_as_code_pipelinerun_count`	Counter	Number of pipelineruns created by pipelines-as-code
`pipelines_as_code_pipelinerun_duration_seconds_sum`	Counter	Number of seconds all pipelineruns have taken in pipelines-as-code
`pipelines_as_code_running_pipelineruns_count`	Gauge	Number of running pipelineruns in pipelines-as-code

Note: The metric pipelines_as_code_git_provider_api_request_count is emitted by both the Controller and the Watcher, since both services use Git providers’ APIs. When analyzing this metric, you may need to combine both services’ metrics. For example, using PromQL:

Kubernetes

Mon, 01 Jan 0001 00:00:00 +0000

Kubernetes #

Pipelines-as-Code works on kubernetes/minikube/kind.

Prerequisites #

You will need to pre-install the pipeline release.yaml file on your kubernetes cluster.

You will need at least a kubernetes version greater than 1.23

Install #

The release YAML to install pipelines are for the released version :

kubectl apply -f https://raw.githubusercontent.com/openshift-pipelines/pipelines-as-code/stable/release.k8s.yaml

and for the nightly :

kubectl apply -f https://raw.githubusercontent.com/openshift-pipelines/pipelines-as-code/nightly/release.k8s.yaml

Verify #

Ensure that the pipelines-as-code controller, webhook, and watcher have come up healthy, for example:

Custom Parameters

Mon, 01 Jan 0001 00:00:00 +0000

Custom Parameters #

Using the {{ param }} syntax, Pipelines-as-Code lets you expand a variable or the payload body inside a template within your PipelineRun.

By default, several variables are exposed according to the event. To view all the variables exposed by default, refer to the documentation on Authoring PipelineRuns.

With the custom parameter, you can specify some custom values to be replaced inside the template.

Utilizing the Tekton PipelineRun parameters feature may generally be the preferable approach, and custom params expansion should only be used in specific scenarios where Tekton params cannot be used.

As an example, here is a custom variable in the Repository CR spec:

Incoming Webhook

Mon, 01 Jan 0001 00:00:00 +0000

Incoming webhook #

Pipelines-as-Code support the concept of incoming webhook URL. It let you trigger PipelineRun in a Repository using a shared secret and URL, instead of creating a new code iteration.

Incoming Webhook URL #

To use incoming webhooks in Pipelines-as-Code, you must configure the incoming field in your Repository CRD. This field references a Secret, which serves as the shared secret, as well as the branches targeted by the incoming webhook. Once configured, Pipelines-as-Code will match PipelineRuns located in your .tekton directory if the on-event annotation of the targeted pipelinerun is targeting a push or incoming event.

Policy on actions

Mon, 01 Jan 0001 00:00:00 +0000

Policy on Pipelines-as-Code Actions #

Pipelines-as-Code uses policies to control which actions can be performed by users who belong to specific teams within an organization, as defined on GitHub or other supported Git providers (currently GitHub and Gitea).

This feature is supported on the following providers

Git Provider Supported

GitHub App ✅️

GitHub Webhook ✅️

Gitea ✅️

GitLab ❌️

Bitbucket Cloud ❌️

Bitbucket Data Center ❌️

Git Provider	Supported
GitHub App	✅️
GitHub Webhook	✅️
Gitea	✅️
GitLab	❌️
Bitbucket Cloud	❌️
Bitbucket Data Center	❌️

Supported Actions #

pull_request - This action triggers the CI in Pipelines-as-Code. Specifying a team restricts the ability to trigger CI to members of that team, regardless of whether they are repository or organization owners or collaborators. However, members listed in the OWNERS file are still permitted to trigger the CI.

CLI tkn-pac

Mon, 01 Jan 0001 00:00:00 +0000

Pipelines-as-Code CLI #

Pipelines-as-Code provides a powerful CLI designed to work as a plug-in to the Tekton CLI (tkn).

tkn pac allows you to:

bootstrap: quickly bootstrap a Pipelines-as-Code installation.
create: create a new Pipelines-as-Code Repository definition.
delete: delete an existing Pipelines-as-Code Repository definition.
generate: generate a simple pipelinerun to get you started with Pipelines-as-Code.
list: list Pipelines-as-Code Repositories.
logs: show the logs of a PipelineRun from a Repository CRD.
describe: describe a Pipelines-as-Code Repository and the runs associated with it.
resolve: Resolve a pipelinerun as if it were executed by pipelines as code on service.
webhook: Updates webhook secret.
info: Show information (currently only about your installation with info install).

Install #

Binary

You can grab the latest binary directly for your operating system from the releases page.

OpenAPI Schema Validation

Mon, 01 Jan 0001 00:00:00 +0000

OpenAPI Schema Validation for Repository CRDs #

Pipelines-as-Code provides OpenAPI schema validation for its Custom Resource Definitions (CRDs), which helps in writing the Repository resources. This page explains what OpenAPI schemas are, their benefits, and how to leverage them in your development environment.

What is OpenAPI Schema Validation? #

OpenAPI schema validation is a mechanism that adds metadata to Kubernetes CRDs, providing:

Type information: Specifies expected data types for fields
Required fields: Marks which fields must be present
Pattern validation: Enforces specific formats (e.g., URLs must start with http:// or https://)
Enumeration validation: Limits fields to a predefined set of values
Field descriptions: Documents the purpose of each field

When you create or modify a Repository resource, the OpenAPI schema helps validate your configuration before it’s applied to the cluster, catching errors early in the development process.

Multiple GitHub Applications Support

Mon, 01 Jan 0001 00:00:00 +0000

Multi-GitHub Application Configuration #

Multi-GitHub Apps Support is a Technology Preview feature only. Technology Preview features are not currently supported and might not be functionally complete. We do not recommend using them in production. These features provide early access to an upcoming Pipelines-as-Code features, enabling you to test functionality and provide feedback during the development process.

Pipelines-as-Code allows multiple GitHub applications to operate on the same cluster, enabling integration with different GitHub instances (e.g., public GitHub and GitHub Enterprise Server).

OpenShift Pipelines 1.15 PAC release

Mon, 01 Jan 0001 00:00:00 +0000

OpenShift Pipelines 1.15 presents several new enhancements to Pipelines-as-Code. Below are the key updates.

Much improved GitOps Commands #

These commands allow you to make quick comments on a pull request to restart a PipelineRun through Pipelines-as-Code.

Commonly used commands include /test pipelinerun to rerun a specific pipelinerun, or /retest to rerun all PipelineRuns.

Trigger PipelineRuns Irrespective of Annotations #

Previously, to re-trigger PipelineRuns, they had to match specific annotations such as pipelinesascode.tekton.dev/on-event set to pull_request. Now, this constraint is removed, allowing you to trigger any PipelineRuns with /test regardless of their annotation status. This is particularly useful if you need to run a pipelinerun selectively before merging a PR, without it automatically consuming resources with each update.

Pipelines-as-Code Release Process

Mon, 01 Jan 0001 00:00:00 +0000

Release process for Pipelines-as-Code #

Clear out the PR needed to be merged.
Wait that CI is connected.
Verify PAC CI cluster is up.
Verify that you have gpg signing setup for your commits.
Prepare to tag the release with a version, you need to choose between a major release/minor or patch release.
If for example you choose to do the release 1.2.3 you tag it locally :

git tag v1.2.3

And pushing it directly to the repo (you need access) :

% NOTESTS=ci git push git@github.com:openshift-pipelines/pipelines-as-code refs/tags/1.2.3

When it started you can follow it on the pac cluster :

tkn pr logs -n pipelines-as-code-ci -Lf

An opinionated CI based on OpenShift Pipelines / Tekton on Pipelines as Code

Overview

Pipelines-as-Code - Installation #

Install Pipelines-as-Code infrastructure #

Git Provider Setup #

Pipelines-as-Code Flows

Diagram of a Pull/Merge Request Flow #

Repository CR

Repository CR #

Concurrency Flow

Concurrency Flow #

Getting started guide.

Getting started with Pipelines-as-Code #

Installation Through Operator

Installation Through Operator #

Manual Installation

Installation #

Operator Install #

Manual Install #

Prerequisite #

Resolver

Pipelines-as-Code resolver #

Authoring PipelineRun

Authoring PipelineRuns in .tekton/ directory #

Matching a PipelineRun

Matching a PipelineRun to a Git provider Event #

Settings

Pipelines-as-Code configuration settings #

Custom certificates

Custom certificates #

OpenShift #

Global Repository Settings

Pipelines-as-Code Global Repository Settings #

Running the PipelineRun

Running the PipelineRun #

GitOps Commands

GitOps Commands #

GitOps Commands on Pull Requests #

PipelineRun status

Status #

GitHub apps #

Private Repositories

Private repositories #

PipelineRuns Cleanup

PipelineRuns Cleanups #

GitHub Apps

Create a Pipelines-as-Code GitHub App #

GitHub Webhook

Use Pipelines-as-Code with GitHub Webhook #

GitLab

Use Pipelines-as-Code with GitLab Webhook #

Create GitLab Personal Access Token #

Bitbucket Cloud

Use Pipelines-as-Code with Bitbucket Cloud #

Create Bitbucket Cloud App Password #

Bitbucket Data Center

Install Pipelines-As-Code on Bitbucket Data Center #

Metrics

Metrics Overview #

Kubernetes

Kubernetes #

Prerequisites #

Install #

Verify #

Custom Parameters

Custom Parameters #

Incoming Webhook

Incoming webhook #

Incoming Webhook URL #

Policy on actions

Policy on Pipelines-as-Code Actions #

Supported Actions #

CLI tkn-pac

Pipelines-as-Code CLI #

Install #

OpenAPI Schema Validation

OpenAPI Schema Validation for Repository CRDs #

What is OpenAPI Schema Validation? #

Multiple GitHub Applications Support

Multi-GitHub Application Configuration #

Authoring PipelineRuns in `.tekton/` directory #