Pipeline Overview

Steps

Steps are the most basic units that you can use to create your pipeline. They represent a single operation that is part of your greater CI/CD process such as running some unit tests for a Python web app, or the compilation of a Java program.

Tekton performs each step in its own container with a container image that you provide. For example, you may use the official Go image to compile a Go program in the same manner as you would on your local workstation (go build).

Once this step is completed, the container is taken down, and everything inside it will be lost unless it's stored in a shared volume.

Using a container for each step is a great way to ensure that your steps will always be reproducible.

For each step, you will need to specify the image to be used and the command to be executed once this image is started.

If you are familiar with the Kubernetes syntax, you will find that writing your steps descriptions is very similar to writing a pod template in Kubernetes. Steps are used to compose tasks.

Tasks

A task is a collection of steps in order.

Tekton runs a task in the form of a Kubernetes pod, where each step becomes a running container in the pod.

A more complex task could have multiple steps to prepare the environment for the larger task. An example would be a test suite that needs some dependencies before running. Installing these dependencies and running the test suites would be two distinct steps in a single task.

The task will run in a single pod, which enables your steps to share a common volume and some resources.

You can configure tasks in many ways. The most common configuration is to add multiple parameters. Adding those parameters to your tasks will let you reuse the same task in a different context. You can find some pre-written tasks in the Tekton Catalog. Tasks for everyday operations such as git clone or run unit test cases, build image, etc.

Pipelines

A pipeline is a collection of tasks in order.

Tekton collects all the tasks, connects them in a directed acyclic graph (DAG), and executes the graph in sequence. In other words, it creates a number of Kubernetes pods and ensures that each pod completes running successfully as desired. Tekton grants developers full control of the process: one may set up a fan-in/fan-out scenario of task completion, ask Tekton to retry automatically should a flaky test exists, or specify a condition that a task must meet before proceeding.

Pipelines describe your CI/CD workflow. A typical pipeline would clone your code, run your tests, build an image, and deploy your application into a cluster.

Tasks in the pipeline could be executed simultaneously for a faster output or sequentially if the task's output is required for the next one to be completed.

If a task is unsuccessful, the pipeline could be halted. This early stop is handy if you have a task that runs a test suite. It could prevent pushing a bugged image into production.

Where to use a step, a task, or a pipeline

A step is the smallest unit available to build your Tekton CI/CD pipelines. It should only do one thing. If you have multiple commands being executed in the context of a step, this is a sign that you should probably convert it into a task.

When writing a task, you should always keep reusability in mind. If a task becomes too complicated and contains too many steps, consider breaking it down into multiple tasks and join them inside a pipeline. Tasks can also have a single step if that is what makes the most sense for that case.

Pipelines are your CI/CD workflows. They will take an input and perform various tasks on it.

Workspaces

A Workspace is a shared volume that can be used across all the tasks running in your pipeline. The main benefit of using a shared volume is that any action that's performed on your source code, such as installing dependencies, will be persisted when other tasks access this same volume.

Workspaces allow Tasks to declare parts of the filesystem that need to be provided at runtime by TaskRuns. A TaskRun can make these parts of the filesystem available in many ways: using a read-only ConfigMap or Secret, an existing PersistentVolumeClaim shared with other Tasks, create a PersistentVolumeClaim from a provided VolumeClaimTemplate, or simply an emptyDir that is discarded when the TaskRun completes.

Workspaces are similar to Volumes except that they allow a Task author to defer to users and their TaskRuns when deciding which class of storage to use.

Workspaces can serve the following purposes:

• Storage of inputs and/or outputs
• Sharing data among Tasks
• A mount point for credentials held in Secrets
• A mount point for configurations held in ConfigMaps
• A mount point for common tools shared by an organization
• A cache of build artifacts that speed up jobs

Understanding TaskRuns and PipelineRuns

TaskRuns and PipelineRuns represent how their counterparts are executed, that is the task and the pipeline. They hold the current status of the task's and pipeline's execution and provide you with more information on their performance, as shown in the following diagram:

Here, you can see the tasks and pipelines that can be defined as templates so that they can be executed. When you use the CLI to start a pipeline (or a task), it will create the associated run.

PipelineRuns

A pipelineRun, as its name implies, is a specific execution of a pipeline. For example, you may ask Tekton to run your CI/CD workflow twice a day, and each execution will become a pipelineRun resource trackable in your Kubernetes cluster. You can view the status of your CI/CD workflow, including the specifics of each task execution with pipelineRuns.

TaskRuns

A taskRun is a specific execution of a task on a cluster. While the task describes the operation that needs to happen in your pipeline, TaskRun describes this task's execution.

A TaskRun will execute all the steps in the task. It will also contain the status of the task's execution and the status of the execution of each step.

TaskRuns are also available when you choose to run a task outside a pipeline, with which you may view the specifics of each step execution in a task.

TaskRuns and pipelineRuns connect resources with tasks and pipelines. A run must include the actual addresses of resources, such as the URLs of repositories, its task or pipeline needs. You may create taskRuns or pipelineRuns manually, which triggers Tekton to run a task or a pipeline immediately. Alternately, one may ask a Tekton component, such as Tekton Triggers, to create a run automatically on demand; for example, you may want to run a pipeline every time a new pull request is checked into your git repository.