Jenkins Pipelines
Pipelines are Jenkins jobs that are enabled by pipeline plugins and which are created using simple scripts using the Groovy programming language-based pipeline DSL (Domain-Specific Language).
Pipelines use the feature of multiple steps to perform both simple and complex tasks based on the parameters that are established. After a pipeline is created, it can build the code and organize the activities needed to drive the application from commit to delivery.
Jenkins Pipeline, or simply Pipeline, is a group of plugins that enables the use and integration of continuous delivery pipelines in Jenkins. Continuous delivery is an automated expression of the process, from version control to delivering software to users and customers.
Jenkins pipelines allow us to specify a complete list of events that occur during the lifecycle of code. It starts with build, then testing, and then deployment. To help in the implementation of certain processes as a continuous delivery pipeline, we can use a group of plugins.
* A single platform that runs the entire pipeline as code is the Jenkin Pipeline.
* All standard jobs specified by Jenkins are manually written into a script and stored in VCS.
* Instead of building multiple jobs for a phase, you can now code all the jobs and put them in a Jenkins file.
Jenkins File
Jenkins files are text files that store pipelines as code. This file is defined using Groovy Domain-Specific Language (DSL). Using this JenkinsFile you can document the steps required to run your Jenkins pipeline. And this JenkinsFile can be checked in the system's SCM. It also allows developers to access, update, and review the code at any time.
There are two types of Jenkins pipeline syntax used to define a JenkinsFile.
1. Declarative
Declarative pipeline syntax provides an easy way to create pipelines. It has a predefined hierarchy for creating pipelines. It allows you to manage any pipeline in a simple and straightforward way.
2. Scripted
With the help of a lightweight executor, the Scripted Jenkins pipeline executes on the Jenkins master. It uses only some resources to translate the pipelines to atomic commands.
Jenkins Pipeline Terms
There are some frequently used terms in the pipeline described below.
1. Pipeline - The pipeline is a group of instructions provided in the format of code for continuous delivery, including necessary instructions for the entire build process.
With pipelines, you can build, test, and deploy applications.
Pipeline { }2. Step - A "step" or a "build step" is a single task that executes a specific process at a given time. The steps tell Jenkins what to do. A pipeline contains a series of steps.
3. Nodes - The machines that Jenkins runs on are called nodes. Node blocks are used in scripted pipeline syntax. A step in it is scheduled to run by adding it to the Jenkins build queue (this step will run as soon as the node's executor’s slot is free).
Create workspaces that are file directory which is specific to a particular job where the resource-intensive processing can occur without affecting your pipeline performances as negative. The workstations last for the duration of their assigned tasks.
node {}4. Stage - A stage block consists of a series of steps in a pipeline. That is, all processes build, test, and deploy converge at one stage. Generally, Jenkins uses a stage block to visualize the pipeline process.
There is also some Jenkins terminology such as "controller", "agent" and "executor" which is helpful in understanding how the pipeline works. These terms are not specific to pipelines:
* controller - A basic installation of the Jenkins on a computer is called as a controller. It handles tasks for the build system. Pipeline scripts are parsed in controllers, and steps wrapped in node blocks are executed in available executors.
* agent - An Agent is a computer that is set up to offload certain projects from the controller. The scope and number of the operations that an agent can do is determined by your configurations. And the operations are performed by the executors.
* executor - The executor is a computational resource that is used for compiling code. The executor can run on agent machines or controllers, either by itself or in parallel with other executors. Jenkins assigns a java.lang.Thread to each executor.
pipeline {
agent any
stages {
stage('Build'){
}
stage(Test){
}
stage(Deploy){
}
}
}To run the pipelines, you need a Jenkins instance equipped with the appropriate plugins. This requires
* Jenkins 1.580.1 or later (Jenkins 2.0 is recommended)
* The core Pipeline plugin
You can refer to the installation of Jenkins from our previous blog about Jenkins.
After installing Jenkins, you can now create the first admin user, as shown below.
You can then log in to Jenkins with this username and password. And you will get a page like the one below.
Steps to create a new pipeline and create builds:
1. Click 'New Item' in the menu to create a new pipeline.
2. Then you will get a window like this: from here, you set an item name, and select Pipeline and click OK.
3. Now a window will open. From here, you can set more configurations for your JenkinsFile.
4. From here, you can select the script type as declarative or script and then code your pipeline in the text box below.
Here I am giving a simple example of a pipeline script:
pipeline {
agent any
stages {
stage('Hello') {
steps {
echo 'Hello World'
}
}
}
}After saving the pipeline script above, the window shown below will open. From here, you can find the "BuildNow" button to build the pipeline.
Click on the Console Output to see the output.
Creating Multi-branch Pipelines
The Multi-branch pipeline allows configuring different tasks for the different branches of the same project. In multi-branch pipelines, Jenkins automatically discovers, manages, and executes jobs for multiple repositories and branches. This removes the need for manual job creation and management, and it is required when a developer adds a new feature to existing products.
A JenkinsFile is always included in the repository root of a multi-branch pipeline project. A sub-project is automatically created by Jenkins for each branch that it finds in a repository with a JenkinsFile.
To create a Multi-branch Pipeline:
1. On the Jenkins home page, you have an option New Item, from there you have the option to enter a name for the job, to select Multibranch Pipeline, and then click OK.
2. The SCM source (which includes GitHub, Git, Subversion, Mercurial, and Bitbucket) should be configured by the given information about the repository, scan credentials, and owner in appropriate fields.
Eg:- If you choose Git as the branch source, you will be asked for the normal connection information, but then rather than enter a fixed refspec ( which is the Git name of the source/destination pair), then you will enter the branch name pattern (you can then check default settings to see any branch).
3. Configuration of the multi-branch pipeline options:
a. API endpoint - An alternate API endpoint is used to use a self-hosted GitHub Enterprise.
b. Checkout credentials - An alternate credential that is used when checking out the code (cloning).
c. Include branches - Regular expression to specify branches to include.
d. Exclude branches - Regular expression to specify branches to exclude. This will take precedence over the contents of included expressions.
e. Property strategy -Define custom properties for each branch optionally.
f. Save.
Jenkins scans the designated repository automatically and creates appropriate branches.
For example, if you want to start from a master branch, then check some changes, then do git checkout -b newFeatureBranch and push some commits, Jenkins will automatically find the new branch in the repository and create a new subproject for it. This subproject will have its own build history independent of the trunk (main line).
After the branch is merged with the main line and deleted, you can request for the automatic removal of its sub-project. For example, if you need to change your Pipeline Script, add a new Jenkins publisher step corresponding to the new reports that your Makefile/pom.xml/etc. is creating, and then you can edit the Jenkinsfile in your need. Pipeline scripts are always synchronized with the source code you are working on, and the checkout scm command checks for the same revision as the loaded script.
This is how we can create a Jenkins pipeline and create builds. For more information regarding Jenkins please refer to the following blogs
What is Jenkins and How to use it
How to Add Git Credentials in Jenkins