When it comes to optimizing development, a good idea is to try to automate processes that are done manually. Obviously, those that require human intervention will not be able to be automated (or at least not completely), but other processes such as test execution on a project or operations such as refresh an environment, deployments or other frequent actions, can save a lot of time (and some mental health for the developers: executing repetitive tasks manually is usually very frustrating).
This is where a tool like Jenkins comes in play. Jenkins is an open source automation software that allows you to implement continuous integration tasks such as automatic processes to deploy environments, test different branches or other operations on different sites of a project.
Jenkins, the task master
Jenkins' basic unit are the tasks, jobs or projects, terms that can be used separately. Each job is a task managed by Jenkins, and thanks to Jenkins' flexibility it can be almost anything: running tests, reloading environments, cleaning temporary files from other processes or any other automatized procedure. For example, imagine a Drupal project: a Drupal site offers a series of operations that normally require either administrator access to the site or access to the console of the server (or servers) where it runs. With Jenkins we could create tasks for each of these operations, so that to perform them you only need to access the Jenkins server. If we concentrate the operations of our projects on Jenkins, the complexity of accessing the operations of all our sites is reduced to accessing Jenkins (well, and to establish a security policy on Jenkins' access to jobs, but that's the subject of another article).
For example, let's say a project offers the following operations for each environment:
- Get a login link.
- Clear caches.
- Cron execution.
- Status report.
- Deploy new code.
- Sync environments (the source will usually be the production environment).
- Do backups.
- Smoking test execution.
- Run tests on a code branch.
Interesting, don't you think? What's less interesting is having to configure each of these tasks. If we only had one project in development there would be no problem: it is configured once and that's it. But when we have several or many projects underway it's tedious and error-prone to have to manually create the same tasks for each project using the Jenkins interface. Wait a minute, didn't we say that in order to speed up development, a good idea is to automate processes? Well, the creation of jobs at Jenkins looks like a manual process that can be automated!
Job DSL plugin, job maker
Among Jenkins' many plugins is Job DSL. This plugin allows you to create several jobs automatically from a main job called seed. It is so called because it is the seed from which other jobs are created. In this way we avoid having to re-create the same jobs for each new project that requires similar tasks, being able to mount all the tasks for each project simply by executing the seed task. Not only that, if we modify the seed task and run it again Jenkins will update the jobs already created. In this way, we could update a single job (our seed task) and automatically update all projects that use the tasks generated by that seed. Fantastic, isn't it?
The seed task basically consists of a file with a Groovy script where the tasks to be created automatically are defined. Even if you have never used Groovy you should not worry, there are many examples and its syntax is very similar to other languages you probably know or you are familiar with. In addition, you can count on the help of the plugin XML JOB to Job DSL, which exports existing tasks to Groovy format, which can serve as a reference to create new tasks from this generated code.
Creating a seed task
Next we are going to show an example of how to generate a basic job using the Job DSL plugin. The first thing you have to do is create a new job of the Freestyle type:
Once created, we must add a «Process Job DSLs» step to the Build section. This section is where you tell Jenkins what the task is and what steps you need to take to complete it. Jenkins offers several types of steps that we won't discuss here because they fall outside the scope of this article. It is enough to know that here we must tell Jenkins where the Groovy file to be executed is:
Where that file comes from depends on how you get Jenkins mounted. Usually, you'll configure a code repository that Jenkins consults and downloads that contains the Groovy file, but we can write the script directly in the task configuration or set a path in the file system where we are going to place the DSL script if we prefer to use a file. The mechanism is secondary, but in the end Jenkins will always end up having the Groovy script with the instructions to create the jobs.
An example of a Groovy script
And what would a Groovy script look like for a seed task? Let's see an example of how to create a task container in Jenkins, with the folder method (in other words, a folder where to put jobs), and inside this folder create a task of pipeline type (is one of the types of jobs Jenkins offers):
String basePath = 'dsljobs'
String repo = 'git@repo:name/project'
folder(basePath) {
description 'DSL generated folder.'
}
pipelineJob("$basePath/status") {
description()
definition {
cpsScm {
scm {
git {
remote {
url(repo)
}
branch("*/dev")
}
}
scriptPath("Jenkinsfile.status")
}
}
}The creation of the folder, as expected, is the block that starts with folder. The next block, pipelineJob, creates the pipeline type job. The type is not relevant, but we use this type of job extensively so we use it here as an example. In the created pipeline task, a URL to a code repository and a Jenkinsfile are defined, which is the file that actually contains the instructions that make up the job to be created. Of course, the DSL file itself can contain the details of the task to be executed instead of referencing another file, but we didn't have a simpler example at hand.
Within the DSL file we can make use of many functions offered by the plugin as part of its API.
Once the seed task is created we can access it from the Jenkins interface, and if we press «Build now» will execute and create the task we've defined inside. Obviously our example is poor: Job DSL is really useful when creating multiple tasks, as the list of operations at the beginning of the article, but the idea is the same: create tasks automatically.
By the way, the execution of the seed task may fail due to a Jenkins security mechanism that prevents the execution of Groovy files that have not been approved. To do this go to the Jenkins configuration, to the section «In-process script Approval». Once the Groovy script is approved, we'll press to build the task again and it should work. This has to be done every time you change the script for security reasons. Although it can be disabled, it is not recommended to do so if we want to keep Jenkins safe.
And that's all for today. Jenkins and Job DSL have many more features, here we've just scratched the surface a little, but I think that what I have seen in this article is enough to see the great possibilities it offers when it comes to automating the creation of tasks for our projects.
