Job Definitions

The job definitions for Jenkins Job Builder are kept in any number of YAML or JSON files, in whatever way you would like to organize them. When you invoke jenkins-jobs you may specify either the path of a single YAML file, or a directory. If you choose a directory, all of the .yaml/.yml or .json files in that directory will be read, and all the jobs they define will be created or updated.

Note

Jenkins Job Builder 2.x plugins are designed to default to generating the xml format for the latest supported version of JJB. This is a change in behaviour from 1.x and below which defaulted to the oldest supported plugin version.

Definitions

Jenkins Job Builder understands a few basic object types which are described in the next sections.

Job

The most straightforward way to create a job is simply to define a Job in YAML. It looks like this:

- job:
    name: job-name

That’s not very useful, so you’ll want to add some actions such as Builders, and perhaps Publishers. Those are described later.

These are job parameters that are common to every type of Jenkins job.

Example:

- job:
    name: job-name
    project-type: freestyle
    defaults: global
    description: 'Do not edit this job through the web!'
    disabled: false
    display-name: 'Fancy job name'
    concurrent: true
    workspace: /srv/build-area/job-name
    quiet-period: 5
    block-downstream: false
    block-upstream: false
    retry-count: 3
    node: NodeLabel1 || NodeLabel2
    logrotate:
      daysToKeep: 3
      numToKeep: 20
      artifactDaysToKeep: -1
      artifactNumToKeep: -1
Job Parameters:
  • project-type: Defaults to “freestyle”, but “maven” as well as “multijob”, “flow”, “pipeline” or “externaljob” can also be specified.

  • defaults: Specifies a set of Defaults to use for this job, defaults to ‘’global’’. If you have values that are common to all of your jobs, create a global Defaults object to hold them, and no further configuration of individual jobs is necessary. If some jobs should not use the global defaults, use this field to specify a different set of defaults.

  • description: The description for the job. By default, the description “!– Managed by Jenkins Job Builder” is applied.

  • disabled: Boolean value to set whether or not this job should be disabled in Jenkins. Defaults to false (job will be enabled).

  • display-name: Optional name shown for the project throughout the Jenkins web GUI in place of the actual job name. The jenkins_jobs tool cannot fully remove this trait once it is set, so use caution when setting it. Setting it to the same string as the job’s name is an effective un-set workaround. Alternately, the field can be cleared manually using the Jenkins web interface.

  • concurrent: Boolean value to set whether or not Jenkins can run this job concurrently. Defaults to false.

  • workspace: Path for a custom workspace. Defaults to Jenkins default configuration.

  • folder: The folder attribute provides an alternative to using ‘<path>/<name>’ as the job name to specify which Jenkins folder to upload the job to.

    Requires the Jenkins CloudBees Folders Plugin.

  • child-workspace: Path for a child custom workspace. Defaults to Jenkins default configuration. This parameter is only valid for matrix type jobs.

  • quiet-period: Number of seconds to wait between consecutive runs of this job. Defaults to 0.

  • block-downstream: Boolean value to set whether or not this job must block while downstream jobs are running. Downstream jobs are determined transitively. Defaults to false.

  • block-upstream: Boolean value to set whether or not this job must block while upstream jobs are running. Upstream jobs are determined transitively. Defaults to false.

  • auth-token: Specifies an authentication token that allows new builds to be triggered by accessing a special predefined URL. Only those who know the token will be able to trigger builds remotely.

  • retry-count: If a build fails to checkout from the repository, Jenkins will retry the specified number of times before giving up.

  • node: Restrict where this job can be run. If there is a group of machines that the job can be built on, you can specify that label as the node to tie on, which will cause Jenkins to build the job on any of the machines with that label. For matrix projects, this parameter will only restrict where the parent job will run.

  • logrotate: The Logrotate section allows you to automatically remove old build history. It adds the logrotate attribute to the Job definition. All logrotate attributes default to “-1” (keep forever). Deprecated on jenkins >=1.637: use the build-discarder property instead

  • jdk: The name of the jdk to use

  • raw: If present, this section should contain a single xml entry. This XML will be inserted at the top-level of the Job definition.

Job Template

If you need several jobs defined that are nearly identical, except perhaps in their names, SCP targets, etc., then you may use a Job Template to specify the particulars of the job, and then use a Project to realize the job with appropriate variable substitution. Any variables not specified at the project level will be inherited from the Defaults.

A Job Template has the same syntax as a Job, but you may add variables anywhere in the definition. Variables are indicated by enclosing them in braces, e.g., {name} will substitute the variable name. When using a variable in a string field, it is good practice to wrap the entire string in quotes, even if the rules of YAML syntax don’t require it because the value of the variable may require quotes after substitution. In the rare situation that you must encode braces within literals inside a template (for example a shell function definition in a builder), doubling the braces will prevent them from being interpreted as a template variable.

You must include a variable in the name field of a Job Template (otherwise, every instance would have the same name). For example:

- job-template:
    name: '{name}-unit-tests'

Will not cause any job to be created in Jenkins, however, it will define a template that you can use to create jobs with a Project definition. It’s name will depend on what is supplied to the Project.

If you use the variable {template-name}, the name of the template itself (e.g. {name}-unit-tests in the above example) will be substituted in. This is useful in cases where you need to trace a job back to its template.

Sometimes it is useful to have the same job name format used even where the template contents may vary. Ids provide a mechanism to support such use cases in addition to simplifying referencing templates when the name contains the more complex substitution with default values.

Default Values for Template Variables

To facilitate reuse of templates with many variables that can be substituted, but where in most cases the same or no value is needed, it is possible to specify defaults for the variables within the templates themselves.

There are 2 ways JJB allows us to define defaults for a parameter in a job-template.

  1. Defining the default variable value in the job-template itself

    With this method we declare the default value of the variable in the job-template itself just once. We can section off the job-template into two sections like this:

    - job-template:
        name: '{project-name}-verify'
    
        #####################
        # Variable Defaults #
        #####################
    
        branch: master
    
        #####################
        # Job Configuration #
        #####################
    
        parameters:
          - string:
              name: BRANCH
              default: '{branch}'
    
        scm:
          - git:
              refspec: 'refs/heads/{branch}'
    

    In this case there is still two branch definitions for the job-template. However we also provide the default value for the {branch} variable at the top of the file. Just once. This will be the value that the job takes on if it is not passed in by a project using the template.

    And, you can do the same in Macro definitions.

  2. Using {var|default}

    In this method we can define the default with the definition of the variable. For example:

    - job-template:
        name: '{project-name}-verify'
        parameters:
          - string:
              name: BRANCH
              default: '{branch|master}'
    

    However where this method falls apart if we need to use the same JJB variable in more than one place as we will have multiple places to define the default value for the template. For example:

    - job-template:
        name: '{project-name}-verify'
        parameters:
          - string:
              name: BRANCH
              default: '{branch|master}'
    
        scm:
          - git:
              refspec: 'refs/heads/{branch|master}'
    

    We can see in this case the {branch|master} variable is defined in two places. Not ideal.

More complex example:

- project:
    name: template_variable_defaults
    jobs:
        - 'template-variable-defaults-{num}':
            num: 1
            disabled_var: true
        - 'template-variable-defaults-{num}':
            test_var: Goodbye World
            num: 2

- job-template:
    # template specific defaults
    # empty value causes disabled_var to be ignored internally
    disabled_var:
    test_var: Hello World
    type: periodic

    # template settings
    name: 'template-variable-defaults-{num}-{type}'
    id: 'template-variable-defaults-{num}'
    disabled: '{obj:disabled_var}'
    builders:
      - shell: |
         echo "Job Name: template-variable-defaults-{num}-{type}"
         echo "Variable: {test_var}"

To use a default value for a variable used in the name would be uncommon unless it was in addition to another variable. However you can use Ids simplify such use cases.

Project

The purpose of a project is to collect related jobs together, and provide values for the variables in a Job Template. It looks like this:

- project:
    name: project-name
    jobs:
      - '{name}-unit-tests'

Any number of arbitrarily named additional fields may be specified, and they will be available for variable substitution in the job template. Any job templates listed under jobs: will be realized with those values. The example above would create the job called ‘project-name-unit-tests’ in Jenkins.

The jobs: list can also allow for specifying job-specific substitutions as follows:

- project:
    name: project-name
    jobs:
      - '{name}-unit-tests':
          mail-to: developer@nowhere.net
      - '{name}-perf-tests':
          mail-to: projmanager@nowhere.net

If a variable is a list, the job template will be realized with the variable set to each value in the list. Multiple lists will lead to the template being realized with the cartesian product of those values. Example:

- project:
    name: project-name
    pyver:
      - 26
      - 27
    jobs:
      - '{name}-{pyver}'

If there are templates being realized that differ only in the variable used for its name (thus not a use case for job-specific substitutions), additional variables can be specified for project variables. Example:

- job-template:
    name: '{name}-{pyver}'
    builders:
      - shell: 'git co {branch_name}'

- project:
   name: project-name
   pyver:
    - 26:
       branch_name: old_branch
    - 27:
       branch_name: new_branch
   jobs:
    - '{name}-{pyver}'

You can also specify some variable combinations to exclude from the matrix with the exclude keyword, to avoid generating jobs for those combinations. You can specify all the variables of the combination or only a subset, if you specify a subset, any value of the omited variable will match:

- project:
    name: project-name
    axe1:
      - axe1val1
      - axe1val2
    axe2:
      - axe2val1
      - axe2val2
    axe3:
      - axe3val1
      - axe3val2
    exclude:
      - axe1: axe1val1
        axe2: axe2val1
        axe3: axe3val2
      - axe2: axe2val2
        axe3: axe3val1
    jobs:
      - build-{axe1}-{axe2}-{axe3}

- job-template:
    name: build-{axe1}-{axe2}-{axe3}
    builders:
      - shell: "echo Combination {axe1}:{axe2}:{axe3}"

The above example will omit the jobs:

  • build-axe1val1-axe2val1-axe3val2

  • build-axe1val1-axe2val2-axe3val1

  • build-axe1val2-axe2val2-axe3val1

To achieve the same without the exclude tag one would have to do something a bit more complicated, that gets more complicated for each dimension in the combination, for the previous example, the counterpart would be:

- project:
    name: project-name_comb1
    axe1:
      - axe1val1
      - axe1val2
    axe2: axe2val1
    axe3: axe3val1
    jobs:
      - build-{axe1}-{axe2}-{axe3}

- project:
    name: project-name_comb2
    axe1:
      - axe1val1
      - axe1val2
    axe2: axe2val2
    axe3: axe3val2
    jobs:
      - build-{axe1}-{axe2}-{axe3}

- project:
    name: project-name_comb3
    axe1: axe1val2
    axe2: axe2val1
    axe3: axe3val2
    jobs:
      - build-{axe1}-{axe2}-{axe3}

- job-template:
    name: build-{axe1}-{axe2}-{axe3}
    builders:
      - shell: "echo Combination {axe1}:{axe2}:{axe3}"

Job Group

If you have several Job Templates that should all be realized together, you can define a Job Group to collect them. Simply use the Job Group where you would normally use a Job Template and all of the Job Templates in the Job Group will be realized. For example:

- job-template:
    name: '{name}-unit-tests'
    builders:
    - shell: unittest
    publishers:
    - email:
        recipients: '{mail-to}'

- job-template:
    name: '{name}-perf-tests'
    builders:
    - shell: perftest
    publishers:
    - email:
        recipients: '{mail-to}'

- job-group:
    name: '{name}-tests'
    jobs:
    - '{name}-unit-tests':
        mail-to: developer@nowhere.net
    - '{name}-perf-tests':
        mail-to: projmanager@nowhere.net

- project:
    name: project-name
    jobs:
    - '{name}-tests'

Would cause the jobs project-name-unit-tests and project-name-perf-tests to be created in Jenkins.

Views

A view is a particular way of displaying a specific set of jobs. To create a view, you must define a view in a YAML file and have a variable called view-type with a valid value. It looks like this:

- view:
    name: view-name
    view-type: list

Views are processed differently than Jobs and therefore will not work within a Project or a Job Template.

View Template

Allow views to also be configured via templates similar to job-templates. This is useful when you have multiple views defined that have similar configuration except for a few variables. View Templates can be passed variables to fill in sections automatically via a project configuration using the new ‘views’ key.

Minimal Example:

- view-template:
    name: '{name}-template-{seq}'
    description: 'testing view templates feature'
    view-type: list
    regex: 'test-view-.*'

- project:
    name: 'test-view'
    views:
        - '{name}-template-{seq}'
    seq:
        - a
        - b
        - c

Macro

Many of the actions of a Job, such as builders or publishers, can be defined as a Macro, and then that Macro used in the Job description. Builders are described later, but let’s introduce a simple one now to illustrate the Macro functionality. This snippet will instruct Jenkins to execute “make test” as part of the job:

- job:
    name: foo-test
    builders:
      - shell: 'make test'

If you wanted to define a macro (which won’t save much typing in this case, but could still be useful to centralize the definition of a commonly repeated task), the configuration would look like:

- builder:
    name: make-test
    builders:
      - shell: 'make test'

- job:
    name: foo-test
    builders:
      - make-test

This allows you to create complex actions (and even sequences of actions) in YAML that look like first-class Jenkins Job Builder actions. Not every attribute supports Macros, check the documentation for the action before you try to use a Macro for it.

Macros can take parameters, letting you define a generic macro and more specific ones without having to duplicate code:

# The 'add' macro takes a 'number' parameter and will creates a
# job which prints 'Adding ' followed by the 'number' parameter:
- builder:
    name: add
    builders:
     - shell: "echo Adding {number}"

# A specialized macro 'addtwo' reusing the 'add' macro but with
# a 'number' parameter hardcoded to 'two':
- builder:
    name: addtwo
    builders:
     - add:
        number: "two"

# Glue to have Jenkins Job Builder to expand this YAML example:
- job:
    name: "testingjob"
    builders:
     # The specialized macro:
     - addtwo
     # Generic macro call with a parameter
     - add:
        number: "ZERO"

Then <builders /> section of the generated job show up as:

<builders>
  <hudson.tasks.Shell>
    <command>echo Adding two</command>
  </hudson.tasks.Shell>
  <hudson.tasks.Shell>
    <command>echo Adding ZERO</command>
  </hudson.tasks.Shell>
</builders>

As you can see, the specialized macro addtwo reused the definition from the generic macro add.

Folders

Jenkins supports organising jobs, views, and slaves using a folder hierarchy. This allows for easier separation of access as well credentials and resources which can be assigned to only be available for a specific folder.

JJB has two methods of supporting uploading jobs to a specific folder:

  • Name the job to contain the desired folder <folder>/my-job-name

  • Use the folder attribute on a job definition, via a template, or through Defaults.

Supporting both an attributed and use of it directly in job names allows for teams to have all jobs using their defaults automatically use a top-level folder, while still allowing for them to additionally nest jobs for their own preferences.

Job Name Example:

- job:
    name: python-jobs/tox-py27
    builders:
      - shell: |
          tox -e py27

Folder Attribute Example:

# 'folders' attribute from defaults should be used (combined with job name) for jobs.

- defaults:
    name: team1
    folder: team1-jobs

- job:
    name: ruby-jobs/rspec
    defaults: team1
    builders:
      - shell: |
          rvm use --create ruby-2.3.0@rspec
          bundle install
          bundle exec rspec

Item ID’s

It’s possible to assign an id to any of the blocks and then use that to reference it instead of the name. This has two primary functions:

  • A unique identifier where you wish to use the same naming format for multiple templates. This allows you to follow a naming scheme while still using multiple templates to handle subtle variables in job requirements.

  • Provides a simpler name for a job-template where you have multiple variables including default values in the name and don’t wish to have to include this information in every use. This also makes changing the template output name without impacting references.

Example:

- project:
    name: test_template_id
    jobs:
        - 'simple-template':
            test_var: Hello World
            type: periodic
            num: 1
        - 'not-as-simple-template':
            test_var: Goodbye World
            type: canary
            num: 2

- job-template:
    name: 'template-test-ids-{num}-{type}'
    id: simple-template
    builders:
      - shell: |
         echo "Template name: {template-name}"
         echo "Job name: template-test-ids-{num}-{type}"
         echo "{test_var}"

- job-template:
    name: 'template-test-ids-{num}-{type}'
    id: not-as-simple-template
    builders:
      - shell: |
         echo "Template name: {template-name}"
         echo "Job name: template-test-ids-{num}-{type}"
      - shell: |
         echo "{test_var}"

Raw config

It is possible, but not recommended, to use raw within a module to inject raw xml into the job configs.

This is relevant in case there is no appropriate module for a Jenkins plugin or the module does not behave as you expect it to do.

For example:

wrappers:
  - raw:
      xml: |
        <hudson.plugins.xvnc.Xvnc>
          <takeScreenshot>true</takeScreenshot>
          <useXauthority>false</useXauthority>
        </hudson.plugins.xvnc.Xvnc>

Is the raw way of adding support for the xvnc wrapper.

To get the appropriate xml to use you would need to create/edit a job in Jenkins and grab the relevant raw xml segment from the config.xml.

The xml string can refer to variables just like anything else and as such can be parameterized like anything else.

You can use raw in most locations, the following example show them with arbitrary xml-data:

- project:
    name: complete002
    version:
        - 1.2
    jobs:
        - 'complete001_{version}'

- job-template:
    name: 'complete001_{version}'
    project-type: maven
    scm:
      - raw:
          xml: |
            <!-- <scm> for raw replaces the whole scm section.
             where as for others the raw part is added to the existing.
            -->
            <scm>
              <scmraw/>
            </scm>
    triggers:
      - raw:
          xml: |
            <triggersraw/>
    wrappers:
      - raw:
          xml: |
            <wrappersraw/>
    builders:
      - raw:
          xml: |
            <buildersraw/>
    publishers:
      - raw:
          xml: |
            <publishersraw/>
    properties:
      - raw:
          xml: |
            <propertiesraw/>
    parameters:
      - raw:
          xml: |
            <parametersraw/>
    notifications:
      - raw:
          xml: |
            <metadataraw/>
    reporters:
      - raw:
          xml:
            <reportersraw/>

Note: If you have a need to use raw please consider submitting a patch to add or fix the module that will remove your need to use raw.

Defaults

Defaults collect job attributes (including actions) and will supply those values when the job is created, unless superseded by a value in the Job definition. If a set of Defaults is specified with the name global, that will be used by all Job, Job Template and Macro definitions unless they specify a different Default object with the defaults attribute. For example:

- defaults:
    name: global
    description: 'Do not edit this job through the web!'

Will set the job description for every job created.

You can define variables that will be realized in a Job Template.

- defaults:
    name: global
    arch: 'i386'

- project:
    name: project-name
    jobs:
        - 'build-{arch}'
        - 'build-{arch}':
            arch: 'amd64'

- job-template:
    name: 'build-{arch}'
    builders:
        - shell: "echo Build arch {arch}."

Would create jobs build-i386 and build-amd64.

In job templates, you can also reference a variable {template-name} in any value and it will be subtitued by the name of the current job template being processed.

Variable References

If you want to pass an object (boolean, list or dict) to templates you can use an {obj:key} variable in the job template. This triggers the use of code that retains the original object type.

For example:

- project:
    name: test_custom_distri
    disabled: true
    distributions: !!python/tuple [precise, jessie]
    architectures: !!python/tuple &architectures
      - amd64
      - i386
    axis_a:
        type: user-defined
        name: architectures
        values: *architectures
    jobs:
      - '{name}-source'

- job-template:
      name: '{name}-source'
      project-type: matrix
      disabled: '{obj:disabled}'
      axes:
        - axis:
            type: user-defined
            name: distribution
            values: '{obj:distributions}'
        - axis: '{obj:axis_a}'

JJB also supports interpolation of parameters within parameters. This allows a little more flexibility when ordering template jobs as components in different projects and job groups.

For example:

- job-template:
    name: '{value-stream}_{project-id}_foo'
    display-name: '{value-stream} {project-id} foo'
    publishers:
        - trigger-parameterized-builds:
            - project: '{downstream}'
              current-parameters: False
              condition: ALWAYS
              git-revision: True

- job-template:
    name: '{value-stream}_{project-id}_bar'
    display-name: '{value-stream} {project-id} bar'
    publishers:
        - trigger-parameterized-builds:
            - project: '{downstream}'
              current-parameters: False
              condition: ALWAYS
              git-revision: True

- job-group:
    name: 'pipeline2'
    project-id: 'p2'
    jobs:
        - '{value-stream}_{project-id}_foo':
            downstream: '{value-stream}_{project-id}_bar'
        - '{value-stream}_{project-id}_bar':

- job-group:
    name: 'pipeline1'
    project-id: 'p1'
    jobs:
        - '{value-stream}_{project-id}_bar':
            downstream: '{value-stream}_{project-id}_foo'
        - '{value-stream}_{project-id}_foo':

- project:
    name: derp
    jobs:
        - 'pipeline1':
            value-stream: 'production'
        - 'pipeline2':
            value-stream: 'experimental'

- defaults:
    name: 'global'
    downstream: ''

By default JJB will fail if it tries to interpolate a variable that was not defined, but you can change that behavior and allow empty variables with the allow_empty_variables configuration option.

For example, having a configuration file with that option enabled:

[job_builder]
allow_empty_variables = True

Will prevent JJb from failing if there are any non-initialized variables used and replace them with the empty string instead.

Tip

Refer to Default Values for Template Variables for details on setting variable defaults.

Variable Inheritance

It is possible in JJB to define defaults for variables at different levels such that it is possible for users of job-templates to override variables defined in the job-template.

Variable priorities for each definition type are as follows:

  1. job-group

  2. project

  3. job-template

  4. defaults

From this list we can immediately see that if we want to make variables in job-templates override-able then using defaults configuration is useless as it has the lowest precedence when JJB is deciding where to pull from.

On the other side of the spectrum, job-groups has the highest precedence. Which unfortunately means if we define a variable in a job-group with the intention of overriding it at the project level then we are out of luck. For this reason avoid setting variables in job-groups unless we want to enforce a setting for a set of jobs and prevent projects from overriding it.

Declaring variable defaults

Refer to Default Values for Template Variables for details on how to declare variable defaults.

Overriding job-template variables

When a project wants to use a job-template it can use override it as follows:

- project:
    name: foo
    jobs:
      - '{project-name}-merge'
      - '{project-name}-verify'

    branch: master

This is the standard way that most folks use and it will set branch: master for every job-template in the list. However sometimes we may want to provide an alternative value for a specific job in the list. In this case the more specific declaration takes precedence:

- project:
    name: foo
    jobs:
      - '{project-name}-merge':
          branch: production
      - '{project-name}-verify'

    branch: master

In this case the verify job will get the value master but the merge job will instead get the branch value production.

Yaml Anchors & Aliases

The yaml specification supports anchors and aliases which means that JJB definitions allow references to variables in templates.

For example:

- _wrapper_defaults: &wrapper_defaults
    name: 'wrapper_defaults'
    wrappers:
      - timeout:
          timeout: 180
          fail: true
      - timestamps

- _job_defaults: &job_defaults
    name: 'defaults'
    <<: *wrapper_defaults

- job-template:
    name: 'myjob'
    <<: *job_defaults

- project:
    name: myproject
    jobs:
      - myjob

The anchors and aliases are expanded internally within JJB’s yaml loading calls and are not limited to individual documents. That means you can’t use the same anchor name in included files without collisions.

A simple example can be seen in the specs full length example with the following being more representative of usage within JJB:

- wrapper_defaults: &wrapper_defaults
    name: 'wrapper_defaults'
    wrappers:
      - timeout:
          timeout: 180
          fail: true
      - timestamps

- job_defaults: &job_defaults
    name: 'defaults'
    <<: *wrapper_defaults

- job-template:
    name: 'myjob'
    <<: *job_defaults

Which will be expanded to the following yaml before being processed:

- wrapper_defaults:
    name: wrapper_defaults
    wrappers:
    - timeout:
        fail: true
        timeout: 180
    - timestamps
- job_defaults:
    name: defaults
    wrappers:
    - timeout:
        fail: true
        timeout: 180
    - timestamps
- job-template:
    name: myjob
    wrappers:
    - timeout:
        fail: true
        timeout: 180
    - timestamps

Custom Yaml Tags

Custom application specific yamls tags are supported to provide enhancements when reading yaml configuration.

Action Tags

These allow manipulation of data being stored in one layout in the source yaml for convenience and/or clarity, to another format to be processed by the targeted module instead of requiring all modules in JJB being capable of supporting multiple input formats.

The tag !join: will treat the first element of the following list as the delimiter to use, when joining the remaining elements into a string and returning a single string to be consumed by the specified module option.

This allows users to maintain elements of data in a list structure for ease of review/maintenance, and have the yaml parser convert it to a string for consumption as any argument for modules. The main expected use case is to allow for generic plugin data such as shell properties to be populated from a list construct which the yaml parser converts to a single string, instead of trying to support this within the module code which would require a templating engine similar to Jinja.

Generic Example:

- job:
    name: sample-job
    builders:
      - shell:
        - string-with-comma: !join:
            - ','
            -
                - item1
                - item2
                - item3

        - string-with-space: !join:
            - ' '
            -
                - item1
                - item2
                - item3

Environment Inject:

- project:
    name: string_join_example
    jobs:
        - 'string-join-data-{name}':
            name: set1
            files: !join:
              - ','
              -
                - /path/to/file1
                - /path/to/file2
                - /path/to/file3
                - /path/to/file4
                - /path/to/file5
                - /path/to/file6
                - /path/to/file7
                - /path/to/file8
                - /path/to/file9
                - /path/to/file10
                - /path/to/file11
                - /path/to/file12
                - /path/to/file13
                - /path/to/file14
                - /path/to/file15
                - /path/to/file16
                - /path/to/file17
                - /path/to/file18
                - /path/to/file19
                - /path/to/file20
        - 'string-join-data-{name}':
            name: set2
            files: !join:
              - ','
              -
                - /another/different/path/to/file1
                - /another/different/path/to/file2
                - /another/different/path/to/file3
                - /another/different/path/to/file4
                - /another/different/path/to/file5
                - /another/different/path/to/file6
                - /another/different/path/to/file7
                - /another/different/path/to/file8
                - /another/different/path/to/file9
                - /another/different/path/to/file10
                - /another/different/path/to/file11
                - /another/different/path/to/file12
                - /another/different/path/to/file13
                - /another/different/path/to/file14
                - /another/different/path/to/file15
                - /another/different/path/to/file16
                - /another/different/path/to/file17
                - /another/different/path/to/file18
                - /another/different/path/to/file19
                - /another/different/path/to/file20

- job-template:
    name: 'string-join-data-{name}'
    properties:
      - inject:
          keep-system-variables: true
          properties-content: |
            FILE_LIST={files}
    builders:
      - shell: |
         echo "Template name: {template-name}"
         echo "Data to be processed:"
         echo "${{INPUT_DATA}}"

While this mechanism can also be used items where delimiters are supported by the module, that should be considered a bug that the existing code doesn’t handle being provided a list and delimiter to perform the correct conversion for you. Should you discover a module that takes arguments with delimiters and the existing JJB codebase does not handle accepting lists, then this can be used as a temporary solution in place of using very long strings:

Extended Params Example:

parameters:
  - extended-choice:
      name: OPTIONS_VALUE
      description: "Available options"
      property-key: key
      quote-value: true
      type: multi-select
      value: "foo|bar|select"
      visible-items: 2
      multi-select-delimiter: '|'
      default-value: foo
      default-property-key: fookey
  - extended-choice:
      name: OPTIONS_FILE
      description: "Available options"
      property-file: /home/foo/property.prop
      property-key: key
      quote-value: true
      type: multi-select
      visible-items: 2
      multi-select-delimiter: '|'
      default-property-file: /home/property.prop
      default-property-key: fookey
  - extended-choice:
      name: OPTIONS_CHECKBOX
      type: checkbox
      value: !join:
        - ','
        -
            - OptionA
            - OptionB
            - OptionC
      visible-items: 2
  - extended-choice:
      name: MULTISELECTOPTIONS
      description: "Available options"
      property-key: key
      quote-value: true
      type: multi-select
      value: !join:
        - '|'
        -
            - foo
            - bar
            - select
      visible-items: 2
      multi-select-delimiter: '|'
      default-value: foo
  - extended-choice:
      name: JSON
      type: json
      groovy-script: >-
          import net.sf.json.JSONObject;
          def jsonEditorOptions = JSONObject.fromObject(/{schema:
          {"type": "object", "title": "Name", "properties":
          {"name": {"type": "string", "propertyOrder" : 1}}}}/);
  - extended-choice:
      name: MULTILEVELMULTISELECT
      type: multi-level-multi-select
      value: !join:
          - ','
          -
              - foo
              - bar
              - baz
  - extended-choice:
      name: MULTILEVELSINGLESELECT
      type: multi-level-single-select
      value: foo

Inclusion Tags

These allow inclusion of arbitrary files as a method of having blocks of data managed separately to the yaml job configurations. A specific usage of this is inlining scripts contained in separate files, although such tags may also be used to simplify usage of macros or job templates.

The tag !include: will treat the following string as file which should be parsed as yaml configuration data.

Example:

- job:
    name: test-job-1
    builders:
      !include: include001.yaml.inc

contents of include001.yaml.inc:

- copyartifact: &copytarball
    project: foo
    filter: "*.tar.gz"
    target: /home/foo
    which-build: last-successful
    optional: true
    flatten: true
    do-not-fingerprint: true
    parameter-filters: PUBLISH=true
- copyartifact:
    <<: *copytarball
    project: bar
    which-build: specific-build
    build-number: 123
- copyartifact:
    <<: *copytarball
    project: baz
    which-build: upstream-build

The tag !include-raw-expand: will treat the given string or list of strings as filenames to be opened as one or more data blob, which should be read into the calling yaml construct without any further parsing. Any data in a file included through this tag, will be treated as string data.

It will expand variables inside the file. If your file contains curly braces, you should double them. Or, you can use tag !include-raw-verbatim:, which does not substitute variables.

Examples:

- job-template:
    name: test-include-raw-verbatim-job-{num}
    builders:
      - shell:
          !include-raw-verbatim: include-raw-hello-world.sh
      - shell:
          !include-raw-verbatim: include-raw-vars.sh

- project:
    name: test-job-template-1
    num: 1
    jobs:
      - 'test-include-raw-verbatim-job-{num}'

contents of include-raw-hello-world.sh:

#!/bin/bash
#
# Sample script showing how the yaml !include-raw-verbatim: tag can be used
# to inline scripts that are maintained outside of the jenkins
# job yaml configuration.

echo "hello world"

exit 0

contents of include-raw-vars.sh:

#!/bin/bash
#
# sample script to check that variables aren't expanded
# when using the !include-raw-verbatim: application yaml tag

VAR1="hello"
VAR2="world"
VAR3="${VAR1} ${VAR2}"

[[ -n "${VAR3}" ]] && {
    # this next section is executed as one
    echo "${VAR3}"
    exit 0
}

Using a list of files:

- job-template:
    name: test-job-include-raw-{num}
    builders:
      - shell:
          !include-raw-verbatim:
              - include-raw-hello-world.sh
              - include-raw-vars.sh

- project:
    name: test-job-template-1
    num: 1
    jobs:
      - 'test-job-include-raw-{num}'

For all the multi file includes, the files are simply appended using a newline character.

You can also use variables in included file paths.

Example:

- wrapper:
    !include: lazy-load-jobs-timeout.yaml.inc

- project:
    name: test
    version:
      - 1.1
    jobs:
      - 'build_myproject_{version}'

- job-template:
    name: 'build_myproject_{version}'
    wrappers:
      !include: lazy-load-wrappers-{version}.yaml.inc
    builders:
      - shell:
          !include-raw-verbatim: echo_vars_{version}.sh

with variable substitution inside included files:

# Variables should be substituted in included raw files
# when their path themselves contains variables.

- job-template:
    name: 'sample-job-{version}'
    builders:
    - shell:
        !include-raw-expand: lazy-load-scripts/subst_vars_{version}.sh

- project:
    name: sample-project
    branch: master
    version:
    - 1.1
    - 1.2:
        branch: feature
    jobs:
    - 'sample-job-{version}'

using a list of files:

- wrapper:
    !include: lazy-load-jobs-timeout.yaml.inc

- project:
    name: test
    num: "002"
    version:
      - 1.1
    jobs:
      - 'build_myproject_{version}'

- job-template:
    name: 'build_myproject_{version}'
    wrappers:
      !include: lazy-load-wrappers-{version}.yaml.inc
    builders:
      - shell:
          !include-raw-verbatim:
            - lazy-load-scripts/echo_vars_{version}.sh
            - include-raw{num}-cool.sh

The tag !include-jinja2: will treat the given string or list of strings as filenames to be opened as Jinja2 templates, which should be rendered to a string and included in the calling YAML construct. (This is analogous to the templating that will happen with !include-raw.)

Examples:

- builder:
    name: test-builder
    builders:
        - shell:
            !include-jinja2: jinja01.yaml.inc

- job:
    name: test-job
    builders:
        - test-builder:
            var: "test variable"
            test_list:
                - a
                - b
                - c

contents of jinja01.yaml.inc:

{{ var }}
{% for item in test_list -%}
{{ item }}
{% endfor %}

The tag !j2: takes a string and treats it as a Jinja2 template. It will be rendered (with the variables in that context) and included in the calling YAML construct.

Examples:

- builder:
    name: test-builder
    builders:
        - shell:
            !j2: |
                {{ var }}
                {% for item in test_list -%}
                {{ item }}
                {% endfor %}

- job:
    name: test-job
    builders:
        - test-builder:
            var: "test variable"
            test_list:
                - a
                - b
                - c

The tag !j2-yaml: is similar to the !j2: tag, just that it loads the Jinja-rendered string as YAML and embeds it in the calling YAML construct. This provides a very flexible and convenient way of generating pieces of YAML structures. One of use cases is defining complex YAML structures with much simpler configuration, without any duplication.

Examples:

- job-template:
    name: test-job-template
    triggers:
    - gerrit:
        projects:
            !j2-yaml: |
                {% for item in triggers %}
                - branches:
                  - branch-compare-type: PLAIN
                    branch-pattern: '{{ item.branch }}'
                  project-compare-type: REG_EXP
                  project-pattern: '{{ item.repositories|join("|") }}'
                {% endfor %}

- project:
    name: test-job-project

    jobs:
    - test-job-template:
        triggers:
        - repositories:
            - a
            - b
            - c
          branch: master
        - repositories:
            - d
            - e
            - f
          branch: stable

Another use case is controlling lists dynamically, like conditionally adding list elements based on project configuration.

Examples:

- job-template:
    name: 'test-job-{variant}'
    properties: !j2-yaml: |
        - rebuild
        {% if discard_old_builds|default %}
        - build-discarder:
            days-to-keep: 7
        {% endif %}

- project:
    name: test-project

    jobs:
    - 'test-job-{variant}':
        variant: abc

    - 'test-job-{variant}':
        variant: def
        discard_old_builds: true

Modules

The bulk of the job definitions come from the following modules.

Module Execution

The jenkins job builder modules are executed in sequence.

Generally the sequence is:
  1. parameters/properties

  2. scm

  3. triggers

  4. wrappers

  5. prebuilders (maven only, configured like Builders)

  6. builders (maven, freestyle, matrix, etc..)

  7. postbuilders (maven only, configured like Builders)

  8. publishers/reporters/notifications