Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Create slave borrowing flow

Description

Create a set of jobs and groovy functions that will enable one Jenkins master to borrow slaves from another Jenkins master.

Acceptance criteria

  1. We define a borrower and a borrowing Jenkins instances. Ans support the use case where both are the same instance

  2. Borrowing is done by setting up a JNLP connection from the slave of the borrower instance to to borrowing instance.

  3. On the borrower instance:

    1. We have a "borrow" job that:

      • Takes slave requirement details, allocates a matching slaves and make it connect via JNLP to the borrowing instance

      • If all matching slaves are already borrowed by other instances, try to call the garbage collector jobs (see below) on borrowing instances to free the slaves.

    2. We have a "return" job that is called by a borrowing instance to return a slave

  4. On the borrowing instance

    1. STDCI tries to borrow slaves from a predefined instance if not matching slaves are available locally

    2. Once borrowed - slaves stay attached to the borrowing instance so they can be reused by other STDCI jobs.

    3. We have a "garbage collector" job that is triggered periodically or by the borrower instance. It looks at borrowed slaves, and frees ones that were not used recently.

Activity

Show:

Barak Korren June 26, 2018 at 3:12 PM

Design details about the slave allocation mechanics in the borrowing instance:

  • STDCI jobs will calculate some consistent checksum for node requirements and use it as a label expression passed to the node() pipeline function.

  • before running the node() function the STDCI jobs will trigger a slave allocator job in the background

  • the slave allocator job will wait a while to let a local slave get allocated

  • if no local slave gets allocated, the slave allocator job will try to borrow a slave from the borrower instance and label it with the label expression.

Barak Korren June 26, 2018 at 1:33 PM

Some design details:

  • The "borrow" job takes the following parameters:

    • The IP and JNLP port of the Jenkins that is borrowing the slave

    • The JNLP secret to register to the Jenkins that is borrowing the slave

    • A YAML/JSON structure with node requirements that includes:

      • A kind field set to "stdci_node_req"

      • An apiVersion field set to "1".

      • The project name.

      • The project clone_ur, branch and refspec.

      • The STDCI thread stage, substage, distro and arch.

      • The STDCI runtime_requirements.

Won't Fix

Details

Assignee

Reporter

Components

Priority

Parent

Created June 26, 2018 at 7:02 AM
Updated August 29, 2019 at 2:12 PM
Resolved July 4, 2019 at 12:21 PM