Better STD-CI for GitHub

Description

We currently have some support for crating STD-CI jobs for projects running on GitHub. There are however, several issues with it:

  • The job templates you use for GitHub are different then the ones you use for Gerrit:

    • Job names need to contain '-github-'

    • In Gerrit you need a separate 'project' entry for the 'build-artifacts' jobs, in GitHub you need a separate one for 'check-patch'.

    • SCM checkout details are specified in the 'project' entry as opposed to a separate SCM file (We might want to adopt this for Gerrit).

  • The 'check-patch' jobs do not have full STC-CI. They don't run the post-build steps.

  • The triggering scheme is strange:

    • All jobs trigger when certain messages are posted in the PR comments.

    • check-patch is triggered automatically when a PR is updated unless the user is not white listed, in which case nag messages are displayed on the PR.

    • STD-CI Jobs can also be triggered by adding a comment on the PR (its the same jobs as opposed to with Gerrit where its a special '*-on-demad' job)

    • Nothing triggers when a PR is merged (As opposed to with Gerrit where 'check-merged' and 'build-artifacts' are triggered on merge).

  • The white-listing scheme is different:

    • The 'jenkins-whiltelist' file does not effect theses jobs at all.

    • GitHub project/organisation membership determines white-listing

    • white-listing is determined by who added the comment to run CI, not who authored the patch.

  • GitHub projects are not currently supported by the change-queue.

Some of these issues and differences had already been tracked in other tickets. This ticket is for concentrating all related work. Other related tickets will be linked to it as blockers.

Activity

Show:
Barak Korren
July 23, 2017, 8:25 AM

How does this work for Lago:

Lago actually has a kind of 'patch gating' scheme. It is achieved by having a few other jobs besides the STD-CI jobs:

  • 'check-patch' is triggered by commenting 'ci test please' on the PR.

  • The trigger configuration on the 'check-merged' and 'build-artifacts' is actually just for manual checking. It runs them when adding a comment with 'ci build please' on the PR.

  • Merging is actually supposed to be done by the CI system:

    1. The process is triggered by commenting 'ci merge please' on the PR.

    2. This comment triggers the '*-trigger' job which triggers the '*-pipeline' job.

    3. The '*-pipeline' job runs the following jobs, each subsequent job is only run if the previous jobs was successful:

      1. The 'check-merged' job

      2. The 'buiild-artifacts' job

      3. The 'deploy-to-*-snapshot' job

    4. If the pipeline is successful, the '*-trigger' job runs a post -build step that merges the Gerrit patch.

There are a few issues with the Lago model:

  • For the merging scheme to test and merge things in the right order, the '*-trigger' job is configured to never run in parallel. This means that this model will not scale to faster moving projects.

  • The current model probably has a race condition where if can be coerced into merging the wrong code by modifying the PR while the pipeline is running.

  • Setting things up for this model requires intimate knowledge of multiple types of jobs. This is far more then we want STD-CI users to need.

Barak Korren
October 19, 2017, 2:39 PM

We now have fully functional, pipeline-base STDCI flow for GitHub projects.

Assignee

Barak Korren

Reporter

Barak Korren

Blocked By

None

Components

Priority

High
Configure