Base experimantal flow on transactional repos

Description

We recently seen various issues with the way the experimental repos are managed:

  • The latest.tested repos is close to unusable because it changes very rapidly and does not keep history around

  • Its impossible to have more then one experimental flow running in parallel because there can be only one latest.under_test repo

  • We've seem various race conditions happening with the experimental repos because they are manipulated both by the 'deploy-to-experimental' jobs and by the 'test-repo' jobs.

  • The experimental repo is manipulated by various secret and undocumented scripts in resources.ovirt.org. This makes things hard to debug.

  • It is close to impossible to reproduce experimental runs because the exact repo used for those runs acn get destroyed as soon as the run is done

We've recently created the technology to maintain transactional mirrors (see OVIRT-575). It is desirable to use the same technique for managing the experimental repos.

Here is a breakdown of the steps to accomplish this task:

  1. The mirrors management script needs YUM repos to sync from. We can actually make each build_artifacts jobs generate its own mini repo by running 'createrepo' in 'exported-artifacts' (OVIRT-1043).

  2. We need to make the 'deploy-to-experimental' job run the mirror management script to sync built artifacts into the relevant repo and create a snapshot (OVIRT-1044).

  3. Once we have the above, we can treat the experimental repo just like any other repo in the OST reposync file. The same work that will be done to inject mirrors into the experimental flow can also inject the experimental repo snapshots (OVIRT-1042).

  4. If we still want the experimental 'latest.tested' to be published on resources.ovirt.org, we can just 'reposync' it from the mirrors server once the OST run is successful. That published repo can include package and metadata history is it can be used by external consumers even if it changes frequently (OVIRT-1045).

Activity

Show:
Barak Korren
May 30, 2017, 9:09 AM

The experimental flow will be replaced by the change-queue flow (). That flow just utilizes the fact that build-artifacts jobs each create their own repos instead of trying to collect all packages in one big repo.

Barak Korren
March 9, 2017, 1:52 PM

Yes. This has nothing to do with the 'tested' repo. This is about change gating. I wrote this when I was thinking about making change gating do everything with the mirrors. I'm not sure I'll end up doing that, but I'll know for sure when I start moving real changes through the gating jobs.

Eyal Edri
March 9, 2017, 1:44 PM

do we still need to update deploy-to-experimental job after we added the 'tested' repo?

Won't Fix
Your pinned fields
Click on the next to a field label to start pinning.

Assignee

Barak Korren

Reporter

Barak Korren