We recently seen various issues with the way the experimental repos are managed:

• The latest.tested repos is close to unusable because it changes very rapidly and does not keep history around

• Its impossible to have more then one experimental flow running in parallel because there can be only one latest.under_test repo

• We've seem various race conditions happening with the experimental repos because they are manipulated both by the 'deploy-to-experimental' jobs and by the 'test-repo' jobs.

• The experimental repo is manipulated by various secret and undocumented scripts in resources.ovirt.org. This makes things hard to debug.

• It is close to impossible to reproduce experimental runs because the exact repo used for those runs acn get destroyed as soon as the run is done

We've recently created the technology to maintain transactional mirrors (see OVIRT-575). It is desirable to use the same technique for managing the experimental repos.

Here is a breakdown of the steps to accomplish this task:

1. The mirrors management script needs YUM repos to sync from. We can actually make each build_artifacts jobs generate its own mini repo by running 'createrepo' in 'exported-artifacts' (OVIRT-1043).

2. We need to make the 'deploy-to-experimental' job run the mirror management script to sync built artifacts into the relevant repo and create a snapshot (OVIRT-1044).

3. Once we have the above, we can treat the experimental repo just like any other repo in the OST reposync file. The same work that will be done to inject mirrors into the experimental flow can also inject the experimental repo snapshots (OVIRT-1042).

4. If we still want the experimental 'latest.tested' to be published on resources.ovirt.org, we can just 'reposync' it from the mirrors server once the OST run is successful. That published repo can include package and metadata history is it can be used by external consumers even if it changes frequently (OVIRT-1045).