Make experimental repos retain versions (or serve static versions of the repo)
Description
duplicates
Activity

Eyal Edri April 5, 2016 at 2:19 PMEdited
The idea is to keep history, and we should have it configurable on how much we want to keep back (ideally i would keep everything at least few months back, if we have the space),
unlike nighties which are rotated frequently, we expect these repos will be used to verify ovirt bugs over a period of a few weeks/month so we should allow some level of history.
In time we'll do adjustments and we'll optimize the retention policy to the best usage of dev+qa consumption.
Yea, we need to keep an update link to latest repo + link to latest release rpm.

Former user April 5, 2016 at 2:12 PM
> Issue is that as the build flow is right now once the repo is created (ovirt-4.0_alpha1) a new ovirt-release rpm has to be created pointing to it (ovirt-release-4.0.0-0.2.alpha1) and added to the repo. Such repo will be referenced within the appliance, node image and live image so it should be something like a "final" repo, not a staging one.
One of the points of the task is:
Question is if that repo that is created weekly, is just the same each time (removed and regenerated) or should it keep a certain amount of them and rolling them out?
The release rpm can be created along with the repo itself in case you need it for each, or we can keep a 'latest' link to the newer repo and point the release rpm there, or anything in between.

Sandro Bonazzola April 5, 2016 at 9:11 AM
> to know what will be the directory structure of the repositories, right now there's some ovirt-4.0_prealpha1/2 and prealpha_latest. It would be helpful if the names maintained the version order when alphabetically sorted, if not, things get a bit more weird.
we're starting with alpha1 today so from now on, it will be alphabetically sorted being alpha < beta < rc
> The created snapshot repo sholud be just 'ovirt_4.0.0_snap1' or should it create one each week? (like incremental numbers or something)
If the above, then how many to keep?
Issue is that as the build flow is right now once the repo is created (ovirt-4.0_alpha1) a new ovirt-release rpm has to be created pointing to it (ovirt-release-4.0.0-0.2.alpha1) and added to the repo. Such repo will be referenced within the appliance, node image and live image so it should be something like a "final" repo, not a staging one.
> the repos allowed when running the repo closure (or a script or similar).
all allowed repos are in ovirt-release40 rpm

Former user March 29, 2016 at 3:00 PM
Also, why are you collecting artifacts in the 'partial' flow patch?

Former user March 29, 2016 at 10:05 AM
I need a couple things (asking you as release manager):
to know what will be the directory structure of the repositories, right now there's some ovirt-4.0_prealpha1/2 and prealpha_latest. It would be helpful if the names maintained the version order when alphabetically sorted, if not, things get a bit more weird.
The created snapshot repo sholud be just 'ovirt_4.0.0_snap1' or should it create one each week? (like incremental numbers or something)
If the above, then how many to keep?
the repos allowed when running the repo closure (or a script or similar).

Eyal Edri March 15, 2016 at 7:20 AMEdited
Last update on needed steps:
Current status:
"Stage 1" (first snapshot repo without images)
a. oVirt nightlies are being published as usual
b. A snapshot was created manually using repoman command, as described in [1]
c ** missing **
d. new release rpm for 4.0 was created manually [2]
e ** missing **
f. new softlink was created for latest pre-alpha releases [3]
"Stage 2" (final repo with images)a. new jobs [4] building node + appliance created using the rpm from [3]
Missing steps:
"Stage 1" (first snapshot repo without images)
new jenkins job that will run the whole flow starting from 1b, work has started here [5] (@sandro, please correct if i'm wrong)
new step for checking repoclosure needs to be added as step 'c', code can be taken from [6].
automate the step of building the release rpm pointing to a specific release dir.
create new release rpm that will always point to latest snapshot link [3]
add sanity for first repo using lago, similar to [7], just using the new release rpm to run on the new snapshot (step 1e)
"Stage 2" (repo including images)
run the job(s) to build appliance, node & ovirt-live using the rpm from [2]. (job for ovirt-live might needs fixing to use release rpm)
push the new images to the new repo, using TBD job
run repoclosure to make sure images are install-able, similar to [6]
run lago sanity tests on whatever is possible (node/appliance/he)
update softlink [3] to point to new repo with images
make repo readonly
move bugs to ON_QA [8]
send email on release with links + changelog
[1] https://ovirt-jira.atlassian.net/browse/OVIRT-422
[2] http://resources.ovirt.org/pub/ovirt-4.0_prealpha2/rpm/el7/noarch/ovirt-release40.rpm
[3] http://resources.ovirt.org/pub/ovirt-4.0_prealpha-latest/
[4] - can you share the links to the jobs?
[5] https://gerrit.ovirt.org/#/c/54626/
[6] http://jenkins.ovirt.org/job/repos_master_check-closure_merged/
[7] http://jenkins.ovirt.org/job/ovirt_master_system-tests/
[8] as first step we'll use the logic of extracting bugs from the commits msg for a list of projects (do we need approval from each maintainer?)

Eyal Edri March 14, 2016 at 11:59 AM
I believe you made some progress for this flow, including adjusting the rhevh and appliance jobs plus adding links + release rpm.
Can you add some info & links so we'll be able to plan the remaining items to finish the automation of the flow?

Eyal Edri February 23, 2016 at 7:49 AM

Eyal Edri February 22, 2016 at 4:04 PM
do you know if anyone from the infra team can handle it?
Details
Assignee
bkorrenbkorrenReporter
Eyal EdriEyal EdriComponents
Priority
High
Details
Details
Assignee

Reporter

We need a more static 4.0 snapshots that will be able to be consumed by people who want to verify a certain build without having a rolling nightly build.
Add a jjb jenkins job that will run manually/once a week and will create a snap 4.0 repo on
resources.ovirt.org from latest nightlies (will need to create the repo on resources.ovirt.org/pub/ovirt-4.0/
Use repoman with the following cmd: "/usr/bin/repoman ovirt_4.0.0_snap1 add conf:ovirt-4.0-snapshots.repoman --keep-latest 1" [1]
This should collect all latest nightlies for 4.0 into a build dir called ovirt_4.0.0_snap1
We'll run repoclosure on it once the copy is finished
run sanity (either manually or via Lago manual if possible)
rhevh team run sanity on the node builds from nightly
Create latest_4.0.0_snapshot link to the repo and update it on each release + create new release rpm with that link.
Move bugs - either move all MODIFIED bugs with target milestone ovirt-4.0.0 or move only bugs with bug-url from the commit msg.
Send email to users/devel with info on release (list of bugs + link to release rpm).
[2] cat ovirt-4.0-snapshots.repoman (the conf file)
rec:http://plain.resources.ovirt.org/pub/ovirt-master-snapshot:latest
rec:http://plain.resources.ovirt.org/pub/ovirt-master-snapshot-static:latest