failures on ovirt-master on multiple projects due to package versioning
Description
Activity

Former user October 23, 2018 at 1:11 PM
This is a completely different error as it seems (a 404) that looks to be related to a stale cache of some sort.
I do not see any mention of snapshot 2018-06-11-20 in the latest mirror snapshot (which is what CI jobs should use). On the contrary, http://mirrors.phx.ovirt.org/repos/yum/epel-el7/2018-06-11-20-43/ seems to point to the "latest" snapshot. @Dafna Ron could you please confirm which job failed with the second issue?

Dafna Ron October 23, 2018 at 12:29 PM
The issue happens on different servers.
I don't think its the servers cache but if so we need to delete the package on all servers.
Also, same jobs show problems accessing the mirrors:
22:21:56 [basic-suite.el7.x86_64] http://mirrors.phx.ovirt.org/repos/yum/epel-el7/2018-06-11-20-43/repodata/05fff37ef16102c687b3613667f57919396858d0a6159e0a472facdd80440579-primary.sqlite.bz2: [Errno 14] HTTP Error 404 - Not Found

Former user October 23, 2018 at 11:24 AM
Looks like it's failing on multiple nodes and the related info message is:
10:07:55 [upgrade-from-release-suite.el7.x86_64] - repo: ovirt-4.2-tested-el7: failed, re-running.
10:07:55 [upgrade-from-release-suite.el7.x86_64] - removing conflicting RPM: /var/lib/lago/ovirt-4.2-tested-el7/noarch/ovirt-ansible-disaster-recovery-1.1.2-1.el7.noarch.rpm
Not sure what it means, is it a failure to remove a local RPM or fetch a new one? @Gal Ben Haim could you please confirm what's the course of action here? I've deleted /var/lib/lago/* on ovirt-srv19 and am rebooting it now, not sure if we should repeat this for all nodes as well.

Former user October 23, 2018 at 11:04 AM
Investigating to see if it's isolated to a single host as there are no signs of metadata corruption or reachability issues with resources.

Dafna Ron October 23, 2018 at 10:40 AM
I was able to wget and repoquery the package:
https://pastebin.com/d0ALuqGF

Dafna Ron October 23, 2018 at 10:31 AM

Dafna Ron October 23, 2018 at 8:38 AM

Dafna Ron October 23, 2018 at 8:34 AM
second failure for otopi:
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10898/
Details
Assignee
Dafna RonDafna Ron(Deactivated)Reporter
Dafna RonDafna Ron(Deactivated)Priority
Medium
Details
Details
Assignee

Reporter

21:38:01 [upgrade-from-release-suite.el7.x86_64] RuntimeError: Failed to run reposync 3 times for repoid: ovirt-4.2-tested-el7, aborting.
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/10896/
@Former user can you please take a look to see if we have issues with the server?