Missing Files on PHX Mirrors

Looks like a cleanup issue again. Files present in metadata should not simply disappear since we generate it after downloading files. Looking into this.

Found in vdsm CI and probably related:

Here’s the error:

The file is not present on-disk:

# ls -l /var/www/html/repos/yum/fedora-updates-fc30/base/Packages/l/ | grep libxcrypt
-rw-rw-r--. 1 jenkins jenkins 132892 Feb 26 13:21 libxcrypt-4.4.15-1.fc30.i686.rpm
-rw-rw-r--. 1 jenkins jenkins 125292 Feb 26 13:21 libxcrypt-4.4.15-1.fc30.x86_64.rpm

At the same time, it’s being downloaded during the sync job:

https://jenkins.ovirt.org/job/system-sync_mirrors-fedora-updates-fc30-x86_64/1022/consoleFull#L103

The only way this can happen is that it’s removed after downloading and running createrepo, yet before the snapshot is created.

Sent a patch to fix this race condition: https://gerrit.ovirt.org/108375

Thanks for the update and the patch!

Resync complete so we should be back on track, I see that the new libxcrypt versions are not deleted any more so the rest of the mirror should be fine, too:

# ls -l /var/www/html/repos/yum/fedora-updates-fc30/base/Packages/l/ | grep libxcrypt
-rw-rw-r--. 1 jenkins jenkins 132892 Feb 26 13:21 libxcrypt-4.4.15-1.fc30.i686.rpm
-rw-rw-r--. 1 jenkins jenkins 125292 Feb 26 13:21 libxcrypt-4.4.15-1.fc30.x86_64.rpm
-rw-rw-r--. 1 jenkins jenkins 133072 Apr 4 14:31 libxcrypt-4.4.16-1.fc30.i686.rpm
-rw-rw-r--. 1 jenkins jenkins 125504 Apr 4 14:31 libxcrypt-4.4.16-1.fc30.x86_64.rpm

Will trigger a re-sync for the rest of Fedora mirrors since they seem to have updates pending as well and we better maintain the base repo on par with the secondary ones.