reposync fails syncing repos on completely unrelated paths (concurrency issue?)

Description

About job
https://jenkins.ovirt.org/job/system-sync_mirrors-centos-kvm-common-el7-x86_64/2934/console

As you can see below

05:35:07 ++ reposync --config=jenkins/data/mirrors-reposync.conf
--repoid=centos-kvm-common-el7 --arch=x86_64
--cachedir=/home/jenkins/mirrors_cache
--download_path=/var/www/html/repos/yum/centos-kvm-common-el7/base
--norepopath --newest-only --urls --quiet

the sync is related to "/var/www/html/repos/yum/centos-kvm-common-el7/base"
using as cache directory "/home/jenkins/mirrors_cache"
but in "/home/jenkins/mirrors_cache" there's
"/home/jenkins/mirrors_cache/fedora-base-fc29":

05:35:16 Traceback (most recent call last):
05:35:16 File "/usr/bin/reposync", line 373, in <module>
05:35:16 main()
05:35:16 File "/usr/bin/reposync", line 185, in main
05:35:16 my.doRepoSetup()
05:35:16 File "/usr/lib/python2.7/site-packages/yum/_init_.py", line
681, in doRepoSetup
05:35:16 return self._getRepos(thisrepo, True)
05:35:16 File "/usr/lib/python2.7/site-packages/yum/_init_.py", line
721, in _getRepos
05:35:16 self._repos.doSetup(thisrepo)
05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 157,
in doSetup
05:35:16 self.retrieveAllMD()
05:35:16 File "/usr/lib/python2.7/site-packages/yum/repos.py", line 88,
in retrieveAllMD
05:35:16 dl = repo._async and repo._commonLoadRepoXML(repo)
05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
1468, in _commonLoadRepoXML
05:35:16 local = self.cachedir + '/repomd.xml'
05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
777, in <lambda>
05:35:16 cachedir = property(lambda self: self._dirGetAttr('cachedir'))
05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
760, in _dirGetAttr
05:35:16 self.dirSetup()
05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
738, in dirSetup
05:35:16 self._dirSetupMkdir_p(dir)
05:35:16 File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line
715, in _dirSetupMkdir_p
05:35:16 raise Errors.RepoError, msg
05:35:16 yum.Errors.RepoError: Error making cache directory:
/home/jenkins/mirrors_cache/fedora-base-fc29 error was: [Errno 17] File
exists: '/home/jenkins/mirrors_cache/fedora-base-fc29'

this looks like a concurrency issue, a lock should be used in order to
prevent two instances to use the same cache directory at the same time or
use separate cache directories for different repos.

Sandro Bonazzola

Assignee

Shlomi Zidmi

Reporter

Sandro Bonazzola

Blocked By

None

Components

Priority

Medium
Configure