mock_runner.sh seems to run slower on newer slaves

Description

After implementing speedup improvements for mock_runner.sh (Specifically enabling it to deploy the chroot by extracting a tarball as opposed to installing it with yum every time), I started monitoring run times for vdsm check_patch jobs.

I've noticed that jobs that are running on the old slaves (fc24-vm*) seem to initialize mock faster then ones running on new slaves (vm*) - 17seconds vs 57 seconds respectively.

I suspect this may be because the older slaves have a different FS mounted on /var/cache/mock

Activity

Show:

Former user December 19, 2016 at 1:20 PM

Closing as duplicate of OVIRT-931. Caching enabled, there should no longer be a difference between new and old workers.

Eyal Edri December 14, 2016 at 3:03 PM

I think its duplicate of the 10-50 min task, feel free to close duplicate if its the case.

Barak Korren December 6, 2016 at 1:48 PM

the problem with logs is that you need to get then "hot" be cause they may disappear fast...

Older slave log (fc24-vm15.phx.ovirt.org):
http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc24-x86_64/5915/

Newer slave log (vm0125.workers-phx.ovirt.org):
http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc24-x86_64/5916/

Then again, on (newer) vm0130.workers-phx.ovirt.org:
http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc24-x86_64/5916/

So not a 100% consistent result, still, we should find out how to get those 40 seconds back...

BTW I was sure the local disk was mounted on older slaves with noatime, looks like that is not the case, maybe we should fix that...

Former user December 6, 2016 at 1:23 PM

some logs would be helpful

Older slaves with local disk hook use ext4 on the secondary disk while newer ones write to rootfs which is XFS. I can try launching a dozen ext4 VMs so that we can compare performance.

Barak Korren December 6, 2016 at 1:16 PM

, any idea what we can do to improve this?

Duplicate

Details

Assignee

Reporter

Components

Priority

Created December 6, 2016 at 1:16 PM
Updated May 25, 2017 at 11:30 AM
Resolved December 19, 2016 at 1:20 PM