Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

network timeouts in CI

Description

Hi,

We are seeing network timeouts in CI today.

Example:
https://jenkins.ovirt.org/blue/organizations/jenkins/oVirt_ovirt-web-ui_standard-check-pr/detail/oVirt_ovirt-web-ui_standard-check-pr/405/pipeline/84/

Total download size: 241 M
Installed size: 340 M
Downloading Packages:

[MIRROR] ovirt-engine-nodejs-8.11.4-1.fc28.x86_64.rpm: Curl error (28):
Timeout was reached for
http://jenkins.ovirt.org/job/ovirt-engine-nodejs_master_build-artifacts-fc28-x86_64/lastSuccessfulBuild/artifact/exported-artifacts/x86_64/ovirt-engine-nodejs-8.11.4-1.fc28.x86_64.rpm
[Operation too slow. Less than 1000 bytes/sec transferred the last 30
seconds]

[MIRROR] ovirt-engine-nodejs-modules-1.8.6-1.fc28.noarch.rpm: Curl error
(28): Timeout was reached for
http://jenkins.ovirt.org/job/ovirt-engine-nodejs-modules_master_build-artifacts-fc28-x86_64/lastSuccessfulBuild/artifact/exported-artifacts/noarch/ovirt-engine-nodejs-modules-1.8.6-1.fc28.noarch.rpm
[Operation too slow. Less than 1000 bytes/sec transferred the last 30
seconds]

[MIRROR] ovirt-engine-yarn-1.7.0-1.fc28.noarch.rpm: Curl error (28):
Timeout was reached for
http://jenkins.ovirt.org/job/ovirt-engine-yarn_master_build-artifacts-fc28-x86_64/lastSuccessfulBuild/artifact/exported-artifacts/noarch/ovirt-engine-yarn-1.7.0-1.fc28.noarch.rpm
[Operation too slow. Less than 1000 bytes/sec transferred the last 30
seconds]

[FAILED] ovirt-engine-nodejs-8.11.4-1.fc28.x86_64.rpm: No more mirrors to
try - All mirrors were already tried without success

Error: Error downloading packages:

–

GREG SHEREMETA

SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX

Red Hat NA

<https://www.redhat.com/>

gshereme@redhat.com IRC: gshereme
<https://red.ht/sig>

Activity

Show:

Former user February 11, 2019 at 12:19 PM

the Jenkins was restarted, please let me know if you still see any issues like this.

Former user February 8, 2019 at 4:29 PM

Sorry for the inconvenience. We are approaching this from several directions: first of all we're working on splitting one huge Jenkins instance into several smaller ones. We're also waiting for a fix to the plugin - Daniel even sent a PR for that. Another improvement would be to serve artifacts independently of the Jenkins itself (currently all the requests go through the JRE and if the app is slow, downloads are slow).

Greg Sheremeta February 8, 2019 at 2:31 PM

Thank you!

Is there a permanent fix?

We just added ovirt CI to the ovirt-web-ui project, and between this and the disks filling up – basically ovirt ci immediately caused us problems upon adopting it. This is very frustrating for the developers, and it doesn't look very good, and we all end up losing confidence in the infrastructure as a result

Former user February 8, 2019 at 1:41 PM

This is likely related to Jenkins slowness caused by a plugin mem leak. I'll try to get it under control - a restart may be needed to free up memory

Cannot Reproduce

Details

Assignee

Reporter

Priority

Created February 7, 2019 at 8:27 PM
Updated August 29, 2019 at 2:12 PM
Resolved July 22, 2019 at 1:59 PM