[VDSM] Tests fail because of network timeout

Description

First time I see this error. The issue seems to be network read timeout
when creating virtual env
in tox test.

If you see this error in your builds, please report.

storage-py27 create:
/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/storage-py27
ERROR: invocation failed (exit code 1), logfile:
/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/storage-py27/log/storage-py27-0.log
ERROR: actionid: storage-py27
msg: getenv
cmdargs: ['/usr/bin/python2', '-m', 'virtualenv',
'--system-site-packages', '--python', '/usr/bin/python2.7',
'storage-py27']

New python executable in
/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/storage-py27/bin/python2.7
Also creating executable in
/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/storage-py27/bin/python
Installing setuptools, pip, wheel...

Complete output from command
/home/jenkins/worksp...e-py27/bin/python2.7 - setuptools pip wheel:
The directory
'/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.cache/pip/http'
or its parent directory is not owned by the current user and the cache
has been disabled. Please check the permissions and owner of that
directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.cache/pip'
or its parent directory is not owned by the current user and caching
wheels has been disabled. check the permissions and owner of that
directory. If executing pip with sudo, you may want sudo's -H flag.
Looking in links: /usr/lib/python2.7/site-packages,
/usr/lib/python2.7/site-packages/virtualenv_support
Collecting setuptools
Downloading https://files.pythonhosted.org/packages/37/06/754589caf971b0d2d48f151c2586f62902d93dc908e2fd9b9b9f6aa3c9dd/setuptools-40.6.3-py2.py3-none-any.whl
(573kB)
Exception:
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-18.1-py2.py3-none-any.whl/pip/_internal/cli/base_command.py",
line 143, in main
status = self.run(options, args)
File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-18.1-py2.py3-none-any.whl/pip/_internal/commands/install.py",
line 318, in run
resolver.resolve(requirement_set)
File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-18.1-py2.py3-none-any.whl/pip/_internal/resolve.py",
line 102, in resolve
self._resolve_one(requirement_set, req)
File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-18.1-py2.py3-none-any.whl/pip/_internal/resolve.py",
line 256, in _resolve_one
abstract_dist = self._get_abstract_dist_for(req_to_install)
File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-18.1-py2.py3-none-any.whl/pip/_internal/resolve.py",
line 209, in _get_abstract_dist_for
self.require_hashes
File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-18.1-py2.py3-none-any.whl/pip/_internal/operations/prepare.py",
line 283, in prepare_linked_requirement
progress_bar=self.progress_bar
File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-18.1-py2.py3-none-any.whl/pip/_internal/download.py",
line 836, in unpack_url
progress_bar=progress_bar
File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-18.1-py2.py3-none-any.whl/pip/_internal/download.py",
line 673, in unpack_http_url
progress_bar)
File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-18.1-py2.py3-none-any.whl/pip/_internal/download.py",
line 897, in _download_http_url
_download_url(resp, link, content_file, hashes, progress_bar)
File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-18.1-py2.py3-none-any.whl/pip/_internal/download.py",
line 617, in _download_url
hashes.check_against_chunks(downloaded_chunks)
File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-18.1-py2.py3-none-any.whl/pip/_internal/utils/hashes.py",
line 48, in check_against_chunks
for chunk in chunks:
File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-18.1-py2.py3-none-any.whl/pip/_internal/download.py",
line 585, in written_chunks
for chunk in chunks:
File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-18.1-py2.py3-none-any.whl/pip/_internal/utils/ui.py",
line 159, in iter
for x in it:
File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-18.1-py2.py3-none-any.whl/pip/_internal/download.py",
line 574, in resp_read
decode_content=False):
File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-18.1-py2.py3-none-any.whl/pip/_vendor/urllib3/response.py",
line 465, in stream
data = self.read(amt=amt, decode_content=decode_content)
File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-18.1-py2.py3-none-any.whl/pip/_vendor/urllib3/response.py",
line 430, in read
raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
File "/usr/lib64/python2.7/contextlib.py", line 35, in _exit_
self.gen.throw(type, value, traceback)
File "/usr/lib/python2.7/site-packages/virtualenv_support/pip-18.1-py2.py3-none-any.whl/pip/_vendor/urllib3/response.py",
line 345, in _error_catcher
raise ReadTimeoutError(self._pool, None, 'Read timed out.')
ReadTimeoutError: HTTPSConnectionPool(host='files.pythonhosted.org',
port=443): Read timed out.
----------------------------------------
...Installing setuptools, pip, wheel...done.
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/virtualenv.py", line 2462, in <module>
main()
File "/usr/lib/python2.7/site-packages/virtualenv.py", line 762, in main
symlink=options.symlink,
File "/usr/lib/python2.7/site-packages/virtualenv.py", line 1015, in
create_environment
install_wheel(to_install, py_executable, search_dirs, download=download)
File "/usr/lib/python2.7/site-packages/virtualenv.py", line 968, in
install_wheel
call_subprocess(cmd, show_stdout=False, extra_env=env, stdin=SCRIPT)
File "/usr/lib/python2.7/site-packages/virtualenv.py", line 854, in
call_subprocess
raise OSError("Command {} failed with error code
{}".format(cmd_desc, proc.returncode))
OSError: Command /home/jenkins/worksp...e-py27/bin/python2.7 -
setuptools pip wheel failed with error code 2
Running virtualenv with interpreter /usr/bin/python2.7

ERROR: Error creating virtualenv. Note that some special characters
(e.g. ':' and unicode symbols) in paths are not supported by
virtualenv. Error details: InvocationError('/usr/bin/python2 -m
virtualenv --system-site-packages --python /usr/bin/python2.7
storage-py27 (see
/home/jenkins/workspace/vdsm_standard-check-patch/vdsm/.tox/storage-py27/log/storage-py27-0.log)',
1)

Build:
https://jenkins.ovirt.org/blue/rest/organizations/jenkins/pipelines/vdsm_standard-check-patch/runs/1131/nodes/127/steps/407/log/?start=0

Assignee

infra

Reporter

Nir Soffer

Blocked By

None

Priority

Medium
Configure