[PLEASE NOTE] copr outage

Description

We seem to be having some kind of an outage in the oVirt PHX data
center that is preventing our hosts in PHX from reaching copr.

This is causing various jobs to fail including all jobs that run OST.

Please avoid running jobs that are known to require connectivity to
copr until further notice.


Barak Korren
RHV DevOps team , RHCE, RHCi
Red Hat EMEA
redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted

Activity

Show:
Evgheni Dereveanchin
September 6, 2017, 1:19 PM

We use Google Public DNS in PHX and fedoraproject.org does not resolve via these servers any more:

jenkins ~]$ dig fedoraproject.org

; <<>> DiG 9.9.4-RedHat-9.9.4-50.el7_3.1 <<>> fedoraproject.org
;; global options: +cmd
;; Got answer:
;; >>HEADER<< opcode: QUERY, status: SERVFAIL, id: 34972
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;fedoraproject.org. IN A

;; Query time: 37 msec
;; SERVER: 8.8.4.4#53(8.8.4.4)
;; WHEN: Wed Sep 06 13:18:16 UTC 2017
;; MSG SIZE rcvd: 46

This SRVFAIL may be related to DNSSEC settings on the domain, for example: http://dnsviz.net/d/fedoraproject.org/dnssec/

Evgheni Dereveanchin
September 6, 2017, 1:21 PM

Tried to escalate this to Fedora's issue tracker on Pagure but it fails to authenticate me with a discovery failure (which may also be related) https://pagure.io/fedora-infrastructure/issues

Evgheni Dereveanchin
September 6, 2017, 1:28 PM

Confirmed with Fedora admins that this is a DNSSEC issue induced by upgrading to a new key. Waiting for this to get fixed.

Evgheni Dereveanchin
September 7, 2017, 8:22 AM

DNSSEC fixed upstream, everything should be back to normal. Closing

Assignee

Evgheni Dereveanchin

Reporter

Barak Korren

Blocked By

None

Priority

Medium
Configure