Re: [ovirt-users] Public Glance repository for oVirt
Description
Activity
Former user October 13, 2021 at 12:15 PM
Pre-reboot mem usage and processes according to top:
top - 05:49:55 up 63 days, 7:16, 1 user, load average: 0.02, 0.04, 0.09
Tasks: 123 total, 2 running, 121 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.5 us, 0.0 sy, 0.0 ni, 99.5 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 1782800 total, 165440 free, 1306576 used, 310784 buff/cache
KiB Swap: 0 total, 0 free, 0 used. 101944 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
7860 keystone 20 0 611020 81440 0 S 0.0 4.6 0:10.75 httpd
7861 keystone 20 0 328900 4416 0 S 0.0 0.2 0:07.00 httpd
7862 keystone 20 0 328900 4416 0 S 0.0 0.2 0:06.95 httpd
7863 keystone 20 0 394436 4416 0 S 0.0 0.2 0:06.98 httpd
7864 keystone 20 0 328900 4416 0 S 0.0 0.2 0:06.82 httpd
7865 keystone 20 0 328900 4416 0 S 0.0 0.2 0:06.76 httpd
7866 apache 20 0 238860 4480 12 S 0.0 0.3 0:00.07 httpd
7867 apache 20 0 238860 4480 12 S 0.0 0.3 0:00.07 httpd
7868 apache 20 0 238860 4480 12 S 0.0 0.3 0:00.06 httpd
7869 apache 20 0 238860 4480 12 S 0.0 0.3 0:00.04 httpd
7870 apache 20 0 238860 4480 12 S 0.0 0.3 0:00.05 httpd
7960 apache 20 0 238860 4480 12 S 0.0 0.3 0:00.59 httpd
10361 root 20 0 0 0 0 S 0.0 0.0 0:00.11 kworker/u8:2
10845 root 20 0 0 0 0 S 0.0 0.0 0:01.02 kworker/u8:1
11508 apache 20 0 238860 4480 12 S 0.0 0.3 0:00.04 httpd
11593 apache 20 0 238860 4480 12 S 0.0 0.3 0:00.04 httpd
11795 glance 20 0 431172 107928 1552 S 0.0 6.1 0:24.25 glance-api
14914 root 20 0 0 0 0 R 0.0 0.0 0:01.96 kworker/0:0
18651 root 20 0 0 0 0 S 0.0 0.0 0:00.19 kworker/1:0
20789 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/0:2
20962 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/1:2
21174 postfix 20 0 90508 4192 3100 S 0.0 0.2 0:00.01 pickup
21202 root 20 0 156936 5840 4408 S 0.0 0.3 0:00.40 sshd
21205 root 20 0 115644 2196 1668 S 0.0 0.1 0:00.04 bash
21305 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/1:1
21364 root 20 0 112940 4284 3260 S 0.0 0.2 0:00.00 sshd
21366 root 20 0 162104 2324 1600 R 0.0 0.1 0:00.02 top
Post-reboot equivalents:
top - 08:14:15 up 2:23, 1 user, load average: 0.01, 0.03, 0.05
Tasks: 119 total, 2 running, 117 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.7 us, 0.3 sy, 0.0 ni, 99.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 1782800 total, 524928 free, 962676 used, 295196 buff/cache
KiB Swap: 0 total, 0 free, 0 used. 592456 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
999 glance 20 0 421700 104872 7872 R 1.7 5.9 1:45.13 glance-api
682 root 20 0 474276 12732 6644 S 1.0 0.7 0:56.94 NetworkManager
9 root 20 0 0 0 0 S 0.3 0.0 0:11.49 rcu_sched
1 root 20 0 128088 6748 4216 S 0.0 0.4 0:02.23 systemd
2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd
3 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/0:0
4 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:0H
5 root 20 0 0 0 0 S 0.0 0.0 0:00.10 kworker/u8:0
6 root 20 0 0 0 0 S 0.0 0.0 0:00.49 ksoftirqd/0
7 root rt 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
8 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_bh
10 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 lru-add-drain
11 root rt 0 0 0 0 S 0.0 0.0 0:00.03 watchdog/0
12 root rt 0 0 0 0 S 0.0 0.0 0:00.03 watchdog/1
13 root rt 0 0 0 0 S 0.0 0.0 0:00.00 migration/1
14 root 20 0 0 0 0 S 0.0 0.0 0:00.01 ksoftirqd/1
16 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/1:0H
18 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kdevtmpfs
19 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 netns
20 root 20 0 0 0 0 S 0.0 0.0 0:00.00 khungtaskd
21 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 writeback
22 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kintegrityd
23 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 bioset
24 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 bioset
25 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 bioset
26 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kblockd
27 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 md
28 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 edac-poller
29 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 watchdogd
36 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kswapd0
37 root 25 5 0 0 0 S 0.0 0.0 0:00.00 ksmd
38 root 39 19 0 0 0 S 0.0 0.0 0:00.07 khugepaged
39 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 crypto
47 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kthrotld
Former user October 13, 2021 at 9:52 AM
The service is back up for now, will review if we need to grow RAM on this host or solve this in some other way as it’s worked for several years up to this point with no issues.
Former user October 13, 2021 at 9:50 AMEdited
Looks like the box is running out of memory:
[5380587.626351] Out of memory: Kill process 11388 (mysqld) score 62 or sacrifice child
[5380587.627302] Killed process 11388 (mysqld), UID 27, total-vm:1725896kB, anon-rss:110984kB, file-rss:0kB, shmem-rss:0kB
I’ve rebooted the system to ensure it is back to clean state.
Former user October 13, 2021 at 9:47 AM
Looks like mysql is down on the glance server.
Checking why this is happening to restore the service.
On Tue, Oct 12, 2021 at 6:45 PM <p.staniforth@leedsbeckett.ac.uk> wrote:
>
> Hello,
> can anyone get to the public openstack image repository ovirt-image-repository?
>
> Also when I try browse the site I get
>
> This site can’t be reached
> https://glance.ovirt.org/ is unreachable.
Thanks for the report.
Adding infra/infra-support. Any clue? This is affecting also CI,
breaking at least
test_undo_preview_snapshot_when_mac_used_reassigns_a_new_mac in
ovirt-system-tests.
Thanks and best regards,
–
Didi