[FIRING:1] LowDiskSpace_NodeExporter_v1_0 templates.ovirt.org:9100 templates-server (/dev/mapper/templates-kubevirt xfs /var/www/html/kubevirt)

Description

Labels:

  • alertname = LowDiskSpace_NodeExporter_v1_0

  • device = /dev/mapper/templates-kubevirt

  • fstype = xfs

  • instance = templates.ovirt.org:9100

  • job = templates-server

  • mountpoint = /var/www/html/kubevirt

Annotations:

  • description = templates.ovirt.org:9100 of job templates-server has less than 10% free disk space

Source: http://prometheus-0:9090/graph?g0.expr=node_filesystem_free_bytes+%2F+node_filesystem_size_bytes%7Bdevice%21%3D%22tmpfs%22%7D+%3C+0.2&g0.tab=1

Activity

Show:
Shlomi Zidmi
October 1, 2020, 7:21 AM

The volume which stores all Kubevirt’s images on templates server is using 96GB out of 110GB possible.

By looking at the following from Kubevirt’s repo:

https://github.com/kubevirt/common-templates/blob/master/automation/test.sh#L110

https://github.com/kubevirt/common-templates/blob/master/automation/test.sh#L138

I can see the image “win01.img“ is not being used and is not listed with all other windows images. If that’s the case we can delete it and free up 25GB. could you please confirm this?

Evgheni Dereveanchin
October 1, 2020, 3:17 PM

As discussed on the infra mtg, these images are manually uploaded so low free space is not an issue. Instead of cleaning it up let’s just extend for now. There should be enough space on the SD so expanding one of the PVs should do the trick.

Shlomi Zidmi
November 16, 2020, 10:02 PM

I've resized the kubevirt directory using some space from other volumes, so far without extending the PVs.
If a similar alert will pop up again, I'll extend the disks.

Assignee

Shlomi Zidmi

Reporter

Alertmanager_Bot

Blocked By

None

Priority

Medium
Configure