Failed to clean docker images

Description

CQ was failed on cleanup due to some containers being left over. Looks like it's some bug that can be fixed in cleanup code:

...
Deleted: sha256:01f26bebce703d4cdf051be97f3f4e7112efb86715d3f14e4853e104bfa4951d
Deleted: sha256:64c5d1920a0ec20e595383bc81eae301ff7c36974b1d205e58302ce9f25dbc75
Error response from daemon: conflict: unable to delete 2454d64ffcd3 (cannot be forced) - image has dependent child images
Error response from daemon: conflict: unable to delete bcec0ae8107e (cannot be forced) - image has dependent child images
ERROR: Failed to clean docker images

Sample job: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6471/consoleFull

Activity

Show:

Former user March 22, 2018 at 6:39 PM

Barak and I were working on a proper solution today using the Docker SDK. It's basically ready but I still need to refine it and test for edge cases.
I'd suggest that for this weekend, we'll send a patch to not fail on docker cleanup since it actually does the job of removing the images, but it fails when the order it deletes them is incorrect.



Any thoughts?
I can prepare such patch quickly.

Former user March 22, 2018 at 2:09 PM

A workaround currently is to go to the problematic slave and run "docker images" to see if any images are left over and delete them using "docker rmi"

However, throughout the several failures there were no such images - probably they are deleted during further global_cleanup runs or service restarts

Dafna Ron March 21, 2018 at 10:30 PM

Former user March 21, 2018 at 5:01 PM

is there an option to somehow sort the images to delete dependent ones first to avoid this from happening?

Fixed

Details

Assignee

Reporter

Priority

Created March 20, 2018 at 1:51 PM
Updated April 1, 2018 at 12:31 PM
Resolved March 27, 2018 at 12:42 PM