Cannot deactivate Storage while there are running tasks on this Storage

Description

we failed test: 007_sd_reattach.deactivate_storage_domain in vdsm project with error: Cannot deactivate Storage while there are running tasks on this Storage. -Please wait until tasks will finish and try again.]". HTTP response code is 409.

I am opening this as a race in case something changed in OST tests which would cause this to repeat.

https://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/2506
https://gerrit.ovirt.org/#/c/92313/2

Activity

Show:

Dafna Ron July 13, 2018 at 12:47 PM

We had another failure today due to this issue.

I created a patch to skip this test and the related tests until this is fixed:
https://gerrit.ovirt.org/#/c/93006/

can you please merge until storage team has a chance to fix the tests?

Eyal Edri July 11, 2018 at 10:24 AM

any update?

Tal Nisan July 10, 2018 at 1:22 PM

you are the current QE contact, can you please have a look?

Eyal Edri July 9, 2018 at 2:44 PM

it seems this test is failing more often than we've seen in the past.
Who from the storage team can assist here to debug this issue and possibly send a fix to the test?

Dafna Ron July 9, 2018 at 8:56 AM

This is reported already to the storage team.
I actually found that the problematic task is copy of template which is taking longer then expected.
We had a discussion about fixing the tests so that tests will be able to re-try and fail with timeout in case of running tasks since this is not something we can solve in code and is failing randomly a lot.

Gal Ben Haim July 8, 2018 at 2:24 PM

Looks like a race.
Probably, there are still running tasks from previous tests that use the storage domain.

Can you please forward it to the storage team?

Dafna Ron July 6, 2018 at 3:56 PM

I think this failure happens because of delay in the network which is causing the storage image copy to be slower -> causing a failure on deactivate of storage domain.
two ways of solving:
1. fix network issues
2. add re-tries on tests


can you please have a look and see if you can confirm/suggest a different reason/solution

Cannot Reproduce

Details

Assignee

Reporter

Priority

Created June 28, 2018 at 8:37 AM
Updated August 29, 2019 at 2:12 PM
Resolved December 15, 2018 at 4:01 PM