If OST was cheap and quick, it would be lovely to run on any patch. It can also be run after a +2 from a maintainer, or after a workflow+1, providing a verified +0.5 to a patch. I don't know if that's what you're aiming at.

OST is probably not going to ever be cheap and quick enough.
I'm aiming at leveraging the same group-testing+bisection tech we use for post-merge OST runs. It was initially designed for pre-merge testing (a.k.a. gating) anyway, we just decided to deliver something that is useful and works before implementing everything we set out to.

If OST was cheap and quick, it would be lovely to run on any patch. It can also be run after a +2 from a maintainer, or after a workflow+1, providing a verified +0.5 to a patch. I don't know if that's what you're aiming at.

I suppose this is not a joke, so I'll answer it seriously. Yes. Vdsm and Engine are too big and complex.

Are there low-hanging fruits there? Could we improve the per-patch checking infrastructure in one way or another to reduce the need to run full OST? Will separate checking "threads" that respond asynchronously help?

Regardless, commit/fail/revert/fix is a frustrating development model. post/test/fix/commit is so much better.

Perhaps we can make efforts to structure patches so that commit/fail/fix can be more common then commit/fail/revert/fix? Will one or both of the following things help?

1. Lower latency of CQ/OST results

2. Posting CQ/OST results back directly to Gerrit

> Could it be that the project contains too much independent stuff in it?

I suppose this is not a joke, so I'll answer it seriously. Yes. Vdsm and Engine are too big and complex.

Regardless, commit/fail/revert/fix is a frustrating development model. post/test/fix/commit is so much better.

BTW I wonder why you need to run OST manually so often. We have change gating, so when merging a breaking change, you're only breaking the particular project you merged a change into. Other, unrelated projects can keep being tested against the last known working version...

Could it be that the project contains too much independent stuff in it?

The process is clunky because the whole idea if having to launch stuff manually in CI is clunky.

We needed to provide a useful tool, so we brought some things together to make this possible. But I wouldn't want to spend efforts around automating a process which is IMO fundamentally broken.

Ideally you would just get OST results for every patch before merging it, but this requires several things:

1. Asynchronous result notification to Gerrit, so you can see check-patch results while OST is still running (We have that in GitHub already, but our Gerrit framework is a bit rigid still)

2. Submitting pre-merged patches to CQ

3. Merge emulation, which is to say, emulate how the code will look if all patches being tested were merged together.

We have bits and pieces of everything except #3 which is by far the most complex thing to do there...

Adding as a blocker