I saw the build queue rise to ~200 tasks today due to these two jobs configured for running only 6 builds in parallel:

ovirt-engine_master_upgrade-from-4.0_el7_created
ovirt-engine_master_upgrade-from-master_el7_created

I bumped the default limit to 10 in https://gerrit.ovirt.org/61319/ but engine-upgrade job override this setting 6:

https://gerrit.ovirt.org/gitweb?p=jenkins.git;a=blob;f=jobs/confs/yaml/templates/engine-upgrade.yaml;hb=refs/heads/master#l23

there's more places where various limits are set - I think we should review them and even remove if possible as the number of slaves is going to increase.

git grep max-total
jobs/confs/projects/defaults.yaml: max-total: 3
jobs/confs/projects/defaults.yaml: max-total: 3
jobs/confs/projects/ovirt/publish-rpms-nightly-3.6.yaml: max-total: 3
jobs/confs/projects/ovirt/publish-rpms-nightly-4.0.yaml: max-total: 3
jobs/confs/projects/ovirt/publish-rpms-nightly-master.yaml: max-total: 3
jobs/confs/yaml/templates/engine-upgrade.yaml: max-total: 6
jobs/confs/yaml/templates/find-bugs.yaml: max-total: 3
jobs/confs/yaml/templates/install-rpm-sanity.yaml: max-total: 3
jobs/confs/yaml/templates/ovirt-node-job-templates.yaml: max-total: 3
jobs/confs/yaml/templates/standard-stage.yaml: max-total: 10
jobs/confs/yaml/templates/standard-stage.yaml: max-total: 1