Support to FTP files for Kimchi Project

Description

Hello,

Kimchi is a sub-project of oVirt and we are planning to make available to users
simple qcow2 image files of Fedora 23, OpenSUSE 42.1 and Ubuntu 16.04 with
Kimchi installed in there.

We need support to store the files and make them available to download, in a
service like FTP. Since Kimchi mailing lists already use oVirt infrastructure,
I'd like to know if it's possible to be supported by oVirt and how to do that.

Thanks and best regards, Paulo.

Paulo Ricardo Paz Vital
Linux Technology Center, IBM Systems
http://www.ibm.com/linux/ltc/

Activity

Show:

Former user August 11, 2016 at 3:33 PM
Edited

Hi ,
Regarding the first part of your 1'st question, your patch needs to be sent to the 'jenkins' project on gerrit.ovirt.org (on the master branch): git://gerrit.ovirt.org/jenkins.git. , does he need special permissions for that? Also I'm not sure about the second part of the question.
Regarding your 2'nd question, all you need is to add a key named 'trigger-times' in the project yaml, and put the desired value.
For example:

  • project:
    name: kimchi_project
    project: kimchi
    stage: build-artifacts
    trigger-times: 'H H/6 * * *'
    ...

Regarding question 3, afaik we only support CentOS and Fedora, can you confirm that?

Eyal Edri August 11, 2016 at 2:46 PM

can you help Paulo on the basics on how to submit a patch to gerrit.ovirt.org to the jenkins repo?

Paulo Ricardo Paz Vital August 11, 2016 at 2:00 PM

Hi . Nice documentation, but I still have some questions:

1- I created the project yaml and the scm yaml files in my local repository. How can I submit them to review? I planning to configure only one of our 4 projects to test in a first moment and, in addition, using a specific branch in my fork in github that will include the necessary files to CI Standard. If this first test works well, is it possible to submit updates (point to projects github repository) and improvements in project's files?
2- One of the fields for the project yaml file is the trigger, that can be on-change or timed. In case of timed (that is what I want to use) how can I set the periodicity to run the job?
3 - Reading the documentation, I could see only references to build artifacts to CentOS (EL) and Fedora. We provide artifacts to Ubuntu (Debian) and OpenSUSE, also. Is it possible to build for these distros?

Eyal Edri August 8, 2016 at 2:22 PM

hey, just one update regarding standard CI , we are in progress of adding documentation on how to add yaml code in here:
https://gerrit.ovirt.org/#/c/61376/5

so you can use it for adding your project

Paulo Ricardo Paz Vital July 28, 2016 at 3:31 PM

Hi

I read the standard CI and have an idea of what is necessary to add in our code to do this. I'm only finishing some other tasks and then will focus in the implementation of all CI stuff from our side.

Eyal Edri July 28, 2016 at 7:08 AM

Hey,
Did you got a chance to read on the standard CI ? do you need help in adding the 1st job?

Eyal Edri June 28, 2016 at 6:19 PM

I think we can support also GitHub repos, same way as Lago is running CI on
jenkins.ovirt.org.
for e.g:
http://jenkins.ovirt.org/job/lago_master_build-artifacts-el7-x86_64/,

But if you're considering moving to Gerrit it will be much easier and
you'll be able to enjoy more advantages of our CI system, such as gerrit
hooks.

e.

On Tue, Jun 28, 2016 at 7:23 PM, Barak Korren (oVirt JIRA) <


Eyal Edri
Associate Manager
RHEV DevOps
EMEA ENG Virtualization R&D
Red Hat Israel

phone: +972-9-7692018
irc: eedri (on #tlv #rhev-dev #rhev-integ)

Barak Korren June 28, 2016 at 4:22 PM

I guess that if you want to remain on GitHub, we could make Gerrit repositories that will only contain the scripts for building the images, or even just simple wrapper scripts that will clone code from GitHub and use it...

You can build anything with standard CI.
One thing to note is that just building something with STD CI already makes it available over HTTP from Jenkins.

The uploading to resource.ovirt.org and repo building is not strictly part if the STD CI per-se its is a deployment stage that typically follows it.
We currently build RPMs in oVirt and have a tool called 'repoman' that knows how to collect them into yum repos. This means that supporting 'yum/dnf' repos is very easy for us. I guess we could also support other kinds of repos but that will take some work because we will need to come up with some tooling for that.

Paulo Ricardo Paz Vital June 28, 2016 at 3:18 PM

Thanks guys by your support. I'm going to read the STD CI and provide the necessary changes in our repository to automate our artifacts building process.

Our GitHub repositories are [1], [2] and [3] and, until now, the decision to use Gerrit in our process is out of scope (personally I like the idea and will rise the question to our Maintainers and community to see what they think about).

One curiosity. If we follow the STD CI, is it possible to build and make available package repositories (we support yum/dnf, apt and zypper) to be used by our users in addition to the images repositories?

[1] https://github.com/kimchi-project/wok.git
[2] https://github.com/kimchi-project/gingerbase.git
[3] https://github.com/kimchi-project/kimchi.git

Eyal Edri June 28, 2016 at 7:07 AM
Edited

Hi Paulo.
We don't add jenkins jobs directly on jenkins, as we use Jenkins Job Builder to add new jobs and also we have something called 'standard ci' which simplify dramatically the process of adding a new project.

to move forward, I suggest you'll post here the source GitHub repos you're using (btw, if you are an oVirt project, you might consider migrating to gerrit.ovirt.org).
In addition, please read [1] to see how the STD CI process works and once you're familiar with it, we can help you add the relevant jobs.

For now it seems only 'build artifacts' will be needed, if you don't have any other sanity jobs.
Also please add to this ticket the commands you're using to build the artifacts (basically you will need to add an 'automation' dir under the root of your projects and add build-artifacts.sh with the command to build it)

[1] http://infra-docs.readthedocs.io/en/latest/CI/Build_and_test_standards.html

Barak Korren June 28, 2016 at 7:05 AM

Barak Korren If oVirt support to qcow2 images to create new VMs, so it's possible to use them on top of oVirt. However, I guess the best options is host them on 'resources.ovirt.org' to make available anyone (using oVirt or not) to download them. About the 'libguestfs.org' style index file we can for sure use the same to support virt-builder or Lago.

oVirt can useQCOW2 images, but its not trivial to make them available for it to use, at least not until BZ1343077 makes it out the door.

About an automated build, we also can provide this, even with only 4 releases by year. We are using the infrastructure of GitHub (where our source control is hosted) to provide the RPMs and DEBs we build, so since a new version is available there, a new task to create an image with them can be set up. For that, I guess we also need access to oVirt Jenkins to create and configure the tasks, if possible.

It it fairly easy to get stuff running on the oVirt Jenkins, all you need is a repo in oVirt Gerrit (if you don't have one, I can make one for you), and have scripts inside that repo that comply with the oVirt CI standards.
Once you have those running and creating images, we can cover the extra mile of getting the images on resources.ovirt.org.

We need to consider how to structure the repos though, would we have one repo for all the images, or one per-image, or perhaps one repo with a branch per-image. With one repo, the build time can be longer, although you could try to parallelise from inside the build script (may be hard to debug). With multiple repos, you use more Gerrit resources and have more manual maintenance, but you can get Jenkins to parallelise for you. With branches, you can have Jenkins parallelise and 'git diff' can make maintenance easier, but you violate the principle of least surprise by making 'master' not mean what people are used to having it mean.

Won't Do

Details

Assignee

Reporter

Blocked By

Priority

Created June 24, 2016 at 6:31 PM
Updated November 29, 2016 at 12:18 PM
Resolved November 29, 2016 at 11:51 AM

Flag notifications