[L2Ork-dev] CI Runners

[Jonathan Wilkes]

Let me take a few more shots at it getting the CI working over the next few
days. There are at least a few odd bugs that have nothing to do with
the complexity of gitlab's CI system. Those bugs are creating failures
and resulting in enormous build times for some of the runners.

Once I have it running smoothly, we can see if it's "good enough." If
not, you may be right that #3 is the only sane route.

Also, I *think* there's a way to make it so that, say, only your merge
requests trigger CI. For GSoC that might be a nice pattern.

-Jonathan


On Wed, Mar 14, 2018 at 4:48 AM, Albert Graef <aggraef at gmail.com> wrote:
> Hi Jonathan,
>
> First, thanks for going through all that tedium and getting me set up for CI. :)
>
> On Fri, Mar 9, 2018 at 12:52 AM, Jonathan Wilkes <jon.w.wilkes at gmail.com> wrote:
>> I wish there were some easier way-- I don't think it's a good design
>> to require me to give myself a bunch of unnecessary permissions on
>> someone else's fork just to allow them to use my runners. But so far
>> that's the only what I found to achieve this.
>
> Well, I've been looking at
> https://docs.gitlab.com/ce/ci/runners/README.html, but I'm still
> struggling trying to understand how runners are set up in Gitlab. On
> Github or Bitbucket, the CI builds run on their servers. But with
> Gitlab you need your own, separate server (either physical or in the
> cloud) on which the runners run if I'm not mistaken. So if any
> collaborators on the project needs to be set up for CI, they will
> always have to ask you to run all those commands on your server,
> right? I don't think that there's any other way to do this in Gitlab,
> unless you're going for shared runners (which I agree is a security
> risk).
>
> So one problem that we have with that kind of setup is the extra work
> it puts on you. The other problem (and that's only going to get worse
> as you give runner access to more people), is that it puts quite a bit
> of load on your servers. Indeed, spawning a runner each time I push
> something to my fork on your Gitlab instance makes me feel a bit
> uneasy now because of that. I handle that by pushing and then quickly
> going into the Gitlab web UI and canceling the pipelines for stuff
> that's not ready yet, but this is cumbersome.
>
> So we need to figure out a workflow for collaborators like me which
> scales up considering the limited server resources we have. It's
> certainly nice to have access to your runners, so that a PR has
> already gone through your CI before you merge. But I think that we
> also need to keep the load on your servers at bay. I see three
> possibilities there:
>
> (1) Provide more runners elsewhere. That's doable, but involves a
> substantial amount of extra (also administrative) work (which
> personally I don't have the time for right now).
>
> (2) Configure the runners so that they only run on protected branches.
> That's not really practical, though, because typically pull requests
> involve a lot of rebasing and squashing, thus rewriting history which
> isn't possible on protected branches by definition.
>
> (3) Set up forks of the main repo elsewhere and only push branches to
> your Gitlab instance when they're (almost) ready to be merged. This
> also has a downside: no CI during development, unless people set that
> up for themselves (which basically amounts to option (1) with all that
> entails).
>
> Nevertheless, I think that (3) is the only workable solution which can
> be implemented in the short term, until we pursue option (1), so
> that's what I'm going for now. I already have a private fork of the
> main repo on Bitbucket which I'm using as a backup right now. I can
> just make that public and use it for development. I also have test
> environments in place for all platforms we support locally anyway, so
> ongoing testing during development isn't an issue for me (in fact
> that's what I'm already doing right now, before I submit pull
> requests).
>
> Cheers,
> Albert
>
> --
> Dr. Albert Gr"af
> Computer Music Research Group, JGU Mainz, Germany
> Email:  aggraef at gmail.com
> WWW:    https://plus.google.com/+AlbertGraef
> _______________________________________________
> L2Ork-dev mailing list
> L2Ork-dev at disis.music.vt.edu
> https://disis.music.vt.edu/listinfo/l2ork-dev