[L2Ork-dev] request for mentorship: build infrastructure

[Albert Graef]

On Wed, Mar 15, 2017 at 1:29 AM, Jonathan Wilkes <jancsika at yahoo.com> wrote:

> Albert provided some comments on the issue tracker about how to
> handle the release process for Purr Data.
>

Just for the record, my comments start with this post here:
https://git.purrdata.net/jwilkes/purr-data/merge_requests/83#note_2538

The TL;DR version is: (1) We need to tag the releases in Purr Data's source
repo, which is easy to do, and (2) we need to upload the binary packages to
permanent cloud storage somewhere, which doesn't seem to be so easy in
Gitlab. Also, Gitlab doesn't present the downloads in a convenient way that
makes it easy for users to find them.

That's why I suggested to use a mirror of the Gitlab source repo on Github
for that purpose. Github has a very nice and simple-to-use release
subsystem, basically you add some release notes to the tag marking the
release, upload the binary packages and you're done.

I've already set up a mirror there, so you can all have a look:
https://github.com/agraef/purr-data. Click on "releases", or follow this
link: https://github.com/agraef/purr-data/releases. All releases are right
there on a single page, with the latest release on top, and it's easy to
find the packages in the Downloads section of each release.

(Jonathan, btw, if you still have the remaining 2.0 binaries lying around
somewhere, please toss them my way so that I can upload them there, thanks.)

I think tagging is fairly straightforward.  Do we need to specify a format
> for the text of the tag? $major.$minor.$bugfix, v$major.$minor.$bugfix,
> etc.
>

Some people use the "v" prefix, but IMHO it's redundant, so I'd prefer just
the naked version number.

Publishing: it seems like I should be able to use gitlab ci for this.
> Currently
> the ci binaries (called "artifacts") last a day and are then removed to
> save
> space.  I could add a rule that when a release tag is pushed then the ci
> builds binaries that never get removed.  Then we'd host the full archive
> of
> releases in gitlab, plus the temporary binaries that come from merges.
>

That would be nice to have. Then instead of uploading the packages at the
Github mirror, one could simply link to the binaries on Gitlab in the
release notes. OTOH, it can't hurt to have a backup of the packages on
Github.


> However, that would mean that I push the release tag to *trigger* the
> build
> of release binaries.  Not sure if that's a problem, or if there's a
> workaround for
> that.
>

I don't see a problem there. While rewriting commit history on the master
branch is generally bad and not permitted (with good reason) the way repos
are typically set up on Gitlab and Github, "repairing" release tags is not
a big deal and can be done at any time, as long as the release isn't really
"out there" yet. So, if you notice that you forgot something or made a
mistake, you just stop the release CI pipeline and delete the build
artifacts, do whatever hot fixes are needed and push them, then do a `git
tag -f` and force-push the tag again when you're done.

Once the CI build is through, if you're fine with everything, you just drop
me (or whoever is our release manager at the time) an email that release
x.y.z is out and your release notes to go along with that, and Mr./Mrs. RM
will take care of putting it up on the Github mirror, at which point it's
going to be available and easy to find for everyone.

The idea there is that we'd use the Github mirror for publishing the
releases (along with a mirror of the source repo), so we can play around on
Gitlab as long as needed until you think it's time for a release. Of
course, once the release landed on Github, it is "out there", so there's no
going back, and any further fixes will need a new release.

I think that this process is sensible and workable, and it allows Gitlab to
be used as the "lab" in which we work on things until they're done, and the
Github mirror as our "outlet" to present the shiny results.

I'm also fine with using github to publish the binaries and/or store the
> older
> versions.  But is there a way to automate the process of uploading the
> binaries
> to github from gitlab ci?
>

Well, we do need to come up with some sensible release notes, I don't see
how we could automate that. :) And after entering those you can just as
well push a button, select the binaries on your local hard drive, and add
them to the release, simple as that. I'm not sure whether Github offers an
API to automate that process, but it's not really needed, as the process is
so easy. And some things are still better done by humans, even in this day
and age. ;-)

Of course this manual publishing of releases won't scale up when you start
doing releases *very* frequently (weekly or even daily), but I don't see
that happening yet, so let's talk about that if and when the need arises.

I can take care of the Github part of the releases for now, until someone
else steps up to become our release master. Any proposals yet? :)

Cheers.
Albert

-- 
Dr. Albert Gr"af
Computer Music Research Group, JGU Mainz, Germany
Email:  aggraef at gmail.com
WWW:    https://plus.google.com/+AlbertGraef
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://disis.music.vt.edu/pipermail/l2ork-dev/attachments/20170315/223d4145/attachment-0001.html>