[L2Ork-dev] request for mentorship: build infrastructure

Fri Mar 17 01:13:13 UTC 2017

Thanks, Albert.
The tags should be pretty easy.  I'll start with those when I get a chance.
I can probably just re-run the ci for the missing 2.0 binaries.  That will at least 
get me Linux and OSX 10.11 binaries.
-Jonathan

On Wed, Mar 15, 2017 at 1:29 AM, Jonathan Wilkes <jancsika at yahoo.com> wrote:

Albert provided some comments on the issue tracker about how to 
handle the release process for Purr Data.

Just for the record, my comments start with this post here: https://git.purrdata.net/jwilkes/purr-data/merge_requests/83#note_2538

The TL;DR version is: (1) We need to tag the releases in Purr Data's source repo, which is easy to do, and (2) we need to upload the binary packages to permanent cloud storage somewhere, which doesn't seem to be so easy in Gitlab. Also, Gitlab doesn't present the downloads in a convenient way that makes it easy for users to find them.

That's why I suggested to use a mirror of the Gitlab source repo on Github for that purpose. Github has a very nice and simple-to-use release subsystem, basically you add some release notes to the tag marking the release, upload the binary packages and you're done.

I've already set up a mirror there, so you can all have a look: https://github.com/agraef/purr-data. Click on "releases", or follow this link: https://github.com/agraef/purr-data/releases. All releases are right there on a single page, with the latest release on top, and it's easy to find the packages in the Downloads section of each release.

(Jonathan, btw, if you still have the remaining 2.0 binaries lying around somewhere, please toss them my way so that I can upload them there, thanks.)

I think tagging is fairly straightforward.  Do we need to specify a format 
for the text of the tag? $major.$minor.$bugfix, v$major.$minor.$bugfix, 
etc.

Some people use the "v" prefix, but IMHO it's redundant, so I'd prefer just the naked version number.

Publishing: it seems like I should be able to use gitlab ci for this.  Currently 
the ci binaries (called "artifacts") last a day and are then removed to save 
space.  I could add a rule that when a release tag is pushed then the ci 
builds binaries that never get removed.  Then we'd host the full archive of 
releases in gitlab, plus the temporary binaries that come from merges.

That would be nice to have. Then instead of uploading the packages at the Github mirror, one could simply link to the binaries on Gitlab in the release notes. OTOH, it can't hurt to have a backup of the packages on Github.

However, that would mean that I push the release tag to *trigger* the build 
of release binaries.  Not sure if that's a problem, or if there's a workaround for 
that.

I don't see a problem there. While rewriting commit history on the master branch is generally bad and not permitted (with good reason) the way repos are typically set up on Gitlab and Github, "repairing" release tags is not a big deal and can be done at any time, as long as the release isn't really "out there" yet. So, if you notice that you forgot something or made a mistake, you just stop the release CI pipeline and delete the build artifacts, do whatever hot fixes are needed and push them, then do a `git tag -f` and force-push the tag again when you're done.

Once the CI build is through, if you're fine with everything, you just drop me (or whoever is our release manager at the time) an email that release x.y.z is out and your release notes to go along with that, and Mr./Mrs. RM will take care of putting it up on the Github mirror, at which point it's going to be available and easy to find for everyone.

 The idea there is that we'd use the Github mirror for publishing the releases (along with a mirror of the source repo), so we can play around on Gitlab as long as needed until you think it's time for a release. Of course, once the release landed on Github, it is "out there", so there's no going back, and any further fixes will need a new release.

I think that this process is sensible and workable, and it allows Gitlab to be used as the "lab" in which we work on things until they're done, and the Github mirror as our "outlet" to present the shiny results.

I'm also fine with using github to publish the binaries and/or store the older 
versions.  But is there a way to automate the process of uploading the binaries 
to github from gitlab ci?

Well, we do need to come up with some sensible release notes, I don't see how we could automate that. :) And after entering those you can just as well push a button, select the binaries on your local hard drive, and add them to the release, simple as that. I'm not sure whether Github offers an API to automate that process, but it's not really needed, as the process is so easy. And some things are still better done by humans, even in this day and age. ;-)

Of course this manual publishing of releases won't scale up when you start doing releases *very* frequently (weekly or even daily), but I don't see that happening yet, so let's talk about that if and when the need arises.

I can take care of the Github part of the releases for now, until someone else steps up to become our release master. Any proposals yet? :)

Cheers.
Albert

-- 
Dr. Albert Gr"af
Computer Music Research Group, JGU Mainz, Germany
Email:  aggraef at gmail.com
WWW:    https://plus.google.com/+AlbertGraef

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://disis.music.vt.edu/pipermail/l2ork-dev/attachments/20170317/d855407b/attachment-0001.html>