[L2Ork-dev] Introduction

Tue Jun 18 14:27:56 EDT 2019

Hi Jeremiah,

Sorry for the delay, I work retail and it's the busy season.

I've replied inline below...

On Fri, Jun 14, 2019 at 12:15 AM Jeremiah Rose
<jeremiah.aaron.rose at gmail.com> wrote:
>
> Thanks Jonathan and Ico for your emails,
>
> I have the mailing list set to digest mode so hopefully this reply works as intended - responses inline.
>
>>
>> > I've been using PD and PD-extended on the Raspberry Pi for a prototype of an electric trumpet that I'm building. At the moment I'm working on a minimal Buildroot OS for the RPi0w to boot directly to Pure Data - that project is over at https://github.com/jeremiahrose/RPi0w-PD and it works good but doesn't do PD yet.
>> >
>> > My immediate goal is to make a Buildroot package for either Purr Data or PD Vanilla (or both!) and if you guys want, I could try to get it pulled into the main Buildroot repo.
>>
>> That would be wonderful.
>>
>> Currently you should be able to build the Purr Data core for RPI zero.
>>
>> The difficulty is that the RPI zero uses a different arm architecture
>> than the rpi3. Most of the Linux cross-compilation
>> documentation is for the other arm archs. While it would be possible
>> to build nw.js-- the GUI toolkit Purr Data uses-- for RPI zero
>> arm it's a pain and probably not particularly useful for a single CPU
>> computer that's trying to also generate audio in realtime.
>
>
> I have Pd Vanilla working well in Buildroot and currently doing tests to submit the patch.
> Yes, I'm a bit intimidated by Purr Data's build process. Is the core (minus GUI and externals) substantially different from Pd Vanilla?

Yes. Pd Vanilla has hard-coded tcl strings in the core. In Purr Data
these are generic calls to a new interface gui_vmess which forwards
messages to the GUI.

Albert improved the build process by making it possible to build just
the core using "make light." Building the rest of the externals
doesn't
actually add much time or complexity-- it's mostly Gem which takes the
bulk of the build time and complexity with dependencies,
multiple backends, etc.

> What's the relationship between Purr Data and Pd Vanilla in terms of git workflow - do you rebase against upstream Pd Vanilla HEAD (like Pd extended did) or was it forked from a previous version?

It was forked from Pd extended and we have patched in changes that
were added in Pd Vanilla.

But the git flow is probably similar. We use gitlab so it's just a
matter of clicking the fork button, making changes and then submitting
a "merge request". (In github this is called a "pull request" but it's
the same thing.)

>
>> > More generally I'm interested in using a headless Pure Data in embedded contexts.
>>
>> If that's the case then you should be able to build Purr Data just
>> fine. The current build script will probably download an nw.js
>> binary for the wrong arm architecture. But running Purr Data with the
>> "-nogui" flag should still work just fine.
>
>
> This will work well enough for me but I don't think Buildroot would accept a patch for a package that contains a non-working binary. Maybe I'll try to compile Chromium for RPi0w anyway and see how it goes ... sounds like a fun night in :-o

It shouldn't be too difficult to add a recipe to the build system to
build just the core without downloading the nw.js binary.
I haven't had a need for that personally but I'll happily merge it if
someone tests it out.

>
>>
>> > I read the conversation with Giulio from the Bela project and I feel like my needs are pretty much aligned with his in regards to separation of GUI and backend.
>> > I would like to use a headless Pure Data on one machine, and then subsequently connect a browser based GUI from another machine to edit the currently running patch. So I'm keen on the HTML GUI used in Purr Data and would like to learn more about how that's implemented, and what I can do to help with any efforts to separate GUI from backend.
>>
>> Giulio did an initial prototype to keep the stream of "motion"
>> messages (or "mousemove" event in the GUI) from propagating to the
>> audio engine.
>> That's a nice start because each "motion" message that makes it to a
>> method call[1] triggers a walk through the linked list of gobjs inside
>> a patch.
>> With his patch you'd only generate gobj list walks on click[2] and on
>> dragging selected objects.
>
>
> Do you have a link to Giulio's work on this? I couldn't find the relevant commits at https://git.purrdata.net/giuliomoro/purr-data

I think it's here:

https://github.com/giuliomoro/purrdata/tree/simpler-motion

Even though Purr Data uses a generic interface to send messages to the
GUI, those messages
are sprinkled throughout the g_*.c code in pd/src. For example: to
create an object box with text
there isn't a single function that sends all the data the GUI needs to
draw it. Instead, there is a
call to draw the border, a call to draw the text, inlets/outlets, etc.
It's even more complicated for
garrays and GOP canvases. There are some ways to incrementally
simplify this but it's slow
work without a spec or testing framework.

>
>>
>> Slight digression-- I'm a bit curious what would happen if
>> canvas_findhitbox iterated through an array of gobjs rather than a
>> linked list. In other words,
>> maintain an array of gobs *in addition* to the current linked list
>> implementation, then measure the performance of array iteration vs.
>> linked-list walk in
>> canvas_findhitbox. That may be just the tip of the iceberg. But even
>> if we separate GUI from core the user will still need to stream
>> messages for things
>> like displacing an object or dragging a slider. In those cases,
>> jumping to the same array index 100 times in a row would be preferable
>> to walking N
>> links 100 times in a row, especially where N is large.
>
>
> Sorry for my ignorance, but are the functions you're quoting here a part of the nw.js implementation? Would it be possible to upstream these improvements to Vanilla or am I thinking about it wrongly?

These are functions currently in both Purr Data and Pd Vanilla.

>
>>
>> As far as the HTML5 GUI-- I used nw.js to retain the multiple toplevel
>> windows of Pd's UI. For a purely browser-based interface with sensible
>> UX
>> I think we need a single container (document, iframe, DOM element,
>> whatever makes sense) through which the user can access and view
>> various
>> canvases available from a given instance. That way the user can click
>> on a subpatch in that container and see the inside of it in the same
>> container.
>> I think we'd have to just relegate each canvas to take up the full
>> space that the container provides-- otherwise we'll spend all our time
>> re-implementing
>> a windowing system inside the browser.
>
>
> From my perspective as an outsider, this sounds like a very sensible approach. Each canvas is a div/iframe/whatever, and allow the web developer to put them wherever she wants.
>
>>
>> But this is all just initial brainstorming on my part. Let me know if
>> you have any questions or suggestions going forward. A lot of this
>> work can
>> be leveraged and used to get Purr Data's backend running as a
>> webassembly binary (and therefore, having the entire instance running
>> inside
>> the browser).
>
>
> That would indeed be very cool. I found another project claiming to do this here: https://github.com/sebpiq/WebPd.
>
> At the moment (again, from my perspective as an outsider), it seems like there is a constellation of overlapping Pure Data projects without very much coordination and a lot of duplicated work. Was this a philosophical decision made by the community, or more of an accident?

WebPd reimplements a subset of the core Pd Vanilla classes in
Javascript. When I last looked the author had implemented some basic
patch
display but no editing or any way to deal with multiple canvases or
even GOP canvases.

I made a conscious decision not to do a grand refactoring with Purr
Data. I did this because Pd has no language spec or any kind of GUI
documentation or guidelines. Previous projects like DesireData ended
with burnout because the author underestimated the complexity of
the GUI/backend interaction. I imagine that adding editing and GOP
display to WebPd would have been similarly time-consuming. The only
way I saw to succeed was to just have the backend to continue spamming
the GUI console with tcl strings until all of them had been converted
to actual GUI functionality.

> I would love to see an ecosystem where I could, for example, use the core Pd Vanilla maintained by Miller as an API and then just compile it against whatever GUI worked for my application, be that nw.js,

You can't do that, partly because of the hard-coded tcl strings in Pd
Vanilla's message-dispatching code. Also, it may appear at first
glance that the
C code is the "server" and the tcl/tk code the "client", but it's not.
The "server" makes a lot of assumptions that depend on the
idiosyncracies
of tk's API.

> Tk, a browser based GUI connecting to a server backend (which is what I need), or a browser based webassembly backend, or whatever. Or maybe that's what's happening already?

Adding the ability in Purr Data to start a headless core on the Pi and
then connect over a socket is probably pretty straightforward. I think
there's already code in Pd Vanilla to start in "-nogui" mode and have
it wait for a connection from a GUI over the socket, so it would just
be a matter of porting that.

Getting Purr Data running fully in the browser (e.g., editable
patches) would require an effort to abstract out all the places where
I'm
currently using node.js to interact with the filesystem, as well as
the places I'm currently using the nw.js windowing interface. I think
I annotated the latter pretty clearly in the code, but I'm not sure
about the node.js stuff.

Best,
Jonathan

> Anyway, just brainstorming on my behalf.
>
> Cheers!
> Jeremiah