[L2Ork-dev] Purr Data: File / Print... PDF

Sat Oct 24 09:28:39 EDT 2020

On Sat, Oct 24, 2020 at 2:59 PM Albert Graef <aggraef at gmail.com> wrote:

> Mac and Windows are looking good as well, so I've thrown it up on the OBS
> preview channel at
> https://build.opensuse.org/package/show/home:aggraef:purr-data-git/purr-data
> .
>

Done, the 20.04 package is here:
https://download.opensuse.org/repositories/home:/aggraef:/purr-data-git/xUbuntu_20.04/amd64/purr-data_2.15.2+git4742+c3462eac-1_amd64.deb

Albert

It would be nice if you could give it a whirl when the build is finished
> and tell me how you like it. (As usual, you can download the 20.04 package
> at
> https://download.opensuse.org/repositories/home:/aggraef:/purr-data-git/xUbuntu_20.04/
> once the build finishes. The revision will be 2.15.2+git4742+c3462eac.)
>
> Jonathan, the MR is now up at
> https://git.purrdata.net/jwilkes/purr-data/-/merge_requests/595. (It
> would be nice if you could take a look at the CI, it seems broken again.)
>
> Albert
>
>
>
> On Sat, Oct 24, 2020 at 12:56 PM Linux ROUEN Normandie <
> linux.rouen at free.fr> wrote:
>
>> ♥‿♥
>> Joseph
>> - - - - - - - - - - - - - - - - - - - -
>> Le 24/10/2020 à 12:46, Albert Graef a écrit :
>>
>> On Sat, Oct 24, 2020 at 3:32 AM Jonathan Wilkes <jon.w.wilkes at gmail.com>
>> wrote:
>>
>>> What I'm talking about is the backend-- the audio engine and message
>>> dispatcher-- taking 2 or 3 minutes to process some data. I believe
>>> Joseph even hit this bug with the exponential list-walking explosion
>>> when he was trying to drag a large selection of objects.
>>>
>>> In those cases, pd-watchdog just keeps printing out messages until the
>>> backend finishes whatever brute-force task it was working on.
>>> Pd-watchdog will never bail in those cases.
>>>
>>
>> Right, that's funny. But the real issue is obviously somewhere in the
>> engine's idle processing, I'd say. As long as the *engine* is busy, it just
>> won't listen to those SIGHUPs, so the watchdog just keeps sending those and
>> printing the "signaling pd" messages along with it.
>>
>> But if the *GUI* is busy, the engine keeps chugging along doing its usual
>> idle processing, which apparently involves reading some sockets via the
>> recv() call. I have no idea where and why exactly that happens, but
>> apparently that recv() call *will* listen to the SIGHUP, and be upset
>> enough about it that it crashes the process, because the SIGHUP isn't
>> handled anywhere. The gdb backtrace seems to indicate that the select()
>> call in sys_domicrosleep() is the culprit, but I suspect that it's actually
>> related to the audio backend in some way, because this crash does or
>> doesn't happen depending on which audio backend is being used. (That
>> discrepancy might also be due to the part of Jack which runs in the client
>> taking care of masking those SIGHUPs, but I can't find anything in the
>> audio backend code of Pd itself which explains the differences between
>> backends there.)
>>
>> Anyway, I was able to solve this by just adding a global method which can
>> be used by the GUI to mark sections when it is busy, during which the
>> engine just keeps the watchdog happy by itself as if running GUI-less. I
>> added those calls to the new print dialog as well as the message operation
>> and the make_index() call which constructs the help index when bringing up
>> the help browser. I *think* that this should cover all cases where this
>> crash may happen.
>>
>> I already tested that on Linux, now testing on Mac and Windows to make
>> sure that the new mechanism doesn't break anything there, so expect a merge
>> request soon. (I also left the sys_domicrosleep() backport from vanilla in
>> there, because I think that it's good to have those fixes as well, even
>> though they didn't help with the issue at hand.)
>>
>> Albert
>>
>>
>>> >
>>> >> Also, given various use cases with long-running number cruncher
>>> >> patches, watchdog *cannot* bail in any of those cases without solving
>>> >> the halting problem.
>>> >
>>> >
>>> > It doesn't try to solve the halting problem. It's a very dumb little
>>> routine which just loops waiting for its socket to dry up. ;-)
>>> >
>>> > I guess that the watchdog is just fine as long as the gui is still
>>> alive and kicking, no matter what the engine does.
>>>
>>> Hm... I'm stumped then. Because I know for one of the problems Joseph
>>> reported, the backend is stuck walking lists in a blocking call that
>>> will keep chugging away for minutes before letting the gui routine do
>>> any socket business. I don't know why this wouldn't cause pd-watchdog
>>> to bail, yet a few seconds of a javascript blocking call would.
>>>
>>> >
>>> > About realtime priorities: Yes, there have been improvements in the
>>> API, but Pd doesn't use them AFAICT.
>>> >
>>> >> So I'm just not sure what its purpose is.
>>> >
>>> >
>>> > I guess that it's a left-over from the bad old days when we all had
>>> single core computers. At that time it was fairly easy to completely lock
>>> up the system with a runaway realtime process. Nowadays, this has become
>>> much harder, usually you can still kill that process.
>>>
>>> Ah, right.
>>>
>>> -Jonathan
>>>
>>> >
>>> > Albert
>>> >
>>> >>
>>> >> -Jonathan
>>> >>
>>> >> On Fri, Oct 23, 2020 at 5:20 PM Jonathan Wilkes <
>>> jon.w.wilkes at gmail.com> wrote:
>>> >> >
>>> >> > On Fri, Oct 23, 2020 at 4:55 PM Albert Graef <aggraef at gmail.com>
>>> wrote:
>>> >> > >
>>> >> > > On Fri, Oct 23, 2020 at 10:03 PM Jonathan Wilkes <
>>> jon.w.wilkes at gmail.com> wrote:
>>> >> > >>
>>> >> > >> > I see that there are some fairly recent code changes in
>>> sys_domicrosleep() in vanilla, though. I'll port those and see whether that
>>> helps.
>>> >> > >>
>>> >> > >> I don't think they will help.
>>> >> > >
>>> >> > >
>>> >> > > You're right, backporting those didn't help.
>>> >> > >
>>> >> > >> The problem is that if pd-watchdog
>>> >> > >> doesn't receive an acknowledgement it will shut down the backend.
>>> >> > >
>>> >> > >
>>> >> > > Yep, that's it. If I modify the watchdog so that it never bails
>>> out then both the message and the print dialog work alright. Of course
>>> that's not the proper solution ;-), but it proves that the watchdog is the
>>> culprit here. So we just have to find a way to keep it happy, see below.
>>> >> >
>>> >> > I'm curious-- is there any extant user report of watchdog saving
>>> them
>>> >> > from having to hard reboot their machine (which is what I guess is
>>> >> > what it's there for)?
>>> >> >
>>> >> > The thing is, this is the first case I've heard of where the GUI has
>>> >> > caused a hangup. And it's a false positive because the blocking call
>>> >> > was triggered by the user purposely and is temporary.
>>> >> >
>>> >> > On the other hand, I've seen my backend lock down the CPU *all the
>>> >> > time*. Exponential explosion is the name of the game all through
>>> >> > g_editor.c, and there are various other freezes from external libs
>>> and
>>> >> > bugs. I have *never* seen watchdog bail in that case-- it just keeps
>>> >> > spitting out "watchdog: signaling pd..." until I ctrl-c out of the
>>> >> > terminal.
>>> >> >
>>> >> > Also-- hasn't Linux realtime priority processing improved
>>> >> > significantly over the past decade to make it less likely to lock up
>>> >> > the machine? (I've only heard in general about improvements, so I'm
>>> >> > not sure about this.)
>>> >> >
>>> >> > >
>>> >> > >> We can change the "message" dialog to something that doesn't
>>> block the
>>> >> > >> js engine. I'm not sure about the print dialog, though.
>>> >> > >
>>> >> > >
>>> >> > > Probably not much that we can do about the print dialog. And
>>> there might be other cases where things may go awry. Search index
>>> generation comes to mind -- we might just have been lucky there because
>>> this is a very quick operation on Linux, but it might give the same issue
>>> on slower computers.
>>> >> > >
>>> >> > > This really calls for a general solution where the gui informs
>>> the engine that it should keep the watchdog happy while it's executing a
>>> callback which may run for a while, and then inform it again when that
>>> callback is finished. That shouldn't be too hard to do, the engine already
>>> does this anyway when running gui-less. I'm working on it.
>>> >> >
>>> >> > I'm curious what happens if we get rid of watchdog and run Purr Data
>>> >> > with realtime priorities on a current distro... :)
>>> >> >
>>> >> > -Jonathan
>>> >> >
>>> >> > >
>>> >> > > Albert
>>> >> > >
>>> >> > >
>>> >> > >>
>>> >> > >> -Jonathan
>>> >> > >>
>>> >> > >> >
>>> >> > >> > Albert
>>> >> > >> >
>>> >> > >> >>
>>> >> > >> >> -Jonathan
>>> >> > >> >> _______________________________________________
>>> >> > >> >> L2Ork-dev mailing list
>>> >> > >> >> L2Ork-dev at disis.music.vt.edu
>>> >> > >> >> https://disis.music.vt.edu/listinfo/l2ork-dev
>>> >> > >> >
>>> >> > >> >
>>> >> > >> >
>>> >> > >> > --
>>> >> > >> > Dr. Albert Gr"af
>>> >> > >> > Computer Music Research Group, JGU Mainz, Germany
>>> >> > >> > Email: aggraef at gmail.com, web: https://agraef.github.io/
>>> >> > >> > _______________________________________________
>>> >> > >> > L2Ork-dev mailing list
>>> >> > >> > L2Ork-dev at disis.music.vt.edu
>>> >> > >> > https://disis.music.vt.edu/listinfo/l2ork-dev
>>> >> > >> _______________________________________________
>>> >> > >> L2Ork-dev mailing list
>>> >> > >> L2Ork-dev at disis.music.vt.edu
>>> >> > >> https://disis.music.vt.edu/listinfo/l2ork-dev
>>> >> > >
>>> >> > >
>>> >> > >
>>> >> > > --
>>> >> > > Dr. Albert Gr"af
>>> >> > > Computer Music Research Group, JGU Mainz, Germany
>>> >> > > Email: aggraef at gmail.com, web: https://agraef.github.io/
>>> >> > > _______________________________________________
>>> >> > > L2Ork-dev mailing list
>>> >> > > L2Ork-dev at disis.music.vt.edu
>>> >> > > https://disis.music.vt.edu/listinfo/l2ork-dev
>>> >> _______________________________________________
>>> >> L2Ork-dev mailing list
>>> >> L2Ork-dev at disis.music.vt.edu
>>> >> https://disis.music.vt.edu/listinfo/l2ork-dev
>>> >
>>> >
>>> >
>>> > --
>>> > Dr. Albert Gr"af
>>> > Computer Music Research Group, JGU Mainz, Germany
>>> > Email: aggraef at gmail.com, web: https://agraef.github.io/
>>> > _______________________________________________
>>> > L2Ork-dev mailing list
>>> > L2Ork-dev at disis.music.vt.edu
>>> > https://disis.music.vt.edu/listinfo/l2ork-dev
>>> _______________________________________________
>>> L2Ork-dev mailing list
>>> L2Ork-dev at disis.music.vt.edu
>>> https://disis.music.vt.edu/listinfo/l2ork-dev
>>
>>
>>
>> --
>> Dr. Albert Gr"af
>> Computer Music Research Group, JGU Mainz, Germany
>> Email: aggraef at gmail.com, web: https://agraef.github.io/
>>
>> _______________________________________________
>> L2Ork-dev mailing listL2Ork-dev at disis.music.vt.eduhttps://disis.music.vt.edu/listinfo/l2ork-dev
>>
>>
>> _______________________________________________
>> L2Ork-dev mailing list
>> L2Ork-dev at disis.music.vt.edu
>> https://disis.music.vt.edu/listinfo/l2ork-dev
>
>
>
> --
> Dr. Albert Gr"af
> Computer Music Research Group, JGU Mainz, Germany
> Email: aggraef at gmail.com, web: https://agraef.github.io/
>

-- 
Dr. Albert Gr"af
Computer Music Research Group, JGU Mainz, Germany
Email: aggraef at gmail.com, web: https://agraef.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://disis.music.vt.edu/pipermail/l2ork-dev/attachments/20201024/fd236e03/attachment-0001.html>