<div dir="ltr"><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Oct 24, 2020 at 2:59 PM Albert Graef <<a href="mailto:aggraef@gmail.com">aggraef@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Mac and Windows are looking good as well, so I've thrown it up on the OBS preview channel at <a href="https://build.opensuse.org/package/show/home:aggraef:purr-data-git/purr-data" target="_blank">https://build.opensuse.org/package/show/home:aggraef:purr-data-git/purr-data</a>.</div></blockquote><div><br></div><div>Done, the 20.04 package is here: <a href="https://download.opensuse.org/repositories/home:/aggraef:/purr-data-git/xUbuntu_20.04/amd64/purr-data_2.15.2+git4742+c3462eac-1_amd64.deb">https://download.opensuse.org/repositories/home:/aggraef:/purr-data-git/xUbuntu_20.04/amd64/purr-data_2.15.2+git4742+c3462eac-1_amd64.deb</a></div><div><br></div><div>Albert<br></div><div><br> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"> It would be nice if you could give it a whirl when the build is finished and tell me how you like it. (As usual, you can download the 20.04 package at <a href="https://download.opensuse.org/repositories/home:/aggraef:/purr-data-git/xUbuntu_20.04/" target="_blank">https://download.opensuse.org/repositories/home:/aggraef:/purr-data-git/xUbuntu_20.04/</a> once the build finishes. The revision will be 2.15.2+git4742+c3462eac.)<div><br></div><div>Jonathan, the MR is now up at <a href="https://git.purrdata.net/jwilkes/purr-data/-/merge_requests/595" target="_blank">https://git.purrdata.net/jwilkes/purr-data/-/merge_requests/595</a>. (It would be nice if you could take a look at the CI, it seems broken again.)</div><div><br></div><div>Albert</div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Oct 24, 2020 at 12:56 PM Linux ROUEN Normandie <<a href="mailto:linux.rouen@free.fr" target="_blank">linux.rouen@free.fr</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<h1>♥‿♥</h1>
<div>Joseph<br>
- - - - - - - - - - - - - - - - - - - -<br>
</div>
<div>Le 24/10/2020 à 12:46, Albert Graef a
écrit :<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Sat, Oct 24, 2020 at 3:32
AM Jonathan Wilkes <<a href="mailto:jon.w.wilkes@gmail.com" target="_blank">jon.w.wilkes@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">What I'm talking about is
the backend-- the audio engine and message<br>
dispatcher-- taking 2 or 3 minutes to process some data. I
believe<br>
Joseph even hit this bug with the exponential list-walking
explosion<br>
when he was trying to drag a large selection of objects.<br>
<br>
In those cases, pd-watchdog just keeps printing out messages
until the<br>
backend finishes whatever brute-force task it was working
on.<br>
Pd-watchdog will never bail in those cases.<br>
</blockquote>
<div><br>
</div>
<div>Right, that's funny. But the real issue is obviously
somewhere in the engine's idle processing, I'd say. As long
as the *engine* is busy, it just won't listen to those
SIGHUPs, so the watchdog just keeps sending those and
printing the "signaling pd" messages along with it.</div>
<div><br>
</div>
<div> But if the *GUI* is busy, the engine keeps chugging
along doing its usual idle processing, which apparently
involves reading some sockets via the recv() call. I have no
idea where and why exactly that happens, but apparently that
recv() call *will* listen to the SIGHUP, and be upset enough
about it that it crashes the process, because the SIGHUP
isn't handled anywhere. The gdb backtrace seems to indicate
that the select() call in sys_domicrosleep() is the culprit,
but I suspect that it's actually related to the audio
backend in some way, because this crash does or doesn't
happen depending on which audio backend is being used. (That
discrepancy might also be due to the part of Jack which runs
in the client taking care of masking those SIGHUPs, but I
can't find anything in the audio backend code of Pd itself
which explains the differences between backends there.)<br>
</div>
<div><br>
</div>
<div>Anyway, I was able to solve this by just adding a global
method which can be used by the GUI to mark sections when it
is busy, during which the engine just keeps the watchdog
happy by itself as if running GUI-less. I added those calls
to the new print dialog as well as the message operation and
the make_index() call which constructs the help index when
bringing up the help browser. I *think* that this should
cover all cases where this crash may happen.</div>
<div><br>
</div>
<div> I already tested that on Linux, now testing on Mac and
Windows to make sure that the new mechanism doesn't break
anything there, so expect a merge request soon. (I also left
the sys_domicrosleep() backport from vanilla in there,
because I think that it's good to have those fixes as well,
even though they didn't help with the issue at hand.)</div>
<div><br>
</div>
<div>Albert<br>
</div>
<div> <br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
><br>
>> Also, given various use cases with long-running
number cruncher<br>
>> patches, watchdog *cannot* bail in any of those
cases without solving<br>
>> the halting problem.<br>
><br>
><br>
> It doesn't try to solve the halting problem. It's a
very dumb little routine which just loops waiting for its
socket to dry up. ;-)<br>
><br>
> I guess that the watchdog is just fine as long as the
gui is still alive and kicking, no matter what the engine
does.<br>
<br>
Hm... I'm stumped then. Because I know for one of the
problems Joseph<br>
reported, the backend is stuck walking lists in a blocking
call that<br>
will keep chugging away for minutes before letting the gui
routine do<br>
any socket business. I don't know why this wouldn't cause
pd-watchdog<br>
to bail, yet a few seconds of a javascript blocking call
would.<br>
<br>
><br>
> About realtime priorities: Yes, there have been
improvements in the API, but Pd doesn't use them AFAICT.<br>
><br>
>> So I'm just not sure what its purpose is.<br>
><br>
><br>
> I guess that it's a left-over from the bad old days
when we all had single core computers. At that time it was
fairly easy to completely lock up the system with a runaway
realtime process. Nowadays, this has become much harder,
usually you can still kill that process.<br>
<br>
Ah, right.<br>
<br>
-Jonathan<br>
<br>
><br>
> Albert<br>
><br>
>><br>
>> -Jonathan<br>
>><br>
>> On Fri, Oct 23, 2020 at 5:20 PM Jonathan Wilkes
<<a href="mailto:jon.w.wilkes@gmail.com" target="_blank">jon.w.wilkes@gmail.com</a>>
wrote:<br>
>> ><br>
>> > On Fri, Oct 23, 2020 at 4:55 PM Albert Graef
<<a href="mailto:aggraef@gmail.com" target="_blank">aggraef@gmail.com</a>> wrote:<br>
>> > ><br>
>> > > On Fri, Oct 23, 2020 at 10:03 PM Jonathan
Wilkes <<a href="mailto:jon.w.wilkes@gmail.com" target="_blank">jon.w.wilkes@gmail.com</a>>
wrote:<br>
>> > >><br>
>> > >> > I see that there are some fairly
recent code changes in sys_domicrosleep() in vanilla,
though. I'll port those and see whether that helps.<br>
>> > >><br>
>> > >> I don't think they will help.<br>
>> > ><br>
>> > ><br>
>> > > You're right, backporting those didn't
help.<br>
>> > ><br>
>> > >> The problem is that if pd-watchdog<br>
>> > >> doesn't receive an acknowledgement it
will shut down the backend.<br>
>> > ><br>
>> > ><br>
>> > > Yep, that's it. If I modify the watchdog
so that it never bails out then both the message and the
print dialog work alright. Of course that's not the proper
solution ;-), but it proves that the watchdog is the culprit
here. So we just have to find a way to keep it happy, see
below.<br>
>> ><br>
>> > I'm curious-- is there any extant user report
of watchdog saving them<br>
>> > from having to hard reboot their machine
(which is what I guess is<br>
>> > what it's there for)?<br>
>> ><br>
>> > The thing is, this is the first case I've
heard of where the GUI has<br>
>> > caused a hangup. And it's a false positive
because the blocking call<br>
>> > was triggered by the user purposely and is
temporary.<br>
>> ><br>
>> > On the other hand, I've seen my backend lock
down the CPU *all the<br>
>> > time*. Exponential explosion is the name of
the game all through<br>
>> > g_editor.c, and there are various other
freezes from external libs and<br>
>> > bugs. I have *never* seen watchdog bail in
that case-- it just keeps<br>
>> > spitting out "watchdog: signaling pd..." until
I ctrl-c out of the<br>
>> > terminal.<br>
>> ><br>
>> > Also-- hasn't Linux realtime priority
processing improved<br>
>> > significantly over the past decade to make it
less likely to lock up<br>
>> > the machine? (I've only heard in general about
improvements, so I'm<br>
>> > not sure about this.)<br>
>> ><br>
>> > ><br>
>> > >> We can change the "message" dialog to
something that doesn't block the<br>
>> > >> js engine. I'm not sure about the
print dialog, though.<br>
>> > ><br>
>> > ><br>
>> > > Probably not much that we can do about
the print dialog. And there might be other cases where
things may go awry. Search index generation comes to mind --
we might just have been lucky there because this is a very
quick operation on Linux, but it might give the same issue
on slower computers.<br>
>> > ><br>
>> > > This really calls for a general solution
where the gui informs the engine that it should keep the
watchdog happy while it's executing a callback which may run
for a while, and then inform it again when that callback is
finished. That shouldn't be too hard to do, the engine
already does this anyway when running gui-less. I'm working
on it.<br>
>> ><br>
>> > I'm curious what happens if we get rid of
watchdog and run Purr Data<br>
>> > with realtime priorities on a current
distro... :)<br>
>> ><br>
>> > -Jonathan<br>
>> ><br>
>> > ><br>
>> > > Albert<br>
>> > ><br>
>> > ><br>
>> > >><br>
>> > >> -Jonathan<br>
>> > >><br>
>> > >> ><br>
>> > >> > Albert<br>
>> > >> ><br>
>> > >> >><br>
>> > >> >> -Jonathan<br>
>> > >> >>
_______________________________________________<br>
>> > >> >> L2Ork-dev mailing list<br>
>> > >> >> <a href="mailto:L2Ork-dev@disis.music.vt.edu" target="_blank">L2Ork-dev@disis.music.vt.edu</a><br>
>> > >> >> <a href="https://disis.music.vt.edu/listinfo/l2ork-dev" rel="noreferrer" target="_blank">https://disis.music.vt.edu/listinfo/l2ork-dev</a><br>
>> > >> ><br>
>> > >> ><br>
>> > >> ><br>
>> > >> > --<br>
>> > >> > Dr. Albert Gr"af<br>
>> > >> > Computer Music Research Group,
JGU Mainz, Germany<br>
>> > >> > Email: <a href="mailto:aggraef@gmail.com" target="_blank">aggraef@gmail.com</a>, web: <a href="https://agraef.github.io/" rel="noreferrer" target="_blank">https://agraef.github.io/</a><br>
>> > >> >
_______________________________________________<br>
>> > >> > L2Ork-dev mailing list<br>
>> > >> > <a href="mailto:L2Ork-dev@disis.music.vt.edu" target="_blank">L2Ork-dev@disis.music.vt.edu</a><br>
>> > >> > <a href="https://disis.music.vt.edu/listinfo/l2ork-dev" rel="noreferrer" target="_blank">https://disis.music.vt.edu/listinfo/l2ork-dev</a><br>
>> > >>
_______________________________________________<br>
>> > >> L2Ork-dev mailing list<br>
>> > >> <a href="mailto:L2Ork-dev@disis.music.vt.edu" target="_blank">L2Ork-dev@disis.music.vt.edu</a><br>
>> > >> <a href="https://disis.music.vt.edu/listinfo/l2ork-dev" rel="noreferrer" target="_blank">https://disis.music.vt.edu/listinfo/l2ork-dev</a><br>
>> > ><br>
>> > ><br>
>> > ><br>
>> > > --<br>
>> > > Dr. Albert Gr"af<br>
>> > > Computer Music Research Group, JGU Mainz,
Germany<br>
>> > > Email: <a href="mailto:aggraef@gmail.com" target="_blank">aggraef@gmail.com</a>, web: <a href="https://agraef.github.io/" rel="noreferrer" target="_blank">https://agraef.github.io/</a><br>
>> > >
_______________________________________________<br>
>> > > L2Ork-dev mailing list<br>
>> > > <a href="mailto:L2Ork-dev@disis.music.vt.edu" target="_blank">L2Ork-dev@disis.music.vt.edu</a><br>
>> > > <a href="https://disis.music.vt.edu/listinfo/l2ork-dev" rel="noreferrer" target="_blank">https://disis.music.vt.edu/listinfo/l2ork-dev</a><br>
>> _______________________________________________<br>
>> L2Ork-dev mailing list<br>
>> <a href="mailto:L2Ork-dev@disis.music.vt.edu" target="_blank">L2Ork-dev@disis.music.vt.edu</a><br>
>> <a href="https://disis.music.vt.edu/listinfo/l2ork-dev" rel="noreferrer" target="_blank">https://disis.music.vt.edu/listinfo/l2ork-dev</a><br>
><br>
><br>
><br>
> --<br>
> Dr. Albert Gr"af<br>
> Computer Music Research Group, JGU Mainz, Germany<br>
> Email: <a href="mailto:aggraef@gmail.com" target="_blank">aggraef@gmail.com</a>,
web: <a href="https://agraef.github.io/" rel="noreferrer" target="_blank">https://agraef.github.io/</a><br>
> _______________________________________________<br>
> L2Ork-dev mailing list<br>
> <a href="mailto:L2Ork-dev@disis.music.vt.edu" target="_blank">L2Ork-dev@disis.music.vt.edu</a><br>
> <a href="https://disis.music.vt.edu/listinfo/l2ork-dev" rel="noreferrer" target="_blank">https://disis.music.vt.edu/listinfo/l2ork-dev</a><br>
_______________________________________________<br>
L2Ork-dev mailing list<br>
<a href="mailto:L2Ork-dev@disis.music.vt.edu" target="_blank">L2Ork-dev@disis.music.vt.edu</a><br>
<a href="https://disis.music.vt.edu/listinfo/l2ork-dev" rel="noreferrer" target="_blank">https://disis.music.vt.edu/listinfo/l2ork-dev</a></blockquote>
</div>
<br clear="all">
<br>
-- <br>
<div dir="ltr">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">Dr. Albert Gr"af<br>
Computer Music Research Group, JGU Mainz, Germany<br>
Email: <a href="mailto:aggraef@gmail.com" target="_blank">aggraef@gmail.com</a>,
web: <a href="https://agraef.github.io/" target="_blank">https://agraef.github.io/</a></div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
L2Ork-dev mailing list
<a href="mailto:L2Ork-dev@disis.music.vt.edu" target="_blank">L2Ork-dev@disis.music.vt.edu</a>
<a href="https://disis.music.vt.edu/listinfo/l2ork-dev" target="_blank">https://disis.music.vt.edu/listinfo/l2ork-dev</a></pre>
</blockquote>
<br>
</div>
_______________________________________________<br>
L2Ork-dev mailing list<br>
<a href="mailto:L2Ork-dev@disis.music.vt.edu" target="_blank">L2Ork-dev@disis.music.vt.edu</a><br>
<a href="https://disis.music.vt.edu/listinfo/l2ork-dev" rel="noreferrer" target="_blank">https://disis.music.vt.edu/listinfo/l2ork-dev</a></blockquote></div><br clear="all"><br>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr">Dr. Albert Gr"af<br>Computer Music Research Group, JGU Mainz, Germany<br>Email: <a href="mailto:aggraef@gmail.com" target="_blank">aggraef@gmail.com</a>, web: <a href="https://agraef.github.io/" target="_blank">https://agraef.github.io/</a></div></div></div></div></div></div>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr">Dr. Albert Gr"af<br>Computer Music Research Group, JGU Mainz, Germany<br>Email: <a href="mailto:aggraef@gmail.com" target="_blank">aggraef@gmail.com</a>, web: <a href="https://agraef.github.io/" target="_blank">https://agraef.github.io/</a></div></div></div></div></div></div></div>