[L2Ork-dev] New scrollbars and other commits, and next steps (was: Excalibur bug)
Jonathan Wilkes
jon.w.wilkes at gmail.com
Mon Jun 15 12:26:53 EDT 2020
Responding inline:
On Sun, Jun 14, 2020 at 10:07 PM Ivica Ico Bukvic <ico at vt.edu> wrote:
>
> OK, two additional merge requests are now up. While I am reasonably
> certain that each commit is self-contained, when it comes to the
> scrollbars, just to be sure, I recommend merging all of my merge requests.
>
> Please note the scrollbar fix is specific to the 0.46.2 branch which in
> my opinion does significantly better in the area of scrollbars but does
> also introduce a few cosmetic issues that will need to be resolved,
> namely dialogs appear not to be centered on their parent window, which I
> suspect is hopefully a simple fix. Also, the menu is white, which I kind
> of like. As far as I can tell the new scrollbars account for all the
> issues we identified so far, including native functionality, namely:
>
> *ability to update whenever the canvas changes (subject to the same
> update rate as the existing implementation, although I seem to recall
> bumping the update rate to 50ms, down from 250ms).
> *ability to detect maximize/restore.
> *scrollbars update when scrolling and resizing.
> *scrollbars are individually clickable and draggable.
> *scrollbars automatically resize to reflect how much room is left to
> either side.
> *scrollbars do their best to maintain the same size between the zoom
> levels (there is some loss due to the fact that: 1) current zoom levels
> are supposedly matching that of chrome, but that does not seem to be
> consistently the case for the negative zoom levels, 2) float point
> (im)precision)
> *the scrolling can be also invoked using the native middle click and
> move (like the browser does).
> *I also wanted to implement the click/grab and drag to scroll feature
> that 1.x has but in order for this to remain consistent across all zoom
> levels, I was unable to find a perfect ratio multiplier, so instead I
> reverted it to the native middle click which seems to work just as well,
> if not better
>
> In addition, the new implementation allows for:
> *ability to stay flush with the patch objects with 0px padding.
> *no race conditions when having situations where scrollbars fight due to
> one stealing width/height from the docuemnt.
> *improved UI consistency across platforms.
> *use of the entire side of the screen.
> *semi-transparency, so you can still see under the scrollbars.
> *responsiveness to the deselect command (and as a result potential
> decrease in the bbox due to the disappearance of an iemgui handle).
Aside from seeing underneath the scrollbar, I think my fix to the excalibur
bug delivers most of these features. Of course it's possible your implementation
does essentially the same as my fix in do_getscroll, so I'll have a look and
see.
>
> Next, we may want to consider working on the following:
> 1) This one will benefit from Jonathan's help: reworking how the
> listeners are added and removed when the scrollbars are rendered visible
> and invisible (currently they are mapped until the patch is closed).
> This is mainly because I still don't quite understand how the event
> listener system works inside the pd_canvas.js. I was also unable to
> locate resize listener in there. Note that pd_canvas.html has noscroll
> and onresize inside its body tag, which may also benefit from being
> integrated into the js file.
>
> 2) figuring out a way where clicking on the scrollbar prevents the click
> to propagate onto the canvas. I noticed this when there was an array
> under it and I ended-up moving both.
I may have a fix for that.
>
> 3) Once these two little things are figured out, we may want to figure
> out finishing implementing other 1.x window features, such as window
> transparency, noresize, ontop, and disable maximize (maximize and
> transparency was not used extensively nor properly documented). For
> this, I would love to learn how once could send messages directly to
> window, effectively invoking functions inside the pdgui.js, for
> instance. 1.x had a hcs external that did this. Does this "just work"
> inside 2.x or is there something we need to add to make that possible?
I'm very reluctant to do that because it turns the entire GUI into a public
interface.
For example-- if I had already done that it would make moving away from
the nw.js api difficult because every user would have some reason that their
patch must call directly into one of nw.js's methods.
On the other hand, we're currently working on this webapp frontend for
GSoC. So it may be helpful as part of that to consider what exactly the
user would need to access in the GUI that they can't currently do. Then
we could start gathering up our own curated public API.
In other words, a filter could be-- can the thing be accessed both in native
Purr Data and in the webapp? That would exclude things like accessing
the filesystem through the GUI, for example.
Still, I think it makes more sense to build in a hook for plugins written
in javascript rather than accessing the GUI through a Pd object.
>
> 4) Ensure that the zooms are consistent across all platforms, so that
> what fits on one edge-to-edge, looks identical on another. I suspect
> this could be a simple albeit time-consuming matter of tweaking zoom values.
I didn't know they weren't. Did you post an issue to the tracker about this?
>
> 5) Once all this is done, I would suggest merging Tweeter (once I update
> it) and then we can use this as an additional carrot to engage new
> potential users.
Sounds good.
>
> Other bugs I noticed that still need attention:
>
> *aforesaid dealing with off-center dialogs
> *tweaking the window theme (0.46.2 looks different for some reason)
> *ensuring that dialogs are not resizable
> *reintroducing iemgui features 1.x has, such as ability for the number2
> object to not have a frame or the > next to the number (used by K12)
> *fixing weird sizes of the various dialog windows and ensuring that they
> are not resizable.
> *dynamic addition and removal of the event listeners for the scrollbars
> and integration of the onscroll and onresize inside pd_canvas.js as
> opposed to the pd_canvas.thml (as noted above)
> *iemguis still have legacy issues when using properties/apply and having
> a label reposition (I believe Jonathan is already working on this)
You might check this since I merged fix-iemgui-label-bug
> *subpatch with an array needs to be properly titled (right now it has no
> title).
> *subpatch plots need to be properly sized to go from edge to edge.
> *fix slow console (e.g. by limiting the number of printed lines?)
> *add ability to add \n to objects and their labels
> *coll ability to dynamically monitor external file or edit the file via gui
>
> And new features (in addition to the GSoC efforts, listed in no
> particular order):
>
> *Catching-up with the vanilla features, where applicable
> *K12 integration
> *Tooltips
> *New HTML-based menu (this should be actually a lot of fun and could
> end-up looking really cool)
For that last one-- can we leverage whatever Hugo is creating for the webapp?
Related-- I'd really like to set up native Purr Data with a flag to run using
the single-page webapp style Hugo is working on. That's just a much better
UX than the toplevel window thing Pd has inherited from 90s motif.
As for the rest-- for the issues that aren't currently on the tracker,
please add
them.
-Jonathan
>
> Best,
>
> Ico
>
> On 6/14/2020 5:29 PM, Jonathan Wilkes wrote:
> > On Sun, Jun 14, 2020 at 5:01 PM Ivica Bukvic <ico at vt.edu> wrote:
> >> True. although if we know what the menu bar will be in terms of size (which appears to be fluctuating 0.46.2 is 25 pixels whereas 0.14.7 is 23, go figure), we shouldn't have to worry about that race condition as long as we assign it proper size and then do not worry about scroll bars because they are already handled within HTML5.
> > It may be possible to fetch it at runtime with the Pd Window. E.g.,
> > fetch the height, create the menu, then set a
> > timeout that's long enough that the race is over.
> >
> > Still, you'll get buggy scrollbar appearance because we cannot trust
> > the initial height values reported for the window.
> >
> > To make it clear-- suppose someone runs `pd-l2ork t*-help.pd`. The
> > timeout duration necessary to ensure that the
> > race doesn't screw up the scrollbars on once of those patches is so
> > long that it would ruin the UX to implement it.
> >
> > -Jonathan
> >
> >
> >
> >
> >
> >> The next step then is to remove the NW menu and build the entire thing in the HTML which shouldn't be too hard
> >>
> >> Best,
> >>
> >> Ico
> >>
> >> --
> >> Ivica Ico Bukvic, D.M.A.
> >> Director, Creativity + Innovation
> >> Institute for Creativity, Arts, and Technology
> >>
> >> Virginia Tech
> >> Creative Technologies in Music
> >> School of Performing Arts – 0141
> >> Blacksburg, VA 24061
> >> (540) 231-6139
> >> ico at vt.edu
> >>
> >> www.icat.vt.edu
> >> www.performingarts.vt.edu
> >> l2ork.icat.vt.edu
> >> ico.bukvic.net
> >>
> >> On Sun, Jun 14, 2020, 16:20 Jonathan Wilkes <jon.w.wilkes at gmail.com> wrote:
> >>> On Sun, Jun 14, 2020 at 3:55 PM Ivica Bukvic <ico at vt.edu> wrote:
> >>>> One response I got on the nw group chat which is something you also mentioned to me once is that we should imppement as much as possible inside HTML5 and forgo reliance on the often buggy nw.
> >>> I agree, but the buggy part of nw is it's cross-platform native window
> >>> menus. The scrollbar behavior is part of HTML5.
> >>>
> >>>> The implementation of custom scrollbars I'm about to finish does exactly that. Not only will it look identical on all platforms but will also be completely oblivious to the race condition you are encountering. This will be even more apparent once we get rid of the node webkit menu bar which is been causing so many problems.
> >>> For this bug it doesn't matter if we use native scrollbars or code
> >>> them manually. If we don't know that the window height
> >>> value we retrieve after load is correct, *and* we don't know how long
> >>> to wait until it becomes correct, we're screwed.
> >>>
> >>> -Jonathan
> >>>
> >>>> Best,
> >>>>
> >>>> Ico
> >>>>
> >>>> --
> >>>> Ivica Ico Bukvic, D.M.A.
> >>>> Director, Creativity + Innovation
> >>>> Institute for Creativity, Arts, and Technology
> >>>>
> >>>> Virginia Tech
> >>>> Creative Technologies in Music
> >>>> School of Performing Arts – 0141
> >>>> Blacksburg, VA 24061
> >>>> (540) 231-6139
> >>>> ico at vt.edu
> >>>>
> >>>> www.icat.vt.edu
> >>>> www.performingarts.vt.edu
> >>>> l2ork.icat.vt.edu
> >>>> ico.bukvic.net
> >>>>
> >>>> On Sun, Jun 14, 2020, 15:51 Jonathan Wilkes <jon.w.wilkes at gmail.com> wrote:
> >>>>> On Sun, Jun 14, 2020 at 2:59 PM Ivica Bukvic <ico at vt.edu> wrote:
> >>>>>> Why not simply subtract the menu at height from the document height?
> >>>>> It's probably different on Windows and Linux, non-existent on OSX, possibly
> >>>>> different on different Linux distros.
> >>>>>
> >>>>> Also it appears to be a race-- so it's possible a big beefy setup could win the
> >>>>> race and now they can't click in the bottom 15 pixels of the patch.
> >>>>>
> >>>>> -Jonathan
> >>>>>
> >>>>>> Best,
> >>>>>>
> >>>>>> Ico
> >>>>>>
> >>>>>> --
> >>>>>> Ivica Ico Bukvic, D.M.A.
> >>>>>> Director, Creativity + Innovation
> >>>>>> Institute for Creativity, Arts, and Technology
> >>>>>>
> >>>>>> Virginia Tech
> >>>>>> Creative Technologies in Music
> >>>>>> School of Performing Arts – 0141
> >>>>>> Blacksburg, VA 24061
> >>>>>> (540) 231-6139
> >>>>>> ico at vt.edu
> >>>>>>
> >>>>>> www.icat.vt.edu
> >>>>>> www.performingarts.vt.edu
> >>>>>> l2ork.icat.vt.edu
> >>>>>> ico.bukvic.net
> >>>>>>
> >>>>>> On Sun, Jun 14, 2020, 14:50 Jonathan Wilkes <jon.w.wilkes at gmail.com> wrote:
> >>>>>>> It appears there's a race condition with the nw.js Menu menubar
> >>>>>>> rendering and nw.js actually
> >>>>>>> updating the DOM.
> >>>>>>>
> >>>>>>> This means we have to rely on a timeout to get accurate window height
> >>>>>>> report, and for
> >>>>>>> some heavy patches this won't be enough.
> >>>>>>>
> >>>>>>> That's why you get the extra height at the bottom of the window for
> >>>>>>> some help patches.
> >>>>>>>
> >>>>>>> I just filed a bug on nw.js google group. Hopefully Roger can fix it
> >>>>>>> our at least give us a
> >>>>>>> callback when it's done rendering.
> >>>>>>>
> >>>>>>> -Jonathan
> >>>>>>>
> >>>>>>> On Sun, Jun 14, 2020 at 1:09 AM Jonathan Wilkes <jon.w.wilkes at gmail.com> wrote:
> >>>>>>>> Excalibur successfully retrieved.
> >>>>>>>>
> >>>>>>>> And the secret CSS incantation is...
> >>>>>>>>
> >>>>>>>> float: left;
> >>>>>>>>
> >>>>>>>> Initial implementation below. I've got the scrollbars hard-coded atm,
> >>>>>>>> but they can easily be measured at startup.
> >>>>>>>>
> >>>>>>>> Also-- I added some overflow offsets so that if a scrollbar is needed,
> >>>>>>>> then there is some padding below the object.
> >>>>>>>> That always drives me crazy when I scroll down and the bottom of the
> >>>>>>>> object is flush with the scrollbar or the
> >>>>>>>> bottom of the window.
> >>>>>>>>
> >>>>>>>> diff --git a/pd/nw/pdgui.js b/pd/nw/pdgui.js
> >>>>>>>> index a24a6f71..7ff9fdab 100644
> >>>>>>>> --- a/pd/nw/pdgui.js
> >>>>>>>> +++ b/pd/nw/pdgui.js
> >>>>>>>> @@ -5774,8 +5774,8 @@ function canvas_params(nw_win)
> >>>>>>>> // the scrollbars from appearing. Here, we just subtract 4 from both
> >>>>>>>> // of them. This could lead to some problems with event handlers but I
> >>>>>>>> // haven't had a problem with it yet.
> >>>>>>>> - min_width = nw_win.window.innerWidth - 4;
> >>>>>>>> - min_height = nw_win.window.innerHeight - 4;
> >>>>>>>> + min_width = nw_win.window.innerWidth;
> >>>>>>>> + min_height = nw_win.window.innerHeight;
> >>>>>>>> // Since we don't do any transformations on the patchsvg,
> >>>>>>>> // let's try just using ints for the height/width/viewBox
> >>>>>>>> // to keep things simple.
> >>>>>>>> @@ -5799,24 +5799,82 @@ function do_getscroll(cid) {
> >>>>>>>> // errors wrt the rendering context disappearing.
> >>>>>>>> gui(cid).get_nw_window(function(nw_win) {
> >>>>>>>> var svg_elem = nw_win.window.document.getElementById("patchsvg");
> >>>>>>>> + var overflow_x,
> >>>>>>>> + overflow_y,
> >>>>>>>> + scrollbar_x_would_overlap_content,
> >>>>>>>> + scrollbar_y_would_overlap_content,
> >>>>>>>> + x_offset = 0,
> >>>>>>>> + y_offset = 0;
> >>>>>>>> var { x: x, y: y, w: width, h: height,
> >>>>>>>> mw: min_width, mh: min_height } = canvas_params(nw_win);
> >>>>>>>> +
> >>>>>>>> + // Let's get to it. First we want to know if our SVG has any child
> >>>>>>>> + // elements overflowing our viewport
> >>>>>>>> +
> >>>>>>>> + overflow_x = width > min_width;
> >>>>>>>> + overflow_y = height > min_height;
> >>>>>>>> +
> >>>>>>>> + // Now let's see if drawing a scrollbar would hide objects at the
> >>>>>>>> + // extremities of the viewport
> >>>>>>>> +
> >>>>>>>> + scrollbar_x_would_overlap_content = height > min_height - 15;
> >>>>>>>> + scrollbar_y_would_overlap_content = width > min_width - 15;
> >>>>>>>> +
> >>>>>>>> + if (overflow_x) {
> >>>>>>>> + // We have a horizontal scrollbar.
> >>>>>>>> + if (scrollbar_x_would_overlap_content) {
> >>>>>>>> + // If there are objects underneath the horizontal scrollbar,
> >>>>>>>> + // we want to make sure that the height of our SVG covers
> >>>>>>>> + // the entire viewport. That way the browser will give us
> >>>>>>>> + // a vertical scrollbar in order to view those objects
> >>>>>>>> + y_offset = 0;
> >>>>>>>> + } else {
> >>>>>>>> + // We don't need a vertical scrollbar after all. So make the
> >>>>>>>> + // canvas 15 pixels shorter so the vertical scrollbar doesn't
> >>>>>>>> + // show up
> >>>>>>>> + y_offset = -15;
> >>>>>>>> + }
> >>>>>>>> + }
> >>>>>>>> +
> >>>>>>>> + if (overflow_y) {
> >>>>>>>> + // vert scrollbar
> >>>>>>>> + if (scrollbar_y_would_overlap_content) {
> >>>>>>>> + // same as above
> >>>>>>>> + x_offset = 0;
> >>>>>>>> + } else {
> >>>>>>>> + x_offset = -15;
> >>>>>>>> + }
> >>>>>>>> + }
> >>>>>>>> +
> >>>>>>>> + // Finally, if we overflow the viewport, let's add some padding so
> >>>>>>>> + // that objects at the edge aren't flush against the scrollbar. At
> >>>>>>>> + // least I find that a real pain at the bottom of the window when
> >>>>>>>> + // trying to add more objects
> >>>>>>>> +
> >>>>>>>> + if (overflow_x) {
> >>>>>>>> + x_offset = 5;
> >>>>>>>> + }
> >>>>>>>> +
> >>>>>>>> + if (overflow_y) {
> >>>>>>>> + y_offset = 5;
> >>>>>>>> + }
> >>>>>>>> +
> >>>>>>>> if (width < min_width) {
> >>>>>>>> width = min_width;
> >>>>>>>> }
> >>>>>>>> - // If the svg extends beyond the viewport, it might be nice to pad
> >>>>>>>> - // both the height/width and the x/y coords so that there is extra
> >>>>>>>> - // room for making connections and manipulating the objects. As it
> >>>>>>>> - // stands objects will be flush with the scrollbars and window
> >>>>>>>> - // edges.
> >>>>>>>> + width += x_offset;
> >>>>>>>> +
> >>>>>>>> if (height < min_height) {
> >>>>>>>> - height = min_height;
> >>>>>>>> + height = min_height
> >>>>>>>> }
> >>>>>>>> + height += y_offset;
> >>>>>>>> +
> >>>>>>>> configure_item(svg_elem, {
> >>>>>>>> viewBox: [x, y, width, height].join(" "),
> >>>>>>>> width: width,
> >>>>>>>> height: height
> >>>>>>>> });
> >>>>>>>> +
> >>>>>>>> });
> >>>>>>>> }
> >>>>>>> _______________________________________________
> >>>>>>> L2Ork-dev mailing list
> >>>>>>> L2Ork-dev at disis.music.vt.edu
> >>>>>>> https://disis.music.vt.edu/listinfo/l2ork-dev
> >>>>>> _______________________________________________
> >>>>>> L2Ork-dev mailing list
> >>>>>> L2Ork-dev at disis.music.vt.edu
> >>>>>> https://disis.music.vt.edu/listinfo/l2ork-dev
> >>>>> _______________________________________________
> >>>>> L2Ork-dev mailing list
> >>>>> L2Ork-dev at disis.music.vt.edu
> >>>>> https://disis.music.vt.edu/listinfo/l2ork-dev
> >>>> _______________________________________________
> >>>> L2Ork-dev mailing list
> >>>> L2Ork-dev at disis.music.vt.edu
> >>>> https://disis.music.vt.edu/listinfo/l2ork-dev
> >>> _______________________________________________
> >>> L2Ork-dev mailing list
> >>> L2Ork-dev at disis.music.vt.edu
> >>> https://disis.music.vt.edu/listinfo/l2ork-dev
> >> _______________________________________________
> >> L2Ork-dev mailing list
> >> L2Ork-dev at disis.music.vt.edu
> >> https://disis.music.vt.edu/listinfo/l2ork-dev
> > _______________________________________________
> > L2Ork-dev mailing list
> > L2Ork-dev at disis.music.vt.edu
> > https://disis.music.vt.edu/listinfo/l2ork-dev
>
> --
> Ivica Ico Bukvic, D.M.A.
> Director, Creativity + Innovation
> Institute for Creativity, Arts, and Technology
>
> Virginia Tech
> Creative Technologies in Music
> School of Performing Arts – 0141
> Blacksburg, VA 24061
> (540) 231-6139
> ico at vt.edu
>
> www.icat.vt.edu
> www.performingarts.vt.edu
> l2ork.icat.vt.edu
> ico.bukvic.net
>
More information about the L2Ork-dev
mailing list