[L2Ork-dev] New developer interested in doing a GSOC18 project

Jonathan Wilkes jon.w.wilkes at gmail.com
Mon Mar 26 09:18:54 EDT 2018 

Hi Pranay,

On Mon, Mar 26, 2018 at 4:44 AM, pranay gupta
<pranayguptastudent at gmail.com> wrote:
>
> Hello Jonathan,
> Thanks for the review. The pointer thing in the aim was a typo on my part and I have already corrected it in my proposal draft.
>
> About the external libraries, I would like to stick to the first approach that you mentioned. i.e prioritise all those libraries that get loaded by default by the libdir_loader. We can extend this once the project gets selected, to the other approaches that you mentioned.
> Below is the list of externals that get loaded, in order.
>
> cyclone
> zexy
> creb
> cxc
> iemlib
> mapping
> markex
> maxlib
> memento
> mjlib
> motex
> oscx
> pddp
> pdogg
> pixeltango
> rradical
> sigpack
> smlib
> unauthorized
> pan
> hcs
> jmmmp
> ext13
> ggee
> ekext
> disis
> lyonpotpourri
> pdlua
>
> Could you please tell me if this list was missing anything?

I don't think Gem is listed there. It ships with the Windows and
Linux binaries but not with OSX.

Other than that, everything should be there.

>
> Also I had one more doubt regarding this, (this may be a pretty beginner doubt)
>
> 1) When I start purr-data after loading the pd-lua library the console throws an error:audio I/O dropout.

I'm honestly not sure about that error.

On the one hand, libraries are loaded at startup synchronously in a
loop. So one would
assume that this would take long enough to cause the backend to miss a
DSP scheduling
deadline and thus output the error.

On the other hand, I don't get that error on my Chromebook with Purr
Data 2.4.9 which
did not ship with pdlua.

My guess is that all the other libraries simply search for a metadata
file in Purr Data's
paths and then add a path when it gets found. Pdlua instead dynamically loads a
library which perhaps takes longer and causes a dropout. (Although 20 libraries
using I/O calls to search through paths for a metafile seems like it
would cause a
dropout, too.)

Anyhow, I'd ignore that error at startup because nobody expects Purr Data to
deliver realtime audio while starting up anyway.

>
> 2) The programs work fine though.
>
>
> 3) Also, on running the external_tests.pd file it shows that all tests are passing.
> I was not able to figure out a reason behind this, It would be great, if you could please help me with this.

Similar to above, that test tries to load all libraries it can find and then
instantiate every object for which it found a constructor-- synchronously.

It usually takes a few seconds. So the scheduler checks the time and sees
that we're a few seconds behind, then prints out the error.

In the CI the tests are run with the "-noaudio" flag (among others) so that
error doesn't happen. But when you run it manually you'll see it.

-Jonathan

>
> Thanks ,
>
> Sincerely,
> Pranay Gupta
>
>
>
>
> On Mon, Mar 26, 2018 at 2:52 AM, Jonathan Wilkes <jon.w.wilkes at gmail.com> wrote:
>>
>> Hi Pranay,
>>
>> On Sun, Mar 25, 2018 at 4:39 PM, pranay gupta
>> <pranayguptastudent at gmail.com> wrote:
>> > Hello Jonathan,
>> > I have started with the proposal and I had a few doubts,
>> > I wanted to ask you if there is a specific list of external libraries that
>> > you want me to include in this project?
>>
>> I don't have a specific list at the moment. However there are a
>> few ways I can think of to filter it down to a manageable set.
>>
>> The first is to focus on the libraries which get loaded by default in
>> the current version of Purr Data we ship. Currently each of these
>> libraries gets loaded by the "libdir loader" which outputs a message
>> to the console at startup:
>>
>> libdir_loader: added 'hcs' to the global objectclass path
>>
>> Second would be to look at the libraries that get installed using
>> the new toplevel make target `make light` and prioritize those,
>> as they are the minimum number needed to have a minimal
>> install.
>>
>> Third would be for the users and devs to tell us which ones
>> they prioritize. But that would be something to do later if the
>> project gets accepted and work on the core has begun.
>>
>> Finally, I think checking c and c++ files in purr-data/externals
>> to see how many currently use "float" instead of "t_float" atm.
>>
>> > Also can you please explain to me a little bit about the testing of
>> > purr-data and the in general?
>>
>> Currently the tests are quite basic-- we check to make sure that each
>> external library will properly load, and that we can instantiate objects
>> from each library without crashes or memory error (using valgrind).
>>
>> Without too much trouble, this can be extended to make sure that
>> each external library which performs DSP computation can actually
>> do so without crashing or memory leaks.
>>
>> There was a project idea for more robust testing but that obviously hasn't
>> been implemented yet.
>>
>> > Also I have attached a link to my proposal draft below. Could you please
>> > review it and tell me if I am going in the right direction with it and how
>> > can I further improve it?
>>
>> That definitely looks like the right direction to me.
>>
>> Under "Aim": you refer to "single precision and double precision
>> pointer." But the
>> fundamental change is to the underlying type for representing numbers in Purr
>> Data, not the pointer size. (Although pointer sizes will obviously change as a
>> consequence of this.)
>>
>> I'm mainly concerned with avoiding ambiguity between this and, say, a project
>> that aims to change an app to run on an architecture where the word size is
>> twice as large.
>>
>> -Jonathan
>>
>> >
>> > Thanks.
>> >
>> > Link to proposal
>> >
>> > Sincerely,
>> > Pranay Gupta
>> >
>> > On Sat, Mar 24, 2018 at 3:27 AM, Jonathan Wilkes <jon.w.wilkes at gmail.com>
>> > wrote:
>> >>
>> >> Hi Pranay,
>> >>
>> >> I'll respond point by point for clarity below.
>> >>
>> >> On Fri, Mar 23, 2018 at 1:33 PM, pranay gupta
>> >> <pranayguptastudent at gmail.com> wrote:
>> >> > Hello Jonathan,
>> >> >
>> >> > I have gone through all the mails and tried to go through some files
>> >> > also,
>> >> > what I understood was that, wherever algorithms have been implemented
>> >> > using
>> >> > single precision I have to change that to double precision and some
>> >> > external
>> >> > libraries that are being called in the core functions might also need
>> >> > changes.
>> >>
>> >> There are basically two categories of work: the core and the external
>> >> libraries (essentially plugins) which ship with Purr Data.
>> >>
>> >> For the core there is prior art with the Pd Double project which can
>> >> be found here:
>> >>
>> >> https://github.com/pd-projects/pd-double
>> >>
>> >> One of the challenges here is that our core code has diverged
>> >> substantially
>> >> from that codebase. However, the overall structure of the code is still
>> >> very
>> >> similar as are the core DSP algorithms and message dispatching/parsing
>> >> code.
>> >>
>> >> The second category-- external libraries-- is a bit more
>> >> difficult because the quality varies considerably for each library.
>> >> Some of them are straightfoward and just require recompiling
>> >> and testing. Some freely mix "t_float" and "float", probably under
>> >> the assumption that the "t_float" type would never change. Still
>> >> others may use type-punning or other tricks that depend on
>> >> "t_float" being equal to "float".
>> >>
>> >> I think a workable approach here is to prioritize a handful of these
>> >> libraries to make ready for double precision and then document
>> >> common types of problems for the rest. That is to say I don't think
>> >> this project idea needs to aim to make every single library ready
>> >> for double precision.
>> >>
>> >> > Also, support for backward compatibility, and an option to the
>> >> > users whether to run purr date using single precision or double
>> >> > precision
>> >> > should be added.
>> >>
>> >> I don't think this is possible. The public API essentially
>> >> requires external library developers to typecast raw pointers to
>> >> arrays of t_float. So either everything is compiled with a
>> >> uniform t_float value, or you get either:
>> >>
>> >> a) data corruption (double-precision mode with single-instance library
>> >> loaded)
>> >> b) data corruption and buffer overflow (single-precision mode with
>> >> double-precision lib)
>> >>
>> >> I'm actually just guessing on the type of errors-- there may be
>> >> some part of the API I'm forgetting which causes crashes in both cases.
>> >>
>> >> > And in case any of the single precision or double precision
>> >> > catches an error, the one that works has to be used.
>> >>
>> >> As above, I don't think that's possible. But we can test both
>> >> precision types in the CI (perhaps switching out one of the older
>> >> Ubuntu runners for a single precision build runner).
>> >>
>> >> > Also, you mentioned
>> >> > about single precision binaries performing better with better algorithms
>> >> > as
>> >> > compared to double precision with a basic implementation when compared
>> >> > to
>> >> > vanilla binaries as per the results of some tests.
>> >>
>> >> I didn't measure any difference on my Rockchip Chromebook.
>> >> But I'd say it would be a good idea to set up some performance
>> >> tests to measure the differences.
>> >>
>> >> >
>> >> > I had a few doubts regarding the project,
>> >> >
>> >> > 1) What should be the main points that I should focus more on in my
>> >> > proposal?
>> >>
>> >> Probably on making it possible to compile Purr Data for both types
>> >> of precision, to make sure the core runs reliably in each case, and
>> >> to make sure some of the more popular external libraries run reliably
>> >> in each and document a process for testing/working on the rest.
>> >>
>> >> >
>> >> > 2) Also will any of the algorithms that are already implemented require
>> >> > any
>> >> > implementation changes when adding support for double precision?
>> >>
>> >> In the core-- yes. These were replaced and tested with new implementations
>> >> in the pd double project. I *think* they are the files assigned to the
>> >> TYPE_PUNNING_SRC variable in purr-data/pd/src/makefile.in.
>> >>
>> >> >
>> >> > 3) And it would be really helpful if you could provide me a few
>> >> > resources
>> >> > that might make it easier for me to understand the project better?
>> >>
>> >> General resources or specific to this project idea?
>> >>
>> >> For the former-- here's a great resource Albert created:
>> >>
>> >> https://agraef.github.io/purr-data-intro/
>> >>
>> >> Let us know if you have any more questions.
>> >>
>> >> -Jonathan
>> >> _______________________________________________
>> >> L2Ork-dev mailing list
>> >> L2Ork-dev at disis.music.vt.edu
>> >> https://disis.music.vt.edu/listinfo/l2ork-dev
>> >
>> >
>> >
>> > _______________________________________________
>> > L2Ork-dev mailing list
>> > L2Ork-dev at disis.music.vt.edu
>> > https://disis.music.vt.edu/listinfo/l2ork-dev
>> _______________________________________________
>> L2Ork-dev mailing list
>> L2Ork-dev at disis.music.vt.edu
>> https://disis.music.vt.edu/listinfo/l2ork-dev
>
>
>
> _______________________________________________
> L2Ork-dev mailing list
> L2Ork-dev at disis.music.vt.edu
> https://disis.music.vt.edu/listinfo/l2ork-dev

More information about the L2Ork-dev mailing list