[L2Ork-dev] regression tests as part of the build

[Jonathan Wilkes]

On Tue, Sep 15, 2020 at 6:04 PM Albert Graef <aggraef at gmail.com> wrote:
>
> Hi Jonathan,
>
> On Tue, Sep 15, 2020 at 5:32 PM Jonathan Wilkes <jon.w.wilkes at gmail.com> wrote:
>>
>> 1. Does that mean that now when a user runs `make` the tests get run?
>
>
> No, they're separate targets, but that's customary. You wouldn't want the tests to be mandatory like that. If you want to test right after the build, just run `make all check`, simple as that. There's also the possibility to run `make all check1` if you only want to do the basic regression test.

Ah, I see. So now I can simplify the CI instructions by doing "make
all check" right after the build. Nice.

>
>> 2. If not, can we change the build system so that it runs the tests as
>> part of the build?
>
>
> See 1. :)

Ok.

>
>> I guess the one tricky part is `make light`. It wouldn't succeed on
>> the external tests.
>
>
> It already fails regression_tests.pd, because that test invokes the cxc/hex2dec external which the light build doesn't include. So `make light check1` returns a nonzero exit status.
>
> The idea of the light build is that it only includes the absolutely essential externals so that Purr Data will work and nothing else (getting as close to a vanilla-like package as possible), and I'd really like it to stay that way.
>
> I'd suggest to simply add a stripped-down version of regression_tests.pd, say regression_tests0.pd, that doesn't need cxc/hex2dec (or maybe replaces that part of the test with something else that's in the core). Then I can add another check0 target, so that you could run `make light check0` to build and test that version.

Let me think a bit about the tests some more. There's already a UX
problem because one has to add a test patch *and* create an
abstraction in regression_tests.pd. It would be much better to be able
to add the test patch and have regression_tests.pd run whatever is in
the directory.

-Jonathan



>
> Albert
>
> --
> Dr. Albert Gr"af
> Computer Music Research Group, JGU Mainz, Germany
> Email: aggraef at gmail.com, web: https://agraef.github.io/
> _______________________________________________
> L2Ork-dev mailing list
> L2Ork-dev at disis.music.vt.edu
> https://disis.music.vt.edu/listinfo/l2ork-dev