Developing validation and testing tools for Jack

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Developing validation and testing tools for Jack

Stéphane Letz
Hi all,

Starting next week, I will have a student for 3 months, working on  
the development of a set of validation and testing tools for Jack.  
Here are some preliminary ideas I have about what tools could be  
interesting to have:

1) Jack API validation: the idea is to develop test applications that  
can validate the Jack API by testing if each function does what it it  
supposed to do. Including all kind of "limit" cases: two slow  
clients, blocked clients... This would be particularly helpful when a  
"big change" is done in the in Jack code, to test that nothing is  
broken.

2) Performance testing tools:  the idea is to develop test  
applications to measure how a set of Jack client behave in different  
conditions: with jackd of (new) jackdmp implementations, with  
different synchronization primitives (fifo, POSIX semaphores...), in  
different connections topology...

3) Jack application validation: the idea is to develop a code layer  
between the real libjack and the application to test that an  
application correctly use the Jack API: follow the correct order when  
using Jack functions, correctly unregister the different ressources  
(ports, client...) [ Yes the "jack_client_close" function is supposed  
to be called but some Jack applications still do not !! ), possibly  
test the correct real-time behavior of the audio process code,  
interaction of state change callback (graph_reorder...) with the main  
application code...

I know that some of you jack developers have already deveIoped this  
kind of tools. It would be interesting to collect them as a starting  
point. I would also be interested to have comments on the 3  
propositions and all other good ideas you will surely have!

Thanks

Stephane


-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server. Download
it for free - -and be entered to win a 42" plasma tv or your very own
Sony(tm)PSP.  Click here to play: http://sourceforge.net/geronimo.php
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel
Reply | Threaded
Open this post in threaded view
|

Re: Developing validation and testing tools for Jack

Rui Nuno Capela
Stéphane Letz wrote:

> Hi all,
>
> Starting next week, I will have a student for 3 months, working on  the
> development of a set of validation and testing tools for Jack.   Here
> are some preliminary ideas I have about what tools could be  interesting
> to have:
>
> 1) Jack API validation: the idea is to develop test applications that  
> can validate the Jack API by testing if each function does what it it  
> supposed to do. Including all kind of "limit" cases: two slow  clients,
> blocked clients... This would be particularly helpful when a  "big
> change" is done in the in Jack code, to test that nothing is  broken.
>
> 2) Performance testing tools:  the idea is to develop test  applications
> to measure how a set of Jack client behave in different  conditions:
> with jackd of (new) jackdmp implementations, with  different
> synchronization primitives (fifo, POSIX semaphores...), in  different
> connections topology...
>
> 3) Jack application validation: the idea is to develop a code layer  
> between the real libjack and the application to test that an  
> application correctly use the Jack API: follow the correct order when  
> using Jack functions, correctly unregister the different ressources  
> (ports, client...) [ Yes the "jack_client_close" function is supposed  
> to be called but some Jack applications still do not !! ), possibly  
> test the correct real-time behavior of the audio process code,  
> interaction of state change callback (graph_reorder...) with the main  
> application code...
>
> I know that some of you jack developers have already deveIoped this  
> kind of tools. It would be interesting to collect them as a starting  
> point. I would also be interested to have comments on the 3  
> propositions and all other good ideas you will surely have!
>
I think this one may fall under the 2) topic above. Yes, its my
dirty-brute-force-approach jack_test4 suite (on attachment).

It lacks the minimal documentation, besides this very note and some
older ones in the mail archives. It has been used to test and benchmark
the performance and development of Ingo's realtime-preempt kernel patch,
specially during its inception (roughly one year ago:).

The main script syntax is like so:

   ./jack_test4_run.sh [secs [clients [ports [period]]]]

where:

   secs    - number of secons of each test run (default 300)
   clients - number of test clients (14)
   ports   - number of i/o ports for each client (4)
   period  - period buffer size in frames (64)

Each test run launches:

   jackd -R -d alsa -d hw:0 -r 44100 -p<period> -n2

and then a given number of jack_test4_client's are started in chain.

The first one generates a 1KHz reference tone that is feeded to the
next, on each of the specified number of its output ports. Each client
syncs with each other and compares the input signal with a
self-generated one, looking for differences which should be zero at all
times. Besides that, it mixes all its inputs to the same number of
output ports which are then fed to the next client. The last client
feeds the whole result signal into the existing terminal/physical jackd
backend input ports, which is usually your soundcard speakers. So you
better be _careful_ with youtr soundcard main volume, as the whole thing
is intended to be audible (a pure, undistorted 1KHz tone).

When it terminates, a summmary is shown giving you a snapshot of your
jackd system behavior. A complete log file is tagged with your current
kernel version. If you have gnuplot installed, a ECG-like chart is also
produced.

Enjoy.
--
rncbc aka Rui Nuno Capela
[hidden email]


jack_test4.3.tar.gz (18K) Download Attachment