should zerocycles be treated as XRUNS?

classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|

should zerocycles be treated as XRUNS?

Florian Paul Schmidt-2

Hi,

During a discussion with Dave Cunningham on #lad today, which was
preceded by a discussion of Dave with Paul Davis, he mentioned to me the
problem of simply not running clients' process callbacks() at times when
the graph is locked and instead feeding a zero cycle to the audio hw.

The real problem lies in the fact that no client ever knows that the
zerocycle has happened. Naturally apps like jack_diplomat suffer from
this as it can severely screw their timings. But other classes of apps
might very well be affected (basically anything that relies on the
assumption that its process callback is actually called unless some
error condition has happened). As the zerocycle doesn't get reported to
any apps as an error state.. Even hd recorders can produce unnoticed
dropouts, cause a cycle or more has possibly simply disappeared.

So basically a quick simple hack to get around this would be to make
jack actually not trylock the graph, but lock it and hope for the best
:) If the timing gets screwed this would automatically get handled as an
xrun thus notifying all apps that something went wrong ;) If it was fast
enough, everything is fine, anyways.

Essentially a zerocycle is quite as bad as a "normal" xrun in some
regards, so it would be quite ok, to treat it with the same error state.

This would also provide a psychological impetus to all people
occasionally staring at jack code on how to make this source of xruns
disappear or at least less likely (by using waitfree techniques to make
the critical graph changing ops deterministic in time or simply faster)..

Simple "fix" kindly provided by Dave:


        if (jack_try_lock_graph (engine)) {
                // engine can't run. just throw away an entire cycle
                MESSAGE("Next XRUN possibly due to graph lock\n");
                jack_lock_graph (engine);
/*
                driver->null_cycle (driver, nframes);
                return 0;
*/      
        }

Thanks,
Florian Schmidt

--
Palimm Palimm!
http://tapas.affenbande.org


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel
Reply | Threaded
Open this post in threaded view
|

Re: should zerocycles be treated as XRUNS?

Florian Paul Schmidt-2
On Wed, 16 Nov 2005 01:11:33 +0100
Florian Schmidt <[hidden email]> wrote:

Yaya, i know it's bad style to reply to oneself, but i don't care :)

> This would also provide a psychological impetus to all people
> occasionally staring at jack code on how to make this source of xruns
> disappear or at least less likely (by using waitfree techniques to make
> the critical graph changing ops deterministic in time or simply faster)..

should have read: "(for example by using waitfree techniques to make the
critical graph changing ops deterministic in time or simply faster).."

>
> Simple "fix" kindly provided by Dave:
>
>
>         if (jack_try_lock_graph (engine)) {
>                 // engine can't run. just throw away an entire cycle
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The above comment should then, of course, disappear, as it's not correct
anymore

>                 MESSAGE("Next XRUN possibly due to graph lock\n");
>                 jack_lock_graph (engine);
> /*
>                 driver->null_cycle (driver, nframes);
>                 return 0;
> */      
>         }

Regards,
Flo

--
Palimm Palimm!
http://tapas.affenbande.org


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel
Reply | Threaded
Open this post in threaded view
|

Re: should zerocycles be treated as XRUNS?

Dave Robillard
In reply to this post by Florian Paul Schmidt-2
On Wed, 2005-16-11 at 01:11 +0100, Florian Schmidt wrote:

> Hi,
>
> During a discussion with Dave Cunningham on #lad today, which was
> preceded by a discussion of Dave with Paul Davis, he mentioned to me the
> problem of simply not running clients' process callbacks() at times when
> the graph is locked and instead feeding a zero cycle to the audio hw.
>
> The real problem lies in the fact that no client ever knows that the
> zerocycle has happened. Naturally apps like jack_diplomat suffer from
> this as it can severely screw their timings. But other classes of apps
> might very well be affected (basically anything that relies on the
> assumption that its process callback is actually called unless some
> error condition has happened). As the zerocycle doesn't get reported to
> any apps as an error state.. Even hd recorders can produce unnoticed
> dropouts, cause a cycle or more has possibly simply disappeared.

Can't you easily tell this happened in a client callback using
jack_last_frame_time (and nframes)?

-DR-




-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel
Reply | Threaded
Open this post in threaded view
|

Re: should zerocycles be treated as XRUNS?

Florian Paul Schmidt-2
On Wed, 16 Nov 2005 14:13:43 +1100
Dave Robillard <[hidden email]> wrote:

> Can't you easily tell this happened in a client callback using
> jack_last_frame_time (and nframes)?

yeah, but basically no client does this. jack_last_frame_time() isn't
even documented in the official api docs (on jackit.sf.net) yet (the
last time i asked to include it has been over a year ago IIRC).

Anyways, by way of the aforementioned fix, all apps that handle xruns
correctly will automagically work again (users of existing hd recorders
will know cycles have been dropped and apps that rely on regular calling
of their processs callbacks for timing purposes will also know something
went wrong).

Plus i think it's the Right Thing To Do (tm) as, as i said before, it
treats zerocycles as error condition explicitely as they should be and
not implicitly which is only possible to detect using an undocumented
relatively new api call.

Also actually this "fix" will decrease the number of zerocycles/xruns
due to graph locking, because in many cases it will just work out all
right in the end (i.e. a client that just reads the graph won't hold the
lock very long, so there's a good chance the contention will go away
quick enough for the process cycle to run. In the cases where it
actually produces xruns, one will

a] see it in the jack output b] and thus care about it and be able to
fix it by improving those functions that hold the graph lock too long.

Thirdly, a quick glance over the null_cycle code in i.e. the alsa driver
didn't convince me that the frame_time code actually handles this case
correctly (drivers/alsa_driver.c:1427ff). I might have missed stuff here
though.

Regards,
Flo

--
Palimm Palimm!
http://tapas.affenbande.org


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel
Reply | Threaded
Open this post in threaded view
|

Re: should zerocycles be treated as XRUNS?

Dave Robillard
On Wed, 2005-16-11 at 11:19 +0100, Florian Schmidt wrote:
> On Wed, 16 Nov 2005 14:13:43 +1100
> Dave Robillard <[hidden email]> wrote:
>
> > Can't you easily tell this happened in a client callback using
> > jack_last_frame_time (and nframes)?
>
> yeah, but basically no client does this. jack_last_frame_time() isn't
> even documented in the official api docs (on jackit.sf.net) yet (the
> last time i asked to include it has been over a year ago IIRC).

It's included in the docs, just not the docs on the site, because the
online docs are insanely old (0.98.1).  That definitely does need to be
fixed.  (hint hint, guys...)

> Anyways, by way of the aforementioned fix, all apps that handle xruns
> correctly will automagically work again (users of existing hd recorders
> will know cycles have been dropped and apps that rely on regular calling
> of their processs callbacks for timing purposes will also know something
> went wrong).
>
> Plus i think it's the Right Thing To Do (tm) as, as i said before, it
> treats zerocycles as error condition explicitely as they should be and
> not implicitly which is only possible to detect using an undocumented
> relatively new api call.
>
> Also actually this "fix" will decrease the number of zerocycles/xruns
> due to graph locking, because in many cases it will just work out all
> right in the end (i.e. a client that just reads the graph won't hold the
> lock very long, so there's a good chance the contention will go away
> quick enough for the process cycle to run. In the cases where it
> actually produces xruns, one will
>
> a] see it in the jack output b] and thus care about it and be able to
> fix it by improving those functions that hold the graph lock too long.

Agreed it's a good idea.  Just saying, clients /can/ detect skipped
cycles.  Just saying. :)

-DR-




-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel
Reply | Threaded
Open this post in threaded view
|

Re: should zerocycles be treated as XRUNS?

Florian Paul Schmidt-2
In reply to this post by Florian Paul Schmidt-2
On Wed, 16 Nov 2005 11:19:32 +0100
Florian Schmidt <[hidden email]> wrote:

> Thirdly, a quick glance over the null_cycle code in i.e. the alsa driver
> didn't convince me that the frame_time code actually handles this case
> correctly (drivers/alsa_driver.c:1427ff). I might have missed stuff here
> though.

I was wrong here. jack_run_cycle() does the frametime stuff before
calling jack_run_one_cycle which does the trylock. So jack_frame_time
and jack_last_frame_time should actually report correct values after a
zerocycle. The other arguments still hold though, IMHO.

Flo

--
Palimm Palimm!
http://tapas.affenbande.org


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel
Reply | Threaded
Open this post in threaded view
|

Re: should zerocycles be treated as XRUNS?

Stéphane Letz
In reply to this post by Florian Paul Schmidt-2

Le 16 nov. 05 à 11:19, Florian Schmidt a écrit :

> On Wed, 16 Nov 2005 14:13:43 +1100
> Dave Robillard <[hidden email]> wrote:
>
>> Can't you easily tell this happened in a client callback using
>> jack_last_frame_time (and nframes)?
>
> yeah, but basically no client does this. jack_last_frame_time() isn't
> even documented in the official api docs (on jackit.sf.net) yet (the
> last time i asked to include it has been over a year ago IIRC).
>
> Anyways, by way of the aforementioned fix, all apps that handle xruns
> correctly will automagically work again (users of existing hd  
> recorders
> will know cycles have been dropped and apps that rely on regular  
> calling
> of their processs callbacks for timing purposes will also know  
> something
> went wrong).
>
> Plus i think it's the Right Thing To Do (tm) as, as i said before, it
> treats zerocycles as error condition explicitely as they should be and
> not implicitly which is only possible to detect using an undocumented
> relatively new api call.
>
> Also actually this "fix" will decrease the number of zerocycles/xruns
> due to graph locking, because in many cases it will just work out all
> right in the end (i.e. a client that just reads the graph won't  
> hold the
> lock very long, so there's a good chance the contention will go away
> quick enough for the process cycle to run. In the cases where it
> actually produces xruns, one will
>
> a] see it in the jack output b] and thus care about it and be able to
> fix it by improving those functions that hold the graph lock too long.
>
> Thirdly, a quick glance over the null_cycle code in i.e. the alsa  
> driver
> didn't convince me that the frame_time code actually handles this case
> correctly (drivers/alsa_driver.c:1427ff). I might have missed stuff  
> here
> though.
>
> Regards,
> Flo
>

I don't like this idea of having the RT possibly block on OSX  
version. RT threads (time constraints on OSX) should never block  
inside their cycle since it may disturb the scheduler. This is at  
least  what Apple says....

Stephane
 

-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
<a href="http://ads.osdn.com/?ad_idv28&alloc_id845&op=click">http://ads.osdn.com/?ad_idv28&alloc_id845&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel
Reply | Threaded
Open this post in threaded view
|

Re: should zerocycles be treated as XRUNS?

Dave Cunningham
On Wed, Nov 16, 2005 at 02:17:15PM +0100, Stéphane Letz wrote:

>
>Le 16 nov. 05 à 11:19, Florian Schmidt a écrit :
>
>>On Wed, 16 Nov 2005 14:13:43 +1100
>>Dave Robillard <[hidden email]> wrote:
>>
>>>Can't you easily tell this happened in a client callback using
>>>jack_last_frame_time (and nframes)?
>>
>>yeah, but basically no client does this. jack_last_frame_time() isn't
>>even documented in the official api docs (on jackit.sf.net) yet (the
>>last time i asked to include it has been over a year ago IIRC).
>>
>>Anyways, by way of the aforementioned fix, all apps that handle xruns
>>correctly will automagically work again (users of existing hd  
>>recorders
>>will know cycles have been dropped and apps that rely on regular  
>>calling
>>of their processs callbacks for timing purposes will also know  
>>something
>>went wrong).
>>
>>Plus i think it's the Right Thing To Do (tm) as, as i said before, it
>>treats zerocycles as error condition explicitely as they should be and
>>not implicitly which is only possible to detect using an undocumented
>>relatively new api call.
>>
>>Also actually this "fix" will decrease the number of zerocycles/xruns
>>due to graph locking, because in many cases it will just work out all
>>right in the end (i.e. a client that just reads the graph won't  
>>hold the
>>lock very long, so there's a good chance the contention will go away
>>quick enough for the process cycle to run. In the cases where it
>>actually produces xruns, one will
>>
>>a] see it in the jack output b] and thus care about it and be able to
>>fix it by improving those functions that hold the graph lock too long.
>>
>>Thirdly, a quick glance over the null_cycle code in i.e. the alsa  
>>driver
>>didn't convince me that the frame_time code actually handles this case
>>correctly (drivers/alsa_driver.c:1427ff). I might have missed stuff  
>>here
>>though.
>>
>>Regards,
>>Flo
>>
>
>I don't like this idea of having the RT possibly block on OSX  
>version. RT threads (time constraints on OSX) should never block  
>inside their cycle since it may disturb the scheduler. This is at  
>least  what Apple says....

How do they define block?  What is "disturbing the scheduler" --
will the OS panic?

nanosleep(5) might be considered to block but shouldn't be a
problem

Futhermore, 1 + 2 can be thought of as blocking until the
completion of the addition operation.

The question is:  How long will the block last, and will that be
too long for the realtime constraint (in this case the amount of
cpu time left after the forthcoming process run)?

I can't see any reason why pthread_mutex_lock() should take any
longer than the worst-case running time for the "biggest"
critical section of the same mutex, assuming that a critical
section is not pre-empted, which we can guarantee by setting the
non-RT thread to SCHED_FIFO policy for the duration of the
critical section.  We can set the priority arbitrarily because
the real RT thread will be suspended waiting for the lock to be
released.  The only problem is if there is a (fairly) cpu
intensive RT thread with priority between the real RT thread and
the temporary RT thread, but is this a likely occurence?  As
soon as the lock is released, the real RT thread will pre-empt
the temporary RT thread so the changeover should be quite
efficient.

Incidently I've tried this out for the graph querying code
jack_port_get_all_connections that usually causes xruns when
qjackctl is running, and it works well.   No xruns under normal
operation (previously there would be some every time qjackctl
drew to the screen, polled, or the user clicked refresh) and
only 1 xrun every few seconds if i run
jack_port_get_all_connections(...) from a client in a while(1)
loop.  Previously this would totally distort the audio from
jack.  This is also *without* changing the code within the
critical section to make it realtime-safe so this may be where
those extra xruns come from.

Of course for anything except a proof-of-concept hack, this
means redesigning the critical sections so that they do not
perform non-realtime-safe calls, but this is not impossible, it
would just take some work.  (Critical sections should be as fast
as possible anyway, to minimise contention.  Making them run in
realtime just makes this requirement a "must" instead of a
"should".)

This is not the ideal solution but I think it would fix a lot of
xruns (they're still xruns even if we say they're "ok" and hide
them under the carpet) without too much redesigning required.

Discussion please! :)


thanks

-Dave Cunningham



-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel
Reply | Threaded
Open this post in threaded view
|

Re: should zerocycles be treated as XRUNS?

Stéphane Letz

Le 16 nov. 05 à 14:59, Dave Cunningham a écrit :

> On Wed, Nov 16, 2005 at 02:17:15PM +0100, Stéphane Letz wrote:
>>
>> Le 16 nov. 05 à 11:19, Florian Schmidt a écrit :
>>
>>> On Wed, 16 Nov 2005 14:13:43 +1100
>>> Dave Robillard <[hidden email]> wrote:
>>>
>>>> Can't you easily tell this happened in a client callback using
>>>> jack_last_frame_time (and nframes)?
>>>
>>> yeah, but basically no client does this. jack_last_frame_time()  
>>> isn't
>>> even documented in the official api docs (on jackit.sf.net) yet (the
>>> last time i asked to include it has been over a year ago IIRC).
>>>
>>> Anyways, by way of the aforementioned fix, all apps that handle  
>>> xruns
>>> correctly will automagically work again (users of existing hd
>>> recorders
>>> will know cycles have been dropped and apps that rely on regular
>>> calling
>>> of their processs callbacks for timing purposes will also know
>>> something
>>> went wrong).
>>>
>>> Plus i think it's the Right Thing To Do (tm) as, as i said  
>>> before, it
>>> treats zerocycles as error condition explicitely as they should  
>>> be and
>>> not implicitly which is only possible to detect using an  
>>> undocumented
>>> relatively new api call.
>>>
>>> Also actually this "fix" will decrease the number of zerocycles/
>>> xruns
>>> due to graph locking, because in many cases it will just work out  
>>> all
>>> right in the end (i.e. a client that just reads the graph won't
>>> hold the
>>> lock very long, so there's a good chance the contention will go away
>>> quick enough for the process cycle to run. In the cases where it
>>> actually produces xruns, one will
>>>
>>> a] see it in the jack output b] and thus care about it and be  
>>> able to
>>> fix it by improving those functions that hold the graph lock too  
>>> long.
>>>
>>> Thirdly, a quick glance over the null_cycle code in i.e. the alsa
>>> driver
>>> didn't convince me that the frame_time code actually handles this  
>>> case
>>> correctly (drivers/alsa_driver.c:1427ff). I might have missed stuff
>>> here
>>> though.
>>>
>>> Regards,
>>> Flo
>>>
>>
>> I don't like this idea of having the RT possibly block on OSX
>> version. RT threads (time constraints on OSX) should never block
>> inside their cycle since it may disturb the scheduler. This is at
>> least  what Apple says....
>
> How do they define block?  What is "disturbing the scheduler" --
> will the OS panic?
>
> nanosleep(5) might be considered to block but shouldn't be a
> problem
>
> Futhermore, 1 + 2 can be thought of as blocking until the
> completion of the addition operation.

By block, i mean having the RT suspend on a condition and having the  
scheduler switch to another thread.

>
> The question is:  How long will the block last, and will that be
> too long for the realtime constraint (in this case the amount of
> cpu time left after the forthcoming process run)?
>
> I can't see any reason why pthread_mutex_lock() should take any
> longer than the worst-case running time for the "biggest"
> critical section of the same mutex, assuming that a critical
> section is not pre-empted, which we can guarantee by setting the
> non-RT thread to SCHED_FIFO policy for the duration of the
> critical section.  We can set the priority arbitrarily because
> the real RT thread will be suspended waiting for the lock to be
> released.  The only problem is if there is a (fairly) cpu
> intensive RT thread with priority between the real RT thread and
> the temporary RT thread, but is this a likely occurence?  As
> soon as the lock is released, the real RT thread will pre-empt
> the temporary RT thread so the changeover should be quite
> efficient.
>
> Incidently I've tried this out for the graph querying code
> jack_port_get_all_connections that usually causes xruns when
> qjackctl is running, and it works well.   No xruns under normal
> operation (previously there would be some every time qjackctl
> drew to the screen, polled, or the user clicked refresh) and
> only 1 xrun every few seconds if i run
> jack_port_get_all_connections(...) from a client in a while(1)
> loop.  Previously this would totally distort the audio from
> jack.  This is also *without* changing the code within the
> critical section to make it realtime-safe so this may be where
> those extra xruns come from.
>
> Of course for anything except a proof-of-concept hack, this
> means redesigning the critical sections so that they do not
> perform non-realtime-safe calls, but this is not impossible, it
> would just take some work.  (Critical sections should be as fast
> as possible anyway, to minimise contention.  Making them run in
> realtime just makes this requirement a "must" instead of a
> "should".)
>
> This is not the ideal solution but I think it would fix a lot of
> xruns (they're still xruns even if we say they're "ok" and hide
> them under the carpet) without too much redesigning required.
>
> Discussion please! :)

Well I have the "redesign thing" (jackmp) working... I just want need  
some more time to document the new design and a release will follow  
soon.

Stephane



-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
<a href="http://ads.osdn.com/?ad_idv28&alloc_id845&op=click">http://ads.osdn.com/?ad_idv28&alloc_id845&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel
Reply | Threaded
Open this post in threaded view
|

Re: should zerocycles be treated as XRUNS?

Dave Cunningham
>By block, i mean having the RT suspend on a condition and
>having the scheduler switch to another thread.

That seems very restrictive, I cannot see why it would be useful
for Apple to require that, it would seem to make communication
between threads much harder...


>>This is not the ideal solution but I think it would fix a lot of
>>xruns (they're still xruns even if we say they're "ok" and hide
>>them under the carpet) without too much redesigning required.
>>
>>Discussion please! :)
>
>Well I have the "redesign thing" (jackmp) working... I just want need  
>some more time to document the new design and a release will follow  
>soon.

Is the jackmp api compatible with jack?  Is it written in c++,
will that be a problem?  Or do you mean that the lock-free
algorithms in jackmp could be implemented in jack with a little
work in translating c++ to c, and that would then solve the
problem.

Having algorithms where the RT thread actually interrupts the
other thread if it arrives second, rather than mutexes being
taken on a first-come first-serve basis, would be much better
than what I proposed since it respects the priorities in the
system...


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel
Reply | Threaded
Open this post in threaded view
|

Re: should zerocycles be treated as XRUNS?

Stéphane Letz

Le 16 nov. 05 à 15:53, Dave Cunningham a écrit :

>> By block, i mean having the RT suspend on a condition and
>> having the scheduler switch to another thread.
>
> That seems very restrictive, I cannot see why it would be useful
> for Apple to require that, it would seem to make communication
> between threads much harder...

This is something special to the RT class of thread on OSX

>
>
>>> This is not the ideal solution but I think it would fix a lot of
>>> xruns (they're still xruns even if we say they're "ok" and hide
>>> them under the carpet) without too much redesigning required.
>>>
>>> Discussion please! :)
>>
>> Well I have the "redesign thing" (jackmp) working... I just want need
>> some more time to document the new design and a release will follow
>> soon.
>
> Is the jackmp api compatible with jack?


Ok course , most of the jack clients I have tested already work:  
ardour, jamin, qjackctl, hydrogen, jack-rack...


> Is it written in c++,
> will that be a problem?

the API stays in C

> Or do you mean that the lock-free
> algorithms in jackmp could be implemented in jack with a little
> work in translating c++ to c, and that would then solve the
> problem.
>
> Having algorithms where the RT thread actually interrupts the
> other thread if it arrives second, rather than mutexes being
> taken on a first-come first-serve basis, would be much better
> than what I proposed since it respects the priorities in the
> system...


No time to answer here, i'm now leaving until next week....

Stephane

-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
<a href="http://ads.osdn.com/?ad_idv28&alloc_id845&op=click">http://ads.osdn.com/?ad_idv28&alloc_id845&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel
Reply | Threaded
Open this post in threaded view
|

Re: should zerocycles be treated as XRUNS?

Dave Robillard
In reply to this post by Dave Cunningham
On Wed, 2005-16-11 at 13:59 +0000, Dave Cunningham wrote:

> On Wed, Nov 16, 2005 at 02:17:15PM +0100, Stéphane Letz wrote:
> >I don't like this idea of having the RT possibly block on OSX  
> >version. RT threads (time constraints on OSX) should never block  
> >inside their cycle since it may disturb the scheduler. This is at  
> >least  what Apple says....
>
> How do they define block?  What is "disturbing the scheduler" --
> will the OS panic?
>
> nanosleep(5) might be considered to block but shouldn't be a
> problem
>
> Futhermore, 1 + 2 can be thought of as blocking until the
> completion of the addition operation.
>
> The question is:  How long will the block last, and will that be
> too long for the realtime constraint (in this case the amount of
> cpu time left after the forthcoming process run)?
>
> I can't see any reason why pthread_mutex_lock() should take any
> longer than the worst-case running time for the "biggest"
> critical section of the same mutex [...]

1+2 does not put your process in a blocked state - you don't give up
your timeslice.  sleep() and pthread_mutex_lock() definitely do, and
there are definitely massive implications of that in the context of a
realtime system.

You can think of 1+2 as "blocking" if you want, but you're just
inventing convenient new definitions of the term "blocking" to suit your
purposes. ;)

-DR-





-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
<a href="http://ads.osdn.com/?ad_idv28&alloc_id845&op=click">http://ads.osdn.com/?ad_idv28&alloc_id845&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel
Reply | Threaded
Open this post in threaded view
|

Re: should zerocycles be treated as XRUNS?

Paul Davis
In reply to this post by Stéphane Letz
> > Is it written in c++,
> > will that be a problem?
>
> the API stays in C
>
> > Or do you mean that the lock-free
> > algorithms in jackmp could be implemented in jack with a little
> > work in translating c++ to c, and that would then solve the
> > problem.
> >
> > Having algorithms where the RT thread actually interrupts the
> > other thread if it arrives second, rather than mutexes being
> > taken on a first-come first-serve basis, would be much better
> > than what I proposed since it respects the priorities in the
> > system...
>
>
> No time to answer here, i'm now leaving until next week....

in addition, i am waving my benign dictator hat here. no discussions of
future adoption of jackdmp until jack 1.0 is released. we have enough on
our already full hands without a language/implementation/design fight
added to the mix. the time will come soon enough for us to talk about
that, but that time is not today or tomorrow.

--p





-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel
Reply | Threaded
Open this post in threaded view
|

Re: should zerocycles be treated as XRUNS?

Dave Cunningham
In reply to this post by Dave Robillard
On Thu, Nov 17, 2005 at 02:10:22AM +1100, Dave Robillard wrote:

>On Wed, 2005-16-11 at 13:59 +0000, Dave Cunningham wrote:
>> On Wed, Nov 16, 2005 at 02:17:15PM +0100, Stéphane Letz wrote:
>> >I don't like this idea of having the RT possibly block on OSX  
>> >version. RT threads (time constraints on OSX) should never block  
>> >inside their cycle since it may disturb the scheduler. This is at  
>> >least  what Apple says....
>>
>> How do they define block?  What is "disturbing the scheduler" --
>> will the OS panic?
>>
>> nanosleep(5) might be considered to block but shouldn't be a
>> problem
>>
>> Futhermore, 1 + 2 can be thought of as blocking until the
>> completion of the addition operation.
>>
>> The question is:  How long will the block last, and will that be
>> too long for the realtime constraint (in this case the amount of
>> cpu time left after the forthcoming process run)?
>>
>> I can't see any reason why pthread_mutex_lock() should take any
>> longer than the worst-case running time for the "biggest"
>> critical section of the same mutex [...]
>
>1+2 does not put your process in a blocked state - you don't give up
>your timeslice.  sleep() and pthread_mutex_lock() definitely do, and
>there are definitely massive implications of that in the context of a
>realtime system.

what are these implications?  are they the same as the
implications of executing a lot of computational intensive code
in a realtime thread?


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel
Reply | Threaded
Open this post in threaded view
|

Re: should zerocycles be treated as XRUNS?

Florian Paul Schmidt-2
In reply to this post by Stéphane Letz
On Wed, 16 Nov 2005 15:34:09 +0100
Stéphane Letz <[hidden email]> wrote:

> Well I have the "redesign thing" (jackmp) working... I just want need  
> some more time to document the new design and a release will follow  
> soon.

Hi Stephane,

did you get rid of the additional period of delay that "plagued" (from
my point of view) jackdmp in previous releases? Forgive my ignorance as
i don't have an apple box i don't follow jackdmp so closely.

Regards,
Flo

--
Palimm Palimm!
http://tapas.affenbande.org


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
<a href="http://ads.osdn.com/?ad_idv28&alloc_id845&op=click">http://ads.osdn.com/?ad_idv28&alloc_id845&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel
Reply | Threaded
Open this post in threaded view
|

Re: should zerocycles be treated as XRUNS?

David-71
In reply to this post by Florian Paul Schmidt-2
On Wed, 16 Nov 2005 11:19:32 +0100
Florian Schmidt <[hidden email]> wrote:

> Anyways, by way of the aforementioned fix, all apps that handle xruns
> correctly will automagically work again (users of existing hd
> recorders will know cycles have been dropped and apps that rely on
> regular calling of their processs callbacks for timing purposes will
> also know something went wrong).
>
> Plus i think it's the Right Thing To Do (tm) as, as i said before, it
> treats zerocycles as error condition explicitely as they should be and
> not implicitly

Florian,

For ignorant readers like me, could you give us a little explanation of
what is a "zero cycle", what they mean to "apps that rely on regular
calling of their processs callbacks for timing purposes" and also what
should an app do to "handle xruns correctly" ?

I thought that it was correct to use the process cycle and its period
in frames as a timebase (as in a sequencer for example), while assuming
that an xrun shouldn't happen on a well-tuned system. Have I missed
something important ? Maybe I'm just too far from intricate issues in
jack's internals to understand your point ...

Thanks a lot.

--
David


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel
Reply | Threaded
Open this post in threaded view
|

Re: should zerocycles be treated as XRUNS?

Florian Paul Schmidt-2
On Thu, 17 Nov 2005 02:39:34 +0100
David <[hidden email]> wrote:

> Florian,
>
> For ignorant readers like me, could you give us a little explanation of
> what is a "zero cycle", what they mean to "apps that rely on regular
> calling of their processs callbacks for timing purposes" and also what
> should an app do to "handle xruns correctly" ?
>
> I thought that it was correct to use the process cycle and its period
> in frames as a timebase (as in a sequencer for example), while assuming
> that an xrun shouldn't happen on a well-tuned system. Have I missed
> something important ? Maybe I'm just too far from intricate issues in
> jack's internals to understand your point ...

Hi David,

well a "zerocycle" is when jack decides to pass a buffer full of zero's
to the hardware. This also means it does not run any of the clients'
process() callbacks for this buffer.

So you might ask, for what reasons would jack decide to do so. I assume
you do know what mutexes are and why interthread synchronization is
necessary. In the case of jack, the central datastructure is its
connection graph. It determines in which order clients' process
callbacks have to be executed, etc.

There are operations on the graph which alter it (i.e. connecting two
ports, creating/destroying a port, reading the graph (though this one
doesn't alter the graph. It still accesses it though) etc.). While these
operations are being executed jack would have to wait until they are
finished to continue processing. At the current time it doesn't wait,
but simply gives up when it sees the graph is locked (i suppose because
the operations take a long time and waiting for them wouldn't really be
worth it for some ops). It then simply produces a zerocycle. This
happens rather often.

So here's a usecase where it matters:

You have a recording program running, and at some point some of the apps
in your jack graph decides it wants to know the connections in the
graph. Now it calls the appropriate function and thus locks the graph.
There's quite a big chance now that this makes jack produce a zerocycle.
This also means (as the process callback of clients is not called during
a zerocycle) that one or more buffers of audio are lost and no error
condition is indicated to the user. So not only is a bit of audio lost,
but also the user doesn't even know about it unless he knows about these
jack internal details and knows he should never let any app do any graph
reading operations, nor fire up, close or connect clients..

Treating the zerocycle as xrun would at least give the user the feedback
that something disturbing the audio stream has happened. It would allow
for neural connections to build up which eventually lead to
understanding the situation (*lightbulb over users head* -> "hmmm, when
i connect apps i get an xrun, so i should better not do this while
recording").

Of course the correct way to remedy this situation would be to fix jack
to make zerocycles unnecessary, but i'm aware that this would take quite
a while and isn't really easy to do.

As this is not really feasible in short time IMHO the user should at
least be notified of zerocycles.

So the two ways we discussed on #lad would be:

a] leave the trylock in case, feed a zerocycle _and_ call the xrun
callback

b] make the trylock a lock and hope for the best as described in my
other mail. If the timing gets screwed, an xrun will be raised
automagically.

About xruns on welltuned systems:

Well, they can always happen due to applications' faults. But zerocycles
are really a built in source of xruns which even the best tuned system
won't do anything about and which (in the current state) the user will
never know about.

Regards,
Flo

--
Palimm Palimm!
http://tapas.affenbande.org


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel
Reply | Threaded
Open this post in threaded view
|

Re: should zerocycles be treated as XRUNS?

Alfons Adriaensen
On Thu, Nov 17, 2005 at 11:51:50AM +0100, Florian Schmidt wrote:

> Of course the correct way to remedy this situation would be to fix jack
> to make zerocycles unnecessary, but i'm aware that this would take quite
> a while and isn't really easy to do.

It shouldn't be too difficult to avoid them when the graph is being read.
only. When this happens, there is no reason to block the process cycle,
you only want to prevent the graph being modified.

This would at least the qjackctl problem. Users my find it acceptable
that processing is disturbed when making new connections, but not
just to refresh a GUI.

Unless there are other things that would make this not work, a second
mutex should do the trick.

Let G be the existing graph lock, and R a new one.


process cycle:  lock G, doit, unlock G

read graph:     lock R, doit, unlock R

modify graph:   lock R, lock G, doit, unlock G, unlock R

The order for the last one is important: a writer should not
block the process cycle until it really can proceed and get
out of the way ASAP. So it should take R first.

Solaris used to have reader-writer locks to do this sort of thing.
I've no idea if they exist in NPTL or Linuxthreads.
 
If the client that reads the graph has itself a process callback
will it still block the cycle (by waiting for the reply rather than
the process trigger), or has this been foreseen ?


--
FA


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel
Reply | Threaded
Open this post in threaded view
|

Re: should zerocycles be treated as XRUNS?

Paul Davis
> Let G be the existing graph lock, and R a new one.
>
>
> process cycle:  lock G, doit, unlock G
>
> read graph:     lock R, doit, unlock R
>
> modify graph:   lock R, lock G, doit, unlock G, unlock R
>
> The order for the last one is important: a writer should not
> block the process cycle until it really can proceed and get
> out of the way ASAP. So it should take R first.
>
> Solaris used to have reader-writer locks to do this sort of thing.
> I've no idea if they exist in NPTL or Linuxthreads.

POSIX P.1003 (pthreads) has rw locks, and yes we could use them here.

> If the client that reads the graph has itself a process callback
> will it still block the cycle (by waiting for the reply rather than
> the process trigger), or has this been foreseen ?

the JACK-created thread that runs process() is not permitted to call
these functions. this is stated in the docs, but i would wager that it
is not as prominent as it should be.

--p




-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel
Reply | Threaded
Open this post in threaded view
|

Re: should zerocycles be treated as XRUNS?

Dave Cunningham
On Thu, Nov 17, 2005 at 07:08:20AM -0500, Paul Davis wrote:

>> Let G be the existing graph lock, and R a new one.
>>
>>
>> process cycle:  lock G, doit, unlock G
>>
>> read graph:     lock R, doit, unlock R
>>
>> modify graph:   lock R, lock G, doit, unlock G, unlock R
>>
>> The order for the last one is important: a writer should not
>> block the process cycle until it really can proceed and get
>> out of the way ASAP. So it should take R first.
>>
>> Solaris used to have reader-writer locks to do this sort of thing.
>> I've no idea if they exist in NPTL or Linuxthreads.
>
>POSIX P.1003 (pthreads) has rw locks, and yes we could use them here.
>
>> If the client that reads the graph has itself a process callback
>> will it still block the cycle (by waiting for the reply rather than
>> the process trigger), or has this been foreseen ?
>
>the JACK-created thread that runs process() is not permitted to call
>these functions. this is stated in the docs, but i would wager that it
>is not as prominent as it should be.

I thought you said the process cycle modifies the graph to
reject failing clients!


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel
12