latencies, signal flows and some deep thinkos

[Paul Davis]

The last three days over on #ardour have involved some brain-melting
discussions of how ardour handles latencies caused by JACK and by
various plugins. During those discussions, some observations about
JACK have slowly emerged, grown and reached the point in my mind where
there is a fully formed crisis.

Although it was never part of the original specification or idea for
JACK, it became clear some time ago that providing an API that allows
clients to measure the latencies in various signal pathways is
important. We added a way for clients to set the latency of their
specific ports, and to determine the total latency for a signal
between a given port and some physical input or output.

This scheme, however, is fundamentally inadequate, because JACK can
only traverse signal pathways represented by port connections. These
are obviously important pathways, but they are not the only ones. A
substantial number of JACK clients have pairs of inputs and outputs
that they create an internal pathway between. The example client is an
example of this - it copies data from its input port to its
output. JACK does not know that there is any signal flow between these
ports, and if you ask for the latency of a signal that will arrive at
the inputs of that client, you will always get zero, even if it
actually continues on through the outputs to some physical output.

In essence, jackd has a totally incomplete view of the signal pathways
in the port graph because it only knows about port connections, and
this prevents the total latency values from ever being correct except
in very simple situations.

However, it gets a little worse. At the moment, the API for checking
total latencies is unidirectional. If you ask about an output port,
you get a number that is supposed to represent the delay between
delivering data to this port and it emerging from a physical/terminal
output. For an input port, its the delay between the data arriving at
a physical/terminal port and the one you are querying about.

I don't think that this is adequate. I suspect that it ought to be
possible to ask the question in either direction:

         "how long has it been since the data arriving at this port
            arrived at a physical/terminal port?"
           
         "how long will it be until the data delivered to this port
            will arrive at a physical/terminal port?"

I have some ideas on how to solve the first part of the problem, and
clearly the second part involves an API change/extension, but I will
leave those suggestions until other people have commented on the
issues.

--p


-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel

[Alfons Adriaensen]

On Thu, May 12, 2005 at 06:58:04AM -0400, Paul Davis wrote:

> Although it was never part of the original specification or idea for
> JACK, it became clear some time ago that providing an API that allows
> clients to measure the latencies in various signal pathways is
> important. We added a way for clients to set the latency of their
> specific ports, and to determine the total latency for a signal
> between a given port and some physical input or output.
>
> This scheme, however, is fundamentally inadequate, because JACK can
> only traverse signal pathways represented by port connections. These
> are obviously important pathways, but they are not the only ones. A
> substantial number of JACK clients have pairs of inputs and outputs
> that they create an internal pathway between. The example client is an
> example of this - it copies data from its input port to its
> output.

Any client that does _not_ do this could be split up into two, one
part having only inputs, and the other only outputs.

> JACK does not know that there is any signal flow between these
> ports, and if you ask for the latency of a signal that will arrive at
> the inputs of that client, you will always get zero, even if it
> actually continues on through the outputs to some physical output.

> In essence, jackd has a totally incomplete view of the signal pathways
> in the port graph because it only knows about port connections, and
> this prevents the total latency values from ever being correct except
> in very simple situations.

Jackd could have a complete view iff all clients could somehow answer
the question: "what is the delay from your input port #A to your output
port #B ?"  (the format of the answer should allow for the ports not
being connected at all).

A complication arises if there is more than one path between two ports,
as can easily happen, either by the connections made by jackd or internally
in a client. In the latter case, even the question formulated above does
not have a definite answer. But this is probably academic.

A more serious problem is that the connections and delays inside a
client need not be fixed. Jackd has no way to find out the values have
changed, unless either it polls for them on every cycle, or there is
some notification mechanism from clients to jackd.

> However, it gets a little worse. At the moment, the API for checking
> total latencies is unidirectional. If you ask about an output port,
> you get a number that is supposed to represent the delay between
> delivering data to this port and it emerging from a physical/terminal
> output. For an input port, its the delay between the data arriving at
> a physical/terminal port and the one you are querying about.

If it is defined in both directions as seems to be implied by the
paragraph above, what do you mean by 'unidirectional' ?

 
> I don't think that this is adequate. I suspect that it ought to be
> possible to ask the question in either direction:
>
> "how long has it been since the data arriving at this port
>    arrived at a physical/terminal port?"
>    
> "how long will it be until the data delivered to this port
>    will arrive at a physical/terminal port?"

If by "how long has it been since/until X" you mean "what is the
difference in time between "now" and X, I think you are mixing up
two problems: 1. relating physical time to samples and 2. taking
graph delays into account. It's probably wiser to keep these
separate.

Also I can't see why you would ever ask that question - "now" withing
a process callback has no useful meaning, as it does not matter when
in the current period some processing is done, as long as it done in
time.

The concept of "now" only has a meaning when you want to timestamp
an external event, for example if you want to know "what sample was
being heard by the user (or being recorded) when (s)he clicked that
mouse button ?"

If you have a 'jack_frame_time ()' timestamp on the event, then this
is easy enough to find out: subtract/add the input/output latency
of the physical port (defined as in your recent post), plus the
delay from the physical port to you application's port.

So we are back to the original problem of finding these delays.

--
FA









-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel

[Paul Davis]

>> In essence, jackd has a totally incomplete view of the signal pathways
>> in the port graph because it only knows about port connections, and
>> this prevents the total latency values from ever being correct except
>> in very simple situations.
>
>Jackd could have a complete view iff all clients could somehow answer
>the question: "what is the delay from your input port #A to your output
>port #B ?"  (the format of the answer should allow for the ports not
>being connected at all).

well, we already have a mechanism that does the computation by
traversing port connections. i was thinking of:

           * adding a connection "type" to the connection data
             structure, with one of two enum values: Explicit and Implicit
           * add jack_port_wire (jack_port_t*, jack_port_t*) which
             creates an Implicit connection between them. clients
             that copy from port A to port B internally would
             call this.
           * jack_port_connect would continue to work as it does,
             but would mark the connection as Explicit

i seem to recall that something like this idea has been proposed before.

>A more serious problem is that the connections and delays inside a
>client need not be fixed. Jackd has no way to find out the values have
>changed, unless either it polls for them on every cycle, or there is
>some notification mechanism from clients to jackd.

this was just added to JACK CVS: jack_recompute_total_latencies() is a
client request that causes jackd to recompute the values.

>> However, it gets a little worse. At the moment, the API for checking
>> total latencies is unidirectional. If you ask about an output port,
>> you get a number that is supposed to represent the delay between
>> delivering data to this port and it emerging from a physical/terminal
>> output. For an input port, its the delay between the data arriving at
>> a physical/terminal port and the one you are querying about.
>
>If it is defined in both directions as seems to be implied by the
>paragraph above, what do you mean by 'unidirectional' ?

the API provides an answer by tracing the signal flow, but the
direction it traces in is defined only by the port type. if you supply
an input port, it traces "backwards" towards a physical/terminal
input; if you supply an output port, it traces "forwards" towards a
physical/terminal output.



-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel

[Alfons Adriaensen]

On Thu, May 12, 2005 at 08:50:36AM -0400, Paul Davis wrote:

> well, we already have a mechanism that does the computation by
> traversing port connections. i was thinking of:
>
>   * adding a connection "type" to the connection data
>              structure, with one of two enum values: Explicit and Implicit
>   * add jack_port_wire (jack_port_t*, jack_port_t*) which
>     creates an Implicit connection between them. clients
>     that copy from port A to port B internally would
>     call this.

It could be interesting to have a delay value associated with
Implicit connections, to allow for clients that introduce an
unwanted delay such as look-ahead limiters. Of course any
delay introduced for 'artistic' reasons should not be included
in this value - we don't want anyone to compensate for it.

>   * jack_port_connect would continue to work as it does,
>     but would mark the connection as Explicit
>
> i seem to recall that something like this idea has been proposed before.

It would certainly allow things to work without having to redesign
the whole lot...

> the API provides an answer by tracing the signal flow, but the
> direction it traces in is defined only by the port type. if you supply
> an input port, it traces "backwards" towards a physical/terminal
> input; if you supply an output port, it traces "forwards" towards a
> physical/terminal output.

Does it take execution order into account, i.e. will it add a period
to the delay if a path goes through a client twice ?

--
FA
 


-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel

[Stéphane Letz]

Le 12 mai 05, à 12:58, Paul Davis a écrit :

> The last three days over on #ardour have involved some brain-melting
> discussions of how ardour handles latencies caused by JACK and by
> various plugins. During those discussions, some observations about
> JACK have slowly emerged, grown and reached the point in my mind where
> there is a fully formed crisis.
>
> Although it was never part of the original specification or idea for
> JACK, it became clear some time ago that providing an API that allows
> clients to measure the latencies in various signal pathways is
> important. We added a way for clients to set the latency of their
> specific ports, and to determine the total latency for a signal
> between a given port and some physical input or output.
>
> This scheme, however, is fundamentally inadequate, because JACK can
> only traverse signal pathways represented by port connections. These
> are obviously important pathways, but they are not the only ones. A
> substantial number of JACK clients have pairs of inputs and outputs
> that they create an internal pathway between. The example client is an
> example of this - it copies data from its input port to its
> output. JACK does not know that there is any signal flow between these
> ports, and if you ask for the latency of a signal that will arrive at
> the inputs of that client, you will always get zero, even if it
> actually continues on through the outputs to some physical output.
>
> In essence, jackd has a totally incomplete view of the signal pathways
> in the port graph because it only knows about port connections, and
> this prevents the total latency values from ever being correct except
> in very simple situations.
>
> However, it gets a little worse. At the moment, the API for checking
> total latencies is unidirectional. If you ask about an output port,
> you get a number that is supposed to represent the delay between
> delivering data to this port and it emerging from a physical/terminal
> output. For an input port, its the delay between the data arriving at
> a physical/terminal port and the one you are querying about.
>
> I don't think that this is adequate. I suspect that it ought to be
> possible to ask the question in either direction:
>
> "how long has it been since the data arriving at this port
>    arrived at a physical/terminal port?"
>
> "how long will it be until the data delivered to this port
>    will arrive at a physical/terminal port?"
>
> I have some ideas on how to solve the first part of the problem, and
> clearly the second part involves an API change/extension, but I will
> leave those suggestions until other people have commented on the
> issues.
>
> --p
>
>

I suggest that before presenting a solution, you describe the problem
you want to solve: I guess latencies compensation in Ardour.
This would help...

Stephane



-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
<a href="http://ads.osdn.com/?ad_ids93&alloc_id281&op=click">http://ads.osdn.com/?ad_ids93&alloc_id281&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel

[Karsten Wiese]

Am Donnerstag, 12. Mai 2005 14:50 schrieb Paul Davis:

> >> In essence, jackd has a totally incomplete view of the signal pathways
> >> in the port graph because it only knows about port connections, and
> >> this prevents the total latency values from ever being correct except
> >> in very simple situations.
> >
> >Jackd could have a complete view iff all clients could somehow answer
> >the question: "what is the delay from your input port #A to your output
> >port #B ?"  (the format of the answer should allow for the ports not
> >being connected at all).
>
> well, we already have a mechanism that does the computation by
> traversing port connections. i was thinking of:
>
>   * adding a connection "type" to the connection data
>              structure, with one of two enum values: Explicit and Implicit
>   * add jack_port_wire (jack_port_t*, jack_port_t*) which
>     creates an Implicit connection between them. clients
>     that copy from port A to port B internally would
>     call this.

Could those additions be superfluous if we said instead:
        * In and Out ports of the same one jack_client are always assumed to be connected implicitly.
        * A Jack application with Ins and Outs that has no signalflow between those should register separate jack_clients.
?

>   * jack_port_connect would continue to work as it does,
>     but would mark the connection as Explicit
>
> i seem to recall that something like this idea has been proposed before.
>
> >A more serious problem is that the connections and delays inside a
> >client need not be fixed. Jackd has no way to find out the values have
> >changed, unless either it polls for them on every cycle, or there is
> >some notification mechanism from clients to jackd.
>
> this was just added to JACK CVS: jack_recompute_total_latencies() is a
> client request that causes jackd to recompute the values.
>
> >> However, it gets a little worse. At the moment, the API for checking
> >> total latencies is unidirectional. If you ask about an output port,
> >> you get a number that is supposed to represent the delay between
> >> delivering data to this port and it emerging from a physical/terminal
> >> output. For an input port, its the delay between the data arriving at
> >> a physical/terminal port and the one you are querying about.
> >
> >If it is defined in both directions as seems to be implied by the
> >paragraph above, what do you mean by 'unidirectional' ?
>
> the API provides an answer by tracing the signal flow, but the
> direction it traces in is defined only by the port type. if you supply
> an input port, it traces "backwards" towards a physical/terminal
> input; if you supply an output port, it traces "forwards" towards a
> physical/terminal output.

-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel

[Alfons Adriaensen]

On Fri, May 13, 2005 at 11:51:06AM +0200, Karsten Wiese wrote:

> >   * adding a connection "type" to the connection data
> >              structure, with one of two enum values: Explicit and Implicit
> >   * add jack_port_wire (jack_port_t*, jack_port_t*) which
> >     creates an Implicit connection between them. clients
> >     that copy from port A to port B internally would
> >     call this.
>
> Could those additions be superfluous if we said instead:
> * In and Out ports of the same one jack_client are always
>         assumed to be connected implicitly.

That is in fact the assumption already made by the graph traversal
algo, and indeed all information is already there to work out a
total latency. The missing part is internal processing delays.

One solution would be to take that assumption, unless it is reset
by the client, and then the client could declare each dependency
separately, including its delay.

> * A Jack application with Ins and Outs that has no signalflow
>         between those should register separate jack_clients.

Yep, that should always be possible. Ardour already uses JACK for
its internal routing to some extent, and if you take that idea
to its limits, it could be split up into at least three 'layers',
top to bottom:

- the capture engine (inputs only)
- the playback engine (outputs only)
- the mixer (in and out)

They could still share a common GUI, and the user need not even
be aware of this structure. E.g. when a track is switched to
input monitoring, the relevant mixer strip is connected
to the same source as the track input in the capture engine.

That would also allow to use each part separately (e.g. just a
multitrack 'tape machine', with a separate mixer) or even having
multiple instances of some. It would also allow zero delay inserts
between the playback engine and the mixer.


(just dreaming of course)


BTW, does JACK allow a single process to have multiple clients ?

--
FA



-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel

[Karsten Wiese]

Am Freitag, 13. Mai 2005 14:08 schrieb Alfons Adriaensen:

> On Fri, May 13, 2005 at 11:51:06AM +0200, Karsten Wiese wrote:
>
> > >   * adding a connection "type" to the connection data
> > >              structure, with one of two enum values: Explicit and Implicit
> > >   * add jack_port_wire (jack_port_t*, jack_port_t*) which
> > >     creates an Implicit connection between them. clients
> > >     that copy from port A to port B internally would
> > >     call this.
> >
> > Could those additions be superfluous if we said instead:
> > * In and Out ports of the same one jack_client are always
> >         assumed to be connected implicitly.
>
> That is in fact the assumption already made by the graph traversal
> algo, and indeed all information is already there to work out a
> total latency. The missing part is internal processing delays.
>
> One solution would be to take that assumption, unless it is reset
> by the client, and then the client could declare each dependency
> separately, including its delay.
>
> > * A Jack application with Ins and Outs that has no signalflow
> >         between those should register separate jack_clients.
>
> Yep, that should always be possible. Ardour already uses JACK for
> its internal routing to some extent, and if you take that idea
> to its limits, it could be split up into at least three 'layers',
> top to bottom:
>
> - the capture engine (inputs only)
> - the playback engine (outputs only)
> - the mixer (in and out)
>
> They could still share a common GUI, and the user need not even
> be aware of this structure. E.g. when a track is switched to
> input monitoring, the relevant mixer strip is connected
> to the same source as the track input in the capture engine.
>
> That would also allow to use each part separately (e.g. just a
> multitrack 'tape machine', with a separate mixer) or even having
> multiple instances of some. It would also allow zero delay inserts
> between the playback engine and the mixer.
>
>
> (just dreaming of course)
>
>
> BTW, does JACK allow a single process to have multiple clients ?
>

Didn't try yet but know an example: look at internal clients:
we can have lots of them and they all live in 1 process, the jackd.
So I believe (and will soon try :-) that its also possible to have multiple clients in 1 external (to jackd) process.

Karsten




-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel

[Stéphane Letz]

Le 13 mai 05, à 14:08, Alfons Adriaensen a écrit :

> On Fri, May 13, 2005 at 11:51:06AM +0200, Karsten Wiese wrote:
>
>>>   * adding a connection "type" to the connection data
>>>              structure, with one of two enum values: Explicit and
>>> Implicit
>>>   * add jack_port_wire (jack_port_t*, jack_port_t*) which
>>>     creates an Implicit connection between them. clients
>>>     that copy from port A to port B internally would
>>>     call this.
>>
>> Could those additions be superfluous if we said instead:
>> * In and Out ports of the same one jack_client are always
>>         assumed to be connected implicitly.
>
> That is in fact the assumption already made by the graph traversal
> algo, and indeed all information is already there to work out a
> total latency. The missing part is internal processing delays.
>
> One solution would be to take that assumption, unless it is reset
> by the client, and then the client could declare each dependency
> separately, including its delay.
>
>> * A Jack application with Ins and Outs that has no signalflow
>>         between those should register separate jack_clients.
>
> Yep, that should always be possible. Ardour already uses JACK for
> its internal routing to some extent, and if you take that idea
> to its limits, it could be split up into at least three 'layers',
> top to bottom:
>
> - the capture engine (inputs only)
> - the playback engine (outputs only)
> - the mixer (in and out)
>
> They could still share a common GUI, and the user need not even
> be aware of this structure. E.g. when a track is switched to
> input monitoring, the relevant mixer strip is connected
> to the same source as the track input in the capture engine.
>
> That would also allow to use each part separately (e.g. just a
> multitrack 'tape machine', with a separate mixer) or even having
> multiple instances of some. It would also allow zero delay inserts
> between the playback engine and the mixer.
>
>
> (just dreaming of course)
>
>
> BTW, does JACK allow a single process to have multiple clients ?
>
> --
> FA
>

jackdmp allows that: multiple clients inside the server as well as
multiple clients inside any external process...

Stephane



-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
<a href="http://ads.osdn.com/?ad_ids93&alloc_id281&op=click">http://ads.osdn.com/?ad_ids93&alloc_id281&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel

[Alfons Adriaensen]

On Fri, May 13, 2005 at 03:06:26PM +0200, St?phane Letz wrote:

> jackdmp allows that: multiple clients inside the server as well as
> multiple clients inside any external process...

I'd be surprised otherwise...

How's the Linux version ? :-)

--
FA




-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel

[Paul Davis]

>> Could those additions be superfluous if we said instead:
>> * In and Out ports of the same one jack_client are always
>>         assumed to be connected implicitly.
>
>That is in fact the assumption already made by the graph traversal
>algo, and indeed all information is already there to work out a
>total latency. The missing part is internal processing delays.

This is not true.

The graph traversal algo does not make this assumption, and the
information required is not available.

I think that Karsten is thinking of very simple clients (from a wiring
perspective) that have, say, 2 input ports and 2 output ports. It
might be reasonable to make this assumption there, but its not
definitely correct. Move up to more complex apps, and it breaks down
entirely. JACK cannot possibly know how the data moves through Ardour,
for example. Even in the case of a 2-in, N-out client (for example,
the Faust-implemented 8 port version of tapiir that Yann Orlarey did
for me at LAC2005), its not clear that JACK can make any assumptions
about the in/out signal flow.

--p


-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel

[Alfons Adriaensen]

On Fri, May 13, 2005 at 09:46:09AM -0400, Paul Davis wrote:

> >That is in fact the assumption already made by the graph traversal
> >algo, and indeed all information is already there to work out a
> >total latency. The missing part is internal processing delays.
>
> This is not true.
>
> The graph traversal algo does not make this assumption, and the
> information required is not available.

Yes and no :-) Consider the following subgraph:

    A ---> B ---> C           (1)

Disregarding loops for a moment, you will agree that the graph traversal
algo ensures that the execution order will be A,B,C.

If it would additionally assume that the outputs of B depend on its
inputs,  ** it could not do any better than what it already does **.
In other words, its behaviour is consistent with making that assumption.
So we can as well say that it makes that assumption, even if that is not
the force that drives it.

Suppose for a moment that the outputs of B do not depend on its inputs.
We could then split up B:

   A ---> B1     B2 ---> C    (2)
   
Now the graph traversel has a choice: either A,B1,B2,C or B2,C,A,B1,
or even A,B2,B1,C.

Since in (1), the graph traversal does not have that choice, it has
implicitly assumed the dependencies in B.


> I think that Karsten is thinking of very simple clients (from a wiring
> perspective) that have, say, 2 input ports and 2 output ports. It
> might be reasonable to make this assumption there, but its not
> definitely correct. Move up to more complex apps, and it breaks down
> entirely. JACK cannot possibly know how the data moves through Ardour,
> for example. Even in the case of a 2-in, N-out client (for example,
> the Faust-implemented 8 port version of tapiir that Yann Orlarey did
> for me at LAC2005), its not clear that JACK can make any assumptions
> about the in/out signal flow.

Could you give a minimal example of that ?

--
FA



-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel

[Alfons Adriaensen]

On Fri, May 13, 2005 at 09:46:09AM -0400, Paul Davis wrote:

> >That is in fact the assumption already made by the graph traversal
> >algo, and indeed all information is already there to work out a
> >total latency. The missing part is internal processing delays.
>
> This is not true.

(...continued)

Isn't the real problem this (assume again A ---> B ---> C):
it could very well be that the outputs of B that C is using do
not depend on those inputs of B that come from A. In other
words, there is no path from A to C, while the graph traversal
(implicitly) assumes there is one.

Hence it could see multiple paths to a PCM, and has no way to
know which ones are valid.


--
FA


-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel

[Stéphane Letz]

Le 13 mai 05, à 17:24, Alfons Adriaensen a écrit :

> On Fri, May 13, 2005 at 09:46:09AM -0400, Paul Davis wrote:
>
>>> That is in fact the assumption already made by the graph traversal
>>> algo, and indeed all information is already there to work out a
>>> total latency. The missing part is internal processing delays.
>>
>> This is not true.
>
> (...continued)
>
> Isn't the real problem this (assume again A ---> B ---> C):
> it could very well be that the outputs of B that C is using do
> not depend on those inputs of B that come from A. In other
> words, there is no path from A to C, while the graph traversal
> (implicitly) assumes there is one.
>
> Hence it could see multiple paths to a PCM, and has no way to
> know which ones are valid.
>
>

Still waiting that *someone* takes some times to explain what problem
has to be solved in the first place with this port latency stuff....

Stephane



-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
<a href="http://ads.osdn.com/?ad_ids93&alloc_id281&op=click">http://ads.osdn.com/?ad_ids93&alloc_id281&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel

[Alfons Adriaensen]

On Fri, May 13, 2005 at 06:21:10PM +0200, St?phane Letz wrote:

> Still waiting that *someone* takes some times to explain what problem
> has to be solved in the first place with this port latency stuff....

It's a subproblem of a more general one, which is finding out the time
associated with the samples that your JACK apps are working on.

Even if you're not interested in absolute timing, you may want that
sounds that follow different paths to the audio HW arrive there in sync.


--
FA


-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel

[Florian Paul Schmidt-2]

On Fri, 13 May 2005 18:21:10 +0200
Stéphane Letz <[hidden email]> wrote:

> Still waiting that *someone* takes some times to explain what problem
> has to be solved in the first place with this port latency stuff....

Imagine the following setup:

[Client A out1] -> [in1 Client B out1] -> alsa_pcm:playback_1
[         out2] -> [in2          out2] -> alsa_pcm:playback_2

Now client B introduces differing latencies for its internal connections
of in1 to out1 and in2 to out2. On the path in1 out1 it adds 0 frames
latency [simple copy for example] and on the path in2 out2 it adds N
frames additional latency [i.e. by having a plugin in that path that is
latent]

Now, if Client A wants that the output it sends out through its
individual output ports will be audible at the same time, it needs to
compensate the signal it sends to out1 by N frames.

For Client A to be able to do this it has to be able to ask jack about
the latencies it can expect when sending audio through in1 and in2 of
Client B respectively (the total latencies for the path from Client B's
inputs to the physical outputs. The physical output latency is actually
constant for all outputs, so it might be or might not be included. But
there might be yet another Client C between B and alsa_pcm doing stuff
similar to B). From the difference of the answers it can compute N.

Flo

--
Palimm Palimm!
http://affenbande.org/~tapas/


-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
<a href="http://ads.osdn.com/?ad_ids93&alloc_id281&op=click">http://ads.osdn.com/?ad_ids93&alloc_id281&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel

[Florian Paul Schmidt-2]

On Fri, 13 May 2005 18:50:32 +0200
Alfons Adriaensen <[hidden email]> wrote:

> On Fri, May 13, 2005 at 06:21:10PM +0200, St?phane Letz wrote:
>
> > Still waiting that *someone* takes some times to explain what problem
> > has to be solved in the first place with this port latency stuff....
>
> It's a subproblem of a more general one, which is finding out the time
> associated with the samples that your JACK apps are working on.

I was going to write a comment on how i agree to you that physical port
latency is something completely different from additional latency
introduced by clients.. but...

When using jack_frame_time as "wallclock" then the notion of port
latency for physical ports existing right now in jack does make some
sense.

Imagine a playback only client. jack runs with 4 periods a 512 frames.
jack computes its output port latency to 3*512 frames. Now the client's
process callback gets called. It fills a buffer of 512 frames. Now the
client is interested in at what exact "time" the first frame of its
current buffer will be audible.

Actually it can find out what time that will be wrt to the
jack_frame_time. During the process callback the client can call
jack_last_frame_time() which gives it the framecount of the first frame
in the current period. The jack_frame_time is based on this frame count
(coupled with the famous DLL ;)). So to find out at what frame_time the
first frame of its current buffer will be audible it simply has to add
3*512 to jack_last_frame_time. When on its signal path there are
additional clients which introduce latencies, these just have to be
added to this..

Flo

--
Palimm Palimm!
http://affenbande.org/~tapas/


-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel

[Florian Paul Schmidt-2]

On Fri, 13 May 2005 19:16:57 +0200
Florian Schmidt <[hidden email]> wrote:

>  During the process callback the client can call
> jack_last_frame_time() which gives it the framecount of the first frame
> in the current period.

actually i'm not 100% sure on the semantics of jack_last_frame_time. It
might also be the framecount after the last process cycle.. Hmm, is that
equivalent? /me wrenches his brain

Anyways, i know for sure that it can be used to put frames in the
process cycle in relation to frame_time() and thus above holds modulo an
offset..

Flo

--
Palimm Palimm!
http://affenbande.org/~tapas/


-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel

[Fons Adriaensen]

On Fri, May 13, 2005 at 07:16:57PM +0200, Florian Schmidt wrote:

> I was going to write a comment on how i agree to you that physical port
> latency is something completely different from additional latency
> introduced by clients.. but...

It's not really different, the total latency is just the sum of the two.
It 'feels' different because for physical latency you are suddenly
referring to physical time, while for internal latencies everything
is 'virtual' - a matter of counters and indices.
 
> Actually it can find out what time that will be wrt to the
> jack_frame_time. During the process callback the client can call
> jack_last_frame_time() which gives it the framecount of the first frame
> in the current period. The jack_frame_time is based on this frame count
> (coupled with the famous DLL ;)). So to find out at what frame_time the
> first frame of its current buffer will be audible it simply has to add
> 3*512 to jack_last_frame_time.

That is exactly what I try to explain in

<http://users.skynet.be/solaris/linuxaudio/downloads/frametime.pdf>

Please ignore the proposal part in this doc - it is being reworked (*).

> When on its signal path there are
> additional clients which introduce latencies, these just have to be
> added to this..

Yep. The problem is to find out if there are any. It looks simple in
simple cases, but once you have a client in the path that has some
direct connections to the soundcard and some that go through other
clients, it is not possible to do this without knowing that client's
internal routing and delays.

--
FA


(*) What is still missing is a relation samples <-> some form of
absolute system time, although you can have a good approximation by
reading both jack_frame_time() and that system time. But you can't
extrapolate that relation, since frame_time never runs at exactly
the nominal frame rate, and you have to keep reading both all the
time. The solution is a second DLL that relates frame_time to some
wall-clock system time. In my original proposal that was the time
given by gettimeofday() converted to double. In the new one that
will probably be the OSC timescale (derived from gettimeofday()).



-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel

[Simon Jenkins]

On Fri, 2005-05-13 at 09:46 -0400, Paul Davis wrote:

> >> Could those additions be superfluous if we said instead:
> >> * In and Out ports of the same one jack_client are always
> >>         assumed to be connected implicitly.
> >
> >That is in fact the assumption already made by the graph traversal
> >algo, and indeed all information is already there to work out a
> >total latency. The missing part is internal processing delays.
>
> This is not true.
>
> The graph traversal algo does not make this assumption, and the
> information required is not available.

I think this assumption is implicit in Jack's design, regardless of
whether there is any line of code which explicitly "makes" the
assumption.

Clients are always executed atomically in Jack: A client must consume
all of its inputs and produce all of its outputs in a single call of the
process callback per process cycle. There is no provision for splitting
a callback into two calls:

A --> (some-of-B) -> C -> (rest-of-B) -> D

when B's internal routing requires it[*].

The above ordering would be required to insert client C into client B's
routing for example. Jack can't do this. Instead, it treats B and C as
if they were connected in a feedback loop. This, I think, is equivalent
to making the assumption that all of B's outputs are dependent on all of
its inputs.

Unfortunately, its not difficult to imagine a modular synth patch using
multiplexer/demultiplexer blocks to rewire itself internally at
extremely high rates. I doubt that Jack could ever completely handle
this case (multiple graph reorderings within a single process cycle?!)
even given the required information. It would have to assume, instead,
that all of the multiplexer blocks' outputs were dependent on all of
their inputs.

Still, Jack could do better than it does now if it could a) see a
client's internal graph AND b) split the client's callback into multiple
calls according the demands of the current graph.

And, erm, it could do a better job of working out the latencies too.  

[*] Rewriting client B as 2 separate clients won't work in non-trivial
cases because deciding which and how many pieces a client must be split
into can depend on the current overall Jack graph, ie not be known at
compile time.  

~ Simon Jenkins
(Bristol, UK)




-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click
_______________________________________________
Jackit-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jackit-devel