JACK and RT cgroups

classic Classic list List threaded Threaded
70 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

Re: JACK and RT cgroups

Paul Davis
On Wed, Dec 15, 2010 at 8:11 PM, Jack O'Quin <[hidden email]> wrote:

> Implicit in this whole discussion is the assumption that there exist
> applications for which the cgroups scheduling is useful. Since it
> clearly does not work for realtime audio, I am curious to know what it
> *is* good for. Can you please post a link to the rationale?

in all fairness to dhaval, cgroups themselves are not an issue here,
and it is reasonably well established what their utility is. the
specific issue is with the creation of groups that do not have RT
scheduling resources allocated to them, and the way that this
intersects with existing mechanisms that grant access to RT
scheduling.

put another way (c/o torben), ulimit/limits.conf controls a user's
maximum available RT priority, but the arrival of cgroups rt
scheduling configuration means that cgroups control whether a task in
a cgroup can actually get any RT scheduling time. if either RT
priority or the cgroup rt usecs param is 0, then no task in that
cgroup gets RT scheduling.

cgroups rt scheduling conf is actually VERY useful, because its holds
the promise of making the issuance of RT scheduling essentially risk
free. you can configure a machine so that by default, RT tasks only
get X% of the scheduling time, meaning that all the stuff we've heard
for years about "its too dangerous for regular users to have RT
scheduling" goes out the window.

what has happened here is just a mismatch between userspace
configuration and tools, and what is actually need by apps and users.

--p
_______________________________________________
Jack-Devel mailing list
[hidden email]
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org
Reply | Threaded
Open this post in threaded view
|

Re: JACK and RT cgroups

David Henningsson-3
In reply to this post by Dhaval Giani
Heh, a long discussion happened over my night it seems :-) Hopefully I'm
not going off-topic by asking a few generic RT questions.

On 2010-12-15 20:03, Dhaval Giani wrote:

>>> line which would do the exact same thing.
>>
>> 1) What would be meaning of e.g.
>>
>>    cpu.rt_runtime_us = 500000;
>>
>
> Ah, ok.
>
> So let me give a basic background on what rt-cgroups are (just so that
> we have a clearer discussion here). So, cgroups basically is a
> mechanism to group processes (arbitrarily). This is represented as a
> ram based filesystem. Creating a group is as simple as doing a mkdir,
> and when you list the files, you get their values. The values that we
> put in the cgconfig.conf set the values in these files.
>
> Now, for the rtcgroup, you have two control files, namely,
> cpu.rt_runtime_us and cpu.rt_period_us . This means that the cgroup
> will get cpu.rt_period_us microseconds every cpu.rt_period_us
> microseconds. Since the default period is 1second, I set the runtime
> to 500ms, so that means every second, that group can get a maximum of
> 50% CPU.

This is interesting. It sounds similar to what rtkit is trying to
achieve (i e handing out rt priority in a safe way), but the strategy is
somewhat different. Could you elaborate on how they relate to each
other? Are they competing technologies?

What about RLIMIT_RTTIME?

On 2010-12-15 19:32, Dhaval Giani wrote:
 > On Wed, Dec 15, 2010 at 7:27 PM, Peter Nelson<[hidden email]>  wrote:
 >>
 >> Actually this affects RealtimeKit and PulseAudio as well.
 >>
 >
 > I haven't seen pulseaudio having issues. (Or at least no complaints
 > from Lennart)

PulseAudio uses rtkit to get RT priority for its threads. I can confirm
that Ubuntu 11.04 - not yet released - is currently affected, the error
is that Rtkit is not permitted to make anything realtime. Now, this is a
development release and we have some time to fix it before the release,
once we've figured out the best way to do so.

Ubuntu 10.10, the last stable version of Ubuntu, is *not* affected.

Lennart isn't complaining because either Fedora is not affected, or he
has been too busy doing systemd and other things.

--
David Henningsson, Canonical Ltd.
http://launchpad.net/~diwic
_______________________________________________
Jack-Devel mailing list
[hidden email]
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org
Reply | Threaded
Open this post in threaded view
|

Re: JACK and RT cgroups

Fons Adriaensen-2
In reply to this post by Paul Davis
On Wed, Dec 15, 2010 at 08:33:29PM -0500, Paul Davis wrote:

> cgroups rt scheduling conf is actually VERY useful, because its holds
> the promise of making the issuance of RT scheduling essentially risk
> free. you can configure a machine so that by default, RT tasks only
> get X% of the scheduling time, meaning that all the stuff we've heard
> for years about "its too dangerous for regular users to have RT
> scheduling" goes out the window.

I'm not convinced I'd want it on a system used for audio production.
An occasional xrun will affect a few ms of signal. Depending on what
you are doing that may be a problem or not. When doing live recording,
there is  a good chance that a glitch of a few ms can be corrected by
some surgical editing. When driving a concert PA, it may actually go
unnoticed. This will *not* be the case if the RT protection kicks in
as the result of a transient problem. None of your audio code will run
for the next scheduling period, which is typically a second. During that
time, the soundcard will typically repeat the last period(s). Can you
imagine the effect when driving a PA system ? It's a disaster waiting
to happen.

> what has happened here is just a mismatch between userspace
> configuration and tools, and what is actually need by apps and users.

I agree that cgroups can be very useful for some type of systems
(which are probably quite different from a typical single-user
audio workstation). If any error was made, it was not by the people
developing cgroups, but by a distro mainly targeting  'personal'
computer users enabling it. I very much hope the upstream default
will be to disable this, so it won't affect all other distros.

Apart from that there's the administration nightmare resulting from
several overlapping system controlling the same thing: ulimts, rtkit,
cgroups,... same as for consolekit/policykit which interfere with
other mechanisms which is why I don't want them.


Ciao,

--
FA

There are three of them, and Alleline.

_______________________________________________
Jack-Devel mailing list
[hidden email]
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org
Reply | Threaded
Open this post in threaded view
|

Re: JACK and RT cgroups

Dhaval Giani
On Thu, Dec 16, 2010 at 12:00 PM,  <[hidden email]> wrote:

> On Wed, Dec 15, 2010 at 08:33:29PM -0500, Paul Davis wrote:
>
>> cgroups rt scheduling conf is actually VERY useful, because its holds
>> the promise of making the issuance of RT scheduling essentially risk
>> free. you can configure a machine so that by default, RT tasks only
>> get X% of the scheduling time, meaning that all the stuff we've heard
>> for years about "its too dangerous for regular users to have RT
>> scheduling" goes out the window.
>
> I'm not convinced I'd want it on a system used for audio production.
> An occasional xrun will affect a few ms of signal. Depending on what
> you are doing that may be a problem or not. When doing live recording,
> there is  a good chance that a glitch of a few ms can be corrected by
> some surgical editing. When driving a concert PA, it may actually go
> unnoticed. This will *not* be the case if the RT protection kicks in
> as the result of a transient problem. None of your audio code will run
> for the next scheduling period, which is typically a second. During that
> time, the soundcard will typically repeat the last period(s). Can you
> imagine the effect when driving a PA system ? It's a disaster waiting
> to happen.
>

Not exactly. Your code will not run only for the duration you specify.
So, if you are fine giving your group 95ms every 100ms, you can do
that, you could also do 9.5ms every 10ms. The 1 second is a default
value, and in order to get stuff working immediately. The question is
this, suppose your rt application misbehaves and takes the entire CPU,
would you like that additional 5% time to kill the process and restart
it or not? Applications can have bugs, and I feel the rt-cgroup
enhancement is a step in the right direction.

>> what has happened here is just a mismatch between userspace
>> configuration and tools, and what is actually need by apps and users.
>
> I agree that cgroups can be very useful for some type of systems
> (which are probably quite different from a typical single-user
> audio workstation). If any error was made, it was not by the people
> developing cgroups, but by a distro mainly targeting  'personal'
> computer users enabling it. I very much hope the upstream default
> will be to disable this, so it won't affect all other distros.
>

No, the problem is not with that, but with a default setup which works
for most users. I know it does not work for you, and there is no
disagreement in that. However, that policy does work for most users
and turning it off is just a matter of adding a line in a config file,
I am not so inclined to change it.

> Apart from that there's the administration nightmare resulting from
> several overlapping system controlling the same thing: ulimts, rtkit,
> cgroups,... same as for consolekit/policykit which interfere with
> other mechanisms which is why I don't want them.
>

So remove them. It is as simple as a yum erase libcgroup.

However, I would much prefer you working with us, to figure out how we
can exploit this feature. (It is not very hard to have code in jack
itself to handle it (as a patch by Torben shows).)

Dhaval
_______________________________________________
Jack-Devel mailing list
[hidden email]
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org
Reply | Threaded
Open this post in threaded view
|

Re: JACK and RT cgroups

Dhaval Giani
In reply to this post by David Henningsson-3
On Thu, Dec 16, 2010 at 8:48 AM, David Henningsson
<[hidden email]> wrote:

> Heh, a long discussion happened over my night it seems :-) Hopefully I'm not
> going off-topic by asking a few generic RT questions.
>
> On 2010-12-15 20:03, Dhaval Giani wrote:
>>>>
>>>> line which would do the exact same thing.
>>>
>>> 1) What would be meaning of e.g.
>>>
>>>   cpu.rt_runtime_us = 500000;
>>>
>>
>> Ah, ok.
>>
>> So let me give a basic background on what rt-cgroups are (just so that
>> we have a clearer discussion here). So, cgroups basically is a
>> mechanism to group processes (arbitrarily). This is represented as a
>> ram based filesystem. Creating a group is as simple as doing a mkdir,
>> and when you list the files, you get their values. The values that we
>> put in the cgconfig.conf set the values in these files.
>>
>> Now, for the rtcgroup, you have two control files, namely,
>> cpu.rt_runtime_us and cpu.rt_period_us . This means that the cgroup
>> will get cpu.rt_period_us microseconds every cpu.rt_period_us
>> microseconds. Since the default period is 1second, I set the runtime
>> to 500ms, so that means every second, that group can get a maximum of
>> 50% CPU.
>
> This is interesting. It sounds similar to what rtkit is trying to achieve (i
> e handing out rt priority in a safe way), but the strategy is somewhat
> different. Could you elaborate on how they relate to each other? Are they
> competing technologies?
>

I know rtkit and cgroups work together. Going through rtkit has been
on my TODO list for a long time. Now that this week I have some spare
bandwidth, I will take a look through it.

> What about RLIMIT_RTTIME?
>
> On 2010-12-15 19:32, Dhaval Giani wrote:
>> On Wed, Dec 15, 2010 at 7:27 PM, Peter Nelson<[hidden email]>  wrote:
>>>
>>> Actually this affects RealtimeKit and PulseAudio as well.
>>>
>>
>> I haven't seen pulseaudio having issues. (Or at least no complaints
>> from Lennart)
>
> PulseAudio uses rtkit to get RT priority for its threads. I can confirm that
> Ubuntu 11.04 - not yet released - is currently affected, the error is that
> Rtkit is not permitted to make anything realtime. Now, this is a development
> release and we have some time to fix it before the release, once we've
> figured out the best way to do so.
>

Right. The simplest way is to have an rtkit cgroup with some rt
runtime assigned to it and assigning threads to that.

> Ubuntu 10.10, the last stable version of Ubuntu, is *not* affected.
>
> Lennart isn't complaining because either Fedora is not affected, or he has
> been too busy doing systemd and other things.
>

Fedora is not affected because they turned the default group by default.

Now that you mention systemd, you are going to face this problem with
systemd as well. With the big hullaballoo over group scheduling being
used to improve interactivity, systemd now assigns all sessions to
their own cgroups. These cgroups do not have any rt_runtime defined
(and should not as well). You are going to face this problem there as
well. So it does need to be fixed sometime soon (very soon infact),
because there will be a lot of screaming and ranting again. And again,
using cpu cgroups *helps* the average desktop user. It might not help
JACK because jack wants to use the real time capabilities, but it
helps the majority and in the opinion of most a step in the right
direction. So, jack will need to learn to with cgroups. The patch by
Torben is an important step, and we will look to fix it up completely
in a few more days. Hopefully the patch will be acceptable to jack.

Thanks,
Dhaval
_______________________________________________
Jack-Devel mailing list
[hidden email]
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org
Reply | Threaded
Open this post in threaded view
|

Re: JACK and RT cgroups

torbenh
In reply to this post by David Henningsson-3
On Thu, Dec 16, 2010 at 08:48:13AM +0100, David Henningsson wrote:

> Heh, a long discussion happened over my night it seems :-) Hopefully
> I'm not going off-topic by asking a few generic RT questions.
>
> On 2010-12-15 20:03, Dhaval Giani wrote:
> >>>line which would do the exact same thing.
> >>
> >>1) What would be meaning of e.g.
> >>
> >>   cpu.rt_runtime_us = 500000;
> >>
> >
> >Ah, ok.
> >
> >So let me give a basic background on what rt-cgroups are (just so that
> >we have a clearer discussion here). So, cgroups basically is a
> >mechanism to group processes (arbitrarily). This is represented as a
> >ram based filesystem. Creating a group is as simple as doing a mkdir,
> >and when you list the files, you get their values. The values that we
> >put in the cgconfig.conf set the values in these files.
> >
> >Now, for the rtcgroup, you have two control files, namely,
> >cpu.rt_runtime_us and cpu.rt_period_us . This means that the cgroup
> >will get cpu.rt_period_us microseconds every cpu.rt_period_us
> >microseconds. Since the default period is 1second, I set the runtime
> >to 500ms, so that means every second, that group can get a maximum of
> >50% CPU.
>
> This is interesting. It sounds similar to what rtkit is trying to
> achieve (i e handing out rt priority in a safe way), but the
> strategy is somewhat different. Could you elaborate on how they
> relate to each other? Are they competing technologies?

they are not competing. since obviously RtKit cant handout the rt
privilieges anymore in ubuntu 11.4
RtKit needs to get fixed to move the threads into the appropriate
(insert some definition of appropriate) cgroup before elevating the
privs.

in my opinion RtKit is pretty much obsolete.
it doesnt really do anything the kernel doesnt do already.

we spent quite some effort to get rid of setuid root binaries,
and just have kernel allow users to use SCHED_FIFO.
we are there on debian squeeze.

user installs jack, gets asked whether he wants rt scheduling in the
audio group and then it just works, out of the box.
the kernel even prevents RT threads from completely burning the cpu
since 2.6.26...

the cgroups now allow a much finer grained tuning of the time slices.
in our experience on the desktop nobody wants finegrained tuning though.
and servers dont really run RT tasks.

dunno... maybe there is some benefit to RtKit, maybe it can know the
correct cgroup to add a task to ?

>
> What about RLIMIT_RTTIME?

never heard about it.
and its a bit hard to find docs on this.
but it really starts to get confusing. we have quite a lot of mechanisms
configuring a single thing.

the problem is that defaults are always dont allow, and you need
permission from all instances.

>
> On 2010-12-15 19:32, Dhaval Giani wrote:
> > On Wed, Dec 15, 2010 at 7:27 PM, Peter Nelson<[hidden email]>  wrote:
> >>
> >> Actually this affects RealtimeKit and PulseAudio as well.
> >>
> >
> > I haven't seen pulseaudio having issues. (Or at least no complaints
> > from Lennart)
>
> PulseAudio uses rtkit to get RT priority for its threads. I can
> confirm that Ubuntu 11.04 - not yet released - is currently
> affected, the error is that Rtkit is not permitted to make anything
> realtime. Now, this is a development release and we have some time
> to fix it before the release, once we've figured out the best way to
> do so.
>
> Ubuntu 10.10, the last stable version of Ubuntu, is *not* affected.
>
> Lennart isn't complaining because either Fedora is not affected, or
> he has been too busy doing systemd and other things.
>
> --
> David Henningsson, Canonical Ltd.
> http://launchpad.net/~diwic
> _______________________________________________
> Jack-Devel mailing list
> [hidden email]
> http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org

--
torben Hohn
_______________________________________________
Jack-Devel mailing list
[hidden email]
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org
Reply | Threaded
Open this post in threaded view
|

Re: JACK and RT cgroups

Paul Davis
In reply to this post by Fons Adriaensen-2
On Thu, Dec 16, 2010 at 6:00 AM,  <[hidden email]> wrote:

> On Wed, Dec 15, 2010 at 08:33:29PM -0500, Paul Davis wrote:
>
>> cgroups rt scheduling conf is actually VERY useful, because its holds
>> the promise of making the issuance of RT scheduling essentially risk
>> free. you can configure a machine so that by default, RT tasks only
>> get X% of the scheduling time, meaning that all the stuff we've heard
>> for years about "its too dangerous for regular users to have RT
>> scheduling" goes out the window.
>
> I'm not convinced I'd want it on a system used for audio production.
> An occasional xrun will affect a few ms of signal. Depending on what
> you are doing that may be a problem or not. When doing live recording,
> there is  a good chance that a glitch of a few ms can be corrected by
> some surgical editing. When driving a concert PA, it may actually go
> unnoticed. This will *not* be the case if the RT protection kicks in
> as the result of a transient problem. None of your audio code will run
> for the next scheduling period, which is typically a second.

given that you have to set rt_runtime_us in order to even get RT
scheduling within a given cgroup, the additional step of setting the
period doesn't seem like much of an extra burden, and really avoids
this issue.

but i would agree that for a custom built system, you would disable
RT_GROUP_SCHED entirely (note: this is not the same as disabling
cgroups, which by themselves don't have any real impact on this
stuff). you would then run a different risk, its true, but its one
that we've actually been facing for nearly a decade: apps locking up
the machine entirely. given that this has been the primary objection
from the kernel guys about making RT scheduling easier to access for
users, i see this as a reasonable tradeoff. you can still build/run a
kernel without RT_GROUP_SCHED for embedded realtime systems (ironic,
eh?), and users of mainstream distros can still get RT scheduling
without any risk of locking their systems.

however, even with that said, in this era where almost all new systems
have at least 2 cores, even this assessment is perhaps a little off.
RT_GROUP_SCHED feels like an a combination of 2 things to me:

    a) an admin-oriented approach to setting up a compute server
    b) a solution to RT-scheduled tasks taking over a uniprocessor

now, its also clearly true that applications are expanding to run on
more cores too (e.g. ardour3, which will expand to any number of
processors), so (b) is not entirely accurate. but the majority of RT
apps are NOT going to run multiple RT threads precisely for the reason
that managing multiple RT threads is quite tricky if you want to stay
RT-safe (technically, on linux at present, its not possible to do in a
truly RT way, though you can get close enough for all practical
purposes).

so again, i think that the future for purpose built kernels for audio
work is to likely to have RT_GROUP_SCHED disabled, but i can't say
that i think its a bad thing that it exists, since it will hopefully
stop the kernel people and distro managers from continually raising
objections to userspace providing easy out of the box access to RT
scheduling (and lets not forget memlock :)

--p
_______________________________________________
Jack-Devel mailing list
[hidden email]
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org
Reply | Threaded
Open this post in threaded view
|

Re: JACK and RT cgroups

torbenh
In reply to this post by Fons Adriaensen-2
On Thu, Dec 16, 2010 at 12:00:36PM +0100, [hidden email] wrote:

> On Wed, Dec 15, 2010 at 08:33:29PM -0500, Paul Davis wrote:
>
> > cgroups rt scheduling conf is actually VERY useful, because its holds
> > the promise of making the issuance of RT scheduling essentially risk
> > free. you can configure a machine so that by default, RT tasks only
> > get X% of the scheduling time, meaning that all the stuff we've heard
> > for years about "its too dangerous for regular users to have RT
> > scheduling" goes out the window.
>
> I'm not convinced I'd want it on a system used for audio production.
> An occasional xrun will affect a few ms of signal. Depending on what
> you are doing that may be a problem or not. When doing live recording,
> there is  a good chance that a glitch of a few ms can be corrected by
> some surgical editing. When driving a concert PA, it may actually go
> unnoticed. This will *not* be the case if the RT protection kicks in
> as the result of a transient problem. None of your audio code will run
> for the next scheduling period, which is typically a second. During that
> time, the soundcard will typically repeat the last period(s). Can you
> imagine the effect when driving a PA system ? It's a disaster waiting
> to happen.

this behaviour is the default for ANY kernel since 2.6.26
-> cat /proc/sys/kernel/sched_rt_runtime_us
950000

now you can configure it for groups.
come on guys.
you are all acting like this is something new.
its not.

it just moved into groups, and since the default setting for a new group
is 0 we start seeing problems.
we just need to make sure, that the default group has it set.



>
> > what has happened here is just a mismatch between userspace
> > configuration and tools, and what is actually need by apps and users.
>
> I agree that cgroups can be very useful for some type of systems
> (which are probably quite different from a typical single-user
> audio workstation). If any error was made, it was not by the people
> developing cgroups, but by a distro mainly targeting  'personal'
> computer users enabling it. I very much hope the upstream default
> will be to disable this, so it won't affect all other distros.
>
> Apart from that there's the administration nightmare resulting from
> several overlapping system controlling the same thing: ulimts, rtkit,
> cgroups,... same as for consolekit/policykit which interfere with
> other mechanisms which is why I don't want them.
>
>
> Ciao,
>
> --
> FA
>
> There are three of them, and Alleline.
>
> _______________________________________________
> Jack-Devel mailing list
> [hidden email]
> http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org

--
torben Hohn
_______________________________________________
Jack-Devel mailing list
[hidden email]
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org
Reply | Threaded
Open this post in threaded view
|

Re: JACK and RT cgroups

Paul Davis
In reply to this post by torbenh
On Thu, Dec 16, 2010 at 6:56 AM, torbenh <[hidden email]> wrote:

> in my opinion RtKit is pretty much obsolete.
> it doesnt really do anything the kernel doesnt do already.

long time readers may be aware that i asked lennart about the
relationship between cgroups and RtKit a year or two ago, and he
conceded after checking on cgroups that RtKit was pretty redundant
given their existence and capabilities.

on the other hand, it sounds as if he has subsequently talked to
dhaval et al, and feels that actually they (cgroups) still don't get
it right for RT scheduling access. but i don't know.

> user installs jack, gets asked whether he wants rt scheduling in the
> audio group and then it just works, out of the box.
> the kernel even prevents RT threads from completely burning the cpu
> since 2.6.26...

this is important point that i forget in my response to fons just now.

> the cgroups now allow a much finer grained tuning of the time slices.
> in our experience on the desktop nobody wants finegrained tuning though.
> and servers dont really run RT tasks.

right, that's why RT_GROUP_SCHED feels to me like a solution for a
compute server, and of little use to just about everyone else.
_______________________________________________
Jack-Devel mailing list
[hidden email]
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org
Reply | Threaded
Open this post in threaded view
|

Re: JACK and RT cgroups

torbenh
In reply to this post by Paul Davis
On Thu, Dec 16, 2010 at 07:08:57AM -0500, Paul Davis wrote:

> On Thu, Dec 16, 2010 at 6:00 AM,  <[hidden email]> wrote:
> > On Wed, Dec 15, 2010 at 08:33:29PM -0500, Paul Davis wrote:
> >
> >> cgroups rt scheduling conf is actually VERY useful, because its holds
> >> the promise of making the issuance of RT scheduling essentially risk
> >> free. you can configure a machine so that by default, RT tasks only
> >> get X% of the scheduling time, meaning that all the stuff we've heard
> >> for years about "its too dangerous for regular users to have RT
> >> scheduling" goes out the window.
> >
> > I'm not convinced I'd want it on a system used for audio production.
> > An occasional xrun will affect a few ms of signal. Depending on what
> > you are doing that may be a problem or not. When doing live recording,
> > there is  a good chance that a glitch of a few ms can be corrected by
> > some surgical editing. When driving a concert PA, it may actually go
> > unnoticed. This will *not* be the case if the RT protection kicks in
> > as the result of a transient problem. None of your audio code will run
> > for the next scheduling period, which is typically a second.
>
> given that you have to set rt_runtime_us in order to even get RT
> scheduling within a given cgroup, the additional step of setting the
> period doesn't seem like much of an extra burden, and really avoids
> this issue.

this "issue" is there since 2.6.26
oh my...
lennart created rtkit, in order to "implement" this feature,
you dont want it, but never ever bothered to turn it off.

>
> but i would agree that for a custom built system, you would disable
> RT_GROUP_SCHED entirely (note: this is not the same as disabling
> cgroups, which by themselves don't have any real impact on this
> stuff). you would then run a different risk, its true, but its one
> that we've actually been facing for nearly a decade: apps locking up
> the machine entirely. given that this has been the primary objection

YOU CAN ONLY LOCK THE MACHINE WITH:
echo 1000000 >/proc/sys/kernel/sched_rt_runtime_us

with the default you cant.



> from the kernel guys about making RT scheduling easier to access for
> users, i see this as a reasonable tradeoff. you can still build/run a
> kernel without RT_GROUP_SCHED for embedded realtime systems (ironic,
> eh?), and users of mainstream distros can still get RT scheduling
> without any risk of locking their systems.
>
> however, even with that said, in this era where almost all new systems
> have at least 2 cores, even this assessment is perhaps a little off.
> RT_GROUP_SCHED feels like an a combination of 2 things to me:
>
>     a) an admin-oriented approach to setting up a compute server
>     b) a solution to RT-scheduled tasks taking over a uniprocessor
>
> now, its also clearly true that applications are expanding to run on
> more cores too (e.g. ardour3, which will expand to any number of
> processors), so (b) is not entirely accurate. but the majority of RT
> apps are NOT going to run multiple RT threads precisely for the reason
> that managing multiple RT threads is quite tricky if you want to stay
> RT-safe (technically, on linux at present, its not possible to do in a
> truly RT way, though you can get close enough for all practical
> purposes).
>
> so again, i think that the future for purpose built kernels for audio
> work is to likely to have RT_GROUP_SCHED disabled, but i can't say
> that i think its a bad thing that it exists, since it will hopefully
> stop the kernel people and distro managers from continually raising
> objections to userspace providing easy out of the box access to RT


> scheduling (and lets not forget memlock :)

RtKit doesnt provide memlock.
thanks RtKit... you can go home now.

>
> --p
> _______________________________________________
> Jack-Devel mailing list
> [hidden email]
> http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org

--
torben Hohn
_______________________________________________
Jack-Devel mailing list
[hidden email]
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org
Reply | Threaded
Open this post in threaded view
|

Re: JACK and RT cgroups

Dhaval Giani
In reply to this post by Paul Davis
On Thu, Dec 16, 2010 at 1:12 PM, Paul Davis <[hidden email]> wrote:

> On Thu, Dec 16, 2010 at 6:56 AM, torbenh <[hidden email]> wrote:
>
>> in my opinion RtKit is pretty much obsolete.
>> it doesnt really do anything the kernel doesnt do already.
>
> long time readers may be aware that i asked lennart about the
> relationship between cgroups and RtKit a year or two ago, and he
> conceded after checking on cgroups that RtKit was pretty redundant
> given their existence and capabilities.
>
> on the other hand, it sounds as if he has subsequently talked to
> dhaval et al, and feels that actually they (cgroups) still don't get
> it right for RT scheduling access. but i don't know.
>

yes, rt-cgroups did not quite meet his need which led to rtkit coming
out. Though we are going to have to look at it pretty soon again :-)

>> user installs jack, gets asked whether he wants rt scheduling in the
>> audio group and then it just works, out of the box.
>> the kernel even prevents RT threads from completely burning the cpu
>> since 2.6.26...
>
> this is important point that i forget in my response to fons just now.
>
>> the cgroups now allow a much finer grained tuning of the time slices.
>> in our experience on the desktop nobody wants finegrained tuning though.
>> and servers dont really run RT tasks.
>
> right, that's why RT_GROUP_SCHED feels to me like a solution for a
> compute server, and of little use to just about everyone else.

Which is also why we set a default of 0 :-). Anyone who would need rt
time would set it up :-).

Thanks!
Dhaval
_______________________________________________
Jack-Devel mailing list
[hidden email]
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org
Reply | Threaded
Open this post in threaded view
|

Re: JACK and RT cgroups

torbenh
On Thu, Dec 16, 2010 at 01:36:47PM +0100, Dhaval Giani wrote:

> On Thu, Dec 16, 2010 at 1:12 PM, Paul Davis <[hidden email]> wrote:
> > On Thu, Dec 16, 2010 at 6:56 AM, torbenh <[hidden email]> wrote:
> >
> >> in my opinion RtKit is pretty much obsolete.
> >> it doesnt really do anything the kernel doesnt do already.
> >
> > long time readers may be aware that i asked lennart about the
> > relationship between cgroups and RtKit a year or two ago, and he
> > conceded after checking on cgroups that RtKit was pretty redundant
> > given their existence and capabilities.
> >
> > on the other hand, it sounds as if he has subsequently talked to
> > dhaval et al, and feels that actually they (cgroups) still don't get
> > it right for RT scheduling access. but i don't know.
> >
>
> yes, rt-cgroups did not quite meet his need which led to rtkit coming
> out. Though we are going to have to look at it pretty soon again :-)
>
> >> user installs jack, gets asked whether he wants rt scheduling in the
> >> audio group and then it just works, out of the box.
> >> the kernel even prevents RT threads from completely burning the cpu
> >> since 2.6.26...
> >
> > this is important point that i forget in my response to fons just now.
> >
> >> the cgroups now allow a much finer grained tuning of the time slices.
> >> in our experience on the desktop nobody wants finegrained tuning though.
> >> and servers dont really run RT tasks.
> >
> > right, that's why RT_GROUP_SCHED feels to me like a solution for a
> > compute server, and of little use to just about everyone else.
>
> Which is also why we set a default of 0 :-). Anyone who would need rt
> time would set it up :-).

the problem is that for a bit low-latency audio, you need rt, and people
using this are musicians or even normal people who like to use voip.

these dont set it up.

>
> Thanks!
> Dhaval
> _______________________________________________
> Jack-Devel mailing list
> [hidden email]
> http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org

--
torben Hohn
_______________________________________________
Jack-Devel mailing list
[hidden email]
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org
Reply | Threaded
Open this post in threaded view
|

Re: JACK and RT cgroups

Paul Davis
In reply to this post by Dhaval Giani
On Thu, Dec 16, 2010 at 7:36 AM, Dhaval Giani <[hidden email]> wrote:
>
> Which is also why we set a default of 0 :-). Anyone who would need rt
> time would set it up :-).

I know you put a smiley on there, but fundamentally, this is the heart
of the attitude that led us (the pro-audio/music creation) community
to have spent so many years endlessly fighting kernel and userspace
decisions. Its actually *precisely* the people who need RT the most
who don't know how to, and don't want to know how to, set it up. How
anyone can justify shipping an OS in 2010 in which anyone needs to
setup anything in order to get deterministic scheduling is ... well, i
think its totally ludicrous.
_______________________________________________
Jack-Devel mailing list
[hidden email]
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org
Reply | Threaded
Open this post in threaded view
|

Re: JACK and RT cgroups

Gabriel M. Beddingfield-2
In reply to this post by Paul Davis
On Thursday, December 16, 2010 06:12:34 am Paul Davis wrote:
> > the cgroups now allow a much finer grained tuning of
> > the time slices. in our experience on the desktop
> > nobody wants finegrained tuning though. and servers
> > dont really run RT tasks.
>
> right, that's why RT_GROUP_SCHED feels to me like a
> solution for a compute server, and of little use to just
> about everyone else.

Yes, when I stumbled in to this about 2 years ago, I came to
the same conclusion (esp. when you read the kernel
documentation for cgroups) -- that this was a solution for
large servers (universities, labs, etc), and had nothing to
do with the desktop.  At that time, I disabled it and simply
knew to watch out for it.

That's part of the reason why nobody complained.  Many of us
couldn't imagine why it would be useful on the desktop or
embedded systems (primarily single-user).

-gabriel
_______________________________________________
Jack-Devel mailing list
[hidden email]
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org
Reply | Threaded
Open this post in threaded view
|

Re: JACK and RT cgroups

Dominique Michel-2
In reply to this post by Dhaval Giani
Le Wed, 15 Dec 2010 20:07:37 +0100,
Dhaval Giani <[hidden email]> a écrit :

> > Turn about again, and i suggest that the jack/RT implications as
> > part of "being fair to all, with no favourites' weren't given any
> > consideration, or you wouldn't be here now.
> >
>
> Someone made a noise, and so I knew about it. Not about jack being a
> favourite. If projectX would have an issue, and I would be informed, I
> would be equally willing to help them. I however would not be in a
> position to keep a look at all projects out there and what issues they
> would face if featureY came into existence.
>
> > I'm not throwing a tantrum, just staggered that something like this
> > could happen in the first place. It just seems so obvious, as a
> > logical process of feature construction, to consider wider
> > implications, as you wrote, for all users, including chaps that use,
> > and/or code for, jack/RT.
> >
>
> Considering the fact that its only JACK facing issues (or at least
> complaining), that too after two years of a feature being in existence
> and after about a year for the default cgroup being created, I think
> the decision was just fine.

I am a non coding project administrator. My distro of choice is gentoo,
so I am able to configure, compile and install a kernel.

As such, I think than the less you can make is to add a text into the
kernel help that address the fact that RT_GROUP_SCHED and CGROUPS will
break any rt enabled software without expropriated setup, and that
with a pointer to the documentation that will show how to do such a
setup.

Udev is a very good piece of software, but almost impossible to setup
for the casual user. I remember the beginning of udev, all the linux
forums was full of threads about how to config udev. It was a real
nightmare for many users. And also, it took years for most distribution
to get it to work out of the box in most cases. So, I also really hope
that this cgroup function will not make like udev, I hope this because
many functions from the rt patchset are slowly incorporated into the
mainline kernel. This fact alone is sufficient for me to consider than
the linux audio community is nor a special or corner case. In fact, rt
can be used with good benefits for every kind of software that need
fast in/out at the hardware level, and it is a lot of them.

The users base of gnu/linux is in fast progression, but issues like
udev at the beginning are only making than potentially new linux users
will only try it, laugh, reboot, issue a "format c:" and forget it for
a whyle. So I really hope the cgroup will not turn into such a farce.

I also think than distributions like ubuntu must understand that such a
new feature is not wanted for a general distribution, that because it is
a certitude than, for the reason explained above, it will break
things for a non negligible part of their potential/existing users.

This is my 2 cents contrib.

Ciao,
Dominique

>
> Dhaval

--
"We have the heroes we deserve."
_______________________________________________
Jack-Devel mailing list
[hidden email]
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org
Reply | Threaded
Open this post in threaded view
|

Re: JACK and RT cgroups

Dhaval Giani
On Thu, Dec 16, 2010 at 7:53 PM, Dominique Michel
<[hidden email]> wrote:

> Le Wed, 15 Dec 2010 20:07:37 +0100,
> Dhaval Giani <[hidden email]> a écrit :
>
>> > Turn about again, and i suggest that the jack/RT implications as
>> > part of "being fair to all, with no favourites' weren't given any
>> > consideration, or you wouldn't be here now.
>> >
>>
>> Someone made a noise, and so I knew about it. Not about jack being a
>> favourite. If projectX would have an issue, and I would be informed, I
>> would be equally willing to help them. I however would not be in a
>> position to keep a look at all projects out there and what issues they
>> would face if featureY came into existence.
>>
>> > I'm not throwing a tantrum, just staggered that something like this
>> > could happen in the first place. It just seems so obvious, as a
>> > logical process of feature construction, to consider wider
>> > implications, as you wrote, for all users, including chaps that use,
>> > and/or code for, jack/RT.
>> >
>>
>> Considering the fact that its only JACK facing issues (or at least
>> complaining), that too after two years of a feature being in existence
>> and after about a year for the default cgroup being created, I think
>> the decision was just fine.
>
> I am a non coding project administrator. My distro of choice is gentoo,
> so I am able to configure, compile and install a kernel.
>
> As such, I think than the less you can make is to add a text into the
> kernel help that address the fact that RT_GROUP_SCHED and CGROUPS will
> break any rt enabled software without expropriated setup, and that
> with a pointer to the documentation that will show how to do such a
> setup.
>

I am still not seeing where RT_GROUP_SCHED and CGROUPS breaks any rt
enabled software. As has been repeated on this thread countless number
of times, there is *no* issue in the kernel. The issue is that by
default the userspace tools create a default group (I still hold to
the view that it is the right thing to do.). An application which uses
rt privileges is *special*. Either,
a) The application should make it much easier for the user to use this
capability (by for example creating the appropriate cgroups and so
on), so that it works out of the box or,
b) Inform the user what he needs to do very clearly and have the user
do the hard work.

What is being repeated again and again that making a one line change
to a configuration file is going to discourage new users. As I have
mentioned again and again, this has been in there for a few months
now, and if it were such a big deal, there would have been tons of bug
reports. Also, this problem is going to come up again once systemd
comes into play. So it has to be fixed by the application as opposed
to people coming about and complaining how Linux developers are
totally against the jack community and are out to get them by making
such changes.

So, as has been done in the past in this thread, there are folks who
have stepped up to the challenge, and there are patches already
floating about. Instead of encouraging them, and supporting them (with
reviews and the like), instead folks are coming, repeating this
conspiracy theory.

The only way to move forward is for rt applications to be aware that
Linux is different, and if you want your application to work out of
the box, some changes need to be made.

This is my last post on non-technical issues in this thread.

Dhaval
_______________________________________________
Jack-Devel mailing list
[hidden email]
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org
Reply | Threaded
Open this post in threaded view
|

Re: JACK and RT cgroups

Alex Stone
On Thu, Dec 16, 2010 at 10:01 PM, Dhaval Giani <[hidden email]> wrote:

> The only way to move forward is for rt applications to be aware that
> Linux is different, and if you want your application to work out of
> the box, some changes need to be made.
>
> This is my last post on non-technical issues in this thread.
>
> Dhaval


Dhaval, so every RT app (jack or alsa) that we currently use,
successfully, (and there are many) will have to be rewritten to
accommodate another group of users within the linux community, simply
because we're "less equal".

Makes an irony of your statement concerning "no special treatment for
any one group" doesn't it.

I can't help think this is driven by potential server sales, and
little else but lip service for the rest.

Still, we have what you've decided we should have, so i guess we'll
need to make the best of it, or not, as the ordinary user decides.

Alex.
_______________________________________________
Jack-Devel mailing list
[hidden email]
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org
Reply | Threaded
Open this post in threaded view
|

Re: JACK and RT cgroups

Fernando Lopez-Lezcano
In reply to this post by Paul Davis
On 12/16/2010 04:44 AM, Paul Davis wrote:

> On Thu, Dec 16, 2010 at 7:36 AM, Dhaval Giani<[hidden email]>  wrote:
>>
>> Which is also why we set a default of 0 :-). Anyone who would need rt
>> time would set it up :-).
>
> I know you put a smiley on there, but fundamentally, this is the heart
> of the attitude that led us (the pro-audio/music creation) community
> to have spent so many years endlessly fighting kernel and userspace
> decisions. Its actually *precisely* the people who need RT the most
> who don't know how to, and don't want to know how to, set it up. How
> anyone can justify shipping an OS in 2010 in which anyone needs to
> setup anything in order to get deterministic scheduling is ... well, i
> think its totally ludicrous.

I very much agree. This periodic "oh no, it is difficult again and we
have to fix it!" thing with regards to rt scheduling will keep pushing
users to other platforms (and after 10+ years is getting tiring :-).

Which begs the basic question: why is it that running in the rt
scheduler is not enabled by default? I imagine the answer being: "normal
users don't need to use rt scheduling". I would say, well, we _are_
normal users. Is this to protect users from a runaway rt process and a
DOS attack? If the answer is yes, then, should we also protect users
from a program that could erase their whole home directory tree if they
run the wrong one?

-- Fernando
_______________________________________________
Jack-Devel mailing list
[hidden email]
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org
Reply | Threaded
Open this post in threaded view
|

Re: JACK and RT cgroups

Fernando Lopez-Lezcano
In reply to this post by Dhaval Giani
On 12/16/2010 03:17 AM, Dhaval Giani wrote:

> On Thu, Dec 16, 2010 at 12:00 PM,<[hidden email]>  wrote:
>> I agree that cgroups can be very useful for some type of systems
>> (which are probably quite different from a typical single-user
>> audio workstation). If any error was made, it was not by the people
>> developing cgroups, but by a distro mainly targeting  'personal'
>> computer users enabling it. I very much hope the upstream default
>> will be to disable this, so it won't affect all other distros.
>
> No, the problem is not with that, but with a default setup which works
> for most users. I know it does not work for you, and there is no
> disagreement in that. However, that policy does work for most users
> and turning it off is just a matter of adding a line in a config file,
> I am not so inclined to change it.

As a packager (Planet CCRMA, rpm based) I would very much prefer that
adding a _file_ somewhere (not replacing a file, or editing a file)
would have the intended effect (ie: disabling the whole system, or
appropriately configuring cgroups to allow for rt shceduling).

If that were the case it would be trivial to create a package that does
the right thing. The user installs it and voila, he/she/it has now
access to rt scheduling as before (and does not need any knowledge of
the internals).

_Changing_ a file, on the other hand, is not so easily done through
packaging. Actually, it is a pain.

-- Fernando
_______________________________________________
Jack-Devel mailing list
[hidden email]
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org
Reply | Threaded
Open this post in threaded view
|

Re: JACK and RT cgroups

Fernando Lopez-Lezcano
In reply to this post by Fernando Lopez-Lezcano
On 12/16/2010 01:52 PM, Fernando Lopez-Lezcano wrote:
> Which begs the basic question: why is it that running in the rt
> scheduler is not enabled by default?

No need to answer this..., sorry for the noise
-- Fernando

> I imagine the answer being: "normal
> users don't need to use rt scheduling". I would say, well, we _are_
> normal users. Is this to protect users from a runaway rt process and a
> DOS attack? If the answer is yes, then, should we also protect users
> from a program that could erase their whole home directory tree if they
> run the wrong one?
_______________________________________________
Jack-Devel mailing list
[hidden email]
http://lists.jackaudio.org/listinfo.cgi/jack-devel-jackaudio.org
1234