Some of the core rock libraries are currently using std::string for
tagging frames and stuff like that. Being a bit hard RT conscious I want
to make sure that these libraries *are* hard RT compatible.
Given that:
* the content of the strings is of bounded size (i.e. one could
presize them)
* it is part of normal C++ types (i.e. we could write specialized copy
operators)
Is it possible to make them hard RT compatible ?
Using the OCL rt_string is not an option as it would add a dependency on
OCL for the "library" part of rock (which is meant to remain
framework-independent)
std::string in hard realtime contexts
On Jul 12, 2011, at 12:08 , Sylvain Joyeux wrote:
> Some of the core rock libraries are currently using std::string for
> tagging frames and stuff like that. Being a bit hard RT conscious I want
> to make sure that these libraries *are* hard RT compatible.
>
> Given that:
> * the content of the strings is of bounded size (i.e. one could
> presize them)
> * it is part of normal C++ types (i.e. we could write specialized copy
> operators)
>
> Is it possible to make them hard RT compatible ?
You have to be incredibly careful to not accidentally use a std::string in a non-RT way, or that the compiler doesn't "helpfully" do it for you ... personally, I'd prefer to see std::string entirely removed from RTT if hard-RT is truly a goal. Maybe v4? ;-)
> Using the OCL rt_string is not an option as it would add a dependency on
> OCL for the "library" part of rock (which is meant to remain
> framework-independent)
I thought rt_string had been moved into RTT proper as part of v2? Is that too much of a dependancy for you? Certainly in v1 it is in OCL.
We'd thought at one point about suggesting to pull out TLSF as a whole, from RTT. This would make TLSF (and rt_string as a trivial extension) available to projects that don't need RTT, but that live in the same subsystem (think a hardware driver that wants to be able to return descriptive error information, but that doesn't want to depend on RTT as a whole).
S
std::string in hard realtime contexts
On 07/12/2011 06:26 PM, S Roderick wrote:
>> Is it possible to make them hard RT compatible ?
>
> You have to be incredibly careful to not accidentally use a std::string in a non-RT way, or that the compiler doesn't "helpfully" do it for you ... personally, I'd prefer to see std::string entirely removed from RTT if hard-RT is truly a goal. Maybe v4? ;-)
I guessed that already ;-). From your experience / careful analysis, is
there a way to do that properly ? The only needed operation, in the end,
is copying around...
>> Using the OCL rt_string is not an option as it would add a dependency on
>> OCL for the "library" part of rock (which is meant to remain
>> framework-independent)
>
> I thought rt_string had been moved into RTT proper as part of v2? Is that too much of a dependancy for you? Certainly in v1 it is in OCL.
RTT is *definitely* too much of a dependency. Even using TLSF is a bit
of a problem, as the library users would need to initialize TLSF in
main() (or does TLSF use "sane defaults" if none are provided ?)
> We'd thought at one point about suggesting to pull out TLSF as a whole, from RTT. This would make TLSF (and rt_string as a trivial extension) available to projects that don't need RTT, but that live in the same subsystem (think a hardware driver that wants to be able to return descriptive error information, but that doesn't want to depend on RTT as a whole).
That's exactly my case. How much work do you think it would be ? Would
the library user need to know about TLSF or would it be transparent for
him ?
std::string in hard realtime contexts
On Jul 12, 2011, at 12:53 , Sylvain Joyeux wrote:
> On 07/12/2011 06:26 PM, S Roderick wrote:
>>> Is it possible to make them hard RT compatible ?
>>
>> You have to be incredibly careful to not accidentally use a std::string in a non-RT way, or that the compiler doesn't "helpfully" do it for you ... personally, I'd prefer to see std::string entirely removed from RTT if hard-RT is truly a goal. Maybe v4? ;-)
> I guessed that already ;-). From your experience / careful analysis, is there a way to do that properly ? The only needed operation, in the end, is copying around...
No, we just don't use std::string if at all possible. We use "const char*" for fixed values, boost::iostream + char[] (fixed size = a weak point) for streaming, and OCL::string (rt_string in v2) for logging, variable sized strings, data ports, etc.
>>> Using the OCL rt_string is not an option as it would add a dependency on
>>> OCL for the "library" part of rock (which is meant to remain
>>> framework-independent)
>>
>> I thought rt_string had been moved into RTT proper as part of v2? Is that too much of a dependancy for you? Certainly in v1 it is in OCL.
> RTT is *definitely* too much of a dependency. Even using TLSF is a bit of a problem, as the library users would need to initialize TLSF in main() (or does TLSF use "sane defaults" if none are provided ?)
TLSF does _not_ have sane defaults. By default it is uninitialized, which is quite poor, but also (in reality) a reasonable design choice. I don't imagine it would take much to have it self-initialize to some default size (probably given as a build-time configuration parameter)
>> We'd thought at one point about suggesting to pull out TLSF as a whole, from RTT. This would make TLSF (and rt_string as a trivial extension) available to projects that don't need RTT, but that live in the same subsystem (think a hardware driver that wants to be able to return descriptive error information, but that doesn't want to depend on RTT as a whole).
> That's exactly my case. How much work do you think it would be ? Would the library user need to know about TLSF or would it be transparent for him ?
TLSF itself is only one source file (IIRC) plus a very small amount of configuration. IIRC Peter integrated some locking into it, for multi-thread safety, which I think is dependant on RTT. Except for that, it should be straightforward.
S
std::string in hard realtime contexts
On 07/12/2011 07:03 PM, Stephen Roderick wrote:
> On Jul 12, 2011, at 12:53 , Sylvain Joyeux wrote:
>
>> On 07/12/2011 06:26 PM, S Roderick wrote:
>>>> Is it possible to make them hard RT compatible ?
>>>
>>> You have to be incredibly careful to not accidentally use a std::string in a non-RT way, or that the compiler doesn't "helpfully" do it for you ... personally, I'd prefer to see std::string entirely removed from RTT if hard-RT is truly a goal. Maybe v4? ;-)
>> I guessed that already ;-). From your experience / careful analysis, is there a way to do that properly ? The only needed operation, in the end, is copying around...
>
> No, we just don't use std::string if at all possible. We use "const char*" for fixed values, boost::iostream + char[] (fixed size = a weak point) for streaming, and OCL::string (rt_string in v2) for logging, variable sized strings, data ports, etc.
>
>>>> Using the OCL rt_string is not an option as it would add a dependency on
>>>> OCL for the "library" part of rock (which is meant to remain
>>>> framework-independent)
>>>
>>> I thought rt_string had been moved into RTT proper as part of v2? Is that too much of a dependancy for you? Certainly in v1 it is in OCL.
>> RTT is *definitely* too much of a dependency. Even using TLSF is a bit of a problem, as the library users would need to initialize TLSF in main() (or does TLSF use "sane defaults" if none are provided ?)
>
> TLSF does _not_ have sane defaults. By default it is uninitialized, which is quite poor, but also (in reality) a reasonable design choice. I don't imagine it would take much to have it self-initialize to some default size (probably given as a build-time configuration parameter)
What does happen if the preallocated size is reached ? Does it grow its
buffer by itself ?
Sylvain
std::string in hard realtime contexts
On Jul 12, 2011, at 13:23 , Sylvain Joyeux wrote:
> On 07/12/2011 07:03 PM, Stephen Roderick wrote:
>> On Jul 12, 2011, at 12:53 , Sylvain Joyeux wrote:
>>
>>> On 07/12/2011 06:26 PM, S Roderick wrote:
>>>>> Is it possible to make them hard RT compatible ?
>>>>
>>>> You have to be incredibly careful to not accidentally use a std::string in a non-RT way, or that the compiler doesn't "helpfully" do it for you ... personally, I'd prefer to see std::string entirely removed from RTT if hard-RT is truly a goal. Maybe v4? ;-)
>>> I guessed that already ;-). From your experience / careful analysis, is there a way to do that properly ? The only needed operation, in the end, is copying around...
>>
>> No, we just don't use std::string if at all possible. We use "const char*" for fixed values, boost::iostream + char[] (fixed size = a weak point) for streaming, and OCL::string (rt_string in v2) for logging, variable sized strings, data ports, etc.
>>
>>>>> Using the OCL rt_string is not an option as it would add a dependency on
>>>>> OCL for the "library" part of rock (which is meant to remain
>>>>> framework-independent)
>>>>
>>>> I thought rt_string had been moved into RTT proper as part of v2? Is that too much of a dependancy for you? Certainly in v1 it is in OCL.
>>> RTT is *definitely* too much of a dependency. Even using TLSF is a bit of a problem, as the library users would need to initialize TLSF in main() (or does TLSF use "sane defaults" if none are provided ?)
>>
>> TLSF does _not_ have sane defaults. By default it is uninitialized, which is quite poor, but also (in reality) a reasonable design choice. I don't imagine it would take much to have it self-initialize to some default size (probably given as a build-time configuration parameter)
> What does happen if the preallocated size is reached ? Does it grow its buffer by itself ?
throw std::bad_alloc IIRC :-(
You can configure it to grow, but by default that uses sbreak/mmap which are non-realtime.
S
std::string in hard realtime contexts
On 07/12/2011 07:46 PM, Stephen Roderick wrote:
>>> TLSF does _not_ have sane defaults. By default it is uninitialized, which is quite poor, but also (in reality) a reasonable design choice. I don't imagine it would take much to have it self-initialize to some default size (probably given as a build-time configuration parameter)
>> What does happen if the preallocated size is reached ? Does it grow its buffer by itself ?
>
> throw std::bad_alloc IIRC :-(
>
> You can configure it to grow, but by default that uses sbreak/mmap which are non-realtime.
OKaaaay.
Given the little requirements I have on these strings (mainly copying
and comparing), I am very strongly thinking that my best option would be
to create a VERY simple base::simple_string type which contains
std::vector<char>. Since vectors are properly handled in the dataflow ...
It would main the integration in orogen straightforward (for rt_string I
would need to create a typelib plugin), and would remove the need to
refactor anything.
Sylvain
std::string in hard realtime contexts
On Jul 13, 2011, at 03:19 , Sylvain Joyeux wrote:
> On 07/12/2011 07:46 PM, Stephen Roderick wrote:
>>>> TLSF does _not_ have sane defaults. By default it is uninitialized, which is quite poor, but also (in reality) a reasonable design choice. I don't imagine it would take much to have it self-initialize to some default size (probably given as a build-time configuration parameter)
>>> What does happen if the preallocated size is reached ? Does it grow its buffer by itself ?
>>
>> throw std::bad_alloc IIRC :-(
>>
>> You can configure it to grow, but by default that uses sbreak/mmap which are non-realtime.
>
> OKaaaay.
>
> Given the little requirements I have on these strings (mainly copying and comparing), I am very strongly thinking that my best option would be to create a VERY simple base::simple_string type which contains std::vector<char>. Since vectors are properly handled in the dataflow ...
So the underlying non-RT behaviour remains, but you don't have to deal with a new type? Is that the advantage?
What changes would make TLSF/rt_string more palatable? I'd hate to see a primary Orocos user choose a different direction over what might be some small changes ...
> It would main the integration in orogen straightforward (for rt_string I would need to create a typelib plugin), and would remove the need to refactor anything.
Hold on, the maker of typegen is leaning _away_ from creating a typelib plugin .... !? ;-)
S
std::string in hard realtime contexts
On 07/13/2011 01:58 PM, Stephen Roderick wrote:
>> Given the little requirements I have on these strings (mainly copying and comparing), I am very strongly thinking that my best option would be to create a VERY simple base::simple_string type which contains std::vector<char>. Since vectors are properly handled in the dataflow ...
>
> So the underlying non-RT behaviour remains, but you don't have to deal with a new type? Is that the advantage?
Which underlying non-RT behaviour ? std::vector *in my use-case*
(predefined set of strings => upper bound on the size) can be
manipulated in a RT-hard way on RTT, so I would get the RT behaviour.
> What changes would make TLSF/rt_string more palatable? I'd hate to see a primary Orocos user choose a different direction over what might be some small changes ...
I don't want the library users to have to deal with corner-cases in
TLSF, i.e. configuration and so on. I.e. I want a string-like container
that allows to store, copy and compare strings (nothing else !) in a
hard-RT way, and that this container requires no further understanding
from the user. My impression is that rt_string is not that container.
>> It would main the integration in orogen straightforward (for rt_string I would need to create a typelib plugin), and would remove the need to refactor anything.
> Hold on, the maker of typegen is leaning _away_ from creating a typelib plugin .... !? ;-)
Well. The whole point of typelib so far in rock is that I did *not* need
to write any special-case plugin for any type. So, hell, yes, if I can
continue avoiding that, I would be a happier man. Now, I'm open for
continuing that discussion and be convinced that doing otherwise would
be a better choice.
Finally, to be fair, I'm not even sure how well orogen/typelib would
right now deal with std::basic_string that differ only by their
allocators. I would have to check that ... (but that is something I have
to check anyway, since -- if it behaves not well -- it is definitely a
bug that needs fixing)
Sylvain
std::string in hard realtime contexts
On Jul 13, 2011, at 08:23 , Sylvain Joyeux wrote:
> On 07/13/2011 01:58 PM, Stephen Roderick wrote:
>>> Given the little requirements I have on these strings (mainly copying and comparing), I am very strongly thinking that my best option would be to create a VERY simple base::simple_string type which contains std::vector<char>. Since vectors are properly handled in the dataflow ...
>>
>> So the underlying non-RT behaviour remains, but you don't have to deal with a new type? Is that the advantage?
> Which underlying non-RT behaviour ? std::vector *in my use-case* (predefined set of strings => upper bound on the size) can be manipulated in a RT-hard way on RTT, so I would get the RT behaviour.
We use the same use case throughout our system with std::vector also (but floating point data), and you just have to ensure that newbies (in particular) don't accidentally use one in a non-RT way. rt_string totally prevents that - the compromise is the TLSF configuration.
>> What changes would make TLSF/rt_string more palatable? I'd hate to see a primary Orocos user choose a different direction over what might be some small changes ...
> I don't want the library users to have to deal with corner-cases in TLSF, i.e. configuration and so on. I.e. I want a string-like container that allows to store, copy and compare strings (nothing else !) in a hard-RT way, and that this container requires no further understanding from the user. My impression is that rt_string is not that container.
Talk to me about a hypothetical rt_string+TLSF that automagically didn't require configuration. What would you hope it to be? How might it work?
>>> It would main the integration in orogen straightforward (for rt_string I would need to create a typelib plugin), and would remove the need to refactor anything.
>> Hold on, the maker of typegen is leaning _away_ from creating a typelib plugin .... !? ;-)
> Well. The whole point of typelib so far in rock is that I did *not* need to write any special-case plugin for any type. So, hell, yes, if I can continue avoiding that, I would be a happier man. Now, I'm open for continuing that discussion and be convinced that doing otherwise would be a better choice.
>
> Finally, to be fair, I'm not even sure how well orogen/typelib would right now deal with std::basic_string that differ only by their allocators. I would have to check that ... (but that is something I have to check anyway, since -- if it behaves not well -- it is definitely a bug that needs fixing)
Fair enough
S
std::string in hard realtime contexts
On 07/13/2011 02:31 PM, Stephen Roderick wrote:
>>>> Given the little requirements I have on these strings (mainly copying and comparing), I am very strongly thinking that my best option would be to create a VERY simple base::simple_string type which contains std::vector<char>. Since vectors are properly handled in the dataflow ...
>>>
>>> So the underlying non-RT behaviour remains, but you don't have to deal with a new type? Is that the advantage?
>> Which underlying non-RT behaviour ? std::vector *in my use-case* (predefined set of strings => upper bound on the size) can be manipulated in a RT-hard way on RTT, so I would get the RT behaviour.
>
> We use the same use case throughout our system with std::vector also (but floating point data), and you just have to ensure that newbies (in particular) don't accidentally use one in a non-RT way. rt_string totally prevents that - the compromise is the TLSF configuration.
MMmmm. So do you have in mind to do an rt_vector as well ? Because,
otherwise, you have to tell people about the workflow for "std::vector
in RT contexts" anyway, no ?
In general, I am thinking of ways in orogen to remove part of the work
when using std::vector (i.e. having an actual predefined workflow for
these kind of types).
I think it is also a matter of trade-off here. Between having to use
custom types (or custom allocators) vs. looking at the few components
that should be hard-RT and making sure they are hard-RT.
>>> What changes would make TLSF/rt_string more palatable? I'd hate to see a primary Orocos user choose a different direction over what might be some small changes ...
>> I don't want the library users to have to deal with corner-cases in TLSF, i.e. configuration and so on. I.e. I want a string-like container that allows to store, copy and compare strings (nothing else !) in a hard-RT way, and that this container requires no further understanding from the user. My impression is that rt_string is not that container.
>
> Talk to me about a hypothetical rt_string+TLSF that automagically didn't require configuration. What would you hope it to be? How might it work?
Well, in principle, the *only* change I should need is to use rt_string
instead of std::string.
I would even be OK with having "near-RT" behaviour [i.e. broken RT
behaviour if the initial memory pool is too small] in the
zero-configuration case, i.e. that if someone wants a complete RT
behaviour he needs to configure TLSF properly (something orogen would do
for the RTT components for instance)