We believe that Orocos is currently severely limited by the lack of an
O(1) dynamic memory allocator, and propose introducing one. We think
that at least the following problems can be remedied through
introduction of such an allocator
1) logging from real-time components
2) use of different logging levels throughout a system
3) use of dynamically sized data
4) real-time reporting
Use of Orocos in larger systems (some of our deployers contain 40+
components, which is certainly not small, though you may not consider
it large :-) ) really does require 1) and 2) to be useful. You have to
be able to log from real-time components, and you have to be able to
set different logging levels throughout the system. Setting a debug
log level for a problem in a single component, currently forces debug
logging from _all_ parts of the system. Subsequent wading through the
log file using grep is tedious, to say nothing of being error prone.
Use of a C++ log4j-derived logging system (eg log4cpp, log4cxx) would
solve 1) and 2). While I would recommend log4cpp over log4cxx due to
dependancies if nothing else, both are limited by use of dynamic
memory allocation (a first pass through log4cpp shows extensive use of
std::string, and some use of std::ostringstream, etc.). Reimplementing
their functionality within a custom Orocos logging system seems
pointless. Integrating a real-time allocator along with appropriate
typing to use the custom allocator, should be much less work.
Dynamically sized data items can currently only be used when their
size is known ahead of time. This is not always possible, or can
become quite unwieldy and/or wasteful of space (when a larger capacity
than necessay must be used). An integrated O(1) allocator would solve
this (at least for many use cases, though not necessarily all).
The existing reporting is quite capable, but limited, as it is
hampered by a lack of separation of the act of sampling the data from
storing the data. This prevents reporting of real-time components due
to non-determinstic times for storing the data (e.g. writing to file).
The log4j-derived libraries already deliver this separation in the
form of categories vs appenders. It is possible (though I haven't
fully worked through this) that the same system could be used for
logging as for reporting, with the existing reporting's sampling code
becoming the component front end that then relies on a log4j style
backend to do the storage (and possibly formatting?). Even if not
using the logging in such a manner, an O(1) allocator could still help
with reporting functionality.
This all leads to a proposal to introduce a O(1) dynamic memory
allocator (e.g. TLSF [1]) into RTT. I'm interested to hear the
communities take on this, and, of course, any alternate suggestions
you might have?
Stephen
Real-time dynamic memory allocation
On Wed, Sep 23, 2009 at 14:59, S Roderick <kiwi [dot] net [..] ...> wrote:
> We believe that Orocos is currently severely limited by the lack of an
> O(1) dynamic memory allocator, and propose introducing one. We think
> that at least the following problems can be remedied through
> introduction of such an allocator
>
> 1) logging from real-time components
> 2) use of different logging levels throughout a system
> 3) use of dynamically sized data
> 4) real-time reporting
I'm replying here to all 20-something mails in this thread.
* The RTT::log() function was designed to be used in the RTT only.
That is, there is a lot of setup code in RTT, and if some
pre-allocation/setup failed, we could log it to the user. This was all
not real-time and very light-weight. The log levels allowed us to hide
some debugging information. RTT itself never calls log() in a thread
or in code that is assumed to be executed in a real-time thread (ie
code that doesn't allocate, like calling a Method or raising an
Event.).
No wonder users are complaining if they try to use log() in their updateHook().
* Logging from components (='user messages' as opposed to reporting
'data flow') is indeed something for the (OMG) Lightweight Log Service
or any other implementation. These loggers offer an interface. We will
need to wrap such a logging framework in a component interface and
'Provide' that interface to other components in your system. Each
component that 'Requires' logging, will need to match that require
interface to the provides interface. This automatic matching is part
of RTT 2.0/WP3 too. In case no such service is provided, the
components' log messages result in a nop (for the insiders,
log_method.ready() returns false).
* Reporting of log messages using data flow ports is a way out too,
but is in my opinion just another way of having the provides/requires
interface, and a bit cumbersome if you're not passing data but error
messages or other texts. The reporting component for data flow logging
is going in the right direction now that the data flow has been fixed
btw.
* real-time allocators are part of my WP3: execution flow refactoring
and message based communication. We will use TLSF. the rt-allocator
can be disabled at compile time, at which point we fall back to
reduced functionality and possible some features which hard-depend
onit become disabled (like Lua).
* The logging component may offer the rt_string 'user type' which uses
TLSF. It'll be part of the logging interface which that component
offers.
All these things that are proposed will help me implementing WP3. So
if someone provides a patch with TLSF and rtstring type, and starts
using that to write up a logging component around some log library.
I'll be more than happy to review, help and accept.
Peter
Real-time dynamic memory allocation
On Sep 24, 2009, at 10:01 , Peter Soetens wrote:
> On Wed, Sep 23, 2009 at 14:59, S Roderick <kiwi [dot] net [..] ...> wrote:
>> We believe that Orocos is currently severely limited by the lack of
>> an
>> O(1) dynamic memory allocator, and propose introducing one. We think
>> that at least the following problems can be remedied through
>> introduction of such an allocator
>>
>> 1) logging from real-time components
>> 2) use of different logging levels throughout a system
>> 3) use of dynamically sized data
>> 4) real-time reporting
>
> I'm replying here to all 20-something mails in this thread.
>
> * The RTT::log() function was designed to be used in the RTT only.
> That is, there is a lot of setup code in RTT, and if some
> pre-allocation/setup failed, we could log it to the user. This was all
> not real-time and very light-weight. The log levels allowed us to hide
> some debugging information. RTT itself never calls log() in a thread
> or in code that is assumed to be executed in a real-time thread (ie
> code that doesn't allocate, like calling a Method or raising an
> Event.).
>
> No wonder users are complaining if they try to use log() in their
> updateHook().
>
> * Logging from components (='user messages' as opposed to reporting
> 'data flow') is indeed something for the (OMG) Lightweight Log Service
> or any other implementation. These loggers offer an interface. We will
> need to wrap such a logging framework in a component interface and
> 'Provide' that interface to other components in your system. Each
> component that 'Requires' logging, will need to match that require
> interface to the provides interface. This automatic matching is part
> of RTT 2.0/WP3 too. In case no such service is provided, the
> components' log messages result in a nop (for the insiders,
> log_method.ready() returns false).
>
> * Reporting of log messages using data flow ports is a way out too,
> but is in my opinion just another way of having the provides/requires
> interface, and a bit cumbersome if you're not passing data but error
> messages or other texts. The reporting component for data flow logging
> is going in the right direction now that the data flow has been fixed
> btw.
>
> * real-time allocators are part of my WP3: execution flow refactoring
> and message based communication. We will use TLSF. the rt-allocator
> can be disabled at compile time, at which point we fall back to
> reduced functionality and possible some features which hard-depend
> onit become disabled (like Lua).
>
> * The logging component may offer the rt_string 'user type' which uses
> TLSF. It'll be part of the logging interface which that component
> offers.
>
> All these things that are proposed will help me implementing WP3. So
> if someone provides a patch with TLSF and rtstring type, and starts
> using that to write up a logging component around some log library.
> I'll be more than happy to review, help and accept.
Ok, so all the above answers the "should we" and "where should it go"
questions. I'll leave the "what should it be" question to another
email thread ...
Stephen
Real-time dynamic memory allocation
2009/9/23 S Roderick <kiwi [dot] net [..] ...>
> We believe that Orocos is currently severely limited by the lack of an
> O(1) dynamic memory allocator, and propose introducing one. We think
> that at least the following problems can be remedied through
> introduction of such an allocator
>
> 1) logging from real-time components
> 2) use of different logging levels throughout a system
> 3) use of dynamically sized data
> 4) real-time reporting
>
> I'm not really in favour of a real-time memory allocator.
- The memory is still a shared resource so you will have to think about a
mechanism of guarding this resource.
- Also you will have to think about exception handling code in real time
because, as Herman stated, you are not sure the allocation will succeed. If
it is just used for logging you could ignore this failure and try again
later. But if it happens while putting some Event (Message) into a queue you
are in trouble...
For 1) we should be thinking of a strategy of decoupling gathering of
logging info with the actual formatting and storing of the logging. This
strategy should not use memory allocation from the 'gathering' side.
I support issue 2).
I can't think of an application that would benefit from 3).
Isn't 4) already covered by the Reporting infrastructure?
Butch.
Real-time dynamic memory allocation
On Sep 24, 2009, at 07:30 , Butch Slayer wrote:
> 2009/9/23 S Roderick <kiwi [dot] net [..] ...>
> We believe that Orocos is currently severely limited by the lack of an
> O(1) dynamic memory allocator, and propose introducing one. We think
> that at least the following problems can be remedied through
> introduction of such an allocator
>
> 1) logging from real-time components
> 2) use of different logging levels throughout a system
> 3) use of dynamically sized data
> 4) real-time reporting
>
> I'm not really in favour of a real-time memory allocator.
> - The memory is still a shared resource so you will have to think
> about a mechanism of guarding this resource.
Agreed, you will have to protect the allocator.
> - Also you will have to think about exception handling code in real
> time because, as Herman stated, you are not sure the allocation will
> succeed. If it is just used for logging you could ignore this
> failure and try again later. But if it happens while putting some
> Event (Message) into a queue you are in trouble...
Which brings us back to a number of different threads of returning a
failure value vs throwing an exception, but overall I agree with you
here.
> For 1) we should be thinking of a strategy of decoupling gathering
> of logging info with the actual formatting and storing of the
> logging. This strategy should not use memory allocation from the
> 'gathering' side.
>
> I support issue 2).
I agree with what you said for both of the above. But perhaps you can
help answer
a) how do you support variable sized logging messages, without placing
an arbitrary length limit
b) how do you provide some of the above _without_ reimplementing
functionality that already exists (and is mature!) in existing logging
frameworks.
Frameworks like the log4j-style (again, I'm just using this as a well-
known example) already do all of this, the only problem being that
they support a) using std::string. This implies dynamic memory
allocation. Hence the need for an O(1) allocator if you don't want to
reimplement their existing functionality (which I don't). This is my
entire chain of thought, in one paragraph.
> I can't think of an application that would benefit from 3).
We have an app that would. Real-time image processing with varying
regions of interest, which implies varying image size. This is a
slightly contrived but realistic scenario, as you can certainly use
the largest memory size and go with that. But in reality, of the 4,
this is the least of my worries (bar its need for a) above).
> Isn't 4) already covered by the Reporting infrastructure?
As I noted to Sylvain, the Reporting code is insufficient. It does not
support use in a real-time setting due to coupling of sampling with
storage (eg to file). It is a very similar, if not the same, issue as
your response to 1).
Also, Marcus' 5) which was support for real-time scripting, is also a
valid use case (actually, we have that scenario right now with needing
to load varying state machines at runtime into what we would like to
be a real-time component).
Stephen
Real-time dynamic memory allocation
On Wed, Sep 23, 2009 at 02:59:48PM +0200, S Roderick wrote:
> We believe that Orocos is currently severely limited by the lack of an
> O(1) dynamic memory allocator, and propose introducing one. We think
> that at least the following problems can be remedied through
> introduction of such an allocator
>
> 1) logging from real-time components
> 2) use of different logging levels throughout a system
> 3) use of dynamically sized data
> 4) real-time reporting
5) scripting with Lua...
If you want a scripting language with a certain expressiveness, there
is no way around this.
Markus
Real-time dynamic memory allocation
On Sep 23, 2009, at 12:29 , Markus Klotzbuecher wrote:
> On Wed, Sep 23, 2009 at 02:59:48PM +0200, S Roderick wrote:
>> We believe that Orocos is currently severely limited by the lack of
>> an
>> O(1) dynamic memory allocator, and propose introducing one. We think
>> that at least the following problems can be remedied through
>> introduction of such an allocator
>>
>> 1) logging from real-time components
>> 2) use of different logging levels throughout a system
>> 3) use of dynamically sized data
>> 4) real-time reporting
>
> 5) scripting with Lua...
>
> If you want a scripting language with a certain expressiveness, there
> is no way around this.
Agreed. So will an O(1) allocator be a requirements for the new
Orocos FSM implementation? Or is this "level of expressiveness" more
than is required for the FSM implementation? Based on some previous ML
traffic it sounded like you were going the route of using Lua for the
implementation, is all ...
Stephen
Real-time dynamic memory allocation
On Wed, Sep 23, 2009 at 08:06:17PM +0200, Stephen Roderick wrote:
> On Sep 23, 2009, at 12:29 , Markus Klotzbuecher wrote:
>
> > On Wed, Sep 23, 2009 at 02:59:48PM +0200, S Roderick wrote:
> >> We believe that Orocos is currently severely limited by the lack of
> >> an
> >> O(1) dynamic memory allocator, and propose introducing one. We think
> >> that at least the following problems can be remedied through
> >> introduction of such an allocator
> >>
> >> 1) logging from real-time components
> >> 2) use of different logging levels throughout a system
> >> 3) use of dynamically sized data
> >> 4) real-time reporting
> >
> > 5) scripting with Lua...
> >
> > If you want a scripting language with a certain expressiveness, there
> > is no way around this.
>
> Agreed. So will an O(1) allocator be a requirements for the new
> Orocos FSM implementation? Or is this "level of expressiveness" more
It will be a requirement for hard-real time scripting.
> than is required for the FSM implementation? Based on some previous ML
It it probably more than we would need, which doesn't mean we
shouldn't take it. I think Lua scripting will be very useful for non
rt tasks too, such as for "smart deployers" and other tools.
> traffic it sounded like you were going the route of using Lua for the
> implementation, is all ...
Yes, I'm going that route. I finished a large amount of the Lua-RTT
bindings and am trying to get a tighter integration with the type
system.
But OTOH I agree with Herman that the core RTT should not be
"polluted" with such allocations. However individual Components should
be allowed to do RT allocations, which actually might even have the
positve side effect of making explicit the fact that the resource
memory (or the absence of it) also needs to be coordinated!
Peter and I recently even discussed if it would be possible to make
all allocations go through such an allocator, for allowing to keep
track of this resource better. However as people will want to use
libraries whose allocations are not under their control, we concluded
it would not really be feasible.
Markus
Real-time dynamic memory allocation
On Wed, Sep 23, 2009 at 2:59 PM, S Roderick <kiwi [dot] net [..] ...> wrote:
> We believe that Orocos is currently severely limited by the lack of an
> O(1) dynamic memory allocator, and propose introducing one. We think
> that at least the following problems can be remedied through
> introduction of such an allocator
>
> 1) logging from real-time components
> 2) use of different logging levels throughout a system
> 3) use of dynamically sized data
> 4) real-time reporting
At least for the first three I see a (one) reason of existance: The
fact that _a lot_ (most?) of the orocos users' are doing 1) and 3)
(e.g. by using/resizing strings/vectors) without even knowing that
there thread will be "switched" to NRT!
Sure, you can blame them for being stupid morons (until you didn't
sleep well a night and make the same error), but if a O(1) dynamic
memory allocator can be a solution to that, I'm in favor.
Would we loose anything?
Klaas
Real-time dynamic memory allocation
On Sep 23, 2009, at 11:45 , Klaas Gadeyne wrote:
> On Wed, Sep 23, 2009 at 2:59 PM, S Roderick <kiwi [dot] net [..] ...> wrote:
>> We believe that Orocos is currently severely limited by the lack of
>> an
>> O(1) dynamic memory allocator, and propose introducing one. We think
>> that at least the following problems can be remedied through
>> introduction of such an allocator
>>
>> 1) logging from real-time components
>> 2) use of different logging levels throughout a system
>> 3) use of dynamically sized data
>> 4) real-time reporting
>
> At least for the first three I see a (one) reason of existance: The
> fact that _a lot_ (most?) of the orocos users' are doing 1) and 3)
> (e.g. by using/resizing strings/vectors) without even knowing that
> there thread will be "switched" to NRT!
3) can be avoided much of the time, so that is mostly education.
IMHO 1) cannot be avoided - you have to be able to debug a component
somehow while it is real-time, which is not possible with the current
implementation (unless you go Sylvain's route of adding ports, and
logger components). And even with Sylvain's suggestion (which sounds
very similar to what we've done also), you still have 2) as an issue.
With a large enough system you need different logging levels in
different places.
> Sure, you can blame them for being stupid morons (until you didn't
> sleep well a night and make the same error), but if a O(1) dynamic
> memory allocator can be a solution to that, I'm in favor.
>
> Would we loose anything?
Nothing immediately comes to mind, no. Though I wonder if debugging
with the likes of valgrind are still possible with a custom dynamic
allocator? That might be an issue.
S
Real-time dynamic memory allocation
On Wed, 23 Sep 2009, Klaas Gadeyne wrote:
> On Wed, Sep 23, 2009 at 2:59 PM, S Roderick <kiwi [dot] net [..] ...> wrote:
>> We believe that Orocos is currently severely limited by the lack of an
>> O(1) dynamic memory allocator, and propose introducing one. We think
>> that at least the following problems can be remedied through
>> introduction of such an allocator
>>
>> 1) logging from real-time components
>> 2) use of different logging levels throughout a system
>> 3) use of dynamically sized data
>> 4) real-time reporting
>
> At least for the first three I see a (one) reason of existance: The
> fact that _a lot_ (most?) of the orocos users' are doing 1) and 3)
> (e.g. by using/resizing strings/vectors) without even knowing that
> there thread will be "switched" to NRT!
> Sure, you can blame them for being stupid morons (until you didn't
> sleep well a night and make the same error), but if a O(1) dynamic
> memory allocator can be a solution to that, I'm in favor.
>
> Would we loose anything?
>
Who can prove that this memory allocator will never run out of memory, and
hence introduce realtime hickups...?
Herman
Real-time dynamic memory allocation
On Sep 23, 2009, at 11:48 , Herman Bruyninckx wrote:
> On Wed, 23 Sep 2009, Klaas Gadeyne wrote:
>
>> On Wed, Sep 23, 2009 at 2:59 PM, S Roderick <kiwi [dot] net [..] ...> wrote:
>>> We believe that Orocos is currently severely limited by the lack
>>> of an
>>> O(1) dynamic memory allocator, and propose introducing one. We think
>>> that at least the following problems can be remedied through
>>> introduction of such an allocator
>>>
>>> 1) logging from real-time components
>>> 2) use of different logging levels throughout a system
>>> 3) use of dynamically sized data
>>> 4) real-time reporting
>>
>> At least for the first three I see a (one) reason of existance: The
>> fact that _a lot_ (most?) of the orocos users' are doing 1) and 3)
>> (e.g. by using/resizing strings/vectors) without even knowing that
>> there thread will be "switched" to NRT!
>> Sure, you can blame them for being stupid morons (until you didn't
>> sleep well a night and make the same error), but if a O(1) dynamic
>> memory allocator can be a solution to that, I'm in favor.
>>
>> Would we loose anything?
>>
> Who can prove that this memory allocator will never run out of
> memory, and
> hence introduce realtime hickups...?
That's a specious argument ... if you run out of memory you have
trouble with any allocator. Doesn't matter if it's O(1) or otherwise.
Now some of the O(1) allocators I've seen will dynamically resize
their memory pool(s), using sbrk(). Which puts you right back into
page faults and non-real-time behaviour. That would have to be dealt
with ...
Stephen
Real-time dynamic memory allocation
On Wed, 23 Sep 2009, S Roderick wrote:
> On Sep 23, 2009, at 11:48 , Herman Bruyninckx wrote:
>
>> On Wed, 23 Sep 2009, Klaas Gadeyne wrote:
>>
>>> On Wed, Sep 23, 2009 at 2:59 PM, S Roderick <kiwi [dot] net [..] ...> wrote:
>>>> We believe that Orocos is currently severely limited by the lack
>>>> of an
>>>> O(1) dynamic memory allocator, and propose introducing one. We think
>>>> that at least the following problems can be remedied through
>>>> introduction of such an allocator
>>>>
>>>> 1) logging from real-time components
>>>> 2) use of different logging levels throughout a system
>>>> 3) use of dynamically sized data
>>>> 4) real-time reporting
>>>
>>> At least for the first three I see a (one) reason of existance: The
>>> fact that _a lot_ (most?) of the orocos users' are doing 1) and 3)
>>> (e.g. by using/resizing strings/vectors) without even knowing that
>>> there thread will be "switched" to NRT!
>>> Sure, you can blame them for being stupid morons (until you didn't
>>> sleep well a night and make the same error), but if a O(1) dynamic
>>> memory allocator can be a solution to that, I'm in favor.
>>>
>>> Would we loose anything?
>>>
>> Who can prove that this memory allocator will never run out of
>> memory, and
>> hence introduce realtime hickups...?
>
>
> That's a specious argument ... if you run out of memory you have
> trouble with any allocator.
Exactly! That's why _real_ realtime does all its allocation at
deployment...
> Doesn't matter if it's O(1) or otherwise.
> Now some of the O(1) allocators I've seen will dynamically resize
> their memory pool(s), using sbrk(). Which puts you right back into
> page faults and non-real-time behaviour. That would have to be dealt
> with ...
I repeat my mantra: RTT should be kept _minimal_, offering only the
features that are really necessary for _hard_ realtime; all other nice
things belong somewhere else: in external middleware, or in non-realtime
parts of Orocos. Please don't forget that the _single_ most important
reason why Orocos is somewhat respected, is its focus on realtime
readiness; I would not like to see this go, because that would make a
competitor spring up rather sooner than later. And rightfully so.
For clarity: I do not object (at all) against a memory allocator, but I do
object against it in RTT. Maybe 2.0 is the right time to add another "sub
project", which contains most of the goodies that we are discussing the
last months, but that are bloating the hard realtime kernel of Orocos...?
Herman
Real-time dynamic memory allocation
On Sep 24, 2009, at 01:57 , Herman Bruyninckx wrote:
> On Wed, 23 Sep 2009, S Roderick wrote:
>
>> On Sep 23, 2009, at 11:48 , Herman Bruyninckx wrote:
>>
>>> On Wed, 23 Sep 2009, Klaas Gadeyne wrote:
>>>
>>>> On Wed, Sep 23, 2009 at 2:59 PM, S Roderick <kiwi [dot] net [..] ...>
>>>> wrote:
>>>>> We believe that Orocos is currently severely limited by the lack
>>>>> of an
>>>>> O(1) dynamic memory allocator, and propose introducing one. We
>>>>> think
>>>>> that at least the following problems can be remedied through
>>>>> introduction of such an allocator
>>>>>
>>>>> 1) logging from real-time components
>>>>> 2) use of different logging levels throughout a system
>>>>> 3) use of dynamically sized data
>>>>> 4) real-time reporting
>>>>
>>>> At least for the first three I see a (one) reason of existance: The
>>>> fact that _a lot_ (most?) of the orocos users' are doing 1) and 3)
>>>> (e.g. by using/resizing strings/vectors) without even knowing that
>>>> there thread will be "switched" to NRT!
>>>> Sure, you can blame them for being stupid morons (until you didn't
>>>> sleep well a night and make the same error), but if a O(1) dynamic
>>>> memory allocator can be a solution to that, I'm in favor.
>>>>
>>>> Would we loose anything?
>>>>
>>> Who can prove that this memory allocator will never run out of
>>> memory, and
>>> hence introduce realtime hickups...?
>>
>>
>> That's a specious argument ... if you run out of memory you have
>> trouble with any allocator.
> Exactly! That's why _real_ realtime does all its allocation at
> deployment...
That is an ivory tower argument, Herman. We do as much of our
allocations up front as we can, but as soon as you want to support
something like, for instance, logging with varying length strings,
_not_ limiting yourself to a fixed length is much easier and less
error prone.
>> Doesn't matter if it's O(1) or otherwise.
>> Now some of the O(1) allocators I've seen will dynamically resize
>> their memory pool(s), using sbrk(). Which puts you right back into
>> page faults and non-real-time behaviour. That would have to be dealt
>> with ...
>
> I repeat my mantra: RTT should be kept _minimal_, offering only the
> features that are really necessary for _hard_ realtime; all other nice
> things belong somewhere else: in external middleware, or in non-
> realtime
> parts of Orocos. Please don't forget that the _single_ most important
> reason why Orocos is somewhat respected, is its focus on realtime
> readiness; I would not like to see this go, because that would make a
> competitor spring up rather sooner than later. And rightfully so.
So you're arguing that a project respected for its hard-realtime-ness
doesn't need a hard real-time memory allocator. Come on ....
> For clarity: I do not object (at all) against a memory allocator,
> but I do
> object against it in RTT. Maybe 2.0 is the right time to add another
> "sub
> project", which contains most of the goodies that we are discussing
> the
> last months, but that are bloating the hard realtime kernel of
> Orocos...?
What other goodies do you have in mind here?
S
Real-time dynamic memory allocation
On Thu, 24 Sep 2009, S Roderick wrote:
> On Sep 24, 2009, at 01:57 , Herman Bruyninckx wrote:
>
>> On Wed, 23 Sep 2009, S Roderick wrote:
>>
>>> On Sep 23, 2009, at 11:48 , Herman Bruyninckx wrote:
>>>
>>>> On Wed, 23 Sep 2009, Klaas Gadeyne wrote:
>>>>
>>>>> On Wed, Sep 23, 2009 at 2:59 PM, S Roderick <kiwi [dot] net [..] ...>
>>>>> wrote:
>>>>>> We believe that Orocos is currently severely limited by the lack
>>>>>> of an
>>>>>> O(1) dynamic memory allocator, and propose introducing one. We
>>>>>> think
>>>>>> that at least the following problems can be remedied through
>>>>>> introduction of such an allocator
>>>>>>
>>>>>> 1) logging from real-time components
>>>>>> 2) use of different logging levels throughout a system
>>>>>> 3) use of dynamically sized data
>>>>>> 4) real-time reporting
>>>>>
>>>>> At least for the first three I see a (one) reason of existance: The
>>>>> fact that _a lot_ (most?) of the orocos users' are doing 1) and 3)
>>>>> (e.g. by using/resizing strings/vectors) without even knowing that
>>>>> there thread will be "switched" to NRT!
>>>>> Sure, you can blame them for being stupid morons (until you didn't
>>>>> sleep well a night and make the same error), but if a O(1) dynamic
>>>>> memory allocator can be a solution to that, I'm in favor.
>>>>>
>>>>> Would we loose anything?
>>>>>
>>>> Who can prove that this memory allocator will never run out of
>>>> memory, and
>>>> hence introduce realtime hickups...?
>>>
>>>
>>> That's a specious argument ... if you run out of memory you have
>>> trouble with any allocator.
>> Exactly! That's why _real_ realtime does all its allocation at
>> deployment...
>
> That is an ivory tower argument, Herman. We do as much of our
> allocations up front as we can, but as soon as you want to support
> something like, for instance, logging with varying length strings,
> _not_ limiting yourself to a fixed length is much easier and less
> error prone.
This is _not_ ivory tower...! Supporting varying length strings is _not_
hard realtime, sigh... Maybe I am getting too old, coming from a time that
people defined "realtime" as "100% deterministic even before the
application starts"... :-) Now, "realtime" seems to mean more and more
"real fast"...
>>> Doesn't matter if it's O(1) or otherwise.
>>> Now some of the O(1) allocators I've seen will dynamically resize
>>> their memory pool(s), using sbrk(). Which puts you right back into
>>> page faults and non-real-time behaviour. That would have to be dealt
>>> with ...
>>
>> I repeat my mantra: RTT should be kept _minimal_, offering only the
>> features that are really necessary for _hard_ realtime; all other nice
>> things belong somewhere else: in external middleware, or in non-
>> realtime
>> parts of Orocos. Please don't forget that the _single_ most important
>> reason why Orocos is somewhat respected, is its focus on realtime
>> readiness; I would not like to see this go, because that would make a
>> competitor spring up rather sooner than later. And rightfully so.
>
> So you're arguing that a project respected for its hard-realtime-ness
> doesn't need a hard real-time memory allocator. Come on ....
Yes, I make that argument, because of the fallacy of a hard realtime
allocation...
>> For clarity: I do not object (at all) against a memory allocator,
>> but I do
>> object against it in RTT. Maybe 2.0 is the right time to add another
>> "sub
>> project", which contains most of the goodies that we are discussing
>> the
>> last months, but that are bloating the hard realtime kernel of
>> Orocos...?
>
> What other goodies do you have in mind here?
All the Communication middleware discussions, in the first place. All the
specific policy related things are very good to have, but they do not
belong in RTT proper.
Herman
Real-time dynamic memory allocation
On Wednesday 23 September 2009 14:59:48 S Roderick wrote:
> We believe that Orocos is currently severely limited by the lack of an
> O(1) dynamic memory allocator, and propose introducing one. We think
> that at least the following problems can be remedied through
> introduction of such an allocator
>
> 1) logging from real-time components
> 2) use of different logging levels throughout a system
> 3) use of dynamically sized data
> 4) real-time reporting
I don't really know what's the state of the OCL components, but I definitely
can do realtime data logging to a file without any memory allocation in the
way. I personally don't care so much about text logging (which is what you
call "logging" I think), so I don't feel it is a compelling reason to add a RT
allocator. If I need logging of internal component state, I just add a new
port with the relevant data structure and "sample-log" it.
Finally, do you have an example of a hard realtime component that needs
dynamically sized data ? (I understand that it is theoretically possible, I'm
just wondering in what situation).
> This all leads to a proposal to introduce a O(1) dynamic memory
> allocator (e.g. TLSF [1]) into RTT. I'm interested to hear the
> communities take on this, and, of course, any alternate suggestions
> you might have?
For the aforementionned reasons, I don't really feel that there is a
compelling reason to introduce one. Now, if you are willing to do the work, I
don't feel a compelling reason to object to it either ;-). What would be the
direct impact on people like me that don't want to use it ? Would it change
anything or would it be only a behind-the-scene thing ?
Real-time dynamic memory allocation
On Sep 23, 2009, at 09:43 , Sylvain Joyeux wrote:
> On Wednesday 23 September 2009 14:59:48 S Roderick wrote:
>> We believe that Orocos is currently severely limited by the lack of
>> an
>> O(1) dynamic memory allocator, and propose introducing one. We think
>> that at least the following problems can be remedied through
>> introduction of such an allocator
>>
>> 1) logging from real-time components
>> 2) use of different logging levels throughout a system
>> 3) use of dynamically sized data
>> 4) real-time reporting
> I don't really know what's the state of the OCL components, but I
> definitely
> can do realtime data logging to a file without any memory allocation
> in the
> way. I personally don't care so much about text logging (which is
> what you
> call "logging" I think), so I don't feel it is a compelling reason
> to add a RT
> allocator. If I need logging of internal component state, I just add
> a new
> port with the relevant data structure and "sample-log" it.
We can do realtime data logging to file too. The problem with this,
and with what you're indicating above, is that it doesn't scale to
larger systems. I don't want to have to add a port to a component to
log. It also doesn't immediately support 2) - different logging levels
in different parts of the system. While you could make it support
that, we're rather not have two independant logging systems; the
custom one, and RTT's.
A concrete example - introducing some new compilance-type controller.
During initial debugging I want to be able to get out a number of
internal values, none of which are needed outside this component and
hence aren't on ports. I either have to add ports for them and then
real-time report them with an additional custom component, or I add
logging statements to dump their values as we go. Also, with a system
problem in mind, the ability to change logging levels throughout the
application, and to do so at runtime, is incredibly useful in tracking
down system-wide issues.
> Finally, do you have an example of a hard realtime component that
> needs
> dynamically sized data ? (I understand that it is theoretically
> possible, I'm
> just wondering in what situation).
Images with varying regions of interest is about the only one we come
across. The other one is strings, representing either states, or
message, or logging ... bringing us back to the logging again. We have
no other dynamically sized data issues - everything else knows its
size at configure time.
>> This all leads to a proposal to introduce a O(1) dynamic memory
>> allocator (e.g. TLSF [1]) into RTT. I'm interested to hear the
>> communities take on this, and, of course, any alternate suggestions
>> you might have?
> For the aforementionned reasons, I don't really feel that there is a
> compelling reason to introduce one. Now, if you are willing to do
> the work, I
> don't feel a compelling reason to object to it either ;-). What
> would be the
> direct impact on people like me that don't want to use it ? Would it
> change
> anything or would it be only a behind-the-scene thing ?
I wouldn't want to introduce a dynamic allocator apart from the fact
that it is an enabling technology for logging, which is a critical
issue in Orocos in our opinion.
I would like to say that it would have no impact on you, and that it
would only be behind the scenes. I can mostly envision how that would
occur, but I'm sure I will have overlooked something.
YMMV
Stephen
Real-time dynamic memory allocation
On Wednesday 23 September 2009 19:54:29 Stephen Roderick wrote:
> We can do realtime data logging to file too. The problem with this,
> and with what you're indicating above, is that it doesn't scale to
> larger systems. I don't want to have to add a port to a component to
> log. It also doesn't immediately support 2) - different logging levels
> in different parts of the system. While you could make it support
> that, we're rather not have two independant logging systems; the
> custom one, and RTT's.
>
> A concrete example - introducing some new compilance-type controller.
> During initial debugging I want to be able to get out a number of
> internal values, none of which are needed outside this component and
> hence aren't on ports. I either have to add ports for them and then
> real-time report them with an additional custom component, or I add
> logging statements to dump their values as we go. Also, with a system
> problem in mind, the ability to change logging levels throughout the
> application, and to do so at runtime, is incredibly useful in tracking
> down system-wide issues.
reporting in general sucks badly in the RTT, and in my opinion that *is* the
problem. Where I really don't agree with you on the software development
process side, is that to make the debugging efficient, you will need to be able
to parse/plot the debugging output of the modules which -- given how it is
done -- will be in non-standard format (it is only text). Forcing it to go
through the port logging framework just simplifies the whole process. Finally,
you can still do the multi-level logging selection, since ports can be created
at runtime based on the logging level.
> > Finally, do you have an example of a hard realtime component that
> > needs
> > dynamically sized data ? (I understand that it is theoretically
> > possible, I'm
> > just wondering in what situation).
>
> Images with varying regions of interest is about the only one we come
> across. The other one is strings, representing either states, or
> message, or logging ... bringing us back to the logging again. We have
> no other dynamically sized data issues - everything else knows its
> size at configure time.
Do you really need to do hard realtime image processing ?
Real-time dynamic memory allocation
On Sep 24, 2009, at 05:27 , Sylvain Joyeux wrote:
> On Wednesday 23 September 2009 19:54:29 Stephen Roderick wrote:
>> We can do realtime data logging to file too. The problem with this,
>> and with what you're indicating above, is that it doesn't scale to
>> larger systems. I don't want to have to add a port to a component to
>> log. It also doesn't immediately support 2) - different logging
>> levels
>> in different parts of the system. While you could make it support
>> that, we're rather not have two independant logging systems; the
>> custom one, and RTT's.
>>
>> A concrete example - introducing some new compilance-type controller.
>> During initial debugging I want to be able to get out a number of
>> internal values, none of which are needed outside this component and
>> hence aren't on ports. I either have to add ports for them and then
>> real-time report them with an additional custom component, or I add
>> logging statements to dump their values as we go. Also, with a system
>> problem in mind, the ability to change logging levels throughout the
>> application, and to do so at runtime, is incredibly useful in
>> tracking
>> down system-wide issues.
> reporting in general sucks badly in the RTT, and in my opinion that
> *is* the
> problem. Where I really don't agree with you on the software
> development
> process side, is that to make the debugging efficient, you will need
> to be able
> to parse/plot the debugging output of the modules which -- given how
> it is
> done -- will be in non-standard format (it is only text). Forcing it
> to go
> through the port logging framework just simplifies the whole
> process. Finally,
> you can still do the multi-level logging selection, since ports can
> be created
> at runtime based on the logging level.
Agreed on the reporting. It's just useful enough that you want to use
it, and just broken enough to make it difficult to use. :-(
Ok, I never said anything about plotting nor parsing data. Nor would I
imagine that many people would consider "text" a non-standard format.
I'm just talking about being able to get logging messages out of real-
time components. No more, no less.
So perhaps you can help educate some of us ...?
a) How do _you_ "efficiently" debug such a component as in the example
I gave above?
b) Can you explain how using a port logging framework "simplifies the
whole process"?
>>> Finally, do you have an example of a hard realtime component that
>>> needs
>>> dynamically sized data ? (I understand that it is theoretically
>>> possible, I'm
>>> just wondering in what situation).
>>
>> Images with varying regions of interest is about the only one we come
>> across. The other one is strings, representing either states, or
>> message, or logging ... bringing us back to the logging again. We
>> have
>> no other dynamically sized data issues - everything else knows its
>> size at configure time.
> Do you really need to do hard realtime image processing ?
Hard vs soft is arguing definitions. But, for some of our realtime
processing the act of a major page fault can induce sufficient delay
to throw the system off in a bad way. Hence no dynamic memory
allocation with the standard allocator. Hence no variable sized
string. Hence no typical logging implementation. Hence back to
difficult debugging.
Cheers
Stephen
Real-time dynamic memory allocation
On Thursday 24 September 2009 13:26:31 Stephen Roderick wrote:
> On Sep 24, 2009, at 05:27 , Sylvain Joyeux wrote:
> > On Wednesday 23 September 2009 19:54:29 Stephen Roderick wrote:
> >> We can do realtime data logging to file too. The problem with this,
> >> and with what you're indicating above, is that it doesn't scale to
> >> larger systems. I don't want to have to add a port to a component to
> >> log. It also doesn't immediately support 2) - different logging
> >> levels
> >> in different parts of the system. While you could make it support
> >> that, we're rather not have two independant logging systems; the
> >> custom one, and RTT's.
> >>
> >> A concrete example - introducing some new compilance-type controller.
> >> During initial debugging I want to be able to get out a number of
> >> internal values, none of which are needed outside this component and
> >> hence aren't on ports. I either have to add ports for them and then
> >> real-time report them with an additional custom component, or I add
> >> logging statements to dump their values as we go. Also, with a system
> >> problem in mind, the ability to change logging levels throughout the
> >> application, and to do so at runtime, is incredibly useful in
> >> tracking
> >> down system-wide issues.
> >
> > reporting in general sucks badly in the RTT, and in my opinion that
> > *is* the
> > problem. Where I really don't agree with you on the software
> > development
> > process side, is that to make the debugging efficient, you will need
> > to be able
> > to parse/plot the debugging output of the modules which -- given how
> > it is
> > done -- will be in non-standard format (it is only text). Forcing it
> > to go
> > through the port logging framework just simplifies the whole
> > process. Finally,
> > you can still do the multi-level logging selection, since ports can
> > be created
> > at runtime based on the logging level.
>
> Agreed on the reporting. It's just useful enough that you want to use
> it, and just broken enough to make it difficult to use. :-(
>
> Ok, I never said anything about plotting nor parsing data. Nor would I
> imagine that many people would consider "text" a non-standard format.
> I'm just talking about being able to get logging messages out of real-
> time components. No more, no less.
Text is a non standard format if a non-human is looking at it. Most of the
time, I personally need to look at actual numerical data (i.e. internal state)
to debug my components, not really at text. Text is required only for
'exceptional' stuff, which can then break hard-RTness
> So perhaps you can help educate some of us ...?
> a) How do _you_ "efficiently" debug such a component as in the example
> I gave above?
> b) Can you explain how using a port logging framework "simplifies the
> whole process"?
Here is how I do it:
* I do create ports that can export internal data, meaning that the data on
those ports is actually not really meant to be read by other components
(note, however, that it could in the end be used for diagnostics purposes).
* while we're not there yet, I actually think that these ports will stay
where they are, to avoid having to recreate them if debugging is needed
* the overhead would, in the end, be zero: the components look at the
connected() status on the ports and export the relevant data only if there
is somebody listening.
* we have a generic logger component that can log anything generated by
orogen, orogen BY DEFAULT adds one of these into each process and our
deployment scripts (Ruby based, using CORBA) log *ALL PORTS* by default.
For the record, this strategy works only because -- thanks to orogen -- adding
new data structures, updating these data structures *and* logging them is dead
simple and quite efficient.
Now, it would be very simple to extend our deployment code to categorize the
ports and log based on the level required by the particular experiments.
> >>> Finally, do you have an example of a hard realtime component that
> >>> needs
> >>> dynamically sized data ? (I understand that it is theoretically
> >>> possible, I'm
> >>> just wondering in what situation).
> >>
> >> Images with varying regions of interest is about the only one we come
> >> across. The other one is strings, representing either states, or
> >> message, or logging ... bringing us back to the logging again. We
> >> have
> >> no other dynamically sized data issues - everything else knows its
> >> size at configure time.
> >
> > Do you really need to do hard realtime image processing ?
>
> Hard vs soft is arguing definitions. But, for some of our realtime
> processing the act of a major page fault can induce sufficient delay
> to throw the system off in a bad way. Hence no dynamic memory
> allocation with the standard allocator. Hence no variable sized
> string. Hence no typical logging implementation. Hence back to
> difficult debugging.
I understand if you need to log strings, but I still don't understand using
images for examples of variable image processing. So, again, do you really
think that the page faults induce too much variability *in you image
processing chain*. AFAIK, there is no realtime camera drivers out there, is
there ?
Sylvain
Real-time dynamic memory allocation
On Sep 24, 2009, at 08:15 , Sylvain Joyeux wrote:
> On Thursday 24 September 2009 13:26:31 Stephen Roderick wrote:
>> On Sep 24, 2009, at 05:27 , Sylvain Joyeux wrote:
>>> On Wednesday 23 September 2009 19:54:29 Stephen Roderick wrote:
>>>> We can do realtime data logging to file too. The problem with this,
>>>> and with what you're indicating above, is that it doesn't scale to
>>>> larger systems. I don't want to have to add a port to a component
>>>> to
>>>> log. It also doesn't immediately support 2) - different logging
>>>> levels
>>>> in different parts of the system. While you could make it support
>>>> that, we're rather not have two independant logging systems; the
>>>> custom one, and RTT's.
>>>>
>>>> A concrete example - introducing some new compilance-type
>>>> controller.
>>>> During initial debugging I want to be able to get out a number of
>>>> internal values, none of which are needed outside this component
>>>> and
>>>> hence aren't on ports. I either have to add ports for them and then
>>>> real-time report them with an additional custom component, or I add
>>>> logging statements to dump their values as we go. Also, with a
>>>> system
>>>> problem in mind, the ability to change logging levels throughout
>>>> the
>>>> application, and to do so at runtime, is incredibly useful in
>>>> tracking
>>>> down system-wide issues.
>>>
>>> reporting in general sucks badly in the RTT, and in my opinion that
>>> *is* the
>>> problem. Where I really don't agree with you on the software
>>> development
>>> process side, is that to make the debugging efficient, you will need
>>> to be able
>>> to parse/plot the debugging output of the modules which -- given how
>>> it is
>>> done -- will be in non-standard format (it is only text). Forcing it
>>> to go
>>> through the port logging framework just simplifies the whole
>>> process. Finally,
>>> you can still do the multi-level logging selection, since ports can
>>> be created
>>> at runtime based on the logging level.
>>
>> Agreed on the reporting. It's just useful enough that you want to use
>> it, and just broken enough to make it difficult to use. :-(
>>
>> Ok, I never said anything about plotting nor parsing data. Nor
>> would I
>> imagine that many people would consider "text" a non-standard format.
>> I'm just talking about being able to get logging messages out of
>> real-
>> time components. No more, no less.
> Text is a non standard format if a non-human is looking at it. Most
> of the
> time, I personally need to look at actual numerical data (i.e.
> internal state)
> to debug my components, not really at text. Text is required only for
> 'exceptional' stuff, which can then break hard-RTness
We have a mix of needs. Sometimes it is numerical data, sometimes it
is program flow which is not purely numeric.
>> So perhaps you can help educate some of us ...?
>> a) How do _you_ "efficiently" debug such a component as in the
>> example
>> I gave above?
>> b) Can you explain how using a port logging framework "simplifies the
>> whole process"?
>
> Here is how I do it:
> * I do create ports that can export internal data, meaning that the
> data on
> those ports is actually not really meant to be read by other
> components
> (note, however, that it could in the end be used for diagnostics
> purposes).
> * while we're not there yet, I actually think that these ports will
> stay
> where they are, to avoid having to recreate them if debugging is
> needed
> * the overhead would, in the end, be zero: the components look at the
> connected() status on the ports and export the relevant data only
> if there
> is somebody listening.
> * we have a generic logger component that can log anything generated
> by
> orogen, orogen BY DEFAULT adds one of these into each process and
> our
> deployment scripts (Ruby based, using CORBA) log *ALL PORTS* by
> default.
This general approach covers a lot of the use cases for debugging
components, and for that it is very capable. However, it doesn't cover
all the common cases. If you're tracing program flow through one or
more components, or have looped or iterative computations within one
component, just having final numeric values isn't enough.
> For the record, this strategy works only because -- thanks to orogen
> -- adding
> new data structures, updating these data structures *and* logging
> them is dead
> simple and quite efficient.
Yeah, the RTT plugin system has a way to go (but it is still useful as
is!!)
> Now, it would be very simple to extend our deployment code to
> categorize the
> ports and log based on the level required by the particular
> experiments.
Got time to knock that out while finishing up the v2 data flow
implementation ...? ;-)
>>>>> Finally, do you have an example of a hard realtime component that
>>>>> needs
>>>>> dynamically sized data ? (I understand that it is theoretically
>>>>> possible, I'm
>>>>> just wondering in what situation).
>>>>
>>>> Images with varying regions of interest is about the only one we
>>>> come
>>>> across. The other one is strings, representing either states, or
>>>> message, or logging ... bringing us back to the logging again. We
>>>> have
>>>> no other dynamically sized data issues - everything else knows its
>>>> size at configure time.
>>>
>>> Do you really need to do hard realtime image processing ?
>>
>> Hard vs soft is arguing definitions. But, for some of our realtime
>> processing the act of a major page fault can induce sufficient delay
>> to throw the system off in a bad way. Hence no dynamic memory
>> allocation with the standard allocator. Hence no variable sized
>> string. Hence no typical logging implementation. Hence back to
>> difficult debugging.
> I understand if you need to log strings, but I still don't
> understand using
> images for examples of variable image processing. So, again, do you
> really
> think that the page faults induce too much variability *in you image
> processing chain*. AFAIK, there is no realtime camera drivers out
> there, is
> there ?
Like I said, slightly contrived. I'm happy to ignore this possibilty
within this discussion. I agree that it is a limited corner case.
Stephen
Real-time dynamic memory allocation
On Wed, 23 Sep 2009, S Roderick wrote:
> We believe that Orocos is currently severely limited by the lack of an
> O(1) dynamic memory allocator, and propose introducing one. We think
> that at least the following problems can be remedied through
> introduction of such an allocator
>
> 1) logging from real-time components
> 2) use of different logging levels throughout a system
> 3) use of dynamically sized data
> 4) real-time reporting
My stance on this issue, for the time being:
- define "realtime"! I don't think logging will ever be the hardest
realtime task in your application, does it? And if it is not the hardest
realtime, it _isn't_ realtime at all :-)
- I have a similar feeling about this as I have about the ongoing
discussion on the "RT IPC": logging seems to me to be the functionality
of dedicated components, and the components that want their data to be
logged should send (asynchronous, non-blocking) messages to those logging
component(s), which can do their stuff in non-realtime...
In summary: this seems to be another example of trying to get
"Coordination" code into the hard realtime "Computation", while there is no
real need for it using appropriate decoupling of the 4Cs...
Herman
>
> Use of Orocos in larger systems (some of our deployers contain 40+
> components, which is certainly not small, though you may not consider
> it large :-) ) really does require 1) and 2) to be useful. You have to
> be able to log from real-time components, and you have to be able to
> set different logging levels throughout the system. Setting a debug
> log level for a problem in a single component, currently forces debug
> logging from _all_ parts of the system. Subsequent wading through the
> log file using grep is tedious, to say nothing of being error prone.
>
> Use of a C++ log4j-derived logging system (eg log4cpp, log4cxx) would
> solve 1) and 2). While I would recommend log4cpp over log4cxx due to
> dependancies if nothing else, both are limited by use of dynamic
> memory allocation (a first pass through log4cpp shows extensive use of
> std::string, and some use of std::ostringstream, etc.). Reimplementing
> their functionality within a custom Orocos logging system seems
> pointless. Integrating a real-time allocator along with appropriate
> typing to use the custom allocator, should be much less work.
>
> Dynamically sized data items can currently only be used when their
> size is known ahead of time. This is not always possible, or can
> become quite unwieldy and/or wasteful of space (when a larger capacity
> than necessay must be used). An integrated O(1) allocator would solve
> this (at least for many use cases, though not necessarily all).
>
> The existing reporting is quite capable, but limited, as it is
> hampered by a lack of separation of the act of sampling the data from
> storing the data. This prevents reporting of real-time components due
> to non-determinstic times for storing the data (e.g. writing to file).
> The log4j-derived libraries already deliver this separation in the
> form of categories vs appenders. It is possible (though I haven't
> fully worked through this) that the same system could be used for
> logging as for reporting, with the existing reporting's sampling code
> becoming the component front end that then relies on a log4j style
> backend to do the storage (and possibly formatting?). Even if not
> using the logging in such a manner, an O(1) allocator could still help
> with reporting functionality.
>
> This all leads to a proposal to introduce a O(1) dynamic memory
> allocator (e.g. TLSF [1]) into RTT. I'm interested to hear the
> communities take on this, and, of course, any alternate suggestions
> you might have?
>
> Stephen
>
> [1] http://rtportal.upv.es/rtmalloc/
Real-time dynamic memory allocation
On Sep 23, 2009, at 09:08 , Herman Bruyninckx wrote:
> On Wed, 23 Sep 2009, S Roderick wrote:
>
>> We believe that Orocos is currently severely limited by the lack of
>> an
>> O(1) dynamic memory allocator, and propose introducing one. We think
>> that at least the following problems can be remedied through
>> introduction of such an allocator
>>
>> 1) logging from real-time components
>> 2) use of different logging levels throughout a system
>> 3) use of dynamically sized data
>> 4) real-time reporting
>
> My stance on this issue, for the time being:
> - define "realtime"! I don't think logging will ever be the hardest
> realtime task in your application, does it? And if it is not the
> hardest
> realtime, it _isn't_ realtime at all :-)
The ability to do something like
log(Info) << "data=" << x << ... << endlog()
and not have to worry about it affect the realtime performance of my
component. That includes having that data end up in a file somewhere.
Now there is a natural separation there that isn't reflected in your
comments above. The decoupling of what to log (and whether to log it),
and where it is logged to. One of the primary problems with the
exising implementation is that these are coupled (the other being use
of std::string, which uses dynamic memory allocations).
> - I have a similar feeling about this as I have about the ongoing
> discussion on the "RT IPC": logging seems to me to be the
> functionality
> of dedicated components, and the components that want their data
> to be
> logged should send (asynchronous, non-blocking) messages to those
> logging
> component(s), which can do their stuff in non-realtime...
> In summary: this seems to be another example of trying to get
> "Coordination" code into the hard realtime "Computation", while
> there is no
> real need for it using appropriate decoupling of the 4Cs...
I agree with part of your sentiment above. The lack of decoupling of
the creation and sending of logging messages, from the act of storage/
transmission of these messages, is part of the overall problem with
the existing logging implementation. That needs to change.
The storage of logging data is a job for dedicated components (the
appenders in log4j-speak). But the question is how do you create,
format and transfer these "messages" to those components. Currently,
RTT's logging doesn't support this, making it useless in debugging
real-time components. We don't want to expand our existing, secondary
logging code that does support real-time logging, because we'll only
end up reinventing the wheel re log4j-style functionality (and you can
pick most any other mature logging framework here, they all have
similar features).
There absolutely is a need for logging in real-time components. Anyone
who says otherwise has never debugged a component with nasty
mathematical or dynamics code inside if, or hasn't come across system
level interactions that required debugging across dozens of
components. Trying to do either of these without being able to log in
realtime, is unnecessarily difficult.
YMMV
Stephen