We would like to be able to get the times in Orocos log files, and Reporting output files, in standard date formats and to higher precision (microsecond). The following seems to be what is required:
1) Changing the RTT::Logger class to be able to output as a date + higher precision is pretty straight forward (changing showTime() ). Integrating that into the system is an open question (ie how do you configure this at runtime, etc), but I feel this is quite doable.
2) Changing OCL's file reporting to output dates seem *really* involved. Fundamentally, as the time is stored as a double, you would need to affect the precision of all doubles involved if you just wanted to store high precision delta-times (and not as actual dates). We've already done this but had to modify RTT's TableMarshaller at the bottom to do so.
Changing to date format seems like either re-writing FileReporting startHook() to use a custom marshaller to special case handle the first column (ie time), or replace FileReporting 's timestamp property with a custom Date property that has appropriate stream operators.
And no, the customer is not interested in post-processing the file to produce the date's. Nor should they have to. Anyway, neither of the existing files provides enough date-related information to actually rebuild the exact date and time (think UTC vs EST).
Is it *really* this involved to simply change the format of one column? I'm hoping I've just lost the plot and missed the simple mechanism for achieving this ...
TIA
Stephen
Changing formatting of Reporting component
On Tue, Jun 23, 2009 at 02:40, <kiwi [dot] net [..] ...> wrote:
> We would like to be able to get the times in Orocos log files, and Reporting output files, in standard date formats and to higher precision (microsecond). The following seems to be what is required:
>
> 1) Changing the RTT::Logger class to be able to output as a date + higher precision is pretty straight forward (changing showTime() ). Integrating that into the system is an open question (ie how do you configure this at runtime, etc), but I feel this is quite doable.
It would also require an extension of the fosi.h file to return the
date in a cross-platform way.I'm not that familiar with standard(?)
C/C++ date functions, so this might be supported already. Otherwise, a
posix-ish seconds-since-epoch approach would be easiest I assume.
>
> 2) Changing OCL's file reporting to output dates seem *really* involved. Fundamentally, as the time is stored as a double, you would need to affect the precision of all doubles involved if you just wanted to store high precision delta-times (and not as actual dates). We've already done this but had to modify RTT's TableMarshaller at the bottom to do so.
>
> Changing to date format seems like either re-writing FileReporting startHook() to use a custom marshaller to special case handle the first column (ie time), or replace FileReporting 's timestamp property with a custom Date property that has appropriate stream operators.
The Reporter is not our finest and certainly not most performant
component. The way it marshals data is very involved. It converts data
sources to properties in order to call the property marshalling
infrastructure to produce the tables. This is not necessary if you
don't require the decomposition of structs. In that case we could
write a 'simple' reporter that just calls on each port:
It's less than half a day work with copy/pasting from the current
reporting component, and will easily allow a per-port format setting
in the simple reporter. It will also be drastically more performant.
Peter
Changing formatting of Reporting component
On Jun 23, 2009, at 03:45 , Peter Soetens wrote:
> On Tue, Jun 23, 2009 at 02:40, <kiwi [dot] net [..] ...> wrote:
>> We would like to be able to get the times in Orocos log files, and
>> Reporting output files, in standard date formats and to higher
>> precision (microsecond). The following seems to be what is required:
>>
>> 1) Changing the RTT::Logger class to be able to output as a date +
>> higher precision is pretty straight forward (changing showTime() ).
>> Integrating that into the system is an open question (ie how do you
>> configure this at runtime, etc), but I feel this is quite doable.
>
> It would also require an extension of the fosi.h file to return the
> date in a cross-platform way.I'm not that familiar with standard(?)
> C/C++ date functions, so this might be supported already. Otherwise, a
> posix-ish seconds-since-epoch approach would be easiest I assume.
boost::posix_time::ptime ... or similar? No new dependancy that way.
>> 2) Changing OCL's file reporting to output dates seem *really*
>> involved. Fundamentally, as the time is stored as a double, you
>> would need to affect the precision of all doubles involved if you
>> just wanted to store high precision delta-times (and not as actual
>> dates). We've already done this but had to modify RTT's
>> TableMarshaller at the bottom to do so.
>>
>> Changing to date format seems like either re-writing FileReporting
>> startHook() to use a custom marshaller to special case handle the
>> first column (ie time), or replace FileReporting 's timestamp
>> property with a custom Date property that has appropriate stream
>> operators.
>
> The Reporter is not our finest and certainly not most performant
> component. The way it marshals data is very involved. It converts data
> sources to properties in order to call the property marshalling
> infrastructure to produce the tables. This is not necessary if you
> don't require the decomposition of structs. In that case we could
> write a 'simple' reporter that just calls on each port:
>
>
> It's less than half a day work with copy/pasting from the current
> reporting component, and will easily allow a per-port format setting
> in the simple reporter. It will also be drastically more performant.
I agree that the current reporting implementation is too involved.
What would be the limitations regarding 'structs' in your proposal
above? As long as the port's type is registered with RTT, it would be
able to output the type correctly, right? One thing, if the above is
doing what I think it is doing, then won't the KDL type plugin do
multi-line output for frames, etc? That would be Bad.
How might per-port formatting be specified/implemented?
If we are talking of significant changes to the reporting components,
then I would like to bring up real-time reporting. Definitely a
missing feature in my books. It is way too hard to sample every cycle
of a real-time controller with reporting. We have bypassed that and
use modified real-time logging instead. Yes, such a change to
reporting would require dealing with non-real-time file I/O (ie
separating sampling from storing).
Cheers
Stephen
Changing formatting of Reporting component
On Tue, Jun 23, 2009 at 13:53, Stephen Roderick<kiwi [dot] net [..] ...> wrote:
> On Jun 23, 2009, at 03:45 , Peter Soetens wrote:
>
>> On Tue, Jun 23, 2009 at 02:40, <kiwi [dot] net [..] ...> wrote:
>>>
>>> We would like to be able to get the times in Orocos log files, and
>>> Reporting output files, in standard date formats and to higher precision
>>> (microsecond). The following seems to be what is required:
>>>
>>> 1) Changing the RTT::Logger class to be able to output as a date + higher
>>> precision is pretty straight forward (changing showTime() ). Integrating
>>> that into the system is an open question (ie how do you configure this at
>>> runtime, etc), but I feel this is quite doable.
>>
>> It would also require an extension of the fosi.h file to return the
>> date in a cross-platform way.I'm not that familiar with standard(?)
>> C/C++ date functions, so this might be supported already. Otherwise, a
>> posix-ish seconds-since-epoch approach would be easiest I assume.
>
> boost::posix_time::ptime ... or similar? No new dependancy that way.
>
>>> 2) Changing OCL's file reporting to output dates seem *really* involved.
>>> Fundamentally, as the time is stored as a double, you would need to affect
>>> the precision of all doubles involved if you just wanted to store high
>>> precision delta-times (and not as actual dates). We've already done this but
>>> had to modify RTT's TableMarshaller at the bottom to do so.
>>>
>>> Changing to date format seems like either re-writing FileReporting
>>> startHook() to use a custom marshaller to special case handle the first
>>> column (ie time), or replace FileReporting 's timestamp property with a
>>> custom Date property that has appropriate stream operators.
>>
>> The Reporter is not our finest and certainly not most performant
>> component. The way it marshals data is very involved. It converts data
>> sources to properties in order to call the property marshalling
>> infrastructure to produce the tables. This is not necessary if you
>> don't require the decomposition of structs. In that case we could
>> write a 'simple' reporter that just calls on each port:
>>
>>
>> It's less than half a day work with copy/pasting from the current
>> reporting component, and will easily allow a per-port format setting
>> in the simple reporter. It will also be drastically more performant.
>
> I agree that the current reporting implementation is too involved. What
> would be the limitations regarding 'structs' in your proposal above? As long
> as the port's type is registered with RTT, it would be able to output the
> type correctly, right? One thing, if the above is doing what I think it is
> doing, then won't the KDL type plugin do multi-line output for frames, etc?
> That would be Bad.
Yeah. But only 3 lines of code extra to fix.
>
> How might per-port formatting be specified/implemented?
I have no idea yet, other than that each column has a 'tag' (like
hello.portname) which can be your key to store options. Since the key
is stored in a boost::tuple, it's trivial to extend and use.
>
> If we are talking of significant changes to the reporting components, then I
> would like to bring up real-time reporting. Definitely a missing feature in
> my books. It is way too hard to sample every cycle of a real-time controller
> with reporting. We have bypassed that and use modified real-time logging
> instead. Yes, such a change to reporting would require dealing with
> non-real-time file I/O (ie separating sampling from storing).
If we have a decent real-time memory allocator in place, your idea can
come into the picture. Heck, even Sylvain's patches actually allow
this (or are close ?). If I'm not mistaken, Sylvain even wrote such a
capable component.
In attachment the 'less than half a day work work'. Maybe it's a wast
of time, but if it helps you or this discussion, it was worth it.
Peter
Changing formatting of Reporting component
We would like to be able to get the times in Orocos log files, and Reporting output files, in standard date formats and to higher precision (microsecond). The following seems to be what is required:
1) Changing the RTT::Logger class to be able to output as a date + higher precision is pretty straight forward (changing showTime() ). Integrating that into the system is an open question (ie how do you configure this at runtime, etc), but I feel this is quite doable.
2) Changing OCL's file reporting to output dates seem *really* involved. Fundamentally, as the time is stored as a double, you would need to affect the precision of all doubles involved if you just wanted to store high precision delta-times (and not as actual dates). We've already done this but had to modify RTT's TableMarshaller at the bottom to do so.
Changing to date format seems like either re-writing FileReporting startHook() to use a custom marshaller to special case handle the first column (ie time), or replace FileReporting 's timestamp property with a custom Date property that has appropriate stream operators.
And no, the customer is not interested in post-processing the file to produce the date's. Nor should they have to. Anyway, neither of the existing files provides enough date-related information to actually rebuild the exact date and time (think UTC vs EST).
Is it *really* this involved to simply change the format of one column? I'm hoping I've just lost the plot and missed the simple mechanism for achieving this ...
TIA
Stephen