On Apr 17, 2013, at 12:26 PM, Edwin Schreuder <edjekroketje [..] ...> wrote:

Dear all,

Recently, I've written a driver for the OROCOS SOEM driver library for the Beckhoff EL5152 (an encoder module). In the scope of my master thesis, I want to log the speed of some generator using this module. Currently, I have this driver that contains an output port which sends the encoder values and in some other component I subscribed to this port.
My intention is to log the values of different sensors to a file for further processing. However, the logging of data has to be done in real-time and the values should be updated with 400 Hz. I have read the full orocos component builder's manual, but only found the standard logging facilities for RTT and it is stated that "Logging can only be done in the non-real-time parts of your application, thus not in the Real-time Periodic Activities !"

I therefore came across http://www.orocos.org/wiki/rtt/examples-and-tutorials/using-real-time-logging and have spent my entire day getting it to work, to no avail.
 
Hi Edwin, sorry to hear about your troubles.

I tried to compile a component (first ran rosrun ocl orocreate-pkg) using OCL::Logging as in the example, but upon compilation I got errors that references could not be found. I therefore added log4cpp and ocl to my manifest.xml file, which resolved some errors. References to OCL::logging could still not be found. I then tried to recompile rtt and ocl with these options using cmake and make:

RTT: ENABLE_RT_MALLOC, OS_RT_MALLOC
OCL: BUILD_RTALLOC, BUILD_LOGGING

Now I could still not compile my component with the logging functions as I got the same errors.

I'm using a ros install according using these instructions: http://www.ros.org/wiki/orocos_toolchain_ros

Long story short: three questions remain for me:
Is the tutorial for real-time logging as mentioned here still operational or depricated: http://www.orocos.org/wiki/rtt/examples-and-tutorials/using-real-time-logging ?
 
Yes, it's operational. The tutorial hasn't been updated for ROS, so I'm not sure whether that is causing any issues. More than likely the tutorial also needs to be scrubbed/improved also. :-(

Could someone push me in a direction of getting the aforementioned logging to work? I need to log maybe 6 32-bit values at 400 Hz.
 
We log a _lot_ more than 6 values at 500Hz, using this approach. You'll be fine.

Is it, in such a case, really impossible to just write my values as tab-separated values in a txt file or as binary representations using the standard output or is this not even worth trying?
 
We use CSV files for some of our work, so that will be fine too.

You do have to enable TLSF options in RTT (as you noted above), enable RTAlloc and Logging in OCL (as above), and that should be all (IIRC). You need the configuration file to configure the logging, and create an appender. Then attach a component to log to the category associated with that appender, and attach a reporting appender which does the writing to disk. Lastly, deploy your system with something that configures TLSF and the OCL::Logging correctly (e.g. deployer-gnulinux).

Other than that, we usually have to tune the size of the TLSF memory buffer, and the appender's buffers, to ensure we don't drop any data. On modern desktops, you can just throw some RAM at this problem and it's fine. We use a TLSF memory buffer of I think 20 MB, run our appenders at nearly the same rate as the producing component, and pop up to several hundred items per cycle. This helps to cope with the non-deterministic file I/O.

HTH
S

Hi Edwin,

On Wed, Apr 17, 2013 at 6:26 PM, Edwin Schreuder <edjekroketje [..] ...> wrote:
> Dear all,
>
> Recently, I've written a driver for the OROCOS SOEM driver library for the
> Beckhoff EL5152 (an encoder module). In the scope of my master thesis, I
> want to log the speed of some generator using this module. Currently, I have
> this driver that contains an output port which sends the encoder values and
> in some other component I subscribed to this port.
> My intention is to log the values of different sensors to a file for further
> processing. However, the logging of data has to be done in real-time and the
> values should be updated with 400 Hz. I have read the full orocos component
> builder's manual, but only found the standard logging facilities for RTT and
> it is stated that "Logging can only be done in the non-real-time parts of
> your application, thus not in the Real-time Periodic Activities !"

We're using the OCL Reporting component to do real-time logging of
ports. Especially the NetcdfReporter is very efficient and quite well
tested.

http://www.orocos.org/stable/documentation/ocl/v2.x/doc-xml/orocos-repor...

>
> I therefore came across
> http://www.orocos.org/wiki/rtt/examples-and-tutorials/using-real-time-lo...
> and have spent my entire day getting it to work, to no avail.

This is only for logging 'events/errors' to users, not for logging
'data streams'.

>
> I tried to compile a component (first ran rosrun ocl orocreate-pkg) using
> OCL::Logging as in the example, but upon compilation I got errors that
> references could not be found. I therefore added log4cpp and ocl to my
> manifest.xml file, which resolved some errors.

That is correct....

> References to OCL::logging
> could still not be found.

You need to add in your CMakeLists.txt file:

orocos_use_package(ocl-logging)

> I then tried to recompile rtt and ocl with these
> options using cmake and make:
>
> RTT: ENABLE_RT_MALLOC, OS_RT_MALLOC
> OCL: BUILD_RTALLOC, BUILD_LOGGING
>
> Now I could still not compile my component with the logging functions as I
> got the same errors.

RT-Logging should be enabled by default in your ROS debian packages,
so no need for recompiling

>
> I'm using a ros install according using these instructions:
> http://www.ros.org/wiki/orocos_toolchain_ros
>
> Long story short: three questions remain for me:
> Is the tutorial for real-time logging as mentioned here still operational or
> depricated:
> http://www.orocos.org/wiki/rtt/examples-and-tutorials/using-real-time-lo...
> ?

It's fine but it lacked the build flags. I added this section.

> Could someone push me in a direction of getting the aforementioned logging
> to work? I need to log maybe 6 32-bit values at 400 Hz.

Use the reporting component ! We use 'kst' for plotting netcdf (.nc)
files. We had to build kst from source though.

> Is it, in such a case, really impossible to just write my values as
> tab-separated values in a txt file or as binary representations using the
> standard output or is this not even worth trying?

Don't try :-)

>
> Apologies for the (maybe) obvious questions.

No problem, glad you asked...

Peter