On Apr 7, 2010, at 15:34 , Peter Soetens wrote:

> On Mon, Mar 29, 2010 at 11:54 PM, S Roderick <kiwi [dot] net [..] ...> wrote:
>> Replaces direct iostream logging to "orocos.log" with logging to log4cpp. Parent application can/must setup log4cpp category(-ies) to send the log entries somewhere useful (eg file, socket).
>>
>> Intent of patch is to allow an application that already uses log4cpp for logging, to have all logging (application + Orocos) go through the same infrastructure. This patch is also (perhaps) a step towards introducing consistent system-wide logging when the new real-time logging is fully integrated into Orocos.
>>
>> Peter, while I think that the patch is clean, I am not sure of any planned approach you have to backwards compatability of RTT::Logger with respect to the new real-time logging framework. I needed this now - it couldn't wait till the developer's workshop.
>
> I have none yet, except that I was considering to make this a RTT 2.2
> feature, since I don't have the time to transform RTT::Logger to
> log4cpp. I looked at both files, couldn't we create the default
> appender in one of the ORO_main setup functions, ie, using the
> StartStopManager mechanism ? I also understood that if we want to get
> the log messages out of the current process, we need a component to do
> that (using data ports). The current Logger solves this by storing max
> 1000 messages in memory until someones starts to pull them. maybe
> something similar could be done here, ie an appender singleton that
> saves the in-process messages until some component starts to empty it
> (and eventually could clean it up or forward everything).

Before I get into all this ... I was thinking of creating a wiki page explaining what a user has to do to use the OCL real-time logging, as well as the log4cpp-based RTT logging. Summarize and walk through the changes needed to user code, and also how everything gets configured. I'd like to get others to look a the overall approach and design, and get some feedback before I go much further. Would this be of benefit to anyone?

Understand, this patch provides a log4cpp-based path for RTT::Logger to log to file. This is _completely_ separate from any OCL::logging-based real-time logging that your components do. This separation also means that you have to configure the two differently, RTT w/ log4cpp, and your component's with the OCL-XML-deployment framework. So no, you do not need a component running to get RTT::Logger messages out. The benefit is that now all the logging is in the same framework ... that's all. And IMHO this is a path to examining how to do the transition from RTT::Logger to use the real-time logging.

I have modified the deployers to accept a log4cpp configuration file to setup the "org.orocos.rtt" category, or to simply default to what currently happens (the only problem is that I haven't fully tied the category's logging priority to RTT's logging level yet. That is tricky, but not complex I think ...). I don't think I would build in the default appender you mention. It will be hard then for user's and OCL::deployers to override. But we do need some reasonable default that satisifies all ... I have simply exposed the OCL::makeDefaultCategory() function. Maybe move that into RTT, and then it is up to the user to call that first thing in ORO_main()?

One possible transition path I see is to refactor the RTT::Logger _implementation_ to provide it's current stdout and remote logging capability via log4cpp (log4cpp requires iostream, so I don't think you can push the printf-logging through it). This would help reduce the code required for the transition, remove some build options, and use log4cpp's configuration to choose appenders (ie file, file and stdout, etc).

In the long run, if we want RTT::Logger to go via the OCL real-time logging, we would need to 1) move the real-time logging from OCL to RTT (this would solve some other problems), 2) provide some default appender as you indicated above, to deal with startup logging messages, and 3) redo _all_ RTT logging statements to not use std::string nor std::stringstream. The last one is a mountain of work ...

> Would anyone (you?) want this all as a RTT 1.12 feature ? I'm trying
> to minimize my 1.x maintenance efforts...

I've already got it in RTT v1. Love git ... :-)

>> Adds an OROSEM_LOG4CPP_LOGGING cmake option (defaults to off). Turning this off replaces all iostream logging calls that wrote to "orocos.log" to calls directly into log4cpp. To get logging statements out, the application _MUST_ set up a log4cpp category associated with RTT::Logger::log4cppCategoryName. Sample code below.
>
> I also want to rename the CMake variables to something like
> RTT_LOGGER_LOG4CPP_LOGGING, ie, make it prettier in the cmake
> configuration tools.

I was actually going to suggest something like this ... we need same for OCL also. Before we go down that track though, I would really like a community discussion of the whole build/configure-file approach Orocos is using. It is currently broken and inconsistent all over the place.

>> WARNINGS
>> - does _NOT_ use the new real-time logging framework
>
> That's indeed still on the agenda...but looks like a 2.0 thing too.

Fine with me. It still needs some work ...

>> - this patch makes RTT::Logger no more real-time safe, nor any more multi-thread safe
>> - both RTT::Logger log levels and also log4cpp priorities dictate what will be logged. You are recommended to set one to debug and use the other to control the actual logging level (NB RTT defaults to always logging at least INFO to file. This constraint remains).
>
> I understood categories must be made on beforehand (they map to data
> ports). This would mean that Logger::In messages could hardly
> practically be converted to categories. Should we drop Logger::In or
> do you see a future for that ?

This is supposed to be taken care of by NDC's in log4j/log4cpp, but I couldn't find a way to make them real-time safe without major change to the log4cpp code. Their developers weren't keen on that - can't blame them. This may need to be looked at again in the future.

>> - using a standard deployer with this patch will cause no output to a log file. One of your components _MUST_ setup a log4cpp category, as noted above. Alternatively (and the next thing on my list) is to use the proposed real-time logging infrastructure to setup the Orocos logging category within the deployment XML file. I think it already supports this.
>
> As written above, I don't mind if some object is sitting ready in the
> RTT to be used by such a component/deployment configuration.

As noted above, I've already modified the deployers to take care of this. It's on github IIRC.

> Thanks for this work, a lot of users are desperately waiting for a
> good (real-time) logger !

I actually have a fully fledged system running all of this now, and are working through some of the system level problems. There are still some constraints and issues, don't get me wrong, but it is beginning to look very good. Nice to have everything within the same logging system too ...

Stephen

On Mar 31, 2010, at 09:44 , S Roderick wrote:

> On Mar 29, 2010, at 17:54 , S Roderick wrote:
>
>> Replaces direct iostream logging to "orocos.log" with logging to log4cpp. Parent application can/must setup log4cpp category(-ies) to send the log entries somewhere useful (eg file, socket).
>>
>> Intent of patch is to allow an application that already uses log4cpp for logging, to have all logging (application + Orocos) go through the same infrastructure. This patch is also (perhaps) a step towards introducing consistent system-wide logging when the new real-time logging is fully integrated into Orocos.
>>
>> Peter, while I think that the patch is clean, I am not sure of any planned approach you have to backwards compatability of RTT::Logger with respect to the new real-time logging framework. I needed this now - it couldn't wait till the developer's workshop.
>>
>> Adds an OROSEM_LOG4CPP_LOGGING cmake option (defaults to off). Turning this off replaces all iostream logging calls that wrote to "orocos.log" to calls directly into log4cpp. To get logging statements out, the application _MUST_ set up a log4cpp category associated with RTT::Logger::log4cppCategoryName. Sample code below.
>>
>> WARNINGS
>> - does _NOT_ use the new real-time logging framework
>> - this patch makes RTT::Logger no more real-time safe, nor any more multi-thread safe
>> - both RTT::Logger log levels and also log4cpp priorities dictate what will be logged. You are recommended to set one to debug and use the other to control the actual logging level (NB RTT defaults to always logging at least INFO to file. This constraint remains).
>> - using a standard deployer with this patch will cause no output to a log file. One of your components _MUST_ setup a log4cpp category, as noted above. Alternatively (and the next thing on my list) is to use the proposed real-time logging infrastructure to setup the Orocos logging category within the deployment XML file. I think it already supports this.
>>
>> Tested in Koala and Snow Leopard.
>>
>> YMMV and all usual warnings apply.
>> Stephen
>>
>> <0001-cmake-Add-support-to-log-through-log4cpp-instead-of-.patch>
>>