How to load plugins in non-deployer applications?

How do we load toolkits and plugins into a GUI application (for example), when we're not using the deployer? We currently load ToolKits using
{{{RTT::Toolkit::Import(KDL::KDLToolkit);}}}
style statements. But you *cannot* load the KDL corba toolkit the same way (at least as far as I can see). What is the correct way of loading toolkits and plugins into applications?

I see related previous forum topics (http://orocos.org/node/551, http://orocos.org/node/572), but nothing specifically addressing this.

I would be happy (for now) with a solution similar to the toolkit import above, though that would obviously not scale to larger systems.

The RTT toolkit and RTT corba toolkit both self-load with something like
{{{
OS::InitFunction RTTLoader( &loadRTT );

OS::InitFunction CorbaLoader( &loadCorbaLib );
}}}
Could/should the above be used?

Another question then is what criteria should we use when deciding which toolkits/plugins should self-load?

TIA
S

How to load plugins in non-deployer applications?

> How do we load toolkits and plugins into a GUI application (for example),
> when we're not using the deployer? We currently load ToolKits using
> {{{RTT::Toolkit::Import(KDL::KDLToolkit);}}}
> style statements. But you *cannot* load the KDL corba toolkit the same way
> (at least as far as I can see). What is the correct way of loading
> toolkits and plugins into applications?

We have not yet defined an API for loading (CORBA) transports, nor from a
dynamic plugin-perspective, nor from a C++ perspective. It should be
similar to Toolkit.hpp (for example calling it Transport.hpp).

This API would rely on functionality from TypeTransporter.hpp ( class
TransportRegistrator ) such that each transport plugin should implement
this class, as done in CorbaLib.cpp, class CorbaLibRegistrator.

A macro like the toolkits now have would allow that the library can also
be loaded by the deployer (which needs to be extended in that case).

Finally, a class similar to RTT::Toolkit is required (or Toolkit is to be
extended !), which allows user code to manually 'Import' a transport as
well. For example: static void Import( TransportRegistrator& tr );

A bit code moving would be necessary, but my feeling says this can be
implemented quite quickly.

>
> I see related previous forum topics (http://orocos.org/node/551,
> http://orocos.org/node/572), but nothing specifically addressing this.
>
> I would be happy (for now) with a solution similar to the toolkit import
> above, though that would obviously not scale to larger systems.
>
> The RTT toolkit and RTT corba toolkit both self-load with something like
> {{{
> OS::InitFunction RTTLoader( &loadRTT );
>
> OS::InitFunction CorbaLoader( &loadCorbaLib );
> }}}
> Could/should the above be used?
>
> Another question then is what criteria should we use when deciding which
> toolkits/plugins should self-load?

The only reason the rtt plugins can self-load is because they are linked
explicitly with the executable. We saw problems with this earlier iirc, so
importing and/or loading from the deployer or C++ should always be
possible.

Peter

How to load plugins in non-deployer applications?

On Dec 18, 2008, at 10:29 , Peter Soetens wrote:

>> How do we load toolkits and plugins into a GUI application (for
>> example),
>> when we're not using the deployer? We currently load ToolKits using
>> {{{RTT::Toolkit::Import(KDL::KDLToolkit);}}}
>> style statements. But you *cannot* load the KDL corba toolkit the
>> same way
>> (at least as far as I can see). What is the correct way of loading
>> toolkits and plugins into applications?
>
> We have not yet defined an API for loading (CORBA) transports, nor
> from a
> dynamic plugin-perspective, nor from a C++ perspective. It should be
> similar to Toolkit.hpp (for example calling it Transport.hpp).
>
> This API would rely on functionality from TypeTransporter.hpp ( class
> TransportRegistrator ) such that each transport plugin should
> implement
> this class, as done in CorbaLib.cpp, class CorbaLibRegistrator.
>
> A macro like the toolkits now have would allow that the library can
> also
> be loaded by the deployer (which needs to be extended in that case).
>
> Finally, a class similar to RTT::Toolkit is required (or Toolkit is
> to be
> extended !), which allows user code to manually 'Import' a transport
> as
> well. For example: static void Import( TransportRegistrator& tr );
>
> A bit code moving would be necessary, but my feeling says this can be
> implemented quite quickly.

It seems that one possibility is to extend RTT::Toolkit to have a
registerTransports() to it, and then use the RTT::Toolkit structure
pretty much as is. The difference between an RTT plugin and its
associated Corba transport library, is just that different functions
within a toolkit are implemented.

The other possibility seems to be add a parallel RTT:Transport
structure, which mirrors the RTT::Toolkit structure. The corresponding
TransportPlugin would have registerTransport() and loadTypes()
functions (the RTT CorbaLib does both, whereas the KDL and my OpenCV
plugins just register transports). This approach I think would force
mod's on the deployer, as well as add static transport objects to each
plugin.

The first approach seems simpler, keeping just one type of plugin, but
it does muddle up a little the type/transport concepts. The second
approach keeps the semantic separation but is more work (IMHO).

What is the semantic difference between a plugin and a toolkit?
Perhaps the answer to this would help chose?

I'm willing to take a first stab at this, once we decide on a basic
approach.

>> I see related previous forum topics (http://orocos.org/node/551,
>> http://orocos.org/node/572), but nothing specifically addressing
>> this.
>>
>> I would be happy (for now) with a solution similar to the toolkit
>> import
>> above, though that would obviously not scale to larger systems.
>>
>> The RTT toolkit and RTT corba toolkit both self-load with something
>> like
>> {{{
>> OS::InitFunction RTTLoader( &loadRTT );
>>
>> OS::InitFunction CorbaLoader( &loadCorbaLib );
>> }}}
>> Could/should the above be used?
>>
>> Another question then is what criteria should we use when deciding
>> which
>> toolkits/plugins should self-load?
>
> The only reason the rtt plugins can self-load is because they are
> linked
> explicitly with the executable. We saw problems with this earlier
> iirc, so
> importing and/or loading from the deployer or C++ should always be
> possible.

Do you remember what problems? One course of action for plugins in
general is maybe to always self-load. Linking into an application (or
dynamically loading when the app starts) would cause self-load, as
would importing within a Deployer XML file. An immediate issue might
be the order that dynamic libraries for an application are loaded,
although I _think_ that if the dependancies are set when each library
is linked that the O/S dynamic loader might take care of this. Of
course, making your own toolkit/plugin self-load is always an option
for each user creating them.

S

How to load plugins in non-deployer applications?

>>
>> It was Ruben who had the problem with KDL: if the self-loading
>> plugin was
>> linked with a component, and the component loaded in the deployer, the
>> toolkit was not doing the self-load thing. We fixed it back then with
>> letting the deployer search for plugins during startup.
>
> Umm ... well ... I don't think you guys did actually fix this as it
> doesn't work for non-standard install locations (like mine). :-(

That's indeed incomplete work.

>
> We have to forcibly load the KDL toolkit and the KDL corba plugin into
> an application. We also import them explicitly within XML deployment
> files.
>
> If we are going to have the deployer self-load plugins, we need to be
> able to set the directory it searches in. This lead you and I a while
> back to talk of an XML configuration of the deployer. But is this the
> right answer? I don't think so. Perhaps the user doesn't want all
> plugins in a directory to be self-loaded? What if they have them from
> multiple directories?

The point of a site-specific XML file is not to just set this one path in
the deployer, but to allow to load a 'standard' XML file which contains
import statements for the deployer.

>
> Isn't a plugin library really just the same as a component library, as
> far as the deployer is concerned. An import statement tells it to load
> the library and then internally, with functionality that already
> exists, the deployer loads any components or plugins or toolkits that
> it finds within that library. The onus is then on the application's
> XML file to specify which libraries/toolkits/plugins/components that
> they want, and to order them correctly to deal with dependancies
> (well, someone has to do it, either the application programmer or the
> dynamic loader).

Almost. A plugin is loaded when found (loadRTTPlugin()). A component isn't.
A plugin is loaded once and stays in process. A component can be
instantiated many times, can be destroyed again and the component library
can even be unloaded. Plugins extend our low-level framework. Components
build our applications.

Peter

How to load plugins in non-deployer applications?

>> ...
> It seems that one possibility is to extend RTT::Toolkit to have a
> registerTransports() to it, and then use the RTT::Toolkit structure
> pretty much as is. The difference between an RTT plugin and its
> associated Corba transport library, is just that different functions
> within a toolkit are implemented.
>
> The other possibility seems to be add a parallel RTT:Transport
> structure, which mirrors the RTT::Toolkit structure. The corresponding
> TransportPlugin would have registerTransport() and loadTypes()
> functions (the RTT CorbaLib does both, whereas the KDL and my OpenCV
> plugins just register transports). This approach I think would force
> mod's on the deployer, as well as add static transport objects to each
> plugin.
>
> The first approach seems simpler, keeping just one type of plugin, but
> it does muddle up a little the type/transport concepts. The second
> approach keeps the semantic separation but is more work (IMHO).

The first option is better anyway, having a single point of access for
very related classes. I would however extend Toolkit with a
static std::vector Transports;
, a getTransports()/hasTransport(string) query and the
Import(TransportRegistrator) method mentioned earlier. The ToolkitPlugin
class is not modified. We use the TransportRegistrator instead (which
should be renamed to TransportPlugin for consistency, and defined in
TransportPlugin.hpp, ToolkitPlugin+macro would/could move to
ToolkitPlugin.hpp and let Toolkit.hpp include both anyway.)

Similarly, an ORO_TRANSPORT_PLUGIN macro is defined which transforms the
transport library into a plugin.

>
> What is the semantic difference between a plugin and a toolkit?
> Perhaps the answer to this would help chose?

The 'generic' plugin API of RTT is defined by

namespace RTT { class TaskContext; }
extern "C" {
bool loadRTTPlugin( RTT::TaskContext* );
std::string getRTTPluginName();
}

(yes we should put this definition in a file Plugin.hpp !). We now only
have one plugin type: the toolkit. After your change, we would have two
plugin types: toolkits and transports. That's what the macro does in
Toolkit.hpp

>
> I'm willing to take a first stab at this, once we decide on a basic
> approach.

I have a pretty clear picture of what I would do. But I'll leave the
honour to you anyway :-)

>>
>> The only reason the rtt plugins can self-load is because they are
>> linked
>> explicitly with the executable. We saw problems with this earlier
>> iirc, so
>> importing and/or loading from the deployer or C++ should always be
>> possible.
>
> Do you remember what problems? One course of action for plugins in
> general is maybe to always self-load. Linking into an application (or
> dynamically loading when the app starts) would cause self-load, as
> would importing within a Deployer XML file. An immediate issue might
> be the order that dynamic libraries for an application are loaded,
> although I _think_ that if the dependancies are set when each library
> is linked that the O/S dynamic loader might take care of this. Of
> course, making your own toolkit/plugin self-load is always an option
> for each user creating them.

It was Ruben who had the problem with KDL: if the self-loading plugin was
linked with a component, and the component loaded in the deployer, the
toolkit was not doing the self-load thing. We fixed it back then with
letting the deployer search for plugins during startup.

Peter

How to load plugins in non-deployer applications?

On Dec 18, 2008, at 15:26 , Peter Soetens wrote:

>>> ...
>> It seems that one possibility is to extend RTT::Toolkit to have a
>> registerTransports() to it, and then use the RTT::Toolkit structure
>> pretty much as is. The difference between an RTT plugin and its
>> associated Corba transport library, is just that different functions
>> within a toolkit are implemented.
>>
>> The other possibility seems to be add a parallel RTT:Transport
>> structure, which mirrors the RTT::Toolkit structure. The
>> corresponding
>> TransportPlugin would have registerTransport() and loadTypes()
>> functions (the RTT CorbaLib does both, whereas the KDL and my OpenCV
>> plugins just register transports). This approach I think would force
>> mod's on the deployer, as well as add static transport objects to
>> each
>> plugin.
>>
>> The first approach seems simpler, keeping just one type of plugin,
>> but
>> it does muddle up a little the type/transport concepts. The second
>> approach keeps the semantic separation but is more work (IMHO).
>
> The first option is better anyway, having a single point of access for
> very related classes. I would however extend Toolkit with a
> static std::vector<TransportRegistrator*> Transports;
> , a getTransports()/hasTransport(string) query and the
> Import(TransportRegistrator) method mentioned earlier. The
> ToolkitPlugin
> class is not modified. We use the TransportRegistrator instead (which
> should be renamed to TransportPlugin for consistency, and defined in
> TransportPlugin.hpp, ToolkitPlugin+macro would/could move to
> ToolkitPlugin.hpp and let Toolkit.hpp include both anyway.)
>
> Similarly, an ORO_TRANSPORT_PLUGIN macro is defined which transforms
> the
> transport library into a plugin.
>
>>
>> What is the semantic difference between a plugin and a toolkit?
>> Perhaps the answer to this would help chose?
>
> The 'generic' plugin API of RTT is defined by
>
> namespace RTT { class TaskContext; }
> extern "C" {
> bool loadRTTPlugin( RTT::TaskContext* );
> std::string getRTTPluginName();
> }
>
> (yes we should put this definition in a file Plugin.hpp !). We now
> only
> have one plugin type: the toolkit. After your change, we would have
> two
> plugin types: toolkits and transports. That's what the macro does in
> Toolkit.hpp
>
>>
>> I'm willing to take a first stab at this, once we decide on a basic
>> approach.
>
> I have a pretty clear picture of what I would do. But I'll leave the
> honour to you anyway :-)

You asked for it ... :-) ... nasty big patch associated, along with
KDL fixes also. This clean builds/runs under Linux for RTT, KDL, BFL,
and OCL.

One thing, I'm not entirely sure of the way you wanted to handle
Plugin.hpp vs ORO_TOOLKIT_PLUGIN(). Let me know if this is (not) what
you had in mind.

Cheers
S

Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm

How to load plugins in non-deployer applications?

On Tuesday 20 January 2009 21:35:55 S Roderick wrote:
> On Dec 18, 2008, at 15:26 , Peter Soetens wrote:
> >
> > I have a pretty clear picture of what I would do. But I'll leave the
> > honour to you anyway :-)
>
> You asked for it ... :-) ... nasty big patch associated, along with
> KDL fixes also. This clean builds/runs under Linux for RTT, KDL, BFL,
> and OCL.
>
> One thing, I'm not entirely sure of the way you wanted to handle
> Plugin.hpp vs ORO_TOOLKIT_PLUGIN(). Let me know if this is (not) what
> you had in mind.

Thanks !

I believe the KDL patch is missing the changes for corbatoolkit.hpp ?

For RTT,

* the TransportPlugin.hpp has a class comment:

Use the XXX macro to have the plugin framework automatically load
all types supported by this plugin (using registerTransport() below).

Did you intend to write ORO_TOOLKIT_PLUGIN() ?

* patch on Types.cpp is missing ?
* Also a patch on AUTHORS is missing :-)

I think it's exactly what I had in mind. But it looks you missed some files...

Peter

How to load plugins in non-deployer applications?

On Jan 21, 2009, at 07:46 , Peter Soetens wrote:

> On Tuesday 20 January 2009 21:35:55 S Roderick wrote:
>> On Dec 18, 2008, at 15:26 , Peter Soetens wrote:
>>>
>>> I have a pretty clear picture of what I would do. But I'll leave the
>>> honour to you anyway :-)
>>
>> You asked for it ... :-) ... nasty big patch associated, along with
>> KDL fixes also. This clean builds/runs under Linux for RTT, KDL, BFL,
>> and OCL.
>>
>> One thing, I'm not entirely sure of the way you wanted to handle
>> Plugin.hpp vs ORO_TOOLKIT_PLUGIN(). Let me know if this is (not) what
>> you had in mind.
>
> Thanks !

> I believe the KDL patch is missing the changes for corbatoolkit.hpp ?
>
> For RTT,
>
> * the TransportPlugin.hpp has a class comment:
>
> Use the XXX macro to have the plugin framework automatically load
> all types supported by this plugin (using registerTransport() below).
>
> Did you intend to write ORO_TOOLKIT_PLUGIN() ?

We have a duplicate mechanism here. Including "Plugin.hpp" provides
the API functions needed for a toolkit, but doesn't implement them.
You have to write the code. Including "Toolkit.hpp" and then declaring
either a transport or a toolkit plugin using ORO_TOOLKIT_PLUGIN() both
defines the API functions and implements the default behaviour of
simply loading them. Is this what we had in mind? Having these two
mechanisms presents possible future confusion for users - do you think
it's worth outlining the above in comments somewhere?

I fixed the comment to indicate use of ORO_TOOLKIT_PLUGIN().

> * patch on Types.cpp is missing ?
> * Also a patch on AUTHORS is missing :-)
>
> I think it's exactly what I had in mind. But it looks you missed
> some files...

Updated patches attached. Beware of the last few lines of the rtt/src/
Types.hpp diff, as they contain part of the patch for the debug info
for debugging corba toolkits. Couldn't get rid of it.

Stephen

Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm

Ruben Smits's picture

How to load plugins in non-deployer applications?

I applied the attached patch for KDL on KDL trunk, i did not realize that the
RTT-patch was already applied, but noticed it by trying to compile trunk/kdl
with trunk/rtt.

svn ci -m "Applying patch of S.Roderick to comply with the changes of the RTT
Toolkit/Plugin system"
Sending src/bindings/rtt/corba/corbatoolkit.cpp
Adding src/bindings/rtt/corba/corbatoolkit.hpp
Sending src/bindings/rtt/toolkit.cpp
Sending src/bindings/rtt/toolkit.hpp
Transmitting file data ....
Committed revision 29947.

Ruben

On Wednesday 21 January 2009 14:47:04 S Roderick wrote:
> On Jan 21, 2009, at 07:46 , Peter Soetens wrote:
> > On Tuesday 20 January 2009 21:35:55 S Roderick wrote:
> >> On Dec 18, 2008, at 15:26 , Peter Soetens wrote:
> >>> I have a pretty clear picture of what I would do. But I'll leave the
> >>> honour to you anyway :-)
> >>
> >> You asked for it ... :-) ... nasty big patch associated, along with
> >> KDL fixes also. This clean builds/runs under Linux for RTT, KDL, BFL,
> >> and OCL.
> >>
> >> One thing, I'm not entirely sure of the way you wanted to handle
> >> Plugin.hpp vs ORO_TOOLKIT_PLUGIN(). Let me know if this is (not) what
> >> you had in mind.
> >
> > Thanks !
> >
> > I believe the KDL patch is missing the changes for corbatoolkit.hpp ?
> >
> > For RTT,
> >
> > * the TransportPlugin.hpp has a class comment:
> >
> > Use the XXX macro to have the plugin framework automatically load
> > all types supported by this plugin (using registerTransport() below).
> >
> > Did you intend to write ORO_TOOLKIT_PLUGIN() ?
>
> We have a duplicate mechanism here. Including "Plugin.hpp" provides
> the API functions needed for a toolkit, but doesn't implement them.
> You have to write the code. Including "Toolkit.hpp" and then declaring
> either a transport or a toolkit plugin using ORO_TOOLKIT_PLUGIN() both
> defines the API functions and implements the default behaviour of
> simply loading them. Is this what we had in mind? Having these two
> mechanisms presents possible future confusion for users - do you think
> it's worth outlining the above in comments somewhere?
>
> I fixed the comment to indicate use of ORO_TOOLKIT_PLUGIN().
>
> > * patch on Types.cpp is missing ?
> > * Also a patch on AUTHORS is missing :-)
> >
> > I think it's exactly what I had in mind. But it looks you missed
> > some files...
>
> Updated patches attached. Beware of the last few lines of the rtt/src/
> Types.hpp diff, as they contain part of the patch for the debug info
> for debugging corba toolkits. Couldn't get rid of it.
>
> Stephen
>
>
>
> Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm

How to load plugins in non-deployer applications?

On Wednesday 21 January 2009 14:47:04 S Roderick wrote:
> On Jan 21, 2009, at 07:46 , Peter Soetens wrote:
> > On Tuesday 20 January 2009 21:35:55 S Roderick wrote:
> >> On Dec 18, 2008, at 15:26 , Peter Soetens wrote:
> >>> I have a pretty clear picture of what I would do. But I'll leave the
> >>> honour to you anyway :-)
> >>
> >> You asked for it ... :-) ... nasty big patch associated, along with
> >> KDL fixes also. This clean builds/runs under Linux for RTT, KDL, BFL,
> >> and OCL.
> >>
> >> One thing, I'm not entirely sure of the way you wanted to handle
> >> Plugin.hpp vs ORO_TOOLKIT_PLUGIN(). Let me know if this is (not) what
> >> you had in mind.
> >
> > Thanks !
> >
> > I believe the KDL patch is missing the changes for corbatoolkit.hpp ?
> >
> > For RTT,
> >
> > * the TransportPlugin.hpp has a class comment:
> >
> > Use the XXX macro to have the plugin framework automatically load
> > all types supported by this plugin (using registerTransport() below).
> >
> > Did you intend to write ORO_TOOLKIT_PLUGIN() ?
>
> We have a duplicate mechanism here. Including "Plugin.hpp" provides
> the API functions needed for a toolkit, but doesn't implement them.
> You have to write the code. Including "Toolkit.hpp" and then declaring
> either a transport or a toolkit plugin using ORO_TOOLKIT_PLUGIN() both
> defines the API functions and implements the default behaviour of
> simply loading them. Is this what we had in mind? Having these two
> mechanisms presents possible future confusion for users - do you think
> it's worth outlining the above in comments somewhere?

Toolkit.hpp should include Plugin.hpp (for the sake of clarity and allowing
the compiler to check for consistency), and the macro should document that it
implements the API of Plugin.hpp when given a toolkitplugin object.

>
> I fixed the comment to indicate use of ORO_TOOLKIT_PLUGIN().
>
> > * patch on Types.cpp is missing ?
> > * Also a patch on AUTHORS is missing :-)
> >
> > I think it's exactly what I had in mind. But it looks you missed
> > some files...
>
> Updated patches attached. Beware of the last few lines of the rtt/src/
> Types.hpp diff, as they contain part of the patch for the debug info
> for debugging corba toolkits. Couldn't get rid of it.

CorbaLib.hpp is still missing ? And what happened with getTransportNames() ?

Peter

PS: the forum now accepts .patch files as well.

How to load plugins in non-deployer applications?

On Jan 21, 2009, at 09:50 , Peter Soetens wrote:

> On Wednesday 21 January 2009 14:47:04 S Roderick wrote:
>> On Jan 21, 2009, at 07:46 , Peter Soetens wrote:
>>> On Tuesday 20 January 2009 21:35:55 S Roderick wrote:
>>>> On Dec 18, 2008, at 15:26 , Peter Soetens wrote:
>>>>> I have a pretty clear picture of what I would do. But I'll leave
>>>>> the
>>>>> honour to you anyway :-)

<sni

>>>> For RTT,

>>>
>>> * the TransportPlugin.hpp has a class comment:
>>>
>>> Use the XXX macro to have the plugin framework automatically load
>>> all types supported by this plugin (using registerTransport()
>>> below).
>>>
>>> Did you intend to write ORO_TOOLKIT_PLUGIN() ?
>>
>> We have a duplicate mechanism here. Including "Plugin.hpp" provides
>> the API functions needed for a toolkit, but doesn't implement them.
>> You have to write the code. Including "Toolkit.hpp" and then
>> declaring
>> either a transport or a toolkit plugin using ORO_TOOLKIT_PLUGIN()
>> both
>> defines the API functions and implements the default behaviour of
>> simply loading them. Is this what we had in mind? Having these two
>> mechanisms presents possible future confusion for users - do you
>> think
>> it's worth outlining the above in comments somewhere?
>
> Toolkit.hpp should include Plugin.hpp (for the sake of clarity and
> allowing
> the compiler to check for consistency), and the macro should
> document that it
> implements the API of Plugin.hpp when given a toolkitplugin object.

Well, that does mean that anyone that includes Toolkit.hpp also
declares those functions, whether they want them or not. That is not
the current behaviour, as you need to use the ORO_TOOLKIT_PLUGIN()
before those functions pop up. Do we want this change?

>> I fixed the comment to indicate use of ORO_TOOLKIT_PLUGIN().
>>
>>> * patch on Types.cpp is missing ?
>>> * Also a patch on AUTHORS is missing :-)
>>>
>>> I think it's exactly what I had in mind. But it looks you missed
>>> some files...
>>
>> Updated patches attached. Beware of the last few lines of the rtt/
>> src/
>> Types.hpp diff, as they contain part of the patch for the debug info
>> for debugging corba toolkits. Couldn't get rid of it.
>
> CorbaLib.hpp is still missing ? And what happened with
> getTransportNames() ?

No change to corbalib.hpp. I've left the two RTT toolkits
(RealtimeToolkit and CorbaLib) as is, in terms of the
ORO_TOOLKIT_PLUGIN and RTT::Tookit::Import() changes. They still self-
load using the StartStopManager.

Yet another patch for getTransportNames() (only) attached. Sorry, I'm
having to manually edit patch files to remove the extra stuff from the
dump-type-info patch for the corba toolkit debugging. It's easy to
mess up ... :-(

> PS: the forum now accepts .patch files as well.

Cool, thanks.
S

Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm

How to load plugins in non-deployer applications?

On Wednesday 21 January 2009 16:17:59 S Roderick wrote:
> On Jan 21, 2009, at 09:50 , Peter Soetens wrote:
> > On Wednesday 21 January 2009 14:47:04 S Roderick wrote:
> >> On Jan 21, 2009, at 07:46 , Peter Soetens wrote:
> >>> On Tuesday 20 January 2009 21:35:55 S Roderick wrote:
> >>>> On Dec 18, 2008, at 15:26 , Peter Soetens wrote:
> >>>>> I have a pretty clear picture of what I would do. But I'll leave
> >>>>> the
> >>>>> honour to you anyway :-)
>
> <sni

>
> >>>> For RTT,
> >>>
> >>> * the TransportPlugin.hpp has a class comment:
> >>>
> >>> Use the XXX macro to have the plugin framework automatically load
> >>> all types supported by this plugin (using registerTransport()
> >>> below).
> >>>
> >>> Did you intend to write ORO_TOOLKIT_PLUGIN() ?
> >>
> >> We have a duplicate mechanism here. Including "Plugin.hpp" provides
> >> the API functions needed for a toolkit, but doesn't implement them.
> >> You have to write the code. Including "Toolkit.hpp" and then
> >> declaring
> >> either a transport or a toolkit plugin using ORO_TOOLKIT_PLUGIN()
> >> both
> >> defines the API functions and implements the default behaviour of
> >> simply loading them. Is this what we had in mind? Having these two
> >> mechanisms presents possible future confusion for users - do you
> >> think
> >> it's worth outlining the above in comments somewhere?
> >
> > Toolkit.hpp should include Plugin.hpp (for the sake of clarity and
> > allowing
> > the compiler to check for consistency), and the macro should
> > document that it
> > implements the API of Plugin.hpp when given a toolkitplugin object.
>
> Well, that does mean that anyone that includes Toolkit.hpp also
> declares those functions, whether they want them or not. That is not
> the current behaviour, as you need to use the ORO_TOOLKIT_PLUGIN()
> before those functions pop up. Do we want this change?

Declaring a function you don't use is completely sane. It happens all the
time. I never heard this before as an argument...

>
> >> I fixed the comment to indicate use of ORO_TOOLKIT_PLUGIN().
> >>
> >>> * patch on Types.cpp is missing ?
> >>> * Also a patch on AUTHORS is missing :-)
> >>>
> >>> I think it's exactly what I had in mind. But it looks you missed
> >>> some files...
> >>
> >> Updated patches attached. Beware of the last few lines of the rtt/
> >> src/
> >> Types.hpp diff, as they contain part of the patch for the debug info
> >> for debugging corba toolkits. Couldn't get rid of it.
> >
> > CorbaLib.hpp is still missing ? And what happened with
> > getTransportNames() ?
>
> No change to corbalib.hpp. I've left the two RTT toolkits
> (RealtimeToolkit and CorbaLib) as is, in terms of the
> ORO_TOOLKIT_PLUGIN and RTT::Tookit::Import() changes. They still self-
> load using the StartStopManager.

OK. Got it.

>
> Yet another patch for getTransportNames() (only) attached. Sorry, I'm
> having to manually edit patch files to remove the extra stuff from the
> dump-type-info patch for the corba toolkit debugging. It's easy to
> mess up ... :-(

Misery for nothing, I applied both patches as one again (will export to svn
soon) :-]

Peter

How to load plugins in non-deployer applications?

>>>
>>> The only reason the rtt plugins can self-load is because they are
>>> linked
>>> explicitly with the executable. We saw problems with this earlier
>>> iirc, so
>>> importing and/or loading from the deployer or C++ should always be
>>> possible.
>>
>> Do you remember what problems? One course of action for plugins in
>> general is maybe to always self-load. Linking into an application (or
>> dynamically loading when the app starts) would cause self-load, as
>> would importing within a Deployer XML file. An immediate issue might
>> be the order that dynamic libraries for an application are loaded,
>> although I _think_ that if the dependancies are set when each library
>> is linked that the O/S dynamic loader might take care of this. Of
>> course, making your own toolkit/plugin self-load is always an option
>> for each user creating them.
>
> It was Ruben who had the problem with KDL: if the self-loading
> plugin was
> linked with a component, and the component loaded in the deployer, the
> toolkit was not doing the self-load thing. We fixed it back then with
> letting the deployer search for plugins during startup.

Umm ... well ... I don't think you guys did actually fix this as it
doesn't work for non-standard install locations (like mine). :-(

We have to forcibly load the KDL toolkit and the KDL corba plugin into
an application. We also import them explicitly within XML deployment
files.

If we are going to have the deployer self-load plugins, we need to be
able to set the directory it searches in. This lead you and I a while
back to talk of an XML configuration of the deployer. But is this the
right answer? I don't think so. Perhaps the user doesn't want all
plugins in a directory to be self-loaded? What if they have them from
multiple directories?

Isn't a plugin library really just the same as a component library, as
far as the deployer is concerned. An import statement tells it to load
the library and then internally, with functionality that already
exists, the deployer loads any components or plugins or toolkits that
it finds within that library. The onus is then on the application's
XML file to specify which libraries/toolkits/plugins/components that
they want, and to order them correctly to deal with dependancies
(well, someone has to do it, either the application programmer or the
dynamic loader).

Thoughts?
S

How to load plugins in non-deployer applications?

>> What is the semantic difference between a plugin and a toolkit?
>> Perhaps the answer to this would help chose?
>
> The 'generic' plugin API of RTT is defined by
>
> namespace RTT { class TaskContext; }
> extern "C" {
> bool loadRTTPlugin( RTT::TaskContext* );
> std::string getRTTPluginName();
> }

As all good programmers know, this is a bad contract. I meant to write:

/**
* @file Plugin.hpp
* This file defines the Orocos plugin API.
* A plugin is a dynamic library which has a unique name and
* can be loaded in a running application. In case the loading
* is done by an Orocos TaskContext, the plugin is notified of
* the loading TaskContext. A plugin can reject to load, in which
* case the library should be unloaded from the application again.
* Once loaded, a plugin remains in the current process until the
* process exits.
*/
#include
namespace RTT { class TaskContext; }
extern "C" {
/**
* Instructs this plugin to load itself into the application.
* Implement in this function any startup code your plugin requires.
* This function should not throw.
*
* @param t The optional TaskContext which is loading this plugin.
* May be zero.
* @return true if the initialisation succeeded, false if the
* plugin could not do so.
*/
bool loadRTTPlugin( RTT::TaskContext* t );

/**
* Return the unique name of this plugin. No two plugins with
* the same name will be allowed to live in a single process.
*/
std::string getRTTPluginName();
}

Probably not perfect, but better anyway.

Peter

How to load plugins in non-deployer applications?

How do we load toolkits and plugins into a GUI application (for example), when we're not using the deployer? We currently load ToolKits using
{{{RTT::Toolkit::Import(KDL::KDLToolkit);}}}
style statements. But you *cannot* load the KDL corba toolkit the same way (at least as far as I can see). What is the correct way of loading toolkits and plugins into applications?

I see related previous forum topics (http://orocos.org/node/551, http://orocos.org/node/572), but nothing specifically addressing this.

I would be happy (for now) with a solution similar to the toolkit import above, though that would obviously not scale to larger systems.

The RTT toolkit and RTT corba toolkit both self-load with something like
{{{
OS::InitFunction RTTLoader( &loadRTT );

OS::InitFunction CorbaLoader( &loadCorbaLib );
}}}
Could/should the above be used?

Another question then is what criteria should we use when deciding which toolkits/plugins should self-load?

TIA
S