Alternative Transports / Master Server

Hey,
we are currently looking into adding new transport layers to orocos.
We haven't choosen the concrete implementation or candidate yet,
the candidates so far are :
- LTP/BT (licklider transmission protocol / Bundle protocol)
- ICE
- DDS
- zeroMq

Noe we stumbled over the issue with the "Master Server"
for now, there is the implicit master server, being the
corba nameserver. By "Master Server" I mean the instance,
which tells you which components are running, and what
their URIs are.
The question now would be how we should move on with
this. Our first idea was, to implement a transport independent
master server as a orocos component and in the long run
switch all transport layers to this one.
Comments are welcome ;-)

Greetings
Janosch

Alternative Transports / Master Server

2014-11-11 12:13 GMT-02:00 Janosch Machowinski <Janosch [dot] Machowinski [..] ...>:
> Hey,
> we are currently looking into adding new transport layers to orocos.
> We haven't choosen the concrete implementation or candidate yet,
> the candidates so far are :
> - LTP/BT (licklider transmission protocol / Bundle protocol)
> - ICE
> - DDS
> - zeroMq
>
> Noe we stumbled over the issue with the "Master Server"
> for now, there is the implicit master server, being the
> corba nameserver.
Can you explain what you mean by that ? There is no "master" server in
RTT the way there is one in e.g. ROS. The only server that exists is a
name server, which is a very narrowed down definition of "master".

The corba client class currently registers tasks on the corba name
server (which made sense given that CORBA was the only name service),
but there's nothing in RTT that mandates it (not even in the CORBA
transport).

It is the job of the deployment tool (either the OCL deployer or
orogen-generated deployments) to register the task wherever they feel
like. I.e. it is a tooling concern (whichever toolchain you use is
really what mandates the task to be registered somewhere specific),
definitely not a transport concern.

In other words, the need for a name server arises on the tooling side,
and the Rock tooling can for instance deal with having many name
servers just fine.

Sylvain

Alternative Transports / Master Server

Am 11.11.2014 um 15:47 schrieb Sylvain Joyeux:
> 2014-11-11 12:13 GMT-02:00 Janosch Machowinski <Janosch [dot] Machowinski [..] ...>:
>> Hey,
>> we are currently looking into adding new transport layers to orocos.
>> We haven't choosen the concrete implementation or candidate yet,
>> the candidates so far are :
>> - LTP/BT (licklider transmission protocol / Bundle protocol)
>> - ICE
>> - DDS
>> - zeroMq
>>
>> Noe we stumbled over the issue with the "Master Server"
>> for now, there is the implicit master server, being the
>> corba nameserver.
> Can you explain what you mean by that ? There is no "master" server in
> RTT the way there is one in e.g. ROS. The only server that exists is a
> name server, which is a very narrowed down definition of "master".
>
> The corba client class currently registers tasks on the corba name
> server (which made sense given that CORBA was the only name service),
> but there's nothing in RTT that mandates it (not even in the CORBA
> transport).
I agree, but to be fair, without the ability to lookup up other components,
orocos would be pretty useless. My proposal was more aiming in the
direction, to have a common interface for component lookup,
independent of the transport layer that one uses.

Apart from this there is the second question, if it makes sense, to add
code that serves component lookup request in a transport independent way.
This would be useful if building transports on pure protocols.

It was already pointed out, there are case, were the transport middleware
already provides the functionality of the name lookup. In this cases
this code
would obviously not be needed.

>
> It is the job of the deployment tool (either the OCL deployer or
> orogen-generated deployments) to register the task wherever they feel
> like. I.e. it is a tooling concern (whichever toolchain you use is
> really what mandates the task to be registered somewhere specific),
> definitely not a transport concern.
Sure, but it would be great, to do something like
./component --useDDS
or
./component --useCORBA

without having to maintain two independent code base on the tooling side.

Janosch

Alternative Transports / Master Server

My impression that you are trying to bundle two separate issues:
- data transport
- name service resolution
For historic reasons, they are bundled in the CORBA transport, but we
should do away with that.

What I meant by "it is a tooling issue" is that:
- the runtime tooling (orocos.rb in Rock) can access any number of
name service resolution protocols, be it bound to a transport (e.g.
CORBA) or not (avahi)
- the deployment-time tooling (orogen deployments in Rock) can
register tasks on any number of name service resolution protocols.

And for what it's worth, not using native name service resolutions
looks ill-conceived as it separates the RTT/Rock tooling from the
transport's tooling (you want to use DDS-enabled tools to inspect your
system ? You can't ! Names are registered on the Rock name service !)

In summary, I really don't see what is achieved by adding
yet-another-name-service in the system. It sounds like it simplifies
the runtime tooling, but does so at the cost of a lot of loss of
interoperability. However, since the runtime tooling already supports
resolving from multiple name services, and will have to if you want to
interoperate with other frameworks (e.g. ROS), the simplification is
really moot from my point of view.

IMO, the same way that any number of transports can be used in
parallel in the same running system, we should aim at being able to
use any number of name services in parallel (we already do that with
ROS and CORBA). Because name services, as transports, have different
runtime, performance and robustness guarantees. avahi takes up to
seconds to propagate new information. CORBA is instant but needs a
centralized server (great for local installs). I personally plan to
not use any for e.g. syskit: since syskit already starts all
components, it can use its own infrastructure to load the CORBA IORs.

Sylvain

Alternative Transports / Master Server

Am 11.11.2014 um 17:36 schrieb Sylvain Joyeux:
> My impression that you are trying to bundle two separate issues:
> - data transport
> - name service resolution
> For historic reasons, they are bundled in the CORBA transport, but we
> should do away with that.
Interesting question, should RTT come with a name service resolution or
not ?

>
> What I meant by "it is a tooling issue" is that:
> - the runtime tooling (orocos.rb in Rock) can access any number of
> name service resolution protocols, be it bound to a transport (e.g.
> CORBA) or not (avahi)
> - the deployment-time tooling (orogen deployments in Rock) can
> register tasks on any number of name service resolution protocols.
Sure, we can add it all inside our own tooling. The question here
is if we should aim at pushing this 'upstream' to RTT, so that the
whole RTT code would benefit from it, not only our own tooling.

Janosch

Alternative Transports / Master Server

If you want to add proper name service support to RTT then IMO you should
replicate what orocos.rb has: a number of name service classes that allow
to register and resolve tasks on existing native name services.

My comment was that the way the corba name service registration is
integrated is harmful and should not be replicated.

Sylvain

Alternative Transports / Master Server

> I personally plan to
> not use any for e.g. syskit: since syskit already starts all
> components, it can use its own infrastructure to load the CORBA IORs.
Forgot to add: /while still registering them on other name services
for the benefit of e.g. rock-display/

Sylvain

Alternative Transports / Master Server

On Tue, 11 Nov 2014, Janosch Machowinski wrote:

> Am 11.11.2014 um 15:47 schrieb Sylvain Joyeux:
>> 2014-11-11 12:13 GMT-02:00 Janosch Machowinski <Janosch [dot] Machowinski [..] ...>:
>>> Hey,
>>> we are currently looking into adding new transport layers to orocos.
>>> We haven't choosen the concrete implementation or candidate yet,
>>> the candidates so far are :
>>> - LTP/BT (licklider transmission protocol / Bundle protocol)
>>> - ICE
>>> - DDS
>>> - zeroMq
>>>
>>> Noe we stumbled over the issue with the "Master Server"
>>> for now, there is the implicit master server, being the
>>> corba nameserver.
>> Can you explain what you mean by that ? There is no "master" server in
>> RTT the way there is one in e.g. ROS. The only server that exists is a
>> name server, which is a very narrowed down definition of "master".
>>
>> The corba client class currently registers tasks on the corba name
>> server (which made sense given that CORBA was the only name service),
>> but there's nothing in RTT that mandates it (not even in the CORBA
>> transport).
> I agree, but to be fair, without the ability to lookup up other components,
> orocos would be pretty useless.

Orocos should not be extended with this functionality, since this would be
a counter-example of "inversion of control": the projects that are
specialised in discovery and communication (Looci, and the growing set of
more recent Internet of Things middlewares) should be able to deploy
themselves into component frameworks, without the latter having to be aware
of this.

> My proposal was more aiming in the
> direction, to have a common interface for component lookup,
> independent of the transport layer that one uses.

There are tons of projects working in this direction, as mentioned above.
It would be a bad idea to add yet another of these, in the very limited
Orocos context, and with the very limited resources available.

> Apart from this there is the second question, if it makes sense, to add
> code that serves component lookup request in a transport independent way.
> This would be useful if building transports on pure protocols.
>
> It was already pointed out, there are case, were the transport middleware
> already provides the functionality of the name lookup. In this cases
> this code would obviously not be needed.

>> It is the job of the deployment tool (either the OCL deployer or
>> orogen-generated deployments) to register the task wherever they feel
>> like. I.e. it is a tooling concern (whichever toolchain you use is
>> really what mandates the task to be registered somewhere specific),
>> definitely not a transport concern.
> Sure, but it would be great, to do something like
> ./component --useDDS
> or
> ./component --useCORBA
>
> without having to maintain two independent code base on the tooling side.

This is not my preferred solution! (Because of the inversion of control
issues.) The way to go is the following:

my_favorite_communication_project --deploy DDS --component-type my_RTT_1

(this is too simple, of course, just illustrating where the "dependencies"
should be dealt with.)

> Janosch

Herman

Alternative Transports / Master Server

Hi,

FYI, Ros is going to use DDS for transportation in the next major version,
see :
http://design.ros2.org/articles/ros_on_dds.html

2014-11-11 15:13 GMT+01:00 Janosch Machowinski <Janosch [dot] Machowinski [..] ...>
:

> Hey,
> we are currently looking into adding new transport layers to orocos.
> We haven't choosen the concrete implementation or candidate yet,
> the candidates so far are :
> - LTP/BT (licklider transmission protocol / Bundle protocol)
> - ICE
> - DDS
> - zeroMq
>
> Noe we stumbled over the issue with the "Master Server"
> for now, there is the implicit master server, being the
> corba nameserver. By "Master Server" I mean the instance,
> which tells you which components are running, and what
> their URIs are.
> The question now would be how we should move on with
> this. Our first idea was, to implement a transport independent
> master server as a orocos component and in the long run
> switch all transport layers to this one.
> Comments are welcome ;-)
>
> Greetings
> Janosch
>
> --
> Dipl. Inf. Janosch Machowinski
> SAR- & Sicherheitsrobotik
>
> Universität Bremen
> FB 3 - Mathematik und Informatik
> AG Robotik
> Robert-Hooke-Straße 1
> 28359 Bremen, Germany
>
> Zentrale: +49 421 178 45-6611
>
> Besuchsadresse der Nebengeschäftstelle:
> Robert-Hooke-Straße 5
> 28359 Bremen, Germany
>
> Tel.: +49 421 178 45-6614
> Empfang: +49 421 178 45-6600
> Fax: +49 421 178 45-4150
> E-Mail: jmachowinski [..] ...
>
> Weitere Informationen: http://www.informatik.uni-bremen.de/robotik
>
> --
> Orocos-Dev mailing list
> Orocos-Dev [..] ...
> http://lists.mech.kuleuven.be/mailman/listinfo/orocos-dev
>

Alternative Transports / Master Server

2014-11-11 15:29 GMT+01:00 Willy Lambert <lambert [dot] willy [..] ...>:

> Hi,
>
> FYI, Ros is going to use DDS for transportation in the next major version,
> see :
> http://design.ros2.org/articles/ros_on_dds.html
>

DDS would definitely be a good choice IMO. Some people in Stockholm (IIRC)
started to work on it, maybe they have a base source code to share...

>
>
>
>
> 2014-11-11 15:13 GMT+01:00 Janosch Machowinski <
> Janosch [dot] Machowinski [..] ...>:
>
>> Hey,
>> we are currently looking into adding new transport layers to orocos.
>> We haven't choosen the concrete implementation or candidate yet,
>> the candidates so far are :
>> - LTP/BT (licklider transmission protocol / Bundle protocol)
>> - ICE
>> - DDS
>> - zeroMq
>>
>> Noe we stumbled over the issue with the "Master Server"
>> for now, there is the implicit master server, being the
>> corba nameserver. By "Master Server" I mean the instance,
>> which tells you which components are running, and what
>> their URIs are.
>> The question now would be how we should move on with
>> this. Our first idea was, to implement a transport independent
>> master server as a orocos component and in the long run
>> switch all transport layers to this one.
>>
>
I don't understand why you need such a master... DDS for instance can
provide dynamic finding of components, then without master. Why do you need
a master in such cases?

> Comments are welcome ;-)
>>
>> Greetings
>> Janosch
>>
>> --
>> Dipl. Inf. Janosch Machowinski
>> SAR- & Sicherheitsrobotik
>>
>> Universität Bremen
>> FB 3 - Mathematik und Informatik
>> AG Robotik
>> Robert-Hooke-Straße 1
>> 28359 Bremen, Germany
>>
>> Zentrale: +49 421 178 45-6611
>>
>> Besuchsadresse der Nebengeschäftstelle:
>> Robert-Hooke-Straße 5
>> 28359 Bremen, Germany
>>
>> Tel.: +49 421 178 45-6614
>> Empfang: +49 421 178 45-6600
>> Fax: +49 421 178 45-4150
>> E-Mail: jmachowinski [..] ...
>>
>> Weitere Informationen: http://www.informatik.uni-bremen.de/robotik
>>
>> --
>> Orocos-Dev mailing list
>> Orocos-Dev [..] ...
>> http://lists.mech.kuleuven.be/mailman/listinfo/orocos-dev
>>
>
>
> --
> Orocos-Dev mailing list
> Orocos-Dev [..] ...
> http://lists.mech.kuleuven.be/mailman/listinfo/orocos-dev
>
>

Alternative Transports / Master Server

On Tue, 11 Nov 2014, Charles Lesire-Cabaniols wrote:

>
>
> 2014-11-11 15:29 GMT+01:00 Willy Lambert <lambert [dot] willy [..] ...>:
> Hi,
>
> FYI, Ros is going to use DDS for transportation in the next major version, see :
> http://design.ros2.org/articles/ros_on_dds.html
>
> DDS would definitely be a good choice IMO. Some people in Stockholm (IIRC) started to work on
> it, maybe they have a base source code to share...

My position on this is twofold:

- Orocos RTT should be 100% decoupled from communication middleware. It
should do one thing best, and that is to provide hard realtime containers
to deploy functions in. Doing communication well is the job of other
projects.

- DDS is just one of the many useful communication middlewares. But it only
offers a very limited set of interaction protocols.

Herman

> 2014-11-11 15:13 GMT+01:00 Janosch Machowinski <Janosch [dot] Machowinski [..] ...>:
> Hey,
> we are currently looking into adding new transport layers to orocos.
> We haven't choosen the concrete implementation or candidate yet,
> the candidates so far are :
>   - LTP/BT (licklider transmission protocol / Bundle protocol)
>   - ICE
>   - DDS
>   - zeroMq
>
> Noe we stumbled over the issue with the "Master Server"
> for now, there is the implicit master server, being the
> corba nameserver. By "Master Server" I mean the instance,
> which tells you which components are running, and what
> their URIs are.
> The question now would be how we should move on with
> this. Our first idea was, to implement a transport independent
> master server as a orocos component and in the long run
> switch all transport layers to this one.
>
>
> I don't understand why you need such a master... DDS for instance can provide dynamic finding of
> components, then without master. Why do you need a master in such cases?
>
>  
> Comments are welcome ;-)
>
> Greetings
>      Janosch
>
> --
>   Dipl. Inf. Janosch Machowinski
>   SAR- & Sicherheitsrobotik
>
>   Universität Bremen
>   FB 3 - Mathematik und Informatik
>   AG Robotik
>   Robert-Hooke-Straße 1
>   28359 Bremen, Germany
>
>   Zentrale: +49 421 178 45-6611
>
>   Besuchsadresse der Nebengeschäftstelle:
>   Robert-Hooke-Straße 5
>   28359 Bremen, Germany
>
>   Tel.:    +49 421 178 45-6614
>   Empfang: +49 421 178 45-6600
>   Fax:     +49 421 178 45-4150
>   E-Mail:  jmachowinski [..] ...
>
>   Weitere Informationen: http://www.informatik.uni-bremen.de/robotik
>
> --
> Orocos-Dev mailing list
> Orocos-Dev [..] ...
> http://lists.mech.kuleuven.be/mailman/listinfo/orocos-dev
>
>
>
> --
> Orocos-Dev mailing list
> Orocos-Dev [..] ...
> http://lists.mech.kuleuven.be/mailman/listinfo/orocos-dev
>
>
>
>

--
Mechanical Engineering Departments, Robotics Research Groups
University of Leuven & Eindhoven University of Technology
<http://people.mech.kuleuven.be/~bruyninc> Tel: +32 16 328056
Vice-President Research euRobotics AISBL <http://www.eu-robotics.net>
Open RObot COntrol Software <http://www.orocos.org>
Associate Editor JOSER <http://www.joser.org>

Alternative Transports / Master Server

Am 11.11.2014 um 15:42 schrieb Charles Lesire-Cabaniols:
>
>
> The question now would be how we should move on with
> this. Our first idea was, to implement a transport independent
> master server as a orocos component and in the long run
> switch all transport layers to this one.
>
>
> I don't understand why you need such a master... DDS for instance can
> provide dynamic finding of components, then without master. Why do you
> need a master in such cases?
This would mean, that you have different client code depending on the
transport that you use. Also one would have the overhead of
maintaining multiple code versions that do the same thing in the end.

Finally, IF you use a pure protocol, as LTP, there is nothing that would
allow component name lookup and you need something like this.

>
> Comments are welcome ;-)
>
> Greetings
> Janosch
>
> --
> Dipl. Inf. Janosch Machowinski
> SAR- & Sicherheitsrobotik
>
> Universität Bremen
> FB 3 - Mathematik und Informatik
> AG Robotik
> Robert-Hooke-Straße 1
> 28359 Bremen, Germany
>
> Zentrale: +49 421 178 45-6611 <tel:%2B49%20421%20178%2045-6611>
>
> Besuchsadresse der Nebengeschäftstelle:
> Robert-Hooke-Straße 5
> 28359 Bremen, Germany
>
> Tel.: +49 421 178 45-6614 <tel:%2B49%20421%20178%2045-6614>
> Empfang: +49 421 178 45-6600 <tel:%2B49%20421%20178%2045-6600>
> Fax: +49 421 178 45-4150 <tel:%2B49%20421%20178%2045-4150>
> E-Mail: jmachowinski [..] ...
> <mailto:jmachowinski [..] ...>
>
> Weitere Informationen:
> http://www.informatik.uni-bremen.de/robotik
>
> --
> Orocos-Dev mailing list
> Orocos-Dev [..] ...
> <mailto:Orocos-Dev [..] ...>
> http://lists.mech.kuleuven.be/mailman/listinfo/orocos-dev
>
>
>
> --
> Orocos-Dev mailing list
> Orocos-Dev [..] ...
> <mailto:Orocos-Dev [..] ...>
> http://lists.mech.kuleuven.be/mailman/listinfo/orocos-dev
>
>