Dear all,
This message is sent to mailinglists in various projects: Orocos, BRICS and
Rosetta. The reason: it's about a development that all projects need, and
could preferably do together. That development is the "realtime scene
graph", in other words, the software support to keep an representation of
the world online, and make it available to all robots, sensors, tools,...
in the system to read from it, and write to it.
There is a first draft available on
<http://people.mech.kuleuven.be/~bruyninc/scenegraph />.
It gives a rough idea about what I see as the major new features, wrt to
existing implementations:
* Real time efficiency
* Deployable as a service
* Unique IDs for all world objects
* Explicit representation of uncertainty
* Short-term memory
Some of these features exist already somewhere, but not the combination. I
think it should definitely be possible to merge different scene graphs, but
also to use (a relevant part of) them in a hard realtime robot controller.
Before I go further with this design, and discuss it with other
stakeholders that have shown intrest (ROS, RoboEarth), I thought it would
be appropriate to first get the opinion from the project partners I am most
closely involved with.
So, please give your opinions about the suggested design:
- are the five focus point ok?
- is the architecture ok? (That is, a "fast object" and "distributed
service" combination, to be designed together.)
- is some particular use case missing?
I am looking forward to your critical comments?
Best regards,
Herman Bruyninckx
[BRICS-all] Some concrete suggestions for a "scene graph" projec
Dear Herman,
Sorry for taking our time, but we have now had some feedback on this from within KUKA and would like to contribute it to the discussion.
Overall we really like your move but would like to suggest to not think just about Scene Graphs, but to think about Scene States. In short if I tell you I am at (x-4,y=5,z=6) that is not nearly as useful as "I am in robot 1's gripper at (4,5,6)" etc. The state of a scene tells us much more about what's going on than the dynamic graph alone.
Use case as you request:
1) Imagine the object in the gripper of the robot above. Using the scene state representation we know that the transformation from the gripper to the object does not change now --> no need to compute this. Could your "spanning tree" handle this?
2) We can imagine that two robots doing something useful together have completely static scene graphs with respect to each other and have no need for dynamic graph communication with each other or any information about a third performing an independent task in the same workspace during this particular scene in this particular scene state. In short, keeping scene state greatly reduces scene graph analysis and computational complexity.--> again, could your spanning tree handle this?
3) If you want to grab a box with a two finger gripper, you normally have 3 grasps (twosides, othertwosides, topbottom). Knowing the state of the box (on a table, I hold the box, you hold the box) determines which grips are appropriate in the situation. --> this can be represented more easily by the scene state than the scene graph, right?
So, we would like to suggest modifying the great ideas you presented in the following ways:
* Scene Graph > Scene state
* Scene Graph object > Scene object
* Scene Graph is an element of Scene State
* Scene State is dynamic as is the scene graph
* Scene state is distributed among "scene state contributors" just as his Scene Graphs are distributed
* More attention to the fact that huge portions of the scene are static (and enumerated in scene state) even in our most dynamic environments!
* The entire project is really about "world model" as you said, but scene graphs are only a part of this. Let's really develop a better world model sharable in offline programming, simulation, robotics and manufacturing in general.
We hope our comments help and restart this discussion. Please feel free to expand/criticise/ask for clarification etc...
Best regards,
Tim
----------------------------------------------------------------------------
Dr. Tim Guhl
Cooperative Research Projects
Research and Predevelopment (R12-V)
KUKA Roboter GmbH
Zugspitzstrasse 140
86165 Augsburg
Germany
Tel. +49 821 797-3270
Mob. +49 173 357 3724
Fax +49 821 797-41-3270
TimGuhl [..] ...
www.kuka-robotics.com
----------------------------------------------------------------------------
KUKA Roboter GmbH
Geschäftsführung / Board of Directors:
Manfred Gundel (CEO), Michael Albert
Sitz und Registergericht / Registered Office:
Augsburg HRB 14914
Diese E-Mail kann vertrauliche und/oder rechtlich geschützte Informationen enthalten. Wenn Sie nicht
der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den
Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe
dieser Mail ist nicht gestattet.
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient
(or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any
unauthorized copying, disclosure or distribution of contents of this e-mail is strictly forbidden.
> -----Ursprüngliche Nachricht-----
> Von: brics-all-bounces [..] ... [mailto:brics-all-
> bounces [..] ...] Im Auftrag von Herman Bruyninckx
> Gesendet: Montag, 11. Oktober 2010 08:34
> An: Rosetta WP 1; Rosetta WP2 mailing list; wp3 [..] ...; Orocos
> Developers; all [..] ...
> Cc: Herman Bruyninckx
> Betreff: [BRICS-all] Some concrete suggestions for a "scene graph"
> project...
>
> Dear all,
>
> This message is sent to mailinglists in various projects: Orocos, BRICS
> and
> Rosetta. The reason: it's about a development that all projects need,
> and
> could preferably do together. That development is the "realtime scene
> graph", in other words, the software support to keep an representation
> of
> the world online, and make it available to all robots, sensors,
> tools,...
> in the system to read from it, and write to it.
>
> There is a first draft available on
> <http://people.mech.kuleuven.be/~bruyninc/scenegraph />.
> It gives a rough idea about what I see as the major new features, wrt
> to
> existing implementations:
> * Real time efficiency
> * Deployable as a service
> * Unique IDs for all world objects
> * Explicit representation of uncertainty
> * Short-term memory
> Some of these features exist already somewhere, but not the
> combination. I
> think it should definitely be possible to merge different scene graphs,
> but
> also to use (a relevant part of) them in a hard realtime robot
> controller.
>
> Before I go further with this design, and discuss it with other
> stakeholders that have shown intrest (ROS, RoboEarth), I thought it
> would
> be appropriate to first get the opinion from the project partners I am
> most
> closely involved with.
>
> So, please give your opinions about the suggested design:
> - are the five focus point ok?
> - is the architecture ok? (That is, a "fast object" and "distributed
> service" combination, to be designed together.)
> - is some particular use case missing?
>
> I am looking forward to your critical comments?
>
> Best regards,
>
> Herman Bruyninckx
> _______________________________________________
> brics-all mailing list
> brics-all [..] ...
> http://mailman.gps-stuttgart.de/mailman/listinfo/brics-all
[BRICS-all] Some concrete suggestions for a "scene graph" projec
On Thu, 18 Nov 2010, BRICS Office wrote:
> Dear Herman,
>
> Sorry for taking our time, but we have now had some feedback on this from
> within KUKA and would like to contribute it to the discussion.
Much appreciated!
> Overall we really like your move but would like to suggest to not think
> just about Scene Graphs, but to think about Scene States. In short if I
> tell you I am at (x-4,y=5,z=6) that is not nearly as useful as "I am in
> robot 1's gripper at (4,5,6)" etc. The state of a scene tells us much
> more about what's going on than the dynamic graph alone.
I do not fully understand the meaning of what you call "Scene State"... The
fact that you have the knowledge about in what gripper something is at a
certain moment is a natural consequence of using the graph: _all_ geometric
information is not just provided stand-alone, but linked to the relevant
primitives (tools, robots, sensors,...) that are represented in the scene
graph. The "arrows" in the scene graph encode these relationships; e.g.,
"being grasped by" would be a relevant relationship that a gripper needs to
offer in a scene graph.
> Use case as you request:
> 1) Imagine the object in the gripper of the robot above. Using the scene
> state representation we know that the transformation from the gripper to
> the object does not change now --> no need to compute this. Could your
> "spanning tree" handle this?
It should, because it is _generated_ (in this particular use case) from the
knowledge that an object has been grasped by the gripper. (Where this
knowledge comes from, and how it is applied to generate the spanning tree,
that's another issue, and one to be dealt with in the components that _use_
the scene graph.
> 2) We can imagine that two robots doing something useful together have
> completely static scene graphs with respect to each other
The _connections_ in the graph would remain the same, but the scene graph
itself would not be static: it will reflect the changes in the positions of
both robots (at least).
> and have no
> need for dynamic graph communication with each other
I guess they _will_ have such a need, especially when you want the motions
of both robots not to hinder each other, or to be coordinated in real time
because they have to operate the same object in the world. (Remember our
Automatica demo with the two LWRs and the box?)
> or any information
> about a third performing an independent task in the same workspace during
> this particular scene in this particular scene state. In short, keeping
> scene state greatly reduces scene graph analysis and computational
> complexity.--> again, could your spanning tree handle this?
Probably I do not understand fully what you mean with "scene state"...
> 3) If you want to grab a box with a two finger gripper, you normally have
> 3 grasps (twosides, othertwosides, topbottom). Knowing the state of the
> box (on a table, I hold the box, you hold the box) determines which grips
> are appropriate in the situation. --> this can be represented more easily
> by the scene state than the scene graph, right?
Some remark? No idea because of "semantic confusion" about scene state :-)
> So, we would like to suggest modifying the great ideas you presented in
> the following ways:
>
> * Scene Graph > Scene state
> * Scene Graph object > Scene object
> * Scene Graph is an element of Scene State
> * Scene State is dynamic as is the scene graph
> * Scene state is distributed among "scene state contributors" just as
> his Scene Graphs are distributed
I see nothing else but suggestions to rename things... That is probably not
what you meant :-)
> * More attention to the fact that huge portions of the scene are static
> (and enumerated in scene state) even in our most dynamic environments!
The "spanning trees" for computation exactly have this goal (as one of many
more goals): if you know that some part of the world will not change, you
should not foresee any computations to "update" that part.
> * The entire project is really about "world model" as you said, but
> scene graphs are only a part of this. Let's really develop a better world
> model sharable in offline programming, simulation, robotics and
> manufacturing in general.
That's my ambition. There must be some semantic confusion amongst us; which
I think is very normal, in this early stage of the ideas.
> We hope our comments help and restart this discussion. Please feel free
> to expand/criticise/ask for clarification etc...
>
> Best regards,
>
> Tim
Thanks a lot!
Herman
[ROSETTA WP2] [BRICS-all] Some concrete suggestions for a "scene
Hi Herman, isn't the KUKA description related to the special (but in standard
applications common) case that geometric relations are 'sufficiently' fixed so
they can be transformed into specific controller actions/states (embedded in
the control without maintaining the graph), and then maintaining the semantics
in a higher level scene graph (e.g. RDF knowledge base, or as part of
engineering tools) that is not real-time capable?
I don't like scene graph either (confusion with graphics needs and with what
you have in CAD tools when the application is engineered), so what about
State Graph?
Cheers Klas
_________________________________________no text inlined below____
Herman Bruyninckx skrev 2010-11-21 16:13:
> On Thu, 18 Nov 2010, BRICS Office wrote:
>
>> Dear Herman,
>>
>> Sorry for taking our time, but we have now had some feedback on this from
>> within KUKA and would like to contribute it to the discussion.
> Much appreciated!
>
>> Overall we really like your move but would like to suggest to not think
>> just about Scene Graphs, but to think about Scene States. In short if I
>> tell you I am at (x-4,y=5,z=6) that is not nearly as useful as "I am in
>> robot 1's gripper at (4,5,6)" etc. The state of a scene tells us much
>> more about what's going on than the dynamic graph alone.
>
> I do not fully understand the meaning of what you call "Scene State"... The
> fact that you have the knowledge about in what gripper something is at a
> certain moment is a natural consequence of using the graph: _all_ geometric
> information is not just provided stand-alone, but linked to the relevant
> primitives (tools, robots, sensors,...) that are represented in the scene
> graph. The "arrows" in the scene graph encode these relationships; e.g.,
> "being grasped by" would be a relevant relationship that a gripper needs to
> offer in a scene graph.
>
>> Use case as you request:
>> 1) Imagine the object in the gripper of the robot above. Using the scene
>> state representation we know that the transformation from the gripper to
>> the object does not change now --> no need to compute this. Could your
>> "spanning tree" handle this?
>
> It should, because it is _generated_ (in this particular use case) from the
> knowledge that an object has been grasped by the gripper. (Where this
> knowledge comes from, and how it is applied to generate the spanning tree,
> that's another issue, and one to be dealt with in the components that _use_
> the scene graph.
>
>> 2) We can imagine that two robots doing something useful together have
>> completely static scene graphs with respect to each other
>
> The _connections_ in the graph would remain the same, but the scene graph
> itself would not be static: it will reflect the changes in the positions of
> both robots (at least).
>
>> and have no
>> need for dynamic graph communication with each other
>
> I guess they _will_ have such a need, especially when you want the motions
> of both robots not to hinder each other, or to be coordinated in real time
> because they have to operate the same object in the world. (Remember our
> Automatica demo with the two LWRs and the box?)
>
>> or any information
>> about a third performing an independent task in the same workspace during
>> this particular scene in this particular scene state. In short, keeping
>> scene state greatly reduces scene graph analysis and computational
>> complexity.--> again, could your spanning tree handle this?
>
> Probably I do not understand fully what you mean with "scene state"...
>
>> 3) If you want to grab a box with a two finger gripper, you normally have
>> 3 grasps (twosides, othertwosides, topbottom). Knowing the state of the
>> box (on a table, I hold the box, you hold the box) determines which grips
>> are appropriate in the situation. --> this can be represented more easily
>> by the scene state than the scene graph, right?
>
> Some remark? No idea because of "semantic confusion" about scene state :-)
>
>> So, we would like to suggest modifying the great ideas you presented in
>> the following ways:
>>
>> * Scene Graph > Scene state
>> * Scene Graph object > Scene object
>> * Scene Graph is an element of Scene State
>> * Scene State is dynamic as is the scene graph
>> * Scene state is distributed among "scene state contributors" just as
>> his Scene Graphs are distributed
>
> I see nothing else but suggestions to rename things... That is probably not
> what you meant :-)
>
>> * More attention to the fact that huge portions of the scene are static
>> (and enumerated in scene state) even in our most dynamic environments!
>
> The "spanning trees" for computation exactly have this goal (as one of many
> more goals): if you know that some part of the world will not change, you
> should not foresee any computations to "update" that part.
>
>> * The entire project is really about "world model" as you said, but
>> scene graphs are only a part of this. Let's really develop a better world
>> model sharable in offline programming, simulation, robotics and
>> manufacturing in general.
>
> That's my ambition. There must be some semantic confusion amongst us; which
> I think is very normal, in this early stage of the ideas.
>
>> We hope our comments help and restart this discussion. Please feel free
>> to expand/criticise/ask for clarification etc...
>>
>> Best regards,
>>
>> Tim
>
> Thanks a lot!
>
> Herman
>
>
> _______________________________________________
> WP2 mailing list
> WP2 [..] ...
> http://box448.bluehost.com/mailman/listinfo/wp2_fp7rosetta.org
-------------------------------------------------------------------------
o--< Klas Nilsson, Dept of Computer Science, Lund University - LTH
\ Box 118, S-221 00 LUND, Sweden | phone:+46-(0)46-2224304
O klas [..] ...,klas [..] ..." title="mailto:klas [..] ...,klas [..] ...">mailto:klas [..] ...,klas [..] ... | mobil:+46-(0)705-365015
+->>[_] http://www.cs.lth.se/~klas | skype:klas_nilsson
|
+-------<< Real-time << control << components << PnP << program << users
[BRICS-all] Some concrete suggestions for a "scene graph" projec
Hello Herman,
we just want to let you know that we (WP3 at BRICS project) are highly
interested in a scenegraph for robotics. Such an approach might be the
right concept to connect our 3D Perception and Modeling (BRICS_3D) and
the Mobile Manipulation (BRICS_MM) libraries.
We already started to incorporate the COLLADA format to the BRICS_MM. On
your webpage you list articulated_systems, node, scene and rigid_body
entities. We think that articulated_systems should be considered
together with the "library_joints, library_kinematics_models and
library_kinematics_scenes" as they are related.
Best Regards,
Sebastian Blumenthal and Alexey Zakharov
Herman Bruyninckx wrote:
> Dear all,
>
> This message is sent to mailinglists in various projects: Orocos, BRICS and
> Rosetta. The reason: it's about a development that all projects need, and
> could preferably do together. That development is the "realtime scene
> graph", in other words, the software support to keep an representation of
> the world online, and make it available to all robots, sensors, tools,...
> in the system to read from it, and write to it.
>
> There is a first draft available on
> <http://people.mech.kuleuven.be/~bruyninc/scenegraph />.
> It gives a rough idea about what I see as the major new features, wrt to
> existing implementations:
> * Real time efficiency
> * Deployable as a service
> * Unique IDs for all world objects
> * Explicit representation of uncertainty
> * Short-term memory
> Some of these features exist already somewhere, but not the combination. I
> think it should definitely be possible to merge different scene graphs, but
> also to use (a relevant part of) them in a hard realtime robot controller.
>
> Before I go further with this design, and discuss it with other
> stakeholders that have shown intrest (ROS, RoboEarth), I thought it would
> be appropriate to first get the opinion from the project partners I am most
> closely involved with.
>
> So, please give your opinions about the suggested design:
> - are the five focus point ok?
> - is the architecture ok? (That is, a "fast object" and "distributed
> service" combination, to be designed together.)
> - is some particular use case missing?
>
> I am looking forward to your critical comments?
>
> Best regards,
>
> Herman Bruyninckx
> _______________________________________________
> brics-all mailing list
> brics-all [..] ...
> http://mailman.gps-stuttgart.de/mailman/listinfo/brics-all
Some concrete suggestions for a "scene graph" project...
On 11/10/10 15:34, Herman Bruyninckx wrote:
> Dear all,
>
> This message is sent to mailinglists in various projects: Orocos, BRICS and
> Rosetta. The reason: it's about a development that all projects need, and
> could preferably do together. That development is the "realtime scene
> graph", in other words, the software support to keep an representation of
> the world online, and make it available to all robots, sensors, tools,...
> in the system to read from it, and write to it.
>
> There is a first draft available on
> <http://people.mech.kuleuven.be/~bruyninc/scenegraph />.
> It gives a rough idea about what I see as the major new features, wrt to
> existing implementations:
> * Real time efficiency
> * Deployable as a service
> * Unique IDs for all world objects
> * Explicit representation of uncertainty
> * Short-term memory
> Some of these features exist already somewhere, but not the combination. I
> think it should definitely be possible to merge different scene graphs, but
> also to use (a relevant part of) them in a hard realtime robot controller.
>
> Before I go further with this design, and discuss it with other
> stakeholders that have shown intrest (ROS, RoboEarth), I thought it would
> be appropriate to first get the opinion from the project partners I am most
> closely involved with.
>
> So, please give your opinions about the suggested design:
> - are the five focus point ok?
> - is the architecture ok? (That is, a "fast object" and "distributed
> service" combination, to be designed together.)
> - is some particular use case missing?
>
> I am looking forward to your critical comments?
First off, I like the idea. Please make sure it's a library, not tied to
a particular framework.
The name "scene graph" is fine. This is what computer games and computer
graphics technologies have used to describe the same thing in virtual
space for decades; I see no reason to use anything different.
I don't quite understand the "short-term memory point." The explanation
needs a little work to explain more clearly what you mean. The bit about
knowing the future plans of others is important to robotics, but I'm not
convinced it belongs in a scene graph. Should plans be in the same data
structure/storage as knowledge about the locations of objects?
I think it is true that the robotics domain has a greater richness of
use cases than others. :) A nuclear power plant has a restricted set of
use cases. So does an ATM, a car's engine controller, your microwave,
the space shuttle, MS Word, and so on. None of them have to deal so
directly with a constantly-changing environment.
The plugin-based architecture makes sense for allowing combined support
of real-time-fast and distributed - you can make the real-time-fast
stuff the core and add a distributed interface as a plugin.
I agree with the need for asynchronous operations. Such an API is also
easy to wrap into a synchronous API if we need it later on. It's much
harder to go the other way.
Geoff
Some concrete suggestions for a "scene graph" project...
On Wed, 13 Oct 2010, Geoffrey Biggs wrote:
> On 12/10/10 15:12, Herman Bruyninckx wrote:
>> On Tue, 12 Oct 2010, Geoffrey Biggs wrote:
>>> First off, I like the idea. Please make sure it's a library, not tied
>>> to
>>> a particular framework.
>>
>> You've read my remark about "Gearbox"? :-) But this point is not under
>> my
>> control, that's what the _implementors_ will make of it...
>
> I actually wrote that line before I even read the webpage. The remark
> about Gearbox got a quiet smile. ;) I don't really care if people put it
> in Gearbox or just distribute it themselves. What I care about is that
> people release stuff in a framework-independent way. That was why I
> helped found Gearbox: to encourage such an approach.
And I am all behind this ambition!
>>> The name "scene graph" is fine. This is what computer games and
>>> computer
>>> graphics technologies have used to describe the same thing in virtual
>>> space for decades; I see no reason to use anything different.
>>>
>>> I don't quite understand the "short-term memory point." The explanation
>>> needs a little work to explain more clearly what you mean.
>>
>> I will. But it is a feature that ROS's libTF has: it does not only store
>> the latest state of the world, but also older states.
>
> That makes more sense. The description made it sound like you wanted to
> store past plans by all robots. Storing previous states of the world is
> highly useful and should be a necessary feature. The API used to look
> them up will also be important.
Yes. And it's a difficult one to get "right". Probably because of the wide
variety in use cases...
The simplest part of that API should just be "database-like queries", such
as: give me the poses of frame XYZ during the last second". The more
complex thing would be queries like: give me the path of frame XYZ during
the last second with a 5cm tolerance". I think the former kind of queries
will be in the "core", the latter would be in one particular "plug in",
that is identified by (i) the type of paths it works with, (ii) the type of
tolerance/uncertainty it works with, and (iii) the specific algorithm it
uses to connect path, tolerance and frame history.
I'm struggling with the best design for "plug ins". So, any concrete
discussion on this topic would be high on my agenda...
>>> The bit about
>>> knowing the future plans of others is important to robotics, but I'm
>>> not
>>> convinced it belongs in a scene graph. Should plans be in the same data
>>> structure/storage as knowledge about the locations of objects?
>>
>> Good remark... I have to think about if further.
>>
>>> I think it is true that the robotics domain has a greater richness of
>>> use cases than others. :) A nuclear power plant has a restricted set of
>>> use cases. So does an ATM, a car's engine controller, your microwave,
>>> the space shuttle, MS Word, and so on. None of them have to deal so
>>> directly with a constantly-changing environment.
>>
>>> The plugin-based architecture makes sense for allowing combined support
>>> of real-time-fast and distributed - you can make the real-time-fast
>>> stuff the core and add a distributed interface as a plugin.
>>>
>>> I agree with the need for asynchronous operations. Such an API is also
>>> easy to wrap into a synchronous API if we need it later on. It's much
>>> harder to go the other way.
>>
>> Indeed.
>>
>> Thanks for the reactions!
>>
>> Herman
>
> Geoff
Herman
Some concrete suggestions for a "scene graph" project...
On 13/10/10 14:15, Herman Bruyninckx wrote:
> On Wed, 13 Oct 2010, Geoffrey Biggs wrote:
>>>> The name "scene graph" is fine. This is what computer games and
>>>> computer
>>>> graphics technologies have used to describe the same thing in virtual
>>>> space for decades; I see no reason to use anything different.
>>>>
>>>> I don't quite understand the "short-term memory point." The explanation
>>>> needs a little work to explain more clearly what you mean.
>>>
>>> I will. But it is a feature that ROS's libTF has: it does not only store
>>> the latest state of the world, but also older states.
>>
>> That makes more sense. The description made it sound like you wanted to
>> store past plans by all robots. Storing previous states of the world is
>> highly useful and should be a necessary feature. The API used to look
>> them up will also be important.
>
> Yes. And it's a difficult one to get "right". Probably because of the wide
> variety in use cases...
> The simplest part of that API should just be "database-like queries", such
> as: give me the poses of frame XYZ during the last second". The more
> complex thing would be queries like: give me the path of frame XYZ during
> the last second with a 5cm tolerance". I think the former kind of queries
> will be in the "core", the latter would be in one particular "plug in",
> that is identified by (i) the type of paths it works with, (ii) the type of
> tolerance/uncertainty it works with, and (iii) the specific algorithm it
> uses to connect path, tolerance and frame history.
That's a good approach. The simple API can be real-time, potentially
fast, and built in. More complex ones can be plugins (or wrapping
interfaces that load the library internnally?) that build on the simple
API to achieve their more complex goals. I was thinking of a database
approach for the simple one, too.
You could even have a plugin that takes SQL queries... :)
> I'm struggling with the best design for "plug ins". So, any concrete
> discussion on this topic would be high on my agenda...
Designing a plugin API is never easy, because of the need to balance
range of plugin capability with plugin ease of implementation.
One thing we did with OpenRTM-aist, which has quite an extensive plugin
capability, was to have a single universal API (actually just one
function) across all plugins for loading and initialising the plugin.
The rest of the API varies depending on the specific area that the
plugin targets. I suspect this is a common approach to plugin design. I
think this would be a good place to start: have a common way for plugins
to load, then differentiate the rest of the plugin API by what area the
plugin targets. This would simplify adding new plugin types later on by
partitioning them out.
Geoff
Some concrete suggestions for a "scene graph" project...
On Wed, 13 Oct 2010, Geoffrey Biggs wrote:
> On 13/10/10 14:15, Herman Bruyninckx wrote:
>> On Wed, 13 Oct 2010, Geoffrey Biggs wrote:
>>>>> The name "scene graph" is fine. This is what computer games and
>>>>> computer
>>>>> graphics technologies have used to describe the same thing in virtual
>>>>> space for decades; I see no reason to use anything different.
>>>>>
>>>>> I don't quite understand the "short-term memory point." The explanation
>>>>> needs a little work to explain more clearly what you mean.
>>>>
>>>> I will. But it is a feature that ROS's libTF has: it does not only store
>>>> the latest state of the world, but also older states.
>>>
>>> That makes more sense. The description made it sound like you wanted to
>>> store past plans by all robots. Storing previous states of the world is
>>> highly useful and should be a necessary feature. The API used to look
>>> them up will also be important.
>>
>> Yes. And it's a difficult one to get "right". Probably because of the wide
>> variety in use cases...
>> The simplest part of that API should just be "database-like queries", such
>> as: give me the poses of frame XYZ during the last second". The more
>> complex thing would be queries like: give me the path of frame XYZ during
>> the last second with a 5cm tolerance". I think the former kind of queries
>> will be in the "core", the latter would be in one particular "plug in",
>> that is identified by (i) the type of paths it works with, (ii) the type of
>> tolerance/uncertainty it works with, and (iii) the specific algorithm it
>> uses to connect path, tolerance and frame history.
>
> That's a good approach. The simple API can be real-time, potentially
> fast, and built in. More complex ones can be plugins (or wrapping
> interfaces that load the library internnally?) that build on the simple
> API to achieve their more complex goals. I was thinking of a database
> approach for the simple one, too.
>
> You could even have a plugin that takes SQL queries... :)
When you mention SQL databases, I do not associate them to "real time". But
it could be one possible _implementation_ of the same concept, and even the
same API.
>> I'm struggling with the best design for "plug ins". So, any concrete
>> discussion on this topic would be high on my agenda...
>
> Designing a plugin API is never easy, because of the need to balance
> range of plugin capability with plugin ease of implementation.
Indeed.
> One thing we did with OpenRTM-aist, which has quite an extensive plugin
> capability, was to have a single universal API (actually just one
> function) across all plugins for loading and initialising the plugin.
That's the easy part :-)
> The rest of the API varies depending on the specific area that the
> plugin targets. I suspect this is a common approach to plugin design. I
> think this would be a good place to start: have a common way for plugins
> to load, then differentiate the rest of the plugin API by what area the
> plugin targets. This would simplify adding new plugin types later on by
> partitioning them out.
You have shifted the discussion back to where I wanted it to go: the
problem is to find the best approach for this "differentiation" while still
guaranteeing compatibility with the core.
But I suggest to first focus on the core. That core should be able to stand
on its own, as a solution that solves already many concrete problems.
Herman
Some concrete suggestions for a "scene graph" project...
On Tue, 12 Oct 2010, Geoffrey Biggs wrote:
> On 11/10/10 15:34, Herman Bruyninckx wrote:
>> Dear all,
>>
>> This message is sent to mailinglists in various projects: Orocos, BRICS and
>> Rosetta. The reason: it's about a development that all projects need, and
>> could preferably do together. That development is the "realtime scene
>> graph", in other words, the software support to keep an representation of
>> the world online, and make it available to all robots, sensors, tools,...
>> in the system to read from it, and write to it.
>>
>> There is a first draft available on
>> <http://people.mech.kuleuven.be/~bruyninc/scenegraph />.
>> It gives a rough idea about what I see as the major new features, wrt to
>> existing implementations:
>> * Real time efficiency
>> * Deployable as a service
>> * Unique IDs for all world objects
>> * Explicit representation of uncertainty
>> * Short-term memory
>> Some of these features exist already somewhere, but not the combination. I
>> think it should definitely be possible to merge different scene graphs, but
>> also to use (a relevant part of) them in a hard realtime robot controller.
>>
>> Before I go further with this design, and discuss it with other
>> stakeholders that have shown intrest (ROS, RoboEarth), I thought it would
>> be appropriate to first get the opinion from the project partners I am most
>> closely involved with.
>>
>> So, please give your opinions about the suggested design:
>> - are the five focus point ok?
>> - is the architecture ok? (That is, a "fast object" and "distributed
>> service" combination, to be designed together.)
>> - is some particular use case missing?
>>
>> I am looking forward to your critical comments?
>
>
> First off, I like the idea. Please make sure it's a library, not tied to
> a particular framework.
You've read my remark about "Gearbox"? :-) But this point is not under my
control, that's what the _implementors_ will make of it...
> The name "scene graph" is fine. This is what computer games and computer
> graphics technologies have used to describe the same thing in virtual
> space for decades; I see no reason to use anything different.
>
> I don't quite understand the "short-term memory point." The explanation
> needs a little work to explain more clearly what you mean.
I will. But it is a feature that ROS's libTF has: it does not only store
the latest state of the world, but also older states.
> The bit about
> knowing the future plans of others is important to robotics, but I'm not
> convinced it belongs in a scene graph. Should plans be in the same data
> structure/storage as knowledge about the locations of objects?
Good remark... I have to think about if further.
> I think it is true that the robotics domain has a greater richness of
> use cases than others. :) A nuclear power plant has a restricted set of
> use cases. So does an ATM, a car's engine controller, your microwave,
> the space shuttle, MS Word, and so on. None of them have to deal so
> directly with a constantly-changing environment.
> The plugin-based architecture makes sense for allowing combined support
> of real-time-fast and distributed - you can make the real-time-fast
> stuff the core and add a distributed interface as a plugin.
>
> I agree with the need for asynchronous operations. Such an API is also
> easy to wrap into a synchronous API if we need it later on. It's much
> harder to go the other way.
Indeed.
Thanks for the reactions!
Herman
Some concrete suggestions for a "scene graph" project...
On 12/10/10 15:12, Herman Bruyninckx wrote:
> On Tue, 12 Oct 2010, Geoffrey Biggs wrote:
>> First off, I like the idea. Please make sure it's a library, not tied to
>> a particular framework.
>
> You've read my remark about "Gearbox"? :-) But this point is not under my
> control, that's what the _implementors_ will make of it...
I actually wrote that line before I even read the webpage. The remark
about Gearbox got a quiet smile. ;) I don't really care if people put it
in Gearbox or just distribute it themselves. What I care about is that
people release stuff in a framework-independent way. That was why I
helped found Gearbox: to encourage such an approach.
>> The name "scene graph" is fine. This is what computer games and computer
>> graphics technologies have used to describe the same thing in virtual
>> space for decades; I see no reason to use anything different.
>>
>> I don't quite understand the "short-term memory point." The explanation
>> needs a little work to explain more clearly what you mean.
>
> I will. But it is a feature that ROS's libTF has: it does not only store
> the latest state of the world, but also older states.
That makes more sense. The description made it sound like you wanted to
store past plans by all robots. Storing previous states of the world is
highly useful and should be a necessary feature. The API used to look
them up will also be important.
>> The bit about
>> knowing the future plans of others is important to robotics, but I'm not
>> convinced it belongs in a scene graph. Should plans be in the same data
>> structure/storage as knowledge about the locations of objects?
>
> Good remark... I have to think about if further.
>
>> I think it is true that the robotics domain has a greater richness of
>> use cases than others. :) A nuclear power plant has a restricted set of
>> use cases. So does an ATM, a car's engine controller, your microwave,
>> the space shuttle, MS Word, and so on. None of them have to deal so
>> directly with a constantly-changing environment.
>
>> The plugin-based architecture makes sense for allowing combined support
>> of real-time-fast and distributed - you can make the real-time-fast
>> stuff the core and add a distributed interface as a plugin.
>>
>> I agree with the need for asynchronous operations. Such an API is also
>> easy to wrap into a synchronous API if we need it later on. It's much
>> harder to go the other way.
>
> Indeed.
>
> Thanks for the reactions!
>
> Herman
Geoff
Some concrete suggestions for a "scene graph" project...
On 10/11/2010 08:34 AM, Herman Bruyninckx wrote:
> Dear all,
>
> This message is sent to mailinglists in various projects: Orocos, BRICS and
> Rosetta. The reason: it's about a development that all projects need, and
> could preferably do together. That development is the "realtime scene
> graph", in other words, the software support to keep an representation of
> the world online, and make it available to all robots, sensors, tools,...
> in the system to read from it, and write to it.
>
> There is a first draft available on
> <http://people.mech.kuleuven.be/~bruyninc/scenegraph />.
> It gives a rough idea about what I see as the major new features, wrt to
> existing implementations:
> * Real time efficiency
> * Deployable as a service
> * Unique IDs for all world objects
> * Explicit representation of uncertainty
> * Short-term memory
> Some of these features exist already somewhere, but not the combination. I
> think it should definitely be possible to merge different scene graphs, but
> also to use (a relevant part of) them in a hard realtime robot controller.
>
> Before I go further with this design, and discuss it with other
> stakeholders that have shown intrest (ROS, RoboEarth), I thought it would
> be appropriate to first get the opinion from the project partners I am most
> closely involved with.
>
> So, please give your opinions about the suggested design:
> - are the five focus point ok?
> - is the architecture ok? (That is, a "fast object" and "distributed
> service" combination, to be designed together.)
> - is some particular use case missing?
>
> I am looking forward to your critical comments?
>
> Best regards,
>
> Herman Bruyninckx
I'm forwarding this internally at DFKI as we are developping such a tool
for a while now ...