Why no shared_ptr's for Ports and Properties

Submitted by markus.klotzbuecher on Fri, 2010-09-03 07:40

RTT-dev

Hi,

Peter, we discussed this some time ago, IIRC the reason
Ports/Properties are not managed by smartptrs is because they're
association with a certain TaskContext is not clear?

For lua scripting this means I can not place them under coordination
of the garbage collector, because that implies they will be destroyed
when unreferenced or when the lua enabled component is destroyed. Both
situations might result in crashes inspite of exclusive use of Lua
(which should not be possible): because a Port/Property was deleted
while it was still added to a TaskContext and either somebody tried to
use it or the TC tried to delete it while beeing deleted itself.

The alternative is to assume that Ports/Properties are never removed
from a TC (there is indeed no API to do that ATM) and that the TC will
delete these when it is destroyed itself (as is currently the
case). For Lua there remains still a small possiblity to leak memory
if a Port/Property is created but not added.

Using smart ptrs for Ports/Properties would solve the problem for me
because I can place the smart ptr itself under the gc and not care
about who else uses it.

Best regards
Markus

Why no shared_ptr's for Ports and Properties

Submitted by peter on Fri, 2010-09-03 08:32.

On Friday 03 September 2010 09:38:05 Markus Klotzbuecher wrote:
> Hi,
>
> Peter, we discussed this some time ago, IIRC the reason
> Ports/Properties are not managed by smartptrs is because they're
> association with a certain TaskContext is not clear?

No. The reason is that a port object is called by user code, and it's more
convenient to let the user create that object than to let the TC created, and
then get a (smart) pointer to it. For example:

  // member:
  InputPort<double> foo;
  // in component constructor:
  this->ports()->addPort(foo, "foo");
  // in update hook:
  double d;
  foo.read( d );

If the TC would manage it, we would get something like this:

  // member:
  InputPort<double>::shared_ptr foo;
  // in component constructor:
  foo.reset( new InputPort<double> );
  this->ports()->addPort( foo, "foo");
  // in update hook:
  foo->read(d);
  // use 'd'

OpenRTC does something like this btw:

  // member that will contain result of read():
  double d;
  // in component constructor:
  this->ports()->addPort(d, "foo");
  // in update hook:
  this->ports()->getPort("foo")->read();
  // use 'd'

I prefer the explicit presence of InputPort and OutputPort, to ease the
reading of the API.

Properties can already managed by PropertyBag ('ownProperty()') by a mechanism
that is second class compared to shared_ptr. The recommended way in 2.0 is to
do:

  // member Property 'd':
  double d;
  // in component constructor:
  this->properties()->addProperty(d, "d").doc("The 'd' parameter");
  // in update hook:
  // use 'd'

Which eliminates code for Property<double>.

>
> For lua scripting this means I can not place them under coordination
> of the garbage collector, because that implies they will be destroyed
> when unreferenced or when the lua enabled component is destroyed. Both
> situations might result in crashes inspite of exclusive use of Lua
> (which should not be possible): because a Port/Property was deleted
> while it was still added to a TaskContext and either somebody tried to
> use it or the TC tried to delete it while beeing deleted itself.

A TC will never delete a port or property itself. You should remove it from
the TC when you delete it though.

>
> The alternative is to assume that Ports/Properties are never removed
> from a TC (there is indeed no API to do that ATM) and that the TC will
> delete these when it is destroyed itself (as is currently the
> case).

This is not the case. It is that class that inherits from TC that must manage
the cleanup.

> For Lua there remains still a small possiblity to leak memory
> if a Port/Property is created but not added.
>
> Using smart ptrs for Ports/Properties would solve the problem for me
> because I can place the smart ptr itself under the gc and not care
> about who else uses it.

I'm in favour of using smart pointer as much as possible. They have the
disadvantage that they do not allow stack-based data, which in case of
properties and ports is common practice.

Peter

Why no shared_ptr's for Ports and Properties

Submitted by snrkiwi on Fri, 2010-09-03 12:12.

On Sep 3, 2010, at 04:29 , Peter Soetens wrote:

> On Friday 03 September 2010 09:38:05 Markus Klotzbuecher wrote:
>> Hi,
>>
>> Peter, we discussed this some time ago, IIRC the reason
>> Ports/Properties are not managed by smartptrs is because they're
>> association with a certain TaskContext is not clear?
>
> No. The reason is that a port object is called by user code, and it's more
> convenient to let the user create that object than to let the TC created, and
> then get a (smart) pointer to it. For example:
>
>

>  // member:
>  InputPort<double> foo;
>  // in component constructor:
>  this->ports()->addPort(foo, "foo");
>  // in update hook:
>  double d;
>  foo.read( d );
>

>
> If the TC would manage it, we would get something like this:
>

>  // member:
>  InputPort<double>::shared_ptr foo;
>  // in component constructor:
>  foo.reset( new InputPort<double> );
>  this->ports()->addPort( foo, "foo");
>  // in update hook:
>  foo->read(d);
>  // use 'd'
>

>
> OpenRTC does something like this btw:
>

>  // member that will contain result of read():
>  double d;
>  // in component constructor:
>  this->ports()->addPort(d, "foo");
>  // in update hook:
>  this->ports()->getPort("foo")->read();
>  // use 'd'
>

> I prefer the explicit presence of InputPort and OutputPort, to ease the
> reading of the API.
>
> Properties can already managed by PropertyBag ('ownProperty()') by a mechanism
> that is second class compared to shared_ptr. The recommended way in 2.0 is to
> do:
>

>  // member Property 'd':
>  double d;
>  // in component constructor:
>  this->properties()->addProperty(d, "d").doc("The 'd' parameter");
>  // in update hook:
>  // use 'd'
>

>
> Which eliminates code for Property<double>.
>
>>
>> For lua scripting this means I can not place them under coordination
>> of the garbage collector, because that implies they will be destroyed
>> when unreferenced or when the lua enabled component is destroyed. Both
>> situations might result in crashes inspite of exclusive use of Lua
>> (which should not be possible): because a Port/Property was deleted
>> while it was still added to a TaskContext and either somebody tried to
>> use it or the TC tried to delete it while beeing deleted itself.
>
> A TC will never delete a port or property itself. You should remove it from
> the TC when you delete it though.
>
>>
>> The alternative is to assume that Ports/Properties are never removed
>> from a TC (there is indeed no API to do that ATM) and that the TC will
>> delete these when it is destroyed itself (as is currently the
>> case).
>
> This is not the case. It is that class that inherits from TC that must manage
> the cleanup.
>
>> For Lua there remains still a small possiblity to leak memory
>> if a Port/Property is created but not added.
>>
>> Using smart ptrs for Ports/Properties would solve the problem for me
>> because I can place the smart ptr itself under the gc and not care
>> about who else uses it.
>
> I'm in favour of using smart pointer as much as possible. They have the
> disadvantage that they do not allow stack-based data, which in case of
> properties and ports is common practice.

I'd be interested to see a real-world example of that, as we have never put a property, port, or any other Orocos interface item, on the stack. We always use a TC. Even in unit testing.
S

Why no shared_ptr's for Ports and Properties

Submitted by peter on Fri, 2010-09-03 12:32.

On Fri, Sep 3, 2010 at 2:10 PM, S Roderick <kiwi [dot] net [..] ...> wrote:
> On Sep 3, 2010, at 04:29 , Peter Soetens wrote:
>
>> On Friday 03 September 2010 09:38:05 Markus Klotzbuecher wrote:
...
>>> Using smart ptrs for Ports/Properties would solve the problem for me
>>> because I can place the smart ptr itself under the gc and not care
>>> about who else uses it.
>>
>> I'm in favour of using smart pointer as much as possible. They have the
>> disadvantage that they do not allow stack-based data, which in case of
>> properties and ports is common practice.
>
> I'd be interested to see a real-world example of that, as we have never put a property, port, or any other Orocos interface item, on the stack. We always use a TC. Even in unit testing.

You have misread my comment. On the stack means writing
'Property<bool> param("param");' somewhere, even as a class member of
an object that is always on the heap (like a TC)... On the heap means:
'Property<bool>* param = new Property<bool>("param");'

I'm betting you hardly ever do case #2 ?

Peter

Why no shared_ptr's for Ports and Properties

Submitted by snrkiwi on Fri, 2010-09-03 12:32.

On Sep 3, 2010, at 08:27 , Peter Soetens wrote:

> On Fri, Sep 3, 2010 at 2:10 PM, S Roderick <kiwi [dot] net [..] ...> wrote:
>> On Sep 3, 2010, at 04:29 , Peter Soetens wrote:
>>
>>> On Friday 03 September 2010 09:38:05 Markus Klotzbuecher wrote:
> ...
>>>> Using smart ptrs for Ports/Properties would solve the problem for me
>>>> because I can place the smart ptr itself under the gc and not care
>>>> about who else uses it.
>>>
>>> I'm in favour of using smart pointer as much as possible. They have the
>>> disadvantage that they do not allow stack-based data, which in case of
>>> properties and ports is common practice.
>>
>> I'd be interested to see a real-world example of that, as we have never put a property, port, or any other Orocos interface item, on the stack. We always use a TC. Even in unit testing.
>
> You have misread my comment. On the stack means writing
> 'Property<bool> param("param");' somewhere, even as a class member of
> an object that is always on the heap (like a TC)... On the heap means:
> 'Property<bool>* param = new Property<bool>("param");'
>
> I'm betting you hardly ever do case #2 ?

Correct. Never I believe.

Though we also never explicitly instantiate interface items as you showed above with Property<bool>. They always live in a TC (well, except for a GUI accessing via CORBA).
S

Why no shared_ptr's for Ports and Properties

Submitted by Butch Slayer on Mon, 2010-09-06 19:24.

Hi,

Smart pointers and singletons are indeed nice design patterns. But they can
also be a pain in the ass and even with smart pointer end up deleting
resource twice. I'd like to see the KISS approach applied to a TC and all
other RTT base components. In my experience with smart pointers is you
should design your software using them from the start. Adding them
afterwards isn't a very good idea, unless you know what you are doing. In
the end we work with C++ here and the programmer is responsible to clean up.

A small example:
TimerThread alpha(1, "Alpha", 0.001);

int main()
{
PeriodicActivity act(shared_ptr(&alpha), 0);
....
}

In the end, alpha is going to be deleted twice... Its a simple example but
the point is that a mistake is easy to make.

Butch.

2010/9/3 Stephen Roderick <kiwi [dot] net [..] ...>

> On Sep 3, 2010, at 08:27 , Peter Soetens wrote:
>
> > On Fri, Sep 3, 2010 at 2:10 PM, S Roderick <kiwi [dot] net [..] ...> wrote:
> >> On Sep 3, 2010, at 04:29 , Peter Soetens wrote:
> >>
> >>> On Friday 03 September 2010 09:38:05 Markus Klotzbuecher wrote:
> > ...
> >>>> Using smart ptrs for Ports/Properties would solve the problem for me
> >>>> because I can place the smart ptr itself under the gc and not care
> >>>> about who else uses it.
> >>>
> >>> I'm in favour of using smart pointer as much as possible. They have the
> >>> disadvantage that they do not allow stack-based data, which in case of
> >>> properties and ports is common practice.
> >>
> >> I'd be interested to see a real-world example of that, as we have never
> put a property, port, or any other Orocos interface item, on the stack. We
> always use a TC. Even in unit testing.
> >
> > You have misread my comment. On the stack means writing
> > 'Property<bool> param("param");' somewhere, even as a class member of
> > an object that is always on the heap (like a TC)... On the heap means:
> > 'Property<bool>* param = new Property<bool>("param");'
> >
> > I'm betting you hardly ever do case #2 ?
>
> Correct. Never I believe.
>
> Though we also never explicitly instantiate interface items as you showed
> above with Property<bool>. They always live in a TC (well, except for a GUI
> accessing via CORBA).
> S
> --
> Orocos-Dev mailing list
> Orocos-Dev [..] ...
> http://lists.mech.kuleuven.be/mailman/listinfo/orocos-dev
>

Why no shared_ptr's for Ports and Properties

Submitted by markus.klotzbuecher on Fri, 2010-09-03 09:52.

On Fri, Sep 03, 2010 at 10:29:40AM +0200, Peter Soetens wrote:
> On Friday 03 September 2010 09:38:05 Markus Klotzbuecher wrote:
> > Hi,
> >
> > Peter, we discussed this some time ago, IIRC the reason
> > Ports/Properties are not managed by smartptrs is because they're
> > association with a certain TaskContext is not clear?
>
> No. The reason is that a port object is called by user code, and it's more
> convenient to let the user create that object than to let the TC created, and
> then get a (smart) pointer to it. For example:
>
>

>   // member:
>   InputPort<double> foo;
>   // in component constructor:
>   this->ports()->addPort(foo, "foo");
>   // in update hook:
>   double d;
>   foo.read( d );
>

>
> If the TC would manage it, we would get something like this:
>

>   // member:
>   InputPort<double>::shared_ptr foo;
>   // in component constructor:
>   foo.reset( new InputPort<double> );
>   this->ports()->addPort( foo, "foo");
>   // in update hook:
>   foo->read(d);
>   // use 'd'
>

Ok, makes sense.

> OpenRTC does something like this btw:
>

>   // member that will contain result of read():
>   double d;
>   // in component constructor:
>   this->ports()->addPort(d, "foo");
>   // in update hook:
>   this->ports()->getPort("foo")->read();
>   // use 'd'
>

>   // member Property 'd':
>   double d;
>   // in component constructor:
>   this->properties()->addProperty(d, "d").doc("The 'd' parameter");
>   // in update hook:
>   // use 'd'
>

>
> Which eliminates code for Property<double>.

Ok.

> > For lua scripting this means I can not place them under coordination
> > of the garbage collector, because that implies they will be destroyed
> > when unreferenced or when the lua enabled component is destroyed. Both
> > situations might result in crashes inspite of exclusive use of Lua
> > (which should not be possible): because a Port/Property was deleted
> > while it was still added to a TaskContext and either somebody tried to
> > use it or the TC tried to delete it while beeing deleted itself.
>
> A TC will never delete a port or property itself. You should remove it from

Are you really sure? If I garbage collect Ports/Props I get double
frees and segmentation faults. I suspected this to happen because Lua
garbage collects all its objects when it gets destructed (in my
lua-Taskcontexts destructor) and afterwards when the TaskContext
itself gets destructed the destructor of the vector of ports is called
which frees all its elements... ?

> the TC when you delete it though.

Well that would be an option, to track to which TC ports/properties
were added and then remove them when they get destructed. Overhead I
would like to avoid if possible.

> > The alternative is to assume that Ports/Properties are never removed
> > from a TC (there is indeed no API to do that ATM) and that the TC will
> > delete these when it is destroyed itself (as is currently the
> > case).
>
> This is not the case. It is that class that inherits from TC that must manage
> the cleanup.
>
> > For Lua there remains still a small possiblity to leak memory
> > if a Port/Property is created but not added.
> >
> > Using smart ptrs for Ports/Properties would solve the problem for me
> > because I can place the smart ptr itself under the gc and not care
> > about who else uses it.
>
> I'm in favour of using smart pointer as much as possible. They have the
> disadvantage that they do not allow stack-based data, which in case of
> properties and ports is common practice.

I get your point!
Thanks!

Markus

Why no shared_ptr's for Ports and Properties

Submitted by bruyninc on Fri, 2010-09-03 08:24.

On Fri, 3 Sep 2010, Markus Klotzbuecher wrote:

> Peter, we discussed this some time ago, IIRC the reason
> Ports/Properties are not managed by smartptrs is because they're
> association with a certain TaskContext is not clear?
>
> For lua scripting this means I can not place them under coordination
> of the garbage collector, because that implies they will be destroyed
> when unreferenced or when the lua enabled component is destroyed. Both
> situations might result in crashes inspite of exclusive use of Lua
> (which should not be possible): because a Port/Property was deleted
> while it was still added to a TaskContext and either somebody tried to
> use it or the TC tried to delete it while beeing deleted itself.

This is a "resource allocation coordination problem"! Seems that a
fundamental solution to this problem lies in providing an explicit
"component" for those resources. I think that makes sense, since Ports are
not to be created/deleted/updated in realtime, do they?

It's the first step towards more 'platform resource awareness' of RTT :-)
And that's going to be a major leap in making Orocos (even) more
professional-grade...

> The alternative is to assume that Ports/Properties are never removed
> from a TC (there is indeed no API to do that ATM) and that the TC will
> delete these when it is destroyed itself (as is currently the
> case). For Lua there remains still a small possiblity to leak memory
> if a Port/Property is created but not added.

Please, make a design that leaves no room for (doubts about) such
assumptions or memory leaks! In other words: provide "singletons" that
coordinate those resources.

> Using smart ptrs for Ports/Properties would solve the problem for me
> because I can place the smart ptr itself under the gc and not care
> about who else uses it.

Your "gc" is then probably knowing too much about a platform resource,
although it shouldn't...?

Herman

Why no shared_ptr's for Ports and Properties

Submitted by markus.klotzbuecher on Fri, 2010-09-03 10:00.

On Fri, Sep 03, 2010 at 10:19:59AM +0200, Herman Bruyninckx wrote:
> On Fri, 3 Sep 2010, Markus Klotzbuecher wrote:
>
> > Peter, we discussed this some time ago, IIRC the reason
> > Ports/Properties are not managed by smartptrs is because they're
> > association with a certain TaskContext is not clear?
> >
> > For lua scripting this means I can not place them under coordination
> > of the garbage collector, because that implies they will be destroyed
> > when unreferenced or when the lua enabled component is destroyed. Both
> > situations might result in crashes inspite of exclusive use of Lua
> > (which should not be possible): because a Port/Property was deleted
> > while it was still added to a TaskContext and either somebody tried to
> > use it or the TC tried to delete it while beeing deleted itself.
>
> This is a "resource allocation coordination problem"! Seems that a
> fundamental solution to this problem lies in providing an explicit
> "component" for those resources. I think that makes sense, since Ports are
> not to be created/deleted/updated in realtime, do they?

No, they don't. But the problem is that scripting holds references to
resources and it doesn't who else does too, consequently it is not
clear when it is safe to destruct these. One solution is to keep track
of the dependencies.

> It's the first step towards more 'platform resource awareness' of RTT :-)
> And that's going to be a major leap in making Orocos (even) more
> professional-grade...
>
> > The alternative is to assume that Ports/Properties are never removed
> > from a TC (there is indeed no API to do that ATM) and that the TC will
> > delete these when it is destroyed itself (as is currently the
> > case). For Lua there remains still a small possiblity to leak memory
> > if a Port/Property is created but not added.
>
> Please, make a design that leaves no room for (doubts about) such
> assumptions or memory leaks! In other words: provide "singletons" that
> coordinate those resources.

Yes. Allowing the possibility for leaks/crashes defeats point of using
a scripting language in the first place.

> > Using smart ptrs for Ports/Properties would solve the problem for me
> > because I can place the smart ptr itself under the gc and not care
> > about who else uses it.
>
> Your "gc" is then probably knowing too much about a platform resource,
> although it shouldn't...?

It's the purpose of a garbage collector to know about resources. In
this case it might know too little!

Best regards
Markus