I am building an app where I want a TaskContext to take action when a buffer is full with data that comes in irregularly via a port. I now have an event port and an updateHook in which data is copied to the buffer, and once the buffer is full, further action is taken. A more elegant solution in my view would be to let a port with a buffer policy generate an event and wake up the TC as soon as it becomes full. Would this in your view be an interesting feature to have and hence should be added to the API?
Theo.
portBufferFullEvent?
On Monday 11 April 2011 13:26:05 t [dot] j [dot] a [dot] devries [..] ... wrote:
> I am building an app where I want a TaskContext to take action when a
> buffer is full with data that comes in irregularly via a port. I now have
> an event port and an updateHook in which data is copied to the buffer, and
> once the buffer is full, further action is taken. A more elegant solution
> in my view would be to let a port with a buffer policy generate an event
> and wake up the TC as soon as it becomes full. Would this in your view be
> an interesting feature to have and hence should be added to the API?
It's a supervision-thing. What you should aim for is that a 'supervisor'
(which may be the TC itself or another TC) is notified of the buffer status and
then can take action, ie, trigger the TC. The irony of this situation is that
we try to model events as much as possible with data ports themselves, since
event callbacks can only call functions within the TC and not accross
components. Remember that 1.x had the same issue.
We're currently still strugling with compilation times and are thinking of
removing certain features ( with a compilation option for starters ) so we're
not that eager to add new features, unless they would simplify things as well.
Summarized: implementing it with a callback that triggers the TC is possible
but does not allow supervision (ie violates the 5C's). implementing it with
additional ports is probably not going to make it into mainline unless some
serious compilation time issues can be fixed.
Peter
portBufferFullEvent?
On Tue, 12 Apr 2011, Peter Soetens wrote:
> On Monday 11 April 2011 13:26:05 t [dot] j [dot] a [dot] devries [..] ... wrote:
>> I am building an app where I want a TaskContext to take action when a
>> buffer is full with data that comes in irregularly via a port. I now have
>> an event port and an updateHook in which data is copied to the buffer, and
>> once the buffer is full, further action is taken. A more elegant solution
>> in my view would be to let a port with a buffer policy generate an event
>> and wake up the TC as soon as it becomes full. Would this in your view be
>> an interesting feature to have and hence should be added to the API?
>
> It's a supervision-thing. What you should aim for is that a 'supervisor'
> (which may be the TC itself or another TC) is notified of the buffer status and
> then can take action, ie, trigger the TC. The irony of this situation is that
> we try to model events as much as possible with data ports themselves, since
> event callbacks can only call functions within the TC and not accross
> components. Remember that 1.x had the same issue.
>
> We're currently still strugling with compilation times and are thinking of
> removing certain features ( with a compilation option for starters ) so we're
> not that eager to add new features, unless they would simplify things as well.
>
> Summarized: implementing it with a callback that triggers the TC is possible
> but does not allow supervision (ie violates the 5C's). implementing it with
> additional ports is probably not going to make it into mainline unless some
> serious compilation time issues can be fixed.
I think the whole problem can be solved without anything extra: the real
issue here is that _two_ peers have to be coordinated, with respect to the
data they put on a Communication. This coordination can (should!) be done
by events: high-water + low water events are fired by the receiver, and the
sender reacts to them. In fact, neither of them has to know anything about
the _cause_ of the events (since that could be an implementation
configuration of the Communication buffer component).
Even one step of abstraction more: the buffer is a scarce resource that is
to be shared by two or more components, hence it deserves its own
full-fledged "Communication component". And as soon as you introduce this
"resource allocation mediator", the resulting design is obvious: let that
mediator coordinate access to its own resource; and hence, the design
suggestion I made above falls out automatically...
Herman
portBufferFullEvent?
On Tue, 12 Apr 2011, Vries, Theo J.A. de [imotec bv] wrote:
>
>
> 2011/4/11 Herman Bruyninckx <Herman [dot] Bruyninckx [..] ...>
> On Mon, 11 Apr 2011, t [dot] j [dot] a [dot] devries [..] ... wrote:
>
> I am building an app where I want a TaskContext to take
> action when a
> buffer is full with data that comes in irregularly via a
> port. I now have
> an event port and an updateHook in which data is copied
> to the buffer,
> and once the buffer is full, further action is taken.
> A more elegant solution in my view would be to let a
> port with a buffer
> policy generate an event and wake up the TC as soon as
> it becomes full.
> Would this in your view be an interesting feature to
> have and hence
> should be added to the API?
>
>
> This is the "low water-high water" pattern for buffered communication. It
> is good to have, but I see no reason to add something to the current API:
> developers have already the freedom to give their events good _names_, and
> that is what is important.
>
> I googled for this "low water-high water" pattern, but could not find anything.
> Do you maybe have a pointer?
I did the same thing as you, and such a search comes out empty... The words
are just too common :-( If I remember correctly, I got this term from the
time I was looking intensively to frameworks like Comedi, for hardware
device interfacing. I think the PCI protocol uses it...
> Anyway, this is an event that should be added to the _Communication_
> framework, not to the "TaskContext". The latter will allow to
> _react_ to
> the event, but it need not have anything extra to _support_ the
> event.
>
> I do not know communication frameworks in robotics that have the
> "low
> water-high water" pattern; but others, outside of robotics (hence,
> the
> _real ones_) do, such as, e.g., Comedi.
>
> Maybe (the new) SmartSoft of Christian Schlegel also has it...
>
> Herman
>
> I have trouble to understand what you say above. Let me explain a bit more about
> my situation and my idea, then maybe you can guide me a bit further.
> I use orocos-rtt straightforwardly, no remote things, so no framework other than
> orocos is involved.
Which is not the desired situation :-) Orocos _needs_ good communication
middleware project, as soon as you leave the single process context.
> I use "addEventPort" to wake up my component when new data
> arrives (coming from another local component). I have seen that the actual work
> involved in this method is implemented in InputPortInterface.
> When using a buffer policy for a port, it makes (more) sense to generate a
> wake-up event when the buffer is full; in my view, a new-data-arrived event of a
> non-buffering port is equivalent to a buffer-full event of a buffered port, both
> mean that action is needed in order to not loose data.
> So my personal preference would be to not change the API, but to change the
> effect of the method addEventPort:
This is a configuration of the policy. Which is the right thing to do, but
it should be done "in the right way". And RTT is not ready enough to fully
support "the right way", because the 5Cs are still a bit too coupled...
> for a non-buffering port, it remains as is,
> but for a buffering port, the event gets generated when the buffer is full.
> The current implementation in my application is, as you can imagine, to wake up
> each time that data arrives, and to only start taking real action when the
> buffer is full. This means that this component is getting active much more often
> than needed and I would like to avoid that.
>
> So how can I take your advise without resorting to another communication
> framework?
Implement the things yourself... But I really see no reason why you should
not use "another communication framework"... Of course, it should be good,
but trying to put everything and the kitchen sink into one single project
is "Bad Practice"! It leads to too much decoupling early on in the
design...
> And what do you / others think of the proposed change to addEventPort?
>
> Theo
Herman
portBufferFullEvent?
Theo,
I didn't receive the original post Herman is replying to. So I'll reply to his
one:
On Wednesday 13 April 2011 09:20:56 Herman Bruyninckx wrote:
> On Tue, 12 Apr 2011, Vries, Theo J.A. de [imotec bv] wrote:
> > 2011/4/11 Herman Bruyninckx <Herman [dot] Bruyninckx [..] ...>
> >
> > On Mon, 11 Apr 2011, t [dot] j [dot] a [dot] devries [..] ... wrote:
> > I am building an app where I want a TaskContext to take
> > action when a
> > buffer is full with data that comes in irregularly via a
> > port. I now have
> > an event port and an updateHook in which data is copied
> > to the buffer,
> > and once the buffer is full, further action is taken.
> > A more elegant solution in my view would be to let a
> > port with a buffer
> > policy generate an event and wake up the TC as soon as
> > it becomes full.
> > Would this in your view be an interesting feature to
> > have and hence
> > should be added to the API?
> >
> > This is the "low water-high water" pattern for buffered communication. It
> > is good to have, but I see no reason to add something to the current API:
> > developers have already the freedom to give their events good _names_,
> > and that is what is important.
> >
> > I googled for this "low water-high water" pattern, but could not find
> > anything. Do you maybe have a pointer?
>
> I did the same thing as you, and such a search comes out empty... The words
> are just too common :-( If I remember correctly, I got this term from the
> time I was looking intensively to frameworks like Comedi, for hardware
> device interfacing. I think the PCI protocol uses it...
>
> > Anyway, this is an event that should be added to the
> > _Communication_ framework, not to the "TaskContext". The latter
> > will allow to _react_ to
> > the event, but it need not have anything extra to _support_ the
> > event.
> >
> > I do not know communication frameworks in robotics that have the
> > "low
> > water-high water" pattern; but others, outside of robotics (hence,
> > the
> > _real ones_) do, such as, e.g., Comedi.
> >
> > Maybe (the new) SmartSoft of Christian Schlegel also has it...
> >
> > Herman
> >
> > I have trouble to understand what you say above. Let me explain a bit
> > more about my situation and my idea, then maybe you can guide me a bit
> > further. I use orocos-rtt straightforwardly, no remote things, so no
> > framework other than orocos is involved.
>
> Which is not the desired situation :-) Orocos _needs_ good communication
> middleware project, as soon as you leave the single process context.
>
> > I use "addEventPort" to wake up my component when new data
> > arrives (coming from another local component). I have seen that the
> > actual work involved in this method is implemented in
> > InputPortInterface.
> > When using a buffer policy for a port, it makes (more) sense to generate
> > a wake-up event when the buffer is full; in my view, a new-data-arrived
> > event of a non-buffering port is equivalent to a buffer-full event of a
> > buffered port, both mean that action is needed in order to not loose
> > data.
> > So my personal preference would be to not change the API, but to change
> > the
> > effect of the method addEventPort:
> This is a configuration of the policy. Which is the right thing to do, but
> it should be done "in the right way". And RTT is not ready enough to fully
> support "the right way", because the 5Cs are still a bit too coupled...
I agree here as well. We already have connection policies, so we can extend
that struct to cover this semantic when buffers are used. We don't have a
system in place that warns the *producer* for buffer empty, since an output
port has no way to know how a buffer is, and it's meaningless, since multiple
clients might be listening and each will have a different buffer status.
>
> > for a non-buffering port, it remains as is,
> > but for a buffering port, the event gets generated when the buffer is
> > full. The current implementation in my application is, as you can
> > imagine, to wake up each time that data arrives, and to only start
> > taking real action when the buffer is full. This means that this
> > component is getting active much more often than needed and I would like
> > to avoid that.
> >
> > So how can I take your advise without resorting to another communication
> > framework?
>
> Implement the things yourself... But I really see no reason why you should
> not use "another communication framework"... Of course, it should be good,
> but trying to put everything and the kitchen sink into one single project
> is "Bad Practice"! It leads to too much decoupling early on in the
> design...
>
> > And what do you / others think of the proposed change to addEventPort?
I think extending the ConnPolicy structure is the simplest thing to do. You
could name the field 'trigger_only_full' (would default to false) or
'trigger_each_sample' (would default to true). It would only matter to buffers.
BTW, people are asking what the difference is between a buffer of size 1 and a
'data' connection. The new API made these indistinguishable.
> >
> > Theo
>
> Herman
Peter
portBufferFullEvent?
2011/4/13 Peter Soetens <peter [..] ...>
> Theo,
>
> I didn't receive the original post Herman is replying to. So I'll reply to
> his
> one:
>
Sorry, I had hit the reply instead of reply to all button I guess.
>
> On Wednesday 13 April 2011 09:20:56 Herman Bruyninckx wrote:
> > On Tue, 12 Apr 2011, Vries, Theo J.A. de [imotec bv] wrote:
> > > 2011/4/11 Herman Bruyninckx <Herman [dot] Bruyninckx [..] ...>
> > >
> > > On Mon, 11 Apr 2011, t [dot] j [dot] a [dot] devries [..] ... wrote:
> > > I am building an app where I want a TaskContext to take
> > > action when a
> > > buffer is full with data that comes in irregularly via a
> > > port. I now have
> > > an event port and an updateHook in which data is copied
> > > to the buffer,
> > > and once the buffer is full, further action is taken.
> > > A more elegant solution in my view would be to let a
> > > port with a buffer
> > > policy generate an event and wake up the TC as soon as
> > > it becomes full.
> > > Would this in your view be an interesting feature to
> > > have and hence
> > > should be added to the API?
> > >
> > > This is the "low water-high water" pattern for buffered communication.
> It
> > > is good to have, but I see no reason to add something to the current
> API:
> > > developers have already the freedom to give their events good _names_,
> > > and that is what is important.
> > >
> > > I googled for this "low water-high water" pattern, but could not find
> > > anything. Do you maybe have a pointer?
> >
> > I did the same thing as you, and such a search comes out empty... The
> words
> > are just too common :-( If I remember correctly, I got this term from the
> > time I was looking intensively to frameworks like Comedi, for hardware
> > device interfacing. I think the PCI protocol uses it...
> >
> > > Anyway, this is an event that should be added to the
> > > _Communication_ framework, not to the "TaskContext". The latter
> > > will allow to _react_ to
> > > the event, but it need not have anything extra to _support_ the
> > > event.
> > >
> > > I do not know communication frameworks in robotics that have the
> > > "low
> > > water-high water" pattern; but others, outside of robotics
> (hence,
> > > the
> > > _real ones_) do, such as, e.g., Comedi.
> > >
> > > Maybe (the new) SmartSoft of Christian Schlegel also has it...
> > >
> > > Herman
> > >
> > > I have trouble to understand what you say above. Let me explain a bit
> > > more about my situation and my idea, then maybe you can guide me a bit
> > > further. I use orocos-rtt straightforwardly, no remote things, so no
> > > framework other than orocos is involved.
> >
> > Which is not the desired situation :-) Orocos _needs_ good communication
> > middleware project, as soon as you leave the single process context.
> >
> > > I use "addEventPort" to wake up my component when new data
> > > arrives (coming from another local component). I have seen that the
> > > actual work involved in this method is implemented in
> > > InputPortInterface.
> > > When using a buffer policy for a port, it makes (more) sense to
> generate
> > > a wake-up event when the buffer is full; in my view, a new-data-arrived
> > > event of a non-buffering port is equivalent to a buffer-full event of a
> > > buffered port, both mean that action is needed in order to not loose
> > > data.
> > > So my personal preference would be to not change the API, but to change
> > > the
> > > effect of the method addEventPort:
> > This is a configuration of the policy. Which is the right thing to do,
> but
> > it should be done "in the right way". And RTT is not ready enough to
> fully
> > support "the right way", because the 5Cs are still a bit too coupled...
>
> I agree here as well. We already have connection policies, so we can extend
> that struct to cover this semantic when buffers are used. We don't have a
> system in place that warns the *producer* for buffer empty, since an output
> port has no way to know how a buffer is, and it's meaningless, since
> multiple
> clients might be listening and each will have a different buffer status.
>
Ok
>
> >
> > > for a non-buffering port, it remains as is,
> > > but for a buffering port, the event gets generated when the buffer is
> > > full. The current implementation in my application is, as you can
> > > imagine, to wake up each time that data arrives, and to only start
> > > taking real action when the buffer is full. This means that this
> > > component is getting active much more often than needed and I would
> like
> > > to avoid that.
> > >
> > > So how can I take your advise without resorting to another
> communication
> > > framework?
> >
> > Implement the things yourself... But I really see no reason why you
> should
> > not use "another communication framework"... Of course, it should be
> good,
> > but trying to put everything and the kitchen sink into one single project
> > is "Bad Practice"! It leads to too much decoupling early on in the
> > design...
> >
> > > And what do you / others think of the proposed change to addEventPort?
>
> I think extending the ConnPolicy structure is the simplest thing to do. You
> could name the field 'trigger_only_full' (would default to false) or
> 'trigger_each_sample' (would default to true). It would only matter to
> buffers.
>
Ok, this part is easy. But then the event firing should be changed
accordingly. I do not understand how this works and where this is
implemented. Could you give me some hints?
>
> BTW, people are asking what the difference is between a buffer of size 1
> and a
> 'data' connection. The new API made these indistinguishable.
>
And that is good, in my view. So a buffer holds last written values, right?
Cheers, Theo.
>
> > >
> > > Theo
> >
> > Herman
>
> Peter
>
portBufferFullEvent?
On 04/13/2011 11:16 AM, Vries, Theo J.A. de [imotec bv] wrote:
>
>
> 2011/4/13 Peter Soetens <peter [..] ...
> <mailto:peter [..] ...>>
>
> Theo,
>
> I didn't receive the original post Herman is replying to. So I'll
> reply to his
> one:
>
> Sorry, I had hit the reply instead of reply to all button I guess.
>
>
> On Wednesday 13 April 2011 09:20:56 Herman Bruyninckx wrote:
> > On Tue, 12 Apr 2011, Vries, Theo J.A. de [imotec bv] wrote:
> > > 2011/4/11 Herman Bruyninckx <Herman [dot] Bruyninckx [..] ...
> <mailto:Herman [dot] Bruyninckx [..] ...>>
> > >
> > > On Mon, 11 Apr 2011, t [dot] j [dot] a [dot] devries [..] ...
> <mailto:t [dot] j [dot] a [dot] devries [..] ...> wrote:
> > > I am building an app where I want a TaskContext to take
> > > action when a
> > > buffer is full with data that comes in irregularly
> via a
> > > port. I now have
> > > an event port and an updateHook in which data is copied
> > > to the buffer,
> > > and once the buffer is full, further action is taken.
> > > A more elegant solution in my view would be to let a
> > > port with a buffer
> > > policy generate an event and wake up the TC as soon as
> > > it becomes full.
> > > Would this in your view be an interesting feature to
> > > have and hence
> > > should be added to the API?
> > >
> > > This is the "low water-high water" pattern for buffered
> communication. It
> > > is good to have, but I see no reason to add something to the
> current API:
> > > developers have already the freedom to give their events good
> _names_,
> > > and that is what is important.
> > >
> > > I googled for this "low water-high water" pattern, but could
> not find
> > > anything. Do you maybe have a pointer?
> >
> > I did the same thing as you, and such a search comes out empty...
> The words
> > are just too common :-( If I remember correctly, I got this term
> from the
> > time I was looking intensively to frameworks like Comedi, for
> hardware
> > device interfacing. I think the PCI protocol uses it...
> >
> > > Anyway, this is an event that should be added to the
> > > _Communication_ framework, not to the "TaskContext". The
> latter
> > > will allow to _react_ to
> > > the event, but it need not have anything extra to
> _support_ the
> > > event.
> > >
> > > I do not know communication frameworks in robotics that
> have the
> > > "low
> > > water-high water" pattern; but others, outside of
> robotics (hence,
> > > the
> > > _real ones_) do, such as, e.g., Comedi.
> > >
> > > Maybe (the new) SmartSoft of Christian Schlegel also has
> it...
> > >
> > > Herman
> > >
> > > I have trouble to understand what you say above. Let me explain
> a bit
> > > more about my situation and my idea, then maybe you can guide
> me a bit
> > > further. I use orocos-rtt straightforwardly, no remote things,
> so no
> > > framework other than orocos is involved.
> >
> > Which is not the desired situation :-) Orocos _needs_ good
> communication
> > middleware project, as soon as you leave the single process context.
> >
> > > I use "addEventPort" to wake up my component when new data
> > > arrives (coming from another local component). I have seen that the
> > > actual work involved in this method is implemented in
> > > InputPortInterface.
> > > When using a buffer policy for a port, it makes (more) sense to
> generate
> > > a wake-up event when the buffer is full; in my view, a
> new-data-arrived
> > > event of a non-buffering port is equivalent to a buffer-full
> event of a
> > > buffered port, both mean that action is needed in order to not
> loose
> > > data.
> > > So my personal preference would be to not change the API, but
> to change
> > > the
> > > effect of the method addEventPort:
> > This is a configuration of the policy. Which is the right thing
> to do, but
> > it should be done "in the right way". And RTT is not ready enough
> to fully
> > support "the right way", because the 5Cs are still a bit too
> coupled...
>
> I agree here as well. We already have connection policies, so we can
> extend
> that struct to cover this semantic when buffers are used. We don't
> have a
> system in place that warns the *producer* for buffer empty, since an
> output
> port has no way to know how a buffer is, and it's meaningless, since
> multiple
> clients might be listening and each will have a different buffer status.
>
> Ok
>
>
> >
> > > for a non-buffering port, it remains as is,
> > > but for a buffering port, the event gets generated when the
> buffer is
> > > full. The current implementation in my application is, as you can
> > > imagine, to wake up each time that data arrives, and to only start
> > > taking real action when the buffer is full. This means that this
> > > component is getting active much more often than needed and I
> would like
> > > to avoid that.
> > >
> > > So how can I take your advise without resorting to another
> communication
> > > framework?
> >
> > Implement the things yourself... But I really see no reason why
> you should
> > not use "another communication framework"... Of course, it should
> be good,
> > but trying to put everything and the kitchen sink into one single
> project
> > is "Bad Practice"! It leads to too much decoupling early on in the
> > design...
> >
> > > And what do you / others think of the proposed change to
> addEventPort?
>
> I think extending the ConnPolicy structure is the simplest thing to
> do. You
> could name the field 'trigger_only_full' (would default to false) or
> 'trigger_each_sample' (would default to true). It would only matter
> to buffers.
>
> Ok, this part is easy. But then the event firing should be changed
> accordingly. I do not understand how this works and where this is
> implemented. Could you give me some hints?
>
>
> BTW, people are asking what the difference is between a buffer of
> size 1 and a
> 'data' connection. The new API made these indistinguishable.
>
> And that is good, in my view. So a buffer holds last written values, right?
There is actually a difference, and we could (1) remove that difference
and (2) get rid of the data connections in the process.
The difference is that buffer connections are "throw new samples when
full" while data connections would be equivalent to buffers of size 1
with a "throw old samples when full" policy.
Given the current buffer implementation (based on chained lists),
implementing both policies inside the buffer implementation would be
trivial *and* would allow throwing away the distinction altogether.
--
Sylvain Joyeux (Dr.Ing.)
Space & Security Robotics
!!! Achtung, neue Telefonnummer!!!
Standort Bremen:
DFKI GmbH
Robotics Innovation Center
Robert-Hooke-Straße 5
28359 Bremen, Germany
Phone: +49 (0)421 178-454136
Fax: +49 (0)421 218-454150
E-Mail: robotik [..] ...
Weitere Informationen: http://www.dfki.de/robotik
-----------------------------------------------------------------------
Deutsches Forschungszentrum fuer Kuenstliche Intelligenz GmbH
Firmensitz: Trippstadter Straße 122, D-67663 Kaiserslautern
Geschaeftsfuehrung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster
(Vorsitzender) Dr. Walter Olthoff
Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes
Amtsgericht Kaiserslautern, HRB 2313
Sitz der Gesellschaft: Kaiserslautern (HRB 2313)
USt-Id.Nr.: DE 148646973
Steuernummer: 19/673/0060/3
-----------------------------------------------------------------------
--
Orocos-Dev mailing list
Orocos-Dev [..] ...
http://lists.mech.kuleuven.be/mailman/listinfo/orocos-dev
portBufferFullEvent?
On Wednesday 13 April 2011 11:21:10 Sylvain Joyeux wrote:
> On 04/13/2011 11:16 AM, Vries, Theo J.A. de [imotec bv] wrote:
> > 2011/4/13 Peter Soetens <peter [..] ...
> > <mailto:peter [..] ...>>
> >
> > Theo,
> >
> > I didn't receive the original post Herman is replying to. So I'll
> > reply to his
> >
> > one:
> > Sorry, I had hit the reply instead of reply to all button I guess.
> >
> > On Wednesday 13 April 2011 09:20:56 Herman Bruyninckx wrote:
> > > On Tue, 12 Apr 2011, Vries, Theo J.A. de [imotec bv] wrote:
> > > > 2011/4/11 Herman Bruyninckx <Herman [dot] Bruyninckx [..] ...
> >
> > <mailto:Herman [dot] Bruyninckx [..] ...>>
> >
> > > > On Mon, 11 Apr 2011, t [dot] j [dot] a [dot] devries [..] ...
> >
> > <mailto:t [dot] j [dot] a [dot] devries [..] ...> wrote:
> > > > I am building an app where I want a TaskContext to
> > > > take action when a
> > > > buffer is full with data that comes in irregularly
> >
> > via a
> >
> > > > port. I now have
> > > > an event port and an updateHook in which data is
> > > > copied to the buffer,
> > > > and once the buffer is full, further action is
> > > > taken. A more elegant solution in my view would be
> > > > to let a port with a buffer
> > > > policy generate an event and wake up the TC as soon
> > > > as it becomes full.
> > > > Would this in your view be an interesting feature to
> > > > have and hence
> > > > should be added to the API?
> > > >
> > > > This is the "low water-high water" pattern for buffered
> >
> > communication. It
> >
> > > > is good to have, but I see no reason to add something to the
> >
> > current API:
> > > > developers have already the freedom to give their events good
> >
> > _names_,
> >
> > > > and that is what is important.
> > > >
> > > > I googled for this "low water-high water" pattern, but could
> >
> > not find
> >
> > > > anything. Do you maybe have a pointer?
> > >
> > > I did the same thing as you, and such a search comes out empty...
> >
> > The words
> >
> > > are just too common :-( If I remember correctly, I got this term
> >
> > from the
> >
> > > time I was looking intensively to frameworks like Comedi, for
> >
> > hardware
> >
> > > device interfacing. I think the PCI protocol uses it...
> > >
> > > > Anyway, this is an event that should be added to the
> > > > _Communication_ framework, not to the "TaskContext". The
> >
> > latter
> >
> > > > will allow to _react_ to
> > > > the event, but it need not have anything extra to
> >
> > _support_ the
> >
> > > > event.
> > > >
> > > > I do not know communication frameworks in robotics that
> >
> > have the
> >
> > > > "low
> > > >
> > > > water-high water" pattern; but others, outside of
> >
> > robotics (hence,
> >
> > > > the
> > > > _real ones_) do, such as, e.g., Comedi.
> > > >
> > > > Maybe (the new) SmartSoft of Christian Schlegel also has
> >
> > it...
> >
> > > > Herman
> > > >
> > > > I have trouble to understand what you say above. Let me explain
> >
> > a bit
> >
> > > > more about my situation and my idea, then maybe you can guide
> >
> > me a bit
> >
> > > > further. I use orocos-rtt straightforwardly, no remote things,
> >
> > so no
> >
> > > > framework other than orocos is involved.
> > >
> > > Which is not the desired situation :-) Orocos _needs_ good
> >
> > communication
> >
> > > middleware project, as soon as you leave the single process
> > > context.
> > >
> > > > I use "addEventPort" to wake up my component when new data
> > > > arrives (coming from another local component). I have seen that
> > > > the actual work involved in this method is implemented in
> > > > InputPortInterface.
> > > > When using a buffer policy for a port, it makes (more) sense to
> >
> > generate
> >
> > > > a wake-up event when the buffer is full; in my view, a
> >
> > new-data-arrived
> >
> > > > event of a non-buffering port is equivalent to a buffer-full
> >
> > event of a
> >
> > > > buffered port, both mean that action is needed in order to not
> >
> > loose
> >
> > > > data.
> > > > So my personal preference would be to not change the API, but
> >
> > to change
> >
> > > > the
> > >
> > > > effect of the method addEventPort:
> > > This is a configuration of the policy. Which is the right thing
> >
> > to do, but
> >
> > > it should be done "in the right way". And RTT is not ready enough
> >
> > to fully
> >
> > > support "the right way", because the 5Cs are still a bit too
> >
> > coupled...
> >
> > I agree here as well. We already have connection policies, so we can
> > extend
> > that struct to cover this semantic when buffers are used. We don't
> > have a
> > system in place that warns the *producer* for buffer empty, since an
> > output
> > port has no way to know how a buffer is, and it's meaningless, since
> > multiple
> > clients might be listening and each will have a different buffer
> > status.
> >
> > Ok
> >
> > > > for a non-buffering port, it remains as is,
> > > > but for a buffering port, the event gets generated when the
> >
> > buffer is
> >
> > > > full. The current implementation in my application is, as you
> > > > can imagine, to wake up each time that data arrives, and to
> > > > only start taking real action when the buffer is full. This
> > > > means that this component is getting active much more often
> > > > than needed and I
> >
> > would like
> >
> > > > to avoid that.
> > > >
> > > > So how can I take your advise without resorting to another
> >
> > communication
> >
> > > > framework?
> > >
> > > Implement the things yourself... But I really see no reason why
> >
> > you should
> >
> > > not use "another communication framework"... Of course, it should
> >
> > be good,
> >
> > > but trying to put everything and the kitchen sink into one single
> >
> > project
> >
> > > is "Bad Practice"! It leads to too much decoupling early on in the
> > > design...
> > >
> > > > And what do you / others think of the proposed change to
> >
> > addEventPort?
> >
> > I think extending the ConnPolicy structure is the simplest thing to
> > do. You
> > could name the field 'trigger_only_full' (would default to false) or
> > 'trigger_each_sample' (would default to true). It would only matter
> > to buffers.
> >
> > Ok, this part is easy. But then the event firing should be changed
> > accordingly. I do not understand how this works and where this is
> > implemented. Could you give me some hints?
> >
> > BTW, people are asking what the difference is between a buffer of
> > size 1 and a
> > 'data' connection. The new API made these indistinguishable.
> >
> > And that is good, in my view. So a buffer holds last written values,
> > right?
>
> There is actually a difference, and we could (1) remove that difference
> and (2) get rid of the data connections in the process.
>
> The difference is that buffer connections are "throw new samples when
> full" while data connections would be equivalent to buffers of size 1
> with a "throw old samples when full" policy.
>
> Given the current buffer implementation (based on chained lists),
> implementing both policies inside the buffer implementation would be
> trivial *and* would allow throwing away the distinction altogether.
I agree here. I even thought you already had implemented that 'throw old
samples away' on your branch... Would you allow to keep the old behavior using
a conn policy field or should we convert it ie, break the behavior ?
IIRC the implementation could still keep data vs buffer since data
(bufsize==1) is an optimized implementation wrt buffers, but I have no figures
to prove this is actually performing better....
Peter
portBufferFullEvent?
On 04/13/2011 11:50 AM, Peter Soetens wrote:
> I agree here. I even thought you already had implemented that 'throw old
> samples away' on your branch... Would you allow to keep the old behavior using
> a conn policy field or should we convert it ie, break the behavior ?
I would add a new policy field, and no, we did not implement it on our
branch.
Right now, I have zero time implementing new things on RTT, just fixing
the issues we have.
Sylvain
portBufferFullEvent?
On Mon, 11 Apr 2011, t [dot] j [dot] a [dot] devries [..] ... wrote:
> I am building an app where I want a TaskContext to take action when a
> buffer is full with data that comes in irregularly via a port. I now have
> an event port and an updateHook in which data is copied to the buffer,
> and once the buffer is full, further action is taken.
> A more elegant solution in my view would be to let a port with a buffer
> policy generate an event and wake up the TC as soon as it becomes full.
> Would this in your view be an interesting feature to have and hence
> should be added to the API?
This is the "low water-high water" pattern for buffered communication. It
is good to have, but I see no reason to add something to the current API:
developers have already the freedom to give their events good _names_, and
that is what is important.
Anyway, this is an event that should be added to the _Communication_
framework, not to the "TaskContext". The latter will allow to _react_ to
the event, but it need not have anything extra to _support_ the event.
I do not know communication frameworks in robotics that have the "low
water-high water" pattern; but others, outside of robotics (hence, the
_real ones_) do, such as, e.g., Comedi.
Maybe (the new) SmartSoft of Christian Schlegel also has it...
Herman
portBufferFullEvent?
On 11/04/11 22:34, Herman Bruyninckx wrote:
> On Mon, 11 Apr 2011, t [dot] j [dot] a [dot] devries [..] ... wrote:
>
>> I am building an app where I want a TaskContext to take action when a
>> buffer is full with data that comes in irregularly via a port. I now have
>> an event port and an updateHook in which data is copied to the buffer,
>> and once the buffer is full, further action is taken.
>> A more elegant solution in my view would be to let a port with a buffer
>> policy generate an event and wake up the TC as soon as it becomes full.
>> Would this in your view be an interesting feature to have and hence
>> should be added to the API?
>
> This is the "low water-high water" pattern for buffered communication. It
> is good to have, but I see no reason to add something to the current API:
> developers have already the freedom to give their events good _names_, and
> that is what is important.
>
> Anyway, this is an event that should be added to the _Communication_
> framework, not to the "TaskContext". The latter will allow to _react_ to
> the event, but it need not have anything extra to _support_ the event.
>
> I do not know communication frameworks in robotics that have the "low
> water-high water" pattern; but others, outside of robotics (hence, the
> _real ones_) do, such as, e.g., Comedi.
>
> Maybe (the new) SmartSoft of Christian Schlegel also has it...
OpenRTM has it. It's part of the DataPort implementation.
Geoff
portBufferFullEvent?
On Tue, 12 Apr 2011, Geoffrey Biggs wrote:
> On 11/04/11 22:34, Herman Bruyninckx wrote:
>> On Mon, 11 Apr 2011, t [dot] j [dot] a [dot] devries [..] ... wrote:
>>
>>> I am building an app where I want a TaskContext to take action when a
>>> buffer is full with data that comes in irregularly via a port. I now have
>>> an event port and an updateHook in which data is copied to the buffer,
>>> and once the buffer is full, further action is taken.
>>> A more elegant solution in my view would be to let a port with a buffer
>>> policy generate an event and wake up the TC as soon as it becomes full.
>>> Would this in your view be an interesting feature to have and hence
>>> should be added to the API?
>>
>> This is the "low water-high water" pattern for buffered communication. It
>> is good to have, but I see no reason to add something to the current API:
>> developers have already the freedom to give their events good _names_, and
>> that is what is important.
>>
>> Anyway, this is an event that should be added to the _Communication_
>> framework, not to the "TaskContext". The latter will allow to _react_ to
>> the event, but it need not have anything extra to _support_ the event.
>>
>> I do not know communication frameworks in robotics that have the "low
>> water-high water" pattern; but others, outside of robotics (hence, the
>> _real ones_) do, such as, e.g., Comedi.
>>
>> Maybe (the new) SmartSoft of Christian Schlegel also has it...
>
> OpenRTM has it. It's part of the DataPort implementation.
Thanks for this information! I do think it does not belong to a _port_ but
to a _communication_, since it is a form of peer-to-peer coordination, and
not a "property" of a single port...
> Geoff
Herman
portBufferFullEvent?
On 13/04/11 19:37, Herman Bruyninckx wrote:
> On Tue, 12 Apr 2011, Geoffrey Biggs wrote:
>> OpenRTM has it. It's part of the DataPort implementation.
>
> Thanks for this information! I do think it does not belong to a _port_ but
> to a _communication_, since it is a form of peer-to-peer coordination, and
> not a "property" of a single port...
Do you mean the connection between two ports? In OpenRTM, it's a
per-connection event.
Geoff
portBufferFullEvent?
On Thu, 14 Apr 2011, Geoffrey Biggs wrote:
> On 13/04/11 19:37, Herman Bruyninckx wrote:
>> On Tue, 12 Apr 2011, Geoffrey Biggs wrote:
>>> OpenRTM has it. It's part of the DataPort implementation.
>>
>> Thanks for this information! I do think it does not belong to a _port_ but
>> to a _communication_, since it is a form of peer-to-peer coordination, and
>> not a "property" of a single port...
>
> Do you mean the connection between two ports?
Yes.
> In OpenRTM, it's a per-connection event.
Sounds good. And to which component does the code belong that takes the
decision to fire high/low water events? I mean: is a "connection" a
full-fledged component in OpenRTM?
> Geoff
Herman
portBufferFullEvent?
On 14/04/11 13:36, Herman Bruyninckx wrote:
> On Thu, 14 Apr 2011, Geoffrey Biggs wrote:
>
>> On 13/04/11 19:37, Herman Bruyninckx wrote:
>>> On Tue, 12 Apr 2011, Geoffrey Biggs wrote:
>>>> OpenRTM has it. It's part of the DataPort implementation.
>>>
>>> Thanks for this information! I do think it does not belong to a
>>> _port_ but
>>> to a _communication_, since it is a form of peer-to-peer
>>> coordination, and
>>> not a "property" of a single port...
>>
>> Do you mean the connection between two ports?
> Yes.
>
>> In OpenRTM, it's a per-connection event.
> Sounds good. And to which component does the code belong that takes the
> decision to fire high/low water events? I mean: is a "connection" a
> full-fledged component in OpenRTM?
I'm not sure if you could call it a "component" in the sense of the word
as used in OpenRTM. It's not a component on the same level as an RTC.
Implementation-wise, it's a class. The port object holds one connection
object instance for each connection it has. An instance of a port object
is held by the RTC.
To the developer, a connection is a separate entity from an RTC or a
port. It has its own individual properties. Naturally, some of these
properties must match the ports involved for the connection to be valid.
The RTC specification is where this concept of a separate entity is
defined, as I recall.
Geoff
portBufferFullEvent?
On Thu, 14 Apr 2011, Geoffrey Biggs wrote:
> On 14/04/11 13:36, Herman Bruyninckx wrote:
>> On Thu, 14 Apr 2011, Geoffrey Biggs wrote:
>>
>>> On 13/04/11 19:37, Herman Bruyninckx wrote:
>>>> On Tue, 12 Apr 2011, Geoffrey Biggs wrote:
>>>>> OpenRTM has it. It's part of the DataPort implementation.
>>>>
>>>> Thanks for this information! I do think it does not belong to a
>>>> _port_ but
>>>> to a _communication_, since it is a form of peer-to-peer
>>>> coordination, and
>>>> not a "property" of a single port...
>>>
>>> Do you mean the connection between two ports?
>> Yes.
>>
>>> In OpenRTM, it's a per-connection event.
>> Sounds good. And to which component does the code belong that takes the
>> decision to fire high/low water events? I mean: is a "connection" a
>> full-fledged component in OpenRTM?
>
> I'm not sure if you could call it a "component" in the sense of the word
> as used in OpenRTM. It's not a component on the same level as an RTC.
> Implementation-wise, it's a class. The port object holds one connection
> object instance for each connection it has. An instance of a port object
> is held by the RTC.
Ok, I see. (Same approach as in Orocos/RTT. Currently.)
Then the problem is, I think, that these two "connection classes" have to
agree with each other, instead of having one component that decides how to
implement the buffering policy, so it is a distributed "decision making".
This is ok (and even desirable in really distributed use cases!), but one
should then not expect that some guarantees can be given, such as "no
message loss".
> To the developer, a connection is a separate entity from an RTC or a
> port. It has its own individual properties. Naturally, some of these
> properties must match the ports involved for the connection to be valid.
I think the interaction between the connection class and the RTC should be
about more than "properties": it should also involve "coordination
behaviour" (FSM). Or not?
> The RTC specification is where this concept of a separate entity is
> defined, as I recall.
>
> Geoff
Herman
portBufferFullEvent?
On 14/04/11 17:54, Herman Bruyninckx wrote:
> On Thu, 14 Apr 2011, Geoffrey Biggs wrote:
>
>> On 14/04/11 13:36, Herman Bruyninckx wrote:
>>> On Thu, 14 Apr 2011, Geoffrey Biggs wrote:
>>>
>>>> On 13/04/11 19:37, Herman Bruyninckx wrote:
>>>>> On Tue, 12 Apr 2011, Geoffrey Biggs wrote:
>>>>>> OpenRTM has it. It's part of the DataPort implementation.
>>>>>
>>>>> Thanks for this information! I do think it does not belong to a
>>>>> _port_ but
>>>>> to a _communication_, since it is a form of peer-to-peer
>>>>> coordination, and
>>>>> not a "property" of a single port...
>>>>
>>>> Do you mean the connection between two ports?
>>> Yes.
>>>
>>>> In OpenRTM, it's a per-connection event.
>>> Sounds good. And to which component does the code belong that takes the
>>> decision to fire high/low water events? I mean: is a "connection" a
>>> full-fledged component in OpenRTM?
>>
>> I'm not sure if you could call it a "component" in the sense of the word
>> as used in OpenRTM. It's not a component on the same level as an RTC.
>> Implementation-wise, it's a class. The port object holds one connection
>> object instance for each connection it has. An instance of a port object
>> is held by the RTC.
>
> Ok, I see. (Same approach as in Orocos/RTT. Currently.) Then the problem
> is, I think, that these two "connection classes" have to
> agree with each other, instead of having one component that decides how to
> implement the buffering policy, so it is a distributed "decision making".
>
> This is ok (and even desirable in really distributed use cases!), but one
> should then not expect that some guarantees can be given, such as "no
> message loss".
OpenRTM is particularly focused on the distributed use cases, which is
why the implementation came out this way. It's a natural consequence, I
think. The connection object instances are created through a single
interface call so it's not too hard to ensure they agree in our use
cases. I wouldn't like to try it in OpenRTM with hard-coded connections,
though. There are no checks in place once the connection is established...
>> To the developer, a connection is a separate entity from an RTC or a
>> port. It has its own individual properties. Naturally, some of these
>> properties must match the ports involved for the connection to be valid.
>
> I think the interaction between the connection class and the RTC should be
> about more than "properties": it should also involve "coordination
> behaviour" (FSM). Or not?
The current version of the spec doesn't make connections that complex.
They're conceptually a separate entity, but that entity is essentially
just a collection of properties and other relevant information to make
the connection. Part of this is due to RTC taking a back seat with
respect to how components communicate. The port implementation is not
defined beyond managing connections, so it is difficult to specify how
the connection itself should behave. This is as it should be at this
level, I think - otherwise we'd have a huge spec to cover as many
possible use cases as we could! :)
At the implementation level in OpenRTM, I'm not sure how much state
management connections do themselves. I suspect it's very little. I do
agree that much of the inter-component coordination should be managed by
a single entity for each connection. This is an ideal worth striving for
to achieve good decoupling and make model-driven development easier.
Geoff
portBufferFullEvent?
On Fri, 15 Apr 2011, Geoffrey Biggs wrote:
> On 14/04/11 17:54, Herman Bruyninckx wrote:
>> On Thu, 14 Apr 2011, Geoffrey Biggs wrote:
>>
>>> On 14/04/11 13:36, Herman Bruyninckx wrote:
>>>> On Thu, 14 Apr 2011, Geoffrey Biggs wrote:
>>>>
>>>>> On 13/04/11 19:37, Herman Bruyninckx wrote:
>>>>>> On Tue, 12 Apr 2011, Geoffrey Biggs wrote:
>>>>>>> OpenRTM has it. It's part of the DataPort implementation.
>>>>>>
>>>>>> Thanks for this information! I do think it does not belong to a
>>>>>> _port_ but
>>>>>> to a _communication_, since it is a form of peer-to-peer
>>>>>> coordination, and
>>>>>> not a "property" of a single port...
>>>>>
>>>>> Do you mean the connection between two ports?
>>>> Yes.
>>>>
>>>>> In OpenRTM, it's a per-connection event.
>>>> Sounds good. And to which component does the code belong that takes the
>>>> decision to fire high/low water events? I mean: is a "connection" a
>>>> full-fledged component in OpenRTM?
>>>
>>> I'm not sure if you could call it a "component" in the sense of the word
>>> as used in OpenRTM. It's not a component on the same level as an RTC.
>>> Implementation-wise, it's a class. The port object holds one connection
>>> object instance for each connection it has. An instance of a port object
>>> is held by the RTC.
>>
>> Ok, I see. (Same approach as in Orocos/RTT. Currently.) Then the problem
>> is, I think, that these two "connection classes" have to
>> agree with each other, instead of having one component that decides how to
>> implement the buffering policy, so it is a distributed "decision making".
>>
>> This is ok (and even desirable in really distributed use cases!), but one
>> should then not expect that some guarantees can be given, such as "no
>> message loss".
>
> OpenRTM is particularly focused on the distributed use cases, which is
> why the implementation came out this way. It's a natural consequence, I
> think. The connection object instances are created through a single
> interface call so it's not too hard to ensure they agree in our use
> cases. I wouldn't like to try it in OpenRTM with hard-coded connections,
> though. There are no checks in place once the connection is established...
>
>>> To the developer, a connection is a separate entity from an RTC or a
>>> port. It has its own individual properties. Naturally, some of these
>>> properties must match the ports involved for the connection to be valid.
>>
>> I think the interaction between the connection class and the RTC should be
>> about more than "properties": it should also involve "coordination
>> behaviour" (FSM). Or not?
>
> The current version of the spec doesn't make connections that complex.
> They're conceptually a separate entity, but that entity is essentially
> just a collection of properties and other relevant information to make
> the connection. Part of this is due to RTC taking a back seat with
> respect to how components communicate. The port implementation is not
> defined beyond managing connections, so it is difficult to specify how
> the connection itself should behave. This is as it should be at this
> level, I think
I fully agree,
> - otherwise we'd have a huge spec to cover as many
> possible use cases as we could! :)
>
> At the implementation level in OpenRTM, I'm not sure how much state
> management connections do themselves. I suspect it's very little. I do
> agree that much of the inter-component coordination should be managed by
> a single entity for each connection. This is an ideal worth striving for
> to achieve good decoupling and make model-driven development easier.
>
> Geoff
Herman