Hi all,

On 04/04/2012 00:45, Peter Soetens wrote:
> 1. updateHook() is called *too many times* (even according to
> specifications), especially when 'OwnThread' operations are implemented
> OR called(!).
> -> This was due to internal bookkeeping of the ExecutionEngine to
> dispatch the asynchronous requests. Users found it confusing that
> updateHook was called during the use of operations. This also kills
> performance of Lua state machines
> sitting in updateHook(), since they are evaluated too many times.

I'm very glad to see this proposition.

We are heavily relying both on operations for asynchronous
communications and the updateHook() calls to manage the simulation
timesteps, so these problems are well known in our team.

I must confess we did not have time back then to submit these issues you
mentioned to orocos-dev@ (schedule constraints), so we used a (kind of)
workaround: an updateHook() "manager", to "cluch" and "decluch" the
updateHook() calls according to the situations.

So, count a +1 for all our team members (make it a +15).

On Apr 3, 2012, at 18:45 , Peter Soetens wrote:

> To all,
>
> This title that late at night, I'm sure I got your attention!
>
> It's serious though. We have observed lately two major issues:
>
> 1. updateHook() is called *too many times* (even according to specifications), especially when 'OwnThread' operations are implemented OR called(!).
> -> This was due to internal bookkeeping of the ExecutionEngine to dispatch the asynchronous requests. Users found it confusing that updateHook was called during the use of operations. This also kills performance of Lua state machines
> sitting in updateHook(), since they are evaluated too many times.
>
> 2. Installing a callback for an event port caused calling that callback AND updateHook(), instead of calling only the callback.
> -> The callback does not replace the call to updateHook(). This is also related to users being surprised that an event port callback is only called periodically in periodic threads, while they thought it would be called immediately upon each event reception, and eventually updateHook periodically (but serialized with the callbacks).
>
> The idea then rose to have updateHook() to only be called after:
> 1) any call to TaskContext::trigger()
> -> for activities which allow this user control

RTT v1 does this also, and causes additional updateHook() as commands and events call this. RTT shouldn't do this IMHO. It should only call updateHook() in the two cases below i.e. events/commands (yes, the RTT v2 Operation equivalent) should not be trigger'ing extra updateHook's.

> 2) the component's activity triggered itself (periodic, file descriptor, ...)
> -> what users expect

+1

> 3) the component is being update()
> -> for slave activity behavior

+1

> I have some unfinished patches in my queue that try to prove this works, but maybe now is already a good time for feedback (just before I go on holidays :-). I think fixing 1) is actually fixing applications instead of breaking them. 2) causes more subtle changes... what do you think ?

Agreed. This is fixing applications by removing extraneous "features" within RTT ...
S

On Wed, Apr 4, 2012 at 12:45 AM, Peter Soetens <peter [..] ...>wrote:

> To all,
>
> This title that late at night, I'm sure I got your attention!
>
> It's serious though. We have observed lately two major issues:
>
> 1. updateHook() is called *too many times* (even according to
> specifications), especially when 'OwnThread' operations are implemented OR
> called(!).
> -> This was due to internal bookkeeping of the ExecutionEngine to dispatch
> the asynchronous requests. Users found it confusing that updateHook was
> called during the use of operations. This also kills performance of Lua
> state machines
> sitting in updateHook(), since they are evaluated too many times.
>
> 2. Installing a callback for an event port caused calling that callback
> AND updateHook(), instead of calling only the callback.
> -> The callback does not replace the call to updateHook(). This is also
> related to users being surprised that an event port callback is only called
> periodically in periodic threads, while they thought it would be called
> immediately upon each event reception, and eventually updateHook
> periodically (but serialized with the callbacks).
>
> The idea then rose to have updateHook() to only be called after:
> 1) any call to TaskContext::trigger()
> -> for activities which allow this user control
> 2) the component's activity triggered itself (periodic, file descriptor,
> ...)
> -> what users expect
> 3) the component is being update()
> -> for slave activity behavior
>
> I have some unfinished patches in my queue that try to prove this works,
> but maybe now is already a good time for feedback (just before I go on
> holidays :-). I think fixing 1) is actually fixing applications instead of
> breaking them. 2) causes more subtle changes... what do you think ?
>

+1

It could explain a lot of weirdness that I have with my applications.

Ruben

>
> Peter
>
>
> --
> Orocos-Dev mailing list
> Orocos-Dev [..] ...
> http://lists.mech.kuleuven.be/mailman/listinfo/orocos-dev
>
>

On 04/04/2012 12:45 AM, Peter Soetens wrote:
> The idea then rose to have updateHook() to only be called after:
> 1) any call to TaskContext::trigger()
> -> for activities which allow this user control
> 2) the component's activity triggered itself (periodic, file descriptor,
> ...)
> -> what users expect
> 3) the component is being update()
> -> for slave activity behavior

+1

I would also vote for *not* calling updateHook just after startHook.
This is the current behaviour as well, and does not make any sense to me
(since the updateHook was never triggered).

On 04/04/2012 12:45 AM, Peter Soetens wrote:
> To all,
>
> This title that late at night, I'm sure I got your attention!
>
> It's serious though. We have observed lately two major issues:
>
> 1. updateHook() is called *too many times* (even according to
> specifications), especially when 'OwnThread' operations are
> implemented OR called(!).
> -> This was due to internal bookkeeping of the ExecutionEngine to
> dispatch the asynchronous requests. Users found it confusing that
> updateHook was called during the use of operations. This also kills
> performance of Lua state machines
> sitting in updateHook(), since they are evaluated too many times.

This is indeed a nasty side-effect. If one requires the updateHook to
run after or during the operation, it should be triggered/called manually.

Was the reasoning behind this to have the latest data-flow information
available (assuming it is read during updateHook() )? Some people might
(unknowingly) depend on this behavior.

>
> 2. Installing a callback for an event port caused calling that
> callback AND updateHook(), instead of calling only the callback.
> -> The callback does not replace the call to updateHook(). This is
> also related to users being surprised that an event port callback is
> only called periodically in periodic threads, while they thought it
> would be called immediately upon each event reception, and eventually
> updateHook periodically (but serialized with the callbacks).

We don't use callbacks, so no potential problem here.

>
> The idea then rose to have updateHook() to only be called after:
> 1) any call to TaskContext::trigger()
> -> for activities which allow this user control
> 2) the component's activity triggered itself (periodic, file
> descriptor, ...)
> -> what users expect
> 3) the component is being update()
> -> for slave activity behavior

Does 1) or 2) include event ports? I assume it does.

>
> I have some unfinished patches in my queue that try to prove this
> works, but maybe now is already a good time for feedback (just before
> I go on holidays :-). I think fixing 1) is actually fixing
> applications instead of breaking them. 2) causes more subtle
> changes... what do you think ?

Go for it :)

Best regards,
Robert

>
> Peter
>

On Wed, Apr 4, 2012 at 12:45 AM, Peter Soetens <peter [..] ...>wrote:

> To all,
>
> This title that late at night, I'm sure I got your attention!
>
> It's serious though. We have observed lately two major issues:
>
> 1. updateHook() is called *too many times* (even according to
> specifications), especially when 'OwnThread' operations are implemented OR
> called(!).
> -> This was due to internal bookkeeping of the ExecutionEngine to dispatch
> the asynchronous requests. Users found it confusing that updateHook was
> called during the use of operations. This also kills performance of Lua
> state machines
> sitting in updateHook(), since they are evaluated too many times.
>
> 2. Installing a callback for an event port caused calling that callback
> AND updateHook(), instead of calling only the callback.
> -> The callback does not replace the call to updateHook(). This is also
> related to users being surprised that an event port callback is only called
> periodically in periodic threads, while they thought it would be called
> immediately upon each event reception, and eventually updateHook
> periodically (but serialized with the callbacks).
>
> The idea then rose to have updateHook() to only be called after:
> 1) any call to TaskContext::trigger()
> -> for activities which allow this user control
> 2) the component's activity triggered itself (periodic, file descriptor,
> ...)
> -> what users expect
> 3) the component is being update()
> -> for slave activity behavior
>
> I have some unfinished patches in my queue that try to prove this works,
> but maybe now is already a good time for feedback (just before I go on
> holidays :-). I think fixing 1) is actually fixing applications instead of
> breaking them. 2) causes more subtle changes... what do you think ?
>

I do like the proposed behavior, so +1. I was very surprised when I learned
--the hard way-- how much updateHook was being triggered, and had to end up
coding around this behavior to emulate what I expected.

I am somewhat concerned about code breakage, though. I don't know how many
people rely on the current behavior.

Adolfo.

> Peter
>
>
> --
> Orocos-Dev mailing list
> Orocos-Dev [..] ...
> http://lists.mech.kuleuven.be/mailman/listinfo/orocos-dev
>
>

On Wed, Apr 04, 2012 at 12:45:01AM +0200, Peter Soetens wrote:
> To all,
>
> This title that late at night, I'm sure I got your attention!
>
> It's serious though. We have observed lately two major issues:
>
> 1. updateHook() is called *too many times* (even according to specifications),
> especially when 'OwnThread' operations are implemented OR called(!).
> -> This was due to internal bookkeeping of the ExecutionEngine to dispatch the
> asynchronous requests. Users found it confusing that updateHook was called
> during the use of operations. This also kills performance of Lua state machines
> sitting in updateHook(), since they are evaluated too many times.
>
> 2. Installing a callback for an event port caused calling that callback AND
> updateHook(), instead of calling only the callback.
> -> The callback does not replace the call to updateHook(). This is also related
> to users being surprised that an event port callback is only called
> periodically in periodic threads, while they thought it would be called
> immediately upon each event reception, and eventually updateHook periodically
> (but serialized with the callbacks).
>
> The idea then rose to have updateHook() to only be called after:
> 1) any call to TaskContext::trigger()
> -> for activities which allow this user control
> 2) the component's activity triggered itself (periodic, file descriptor, ...)
> -> what users expect
> 3) the component is being update()
> -> for slave activity behavior
>
> I have some unfinished patches in my queue that try to prove this works, but
> maybe now is already a good time for feedback (just before I go on holidays
> :-). I think fixing 1) is actually fixing applications instead of breaking
> them. 2) causes more subtle changes... what do you think ?

+1

This seems the most "correct" behavior.

Markus

Hi Peter,

rather than changing the behavior, it sounds to me like a bug fixing ;)

+1 to the changes you proposed.

Davide

On Wed, Apr 4, 2012 at 12:45 AM, Peter Soetens <peter [..] ...>wrote:

> To all,
>
> This title that late at night, I'm sure I got your attention!
>
> It's serious though. We have observed lately two major issues:
>
> 1. updateHook() is called *too many times* (even according to
> specifications), especially when 'OwnThread' operations are implemented OR
> called(!).
> -> This was due to internal bookkeeping of the ExecutionEngine to dispatch
> the asynchronous requests. Users found it confusing that updateHook was
> called during the use of operations. This also kills performance of Lua
> state machines
> sitting in updateHook(), since they are evaluated too many times.
>
> 2. Installing a callback for an event port caused calling that callback
> AND updateHook(), instead of calling only the callback.
> -> The callback does not replace the call to updateHook(). This is also
> related to users being surprised that an event port callback is only called
> periodically in periodic threads, while they thought it would be called
> immediately upon each event reception, and eventually updateHook
> periodically (but serialized with the callbacks).
>
> The idea then rose to have updateHook() to only be called after:
> 1) any call to TaskContext::trigger()
> -> for activities which allow this user control
> 2) the component's activity triggered itself (periodic, file descriptor,
> ...)
> -> what users expect
> 3) the component is being update()
> -> for slave activity behavior
>
> I have some unfinished patches in my queue that try to prove this works,
> but maybe now is already a good time for feedback (just before I go on
> holidays :-). I think fixing 1) is actually fixing applications instead of
> breaking them. 2) causes more subtle changes... what do you think ?
>
> Peter
>
>
> --
> Orocos-Dev mailing list
> Orocos-Dev [..] ...
> http://lists.mech.kuleuven.be/mailman/listinfo/orocos-dev
>
>

2012/4/4 Peter Soetens <peter [..] ...>:
> To all,
>
> This title that late at night, I'm sure I got your attention!
>
> It's serious though. We have observed lately two major issues:
>
> 1. updateHook() is called *too many times* (even according to
> specifications), especially when 'OwnThread' operations are implemented OR
> called(!).

what do you mean by "implemented" ? During the time you do the "addOperation" ?
what do you mean by called ? Doest that mean that : "I an aperiodic
component it triggers the updateHook ?" or/and "even in a periodic
component it triggers the updateHook out of the cycle" ?

> -> This was due to internal bookkeeping of the ExecutionEngine to dispatch
> the asynchronous requests. Users found it confusing that updateHook was
> called during the use of operations. This also kills performance of Lua
> state machines
> sitting in updateHook(), since they are evaluated too many times.
>

x-( really ? Woaw. To illustrate your motivations, I'm very surprise
to discover those execution of the updateHook, and really really
really afraid about my app health ! It's for this kind of background
computations that I throwed ROS away of high timing features, I'm a
bit horified ;p

> 2. Installing a callback for an event port caused calling that callback AND
> updateHook(), instead of calling only the callback.
> -> The callback does not replace the call to updateHook(). This is also
> related to users being surprised that an event port callback is only called
> periodically in periodic threads, while they thought it would be called
> immediately upon each event reception, and eventually updateHook
> periodically (but serialized with the callbacks).
>
> The idea then rose to have updateHook() to only be called after:
> 1) any call to TaskContext::trigger()
> -> for activities which allow this user control
> 2) the component's activity triggered itself (periodic, file descriptor,
> ...)
> -> what users expect
> 3) the component is being update()
> -> for slave activity behavior
>
> I have some unfinished patches in my queue that try to prove this works, but
> maybe now is already a good time for feedback (just before I go on holidays
> :-). I think fixing 1) is actually fixing applications instead of breaking
> them. 2) causes more subtle changes... what do you think ?
>

To my point of view :
_ It's an emergency to correct 1. And I am very impatient to
_ 2 would be a good thing for me as I am not using those callbacks
because of that strange updateHook triggering. So it won't break
anything on my side.

> Peter
>
>
> --
> Orocos-Dev mailing list
> Orocos-Dev [..] ...
> http://lists.mech.kuleuven.be/mailman/listinfo/orocos-dev
>