[Bug 882] New: Port invalidated during write()

http://bugs.orocos.org/show_bug.cgi?id=882

Summary: Port invalidated during write()
Product: Toolchain
Version: unspecified
Platform: All
OS/Version: GNU/Linux
Status: NEW
Severity: normal
Priority: P3
Component: RTT
AssignedTo: orocos-dev [..] ...
ReportedBy: bert [dot] willaert [..] ...
CC: orocos-dev [..] ...
Estimated Hours: 0.0

I am having trouble when streaming an OrocosPort to ROS. Very often I get the
following message:

Deployer [S]> 7.884 [ ERROR ][Master] A channel of port Joint_pos_port has been
invalidated during write(), it will be removed

After this, the associated rostopic is still there but rostopic echo or rxplot
doesn't do anything anymore. I can 'restart' the streaming by doing the
following in the Deployer:

var ConnPolicy cp_ros cp_ros.transport = 3 cp_ros.name_id = "/theta_to_ros"
stream("Daq.Joint_pos_port", cp_ros)

but 3 - 300 seconds later, the same thing happens again... I installed the
orocos_toolchain_ros as Debian pkg beginning of June.

[Bug 882] Port invalidated during write()

http://bugs.orocos.org/show_bug.cgi?id=882

Peter Soetens <peter [..] ...> changed:

What |Removed |Added
----------------------------------------------------------------------------
Attachment #718 is|0 |1
obsolete| |

--- Comment #4 from Peter Soetens <peter [..] ...> 2011-07-26 16:18:08 CEST ---
Created attachment 719
--> http://bugs.orocos.org/attachment.cgi?id=719
rtt_ros_integration: fixup/rewrite publisher activity - 2

Fixed protected member issue.

(Thanks Steven :-)

[Bug 882] Port invalidated during write()

http://bugs.orocos.org/show_bug.cgi?id=882

--- Comment #3 from Peter Soetens <peter [..] ...> 2011-07-26 15:48:24 CEST ---
Created attachment 718
--> http://bugs.orocos.org/attachment.cgi?id=718
rtt_ros_integration: fixup/rewrite publisher activity

- A single channel can not starve other channels anymore
- Connections are never invalidated, message drops may occur
if no buffering is present in the channel
- Usage of weak_ptr in order to cleanup the publisher if the
last channel dies
- Document Instance() wrt thread-safety and advised usage pattern

[Bug 882] Port invalidated during write()

2011/7/26 Peter Soetens <peter [..] ...>:
>
> http://bugs.orocos.org/show_bug.cgi?id=882
>
> --- Comment #3 from Peter Soetens <peter [..] ...> 2011-07-26 15:48:24 CEST ---
> Created attachment 718
>  --> http://bugs.orocos.org/attachment.cgi?id=718
> rtt_ros_integration: fixup/rewrite publisher activity
>
>    - A single channel can not starve other channels anymore
>    - Connections are never invalidated, message drops may occur
>      if no buffering is present in the channel
>    - Usage of weak_ptr in order to cleanup the publisher if the
>      last channel dies
>    - Document Instance() wrt thread-safety and advised usage pattern

*Testing*
Looks like the patch breaks the RTT ROS typekit compilation
"...
error: ‘void ros_integration::RosPublishActivity::addPublisher(ros_integration::RosPublisher*)’
is private
..."

Steven

>
> --
> Configure bugmail: http://bugs.orocos.org/userprefs.cgi?tab=email
> ------- You are receiving this mail because: -------
> You are on the CC list for the bug.
> You are the assignee for the bug.
> --
> Orocos-Dev mailing list
> Orocos-Dev [..] ...
> http://lists.mech.kuleuven.be/mailman/listinfo/orocos-dev
>

[Bug 882] Port invalidated during write()

http://bugs.orocos.org/show_bug.cgi?id=882

--- Comment #2 from Peter Soetens <peter [..] ...> 2011-07-26 15:47:11 CEST ---
(In reply to comment #1)
> (In reply to comment #0)
> > I am having trouble when streaming an OrocosPort to ROS. Very often I get the
> > following message:
> >
> > Deployer [S]> 7.884 [ ERROR ][Master] A channel of port Joint_pos_port has been
> > invalidated during write(), it will be removed
> >
> > After this, the associated rostopic is still there but rostopic echo or rxplot
> > doesn't do anything anymore. I can 'restart' the streaming by doing the
> > following in the Deployer:
> >
> > var ConnPolicy cp_ros cp_ros.transport = 3 cp_ros.name_id = "/theta_to_ros"
> > stream("Daq.Joint_pos_port", cp_ros)
> >
> > but 3 - 300 seconds later, the same thing happens again... I installed the
> > orocos_toolchain_ros as Debian pkg beginning of June.
>
> The bug is in the rtt_ros_integration package in src/ros_publish_activity.hpp:
>
>

>   82       bool requestPublish(RosPublisher* chan){
>   83           //Logger::In in("RosPublishActivity");
>   84           //log(Debug)<<"Requesting publish"<<endlog();
>   85           bool retval = pending_queue.enqueue(chan);
>   86           return retval&&this->trigger();
>   87       }
> 

>
> It should just return true (trigger will always return true) and ignore retval.
> This means we drop packets if the ROS transport can not follow. Seems
> reasonable to me, and much better than blowing up the connection 'now and
> then'.

What seemed like a one-liner fix turned out to be a rewrite of the
RosPublisherActivity. The current fifo queue is plain broken. If we drop
requests, the new message will never be published if the drop was caused by a
full buffer because of another channel hammering it. This is unacceptable,
since *eventually* the last message of any given channel must come through,
only *intermediate* data may be dropped, but never the last one.

Fix is coming through, will push to master for testing.

[Bug 882] Port invalidated during write()

http://bugs.orocos.org/show_bug.cgi?id=882

Peter Soetens <peter [..] ...> changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |ASSIGNED
CC| |peter [..] ...

--- Comment #1 from Peter Soetens <peter [..] ...> 2011-07-25 23:46:37 CEST ---
(In reply to comment #0)
> I am having trouble when streaming an OrocosPort to ROS. Very often I get the
> following message:
>
> Deployer [S]> 7.884 [ ERROR ][Master] A channel of port Joint_pos_port has been
> invalidated during write(), it will be removed
>
> After this, the associated rostopic is still there but rostopic echo or rxplot
> doesn't do anything anymore. I can 'restart' the streaming by doing the
> following in the Deployer:
>
> var ConnPolicy cp_ros cp_ros.transport = 3 cp_ros.name_id = "/theta_to_ros"
> stream("Daq.Joint_pos_port", cp_ros)
>
> but 3 - 300 seconds later, the same thing happens again... I installed the
> orocos_toolchain_ros as Debian pkg beginning of June.

The bug is in the rtt_ros_integration package in src/ros_publish_activity.hpp:

  82       bool requestPublish(RosPublisher* chan){
  83           //Logger::In in("RosPublishActivity");
  84           //log(Debug)<<"Requesting publish"<<endlog();
  85           bool retval = pending_queue.enqueue(chan);
  86           return retval&&this->trigger();
  87       }

It should just return true (trigger will always return true) and ignore retval.
This means we drop packets if the ROS transport can not follow. Seems
reasonable to me, and much better than blowing up the connection 'now and
then'.

Peter

[Bug 882] Port invalidated during write()

2011/7/25 Peter Soetens <peter [..] ...>:
>
> http://bugs.orocos.org/show_bug.cgi?id=882
>
> Peter Soetens <peter [..] ...> changed:
>
>           What    |Removed                     |Added
> ----------------------------------------------------------------------------
>             Status|NEW                         |ASSIGNED
>                 CC|                            |peter [..] ...
>
> --- Comment #1 from Peter Soetens <peter [..] ...> 2011-07-25 23:46:37 CEST ---
> (In reply to comment #0)
>> I am having trouble when streaming an OrocosPort to ROS. Very often I get the
>> following message:
>>
>> Deployer [S]> 7.884 [ ERROR ][Master] A channel of port Joint_pos_port has been
>> invalidated during write(), it will be removed
>>
>> After this, the associated rostopic is still there but rostopic echo or rxplot
>> doesn't do anything anymore. I can 'restart' the streaming by doing the
>> following in the Deployer:
>>
>> var ConnPolicy cp_ros cp_ros.transport = 3 cp_ros.name_id = "/theta_to_ros"
>> stream("Daq.Joint_pos_port", cp_ros)
>>
>> but 3 - 300 seconds later, the same thing happens again... I installed the
>> orocos_toolchain_ros as Debian pkg beginning of June.
>
> The bug is in the rtt_ros_integration package in src/ros_publish_activity.hpp:
>
>

>  82       bool requestPublish(RosPublisher* chan){
>  83           //Logger::In in("RosPublishActivity");
>  84           //log(Debug)<<"Requesting publish"<<endlog();
>  85           bool retval = pending_queue.enqueue(chan);
>  86           return retval&&this->trigger();
>  87       }
> 

>
> It should just return true (trigger will always return true) and ignore retval.
> This means we drop packets if the ROS transport can not follow. Seems
> reasonable to me, and much better than blowing up the connection 'now and
> then'.

+1

Steven

>
> Peter
>
> --
> Configure bugmail: http://bugs.orocos.org/userprefs.cgi?tab=email
> ------- You are receiving this mail because: -------
> You are on the CC list for the bug.
> You are the assignee for the bug.
> --
> Orocos-Dev mailing list
> Orocos-Dev [..] ...
> http://lists.mech.kuleuven.be/mailman/listinfo/orocos-dev
>