Dissecting Command and Method: blocking/non-blocking vs synchronous/asynchronous

Table of Contents

Notes on naming

(Please feel free to edit/comment etc. This is a community document, not a personal document)

Notes on naming

The word service is used to name the offering of a C/C++ function for others to call. Today in Orocos Components offer services in the form of 'RTT::Method' or 'RTT::Command' objects. Both lead to the execution of a function, but in a different way. Also, despite the title, it is advised to refrain from using the terms synchronous/asynchronous, because they are relative terms and may cause confusion if the context is not clear.

An alternative naming is possible: the offering of a C/C++ function could be named 'operation' and the collection of a given set of operations in an interface could be called a 'service'. This definition would line up better with service oriented architectures like OSGi.

Purpose

This page collects the ideas around the new primitives that will replace/enhance Method and/or Command. Although Method is a clearly understood primitive by users, Command isn't because of its multi-threaded nature. It is too complex to setup/use and can lead to unsafe applications (segfaults) if used incorrectly. To get these primitives better, we re-look at what users want to do and how to map this to RTT primitives.

What users want to do

Users want to control which thread executes which function, and if they want to wait(block) on the result or not. This all in order to meet deadlines in real-time systems. In practice, this boils down to:

When calling services (ie functions) of other components, one may opt to wait until the service returns the result, or not and optionally collect the result later. This is often best decided at the caller side, because both cases will cause different client code for sending/receiving the results

When implementing services in a component, the component may decide that the caller's thread executes the function, or that it will execute the function in it's own thread. Clearly, this can only be decided at the receiver side, because both cases will cause a different implementation of the function to be executed. Especially with respect to thread-safety.

Dissecting the cases

When putting the above in a table, you get:

Calling a service (a function)
Wait? \ Thread?	Caller	Component
Yes	(Method)	(?)
No	X	(Command)

For reference, the current RTT 1.x primitives are shown. There are two remarkable spots: the X and the (?).

The X is a practically impossible situation. It would involve that the client thread does not wait, but its thread still executes the function. This could only be resolved if a 'third' thread executes the service on behalf of the caller. It is unclear at which priority this thread should execute, what it's lifetime and exclusivity is and so on.

The (?) marks a hole in the current RTT API. Users could only implement this behaviour by busy-waiting on the Command's done() function. However, that is disastrous in real-time systems, because of starvation or priority inversion issues that crop up with such techniques.

Another thing you should be aware of that in the current implementation, caller and component must agree on how the service is invoked. If the Component defines a Method, the caller must execute it in its own thread and wait for the result. There's no other way for the caller to deviate from this. In practice, this means that the component's interface dictates how the caller can use its services. This is consistent with how UML defines operations, but other frameworks, like ICE, allow any function part of the interface to be called blocking or non-blocking. Clearly, ICE has some kind of thread-pool behind the scenes that does the dispatching and collects the results on behalf of the caller.

Backwards compatibility - Or how it is now

Orocos users have written many components and the primary idea of RTT 2.0 is to solve issues these components still have due to defects in the current RTT 1.x design. Things that do work satisfactory should keep working without modification of the user's design.

Method

It is very likely that the RTT::Method primitive will remain to exist as it is today. Little problems have been reported and it is easy to understand. The only disadvantage is that it can not be called 'asynchronously'. For example: if a component defines a Method, but the caller does not have the resources to invoke it (due to a deadline), it needs to setup a separate thread to do the call on its behalf. This is error prone. Orocos users often solve this by defining a command and trying to get the result data back somehow (also error prone).

Command

Commands serve multiple purposes in today programming with Orocos.

First, they allow thread-safe execution of a piece of code in a component. Because the component thread executes the function, no locking or synchronization primitives are required.
Second, they allow a caller to dispatch work to another component, in case the caller does not have the time or resources to execute a function.
Third, they allow to track the status of the execution. The caller can poll to see if the function has been queued, executed, what it returned (a boolean) etc.
Fourth, they allow to track the status of the 'effect' of the command, past its execution. This is done by attaching a completion condition, which returns a bool and can indicate if the effect of the command has been completed or not. For example, if the command is to move to a position, the completion condition would return true if the position is reached, while the command function would have only programmed the interpolator to reach that position. Completion conditions are not that much used, and must be polled.

A simpler form of Command will be provided that does not contain the completion condition. It is too seldomly used.

It is to the proposals to show how to emulate the old behavior with the new primitives.

Proposals

Each proposal should try to solve these issues:

The ability to let caller and component choose which execution semantics they want when calling or offering a service (or motivate why a certain choice is limited):

The ability to wait for a service to be completed
The ability to invoke a service and not wait for the result
The ability to specify in the component implementation if a function is executed in the component's thread
The ability to specify in the component implementation if a function is executed in the caller's thread

And regarding easy use and backwards compatibility:

Show how old-time behavior can be emulated with the new proposal
Show which semantics changed
How these primitives will be used in the scripting languages and in C++

And finally:

Define proper names for each behavior.

Proposal 1: Method/Message

This is one of the earliest proposals. It proposes to keep Method as-is, remove Command and replace it with a new primitive: RTT::Message. The Message is a stripped Command. It has no completion condition and is send-and-forget. One can not track the status or retrieve arguments. It also uses a memory manager to allow to invoke the same Message object multiple times with different data.

Emulating a completion condition is done by defining the completion condition as a Method in the component interface and requiring that the sender of the Message checks that Method to evaluate progress. In scripting this becomes:

// Old:
  do comp.command("hello"); // waits (polls) here until complete returns true
 
// New: Makes explicit what above line does:
  do comp.message("hello"); // proceeds immediately
  while ( comp.message_complete("hello") == false ) // polling
     do nothing;

In C++, the equivalent is slightly different:

// Old:
  if ( command("hello") ) {
     //... user specific logic that checks command.done() 
  }
 
// New:
  if ( message("hello") ) { // send and forget, returns immediately
     // user specifc logic that checks message_complete("hello")
  }

Users have indicated that they also wanted to be able to specify in C++:

  message.wait("hello"); // send and block until executed.

It is not clear yet how the wait case can be implemented efficiently.

The user visible object names are:

RTT::Method to add a 'client thread' C/C++ function to the component interface or call one.
RTT::Message to add a 'component thread' C/C++ function to the component interface or call one.

This proposal solves:

A simpler replacement for Command
Acceptable emulation capacities of old user code
The invocation of multiple times the same message object in a row.

This proposal omits:

The choice of caller/component to choose independently
Solving case 'X' (see above)
How message.wait() can be implemented

Other notes:

It has been mentioned that 'Message' is not a good and too confusing name.

Proposal 2: Method/Service

This proposal focuses on separating the definition of a Service (component side) from the calling of a Method (caller side).

The idea is that components only define services, and assign properties to these services. The main properties to toggle are 'executed in my thread or callers thread, or even another thread'. But other properties could be added too. For example: a 'serialized' property which causes the locking of a (recursive!) mutex during the execution of the service. The user of the service can not and does not need to know how these properties are set. He only sees a list of services in the interface.

It is the caller that chooses how to invoke a given service: waiting for the result ('call') or not ('send'). If he doesn't want to wait, he has the option to collect the results later ('collect'). The default is blocking ('call'). Note that this waiting or not is completely independent of how the service was defined by the component, the framework will choose a different 'execution' implementation depending on the combination of the properties of service and caller.

This means that this proposal allows to have all four quadrants of the table above. This proposal does not detail yet how to implement case (X) though, which requires a 3rd thread to do the actual execution of the service (neither component nor caller wish to do execute the C function).

This would result in the following scripting code on caller side:

//Old:
  do comp.the_method("hello");
 
//New:
  do comp.the_service.call("hello"); // equivalent to the_method.
 
//Old:
  do comp.the_command("hello");
 
//New:
  do comp.the_service.send("hello"); // equivalent to the_command, but without completion condition.

This example shows two use cases for the same 'the_service' functionality. The first case emulates an RTT 1.x method. It is called and the caller waits until the function has been executed. You can not see here which thread effectively executes the call. Maybe it's 'comp's thread, in which case the caller's thread is blocking until it the function is executed. Maybe it's the caller's thread, in which case it is effectively executing the function. The caller doesn't care actually. The only thing that has effect is that it takes a certain amount of time to complete the call, *and* that if the call returns, the function has been effectively executed.

The second case is emulating an RTT 1.x command. The send returns immediately and there is no way in knowing when the function has been executed. The only guarantee you have is that the request arrived at the other side and bar crashes and infinite loops, will complete some time in the future.

A third example is shown below where another service is used with a 'send' which returns a result. The service takes two arguments: a string and a double. The double is the answer of the service, but is not yet available when the send is done. So the second argument is just ignored during the send. A handle 'h' is returned which identifies your send request. You can re-use this handle to collect the results. During collection, the first argument is now ignored, and the second argument is filled in with the result of the service. Collection may be blocking or not.

//New, with collecting results:
  var double ignored_result, result;
 
  set h = comp.other_service.send("hello", ignored_result);
 
  // some time later :
  comp.other_service.collect(h, "ignored", result); // blocking !
 
  // or poll for it:
  if ( comp.other_service.collect_if_done( h, "ignored", result ) == true ) then {
     // use result...
  }

In C++ the above examples are written as:

//New calling:
  the_service.call("hello", result); // also allowed: the_service("hello", result);
 
//New sending:
  the_service.send("hello", ignored_result);
 
//New sending with collecting results:
  h = other_service.send("hello", ignored_result);
 
  // some time later:
  other_service.collect(h, "ignored", result); // blocking !
 
  // or poll for it:
  if ( other_service.collect_if_done( h, "ignored", result ) == true ) {
     // use result...
  }

Completion condition emulation is done like in Proposal 1.

The definition of the service happens at the component's side. The component decides for each service if it is executed in his thread or the callers thread:

  // by default creates a service executed by caller, equivalent to defining a RTT 1.x Method  
  RTT::Service the_service("the_service", &foo_service );
 
  // sets the service to be executed by the component's thread, equivalent to Command
  the_service.setExecutor( this );
 
  //above in one line:
  RTT::Service the_service("the_service", &foo_service, this );