Just something I'm wondering about: would it be possible to implement zero-copy dataflow ports? Could I send a shared_ptr or something similar over a port to another component? Would this work?
Provided both are in the same process, proper precautions are taken (locking), etc.
If not, has the 'isolation' of components something to do with this? Ie: sharing pointers introduces a form of interdependency between components, which may be unwanted.
Extending on the above: are pass-by-reference service calls possible? An OperationCaller with a reference argument seems to compile, I haven't tried running it though.
thanks for your comments,
zero-copy dataflow?
| Submitted by gah on Thu, 2012-08-16 13:58 |
zero-copy dataflow?
Hi g ah,
On Thu, Aug 16, 2012 at 01:58:08PM +0000, g ah wrote:
>
> Just something I'm wondering about: would it be possible to
> implement zero-copy dataflow ports? Could I send a shared_ptr or
> something similar over a port to another component? Would this work?
Yes, this has been done. But it should happen late in your development
process, i.e. you should really know that you need it.
> Provided both are in the same process, proper precautions are taken (locking), etc.
> If not, has the 'isolation' of components something to do with this?
> Ie: sharing pointers introduces a form of interdependency between
> components, which may be unwanted.
Not sure what you mean here. Yes, sending a pointer (shared or not)
introduces a possible race condition, so it is up to you to take care
that once you've sent the pointer you don't muck with it on the other
side.
> Extending on the above: are pass-by-reference service calls
> possible? An OperationCaller with a reference argument seems to
Yes.
> compile, I haven't tried running it though.
Markus
zero-copy dataflow?
Markus Klotzbuecher wrote:
>> Just something I'm wondering about: would it be possible to
>> implement zero-copy dataflow ports? Could I send a shared_ptr or
>> something similar over a port to another component? Would this work?
>
> Yes, this has been done. But it should happen late in your development
> process, i.e. you should really know that you need it.
just curious again: what would happen if such a component (which is sending out pointers) is used in combination with the corba transport, in order to communicate between separate processes? I'm assuming that won't work, but am unsure how 'intelligent' the serialiser in orocos is.
Might be a bit unfair to assume it handles those situations: after all, a shared_ptr is just another datatype.
zero-copy dataflow?
On 08/16/2012 04:22 PM, g ah wrote:
> Markus Klotzbuecher wrote:
>>> Just something I'm wondering about: would it be possible to
>>> implement zero-copy dataflow ports? Could I send a shared_ptr or
>>> something similar over a port to another component? Would this work?
>> Yes, this has been done. But it should happen late in your development
>> process, i.e. you should really know that you need it.
> just curious again: what would happen if such a component (which is sending out pointers) is used in combination with the corba transport, in order to communicate between separate processes? I'm assuming that won't work, but am unsure how 'intelligent' the serialiser in orocos is.
> Might be a bit unfair to assume it handles those situations: after all, a shared_ptr is just another datatype.
Out of the box this would not work because you have to write a typekit
for every new data type which shall be transported. If there would be a
typekit which is aware of the shared_ptr and the object stored insight
this shared_ptr the corba transport would do the right thing.
Have a look at
http://www.orocos.org/stable/documentation/rtt/v2.x/api/html/classRTT_1_...
which is used by our components to do exactly what you want to do (for
clarification: for each template instance a custom typekit is needed) .
Alex
zero-copy dataflow?
On Thu, 16 Aug 2012, Alexander Duda wrote:
> On 08/16/2012 04:22 PM, g ah wrote:
>
> Markus Klotzbuecher wrote:
>
> Just something I'm wondering about: would it be possible to
> implement zero-copy dataflow ports? Could I send a shared_ptr or
> something similar over a port to another component? Would this work?
>
> Yes, this has been done. But it should happen late in your development
> process, i.e. you should really know that you need it.
>
> just curious again: what would happen if such a component (which is sending out pointers) is use
> d in combination with the corba transport, in order to communicate between separate processes? I
> 'm assuming that won't work, but am unsure how 'intelligent' the serialiser in orocos is.
> Might be a bit unfair to assume it handles those situations: after all, a shared_ptr is just ano
> ther datatype.
>
> Out of the box this would not work because you have to write a typekit for every new data type
> which shall be transported. If there would be a typekit which is aware of the shared_ptr and the
> object stored insight this shared_ptr the corba transport would do the right thing.
>
> Have a look athttp://www.orocos.org/stable/documentation/rtt/v2.x/api/html/classRTT_1_1extras_1_1ReadOnlyPoin
> ter.html which is used by our components to do exactly what you want to do (for clarification:
> for each template instance a custom typekit is needed) .
I would advice against using pointers for data port _at the RTT component
level_: this kind of runtime efficiency optimization (while very important)
should not be part of the early development of a system, but be done by a
"deployment tool". There are no such tools currently already available in
the Orocos context (or in any other context that I am aware of).
In other words: frameworks like Orocos do not (yet) separate cleanly
enough the application building phase from the deployment phase: in the
former, one should not worry too much about efficiency, but about logical
correctness of the functionality; the efficiency should be added when one
knows the exact platform (OS, OS-less, FPGA,...) on which the application
will run. Only then, one can make the optimally efficient policy _without_
reducing reusability in other computational platform contexts.
But again: tooling to support this is badly needed, sigh...
> Alex
Herman
zero-copy dataflow?
On Mon, Aug 20, 2012 at 11:02:01AM +0200, Herman Bruyninckx wrote:
> I would advice against using pointers for data port _at the RTT component
> level_: this kind of runtime efficiency optimization (while very important)
> should not be part of the early development of a system, but be done by a
> "deployment tool". There are no such tools currently already available in the
> Orocos context (or in any other context that I am aware of).
An extreme case would be to pass data by registers of the CPU. Unlikely, but
possible (especially when the components are running within the same thread).
A "deployment tool", which is close to handle this kind of optimization is Ada
compiler. In Ada the programmer usually do not have to care about the storage
for a variable (stack/register) and the compiler automagically finds the "best"
way to handle data.
zero-copy dataflow?
On Mon, 20 Aug 2012, Piotr Trojanek wrote:
> On Mon, Aug 20, 2012 at 11:02:01AM +0200, Herman Bruyninckx wrote:
>> I would advice against using pointers for data port _at the RTT component
>> level_: this kind of runtime efficiency optimization (while very important)
>> should not be part of the early development of a system, but be done by a
>> "deployment tool". There are no such tools currently already available in the
>> Orocos context (or in any other context that I am aware of).
>
> An extreme case would be to pass data by registers of the CPU. Unlikely, but
> possible (especially when the components are running within the same thread).
THis is _not at all_ unlikely: the current trend to "programmable hardware"
(FPGA, GPU, multicore,...) is making this use case _a lot_ more realistic.
I really believe robotics _should_ move there, away from the "everything is
a process" approach that was (implicitly) introduced into the robotics
community by the advent of ROS...
> A "deployment tool", which is close to handle this kind of optimization is Ada
> compiler. In Ada the programmer usually do not have to care about the storage
> for a variable (stack/register) and the compiler automagically finds the "best"
> way to handle data.
Ada lacks multi-hardware resource models, doesn't it?
The FPGA and GPU providers are better sources of inspiration of the tooling
that is needed (and already exists) in this context. I think...
> Piotr Trojanek
Herman
zero-copy dataflow?
On Mon, Aug 20, 2012 at 02:04:37PM +0200, Herman Bruyninckx wrote:
> >An extreme case would be to pass data by registers of the CPU. Unlikely, but
> >possible (especially when the components are running within the same
> >thread).
>
> THis is _not at all_ unlikely: the current trend to "programmable hardware"
> (FPGA, GPU, multicore,...) is making this use case _a lot_ more realistic.
> I really believe robotics _should_ move there, away from the "everything is
> a process" approach that was (implicitly) introduced into the robotics
> community by the advent of ROS...
Fully agree.
> >A "deployment tool", which is close to handle this kind of optimization is Ada
> >compiler. In Ada the programmer usually do not have to care about the storage
> >for a variable (stack/register) and the compiler automagically finds the "best"
> >way to handle data.
>
> Ada lacks multi-hardware resource models, doesn't it?
There is a concept of 'partition' in Ada (in particular in the distributed
systems annex), which is probably not exactly what you mean, but it is much
closer to the 'hardware resource model' than anything from the C/C++ world.
The Ada deployment configuration assigns the library units (parts of the user
application) to the individual partitions. Partition usually means 'a process'
and there are some restrictions when coding a multi-partition application
(e.g., regarding the use of "pointers"). The toolchain takes care of most of
the work, e.g., generating the data marshalling and the networking code if
required.
This already works for heterogenous distributed applications (e.g.,
Linux/Windows setups), but there are no tools for setups like Linux/RTAI.
However, the "distributed resource model" is already there.
> The FPGA and GPU providers are better sources of inspiration of the tooling
> that is needed (and already exists) in this context. I think...
I am not an expert in these fields... Any examples?
zero-copy dataflow?
On Mon, 20 Aug 2012, Piotr Trojanek wrote:
[...]
>>> A "deployment tool", which is close to handle this kind of optimization is Ada
>>> compiler. In Ada the programmer usually do not have to care about the storage
>>> for a variable (stack/register) and the compiler automagically finds the "best"
>>> way to handle data.
>>
>> Ada lacks multi-hardware resource models, doesn't it?
>
> There is a concept of 'partition' in Ada (in particular in the distributed
> systems annex), which is probably not exactly what you mean, but it is much
> closer to the 'hardware resource model' than anything from the C/C++ world.
Thanks for the information! My last hands-on Ada experience dates from the
1980s... :-(
> The Ada deployment configuration assigns the library units (parts of the user
> application) to the individual partitions. Partition usually means 'a process'
> and there are some restrictions when coding a multi-partition application
> (e.g., regarding the use of "pointers"). The toolchain takes care of most of
> the work, e.g., generating the data marshalling and the networking code if
> required.
This comes indeed very close to the different kind of "components" that we
(most often still implicitly) use in robotics:
- components as a "port-based scope" of programming functionality,
- components as "containers" that can be deployed as one binary unit on a
software infrastructure platform,
- components as the original meaning of pieces of hardware that are
inseparable as a "product".
> This already works for heterogenous distributed applications (e.g.,
> Linux/Windows setups), but there are no tools for setups like Linux/RTAI.
> However, the "distributed resource model" is already there.
>> The FPGA and GPU providers are better sources of inspiration of the tooling
>> that is needed (and already exists) in this context. I think...
>
> I am not an expert in these fields... Any examples?
The Xilinx tools. CUDA or OpenCL. They have the primitives of "CPU", "Memory",
"BUS", "time" to which all code has to be "translated", but the primitives
can have _very_ different properties wrt performance etc.
> Piotr Trojanek
Herman
zero-copy dataflow?
On Mon, Aug 20, 2012 at 04:06:40PM +0200, Herman Bruyninckx wrote:
> Thanks for the information! My last hands-on Ada experience dates from the
> 1980s... :-(
There were a lot of changes since then :-) In my opinion one of the most
interesting feature of Ada (in the context of this discussion) is the ability
to "subset" the language, e.g., to disallow the use of pointers to data located
in other components. There are much more language constructs, which can be
prohibited, e.g., exceptions, recursion, some forms of threading, etc.
This is quite opposite to the idea of embedded DSLs, where the goal is to
create a new language on top of the host language. In Ada one can build a "DSL"
by disallowing certain constructs of the host language. For example, there is
'pragma Remote_Types', which in practice provides an IDL-like dialect for
specification of data types, which can be transmitted between components of a
distributed application.
> >The Ada deployment configuration assigns the library units (parts of the user
> >application) to the individual partitions. Partition usually means 'a process'
> >and there are some restrictions when coding a multi-partition application
> >(e.g., regarding the use of "pointers"). The toolchain takes care of most of
> >the work, e.g., generating the data marshalling and the networking code if
> >required.
>
> This comes indeed very close to the different kind of "components" that we
> (most often still implicitly) use in robotics:
> - components as a "port-based scope" of programming functionality,
> - components as "containers" that can be deployed as one binary unit on a
> software infrastructure platform,
> - components as the original meaning of pieces of hardware that are
> inseparable as a "product".
Of course Ada can not be used as a drop-in replacement for component models and
runtimes such as Orocos. For example, there is no concept of 'data port' and
the preferred way is to wire all the components statically at compile time. I
believe, however, that there is a lot of "best-practice" in the design of this
language. Much more than in the case of C/C++...
> >>The FPGA and GPU providers are better sources of inspiration of the tooling
> >>that is needed (and already exists) in this context. I think...
> >
> >I am not an expert in these fields... Any examples?
>
> The Xilinx tools. CUDA or OpenCL. They have the primitives of "CPU", "Memory",
> "BUS", "time" to which all code has to be "translated", but the primitives
> can have _very_ different properties wrt performance etc.
I could not find these primitives when looking at OpenCL documentation (?), but
it is also not easy to guess where to find them in the case of Ada. Just for a
reference: "CPU" would be probably "active partition", "Memory" - "storage
pool", "BUS" - "passive partition" (this kind of partition can not host
threads, but only static data and synchronization monitors). Finally "time" is
already well supported by the language.
zero-copy dataflow?
On Sat, Aug 25, 2012 at 11:13:39PM +0200, Piotr Trojanek wrote:
> On Mon, Aug 20, 2012 at 04:06:40PM +0200, Herman Bruyninckx wrote:
> > Thanks for the information! My last hands-on Ada experience dates from the
> > 1980s... :-(
>
> There were a lot of changes since then :-) In my opinion one of the most
> interesting feature of Ada (in the context of this discussion) is the ability
> to "subset" the language, e.g., to disallow the use of pointers to data located
> in other components. There are much more language constructs, which can be
> prohibited, e.g., exceptions, recursion, some forms of threading, etc.
>
> This is quite opposite to the idea of embedded DSLs, where the goal is to
> create a new language on top of the host language. In Ada one can build a "DSL"
> by disallowing certain constructs of the host language. For example, there is
Depending on the host language, this can be easily done with embedded
DSL too, typically by executing the DSL code in a new, restricted
lexical environment.
> 'pragma Remote_Types', which in practice provides an IDL-like dialect for
> specification of data types, which can be transmitted between components of a
> distributed application.
>
> > >The Ada deployment configuration assigns the library units (parts of the user
> > >application) to the individual partitions. Partition usually means 'a process'
> > >and there are some restrictions when coding a multi-partition application
> > >(e.g., regarding the use of "pointers"). The toolchain takes care of most of
> > >the work, e.g., generating the data marshalling and the networking code if
> > >required.
> >
> > This comes indeed very close to the different kind of "components" that we
> > (most often still implicitly) use in robotics:
> > - components as a "port-based scope" of programming functionality,
> > - components as "containers" that can be deployed as one binary unit on a
> > software infrastructure platform,
> > - components as the original meaning of pieces of hardware that are
> > inseparable as a "product".
>
> Of course Ada can not be used as a drop-in replacement for component models and
> runtimes such as Orocos. For example, there is no concept of 'data port' and
> the preferred way is to wire all the components statically at compile time. I
> believe, however, that there is a lot of "best-practice" in the design of this
> language. Much more than in the case of C/C++...
I agree! How well in your opinion would Ada be suited to implement a
component model runtime?
> > >>The FPGA and GPU providers are better sources of inspiration of the tooling
> > >>that is needed (and already exists) in this context. I think...
> > >
> > >I am not an expert in these fields... Any examples?
> >
> > The Xilinx tools. CUDA or OpenCL. They have the primitives of "CPU", "Memory",
> > "BUS", "time" to which all code has to be "translated", but the primitives
> > can have _very_ different properties wrt performance etc.
>
> I could not find these primitives when looking at OpenCL documentation (?), but
> it is also not easy to guess where to find them in the case of Ada. Just for a
> reference: "CPU" would be probably "active partition", "Memory" - "storage
> pool", "BUS" - "passive partition" (this kind of partition can not host
> threads, but only static data and synchronization monitors). Finally "time" is
> already well supported by the language.
Markus
zero-copy dataflow?
On Mon, Aug 27, 2012 at 09:35:49AM +0200, Markus Klotzbuecher wrote:
> > This is quite opposite to the idea of embedded DSLs, where the goal is to
> > create a new language on top of the host language. In Ada one can build a
> > "DSL" by disallowing certain constructs of the host language.
>
> Depending on the host language, this can be easily done with embedded
> DSL too, typically by executing the DSL code in a new, restricted
> lexical environment.
I think, that there is a subtle difference between the "restricted environment"
and "restricted language" approaches. The former ("eval-like") approach
restricts the user only from referencing some of the objects (e.g., variables
or procedures), which are already within the host lexical environment. It does
not, however, prohibit the duplication of such objects within the restricted
environment. In other words, it makes things harder, but not impossible.
AFAIK it is not possible to disallow such constructs as exceptions or recursion
by the "restricted environment" approach. There are some dynamic languages
(e.g., LISP), where exception mechanism is implemented in the form of a
library, so it is probably possible to exclude it from the lexical scope. The
user, however, can still "reimplement" such mechanism within the restricted
environment. I think, that recursion is even more difficult to restrict...
> > Of course Ada can not be used as a drop-in replacement for component models
> > and runtimes such as Orocos. For example, there is no concept of 'data
> > port' and the preferred way is to wire all the components statically at
> > compile time. I believe, however, that there is a lot of "best-practice" in
> > the design of this language. Much more than in the case of C/C++...
>
> I agree! How well in your opinion would Ada be suited to implement a
> component model runtime?
In my opinion it is much better suited for this task than C++! Many of the
features of Orocos are simply already there, e.g., (1) a portable real-time
runtime, (2) all the stuff related to distributed applications (e.g.,
serialization and CORBA interfacing), (3) gccxml-like parsing of code as for
typekit.
The problem is rather in the satellite code, e.g., device drivers or math
libraries. Ada supports calling of external C/C++ code and it is possible to
automatically generate the required bindings. However, a piece of code, which
was designed for a particular language, will be always easier to use in that
language...
On the one hand it is not possible for the robotics community to switch to Ada.
On the other hand, I am afraid, that at some level of complexity of the control
system it will be not possible to certify a robot, which was programmed in C++.
zero-copy dataflow?
On 27 August 2012 22:08, Piotr Trojanek <piotr [dot] trojanek [..] ...> wrote:
> On the one hand it is not possible for the robotics community to switch to Ada.
> On the other hand, I am afraid, that at some level of complexity of the control
> system it will be not possible to certify a robot, which was programmed in C++.
This is why standards such as MISRA C++ exist.
Geoff
zero-copy dataflow?
On Tue, Aug 28, 2012 at 04:41:08PM +0900, Geoffrey Biggs wrote:
> On 27 August 2012 22:08, Piotr Trojanek <piotr [dot] trojanek [..] ...> wrote:
> > On the one hand it is not possible for the robotics community to switch to Ada.
> > On the other hand, I am afraid, that at some level of complexity of the control
> > system it will be not possible to certify a robot, which was programmed in C++.
>
> This is why standards such as MISRA C++ exist.
Yes, but MISRA C (and C++ probably too) is very poor comparing to the tools
from the Ada world... In particular, it does not address concurrency/threading,
right?
Another story is that MISRA is not free, so it is difficult to promote within
the academic/open-source communities.
zero-copy dataflow?
On 28 August 2012 17:05, Piotr Trojanek <piotr [dot] trojanek [..] ...> wrote:
> On Tue, Aug 28, 2012 at 04:41:08PM +0900, Geoffrey Biggs wrote:
>> On 27 August 2012 22:08, Piotr Trojanek <piotr [dot] trojanek [..] ...> wrote:
>> > On the one hand it is not possible for the robotics community to switch to Ada.
>> > On the other hand, I am afraid, that at some level of complexity of the control
>> > system it will be not possible to certify a robot, which was programmed in C++.
>>
>> This is why standards such as MISRA C++ exist.
>
> Yes, but MISRA C (and C++ probably too) is very poor comparing to the tools
> from the Ada world... In particular, it does not address concurrency/threading,
> right?
It does not. I completely agree that Ada is a much better tool and
approach. I think that if you have to artificially restrict the tool,
rather than the tool doing it for you and guaranteeing compliance,
there's too much risk of faults. However, this does not mean that it
will not be possible to certify a robot written using C++ *provided*
MISRA or something similar is followed. Someone trying to do so should
probably expect higher development and certification costs, though.
> Another story is that MISRA is not free, so it is difficult to promote within
> the academic/open-source communities.
Alas, no, it is not.
Geoff
zero-copy dataflow?
> On 28 August 2012 17:05, Piotr Trojanek <piotr [dot] trojanek [..] ...> wrote:
>> On Tue, Aug 28, 2012 at 04:41:08PM +0900, Geoffrey Biggs wrote:
>>> On 27 August 2012 22:08, Piotr Trojanek <piotr [dot] trojanek [..] ...> wrote:
>>> > On the one hand it is not possible for the robotics community to
>>> switch to Ada.
>>> > On the other hand, I am afraid, that at some level of complexity
>>> of the control
>>> > system it will be not possible to certify a robot, which was
>>> programmed in C++.
>>>
>>> This is why standards such as MISRA C++ exist.
>>
>> Yes, but MISRA C (and C++ probably too) is very poor comparing to the tools
>> from the Ada world... In particular, it does not address
>> concurrency/threading,
>> right?
>
> It does not. I completely agree that Ada is a much better tool and
> approach. I think that if you have to artificially restrict the tool,
> rather than the tool doing it for you and guaranteeing compliance,
> there's too much risk of faults. However, this does not mean that it
> will not be possible to certify a robot written using C++ *provided*
> MISRA or something similar is followed. Someone trying to do so should
> probably expect higher development and certification costs, though.
A great piece of work done at DFKI in Bremen which fits to the
discussion of this thread:
"Guaranteeing Functional Safety: Design for Provability and
Computer-Aided Verification" Holger Täubig, Udo Frese, Christoph
Hertzberg, Christoph Lüth, Stefan Mohr, Elena Vorobev, Dennis Walter.
In Autonomous Robots, Springer, volume 32, number 3, pages 303-331,
Apr/2012.
Nico
>
>> Another story is that MISRA is not free, so it is difficult to
>> promote within
>> the academic/open-source communities.
>
> Alas, no, it is not.
>
> Geoff
> --
> Orocos-Users mailing list
> Orocos-Users [..] ...
> http://lists.mech.kuleuven.be/mailman/listinfo/orocos-users
>
zero-copy dataflow?
On Tue, Aug 28, 2012 at 10:42:49AM +0200, Nico Hochgeschwender wrote:
> A great piece of work done at DFKI in Bremen which fits to the discussion of
> this thread:
>
> "Guaranteeing Functional Safety: Design for Provability and
> Computer-Aided Verification" Holger Täubig, Udo Frese, Christoph
> Hertzberg, Christoph Lüth, Stefan Mohr, Elena Vorobev, Dennis Walter.
> In Autonomous Robots, Springer, volume 32, number 3, pages 303-331,
> Apr/2012.
I have never worked with MISRA C (which is used in the above research) -- I was
only reading those guidelines, which I could find for free. My impression is
that many of the guidelines simply reinvent the restrictions, which are a core
of Ada. Similarly, for me the approach to annotate C functions with pre- and
post-conditions (as in the above paper) is not really different from SPARK (a
toolset for high-assurance software in Ada). The difference is rather in the
availability and the maturity of the tools.
It would be interesting to see some evaluation of verifying an existing
robotics software, i.e., whether it is "better" to refactor it according to
MISRA C or rewrite it in Ada.
zero-copy dataflow?
On Tue, Aug 28, 2012 at 12:20:54PM +0200, Piotr Trojanek wrote:
> On Tue, Aug 28, 2012 at 10:42:49AM +0200, Nico Hochgeschwender wrote:
> > A great piece of work done at DFKI in Bremen which fits to the discussion of
> > this thread:
> >
> > "Guaranteeing Functional Safety: Design for Provability and
> > Computer-Aided Verification" Holger Täubig, Udo Frese, Christoph
> > Hertzberg, Christoph Lüth, Stefan Mohr, Elena Vorobev, Dennis Walter.
> > In Autonomous Robots, Springer, volume 32, number 3, pages 303-331,
> > Apr/2012.
>
> I have never worked with MISRA C (which is used in the above research) -- I was
> only reading those guidelines, which I could find for free. My impression is
> that many of the guidelines simply reinvent the restrictions, which are a core
> of Ada. Similarly, for me the approach to annotate C functions with pre- and
> post-conditions (as in the above paper) is not really different from SPARK (a
> toolset for high-assurance software in Ada). The difference is rather in the
> availability and the maturity of the tools.
But unfortunately, most of the time, it will be cheaper (apparently,
at least) to force the hordes of mediocre C++ programmers (like
myself) to stop using dangerous constructs instead of training them in
Ada.
> It would be interesting to see some evaluation of verifying an existing
> robotics software, i.e., whether it is "better" to refactor it according to
> MISRA C or rewrite it in Ada.
That would be interesting indeed, though not easy to carry out without
personal preferences distorting the evaluation.
Markus
zero-copy dataflow?
Markus Klotzbuecher wrote:
> Hi g ah,
hi, thank you for your reply
> On Thu, Aug 16, 2012 at 01:58:08PM +0000, g ah wrote:
>> Just something I'm wondering about: would it be possible to
>> implement zero-copy dataflow ports? Could I send a shared_ptr or
>> something similar over a port to another component? Would this work?
>
> Yes, this has been done. But it should happen late in your development
> process, i.e. you should really know that you need it.
Are you implying that it should only be done in case of performance issues? Ie: it is actually frowned upon, but ok under certain circumstances?
>> Provided both are in the same process, proper precautions are taken (locking), etc.
>> If not, has the 'isolation' of components something to do with this?
>> Ie: sharing pointers introduces a form of interdependency between
>> components, which may be unwanted.
>
> Not sure what you mean here. Yes, sending a pointer (shared or not)
> introduces a possible race condition, so it is up to you to take care
> that once you've sent the pointer you don't muck with it on the other
> side.
what I was trying to say is that sharing pointers to datastructures implies that both components have knowledge of that datastructure, and access. You're essentially poking a hole in the encapsulation enforced by the component concept.
This as opposed to using dataports that carry 'intermediate' types that have been explicitly designed to be shared among components.
>> Extending on the above: are pass-by-reference service calls
>> possible? An OperationCaller with a reference argument seems to
>> compile, I haven't tried running it though.
> Yes.
but the same restrictions as for dataflow ports hold, I assume?
zero-copy dataflow?
On Thu, 16 Aug 2012, g ah wrote:
>
> Markus Klotzbuecher wrote:
>> Hi g ah,
> hi, thank you for your reply
>
>> On Thu, Aug 16, 2012 at 01:58:08PM +0000, g ah wrote:
>>> Just something I'm wondering about: would it be possible to
>>> implement zero-copy dataflow ports? Could I send a shared_ptr or
>>> something similar over a port to another component? Would this work?
>>
>> Yes, this has been done. But it should happen late in your development
>> process, i.e. you should really know that you need it.
> Are you implying that it should only be done in case of performance
> issues? Ie: it is actually frowned upon, but ok under certain
> circumstances?
Indeed... "Don't do this at home" :-)
>>> Provided both are in the same process, proper precautions are taken (locking), etc.
>>> If not, has the 'isolation' of components something to do with this?
>>> Ie: sharing pointers introduces a form of interdependency between
>>> components, which may be unwanted.
>>
>> Not sure what you mean here. Yes, sending a pointer (shared or not)
>> introduces a possible race condition, so it is up to you to take care
>> that once you've sent the pointer you don't muck with it on the other
>> side.
> what I was trying to say is that sharing pointers to datastructures
> implies that both components have knowledge of that datastructure, and
> access. You're essentially poking a hole in the encapsulation enforced by
> the component concept.
Yes! So, it should _only_ be used in a "composition" ("scope",
"closure",...) of components where you know _at deployment time_ that the
types of ports match, for the full 100%. That's why I advocate this
approach to be used only at deployment time.
> This as opposed to using dataports that carry 'intermediate' types that
> have been explicitly designed to be shared among components.
It _is_ indeed contrary to the core idea of using component based design!
Herman
>>> Extending on the above: are pass-by-reference service calls
>>> possible? An OperationCaller with a reference argument seems to
>>> compile, I haven't tried running it though.
>> Yes.
> but the same restrictions as for dataflow ports hold, I assume?
>