Hello,
we (ASSET Institute, TU Delft) started the development of a set of
GUI components for our kite-power control system.
Many of these components are generic and might be useful in a
different context, too.
Components
----------------
Components, that are currently in development:
- Clock (emits a fast clock and a slow clock, using UDP packages)
- CarSim (very simple model of a car, that is moving on a straight line)
- FileReader (reads csv files, sends UDP packages on eauch clock event)
- FuncGen (generates sinus, triangle and sqare signals)
- Oscilloscope (displays scalar and vector data streams)
- UDPLogger (logs UDP messages and displays them in a human readable format on the screen)
- UDPClient (emits UDP messages of type SCALAR, VECTOR or EVENT)
The components can be used for simulations with up to 100 Hz sampling rate
without using OROCOS, or together with orocos for much higher sampling rates.
All components are implemented as stand-alone executables. They shall have a default
screen size of 1/8 or 1/4 of a 1280x1024 screen, so that up to 8 of the small
components can be used at the same time on one screen.
Configuration files are used, to define the port numbers and ip numbers of each
input and output. Any number of inputs can be connected to one output.
Communication protocol:
---------------------------------
Requirements:
- it must be possible, to use it for the communication between
processes on one computer, but also an a LAN segment between
different computers
- the protocol must be able to transport messages with a size
of up to 32 double values
- multicasting (one output, multiple inputs) must be possible
- hard real-time (bounded latency of e.g. 3 ms) must be possible
- all messages must be unambiguous
- the protocol shall be simple and easy to use
Current implementation:
- uses UDP packages
- binary or ASCII message format
- messages have the type SCALAR, VECTOR or EVENT
- all numbers are transferred as double values
- lost packages can be detected
- all packages have a time-stamp, so the latency of each transfer
can be calculated with a microsecond resolution (if you have
synchronized clocks)
- on the input side a message- queue is used, to avoid package losses,
if the processing needs too much time (normally this should not be
needed, but it is needed for example for slow display components)
- it uses hard coded enumerations for the physical quantity, and
the defined events
- runs an Linux with rt-preemt patches or xenomai
Any comments welcome.
Regards:
Uwe Fechner
GUI components for real-time control and simulation
On Sun, 21 Aug 2011, Uwe Fechner wrote:
> Thank you for you comments.
>
> I published a first roadmap and screen-shot:
>
> http://www.kieltech.de/uweswiki/GUI%20components%20for%20real-time%20simula
> tion%20and%20control/RoadMap
>
> http://www.kieltech.de/uweswiki/GUI%20components%20for%20real-time%20simula
> tion%20and%20control/ScreenShots
Thanks a lot for sharing this information! I hope some synergistic
cooperation can grow out of this :-)
> Best regards:
>
> Uwe Fechner
Herman
> Am 19.08.2011 10:47, schrieb Piotr Trojanek:
> On Fri, Aug 19, 2011 at 10:24, Uwe Fechner
> <u [dot] fechner [..] ...> wrote:
> What are "reusable components"?
> And how to they differ from "classes"?
>
>
> Class is strictly related to object-oriented programming. Component is
> a similar concept,
> but more high-level and independent of the programming paradigm.
>
> You can create components in Lisp, Prolog or even assembler, as long
> as you encapsulate
> a set of related functions or data (to quote WikiPedia definition).
> Obviously, object-oriented
> languages support component-oriented programming.
>
> On the other hand, I think the amount of changes, that are
> possible without recompiling are limited. One example:
> If you want to exchange the communication protocol, from
> my point of view it would be acceptable, if you have to
> re-implement an abstract class and recompile.
>
>
> With a well-defined interfaces this can be done with dynamic linking
> and no recompilation
> is required. You can have, e.g., libtransport_tcp.so and
> libtransport_udp.so libraries and
> choose (or even switch) them at runtime.
>
> --
> Piotr Trojanek
>
>
>
>
>
--
K.U.Leuven, Mechanical Eng., Mechatronics & Robotics Research Group
<http://people.mech.kuleuven.be/~bruyninc> Tel: +32 16 328056
EURON Coordinator (European Robotics Research Network) <http://www.euron.org>
Open Realtime Control Services <http://www.orocos.org>
Associate Editor JOSER <http://www.joser.org>, IJRR <http://www.ijrr.org>
GUI components for real-time control and simulation
On Thu, 18 Aug 2011, Uwe Fechner wrote:
> Hello,
>
> we (ASSET Institute, TU Delft) started the development of a set of
> GUI components for our kite-power control system.
>
> Many of these components are generic and might be useful in a
> different context, too.
>
> Components
> ----------------
> Components, that are currently in development:
>
> - Clock (emits a fast clock and a slow clock, using UDP packages)
Orocos/RTT has the "TimeService" for this purpose.
I would strongly suggest to keep the function of a component (providing
time ticks in this case) separate from the communication platform (UDP in
your case).
> - CarSim (very simple model of a car, that is moving on a straight line)
This is already a complete application, since the model is not enough to
keep the car on a straight line, so there are a controller and an integrator component involved, somewhere, isn't it?
> - FileReader (reads csv files, sends UDP packages on eauch clock event)
> - FuncGen (generates sinus, triangle and sqare signals)
FuncGen is a too generic name; I would prefer the name of the function that
is generated.
> - Oscilloscope (displays scalar and vector data streams)
> - UDPLogger (logs UDP messages and displays them in a human readable format on the screen)
> - UDPClient (emits UDP messages of type SCALAR, VECTOR or EVENT)
>
> The components can be used for simulations with up to 100 Hz sampling rate
> without using OROCOS, or together with orocos for much higher sampling rates.
Performance will most probably depend more on the choice of your
communication architecture than on the use of Orocos :-)
> All components are implemented as stand-alone executables.
That's a pity. Why haven't you chosen to go for a more decoupled design,
where functionality is independent of the "process container"?
> They shall have a default
> screen size of 1/8 or 1/4 of a 1280x1024 screen, so that up to 8 of the small
> components can be used at the same time on one screen.
>
> Configuration files are used, to define the port numbers and ip numbers of each
> input and output. Any number of inputs can be connected to one output.
>
> Communication protocol:
> ---------------------------------
>
> Requirements:
> - it must be possible, to use it for the communication between
> processes on one computer, but also an a LAN segment between
> different computers
> - the protocol must be able to transport messages with a size
> of up to 32 double values
> - multicasting (one output, multiple inputs) must be possible
> - hard real-time (bounded latency of e.g. 3 ms) must be possible
> - all messages must be unambiguous
> - the protocol shall be simple and easy to use
>
> Current implementation:
> - uses UDP packages
> - binary or ASCII message format
> - messages have the type SCALAR, VECTOR or EVENT
> - all numbers are transferred as double values
Have you thought about using one of the (too many) "serialization"
frameworks: Google Protocol Buffers, Apache Thrift, ZeroMQ, LabComm,...?
> - lost packages can be detected
> - all packages have a time-stamp, so the latency of each transfer
> can be calculated with a microsecond resolution (if you have
> synchronized clocks)
> - on the input side a message- queue is used, to avoid package losses,
> if the processing needs too much time (normally this should not be
> needed, but it is needed for example for slow display components)
> - it uses hard coded enumerations for the physical quantity, and
> the defined events
> - runs an Linux with rt-preemt patches or xenomai
>
> Any comments welcome.
What suggestions do you have about code that you think about releasing, or
best practices that you have learned, or pitfalls to avoid, or...?
> Regards:
>
> Uwe Fechner
Herman
GUI components for real-time control and simulation
Hello Hermann,
I want to answer some of your remarks quickly:
> Orocos/RTT has the "TimeService" for this purpose.
>
> I would strongly suggest to keep the function of a component (providing
> time ticks in this case) separate from the communication platform (UDP in
> your case).
Well, if I have a number of components, that work together, than it is convenient to have
a central GUI component to send the events START, STOP, PAUSE, CONTINUE and to change
the sampling intervals (you might want to increase it during testing, until the controller
becomes unstable, or decrease it, until you start to lose packages etc.).
>> - CarSim (very simple model of a car, that is moving on a straight line)
> This is already a complete application, since the model is not enough to
> keep the car on a straight line, so there are a controller and an integrator component involved, somewhere, isn't it?
>
We need it as test signal generator for the winch controller, that shall be developed first.
It has an integrator, but no controller. Just three input values (acceleration, max. velocity
and max. time.
>> - FileReader (reads csv files, sends UDP packages on eauch clock event)
>> - FuncGen (generates sinus, triangle and sqare signals)
> FuncGen is a too generic name; I would prefer the name of the function that
> is generated.
Well, you can buy an electronic instrument, that is called "function generator". This is, why I
chose this name. And - of course - it shall be able to generate different kind of functions, so I
cannot use
the name of a fixed function. It shall be possible to change the function during the tests in
real-time.
> All components are implemented as stand-alone executables.
> That's a pity. Why haven't you chosen to go for a more decoupled design,
> where functionality is independent of the "process container"?
This is not a final decision. What kind of "process container" would you suggest?
My goal is, that students can write additional components during their thesis.
It is important, that it is easy to write these components, and that the whole system
stays stable, if a component, that is not essential for the stability of the system crashes.
> Have you thought about using one of the (too many) "serialization"
> frameworks: Google Protocol Buffers, Apache Thrift, ZeroMQ, LabComm,...?
>
I just looked at "Google protocol buffers", and they look interesting. Important for me is the
possibility to do co-simulations with simulink, and the simulink version, that we have
here has only default components to read and write vectors of the same data-type over
UDP. But there is a Google protocol buffer library for Matlab, perhaps it is possible to
make a Simulink component out of it. We are doing soft-real-time co-simulations with
our C++ components and Matlab at 50 Hz sampling rate.
There would also be the possibility, to use "Google protocol buffers" between the GUI and
OROCOS, and a different serialization for the Simulink co-simulation.
>
> What suggestions do you have about code that you think about releasing, or
> best practices that you have learned, or pitfalls to avoid, or...?
I think, that I will publish a little bit about my experiences on my wiki:
http://www.kieltech.de/uweswiki/FrontPage
I think, that it is too early, to publish any code yet, but hopefully we can start doing it
in one or two months (I need the permission of my supervisor).
Best regards:
Uwe
GUI components for real-time control and simulation
On Thu, 18 Aug 2011, Uwe Fechner wrote:
> Hello Hermann,
>
> I want to answer some of your remarks quickly:
>
>> Orocos/RTT has the "TimeService" for this purpose.
>>
>> I would strongly suggest to keep the function of a component (providing
>> time ticks in this case) separate from the communication platform (UDP in
>> your case).
> Well, if I have a number of components, that work together, than it is
> convenient to have a central GUI component to send the events START,
> STOP, PAUSE, CONTINUE and to change the sampling intervals (you might
> want to increase it during testing, until the controller becomes
> unstable, or decrease it, until you start to lose packages etc.).
Absolutely, that's what I am saying: this is _functionality_, and that
should be kept separate from the _communication_ that is between the GUI
and the application it is GUIing :-)
>>> - CarSim (very simple model of a car, that is moving on a straight line)
>> This is already a complete application, since the model is not enough to
>> keep the car on a straight line, so there are a controller and an integrator component involved, somewhere, isn't it?
>>
> We need it as test signal generator for the winch controller, that shall
> be developed first.
> It has an integrator, but no controller.
Ok, I understand. What integrator code/component/library do you use?
> Just three input values
> (acceleration, max. velocity and max. time.
>>> - FileReader (reads csv files, sends UDP packages on eauch clock event)
>>> - FuncGen (generates sinus, triangle and sqare signals)
>> FuncGen is a too generic name; I would prefer the name of the function that
>> is generated.
> Well, you can buy an electronic instrument, that is called "function
> generator". This is, why I chose this name.
Again, I want to stress the difference between an _application_ ("product")
and the components that are behind it. This thread is about the latter,
definitely, and trying to find out with the community _how_ to make these
components as reusable as possible in as many applications as possible.
And choosing the right name is a necessary step in that direction :-)
> And - of course - it shall be able to generate different kind of
> functions, so I cannot use the name of a fixed function. It shall be
> possible to change the function during the tests in real-time.
>> All components are implemented as stand-alone executables.
>> That's a pity. Why haven't you chosen to go for a more decoupled design,
>> where functionality is independent of the "process container"?
> This is not a final decision. What kind of "process container" would you
> suggest?
None! Because that is exactly what all "product" designers should be
allowed to choose for themselves. Orocos is all about providing some
building blocks to get to that goal as good as possible, but it stops at
these "product"-level choices such as (i) the communication middleware, and
(ii) the "process" container.
> My goal is, that students can write additional components during their
> thesis. It is important, that it is easy to write these components,
My experience is that you make it easier for them if you give the good
example of how (i) to make decoupled components, _and_ (ii) to compose them
into a system. I would advise against following the opposite direction,
that is, first making some "product"-level decisions ("process", UDP
transport, naming that suggests more than what is behind the surface,...),
and then expecting students (or others) to add components to that.
> and
> that the whole system stays stable, if a component, that is not essential
> for the stability of the system crashes.
>
>> Have you thought about using one of the (too many) "serialization"
>> frameworks: Google Protocol Buffers, Apache Thrift, ZeroMQ, LabComm,...?
>>
> I just looked at "Google protocol buffers", and they look interesting.
> Important for me is the possibility to do co-simulations with simulink,
> and the simulink version, that we have here has only default components
> to read and write vectors of the same data-type over UDP. But there is a
> Google protocol buffer library for Matlab, perhaps it is possible to make
> a Simulink component out of it. We are doing soft-real-time
> co-simulations with our C++ components and Matlab at 50 Hz sampling rate.
Interesting use case!
> There would also be the possibility, to use "Google protocol buffers"
> between the GUI and OROCOS, and a different serialization for the
> Simulink co-simulation.
>> What suggestions do you have about code that you think about releasing, or
>> best practices that you have learned, or pitfalls to avoid, or...?
> I think, that I will publish a little bit about my experiences on my wiki:
> http://www.kieltech.de/uweswiki/FrontPage
Thanks! I do find it a pitty that the first thing you put on such a wiki
are installation instructions... :-( If you think the ones on the "main"
site are not good, please provide patches to those, instead of "polluting"
the WWW with alternative documentation. This leads to a tremendous
fragmentation, and to a decreased confidence of new users in the project.
> I think, that it is too early, to publish any code yet, but hopefully we
> can start doing it in one or two months (I need the permission of my
> supervisor).
I will talk to him and convince him! :-)
> Best regards:
>
> Uwe
Herman
GUI components for real-time control and simulation
Hello,
you wrote:
> Again, I want to stress the difference between an _application_ ("product")
> and the components that are behind it. This thread is about the latter,
> definitely, and trying to find out with the community _how_ to make these
> components as reusable as possible in as many applications as possible.
What are "reusable components"?
And how to they differ from "classes"?
I think, that we don't have a common understanding
of these terms yet.
>From my point of view, a reusable GUI component needs
to have a GUI, that means, it must be visible.
The most easy way, to implement a GUI is an executable,
that displays a window.
In this case no other framework to run the component
is needed, than a window manager.
It should be possible, to make some changes to the
component without recompiling, just by editing configuration
files.
One example could be a configuration file, that defines the
names of the valid events. Another example would be a
configuration file, that defines the connections between
the components.
On the other hand, I think the amount of changes, that are
possible without recompiling are limited. One example:
If you want to exchange the communication protocol, from
my point of view it would be acceptable, if you have to
re-implement an abstract class and recompile.
What is the application, if a component is already a program,
that can run on it's own?
Well, the application is a set of components together with
a set of configuration files. To start these set of components,
a simple bash script or a separate starter program might
be used. In general the application will be build of a number
of reusable components AND some application specific
components.
A little bit more about GUI components:
Having separate executables is probably the easiest way
for GUI components, but you can achieve a higher level
of reusability, if you build implementations of the
QWidged class. Then you could build monolithic application,
that uses a number of these GUI components.
But this would reduce the robustness of the application,
because now all GUI components run in the same address
space.
Why are Simulink and LabView so successful?
Because they provide a large set of reusable components,
that can be configured by changing properties.
What are the limitations of these programs:
- expensive
- closed source
- not designed for realtime- control
- not usable for distributed real-time control
My vision for the GUI-components project:
- a component library with ready-to-use components
(no compilation needed)
- can easily be used in a distributed environment
(by default the latency of all messages shall
be monitored)
- a little bit of adaption shall be possible by
editing configuration files
(actually, composing the application from the
components shall be possible in this way; later
a GUI-based connection editor could be added)
- bigger changes shall be possible by reimplementing
abstract classes and recompiling without making it
difficult, to update the original components
- scriptable with LUA
- it should be possible, to write components in
other languages than C++, e.g. Java
- co-simulation with Simulink must be possible
- good integration with OROCOS
These ideas are by no means complete.
Best regards:
Uwe Fechner
GUI components for real-time control and simulation
On Fri, 19 Aug 2011, Uwe Fechner wrote:
> Hello,
>
> you wrote:
>
>> Again, I want to stress the difference between an _application_ ("product")
>> and the components that are behind it. This thread is about the latter,
>> definitely, and trying to find out with the community _how_ to make these
>> components as reusable as possible in as many applications as possible.
>
>
> What are "reusable components"?
> And how to they differ from "classes"?
Think about the difference between transistors and chips, in the context of
computer hardware. The goal and workflow of making chips from transistors
is different from, and complementary to, that of making a PC motherboard
from chips. There is room for both and the boundaries are not sharp.
Another way of thinking about when to use components and when classes is as
follows: you want to make some piece of software functionality that you
want people to use without having to recompile, or without any dependency
on specific other class libraries; just connect the 'wires', make sure both
ends agree on the protocol(s) on the wire, and there you go.
> I think, that we don't have a common understanding of these terms yet.
I'm sure we don't. :-) It's the number one discussion point in most of the
talks I give about these things... But it is also not easy to give a
FAQ-like answer.
>> From my point of view, a reusable GUI component needs
>> to have a GUI, that means, it must be visible.
>
> The most easy way, to implement a GUI is an executable,
> that displays a window.
>
> In this case no other framework to run the component
> is needed, than a window manager.
>
> It should be possible, to make some changes to the
> component without recompiling, just by editing configuration
> files.
>
> One example could be a configuration file, that defines the
> names of the valid events. Another example would be a
> configuration file, that defines the connections between
> the components.
>
> On the other hand, I think the amount of changes, that are
> possible without recompiling are limited. One example:
> If you want to exchange the communication protocol, from
> my point of view it would be acceptable, if you have to
> re-implement an abstract class and recompile.
>
> What is the application, if a component is already a program,
> that can run on it's own?
> Well, the application is a set of components together with
> a set of configuration files. To start these set of components,
> a simple bash script or a separate starter program might
> be used. In general the application will be build of a number
> of reusable components AND some application specific
> components.
>
> A little bit more about GUI components:
> Having separate executables is probably the easiest way
> for GUI components, but you can achieve a higher level
> of reusability, if you build implementations of the
> QWidged class. Then you could build monolithic application,
> that uses a number of these GUI components.
>
> But this would reduce the robustness of the application,
> because now all GUI components run in the same address
> space.
>
> Why are Simulink and LabView so successful?
>
> Because they provide a large set of reusable components,
> that can be configured by changing properties.
>
> What are the limitations of these programs:
> - expensive
> - closed source
> - not designed for realtime- control
> - not usable for distributed real-time control
>
> My vision for the GUI-components project:
>
> - a component library with ready-to-use components
> (no compilation needed)
> - can easily be used in a distributed environment
> (by default the latency of all messages shall
> be monitored)
> - a little bit of adaption shall be possible by
> editing configuration files
> (actually, composing the application from the
> components shall be possible in this way; later
> a GUI-based connection editor could be added)
> - bigger changes shall be possible by reimplementing
> abstract classes and recompiling without making it
> difficult, to update the original components
> - scriptable with LUA
> - it should be possible, to write components in
> other languages than C++, e.g. Java
> - co-simulation with Simulink must be possible
> - good integration with OROCOS
>
> These ideas are by no means complete.
Thanks for sharing them! I think I understand most of them (and agree with
them).
> Best regards:
>
> Uwe Fechner
Herman
GUI components for real-time control and simulation
On Fri, Aug 19, 2011 at 10:24, Uwe Fechner <u [dot] fechner [..] ...> wrote:
> What are "reusable components"?
> And how to they differ from "classes"?
>
Class is strictly related to object-oriented programming. Component is a
similar concept,
but more high-level and independent of the programming paradigm.
You can create components in Lisp, Prolog or even assembler, as long as you
encapsulate
a set of related functions or data (to quote WikiPedia definition).
Obviously, object-oriented
languages support component-oriented programming.
On the other hand, I think the amount of changes, that are
> possible without recompiling are limited. One example:
> If you want to exchange the communication protocol, from
> my point of view it would be acceptable, if you have to
> re-implement an abstract class and recompile.
>
With a well-defined interfaces this can be done with dynamic linking and no
recompilation
is required. You can have, e.g., libtransport_tcp.so and libtransport_udp.so
libraries and
choose (or even switch) them at runtime.
GUI components for real-time control and simulation
On 08/18/2011 11:23 AM, Herman Bruyninckx wrote:
> On Thu, 18 Aug 2011, Uwe Fechner wrote:
>
>> Hello,
>>
>> we (ASSET Institute, TU Delft) started the development of a set of
>> GUI components for our kite-power control system.
>>
>> Many of these components are generic and might be useful in a
>> different context, too.
>>
>> Components
>> ----------------
>> Components, that are currently in development:
>>
>> - Clock (emits a fast clock and a slow clock, using UDP packages)
>
> Orocos/RTT has the "TimeService" for this purpose.
>
> I would strongly suggest to keep the function of a component (providing
> time ticks in this case) separate from the communication platform (UDP in
> your case).
>
>> - CarSim (very simple model of a car, that is moving on a straight line)
>
> This is already a complete application, since the model is not enough to
> keep the car on a straight line, so there are a controller and an integrator component involved, somewhere, isn't it?
>
>> - FileReader (reads csv files, sends UDP packages on eauch clock event)
>> - FuncGen (generates sinus, triangle and sqare signals)
>
> FuncGen is a too generic name; I would prefer the name of the function that
> is generated.
>
>> - Oscilloscope (displays scalar and vector data streams)
>> - UDPLogger (logs UDP messages and displays them in a human readable format on the screen)
>> - UDPClient (emits UDP messages of type SCALAR, VECTOR or EVENT)
>>
>> The components can be used for simulations with up to 100 Hz sampling rate
>> without using OROCOS, or together with orocos for much higher sampling rates.
>
> Performance will most probably depend more on the choice of your
> communication architecture than on the use of Orocos :-)
>
>> All components are implemented as stand-alone executables.
>
> That's a pity. Why haven't you chosen to go for a more decoupled design,
> where functionality is independent of the "process container"?
>
>> They shall have a default
>> screen size of 1/8 or 1/4 of a 1280x1024 screen, so that up to 8 of the small
>> components can be used at the same time on one screen.
>>
>> Configuration files are used, to define the port numbers and ip numbers of each
>> input and output. Any number of inputs can be connected to one output.
>>
>> Communication protocol:
>> ---------------------------------
>>
>> Requirements:
>> - it must be possible, to use it for the communication between
>> processes on one computer, but also an a LAN segment between
>> different computers
>> - the protocol must be able to transport messages with a size
>> of up to 32 double values
>> - multicasting (one output, multiple inputs) must be possible
>> - hard real-time (bounded latency of e.g. 3 ms) must be possible
>> - all messages must be unambiguous
>> - the protocol shall be simple and easy to use
>>
>> Current implementation:
>> - uses UDP packages
>> - binary or ASCII message format
>> - messages have the type SCALAR, VECTOR or EVENT
>> - all numbers are transferred as double values
>
> Have you thought about using one of the (too many) "serialization"
> frameworks: Google Protocol Buffers, Apache Thrift, ZeroMQ, LabComm,...?
+1 on Google Protocol Buffers. We have used it before in (non-realtime)
datalogging components to serialize complex objects to strings and then
these strings can be sent out over a socket. (protobufs doesn't provide
an API for serializing directly to sockets). The advantage of protobufs
is that they can have 'extensions' [
http://code.google.com/apis/protocolbuffers/docs/proto.html#extensions ]
and can be self-describing [
http://code.google.com/apis/protocolbuffers/docs/techniques.html#self-de...
] and this may be useful depending on your design.
/Sagar
>> - lost packages can be detected
>> - all packages have a time-stamp, so the latency of each transfer
>> can be calculated with a microsecond resolution (if you have
>> synchronized clocks)
>> - on the input side a message- queue is used, to avoid package losses,
>> if the processing needs too much time (normally this should not be
>> needed, but it is needed for example for slow display components)
>> - it uses hard coded enumerations for the physical quantity, and
>> the defined events
>> - runs an Linux with rt-preemt patches or xenomai
>>
>> Any comments welcome.
>
> What suggestions do you have about code that you think about releasing, or
> best practices that you have learned, or pitfalls to avoid, or...?
>
>> Regards:
>>
>> Uwe Fechner
>
> Herman
GUI components for real-time control and simulation
On Thu, Aug 18, 2011 at 12:58, Sagar Behere <sagar [dot] behere [..] ...> wrote:
> +1 on Google Protocol Buffers. We have used it before in (non-realtime)
> datalogging components to serialize complex objects to strings and then
> these strings can be sent out over a socket. (protobufs doesn't provide
> an API for serializing directly to sockets). The advantage of protobufs
> is that they can have 'extensions' [
> http://code.google.com/apis/protocolbuffers/docs/proto.html#extensions ]
> and can be self-describing [
>
> http://code.google.com/apis/protocolbuffers/docs/techniques.html#self-de...
> ] and this may be useful depending on your design.
>
Exactly the same can be done with boost::serialization's XML archive.
I see no reason to use protobufs if you are using C++ only.
Boost::serialization
allows to handle much more complex structures (e.g., STL collections) much
easier than protobufs.
GUI components for real-time control and simulation
On Thu, 18 Aug 2011, Piotr Trojanek wrote:
> On Thu, Aug 18, 2011 at 12:58, Sagar Behere <sagar [dot] behere [..] ...> wrote:
> +1 on Google Protocol Buffers. We have used it before in
> (non-realtime)
> datalogging components to serialize complex objects to strings and
> then
> these strings can be sent out over a socket. (protobufs doesn't
> provide
> an API for serializing directly to sockets). The advantage of
> protobufs
> is that they can have 'extensions' [
> http://code.google.com/apis/protocolbuffers/docs/proto.html#extensions
> ]
> and can be self-describing [
> http://code.google.com/apis/protocolbuffers/docs/techniques.html#self-de...
> ion
> ] and this may be useful depending on your design.
>
>
> Exactly the same can be done with boost::serialization's XML archive.
>
> I see no reason to use protobufs if you are using C++ only.
The world is using (much!) more than C++! I find it an extremely important
aspect of a middleware that it is as programming language agnostic as
possible.
> Boost::serialization
> allows to handle much more complex structures (e.g., STL collections) much
> easier than protobufs.
There are much older and mature projects that do that for more than a
decade already, e.g., NetCDF or HDF5.
I would suggest not to fall in the trap of "my favority has this feature
that your favorite does not have", and concentrate on the minimal set of
features that _must_ be there.
Herman
GUI components for real-time control and simulation
On Thu, Aug 18, 2011 at 15:00, Herman Bruyninckx <
Herman [dot] Bruyninckx [..] ...> wrote:
> Boost::serialization
>> allows to handle much more complex structures (e.g., STL collections) much
>> easier than protobufs.
>>
>
> There are much older and mature projects that do that for more than a
> decade already, e.g., NetCDF or HDF5.
>
Not really. The NetCDF, HDF5 (and also XDR, XML, etc.) are just the data
formats.
They do not provide any "framework" for serialization. By a "framework" I
mean
a unified API, which is independent of the underlying data format.
Of course you can use "your favorite" IDL, but this is exactly what you want
to avoid...
Whatever you choose, the way to go is to decouple communication from
computation.
But eventually you will have to choose one of the IDLs/formats/serialization
frameworks anyway.
> I would suggest not to fall in the trap of "my favority has this feature
> that your favorite does not have", and concentrate on the minimal set of
> features that _must_ be there.
>
One of minimal features was gives as hard real-time, so serialization with
Google Protocol Buffers
will not work for this. Apache Thrift probably too. Not sure about NetCDF
and HDF5. XDR will do
the job and boost::serialization is just a IDL-less wrapper for C++.
ZeroMQ provides only message-oriented transports (without any
serialization).
GUI components for real-time control and simulation
On Thu, Aug 18, 2011 at 11:23, Herman Bruyninckx <
Herman [dot] Bruyninckx [..] ...> wrote:
> On Thu, 18 Aug 2011, Uwe Fechner wrote:
> > Configuration files are used, to define the port numbers and ip numbers
> of each
> > input and output. Any number of inputs can be connected to one output.
> >
> > Communication protocol:
> > ---------------------------------
> >
> > Requirements:
> > - it must be possible, to use it for the communication between
> > processes on one computer, but also an a LAN segment between
> > different computers
> > - the protocol must be able to transport messages with a size
> > of up to 32 double values
> > - multicasting (one output, multiple inputs) must be possible
> > - hard real-time (bounded latency of e.g. 3 ms) must be possible
>
There are solutions like "real-time ethernet" for this, but 3ms on a
separate
LAN segment should not be difficult. If you want to use the same cables also
for non-real-time traffic, than some QoS IP stack configuration can be
helpful.
> > - all messages must be unambiguous
> > - the protocol shall be simple and easy to use
> >
> > Current implementation:
> > - uses UDP packages
>
UDP, in general, gives you unordered data delivery, so you should be careful
and handle
this at protocol or application level.
> > - binary or ASCII message format
>
I would not recommend using ASCII for double values, since you can easily
loose data precision.
Moreover, if you encode 8-byte double value as string, then it will take
much more space.
What we use is a boost::archive for XDR marshaling. It is real-time (i.e.,
zero allocation) and works
for 32/64-bit as well as for cross-endian scenarios:
https://github.com/wut-rcprg/mrrocpp/tree/master/src/base/lib/xdr
> - messages have the type SCALAR, VECTOR or EVENT
> > - all numbers are transferred as double values
>
Is this not just a wasting of bandwidth?
>
> Have you thought about using one of the (too many) "serialization"
> frameworks: Google Protocol Buffers, Apache Thrift, ZeroMQ, LabComm,...?
>
ZeroMQ is worth to look at. In particular it offers "reliable multicast
transport".
In the 1.x branch there was also support for SCTP, which probably better
suits
your needs than UDP. However, it is only "transport" library, without any
support
for "serialization".
Google Protocol Buffers does not give you real-time performance, as far as I
know.
> > - lost packages can be detected
>
> - all packages have a time-stamp, so the latency of each transfer
> > can be calculated with a microsecond resolution (if you have
> > synchronized clocks)
>
You can take a look at the linux/Documentation/networking/timestamping.txt
if you require this level of accuracy. It is used by the Precision Time
Protocol
utilities which you can use for the clock synchronization.
> > - on the input side a message- queue is used, to avoid package losses,
> > if the processing needs too much time (normally this should not be
> > needed, but it is needed for example for slow display components)
>
This can be done at the InputPort's Buffer policy level.
> > - it uses hard coded enumerations for the physical quantity, and
> > the defined events
>
This should be easy if you use boost::serialization and have a separate,
i.e. "strong-typed", classes for these quantities.
> > - runs an Linux with rt-preemt patches or xenomai
>
I would suggest to start with rt-preempt and go to xenomai only if you need.
> > Any comments welcome.
>
Hope this helps.