typekit's typelib registry

Hello,

On my amd64 host, I cross-compile my typekit and `make install` it to
/home/sagar/foo/bar/orocos/install

Then I

cd /home/sagar/foo/bar/orocos;
rsync -aP . sagar@gumstix:orocos/;
(above command copys the orocos dir, together with the install dir to
the target embedded platform, but not at the same location as on the host)

Now, when I start the deployer on the target platform, I get a bunch of
messages like

-----

1.666 [ ERROR ][TypekitRepository::Import] cannot load the typekit's
Typelib registry from
1.666 [ ERROR ][TypekitRepository::Import]
/home/sagar/foo/bar/orocos/install/share/orogen/MortDataTypes.tlb
1.667 [ ERROR ][TypekitRepository::Import] remember to do 'make
install' before you use the oroGen-generated libraries ?

-----

Thus, it looks for the typelib's .tlb file at the same absolute path as
on the host. I could of course create symlinks on the target platform,
or even use the exact same filesystem paths on the host and target. But
is there a cleaner way to get rid of these errors?

Thanks in advance,
Sagar

typekit's typelib registry

On 05/14/2013 11:04 PM, Sagar Behere wrote:
> Hello,
>
> On my amd64 host, I cross-compile my typekit and `make install` it to
> /home/sagar/foo/bar/orocos/install
>
> Then I
>
> cd /home/sagar/foo/bar/orocos;
> rsync -aP . sagar@gumstix:orocos/;
> (above command copys the orocos dir, together with the install dir to
> the target embedded platform, but not at the same location as on the host)
>
> Now, when I start the deployer on the target platform, I get a bunch of
> messages like
>
> -----
>
> 1.666 [ ERROR ][TypekitRepository::Import] cannot load the typekit's
> Typelib registry from
> 1.666 [ ERROR ][TypekitRepository::Import]
> /home/sagar/foo/bar/orocos/install/share/orogen/MortDataTypes.tlb
> 1.667 [ ERROR ][TypekitRepository::Import] remember to do 'make
> install' before you use the oroGen-generated libraries ?
>
> -----
>
> Thus, it looks for the typelib's .tlb file at the same absolute path as
> on the host. I could of course create symlinks on the target platform,
> or even use the exact same filesystem paths on the host and target. But
> is there a cleaner way to get rid of these errors?

when I created the proper symlinks, I do not get the errors above, but
the deployer segfaults during startup. The last line in orocos.log is
(MortDataTypes is the name of my Typekit, which I cross-compiled without
corba..)

1.332 [ Info ][TypekitRepository::Import] Loading Transport
Typelib:///orogen/MortDataTypes.

I do not have gdb on the embedded target, so I did `strace
deployer-xenomai` and the last few lines are

----
open("/home/sagar/foo/bar/orocos/install/share/orogen/MortDataTypes.tlb", O_RDONLY|O_LARGEFILE)
= 9
read(9, "<?xml version=\"1.0\"?>\n<typelib>\n"..., 8191) = 6011
read(9, "", 8191) = 0
futex(0xb3cd0778, FUTEX_WAKE_PRIVATE, 2147483647) = 0
gettimeofday({1368566882, 271203}, NULL) = 0

typekit's typelib registry

At runtime, typelib's tlb file is only required for the mqueue transport
and rock's data logger. If you don't need those, you can disable.

Typelib is, indeed, not knowing about cross compilation, which means:

1/ on the compilation side, we were so far setting the
CMAKE_INSTALL_PREFIX to the target's install path and then use DESTDIR
to install locally on a 'target install folder'. Unfortunately, autoproj
does not support this natively. It is a very small change, and we need
to improve the cross-compilation workflow anyways. If there is a better
method, I am all ears.

2/ on the generation side, the tlb is generated using the host
architecture, which means that it simply cannot work on a different
target. Fixing that while using gccxml seemed to be really a pain (you
have to recompile gccxml for the target architecture), so it will wait
for a clang-based replacement of the current gccxml parser.

typekit's typelib registry

On 05/15/2013 10:17 AM, Sylvain Joyeux wrote:
> At runtime, typelib's tlb file is only required for the mqueue transport
> and rock's data logger. If you don't need those, you can disable.
>
> Typelib is, indeed, not knowing about cross compilation, which means:
>
> 1/ on the compilation side, we were so far setting the
> CMAKE_INSTALL_PREFIX to the target's install path and then use DESTDIR
> to install locally on a 'target install folder'. Unfortunately, autoproj
> does not support this natively. It is a very small change, and we need
> to improve the cross-compilation workflow anyways. If there is a better
> method, I am all ears.
>
> 2/ on the generation side, the tlb is generated using the host
> architecture, which means that it simply cannot work on a different
> target. Fixing that while using gccxml seemed to be really a pain (you
> have to recompile gccxml for the target architecture), so it will wait
> for a clang-based replacement of the current gccxml parser.

Is it possible to generate a target architecture .tlb using a target
architecture binary executing on the host? I use qemu to transparently
execute arm binaries (both static and dynamic) on my amd64 host.

See, for example (some info out of date)
http://wiki.debian.org/QemuUserEmulation

Regards,
Sagar

typekit's typelib registry

On 05/15/2013 10:23 AM, Sagar Behere wrote:
> On 05/15/2013 10:17 AM, Sylvain Joyeux wrote:
>> At runtime, typelib's tlb file is only required for the mqueue transport
>> and rock's data logger. If you don't need those, you can disable.
>>
>> Typelib is, indeed, not knowing about cross compilation, which means:
>>
>> 1/ on the compilation side, we were so far setting the
>> CMAKE_INSTALL_PREFIX to the target's install path and then use DESTDIR
>> to install locally on a 'target install folder'. Unfortunately, autoproj
>> does not support this natively. It is a very small change, and we need
>> to improve the cross-compilation workflow anyways. If there is a better
>> method, I am all ears.
>>
>> 2/ on the generation side, the tlb is generated using the host
>> architecture, which means that it simply cannot work on a different
>> target. Fixing that while using gccxml seemed to be really a pain (you
>> have to recompile gccxml for the target architecture), so it will wait
>> for a clang-based replacement of the current gccxml parser.
>
> Is it possible to generate a target architecture .tlb using a target
> architecture binary executing on the host? I use qemu to transparently
> execute arm binaries (both static and dynamic) on my amd64 host.
Definitely. The only thing left will then be to modify typelib to make
the gccxml command configurable (it is currently hardcoded in
bindings/ruby/lib/typelib/gccxml.rb 577), and have a way to pass that
information through typegen and the CMake stuff.

What would be best in your opinion ? Environment variable ? Command-line
option ?

Sylvain

typekit's typelib registry

On 05/15/2013 10:28 AM, Sylvain Joyeux wrote:
> On 05/15/2013 10:23 AM, Sagar Behere wrote:
>> On 05/15/2013 10:17 AM, Sylvain Joyeux wrote:
>>> At runtime, typelib's tlb file is only required for the mqueue transport
>>> and rock's data logger. If you don't need those, you can disable.
>>>
>>> Typelib is, indeed, not knowing about cross compilation, which means:
>>>
>>> 1/ on the compilation side, we were so far setting the
>>> CMAKE_INSTALL_PREFIX to the target's install path and then use DESTDIR
>>> to install locally on a 'target install folder'. Unfortunately, autoproj
>>> does not support this natively. It is a very small change, and we need
>>> to improve the cross-compilation workflow anyways. If there is a better
>>> method, I am all ears.
>>>
>>> 2/ on the generation side, the tlb is generated using the host
>>> architecture, which means that it simply cannot work on a different
>>> target. Fixing that while using gccxml seemed to be really a pain (you
>>> have to recompile gccxml for the target architecture), so it will wait
>>> for a clang-based replacement of the current gccxml parser.
>>
>> Is it possible to generate a target architecture .tlb using a target
>> architecture binary executing on the host? I use qemu to transparently
>> execute arm binaries (both static and dynamic) on my amd64 host.
> Definitely. The only thing left will then be to modify typelib to make
> the gccxml command configurable (it is currently hardcoded in
> bindings/ruby/lib/typelib/gccxml.rb 577), and have a way to pass that
> information through typegen and the CMake stuff.
>
> What would be best in your opinion ? Environment variable ? Command-line
> option ?

I normally use 'ccmake -i' which gives me a decent ncurses UI where I
can set CMAKE options, paths etc. If the path to the gccxml command can
be set that way, that would be nice. For example, that is the way to set
the path to TYPEGEN_EXE in the current build system. This way, cmake can
pick it up even from an env. variable, if one is specified.

I will be glad to test this, if you like.

/Sagar

typekit's typelib registry

Hi Sagar,

On Tue, May 14, 2013 at 11:34 PM, Sagar Behere <sagar [dot] behere [..] ...>wrote:

> On 05/14/2013 11:04 PM, Sagar Behere wrote:
>
>> Hello,
>>
>> On my amd64 host, I cross-compile my typekit and `make install` it to
>> /home/sagar/foo/bar/orocos/**install
>>
>> Then I
>>
>> cd /home/sagar/foo/bar/orocos;
>> rsync -aP . sagar@gumstix:orocos/;
>> (above command copys the orocos dir, together with the install dir to
>> the target embedded platform, but not at the same location as on the host)
>>
>> Now, when I start the deployer on the target platform, I get a bunch of
>> messages like
>>
>> -----
>>
>> 1.666 [ ERROR ][TypekitRepository::Import] cannot load the typekit's
>> Typelib registry from
>> 1.666 [ ERROR ][TypekitRepository::Import]
>> /home/sagar/foo/bar/orocos/**install/share/orogen/**MortDataTypes.tlb
>> 1.667 [ ERROR ][TypekitRepository::Import] remember to do 'make
>> install' before you use the oroGen-generated libraries ?
>>
>> -----
>>
>> Thus, it looks for the typelib's .tlb file at the same absolute path as
>> on the host. I could of course create symlinks on the target platform,
>> or even use the exact same filesystem paths on the host and target. But
>> is there a cleaner way to get rid of these errors?
>>
>
This is typelib not knowing about cross compilation. This needs to be fixed
in that library, by using the correct cmake variables.

>
> when I created the proper symlinks, I do not get the errors above, but the
> deployer segfaults during startup. The last line in orocos.log is
> (MortDataTypes is the name of my Typekit, which I cross-compiled without
> corba..)
>
> 1.332 [ Info ][TypekitRepository::Import] Loading Transport
> Typelib:///orogen/**MortDataTypes.
>
> I do not have gdb on the embedded target, so I did `strace
> deployer-xenomai` and the last few lines are
>
> ----
> open("/home/sagar/foo/bar/**orocos/install/share/orogen/**MortDataTypes.tlb",
> O_RDONLY|O_LARGEFILE) = 9
> read(9, "<?xml version=\"1.0\"?>\n<typelib>\**n"..., 8191) = 6011
> read(9, "", 8191) = 0
> futex(0xb3cd0778, FUTEX_WAKE_PRIVATE, 2147483647) = 0
> gettimeofday({1368566882, 271203}, NULL) = 0

typekit's typelib registry

On 05/14/2013 11:43 PM, Peter Soetens wrote:
> Hi Sagar,
>
>
> On Tue, May 14, 2013 at 11:34 PM, Sagar Behere <sagar [dot] behere [..] ...
> <mailto:sagar [dot] behere [..] ...>> wrote:
>
> On 05/14/2013 11:04 PM, Sagar Behere wrote:
>
> Hello,
>
> On my amd64 host, I cross-compile my typekit and `make install`
> it to
> /home/sagar/foo/bar/orocos/__install
>
> Then I
>
> cd /home/sagar/foo/bar/orocos;
> rsync -aP . sagar@gumstix:orocos/;
> (above command copys the orocos dir, together with the install
> dir to
> the target embedded platform, but not at the same location as on
> the host)
>
> Now, when I start the deployer on the target platform, I get a
> bunch of
> messages like
>
> -----
>
> 1.666 [ ERROR ][TypekitRepository::Import] cannot load the
> typekit's
> Typelib registry from
> 1.666 [ ERROR ][TypekitRepository::Import]
> /home/sagar/foo/bar/orocos/__install/share/orogen/__MortDataTypes.tlb
> 1.667 [ ERROR ][TypekitRepository::Import] remember to do 'make
> install' before you use the oroGen-generated libraries ?
>
> -----
>
> Thus, it looks for the typelib's .tlb file at the same absolute
> path as
> on the host. I could of course create symlinks on the target
> platform,
> or even use the exact same filesystem paths on the host and
> target. But
> is there a cleaner way to get rid of these errors?
>
>
> This is typelib not knowing about cross compilation. This needs to be
> fixed in that library, by using the correct cmake variables.
>
>
> when I created the proper symlinks, I do not get the errors above,
> but the deployer segfaults during startup. The last line in
> orocos.log is (MortDataTypes is the name of my Typekit, which I
> cross-compiled without corba..)
>
> 1.332 [ Info ][TypekitRepository::Import] Loading Transport
> Typelib:///orogen/__MortDataTypes.
>
> I do not have gdb on the embedded target, so I did `strace
> deployer-xenomai` and the last few lines are
>
> ----
> open("/home/sagar/foo/bar/__orocos/install/share/orogen/__MortDataTypes.tlb",
> O_RDONLY|O_LARGEFILE) = 9
> read(9, "<?xml version=\"1.0\"?>\n<typelib>\__n"..., 8191) = 6011
> read(9, "", 8191) = 0
> futex(0xb3cd0778, FUTEX_WAKE_PRIVATE, 2147483647) = 0
> gettimeofday({1368566882, 271203}, NULL) = 0
> --- SIGSEGV (Segmentation fault) @ 0 (0) ---
> +++ killed by SIGSEGV +++
> Segmentation fault
> -----
>
> the MortDataTypes.tlb file is attached if it helps.
>
> The weird thing is, without the proper filesystem path, I get the
> bunch of errors originally noted, yet my application seems to be
> working, and the taskbrowser shows the correct types from the
> typelib as the DataFlowPort datatypes (i.e. I don't see (unknown_t)
> there)
>
> This is the toolchain-2.6 branch, cross-compiled for armel on an
> amd64 host machine.
>
> What steps to take to debug this?
>
>
> We'd have to check if typelib is actually useful in a cross-compiled
> environment (maybe the .tlb reflects host instead of target).

Does typelib have anything to do with orogen/typegen? I am actually
using the host's (amd64) orogen/typegen, even if the rest of the system
is cross-compiled (my cross-compilation method[1] fails to build orogen,
so I simply complied the toolchain natively in another dir to get access
to the orogen tool, assuming that all it does is generate c++ code
templates). Could that be the cause?

> Maybe try to add -n corba,typelib instead of -n corba and it will
> disable typelib support for you...until we get this sorted out.

Adding -n corba,typelib caused the build of the mqueue transport to fail
(apparently it looked for some typelib headers) so I did -n
corba,typelib,mqueue which left me with only
libMortDataTypes-typekit-xenomai.so

However, that worked. The deployer no longer segfaults :)

I wonder if compiling the typelib, corba and mqueue transports natively
and then make install'ing them into the same orocos/install/ folder as
the cross-compiled code will work. I'll try that tomorrow. Fortunately,
one does not have to compile typekits as often as application code.

Thanks and regards,
Sagar

[1]: Cross-compiling the orocos toolchain:
http://techblog.sagar.se/blog/2012/06/30/cross-compiling-the-orocos-tool...