We have a patch on trunk that fixes multi-threaded parsing on Linux, but ruins
the whole parsing on windows (the parser ends up in an endless loop or crashes
the executable).
The situation was this: there was a global variable 'skip_eol' that told all
parser objects if in the current parsing context, the end of line needed to be
skipped or not. It worked, but if two threads were parsing at the same time,
it clearly fails.
The fix was to make this a variable of the CommonParser object and pass a
reference or pointer to this object to any parser. They would then query the
CP what the current state of the context was. And all was fine again. At least
on Linux. This change causes a catastrophe when compiled with MSVS2005.
Windows Wizards: What could be the cause ?
To test this patch, apply it to 1.10.3 or 1.8... and run the program-test. It
will never return. Using boost 1.38. This patch was also part of 1.10.4 btw.
I also attached two other patches that you should apply to avoid seeing other
trouble.
Peter
| Attachment | Size |
|---|---|
| 0001-Fix-multi-threaded-parser-use-bug.patch | 38.42 KB |
| 0001-win32-turn-of-optimization-in-two-files-that-cause-c.patch | 1.5 KB |
| bisect-fixups.patch | 1.02 KB |
Good patch for Linux turns to bad patch for win32
On Thursday 12 August 2010 14:56:42 Peter Soetens wrote:
> We have a patch on trunk that fixes multi-threaded parsing on Linux, but
> ruins the whole parsing on windows (the parser ends up in an endless loop
> or crashes the executable).
>
> The situation was this: there was a global variable 'skip_eol' that told
> all parser objects if in the current parsing context, the end of line
> needed to be skipped or not. It worked, but if two threads were parsing at
> the same time, it clearly fails.
>
> The fix was to make this a variable of the CommonParser object and pass a
> reference or pointer to this object to any parser. They would then query
> the CP what the current state of the context was. And all was fine again.
> At least on Linux. This change causes a catastrophe when compiled with
> MSVS2005.
>
> Windows Wizards: What could be the cause ?
More news on this front. It appeared that even without this patch, things
still went very wrong. After some more bisecting, a grave bug in 'data source
adaptors' turned up. We were using plain pointers to shared objects, that got
freed before. It should have crashed seriously on Linux too. It's a mystery
why it doesn't.
The patch in attachment shows what was wrong. This bug only surfaced during
1.10.0, because of adding the automatic type conversion feature in the type
system. I believe older version are not affected. This would explain why MSVS
builds of 1.8 worked fine.
Peter
Good patch for Linux turns to bad patch for win32
On Thu, Aug 12, 2010 at 02:56:42PM +0200, Peter Soetens wrote:
> We have a patch on trunk that fixes multi-threaded parsing on Linux, but ruins
Huh, why on earth would you need multi-threaded parsing at all?
Performance of parsing rtt scripts and statemachines ???
Markus
Good patch for Linux turns to bad patch for win32
On Thursday 12 August 2010 16:10:17 Markus Klotzbuecher wrote:
> On Thu, Aug 12, 2010 at 02:56:42PM +0200, Peter Soetens wrote:
> > We have a patch on trunk that fixes multi-threaded parsing on Linux, but
> > ruins
>
> Huh, why on earth would you need multi-threaded parsing at all?
>
> Performance of parsing rtt scripts and statemachines ???
Loading state machines in different components at the same time can happen in a
running application (process). It had happened in the field. The other solution
was to put a big global mutex around the parsers, which doesn't work in
environments like Xenomai, unless some care is taken to delay initialisation
until after the RTT started up Xenomai services.
Peter
Good patch for Linux turns to bad patch for win32
On Thu, Aug 12, 2010 at 04:48:54PM +0200, Peter Soetens wrote:
> On Thursday 12 August 2010 16:10:17 Markus Klotzbuecher wrote:
> > On Thu, Aug 12, 2010 at 02:56:42PM +0200, Peter Soetens wrote:
> > > We have a patch on trunk that fixes multi-threaded parsing on Linux, but
> > > ruins
> >
> > Huh, why on earth would you need multi-threaded parsing at all?
> >
> > Performance of parsing rtt scripts and statemachines ???
>
> Loading state machines in different components at the same time can happen in a
> running application (process). It had happened in the field. The other solution
> was to put a big global mutex around the parsers, which doesn't work in
> environments like Xenomai, unless some care is taken to delay initialisation
> until after the RTT started up Xenomai services.
Yuck! Then #ifdef the big_parser_lock only for windows?
Markus