Question about delimited string behavior

[felixp7]

Having recently started work on a project that will rely extensively on delimited strings, I noticed something that could be just as easily a quirk or a bug. Running the following code:

FOR I$ IN "1 2 \"3 4\" 5 6"
PRINT I$
NEXT
prints out the expected results:

1
2
"3 4"
5
6

But if I insert a space after the closing quote:

FOR I$ IN "1 2 \"3 4 \" 5 6"
PRINT I$
NEXT

BaCon no longer sees it...

1
2
"3 4 " 5 6

...unless I double it, in which case it functions normally again.

FOR I$ IN "1 2 \"3 4 \"\" 5 6"
PRINT I$
NEXT
To wit:

1
2
"3 4 ""
5
6

So, is this intended behavior? Because it surprised me at first. At least BaCon preserving the double quotes sort of makes sense, and is easy to work around.

Configuration: BaCon 3.9.2 over GCC 7.3.0 on 64-bit. This happens under OPTION QUOTED (of course), with and without OPTION COLLAPSE.

[Pjot]

Thanks for letting know felixp7,

I am looking into this issue and will keep you updated.

BR
Peter

[Pjot]

So I found the cause of this weird behavior. Again it has to do with legacy code, which was created at the time when BaCon did not have any awareness of delimited strings at all (version 2.2 in 2013).

Back then, the idea was to have the possibility to step through elements in a string, in which those elements are space separated. When the more advanced implementation of the delimited string engine came in - later, in 2016 - the code of the FOR..IN implementation was never changed to make use of this very same engine.

To show my point, if you run this code:

FOR i = 1 TO 5
   PRINT TOKEN$("1 2 \"3 4 \" 5 6", i)
NEXT


You will see that the results are as expected. However, TOKEN$ uses the extensively tested delimited strings core engine which I just mentioned.

In the upcoming 3.9.3 release all the legacy code in FOR..IN was replaced, so the FOR..IN function now also uses that very same delimited string core engine, and the results now are in line with all the other delimited string functions.

If you fetch the latest beta it should work fine.

BR
Peter