|
Post by Pjot on Aug 25, 2019 20:07:38 GMT 1
Hi Joe, After a long debugging session I found the cause of the issue with the CLang compiler. It turns out that GCC and CLang parse a line of C code in a different way. The GCC compiler starts from the very end of the line working towards the beginning, while CLang performs its parsing the other way around.
This has consequences for the code at lines 151 and 153. In there, the string concatenation includes a single CHOP$ function, and a custom function. Both use the intermediate string result buffers. So in GCC, first the custom function is executed, using many of the buffers, and the result is returned, and after this the CHOP$ function, which only uses one buffer. In CLang first the CHOP$ is evaluated, and then the custom function, using many of the buffers, thereby overwriting the temporary result from CHOP$. So the program works with GCC but this is pure coincidence, because if the functions were swapped, the situation would be different, and the JSON parsing would fail with GCC in the same manner as with CLang.
The problem only happens in case of string concatenations. It reveals a flaw in the way BaCon handles temporary results from functions, and the implementation needs a revision to solve this issue in a structural way.
For now I made a workaround, and the updated JSON parser actually should produce the correct result when compiled with CLang. Regarding your request: according to this JSON website the layout of a JSON string may or may not contain spaces. Your code should be able to handle any kind of formatting. I try to solve this problem in my parser by deleting all superfluous markup bytes like carriage returns, newlines, spaces etc before starting the actual parsing. Thanks again for pointing out the CLang problems, Best regards Peter
|
|
|
Post by bigbass on Aug 26, 2019 3:56:27 GMT 1
Hello Peter Your code works correctly with CLang now Thanks for all your hard work on this it is very useful to have JSON readable and parsed in bacon with your baconized parser functions! I guess for fun or I just wanted to understand JSON and see if it could be done in c++ with bacon there were a few things that had to be done first 1.) a c++ here doc that could be assigned to a string 2.) a way to SPLIT and filter that RAW c++ string and work with bacon 3.) make a simple demo of JSON beautifier The code was commented to explain the work arounds in more detail This website was used as the reference of what the output should look like jsonformatter.org/c55141A big Thanks again for getting it to work with CLang! Joe PRAGMA INCLUDE <iostream> PRAGMA INCLUDE <ostream> PRAGMA INCLUDE <string>
PRAGMA LDFLAGS PRAGMA COMPILER g++ PRAGMA OPTIONS -Wno-write-strings -Wno-pointer-arith -fpermissive 'PRAGMA BACONLIB OPTION PARSE FALSE
'--- raw-string literal example so that we dont read in a file '--- can now be used also as a "here doc" saved as the variable json '--- std::string json = R"( some multi line data)"; std::string json = R"({ "name":"John", "age":30,"cars": \ [{ "name":"Ford", "models": [ "Fiesta", "Focus", "Mustang" ] } \ ,{ "name":"BMW", "models": [ "320", "X3", "X5" ] } \ ,{ "name":"Fiat", "models": [ "500", "Panda" ] }]})" ; '--- show that we can output this easily in c++ std::cout << json << "\n"; '--- we can cheat if we use a string pointer ptr '--- to the RAW string address from c++ DECLARE *ptr TYPE STRING ptr = &json
'--- how to get around the conversion of types '--- since we have the addressed memory already allocated in c++ '--- We can now SPLIT a raw string literal in BaCon '--- if we point to the address of the raw string literal '--- declared as a STRING a nice discovery PRINT PRINT "=========*ptr to raw string literal JSON beautifier===========" PRINT LOCAL dimension
SPLIT *ptr BY "," TO array$ SIZE dimension FOR i = 0 TO dimension -1 '--- a simplified JSON beautifier filtered$= REPLACE$(array$[i], "{", "{ \n") filtered$= REPLACE$(filtered$, "}", "\n }") filtered$= REPLACE$(filtered$, "[", "[ \n") filtered$= REPLACE$(filtered$, "]", "\n ] ") PRINT filtered$ NEXT
' Declare array DECLARE Cars$ ASSOC STRING
' Initialize some name value pairs Cars$("item0") = array$[0] Cars$("item1") = array$[1] Cars$("item2") = array$[2] Cars$("item3") = array$[3] Cars$("item4") = array$[4] Cars$("item5") = array$[5]
' Lookup the index names (item1, item2, item3) LOOKUP Cars$ TO item$ SIZE size
PRINT PRINT "========= a bacon associative array special use =====" PRINT
i = 4 PRINT Cars$(item$[i-1]) & Cars$(item$[i]) & Cars$(item$[i+1])
PRINT PRINT "========= indexed associative array ===============" PRINT ' Print values we could filter easily here also if needed ' you usually use i = 0 but wanted to start on the fourth array FOR i = 3 TO size-1 PRINT Cars$(item$[i]) NEXT
terminal output { "name":"John", "age":30,"cars": [{ "name":"Ford", "models": [ "Fiesta", "Focus", "Mustang" ] } ,{ "name":"BMW", "models": [ "320", "X3", "X5" ] } ,{ "name":"Fiat", "models": [ "500", "Panda" ] }]}
=========*ptr to raw string literal JSON beautifier===========
{ "name":"John" "age":30 "cars": [ { "name":"Ford" "models": [ "Fiesta" "Focus" "Mustang" ] } { "name":"BMW" "models": [ "320" "X3" "X5" ] } { "name":"Fiat" "models": [ "500" "Panda" ] } ] }
========= a bacon associative array special use =====
"models": [ "Fiesta" "Focus" "Mustang" ] }
========= indexed associative array ===============
"models": [ "Fiesta" "Focus" "Mustang" ] }
------------------ (program exited with code: 0) Press return to continue
|
|
|
Post by bigbass on Aug 27, 2019 19:06:11 GMT 1
some improvements
the important part was to auto generate a bacon associative string array from the raw string literal
which converts array$[] to a Cars$("item") automatically as a numbered string
and a HASDELIM example too
PRAGMA INCLUDE <iostream> PRAGMA INCLUDE <ostream> PRAGMA INCLUDE <string>
PRAGMA LDFLAGS PRAGMA COMPILER g++ PRAGMA OPTIONS -Wno-write-strings -Wno-pointer-arith -fpermissive PRAGMA BACONLIB OPTION PARSE FALSE
'--- raw-string literal example so that we dont read in a file '--- can now be used also as a "here doc" saved as the variable json '--- std::string json = R"( some multi line data)"; std::string json = R"({ "name":"John", "age":30,"cars": \ [{ "name":"Ford", "models": [ "Fiesta", "Focus", "Mustang" ] } \ ,{ "name":"BMW", "models": [ "320", "X3", "X5" ] } \ ,{ "name":"Fiat", "models": [ "500", "Panda" ] }]})" ; '--- show that we can output this easily in c++ std::cout << json << "\n"; '--- we can cheat if we use a string pointer ptr '--- to the RAW string address from c++ DECLARE *ptr TYPE STRING ptr = &json ' Declare array DECLARE Cars$ ASSOC STRING
'--- how to get around the conversion of types '--- since we have the addressed memory already allocated in c++ '--- We can now SPLIT a raw string literal in BaCon '--- if we point to the address of the raw string literal '--- declared as a STRING a nice discovery PRINT PRINT "=========*ptr to raw string literal JSON beautifier===========" PRINT LOCAL dimension
SPLIT *ptr BY "," TO array$ SIZE dimension FOR i = 0 TO dimension -1 '--- a simplified JSON beautifier IF HASDELIM(array$[i] ,":") OR HASDELIM(array$[i] ,"]") THEN filtered$ = REPLACE$(array$[i], "{", "{ \n") filtered$= REPLACE$(filtered$, "}", "\n }") filtered$= REPLACE$(filtered$, "[", "[ \n") filtered$= REPLACE$(filtered$, "]", "\n ] ") PRINT filtered$ '--- make a dynamic bacon associative array '--- automatically generated for you ! itemline$ = CONCAT$("item", STR$(i)) Cars$(itemline$) = array$[i] END IF NEXT
' Lookup the index names (item1, item2, item3) LOOKUP Cars$ TO item$ SIZE size
PRINT PRINT "======automatically generated bacon ASOC STRING item10 ===========" PRINT PRINT Cars$("item10") PRINT
PRINT PRINT "========= a bacon associative array special use =====" PRINT
i = 4 PRINT Cars$(item$[i-1]) & Cars$(item$[i]) & Cars$(item$[i+1])
PRINT PRINT "========= indexed associative array ===============" PRINT ' Print values we could filter easily here also if needed ' you usually use i = 0 but wanted to start on the fourth array FOR i = 3 TO size-1 PRINT Cars$(item$[i]) NEXT
|
|
|
Post by Pjot on Aug 29, 2019 6:51:41 GMT 1
Thanks Joe, Finally I have solved this issue, the fix is simple and elegant and should work in all situations for all compilers, even in nested string concatenations on the same line of code. The downside is a slight performance penalty, depending on the amount of string concatenations available in the program. However, looking some other improvements in this beta, for example the performance improvements in LAST$ and FIRST$ and also in the RETURN statements, the overall experience should not be that bad. But I will try to improve this solution further. The original JSON parser now should compile and run correctly, both in case of GCC and CLang. Best regards Peter
|
|
|
Post by bigbass on Sept 2, 2019 16:39:26 GMT 1
Hello Peter using your latest bacon 3.9.2 and your latest JSON parser I am getting the correct output compared to your output result on page one of this thread basic-converter.proboards.com/post/12418this is good news! all is well now when using clang Joe P.S I would like to add some notes about reading your parser ASCII values used in json2delim.bac http://basic-converter.org/json2delim.bac so you can decode what is happening
-------------------------------------- ' Skip special json characters 123, 125, 91, 93, 44, 58 { } [ ] , : -------------------------------------- ' Skip digits and dot and minus 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 45, 46 0 1 2 3 4 5 6 7 8 9 - . ------------------------------------------------ ' Skip true, false, null and more 97, 101, 117, 102, 108, 110, 114, 115, 116 a e u f l n r s t ----------------------------------------------- 92 \ back slash ----------------------------------------------- 34 " double quote -----------------------------------------------
|
|
|
Post by Pjot on Sept 2, 2019 17:38:20 GMT 1
Thanks Joe, This is good news indeed I have added the actual ASCII signs in the comments of the json2delim.bac file. BR Peter
|
|
|
Post by Pjot on Sept 4, 2019 21:00:37 GMT 1
The downside is a slight performance penalty, depending on the amount of string concatenations available in the program. So I have found a better solution which also restores the original performance. Latest is in the fossil repo. BR Peter
|
|