howto split ARGUMENT$ properly?

[rikky]

hello

I have this variable in bash containing newline, quote and single quote :

blabla="This is a single quote: '
This is a double quote: "'"'

echo "$blabla"
response :

This is a single quote: '
This is a double quote: "
I have a (ba)sh script ARGUMENT.sh : 

#!/bin/sh

# store arguments in a special array
args=("$@")
# get number of elements
ELEMENTS=${#args[@]}

# echo each element in array  
# for loop
for (( i=0;i<$ELEMENTS;i++)); do
  echo
  echo "$i:"    
  echo "${args[${i}]}"
done
I can do ARGUMENT.sh with the weirdest arguments, and it gives the right output :

ARGUMENT.sh  "$blabla" "hela hola" twee
output :

0:
This is a single quote: '
This is a double quote: "

1:
hela hola

2:
twee
However, whatever I try with BaCon, I cannot get something useful together.
I tried various methods, I'll put some compilation below :

'OPTION QUOTED TRUE
'OPTION QUOTED FALSE
'OPTION DELIM CHR$(1)

PRINT
PRINT "ARGUMENT: " & ARGUMENT$
PRINT

'FOR i = 1 TO AMOUNT(ARGUMENT$,CHR$(1))
'  PRINT
'  PRINT STR$(i) & ":"
'  PRINT TOKEN$(ARGUMENT$,i,CHR$(1))
'NEXT i

FOR i = 1 TO AMOUNT(ARGUMENT$)
 PRINT
 PRINT STR$(i) & ":"
 PRINT TOKEN$(ARGUMENT$,i)
NEXT i

The output of the above is:

ARGUMENT: ARGUMENT "This is a single quote: '
This is a double quote: "" "hela hola" twee

1:
ARGUMENT

2:
"This is a single quote: '
This is a double quote: "" "hela

3:
hola" twee

I get a lot of quotes too many, and OPTION DELIM unfortunately doesn't work BEFORE the ARGUMENT$ is grapped by BaCon.

 Rik.

EDIT PS: The output from the above BaCon program  is for me completely illogical. 

EDIT 2  : made the BaCon script output nicer, but didnot change the program itself.

[vovchik]

Dear Rik,

In c we have argc and argv - and it seems that ARGUMENT$ is similar:

DECLARE i TYPE int
DECLARE myarg$ TYPE STRING
FOR i = 0 TO argc - 1
myarg$ = (char*)argv[i]
PRINT myarg$
NEXT i
PRINT ARGUMENT$


Compile as argtest and run in a terminal:

export blabla="silly"; ./argtest "$blabla" "hela hola"

With kind regards,
vovchik

[Pjot]

Hi rikky,

As vovchik has mentioned in the previous post, we can use the C standard argv[] array as well (undocumented).

But there is a slight difference here, as BASH uses an plain array to store the arguments and BaCon uses a delimited string.

Nevertheless, it would be nice if we could parse the arguments in a delimited string also. The problem comes from the situation that one of the arguments contains a "-symbol.

As I am in the process of redesigning the internals of BaCon, where ARGUMENT$ is on my list also, I now have redesigned the argument handling in such a way that it (1) no longer uses global help variables and (2) also will 'unflatten' any delimiter symbols if they are found.

If you obtain the new beta then you'll see the difference. This is my output now:

# cat rikky.bac
PRINT "ARGUMENT: -", ARGUMENT$, "-"
PRINT "====================="
PRINT TOKEN$(ARGUMENT$, 1)
PRINT "====================="
PRINT TOKEN$(ARGUMENT$, 2)
PRINT "====================="
PRINT TOKEN$(ARGUMENT$, 3)
PRINT "====================="
PRINT TOKEN$(ARGUMENT$, 4)
#
#
# ./rikky "$blabla" "hela hola" twee
=====================
ARGUMENT: -./rikky "This is a single quote: '
This is a double quote: \"" "hela hola" twee-
=====================
./rikky
=====================
"This is a single quote: '
This is a double quote: \""
=====================
"hela hola"
=====================
twee


If you need to use the argument without the \" please use the FLATTEN$ function to get rid of all the double quotes. E.g.:

PRINT FLATTEN$(TOKEN$(ARGUMENT$, 2))


Best regards
Peter

[rikky]

Ah,
FOR i = 1 TO AMOUNT(ARGUMENT$)
 PRINT
 PRINT STR$(i) & ":"
 PRINT FLATTEN$(TOKEN$(ARGUMENT$,i))
NEXT i

output is now:

1:
/home/pi/test

2:
This is a single quote: '
This is a double quote: "

3:
hela hola

4:
twee

Thanks.

 Rik.

[rikky]

So Vovchik send me a jason text file to test.
And I had already noticed that in Jason texts there is sometimes also a "discription" : "something"
And this "something" might contain backslashed characters like the quote, \"

So I adapted the testfile with a backslashed quote

{"widget": {
   "debug": "on",
   "window": {
       "title": "Sample Konfabulator Widget",
       "name": "main_window",
       "width": 500,
       "height": 500
       "description": "discription with a quote\""  
}}}  

And then tried to UNJSON it the stupid way

UNJASON "$(cat file)"

so basicaly the c program from above gives the right answer.
The Vovchik solution from above also gives the right answer.

VOVCHIK.bac
DECLARE i TYPE int
DECLARE arg$ TYPE STRING
FOR i = 0 TO argc - 1
arg$ = (char*)argv[i]
PRINT
PRINT STR$(i) & ":"
PRINT arg$
NEXT i

VOVCHIK "$(cat file)" 
response:

0:
VOVCHIK

1:
{"widget": {
   "debug": "on",
   "window": {
       "title": "Sample Konfabulator Widget",
       "name": "main_window",
       "width": 500,
       "height": 500
       "description": "discription with a quote\""  
}}}
The ARGUMENT.bac program from above however crasches.

ARGUMENT "$(cat file)"
response:

1 :
ARGUMENT

2 :
{"widget": {
   "debug": "on",
   "window": {
       "title": "Sample Konfabulator Widget",
       "name": "main_window",
       "width": 500,
       "height": 500
       "description": "discription with a quote\""

3 :


4 :

}}}

5 :


6 :

 Rik.

[Pjot]

Hi rikky,

Well, for me the program does not crash. Which BaCon version are you using? On my machine the result is:

{"widget": {
   "debug": "on",
   "window": {
       "title": "Sample Konfabulator Widget",
       "name": "main_window",
       "width": 500,
       "height": 500
       "description": "discription with a quote\""

3:


4:

}}}


And believe it or not, this is the correct output. Reason is that the JSON file contains spaces at the end of the line with "description".

But if you remove those spaces then your program works as it should (corrected JSON file attached).

Your confusion seems to come from mixing up arrays versus delimited strings.

The program from vovchik handles the arguments to the program as elements in an array. Printing the second element in the array of arguments will print the contents of the original JSON file.

Your program however uses TOKEN$, and therefore, it looks at the ARGUMENT$ variable as a delimited string. Obviously, the result will be the element based on the default delimiter, which is a single space. Therefore it returns 4 tokens because there are spaces at the end of the "description" line.

Alternatively, you can capture all tokens except the first one, using the LAST$ function.

PRINT FLATTEN$(LAST$(ARGUMENT$, 1))


This will return all items in the delimited string except the first one.

HTH
Peter

PS note that my JSON parser also invokes a cleaning function called "Cleanup_JSON$" which first removes all redundant spaces from the JSON string.

Attachments:

rikky.json (227 B)

[rikky]

Well, there is also the most simple bash ARGUMENT program,  in this case:

echo $1
which also gives the right answer.

Note that there are quotes around the "$(cat file)"
And note that if you remove the backslash out of the file, you can put as many quotes and spaces in there as you want, and they still make part of the TOKEN$(ARGUMENT$(2))

And what I was looking for was a way to beat bash.
(The UNJASON thing is just a continuation of that project.)
For if you want to do delimited string things in bash you're doomed.
It's horribly difficult.
I think I eventually succeeded with the BaCon4bash4dummy's project, even though I might be
the only one using it.

Thing is that BaCon4bash4dummy's s*cks, if the ARGUMENT$ doesn't behave the way it should behave.
How can you beat sed and awk, if those programs receive other information then that I get?
If there is a backslash on the commandline, I want to know about it.
I do NOT want to get other arguments then that are meant in the commandline.

export QUOTED=FALSE
TOKEN "$(cat file1)" 2 "$(cat file2)"  
means that I get the second part of file1, where the contents of file2 is the delimiter.

Try this in bash or sed or awk.

Now if one of these files contains a backslashed quote the whole project falls apart,
NOT only for TOKEN but also for all the other commands.
You can not have something that works for 98%.

So the newest VOVCHIK variant of TOKEN actually works.
The ARGUMENT variant doesn't

I have a lot of updating to do.

Without BaCon, by the way, I would not have been able to compete with the bash cracks anyway, 

So Thanks
 Rik.

EDIT:
TOKEN "$(cat file1)" "$(cat file2)" 2    =>   TOKEN "$(cat file1)" 2 "$(cat file2)"  

Attachments:

TOKEN.tgz (4.08 KB)

[rikky]

It's 5 in the morning and I shock waked up.

I suddenly see what you mean.
Finally grasped it.

In order to get BaCon to do what it already does with the ARGUMENT$, BaCon is treating it as a delimited string with an OPTION QUOTED to TRUE.

The quoted backslash realy ruins this.

What I want is simply not possible. 

 Rik.

[Pjot]

Dec 27, 2020 5:32:06 GMT 1 rikky said:

It's 5 in the morning and I shock waked up.

I suddenly see what you mean.
Finally grasped it.

Sounds to me that programming is bad for you Though I recognize the experience myself as well, of course. The best ideas come up in bed, shower or the bathroom

The reason that BASH works like this is that it looks at arguments in terms of an array.

For BaCon, the ARGUMENT$ primarily is a string with all arguments in plain text. So by nature it is not an array. You can use the good old string functions, or also delimited string functions. But the behavior is different.

BR
Peter

[bigbass]

Hello Rik

This is the same code as vovchik's example
I just used WHILE WEND and cheated with an alias $
so that we are just looking at it another way for a demo

and we can remove the cat command also


call this split.bac

ALIAS argv TO ARGS$

i = 1
WHILE (i <= argc -1)
PRINT i -1 ,":"
PRINT  ARGS$[i], NL$
INCR i
WEND



call this j.txt

{"widget": {
   "debug": "on",
   "window": {
       "title": "Sample Konfabulator Widget",
       "name": "main_window",
       "width": 500,
       "height": 500
       "description": "discription with a quote\""  
}}}  



then the run code would be
./split  $(<j.txt)



an extra easy way to get key value pairs
grep '"name":' j.txt


or
grep '"description":' j.txt


sed can do this also
replace the comma for a newline

sed 's/,/\n/g' <<< $(< j.txt)



Joe

[rikky]

Nice.

So now it becomes :

ALIAS argv TO ARGS$
ALIAS argc-1 TO ARGLEN
After which you have your arguments available throughout your program in the form of ARGS$[<number>]

Except then for inside SUB's and FUNCTION's, I noticed.
For then you get an error: 'argv' undeclared (first use in this function)

So for inside SUB/FUNCTIONS you still need another variable.

Putting them into another array is not possible (for me)

DECLARE array$[ARGLEN]

error: 'argc' undeclared here (not in a function)

and declaring argc first gives : error: variably modified 'array__b2c__string_var' at file scope

Also ARGLEN cannot be used inside SUB/FUNCTIONS : error: 'argc' undeclared (first use in this function)

NOR being declared globaly GLOBAL ARGLEN :  error: lvalue required as left operand of assignment

SO that leaves the associative array.
OPTION EXPLICIT TRUE

ALIAS argv TO args$
ALIAS argc-1 TO arglen

GLOBAL var
DECLARE ARGS$ ASSOC STRING

var = 1
WHILE (var <= arglen)
 ARGS$(STR$(var)) = args$[var]
 INCR var
WEND

GLOBAL ARGLEN
ARGLEN = NRKEYS(ARGS$)
And now it is usable inside SUB/FUNCTIONS also in the form of ARGS$("<number>")
It doesn't look as nice though as ARGS$[<number>]


sed 's/,/\n/g' <<< $(< j.txt)   =>=>=>  REPLACE "," $'\n' <<< $(<j.txt) :

{"widget": {
   "debug": "on"

   "window": {
       "title": "Sample Konfabulator Widget"

       "name": "main_window"

       "width": 500

       "height": 500
       "description": "discription with a quote\""  
}}}

 Rik.

[vovchik]

Dear Rik,

What about this?

ALIAS argv TO ARGS$
ALIAS argc TO ARGCNT
DEF FN ARGC() = argc
DEF FN ARGPARSE$(i) = IIF$(ARGC(), ARGS$[i], "")

' ------------------
FUNCTION PARSEARG$(int ARGCNT, STRING ARGS$[])
' ------------------
LOCAL i TYPE int
LOCAL result$ TYPE STRING
result$ = NL$
FOR i = 1 TO ARGCNT - 1
result$ = result$ & NL$ & ARGS$[i]
NEXT i
RETURN CHOP$(result$)
END FUNCTION

PRINT PARSEARG$(ARGCNT, ARGS$)

I think all of this should work. Joe might have an idea or two, as well...and Peter, of course.

With kind regards,
vovchik

[rikky]

ALIAS argv TO ARGS$
ALIAS argc TO ARGLEN
DEF FN ARGC() = argc
DEF FN ARGPARSE$(i) = IIF$(ARGC(), ARGS$[i], "")

FUNCTION ARGSPRINT$(var,ARGS$[])
 
 RETURN ARGS$[var]
 
END FUNCTION

SUB PRINTARGS$(var,ARGS$[])

 PRINT ARGS$[var]
 
END SUB

PRINT
FOR i = 1 TO ARGLEN -1
 PRINT ARGS$[i]
NEXT i

PRINT
FOR i = 1 TO ARGLEN -1
 PRINT ARGSPRINT$(i,ARGS$)
NEXT i

PRINT
FOR i = 1 TO ARGLEN -1
 PRINTARGS$(i,ARGS$)
NEXT i

PRINT

./test "1 2 \"3" "2" 3
response:

1 2 "3
2
3

1 2 "3
2
3

1 2 "3
2
3
 Rik.

[bigbass]

Hello Rik
Glad to see you are getting
good results




vovchik the function works for me also
thanks looks and works great

I did a TOTAL_SIZE macro for c++
some time ago cant remember why I had to do it
it was in the last post of basic-converter.proboards.com/post/13757

now I see the dot member syntax wasn't
working in c++

Joe

[alexfish]

Hello All

here is a search char function I use with bacon

it returns rest of string beyond the char found

have fun
+
BR
Alex


LOCAL B$ TYPE STRING
B$ = B$ & "test" & CHR$(34) & " these " & CHR$(39) & " bits"
PRINT B$

LOCAL C$ TYPE STRING

C$ = strchr(B$,39)

IF LEN(C$) THEN
PRINT C$
ELSE
PRINT "not found"
END IF

C$ = strchr(B$,34)

IF LEN(C$) THEN
PRINT C$
ELSE
PRINT "not found"
END IF