White space between INSTR and it's paren "(" causes (SOLVED)

[clamzonprozac]

Howdy Peter!
   Here's another one for you. I was able to contain this bug today.
  The snippet below demonstrates the problem  consistently
 (at least on my machine, in the month of July) ...

' INSTR is flagged with a compile time error depending on the
' position of the left paren... but only for the first
' instance of INSTR in StripCRLF$(). White space = gives
' error, no space = okay!

FUNCTION StripCRLF$(t$)
' Strips ending Line Feed character, or Carrage Return + Line Feed pairs from a string
IF INSTR (t$,CHR$(0x0A),LEN(t$)) > 0 THEN
PRINT "FOUND LINE FEED"
t$ = MID$(t$,1,LEN(t$)-1)
IF INSTR (t$,CHR$(0x0D),LEN(t$)) > 0 THEN
t$ = MID$(t$,1,LEN(t$)-1)
ENDIF
ENDIF
RETURN t$
END FUNCTION

foo$ = "Hello World" + CHR$(13) + CHR$(10)
goo$ = StripCRLF$(foo$)

the error message reads...
strip_crlf.bac:9:5: note: in expansion of macro 'INSTR'

Kind Regards,
-Roger

[Pjot]

Hi Roger,

Well, this is as designed - all functions in BaCon *must* have the parentheses attached.

So, it is: INSTR(...) , and not INSTR (...)

In fact, BaCon functions are defined as a macro in C. So when BaCon converts the code, it simply will copy the function names to the generated C code.

Then, the C compiler will run its macro preprocessor (CPP), which substitutes the macros. After that, GCC starts compiling and will simply exit at the first unknown token, which, in this case, is 'INSTR'.

BR
Peter

[clamzonprozac]

Hi Peter,
   I slept on it for a few days. I'll have to fall back on my experience with compilers that delete white space unless there's a good reason to keep it. So it
   wouldn't matter if one typed zero spaces or a dozen between a keyword and a "special" character, the result should be the same, IMHO.  The kind of
   behavior you describe "should" be commented on (somewhere) in the main BaCon doc.

  Breaking for a moment to explain myself: I'm a retired electronics hardware engineer who  is also a recent refugee from Windows and for the past
  nearly 40 years has written code in variants of Pascal, C, assembly language, and BASIC. The Pascal exposure may explain why I'm tend to liberally
  use parentheses.

 So, this evening I stubbed my toe on this bit of code, boiled down to be an example. Its about using parentheses in an IF conditional statement and also
 using a backslash for line continuation. If I remove the parens and keep the backslash compilation completes, if not there's no joy. Or, if I keep the parens
and omit the  backslash things also work

--------------
'line_continue.bac
GLOBAL RECORD txtrec[100]
LOCAL t_stat TYPE int
END RECORD

GLOBAL RECORD cmprec[100]
LOCAL c_stat TYPE int
END RECORD

GLOBAL i, a, b TYPE int

i = 1 : a = 2 : b = 2
txtrec.t_stat = 4
cmprec.c_stat = 4

' Both tests fail compilation (claimed missing THEN) with the addition of line continuation,
' otherwise they evaluate to TRUE.

IF (txtrec.t_stat <> -1) AND \
   (txtrec.t_stat = cmprec.c_stat) THEN
      PRINT "first test"
ENDIF

IF (a <> -1) AND \
    (a = b) THEN
      PRINT "second test"
ENDIF

--------
Now, a  few minutes later: testing a bit more I  commented out the first test, and it compiled! I removed the comment pair (/* *\) and I've not been able to recreate the problem EXCEPT in the source  where the failure was first seen. It's maddening, and  acts like there's an un-intialized "something"  going on.

Maybe the issue will show up for you?

Kind Regards,
-Roger

[Pjot]

Hi Roger,

First, your demo program doesn't compile for me at all, as it uses arrays of records. I had to remove the [100] index indications first.

Then, everything compiled with no problem, as is.

There are some things which we can check:

• Can you crosscheck the actual BaCon you are using? A lot of times people run into problems because they unknowingly use some binary which happens to be in their $PATH, and which takes precedence above the BaCon version they actually think they're using. For me, after plain copying and pasting your code from the forum into VIM, and removing the '[100]' parts, and saving - compilation works with the latest beta.
• Can you make sure there are no strange white spaces (like 0xA0, 0x01, 0x08, 0x11 etc) somewhere in your code?
• For the line continuation, it *has* to end with <space><backslash>, so literally " \", and there should really be nothing else behind that. If a tab or a space is behind that last backslash, then a compile error will occur.

HTH
Peter

[Pjot]

Thinking over your other comments:

I slept on it for a few days. I'll have to fall back on my experience with compilers that delete white space unless there's a good reason to keep it. So it wouldn't matter if one typed zero spaces or a dozen between a keyword and a "special" character, the result should be the same, IMHO. The kind of behavior you describe "should" be commented on (somewhere) in the main BaCon doc.

Thing is, the whole BASIC-to-C-converter project started in 2009, solely based on Kornshell script. There was not a BaCon version of BaCon at that moment.

So, the problem with shell scripting is, that it simply cannot distinguish whether or not a space appears in between double quotes. If a space appears between double quotes, then it is part of a string and should be left untouched.

Of course, we could have chosen for external tooling to do the bulk of the parsing. However, the intention of BaCon was (and is) to be able to use it at a variety of Unix-like platforms. Therefore, the project always tries to avoid as many external dependencies as possible. Such external tools may or may not be available at other platforms, and if they are, then they usually work in a different way than expected.

This means that BaCon has to do all parsing by itself. In shell scripting, it means that it will look at each and every byte of the source code. It is very picky when it comes to parenthesis, 'loose' spaces, position of comma's, and so on.

When BaCon evolved to a more elaborate set of instructions, it was possible to implement BaCon in itself. And a lot of functions in BaCon are more powerful compared to shell scripting. For example, the whole delimited string concept includes the automatic exception of characters in between double quotes. It can parse text with just a few functions.

However, the BaCon and shell script implementation are deliberately kept the same as much as possible. BaCon always will stick to the limitations of the shell script implementation as much as possible (there are a few exceptions where this simply cannot be done).

The second thing is, you will discover that a lot of Unix-like platforms have their peculiarities when it comes to (redundant) white space characters. A wide-known issue is the Windows '\r\n' combination, which POSIX platforms usually write as '\n', and certain versions of MacOSX as '\r'. The use of white spaces in text is not univocal and universal. BaCon *has* to be picky when it should function properly on all those platforms.

You mention you're a recent refugee of Windows. Of course, within the various Windows versions, things are easy. Basically, you always face the same platform, and compilers / converters / interpreters do not need to take such differences into account.

Now that you have converted to Unix-like platforms, you'll find that things are not that simple. A white space at an unexpected position may cause headaches. Therefore, please make sure to use a very strict and reliable editor. My choice is VIM, but I know that people use Geany or Nano as well. Please note that BaCon contains a limited built-in editor also (the '-e' command line option).

Thirdly, BaCon is a "semantic" and "lazy" converter. Semantic in the sense that it tries to convert the intention of the code. Lazy in the sense that it tries to perform the conversion in as few steps as possible: expressions are passed as-they-are to the C compiler, and functions are macro names. Of course, such decision has impact on the way you have to write your code.

Even though BaCon presents itself as BASIC, in a sense, it might be more correct to say: BaCon is a very elaborate pre-processor to the C language. If you keep this in mind, then your coding should be more easy. Even in Windows, you need to write a C macro with parenthesis attached to the macro name.

---

Of course I am happy that you have converted to the Unix approach. I ran away from Windows myself back in 1998, and started using Linux ever since. Back then, I felt like learning everything I knew about computers and programming all over again. We're talking about kernel 2.0, and Linux was not that simple. You could not just install it with some automatic wizard, like nowadays. You had to do manual disk partitioning, manual kernel compilation, and manual hardware configuration. Things have become easier over the years, but Linux still is not 'like Windows'. I understand this can be disappointing, but eventually, you'll see that your patience, sweat and tears will pay off in terms of performance, stability and reliability.

HTH
Peter

[clamzonprozac]

Hi Peter,

File inspection with a hex editor revieled that its too easy to slip in even a plain space (0x20) between the line continuation backslash and a newline(0x0A),

which can cause a bunch of grief.

Thank you for your time and effort in your reply to my comments. It helped in more ways than one.

Kind Regards,

-Roger

[clamzonprozac]

Hi Peter,
As if your day didn't hold enough, here's an issue involving character conditionals, parentheses, and boolean tests.
Something seems broken ...

' a test of evaluating single character string conditionals

GLOBAL a$, c$, s1$, s2$, s3$
GLOBAL k TYPE int
s1$ = "" : s2$ = "" : s3$ = ""
a$ = "+ -    01234.56789xyz"
FOR k  = 1 TO LEN(a$)
 c$ = MID$(a$,k,1)
 IF c$ >= "0" AND c$ <= "9" THEN
   s1$ = s1$ + c$
 ENDIF
 IF (c$ = ".") OR (c$ >= "0" AND c$ <= "9") THEN
   s2$ = s2$ + c$
 ENDIF
 IF (c$ = ".") OR (c$ = "-") OR (c$ = "+") OR (c$ >= "0" AND c$ <= "9")  THEN
   s3$ = s3$ + c$
 ENDIF
NEXT
PRINT "[",s1$,"]   <-- should be only numbers 0..9"
PRINT "[",s2$,"]   <-- should be decimal point, and numbers 0..9"
PRINT "[",s3$,"]   <-- should be signs, decimal point, and numbers 0..9"


Output:
sh-5.0# ./char_cond
[0123456789] <-- should be only numbers 0..9
[+ - .9] <-- should be decimal point and numbers 0..9
[+ - .9] <-- should be signs, decimal point and numbers 0..9


Oh, and I'm using a BaCon about a month old.
What's happening here?
Kind Regards,
-Roger

[Pjot]

Thanks Roger,

In BaCon, it usually isn't required to use parenthesis when comparing strings. In fact, the program would have worked perfectly fine without any parenthesis:

FOR k = 1 TO LEN(a$)
 c$ = MID$(a$,k,1)
 IF c$ >= "0" AND c$ <= "9" THEN
   s1$ = s1$ + c$
 ENDIF
 IF c$ = "." OR c$ >= "0" AND c$ <= "9" THEN
   s2$ = s2$ + c$
 ENDIF
 IF c$ = "." OR c$ = "-" OR c$ = "+" OR c$ >= "0" AND c$ <= "9" THEN
   s3$ = s3$ + c$
 ENDIF
NEXT


This would have given you the result you're looking for, also when converting with the current 4.8 stable release.

By the way, there is a BaCon construct to check if a value lies in between a certain range, called BETWEEN (opposite being BEYOND):

FOR k = 1 TO LEN(a$)
 c$ = MID$(a$,k,1)
 IF c$ BETWEEN "0" AND "9" THEN
   s1$ = s1$ + c$
 ENDIF
 IF c$ = "." OR c$ BETWEEN "0" AND "9" THEN
   s2$ = s2$ + c$
 ENDIF
 IF c$ = "." OR c$ = "-" OR c$ = "+" OR c$ BETWEEN "0" AND "9" THEN
   s3$ = s3$ + c$
 ENDIF
NEXT


I myself use parenthesis usually in case of (numeric) calculations or binary equations. But for string comparisons? I guess it is a matter of taste - that's why I never ran into this issue

Having said that, from syntax point of view, it is valid code and accepted during conversion stage. So it should work and it is confusing that the binary ends up producing these results.

Particularly for string equations, the parser now accepts more complicated parenthesis constructs as demonstrated in your program.

If you have time, please verify the latest beta with your code.

Thanks again,
Peter

[clamzonprozac]

Hi Peter,
   Man, that was quick. Thank you! Your fix works exactly as expected. Yes, I agree, parens are a matter of taste and now we
   can have it both ways! BTW: regarding the keyword BETWEEN, viscerally, when I see it I think of the valid range as
  actually being inside of the end values... as in the major part of my head is between my ears. 
 Kind Regards... and have a Great Day,
-Roger

[clamzonprozac]

Hi Peter,
  Continuing on from last night's successful test of the paren enhancement fix, I moved over to the project I've been working on
  and attempted a recompile. It failed with this log message:

In file included from HUG_motif_demo.bac.h:2,
                from HUG_motif_demo.bac.c:2:
HUG_motif_demo.bac.generic.h:3:10: fatal error: /usr/include/Xm/Xm.h/usr/include/Xm/ScrolledW.h/usr/include/Xm/BulletinB.h/usr/include/Xm/PushB.h/usr/include/Xm/ArrowB.h/usr/include/Xm/ToggleB.h/usr/include/Xm/Text.h/usr/include/Xm/Label.h/usr/include/Xm/DrawingA.h/usr/include/Xm/Separator.h/usr/include/Xm/FileSB.h/usr/include/Xm/Frame.h/usr/include/Xm/MessageB.h/usr/include/Xm/DrawnB.h/usr/include/X11/XKBlib.h: No such file or directory
   3 | #include "/usr/include/Xm/Xm.h/usr/include/Xm/ScrolledW.h/usr/include/Xm/BulletinB.h/usr/include/Xm/PushB.h/usr/include/Xm/ArrowB.h/usr/include/Xm/ToggleB.h/usr/include/Xm/Text.h/usr/include/Xm/Label.h/usr/include/Xm/DrawingA.h/usr/include/Xm/Separator.h/usr/include/Xm/FileSB.h/usr/include/Xm/Frame.h/usr/include/Xm/MessageB.h/usr/include/Xm/DrawnB.h/usr/include/X11/XKBlib.h"
     |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile.bacon:6: HUG_motif_demo.bac.o] Error 1   So this failure was generated from an area of code that does a " PRAGMA INCLUDE... ".  I went back to my prior Bacon version (compiled 9 July), and I get
   compilation without error.  For completeness I have to add that I'm running Puppy Linux (Fossapup v9.54(i think that's the minor version)), and I have to
  ./configure with  --prefix=/usr  . Done on latest compile,  older one was copied by hand, there are no other  diffs from your build documentation.

   For  future reference, this issue was found in bacon-c75971f107. Please let me know if it would be useful to regress to a prior commit.
 Kind regards,
 -Roger

[Pjot]

Well, you're ahead of me. I bet that these include lines were separated by " \" and that you have used INCLUDE to add them to your code?

I was verifying an experimental fix regarding your complaint about the " \", which sometimes did have a space character behind it. As you can see, input lines are chopped off now, however, the INCLUDE line parser does not add them to the code correctly yet.

Looking over the regression tests from yesterday, the main line parser worked fine, so I guess that fix also can be applied to the INCLUDE line parser.

The latest beta should support that as well.

BR
Peter

[clamzonprozac]

Howdy Peter,
   I had to modify some array declarations ( probably some improvements I missed ) but the INCLUDE... is now working, and
  my project is compiling and running as expected. Its a good day. Thank you!
Kind Regards,
-Roger

[clamzonprozac]

Hi Peter,
     I've stumbled on an issue with LOCAL in a SUB. It seems that BaCon is complaining that it sees LOCALS with the same identifier names which have
    also been defined as GLOBAL. Unless I'm going crazy (and that could always happen) , LOCALs shouldn't collide with GLOBALs.  The documentation gives
    me comfort in believing that. But, here, take a look...
' local vs global collision
GLOBAL s1$
GLOBAL test TYPE int
s1$ = "abcd "
test = 101


FUNCTION RPad$(s$,c$, int cnt)
' Right pad s$ with c$ until length of returned string matches cnt
LOCAL test TYPE int
LOCAL s1$
 test = 999
 s1$ = s$
 IF c$ = "" THEN : ' avoids an endless loop incident
    RETURN s1$
 END IF
 WHILE LEN(s1$) < cnt
    s1$ = s1$ + c$
 WEND
 RETURN s1$
END FUNCTION

PRINT " RPad$() =[",RPad$(s1$," ",10),"]"


BTW: I noticed this issue this evening while compiling with 4.8 something. I'm now using BaCon 5.0. Same thing happens.

As always, thank you for a really fun and useful tool!

Kind regards,
-Roger

[Pjot]

Hi Roger,

Well, I hate to be the bringer of bad news, but... this is a feature, not a bug. This has been a conscious and deliberate decision.

In the past, there have been many discussions among BaCon users about the usefulness of Variable Shadowing.

Some languages support it (like C), and some don't (like a few old-school BASIC's). Which one should have the advantage?

Next to this, sometimes forum users post large programs which "don't work", reason being that they have mixed up their global variable names with local ones. Unfortunately, it can take some time to find this out

So I would like to help out all users - for free - but there are also other things in life I have to attend to

Therefore, I selfishly disabled variable shadowing altogether - it saves a lot of time in debugging other peoples code and tracing down issues in BASIC code.

Note that the C code generated by BaCon does not use variable shadowing either.

Best regards
Peter

[clamzonprozac]

Hi Peter,
   Well, then, (from being a Pascal user for so many years) it just "makes sense" to have the scoping rules that make for what is now
   given the name "Variable Shadowing".  Maybe I'm misunderstanding something key here,  but without shadowing what is the point of
   having a variable be "LOCAL"?  Even the BaCon documentation of the keyword LOCAL, to this day, states "This statement only has sense
  within functions,  subroutines or records. It defines a local variable <var> with C type <type> which will not be visible to other functions,
  subroutines or records, nor to the main program."   I notice there's an OPTION LOCAL TRUE/FALSE switch for implicit variables but, to my
  understanding, it clearly doesn't cover explicitly declared variables.

  I haven't inspected BaCon's emitted "C" code but the difference between a GLOBAL and LOCAL variable might be the appending to a variable
  identifier  a unique string that changes from SUB to SUB and FUNCTION to FUNCTION?

  (Sigh) I understand the need to keep other folks happy but since you once had the scoping as above, (and if the capability is still intact)
  couldn't it become yet another OPTION switch?

  I say all of this (yes, easy for me to say) while appreciating that it may be be something of a non-trivial headache for you.

 And I just returned from a snack break that included thoughts of a custom pre-processor (what a way to start the day)...

 Kind Regards,
 -Roger