Importing BaCon to OpenEmbedded (solved)

[Pjot]

Hi Barry,

Thanks for the explanation of your setup and also for the logs. From the first lines of the logging I can already see why things are not working.

Could you maybe send a link to an article or posting where those Debian developers declare 'which' as deprecated? I searched for this but could not find their statement.

Regarding compilation, your logging shows that the shell script cannot generate the binary for dependent functions using 'cc' as C compiler:

Creating lexical analyzer... bacon.sh: line 9130: cc: command not found
done.
bacon.sh: line 9134: build/bacon.bac.lex.exe: No such file or directory


During the conversion process, BaCon creates a 'lex' file with the functions to search for in the generated code. Then, this file needs to be compiled to an executable 'bacon.bac.lex.exe', which will be executed to select which C code needs to be added to the final binary.

This is the related Shell script code (lines 9128-9131):

echo -n "Creating lexical analyzer... "
${LEX} -o ${g_BACONLEXER}.c ${g_BACONLEXER}
${g_CCNAME} ${g_BACONLEXER}.c -o ${g_BACONLEXER}.exe
echo "done."


If you have not provided the name for the C compiler then BaCon assumes 'cc'.

For now, your compilation should work by explicitly adding the name for the C compiler as an argument. For example, if 'clang' is your compiler:

# bash bacon.sh -c clang bacon


Of course you can add any other option on the command line as well. Alternatively, you can add a symlink 'cc' pointing to your compiler also.

Please let me know the outcome.

BR
Peter

[barryk]

It works!

I didn't even know about that "-c" option!

Thanks for looking into it.

Now, need to see if can use that native 'bacon' to compile for the target.

Here is where it is stated that 'which' is deprecated:

wiki.debian.org/NewInBookworm

Interesting, "type -a pwd" does work, whereas "type -p pwd" doesn't.

[Pjot]

Thanks Barry,

Great to hear it all works for you now. In case of the 'autoconf' scripts, the detected C compiler would be set in the Makefile and then passed on to the BaCon shell script in a similar manner.

And thanks for the link, I could not find it in Debian channels, but eventually I did find this long story on stackexchange, which tries to explain the history of 'which'.

To me, as an old-school Unix user coming from somewhere back in the '90s, it is a bit of a shock to see that 'which' should be avoided.

So I decided to proceed with the recommended 'command -v' instead, as it is POSIX compliant and works for BASH, KSH and also ZSH. Updated BaCon code in the repo.

Best regards

Peter

PS Regarding your 'pwd' issue, I assume it is caused by the situation that 'pwd' also is a built-in keyword for BASH, similar to 'echo'. Anyway, all works now with the recommended 'command -v'.

[barryk]

Hmmm, it has been awhile, things have changed.
'bacon.sh' no longer has the "-a" option, to rebuild 'libbabcon.a'

bacon.sh seems to ignore it.

I do recall very old versions of bacon that did not have that external libabcon.a; has bacon gone back to that?

[Pjot]

Hi Barry,

The thing is that BaCon needs to know which code should be available in the resulting binary, and which code can be omitted.

In the 3.x series, BaCon would create an external static library called 'libbacon.a'. From this, the linker took the object files which were required to build the binary, thus leaving out (compiled) code for functions which are not used in the program.

In the 4.x series, this mechanism has changed. Instead, BaCon now looks for the availability of either 'flex' or 'lex'. Based on a few lexer rules it will create a binary, which, in its turn, can generate the list of required C functions to compile into the resulting binary (you saw this in the previous compile hurdle).

Both approaches are meant to make sure no redundant functions and no redundant code would be added to the resulting binary.

In BaCon 4.5, if neither 'flex' nor 'lex' is found on the system, then BaCon will bluntly add all C code, so everything, for all functions and statements. This of course will produce an unnecessary large binary - but it will work.

Best regards
Peter

[barryk]

I have hit another roadblock.

There are two recipe files, 'bacon-native_4.5.bb' and 'bacon_4.5.bb'
The first one compiles 'bacon' binary to run on the host system. That works.
The second one compiles 'bacon' binary to run on the target system. That doesn't work.

In both recipes, this is what is being executed, works in first recipe, fails in second:

bash bacon.sh -n -p -d build -c ${CC/ */} bacon.bac

In the second case, here is the failure:

Converting 'bacon.bac'... done, 9972 lines were processed in 337 seconds.
Creating lexical analyzer... build/bacon.bac.lex.c:19:10: fatal error: stdio.h: No such file or directory
  19 | /* begin standard C headers. */
     |          ^~~~~~~~~
compilation terminated.
done.
bacon.sh: line 9134: build/bacon.bac.lex.exe: No such file or directory

In the first case, I think that bacon.sh must be finding stdio.h in the host system. Maybe.

For the second recipe, here is the content of CC:

x86_64-poky-linux-gcc  -m64 -march=nocona -mtune=nocona -mno-sse3 -mfpmath=sse -fstack-protector-strong \
-D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security \
--sysroot=/mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9/recipe-sysroot

"-c ${CC/ */}" is only passing "x86_64-poky-linux-gcc"
It seems that I have to inform bacon.sh about that "--sysroot" parameter, as that's where the usr/include is located.

How to do that?

Answering my own question:
it looks like just pass in all of CC, by putting it in quotes: -c "${CC}"
Trying t now...

[barryk]

Yes! Almost there. I will post the final recipes when all fixed.

Compiled 'bacon' for the target cpu ok, but a fail for bacongui-gtk:

| Converting 'bacongui-gtk.bac'... done, 3450 lines were processed in 0.172 seconds.
| Creating lexical analyzer... In file included from /mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9/recipe-sysroot/usr/include/bits/libc-header-start.h:33,
|                  from /mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9/recipe-sysroot/usr/include/stdio.h:27,
|                  from build2/bacongui-gtk.bac.lex.c:19:
| /mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9/recipe-sysroot/usr/include/features.h:397:4: warning: #warning _FORTIFY_SOURCE requires compiling with optimization (-O) [-Wcpp]
|   397 | #  warning _FORTIFY_SOURCE requires compiling with optimization (-O)
|       |    ^~~~~~~
| done.
| Compiling 'bacongui-gtk.bac'... x86_64-poky-linux-gcc  -m64 -march=nocona -mtune=nocona -mno-sse3 -mfpmath=sse -fstack-protector-strong  -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security --sysroot=/mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9/recipe-sysroot   -O2 -pipe -g -feliminate-unused-debug-types -fmacro-prefix-map=/mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9=/usr/src/debug/bacon/4.5-r9                      -fdebug-prefix-map=/mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9=/usr/src/debug/bacon/4.5-r9                      -fdebug-prefix-map=/mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9/recipe-sysroot=                      -fdebug-prefix-map=/mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9/recipe-sysroot-native=  -c bacongui-gtk.bac.c
| bacongui-gtk.bac:1595:39: error: 'DATA_PATH' undeclared (first use in this function)
|
| INFO: see full error report (y/[n])? WARNING: /mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9/temp/run.do_compile.7569:1 exit 2 from 'bacon -d build2 -c "x86_64-poky-linux-gcc  -m64 -march=nocona -mtune=nocona -mno-sse3 -mfpmath=sse -fstack-protector-strong  -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security --sysroot=/mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9/recipe-sysroot" -o " -O2 -pipe -g -feliminate-unused-debug-types -fmacro-prefix-map=/mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9=/usr/src/debug/bacon/4.5-r9                      -fdebug-prefix-map=/mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9=/usr/src/debug/bacon/4.5-r9                      -fdebug-prefix-map=/mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9/recipe-sysroot=                      -fdebug-prefix-map=/mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9/recipe-sysroot-native= " -l "-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed -fstack-protector-strong -Wl,-z,relro,-z,now" bacongui-gtk.bac'
| ERROR: Execution of '/mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9/temp/run.do_compile.7569' failed with exit code 2
ERROR: Task (/mnt/build/oe-builds/oe-quirky/meta-quirky/recipes-quirky/bacon/bacon_4.5.bb:do_compile) failed with exit code '1'


What is "DATA_PATH"?

[barryk]

bacongui-fltk also fails:

| Compiling 'bacongui-fltk.bac'... x86_64-poky-linux-gcc  -m64 -march=nocona -mtune=nocona -mno-sse3 -mfpmath=sse -fstack-protector-strong  -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security --sysroot=/mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9/recipe-sysroot   -O2 -pipe -g -feliminate-unused-debug-types -fmacro-prefix-map=/mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9=/usr/src/debug/bacon/4.5-r9                      -fdebug-prefix-map=/mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9=/usr/src/debug/bacon/4.5-r9                      -fdebug-prefix-map=/mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9/recipe-sysroot=                      -fdebug-prefix-map=/mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9/recipe-sysroot-native=  -Wno-write-strings -Wno-pointer-arith -c bacongui-fltk.bac.c
| /mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9/recipe-sysroot/usr/include/FL/Enumerations.H:663:18: error: unknown type name 'Fl_Boxtype'
|
| INFO: see full error report (y/[n])? WARNING: /mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9/temp/run.do_compile.12832:1 exit 2 from 'bacon -d build2 -c "x86_64-poky-linux-gcc  -m64 -march=nocona -mtune=nocona -mno-sse3 -mfpmath=sse -fstack-protector-strong  -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security --sysroot=/mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9/recipe-sysroot" -o " -O2 -pipe -g -feliminate-unused-debug-types -fmacro-prefix-map=/mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9=/usr/src/debug/bacon/4.5-r9                      -fdebug-prefix-map=/mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9=/usr/src/debug/bacon/4.5-r9                      -fdebug-prefix-map=/mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9/recipe-sysroot=                      -fdebug-prefix-map=/mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9/recipe-sysroot-native= " -l "-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed -fstack-protector-strong -Wl,-z,relro,-z,now" bacongui-fltk.bac'
| ERROR: Execution of '/mnt/build/oe-builds/oe-quirky/build-amd64/tmp/work/nocona-64-poky-linux/bacon/4.5-r9/temp/run.do_compile.12832' failed with exit code 2
ERROR: Task (/mnt/build/oe-builds/oe-quirky/meta-quirky/recipes-quirky/bacon/bacon_4.5.bb:do_compile) failed with exit code '1'


The error is: unknown type name 'Fl_Boxtype'
I really don't need the editor. In EasyOS we use geany.

[barryk]

Here are the final recipes, for anyone else who uses OpenEmbedded.
Firstly, file 'bacon-native_4.5.bb'. OE supports auttools, and I could have compiled with the makefile, however, simple to do it this way:

# Recipe created by recipetool
# recipetool create -o bacon_4.5.bb https://www.basic-converter.org/stable/bacon-4.5.tar.gz

HOMEPAGE = "http://basic-converter.org/"
SUMMARY = "A BASIC compiler with highlevel GUI IDE."

LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://LICENSE;md5=45bf18d7318cf67fb528718d3aa04efe"

SRC_URI = "https://www.basic-converter.org/stable/bacon-${PV}.tar.gz"

SRC_URI[md5sum] = "0ace921eae6cc8a50880a69bd512ecfa"
SRC_URI[sha256sum] = "a47083581cdbae0c51076195afad102c193b86afa0c3f0ec954af51a7d8c662a"

DEPENDS = "flex-native coreutils-native util-linux-native"

inherit native

do_configure() {
true
}

do_compile_prepend() {
#remove pwd test from script...
sed -i -e 's%which pwd%which head%' bacon.sh
}

do_compile() {
mkdir -p build
bash bacon.sh -d build -c "${CC}" -o "${CFLAGS}" -l "${LDFLAGS}" bacon.bac
}

# note: in -native recipes, must use $bindir, NOT /usr/bin!
do_install() {
install -d ${D}${bindir}
install -m755 build/bacon ${D}${bindir}
install -m755 bacon.sh ${D}${bindir}
}

That do_compile_prepend() won't be needed with the next version of bacon (after 4.5).
The above recipe creates binary 'bacon' that will work on the host system.

The file 'bacon_4.5.bb' will compile binary 'bacon' for the target system:

# Recipe created by recipetool
# recipetool create -o bacon_4.5.bb https://www.basic-converter.org/stable/bacon-4.5.tar.gz

HOMEPAGE = "http://basic-converter.org/"
SUMMARY = "A BASIC compiler with highlevel GUI IDE."

LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://LICENSE;md5=45bf18d7318cf67fb528718d3aa04efe"

SRC_URI = "https://www.basic-converter.org/stable/bacon-${PV}.tar.gz"

SRC_URI[md5sum] = "0ace921eae6cc8a50880a69bd512ecfa"
SRC_URI[sha256sum] = "a47083581cdbae0c51076195afad102c193b86afa0c3f0ec954af51a7d8c662a"

DEPENDS = "flex-native bacon-native coreutils-native util-linux-native"

do_configure() {
true
}

do_compile() {
mkdir -p build
bacon -d build -c "${CC}" -o "${CFLAGS}" -l "${LDFLAGS}" bacon.bac
}

do_install() {
install -d ${D}${bindir}
install -m755 build/bacon ${D}/usr/bin
#remove pwd test from script...
sed -i -e 's%which pwd%which head%' bacon.sh
install -m755 bacon.sh ${D}/usr/bin
install -d ${D}/usr/share/BaCon
install -m644 bacon.bac ${D}/usr/share/BaCon
install -m644 bacongui-gtk.bac ${D}/usr/share/BaCon
install -m644 bacongui-fltk.bac ${D}/usr/share/BaCon
install -d ${D}/usr/share/pixmaps
install -m644 icons/BaCon24.png ${D}/usr/share/pixmaps/BaCon.png
}

...this compiles very fast, as it uses 'bacon' from the first recipe.

Next, I have to verify that the recipe works on a very different target, say i686 or aarch64.

Thanks for the help. I was messing around, then the final solution was actually very simple.

[Pjot]

Hi Barry,

Thanks for sharing all your code us.

You can of course mess around with compile schemes and recipes, but please note that the Makefile generated by './configure' is the only one which should work on all platforms and in all circumstances.

So if you run into trouble, simply run './configure' on a platform where you know for sure everything works out-of-the-box and compare the created Makefile with your scripts. It probably will help a lot to see how it all works.

BR
Peter

[barryk]

Peter,
Yes, I should use autotools in the OE recipe.

Right now, OpenEmbedded compiling on a x86_64 host OS, for i686 target. Attempting compile with the Makefile, but compile fails. It also gives this warning:

| Converting 'bacon.bac'... 10058
| Converting 'bacon.bac'... done, 10058 lines were processed in 0.388 seconds.
| Creating lexical analyzer... In file included from /mnt/build/oe-builds/oe-quirky/build-i686/tmp/work/i686-poky-linux/bacon/4.5.0.1-r4/recipe-sysroot/usr/include/bits/libc-header-start.h:33,
|                  from /mnt/build/oe-builds/oe-quirky/build-i686/tmp/work/i686-poky-linux/bacon/4.5.0.1-r4/recipe-sysroot/usr/include/stdio.h:27,
|                  from build/bacon.bac.lex.c:19:
| /mnt/build/oe-builds/oe-quirky/build-i686/tmp/work/i686-poky-linux/bacon/4.5.0.1-r4/recipe-sysroot/usr/include/features.h:397:4: warning: #warning _FORTIFY_SOURCE requires compiling with optimization (-O) [-Wcpp]
|   397 | #  warning _FORTIFY_SOURCE requires compiling with optimization (-O)
|       |    ^~~~~~~
| done.

Well, I suppose that I could edit CC, CFLAGS and LDFLAGS in the Makefile. Here they are:

CC              = i686-poky-linux-gcc  -m32 -march=i686 -fstack-protector-strong  -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security --sysroot=/mnt/build/oe-builds/oe-quirky/build-i686/tmp/work/i686-poky-linux/bacon/4.5.0.1-r4/recipe-sysroot
CFLAGS          =  -O2 -pipe -g -feliminate-unused-debug-types -fmacro-prefix-map=/mnt/build/oe-builds/oe-quirky/build-i686/tmp/work/i686-poky-linux/bacon/4.5.0.1-r4=/usr/src/debug/bacon/4.5.0.1-r4 -fdebug-prefix-map=/mnt/build/oe-builds/oe-quirky/build-i686/tmp/work/i686-poky-linux/bacon/4.5.0.1-r4=/usr/src/debug/bacon/4.5.0.1-r4  -fdebug-prefix-map=/mnt/build/oe-builds/oe-quirky/build-i686/tmp/work/i686-poky-linux/bacon/4.5.0.1-r4/recipe-sysroot=  -fdebug-prefix-map=/mnt/build/oe-builds/oe-quirky/build-i686/tmp/work/i686-poky-linux/bacon/4.5.0.1-r4/recipe-sysroot-native=
LDFLAGS         = -Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed -fstack-protector-strong -Wl,-z,relro,-z,now


What does "-D_FORTIFY_SOURCE=2" actually do? Could I just take it out?
Anyway, why is there a warning, as CFLAGS does have "-O2"?

[barryk]

I took out FORTIFY_SOURCE. In the Makefile, I also changed to running the native bacon executable, instead of bacon.sh, like this:

do_compile_prepend() {
#use native bacon binary...
sed -i -e 's%^BSHELL .*%BSHELL = %' Makefile
sed -i -e 's%\./bacon\.sh%bacon%' Makefile
#this causes a warning, remove...
sed -i -e 's%\-D_FORTIFY_SOURCE=2%%' Makefile
}

But compile still fails, without any information, except for an error number:

Converting 'bacon.bac'... done, 10058 lines were processed in 0.386 seconds.
Creating lexical analyzer... done.
make: *** [Makefile:22: bacon] Error 141

Does that "141" give a clue what is wrong?

The line that is failing in Makefile:

$(BSHELL) bacon -c "$(CC)" -o "$(CFLAGS)" -l "$(LDFLAGS)" -d build bacon.bac;

[Pjot]

Aug 22, 2022 2:00:30 GMT 1 barryk said:

What does "-D_FORTIFY_SOURCE=2" actually do? Could I just take it out?
Anyway, why is there a warning, as CFLAGS does have "-O2"

Hi barry,

You seem to stumble into an issue with your optimization flags on the one hand and the code generated by the 3rd party 'Flex' tool on the other. As it is just a warning, I wouldn't be too worried. The resulting binary should have been created in any case (I see a 'done' at the end).

If the binary is not there then I also would take '-D_FORTIFY_SOURCE=2' flag which was put there by flex. Or tweak the list of CFLAGS until the warning disappears (maybe change -O2 to plain -O).

Anyway, the flag seems to handle buffer overflow detection and stuff like that.

The output of that second compile doesn't say much unfortunately... do you maybe have a logfile?

Thanks
Peter

[barryk]

Hi Peter,
Unfortunately, there is nothing more in the compile log. I modified the recipe so the Makefile will convert to C only, then run the C compiler separately:

do_compile_prepend() {
#use native bacon binary, and convert only...
sed -i -e 's%^BSHELL .*%BSHELL = %' Makefile
sed -i -e 's%\./bacon\.sh%bacon -n -p %' Makefile
#this causes a warning, remove...
sed -i -e 's%\-D_FORTIFY_SOURCE=2%%' Makefile
}

do_compile() {
oe_runmake
#change to folder with C source and compile...
cd build
${CC} -fPIC -O2 ${CFLAGS} -fno-var-tracking-assignments -c bacon.bac.c
${CC} -o bacon bacon.bac.o -L. -lm ${LDFLAGS}
cd ..
}

"oe_runmake" is a wrapper around make. Part of the function of the wrapper is to log everything to a file.
It doesn't reach that "cd build", the build aborts, as reported before:

| Converting 'bacon.bac'... done, 10058 lines were processed in 0.392 seconds.
| Creating lexical analyzer... done.
| make: *** [Makefile:22: bacon] Error 141

So, it is bacon, it seems, that is returning an error number, this line in the Makefile:

bacon -n -p  -c "$(CC)" -o "$(CFLAGS)" -l "$(LDFLAGS)" -d build bacon.bac;

The conversion has worked, it has created build/bacon.bac.c, and it looks ok.

The Makefile has aborted because 'bacon' has returned an error code. I presume it is that "141".

Is it an error to be passing in $CFLAGS and $LDFLAGS if bacon is only doing a conversion, not running the C compiler?
Don't think so, because had that same 141 error before, when bacon did not have that "-n".

[Pjot]

Hi Barry,

Can you send me the complete Makefile you're using? Then maybe I can reproduce the problem.

Thanks,
Peter