Importing BaCon to OpenEmbedded (solved)

[barryk]

No, bacon.bac.lex.exe is a 32-bit binary.

OE is a cross-compile environment, and I also compile for aarch64 target, as have EasyOS running on the rpi4.

In that case, build/bacon.bac.lex.exe will be an aarch64 binary.

So, if I leave out flex dependency, will that mean bacon will create bigger executables?

[Pjot]

Aug 24, 2022 14:17:39 GMT 1 barryk said:

So, if I leave out flex dependency, will that mean bacon will create bigger executables?

No, it will just mean that the BaCon binary itself will be bigger. Subsequent binaries created by BaCon will be the usual small size.

Regarding the error:

./bacon.sh: line 9214: build/bacon.bac.lex.exe: No such file or directory


I really do not see how a script like 'bacon.sh' cannot execute a binary. It is there and 32bit and should work in 64bit. Does 'ldd bacon.bac.lex.exe' show all dependencies? If a dependency fails then it cannot be executed.

Best regards
Peter

[barryk]

Hi Peter,
My host OS does not have 32-bit libraries, so build/bacon.bac.lex.exe cannot be executed.

OpenEmbedded/Yocto is a cross-compiler, usually running on a x86_64 host OS, with many possible target CPUs. In my case, I compile for x86_64 (nocona), i686 and aarch64 targets. Awhile back, I was also compiling for armv7 target.

So build/bacon.bac.lex.exe will be created for the target CPU, hence will not run in the host OS. Only exception is the nocona target.

Yes, x86 32-bit build/bacon.bac.lex.exe could be made to run, if I had 32-bit libc in the host, but that is only for that one target CPU.

I have completed the i686 build using bacon 3.9.3, and all of the utilities work. I created 32-bit i686 EasyOS built from the packages compiled in OE:

bkhome.org/news/202208/easyos-32-bit-i686-dunfell-series-version-435.html

Tested 'picscale', 'popup' and some other utilities written in bacon, in the 32-bit OS, they work.

Thanks for looking into this. There doesn't seem to be any quick fix for the 4.x versions.
Perhaps you could bring back libbacon.a as an option, instead of using flex/lex?

Or, as you said, just not have flex as a dependency, so it won't be used. But then, the same would have to apply to cross-compiling all the utilities, such as picscale. They would all be bigger binaries.

[barryk]

Aug 24, 2022 13:25:26 GMT 1 Pjot said:

Anyway, using flex is optional. It is only there to optimize the resulting binary, but we can live without. If you - temporary - rename your flex utility and try again then BaCon should not bother.

Yes and no. OE compiles each package in an environment that is almost completely isolated from the host OS.
You can think of it as like a chroot -- a chrootable rootfs is put together with "-native" packages, for example "gcc-native", and bacon will be compiled inside that rootfs.

So the existence or otherwise of 'flex' in the host OS makes no difference.

It so happens that neither 'flex-native' nor 'indent-native' are included by default in that rootfs, so the recipe for bacon will have to specify them:

DEPENDS += "flex-native indent-native"

If not listed in the DEPENDS variable, then they won't be in the rootfs.

OE doesn't actually use a chroot, does the equivalent by other means, but the concept of a chroot clarifies what is happening.

In fact, OE builds a "chrootable rootfs" for each package that it compiles, each one tailored to the needs of that package. This results in an incredible amount of duplication, and an OE build can typically require about 350GB drive space.

Though, I did read, they try to use hardlinks where possible. Also, there is an option to destroy each "chrootable rootfs" after successful build, but that is not the default.

[barryk]

Peter,
There may be a simple fix for that build/bacon.bac.lex.exe
Ditto for other packages, for example, picscale generates build/picscale.bac.lex.exe

The variables in Makefile, CC, CFLAGS, etc, are all for the target CPU.

Bacon needs to be told that it is in a cross-compile environnment, and use different variables to compile a binary that will run in the host OS.

OE supplies those variables, they are all prefixed with "BUILD_":

basic-converter.proboards.com/post/14610

The simplest fix would be for bacon to see if "BUILD_CC" etc are set, then compile *.lex.exe using those.

Or, maybe have another commandline option, say "-u", informing bacon that this is a cross-compile environment.
Then, if commandline does have "-u", check for "BUILD_CC" etc to compile *.lex.exe

I know that BUILD_CC is OE-specific, but other cross-compile systems, such as Buildroot, would have equivalent variables, and they can be assigned to the OE variable names.

[barryk]

Had a very quick look at bacon.sh, looks doable to create a patch
for the *.lex.c compile
I could probably do that, just for the requirements of OE.

[Pjot]

Aug 25, 2022 6:23:55 GMT 1 barryk said:

Bacon needs to be told that it is in a cross-compile environnment, and use different variables to compile a binary that will run in the host OS.

OE supplies those variables, they are all prefixed with "BUILD_":

basic-converter.proboards.com/post/14610

The simplest fix would be for bacon to see if "BUILD_CC" etc are set, then compile *.lex.exe using those.

I know that BUILD_CC is OE-specific, but other cross-compile systems, such as Buildroot, would have equivalent variables, and they can be assigned to the OE variable names.

Well, as I mentioned before, the C code generated by BaCon does not actually need optimization by flex.

So the most easy way would be to add a parameter like '-b' for 'bare', so BaCon never invokes any external utility (not flex, not indent). Then the generated code will contain the full library of functions and the resulting binary will be bigger than necessary. But it will work.

As BaCon is designed to be as generic as possible with the most conservative code and tools, so it can run everywhere, even on my Android phone, I am not sure how meaningful it is to start implementing platform dependent quirks. I have always tried to avoid this where possible, therefore you will not find any ASM code, GNU extensions, Linux API calls, etc in the C code.

Adding platform dependent exceptions will cause BaCon to loose its generic character, and besides this it also will be impossible to maintain on the long-term.

If you have a suggestion though on how to workaround the flex hurdle with scripts or external Makefiles or CMake or even Ant or SCons or anything else, then it's no problem to ship such script with the source package.

Best regards

Peter

[barryk]

Hi Peter,
A "-b" option is of no interest to me, as I can easily achieve the effect by just leaving 'flex-native' and 'indent-native' out of the DEPENDS variable.

Down the track, I might create patches for bacon.sh and bacon.bac, so as to be able to have 'flex-native' dep in an OE build.

[bigbass]

Hello barryk and Peter



maybe one more try ?

Disclaimer
there is no way I can test on oe
but there are many things that are the same in the process of a crossbuild


I compiled picscale and bacon in a different crossbuild




barryk forgive me if the syntax is not 100% oe compatible for the reason I gave earlier
but the next idea is implied you will need to adjust it



page three you wrote basic-converter.proboards.com/post/14611

have confirmed, bacon 3.9.3 compiles OK in OE.

As back in 2017, when I was compiling 3.5.4, cannot get autotools to work, so resorted to doing the build manually. So, this is 'bacon_3.9.3.bb' recipe file:

that was the 3.9.3 bacon



I think this should work with the new bacon
but have no way to test it



do_compile() {
   mkdir -p build
   # -n convert to C only, -a rebuild libbacon.a, -p preserve temporary files,
   # -y automatically delete temporary files ...
   #bash bacon.sh -y  -d build bacon.bac
   #...this works for x86_64 target. but for armv7 target may need to do this...
   bash bacon.sh -n -p  -d build bacon.bac
   cd build
   
   # here the crossbuild compiler needs to be used too for CC for the compile and build
   ${CC} -fPIC -O2 ${CFLAGS} -c bacon.bac.c
   
   # make sure that we dont link the HOST system libs with the crossbuild  
   ${CC} -o bacon bacon.bac.o  "/usr/${CROSSBUILD_ARCH}-linux-gnu/lib/libm.so"  
   cd ..
}





maybe manually it can be fixed
anyway its worth a try
just trying to help

Joe


P.S

this worked for me in bash

cd build

#COMPILE
/usr/bin/x86_64-linux-gnu-gcc-10  -c bacon.bac.c


#BUILD  add libm.so  from the x86_64 crossbuild
/usr/bin/x86_64-linux-gnu-gcc-10 -o bacon bacon.bac.o  /usr/x86_64-linux-gnu/lib/libm.so


#STRIP
/usr/x86_64-linux-gnu/bin/strip  bacon


[barryk]

Hi Joe,

bash bacon.sh -n -p  -d build bacon.bac

In OE, CC, CFLAGS, LDFLAGS will all be for the target CPU, so lexical analysis exe will not run.

However, BUILD_CC etc are for the host system, so if we did, say:

CC="$BUILD_CC" CFLAGS="$BUILD_CFLAGS" LDFLAGS="$BUILD_LDFLAGS" bash bacon.sh -n -p -d build bacon.bac

Then the lexical analyzer exe will run.
Then maybe the lexical analyzer exe could be used for another build, this time for the target CPU...

However, what assumptions would bacon.sh be making that are inappropriate, if say the host OS is x86_64 and the target is, say armv6?
bacon.sh would not even know that it is compiling for a 32-bit target.

How generic is the lexical analyzer? if compiled for x86_64, could the exe be used to analyze if do a subsequent armv6 build?
...that would be a question for Peter

Hmmm, might not have explained it very well, but hope you can get the drift of what I am asking.

[Pjot]

The C code generated by the 'flex' binary is generic C code and should be compilable by the regular C compilers.

However, BaCon did not pass CFLAGS and LDFLAGS as compile options for the lexer.

For example, if, in my 64bit Mint, I would compile the program "sinus.bac" for 32 bit Intel, it looks as follows:

$ bacon -p -o -m32 -l -m32 sinus
Do you want to delete them ([y]/n)? y
Temporary files were deleted.
Converting 'sinus.bac'... done, 51 lines were processed in 0.003 seconds.
Creating lexical analyzer using flex... done.
Applying indentation... done.
Compiling 'sinus.bac'... cc  -m32 -c sinus.bac.c
cc -o sinus sinus.bac.o -m32    -lm
Done, program 'sinus' ready.


Looking at the generated binaries:

$ file sinus
sinus: ELF 32-bit LSB pie executable, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, BuildID[sha1]=cf8df3cf908e5b308ac7f8553614d83066c23b28, for GNU/Linux 3.2.0, not stripped
$ file sinus.bac.lex.exe
sinus.bac.lex.exe: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=34cef3cfa05edcc78230f555b15ddd45145fe3fd, for GNU/Linux 3.2.0, not stripped


In other words, the resulting binary is 32bit, but the generated lexer binary still is 64bit. I guess this is the problem barry is facing with his cross compilation. It is impossible to execute the 64bit lexer in his 32bit cross compilation environment.

So now I have made sure that the CFLAGS and LDFLAGS variables used to compile BaCon also are used when compiling the lexer. Then the lexer is 32bit as well and should work there too.

The lastest beta contains this fix, maybe it helps.

BR
Peter

[bigbass]

Hello Barry and Peter


for me it is easier to explain with a working code example
that I wrote today
of course you would need to adapt it to your open embedded syntax
but you could also test this on a raspberry pi3 in a generic setting

crossbuild-bacon.tar.gz (284.34 KB)

hope something could be useful
thank you Peter for all your work
thank you Barry for trying to use a recent bacon version

Joe

[barryk]

Hi Peter and Joe,
What would work in OE is if can use CC="$BUILD_CC" etc as suggested earlier, then retain the lexer exe for a repeat build using the normal cross-compile CC etc.

That would require a commandline flag for bacon, "do not compile lexer exe if it already exists".
Then, the second compile will use the existing lexer exe.

Does that seem reasonable?

[barryk]

Sept 1, 2022 17:27:14 GMT 1 Pjot said:

In other words, the resulting binary is 32bit, but the generated lexer binary still is 64bit. I guess this is the problem barry is facing with his cross compilation. It is impossible to execute the 64bit lexer in his 32bit cross compilation environment.

No, that isn't correct.
OE will have no problem executing a x86_64 lexer exe. The host OS is x86_64.

In OE, the CC, CFLAGS, etc., are all set for the target, say armv6
That is why lexer exe is being built for the target, hence won't run.

If I did a compile with CC="$BUILD_CC" etc, then lexer exe would be x86_64 and would run.

That is what I am getting at. compile lexer exe so that it will run, without progressing to compiling the C code.

Then repeat the build, this time using the normal cross-compile CC, etc., but with a flag so will use the pre-existing lexer exe.
This time, progress to compile the C code.

[barryk]

Hi Peter,
No need to worry about fixing it for me, I can very easily apply patches to bacon.sh and bacon.bac

In the case of bacon.sh, all that I need to do is replace this line:

${g_CCNAME} ${g_BACONLEXER}.c -o ${g_BACONLEXER}.exe

With this:

${BUILD_CC} ${BUILD_CFLAGS} ${g_BACONLEXER}.c -o ${g_BACONLEXER}.exe ${BUILD_LDFLAGS}

Similar with bacon.bac

Problem solved.