Word counting 2

[Deleted]

Sorry for bringing back this ancient topic (http://basic-converter.proboards.com/thread/240/word-counting), but this is a good example of how BaCon is getting better and better. This version requires BaCon 2.4.0:


DECLARE kawalki$[100000] TYPE STRING
DECLARE zastapione$[100000] TYPE STRING
DECLARE wszystkie ASSOC int

PRINT "Reading the file..."

OPEN "Bible.txt" FOR READING AS ksiazka
x = 0
WHILE NOT(ENDFILE(ksiazka)) DO
READLN akapit$ FROM ksiazka
kawalki$[x] = LCASE$(akapit$)
INCR x
WEND
CLOSE FILE ksiazka

PRINT "Counting..."


FOR y = 0 TO x
zastapione$[y] = REPLACE$(kawalki$[y], "[^a-z]", " ", TRUE)
NEXT

wsz = 0
uni = 0
FOR y = 0 TO x
FOR slowo$ IN zastapione$[y]
IF LEN(slowo$) > 1 OR slowo$ = "a" OR slowo$ = "i" THEN
IF ISKEY(wszystkie, slowo$) = TRUE THEN
INCR wszystkie(slowo$)
INCR wsz
ELSE
wszystkie(slowo$) = 1
INCR uni
INCR wsz
ENDIF
ENDIF
NEXT
NEXT

PRINT "All words - ", wsz
PRINT "Unique words - ", uni

PRINT "Sorting and writing to the file..."

SORT wszystkie

OPEN "words2.txt" FOR WRITING AS wynik
WRITELN "All words - " & STR$(wsz) TO wynik
WRITELN "Unique words - " & STR$(uni) TO wynik
WRITELN " " TO wynik
LOOKUP wszystkie TO klucze$ SIZE rozmiar
FOR x = rozmiar - 1 TO 0 STEP -1
   WRITELN klucze$[x] & " - " & STR$(wszystkie(klucze$[x])) TO wynik
NEXT
CLOSE FILE wynik

PRINT "Done!"


It may be not the fastest solution*, but, because it uses regular expressions and associative arrays, it's shorter and easier to read. And finally, it produces exactly the same result like Python, Perl, Ruby and other languages.

*Associative arrays are not as fast as in some other languages.

[Deleted]

Doesn't work with BaCon 3.5.1.

*** Error in `./words': double free or corruption (fasttop): 0x0000000000eedfa0 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7f91e838c7e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x7fe0a)[0x7f91e8394e0a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7f91e839898c]
./words[0x4027ee]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f91e8335830]
./words[0x4012f9]
======= Memory map: ========
00400000-00406000 r-xp 00000000 00:27 7078830 /home/tomek/Documents/words/words
00605000-00606000 r--p 00005000 00:27 7078830 /home/tomek/Documents/words/words
00606000-00607000 rw-p 00006000 00:27 7078830 /home/tomek/Documents/words/words
00607000-00790000 rw-p 00000000 00:00 0
00e0a000-00f09000 rw-p 00000000 00:00 0 [heap]
7f91e4000000-7f91e4021000 rw-p 00000000 00:00 0
7f91e4021000-7f91e8000000 ---p 00000000 00:00 0
7f91e80ff000-7f91e8115000 r-xp 00000000 08:01 7475703 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f91e8115000-7f91e8314000 ---p 00016000 08:01 7475703 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f91e8314000-7f91e8315000 rw-p 00015000 08:01 7475703 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f91e8315000-7f91e84d4000 r-xp 00000000 08:01 7472360 /lib/x86_64-linux-gnu/libc-2.23.so
7f91e84d4000-7f91e86d4000 ---p 001bf000 08:01 7472360 /lib/x86_64-linux-gnu/libc-2.23.so
7f91e86d4000-7f91e86d8000 r--p 001bf000 08:01 7472360 /lib/x86_64-linux-gnu/libc-2.23.so
7f91e86d8000-7f91e86da000 rw-p 001c3000 08:01 7472360 /lib/x86_64-linux-gnu/libc-2.23.so
7f91e86da000-7f91e86de000 rw-p 00000000 00:00 0
7f91e86de000-7f91e87e6000 r-xp 00000000 08:01 7472365 /lib/x86_64-linux-gnu/libm-2.23.so
7f91e87e6000-7f91e89e5000 ---p 00108000 08:01 7472365 /lib/x86_64-linux-gnu/libm-2.23.so
7f91e89e5000-7f91e89e6000 r--p 00107000 08:01 7472365 /lib/x86_64-linux-gnu/libm-2.23.so
7f91e89e6000-7f91e89e7000 rw-p 00108000 08:01 7472365 /lib/x86_64-linux-gnu/libm-2.23.so
7f91e89e7000-7f91e8a0d000 r-xp 00000000 08:01 7472348 /lib/x86_64-linux-gnu/ld-2.23.so
7f91e8bed000-7f91e8bf0000 rw-p 00000000 00:00 0
7f91e8c09000-7f91e8c0c000 rw-p 00000000 00:00 0
7f91e8c0c000-7f91e8c0d000 r--p 00025000 08:01 7472348 /lib/x86_64-linux-gnu/ld-2.23.so
7f91e8c0d000-7f91e8c0e000 rw-p 00026000 08:01 7472348 /lib/x86_64-linux-gnu/ld-2.23.so
7f91e8c0e000-7f91e8c0f000 rw-p 00000000 00:00 0
7fff9a38f000-7fff9a3b0000 rw-p 00000000 00:00 0 [stack]
7fff9a3d5000-7fff9a3d7000 r--p 00000000 00:00 0 [vvar]
7fff9a3d7000-7fff9a3d9000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
ERROR: signal ABORT received - internal error. Try to compile the program with TRAP LOCAL to find the cause.

This part seems to be reponsible:

wsz = 0
uni = 0
FOR y = 0 TO x
FOR slowo$ IN zastapione$[y]
IF LEN(slowo$) > 1 OR slowo$ = "a" OR slowo$ = "i" THEN
IF ISKEY(wszystkie, slowo$) = TRUE THEN
INCR wszystkie(slowo$)
INCR wsz
ELSE
wszystkie(slowo$) = 1
INCR uni
INCR wsz
ENDIF
ENDIF
NEXT
NEXT

[Deleted]

OK. I've figured it out. The problem was the x variable. Both lines with for y = 0 to x need to be changed to for y = 0 to x - 1. However... I'm pretty sure it simply worked with older versions of BaCon, while it shouldn't have. Also, the error message looked pretty scary.

[Pjot]

Hi Tomaaz,

Well, that could be - anyway, the BaCon array declarations, which resemble C, are pretty confusing. I make mistakes with them myself all the time. E.g. DECLARE x[10] means that there are elements from 0-9.

You can write to element x[10] though, sometimes this actually works.

So actually, I wonder if it really worked with the older BaCon version, I mean, it seemed as if it worked, but sometimes, the Linux kernel is kind enough to not throw an exception. This is pure luck.

But sometimes the kernel does throw an exception.

To really check a compiled program for pointer issues, I always use Valgrind. It will show illegal memory writes, uninitialized pointers, memory leaks, and what not.

In the end, we're dealing with compiled C code, and hence we also have to deal with those peculiarities...

Thanks and regards
Peter