Array benchmark

[Deleted]

I've found another "interesting" topic on BP.org: forum.basicprogramming.org/index.php?topic=2608.msg18990#msg18990. JRS says that BaCon takes more than an hour and a half to complete that task, so I've found wordcitiespop.txt file ( www.maxmind.com/download/worldcities/ ) that is 144 MB big and written my own program:

OPTION BASE 1
GLOBAL all$[100000000] TYPE STRING

OPEN "worldcitiespop.txt" FOR READING AS ksiazka
counter = 0

WHILE NOT(ENDFILE (ksiazka)) DO
READLN akapit$ FROM ksiazka
SPLIT akapit$ BY "," TO slowa$ SIZE siz
FOR x = 1 TO siz
all$[counter + 1] = slowa$[x]
NEXT X
counter = counter + siz
WEND

CLOSE FILE ksiazka

PRINT counter

The result is 22217713 (more than 22 millions) and, on my computer, it takes BaCon... 11 sec. to complete the task.

EDIT: Here is a version that uses REDIME statement to change a final size of the main array:

OPTION BASE 1
GLOBAL all$ TYPE STRING ARRAY 100000000

OPEN "worldcitiespop.txt" FOR READING AS ksiazka
counter = 0

WHILE NOT(ENDFILE (ksiazka)) DO
READLN akapit$ FROM ksiazka
SPLIT akapit$ BY "," TO slowa$ SIZE siz
FOR x = 1 TO siz
all$[counter + 1] = slowa$[x]
NEXT X
counter = counter + siz
WEND

CLOSE FILE ksiazka

PRINT counter
REDIM all$ TO counter

This one is a bit slower. It takes 18 sec. to complete the task.

Peter, please post my examples on BP.org (I was banned from that forum) to show everybody that what JRS says is simply not true (he just doesn't know how to use BaCon). You could also write much faster example using more advanced techniques, but I don't think there is a point to waste your time for it.

Regards!

[vovchik]

Dear Tomaaz,

Dzięki za oświecenie. Dlaczego on (J. Spikowski) to robi? Nie pierwszy raz

With kind regards,
vovchik

[Deleted]

Jun 3, 2012 14:41:42 GMT 1 vovchik said:

Dzięki za oświecenie. Dlaczego on (J. Spikowski) to robi? Nie pierwszy raz

Vovchik, are you from Poland?

[Pjot]

Hi Tomaaz,

Thanks for noticing, but let him be. He obviously is demonstrating his childish behavior on BP.org because nobody is interested in his ScriptBasic evangelism. The fact that he canceled his account on this forum shows that indeed he is not interested in BaCon at all, but just wanted to use the threads here to promote his preoccupations.

But it is not worth the energy. It's boring even. Let's stick to things which are fun.

I think your program does not do the correct thing, by the way. Shouldn't you CONCAT the all$[counter] variable?

Regards
Peter

[Deleted]

Peter, the goal is to convert a text file to a string array. This is what my program does. It creates all$ array that contains the whole text file in 22217713 elements. JRS's ScriptBasic and BaCon examples are doing exactly the same thing. His BaCon example is doing it a different way (that is much slower), but the result is exactly the same.

admin said:

I think your program does not do the correct thing, by the way. Shouldn't you CONCAT the all$[counter] variable?

No, because the task is to create an array. JRS builds a huge string (I think this is the slow part of his program) and then split it for a huge array. I split smaller strings for a smaller arrays and build a final one (which is huge) from them. I don't use CONCAT and this is why my program is much faster (I think so).

I know it's boring, but I still think you should post it on BP.org. Not for JRS. For other members. People shouldn't think that BaCon needs an hour and a half to complete that task.

[Pjot]

Ok Tomaaz, sorry for my confusion, it's just that I did not pay good attention because this discussion delivers a lot of negative energy. And I like to enjoy my Sundays

Anyway, I have posted it (and modified it slightly) so we'll see...

Thanks again!
Peter

[Pjot]

Tomaaz, this one is even faster, takes 9 seconds on my slow EEEPC (1.6 Ghz). Though it doesn't create an array, it produces the same output.


CONST file$ = "worldcitiespop.txt"

' Declare memory
fsize = FILELEN(file$)
mem = MEMORY(fsize)

' Get the complete text into memory
OPEN file$ FOR READING AS cities
GETBYTE mem FROM cities SIZE fsize
CLOSE FILE cities

' Check on commas
term = 0
FOR x = 0 TO fsize-1
IF PEEK(mem+x) = ASC(",") THEN INCR term
NEXT

PRINT term

FREE mem


[vovchik]

Dear Peter,

Master Fuke travelled around saying, “If a clear mind comes, I let it be clear. If a deluded mind comes, I let it be deluded."

With kind regards,
vovchik

[Pjot]

A very deep gassho to you, my friend

_()_

[Deleted]

Jun 3, 2012 18:20:53 GMT 1 Pjot said:

And I like to enjoy my Sundays

Sure, that's a better thing to do, but I couldn't just ignore "an hour and a half and still not completed...". To be honest, I made a small mistake in my examples (it's fixed now - should be all$[counter + 1] = slowa$[x] instead of all$[counter] = slowa$[x]), but, as it doesn't affect the speed of the program at all, there is no point to change anything on BP.org. And I think that you can close this topic now.

Enjoy the rest of your Sunday!

[Deleted]

Of course, JRS checked my program and found out it doesn't work properly: forum.basicprogramming.org/index.php?topic=2608.msg18995#new. Can someone post a fixed version, please? Here it is:

OPTION BASE 1
GLOBAL all$[100000000] TYPE STRING

OPEN "worldcitiespop.txt" FOR READING AS txtfile
counter = 0

WHILE NOT(ENDFILE (txtfile)) DO
READLN line$ FROM txtfile
SPLIT line$ BY "," TO term$ SIZE size
FOR x = 1 TO size
all$[counter + x] = term$[x]
NEXT X
INCR counter, size
WEND

CLOSE FILE txtfile

PRINT counter

Guys, I won't bore you any more with it. I promise!