Text vs Sqlite Benchmark

[2lss]

I need a good method for storing some data (for another program) so I made this program to compare plain text files vs. a sqlite database.

Basically it just enters random strings of data in the form of a table into either a text file or a sqlite database. From there you can read the files to get an idea of which method is faster. The downside is that you have to use the 'time' shell command due to the fact that the bacon SECONDS function only returns whole seconds.

EDIT - Per vovchik's suggestion, I have added TIMER for benchmarking. No need to use the 'time' shell command.

From what I see, it takes about the same amount of time to read a text file versus a sqlite database of the same dimensions, however when writing, the sqlite database is much slower than the text file. Overall I think the moral of the story is that if your reading from the sqlite database then your ok, but writing large amounts of data may take awhile.

Make sure to use the patched version of the sqlite bacon include file. I had to change a few things to suppress the column headings.

Examples:

./txtvsdb -r 100 -c 10 -s 5 --write-text
Write a table to a text file that has 100 rows, 10 columns, and each string is 5 random characters long.

./txtvsdb -r 100 -c 10 -s 5 --write-db
Same as above but writes to a sqlite database instead

./txtvsdb --read-text -v
Reads the contents of the text file and stores the values into an associative array. The -v flag causes the values to be printed.

./txtvsdb --read-db
Same as above but with a sqlite database and no output.

Any thoughts or suggestions welcome ;D

EDIT - There is no way for my to update the attachment. See post five for the new version.

Attachments:

[vovchik]

Dear 2lss,

Use TIMER for microsecond granularity. it is there precisely to time the duration of routines....

With kind regards,
vovchik

[2lss]

Thanks vovchik! That will work great!

I always skip over that function when looking through the docs. It reminds me of TIMEOUT in hug.

[vovchik]

Dear 2lss,

I tend to skip it too, but saw somebody make good use of it in timing alternative routines and it has now stuck.

With kind regards,
vovchik

[2lss]

Thanks to vovchik's suggestion, it can now time the benchmark without using the external 'time' command!


Here is the updated version

[2lss]

Here is the updated version

Attachments:

[gcav]

Jun 19, 2013 22:53:44 GMT 1 2lss said:

Here is the updated version

I have not update the PBL Isam library in a while. When I have a little bit of time, I will
add the ISAM access to 2lss source code, so we can compare the flat file, sql, and PBL Isam implementation...

gc