Compression and decompression

[bigbass]

Hello Peter

after some sleep I tried again

I was careful to copy and test several times
before posting .It seems that POKE and the result of ASC
is the cause somewhere


because if I split the two commands up
it complies and works correctly now


line 51 original code causes the error

OPTION MEMTYPE unsigned char
FOR y = 1 TO LEN(out$)
POKE result+byte, ASC(MID$(out$, y, 1))
INCR byte
NEXT



splitting the ASC code to get a result first then using the result
with POKE now works



 
OPTION MEMTYPE unsigned char
FOR y = 1 TO LEN(out$)
ASCII_PART = ASC(MID$(out$, y, 1))  
POKE result+byte, ASCII_PART
INCR byte
NEXT





Thanks for the code it was tested to work correctly
for me but still don't no why ASC and POKE had a problem
without digging deeper in the code or I need to install
the latest beta will try with that next




UPDATED installed latest beta
you fixed it somewhere between the official stable and latest beta!
all is fine

Joe


pi@raspberrypi:~/Public/New $ ./decomp myinstalled_packages.txt.lzw
Done in 375 msecs.
pi@raspberrypi:~/Public/New $

[Pjot]

Hi Joe,

From your logging I could see it was a '(' and/or ')' issue in the POKE statement, which indeed has been improved in 3.8 and 3.8.1.

Great that is was resolved!

Pls note that it is my intention to release 3.8.1 on January 1, 2019. As always, do not hesitate to mention any bugs or issues here. Including errors in documentation.

Best regards,
Peter

[Pjot]

All,

The compression techniques mentioned before almost always generate binary data as a result.

However, now I needed to compress ASCII text, but in such a way that the result is ASCII text as well. So compressing from ASCII to ASCII. I have been looking for an algorithm which could do such a thing in a small and fast way.

Actually, we could make this a contest: create a program which compresses the 96 printable ASCII characters to a result which consist of 96 printable characters.

The below code is inspired by the Predictor Compression Protocol with some modifications. The idea is to predict text based on previous snippets of text.

The code simply takes 6 characters at a time, stores this in a associative array, assigning the 7th character. Then it shifts to characters 2-7 and assigns the 8th character, and so on.

• During compression, it builds this list with associative values, thereby checking how many characters of the current chunk can be predicted, and it stores an indicator byte plus the characters is could not predict.
• During decompression, it builds the same list with associative values, and depending on the indicator byte, it produces characters from the associative list or from the input stream.

This approach has some very nice benefits:

1. Output is in ASCII
2. No need to store an index table to map compressed parts of the data
3. Very small and simple implementation, even works in Shell script (see below)
4. Reasonable compression ratios, for computer source code it usually is up to 55% to 60% compression

This is the code for PCP compression. As with BaCon, it is MIT license

CONST BS = 6

FUNCTION pcp_encode$(data$)

   LOCAL Gues$ ASSOC STRING
   LOCAL txt$, result$, c$, pr$, old$, pcp$
   LOCAL i, flag

   WHILE TRUE
       txt$ = MID$(data$, 1, BS)

       flag = 0
       result$ = ""

       FOR i = 1 TO LEN(txt$)
           c$ = MID$(txt$, i, 1)
           pr$ = RIGHT$(pr$, BS-1) & old$

           IF Gues$(pr$) = c$ THEN
               flag = flag | BIT(i-1)
           ELSE
               Gues$(pr$) = c$
               result$ = result$ & c$
           ENDIF

           old$ = c$
       NEXT

       pcp$ = pcp$ & CHR$(flag+33) & result$

       IF LEN(data$) <= BS THEN BREAK

       data$ = MID$(data$, BS+1)
   WEND

   RETURN pcp$

ENDFUNCTION

FUNCTION pcp_decode$(pcp$)

   LOCAL Table$ ASSOC STRING
   LOCAL pq$, c$, total$
   LOCAL i, flag

   WHILE LEN(pcp$) > 0

       c = ASC(LEFT$(pcp$, 1))-33
       pcp$ = MID$(pcp$, 2)

       FOR i = 1 TO BS

           IF c & BIT(i-1) THEN
               c$ = Table$(pq$)
           ELSE
               c$ = LEFT$(pcp$, 1)
               pcp$ = MID$(pcp$, 2)

               Table$(pq$) = c$
           ENDIF

           total$ = total$ & c$
           pq$ = RIGHT$(pq$, BS-1) & c$
       NEXT
   WEND

   RETURN total$

ENDFUNCTION

'------------------------------------------------------------------------------------------
' Main
'------------------------------------------------------------------------------------------

' Get filename
IF AMOUNT(ARGUMENT$) < 2 THEN
   PRINT "Usage: ", ME$, " <file>"
   END 1
ENDIF

ascii$ = LOAD$(DIRNAME$(ME$) & "/" & TOKEN$(ARGUMENT$, 2))

compr$ = pcp_encode$(ascii$)

SAVE compr$ TO DIRNAME$(ME$) & "/" & TOKEN$(ARGUMENT$, 2) & ".pcp"

'------------------------------------------------------------------------------------------

ret$ = pcp_decode$(compr$)

SAVE ret$ TO DIRNAME$(ME$) & "/" & TOKEN$(ARGUMENT$, 2) & ".pcp.org"


BR
Peter

[Pjot]

And this is a PCP decoder, but in Shell Script (didn't make an encoder yet).

I am only showing this to prove my point: the PCP compression is really easy and small with a relatively good benefit.

This code needs a filename as argument:

#!/usr/bin/env bash
#
# Works with BASH, ZSH, and KSH93 - PvE. MIT License.
#

set -f

function pcp_decode
{
   typeset -A Table
   typeset Pcp Ch Total Pq="      "
   typeset -i i v

   Pcp="${1}"

   while [[ ${#Pcp} -gt 0 ]]
   do
       ((v=$(printf "%d" "'${Pcp:0:1}")-33))
Pcp=${Pcp:1}

       for i in $(seq 1 6)
       do
           if [[ $((${v} & 1)) -ne 0 ]]
           then
               Ch=${Table[${Pq}]}
           else
Ch="${Pcp:0:1}"
       Pcp="${Pcp:1}"
               Table[${Pq}]="${Ch}"
           fi

   Total="${Total}${Ch}"
           Pq="${Pq:1:5}${Ch}"

           ((v=${v}>>1))
       done
   done

   echo "${Total}"
}

echo "$(pcp_decode "$(cat ${1})")"