Vosk Speech Recognition

[juppel]

Hello,
I'm trying to convert the C++ code from vosk to Bacon Basic. But I have no idea how to do it. I've already started partially and so far there are no errors when converting. But the question is, is my approach correct?
How can I do this if I want to process the output in Baconbasic?

#include "vosk_api.h"
#include <stdio.h>

int main() {
   FILE *wavin;
   char buf[3200];
   int nread, final;

   VoskModel *model = vosk_model_new("model");
   VoskRecognizer *recognizer = vosk_recognizer_new(model, 16000.0);

   wavin = fopen("test.wav", "rb");
   fseek(wavin, 44, SEEK_SET);
   while (!feof(wavin)) {
        nread = fread(buf, 1, sizeof(buf), wavin);
        final = vosk_recognizer_accept_waveform(recognizer, buf, nread);
        if (final) {
            printf("%s\n", vosk_recognizer_result(recognizer));
        } else {
            printf("%s\n", vosk_recognizer_partial_result(recognizer));
        }
   }
   printf("%s\n", vosk_recognizer_final_result(recognizer));

   vosk_recognizer_free(recognizer);
   vosk_model_free(model);
   fclose(wavin);
   return 0;
}

 to

PRAGMA INCLUDE </vosk-api-master/c/src/vosk_api.h>
PRAGMA LDFLAGS vosk
PROTO vosk_recognizer_result
PROTO vosk_model_new

PROTO vosk_recognizer_accept_waveform
PROTO vosk_recognizer_partial_result
PROTO vosk_recognizer_final_result
PROTO vosk_recognizer_free

PROTO vosk_model_free


DECLARE buf[3200] TYPE char '??

LOCAL final TYPE int
LOCAL nread TYPE int


VoskModel *model = vosk_model_new("model")
VoskRecognizer *recognizer = vosk_recognizer_new(model, 16000.0)
wavin = fopen("test.wav", "rb")
final = vosk_recognizer_accept_waveform(recognizer, buf, nread)

'PRINT  vosk_recognizer_result(recognizer) 'Error Wai?
vosk_recognizer_result(recognizer) 'OK



[Pjot]

Hi juppel,

As it is a very small program, I could quickly convert it as follows:

PRAGMA OPTIONS -I.
PRAGMA INCLUDE <vosk-api/src/vosk_api.h>
PRAGMA LDFLAGS -L. vosk

ALIAS vosk_recognizer_result TO vosk_result$
ALIAS vosk_recognizer_partial_result TO partial_result$
ALIAS vosk_recognizer_final_result TO final_result$

DECLARE *wavin TYPE FILE
DECLARE buf[3200] TYPE char
DECLARE nread, final TYPE int

DECLARE *model TYPE VoskModel
DECLARE *recognizer TYPE VoskRecognizer

model = vosk_model_new("model")
recognizer = vosk_recognizer_new(model, 16000.0)

OPEN "test.wav" FOR READING AS wavin

SEEK wavin OFFSET 44

WHILE NOT(ENDFILE(wavin))
   GETBYTE buf FROM wavin CHUNK 3200 SIZE nread
   final = vosk_recognizer_accept_waveform(recognizer, buf, nread)

   IF final THEN
       PRINT vosk_result$(recognizer)
   ELSE
       PRINT partial_result$(recognizer)
   ENDIF
WEND

PRINT final_result$(recognizer)

CALL vosk_recognizer_free(recognizer)
CALL vosk_model_free(model)
CLOSE FILE wavin


Please note that I did not really test its functionality, as there is no "libvosk.so" on my system to link with (and I do not have the time right now to go through the installation procedure).

Also, I do not have a "test.wav" to use for this program.

But the above code compiles for me, and should help you on your way.

BR
Peter

[juppel]

Thank you very much. That works wonderfully.
Now I have a rough idea of ​​how something like that is done.
The code doesn't look easy to me. But I've learned something.

Thank you very much

juppel

[juppel]

good morning,

I have a few string functions here that are important for speech recognition for further processing

FUNCTION FreeHolderInstr(STRING hayst$, STRING need$) TYPE int
LOCAL lo
LOCAL from$
IF INSTR(need$, "*") THEN
IF LEFT$(need$, 1)="*" AND RIGHT$(need$, 1)="*" THEN
lo=LEN(need$)-1
from$=LEFT$(need$, lo)
from$=RIGHT$(from$, lo-1)
RETURN INSTR(hayst$, from$)
ENDIF
IF LEFT$(need$, 1)="*" THEN
lo=LEN(need$)-1
from$ = RIGHT$(need$, lo) + " "
RETURN INSTR(hayst$, from$)
ENDIF
IF RIGHT$(need$, 1)="*" THEN
lo=LEN(need$)-1
from$ = " " + LEFT$(need$, lo)
RETURN INSTR(hayst$, from$)
ENDIF
ELSE
RETURN ISTOKEN(hayst$, need$)
ENDIF
ENDFUNCTION

FUNCTION CheckMakroTrigger(STRING SpeechText$, STRING makroText$) TYPE int
LOCAL AmountTrigger, AmountTriggerFound, AmountSpeech, LastPosInSpeech
LOCAL Trigger$, OrTrigger$

' COLLAPSE$ entfernt die doppelten Leerzeichen
' CHOP$ entfernt die Leerzeichen am Anfang und Ende
makroText$=COLLAPSE$(makroText$, " ")
makroText$=CHOP$(makroText$)
makroText$=LCASE$(makroText$)
SpeechText$=" " + LCASE$(SpeechText$) + " "

AmountTrigger = AMOUNT(makroText$, " ")
AmountSpeech = AMOUNT(SpeechText$, " ")
' PRINT "Triggeranzahl: ";AmountTrigger

FOR Trigger$ IN makroText$ STEP " "
' PRINT "Trigger: ";Trigger$
IF FreeHolderInstr(SpeechText$, Trigger$) THEN
INCR AmountTriggerFound
LastPosInSpeech = MAX(LastPosInSpeech,ISTOKEN(SpeechText$, Trigger$))
ENDIF

IF INSTR(Trigger$, "/") THEN
FOR OrTrigger$ IN Trigger$ STEP "/"
' PRINT "Or Trigger: ";OrTrigger$
IF FreeHolderInstr(SpeechText$, OrTrigger$) THEN
INCR AmountTriggerFound
LastPosInSpeech = MAX(LastPosInSpeech,ISTOKEN(SpeechText$, OrTrigger$))
BREAK
ENDIF
NEXT
ENDIF
NEXT
' PRINT "AmountTriggerFound: ";AmountTriggerFound
' PRINT
LastTriggerFound$ = TOKEN$(SpeechText$, LastPosInSpeech)
' PRINT "LastPosInSpeech: ";LastPosInSpeech
IF AmountTrigger=AmountTriggerFound AND AmountSpeech >= AmountTriggerFound THEN RETURN 1
ENDFUNCTION



Speech$="light on"

IF CheckMakroTrigger(Speech$, "*ight* on/switch") THEN
PRINT "letzter Trigger der gefunden wurde: "; LastTriggerFound$
PRINT "OK"
ELSE
PRINT "leider nicht gefunden!!"
ENDIF

'in German

Speech$="Licht bitte einschalten"

IF CheckMakroTrigger(Speech$, "*icht* ein*/an*") THEN  :'ein or an
PRINT "letzter Trigger der gefSometimes it understands night on instead of llight on or in German
switch onunden wurde: "; LastTriggerFound$
PRINT "OK"
ELSE
PRINT "leider nicht gefunden!!"
ENDIF


Sometimes it understands night on instead of llight on or in German: "nicht anschalten" or "Licht einschalten" or "Licht an or Licht ein" then my light goes on

It would be cool if ISTOKEN had such a placeholder function. ISTOKEN("lights on", "light*")  :-)

or is there already something?

nice Day

Juppel

[Pjot]

Hi Juppel,

In case of delimited strings, you could use MATCH here:

a$ = "lights on"
IF MATCH(a$, "lights *") THEN PRINT "Found"


HTH
Peter

[juppel]

Hi,

but ein need

vosktext$ = "Hello computer lights on"
'or misunderstands trigger
vosktext$ = "Hello computer night on"
IF MATCH(vosktext$, "* *ight* on *") THEN PRINT "Found" 'would be great if it were possible
vosk sometimes misunderstands something e.g. nights on or light instead of lights

my function works, but it's just an idea.
But it still helped me. Thanks
best regards

juppel

[Pjot]

No worries, the MATCH function is meant to match the items in a delimited string. But a delimited string is a string as well, it is just a special kind of string.

Therefore, you can use REGEX as well to match your string against regular expressions:

vosktext$ = "Hello computer night on"

IF REGEX(vosktext$, ".* ?ight.* on") THEN PRINT "Found"


You may have to fiddle around with the regex syntax a bit. By default, BaCon uses the POSIX ERE syntax described here but you can change this to PCRE if you're interested. This is explained in the manual - see the chapter about regular expressions.

HTH
Peter

[juppel]

hello

REGEX is great, it really helped a lot.

now I have written a voice control in Bacon Basic. This was for my blind friend. You can control the entire PC and devices with it.

vosk is used for recognition (stt). I work with the python script microphone.py, whose output I use for my speech conversion.
Unfortunately I couldn't do it with Basic. I should be able to read the microphone with Basic

It can also output a response set if appropriate programs are set up. Texts can be written or messages can be sent. Everything is explained again in the file. I had already written the program in freebasic and wanted to translate it for baconbasic, which works so much better and saves half the code.

The readme file shows which files can be found where and which programs are still needed!

have fun trying it out.

Attachments:

micestt.tar.gz (72.43 KB)

[Pjot]

Thanks juppel,

Unfortunately I couldn't do it with Basic. I should be able to read the microphone with Basic

BaCon has no native graphics or sound processing capabilities, so it has to make use of an external library (same as Python).

As BaCon converts to C, you have to look for a library in C which can process sound. I myself like the OpenAL library a lot, but I am not sure if it can capture a microphone.

The following page however contains a lot of resources all of which can be used by BaCon: Single File Libraries

These are small libraries, lots of them with permissive licenses, perfect for the tiny tools BaCon can produce.

So my choice there would be miniaudio which comes with a small set of C API calls.

BR
Peter