Problems with speech recognition

Blane

Hello bugsounet, first of all many thanks for your great work. Do you have any idea where the problem could be? The google assistant does not respond to any voice input. In the terminal output of gateway there is a warning from arecord(screenshot1) every second. As I understand it does not find the micro. So I started looking for the device name in your files and found the following entry in three files:

/MagicMirror/modules/EXT-Detector/components/porcupine.js
./MagicMirror/modules/EXT-Detector/node_modules/@bugsounet/snowboy/index.js:
./MagicMirror/modules/EXT-Detector/node_modules/@bugsounet/snowboy/sample.js:

device=“plughw:1”

When I tried to start a recording with arecord --device =plughw:1 --format… I tried to start a recording and got an error. So I searched for my device name, it is “plughw:2,0”. At least I can start a recording with arecord --device =plughw:2,0 --format…(screenshot2) and record sound. After I have adjusted the entries in the three files (screenshot3), unfortunately the message still appears and the voice input still does not work. Maybe you have an idea what else it could be?

bugsounet

Hi,

there is no micConfig to do
why ?

since last version of buster and now bulleyes, RPI os (like all linux debian compliant) use a program named pulseaudio
this program is able to::

determinate what recorder is plugged (with device name)
record and use all recorder at the same time without comflit of take hand

the device name is default

You can discover it with arecord -L command

default
    Playback/recording through the PulseAudio sound server

After if you modify .asoundrc file , you break the configuration and it can’t works
If you remove /disable or uninstall pulse audio server, it will not works too

Finaly, I force this configuration (and you can’t change it)
mic: { recorder: 'auto', device: 'default' }

Blane

I had no intention of changing the micro configuration. The voice input did not work and then I came across the warning in the gateway terminal output. I just wonder why the voice input does not work. I have installed and configured everything according to your instructions. When I make a request to google via the gateway tools, it is answered and displayed on the mirror.

bugsounet

first thing to verify is this:

Gateway v3.0.1 (230316.fr) 

pi@raspberrypi:~/MagicMirror $ ps aux | grep pulse
pi         686  0.2  0.4 648580  8612 ?        S<sl Mar15   3:49 /usr/bin/pulseaudio --daemonize=no --log-target=journal
pi        6775  0.0  0.0   4892   572 pts/0    S+   22:47   0:00 grep --color=auto pulse

and this:

pi@raspberrypi:~ $ arecord -L               
null
    Discard all samples (playback) or generate zero samples (capture)
default
    Playback/recording through the PulseAudio sound server
lavrate
    Rate Converter Plugin Using Libav/FFmpeg Library
samplerate
    Rate Converter Plugin Using Samplerate Library
speexrate
    Rate Converter Plugin Using Speex Resampler
jack
    JACK Audio Connection Kit
....

and OS version:

pi@raspberrypi:~ $ uname -a
Linux raspberrypi 5.15.84-v7l+ #1613 SMP Thu Jan 5 12:01:26 GMT 2023 armv7l GNU/Linux

pi@raspberrypi:~ $ cat /etc/os-release   
PRETTY_NAME="Raspbian GNU/Linux 11 (bullseye)"
NAME="Raspbian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

and verify if:

.soundrc is NOT created

pi@raspberrypi:~ $ cd ~
pi@raspberrypi:~ $ cat .soundrc
cat: .soundrc: No such file or directory

Blane

Here are the outputs

admin@raspberrypi:~/MagicMirror $ ps aux | grep pulse
admin      721  0.0  0.4 920580  7928 ?        S<sl 15:58   0:16 /usr/bin/pulseaudio --daemonize=no --log-target=journal
admin    13587  0.0  0.0   3536  1624 pts/0    S+   22:54   0:00 grep --color=auto pulse

admin@raspberrypi:~ $ arecord -L
null
    Discard all samples (playback) or generate zero samples (capture)
default
    Playback/recording through the PulseAudio sound server
lavrate
    Rate Converter Plugin Using Libav/FFmpeg Library
samplerate
    Rate Converter Plugin Using Samplerate Library
speexrate
    Rate Converter Plugin Using Speex Resampler
jack
    JACK Audio Connection Kit
oss
    Open Sound System
pulse
    PulseAudio Sound Server
upmix
    Plugin for channel upmix (4,6,8)
vdownmix
    Plugin for channel downmix (stereo) with a simple spacialization
usbstream:CARD=b1
    bcm2835 HDMI 1
    USB Stream Output
usbstream:CARD=Headphones
    bcm2835 Headphones
    USB Stream Output
hw:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    Direct hardware device without any conversions
plughw:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    Hardware device with all software conversions
sysdefault:CARD=Device
    USB PnP Sound Device, USB Audio
    Default Audio Device
front:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    Front output / input
dsnoop:CARD=Device,DEV=0
    USB PnP Sound Device, USB Audio
    Direct sample snooping device
usbstream:CARD=Device
    USB PnP Sound Device
    USB Stream Output
usbstream:CARD=UACDemoV10
    UACDemoV1.0
    USB Stream Output

admin@raspberrypi:~ $ uname -a
Linux raspberrypi 5.15.84-v7l+ #1613 SMP Thu Jan 5 12:01:26 GMT 2023 armv7l GNU/Linux

admin@raspberrypi:~ $ cat /etc/os-release 
PRETTY_NAME="Raspbian GNU/Linux 11 (bullseye)"
NAME="Raspbian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

admin@raspberrypi:~ $ cat .soundrc
cat: .soundrc: No such file or directory

bugsounet

ok perfect !

Can you retry with this:

pi@raspberrypi:~ $ cd ~ && cat .asoundrc 
cat: .asoundrc: No such file or directory

sorry

bugsounet

After, I think to this:

Maybe you use another process or module in MagicMirror that uses your microphone with old protocol.
In this case, your micro phone will be not available

You can try this:

turn off MagicMirror app
open a terminal and try this:

arecord -D default -q -r 16000 -c 1 -t wav -f S16_LE -d 3 test.wav && aplay test.wav

it will use pulse audio for opening the mic, record 3 seconds and play test.wav file

tell me if any error ?

If error can give me the result of this:

pi@raspberrypi:~ $ ps aux | grep record

Blane

Thank you very much for your support, unfortunately I can’t work on the mirror again until Monday. I will try and post the outputs on Monday

Blane

Hello bugsounet, tried what you wrote, the file .asoundrc does not exist and the recording started and played back without error.

cd ~ && cat .asoundrc 
cat: .asoundrc: No such file or directory

admin@raspberrypi:~ $ arecord -D default -q -r 16000 -c 1 -t wav -f S16_LE -d 3 test.wav && aplay test.wav
Wiedergabe: WAVE 'test.wav' : Signed 16 bit Little Endian, Rate: 16000 Hz, mono

admin@raspberrypi:~ $ ps aux | grep record
admin     3575  0.0  0.0   3536  1552 pts/0    S+   12:53   0:00 grep --color=auto record

bugsounet

Hi,

For me it’s good.

So, Can you try to disable all modules in your config except:

MMM-GoogleAssistant
EXT-Detector
Gateway
EXT-Alert

and retry ?

Blane

Hello bugsounet, unfortunately, nothing has changed. The file is still not present and the recording worked and was played back. After deactivating the modules, the MagicMirror just stays black and I can no longer reach the gateway

admin@raspberrypi:~ $ cd ~ && cat .asoundrc 
cat: .asoundrc: No such file or directory

admin@raspberrypi:~ $ arecord -D default -q -r 16000 -c 1 -t wav -f S16_LE -d 3 test.wav && aplay test.wav
Wiedergabe: WAVE 'test.wav' : Signed 16 bit Little Endian, Rate: 16000 Hz, mono

admin@raspberrypi:~ $ ps aux | grep record
admin     2602  0.0  0.0   3536  1660 pts/0    S+   23:48   0:00 grep --color=auto record

bugsounet

@bugsounet said in Problems with speech recognition:

Hi,

For me it’s good.

So, Can you try to disable all modules in your config except:

MMM-GoogleAssistant

EXT-Detector

Gateway

EXT-Alert

and retry ?

It’s mean Start only MagicMirror with this
And try if it’s works.
I suspect another module that use your
Microphone Because command lines works

Blane

Hi, I have now disabled all modules except yours. Unfortunately, the speech recognition still does not respond. It still comes to the warning in the terminal output from the gateway

[22.03.2023 15:03.37.228] [LOG]   [LPCM16] WARN: arecord: main:830: Error when opening the device: File or directory not found

Here is my config file

/*** GENERATED BY @bugsounet Gateway v3.0.0 ***/
/*** https://forum.bugsounet.fr **/

var config = {
  address: 'localhost',
  port: 8080,
  basePath: '/',
  ipWhitelist: [
    '127.0.0.1',
    '::ffff:127.0.0.1',
    '::1'
  ],
  useHttps: false,
  httpsPrivateKey: '',
  httpsCertificate: '',
  language: 'de',
  locale: 'de-DE',
  logLevel: [
    'INFO',
    'LOG',
    'WARN',
    'ERROR'
  ],
  timeFormat: 24,
  units: 'metric',
  modules: [
    /***{
      module: 'alert'
    },
    {
      module: 'updatenotification',
      position: 'top_bar'
    },
    {
      module: 'clock',
      position: 'top_left',
      config: {
        timeFormat: 24,
        timezone: 'Europe/Berlin'
      }
    },
    {
      module: 'calendar',
      header: 'Kalender',
      position: 'top_left',
      config: {
        calendars: [
          {
            symbol: 'calendar-alt',
            url: ********
          }
        ]
      }
    },
    {
      module: 'compliments',
      position: 'lower_third'
    },
    /***{
      module: 'weather',
      position: 'top_right',
      config: {
        weatherProvider: 'openweathermap',
        type: 'current',
        location: 'Kleinaitingen',
        locationID: '2889814',
        apiKey: '***********'
      }
    },
    {
      module: 'weather',
      position: 'top_right',
      header: 'Weather Forecast',
      config: {
        weatherProvider: 'openweathermap',
        type: 'forecast',
        location: 'Kleinaitingen',
        locationID: '2889814',
        apiKey: '*************'
      }
    },
    {
      module: 'newsfeed',
      position: 'bottom_bar',
      config: {
        feeds: [
          {
            title: 'ARD News',
            url: 'http://www.tagesschau.de/xml/rss2'
          }
        ],
        showSourceTitle: true,
        showPublishDate: true,
        broadcastNewsFeeds: true,
        broadcastNewsUpdates: true
      }
    },
    {
      module: 'MMM-SmartTouch',
      position: 'bottom_center',
      config: {}
    },
    {
      module: 'MMM-Face-Recognition-SMAI',
      position: 'top_right',
      config: {}
    },***/
    {
      module: 'MMM-GoogleAssistant',
      configDeepMerge: true,
      config: {
        debug: false,
        stopCommand: 'stop',
        assistantConfig: {
          lang: 'de-DE',
          latitude: 48.21953,
          longitude: 10.86797
        },
        responseConfig: {
          useFullscreen: false,
          chimes: {},
          imgStatus: {},
          zoom: {}
        },
        recipes: []
      }
    },
    {
      module: 'Gateway',
      config: {
        debug: false,
        port: 8081,
        username: 'admin',
        password: 'admin',
        noLogin: false,
        usePM2: false,
        PM2Id: 0,
        useMapping: false,
        portMapping: 8081
      }
    },
    {
      module: 'EXT-Detector',
      position: 'top_left',
      disabled: false,
      configDeepMerge: true,
      config: {
        debug: false,
        useIcon: true,
        touchOnly: false,
        porcupineAccessKey: '*************',
        porcupineCustomModel: null,
        detectors: [
          {
            detector: 'Snowboy',
            Model: 'jarvis',
            Sensitivity: null
          },
          {
            detector: 'Porcupine',
            Model: 'ok google',
            Sensitivity: null
          },
          {
            detector: 'Porcupine',
            Model: 'hey google',
            Sensitivity: null
          }
        ]
      }
    },
    {
      module: 'EXT-Alert',
      config: {
        debug: false,
        ignore: []
      }
    },
    {
      module: 'EXT-UpdateNotification',
      position: 'top_bar',
      configDeepMerge: true,
      disabled: false,
      config: {
        debug: false,
        updateInterval: 600000,
        startDelay: 60000,
        ignoreModules: [],
        updateCommands: [],
        notification: {
          useTelegramBot: true,
          sendReady: true,
          useScreen: true,
          useCallback: true
        },
        update: {
          autoUpdate: true,
          autoRestart: true,
          usePM2: false,
          PM2Name: '0',
          defaultCommand: 'git --reset hard && git pull && npm install',
          logToConsole: true,
          timeout: 120000
        }
      }
    }
  ]
}

/*************** DO NOT EDIT THE LINE BELOW ***************/
if (typeof module !== 'undefined') {module.exports = config;}

bugsounet

and Detector wake words works ?

Blane

No, it does not respond to any voice input at all.The only thing that works, is when I make a request to Google through the gateway tools, it is answered and displayed on the mirror.

Blane

Hi, I just saw that there is a new error in the terminal output.

[22.03.2023 15:21.54.957] [LOG]   [LPCM16] WARN: ALSA lib pcm_dsnoop.c:638:(snd_pcm_dsnoop_open) unable to open slave

pitchx

have you tried this solution ?

Blane

Hi bugsounet, the topic can be closed, I started the project completely from scratch, reinstalled everything and started with Google Assistant first. Now the voice input works, thanks for your work and help

Solved Problems with speech recognition