While this is certainly a cool concept, local voice assistants like this are currently a novelty. Cool to play around with, though!
You can expect around 5 seconds processing time to start generating the response to a basic question on a very basic model like Llama 3 8B.
For context, using Moondream2 (as recommended) on a RasPi 5, it takes around 50 seconds to process an image taken by the Camera and start generating a description.
I look forward to more voice assistants that arent as laughably broken as Google/Alexa.
My Google Nest can’t even subtract from a timer (it can add, though!). Such a blatant surveillance device.
“Ok, Google, add -5 minutes to the timer”
It either explodes, or you start getting notifications in the past. Seems like a possible win, either way.
While this is certainly a cool concept, local voice assistants like this are currently a novelty. Cool to play around with, though!
You can expect around 5 seconds processing time to start generating the response to a basic question on a very basic model like Llama 3 8B.
For context, using Moondream2 (as recommended) on a RasPi 5, it takes around 50 seconds to process an image taken by the Camera and start generating a description.
Interesting, using whisper-fast on Home Assistant on my server computer takes like 2-3 seconds to process and delivery an output in English.
Useful in the smart home space.
Laughably broken in most other languages other than English, but then again, google and Alexa barely work in other languages.