related live translation projects
For planning a universal solution #161 we investigate some relaated proects and their approach.
RTranslator
Android app: bi-directional audio live translation via bluetooth and Google API https://github.com/niedev/RTranslator
Polyglot for OBS
Live translation https://obsproject.com/forum/resources/polyglot-real-time-local-translation-ai-service-for-obs.1818/
LocalVocal for OBS
Live transcriptions https://github.com/occ-ai/obs-localvocal
OBS live translation
Uses the webkitSpeechRecognition provided by Chrome Browser to transform the input speech audio to text. See https://github.com/eddieoz/OBS-live-translation
live captioning & translation of video calls
using the agora.io SDK & service https://dev.to/akshatvg/build-a-live-translated-transcriptions-service-within-your-video-call-web-app-4dhl
FreeSwitch
FreeSwitch is the audio gateway in BBB (and we use it at fairkom with FusionPBX). There exist a number of modules for transcription, TTS (also for a Whisper stream), translation https://github.com/jambonz/freeswitch-modules - uses a JS library to hook into freeswitch https://www.npmjs.com/package/drachtio-fsmrf
Overview article https://sheerbit.com/text-to-speech-and-speech-to-text-in-freeswitch/ - sheerbit has no github repo - sent them a mail if they are interested to cooperate
Transposer
Universal language processing queue https://git.fairkom.net/emb/displ.eu/transposer/
Subtitling & translation
Whisper for BBB: https://github.com/bigbluebutton-bot/bbb-translation-bot