Hermes Intelligence · Decision-grade (T-deep)

Meeting Transcription Engine — Refresh & Verify

meet.boas.dev · 2026-07-07 · extends the 2026-05-18 landscape arc
Owning board: metis #26 → Epic #110 Track D · Phase-1 infra #41

Lane A — fast-path Lane B — bilingual IS/EN Lane C — freshness + claim-verify

One-line verdict. The powerful live-transcription engine you want already exists and is already chosen (skynet + jigasi). It hasn't shipped because of infrastructure — a GPU host and the self-host Jitsi cutover — not tool choice. Meetily is the stopgap standing in exactly that gap. Don't re-tool; unblock the infra.

TL;DR

Right tool, already picked. skynet + jigasi + IS-Whisper + local LLM — ratified by athena, PRD'd by metis (#110). 7 weeks of freshness scanning found nothing that displaces it.
Blocker = infra, not tooling. meet.boas.dev today is JaaS (8x8-hosted), not self-hosted. Jigasi requires a self-operated Prosody/Jicofo/JVB stack (XMPP-config access JaaS won't expose). Gate: GPU host (metis-051) + self-host cutover (#41).
Speaker "magic" is REAL & code-verified. Jigasi maps each speaker's RTP stream → XMPP-roster name, tags every chunk before ASR. Beats acoustic diarization — but PARTIAL: a silent-drop failure mode on unmapped SSRCs, no accuracy study. Instrument + keep a fallback.
A "ship today" path exists but breaks self-host. JaaS-native transcription is already speaker-attributed, ~$0.06/min — but audio transits Google Cloud STT. Bridge-only, if the mandate flexes.
Bilingual IS/EN — the in-meeting language toggle is CONFIRMED (with a small build). Operator wants a deliberate mid-meeting language switch (no mid-sentence). jigasi already does the live half for free; skynet needs a scoped patch to 2 Python files (it latches language at first-chunk). No model-reload cost. Google Cloud ruled out → self-host skynet+jigasi is the only path.
Unclosed gap: no IS checkpoint publishes an English WER. Cheap to close — run an EN eval before locking the model.
Freshest catch: pyannote community-1 shipped today (2026-07-07) — new default for the 522 retro transcripts (#110-D4).

1 · The blocker is infra, not tooling

Every Meetily complaint is exactly what skynet+jigasi fixes:

Meetily limitation	skynet + jigasi
Processes background OS audio	Joins the conference server-side; transcribes the live call
No speaker attribution (one mixed stream)	Per-participant RTP stream → XMPP-roster name, before ASR
Doesn't slot into the pipeline	`/summarize` + `/action-items` → write-back to the meetings, CRM, and task tables
Single-machine operator dependency	Server-side service beside the Jitsi deploy

Why it's not live: per metis #26 — Epic #110 all at 🔬/❌, D1 gated on metis-051 GPU host; meet.boas.dev still on JaaS 8x8.vc (Phase-1 self-host cutover to jitsi.boas.dev not done, #41). Jigasi cannot attach to JaaS — it needs Prosody VirtualHost + brewery-MUC + Jicofo config that 8x8's closed service doesn't expose. The self-host dependency is hard, not arbitrary.

2 · Speaker attribution — VERIFIED (code-level), PARTIAL in practice

MECHANISM CONFIRMED JVB is an SFU → Jigasi gets N separate RTP streams (one SSRC/speaker, no mixing). Transcriber.findParticipant(ssrc) routes each buffer to its own Participant carrying getName()/getEmail() from XMPP presence; skynet just echoes the speaker_id|lang tag. Stronger than acoustic diarization for "who said this."

PARTIAL Caveats the build must handle: (1) unmapped-SSRC segments are silently dropped (GitHub issue #91, open; jigasi has no release trail to confirm a fix) → instrument for drops; (2) getName() falls back to literal "Fellow Jitser" for presence-less participants; (3) no independent accuracy study — forum traffic is setup pain, not misattribution. Keep a diarization fallback.

Cross-lane note: a secondary source claimed Jigasi "mixes all audio into one stream." The jigasi source code refutes this — per-SSRC separation, no mix step. Primary source wins.

3 · "Ship today" options — and the self-host wall

Option	Today?	Speakers?	Self-host / privacy	Effort	Cost
JaaS-native transcription	Yes	Yes	No — Google Cloud STT hop	Low	$0.06/min
Jigasi → JaaS	No	—	needs Prosody/Jicofo access	—	—
Custom participant-bot	Possible, no OSS tool	If built	STT stays local	Med-high build	infra
Phase-1 self-host + skynet/jigasi (ratified)	Blocked (GPU+cutover)	Yes	Fully self-hosted	scoped	infra

Under self-host mandatory, JaaS-native is disqualified as a standing solution — viable only as a deliberate bridge. No off-the-shelf tool reaches live + speaker-labeled + self-hosted without the cutover or a custom bot. Recall.ai / Vexa / Attendee / MeetingBaas — none support Jitsi.

4 · Bilingual IS/EN — the new axis

skynet's streaming_whisper requires a per-stream language hint (no auto-detect) — and that constraint is the answer:

RECOMMENDED Per-participant routing: stock whisper-large-v3(-turbo) on EN-primary speakers (zero forgetting risk), IS fine-tune on IS-primary speakers. Different checkpoints per stream — a config change, not code.
Prefer the LoRA adapter jonasaise (frozen base → preserves EN) over full fine-tunes sam8000 / LVL (best IS WER but EN-forgetting risk).
GAP No IS checkpoint publishes an English WER. Close it cheaply before locking the model — run a few hundred EN utterances through each candidate.
NOT NEEDED One speaker code-switching mid-sentence: no clean solution — but operator has ruled this out of scope.

CONFIRMED-with-build In-meeting language toggle (Lane D, source-verified across jigasi + lib-jitsi-meet + skynet master). A participant fires a command to flip IS↔EN; transcription switches from that point forward.

jigasi side = FREE. `setTranscriptionLanguage()` (already public) → XMPP presence → jigasi re-reads it live every presence update → rebuilds the per-chunk `speaker_id|lang` header fresh. Zero build.
skynet side = the only work. `streaming_whisper` latches each participant's language at first-chunk (`State.lang`) and never re-reads it → scoped patch to 2–3 Python files (re-read lang per chunk + flush buffer at the switch). No model reload — one shared multilingual model, `language=` is a per-call arg.
Effort: small-to-moderate. UI control (reuse shipped API) is cheap; the skynet patch is the real work but contained. Caveat: jigasi prefers translation-lang over source-lang — the toggle must leave the "translate to" picker unset.

5 · Freshness — nothing displaces the stack

skynet 2026.5.0 still current; 2026.2.0 added per-participant transcription stats (corroborates §2).
Vexa still no Jitsi. WhisperLive newest feature is June 2025 (a search summary mis-dated it 2026 — caught).
pyannote community-1 (v4.0) dropped 2026-07-07 — new default for #110-D4's 522 retro transcripts (no DER numbers yet → benchmark first).
NVIDIA streaming ASR (Nemotron/Parakeet) newer but a bigger backend swap + snippet-tier confidence; Sortformer diarizer capped at 4 speakers → unfit for real meetings.

6 · Recommendation & action plan

Headline: don't re-tool. Unblock the infra, close two small gaps, update the PRD with the verified caveats.

For metis (build owner, #110)

Treat GPU host (metis-051) + Phase-1 self-host cutover (#41) as the critical path to live transcription.
Adopt per-participant language routing in streaming_whisper (stock v3 EN / IS fine-tune IS). Update PRD ASR section (was IS-single-model).
Close the EN-WER gap before locking the IS checkpoint (extend metis-052 eval).
Downgrade roster-attribution to PARTIAL — instrument + fallback in the PRD.
Set pyannote community-1 as the D4 retro-diarization default (benchmark DER first).

For the operator — one decision (bridge policy while self-host lands)

Option	What you get	Trade
(a) Stay on Meetily	No third-party hop; keep waiting	No speaker attribution in the interim
(b) Bridge on JaaS-native	Live, speaker-attributed, ~1 day, $0.06/min	Audio → Google Cloud STT (flex self-host, time-boxed)
(c) Custom participant-bot	Self-host STT, no wait	Real build; ≈ cost of finishing cutover, no durable result — dominated

Hermes rec: (a) or (b) depending on urgency for live speaker-attributed notes — and push metis-051 + #41 as the actual fix. (c) is dominated.

Hermes research · research/meeting-transcription-refresh-2026-07-07/ · commit 01c964b
Full per-lane citations (all fetched 2026-07-07): sources/A-fast-path.md · B-bilingual-streaming.md · C-freshness-verify.md
Two checks before a build decision rests on them: (1) EN-WER of the chosen IS checkpoint, (2) NVIDIA streaming-ASR dates. Neither gates the headline (code-verified + deep-read tiers).