I'm making my own, for personal use. I did a survey of many and they all (that I could find) skip the fundamentals.
The major issues that I've run into:
- Crash recovery. Most of these apps are incredibly buggy and crash all the time, taking the recorded audio with them. Macwhisper is incredibly bad at this.
- Disk space. Many of these apps save wav files to disk. After a few hours of meetings, you may end up with gigabytes eaten.
- Microphone bleed. People don't always use headphones, the system mic will pick up the speaker sounds, causing duplicate (approximately) transcriptions.
I've yet to find a solution that handles all these correctly, let alone having high quality transcriptions.
Anyway, most of these apps are built around https://github.com/FluidInference/FluidAudio, if anyone is curious. Their readme has a big list of similar apps as well.
I think I've got the other two bits covered. I pushed an update yesterday that adds active echo cancellation so that audio playing through the speakers (or leaky headphones) won't get transcribed twice if it is picked up by the microphone. It can be disabled in preferences, but it's on by default.
The disk space issue is one that I considered as well. By default, Trace deletes the actual audio recordings as soon as transcription is successfully completed, so the idea is you keep just the markdown transcript rather than the gigabytes of raw audio. If you want, there's a preference to disable the auto-deletion. There's a bit more on the support page here https://traceapp.info/support (search for "Auto-deletion of audio").
FluidAudio is a big part of this and is actually used in two places during a session. It runs the Parakeet EOU model for the instant recap (which isn't hugely accurate, but it's good enough for the job) and after the call it's also used to transcribe the recording, depending on which engine you've selected (Trace offers a fast and an accurate one). If the fast engine is selected, we use FluidAudio with the Parakeet-TDT 0.6b v3 model for transcription, which then goes through Pyannote and WeSpeaker for diarization. If the accurate engine is selected, we use WhisperKit with the Whisper large-v3-turbo model for transcription, and SpeakerKit for diarization.
- Journaling file structures (telegraph what you're about to write, then write it, then signal completion)
- memmap your important data structures to a file (they will be flushed to disk no matter how your app dies - short of a power loss)
- post-crash dump (put last-minute writers in a crash handler to save it to disk)
A journaling file structure is the most secure, because it's designed with the assumption that writing will eventually fail. memmapped structs are easy and cheap, and get you 99% of the way there (only power loss will lose your data). Crash-time writing is doable with a crash handler like KSCrash, but there are many ways an app can crash without triggering a crash handler (thermal kill, exceeding quota, memory jetsam, etc). You also need to write your data in a signal-safe manner.
- crash recovery: part one is use ADTS aac (even if process crashes, audio is saved up until it does). Part two is isolating the transcription/summaries in separate XPC services.
- disk space: AAC 64kbps mono soles it. Could use Opus for further reduction but both are small.
- speaker bleed: macOS voice isolation processing solves this. It’s a nightmare to get setup, but works great once done.
- library: using argmax SDK - by a bunch of ex-Apple on device AI folks.
It it wasn’t for CoreAudio, I’d say it was easy to make. Argmax, Whisper, and llama.cpp - wrapped in the right architecture, mostly just work.
I’m having fun nerding out on the details like custom vocabulary (get the names of the people in here meeting right), inferring speaker names from transcript, calendar integration, nice UI, etc.
Wait really? I honestly would have thought this was a solved problem by now, especially high quality transcriptions bit, just out of curiosity, is the problem that the quality isn't high enough?
Due to audio quality, transcription sometimes produces garbled output or understands something wrong. FluidVoice offers the option to use a LLM to „interpret“ the text to rescue garbled audio through context. Do you also plan to support something like this? This would be a great feature!
> Which languages does Trace support? English only, for now. Both transcription models, Fast and Accurate, are built for English audio. A recording in another language will still produce a transcript, but it won’t be accurate: the model maps whatever it hears onto English words, so the result comes out garbled rather than failing outright.
> If transcribing other languages matters to you, get in touch (see Contact below).
I just purchased it. What's the best way to give you feedback? (Do you want any?)
From the top of my head: - will the mic switch automatically when I am at my office? Or do I have to change settings every time? Maybe a preference of what's available + auto switch would be good. - I personally don't need the hot key. Menu bar icon would be fine. - Download the model is a long process. Put it into the installer, not into the bar on the bottom - Speaker correction would be amazing. If it could "Learn" the speakers based on voice. - Overall neat app. Good animations and UX
**Speaker 1** [00:00] What if I fell to the floor?
**Microphone** [00:02] Yes, this is Phil, I'm just speaking, this should be my voice, and there's music in the
**Speaker 1** [00:05] Couldn't tell this anymoreFor the switching, do you mean if you hot-swap during a call? The mic should auto-switch if you've got System default selected, but feel free to give it a go and report back. If it doesn't do what we expect I can absolutely take a look at changing the behaviour.
Learning speakers is also on the to-do list.
P.S. Great choice in test audio. What a banger.
I would be more willing to purchase if it was open source and I could build from source to try it first.
Not the subsidized subs
What's your diarization pipeline? Pyannote?
I'd taken a different approach that used a LLM clean-up pass to summarize and progressively compress the transcript for ultra-long content, but I like the idea of targeted "pay attention here" flags.
hello@traceapp.info
In my experience, medium is often the sweet spot for English accuracy vs speed, especially if following-up with a post-processing pass. The large options are all fine, but can severely slow it down. There are some speed checks on my website if you're curious (link not posted because I don't want to hijack another post's app).
Add "open source" if you wish as well.