There are, broadly, two kinds of audio recordings that linguists want to transcribe. One is native speakers telling traditional stories, where they're speaking naturally and taking the natural shortcuts (such as "wanna" and "gonna" in English). The other is native speakers reading words (or short example sentences) very carefully and distinctly, so that the linguist can listen to the recording over and over to learn how to pronounce the word right. In those recording, they'll say "want to" and "going to" rather than "wanna" and "gonna".

Thanks for the pointer; I'll check out that model and see if it handles the "slowly and carefully" type of recording better than the "natural speaking" type. (And depending on what kinds of errors the model makes, even the recordings where it makes errors can prove useful: for example, a linguist studying regional variations in speech would want the model to produce the IPA for "gonna" rather than "going to").

Dialects degenerate phonemes that would otherwise occupy identity relations between different utterances of the same "word" (word/concept mutable hyperobject as is the standard in any socially-relevant spoken language) which would be a bit of irony in this thought experiment since common knowledge dictates that more data samples must be present in the dataset (not less, as in rarely-spoken languages) to associate separate pronunciations of utterances representing the same underlying concept. However very-rarely-spoken languages probably don't have distinct dialects since so much focus is put on mutual intelligibility with the few members of the group that remain fluent in that language. It's not outside the realm of possibility that small speaking communities nonetheless fractionate into dialectical specialities but that seems increasingly unlikely as the fervor for preserving/recognizing dying languages increases, and global instant communication continues to become more commonplace.

Example: Schwabisch is wild and would be phonetically transcribed very differently from Hochdeutsch which is its ostensible language progenitor (technically more a cousin than an ancestor in the lineage of language evolution), but if the goal is merely to focus the model purely on phonetic transcription then you can add additional post-processing layers which map sounds to core concepts shared across dialects for actual translation. But I like your idea of interacting with the intermediate elements to familiarize yourself at least with the phonetic patterns, we humans are still thinkers enough to infer patterns of grammar and semantics from these building blocks just as we have done for the entire history of the species/lineage before written representations of language came along (relatively late -- evidence of script cropped up only once civilization had centralized to a sufficient degree to make economics non-local and non-trivial).

tl;dr the big words: it's not til you collect enough spoken samples of the dead(ish/dying) language being spoken that the local idiosyncracies are discovered, luckily linguists are smart enough to probably anticipate and certainly post-process language snippets to grasp the common structures for this or that given language.

FWIW someone wrote their PhD thesis on the Iu Mien language: https://opal.latrobe.edu.au/articles/thesis/An_Iu_Mien_gramm... It's not a textbook, but that doesn't mean you can't try to use it as a textbook. For example, there are some thoroughly-analyzed example texts. Also, the acknowledgments section mentions the Iu Mien Literacy Project, which has published a short course on the very basics, as it turns out: https://iumienliteracyprojects.com/resources/ I suspect the 925-page PDF contains some more hints to additional resources like that. :)

The linguists I know are not necessarily a representative sample... but they're mostly interested in just that: maximally faithful IPA representation. They want to know if the speaker switches back and forth between aspirated p and unaspirated p on the same word, because that tells them something about the language — that aspirated consonants are not meaningful.

Linguists studying the sounds of a language, its phonology, often want to find "minimal pairs", words that differ by only a single sound. For example, din and tin in English. You record a native speaker saying both words, and telling you their meaning, and then you play back either recording A or recording B to other native speakers and ask them which word it is. If they can identify the word every time, then you've found two sounds that are meaningfully distinct in this language. (Some languages don't distinguish the d and t sounds, but English does). But if the native speakers go 50/50 on which word it is, or ask to hear it in a sentence for clarification because it could be two or three different words, then you've found a pair of sounds that this language does not distinguish. (Note that you're playing the words in isolation, because sentence context might make it obvious which one it is, e.g. you can't tell if an English speaker is saying their or there until you hear more words of the sentence).

So yes, the linguists that I know (who, again, are not necessarily a representative sample) are interested in as faithful an IPA representation as they can get, because that inconsistent transcription will give them many clues about the language. It still all has to be checked, because that switching back and forth between aspirated and unaspirated p (for example) could have been an artifact of a poor-quality microphone not picking up the aspiration, or a windy day causing aspiration sounds that the speaker never said, or the speech-to-text model making a mistake. But I watched my wife listen to the same two-second recording on loop over and over, trying to be certain of which sound she was hearing in the middle of the word. Double-checking the output of the model would (in most cases) only require listening to the audio once or twice, not half-a-dozen times like she typically did while researching her thesis. At the time, LLMs were not really a thing yet, but if she were doing her thesis today I bet a speech-to-IPA model would have saved her quite a lot of time — but only if it output every distinction, even the ones not meaningful in the target language. The "maximally faithful" representation, as you put it.