For an LLM, just being able to play an entire game of chess without illegal moves and without inventing pieces that aren't on the board is an achievement. Even more so for a live model. Then again, who knows how much the harness helped here
Seems more than good enough for a live model though! I can imagine this demo being extended to be a lot nicer to play with. You can just feed the model engine analysis and it can make as high of quality moves as needed. No longer any correlation between the model's understanding of the position and the moves that would be made but I think that's still a really nice improvement when thinking about this as adding live voice interaction to existing chess vs computer functionality rather than adding chess to possible interactions with the latest live voice model.