> There is a long-lived and incredible community of classic gaming emulation work online for it to reference. And that community has created…conformance tests! Opus found them and built against one, that has >3k golden instructions
replyI'm doing something similar but not for an emulator, using tests other projects in a similar space use to validate things.
I also know a friend that feeds Claude Gherkin formatted test criteria, and apparently that yields insanely good Claude output.
I said last year to people that I swear BDD is going to make a comeback. Hasn’t yet, but I still reckon it might!
reply Claude Gherkin
They are talking about Cucumber, a behavior driven development platform. Gherkin is a syntax for writing test cases:
replyhttps://cucumber.io/docs/gherkin/
Very neat, seems quite appropriate for the way people currently work with LLMs.
I'd just ask Claude about writing test specs in Gherkin. I'm going to try this Monday.
replyTruly amazing. Quite thought provoking how it was helped by having high quality oracles. Eg. Qwenputer & good NES reference materials.
replyI guess humans need their oracles too: teachers, encyclopaedias, degrees, phds.
Bravo, awesome project, really creative!
[flagged]
reply