No patient/customer data included in the screenshots - in production we’d basically find some “region” of the screen to screenshot that would help an LLM say yes/no to - ex: “is the nav bar green and does it say Insert Note”
replyOk, that seems doable if working perfect, but what is the tool output "i can see the patient list with search functionality and 30 patients" thing in the demo ? Is this not vision detected ? Or are you digging into windows api (making non standard windows components/widgets non working)?
replyah for the fake EMR demo I was a bit less strict with claude, it was vision detected likely - our MCP has tools for taking screenshots of the screen and inspecting the windows accessibility tree etc
reply