I have an automation that will grab an image from that camera and send it to the Anthropic API with a prompt describing where the bins are, and the color of each bin. The prompt comes back with JSON indicating which bins it can see. If it can see e.g. the yard waste bin, and it's time to take the yard waste bin to the curb, it will send me a notification.
It also reminds me when it's time to return them to their "resting" position after pickup- unless I already have.
Home Assistant orchestrates this all. I also have it hooked up to a "Bindicator"- a white 3D printed bin with a color-changing LED inside, controlled by an ESP8266 using ESPHome. When it's time to move the bin it lights up the color of the bin that needs to be moved. If I haven't moved the bins back after a day it turns orange to remind me.
Claude configured the automations for me via MCP but it's not particularly complicated. It's pretty reliable, though sometimes just because of where the doorbell camera is it can't clearly see the bins. If it can't make a ruling on the bin location it just falls back to notifying me so I will sometimes get a reminder I don't need.
- how often do you have to change your cameras' batteries, or are you accepting ugly wiring for powering from mains?
- how many cameras do you have installed to cover your home?
- IP cameras are flaky, too
- constantly sending images of one's private space to a cloud LLM is Orwellian
- are you polling your LLM minutewise "Did the cat vomit on the floor"?
Meh (a) I want security and cat-monitoring cameras anyway (b) I'm willing to take the effort to install cameras neatly since they're capable of serving as 100 different sensors
> constantly sending images of one's private space to a cloud LLM is Orwellian
I agree. I have a local server with a GPU. The security side of things records to a local NVR. Nothing goes to the cloud.
> IP cameras are flaky, too
haven't had nearly as many problems here
> are you polling your LLM minutewise "Did the cat vomit on the floor"?
Kind of, but it only re-polls if the camera detected motion, and then every 15 minutes as a catch-all.
I’ve not played with vision models much. I did some experiments with OpenCV and Processing many years ago, but times have changed.
I think you just wrote me next weekend project ;)
why are so few hacking their cognitive abilities (basic short-term memory)?
I don't have a problem with short or long term memory, in certain contexts. I can hold entire problem definitions and codebases in my head while working, but I often set down my keys on the way to pick up my water bottle and get to my car only to have no idea where they are. I can remember details of specifications, language features, video game mechanics, etc. for years, but often can't remember someone's name.
There's a whole cottage industry of social media telling me this is ADHD but I don't know if that's real or just for clicks, and if I can improve these issues.
Many, having unfortunately been all the way at the worst-version of that – and figured out the way back.
No guarantees I can help, obviously, but this rhymes with the types of intermittency I dealt with (if you start going into rooms and forgetting what you went in for, that really sucks and is not “normal” - despite being common.)
Give a way to reach you? If you’re motivated, you can probably optimize.