I'm coming at it from more of the data infrastructure side (e.g. send all of your logs and metrics to a cheap Iceberg catalog in the cloud so you have a central place to query[1]) but also check out https://github.com/tobilg/ai-observer -- duckdb popping up everything to make this interesting and easy.