the patter matcher is interesting to also collapse log lines and compare that between runs, thank you!
replyIn my tool I was going more of a premise that it's frequently difficult to even say what you're looking for so I wanted to have some step after reading logs to say what should be actually analyzed further which naturally requires to have some model
very interesting, curious if there is any downside to running this at scale (compute?)
replyI'd assume it probably depends how large and varied your logs are?
replyBut, my guess, I could see an algorithm like that being very fast. It's basically just doing a form of compression, so I'm thinking ballpark, like similar amount to just zipping the log
Can't be anything CLOSE to the compute cost of running any part of the file through an LLM haha