upvote
Panagram is a scam.
reply
(This is the part where you provide extensive extraordinary evidence to your claim)
reply
Scam might be too strong a word but it certainly has far higher false positive rates than they are claiming, and their output is at best misleadingly presented: https://freddiedeboer.substack.com/p/i-wouldnt-say-pangram-i...
reply
This is on Pangram 3 which is very very old now and the founder responded below

https://freddiedeboer.substack.com/p/i-wouldnt-say-pangram-i...

What about on Pangram 4?

https://www.pangram.com/blog/pangram-4-technical

reply
No, they are the ones making claims, especially their CEO saying things like a 1/10000 false positive rate. Their own testing showed a 2% rate, which is insanely high when you talk about the number of papers students turn in. Worse their testing methodology compared it with pre-llm documents and not post llm documents that were human written (much harder and more expensive to verify), by treating language as static.
reply
You're saying because it has some false positives that Pangram is 100% a scam?

Is their research also a scam too?

https://pangram-public.s3.us-east-1.amazonaws.com/pdf/pangra...

https://www.pangram.com/blog/pangram-4-technical

If so, what is the best one out there other than Pangram then?

reply
Pangram is subjectively very useful and I personally subscribe, but the burden of proof is on them. The product is very much "trust me bro" and I fear that if they ever try to improve recall both their precision and reputation will tank.
reply
Then what is the best way to know that something is AI generated slop then?
reply
The question you're asking has nothing to do with who has the burden of proof when it comes to claims about Pangram, but I'll answer it anyway.

Today, the best way is probably Pangram. Tomorrow, it might not be, especially if they try to push their recall up.

You might have to make peace with the fact that there may not always be a tool that does what you want.

reply
So Pangram is the best one right now, that all I need to know, and I can safely assume that the Claude AI marks will make it even stronger.

Thanks!

> But the burden of proof is on them...

I mean is this enough proof?

https://www.pangram.com/blog/pangram-4-technical

https://pangram-public.s3.us-east-1.amazonaws.com/pdf/pangra...

Or is this marketing, a public stunt or not real research?

I think this is enough for me to know they are actually improving their AI slop detector.

reply