This is not a SFW instance. NSFW communities will be tagged, and efforts will be made to keep SFW communities clean, but proceed at your own comfort level.
Creative writing is orders of magnitude easier to detect. The biggest problem for AI writing detection tools is the use of translation — so generating text, translating it into another language, and translating it back — but this ends up as a false-negative issue. False positives are very rare.
More importantly, we don’t need GPTZero to detect AI writing because professionals can identify it on sight. Add context clues (students are dumb) and suddenly it’s trivial.
EDIT: I need you to understand that college students are terrible writers. In all my decades and literally thousands of students, I’ve met maybe three undergrads whose writing was more than proficient, and that’s in an environment where the average SAT scores are over 1550. The idea I’m going to mistake your writing for AI (which sounds horrible and robotic, btw) because of its “high quality” is simply unfathomable.
Thank you for linking me to that study! Unfortunately google is a piece of shit and they’re calling me a bot and refusing to talk to me even though I’m on a standard residential internet connection. Could you tell me the title/authors so I can look for it elsewhere?
I’m particularly interested in seeing what the actual false positive rates are beyond a colloquial “1%”. Because 1% is, in the context of accusing academic cheating, absolutely unacceptable dog shit.
Take a basic example intro class with 350 students, with only two tests per semester. With “only” a 1% false positive rates you will be destroying the lives of more than a dozen innocent kids every school year. And thats only for one class, students are taking many classes per semester.
Well, an important detail is that, as your professor I would already suspect you cheated. The only reason to use such a tool is to make it harder to deny.
Imagine an Ivy League classroom of 100 students, as you said. First of all, they all suck at writing in their own quirky ways. By contrast, AI has no rhythm, clumsiness, or voice. It writes like a corporation. If you’re being accused of using AI, that’s not a compliment.
Anyways, here’s another thing about these studies. Counterintuitively, a 1% false positive for a tool like GPTZero doesn’t mean that if I feed it 100 human essays from my creative writing class it’ll get one of them wrong.
It’s more like if there are 100 “types” of creative writing, 1 of them would be wrong consistently and the other 99 almost never. For these tools, there are types of writing they can’t ever figure out, like blind spots (probably from lack of data). A college essay is not going to be such a blind spot.
Here’s a known blind spot: if you have two humans collaborating on a paper, GPTZero will fail. So one person writes, the other extensively re-writes — like a translation. You end up with an inauthentic mess that throws up false positives.
If you’re looking for studies, go to Google scholar and type “GPTZero accuracy specificity” or something make sure you set it to 2025 or later, since GPTZero is rapidly improving. Most studies don’t target creative writing though, because like I said it basically doesn’t get those wrong and scientific articles are more difficult.
I’m an expert in my field. Do you know what specialization looks like? If you spent 70% of your waking life reading you’d be pretty fucking confident about reading as well. Literacy, how terrifying.
The false positive rate is pretty low, but a false negative of 35% is pretty concerning in terms of whether they’re fit for purpose. Like it’ll rarely misclassify human writing as AI generated, but it’s somewhat closer to a coin flip in correctly identifying AI generated language. It’s a signal, but I imagine it’s a poor substitute for the judgement of a person who has spent time teaching teenagers / young adults to write.
Which to the original point isn’t totally snake oil, but it’s maybe not a great tool for the professionals who it’s targeting.
The false negative rate for creative writing is exceedingly low as well, sub 1%. False negatives aren’t the critique of this meme, and the examples I gave for GPTZero were worst case scenarios for very specific types of technical writing.
You posted this image in response to a statement saying that AI detectors are snake oil:
They may have hated Jesus because he told them the truth, it’s possible.
OR – and you can just hear me out here – they may have hated Jesus because he talked shit and then added irrelevant religious propaganda to falsely insert an authority figure that would preclude and/or divert righteous disagreement with what he was claiming.
I wouldn’t know, I wasn’t there. Neither were you, when it comes down to it.
I don’t know whether you posted that off-topic religious mess in any kind of sincerity, but when anyone posts anything that looks like it came out of a Chick tract I instantly and forever assume that they are Mike Lindell the My Pillow guy.
Just so you know. This type of religious content may not be working for you in the manner in which you intended.
I appreciate the explanation but it didn’t read that way, and to me it still doesn’t. All the examples in the link you gave use that image as a template and not the meme itself. In the meme, in the white space just to the left of Mary with a beard hollering Shut up! someone writes in a second speech bubble. This not only takes it into meme territory, but well out of Chick tract territory.
OP didn’t bother. Also, OP has now edited out the rest of his original comment, which featured a second image of a table of some kind. This is what it looked like when I responded to it:
So I’m just gonna stay with my original Mike Lindell the My Pillow guy estimation, if only because I can still say it hasn’t failed me yet. But thanks for the effort: I will endeavor to retain the education you have labored to provide. 👍
It’s really just that AI detectors are snake oil
They’re not. GPTZero has a false positive rate under 1% for creative writing.
That’s a bold claim with no proof.
Worst performance for GPTZero is technical medical papers with a 10% false positive rate. Here’s the study from S. Korea.
Scientific scholarly writing: <4% false positive.
Creative writing is orders of magnitude easier to detect. The biggest problem for AI writing detection tools is the use of translation — so generating text, translating it into another language, and translating it back — but this ends up as a false-negative issue. False positives are very rare.
More importantly, we don’t need GPTZero to detect AI writing because professionals can identify it on sight. Add context clues (students are dumb) and suddenly it’s trivial.
EDIT: I need you to understand that college students are terrible writers. In all my decades and literally thousands of students, I’ve met maybe three undergrads whose writing was more than proficient, and that’s in an environment where the average SAT scores are over 1550. The idea I’m going to mistake your writing for AI (which sounds horrible and robotic, btw) because of its “high quality” is simply unfathomable.
Thank you for linking me to that study! Unfortunately google is a piece of shit and they’re calling me a bot and refusing to talk to me even though I’m on a standard residential internet connection. Could you tell me the title/authors so I can look for it elsewhere?
I’m particularly interested in seeing what the actual false positive rates are beyond a colloquial “1%”. Because 1% is, in the context of accusing academic cheating, absolutely unacceptable dog shit.
Take a basic example intro class with 350 students, with only two tests per semester. With “only” a 1% false positive rates you will be destroying the lives of more than a dozen innocent kids every school year. And thats only for one class, students are taking many classes per semester.
Well, an important detail is that, as your professor I would already suspect you cheated. The only reason to use such a tool is to make it harder to deny.
Imagine an Ivy League classroom of 100 students, as you said. First of all, they all suck at writing in their own quirky ways. By contrast, AI has no rhythm, clumsiness, or voice. It writes like a corporation. If you’re being accused of using AI, that’s not a compliment.
Anyways, here’s another thing about these studies. Counterintuitively, a 1% false positive for a tool like GPTZero doesn’t mean that if I feed it 100 human essays from my creative writing class it’ll get one of them wrong.
It’s more like if there are 100 “types” of creative writing, 1 of them would be wrong consistently and the other 99 almost never. For these tools, there are types of writing they can’t ever figure out, like blind spots (probably from lack of data). A college essay is not going to be such a blind spot.
Here’s a known blind spot: if you have two humans collaborating on a paper, GPTZero will fail. So one person writes, the other extensively re-writes — like a translation. You end up with an inauthentic mess that throws up false positives.
If you’re looking for studies, go to Google scholar and type “GPTZero accuracy specificity” or something make sure you set it to 2025 or later, since GPTZero is rapidly improving. Most studies don’t target creative writing though, because like I said it basically doesn’t get those wrong and scientific articles are more difficult.
I’m glad I never had you as a professor, your overconfident arrogance is terrifying.
I’m an expert in my field. Do you know what specialization looks like? If you spent 70% of your waking life reading you’d be pretty fucking confident about reading as well. Literacy, how terrifying.
All the experts I’ve known didn’t need to tell me. It was just obvious.
Props for responding constructively.
The false positive rate is pretty low, but a false negative of 35% is pretty concerning in terms of whether they’re fit for purpose. Like it’ll rarely misclassify human writing as AI generated, but it’s somewhat closer to a coin flip in correctly identifying AI generated language. It’s a signal, but I imagine it’s a poor substitute for the judgement of a person who has spent time teaching teenagers / young adults to write.
Which to the original point isn’t totally snake oil, but it’s maybe not a great tool for the professionals who it’s targeting.
The false negative rate for creative writing is exceedingly low as well, sub 1%. False negatives aren’t the critique of this meme, and the examples I gave for GPTZero were worst case scenarios for very specific types of technical writing.
You posted this image in response to a statement saying that AI detectors are snake oil:
They may have hated Jesus because he told them the truth, it’s possible.
OR – and you can just hear me out here – they may have hated Jesus because he talked shit and then added irrelevant religious propaganda to falsely insert an authority figure that would preclude and/or divert righteous disagreement with what he was claiming.
I wouldn’t know, I wasn’t there. Neither were you, when it comes down to it.
I don’t know whether you posted that off-topic religious mess in any kind of sincerity, but when anyone posts anything that looks like it came out of a Chick tract I instantly and forever assume that they are Mike Lindell the My Pillow guy.
Just so you know. This type of religious content may not be working for you in the manner in which you intended.
It’s literally from a Chick Tract. It’s an ancient meme… circulating for the past 10 years or so.
KYM
Whether the person posting it is right or wrong, it’s not really an endorsement of religion. It’s like “scapegoat”, or “omg”.
I appreciate the explanation but it didn’t read that way, and to me it still doesn’t. All the examples in the link you gave use that image as a template and not the meme itself. In the meme, in the white space just to the left of Mary with a beard hollering Shut up! someone writes in a second speech bubble. This not only takes it into meme territory, but well out of Chick tract territory.
OP didn’t bother. Also, OP has now edited out the rest of his original comment, which featured a second image of a table of some kind. This is what it looked like when I responded to it:
So I’m just gonna stay with my original Mike Lindell the My Pillow guy estimation, if only because I can still say it hasn’t failed me yet. But thanks for the effort: I will endeavor to retain the education you have labored to provide. 👍
someone disagrees with you on the internet? just lie! they arent going to check anyway and everyone believes lies on the internet.