Detecting ChatGPT and AI-Generated Text in School Papers

The race is on to develop tools and techniques that can detect AI-generated copy. Educators and institutions of higher ed are especially vulnerable to the throes of machine-generated content. Now, a group of researchers at Stanford may be one step closer to a useful solution.

The use of large language models (LLMs) is skyrocketing, and with good reason; it’s really good …

Recently, a team of researchers at Stanford proposed a new method called DetectGPT, which aims to be among the first tools to combat generated text in higher education. The method is based around the idea that text generated by LLMs typically hover around specific regions of the negative curvature regions of the model’s log probability function. Through this insight, the team developed a new barometer for judging if text is machine-generated which doesn’t rely on training an AI or collecting large datasets to compare the text against. We can only guess this means human written text occupies positive curvature regions, but the source is not clear on this.

This method, called “zero-shot”, allows DetectGPT to detect machine written text without any knowledge of the AI that was used to generate it. It operates in stark contrast to other methods which require training ‘classifiers’ and datasets of real and generated passages.

The team tested DetectGPT on a dataset of fake news articles (presumably anything that came out of CNET over the last year) and it outperformed other zero-shot methods for detecting machine-generated text. Specifically, they found that DetectGPT improved the detection of fake news articles generated by 20B parameter GPT-NeoX from 0.81 AUROC for the strongest zero-shot baseline to 0.95 AUROC for DetectGPT. Honestly, this is all French to me, but it purports a substantial improvement in detection performance and suggests that DetectGPT may be a promising way to scrutinize machine-generated text moving forward.

Beware the ‘Storification’ of the Internet

The Atlantic:

There is a growing trend in American culture of what the literary theorist Peter Brooks calls “storification.” Since the turn of the millennium, he argues in his new book, Seduced by Story: The Use and Abuse of Narrative, we’ve relied too heavily on storytelling conventions to understand the world around us, which has resulted in a “narrative takeover of reality” that affects nearly every form of communication—including the way doctors interact with patients, how financial reports are written, and the branding that corporations use to present themselves to consumers. Meanwhile, other modes of expression, interpretation, and comprehension, such as analysis and argument, have fallen to the wayside.

The danger of this arises when the public fails to understand that many of these stories are constructed through deliberate choices and omissions. Enron, for instance, duped people because it was “built uniquely on stories—fictions, in fact … that generated stories of impending great wealth,” Brooks writes. Other recent scams, like those pulled off by Purdue Pharma, NXIVM, and Anna Delvey, succeeded because people fell for tales the perpetrators spun. In other words, we could all benefit from a lesson in close reading and a dose of skepticism.

Report: Texans Suffering from Inequitable Broadband Access

Texas’ first-ever comprehensive, community-driven broadband internet analysis, which was released today, paints a dismal picture for those marginalized citizens who have to go without healthcare, economic opportunity, education and online communication due to broken infrastructure. Their testimony and stories are moving.

Hats off to my Texas Comptroller of Public Accounts colleagues for their work on this powerful report. I was excited to work on this, and being able to contribute was one of the most rewarding times of my career. Check it out, and press your lawmakers and communities to do more. 

The End of the iPod

After more than 20 years, one of the most iconic devices of all time is no more:

Over the years, Apple introduced multiple iterations of the iPod, including iPod mini, multiple versions of the iPod nano, the iPod shuffle, and the iPod touch ... Apple said that it is discontinuing the iPod because the iPod's capabilities have been built into the entire Apple product lineup, from the Mac to the iPhone to the Apple Watch.

My favorite iPod was the fourth generation model, released in 2004.

Apple’s Newsroom posts a farewell:

Music has always been part of our core at Apple, and bringing it to hundreds of millions of users in the way iPod did impacted more than just the music industry — it also redefined how music is discovered, listened to, and shared,” said Greg Joswiak, Apple’s senior vice president of Worldwide Marketing. “Today, the spirit of iPod lives on. We’ve integrated an incredible music experience across all of our products, from the iPhone to the Apple Watch to HomePod mini, and across Mac, iPad, and Apple TV. And Apple Music delivers industry-leading sound quality with support for spatial audio — there’s no better way to enjoy, discover, and experience music.

For those who missed it, it’s hard to describe how exciting this product line was for pre-iPhone era Apple fans. It truly changed the music buying and listening experience.