Is text-generating AI an industry killer or just another wave of hype?
If you have been reading all the hype about the latest artificial intelligence chatbot, ChatGPT, you might be excused for thinking that the end of the world is nigh.
The clever AI chat program has captured the imagination of the public for its ability to generate poems and essays instantaneously, its ability to mimic different writing styles, and its ability to pass some law and business school exams.
Teachers are worried students will use it to cheat in class (New York City public schools have already banned it). Writers are worried it will take their jobs (BuzzFeed and CNET have already started using AI to create content). The Atlantic declared that it could “destabilize white-collar work.” Venture capitalist Paul Kedrosky called it a “pocket nuclear bomb” and chastised its makers for launching it on an unprepared society.
Even the CEO of the company that makes ChatGPT, Sam Altman, has been telling the media that the worst-case scenario for AI could mean “lights out for all of us.”
But others say the hype is overblown. Meta’s chief AI scientist, Yann LeCun, told reporters ChatGPT was “nothing revolutionary.” University of Washington computational linguistics professor Emily Bender warns that “the idea of an all-knowing computer program comes from science fiction and should stay there.”
So, how worried should we be? For an informed perspective, I turned to Princeton computer science professor Arvind Narayanan, who is currently co-writing a book on “AI snake oil.” In 2019, Narayanan gave a talk at MIT called “How to recognize AI snake oil” that laid out a taxonomy of AI from legitimate to dubious. To his surprise, his obscure academic talk went viral, and his slide deck was downloaded tens of thousands of times; his accompanying tweets were viewed more than two million times.
Narayanan then teamed up with one of his students, Sayash Kapoor, to expand the AI taxonomy into a book. Last year, the pair released a list of 18 common pitfalls committed by journalists covering AI. (Near the top of the list: illustrating AI articles with cute robot pictures. The reason: anthropomorphizing AI incorrectly implies that it has the potential to act as an agent in the real world.)
Narayanan is also a co-author of a textbook on fairness and machine learning and led the Princeton Web Transparency and Accountability Project to uncover how companies collect and use personal information. He is a recipient of the White House’s Presidential Early Career Award for Scientists and Engineers.
Our conversation, edited for brevity and clarity, is below.
This actually matches what the philosopher Harry Frankfurt has called bullshit, which is speech that is intended to persuade without regard for the truth. A human bullshitter doesn’t care if what they’re saying is true or not; they have certain ends in mind. As long as they persuade, those ends are met. Effectively, that is what ChatGPT is doing. It is trying to be persuasive, and it has no way to know for sure whether the statements it makes are true or not.
What’s been different in the last five years is that, because of a new technology called transformers and other related technologies, computers have gotten good at reversing the perception task of identifying a cat or dog. This means that, given text prompts, they can actually generate a plausible image of a cat or a dog or even fanciful things like an astronaut riding a horse. The same thing is happening with text: Not only are models taking a piece of text and classifying it, but given a prompt, these models can essentially run classification in reverse and produce plausible text that might fit into the category given.
However things are improving. It is possible, at least to some degree, to take the work of human content moderators and train models to make those judgments about whether an image represents nudity or hate speech. There will always be inherent limitations, but content moderation is a dreadful job. It’s a job that’s filled with the trauma of looking at images of gore and beheadings and all kinds of horrible things day in and day out. If AI can minimize the human labor, that’s a good thing.
I think there are certain aspects of the content moderation process that should not be automated. Deciding the line between acceptable and unacceptable speech is time-consuming. It’s messy. It needs to involve input from civil society. It’s constantly shifting and culture-specific. And it needs to be done for every possible type of speech. Because of all that, AI has no role here.
What all of these have in common is statistical predictions based on rough patterns and correlations in the data about what a person might do. These predictions are then used to some degree to make decisions about them, and in many cases, deny them certain opportunities, limit their autonomy, and take away the opportunity for them to prove themselves and show they’re not defined by statistical patterns. There are many fundamental reasons why we might want to consider most of these AI applications to be illegitimate and morally impermissible.
When an intervention is made based on a prediction, we need to ask, “Is that the best decision we can make? Or is the best decision one that doesn’t correspond to a prediction at all?” For instance, in the criminal-risk prediction scenario, the decision that we make based on predictions is to deny bail or parole, but if we move out of the predictive setting, we might ask, “What is the best way to rehabilitate this person into society and decrease the chance that they will commit another crime?” It opens up the possibility of a much wider set of interventions.
Even with something as profound as the internet or search engines or smartphones, it’s turned out to be an adaptation, where we maximize the benefits and try to minimize the risks, rather than some kind of revolution. I don’t think large language models are even on that scale. There can potentially be massive shifts, benefits, and risks in many industries, but I cannot see a scenario where this is a “sky is falling” kind of issue.
Julia Angwin is editor-at-large and founder of The Markup, which originally published this article. It is republished here under the Creative Commons Attribution-NonCommercial-NoDerivatives license.
Leave a Reply