I recently encountered a blog post with the most fantastic title: Despite decades of effort, scientists have still not found a substitute for reading the paper. The post was written by Jeremy Yoder, an Associate Professor in the Department of Biology at California State University Northridge, on his blog, Denim and Tweed. Yoder comments on the difficulty of keeping up with the sheer volume of academic papers, and of the attempts by publishers and journals (and tech firms) to help us off-load this chore by utilizing LLMs which can summarize the content in various ways. As you can perhaps tell from the title of his post, Yoder is not fully on board with this, and argues that in fact, academic articles are inherently structured in a way which allows us to capture the key facts and insights without the use of these tools – title, abstract, introduction, methods, results, discussion, etc. In addition, the various tools being used have many flaws; as Yoder so eloquently puts it: “LLM-generated summaries flow in the grooves worn by the most common phrasings in their training corpus.”
The article is great; if you are interested in this kind of thing, I suggest you read it. (It is short enough that you won’t even be tempted to ask a chatbot to summarise it for you). He concludes:
“The problem these various summaries and addenda are trying to solve is the problem of communication between human minds. Scientific articles—however stilted and structured—are written by human beings to convey information to other human beings. That communication is inherently imperfect, because both the writers and the readers are imperfect, limited, beings. No technological intervention can change this, and no tool or editorial presentation absolves a scientist from the responsibility to use her own scholarly judgement about what is interesting and relevant and important in a paper she’s read.
At the end of the day, you still have to read the paper, and figure out what it means for yourself.”
I have been thinking about this post a lot, especially in the context of the many conversations taking place at the university around the use of AI: by students, by staff, in research, in teaching, in marking, by employers, in disability contexts, etc. But yesterday, I found myself thinking about it in an entirely new context: that of writing this blog. You see, I received an email which went like this:
Hi Kelly (Knitigating Circumstances),
I’m [redacted], the developer behind [redacted] — I’ve been reading your blog. I built a tool that learns your writing style, turns your photos into full articles, and publishes straight to WordPress/Ghost/Blogger. I’d love for you to try it.
It then goes on to offer me a free month of the service, after which I can pay for it monthly. Let me summarise:
- You wish to steal my content, and then sell it back to me.
- You wish to sell me a tool which is already freely available (such as ChatGPT).
The thing that most annoys me about this (and there are many things that annoy me about this) is the assumption that I would want to do this. I write this blog because I like to write (I’m certainly not making money off it). I am proud of the work I put into it, and that it speaks with my voice. If I decide that I no longer wish to write it, I will stop. I certainly don’t want a machine writing it for me. And when I read a blog, I read it because I enjoy the voice of the writer – a living, speaking, creative individual with opinions to challenge and inspire me.
I also expect that when I read a blog written by an expert in their field, that I can trust the content. It is astounding how many blogs and podcasts are automatically generated and not subject to any editorial oversight at all. One of the best things I have read about this comes from the blog of knitting designer and publisher, Kate Davies, in her absolutely brilliant post Knitting Bullshit, published last April. Kate read and listened to scores of AI-generated knitting content and came away appalled. She writes about this AI slop: “I don’t think we need any further examples of this content to understand just how badly and how baldly it has addressed itself to the extraordinary creative practice and the vibrant global community of which I am proud to be a part, hollowed it out, and transformed it into Bullshit of the purest, most unalloyed kind.”
The author John Scalzi writes frequently about creativity and AI-generated content on his blog Whatever. One brilliant example is his post Generative “AI”: The Guitar Hero of Creativity. This post is not a hit job on Guitar Hero (which Scalzi loves), but rather uses it as a very clever metaphor. He writes:
“Just like being good at Guitar Hero does not make one a guitar player, being able to prompt doesn’t easily make one a creator. At best, it tells what is essentially a statistical engine what you want it to output based on the vast database of other people’s work that it is drawing from. Nothing it cobbles together gives you the experience of creation; the only benefit of that crank of the “AI” flywheel is to the company that owns the “AI.” The image or music or video or text is not from you, it’s just something you asked for. You’re not the creator. You are, perhaps, the creator’s manager.”
Kate Davies and John Scalzi and thousands of other creative people write wonderful content. I would prefer to read their voices over Grok or Gemini or Claude. Just as there is no substitute for reading the paper, there is also no substitute for writing it yourself. For that matter, there is no substitute for a walk in the woods, or curling up with a knitting project. I intend to do both of those things next (not necessarily at the same time).
