Discussion about this post

User's avatar
Rob's avatar

CASE ONE: In a class I teach on Transmedia Storytelling, I assign students to do a case study of a transmedia project. I have a list of possible projects to choose from. One of the students picking toward the end had the case she wanted chosen by someone else, so she asked me if there was another similar example of a nonprofit project created by a woman. Reasonable request, but I didn't have one off the top of my head, so I told her I'd get back to her.

Aha! A job for ChatGPT! I put in the parameters of what I was looking for and it gave me back a list of five - two of which were on my syllabus already, one that was inappropriate for unrelated reasons, and two promising candidates I had not been aware of. One looked especially interesting - it was a project on sex trafficking in conflict zones done by an NGO in Central African Republic in 2016. I was intrigued and tried to look it up.

There was no sign of it online, although the organization was real. I tried the Wayback Machine in case they'd taken it down from their site. Nope. I asked ChatGPT who the creators were and it gave me several names, including an artist from CAR I knew because I'd interviewed him recently. I checked his website and portfolio but saw no mention of this project, so I emailed him. He replied that he had no idea what I was talking about.

Turns out ChatGPT had made the whole thing up, but had done so in such a convincing way that almost anyone would have taken it as true. It had links, citations, used plausible information. On this technical subject, I may be one of a handful of people in the world who is an "expert" but it nearly fooled me, and would have if I had not put in more time verifying the work than it would have taken me, in the end, to do my own research the old fashioned way.

CASE TWO: I'm developing a client presentation on basic marketing strategy for a nonprofit I'm doing some consulting for. Since Microsoft has "upgraded" my Office 365 to include Copilot, I thought I'd try to get some value out of it. I ask PowerPoint to generate an infographic of the customer journey in four steps: attention, engagement, conversion and loyalty. This is a pretty routine visualization and I figure that if it were trained on image data like charts and business graphics, it wouldn't have any trouble creating an original image.

Nope. No amount of prompting could get it to generate anything that wasn't incoherent gibberish: not only fundamentally incorrect conceptually, but also ugly, full of weird and unintelligible text, and garishly designed.

After wasting 45 minutes trying to get the AI to amplify my productivity and execute the details of my strategic vision, I gave up and built the whole thing using clip art in about 10 minutes.

CASE THREE: I'm working on a report for a client analyzing some survey data. Unfortunately the person who conducted the survey doesn't know how or why to use spreadsheets, so the Excel files they gave me were nearly impossible to do any kind of analysis on. The data was spread across 4 different sheets, all formatted differently. After spending a little time trying to fix it manually, I dumped them all into ChatGPT to see if it could help me spot the patterns in the data that I needed to use in the report.

Things start out promising, as it did seem able to correlate the different sheets and track the kind of stuff I was trying to look for. But eventually, it starts giving me contradictory answers, citing information that was plainly wrong, giving me rankings of 26 states when I asked for 50, and other weirdness.

I start wondering if any of its answers are reliable or authoritative. I worry about using any of this as the basis for my analysis and discussion of the findings - that is, my own insights based on my experience and interdisciplinary understanding. If the data isn't correct, then nothing I add to it will be true or meaningful. Indeed I may be misleading people or drawing false implications.

All of these strike me as completely ordinary scenarios for knowledge workers seeking tools to enhance their existing expertise, and in all of them, using advanced AI tools for the purposes they are apparently intended for was useless or worse. Most actual AI technical folks I've talked to about this tell me these kinds of problems are fundamental to the way the models are built and can't be easily fixed. In fact, the latest OpenAI models are more prone to hallucination than previous ones.

I do see value in AI's ability to spot patterns in large data sets for things like pharmaceutical research or improving the performance of complex systems at scale (like optimizing delivery routes for packages or reducing waste in manufacturing processes). But in real world cases of high-end knowledge work, if these tools can't handle the simple, routine problems, I don't see how companies are going to recoup the trillions they are investing.

Expand full comment
3 more comments...

No posts