It’s no surprise that businesses—particularly those that focus on marketing, direct-to-consumer sales, and entertainment—have jumped feet-first onto the artificial intelligence (AI) bandwagon. History tells us that any significant technical frontier that opens briefly to the average Joe will spawn a brief, mad, Wild West-like dash to be the first to figure out how to profit. Soon, a small handful of winners will be declared, and things will tighten back down and return to business as usual.

What’s puzzling is that education and corporate training functions—groups whose responsibility is to the people they serve, not to profits—seem to have been so quick, so vocal, and so uncritical in wanting to follow suit.

What does AI potentially offer the E & T space?

The educational and training benefits often touted for AI tend to fall into one of 3 categories:

1. Vague and theoretical. Marketers, industry pundits, and the folks creating AI tools loudly assure us that AI can “transform” education; that it can help us “engage” and “empower” learners by “personalizing” and “enhancing” instructional materials. (Unfortunately, they don’t explain how to execute on any of these glittering generalities.)
2. More of what we already have plenty of, such as the ability to provide learners with immediate feedback (um…. auto graded quizzes, discussion boards, and instant messaging?) and software options that help us draft and create instructional materials. This latter option is what most of us in E & T who are experimenting with AI—primarily in the form of chatbots—are using it for: as one of many options for drafting instructional materials. But if we reflect on #3 below, we may want to put a pause on some of that.
3. Outright lies, lies by omission, and dangerous assumptions. Some pro-AI sources imply or even state outright that AI can somehow teach critical thinking and digital literacy skills. Others simply assume that AI needs to be incorporated, RIGHT NOW, into all schools everywhere. All these stances have the potential to erode learners’ fundamental ability to differentiate truth from fiction in all contexts, digital or otherwise.

It’s the third category that’s the focus of this post.

Chatbots are unvettable (and, therefore, untrustworthy) sources

With the possible exception of proprietary tutoring chatbots (such as Khanmigo and others made available by specific organizations for specific, stated purposes), chatbots are inappropriate for creating learner-facing materials, and the reason has nothing to do with technology or access.  Chatbots fail all of the tests we’ve historically used to vet sources, and it’s unequivocally wrong to provide unvetted materials to learners who are relying on us to provide high-quality instruction on a specific topic presented in a specific context.

Below is a breakdown of the five criteria traditionally used to vet a source.

Chatbots can’t satisfy even one of the 5 traditional criteria for vetting sources—making them even less trustworthy than traditional search engine results.

CRAAP: The 5 traditional criteria for vetting sources (and how AI stacks up)

Currency, relevance, authority, accuracy, and purpose are nothing more or less than the defining attributes of trustworthy sources. If a source meets all 5 of the following criteria, then—and only then—can we say that source is trustworthy.

These 5 criteria are formally addressed in nonfiction books, academic papers, and the reference section/footnotes that English teachers require in essay assignments. They’re not formally addressed (but, hopefully, are considered) in other research situations, such as informal primary research and googling.

Whether they’re formally addressed and articulated or not, the following 5 criteria determine how we differentiate between what’s true and what’s not true.

Currency is important because expert knowledge of some topics changes over time. For example, we know now that toads don’t spontaneously generate and that malaria isn’t caused by “bad air” (both of which were considered facts in previous centuries). Relevance is important because a fact appropriate for one audience (such as washing and bandaging a wound for an audience of parents and school nurses) may be inappropriate or inadequate for another (such as an audience of highly trained doctors). Authority is important because information from a trusted, experienced source (such as a professional mechanic) is more useful/valuable than information from an inexperienced or unknown source (such as a person who has never actually fixed a car). Accuracy is important because without it, there can be no facts. Purpose is important because it allows us to evaluate whether information is skewed to a hidden agenda.

1. Currency.  There’s no way to identify, without independent research, whether a chatbot response contains the most current information on a given topic. Granted, currency isn’t a big deal in some contexts (the works of Epictetus, for example). But it’s a very big deal in subjects such as medicine, technology, and anything to do with governmental regulations—all volatile subjects characterized by rapid, meaningful change that ties currency to up-to-the-minute accuracy.
2. Relevance. There’s no way to identify whether a chatbot response is actually relevant in the context we have in mind—even if we’ve carefully followed the chatbot’s instructions around wording and tried to phrase our question unambiguously. This is the case because a chatbot has only the requestor’s question to go on (and not an understanding of the requester’s current knowledge/background or reason for asking). And unlike AI precursors, chatbots’ ability to accept freeform prompts makes irrelevance nearly guaranteed. (If you’ve ever asked a chatbot the same question 15 different ways and gotten 15 different answers, you’ll understand how difficult it can be to communicate context to a chatbot.)
3. Authority. There’s no way to identify whether a chatbot response is authoritative, because there is no verifiable “author” involved. In fact, when we’re dealing with chatbots, we have no idea how or where the answer was sourced or even if it was sourced, because answers can be constructed wholesale by chatbot algorithms based on proprietary criteria. (Side note: Judging by the flood of AI “writer” job requisitions on the job boards these days, there’s a good chance that tomorrow’s “authorities” will be random tech writers with no subject expertise who were paid well below professional rates to come up with subject-specific questions and answers.) And unlike an authority on a given topic, if the response a chatbot generates is wrong, there’s no recourse.
4. Accuracy. There’s no easy way to identify how much of a chatbot response, if any, is accurate. Using traditional means (such as dictionaries and expert-authored books produced by reliable publishers) we could verify the accuracy of every aspect of every chatbot response we receive independently, outside the tool. But from a practical perspective, few of us will do so. After all, ostensibly we’re using a chatbot to save time. If we’d wanted to look things up independently, we would have done so instead of leaning on a chatbot!
5. Purpose. The algorithms chatbots use to construct, phrase, and display one answer over another are proprietary. Simply put, we have no idea why a given chatbot produces the replies it produces or whether it produces different replies for others (and, if so, what criteria is being used to differentiate replies). This is emphatically not the case with an authoritative human source, each of whom can be relied on to have a clear purpose or “angle.”  If we ask both the representative of a tobacco company and our family physician whether cigarettes are good for us, for example, we can be sure the answers will vary and why they will vary because we know the tobacco company representative’s purpose is to drive cigarette sales, and we also know the family physician’s purpose is to give us the unvarnished facts we need to keep our bodies healthy. Because we can’t guess the motives of the folks who create AI tools (or even identify the actual creators, in many cases), we have no idea what’s driving their algorithms. This means we can’t identify a chatbot’s purpose for providing the answers it provides.

Pro Tip: If you’re unfamiliar with the CRAAP approach to vetting sources described above, a) You’re likely not an English teacher, and b) You’re welcome! This approach allows us to separate fact from fiction (or, at least, fact from “no idea whether it’s true or not”) quickly and reliably in all situations.  Plus, the rudeness of the acronym makes it a favorite of the middle school set (the age at which CRAAP and other online literacy skills are often taught) and easy to remember for all of us.

The bottom line

When the dust settles around AI, some business owners will be better off, and some will be worse.

But the stakes area a lot higher for us educators and trainers. When the dust settles around AI, do we really want to be left with a generation that can’t vet sources not because they don’t know how, but because they don’t believe they have to?

What’s YOUR opinion?

Are you applying, or thinking about applying, AI in the classroom or the training room? What alternatives, if any, is AI replacing in your repertoire? Do you see any downside to using, or teaching students to use, sources that can’t be vetted?   Please consider leaving a comment and sharing your hard-won experience with the learning community.