Should you trust AI? If you don’t know of an instance where it has told you something that is flagrantly untrue, you either don’t ask it much or you really, really need to catch yourself on (as they might say in Ulster).
This last two years has seen an explosive growth in content online – fuelled by AI. It’s an extraordinary tool. It’s fantastic. Research is quicker, documentation is quicker, content creation of all types is much, much quicker.
… and for the moment, at least, AI is largely free.
This is like crack cocaine, everyone everywhere is pumping out vast volumes of Google-able, shareable words (and images … and video). This gear is good: plausible, grammatically compliant (if you can bear an Oxford comma) and delivered with a convincing confidence – the urge to trust AI, the instinct to trust AI is strong.
And it is also, frequently, talking bollocks.
Table of Contents
Consider this …
My spouse and I made these searches this summer.
Same search engine, across the same table, same sixty seconds, identical phones … almost identical queries. Google’s AI overview provided directly contradictory information to each of us.
This is Google. This is not a source from which you can reliably glean actionable facts.
For your information. There are black squirrels on the Isle of Arran (and Kintyre).
I know, I have been there, I have seen them.
Cute wee things.
Given the time advantages of AI, the pressure on you to use it will be strong – all your competitors will be reaping the benefits. If you aren’t yet, sooner or later it’s probably inevitable that you will, at some point, need to knock out a quick ‘how to’, a snippet of metadata or a short code excerpt.
So, how to avoid broadcasting said bollocks to the internet with your name on it?
How to stop AI making an arse of you?
Here are five, far from exhaustive, tips to help you preserve your dignity.
1. Read the Damn Thing!
It shouldn’t need saying, it really shouldn’t. But there’s at least one site we host where the website is very much a side-hustle. The owner is allowed to contribute to it during his lunchtime, so he does. I’ve witnessed him adding two posts in the course of one phone call – I advised him not to, but it’s his site.
As yet, AI doesn’t click “Buy Now”, LLMs don’t make inquiries … even when they do, they won’t be the conversions you want. You may be generating content to gain visibility online – but ultimately, you need real human beings to respond. You need people to react to your online presence in the manner you wish.
If you want sales calls, or inquiries from your site, those people need to read your content, at least some of it.
If you can’t be bothered, why should they?
Quite apart from that, if you don’t read it, you won’t know whether or not your website, your content, is telling you what you want your customers want to hear? … or whether you are publishing laughably untrue?
You don’t want your customers to trust AI; you want them to trust you.
2. Check Your Sources
Kim Kardashian didn’t, and now she’s blaming ChatGPT for making her fail her law degree. It may sound unbelievably pious to recommend she take some responsibility, but I guess I do – if I must hear about her at all, I do.
Knowing what is true is not entirely straightforward. Edgar Allen Poe is credited with the expression:
“Believe half of what you see and nothing of what you hear.”
Ironically, checking the sources reveals that he probably does not deserve the credit.
But the principle holds.
ChatGPT 3.5 is famously poor for the accuracy of its citations. Accuracy matters in healthcare: according to the US National Library of Medicine:
“Among these references, 47% were fabricated, 46% were authentic but inaccurate, and only 7% were authentic and accurate.”
According to the same website:
“ChatGPT-4 had an average reference accuracy score of 66.7%. Of the 30 references, only 43.3% were accurate and deemed “true” while 56.7% were categorized as “false” (43.3% inaccurate and 13.3% nonexistent).”
The figures for 3.5 are widely available; those for ChatGPT 4, we could only find the source above.
References are a key means of demonstrating authority – for ChatGPT, at least, the references appear to be created to support the LLM’s proposition, rather than the LLM compiling external sources to formulate an argument.
I, for one, would sooner trust an academic journal than trust AI.
3. Do Your Own Research
So, if you can’t trust AI-generated citations, can you trust the argument itself?
Broadly yes, but who wants to be believed just 66% of the time? That is not trust.
As you can see above from black squirrel-gate above, LLMs generate content gleaned from what it considers authoritative online content, but it has no way of knowing what is actually true.
LLM-generated content can illuminate potential ideas and pathways into arguments. But you really do need to take responsibility for what you assert online. You need to check the validity of both the assertion itself and the facts that inform it.
That means reading, online and off – try it, it will build your expertise. You might even enjoy it.
If you really have first-hand experience, if you’ve actually seen a furry-tailed, melanistic blighter skipping over a Caledonian farm track, then you have authority. Proclaim it with confidence.
4. Have Something to Say
Again, this shouldn’t need saying. But as people rush to stuff their websites with indexable content, the temptation to bloat your site with pages addressing the most frequently sought search terms is a heady one – especially if it can be done with ease.
But again, human time is precious, and ultimately it is humans who will make the decisions that will cause your business to thrive or fail. Before you post, you should ask yourself: ‘If I didn’t know it already, would I want to read this?’ If the answer is ‘No’, then ask if your content will generate trust or authority.
Ask yourself if you respect your prospective customers.
Attempting to be ubiquitous online, without a genuine belief that what you have to say is worth hearing, is just gainsay, just attempting to shout the loudest. If you want to build trust, if you want to display authority, it’s not a good look.
5. Put a Shift In
If all this looks like a lot more work than stomping over to Gemini and typing in “Hey Gemini, please give me 1,250 words on why Little Fire Digital rocks (UK spelling, no Oxford commas)”, that’s because it is.
AI can help you organise thoughts, polish grammar and reveal insights, but as ChatGPT itself pointed out (complete with M-dash):
“AI is a phenomenal accelerator of creativity — but it’s not a substitute for critical thinking.”
AI doesn’t know when it’s wrong. It doesn’t care if it makes an arse of you.
Just because AI lets you publish content without intent or effort, it doesn’t mean you should. Authority and accuracy come from research and experience. Research and experience drive insight and knowledge. Put a shift in.
Why Does It Matter?
Mistakes Are Expensive
Trust is hard-earned and easily lost. On October 15 1987, Michael Fish told viewers of the evening weather forecast:
“Earlier on today, a woman rang the BBC and said she’d heard that there was a hurricane on the way. Well if you’re watching, don’t worry – there isn’t.”
The next morning, over 15 million trees had been blown down across the south-east of England alone.
I remember it well (I‘m that old): Michael dismissing the prediction with a patrician chuckle and the walk to college the next morning, literally climbing over shattered tree stumps. He had a long career, Mr Fish, he was an expert meteorologist, but he’s only remembered for one thing.
Twenty-five years later, he pointed out:
“The computer of course was the thing that did the forecasts but you can’t blame that … because it was unfortunate that the computer lacked a huge amount of data from the area that the storm was developing.”
Clearly, computers making an arse of you pre-dates AI. But AI lets you do everything bigger, better and faster.
Mistakes Cost Money – A Lot of Money
Deloitte is the largest professional services network in the world by revenue and number of employees.
Deloitte recently was forced to refund the Australian government $90,000 AUD as it became apparent that a report it had compiled, largely with ChatGPT, was riddled with errors.
$90,000 is a lot of arse – Deloitte may well have slipped off quite a few nations’ speed-dial.
The Epistemic Crisis
Back in the olden days – you know, 2017 – we were gnashing our teeth in despair as prominent politicians claimed that acres of empty paving were indicative of the “greatest rally in American history” and of a plethora of other lies across the modern political landscape.
I mean, this is nothing new …
“… the deliberate falsehood and the outright lie used as legitimate means to achieve political ends, have been with us since the beginning of recorded history”
Hannah Ahrendt, Crises of the Republic 1974
But the extent of public untruth has reached the point where belief or denial of palpably visible facts is, for many, a matter of personal choice, driven by personal prejudice.
The very notion of a “post-truth” world no longer seems to horrify anybody.
The incumbent government accepts the widely held position that the climate is undergoing rapid, anthropogenic change; the current, most popular opposition party states that this is ‘unproven’. And, as for the leader of the free world …
“This ‘climate change,’ it’s the greatest con job ever perpetrated on the world”
Donald Trump
This matters. For entire nations to base policy on antithetical interpretations of the visible world is at best wasteful and, at worst, catastrophic.
And those glaciers have definitely gone somewhere.
I know, I have been there, I have seen them … or what remains of them.
Misinformation – you’re either part of the solution or part of the problem.
Trust AI at Your Peril
You need knowledge to act. You need good knowledge to act well. If you check the weather forecast before you choose what to wear, you want it to be accurate. If you want to get to where you need to go, you want your maps to be precise.
Remember the early days of SatNav, when the news (remember trusting the news?) would detail reports of drivers finding themselves on beaches, on railway lines, on clifftops as their widgets led them astray …
… and that was just the maps.
AI will hand you a great deal of rope, please don’t use it to hang yourself.
You owe it to yourself; you owe it to all of us.