Love AI. Don't Trust AI. Here's how to verify.

AI Adoption

31 Mar

AI gets things wrong. We know that.

It makes things up. It hallucinates. It wants to please us, so it will answer a question even if it doesn't really know the answer.

You can tell it: don't make things up. Be accurate. Give me a confidence level. But that's still the AI telling you what it thinks. It doesn't actually check itself.

So here's what I want to explore with you: what kinds of problems come up, and what can we do to stay safe?

How do we make sure our team knows how to verify what AI gives them?

What do we do if someone shared the wrong thing with the wrong AI?

And how do we build a culture where people feel safe to speak up when something goes wrong?

Let me walk you through it.

It happened in a session I ran this week

I asked the room: has anyone had AI give them a wrong answer?

One accountant put her hand up. She'd asked Copilot to find a tax reference to back up advice she was giving a staff member. It came back with a citation. She used it. Then she clicked it. It went nowhere.

She stopped using Copilot after that.

I understand why. But that's not the right response.

Another person in the same room shared something different. They were uploading a spreadsheet and asking Copilot to summarise the data. They ran it four times. Each time, one or two numbers were wrong. Never the same numbers. They still don't know where those numbers came from.

Their conclusion: it's really important that you check all of the data.

Yes. Exactly.

The Olympics example

Someone in one of my Copilot workshops shared this, and allowed me to pass it on.

They asked AI: who won gold in the 200 metres at the 2000 Olympics?

It came back with a name. Confident. Well-formatted.

Then someone in the room asked: what about the women's?

Oh, yes, said the AI. Here's who won that.

Then someone asked: could this have been a different sport?

Oh yes, said the AI. It could have been swimming, athletics, or canoeing.

The first answer wasn't wrong. It wasn't a hallucination. It was incomplete. And the AI didn't flag that. It just answered the question it was asked, as if that was the whole picture.

That's what we have to watch for. Not just the wrong answer. The answer that's right as far as it goes, but doesn't tell you what it left out.

The Deloitte lesson

This is an Australian story, which makes it even more relevant.

Deloitte was paid $440,000 by the Australian government to produce a 237-page report on welfare compliance. It was a serious, professional piece of work. I'm sure a lot of people worked very hard on it. The analysis was sound. The recommendations were solid.

But scattered through the footnotes were references to academic papers that don't exist, and a quote attributed to a federal court judge that was completely fabricated.

A Sydney University researcher spotted it almost immediately. He recognised a colleague's name in a citation, looked it up, and the book didn't exist. He said he knew "instantaneously" it was a hallucination.

Deloitte had used AI to help with research and citations. Nobody checked. They had to issue a corrected version and refund part of the fee. An Australian senator said they had done "the kinds of things that a first-year university student would be in deep trouble for."

The lesson is not: don't use AI.

The lesson is: the main work was fine. It was the bits nobody checked that caused the damage.

So what does this mean?

Love AI means: use it. Use it a lot. Use it for everything that makes sense.

Don't trust AI means: before you act on it, before you send it to a client, before you make a decision based on it, check.

These two things are not in conflict. A good doctor loves their diagnostic tools and still reads the results carefully.

The two kinds of tasks

There are two kinds of tasks we bring to AI.

The first kind is tasks in areas where you are the expert. You can look at the response and quickly assess it. You know enough to say yes, that's right, or no, that's off. The AI did the work, the writing, the organising. You reviewed it. That's a good use of AI.

The second kind is tasks where you're not the expert, where you needed help. These are the ones to be careful about. Because if you don't know the answer, you can't always tell when it's wrong. You can't make legal, financial, or medical decisions based on an AI response you can't actually evaluate.

This doesn't mean don't use AI for things you don't know. It means: when you don't know, verify before you act.

Five ways to verify

Check the sources. When AI gives you references, click them. Not just to see if they exist. Read enough to confirm they say what you think they say. Australian. Recent. Actually relevant.

Ask it to check itself. Before you accept a response, ask: Can you check this for accuracy? Is there anything here you're not confident about? It won't catch everything, but it catches more than nothing.

Use reasoning mode. In Copilot it's called Think Deeper. In ChatGPT it's the o-series models. These think before they answer. Hallucinations drop significantly. Use reasoning mode for anything that matters.

Ask another AI. They're trained differently. They make different mistakes. If you're not sure about something one AI gave you, paste it into another and ask: What do you think of this? They are very good at improving on each other.

Ask a human expert. This is the most important one and the one we skip most often. If you're outside your area of expertise, show it to someone who actually knows. You've done the legwork. It takes them 30 seconds to validate.

What to do when something goes wrong

Mistakes happen. Someone accidentally uploads a document to the wrong AI. Someone realises too late they were in the personal Copilot, not the work one.

Here is what I say to my kids. If you've done something wrong and you tell me, you have immunity. If I find out another way, that's a different story.

Same principle applies here. If something goes wrong, tell someone. It gets managed. You will be fine. The firms that stay safe are the ones where people feel safe to speak up.

AI is not going anywhere. Neither are the mistakes. The ones who stay safe are the ones who talk about it.

One more thing

If you think your team could benefit from a refresher on this, I have a short video series for you. Five videos and a downloadable PDF. It covers how to verify AI responses, what AI often gets wrong, how to handle confidential information, and what to do if something goes wrong. Some of my clients use it as part of their staff onboarding.

You can get it at inbal.com.au/aiform

—

Inbal Rodnay

Guiding Firms in AI Adoption and Automation

Keynote speaker | AI Workshops | Executive briefings | The Tech Savvy Firm

Want to receive these updates straight to your inbox? Click here: www.inbal.com.au/join

When you are ready, here is how Inbal can help: