Copilot Vision: When AI Joins You on Screen
Today we’re one step closer to AI that truly sits on our devices, can see everything we’re doing, and can chat with us about it.
No more screenshots, no more uploads, no need to give it context.
It just lives with us.
This is where we’re going.
AI comes to our world rather than us having to go to it.
In the video, I share Microsoft’s new capability called Copilot Vision.
Inside Copilot Vision
Microsoft Copilot is installed directly on the device, and the magic happens when we activate Vision through the glasses icon.
You can share your screen or a single app, and Copilot instantly recognises what you’re working on.
For instance, when viewing a QuickBooks dashboard, Copilot identifies it and asks whether you’d like help with something in the app.
I asked, “How do I schedule a report so that it gets sent to me by email every week?”
From there, Copilot begins guiding step by step.
When I ask it to highlight where to click, it shows a bright clear circle around it.
It can point out exactly where to go next, like the Reports tab on the left side.
It’s clear, visual, and fast to respond.
The interaction feels natural, like having an assistant right beside me, ready to help in real time.
I actually used it a few times today with QuickBooks Online and OBS, the app I use to record my videos.
The difference is, QuickBooks Online runs in a browser, while OBS is an installed app like Excel or Word, and Copilot can handle both.
Why It Matters
I love that AI comes to where I am, into my browser, into my desktop, onto my device, instead of me having to go to it, log in, upload files, or explain what’s going on.
In the settings, you can make it auto start on login, or set it to respond when you say, “Hey Copilot.” so the entire interaction is almost hands-free.
We can also allow it to search and read files on our PC to find things for us and be further grounded in our context.
Where It’s Heading
We’re seeing a few directions now.
Copilot Vision sits on the device and works with everything we’re doing.
It’s not yet doing tasks for us, but Microsoft has already shown where it’s going, like renaming files, giving images meaningful names, or organising folders.
We’re also seeing AI in browsers, able to interact directly with what’s on the sites we visit.
Which one will win?
Not sure.
Maybe both will coexist.
For now, it’s worth trying both so you can get comfortable with them.
Copilot Vision is already a useful feature to explore for tech support, brainstorming and generally reviewing work.
A Note on Privacy
When you start, you’ll be asked to acknowledge Microsoft’s privacy statement.
It covers things like what Copilot Vision can access, what it does with our information and where it is stored.
It’s worth reading through before you begin.
Have fun!
—-
Inbal Rodnay
Guiding Firms in Adopting AI and Automation
Keynote speaker | AI Workshops | Executive briefings | Consulting CIO
Want to receive these updates straight to your inbox? Click here: www.inbal.com.au/join
When you are ready, here is how Inbal can help:
Bring your entire team on the AI journey in just 30 days. This program is designed to give your team a solid foundation in using generative AI in responsible and impactful ways. Inbal helps you choose your AI tools, create an AI policy and train your team.
Want the confidence to set strategy and lead but don't have time to keep up with all the changes in tech?
Tailored for your needs, Inbal will works with you through one-on-one sessions to develop your technology literacy and keeps you up to date.
For CEOs, partners and business leaders. Everything you need to know about AI without the noise. Inbal shares the state of AI, recommends tools, and answers your questions about strategy, implementation and safe use.
Only what's real, no hype, no noise.
This is a one-off session for your entire leadership team.
