There’s a new worry creeping into conversations with business owners, and it’s an understandable one. We’ve spent years telling businesses how important it is to be found on Google, and now we’re telling them they need to think about being found by AI as well.
So, naturally, people are starting to test it.
They open ChatGPT, Claude or another AI assistant, give it their website address and ask it to have a look. Sometimes it works beautifully. Other times, the AI comes back saying it can’t access the website.
Which immediately raises a rather uncomfortable question. If an AI platform can’t access your website, does that mean your business is invisible to people using AI to find products and services?
Fortunately, the answer isn’t quite that straightforward. Although, as is often the case with anything involving AI search at the moment, there are a few moving parts to understand.
What Does Being Visible in AI Search Actually Mean?
When we talk about AI search visibility, we’re really talking about whether your business appears when potential customers ask AI platforms questions related to the products and services you offer.
Imagine you run a specialist engineering company. Someone might ask ChatGPT which UK businesses manufacture a particular component, use Perplexity to research suppliers with experience in their industry, or ask Gemini to compare different manufacturers.
They may never have heard of your business, and they certainly won’t be giving the AI your website address. They’re looking for information, recommendations and options to consider.
Those are the opportunities we’re interested in when we talk about improving a business’s visibility in AI search.
Related: What is Answer Engine Optimisation (AEO)?
Of course, there are also situations where someone specifically asks an AI assistant to visit your website. They might want to compare your offering with a competitor’s or check whether you provide a particular service. In those circumstances, it’s clearly beneficial if the AI can retrieve your public content.
But being able to access a particular webpage and choosing to mention a business in response to a customer’s question are two different things.
And that’s where some of the confusion starts.
Not All AI Bots Are Doing the Same Job
One of the things worth understanding is that AI platforms don’t necessarily use a single system to access information from websites. Different automated systems may have different purposes, and website security settings can affect them in different ways.
OpenAI, the company behind ChatGPT, provides a useful example.
It identifies three systems with distinct functions. OAI-SearchBot is used to crawl publicly available website content for potential inclusion in ChatGPT Search. ChatGPT-User is associated with situations where a user specifically asks ChatGPT to retrieve information from a webpage. GPTBot, meanwhile, is primarily associated with collecting information that may be used to train AI models.
So, if you ask ChatGPT to look at your website and it can’t access the page, that doesn’t automatically mean OAI-SearchBot is also being blocked.
And even if all three can access your website, that doesn’t guarantee your business will appear when someone asks ChatGPT for a recommendation.
Other platforms, including Claude, Perplexity, Gemini and Microsoft’s Copilot, have their own approaches to retrieving information, using search indexes and accessing website content. Some publish information about their crawlers and user-triggered access systems, while others rely partly on existing search infrastructure.
There isn’t one universal AI bot that every platform uses, nor is there one setting that guarantees visibility everywhere.
Which is mildly inconvenient for those of us who would quite like a simple instruction manual.
The practical point is that an access problem with one AI platform, or even one particular request, doesn’t necessarily tell us what’s happening across the wider AI search landscape.
Your Website Security Has a Job to Do
Before we start worrying about whether every AI platform can access our websites, it’s worth remembering why websites have security measures in the first place.
Websites receive an enormous amount of automated traffic, and not all of it is particularly friendly. Alongside legitimate search engine crawlers and other automated services, there are bots attempting to identify vulnerabilities, access restricted information, scrape content or generally cause trouble.
Security systems such as firewalls, WordPress security plugins and services like Cloudflare help manage this activity. They assess incoming requests and decide whether to allow, challenge or block them.
Occasionally, perfectly legitimate requests get caught up in those rules.
An AI assistant trying to retrieve a public webpage might be blocked because of how the request is made or how the security system classifies it. That doesn’t mean the AI platform is doing anything malicious, and it doesn’t necessarily mean there’s anything fundamentally wrong with your website.
But it does create a challenge. We want legitimate search engines and AI crawlers to access the public information that helps customers discover our businesses, without weakening the protections designed to keep harmful activity out.
And simply allowing anything that identifies itself as an AI crawler isn’t a particularly sensible solution. Malicious bots can impersonate legitimate ones, which is why verification and properly configured security rules matter.
There’s also a significant difference between making information about your products and services publicly accessible and protecting customer details, account information or other sensitive data. The two shouldn’t be confused.
If legitimate AI search crawlers are consistently being blocked, that’s worth investigating with your developer or hosting provider. But turning off security features because an AI assistant couldn’t open your homepage is rather like removing the locks from your office because a delivery driver couldn’t get through the front door.
It might solve one problem, but you’ve potentially created a much bigger one.
Being Accessible to AI Doesn’t Guarantee Being Visible in AI Search
This is the distinction I think businesses really need to understand.
You could have a technically excellent website that legitimate AI crawlers can access without any difficulty, yet still find that your competitors are being mentioned in AI-generated answers while your business barely features.
That’s because accessibility and visibility aren’t the same thing.
AI platforms may draw on information from business websites, search indexes, industry publications, third-party references and other publicly available sources when generating answers. Exactly how they select and prioritise that information varies between platforms, and much of the process remains opaque.
But simply making information available doesn’t guarantee it will be used.
The reverse is also possible. An AI platform might mention your business in response to a relevant question even when a particular attempt to access your website fails. It may have information available through other sources, search results or previously acquired data.
That doesn’t make technical accessibility unimportant. If your website is consistently inaccessible to legitimate crawlers, it could affect how effectively your content is discovered and kept up to date.
It just means that one failed access request isn’t a particularly reliable way to assess your overall AI search visibility.
How Do You Know Whether Your Business Is Appearing in AI Search?
Rather than focusing exclusively on whether an AI assistant can open a particular page, we think businesses need to start looking at the questions their potential customers are actually asking.
If you’re a commercial insurance broker, for example, you might want to understand which businesses appear when someone asks an AI platform to recommend brokers specialising in a particular industry. If you manufacture industrial equipment, you might be interested in how your company appears when someone researches suppliers or compares product specifications.
The questions will vary depending on the business, but the principle is the same. We want to understand whether the company is appearing in commercially relevant conversations, how accurately it’s being represented and which competitors are receiving visibility.
At WSI, we’re increasingly using specialist AI visibility monitoring platforms, including Rankscale, to investigate this. These tools allow us to test selected prompts across AI platforms, monitor brand mentions, examine citations and identify topics where competitors may be appearing more frequently.
They’re useful, but they do have limitations. They can’t see every question being asked by every AI user, and they can’t tell us with certainty why a particular platform selected one source over another. The results also vary depending on the questions being tested, the platform and other factors.
What they give us is a way to monitor patterns over time, rather than drawing sweeping conclusions from one or two manual tests.
And, importantly, the findings need to lead to action.
Sometimes that means addressing technical problems that are preventing crawlers from accessing useful content. Sometimes it means improving the information on a website so that products, services and expertise are explained more clearly. In other cases, it may mean strengthening a business’s presence across relevant third-party websites or developing content that better answers the questions customers are asking.
Much of this builds on the foundations of good Search Engine Optimisation (SEO). The difference is that we’re now having to consider how those foundations influence visibility across several different AI-driven search experiences, rather than focusing solely on traditional search rankings.
The Rules Are Still Being Written
One of the challenges with AI search is how quickly everything is changing.
ChatGPT, Gemini, Claude, Perplexity, Copilot and the other platforms are continually developing how they retrieve information and generate responses. New features appear, existing systems change and the methods we use to measure visibility have to evolve alongside them.
It’s all a bit of the Wild West, which makes it an interesting space to work in, although occasionally it would be lovely if everyone could stop changing things for five minutes.
We’ve spent decades learning how traditional search engines work, and even there, nobody outside those organisations knows every detail of their algorithms. AI search introduces another layer of complexity, particularly because the systems involved aren’t always transparent about how they select and use information.
Related: SEO and AI Overviews
At WSI, we’re spending a lot of time researching, testing and monitoring what actually happens when businesses appear in AI-generated answers. Some of what we’re seeing reinforces established SEO principles, while other findings are making us rethink how we approach certain aspects of visibility.
I think it’s important to be open about the fact that we’re still learning. Anyone promising a guaranteed formula for getting your business recommended by every AI platform is making a rather ambitious claim.
For now, the sensible approach is to keep testing, measure what we can, investigate what we don’t understand and adjust as the technology develops.
So, Is Your Website Blocking AI?
If an AI assistant tells you it can’t access your website, it’s worth finding out why. There may be a straightforward technical explanation, or it may highlight an issue that needs attention.
But don’t assume that one failed request means your business is invisible across AI search. And don’t start dismantling your website security in an attempt to make every automated visitor feel welcome.
The more useful question is whether your business is appearing when potential customers use AI to research the products and services you offer.
Are you being mentioned alongside your competitors? Is the information about your business accurate? Are AI platforms finding the content that demonstrates your expertise and helps customers understand what makes you a suitable choice?
Those are the questions that connect AI search visibility to actual business opportunities.
Because ultimately, we’re not trying to make our websites popular with robots. We’re trying to make our businesses easier for potential customers to discover, understand and choose.
The robots are just part of the process.
