Technical Website Audit Checklist for AI Search Visibility in India

Why technical issues block AI visibility

AI engines gather information about businesses by processing web content. If your website blocks their crawlers, loads too slowly to be processed reliably, has no structured data, or delivers incorrect business information, the AI either cannot read your site or cannot trust what it reads.

This is a different kind of problem from content quality. A business owner can write excellent service descriptions but still be invisible on AI search because their robots.txt blocks AI crawlers, or their schema markup has errors, or their website takes 12 seconds to load on a mobile connection.

Use this checklist to identify and fix the technical barriers. Each section can be audited using free tools available to any business owner.

Section 1: Crawlability and indexation

Before AI engines can use your content, they need to be able to access and read it.

Check your robots.txt file

Your robots.txt file tells crawlers which parts of your site they can access. Access your file at yourdomain.com/robots.txt. Look for any lines that say "Disallow: /" which would block all crawlers. Also check for rules that might accidentally block AI crawlers.

The AI crawlers you should explicitly allow are: Googlebot, Bingbot, OAI-SearchBot (OpenAI/ChatGPT), Claude-Web (Anthropic), PerplexityBot, and Applebot. A robots.txt that allows all of these looks like:

User-agent: *
Allow: /

User-agent: OAI-SearchBot
Allow: /

User-agent: Claude-Web
Allow: /

User-agent: PerplexityBot
Allow: /

Sitemap: https://yourdomain.com/sitemap.xml

Check your sitemap

Your sitemap should list all the pages you want AI engines and search engines to find. Access it at yourdomain.com/sitemap.xml. Verify that all your important pages are listed and that the URLs are correct. If you do not have a sitemap, create one. Most website builders have a plugin or setting for this.

Check which pages are indexed by Google

Go to Google Search Console and check the "Pages" report. This shows which of your pages Google has indexed and which have been excluded. Pages that Google cannot index are also unlikely to appear in AI-generated answers.

Check for redirect chains

If your website has changed domain or URL structure, you may have redirect chains where page A redirects to page B which redirects to page C. These slow down crawlers and can cause pages to be skipped. Each redirect should go directly to the final destination URL.

Section 2: Schema markup

Schema markup is the most direct way to communicate your business identity to AI engines. Without it, AI must infer everything from your text alone.

Check existing schema

Go to Google's Rich Results Test at search.google.com/test/rich-results and enter your website URL. This tool shows you what structured data Google finds on your site and whether it is valid. If no structured data is found, you have none and need to add it.

Required schema for local Indian businesses

Every Indian business website should have at minimum:

• LocalBusiness or ProfessionalService schema on the home page, including: full business name, street address, city, state, postal code, country (IN), phone number, email, and opening hours
• BreadcrumbList schema on all pages to help AI understand your site structure
• FAQPage schema on any page that has a FAQ section
• BlogPosting schema on each blog article, with author name, datePublished, and dateModified

Validate your schema

After adding schema, validate it at schema.org/validator. Common errors include missing required fields, incorrect property names, and invalid date formats. All errors should be fixed. Warnings are acceptable but worth reviewing.

Add sameAs links to your schema

The sameAs property in schema markup links your website to your profiles on other platforms. For Indian businesses, include links to your LinkedIn company page, your Google Business Profile (maps.google.com link), and your JustDial listing if applicable. This helps AI engines connect your website to your business's presence across the web.

Section 3: Page speed

A slow website is a problem for AI crawlers and for human visitors. AI crawlers have time limits on how long they wait for a page to load. If your page takes more than a few seconds, the crawler may give up before processing your content.

Test your page speed

Go to pagespeed.web.dev and enter your website URL. Run the test on both Mobile and Desktop. Note the Performance score. For AI visibility purposes, aim for a score above 70 on mobile. Scores below 50 indicate significant problems that are likely preventing full crawl of your content.

Common speed problems on Indian websites

• Large uncompressed images: Images that are 2MB or larger slow pages significantly. Compress images to under 200KB using tools like squoosh.app or tinypng.com.
• Shared hosting with slow servers: Budget shared hosting in India often has high response times. If your server response time is above 600ms (shown in PageSpeed Insights), consider upgrading your hosting.
• No content delivery network (CDN): A CDN serves your website from servers closer to the user. Cloudflare's free plan is sufficient for most Indian business websites.
• Render-blocking scripts: JavaScript files that load before your page content block the page from displaying quickly. Your developer can fix this by moving scripts to the bottom of the page or adding the "defer" attribute.

Section 4: Mobile usability

The majority of internet users in India access the web on mobile devices. Google and AI engines that use Google's data prioritise mobile-friendly websites.

Test mobile usability

In Google Search Console, go to the "Mobile Usability" report. This shows specific mobile problems on your pages. Common issues include text that is too small to read, clickable elements that are too close together, and content that is wider than the screen.

Also test your website on an actual mobile phone on a 4G connection, not just in a desktop browser's developer tools. Real-world mobile performance often differs from simulated results.

Check viewport configuration

Your website's HTML should have the following tag in the <head> section:

<meta name="viewport" content="width=device-width, initial-scale=1.0">

Without this tag, mobile browsers display your website at desktop width, making text tiny and navigation unusable.

Section 5: HTTPS and security

An insecure website (HTTP rather than HTTPS) is a trust signal problem for both search engines and AI engines.

Confirm HTTPS is active

Visit your website and look at the browser address bar. You should see a padlock icon and your URL should begin with "https://" not "http://". If you see "Not Secure" or your URL begins with "http://", contact your hosting provider to enable SSL. Most hosting providers in India now offer free SSL certificates through Let's Encrypt.

Check for mixed content

Mixed content occurs when an HTTPS page loads resources (images, scripts, stylesheets) over HTTP. This triggers browser security warnings. Use a tool like Why No Padlock (whynopadlock.com) to check your pages for mixed content issues.

Section 6: NAP consistency and business data

AI engines cross-reference your business information across multiple sources. Inconsistency in your Name, Address, and Phone number reduces their confidence in your business data.

Create a NAP reference document

Write down the exact format you want to use for your business name, address, and phone number. Example:

• Business Name: OptiScale Advisors (not "Optiscale" or "Opti Scale Advisors")
• Address: 13, Rayakotta Road, Old Rayakottah Hudco, Hosur, Tamil Nadu 635109
• Phone: +91 73972 25523 (not "7397225523" or "073972 25523")

Use this exact format everywhere: your website, Google Business Profile, JustDial, Sulekha, and any other directory where your business appears.

Audit your directory listings

Search for your business name on Google. Look at all the places it appears. Check each one for NAP consistency. Update any that differ from your reference document.

Section 7: AI bot access

This is a newer concern that many website owners are unaware of. AI companies use specific crawlers to gather content for their models. If these crawlers are blocked, the AI cannot learn about your business from your website.

Check your robots.txt for AI bot rules

Some website security plugins and server configurations block AI crawlers by default, either intentionally or accidentally. Check your robots.txt for any rules that might block OAI-SearchBot, PerplexityBot, Claude-Web, or Anthropic-AI. If you find rules blocking these, remove them unless you have a specific reason to keep them.

Check your llms.txt file

Some websites are beginning to add an llms.txt file (similar to robots.txt but specifically for AI systems). This file tells AI engines which content on your site is most important and relevant for them to process. While not yet universal, adding an llms.txt file is a forward-looking step for businesses serious about AI visibility.

How to prioritise fixes

If you are running this audit yourself and find multiple issues, prioritise in this order:

1. Crawlability issues first. If AI crawlers cannot access your site, nothing else matters. Fix robots.txt blocks and indexation problems before anything else.
2. HTTPS second. An insecure site is a hard barrier for both search engines and AI engines.
3. Schema markup third. This has the most direct impact on AI engines' ability to understand and cite your business correctly.
4. NAP consistency fourth. Audit and fix directory listing inconsistencies systematically.
5. Page speed fifth. Important, but less urgent than the above unless your scores are extremely low (below 30).
6. Mobile usability sixth. Fix mobile issues identified in Search Console after the higher-priority items are resolved.

If you would prefer a professional audit rather than doing this yourself, our AI Visibility Audit covers all of these technical factors plus content analysis and AI platform testing, delivered as a report with specific fixes within 48 hours.

Frequently asked questions

How often should I run a technical audit?

Run a full technical audit twice a year. However, check your Google Search Console "Pages" report monthly for any new indexation issues, as these can emerge when you add new pages or make changes to your website.

Do I need a developer to fix these technical issues?

Some issues (like page speed and schema markup) typically require a developer. Others (like robots.txt, your Google Business Profile, and directory listings) you can fix yourself. Start with what you can do independently and then engage a developer for the technical items.

My website is on Wix or WordPress. Does that change anything?

The checklist applies regardless of your platform. Wix and WordPress both have plugins and settings to manage robots.txt, sitemaps, and schema markup. The specific steps differ by platform but the requirements are the same.

What is the fastest single thing I can do to improve AI visibility today?

Verify and complete your Google Business Profile. It takes 30 minutes and has the highest immediate impact of any single action for local AI search visibility in India.

---

Related reading:

• How to optimise website content for AI search
• How to check if your business appears on ChatGPT
• AI Visibility Audit service - Rs. 15,000