Technical SEO still decides whether AI can find you

Assistants that search the web choose from pages a crawler can fetch and a search index can return. If your best comparison is noindexed, stuck in a staging basic-auth wall, or canonicalized to a thinner URL, it is not a candidate. The model may still mention you from older memory, and the mention will lag the product. The citation, the part you can influence this month, needs a URL that exists in the open index.

Check indexation on the revenue URLs first, not on the blog as a whole. Search Console coverage, a site: query, and a crawl that lists status codes will tell you whether the alternatives page, the integration pages, and the docs intro are actually available. A site can have thousands of indexed blog posts and one commercial page stuck in a parameter trap. That site will look healthy in a content report and absent in the answers that create pipeline.

Robots rules deserve a human read after every redesign. Teams block /resources or a JavaScript folder and accidentally hide the templates that render the copy. They allow the HTML and block the only path that loads the text. Render the page the way a crawler will and read the extracted text. If the answer you hope to be quoted is missing from that extract, markup and widgets are hiding it.

AI products fetch pages with their own crawlers, and robots.txt treats each one separately. Some fetch pages for search features, such as OAI-SearchBot for ChatGPT search, PerplexityBot, and Claude-SearchBot. Some fetch a page when a user asks about it, such as ChatGPT-User and Perplexity-User. Others collect training data, such as GPTBot and ClaudeBot. Google's AI Overviews and AI Mode still depend on Googlebot and Google's index, and Copilot on Bingbot and Bing's. Blocking the training crawlers is a reasonable business choice. Blocking the search and user crawlers by accident takes you out of the answers that cite live pages, and a copied robots.txt or a security plugin does that more often than anyone admits.

Robots.txt is only the first gate. A noindex meta tag or an X-Robots-Tag header keeps a page out of search features. Nosnippet, or a very small max-snippet, keeps it from being quoted. Bing reads noarchive as "leave this out of Copilot answers" and nocache as "title and snippet only." And a page whose text only appears after JavaScript runs is blank to many AI fetchers. Check each of these on the pages you most want quoted: home, pricing, product, comparison, and your facts page.

The GEO agent runs this audit on your key pages: robots rules for 38 AI bots, meta robots and X-Robots-Tag, snippet limits, Bing's noarchive and nocache, JavaScript-only content, sitemaps, canonicals, and whether each page is indexed. It fetches as its own named crawler, follows robots.txt, and never poses as another bot. The verdict for each AI bot comes from reading your rules, not from impersonating it.

Duplicate commercial pages split signals. A versus page, a tracking-parameter copy, a print view, and a campaign landing page that repeats the same comparison teach the index to pick one and ignore the others. Sometimes it picks the worst one. Set a canonical you would be happy to see cited. Link internally to that canonical only. Retire the copies or give them a distinct intent.

Faceted navigation and tag archives create the same problem at scale on content sites. If they are indexable, they compete with the guide you wrote on purpose. If they are your only internal links to a revenue page, blocking them orphans the page. The fix is deliberate HTML links from indexed, useful URLs, plus a crawl rule that keeps the infinite combinations out of the index. This is ordinary technical SEO. It decides which URL an AI Overview is allowed to find.

International sites add a country version of the same risk. Hreflang and a clear country URL structure tell engines which language and market a page serves. Mixed currencies and mixed claims on a single URL produce mixed answers. If you sell in specific countries, the GEO prompts should be tracked in those markets, and the indexed page for that market should be the one whose facts match the offer.

Crawlers discover pages by following links. So do people. A revenue page linked only from a sitemap and a footer column is a whisper. A revenue page linked from the header, from the relevant docs, and from the guides that mention the job is a statement. Anchor text should say what the page is. "Learn more" teaches nothing. "Payout API vs building bank transfers in-house" teaches the destination and the query.

Click depth matters more than a mystical link-equity formula. If a new visitor and a crawler can reach the comparison in one or two clicks from the homepage, you have done the structural job. If it takes five clicks through a blog year archive, you have buried the only page that can catch demand. Audit the money URLs for inlinks every month. Orphans are a publishing bug, not a content-quality mystery.

HTML links beat onclick navigation that never renders an href. Some app-like marketing sites look linked and ship a crawl with a homepage and almost nothing else. Run a crawler that does not execute your design system's assumptions. The list of URLs it finds is the list a cautious engine can find. Everything important that is missing from that list needs a plain anchor.

Core Web Vitals are not a GEO spell. They are a condition for a page being pleasant enough to rank and to use. Heavy hero videos, unoptimized illustration, and chat widgets that block rendering hit the templates you care about most, because those templates carry the brand. Fix the commercial template once and every comparison benefits. Chasing a perfect score on a blog post while the pricing page stutters is the wrong order.

Stability includes not breaking the text extraction. Client-side rendering that shows a headline to a browser and an empty container to a simple fetch will starve anything that does not run your full JavaScript budget. Server-render the words. Hydrate the extras. This single architectural choice does more for AI visibility than a year of "AI keyword" plugins.

Watch regressions weekly. A release can add a megabyte, a noindex, or a canonical bug on a Thursday and sit unnoticed until the quarter's rankings slide. The SEO agent is built to flag crawl, speed, indexation, internal links, and structured data as they drift, because humans do not re-audit a template for fun. On the GEO side, the agent watches the pages engines cite, your own key pages, and any placements, and flags a page that changes by a fifth or more, redirects, disappears, or gets newly blocked. The point of the watch is a short list, not a 90-page PDF nobody assigns.

Schema does not make a claim true and it does not force a citation. It translates claims you already show into a form machines misread less often. Organization schema with a consistent name and URL. Product or SoftwareApplication schema when the page is actually about the product. FAQ schema only for questions visible on the page. Breadcrumb schema so the hierarchy is explicit. Article schema on essays like this one, with a real date.

Keep the schema aligned with the visible text. Marking up a price the page does not show, or a review score you invented, is how rich-result eligibility turns into a trust problem. If the facts change, change the HTML and the JSON-LD together. Generated schema from a plugin that guesses types will mark a comparison page up as a blog post and a docs page up as an advertisement. Review it on the templates that matter.

Structured data still matters less than the page it describes. Third-party comparisons, consistent facts, answer-first pages, and expert mentions do more to change an answer, and that ordering keeps teams from spending a quarter on markup while the comparison page is still a slogan. The GEO agent builds Organization, Product or SoftwareApplication, FAQ, and LocalBusiness markup straight from your approved facts, with no model writing it, so the markup states exactly what the facts file says. Ship schema as part of the template work in this article. Spend the argument on the sentence the schema describes.

Engines resolve a company through names, URLs, profiles, and repeated descriptions. Technical choices affect that resolution. One canonical domain. HTTPS. A stable about URL. A logo and name that match the organization markup. Redirects that preserve the old URL with a single hop. A messy domain history, a blog on a subdomain nobody links to, and three legacy domains still indexed will scatter the entity.

Editorial choices finish the job. The same one-sentence category description in the footer, the about page, the docs, and the profiles. The same product name in the title tag and the H1. Titles that change every week for "CTR tests" shred recognition. Test titles when you have a measurement plan. Do not rotate them because a tool suggested a synonym.

When two product lines exist, give them distinct names and distinct URLs. A model that cannot tell your API from your dashboard will recommend the wrong one and your sales team will hear it. Technical IA is positioning. The information architecture you ship is the taxonomy an answer will inherit.

An XML sitemap should list the canonical URLs you want indexed, with accurate lastmod when the content truly changed. It is a hint, not a command. Stuffing it with redirects, noindex URLs, and parameter duplicates wastes the hint. HTML sitemaps are optional if the internal links are real. A sitemap cannot rescue a page nothing links to.

Server logs and crawl stats show whether the engines you care about are fetching the new URL. A fetch is not a citation and it is not a ranking. It is evidence the door is open. If a new commercial page receives no fetches after a week, the internal links or the sitemap or the host response is wrong. Look there before you rewrite the prose a third time.

Ignore proprietary scores that promise "AI readiness" from a checklist of meta tags. Readiness is indexation, a canonical that matches the page you want quoted, internal links, rendered text, speed that does not embarrass the template, and schema that matches the visible facts. The rest of GEO is editorial and distribution, covered in the citation guide. Technical work makes that editorial eligible.

Start with a crawl of the production host. Export status codes, indexability, canonicals, titles, H1s, and word count of rendered text for every URL that contains a revenue intent. Flag anything commercial that is non-200, non-indexable, canonicalized elsewhere, or thin in extracted text. Fix that set before you commission new articles.

Second, map internal inlinks to that set. Add HTML links from the homepage and from the docs or guides that share the topic. Remove competing duplicates. Third, run the commercial template through a performance pass and a schema pass, and confirm the JSON-LD price, name, and description match the screen. Fourth, request indexing for the corrected URLs and watch fetches.

Put this pass on a weekly schedule, because the next release will nick it. Pair it with the prompt baseline. When an engine cites a third party and ignores your better page, you can now rule out "they cannot see us" and move to "they do not trust us yet," which is an outreach and proof problem. When they cannot see you, no amount of prose will move a live-search answer. Technical SEO is how you earn the right to have the content argument.