How to rank in LLMs and get recommended by AI agents: the complete playbook (2026)
AI assistants like ChatGPT, Claude, and Perplexity are becoming the first place buyers ask for software recommendations — and AI agents are starting to make purchases autonomously. Ranking in LLMs requires three things: (1) letting AI crawlers read your site, (2) making your content machine-legible and structured, and (3) being present in the sources LLMs trust. This guide covers all three, step by step.
How do LLMs decide which products to recommend?
LLMs draw from two layers — their training data (what the web said about you historically) and their live search/retrieval layer (what their crawlers can read right now). You influence the first through presence in trusted sources, and the second through technical legibility.
The retrieval layer runs on named crawlers: OpenAI ships GPTBot (training) and OAI-SearchBot plus ChatGPT-User (live browsing); Anthropic ships ClaudeBot; Perplexity ships PerplexityBot and Perplexity-User; Google ships Google-Extended for Gemini. When a user asks “best invoicing tool for freelancers,” the assistant issues a search, fetches the top pages, and synthesizes a retrieval-augmented answer with citations. Agents go further: to actually act — compare, sign up, or buy — they need structured data (pricing schema, an OpenAPI spec, an MCP endpoint, an ACP product feed) that turns your site into a callable API.
How do you allow AI crawlers in robots.txt?
Add explicit Allow rules for GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot, and Google-Extended. Blocking them — or leaving ambiguous rules — makes your site invisible to AI search and answer engines.
Most SaaS sites either block AI bots by copy-pasting a “defensive” robots.txt from a blog post, or leave the default that some CDNs ship with. Both mistakes cost you citations. Serve the block below at /robots.txt. If you use a WAF (Cloudflare, Vercel, Fastly), also whitelist those user agents at the edge — a 403 counts as blocked.
robots.txtUser-agent: * Allow: / User-agent: GPTBot Allow: / User-agent: OAI-SearchBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: ClaudeBot Allow: / User-agent: PerplexityBot Allow: / User-agent: Google-Extended Allow: / Sitemap: https://yourdomain.com/sitemap.xml
What is llms.txt and how do you write one?
llms.txt is a plain-Markdown file at the root of your domain that tells LLMs which pages matter, in what order, and what each one is about. It is the AEO equivalent of a sitemap written for a human reader.
The spec lives at llmstxt.org. Serve it at /llms.txt. Optionally also publish /llms-full.txt with the full text of your most important pages concatenated, so an LLM can ingest your product in one fetch.
llms.txt# Acme Invoicing > Acme is invoicing for freelancers. Send invoices, accept cards and ACH, and get paid in 2 days on average. ## Core pages - [Product overview](/): What Acme does, who it's for, the 30-second pitch. - [Pricing](/pricing): Free plan, Pro at $12/mo, Business at $39/mo. No sales calls. - [API docs](/docs/api): REST + webhooks. OpenAPI at /openapi.json. - [Integrations](/integrations): Stripe, QuickBooks, Zapier, MCP server. ## Trust - [Security](/security): SOC 2 Type II, GDPR, data residency. - [Changelog](/changelog): Weekly product updates.
Which structured data (JSON-LD) do LLMs actually use?
At minimum: Organization on every page, SoftwareApplication with a nested Offer on your product/pricing page, and FAQPage on your FAQ. LLMs and their retrieval layers use schema.org markup to extract facts they can quote without hallucinating.
Put JSON-LD in a <script type="application/ld+json"> tag in your <head>. Include real, numeric prices — not “contact us”. Validate with Google's Rich Results Test and Schema.org validator.
json{ "@context": "https://schema.org", "@type": "SoftwareApplication", "name": "Acme Invoicing", "applicationCategory": "BusinessApplication", "operatingSystem": "Web", "offers": [ { "@type": "Offer", "name": "Pro", "price": "12.00", "priceCurrency": "USD", "priceSpecification": { "@type": "UnitPriceSpecification", "price": "12.00", "priceCurrency": "USD", "unitCode": "MON" } } ], "aggregateRating": { "@type": "AggregateRating", "ratingValue": "4.8", "reviewCount": "312" } }
Why must you serve real HTML instead of a JavaScript shell?
Most AI crawlers do not execute JavaScript. If your page ships an empty <div id='root'></div> and hydrates on the client, GPTBot and ClaudeBot see nothing. Serve server-rendered or prerendered HTML.
Test it in 5 seconds with curl. If the response body does not contain your headline, pricing, and core copy, the AI crawlers see the same emptiness. Fix with SSR (Next.js, TanStack Start, Remix, SvelteKit), static generation, or a prerender service (Prerender.io, Rendertron).
bashcurl -sA "GPTBot" https://yourdomain.com/ | grep -i "your headline" # If nothing prints, LLMs can't see your content.
How do you publish machine-readable pricing?
Put real numbers on a public /pricing page, mark them up with schema.org Offer, and never hide plans behind “contact sales”. Agents cannot evaluate — let alone buy — what they cannot read.
A parseable HTML table with plan name, price, billing cadence, and included limits beats a beautiful pricing widget that renders in JS. If enterprise pricing is genuinely custom, publish a starting price (“from $X/user/mo”) so the model has something to quote. “Book a demo” alone is a competitive gift to whoever publishes numbers.
How do you expose your API for agents?
Publish API docs at a predictable URL (usually /docs or /api), and serve an OpenAPI spec at a discoverable location like /openapi.json. Agents integrate with what they can read; undocumented APIs do not exist for them.
Link the OpenAPI file from your docs page and from /llms.txt. Prefer a single canonical spec over three partial ones. Include authentication, rate limits, and example requests — those are the fields an agent needs to actually call you.
What are agent manifests (MCP and ACP) and when do you need them?
Agent manifests declare capabilities an autonomous agent can invoke. MCP (Model Context Protocol) exposes tools and data to any MCP-aware model; ACP (Agentic Commerce Protocol) exposes products, pricing, and checkout so agents can buy. Publish them if you want to be actionable, not just discoverable.
Read the specs: modelcontextprotocol.io and agenticcommerce.dev. A common pattern is a public MCP endpoint plus a /.well-known/agents.json that lists it. For e-commerce or usage-based SaaS, an ACP product feed lets agents transact without a human handoff.
Where do LLMs actually look when they recommend software?
Third parties, not your homepage. G2, Capterra, comparison articles (“X vs Y”), Reddit and Hacker News threads, and category directories dominate what models cite. LLMs trust corroborating sources more than self-description.
Claim and complete your profiles on G2, Capterra, GetApp, Product Hunt, and every credible directory in your category. Seed honest comparison content (yours and others') that names competitors — LLMs are pattern-matching on those phrasings. Answer questions in the communities your buyers already read; a well-cited Reddit answer outranks a landing page.
Why must your story be consistent everywhere?
LLMs hedge when signals conflict. If your G2 profile says “project management” and your site says “work OS”, and your Twitter bio says “collaboration platform”, the model picks a competitor it is more certain about.
Lock in one name (spelling, casing), one one-line description, one category, and one price point. Propagate them to your site, directories, socials, and every guest post. Boring consistency outperforms clever variation. Re-audit quarterly.
How do you measure and iterate on LLM visibility?
Ask each major assistant your category's core queries — “best [category] tools for [ICP]” — and log who gets cited, in what order, with which source. Re-scan monthly, and track deltas after each fix. If nothing changes, the fix was cosmetic.
Run the same 5–10 prompts in ChatGPT, Claude, Perplexity, and Google's AI Overview each month. Note whether you are mentioned, cited, or ignored, and which URLs each answer links to. Re-run the AIScoring scan on your site and top three competitors. The gap between your check list and theirs is your backlog.
Frequently asked questions
How do I get ChatGPT to recommend my product?
Do three things: (1) allow GPTBot, OAI-SearchBot, and ChatGPT-User in robots.txt so OpenAI's crawlers can read you; (2) make your product page and pricing machine-legible with server-rendered HTML and schema.org SoftwareApplication + Offer markup; (3) build a citation footprint on the sources ChatGPT quotes — G2, Capterra, comparison articles, Reddit, and category directories. ChatGPT cites third-party corroboration more than self-description.
What is llms.txt and do I need it?
llms.txt is a Markdown file at the root of your domain (spec at llmstxt.org) that lists the pages you want LLMs to read, in priority order, with descriptions. It is not yet an official web standard, but adoption is growing fast and it is trivial to publish. Yes — add one.
Does blocking GPTBot hurt my business?
Yes, in 2026 it does. Blocking GPTBot removes you from ChatGPT's search and browsing surface, which is now a top-of-funnel channel comparable to organic Google search for many B2B categories. If your concern is training data usage, block for that purpose specifically (many models honor separate signals) rather than blocking search retrieval.
How long does it take to appear in LLM answers?
Retrieval-based answers (Perplexity, ChatGPT search, Claude web) can pick up changes within days once your site is crawled. Training-data-based answers (a model's built-in knowledge) update on model release cycles — months to a year. Optimize for both, but expect early wins on the retrieval layer.
What's the difference between SEO and AEO/GEO?
SEO optimizes to rank in a list of blue links a human clicks. AEO (Answer Engine Optimization) and GEO (Generative Engine Optimization) optimize to be quoted inside an AI-generated answer. AEO cares about extractable direct answers, structured data, and third-party citations; SEO cares about keywords, backlinks, and click-through. They overlap, but the tactics diverge.
Can AI agents actually buy software today?
In early production, yes — but the market is nascent. The Agentic Commerce Protocol launched in 2025, MCP tool-use is now widespread in developer workflows, and Stripe, Shopify, and others have shipped agent-checkout primitives. Autonomous purchase of SaaS by agents is real but small in 2026; most analysts expect meaningful volume by 2028. Being ready now is cheap insurance.
How do I know if my site is agent-ready?
Run a scan at aiscoring.io. You'll get a grade (A–F), 12 transparent checks covering AI crawler access, llms.txt, structured data, agent manifests, SSR, pricing legibility, and API discoverability, plus a fix list ordered by impact.
The 5 most common mistakes
- 01Blocking AI crawlers while pouring budget into content marketing — you're paying to write for readers you've locked out.
- 02Hiding pricing behind “book a demo” — agents cannot evaluate or purchase what they cannot read, so the model recommends a competitor with public prices.
- 03Shipping a JavaScript-only shell — AI crawlers rarely execute JS, so an unhydrated is what they cite.
- 04Inconsistent naming, categorization, or positioning across your site, directories, and socials — LLMs hedge on ambiguity and pick a clearer competitor.
- 05Treating agent-readiness as a one-time project — crawler policies, schema, and third-party citations decay; audit and re-scan monthly.
See where your site stands in 10 seconds
The AIScoring scanner runs all 12 checks in this guide against any SaaS URL and gives you an A–F grade plus a prioritized fix list.