Free Buyer's-Eye Audit
Menu
Free Buyer's-Eye Audit

Results · AI test · Method

AI search test: how we check what assistants say about a manufacturer

We ask AI assistants the questions a buyer's engineer would ask about a company, and check every answer against the company's confirmed record. This page sets out the method, and how to run a rough first pass on your own plant. Results appear here only with the client's written permission, each with its date and the assistants we asked.

Get your free Buyer's-Eye Audit How AEO works

Two tests, kept apart

This page describes the AI assistant test. We ask named assistants about one company on a stated date, and score each answer against what the company confirmed in its Blueprint.

It is separate from the AI Knowledge Test, which grades a website's own content against ten buyer questions and asks no assistant anything. The two measure different things, so we never add their results together. Both methods are on our methodology page.

Method

Four steps, run the same way every time so a re-test can be compared with the first run.

  1. OP 10

    Questions

    The questions a buyer's engineer would ask before a call: limits, specs, certifications and fit for the application.

  2. OP 20

    Assistants

    Named assistants, each asked the same wording, on a stated date.

  3. OP 30

    Scoring

    Each answer marked correct, partial, wrong or 'I don't know', against the company's own confirmed record.

  4. OP 40

    Checks

    Every contradiction checked twice, and every count reconciled before it's published.

Why assistants guess

An AI assistant answers from what it can read. When a website doesn't state a limit, a spec or a certification, the assistant fills the gap with what's typical for that kind of company, and says it with the same confidence as a fact.

A confident wrong answer costs more than 'I don't know'. An engineer who hears the wrong line speed or the wrong certification moves on to the next name, and the company never learns the question was asked. Nothing in its analytics shows the call that didn't come.

The fix starts on the website. Each wrong answer traces back to a sentence that isn't there, or a number locked in a PDF or a photo.

What the fix looks like

We put the missing limits, specs and proof on the pages buyers and assistants read. Each fact goes up as plain text, with its unit, after the client confirms it.

Then we ask the same questions of the same assistants on a stated date, and set what changed beside the first run.

Run a first pass on your own plant

You can run a rough version of this test on your own company in about an hour.

  1. OP 10

    Write ten questions

    The ones your buyers' engineers ask before a call: biggest part, tightest tolerance, materials, certifications, lead time.

  2. OP 20

    Ask two or three assistants

    Use the same wording in each, and write down the date.

  3. OP 30

    Score against your floor

    Correct, partial, wrong or 'I don't know', judged against what your plant actually runs.

  4. OP 40

    Trace each miss

    Find the page where the right answer should be. Usually it isn't there yet.

See the lesson

Sheet 401, The Hoist Says 15 Tons: as said and released (illustration)Sheet 401, The Hoist Says 15 Tons, drawn as two panels. Illustration: the shop, Granite Ridge Machine Works, and every figure are invented. Left, REV A, as said: a bridge crane hook block whose rated load plate reads SWL 15 T. Beside it a lift scale carries a red dashed ghost dimension, 20 to 30 T, marked AI, while the true 15 T is only pencilled, not on the site. An AI assistant card answers an engineer who asked who can machine a 25-ton casting: Granite Ridge routinely handles 20 to 30 tons on a 40 to 45 foot mill, no certifications. The shop's capabilities page states its lift, mill travel and certifications nowhere. A red stamp reads: As AI describes you, not for construction. Caption: the site is silent, so the AI guesses 20 to 30 tons. Right, REV F, released: the same hook block. The capacity page states max lift SWL 15 T, mill travel 240 inches, ISO 9001 and ITAR registered, sourced to the rated load plate. The AI's 20 to 30 T dimension is struck through and the 15 T dimension is inked in gold. A note reads Ask AI again: planned. Plate: Released, illustration, not a promise. Caption: released, the capacity page says SWL 15 T.665544332211DDCCBBAAREV AAS SAIDAS AI DESCRIBES THE SHOPREV FRELEASEDTHE SHOP AS BUILTSHTTITLESUBJECTREVSCALEDRAWNSTATUS401The Hoist Says 15 TonsWhen your site is silent, AI guesses.GRANITE RIDGE MACHINE WORKSgraniteridge-mw.test · BAY 2 HOISTFNTSINDUSTRIALMARKETINGILLUSTRATIONINVENTED FIGURESBRIDGE GIRDER · BAY 2SWL 15 T051015202530 TRATED LIFT20–30 T (AI)15 T · NOT ON SITEAI ASSISTANT · 10:40 PMQ WHO CAN MACHINE A25-TON CASTING?A GRANITE RIDGEROUTINELY HANDLES20 TO 30 TONS,ON A 40–45 FT MILL.NO CERTIFICATIONS.graniteridge-mw.testCAPABILITIESLIFTNOT STATEDCERTSNOT STATEDTRAVELNOT STATEDAS AI DESCRIBES YOUNOT FOR CONSTRUCTIONBRIDGE GIRDER · BAY 2SWL 15 T051015202530 TRATED LIFT20–30 T (AI)SWL 15 T · STATEDgraniteridge-mw.test/capacityCAPACITY & PROOFMAX LIFTSWL 15 TMILL TRAVEL240 INCERTIFICATIONSISO 9001ITAR REGISTEREDSOURCERATED LOAD PLATE, BAY 2CERTIFICATES LINKEDASK AI AGAIN: PLANNEDRELEASEDILLUSTRATION, NOT A PROMISEThe site is silent, so AI guesses 20 to 30 tons.Released: the capacity page says SWL 15 T.

The Hoist Says 15 Tons

Watch the story 6 min

What happens

  1. 01An engineer asks an AI assistant who can machine a big casting
  2. 02The assistant answers with confident numbers
  3. 03The hoist plate on the floor says otherwise
  4. 04The shop never hears the answer was given

Questions about the test

Can you run this test on our company?

Yes. It runs inside a program, after the Blueprint, because we need your confirmed record to score the answers against. We scope it on the review call after your free Buyer's-Eye Audit.

Will fixing our pages make AI assistants recommend us?

We can't promise that, and nobody can. What we can do is put your facts where assistants read them, then ask the same questions on a stated date and show you what changed.

Related

Find out what AI says about your plant

Start with the free Buyer's-Eye Audit, and we'll scope it on the review call after it. Every program starts with the paid Buyers Blueprint, which gives us the record to check the answers against.

Get your free Buyer's-Eye Audit Start with a Blueprint