AI engines change underneath you. A new model ships, or the way an engine decides when and how to search the web changes, and the answers your buyers get about your category can look different the next morning.
The tension is that nothing on your side moved, yet your visibility did. This post covers what a release can change, how to respond in the first week, and what to leave alone.
What changes when an AI engine releases a new model?
A new model can change which sources an engine cites, which brands it names, how it words its answers and which facts it relies on. Any of those can move your visibility without anything changing on your side.
Different sources
Engines that search before answering decide what to search for, how many results to read and which to trust. A new model can make those decisions differently, so the pages cited for the same prompt may change: a review site that used to anchor answers may give way to a community thread, or to a competitor’s comparison page.
Different brands named
When the sources change, the shortlist often changes with them. A model may also name more or fewer options by default, which on its own can move a brand in or out of an answer.
Different wording
The way a model describes your product can shift: what it leads with, which caveats it adds, how it positions you against competitors. That affects sentiment and position even when visibility holds.
Facts from different training data
A model trained on a different snapshot of the web may know about your newer features, or may repeat older ones. Pricing, plan names and capabilities are where this shows up first, and where an inaccuracy costs the most.
Your visibility can shift even when your site did not
Visibility is measured in the answer, and the answer is produced by the engine. When the engine changes how it searches, reads or recalls, the output changes even though your inputs stayed the same.
This is the same mechanism that makes different engines give different answers to the same question, explained in why AI engines disagree. A model release is, in effect, one engine starting to disagree with its own earlier self.
What should you do in the first week after a release?
In the first week, re-run your prompt set, compare each engine against its own baseline, check sources and accuracy, and collect several days of data before drawing a conclusion.
- Confirm what changed. Note which engine changed and when the change reached the surface your buyers use. Releases do not always reach every user or every mode at once, so the announcement and the change in answers may not line up.
- Re-run your prompt set. Run the full tracked set on the affected engine, not a handful of prompts you happen to remember. The set you already track is the only thing you can compare fairly against history.
- Compare per engine against the baseline. Look at the affected engine’s visibility, share of voice and position against its own recent history. Keep the other engines separate; they are your control group.
- Check the sources. For prompts that moved, read which pages are now cited and which dropped out. A source change usually explains a brand change, and it tells you where to look.
- Check accuracy. Read the answers to your branded prompts in full. Look for outdated pricing, missing features and claims that were correct last month and are not now.
- Keep collecting. Let daily runs continue through the week before deciding anything, and annotate the release on your trend lines so anyone reading the chart later knows why the line moved.
What not to do after a release
Do not rewrite pages, change your prompt set or report a result based on a single day of answers. The first days after a release are the noisiest time to make decisions.
- Overreacting to one day. Answers vary between runs even when nothing has changed. One bad morning is not a trend.
- Changing the prompt set. Adding or removing prompts during the comparison breaks it. If the release suggests new prompts worth tracking, add them after the week, as a dated batch. Which prompts to track covers why a fixed set matters.
- Blending engines. An average across engines dilutes the change on the affected one and hides it.
- Rewriting content reflexively. If the model now prefers different sources, the fix may be a third-party listing, not your own page. Diagnose before editing.
- Announcing early. A gain in the first days is as likely to fade as a drop is.
How do you tell a real shift from normal variance?
A real shift persists across days, appears across many prompts and exceeds the movement you normally see in your baseline. Normal variance is short-lived and scattered.
- Does it persist? A change that holds through the week is signal. A change that appears once and reverses is noise.
- Is it broad? Movement across many prompts, or across a whole group such as comparison prompts, points to a change in the engine. Movement on one prompt usually does not.
- Is it bigger than usual? If your visibility on an engine normally wobbles from day to day, a move of that size after a release tells you nothing. A move well outside that range does.
- Is it confined to one engine? If the other engines held steady, the release is the likely cause. If they all moved, look for something else.
Once a shift is confirmed, act on its cause. If new sources are cited, check whether you are represented in them accurately. If facts are wrong, correct them on your own pages and on the third-party pages being cited. If a competitor now appears, read what the engine cites for them; that is usually a clear map of what the model finds persuasive.
What to do first
Releases cannot be predicted in detail, so the highest-return work happens before one arrives. In rough order of return on effort:
- Fix your prompt set now, so there is a stable instrument to compare against when answers change.
- Run it daily per engine from a fresh session and keep the full answers. Noma, for instance, asks each tracked prompt once a day on ChatGPT, Gemini, Perplexity, Claude and Grok, stores the full answers, and lets you re-run any prompt on demand.
- Learn your normal day-to-day range per engine, so you can recognize a move outside it.
- Keep a short list of facts about your brand (pricing, plans, key capabilities) to check against every engine after any release.
- Decide in advance who reviews the results in the week after a release, and who acts on them.
The teams that handle releases calmly are not the ones who guessed what would change. They are the ones who can compare this week against last month, engine by engine, and see exactly what did.