Define the reason
Write what is wrong or unclear before writing the replacement copy.
A small method for website edits and later checks
Keep a record before you rewrite the page.
When a search answer is wrong or disappointing, the urge is to edit immediately and check again later. The missing piece is usually the record in between: what was wrong, what the page said before, what changed, and which later observation would actually be comparable.
A change log should answer:
What changed?
Why?
What else changed?
What did we observe later?
What still cannot be attributed?
Before the edit
Save enough context to make the next check meaningful
Start with the current page and the reason you are changing it. If the trigger is a factual error, record the error plainly and correct it. Do not leave an incorrect price, service condition or eligibility rule online just to preserve a cleaner experiment.
Then save the old wording or the location of a saved version. Record the question set you care about and the conditions under which you observed the search answer. A later result is easier to interpret when those conditions are visible rather than reconstructed from memory.
Keep a stable question set
Use the same important customer questions for the baseline and repeat checks.
Save the baseline
Keep the answer, date and conditions. If no answer appeared, record it as unassessed rather than forcing a zero into a score.
Document the actual edit
Save the old and new wording, change date and any related technical or design work.
Plan repeat checks
Choose dates or triggers that are practical, then keep the observation conditions as comparable as you reasonably can.
Separate observation from cause
Write what changed in the answers, then write what remains uncertain. Those are different sentences.
Worked example · fictional
Cedar Bookkeeping
Everything below is made up for illustration. This is not an experiment run by Alex and the timeline is not presented as real search data.
Old wording
“We support growing businesses with straightforward bookkeeping and year-round financial admin.”
New wording
“Cedar Bookkeeping provides monthly bookkeeping for UK limited companies and sole traders. It does not provide statutory audit services.”
Reason for the edit: the old paragraph did not state who was eligible for the service and left room for readers to infer a broader accounting offer than the business intended.
A
Baseline · 3 June · fictional
Save the page and answer conditions
Question set saved. UK market, English, fresh logged-out sessions, search enabled where available. One answer describes Cedar as “accounting and audit support”; another does not mention the business. The missing mention is recorded as unassessed for position, not as position zero.
B
Change · 5 June · fictional
Clarify service eligibility
The service paragraph is replaced with the new wording. No pricing, navigation or structured-data changes are planned in this fictional example.
C
Repeat check · 12 June · fictional
Run the same questions again
The audit wording disappears from the observed answer and Cedar is described as a bookkeeping service for small UK businesses. A second question still does not mention the business.
D
Conclusion · fictional
Record the observation without claiming the cause
The wording after the edit is more consistent with the page. That is an observation. It does not demonstrate that the page edit caused the answer to change; the answer system, retrieval set and other external sources may also have changed.
Keep the messy parts
Log simultaneous changes instead of pretending they did not happen
Correct facts first
If a customer-facing fact is wrong, fix it even if that ruins a neat before/after test. The change log exists to preserve the context around a necessary edit, not to delay the correction.
Name other interventions
If the same week includes a redesign, a pricing update, a new location page and a copy change, write all four down. Do not quietly treat the later answer as the result of the one change you happened to care about.
Keep absence separate from failure
If an answer was unavailable or the prompt was not run, mark the observation unassessed. When an answer appears, record search use separately: an answer without browsing can still contain a mention or claim you can assess. Missing evidence is not a measured zero.
Decide what the log is for
The useful outcome may be “the factual error is fixed and later answers look better, attribution unresolved.” That is less dramatic than a growth claim and much easier to defend.
For repeated observations, optional tracking views can reduce manual collection. Search visibility tracking and position tracking are relevant when you want to keep a prompt set under observation; they do not turn a before/after pattern into proof of causation.
Takeaway
Use the blank change-log worksheet
It keeps the page, reason, old/new wording, other changes, baseline, repeat checks, observations, uncertainty, decision and next action in one place.
Tools and reading
Useful when the log becomes a larger measurement job
I contribute to Beamtrace at Elfsight. The audit tool and research briefings linked here are also my independent work.
Beamtrace
- Track search visibility — monitor whether a business appears in recorded AI answers for a chosen prompt set.
- Track positions in answers — compare where a brand appears for repeated prompts over time.
- Compare competitors — review which alternatives appear in the same answer contexts.
- Follow brand mentions — inspect when and how a brand appears across monitored answers.
- Investigate cited sources — review the pages and references associated with monitored answers.
- Check how the business is described — review reputation themes and monitored descriptions rather than one isolated answer.
Independent audit and research
- Check whether a page can be read — inspect access and retrieval conditions separately from the content-change log.
- Browse the research archive — practical briefings that retain the original study sources and limitations.
- Read about query language and market selection — a reminder to record language and market when comparing answer observations. Original paper.
- Read about body-copy rewrites and answer visibility — a benchmark suggesting that copy-only rewriting can have different effects across retrieval, reranking and citation stages. It is benchmark evidence, not live-platform proof. Original paper.
- Read about search use and publisher referrals — a seven-day experiment showing why answer visibility and website traffic should be logged as different outcomes. Original paper.
Record changes visitors can see
Customer reviews
- Show customer reviews with JustReviews — display existing reviews on your website. Log a change to the sources, layout or selection so the next comparison has a clear baseline.
Visitor questions
- Answer questions from your website content — a chatbot can use your pages, files and FAQs. When that content changes, repeat a relevant question and keep the answer with the change record.