AI Chatbot Outdated Answers: A CMS Problem, Not AI

If your chatbot learns your website from a nightly crawl, it can be wrong for up to 24 hours.

Not subtly wrong, either. Wrong in the way that sends a resident to an office that closed, or a student to an application window that shut on Friday.

Outdated chatbot answers usually get blamed on the model. Teams rewrite the system prompt, add a disclaimer, or start shopping for a different vendor. The model is rarely the problem. What's broken is the path between the content management system and the assistant's knowledge base, and that path is normally settled during procurement, before anyone frames it as an architecture decision.

There's a different pattern, and on Drupal in 2026 most of it is configuration rather than engineering. It's also not always worth building. This article spends as much space on that case as on the pipeline.

Why Does an AI Chatbot Give Outdated Answers When the Website Is Correct?

Because the assistant isn't reading your website. It's reading a copy of your website, made at some point in the past, and it has no way to know how old that copy is.

A crawl-fed assistant works on a schedule. The vendor fetches your pages, splits them into chunks (short passages, usually a few hundred words each), converts every chunk into a vector embedding (a numeric representation of what the passage means), and stores those embeddings in a vector index the assistant searches at question time. Between runs, that store is frozen. Your worst-case error window is the full crawl interval, and it's longer than that for pages the crawler reaches late in its queue.

That whole arrangement is retrieval-augmented generation, or RAG: the model doesn't answer from its training data; it answers from whatever passages the retrieval step hands it. Which means the accuracy of the answer is capped by the freshness of the index, no matter how good the model is.

Image

The pattern is already visible in the sector. A CalMatters investigation published in March 2026 found California community college chatbots giving students unclear or wrong answers about campus services. Districts running manually maintained question libraries return nothing useful when a question isn't in the library, and bots that read the college website inherit whatever is stale on it.

Retrieval quality matters more than most procurement processes assume. The Open Data Institute's CitizenQuery-UK benchmark tested six models, from frontier systems to small open-weight ones, against more than 22,000 synthetic citizen queries mapped to authoritative GOV.UK answers. The models were generally accurate, but some answers were significantly wrong, with extremely high variance and a long tail of unreliable responses.

The clearest failures came from the smaller open models. Feed any model that sets a knowledge base that is a day behind, and you have stacked a retrieval failure on top of a generation failure.

This is landing now because the deployments are landing now. Public sector AI use cases grew 50% from 2024 to 2026 in research commissioned by Twilio, and answering resident questions or resolving issues was among the three most-cited use cases. In the UK, the Government Digital Service is putting a retrieval-augmented chatbot into the GOV.UK app first and the wider website after.

What Is Publish-Time Sync, and How Is It Different From a Crawl?

Publish-time sync makes the CMS the event source. When an editor saves, publishes, unpublishes, or deletes content, the CMS re-chunks the affected item, generates embeddings, and writes them straight into the vector index the assistant retrieves from. The lag is seconds.

Three ways publish-time sync differs from a scheduled crawl:

• It's event-driven, not schedule-driven. Nothing waits for a queue position. The change and the index write are part of the same save.
• The CMS can see structure a crawler can't. Workflow state, language, which specific field changed, which other content references the changed item. A crawler sees rendered HTML and infers the rest.
• Deletions propagate. A crawler is good at discovering additions and slow at noticing removals. An event-driven pipeline treats an unpublish as a first-class event.

Scheduled crawl compared with publish-time sync

What Is a Tolerable Staleness Window?

Your tolerable staleness window is the longest a wrong answer can sit in the assistant before it costs someone money, a deadline, or an entitlement. It is a per-content-type number, not a site-wide one: a fee schedule and a history-of-the-department page do not have the same window. Setting it per content type is the decision this entire architecture turns on, because it is what tells you whether a nightly crawl is comfortably fine or a 24-hour gap is live exposure.

Which Drupal Modules Make Publish-Time Sync Work?

Publish-time sync on Drupal takes five pieces, and the one that actually triggers the sync is a checkbox.

• AI (core). The provider-agnostic layer that talks to OpenAI, Anthropic, Gemini, or a self-hosted model without leaking provider-specific code into everything else.
• AI Search. A Search API backend for vector databases. It splits content into chunks, converts each into an embedding, and stores them in a vector database (the datastore that holds those embeddings and answers similarity queries). This is what underpins the retrieval-augmented generation workflows behind assistants and chatbots. It began as a submodule of AI core in 2024 and is now its own project, which is the version to use in production because it is where active development happens.
• A vector database provider module. Milvus, Pinecone, Postgres with pgvector, SQLite, OpenSearch, Qdrant and Azure AI Search all have provider modules, at varying levels of maturity. For a government estate, this is a data residency decision before it is a performance one: pgvector or self-hosted Milvus keeps vectors inside your boundary; managed services move them outside it.
• Search API's indexing settings. This is the actual sync trigger. “Index items immediately” sends new and changed content to the index when the content is saved rather than waiting for the next cron run, and “Track changes in referenced entities” re-indexes content that references something that changed. The second matters more than it sounds: when a fee taxonomy term is updated, every page that references it is stale too.
• The assistant layer. The AI Assistants module's RAG action pulls relevant content out of the vector database during a chat, so the retrieval step reads the index your CMS just wrote to.

How Does Publish-Time Sync Behave on a Multilingual Site?

Each translation is tracked as its own item in the index, so an Arabic or Spanish update syncs on its own save instead of waiting for a crawler to rediscover a separate URL. On a bilingual government estate, that removes a failure mode which is hard to even detect: the English page corrected, the Arabic page still wrong, and no signal anywhere that the two have diverged.

What Changed in Drupal AI During 2026?

Two releases in the first half of 2026 moved publish-time sync from custom integration work to configuration.

The Drupal AI 1.3.x series (currently 1.3.10), first released 12 March 2026, added configurable guardrails that run before or after any AI request and observability that exports spans, traces, and metrics through OpenTelemetry (the open standard for application monitoring). It also added rerank as a provider operation type, but the Search API processor that applies it to search results doesn't arrive until 1.5.x.

Drupal AI 1.4.x (currently 1.4.5) followed on 26 May 2026 with global and streaming guardrails, plus prompt length limits aimed at denial-of-wallet attacks, where an attacker inflates your API bill with oversized prompts. The 1.4.x line also hardened the guardrails 1.3.x introduced for large-scale, public-facing deployments. Check the module's current release notes before treating any version detail here as fixed.

The practical effect is that accepting a vendor crawl is no longer the only defensible option, and the audit trail a public sector procurement asks for now lives in your own platform.

Vardot is a Drupal Diamond Certified Partner and a Gold Sponsor of the Drupal AI Initiative, which is why this article can be specific about what these modules do and what they do not do yet.

What Does Publish-Time Sync Cost to Run?

More than the build, which is the number that usually gets estimated. Every content change has to be re-chunked and re-embedded, and that fact drives all five of the running costs below.

• Embedding calls on every save. A long policy page is many chunks, and each chunk is an API call. A bulk edit across a content type can turn one editorial action into thousands. The real control is the same "Index items immediately" setting that makes sync fast: turn it off before a bulk edit and Search API queues the items for cron instead of embedding each one inline, working through them in batches at the cron limit you've set.
• A second production datastore. The vector database needs uptime, backups, upgrades, access control, and in the public sector, its own residency review.
• Re-embedding events. Change your chunking strategy or your embedding model and the entire corpus is re-embedded. Budget for that happening at least once.
• Someone owning retrieval quality. Chunk size, contextual fields, reranking thresholds. This is a standing job, not a launch task.
• Editor experience. Synchronous embedding on save adds latency to the editorial workflow. Queuing it removes the latency and reintroduces a small lag, which is a trade-off worth making deliberately rather than discovering in week three.

When Is a Nightly Crawl Actually the Right Choice?

Often enough that it deserves a real case rather than a caricature. Four conditions make the scheduled crawl the better architecture, and any one of them is reason enough to pause the pipeline.

1. Your Content Changes Slower Than Your Crawl Interval

If your answerable content turns over quarterly and none of it is time-bound, a nightly crawl is never meaningfully wrong. Building a real-time pipeline to eliminate a gap that nothing falls into is spending against a risk you don't carry.

2. The Answers People Ask About Don't Live in Your CMS

If most questions resolve to permit status, financial aid disbursement, or case progress, those answers sit in a permitting system, a student information system, or a benefits portal. Publish-time sync on the CMS fixes the smaller half of the problem and leaves the frustrating half untouched. Integrate the system of record first.

3. You Don't Have a Real Unpublish Practice

If pages get orphaned instead of retired, and nobody owns retiring them, faster sync propagates the same governance gap at higher speed. Fix the editorial workflow before you shorten the loop.

4. You Can't Staff a Second Production Datastore

A vector database nobody can back up, patch, or be paged about at 2 am is a risk you have moved rather than removed. If your team is already at capacity, the honest choice is a scheduled crawl on a shorter interval, plus better content hygiene.

Where We Land: Staleness Is a Retraction Problem

A 24-hour crawl interval is the symptom, not the failure. Our view is that the failure that actually damages a public sector assistant isn't the answer that arrived late. It's the answer that should have been withdrawn.

What Happens to Content You Unpublish or Delete?

In a crawl-fed assistant, usually nothing, for a while. Crawlers are good at finding new things: a new page gets linked from the homepage, the news feed, and the sitemap, and it enters the index on the next pass. Crawlers are bad at forgetting. A page pulled from navigation is still sitting at its URL, a superseded PDF is still where it was, and content deleted from the CMS keeps being served from the assistant's index until the crawler revisits and learns it's gone. Publish-time sync treats the unpublish itself as the event, so the passage leaves the index in the same operation.

Public sector content is asymmetric in exactly the wrong direction. The highest-risk statements you publish are the ones you later take back: the closed application window, the cancelled clinic, the fee schedule superseded on 1 July, the guidance note withdrawn after legal review. A crawler cannot see any of those events. It can only see their aftermath, eventually.

So the thing worth syncing at publish time isn't really the new content. It's the workflow state, which your CMS already tracks and a crawler has no access to. Reframed that way, you aren't building a faster crawler. You're handing the assistant the editorial state machine you already run.

That makes the dependency editorial rather than technical. Varbase, our enterprise Drupal distribution, ships the moderation and workflow baseline preconfigured and stays aligned with Drupal's official roadmap. But the configuration matters far less than whether your editors genuinely retire content rather than quietly unlinking it.

Five Questions That Tell You Whether to Build Publish-Time Sync

Five questions to answer against facts you already have:

• Can a wrong answer cost someone money, a deadline, or an entitlement? If yes, your tolerable staleness window is minutes, and no crawl interval buys you that.
• Do you retract content, or only add it? Count what you unpublished, archived, or superseded last quarter. Near zero means the crawl gap is an inconvenience. Routine means it's exposure.
• Is your CMS the system of record for what people actually ask? If the top questions resolve to another system, that integration outranks this one.
• Does your editorial workflow have a used, enforced unpublish state? If not, sync will surface your governance gap faster, not close it.
• Can you run a vector database inside your compliance boundary? Self-hosted keeps vectors under your control. Managed moves this from an architecture decision to a procurement one.

What the answers imply: yes to 1 and 2 means build it, and the cost is justified. No to both means a scheduled crawl is correct, and the budget belongs in content quality instead. Yes to 1 but no to 4 means fix the workflow first, because the pipeline will only accelerate what's already wrong.

Vardot doesn't sell a chatbot product and doesn't resell a vector database, so both architectures are the same category of work for us. That's why this article can say the cheaper one is frequently the right one, and it's the same advice we give on the platforms we build and operate for UNHCR, UNICEF, and Georgetown University, where a wrong answer has a consequence for an actual person.

If you're scoping a resident-facing or student-facing assistant this year, the decision worth getting right isn't which vendor. It's which parts of your content have to be current within minutes and which don't.

Our retrieval architecture review runs as the Align and Blueprint stages of the Vardot Delivery System: we map your content types against tolerable staleness windows and tell you which portion of the estate justifies the pipeline. Sometimes the finding is that none of it does.