TL;DR: Enterprise AI runs on enterprise data, and that includes the unstructured data in the form of documentation that is poorly curated. No taxonomy, revisions sitting next to finals with no way to tell which one is current or still relevant. If data is the DNA of modern business, documentation is the dominant chromosome that can make the difference between robust health and questionable viability.
I post a lot about the value of architecture and training in AI adoption, and share posts by people focused on the nuances of prompting because these things will make a difference. Data cleanliness is a topic I usually just point people toward other people’s thinking on because my days focus more on how data moves and evolves than on structuring and managing it. As a long-time generalist, I have a deep appreciation of the added value specialists bring to the process. Documentation is a different kind of data, with its own unique headaches. I’ve harped on some of those issues for years: why self-documenting code doesn’t produce a self-documenting solution (From Agile to Fragile in 60 Sprints), why nothing gets read that never gets written down (If It Is Not Written Down It Does Not Exist), and why a taxonomy nobody maintains is worse than no taxonomy at all (Failure to plan communications is communicating a plan to miscommunicate, one that I wrote at the dawn of the current age of AI without realizing how it would soon become even more important).
Then it occurred to me that documentation and data cleanliness are the same conversation, and I hadn’t heard many people say so directly. Documentation is data, and not just any part of it. It’s the chromosome that expresses the rest of your enterprise data, the one that decides whether everything built on top of it turns out healthy. It’s also usually the part in the worst shape. That’s the case I want to make here. (Confession: the connection came from something I read that mentioned it in passing, and I never noted the source. Whoever you are, kudos.)
AI needs three things from your data, in order:
- Access.
- Understanding.
- The ability to apply what it understands to the specific context of whatever someone just asked it.
Skip a step and the most expertly crafted prompt or the best planned architecture in the world still only gets you an expensive autocomplete with a confident and flattering tone, because both of them are working with whatever access and understanding they’ve been handed.
There’s a lot written about the mechanics of step one: RAG, vector stores, and running SharePoint exports through Pandoc before they ever reach a prompt, since raw Word and PDF files carry a lot of baggage a model doesn’t need to see. Step two has gotten a lot less attention. Understanding requires that all of the documentation your AI is referencing provides meaning and consistent messaging, and the usual disarray of enterprise documentation doesn’t.
AI is a (New) Good Reason to Clean Up Your Documentation
This was already a problem in 2022, back when the audience for the complaint was a project manager, not a language model. Templates that auto-update their “last modified” date every time the file gets saved, whether or not the content actually changed, so the date stops meaning anything (Replace Auto Dates in Templates). It gets worse at the platform level. SharePoint and Teams will happily stamp a document “Modified” the moment someone opens it, whether they changed a single character or not, because the file was never stored with the “open as read-only” flag set. Nobody sets it, because almost nobody thinks about it, and now your most trustworthy-looking piece of metadata is lying to you and to anything reading it downstream.
Then there’s version chaos stacked on top of date chaos. A SharePoint draft with one clear owner on paper, quietly edited by people who didn’t know that, discovered only after someone had to re-verify the entire document line by line (Recovering Previous Versions from SharePoint). That gripe about config files requiring “reading documentation, which is only read less than it is written” is sixteen years old (Dynamic Log Location for log4j). That was a joke about developers skipping the manual. It reads differently now that the thing skipping the manual is a model that can’t lean over and ask a coworker what the doc meant to say.
None of that is a new problem. What’s new is that AI has zero tolerance for it. A human can walk into a shared drive, eyeball three files named some variation of “Process_Final_v2_ACTUAL_FINAL,” and guess correctly which one to trust, because they have context: they remember the meeting, they know who owns the process, they can just ask. AI doesn’t get that fallback. It reads what’s in front of it and treats every file as equally authoritative, including the wrong one.
If This Sounds Familiar…
Turns out I’m not the only one who noticed. Other people are seeing the same problem from angles most enterprise practitioners rarely get access to. Amit Shivpuja, who runs data and AI enablement at Walmart, wrote in Forbes that he watched a fully governed, well-modeled AI program produce inconsistent results anyway, and traced it to what he calls the missing documentation layer: the context that should have been captured during requirements, design, and testing, but instead lived in a Slack thread or in someone’s head (The Hidden Barrier To Enterprise AI: The Missing Documentation Layer). His diagnosis lines up with a decade of watching the same pattern play out: humans compensate for missing documentation with tribal knowledge. AI can’t.
The gap between AI investment and AI payoff backs this up at scale. 79% of organizations report real challenges getting AI to deliver, a double-digit jump from the year before, even as most are raising budgets to feed it (WRITER). Separately, only 32% of organizations report sustained business impact from AI despite 86% of the C-suite increasing investment, according to an Accenture survey (Forbes). Nobody breaks that spend down into prompt engineering versus architecture versus training, but odds are good most of it lands in exactly those three buckets. It’s not usually the prompts, the architecture, or the training that’s failing. It’s the context underneath all three.
How to Get Started
None of that makes training, architecture, or prompt engineering optional. They’re not, and treating them as afterthoughts would be its own kind of mistake. Training your people is worth the time and the awkward learning curve that comes with it, every time. Architecture done right is what lets a foundation hold up for years while the technology running on top of it changes every few weeks, so it’s worth building solid instead of patching forever. And prompt engineering isn’t dead, whatever this month’s headlines are claiming. Knowing how to ask well is still what gets the new, flashier capabilities to actually do what you meant instead of what you typed.
But the potential of all that value depends on how well it works with your enterprise data. We’re past the point of needing to prove AI will benefit the business. We’re all now in the throes of how it will benefit the business, and that how stays severely limited until you put your data in order. Here are a few tips to get you started.
Make documentation part of done, not an afterthought to done. Shivpuja’s Forbes piece gets this right: a story or a feature isn’t finished until the meaning, the rules, and the assumptions behind it are captured somewhere a model can find them. That’s a process change, not a tooling purchase.
Default to read-only. Force intent for edits. If a document is finished, save it that way, and make someone actively choose to reopen it for editing. This has been true since 2022, and the fix hasn’t gotten any harder to implement. Just more expensive to keep skipping.
Give it a taxonomy and a living Read Me, and actually maintain both. This isn’t new advice. Generative AI just raised the stakes on it: organize things so someone new can find their way around without a tour guide, pin an explanation of that structure somewhere obvious, and keep it current as the team and the work change (Failure to plan communications). A taxonomy that was accurate in 2022 and hasn’t been touched since is worse than no taxonomy at all, because it still looks trustworthy.
Attach context to the asset, not to a folder that might get reorganized next quarter. Documentation living next to the process or dataset it describes survives longer than documentation living in a wiki page someone has to remember exists.
Convert for the machine, not just for the human. Pandoc, or whatever your equivalent is, exists to clear that baggage out before it hits a vector store. Word and PDF files are full of formatting decisions that made sense to a human editor and mean nothing to an LLM. Fifteen minutes of conversion saves a RAG pipeline from tripping over someone’s decade-old formatting habits. This is architecture work too, for what it’s worth, just the boring kind that doesn’t show up in a vendor pitch.
Set a review cadence, and mean it. One knowledge-management vendor’s own market analysis cites a 2025 Gartner study putting the number at 60% of internal knowledge articles going stale within six months, with only 14% of teams auditing content on any real schedule (source, with a grain of salt: it’s a page selling a fix for the exact problem it’s describing, but the shape of the number matches what shows up across most enterprise environments). A tool that flags stale content is nice. A team that actually looks at the flag on a schedule is the part that works. Buying a self-updating knowledge base without building the habit of using it just moves the swamp to a nicer-looking pond.
If Data Debt were a Thing…
There’s a term for this that architects already understand: technical debt. Every shortcut taken to ship faster accrues interest, and the bill always comes due, usually at the worst possible time and for more than the original shortcut would have cost to do right.
Documentation disarray runs on the same math. Call it data debt. It accrues every time a taxonomy goes unmaintained, a Read Me goes stale, or a “final” draft ships without anyone reconciling it against the other four drafts sitting next to it. None of that shows up on a balance sheet, so nobody budgets against it. But your prompts, your architecture, and your training program are all paying interest on it anyway, every time they inherit whatever your documentation can actually support.
Pay it down early and it stays cheap: a taxonomy tightened now, a template’s date field fixed before it propagates through a hundred more copies. Let it ride, and the interest compounds. More conflicting drafts pile up. More tribal knowledge walks out the door with the people who had it. More AI output gets built on documentation that was already lying to you, and the eventual fix means untangling years of it instead of an afternoon of it. Ignore it long enough and it doesn’t just get expensive, it bankrupts the whole initiative: the AI program that never delivered, the budget that got pulled, the “we tried AI and it didn’t work” verdict that was actually a documentation problem wearing an AI costume.
Fix the documentation first. It’s not sexy, nobody gets interviewed on a podcast for cleaning up a taxonomy, and it’s exactly the debt payment that keeps the rest of the investment solvent.
© Scott S. Nelson
