TL;DR: Your GenAI output is failing because your local workspace is a disaster. If your desktop is a dumping ground, your enterprise data lake is guaranteed to be a swamp. Stop blaming the model, establish a strict folder taxonomy, and kill the bad data habits before they scale.

For my regular reader, you know I can’t resist a pun, and the initial research note for this post was “Timely topic title: Take the Tax out of Taxonomy”. You also know I digress, so I thought I would get it out of the way at the start. Done. Moving on to the next level…

You are paying a massive hallucination tax. You bought a premium AI subscription or deployed a desktop agent. You pointed it at a project directory full of deprecated drafts, unstructured notes, “versioned” files, maybe even some sample code. Now the AI is confidently generating output based on requirements from three years ago, and maybe Wednesday’s lunch order.

The AI users assume a better foundation model or highly complex prompt engineering will fix output inaccuracy. They will not. According to the research paper A Comprehensive Taxonomy of Hallucinations in Large Language Models, hallucinations are not merely a bug, but a theoretically inevitable feature of computable LLMs, irrespective of architecture or training.

You cannot patch out hallucinations with a clever system prompt. You have to restrict their oxygen.

Generative AI operates entirely on the context it is fed. When you open a workspace (or upload a zip file, or point it at SharePoint), the model uses the folder structure to understand relationships. It assumes every file in the provided directory is equally valid, current, and relevant.

To get faster, accurate output, you must adopt a standardized, hierarchical folder taxonomy. This is not a housekeeping chore. It is a strict data contract for your AI. The academic consensus supports this structural approach. As outlined in A Systematic Framework for Enterprise Knowledge Retrieval, transforming a static blob of data into a navigable, context-rich knowledge architecture significantly improves model accuracy and reduces retrieval latency.

The Prep Station Metaphor

Think of an LLM as a highly skilled line cook with zero short-term memory. If you ask the cook to make an omelet, but point them to a kitchen counter where the fresh eggs are mixed in a pile with old receipts, bleach, and rotten produce, the resulting meal will be toxic.

You have to prep the station before you ask for the work.

This requires changing how you manage your local environments. You must segment your files and organize your folders explicitly by client, project, or (and sometimes “and”) specific activity. When the AI opens that specific folder, the taxonomy forces it to focus strictly on the given task.

The Micro-Macro Data Contagion

Local file structures often mirror enterprise data architecture. If your team’s shared drive is a chaotic dumping ground of nested, unnamed folders, your enterprise data lake is likely more of a content swamp.

Organizations often fund massive, top-down enterprise data transformation projects. They deploy tools to wrangle petabytes of unstructured data. Consultants are brought in to describe how it should be done, walk you through clean up, and leave you with a perfectly indexed wiki on how to maintain it.

The reason other organizations don’t do this kind of clean up? Aside from the few that don’t need it, the rest have someone recruited from an organization that did need it, then did it…then did it again a few years later. At least some had the excuse of acquisitions as a cause. The rest just forgot to make being organized part of the organization’s culture.

The report What Is Data Taxonomy? Use Cases & Best Practices points out that taxonomy programs do not fail because the classification structure was wrong. They fail because nobody owned it after launch, or the controlled vocabulary was written for data engineers rather than the business users who needed to adopt it. A taxonomy that nobody actively owns becomes outdated within twelve months.

If you build a pristine enterprise knowledge graph but your teams still save raw client notes to a local desktop folder named “Misc”, your clean data architecture will erode. Bad habits always defeat good infrastructure.

Start locally. Expand globally. Treat your team’s shared folder as the training ground for enterprise AI.

Here is the implementation baseline for engineering a reliable folder taxonomy.

1. Force Local Discipline: The guide Document Taxonomy Simplified notes a critical reality: AI can read full text, but without consistent indexing and classification, it has a harder time understanding which documents are current or relevant for a specific question. Humans must define the taxonomy. Organizations that rely solely on AI risk amplifying bad data.
2. Build a Strict Domain Hierarchy: Segment folders strictly by project and lifecycle status. Your AI should never have read access to a “Drafts” directory when you are asking it to write production documentation.
3. Establish the Data Contract: Metadata like document type, owner, client, date, and status tells AI not just what a document says, but how it should be used. This context improves AI ranking and reduces irrelevant hits that happen to share keywords.
4. Separate Human and AI-Native Formats: Segment your directories into files meant for humans and files meant for the AI. Lean towards using markdown files, text files, and CSV files for AI consumption. Keep heavy, formatting-rich files in a separate reference folder that the AI does not scan unless explicitly commanded.
5. Isolate Contextual Boundaries: Open-ended prompts can generate answers that blend multiple disciplines or outdated content. When your library is indexed by department, project, and lifecycle stage, AI can narrow its focus and answer questions within the right slice of your information.

You’ll note that there is a lack of solid reference examples of good taxonomies. This is, again, related to data cleanliness being driven by culture. The same taxonomy may or may not work for another organization. But a solid taxonomy based on how the organization thinks and processes can easily be maintained through training, communication, and the occasional reminder (preferably automated).

The ROI of a strict folder taxonomy is immediate. Output precision goes up. Token waste goes down.

If your AI is only as reliable as the context it receives, your unstructured file storage is an active threat to your workflow. Build the hierarchy locally. Clean up the directories.

Credit to Dylan Davis‘s video 5 Changes That Make ChatGPT & Claude 10x Better for sparking this research.

© Scott S. Nelson

It’s not because they are more innovative.

There is a common myth in tech that smaller, nimbler companies always win the adoption race. But with Generative AI, we are seeing the opposite. While startups are still “tinkering,” enterprises are productionizing. According to recent data shared by Nathaniel Whittemore (a.k.a. NLW, host of the AI Daily Brief & CEO, Super.ai) at the at the AI Engineer World’s Fair, full production deployment of AI agents in billion-dollar enterprises jumped from 11% to 42% in just the first three quarters of 2024 [03:15]. Why? It comes down to a brutal reality of economics, automation, and what I call the “2% vs. 20% ROI Gap.”

AI is Automation (Just Less Consistent)

Many AI enthusiasts argue that automation isn’t AI. That’s true in the sense that not all fruits are apples, but all apples are fruits. AI is automation. The primary difference? Traditional automation is deterministic (consistent); AI is probabilistic (less consistent, but more capable). Smaller companies are already masters of traditional automation because they have to be. They use it to survive with fewer people. But for a massive corporation, the “low-hanging fruit” of basic automation hasn’t even been picked yet. This creates a massive opportunity for Information Gain—the ability to apply AI to “messy” processes that were previously too expensive to automate.

The Math: The 2% vs. 20% Rule

The biggest “moat” for big business isn’t their data or their brand—it’s their Scale ROI. Because a large company doesn’t need significantly more resources than a small company to build a single AI agent or workflow, the math of deployment looks very different:

• For the Small Business: To pay for the initial R&D and resource overhead, a new AI tool might need to deliver a 20% improvement in efficiency just to break even.
• For the Enterprise: Because they are applying that tool across thousands of employees or millions of transactions, a mere 2% improvement creates an ROI that justifies the entire department.

Furthermore, as NLW points out, these large organizations are moving toward Systemic Adoption [17:00]. They aren’t just doing “spot experiments”; they are thinking cross-disciplinarily. They can afford to go slower, spend more on high-quality resources, and leverage volume discounts that drive their production costs down even further.

The “Risk Reduction” Transformation

Interestingly, while most companies start with “Time Savings” (the default ROI metric), the real “transformational” wins are happening elsewhere. NLW’s study found that Risk Reduction—while the least common primary goal—was the most likely to result in “transformational” impact [14:59]. Large companies have massive back-office, compliance, and risk burdens. AI can handle the sheer volume of these tasks in ways a human team never could [15:17]. This is a “moat” that small businesses simply don’t have to worry about yet.

The Cycle: From Moat to Commodity

This scale is the moat that gives big business a temporary advantage. But here is the irony: The more they use that advantage, the faster the moat shrinks. As enterprises productionize these efficiencies, they effectively commoditize them. What cost a Fortune 500 company $1 million to develop today will be a $20/month SaaS plugin for a small business tomorrow. We are in a cycle of:

1. Hype: Everyone talks.
2. Value: Big companies productionize at scale.
3. Cheap: The tech becomes a commodity.
4. Reverse Leverage: Small, disruptive players use those same cheap tools to move faster and out-innovate the giants.

The giants are winning the production race today, but they are also building the very tools that the next generation of “disruptors” will use to tear them down.

© Scott S. Nelson