TL;DR: IT and software engineering teams don’t adopt GenAI just because they have access to it. They adopt it once they learn where it fits, which engineering habits still apply, which ones quietly break, and why using AI isn’t a threat to their livelihood. Adopting it responsibly means spending tokens deliberately instead of maxing them out, testing probabilistic output differently than deterministic code, keeping rules that must always hold out of the model’s hands, shipping only code the team understands well enough to maintain, and configuring the tool’s instruction files and hooks on purpose. AI tools easily help engineers produce more code. But how much of that code actually ships and how much of it really shouldn’t? More importantly, what do you do when you know the difference and are ready to move up to the next level of agentic productivity?
Engineers Get the Tools First and the Training Last
IT and engineering teams are usually the first in an organization to get GenAI. They get coding assistants, API keys, and an unqualified expectation to triple their previous output with no ill side effects. (For those who were part of an enterprise Agile “transformation,” this may sound familiar… and not all corporate cultures have evolved since, despite a few gigabytes of files “capturing” lessons learned from the retrospectives.)
The tools have certainly arrived. About 90% of technology professionals now use AI at work, and over 80% believe it has increased their productivity, yet about 30% of developers still report little to no trust in AI-generated code (from the 2025 DORA findings cited in Balancing AI tensions: Moving from AI adoption to effective SDLC use). Wide use combined with low trust means teams use the tools without knowing how far to rely on them.
Engineers learn prompting quickly, wire up an API call in an afternoon, and have a demo by Friday (or lunch, depending on ambition, skill, and risk tolerance). What they don’t pick up from the “Quick Start” guides and YouTube promos is the judgment underneath: what the model should own, how to test something that doesn’t return the same answer twice, and how to tell a real improvement from a lucky run. Those gaps don’t show up in the demo. They show up in production, in the token bill, or in a post-production incident audit.
DORA’s 2025 research describes AI’s primary role as an amplifier that magnifies an organization’s existing strengths and weaknesses, with the greatest returns coming from improving the underlying organizational system rather than from the tools alone (see DORA | State of AI-assisted Software Development 2025…and yes, I know that “2025” is considered dated, but rigorously reported data still lags behind Age of AI evolution rates). A team with weak testing discipline gets weak testing at higher speed.
The training Anthropic provides for its Claude Certified Architect – Professional (CCAR-P) certification (similar public reference: Four role-based certifications for the people who put Claude to work for customers) frames the missing skills as four “AI Fluency competencies”: Delegation, Description, Discernment, and Diligence (similar public reference: AI Fluency: Framework & Foundations).
- Delegation means deciding “not just whether Claude can do it, but whether Claude should own it” (CCAR-P training, Module 1, Claude Platform & Solution Design, “Delegation: deciding what Claude is trusted to own”).
- Description means telling the model precisely what you want: the scope of the task, the format of the output, and the constraints that bound it (CCAR-P training, Module 1, Claude Platform & Solution Design, “Description: the 4D competency applied to prompt design”).
- Discernment means judging model output “rather than accepting it at face value” (CCAR-P training, Module 2, Enterprise Integration & Production, “Discernment: judging the output”).
- Diligence means taking responsibility for verifying and vouching for AI output, including whether the author can explain what shipped (CCAR-P training, Module 5, Team Enablement and Operational Productivity, “Improving developer workflows with AI tooling”).
None of them is a prompting skill, and none of them shows up in the typical enablement session.
One team I worked with began merging story code three times faster after receiving their Claude licenses, yet actual delivered velocity stayed flat. The problem was that “LGTM” (looks good to me, if you’re a non-dev reading this) results did look good by the old standards. Variables and functions had clear names, the code was well-organized and easily read, and unit tests passed with flying colors. But seven times out of ten the results were bounced back by QA for missing some of the acceptance criteria, and two of the three that passed QA were sent back by product because of a problem that would have been recognized by a developer not feeling pressured to triple their output or caught by a QA tester who hadn’t had their workload suddenly quadrupled (don’t forget, the failed code had to be re-tested).
I coached the team in using /plan mode. Planning is a feature that Claude introduced in February of 2025. While Claude has evolved beyond the need to pre-plan its work, the human prompters have not. Plan mode forces the developer to think through what they are asking the coding agent to do. That, coupled with the /grill-me skill, slowed down the time from the start of implementing a story to the creation of a pull request, but tripled the number of stories that passed QA and product review the first time through, resulting in a true doubling of team velocity.
More Code Isn’t More Software
The tools work, and the evidence on that is strong. A study of more than 500,000 GitHub developers found that autocomplete, interactive coding agents, and autonomous coding agents raised commits by 30%, 180%, and 240% respectively, as cumulative effects of adopting each generation (from the Abstract of Writing Code vs. Shipping Code: Productivity Effects Across Generations of AI Coding Tools).
The gains shrink sharply on the way to production. That same 240% increase in commits “falls to 80% for the number of projects, and to 30% for actual releases” (Abstract). Interactive agents such as Claude Code raised lines of code by 958% and pull requests by 86%, but releases by only 20% (Section 1, Introduction). After adopting interactive agents, developers added 7.0 times as many lines of code but deleted 12.2 times as many, created 79% more branches, and saw the share of new repositories abandoned after their first month climb from 60% to 69% between 2021 and 2026 (Section 6.2, Mechanisms Underlying Attenuation). The paper describes the binding constraint as “shifting from writing code to reviewing, integrating, and distributing it” (Section 8, Conclusion).
DORA sees the same trade-off from the team’s side. It found that higher AI adoption is associated with increases in both delivery throughput and delivery instability, and it describes a “verification tax,” where time saved writing code gets spent auditing it. It also flags a gap between prototype and production: AI speeds up prototyping, but integrating the result into production can cancel out those gains (see Balancing AI tensions).
The NBER paper adds a detail that matters for training. The growing gains from interactive agents were “driven primarily by improvements in the frontier models that power them, rather than by within-developer learning over time” (Writing Code vs. Shipping Code, Section 5.4, Results: Dynamic Treatment Effects). Engineers don’t need much training to write more code with these tools; the models handle that. Training has to target the review, integration, testing, and design decisions that decide whether the extra code ships.
Good Engineering Instincts, Wrong Kind of System
Most of the costly mistakes in GenAI adoption come from experienced engineers applying sound instincts to a system that behaves differently from the ones they were trained on. Simplify the design. Cut integrations. Trust what passes in testing. Those habits work for deterministic code. With a probabilistic model, they can quietly backfire.
One scenario from Anthropic’s architect certification training shows the problem clearly. A team had a rule that any claim over £5,000 goes to a senior adjuster, enforced by a SQL check. The architect proposed letting the model extract the amount and route the claim instead, which meant one less integration and a cleaner design. Three months later, an audit found 41 misrouted claims. The amounts had been written in sentences like “damages estimated around five thousand pounds,” and the model read them as estimates rather than threshold triggers (CCAR-P training, Module 1, Claude Platform & Solution Design, “A scoping call, transcribed”).
Every engineer knows the urge to remove a dependency. What this team hadn’t been taught is that a deterministic rule, one that has to be right every time, doesn’t belong in a system that’s right most of the time. Nobody wrote test cases for the routing either, because it was treated as something the model would just handle. The SQL check would have been tested as a matter of course.
The same mistake shows up in how teams configure their coding assistants. Anthropic’s Claude Code documentation is explicit that Claude treats CLAUDE.md files “as context, not enforced configuration,” and that blocking an action takes a hook (see How Claude remembers your project). Allowing or blocking specific tool calls belongs in the permissions and hooks in settings.json (see Explore the .claude directory). Writing “never deploy to production without approval” into a Markdown file is a reminder. The deployment platform still has to require the approval. Teams that haven’t been trained on that distinction treat a policy written in prose as if it were an enforced control.
Reaching for the Agent Because It Sounds Like Progress
Engineers like flexible designs, and agents feel like the most flexible option available. Another Anthropic training scenario is a 90-day retrospective from a team that picked an agent for exactly that reason. One team member said they’d added so many guardrail tools by month two that they’d “basically rewritten the workflow inside the agent loop, minus the logging.” When they finally mined the traces, the real paths through the system “fell into only four shapes. Four.” Their conclusion was that a router and four chains would have “saved ourselves six months” (CCAR-P training, Module 1, “When the team wanted flexibility and got non-determinism”).
That’s six months of engineering time spent because nobody on the team had been trained to tell a task that feels open-ended from one that actually needs an agent. The training puts the test plainly: if the steps could have been written in code, a workflow would have done the job (similar public reference: Building effective agents).
“Use the Best Model” Feels Like Diligence
When engineers don’t know how to choose a model tier, they choose the safest-sounding one. In one Anthropic training scenario, a team ran every call on Opus. Ninety days after launch, monthly cost was running at seven times the original forecast, and median latency was 2.3 seconds against an 800-millisecond target, with no change in customer satisfaction (CCAR-P training, Module 1, “When defaulting to Opus everywhere produced a 7× cost overrun”).
Once the team built an eval set, it moved classification to Haiku and summarization to Sonnet, and kept Opus only for the final response. Monthly cost dropped 71%, and median latency fell to 940 milliseconds (CCAR-P training, Module 1, “90 days after launch”). The training’s verdict: “The absence of a deliberate choice is not neutral.”
Defaulting to the top tier is one form of a broader habit sometimes called “token maxxing”: the most capable model, the largest context, extended thinking switched on everywhere, and retries until something sticks, as if spending more were the same as being careful. It feels responsible, and it isn’t. Every token spent on a step that didn’t need it is budget that a step which did need it no longer has, and the bill gives no hint of which was which until someone measures.
Cheapest isn’t automatically right either. The FinOps Foundation’s guidance is to define quality criteria for each task, test candidate models, and choose the least expensive one that meets the bar. It tracks over-provisioning with a “model fit” KPI, the share of requests sent to a more capable model than the task needs, and gives an example where 12,000 of 60,000 requests, or 20%, were over-provisioned (see FinOps for AI Overview). The same guidance measures cost per successful outcome: total cost of all runs divided by the runs that met the quality bar, as in its example of $1,200 / 8,000 = $0.15. A cheap model that needs three tries isn’t cheap.
Engineers already profile code before optimizing it. Model-tier selection is the same discipline, measuring per step and choosing per step, but teams only apply it once someone shows them it’s an engineering decision and not a vendor setting.
Testing Habits That Don’t Transfer
Testing is where untrained teams struggle most, because their testing instincts were built for code that returns the same output every time.
Another Anthropic training scenario (a composite of a common pattern) describes a team that tested a new system prompt on 50 sessions against 50 on the old version. The new prompt scored a 68% task success rate against 62%, so the team shipped it. Two weeks later it had settled at 61%, and “the apparent 6-point gain had disappeared” (CCAR-P training, Module 2, Enterprise Integration & Production, “The 50-session winner that wasn’t”). For comparison, the same training works through a properly sized test in which detecting a 5-point improvement from a 70% baseline “with 80% power and 5% significance requires approximately 1,500 sessions per group” (CCAR-P training, Module 2, “Production readiness builder”).
The same gap shows up in load and failure testing. In one post-launch review, “the long documents in the tail were consuming 80% of the total token spend,” and a 529 error at peak on day three took the whole workflow down because nobody had tested what happens when the call fails (CCAR-P training, Module 2, “The demo cost profile that became the production bill”). Uneven input distributions and fallback paths are standard engineering concerns. Teams new to GenAI tend to set them aside, as if the rules had changed.
Anthropic’s training also gives engineers a familiar way in: set eval thresholds the way you’d set acceptance criteria, based on the business requirement, and “Don’t just choose what your first prototype happens to achieve” (CCAR-P training, Module 2, “Evals as acceptance criteria”; similar public reference: Define success criteria and build evaluations). Framed that way, evals look like something engineers already know how to do.
Feeling Faster Isn’t Being Faster
Teams also misread their own productivity. In METR’s randomized study from early 2025, 16 experienced open-source developers took 19% longer to finish tasks on their own repositories when they were allowed to use AI tools (see Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity).
Things have moved since then. METR’s late-2025 follow-up covered 57 developers, 143 repositories, and more than 800 tasks. It measured an 18% speedup for developers returning from the original study (confidence interval -38% to +9%) and a 4% speedup for new developers (-15% to +9%) (see We are Changing our Developer Productivity Experiment Design). METR treats those figures as a likely lower bound and calls its own data “very weak evidence” for the size of the change. Part of the reason is that 30% to 50% of surveyed developers said they skipped tasks they didn’t want to do without AI. METR also notes that self-reported speedups are unreliable.
Two things follow for adoption. The gains are real and growing, but they’re smaller and less certain than they feel, and the only way to know what a team is actually getting is to measure it. Training should also teach engineers to recognize when doing the work by hand is faster. Knowing when not to use the tool is part of being fluent with it.
Review Is Now the Bottleneck, and It’s Easy to Rubber-Stamp
If writing code is no longer the constraint, review is. The NBER paper notes that the bottleneck should be tighter inside companies than in open source, “where shipping code involves review and approval processes that open-source developers largely avoid,” and it cites enterprise data identifying code review as the binding constraint (Writing Code vs. Shipping Code, Section 8, Conclusion). DORA describes the same tension: authors can generate large changes quickly, and the review burden lands on their colleagues (see Balancing AI tensions).
Under that pressure, review tends to give way. An Anthropic training scenario describes a reviewer facing 400 items a day with only the model’s output and an approve button: “After the first hour I have to just hit approve to keep up with the pace” (CCAR-P training, Module 3, Responsible AI, Safety, and Risk for Architects, “When routing everything to review makes review meaningless”).
The fix is an engineering one: route items by stakes, and give reviewers the inputs and the reason each item was flagged. DORA’s recommendations point the same way. It advises AI-driven checks that reach authors while they’re writing, splitting large AI-generated changes into small batches that are easier to review and test, investing in test automation, and considering context-aware review agents to enforce standards (see Balancing AI tensions). Some of that discipline can be packaged into the tooling itself. The community-built Popquiz skill for Claude Code, created by scoobydrew83, targets what its author calls “comprehension debt”: code that ships without anyone understanding it. Before agent-generated or unfamiliar code merges, it walks the reviewer through an explanation of the change and ends with a written quiz that has to be answered before an approval is allowed. It even fails reviews that approve with rubber-stamp phrasing. It triggers on the phrases reviewers use when they’re in a hurry, such as “LGTM,” “ship it,” and “approve this PR.” It’s a third-party skill, so it should go through the same review as any other shared artifact before the team adopts it. Review that approves everything is not oversight, whoever the reviewer is.
Speed Today, Skills Tomorrow
There’s a quieter cost that training has to cover too. DORA points out that AI can help engineers learn unfamiliar areas quickly, but it can also lead to shallow understanding and skills that erode. Its recommendations include pairing junior engineers with senior mentors to review AI-driven design decisions, and deliberately keeping some complex components manual (see Balancing AI tensions).
This is the part of training that has nothing to do with the tool. An engineer who never had to work through a hard design problem won’t be able to review the model’s attempt at one. The review bottleneck above depends on reviewers whose judgment came from doing the work themselves. It’s also a maintenance problem. Code nobody on the team understands is code nobody else can safely change, debug during an outage, or hand to the next hire. Shipping only what someone can explain is part of adopting AI responsibly, because comprehension skipped today gets paid for later by whoever inherits the code.
Lumpy Adoption Never Becomes a Team Practice
Hand out licenses without a plan and adoption doesn’t spread evenly. A few engineers go all in, most use the tool occasionally, and some don’t use it at all. Anthropic’s architect certification training calls this “lumpy adoption”: a few developers use AI tooling heavily while the rest barely touch it, so the team never realizes the tool’s real gain and the practice never becomes standard (CCAR-P training, Module 5, Team Enablement and Operational Productivity, “Improving developer workflows with AI tooling”).
The unevenness costs more than unused licenses. The heavy users generate the large changes that DORA warns about, and the light users end up reviewing them without the experience to judge them. Estimates drift because half the team works at one pace and half at another. Each heavy user also builds a private setup that nobody else can reproduce. The same training module names a related failure: teams that never get past using Claude as a question-and-answer box, because nobody set up the more valuable workflows. Its summary is blunt: “Providing a team with access is not adoption.”
The remedy the training describes is a rollout that’s planned rather than left to early adopters. In its example, a 200-person engineering org rolling out Claude Code across four departments starts with one champion per department. Each champion gets two weeks to convert a real workflow, such as a code-review assist or a test-generation step, then runs a 45-minute session for a first batch of five peers. By the broad rollout, every department has a working example, a local expert, and a shared CLAUDE.md the champion has already tuned. The training notes that the same rollout done as a single mass email would likely have produced a wave of confused first-time prompts and a quiet return to old habits (CCAR-P training, Module 5, “Configuring Claude tooling and environments for teams”).
Champions are the training plan. They turn one person’s fluency into the team’s baseline, which is the same job done by the shared Markdown files described below.
Hours Saved Aren’t Dollars Saved
The ROI case for GenAI usually multiplies hours saved by salary and calls the result savings. Finance will eventually see through that. If a team finishes maintenance work sooner, the organization gains capacity, but payroll doesn’t change unless spending actually changes. A credible business case keeps capacity that gets reassigned to valuable work separate from spending that’s actually avoided, such as reduced contractor demand, and from direct cost cuts.
That distinction is where many programs fail. As cited by SmartDev, McKinsey’s 2025 survey found that 88% of organizations use AI in at least one function, but only about 39% report measurable profit impact at the enterprise level (see Real Cost of Generative AI: What SMEs Actually Pay). The same article points to McKinsey’s finding that workflow redesign was strongly associated with measurable financial impact, and it notes that ongoing costs often exceed initial development for most enterprise AI initiatives. Licenses and tokens are the visible part of the bill. Integration, workflow redesign, evaluation, governance, support, and training make up the rest.
The FinOps Foundation’s data points to ownership as the difference. In its State of Tokenomics survey, proving value or ROI was the most commonly named challenge, at 43%, and organizations with defined ownership of AI economics were 3.7 times more likely to show value to the CFO (see FinOps for AI Overview). Its practical guidance is to tag experimentation separately from production, map each API key to one team, application, and cost center, and set a functional baseline alongside the cost baseline. Its infrastructure guidance adds that observed costs should be checked against the business case’s original assumptions (see Choosing an AI Approach and Infrastructure Strategy).
The training also isn’t only for engineers. The FinOps Foundation recommends teaching FinOps basics to the people building with AI and AI cost basics to finance, procurement, and leadership (see FinOps for AI Overview). A manager who can’t tell activity from realized benefit will approve the wrong expansion.
The Markdown Files Nobody Told the Team About
Hand a team Claude Code licenses without training and they get a capable assistant, but not much of what makes it work well for a team. Claude Code reads a set of Markdown files that hold project instructions, scoped engineering rules, reusable procedures, specialist subagents, and memory that carries across sessions (see Explore the .claude directory). Each file’s name, location, and frontmatter decide whether Claude sees it, when it sees it, and what it does with it. Nothing about those rules is obvious from the file names.
Untrained teams tend to fall into one of three patterns.
They don’t use the files at all. Claude still works, but it has to infer conventions from the code and the conversation. Every engineer re-explains the build command, the architecture boundaries, and what “done” means, in every session. That’s the prompt-engineering equivalent of tribal knowledge.
Everyone writes their own. One engineer runs /init, commits the result without reviewing it, and moves on. Another keeps a private CLAUDE.local.md with a better test workflow that nobody else knows about. A third copies a skill from a blog post. Each developer’s sessions now behave differently, and the team’s best practices live on one person’s laptop. A workflow isn’t reproducible if it quietly depends on one senior developer’s private files. Anthropic’s architect certification training describes the alternative as a shared team baseline (a project CLAUDE.md, an agreed set of tools, and a permission posture) that every member starts from, instead of personal setups that drift apart (CCAR-P training, Module 5, “Configuring Claude tooling and environments for teams”).
They write them wrong. This one is the most expensive, because the team believes guidance is active when it isn’t, or that guidance is enforced when it’s only advice. The common mistakes are all predictable from how the files load:
- Bloated instructions. A 900-line
CLAUDE.md uses up context on every session and lowers how closely Claude follows it. Anthropic recommends keeping each CLAUDE.md under 200 lines (see How Claude remembers your project).
- Stale instructions. An old test command or a reference to a decommissioned service is worse than no instruction, because it supplies the wrong assumption every time.
- Contradictions. Root and nested
CLAUDE.md files are concatenated into context rather than overriding each other, and when two instructions conflict, the Claude Code documentation warns that Claude may pick one arbitrarily (see How Claude remembers your project). A billing service file that contradicts the root file doesn’t win; it just adds a coin flip.
- Broken path rules. A rule whose path glob matches nothing never activates. A rule whose YAML frontmatter doesn’t parse loads as if it had no
paths at all (see How Claude remembers your project), so guidance meant only for Terraform files gets applied to every task.
- Skill name collisions. When names match, enterprise skills take precedence over personal ones, and personal skills over project ones (see Extend Claude with skills). A developer’s personal
release-review skill silently replaces the team’s version.
- Unreviewed memory. Claude’s auto memory is machine-local and isn’t shared across machines or cloud environments (see How Claude remembers your project). It’s one developer’s working notes, not approved policy. If a mistaken correction gets saved, it carries into later sessions until someone finds and removes it.
- Policy written in prose. “Never touch production” in a Markdown file is a reminder, not a control.
None of this shows up in a demo. It shows up as inconsistent output between developers, verification claims that weren’t actually checked, and senior engineers wondering why the assistant “keeps forgetting” a rule that was never loaded.
What Deliberate Looks Like
The difference between a weak instruction file and a useful one is easy to see side by side. Here’s a CLAUDE.md that an untrained team typically produces:
# Project Notes
Write clean, high-quality code. Follow best practices.
Be careful with the database.
Run tests with: npm run test:all (old script, removed in 2024)
Use the payments-v1 service for billing. (decommissioned)
API key for staging: sk-staging-4f9a...
I prefer tabs and verbose explanations.
## Everything about Terraform, Kubernetes, React, the billing
## service, our 2019 architecture review, and the full style guide
...(700 more lines)...
Almost every line has a problem. The vague standards (“clean,” “careful”) can’t be tested. The commands and services are out of date. There’s a secret in a committed file, a personal preference written as a team rule, and enough unrelated material to bury the parts that matter.
Here’s the version a trained team writes:
# Project instructions
## Commands
- Install dependencies: npm ci
- Check types: npm run typecheck
- Run unit tests: npm test
- Run lint checks: npm run lint
## Architecture
- HTTP handlers call application services.
- Application services access persistence through repository interfaces.
- Do not introduce an alternative persistence framework without approval.
## Change verification
- Add or update tests for changed behavior.
- Report the checks executed and their actual results.
- If a check cannot run, state the reason; do not claim it passed.
## Specialized guidance
- Infrastructure conventions are in infra/CLAUDE.md.
- Use the migration-review skill for database migration reviews.
It’s short, every instruction can be checked, and it points to specialized guidance instead of including it. The “Change verification” section deals directly with the Discernment problem from earlier: it tells Claude to report what it actually ran, not to claim success.
Scoped rules show the same contrast. A poorly written one:
---
paths: infra/**/*.tf
- "**/*"
---
Follow Terraform best practices.
The YAML is malformed, so the rule loads unconditionally. Even if it parsed, **/* matches every file, and “best practices” tells Claude nothing. A well-written one:
---
paths:
- "infra/**/*.tf"
---
# Infrastructure conventions
- Use the repository's approved module versions.
- Preserve mandatory resource tags.
- Explain any change affecting public network exposure.
- Report validation results and identify checks not executed.
It loads only when Claude touches Terraform files, the infrastructure team owns it, and every line is something a reviewer can verify.
Skills follow the same pattern. A skill described as “helps with databases” either never triggers or triggers on everything. A skill with a precise description, explicit steps, and disable-model-invocation: true runs only when an engineer asks for it:
---
name: migration-review
description: Review a database migration for compatibility and operational risk.
disable-model-invocation: true
---
# Migration review
1. Identify schema changes and affected application versions.
2. Check backward compatibility during rolling deployment.
3. Assess locking, data volume, and runtime risks.
4. Review recovery and rollback assumptions.
5. Identify missing tests and operational checks.
6. Produce findings with evidence and unresolved questions.
Do not execute the migration.
That last line is still guidance, not enforcement. The database permissions are what actually prevent the migration from running.
Hooks are the piece that closes that gap inside the tool itself. A hook is a script configured in settings.json that runs before or after Claude uses a tool (see Explore the .claude directory). Because the hook runs every time, it doesn’t depend on Claude remembering an instruction. Anthropic’s documentation is direct about the distinction: to block an action regardless of what Claude decides, use a PreToolUse hook rather than a line in CLAUDE.md (see How Claude remembers your project).
The responsible split is simple to state and easy to get wrong without training. Put preferences and conventions in Markdown. Put anything that must always or never happen in a hook: blocking edits to lockfiles, generated code, or production configuration; refusing destructive shell commands; running the formatter and the relevant tests after every edit. Hook scripts are code that runs on every developer’s machine, so they belong in version control and go through the same review as everything else. A team that knows when to use a hook stops arguing about whether Claude “followed the rules” and starts proving it.
A Practical Repository Layout
Put together, a team using these files deliberately ends up with a repository that looks something like this (an illustrative layout, not a required structure):
repository/
├── CLAUDE.md
├── AGENTS.md # Optional shared multi-tool instructions
├── CLAUDE.local.md # Gitignored
├── .gitignore
├── .claude/
│ ├── settings.json # Shared configuration; not Markdown
│ ├── settings.local.json # Private configuration
│ ├── rules/
│ │ ├── testing.md
│ │ ├── api-design.md
│ │ └── terraform.md
│ ├── skills/
│ │ ├── migration-review/
│ │ │ ├── SKILL.md
│ │ │ └── reference.md
│ │ └── incident-analysis/
│ │ └── SKILL.md
│ ├── agents/
│ │ └── security-reviewer.md
│ └── output-styles/
│ └── engineering-review.md
├── services/
│ └── billing/
│ └── CLAUDE.md
├── infra/
│ └── CLAUDE.md
└── docs/
├── architecture.md
├── adr/
└── runbooks/
The structure shows who owns each piece. The root CLAUDE.md holds what every change needs. Each service and infrastructure team maintains its own nested file and scoped rules. Reusable procedures live in skills, and specialist reviewers are agents with restricted tools. Personal preferences stay in gitignored files, and permissions live in settings.json, where they’re enforced. Ordinary documentation stays in docs/. A runbook doesn’t become a standing instruction just because it’s a Markdown file; Claude reads it when it’s imported, referenced, or encountered during the work.
The payoff comes from how the team manages these files, as much as from the files themselves. Shared instructions, rules, skills, and agents get committed, reviewed through pull requests (with CODEOWNERS where it helps), tested against representative tasks, and rolled back when a change makes things worse. The files also need upkeep, and Claude Code can help with that: running /doctor prompt-audit has Claude check instruction files for instructions written for older models, references to files or commands that no longer exist, and files that contradict each other (see How Claude remembers your project). When an engineer’s private memory captures something useful, it gets promoted into a reviewed rule or skill so the whole team benefits. That’s where cooperation turns into capability. What one engineer figured out about the billing service, or how to review a risky migration, becomes how every developer’s sessions work, including new hires on their first day.
Managed this way, the files support every outcome this post has covered. Quality improves because standards and verification steps are in place at the point of work. Productivity improves because procedures get reused instead of rebuilt from prompts. Cost goes down because scoped rules and on-demand skills keep irrelevant material out of context. And the training investment holds up, because what people learned is written down, versioned, and shared rather than leaving with whoever learned it.
What Engineering Teams Need to Be Trained On
Enablement for IT and engineering teams usually stops at tool features and prompting tips. The NBER findings suggest that’s the part teams need least help with. The cases above point to a different curriculum:
- Delegation: deciding which logic stays deterministic, which work the model owns, and where people stay in charge.
- Pattern selection: the difference between an augmented call, a workflow, and an agent, and when each one earns its added complexity.
- Model and cost decisions: choosing model tiers step by step, backed by evals, measuring cost per successful outcome, modeling costs at production volume with realistic inputs, and recognizing token maxing for what it is.
- Testing probabilistic systems: evals as acceptance criteria, experiments big enough to trust, failure-path testing, and checking that generated tests actually exercise the requirements.
- Review and integration: review flows sized to the volume AI now generates and given enough context to catch problems, small batches, early feedback, and judgment about which experiments are worth carrying through to release.
- Guidance vs. enforcement: knowing what belongs in
CLAUDE.md, rules, and skills, and what has to be enforced through hooks, permissions, CI, and deployment gates, including how, when, and why to write a hook.
- Skill preservation: mentoring, and deliberately manual work, so reviewers keep the judgment that review depends on.
- Selective non-use: recognizing when doing the work by hand is faster and measuring actual results instead of trusting how fast it felt.
Format matters as much as content. Training works best when it’s role-specific and hands-on with representative work, not a generic prompting demo, and when it’s treated as part of the operating model rather than a one-time onboarding event. It also works best when it’s rolled out through champions and small batches rather than a single all-hands switch-on, so fluency spreads deliberately instead of pooling with early adopters. DORA’s advice to measure impact, not output, sets the bar: lines of code accepted is a volume metric, not a result (see Balancing AI tensions). In practice that means a sequence: establish baselines for delivery speed, quality, and cost; pick bounded use cases with clear acceptance criteria; train against real work; build validation and spending controls into the existing workflow; pilot and measure, including verification effort and rework; then scale, revise, or stop each use case based on the results.
Not every engineer needs an architect certification. The ones making these design calls need more than a kickoff session, and the rest of the team needs enough fluency to notice when a “simpler” design is really a riskier one.
Training Is How Adoption Actually Happens
Engineering teams don’t fail at GenAI because they lack talent. They fail when they’re handed new tools and left to apply old habits. The tools already boost output: 240% more commits. Only 30% more releases come out the other end. The difference is in review, integration, testing, and design decisions, which is the same place the 41 misrouted claims, the six months spent on an agent, and the 7× overrun came from. That’s what adopting AI responsibly looks like in practice: spending tokens where they earn their keep, testing probabilistic output for what it is, keeping must-hold rules in deterministic code and hooks, shipping only what the team can explain and maintain, and managing the files that shape the tool’s behavior like any other code. Training aimed there is what turns more code into more shipped software, and it’s the line item that makes the license spend pay back.
If you found this interesting, please share.
