Architecture

SaaS Has No Box

Scott S NelsonLeave a comment

TL;DR: SaaS doesn’t remove implementation work, it relocates it: from servers and patches to configuration, nested declarations, and complex weaving of separate-but-related features to deliver enterprise capabilities over custom code written against vendor interfaces. Demos can be deceiving, even if not intentionally so. More often it’s the expectation that everything you need is available “out of the box” that makes SaaS look like a magic bullet to enterprise product owners instead of just another tool that takes training and skill to build with and effort to maintain.

When There Is No Box

SaaS can be the perfect architectural choice for an organization to focus on business value instead of unique technology capabilities. You don’t buy SaaS, you lease it, and that distinction is its value proposition. The vendor manages technology refresh cycles as part of the subscription  through continuous updates instead of IT trying to keep up on endless patch cycles and observability-driven improvements.  Traditional software, whether licensed on-prem or custom built (even “cloud native”), is a distraction for some organizations, and an outright burden for others, especially when the maintenance isn’t budgeted for during the design and build phases. Organizations that make the wrong choice end up in endless cycles of replacing systems through CapEx as a result of under-funded OpEx. These cycles are as disruptive to the users as they are to the support teams, both having to reskill while losing they “muscle memory” built up with the previous system. SaaS is often the solution for these organizations, but not always. Technical debt can written declaratively as well as programmatically, especially when enterprise SaaS is confused with enterprise software.

The gray area between SaaS and software is what it takes to turn a collection of separate-but-related features into an enterprise capability: level of effort, support, and everything a vendor demo skips over. Demos are built for the business buyer with a checkbook, not the developer with a deadline. A single capability, shown on its own, looks like a checkbox. Implementing it by wiring together ten other features, each documented as if the others didn’t exist is a different job entirely, and it’s the job nobody demos.

Bottom line: Nothing is “out of the box” with SaaS. SaaS has no box. It has a tenant, a contract, a configuration console, a few license tiers, an identity integration, and eventually a backlog. No box.

The wrong assumption is that enterprise SaaS is a pre-built application, so implementation is mostly a matter of switching it on (an understandable confusion because some non-enterprise SaaS does work that way). A more accurate model: enterprise SaaS is a product (and, with successful vendors, a collection of acquired, competing, products) sitting on a platform shared by thousands of other customers. The product supplies standard capabilities. The platform supplies features to configure, automate, extend, secure, integrate, monitor (sometimes) and govern them. Both matter, and they’re different implementation problems.

While there’s no box, SaaS still delivers many of the same benefits standard software has always had over custom builds, and a small company running a fairly standardized process can get close to production-ready with limited setup (warning: no organization with an IT staff ever has a limited setup). Those are the things business users are so used to that they forget to count them: security, role management, standard integrations, audit logging, backup and disaster recovery, single sign-on, and standard compliance (support) built in rather than bolted on. A department adds users, roles, approvals, reports, and a few local variations, and configuration becomes real work. An enterprise running the same product across business units, regulations, and integrations turns it into a configurable platform that needs architecture and governance, whether or not anyone wrote a line of code. Again, the confusion is mistaking standard features as “out of the box”. They are parameters necessary to use a well designed platform to protect both the customer and the vendor.

Cloud Has Changed the Meaning of OOTB

“Out of the box” used to mean a shrink-wrapped disk that installed a working application on a server you owned. Cloud didn’t kill the phrase, it’s just no more literal than dialing a phone or pasting text. The concepts are still the same, though sometimes mis-applied because the meaning has been lost in translation.

To a vendor, OOTB usually means the feature exists in the licensed edition, fits the product’s intended model, and can be enabled or demonstrated without a separate application. That’s an honest intent. It’s also incomplete. The platform including the building blocks is not the same as an organization putting the thing into production next week with its own data, users, policies, integrations, processes, and exceptions.

To a business stakeholder, OOTB often means something closer to done: complete, supported, included in the purchase, ready with minimal effort, sensible about exceptions to processes that “everyone” follows. Fair enough for a consumer app, but it falls apart fast in enterprise SaaS, because “normal” is in the eye of the beholder (or buyer), and the vendor is working from a normal defined by the lowest common denominator across their entire customer base.

To an engineer or an experienced administrator, it means the capability exists, there’s probably a supported configuration path, and there may also be three licensing constraints, a technical prerequisite nobody mentioned, and a limitation that won’t surface until a user asks a perfectly reasonable question in production.

The reason those three points of view keep colliding is simple: a feature demo is a planned, orchestrated, and practiced event to showcase what a product does. An enterprise capability is the business secret sauce baked into solution that relies on twice as many features and sometimes held together by bubblegum and baling wire. One is maybe out of the box. The other never saw a box, and the instructions were in a different language for a different purpose and required a lot of creativity that often goes unrecognized.

Enterprise SaaS is less a finished solution than a pile of Legos with an incomplete picture and no instructions (but sometimes a blog post, which isn’t always going to mention the complexity required or the dozens of failed attempts that came before it). The pieces are good. Plenty fit together well. But the pieces being in the box, metaphorically, since SaaS still doesn’t have one, doesn’t tell you what to build, whether the model survives the next upgrade, or who fixes it when someone changes a foundational rule six months from now.

Take a routing requirement. In a demo: a new request comes in, gets assigned by region, triggers a notification, and that’s it. In production: region comes from more than one system, the request touches multiple accounts, the assigned team depends on product line and contract status, some records are role-restricted, the notification has to respect local hours, and the downstream system is occasionally down. The feature is the same one from the demo. The demo was not OOTB, and neither is the solution the business needs. If you’re lucky, you might get details of how to duplicate the demo. You will not get instructions on how to make the version you actually need. And what seems the hardest for the non-technical folks to understand is that it is much more effort to create the configuration without direction than it is to develop with custom code. The custom code has no multi-tenant constraints.

OOTB is a capability statement. It is not an implementation estimate, an operational guarantee, or a substitute for architecture. And it isn’t available with SaaS (though some enterprising companies build add-ons that are, but they are usually even more constrained than the SaaS itself).

The Simpler the Configuration, the More Complex the Code It Hides

The difference between abstraction and misdirection is that the magician will often tell you he is misdirecting you. To identify abstraction, you need to think like an engineer, not business user.

Abstraction is not a bad thing. The steering wheel of a car is an abstraction from the complex systems necessary to change the orientation of the front wheels, and even developers who can read binary would still rather use method that abstracts the library of components that abstract the bits away from streaming data from one location to another.

But you can make better decisions about how to steer the car or which streaming library to use if you understand that it is an abstraction and actions have deeper consequences.

Configuration and declarative programming are that same kind of abstraction, a steering wheel for the platform instead of a wrench in its engine. An out-of-the-box capability is a feature the platform supplies through exactly that abstraction. It may work as-is for the standard case, but it usually needs configuration, permissions, data setup, licensing, and training before it’s useful, and it may not match the current process exactly.

Configuration means shaping standard behavior through supported settings: roles, page layouts, field validation, reports, routing rules, approval structures, notifications, integration endpoints. Once it affects production behavior, it’s implementation work, full stop, with requirements, design review, test cases, deployment control, documentation, and an owner. Calling it “not development” is technically defensible in the narrowest sense. It’s also how a team ends up with 437 rules, three administrators who understand half of them, and no confident answer for what happens when a customer’s status changes (see Agile is not Ready, Fire, Aim for more on this).

Declarative development, workflow tools, rules engines, visual process designers, formula fields, event triggers, is a platform-provided way to define logic without conventional code. It’s still code. The platform abstracts the syntax away from the person assembling it. It doesn’t abstract away the consequences. A few boxes connected by arrows can represent a substantial amount of execution logic, error handling, and dependency chains the calm canvas never shows, until production has other opinions, usually at an inconvenient hour. Stack enough simple, individually reasonable configurations and the system ends up simple nowhere and complex everywhere at once (see 5 Considerations About Low-Code/No-Code).

That’s the mechanism that makes SaaS scale, not a flaw to engineer away. A platform flexible enough to fit a hospital system, a bank, and a mid-size distributor without three separate codebases has to carry that flexibility as complexity somewhere. The vendor pushes it into configuration and declarative tooling instead of custom code. That doesn’t make the complexity optional, it just changes who manages it and what tools they get to do it with.

Custom development, code within supported extension mechanisms, custom applications on the platform, middleware, integration services, isn’t automatically a failure of SaaS. A five-line extension with source control, automated tests, logging, and a documented owner can be safer than a heroic visual flow nobody wants to touch because the canvas now resembles an airport terminal map. The safer default is buy-before-build: standard components first, AppExchange next, custom code only when nothing else genuinely fits (see Enterprise Salesforce Team Recipe for a longer rant on this aspect). Either way, the distinction that matters isn’t code versus clicks. It’s whether the result can be understood, tested, secured, upgraded, and supported by whoever inherits it. Microsoft’s own guidance on low-code governance is telling here: it talks about guiding “both professional and citizen developers who build custom applications for business needs” (see What Is Low-Code Governance and Why Is It Necessary?). A low-code solution is still a custom application, and it needs roles, controls, and oversight regardless of how few characters got typed to build it.

“Can Be Done” != “Is Supported”

This is worth repeating more than once: almost anything is technically possible with enough extensions, workarounds, APIs, and questionable judgment. Whether it’s supported is a different question entirely (insert Jeff Goldblum Jurassic Park meme here).

Vendor-supported means the mechanism is documented, stood behind, and preserved through normal releases. Whether your own teams can support it comes down to whether they can diagnose it, test it, change it, and hand it off. And it only stays sustainable if the whole thing remains economically and operationally reasonable as the platform, the process, and the team keep changing around it. A demo answering yes to “can it be done” tells you nothing about the other three.

Undocumented API behavior, browser automation standing in for a real integration, a customization that quietly modifies vendor-delivered files, a workflow built on a feature the vendor has already announced it’s retiring, a marketplace add-on with one maintainer and an optimistic roadmap: all technically working. A documented, versioned extension point is a different commitment than any of that.

The cost and risk of “custom” depends less on the label and more on the stability of what it’s built on. A stable foundation has published APIs with deprecation notices, documented extension points, sandboxes, deployment and rollback mechanisms, and a realistic talent pool that can maintain it. An unstable one has undocumented internals, screen scraping, direct database access outside vendor guidance, fragile third-party dependencies, and configuration too tangled to inventory. A small customization on a documented extension point can have a modest lifecycle cost. A small customization on undocumented behavior can turn into a recurring upgrade project. The line count, or the number of boxes in a flow, tells you less than the stability of the surface it’s built on.

Standard declarative features aren’t automatically exempt from this either. Salesforce ended support for Workflow Rules and Process Builder after December 31, 2025. Existing automation can keep running. Salesforce’s own documentation puts the distinction plainly: “Your active Workflow Rules and Process Builder processes continue to run as they do today,” followed immediately by the limitation that “customer support is not available, and bugs will not be fixed” (from Workflow Rules and Process Builder End of Support). Salesforce’s own stated reasoning for the consolidation was that spreading automation across multiple tools “hampers both maintenance and innovation” (see Go with the Flow: What’s Happening with Workflow Rules and Process Builder?), which is a reasonable product decision. It doesn’t change who pays for inventorying, testing, migrating, and retraining on what got built on the old feature. “We didn’t write code” was never a lifecycle strategy.

Which is where architects earn their keep, not by vetoing low-code or routing every request through a custom-dev project, but by distinguishing product configuration from business-critical custom behavior, spotting dependency and lifecycle risk before it’s load-bearing, and asking what happens when the vendor retires the thing three years from now. The job isn’t making the solution more complicated. It’s making the complexity visible early enough that the organization can choose where to carry it.

The Ocean View Capability

Every enterprise SaaS platform has Ocean View capabilities. They look excellent from the balcony. The demo is clean, the data is cooperative, the presenter knows the route, and the integration has already been arranged. The value is obvious.

Shiny things can be dazzling, but don’t mistake them for a trick. A ninety-second demo has to compress a hard problem into something presentable, and that compression is what quietly removes the cases that would make the problem visible.

Then implementation starts, and someone asks about identity federation, field-level security, migration errors, audit history, a regional exception, an approval delegation, an API failure, and the department that’s been running its actual process out of a shared mailbox since 2014. The ocean’s still there. It’s just farther from the lobby than the brochure implied.

Vendor blogs are good for learning what a feature is intended to do. They’re written to explain a capability, not to catalog every limitation, so they lean on clean prepared data, follow the happy path, and rarely dwell on edition-specific licensing, exception handling, or recovery scenarios. A vendor blog can show that a feature exists. It rarely proves the feature fits your operating model.

Demos are orchestrated on purpose, not deceptively, just deliberately: prepared dataset, existing roles and permissions, the expected route through the workflow, integrations already wired, no duplicate records, no outages, no data-quality problems, no last-minute policy exception. Call it theater, not fraud. It’s designed to explain a capability quickly, not reproduce a customer’s production environment. The mistake is mistaking a successful demo for a fit assessment.

The fix is straightforward, if less exciting than watching a polished walkthrough: ask the vendor to run the organization’s actual process with representative data, not the generic version. Require the exception path, not just the happy one. Test the real identity, role, and approval requirements, and find out what happens when an integration or automation fails. Get the licensing tiers, support boundaries, and lifecycle commitments in writing. The best single question in the room is usually, “Show me what happens when it doesn’t work.”

Gregor Hohpe, (the definitive author on architecture, IMO) states the reality experienced architects live in more plainly than almost anyone writing about this, goes further in The Quest for Low-Code. His starting point: there’s no magic in software. Every code-reduction mechanism implies a trade-off, and the evaluation job is naming which one you’re accepting. From there his checklist gets concrete: is what’s left actually low cognitive load, or just low line-count with the complexity relocated into a jumble of custom syntax buried in quotes inside a YAML file? How does the solution evolve, is there version control, can you roll back, does it hold up past the scale of the demo? And what happens on the unhappy path, do the error messages help or just leak the tool’s internals? His three vendor exercises put that last question to the test directly: introduce an error on purpose and read what the message actually tells you, leave the room and have the vendor break the thing and debug it live, or take the keyboard yourself and see how much hand-holding it takes to build something real.

Configure Deliberately, Extend Carefully

SaaS is frequently the right call. It reduces unnecessary ownership, accelerates access to mature capability, and spares an organization from rebuilding commodity software it has no business rebuilding. Standard functionality should be the default. Sometimes the business process should adapt to a proven product model instead of demanding a bespoke replica of every legacy exception it’s accumulated out of habit.

None of that means every use case gets covered for free. The same flexibility that lets the platform serve wildly different businesses is what makes your specific business’s edge cases someone’s implementation problem, not a checkbox someone forgot to include. And custom code isn’t automatically the harder path. A clean extension against a documented interface can take less time and less risk than the half-dozen declarative features it would take to fake the same behavior through configuration. The label on the technique, code or clicks, was never a reliable way to estimate the work.

Watch for the case study that shows someone getting a complex capability working by wiring together three or four platform features. It’s real, but it’s also incomplete: what made it into the write-up is the wiring, not the weeks of trial, error, and platform-specific knowledge that came before it worked cleanly enough to demo. A capability that took a seasoned admin forty hours to get right the third time doesn’t become a ten-minute task because the polished version only takes ten minutes to watch someone build.

And the decision to go with SaaS is not where the architecture ends. A workflow assembled in a visual designer can still affect revenue, customer commitments, regulatory obligations, and a support team’s weekend. An integration configured through a menu can still fail at 2:00 AM. A field renamed without documentation can still turn next year’s migration into archaeology.

Use the standard product where it genuinely fits. Configure it with discipline, and price that discipline honestly: time, effort, and cost scale with how far the requirement sits from the common case, because the common case is what SaaS was built to solve cheaply. The more unique the business need, the more of that cost the organization carries itself, whether it ends up in configuration, declarative automation, or custom code. Treat declarative automation as software, because it is. Extend stable, documented foundations when the requirement earns it, code included. And the next time someone says a capability is available out of the box, ask who it’s actually out of the box for.

In enterprise SaaS, nothing is truly out of the box. The capability exists. What it costs to make that capability yours, in time, effort, and risk, is an architecture decision shaped by how far your need sits from the common case, not a vendor claim and not a line item copied from someone else’s case study.

If you found this interesting, please share.

Meet Deadlines and Manage Technical Debt with AI-Assisted Architecture

Scott S NelsonLeave a comment

tl;dr: New platform. Deadline. The instinct is to move fast and clean it up later. That’s where technical debt is born. A well-constructed Claude project, loaded with curated platform documentation and queried with the experience to know what to ask and how to evaluate the responses, tactically compresses the ramp-up without sacrificing strategic design principles.

The Sharp Fork in the Road

Every architect and engineering lead who has given a project to deliver on a new platform or with new technology under deadline pressure knows this fork. Pushing for proper preparation can get you marked (ironically) as a risk from the leadership perspective. Plowing forward using old techniques without understanding the new nuances keeps you up at night…either knowing you are missing something up front, or fixing what you didn’t know during the final death-march phase of a waterfall project that just happens to use Kanban boards, daily stand ups, and sprint ceremonies.

One path: move fast. Learn just enough to ship. Ask support when you hit a wall. Request exceptions when you hit limits. Get it working and tell yourself you’ll revisit the architecture when there’s more time. (There is never more time.) What you build in that mode becomes the foundation everything else is built on, and the cost of fixing it compounds with every sprint.

The other path: slow down. Read the documentation properly. Understand the platform’s constraints before you design around them. Make the right call the first time. This is correct and often impractical. Deadlines are real. The platform is new. The documentation is dense. The team is waiting.

The Contentstack project that prompted this post took the first path and ran into a SaaS governance constraint that happens to be measured recursively. The first time it was hit, the response was typical for teams working with a new SaaS vendor and release date that was set before the first line of code was written: Ask for an exception. Which was granted, hit again and raised again. Fortunately, the third time it happened, an experienced vendor support manager recommended reviewing best practices to avoid the issue. And an experienced architect was on the receiving end of that suggestion, one who had previously dealt with a Salesforce solution that went down three months after launch from relying on similar exceptions.

This post is not about Contentstack architecture. It is about the challenge many teams face with balancing target dates and defensive design decisions, and a tool set to apply in order to keep from tipping too far in either direction.

Claude as a Platform Research Partner

Giving Claude access to a curated set of platform documentation and then working interactively to explore solutions is not a replacement for architectural experience. It is an accelerant for it. It is also not a way to do away with architects or the inclusion of design tasks at the feature or story level. It is how to fulfill the expectation that AI can provide ROI immediately when applied by experienced technologists.

These distinctions matter. It’s never about “ask Claude what to do” (because if you need to ask “what” you won’t know how to ask “how” when the time comes). It is “I understand how systems like this behave, I know which constraints are likely to compound, and I need to move through the analysis faster than I could alone.” Experienced architects and engineers bring the judgment: familiarity with how content models fail at scale, how schema resolvers typically handle recursion, how vendor-imposed limits usually reflect real constraints in the underlying system. Claude brings the recall, the scripting, the cross-referencing, and the tireless patience for the kind of recursive schema analysis that would take a senior engineer the better part of a day.

For those that follow my posts you know that I will often describe theoretical solutions backed by a combination of personal experience where they would have worked linked to examples from others who demonstrated that they work. In this case the experience came before the theory, working backwards from a result where I noticed the process while documenting the solution (because, hey, that is what architects do after they solve something).

The working example was with a Contentstack implementation. It took one focused 2-hour session to identify an obscure root cause, define a strategic solution, discover other areas to apply the same solution, and identify where the solution would cause more harm than good. A second 30-minute session was applied after the first round of refactoring to validate the impact and prioritize the remaining effort. Before Generative AI, this would have been several days of effort that would not have been attempted until the risk was realized in production delay.

The Project is the Architecture

Before a single question gets asked, the project has to be built. This is not setup overhead. This is the work.

A blank Claude chat window and a well-constructed project will give you very different results on the same question. The difference is not the AI. It is the knowledge boundary, the taxonomy, the instructions, and the accumulated session output. Strip those away and you have a general-purpose assistant guessing at context. Keep them and you have something that behaves like a senior researcher who has been on the project for months.

What goes in the project folder:

Downloaded documentation as markdown files, not links. Links go stale, require fetches, and introduce latency. Pull the platform docs that matter, save them as markdown, put them in the folder. For Contentstack: the Global Fields limitations page, the Content Modeling Best Practices guide, the Custom Fields documentation. Not every page in the docs. The ones relevant to the work. Knowing which ones matter is the first place architectural experience shows up.

Actual data from the platform. In this case, exported stack JSON. Claude can read it directly in the sandbox, run scripts against it, and cross-reference findings against the loaded documentation in the same session. That combination of curated docs and live data is what makes the diagnosis precise instead of speculative.

Session summaries. After each working session, have Claude produce a structured summary: the original problem, the conclusions, the evidence, the next steps. That file becomes the cold-start document for the next session. You don’t re-explain the context. You hand Claude the prior session’s output and continue. The knowledge compounds.

At some point (again, much of this requires human intuition gained through real-world experience), have Claude work with you to turn the summaries into a skill scoped to the specific platform, technology, or tool so that when they are in context these lessons learned will be applied automatically going forward.

The Taxonomy Is Not an Afterthought

Separate downloaded reference content from working session output. Nest folders by topic. /reference/, /sessions/, /data/ serve different purposes and should live in different places. This is not pedantry. It is how you make the project instructions work correctly, and how you find things six weeks later without rebuilding context from scratch.

If the platform has extensive documentation, don’t try to enumerate allowed URLs in the project instructions directly. Create a reference-urls.md, or per-topic files like contentstack-docs-urls.md, with an annotated, categorized list of approved sources. Claude works from the list. You maintain the list. It stays current and searchable.

The discipline compounds the same way the session summaries do. A well-organized project from session three makes session fifteen faster than session one.

The Project Instructions Are the Rules of Engagement

The instructions define how Claude behaves inside this knowledge space. Three things they need to do:

Challenge assumptions. If a question implies something not supported by the loaded documentation, say so. Don’t fill gaps with plausible-sounding answers. The most dangerous thing a research assistant can do is answer confidently on insufficient evidence. This instruction eliminates a whole category of hallucination risk before it starts.

Point out mistakes. If the framing of a problem is wrong, say so. This is the instruction most people skip and then complain about later. You want an assistant that pushes back, not one that validates your bad hypothesis and helps you build a case on sand.

Limit web searches to specific URLs. Unconstrained web search in a technical investigation introduces noise: outdated content, inconsistent sourcing, SEO-optimized answers that aren’t accurate. Lock it down. Specify which domains are permitted. For a Contentstack project, that’s contentstack.com/docs. Everything else requires explicit permission. If the approved URL list is long, store it in a markdown file in the project folder and point the instructions at it.

This Requires an Architect

Here is the part that does not get said enough.

You cannot point Claude at an unfamiliar platform, load a few docs, and expect it to diagnose architecture problems. You can try. What you’ll get is fluent, confident, and partially wrong.

There are many engineers capable of setting this up. The value of an architect doing the work is separation of concerns in roles. The architect’s role is to nail down processes and choices that allow engineers to focus on the best way to apply them.

In our Contentstack use case, the single session worked because the person directing it brought a deep understanding of adjacent technologies and the experience to know both what to ask and how to evaluate the responses. Specifically:

  • Recognizing that the error message pointed to a schema limit, not a code problem, because that’s how content platform resolvers typically surface constraint violations
  • Understanding that “recursive” in the documentation meant multiplicative compounding, not additive, based on how similar systems handle nested references
  • Knowing the fix had to leave the content model intact for authors, which ruled out several otherwise obvious approaches
  • Reading a Claude-generated Python script’s output and recognizing that the confident result provided the first time was due to looking in the wrong parts of the schema
  • Looking at a before/after instance table and determining whether the fix was actually complete or just moved the problem

None of that knowledge lives in the documentation itself. It transfers in from adjacent experience: content modeling, schema design, how platform resolvers work under the hood. Claude surfaces the platform-specific detail. The architect determines what it means.

The tool doesn’t replace experience. It supercharges it with speed and specific knowledge.

The Interaction Pattern

What the Contentstack session actually looked like, stripped of the platform specifics:

  1. State the problem. Provide the evidence: the error message, the exported schema, the documentation.
  2. Claude generates a hypothesis. Test it against the data.
  3. Diagnostic script written and run in the sandbox.
  4. Root cause confirmed. Fix designed. Impact predicted before any schema changes are made.
  5. Fix implemented. Follow-up session loads the new export and verifies the result.
  6. Summary file created. Next session’s candidates identified.

No magic. An architect with relevant adjacent experience, a fast and patient research partner, and a well-stocked project folder.

Prompts That Did Actual Work

These are worth examining because the techniques transfer to any platform.

“Describe in detail the cause of home_page_template having 24 instances, and instances of what?”

The second half of that question is the important part. Asking Claude to define what it is counting before giving the count forces precision on both sides. In technical sessions on an unfamiliar platform, jargon can mask shallow understanding without anyone noticing until the fix doesn’t work. The ability to ask that follow-up, to know that “instances” needed a definition before the number meant anything, comes from having debugged similar problems elsewhere. Use this pattern whenever an answer could be technically correct but operationally ambiguous.

“Create a summary file to feed to the next analysis session that includes the conclusions from this session combined with the original inputs. Format and sequence the file so that the next session can be as efficient as possible.”

Besides being familiar with adjacent technology, experience solving complex issues with Generative AI is why this is an approach for architects and engineers. Yes, Claude will now start compacting sessions on its own to improve efficiency, but having the sense that it is time to move to a new session is again an area where human experience beats relying entirely on the AI.

This prompt converts a working session into a durable asset. The phrase “format and sequence for efficiency” is carrying real weight: it tells Claude to think about how the file will be consumed, not just what it contains. The output becomes the cold-start document for the next session. Without it, every session re-derives context the previous one already established.

“Read the attached to get full context of the original issue, then review the contents of [folder] and determine if and how the issue has been improved.”

Sequencing does the work here. Claude gets the full prior-session summary before it touches the new data, so “improved” arrives with a precise definition attached. Without that order, it analyzes the new export without knowing what it’s comparing against. Prime with context before assigning the task, every time.

All three follow the same pattern. Context before task. Output format stated up front. It is not a methodology. It is just how you would brief a colleague who needs to be useful on short notice.

The Setup Is the Differentiator

Two teams, same platform, same error.

Team A has Claude. No curated project, no loaded docs, no taxonomy, no instructions. They get generic answers that feel helpful until they don’t hold up under the actual constraints of the platform.

Team B has a project built by someone with deep experience in adjacent technologies, content modeling, schema design, API behavior under constraint, who knows both what to ask and how to evaluate what comes back. Downloaded reference docs. Exported platform data. Session summaries that carry forward. Instructions that push back on bad assumptions.

Team B gets a root cause analysis, a fix, and a forward-looking roadmap. More importantly, they get it without accumulating the kind of structural debt that shows up six months later as an emergency.

A Note about Choosing Cowork

What I’m describing is not the typical use case for Claude’s project-based workspace. It is aimed at knowledge workers automating routine tasks: organizing files, generating reports, drafting communications. Productivity stuff. This is not that.

My choice of Cowork is based on my day-to-day work being mostly in documents and decks. This could also likely be done using Claude Code in an IDE for those that prefer that interface.

I became aware of how far outside the lines I was operating when someone asked what tool I was using, I explained it, and I watched the look on their face. You know the look.

I have been here before. I spent years using JMeter for continuous functional and regression API testing, which is not what JMeter is for. JMeter is a load and performance testing tool, and there are entire communities of people who will tell you this. They are correct and also missing the point, because once you understand how JMeter handles realistic randomized inputs and configuration-driven test selection, you end up with one codebase doing the work of four. I wrote about it. People told me I was doing it wrong. The tests kept passing, so.

It is common to analogize the similarities between physical tools and technical tools. “When all you have is a hammer, everything looks like a nail”, and “You can use a screwdriver as a chisel, but you really shouldn’t.” I’ve often used those myself. But the opposite analogies are also true. Most tools can be a weapon, and many tools can have multiple uses. While screwdrivers are still terrible chisels, some are great prybars, hole punches, and, yes, weapons. Same with software. Excel has spellcheck, but I’d never paste text into it before posting to a blog, but I have used formulas to parse text rather than writing a script to apply regex rules because it is faster and just as accurate. Use your tools to the extent of their value, and don’t underestimate their value or your ability to innovate.

If you found this interesting, please share.

How to Foster AI Adoption from the Bottom Up

Scott S NelsonLeave a comment

There is a lot of commentary about how AI initiatives are “failing”. Some measure it as ROI, which is a fair yardstick. Others point at the lack of adoption, which every technology goes through. The blame for these “failures” is often placed on leadership, which is fair given the meaning of the term. Speaking of terms, derivatives of “fail” have been in quotes so far for a reason: if you think of these things as a fail, then you are either sitting on the sidelines or throwing in the towel on the opportunities that AI offers. The only way to fail with AI is to give up on discovering how it will help your business. That will likely be followed by another type of fail.

Like everything else, failure is also a continuum. A prompt that returns an irrelevant result can technically be considered a fail, again challenged by the assumption that one can truly fail if they take the result as a lesson and do something else. At the other end of the spectrum is an agent that Deletes Company’s Entire Database, which is as close to a true fail one can get. There is no getting away from the fact that some people, teams, even companies, are just not very good at what they do, and capitalistic Darwinism will deal with them regardless of whether they are adopting AI or not (though AI will speed up the process).

Those true fails are a very small percentage of the world. The rest is a result of a type of hubris that (probably not coincidently) has seen a lot of attention in the business- and pop-psychology mediums lately, the Dunning-Kruger effect. Well, actually, just similar. The Dunning-Kruger effect is mostly about those that think they are doing better than they really are. The precursor to the failures that occur from attempting to follow early adopters doesn’t really have a term. However, think of this analogy: a person shows up to open mike night at a comedy club and sees a few folks pull off really good sets. They sign up to give it a try and find out the hard way that there is a lot more to getting an audience to laugh than just telling jokes.

So, lots of companies piled on to the AI bandwagon after having seen others succeeding with what looks from the outside as little or no effort. From the inside, these front runners have been playing with AI for years before ChatGPT woke the world up to the value of LLMs and Generative AI. They already had goals in mind of what they would do with the next evolution of the technology and they applied the latest and greatest with the same rigor they had been working with the latest but-maybe-not-the-greatest capabilities previously. The Lesson: Have a goal, make a plan, and be flexible moving from current state to target.

Back to the leadership blame. Again, some leadership is just not that great. The rest become great by engaging the company as a whole to define goals, make plans, and support the flexibility needed in achieving those goals when following a roadmap that traverses new territory. With AI, that leadership could start with a survey about “What tasks do you perform that are tedious?”

It’s especially important for leadership to ask this of the company as a whole because they are generally insulated from the types of mundane work that takes away from the ability to be productive. Some common areas are time and expense reporting. Most time reporting systems have a way to simplify this for recurring entries, but for those with varying tasks it would boost productivity and morale to have an agent that parses emails, IMs, and calendars to draft the weekly time submission and remind the person to review and submit. Or expense reports drafted by uploading receipts and the driving meeting invite or email. The ROI of such agents will vary greatly between enterprises, from a few dollars per person to a few million per year.

By polling the entire company for input on where AI can free people up from repetitive, simple, and necessary tasks that reduce the time, energy, and interest they could spend on work that is more fulfilling and more profitable, an enterprise can rapidly create a backlog of ROI-driven AI initiatives. The initial ROI may be small and sometimes difficult to measure. There are two key differences these initiatives would have over the 95% that “fail”:

  1. ROI-Driven Goal: The initiative would have an ROI-driven goal from the start, avoiding the trap of, “We’re doing this because everyone is doing it.”
  2. Built-in Adoption: Adoption is built in, because the goal was defined by the very people who will benefit directly and immediately from the results.

Once business success is demonstrated, more ideas will flow. At which point, it’s a good thing you defined some governance first. You did, right? Well, that’s another article.

As an addendum to this post, I asked Google Gemini for some better questions to solicit ideas for AI initiatives. Here is what it responded with:

That is a good starting point, but the question “What tasks do you perform that are tedious?” is limited because it only focuses on negative experience (tedium) and ignores major areas where AI offers massive value.

Here are five better questions to ask when identifying opportunities for AI, along with the reasoning behind why they are more effective:

5 Better Questions for Identifying AI Opportunities

1. “What tasks do you currently perform that involve interpreting or synthesizing a large volume of unstructured data?“

Why It’s Better:

  • This targets the core strength of modern AI, especially Large Language Models (LLMs) and computer vision. AI excels at processing and making sense of data humans find overwhelming:

    • Examples: Summarizing thousands of customer feedback emails, extracting key terms from legal contracts, or analyzing unstructured notes from sales calls.

    • This question moves beyond “tedious” (which a human might not mind doing) and identifies tasks where AI provides superhuman efficiency and insight by handling data volume and complexity.

2. “Where in your process are decisions made based on historical patterns, forecasts, or complex optimization?“

Why It’s Better:

  • This targets Predictive and Prescriptive AI. These systems go beyond automation; they actively optimize the business for better outcomes.

  • Examples: Optimizing logistics routes (a complex math problem), predicting equipment failure (preventative maintenance), or determining the best pricing strategy based on market conditions.

  • This question focuses on driving direct business value (cost savings, revenue growth, risk reduction), which is a stronger justification for AI investment than simply reducing tedium.

3. “What tasks are currently handled by a person that involve repetitive, rules-based digital actions?“

Why It’s Better:

  • This clearly identifies opportunities for Robotic Process Automation (RPA), which is often the lowest-hanging fruit for AI implementation.

  • Examples: Copying data between two systems, generating routine reports, or verifying customer identity against a checklist of rules.

  • The benefit here is accuracy and speed. While these might be “tedious,” they are more importantly tasks where humans introduce errors, meaning AI delivers improved quality and compliance.

4. “What tasks do you perform where the output quality is inconsistent or highly dependent on individual human experience?“

Why It’s Better:

  • This targets AI’s ability to drive standardization and quality. Humans, no matter how skilled, have variability. AI provides consistent output.

  • Examples: Writing personalized yet standardized responses to common customer service inquiries, determining the quality of manufactured parts (visual inspection), or classifying documents.

  • This question focuses on the goal of scaling expertise—making the output of your best employee available to everyone, consistently.

5. “What are the most frequent points of friction or delay in your customer or employee experience?”

Why It’s Better:

  • This focuses on impact and user experience, rather than just internal tasks. AI is a powerful tool for improving interactions.

  • Examples: Implementing a chatbot to answer 80% of routine questions instantly (reducing wait times), automatically tagging inbound customer support tickets (speeding up routing), or providing real-time personalized recommendations.

  • This shifts the focus to strategic business improvement and reveals problems that can be solved with AI-powered interfaces, not just back-office automation.

If you found this interesting, please share.

Is Your Data Ready for AI?

Scott S NelsonLeave a comment

These days, everyone is either trying AI (rare), considering AI (most common), tried it before they were ready (with mixed results) or just AI curious (which doesn’t necessarily preclude the other possibilities). Sooner or later, your organization is going to be in the trying category, and then you will be either in the group of folks that excelled with it or stumbled. One of the key factors that will determine that result is the quality of your data going in, and the integrity of your data moving forward.

Let’s take a little time now to consider the relationship between data quality, data integrity, and generative technologies, and then think about potential how to improve the odds of landing in the successful adopter side of the coming AI divide.

The Human Edge: Fuzzy Thinking and Pattern Recognition

The current differentiation between AI and human intelligence lies in our capacity for fuzzy thinking and nuanced pattern recognition. Humans possess an innate ability to identify when information doesn’t fit a pattern or context, a skill that AI systems are still developing. While AI can process vast amounts of data at incredible speeds, it may struggle with contextual understanding and adaptability in novel situations.

This limitation in AI’s cognitive flexibility can lead to inefficiencies, particularly when dealing with complex, real-world scenarios. As AI systems attempt to process and make sense of imperfect or inconsistent data, they will consume more computational resources, leading to higher operational costs.

The Rising Costs of Using AI Inefficiently

The inefficiencies in AI processing are already manifesting at a macro level. Major tech companies and AI research institutions are reporting significant increases in power consumption as they scale up their AI offerings and user base. These escalating costs will (eventually and inevitably) be passed on to consumers, likely in the form of changes to service billing structures. Consider the current use of paying per token where either the cost per token will go up or the number of tokens require to complete common operations, or both. Think of how coffee used to be sold in 1-lb bags and now we pay more per bag where the bag now holds 10 ounces. AI may become the first digital form of shrinkflation.

Garbage In, Garbage Out…More Garbage In?

Recognizing these challenges, forward-thinking organizations are prioritizing data cleanup as an important first step on their AI adoption journey. However, it’s important to note that data integrity is not the result of a a one-time effort. It requires ongoing policies, procedures, processes to support what is likely the most import commodity any organization owns.

When data stores are initially created, they are typically clean and well-structured (don’t get me started on garbage test data, that is a separate article…coming soon!). The data becomes messy over time (how much time depends on many factors) simply through regular use (and sometimes irregular, but that is also beyond the scope of this post). When AI is added to that use, trained on that same use, it will get messier faster unless the processes that led to the mess are also addressed.

It may be tempting to consider this a training issue. Inadequate training can certainly lead to bad data, but good training may not be sufficient to correct the problem. This is because training is costly to create, costly to deliver, will need to be delivered again for every new team member, will likely need to be repeated periodically for all team members, and still may not always be remembered or followed.

The most reliable and cost-effective way to improve those processes is to automate those that can be automated. Automation may cost more to create than the training process, but then it is one-and-done until the process itself needs to change. The key to cost-effective automation is determining when it is still OK to kick an edge case out for a human to deal with it and have a good process for the human to be notified and the task tracked to completion.

Automation offers several advantages over traditional training methods:

  1. Consistency: Automated processes perform tasks the same way every time, reducing human error.
  2. Scalability: Once implemented, automated processes can handle increasing volumes of data without proportional increases in cost.
  3. Long-term cost-effectiveness: While initial implementation may be costly, automation provides ongoing benefits without the need for repeated training sessions.

Moving forward

Once the organization’s data has been cleaned up and processes put in place to maintain the integrity of that data, automated where possible, then the opportunity to get ahead of the competition through generative technologies is real for your organization. Like many adventures into new territory, there will be plenty of new challenges that will require urgent attention and decisive action. Preparing for what is known and predictable first will leave more resources for managing the unexpected.

And remember, most people heading into new territory seek the help of an experienced guide. Being new territory, it isn’t so important that the guide be experienced with the specific territory, but that they have experience of venturing into other new areas and have lived to tell about it.

Shout out to Jon Ewoniuk and his new podcast The 360 Salesforce Mastermind Podcast. This article was inspired by his first episode, where his guest spoke about niches (mine being a leadership in digital innovation and automation adoption) and the importance of good data to support generative technologies.

If you found this interesting, please share.
Freepik rendering of the prompt 6 cats in a line one whispering to the next playing the telephone game

Realizing Agile Efficiency

Scott S NelsonLeave a comment
(Feature image by Freepik)

TL;DR: Fostering a culture of trust that leads to calm collaboration up front will yield the benefits that Agile principles promise.

Preface: While agile is in the title of this post, no claim is made that the post is about how to do agile or how SAFe is or is not agile. It is about how the Manifesto for Agile Software Development is self-clarifying in that it concludes with “while there is value in the items on the right, we value the items on the left more.” (italics mine), and how the value of the items on either side should be measured by their effectiveness in a given organization and the organizations influence on the “self-organizing teams” referenced in the Principles behind the Agile Manifesto. That said…

The value of architecture, documentation, and design reviews in SAFe was illustrated in a scenario that played out over several weeks.

The situation started with the discovery that a particular value coming from SAP had two sources. Well, not a particular value from the perspective of the source. The value had the same name, was constrained to the same list of options, but could and did have different values depending on the source, both of which were related to the same physical asset. For numerous reasons not uncommon to SAP implementations that have evolved for over a decade, it was much more prudent to fetch these values from SAP in batches and store them locally.

The issue of the incorrect source was identified by someone outside the development team when it was found to be commonly missing from the source selected for work prioritization. For various reasons that will be common across a variety of applications that support human workflow, this was considered something that needed to be addressed urgently.

The developer who had implemented the fetch to the correct source was tapped to come up with a solution. Now, one thing about this particular application is that it was a rewrite of a previous version where the value of “Working software over comprehensive documentation” was adhered to without considering the contextual reality that the team developing release one would neither be the team working on the inevitable enhancements nor ever meet that team. The re-write came about when the system was on its third generation of developers and every enhancement was slowed because there was no way to regression test all of the undocumented parts. Unsurprisingly, the organizational context that resulted in the first version missing documentation also resulted in some tables schemas being copied wholesale from the original application and not reviewed because requirements were late, resources were late, and the timeline was unchanged. So, with no understanding of why not to, the developer provided a temporary solution of copying the data from one table to the other because it had only been communicated that the data from one source was the correct data for the prioritization filter. Users were able to get their correctly prioritized assignments and  the long-term fix went to the backlog.

As luck and timing would have it, when the design phase of the long term fix was picked up by the architect, the developer was on vacation. Further, while this particular developer had often made time to document his designs, the particular service the long-term fix depended on was one of the few that were not documented. Still further, it had been re-design as another service had been discovered to obtain the same data more reliably. But all of the data currently loaded was from the previous version, so even the attempt of reverse engineering the service to get sample data for evaluation was not possible. These kinds of issues can lead to frustration, which in turn dampens creative thinking, which is to say that had the architect looked at the data instead of following the assumption from the story that the data wasn’t yet readily available, he would have discovered that it was already present.

Eventually the source of the correct value was identified and a design created that would favor the correct value over the incorrect value but use the incorrect value if the correct one was not available to allow for the assignments to continue because sometimes the two actual values were the same (which is inspiration about a future post discussing the value of MDM). The design also included updating to the correct value if it became available after the initial values were set. The architect, being thorough, noted in the design a concern about what should be done when the correct value came into the system after the record that was prioritized based on that value has been assigned and processed by a user. After much back and forth, it was finally communicated that while the data was retrieved from the same system and labeled with the same name, the two values were not different because one was incorrect but because they were in fact to separate values meant for two different viewpoints. Which means that the design of attempting to choose and store a single correct value in both tables was invalid and that the records altered for the work-around were now (potentially) invalid. This made the correct solution a (relatively) simple change to the sorting query.

With the full 20/20 vision of hindsight, it is now clear that if the team did not feel that ever issue needed to be treated as an emergency and all of the product, design, and development stakeholders had discussed the issue prior to taking action, about 80 hours of work would have been reduced to 4 hours. Yes, there were other factors that impacted the need of 80 hours to deal with what is a fairly minor flaw, but those factors would not have come in to play had the questions been asked up front and clarity reached through collaboration.

If you found this interesting, please share.