Best Practices in IT

Most of my IT career has been as a consultant and a manger of consultants. Over the years I have established some patterns to follow that I find helpful in making projects and technology easier in the long run. I sometimes share these in mentoring sessions and have heard often enough “you should write that down” where I started to.

I am still writing these concepts down, and while they originate from a consulting perspective they translate to every day practices beyond consulting.

At the time of this writing (not reflected by the pub date), I am re-vamping this site. Part of the update is moving this content from a standard page to a category page. The navigation will change to reflect that when it is either complete or well along.

Replacing your Proposal Team with ChatGPT

September 9, 2025September 9, 2025Scott S NelsonLeave a comment

I’ve heard of some businesses that have completely automated their RFP response process using Agentic AI. To reach that level of automation, you either need a very narrow set of services or a very generous budget to address all the quirks and exceptions.

I have neither of those.

Before I go on, I want to point out that while I will definitely continue to use Generative AI with all of my documentation as tool to improve quality, I much prefer working with a human team that is AI-augmented rather than just AI. It is a strain being the only one managing the human factor of work that is meant to drive decisions. The title is not a suggestion; it is a description of how to cope when it is necessary.

What I do have is access to a few Generative AI tools. For various reasons I won’t get into here, ChatGPT Projects is the best fit for the workflow I have adopted (and still refining). Projects are ChatGPT’s (poor) answer to NotebookLM and Perplexity Spaces

(see my earlier post about Organizing AI Augmentation with Notebooks).

Projects are useful in that they keep related prompts and files in one place, but they don’t really cross-reference or allow for collaboration. It does come with that fine print at the bottom of the screen stating:

“OpenAI doesn’t use [NAME OF COMPANY PAYING SUBSCRIPTION FEE] workspace data to train its models.”

Which is the main one of those reasons I said I wouldn’t get into (oops!).

I recently worked on a proposal at a time when most of the people who would usually help were busy with other things, so I settled into working mostly with ChatGPT like an eager-but-green proposal teammate (the AI being the green one, not me…no matter what that LLM wrapper says).

Setting the Stage

For this particular proposal, the prep work didn’t look all that different from the old manual process. It starts with a short document to capture the proposal’s guiding themes: my company’s strengths, differentiators, and the ideas that needed to shine through in both tone and substance. The document was mostly drafted by practice leadership and refined with a few folks familiar with client, project types, or both.

Next came the outline. Depending on the RFP structure, I sometimes let ChatGPT take the first crack at building an outline from the document, then refine it interactively. Other times, the RFP format or flow is not friendly to automate parsing, even for a well-trained AI (or so I assume, as I haven’t attempted to train one that deeply yet). In this case I build the first draft of the outline myself, then hand it to ChatGPT to check against the original RFP. That combination of back-and-forth has become standard practice.

Draft One: Enter the AI Intern

Once the outline was in good shape, ChatGPT proactively offered to populate the template once it was refined, which fits with the persona I have of it as an eager, educated, and inexperienced intern or junior associate. And given the quality of its suggestions, it is tempting to respond with a “Yes” and let ‘er rip. But tempered experience had me opt for prompting it to do so one section at a time, and waiting for feedback or confirmation before moving on to the next section. In this manner, I was able to put together a pretty decent first draft much faster than doing it entirely on my own (or even with a “real” eager, educated, and inexperienced intern or junior associate, whom I also would not want to do a full draft before getting some feedback).

I would say it was about 50/50 of accepting the first draft of a section versus a revision. As with any Generative AI augmented content generation, most of the issues stemmed from missing levels of details in my prompts versus ChatGPT misunderstanding the intent. Speaking of understanding the intent, I attached the entire proposal (again, because, like I said, I know notebooks and spaces and projects ain’t those), the outline, and the context document after it asked to write the proposal for me, and tempering the response to its offer with “Yes, but…” and then instructions to do it a section at a time and refer to the files.

Staying Sane (a.k.a. Breaks Matter)

As many proponents of utilizing Flow will tell you, it can be very beneficial to take breaks every 60 to 120 minutes (while most of the gurus on the topic seem to gravitate to the 90 minute mark, I hold fast that it varies by person and context, mangling Bruce Lee’s advice to “be like water”, in this case by seeking your own level). Without breaks, your ability to be objective about the quality of GenAI outputs will start to degrade and tilt where your bias is, i.e., past one’s threshold of real focus, some will start accepting every output while others will either keep refining the prompts for sections over and over or just re-write it by hand.

The Human Touch

After ChatGPT’s draft, it was time for the what passes as human intelligence (I used to call coffee my “artificial intelligence” until the term started being used by everyone to refer to what we currently call AI). I have enough experience (and ego) around writing proposals, and made some minor edits of the first AI generated draft. Once that first draft was completed, I dove in to give it a serious human touch, reading through the entire draft and making notes of changes I thought it needed. That read through without editing may seem counterintuitive, but it is necessary because something that jumps out at me as being incomplete, inaccurate, or just plain wrong may be clarified later in the document. After a top to bottom read and making notes of changes, I then work through the notes to actually make the changes, skipping or revising those changes with the full context of the document.

Then it’s ChatGPT’s turn again. I have it go through the document, essentially repeating what I had just done. This is a process I have worked on in other forms of writing as well, and I have a general prompt that I tweak as needed:

Check the attached [PROPOSAL FILENAME] for spelling errors, grammar issues, overall cohesiveness, and that it covers all points expected as a response to [RFP FILENAME].

Only provide detailed descriptions of any corrections or recommended changes so that I can select the changes I agree with. Think hard about this (thanks to Jeff Su‘s YouTube channel for this addition!)

And then I work my way through the response. This same prompt is re-run with updated versions of the proposal until I am satisfied that this stage has yielded as much benefit as it can.

Tightening the Screws

Finally, (or almost so) I have ChatGPT draft the executive summary. In the case of a really big RFP response, I will first have it draft the section summaries. These summaries are necessary to any proposal. In fact, they often make or break the proposal, possibly because they are the only parts the decision makers read, sometimes along with reviews done by others. If the summaries don’t come easy, or don’t sound right based on that original context document, I will go through and collaboratively revise the relevant sections until the summaries flow.

The Final Check

Finally, I try my best to find another human to check the whole of the result. If I’m lucky, I get additional input. If I’m really lucky, they’ve brought their own GenAI-assisted reviews into the mix.

GenAI has had a major impact on my writing output. The flow I use for proposals isn’t all that different from the flow I use to write blog posts or other content. I do a number of stream-of-consciousness sessions (the number varying on the complexity and length of the content), and then start refining it. I used that approach before GenAI, and the key difference that GenAI has made in my process is that I have learned to do less self-editing during those initial brain dumps, because I know that I have a tireless editor to review and give me feedback during the editing phase. Plus, the editor can be coached in both my intent and style to help me improve beyond just the level of “not clear”, and “i before e except after c or when the dictionary says otherwise”.

10% innovation, 90% stay out of their way

December 13, 2024December 13, 2024Scott S NelsonLeave a comment

Google had the “20% Time Project”, which was an initiative to encourage people to innovate unsupervised. According to a page in Wikipedia (found using Perplexity.ai, as most of my research these days), this resulted in things like AdSense and Google News. The Wikipedia entry and other research shows that it had its issues, too. Few people did any percent, and some called it the “120% Project” because the work happened beyond the regular work day.

Official or not, many companies have some type of program like this, and I’ve worked at a few where they were both formal and informal. The formal ones have some type of financial incentive to them, and the informal ones are generally a good way to accelerate career growth. There are many ways to measure the success of such programs. Increases in employee engagement; broadening of skills (which increases organizational capabilities); competitive edge as better (or even disruptive) innovations evolve; the attraction talented new employees; and things people haven’t even thought of yet because of the nature of the topic.

The innovations that pay off go through some stages that are similar to regular old projects. There is the discomfort stage, where something is identified as bothersome at some level and wouldn’t it be great if something were done about that. There is the ideation phase, where someone thinks that they have a way to make it less bothersome (or more cool, which is a legitimate bypass of the discomfort stage). There is the experimental stage where various things are tried to fix the problem or improve the solution (or sometimes both). Then there is the demonstration stage, where someone shows others (sometimes a someone from an earlier stage) what the experiments have yielded. The demonstration and experimental stages may iterate for a while, until either the commitment or abandonment stage occurs. Those last two are not only similar to regular old projects, they are the start of regular old projects.

One interesting thing about the stages of an innovation is that the “someone” at each stage may or may not be the same someone. There are solopreneurs where they are the someone from start to finish, or at least starting at the experimental stage. Which final stage comes in to play has lots of different influences. Some innovations are clearly awesome from the start, while others seem great in isolation but have impractical issues during or after the ideation stage. Others can be killed by skipping or re-ordering stages. As mentioned, some stage can be skipped because they may not be necessary (though I believe the discomfort stage always occurs and is just not always recognized as such). The biggest problem can occur in re-ordering the stages, especially moving that commitment stage anywhere earlier. It is usually helpful to set a time frame for the demonstration stages. Having an expectation around when the demonstration will be provides the urgency that is sometimes necessary to shift from motivation to action. Gathering estimates and milestones before that first demo is a good way to influence the lifecycle to ending in abandonment. It takes away the sense of ownership, playfulness, and excitement and turns it into a job too soon.

This problem has been observed in several psychological studies, none of which I had the foresight to take note of when I heard of them but Perplexity did manage to find a decent article on the “Overjustification Effect” , which is to say it isn’t just my opinion. One simplification of the phenomenon is that something that is interesting and motivating in and of itself becomes much less so when someone else tells you it is, or tells you how to be interested, or demands that you explain exactly how you are interested.

There is a related effect (which I am too lazy to find references to, but trust me, I’m not making it up) where someone that is instinctively talented at something is studied to determine how they do that thing they do through questioning and validating those unconscious processes, cease to be good at it. Usually it is temporary, but sometimes it is permanent.

All of which is to say that if you want your team to innovate, let them come up with what they will be innovative about, come to an agreement on a time frame where they can demonstrate progress, and then get out of their way. Trying to set detailed steps and milestones for innovation will greatly lower the odds of getting to a point where defining detailed steps and milestones will lead to an innovated solution.

Is Your Data Ready for AI?

October 24, 2024Scott S NelsonLeave a comment

These days, everyone is either trying AI (rare), considering AI (most common), tried it before they were ready (with mixed results) or just AI curious (which doesn’t necessarily preclude the other possibilities). Sooner or later, your organization is going to be in the trying category, and then you will be either in the group of folks that excelled with it or stumbled. One of the key factors that will determine that result is the quality of your data going in, and the integrity of your data moving forward.

Let’s take a little time now to consider the relationship between data quality, data integrity, and generative technologies, and then think about potential how to improve the odds of landing in the successful adopter side of the coming AI divide.

The Human Edge: Fuzzy Thinking and Pattern Recognition

The current differentiation between AI and human intelligence lies in our capacity for fuzzy thinking and nuanced pattern recognition. Humans possess an innate ability to identify when information doesn’t fit a pattern or context, a skill that AI systems are still developing. While AI can process vast amounts of data at incredible speeds, it may struggle with contextual understanding and adaptability in novel situations.

This limitation in AI’s cognitive flexibility can lead to inefficiencies, particularly when dealing with complex, real-world scenarios. As AI systems attempt to process and make sense of imperfect or inconsistent data, they will consume more computational resources, leading to higher operational costs.

The Rising Costs of Using AI Inefficiently

The inefficiencies in AI processing are already manifesting at a macro level. Major tech companies and AI research institutions are reporting significant increases in power consumption as they scale up their AI offerings and user base. These escalating costs will (eventually and inevitably) be passed on to consumers, likely in the form of changes to service billing structures. Consider the current use of paying per token where either the cost per token will go up or the number of tokens require to complete common operations, or both. Think of how coffee used to be sold in 1-lb bags and now we pay more per bag where the bag now holds 10 ounces. AI may become the first digital form of shrinkflation.

Garbage In, Garbage Out…More Garbage In?

Recognizing these challenges, forward-thinking organizations are prioritizing data cleanup as an important first step on their AI adoption journey. However, it’s important to note that data integrity is not the result of a a one-time effort. It requires ongoing policies, procedures, processes to support what is likely the most import commodity any organization owns.

When data stores are initially created, they are typically clean and well-structured (don’t get me started on garbage test data, that is a separate article…coming soon!). The data becomes messy over time (how much time depends on many factors) simply through regular use (and sometimes irregular, but that is also beyond the scope of this post). When AI is added to that use, trained on that same use, it will get messier faster unless the processes that led to the mess are also addressed.

It may be tempting to consider this a training issue. Inadequate training can certainly lead to bad data, but good training may not be sufficient to correct the problem. This is because training is costly to create, costly to deliver, will need to be delivered again for every new team member, will likely need to be repeated periodically for all team members, and still may not always be remembered or followed.

The most reliable and cost-effective way to improve those processes is to automate those that can be automated. Automation may cost more to create than the training process, but then it is one-and-done until the process itself needs to change. The key to cost-effective automation is determining when it is still OK to kick an edge case out for a human to deal with it and have a good process for the human to be notified and the task tracked to completion.

Automation offers several advantages over traditional training methods:

Consistency: Automated processes perform tasks the same way every time, reducing human error.
Scalability: Once implemented, automated processes can handle increasing volumes of data without proportional increases in cost.
Long-term cost-effectiveness: While initial implementation may be costly, automation provides ongoing benefits without the need for repeated training sessions.

Moving forward

Once the organization’s data has been cleaned up and processes put in place to maintain the integrity of that data, automated where possible, then the opportunity to get ahead of the competition through generative technologies is real for your organization. Like many adventures into new territory, there will be plenty of new challenges that will require urgent attention and decisive action. Preparing for what is known and predictable first will leave more resources for managing the unexpected.

And remember, most people heading into new territory seek the help of an experienced guide. Being new territory, it isn’t so important that the guide be experienced with the specific territory, but that they have experience of venturing into other new areas and have lived to tell about it.

Shout out to Jon Ewoniuk and his new podcast The 360 Salesforce Mastermind Podcast. This article was inspired by his first episode, where his guest spoke about niches (mine being a leadership in digital innovation and automation adoption) and the importance of good data to support generative technologies.

Freepik rendering of the prompt 6 cats in a line one whispering to the next playing the telephone game

Realizing Agile’s Efficiency

July 11, 2024July 17, 2024Scott S NelsonLeave a comment

(Feature image by Freepik)

TL;DR: Fostering a culture of trust that leads to calm collaboration up front will yield the benefits that Agile principles promise.

Preface: While agile is in the title of this post, no claim is made that the post is about how to do agile or how SAFe is or is not agile. It is about how the Manifesto for Agile Software Development is self-clarifying in that it concludes with “while there is value in the items on the right, we value the items on the left more.” (italics mine), and how the value of the items on either side should be measured by their effectiveness in a given organization and the organizations influence on the “self-organizing teams” referenced in the Principles behind the Agile Manifesto. That said…

The value of architecture, documentation, and design reviews in SAFe was illustrated in a scenario that played out over several weeks.

The situation started with the discovery that a particular value coming from SAP had two sources. Well, not a particular value from the perspective of the source. The value had the same name, was constrained to the same list of options, but could and did have different values depending on the source, both of which were related to the same physical asset. For numerous reasons not uncommon to SAP implementations that have evolved for over a decade, it was much more prudent to fetch these values from SAP in batches and store them locally.

The issue of the incorrect source was identified by someone outside the development team when it was found to be commonly missing from the source selected for work prioritization. For various reasons that will be common across a variety of applications that support human workflow, this was considered something that needed to be addressed urgently.

The developer who had implemented the fetch to the correct source was tapped to come up with a solution. Now, one thing about this particular application is that it was a rewrite of a previous version where the value of “Working software over comprehensive documentation” was adhered to without considering the contextual reality that the team developing release one would neither be the team working on the inevitable enhancements nor ever meet that team. The re-write came about when the system was on its third generation of developers and every enhancement was slowed because there was no way to regression test all of the undocumented parts. Unsurprisingly, the organizational context that resulted in the first version missing documentation also resulted in some tables schemas being copied wholesale from the original application and not reviewed because requirements were late, resources were late, and the timeline was unchanged. So, with no understanding of why not to, the developer provided a temporary solution of copying the data from one table to the other because it had only been communicated that the data from one source was the correct data for the prioritization filter. Users were able to get their correctly prioritized assignments and the long-term fix went to the backlog.

As luck and timing would have it, when the design phase of the long term fix was picked up by the architect, the developer was on vacation. Further, while this particular developer had often made time to document his designs, the particular service the long-term fix depended on was one of the few that were not documented. Still further, it had been re-design as another service had been discovered to obtain the same data more reliably. But all of the data currently loaded was from the previous version, so even the attempt of reverse engineering the service to get sample data for evaluation was not possible. These kinds of issues can lead to frustration, which in turn dampens creative thinking, which is to say that had the architect looked at the data instead of following the assumption from the story that the data wasn’t yet readily available, he would have discovered that it was already present.

Eventually the source of the correct value was identified and a design created that would favor the correct value over the incorrect value but use the incorrect value if the correct one was not available to allow for the assignments to continue because sometimes the two actual values were the same (which is inspiration about a future post discussing the value of MDM). The design also included updating to the correct value if it became available after the initial values were set. The architect, being thorough, noted in the design a concern about what should be done when the correct value came into the system after the record that was prioritized based on that value has been assigned and processed by a user. After much back and forth, it was finally communicated that while the data was retrieved from the same system and labeled with the same name, the two values were not different because one was incorrect but because they were in fact to separate values meant for two different viewpoints. Which means that the design of attempting to choose and store a single correct value in both tables was invalid and that the records altered for the work-around were now (potentially) invalid. This made the correct solution a (relatively) simple change to the sorting query.

With the full 20/20 vision of hindsight, it is now clear that if the team did not feel that ever issue needed to be treated as an emergency and all of the product, design, and development stakeholders had discussed the issue prior to taking action, about 80 hours of work would have been reduced to 4 hours. Yes, there were other factors that impacted the need of 80 hours to deal with what is a fairly minor flaw, but those factors would not have come in to play had the questions been asked up front and clarity reached through collaboration.

AI as a mental crutch

May 3, 2024Scott S NelsonLeave a comment

(Feature image created with DALL-E, providing feedback on my image proompting skills)

Every couple of years I find myself building a new Linux virtual machine baseline for some project. Even though I’ve documented the process thoroughly the last few times there is always some quirk that has me starting mostly from scratch each time. This time I started off with setting the home page in Firefox to perplexity.ai and using it to find all of those mundane commands I forget three weeks after going back to Windows for my day to day work.

This time I hit a snag pretty early in that I was getting an error that made no sense to me (the specifics of which aren’t relevant to this train of thought). Perplexed, I asked Perplexity “wtf?” in my best prompt formatting (which, admittedly, is a WIP) and it gave me a few things to try. Some (not all) of them made perfect sense and I gave them a try. They failed.

I compared everything I was looking at against a similar appliance and didn’t see any obvious differences. I tried variations of prompts with Perplexity to get a more directly relevant response, which either resulted in what had already been suggested or even less relevant responses (I did mention my prompting skills need, work, right?).

I then tried ChatGPT, which gave me the same answers that differed only in their verbosity and longer pauses between response blocks.

Finally, I ran the same search I started with in Google, which returned the usual multiple links from our old friend Stack Overflow. I did like I did before using GPT backed by LLMs and narrowed the time frame down to the last year to eliminate answers 10 years out of date (and sometimes links to my own past explanations that are equally out of date) and found a summary that looked closer to my actual problem than the bulk of the answers (which were clearly the source of the responses from both GPT sources I had tried earlier).

And there was my answer. Not just to this one problem, but to the kind of sloppy approach I had fallen into using AI. The thread started with an exact description of the same problem, with lots of the same answers that had been of no help. And then the original poster replied to his own thread with the solution (a habit of frequent Stack Overflow contributors I have always admired and sometimes remember to emulate), along with how he wound up in the situation. Again, the specific error isn’t relevant to this tale, but the source is using the the first search result that seems to answer the question rather than reading it all the way through and seeing the subtle difference between what was needed and what was provided.

No AI response will tell you about the screw ups that caused the problem (are they embarrassed for their human creators or just don’t think it’s relevant?) and the path to realizing the mistake and then recovering (and learning). But real people will and that is how we learn from each other.

So having copilot proof your work is great and using promoting to get a start on something you’re stuck on is a great productivity boost. But relying solely on the technology to do all the work is how we wind up forgetting how to think and learn and build better technology to give us time to think and learn. In short, don’t trade the mental crutch for a creative wheelchair.

What IT Is

"Technology should make things easier. I make technology easier." – Scott S Nelson