The future is cloudy with a chance of success

(Originally published at InfoWorld.)

I would have titled this post “How to be a rainmaker in the cloud” except the term rainmaker often refers to the selling process, which is already succeeding and that success is a key contributing factor to why so many cloud initiatives are all wet. If nothing else, the popularity of cloud services has made the use of metaphors in technical articles much easier!

In this post I want to talk about some of the slipperier aspects of cloud services; how to reap the most benefits; and ways to identity potential pitfalls before a sinking feeling sets in.

AI, big data and the cloud

A great expression going around about AI and big data is that they are “…like teenage sex: everyone talks about it; nobody really knows how to do it; everyone thinks everyone else is doing it; so everyone claims they are doing it”. I want to add to that that “and most of those that are doing are not having nearly as much fun as they could be”. The same can be said for the cloud, even though it has been around a lot longer and is relatively more mature. And many people actually already have it, though they might not know it, so maybe it is more like insanity.

These three technologies were all drivers for each other. The ease of getting started in the cloud (the quality of which we will ignore for now) was a multiplier for the data accumulation that had already been growing exponentially. The amount of data was so big it needed a separate category of, well big. Then, trying to manage that much data while it was still of business value (or even determine if it is of value) quickly became too much for human manipulation or even basic algorithms, so more complex algorithms were created, followed by algorithms that create algorithms and then AI became a battle cry to save us all from drowning in the data lakes (that we probably wouldn’t have created if we had true AI to start with).

The lack of quality in initial cloud forays that we ignored at the start of the last paragraph combined with the ease of getting into the cloud is what led the way for so many being over whelmed with data.  The truth is most cloud initiatives that have real business value are still in their early stages. The early adopters (that are now trying to hire more data scientists than then there are in order to help dam the floods) have given the rest of us a great example of what not to do. So let’s use their hindsight to build our vision.

If you don’t know what it is, don’t handle it with familiarity

Anyone with kids or animals learns fairly early that they should never pick anything up from the floor bare-handed if they are not positive of what it is. Technology deserves the same wary respect. If some says “everyone is moving to the cloud”, I know they are either misinformed or not being truthful (sometimes both). First, because everyone rarely does anything at the same time, no matter how much marketers and salespeople tell us otherwise. And second, because lots of us have been there for years and just didn’t think of it that way. Financial services is one very common area where storage, processing and source-of-truth data has been entrusted by the customers to the service provider and accessed over the internet since internet was spelled with a capital “I”.

The truth is, there are many different types of cloud services and the value of a given service and provider varies according to the enterprise needs. I know this is really obvious and what I am calling out here is how it is often forgotten after seeing a competitor’s press release or the conclusion of a well-rehearsed and intricately orchestrated sales demo. There is no doubt in my mind that cloud services will benefit most (maybe even all) businesses. But not every type of cloud service is needed by every business and how one business uses a specific service is not how every business should even if they will benefit from the same service.

Data storage is a great example, because it has become a universal need. There are businesses that can and should keep all of their data in cloud services. The plural is on purpose, because if you are going to commit your entire business to the cloud you need to have some kind of back up. On the opposite end of the spectrum there are many small companies that have neither the volume nor the budget to properly maintain all data safely in the cloud. In the middle (and the latter example is far more common than the first) there are the majority of businesses that will benefit storing certain types of data in the cloud and applying both MDM synchronization to ensure availability and continuity.

The green field is not always greener

The biggest hurdle in guiding enterprise technology is not getting buy-in on new technology; it is managing the false sense of security that comes with having been convinced to adopt the new technology.  

Part of the reason stake holders expect the latest cloud offering to save the planet is over-selling on the part of IT, vendor sales and marketing, and industry hype (which is greatly fueled by vendor marketing in much the same manner as a recursive process with a bug).

Another equally significant part is human nature. Once a decision has been made or a belief adopted, the subconscious mind will often vigorously defend that decision or belief whenever it is challenged, deepening the associated conviction. This is why we so often seen a mass movement towards solutions as they gain popularity even if the solution is not always what is called for. People who want to slow or redirect the changes for very good reasons seldom take the psychology of newly adopted convictions into account and the result is a more energized drive in the direction that may not be right (or right at the time).

Measure twice, cut once…every single time!

To avoid the rush to greener cloud pastures, IT and business need to work together to define and agree on business and technical goals. Once the goals are agreed to, an analysis of how a given cloud solution will further those goals should be supported by a pilot or proof of concept before any large commitment is made. The results of the analysis must vigorously seek any side effects that have a negative business aspect in addition to how the goals are met (or not). Finally, if the solution is adopted, the analysis needs to be reviewed and revised for every case. Again, no rocket science or epiphany here, just common sense that is not-so-common as to not benefit from repetition.

Walk, don’t run, to your nearest gateway

To offset the cautionary tone of this post, it bears repeating that there are many benefits to cloud services and some combination of cloud services will very likely benefit any enterprise. Some key concepts to keep in mind in order to realize the most benefits (and dodge the potential pitfalls of a bad combination) are: The forecast for enterprise architecture is increasing clouds. Enjoy the shade and keep to high ground away from potential flood damage!

Facebooktwittergoogle_plusredditlinkedinmail
© Scott S. Nelson

Too big to survive: There is no bailout for technical debt

The only difference between technical debt and financial debt is that costs are more often known in advance when taking on financial debt. Both types of debt are a tool when used intelligently with purpose and a plan to manage it and can take a devastating toll when used recklessly or imposed through misdirection or miscommunication.

Acceptable vs unnecessary debt

The original heading here was “Necessary vs unnecessary debt”. On further reflection, though, I realized that the only good reasons for incurring debt are time drive. If time is removed as a factor there is no reasonable need for debt. So then it becomes a question of when time is important enough of a factor to make debt acceptable.  The only context I can think of where time is universally an acceptable driver for debt is in an emergency situation.

Beyond an emergency, the evaluation for whether debt is acceptable because of time becomes a value proposition. In our personal lives, the first car and house is generally considered to be a good reason to accept debt because both have a large enough cost where they are likely to become more expensive over time, making it harder and harder to save for them in a reasonable period of time.

Similarly, building in-house custom applications rather than waiting for a Common Off The Shelf (COTS) solution that will incur technical debt in minimally reviewed code and the inevitable maintenance costs is worth it for functionality that is key to business value. Having worked for software vendors, I can honestly say that it if it isn’t already Generally Available (GA) as at least a patch one then it should still be considered unavailable as a COTS solution.

The other common time driver that should generally not be an acceptable reason to take on debt is impatience. Using a home equity loan to buy the latest television is a poor financial decision and implementing a new solution without a thorough evaluation and proper training is a gamble that will usually result in higher maintenance cost or a potential system failure.

The old adage “patience is a virtue” is not only true, it is a vast understatement of the value of patience.

Stop debt before it happens

The reason technical debt is becoming an increasing concern at many companies is because it tends to grow exponentially, just like financial debt. And for the same reasons. Of the three drivers for debt mentioned previously (emergency, long-term value, short-viewed impatience), the most frequent cause is the least necessary. Impatience. Problems arising from bad habits will grow until the habit has been replaced by actions that have a more positive effect.

Without getting too psychological here, impatience is a result of either wanting very much to move towards a reward or away from loss. For some odd reason, the drive forward doesn’t seem to repeat in the same context nearly as much as the drive to move away from. In technology, the drive to move away from is so common that the three key emotions related with impatience driven by escape have an acronym: FUD (fear, uncertainty, doubt).  In the case of IT decisions all three are essentially redundant, or at least a sequence. Fear driven by uncertainty and/or doubt. When the decision is around taking on technical debt, the fear is that business owners or customers will be upset if the feature is delayed or reduced and the uncertainty and doubt are the result of either not asking these stakeholders or asking only half the question.

Asking a stakeholder “Is it a problem if feature X is not in the release?” will frequently have a different answer than “Would you prefer we include feature X in a later release or risk certain delays to all future feature releases by pushing it before we have time to include it in a maintainable manner”? My experience is that most of the time neither question is asked and it is just assumed the world will end if users don’t have access right now to a new option that only 3% will ever use. It is also my experience that when the tradeoff of reliability and stability versus immediacy is explained to stakeholders they usually opt for the delay. I know many people believe that businesses have lost sight of long term implications and I believe that in many cases it not because they are deliberately ignoring them but because the people that should tell them when and why to be cautious are afraid of saying anything that will be considered “negative”.

To summarize, the best way to reduce the accumulation of technical debt is to have open, honest communication with stake holders about when decisions involve technical debt, the consequences of that debt, and the options for avoiding taking on the debt. Then, if the decision is to still choose the right now over the right way, immediately request buy-in for a plan, timeline and budget to reduce the technical debt. Again, my experience is that when the business is presented with a request to ensure functional reliability they frequently say yes.

Getting out of unavoidable or accepted debt

Taking on some technical debt is inevitable. This is why the modifiers usually, most often, and frequently were used in the previous section rather than more-comforting-yet-inaccurate always, definitely, and every time. Even in a theoretically perfect process where business always opts for debt-free choices and emergencies never happen, there are still going to be debt-inducing choices made either from lack of information or usage of imperfect vendor releases.

In the case where the debt is incurred unknowingly, once it is discovered be sure to document it, communicate and plan for its correction. The difference with cases where the debt is taken on knowingly because it is unavoidable without a much larger cost in vendor change, monitor the item with every project and when there is a reasonable option to correct it, do it. I once had to build something that was a bit kludgey because the vendor application clearly missed an implication of how a particular feature was implemented. We created a defect in the defect tracker which was reviewed in every release. 18 months later, the vendor found the error, corrected it and we replaced the work-around with the better approach in the next release. For major enterprises it is a good idea to raise a support case with the vendor when such things are identified, which I did not do at the time because the company I was managing this application for was too small to get vendor attention and the feature was not in broad use.

Originally published at InfoWorld.Facebooktwittergoogle_plusredditlinkedinmail


© Scott S. Nelson