Software Time and Complexity

Is your CI/CD missing the third C?

Scott S NelsonLeave a comment

If you are reading this post you know that both C’s in CI/CD represent “Continuous” (and if you didn’t already know that, your curiosity is admirable…and, no, “curiosity” isn’t the third C).

Getting to CI/CD can be a challenge, especially if it isn’t set up as part of the first release. (Challenge is not our third C, either, though it is becoming apparent there are many words that start with C related to software systems, so we’ll ignore the false possibilities from here on out). Adding to the procedural challenge of timing, there is also the conceptual challenge of semantics. Some organizations define the continuous Cs as every time a commit is made to the repository; others say it begins with a merge after a review; some say that if it is done “correctly” (are you still counting the “C”s?) there is no need for a review; still others will say that hourly (or daily, or twice daily, etc.) counts; some say you only really need I or D continuously (and others will certainly debate that); and there are bound to be more that don’t come to mind at the moment. What those contiguous definitions of continuous have in common is that they are all correct, depending on the context.

The cruel thing about continuous is that, technically, if it ever isn’t, it never was and cannot be again. Which is not correct, because even when done perfectly, there will be the occasional hiccup and things will need to be re-calibrated to continue being continuous.

Except when they don’t.

When they don’t, there can be several symptoms that are confused with causes. At some point, commits became synonymous with the cards on the Kanban. Sometimes it’s because management thought it would be useful to automate tracking with branches, or people used build failures as a measurement of skill rather than assignment of bug fixes, or PRs are preferred over testing—despite the fact that this approach often leads to more production issues and longer MTTR.

So what prevents those problems (and others) from discontinuing (or preventing the adoption of) CI/CD? It’s the third C, and…

The third C is Culture.

CI/CD requires the adoption of a “shift left” mentality, which in turn requires a culture of fearless experimentation where testing before committing is done as a point of pride, and every time the tests pass is a reason to commit (or “another test passes”, if you are following TDD). This also requires using feature flags that are either centrally managed (which supports runtime management in production) or developer managed, which requires automated testing of the flag.

A successful CI/CD culture depends on the team’s attitude—an attitude built on one part pride, one part professionalism, one part discipline (maybe two parts), and is most easily maintained when it includes the participation of leadership (perhaps the next post will be about the missing “P’s” in technology).

So, after that long walk through what was really a short concept, the key takeaway is that if your team is struggling with starting or sustaining a true (because there are many that are in name only, and you know which ones) continuous pipeline, focus on fostering a cultural shift within your team before investing in tools.

Facebooktwitterredditlinkedinmail
© Scott S. Nelson

Why People Never Listen (to Me)

Scott S NelsonLeave a comment

I received my weekly James Clear newsletter today. I almost filed with all the other newsletters I subscribe to and just never seem to have time to read. But then I remembered that I’m on PTO, and Clear lives up to his name, so I jumped in. Then stopped in my tracks when I got to:

“You have to work hard to discover how to work smart. You won’t know the best solutions until you’ve made nearly all the mistakes.”

My first thought was “Hmm”, followed by “that makes sense”, and then “but what is the point of trying to teach people how to do things?”.

I mentally spun on this for a while. I know that I learn from others (directly or indirectly via person, print, or video) faster if I have had some experience with the topic and struggled with doing or understanding, or at least dissatisfied with the results. I’m also familiar with constructivism and similar concepts where some level familiarity, even if only having heard the term in passing, makes it easier for the mind to grasp and incorporate details when they are presented.

Then I realized that what I struggle with is some of the qualitative terms in the statement, i.e.,

“You have to work hard to discover how to work smart. You won’t know the best solutions until you’ve made nearly all the mistakes.”

Is it impossible to learn how to work smart without having worked hard? If a student, mentee, trainee, etc. trusts the person teaching them and is highly and intrinsically motivated to learn, I think they can do so without the prior high level of effort. That said, I think that the skill of teachers (generically rather than academically, and based on skill of transferring knowledge as well as knowledge of skill) and the motivation of learners have been steadily declining and that this circumstance as an exception has become fairly rare.

As to the second of Mr. Clear’s sentences, I might be putting too fine of a point on my umbrage. I believe that one will appreciate how much better a solution is than alternatives if they have witnessed the outcomes of the others, and that appreciation will increase with experience over mere observation. I do, however, take issue with “nearly all” at several levels. The first is the fuzziness of it. Is it all but 1? Is it 99%? And I really truly believe that some, with that fuzzy level being at least less than half, of the mistakes is more than adequate for some people, especially those that are either impatient (while motivated) or highly self critical (a motivation that tries the patience).

My  protests aside, I think James Clear’s statement is accurate often enough to be accepted as a general (if not a hard and fast) rule. I am positive in my belief that his experience with people in this context is much broader than my own.

I do tend to wander in my writing, so let’s wrap up the thought on why people never listen. They do, but the absorption, retention, and results of that listening vary between individuals and over time. Some folks will get it right away because they are motivated and trust the source (and sometimes that motivation is just lack of self trust); some will have to prove it right before they believe it, and then they may experience the source has their own actions (not all, but some); some will filter it through their own thinking until they reach a threshold where they will try anything, even something that is presented as proven already to be effective; and some will just never get it, for a variety of reasons that could be it’s own book, let alone blog post.

No matter the motivation or outcome, if you are trying to share wisdom, be wise enough to know that you may need to both vary and repeat your message to eventually be heard. If you are trying to learn something, you may want to apply what you know earlier and more often so that you are well prepared to know more.

Facebooktwitterredditlinkedinmail
© Scott S. Nelson

Simplifying Deploy and Destroy with the SF CLI

Scott S NelsonLeave a comment

Personally, I never had any problems  with the SFDX CLI, which is why I was surprised they were replacing it with the SF CLI. What I now sometimes struggle with is translating all of my cheat sheet references from one to the other, and the sifting through the plethora of search results that still refer to the SFDX way of doing things. OK, I know I can minimize the second part by setting the date range in my Google searches, but when I’m being lazy (or in a hurry, both of which are more common than I prefer) I use Perplexity.ai rather than Google.

Most recently my conversion conundrum cropped up with removing components that I had deployed through the CLI. I found a great SFDX syntax in Perplexity literally one day before it was slated to be retired. Here is the relevant part of the session that got me to the solution:

Question: cli to delete objects from an org

Answer:
To delete objects from a Salesforce org using the CLI, you can use the destructive changes process. Here’s how to do it:

Step 1: Create Destructive Changes XML

Create a file named destructiveChanges.xml with the following structure:
<?xml version="1.0" encoding="UTF-8"?>
<Package xmlns="http://soap.sforce.com/2006/04/metadata">
    <types>
        <members>ObjectName1__c</members>
        <members>ObjectName2__c</members>
        <name>CustomObject</name>
    </types>
</Package>
Replace ObjectName1__c and ObjectName2__c with the API names of the objects you want to delete.

Step 2: Create Empty Package XML

Create a file named package.xml with the following content:
xml
<?xml version="1.0" encoding="UTF-8"?>
<Package xmlns="http://soap.sforce.com/2006/04/metadata">
    <version>62.0</version>
</Package>

Step 3: Deploy Destructive Changes

Run the following Salesforce CLI command:
bash
sfdx force:mdapi:deploy -d . -w -1 -u YourOrgAlias
This command deploys the destructive changes to your org:
  • -d . specifies the current directory as the deployment source
  • -w -1 waits for the deployment to finish
  • -u YourOrgAlias specifies the target org alias

Additional Considerations

  • Ensure you have the necessary permissions to delete objects in your org.
  • Deleting objects will also delete all associated records and metadata (fields, validation rules, etc.).
  • Some objects may have dependencies that prevent deletion. You might need to remove these dependencies first.
  • Always test this process in a sandbox environment before applying it to production.
  • Consider backing up your metadata and data before performing destructive changes.
If you need to delete other types of metadata (like Apex classes, Visualforce pages, etc.), you can include them in the destructiveChanges.xml file using the appropriate metadata type names.
Remember, this process permanently deletes the specified objects and their data from your org, so use it with caution.

When I ran it, it worked perfectly, and output a warning that it would no longer work the next day (I did not preserve the message content, but I can tell you the date was 11/4/24 because UpNote metadata is useful). As I mentioned, I’m often in a hurry, so I often make mistakes, and what has kept me in technology since the original Wolfenstein was released is always making sure I have a way of undoing whatever I do. So I decided to go find the cool new sf CLI solution. Which was not terribly clear to me. I found the documentation a bit confusing. Going back to the old-fashioned Google search (trust me, it pains me more to write “old-fashioned Google search” than it does for you to read it), I found a Trailblazer Community post by Max Goldfarb that greatly clarified how to make the new sf call.

I have since refined the process clarified by Max once I read the documentation while in less than my usually rushed efforts. First, I keep that step 2 XML that Perplexity gave me stored in my repo as antiPackage.xml, the full path being manifest\destructiveArchive\antiPackage.xml. I could then generate the destructive xml with:

sf project generate manifest --source-dir force-app/main/default/[WHATEVER-I-WANT-TO-DESTROY] --type destroy

and then run my simplified killer command:

sf project deploy start --manifest manifest/destructiveArchive/antiPackage.xml --pre-destructive-changes manifest/destructiveArchive/destructiveChangesALL.xml -o [ALIAS]

and boom! stuff removed. I even keep a set of destructive xmls in a special folder (you may have noticed it in the command above). I continued to use this happily for a few weeks.

Today I was messing with a single component that was giving me problems (or, more accurately, I was giving it problems by building it incorrectly) and started by removing just it. Then I went about making my fixes and, feeling less hectic than usual, decided to have a package.xml just for it so the deployment would go faster (yes, I know I can do that with a right-click, but once I am comfortable with a CLI, I prefer it, and it lets me make sure that my eventual release script will have fewer issues). It was very small, which made the content leap out at me more, and, so I realized…There is no difference between the XML output with or without the “–type destroy” switch!

So now I just keep one XML for each context in manifest\deployArchive and reference it for both deploy and destroy.

Facebooktwitterredditlinkedinmail
© Scott S. Nelson

10% innovation, 90% stay out of their way

Scott S NelsonLeave a comment

Google had the “20% Time Project”, which was an initiative to encourage people to innovate unsupervised. According to a page in  Wikipedia (found using Perplexity.ai,  as most of my research these days), this resulted in things like AdSense and Google News. The Wikipedia entry and other research shows that it had its issues, too. Few people did any percent, and some called it the “120% Project” because the work happened beyond the regular work day.

Official or not, many companies have some type of program like this, and I’ve worked at a few where they were both formal and informal. The formal ones have some type of financial incentive to them, and the informal ones are generally a good way to accelerate career growth. There are many ways to measure the success of such programs. Increases in employee engagement; broadening of skills (which increases organizational capabilities); competitive edge as better (or even disruptive) innovations evolve; the attraction talented new employees; and things people haven’t even thought of yet because of the nature of the topic.

The innovations that pay off go through some stages that are similar to regular old projects. There is the discomfort stage, where something is identified as bothersome at some level and wouldn’t it be great if something were done about that. There is the ideation phase, where someone thinks that they have a way to make it less bothersome (or more cool, which is a legitimate bypass of the discomfort stage). There is the experimental stage where various things are tried to fix the problem or improve the solution (or sometimes both). Then there is the demonstration stage, where someone shows others (sometimes a someone from an earlier stage) what the experiments have yielded. The demonstration and experimental stages may iterate for a while, until either the commitment or abandonment stage occurs. Those last two are not only similar to regular old projects, they are the start of regular old projects.

One interesting thing about the stages of an innovation is that the “someone” at each stage may or may not be the same someone. There are solopreneurs where they are the someone from start to finish, or at least starting at the experimental stage. Which final stage comes in to play has lots of different influences. Some innovations are clearly awesome from the start, while others seem great in isolation but have impractical issues during or after the ideation stage. Others can be killed by skipping or re-ordering stages. As mentioned, some stage can be skipped because they may not be necessary (though I believe the discomfort stage always occurs and is just not always recognized as such). The biggest problem can occur in re-ordering the stages, especially moving that commitment stage anywhere earlier. It is usually helpful to set a time frame for the demonstration stages. Having an expectation around when the demonstration will be provides the urgency that is sometimes necessary to shift from motivation to action. Gathering estimates and milestones before that first demo is a good way to influence the lifecycle to ending in abandonment. It takes away the sense of ownership, playfulness, and excitement and turns it into a job too soon.

This problem has been observed in several psychological studies, none of which I had the foresight to take note of when I heard of them but Perplexity did manage to find a decent article on the “Overjustification Effect” , which is to say it isn’t just my opinion. One simplification of the phenomenon is that something that is interesting and motivating in and of itself becomes much less so when someone else tells you it is, or tells you how to be interested, or demands that you explain exactly how you are interested.

There is a related effect (which I am too lazy to find references to, but trust me, I’m not making it up) where someone that is instinctively talented at something is studied to determine how they do that thing they do through questioning and validating those unconscious processes, cease to be good at it. Usually it is temporary, but sometimes it is permanent.

All of which is to say that if you want your team to innovate, let them come up with what they will be innovative about, come to an agreement on a time frame where they can demonstrate progress, and then get out of their way. Trying to set detailed steps and milestones for innovation will greatly lower the odds of getting to a point where defining detailed steps and milestones will lead to an innovated solution.

Facebooktwitterredditlinkedinmail
© Scott S. Nelson