Creating an iDoc Mapping in Informatica Cloud Data Integration

One type of mapping I find particularly complicated to build is one using the SAP iDoc connector. The complicate part is achieving the correct value nesting. It is complicated because when you create the mapping, the nesting is not visible. It is created by the use of Primary and Foreign key fields (GPK and GFK in the mapplet).

I have seen several approaches to this that work and the approach I am going to share here is the one that I came up with and use because I understand it. YMMV.

First, I create a list of GPK and GFKs from the mapplet XML. These values can be found in the TRANFORMATION nodes within the MAPPLET node.

With this list in hand, I create the document GPK with a field that will be unique per iDoc. In this case I have a unique entity ID in my source and a date field that will make the entry for the entity unique. I prefix this key with the OTYPE because I may have multiple entries for the same object on the same date but different OTYPEs.

Creating the GPK_DOCNUM Value
Creating the GPK_DOCNUM Value

Then, for each immediate child, I define its GFK and GPK using the node names as prefixes:

Creating the GPK_E2PLOGI001 Value
Creating the GPK_E2PLOGI001 Value
Creating GPK_E2PITYP001 Value
Creating GPK_E2PITYP001 Value
Defining First Child of E2PITYP001
Defining First Child of E2PITYP001

The above examples goes three levels deep. If your iDoc requires nest your nodes in a different pattern, generate your GPK values in such as way that they can be properly created in both the parent and child node.

For ease of maintenance, I name the fields in the expressions the same as they are defined in the mapplet so that I can use Automatic field mapping, which is eliminates the need to manually re-map anytime I make a change:

Auto-mappping Fields for ease of Maintenance
Auto-mapping Fields for ease of Maintenance

I hope this saves you some of the time it took to get me to this approach.


How Salesforce Supports Citizen Development

Citizen development is really a responsible response to the dilemmas created by Shadow IT. Now that technology is available to those with minimal technical knowledge business users will implement solutions without the help of the IT department. The best thing IT can do about this is mentor the business users in ways that will support what business is going to do anyway in a way that will not lead to enterprise-level headaches. Salesforce is at the forefront in helping business and IT with this new paradigm.

The number of times I revised the title of this post is a sign of the times in technology. Those not steeped in the gray arts of technology may think that since computers process 1’s and 0’s that going from thought waves to software is a linear and clearly defined path. The more the technology evolves the less true that is. I started with the title of “How Salesforce Enables Citizen Development”, but a key premise of this post is that it is not a check box in the system administrator’s console, which the term “enables” insinuates. “Citizen Development with Salesforce” was considered and rejected because it has a tone that suggests that there is no longer a need for highly trained Salesforce administrators, architects and developers. Not only do I disagree with that premise, I more emphatically caution against the invalid assumption that such a void would result in cost savings. These nuances of title may seem like a lot of over-thinking except that as both a writer and reader I am all-too-aware of the tendency to base a fully formed opinion on the title alone.

I was recently asked to sum up the benefits of citizen development and came up with the following:

  • User-owned Solutions
  • Reduced IT Bottlenecks
  • Streamlined Process
  • Lower Costs to Deliver

Salesforce supports citizen development by providing a platform with capabilities that can be accessed and utilized with a minimum of training and experience. The unbridled optimist will look at the preceding sentence and imagine a world where every business user can build applications that are easy to use and will contribute to productivity at a lower cost.

Citizen Development Bumper Sticker Policies
Citizen Development Bumper Sticker Policies

The realist would (and should) take umbrage with the word “every”. Putting aside the variance in individual capabilities, there are other key factors that make “every business user” a dangerous assumption. Two key factors are time and inclination. It takes both to perform any one of the following critical tasks for a successful application:  Determine the full range of business requirements an application should address; analyze the variety of technical solutions and appropriately select the best fit for the requirements;  review the existing functionality within the organization for potential reuse and impact; train and support other users in the resulting application; and maintain proper data governance to ensure both adequate security and cost controls.

So, perhaps a better statement of how Salesforce supports citizen development would be “Salesforce provides the tools for an enterprise to enable business users to build applications with little or no IT support when proper governance processes are established and followed”. This phrase doesn’t fit on a bumper sticker as easily as “Clicks not code”. Perhaps “IT doesn’t go away. IT gets out of the way” almost fits, though.

The “lower cost to deliver” benefit is based on the streamlined process of citizen development, i.e., no need for business to create a full specification to hand off to IT for implementation since business will own the development. In an enterprise where the IT team is continuously backed up, this will lead to faster time to delivery as well. In cases where the scenario is simple or common enough to be configured in a generic manner, a great deal of time can be saved. However, this should not be confused with the false assumption that configuration over coding is inherently faster. Sometimes it is and sometimes it is not. Declarative programming must be provided in a way that is maintainable by the vendor and generic for the customer. For a skilled developer, custom development can be completed in far less time than it takes to configure a collection of generic options to something as simple as loop through a specific set of data looking for a specific output.

If it sounds like citizen development is a bad idea that is neither the case nor the intention. Application development is like raising a child… it takes a village with each member contributing their specialty at the best time and in the appropriate context. A governance group to provide guidelines, consider exceptions and enforce adherence; Architecture and security specialists to determine the best way to ensure compliance; Developers to provide reusable components when they are not readily available from the App Exchange; Trained Salesforce system administrators to enable appropriate permissions, configure necessary integrations, and manage production deployments.

In short, all of the roles that an organization following best practices for platform use will have in place anyway. On the one hand, supporting citizen development adds some additional tasks to those who support the platform. On the other hand, properly supported citizen development frees up platform support personnel to better focus on the tasks that most need their skills while improving relations between business and IT by enabling business to more self-supporting.

Originally published at InfoWorld


Salesforce Community with Custom Objects in a Flow

Last year I wrote about Revisiting the Question of Build versus Buy for Web Portal Solutions, which was greatly inspired by the reduction in solid portal offerings that began about a decade ago. The dearth in offerings is a result of acquisitions by vendors who prefer that portals work only (or, at least, easily) with their own products. While I’m not a big fan of architectures that lead to vendor lock-in, there are situations where the portal is focused on exposing a certain view of a specific application and it is best if that is built by the same vendor as the application itself.  Case in point: Salesforce Community Cloud.

Salesforce has been rapidly improving their Community Cloud product with more features and easier management. I believe this is partly fueled by the demise of pure-play portals that can surface Salesforce functionality using the great Salesforce APIs (Liferay being the exception) and a smart move to increase Salesforce revenue while reducing customer costs for high API traffic. Whatever the drivers, I like where it is going. Though, like any technology on a rapid growth curve, there are some bumps along the way. One such bump is puzzling out all of the different permissions involved to provide various functionalities. Generally these bumps are easily overcome thanks to the active SFDC user community. Recently I ran across something that was not so easily solved and thought I would share it here.

The requirement is to create a workflow that ties together the standard Case object with a custom object. The goal is to make the user experience of working with a long form created previously to support the custom object by breaking down the numerous fields into user-friendly chunks that will result in a Case being created along with the custom object as a related object. With the normal amount of “oops” and “darns”, I managed to put together a Visual Work Flow that would take the inputs and create the Case object and then the related custom object.

A Visual Work Flow to take inputs and create a Case object and related custom object
A Visual Work Flow to take inputs and create a Case object and related custom object

I then created a Community page in a community using the Lightning Community template Napili and placed a Flow component with the new workflow configured.

Adding a Visual Workflow to a Lightning Community
Adding a Visual Workflow to a Lightning Community

The first bump I hit was that the Case failed to be created, which I easily figured out from the various Salesforce user communities was because I was using the wrong ID for the contact (the User ID is different than the Contact ID for community users). Once this was fixed, I then found that the custom object was not being created. Back to the trusty community posts where I found the sage advice to grant access to the object to the profile referenced in the Community Builder Settings. Well, not so sage as in my haste I forgot that that fields only applies to “guest” users, i.e., users not logged in.

To make a very long story short, the default profile granted to customer community users did not have permission to access the object (even though, during creation, I granted necessary access to all profiles) and the default profile is read-only. The fix was to clone the default community user profile, grant that profile the necessary permissions, and then use that profile for my community users. Who then could not log in! Oh, yes, even though everything about the default profile was cloned, the new profile still needed to be granted access to the community using the Community Administration view.

Granting access to a new profile in the Community Administration view.
Granting access to a new profile in the Community Administration view.

And, finally, the workflow could flow to the end.

Lightning Community Workflow Complete View
Lightning Community Workflow Complete View

Originally published at InfoWorld


From Agile to Fragile in 60 sprints

The adoption of agile software development methodologies has been a necessary evolution to support the explosive demand for new and expanded capabilities.   There is no doubt that without the broad adoption of agile practices much of the growth in technology, and all of those aspects of everyday life that is driven by technology, simply would not have happened.

Still, too much of a good thing applies. Another old adage that comes to mind is “You can have it better, cheaper, faster. Pick any two”. Many organizations have insisted on all three. How did they do it? They sacrificed the documentation.

I’m not talking about saving shipping costs and trees by making manuals virtual, and then saving bandwidth by replacing the documents download with the install files with links to online documentation (which has its own issues in this world of massive M&A). I’m talking about all those wonderful references that development teams, sometimes backed by technical writers, produced so that others may pick up where they left off to maintain and enhance the final applications. Yes, that documentation.

Self-Documenting Code does not make a Self-Documenting Solution

While no one can honestly disagree with the value put forth in the Manifesto for Agile Software Development : “Working software over comprehensive documentation”, I also don’t think the intention was that documentation impedes working software.   Still, the manifesto has fed the meme (the original definition, not the funny GIFs) “Good code is self-documenting”. When I hear this, my response is “True; and knowing what code to read for a given issue or enhancement requires documentation”.  My response lacks the desired impact for two reasons: It doesn’t easily fit on a bumper sticker and it requires putting time and effort into a task that many people do not like to do.

The danger of little or no documentation is that the application becomes dependent on “tribal knowledge”. In a perfect enterprise, this is a dependable approach because employee turnover is low and when people do depart they always do so with adequate notice and thoroughly train their replacements. I have heard these enterprises exist, though I have never spent any time working with one of them.  I did, however, recently work with a business intelligence group where their entire ETL staff departed within a few weeks of each other after a few years of furiously building hundreds of data integrations in a dozen different business areas and then spent less than 9 hours in “knowledge transfer” sessions with my team who were tasked with keeping the lights on until a new crew was hired and trained. There was not one page of documentation at the start of the knowledge transfer and I have yet to find a line of documentation in any of the code.

I’m not advocating the need for waterfall-style detailed design documents. In some ways, those can be worse than no documentation because they are written before the code and configurations they are intended to describe are created and fail to be updated when the actual implementation deviates. In an agile world, writing the documentation after the implementation is a sound approach that will support the manifesto value of “Working software over comprehensive documentation” by being just enough documentation to facilitate maintaining the software in the future.

Meeting between the Lines

How much is just enough? That is going to vary by both application (and/or system) and enterprise. Some applications are so simple that documentation in the code to supplement the “self-documenting” style is sufficient. More complex solutions will need documentation to describe things from different aspects, and the number of aspects is effected by whether maintenance is done by the development team or a separate production support group. The litmus test for whether your documentation is adequate is to take a look at it from the perspective of someone who has never heard of your application and needs to be productive in maintaining or enhancing it in less than a day. If you have difficulty in adopting that point of view (many people do, and double as many developers), have someone outside your team review the documentation.

I find the following types of documents to be a minimum to ensure that a system can be properly managed once released to production:

  • Logical System Architecture
  • Physical System Architecture
  • Component Relation Diagrams
  • Deployment Procedures

Again, the level of detail and need for additional documentation is going to be driven by complexity and experience. Another factor is how common the relevant skills are. If the candidate pool for a particular platform or framework is shallow, more detail should be provided to act as springboard for people that may be learning the technology in general while diving into the particular implementation.

Yes, there are Exceptions

Conversely, some solutions are true one-offs that are filling a very specialized need that is unlikely to evolve and may have a short lifespan. These implementations only really need sufficient reference to migrate them to another environment or decommission them to free up resources while not negatively impacting other systems. I do caution you to really be sure that an application falls into this category before deciding to minimize the documentation.  What comes to my mind when I think of such decisions is massive amount of resources dedicated to dealing with two-digit years in 1999 to address applications that were not expected to still be in use when they were developed 10 or 20 years previously.

A Final Appeal

At the beginning I agreed with the manifesto value of working code prioritized over comprehensive documentations. In the days when most software life cycles began with tons of documentation and meetings to review the documents and meetings to review the results of the review, a great deal more beneficial build and test activities could have been done in that time instead. My experience in documenting the results of agile and other iterative processes toward the end of the development cycle and then reviewing that documentation with people outside the team is that design flaws are discovered when looking at the solution as a whole rather than the implications to individual stories in a sprint. The broader perspective that waterfall tried to create (and often failed since most waterfall documentation does not match the final implementation) can be achieved better, cheaper and faster when documenting at the end of the epic. In this one case, picking cheaper and faster yields better.

Documenting the fruits of your software and application implementation labors may not be the most exciting part of your team’s work, but the results of not documenting can become the most painful experience for those that follow…or your next gig!

Originally published at InfoWorld


5 easy steps to install custom components in Salesforce Trailhead playground orgs

Salesforce Trailhead trainings are a great way to learn Salesforce. Some of the Hands-on Challenges require installing components. If you are using a Developer org to run these, the instructions are easy to follow. However, if you are using Trailhead Playground org, it is kind of a pain to install some components. There is a link thoughtfully provided by Salesforce for how to do this with eleven steps that I find a bit too time-consuming and confusing. I have found a slightly different approach that seems (at least to me) simpler. I will leave it to you decide which you prefer.

The instructions for installing the component (as in the screenshot below)will often be provided well before the challenge and the hint to avoid the frustration of trying to log into a Playground org when prompted by the standard component installation URL.

Example Trailehead Package Insturction
Example Trailhead Package Instruction

Step 1: Decide for yourself whether you will read through the full lesson or skip right to the challenge and when you get to the challenge, open your Trailhead Playground org in a new window by right-clicking on the Launch button (as pictured below).

Open your Trailhead Playground org in a new window by right-clicking on the Launch button
Open your Trailhead Playground org in a new window by right-clicking on the Launch button

Step 2: Log in to your Trailhead Playground org.

Step 3: Go back to the lesson screen and copy the component installation URL without the domain, i.e., “packaging/installPackage.apexp?p0=04tj0000001mMYP”

Copy the component installation URL without the domain
Copy the component installation URL without the domain

In some cases the installation instructions will have a link without the URL on the page. In this case, right-click on the link and copy the target to get the path, pasting it into a text editor to extract the portion following the domain.

Right-click on the link and copy the target to get the path
Right-click on the link and copy the target to get the path

Step 4: With the package path in your clipboard, paste it after the domain name of your Playground in the window you have already logged into and press enter.

With the package path in your clipboard, paste it after the domain name of your Playground in the window you have already logged into and press enter.
With the package path in your clipboard, paste it after the domain name of your Playground in the window you have already logged into and press enter.

Step 5: Once the installation screen will come up, and you can continue as instructed in the Trailhead lesson.

Once the installation screen will come up, and you can continue as instructed in the Trailhead lesson.
Once the installation screen will come up, and you can continue as instructed in the Trailhead lesson.

I have tested this on both Chrome and Firefox running in Windows 7. Your results may vary with a different combination of browsers and O/S.

Originally published at InfoWorld

01/27/2018 Update: The new Salesforce Trailhead UI may take you to the log in page. No worries. Copy the login URL and strip the characters before and including “startURL=”, past into, and strip all of the characters including and following the first “&”. to get the package URL.


Finding and Fixing the Cause of 0 Byte Files from FTP Source Events in ICRT

Ran into this recently and wanted to share it with my loyal reader… An Informatica Cloud Application Integration solution that worked wonderfully through months of UAT and flawlessly the first few weeks in production suddenly produced complaints from stake holders because the files output as the end result of the process were empty.

Looking at the process logs, there were no errors shown. Every technologist’s favorite finding (sarcasm, for those not blooded in debugging production issues).

The application functionality is relatively simple: Listen for new files in a remote folder via SFTP, download the files and rename them for processing by a Data Integration Mapping Configuration Task then run the MCT. I’ll spare you the details of trying to get the details of the missing files except that once they were obtained the pattern I noticed was that the missing files were the largest in the brief history of the application. Not much more deduction required to figure out that in the source environment the files were being written directly to the outgoing folder rather than being created elsewhere and dropped in when complete. The larger files took longer to write than the polling interval so they were being brought down in name only (pun intended).

It took a while of reading and re-reading the documentation for the FTP connector to determine that the best strategy for this issue was to use the File Read Lock Settings for the event source, setting the Read Lock value to “changed” and running tests to the get the correct values for the other File Read Lock Settings.

ICRT FTP File Read Lock Settings

Problem solved!


The Differences between IT Consultants and Contractors

I try to post original content. Sometimes that originality may only be in the presentation of the information, in which case I am attempting to provide (I hope) a clearer understanding or a simpler approach. Because of this personal rule of conduct, I first researched this topic to which I have thought and spoke about for quite some time and was very surprised at what I found. What is already out there on the subject of comparing contractor and consultant roles is sometimes contradictory and has some distinctions that I think are based on only thinking about individuals rather than encompassing companies that provide both services as well. Rather than argue the points others have made (which I don’t necessarily disagree with in certain, specific contexts) I will present my thoughts and experience and leave it to you if you wish to research further.

What’s the Difference?

In short, the basic difference between the two is simple: A contractor is an individual who possesses a specific skill set that they will utilize to your specification, where a consultant is an individual who has experience with developing a solution within a domain where you need assistance.

The basic difference is also inadequate to understanding which one you need for a given project (or aspect of a project) and how to work with them to your best advantage, so let’s dive a little deeper into the more subtle differences.

While you may work with both as individuals it is more common to work with them in groups. A group of consultants will be a team assembled on your behalf by a consulting company (AKA partner, group, professional services provider, etc.) and should be self-managing. A group of contractors may come from the same agency but will require management (which may also be contracted).

Consultants can help you define the problem and work with you to develop a plan to get from current state to target state.  Frequently they also perform and/or manage the tasks and deliverables of the plan. Consultants can direct contractors to execute to the plan, and will often provide those contractors as well.

Another difference is that for a contractor to be valuable, they must be deeply familiar with a specific aspect of the project, where consultants need only be familiar with the general domain of the project. One of the best reasons for engaging consultants is their proven ability to navigate through the unknown.

Working with Consultants vs. Contractors

One difference not included above is cost. There are many different fee structures for either, though they can all be broken down (for the sake of comparison) to cost per hour of effort. Consultants are almost always a higher hourly cost. The difference is usually reflected in the value provided during that time, meaning that you will get more benefit for each hour of consulting. They key word in the previous sentence is usually.

There are two common scenarios where the value is not always higher with consultants.  The first is when it is the wrong consultant.  The wrong consultant can be engaged for any number of reasons, and once this is determined than it should be corrected. This, however, is not the most common reason for missing out on the full value of a consultant.

The most common reason for not realizing the maximum value of a consultant or consulting team is working with them like they are contractors. Consultants should be actively involved at all levels of the project. During requirements definition they can provide their experience with what similar projects have missed including early on, and help determine prioritization through an understanding of the effort involved in delivering a requirement. Consultants will be able to apply experience in planning, knowing what tasks can be done in parallel to support timelines and where risks are most likely to occur along with mitigation approaches. Once the delivery phase has begun, consultants will recognize issues and opportunities during regular reviews that might go unnoticed by those who have not done similar projects in the past. Every consulting company I have worked with has a project management practice, and if it is a team of consultants engaged on a project it will generally yield the most value if the part of that team is a project manager who will, among other contributions, help the client to realize the maximum benefit of working with the consulting team.

Having one or more consultants on a project and then tasking them the same way as contractors is like rowing a power boat. It can still get from one place to another, but the boat is under-utilized, the journey will take more work than required, and it will not be nearly as much fun!

Which is Best for Your Project?

If your project involves technologies that your enterprise is already comforably familiar with and you just need more hours in the day, contractors should fill the need nicely. You may be implementing a larger project where an isolated area is outside of your experience and a contractor can fill that gap and train your people on how to maintain it afterwards. Or the project you are working on is scaling out your technical landscape and you will need to keep on someone afterwards for maintenance, so contracting can be a “try before you buy” approach to determine the right candidate.

If there is a concern about whether the project is the right thing to do or the technologies are the right ones to use, consultants can bring experience and a fresh viewpoint to increase confidence. If a project will introduce more than one or two completely new aspects to the enterprise, engaging a consultant should certainly be considered. The nature of consulting makes them familiar and comfortable with the unknown. For many organizations, internal teams need to be more focused on the day-to-day operations and introducing change to the technical landscape can be better served by professionals for whom change is the day-to-day operation.


A Quick Tutorial to Migrate Informatica Cloud ICS Objects between Orgs

Screenshots with captions:

Administer > Migrate Objects
In the Target Org, From the Administer Menu, Select Migrate Objects
Start Migration
On the Migrate Objects Page Click Start Migration
Log into Source Org as Admin
Click the Log In… Button
Enter Credentials when Prompted
Enter credentials for an Administrator Account in the Source Org
Click Add Objects
Once logged in you can click the Add Objects button
Select Objects to Migrate.

Save time by selecting objects at the top of  related hierarchies, such as Task Flows will automatically select the tasks, and any objects required to support the task (such as Mappings and Connections) where selecting an items lower in the hierarchy (such as Connections) will not automatically select their parents. Close the dialog by clicking the OK button (not pictured).

Back on the Migrate Objects Page Click the OK Button
Choose Carefully Whether to Overwrite or Not

If you are migrating updated objects you will see the prompt above. You may wish to rename the existing objects before migrating. You may also want to delete the existing objects as the Overwrite behavior may not be what you expect.

Once the migration is complete, you will need to review your objects to confirm or correct any Org-specific values such as the Secure Agent name or credentials for Connections that require them.