How Does Technical Debt Impact Salesforce Orgs?

How Does Technical Debt
Impact Salesforce Orgs?

With many Salesforce Orgs hitting the 10-year mark, the crippling effects of technical debt are becoming more common. Technical debt can result in slow performance, reduced agility, sluggish adoption, and the failure to meet business requirements. This complex problem is leaving Salesforce admins and IT executives alike wondering what to do.

To help combat the situation, Salesforce Premium Support now offers services geared towards tackling technical debt. As part of this, Salesforce recommends that customers devote a percentage of each release cycle to clean-up work. But while everyone agrees that technical debt is bad, there hasn’t been much discussion regarding exactly how technical debt reduces Org performance and agility. Without understanding this, it is difficult to know how to fix the problem or avoid it in the first place.

In software development, technical debt is often defined as the cost of choosing an easy solution now instead of a better approach that might take longer. Salesforce analysts like to associate technical debt with the complexity that results from extensive customization. The argument is basically that customizations cause complexity, and that results in technical debt which reduces Org agility.

At Metazoa, we don’t think that either of these definitions explain what is happening. First of all, Salesforce Orgs need to be highly customized in order to address important business requirements. Complex Salesforce Orgs can work just fine if they are properly managed. And secondly, we have found that the loss of agility and performance in older Salesforce Orgs can often be traced to specific malfunctions that can be fixed. We do not think the problem is too many customizations, but rather vital systems in the Org that have become unhealthy.

When customers come to us with technical debt issues, they often think that the only option is to clone the Org and start over. In many cases, common developer tools will no longer work on their Org, and various Salesforce APIs fail to respond in a timely fashion. These customers cannot transition to Lightning Experience or take advantage of other new Salesforce products. However, instead of starting over, the first step in removing technical debt is to identify which systems need repair.

Org Visualization

To get a good sense of where the problems are, you must first get an asset count for each metadata type. The List Metadata API can do this rather easily. However, this API will only return the first 100,000 assets. If you exceed that limit, you will need to get smarter and split metadata types across API calls to get the desired results.

Once you have an asset count for each metadata type, you then need to take a comprehensive metadata snapshot of the Org. This can be difficult because the Retrieve Metadata API has some governor limits. By conducting multiple retrieves in parallel, and stitching the results together, millions of assets can be assembled. This process can take some time because of Org size and API response times.

Once an overview of all the Org metadata is available, many problems will become apparent. Why are there 4000 inactive Assignment Rules? How did we end up with 7000 Groups? During this process, there will be business value judgments involved, not just technical issues. For example, you might need to consider how the Org should be structured to achieve corporate requirements.

To answer these questions, reports can be assembled to help administrators visualize complexity and understand where the bottlenecks are. Metazoa provides some great free reports on the AppExchange that can help, including Profiles and Permissions, Field and Picklist Usage, and the Data Dictionary Report. You can also create your own reports with the Metadata API and the Tooling API. Compliance reports can be used on the front end to identify the problems and on the back end to document the changes to the Org.

Profiles

One of the most serious problems is gigantic amounts of Profile data. Let’s say you have an Org with 500 Profiles and 1,000 Objects and 500 fields per Object. This Org will have 250 million field level security (FLS) permissions. Each one of these takes up about 250 bytes of metadata. If you download the profiles, the file size will be approximately 60 GB! That’s a ton of data, and it shuts down most developer tools like Salesforce DX, Visual Studio Code, and Workbench.

The data size is not the only problem. The Profile page in the Setup menu will take several minutes to load, and when the page finally does load there will be an overwhelming number of permissions displayed which introduces security compliance delays in addition to implementation issues. The best practices for resolving these issues include:

  • Evaluate aggregate org security permissions
  • Identify and remove underutilized Fields
  • Identify and merge similar Profiles
  • Transition to Permission Sets

Permission Sets

Each assignment between a Permission Set and a User is stored as an internal record in Salesforce. These assignments control access to Objects, Fields, Apex Classes, etc. If there are many Users and many Permission Sets, then the total number of assignments can become very large. There can be great difficulty documenting the combined effect of all these permissions for an individual User. Editing all these assignments through the Setup Menu can be tedious and error prone.

When you assign Permission Sets to a Permission Set Group, there is a background process that calculates the new aggregate permissions for the group. This recalculation process can cause performance problems and delays in Orgs with lots of active assignments. The best practices for resolving these issues include:

  • Evaluate combined security permissions by User
  • Identify and merge similar Permission Sets
  • Reduce the number of assignments with Permission Set Groups

Unused Assets

There are several different metadata asset types that can end up being left behind because of Org configuration settings. For example, maybe the asset is not referenced by any other asset. This can happen with Groups, Queues, and Email Templates. Perhaps the asset is not assigned to any user. This can happen with Profiles, Permission Sets, and Roles. In addition, assets can be forgotten because they were deactivated or hidden. This can happen with Workflows, Assignment Rules, Approval Processes, and Flows.

There are analogous problems on the data side, where Fields and Picklist Values can be underutilized. Reports and Dashboards can also go stale. Admins should take advantage of the compliance information that Salesforce provides for each Custom Field. This information includes the Description, Help Text, Business Status, Security Classification, and Compliance Group. Special attention should be given to the date the asset was created and last modified, and which admins were involved. Deleting all these unused assets is a great first step towards Org cleanup and optimization!

Inactive Users

Once a User is created, they can be made inactive, but they cannot be deleted. Over time, Orgs can end up with more inactive users than active ones. There are over 50 different ways that an inactive user can remain connected to a Salesforce Org. Inactive users can still manage active users, receive system emails, remain team members, and serve as the running user for Dashboards and other important enterprise systems.

These zombie users complicate Org management and software development. They also pose multiple security threats. Tracking down and removing all the connections between inactive users and the Salesforce Org can be a very complicated process. The connections can be by username, user email, and user ID. The best practice is to identify these Users and either delete their connections to the Org or replace them with an active user.

Record-Level Security

Another common problem within Salesforce Orgs is simply having too many assets of some metadata type. We once worked with a customer that had over 100,000 User Roles. Sometimes having that many Roles is unavoidable due to Portal or Experience Cloud setup, but other times it’s just technical debt.

When the Role hierarchy changes, this triggers a recalculation of Record-Level Security. As you can imagine, this customer’s Org suffered from massive performance issues due to constant security recalculations. In this case, the answer was cleaning up the Role hierarchy and then reassigning all Users to the correct Role. Salesforce admins should pay special attention to changes that trigger Record-Level Security calculations. These include:

  • Role Hierarchy
  • Territory Hierarchy
  • Sharing Rules
  • Team Membership
  • Manual Sharing
  • Programmatic Sharing
  • Groups and Queues

Second Generation Packages

Older applications are often unpackaged, and they tend to directly customize various aspects of the Org and add lots of unnecessary complexity to it. To remedy this, release managers should dedicate some time in each release cycle to packaging these legacy projects. Packages encapsulate functionality, isolate bugs, simplify releases, improve testing, and provide a foundation for agile development.

The great thing about managed packages is that they add an additional layer of separation with namespaces. Unmanaged packages can accomplish this as well with strong naming conventions. Developing packaged applications is the wave of the future, but even the most advanced DevOps practice can’t succeed if the Org is saddled with technical debt.

The Healthy Org

An analogy with healthcare is helpful. If your doctor gave you and MRI, and said that there were many connections in your brain, and that your brain was quite complex, would you think that was a medical problem? Or a sign of intelligence? On the other hand, if the doctor said some of your internal organs were abnormally large, or that your white blood cell count was too high, then you would be worried. At Metazoa, we have analyzed hundreds of Orgs, and we have found that the Orgs with technical debt tend to be less complex than healthy Orgs. They are dead in the water, not able to adopt new Salesforce features, and in need of repair. The conventional wisdom that technical debt is caused by too many customizations is simply wrong.

Conclusion

Salesforce has provided many powerful options for customizing an Org and solving important business problems. But without proper management, a dynamic Salesforce Org can become crippled with technical debt over time. But don’t worry, technical debt is a problem that can be fixed. Fortunately, there are some powerful tools available that can both find technical debt and fix the problems as well. Please check out our Org management product Snapshot on the AppExchange if you need to tackle technical debt!


The New Lightning Security Report

The Lightning Experience brings modern web technologies into the Salesforce user interface. You can override many aspects of the Lightning Experience with Visualforce Pages, Lightning Components, and Flexipages. There are multiple levels of override, so the Lightning Experience can be customized extensively. But all of these customizations can be difficult to keep track of. For example, when a particular User clicks on a certain Object, what Page will be Viewed? These customizations are also notoriously difficult to deploy between Orgs. To address these issues, Snapshot has introduced a new Lightning Security Report with Smart Deploy. This report will give you an overview of all Lightning customizations, allow you to quickly edit these customizations, and then easily deploy the changes to your Org.

Visualforce Pages in Salesforce Classic

In the classic version of Salesforce, Visualforce Pages (Apex Pages) could be used to override various Actions in the user interface. The picture below shows the Account object in the Setup Menu. As you can see, the Edit and View Actions have been overridden. These Action Overrides are stored in the Custom Object.


Introducing Lightning Components

Later on, Salesforce extended the Custom Object Action Override to also include Lightning Components (Aura Bundles). The View, Tab, New, and Edit Actions can be overridden with Lightning Components. Lightning Components are used in small form factors like Mobile Phones or large form factors like the Desktop Experience. If you do not specify a Lightning Component, then the Mobile and Desktop Lightning interface will use the classic Visualforce Page.


Here Come Flexipages

When the Lightning Experience was introduced, Salesforce need a responsive replacement for the classic Page Layout. Flexipages are the new solution for customizing Lightning Experience pages. You can launch a Flexipage Tab Action when the user clicks on the Home Page for any application. Flexipages can also be launched for the View Action when a particular Record is displayed. You can specify different Flexipages for each Custom Application, Record Type, and Profile combination. Because of this, the Action Override information for Flexipages is stored in the Custom Application.

Let’s look at the Tab Action first. The Tab Action controls what Page is displayed when a particular User clicks on the Home Tab. The default option for the Home Page is the Custom Tab called standard-home. A Flexipage of type HomePage can be assigned to override this tab. The Flexipage can be assigned as the Org Default, an Application Default, or as a combination of Application and Profile. The Activation Dialog below in Lightning App Builder shows the assignment options. Note that Org Default can be overridden by App Default which can be overridden by App and Profile.

All of this pertains to overriding the Home Page Tab. If you want to override a Custom Object Tab, then you can do that by editing the Tab Action for the given Custom Object. This is a bit confusing, because there are two different Tabs involved. The Home Page Tab is associated with the Custom Application, and the Custom Object Tab is associated with the Custom Object.

Now let’s take a look at the View Action. The View Action controls what screen is displayed when a particular User attempts to view an Object. A Flexipage of type RecordPage can be assigned as the Org Default, an Application Default, or as a combination of Application, Record Type, and Profile. The Activation Dialog below in Lightning App Builder shows the assignment options. Note that Org Default can be overridden by App Default which can be overridden by App, Record Type, and Profile.

The Org Default Flexipage will override any Visualforce Page or Lightning Page that is currently defined for the View Action. The App Default will override this Flexipage for a specific Custom Application. The App, Record Type, and Profile combination will override all other Pages when the Object is Viewed.

Overwhelming Overrides

While Lightning Experience provides great customization options, managing this complexity is mind bending. For example, when a particular User clicks on a certain Object, what Page will be Viewed? In order to answer this question, you would need to understand the five levels of overrides that are involved:

  • Classic Visualforce Page
  • Lightning Component
  • Org Default Flexipage
  • Application Default Flexipage
  • Application, Record Type, and Profile Flexipage

Detail View

The new Lightning Security Report clarifies this confusing situation. The detail view displays all of your action overrides on a single page. The reports are divided by Form Factor:

  • All (Large and Small)
  • Lightning Desktop (Large)
  • Mobile Phone (Small)

The report shows the Override Type sorted by order of appearance in the user interface. The Override Source column shows the name of the Metadata Asset that contains the override information. This is useful when these assets are deployed. The Content Type and Content Source show the actual Visualforce Page, Lightning Page, or Flexipage that will be seen in the user interface. The Record Type column shows the associated Record Type if any. Sometimes this column just has a Custom Object name for the App Default override. When there is a Tab Action, this column has the Tab name standard-home. The Profile Name column shows the Profile Name. The last column has the Action Name and Form Factor. Right click the table to toggle the visibility of various characteristics. You can Show Labels to include all the various labels in the report.

The report is color coded by the type of object that contains the override information. This also relates to the type of override. Here is a legend for reading the report:

  • White – Custom Tab – Org Default
  • Green – Custom Object – Page Component
  • Yellow – Custom Application – Application Default
  • Red – Custom Application – App, Record Type, and Profile

Summary View

The last part of the report list provides summary tables for the View Action and Tab Action by Form Factor. The Minimum, Medium, and Maximum options control the amount of information in the table. There is no Tab Action for the Small Form Factor, so that option is not included. The summary report gives you a three-dimensional view with Custom Applications and Profiles down the left and Record Types across the top. This provides all of the information about what View will actually be displayed when a particular User clicks on any Object. There are catch-all areas for Other Profiles and Other Applications that are not explicitly controlled by an override.

Summary Tab

The last part of the report list shows a summary view of all the Tab Actions. As mentioned earlier, there are two kinds of Tab Actions. The Home Page Tab Actions occupy one column of the report. They depend on the Custom Application and Profile settings. The other columns will show the Tab Action for particular Custom Objects. This is controlled by the Visualforce Page, Lightning Page, or Flexipage that has been assigned to the Custom Object. This information does not depend on Custom Application or Profile.

Editing Assignments

At the upper left of the Lightning Security Report there is a button to hide and show the Editing Palette. When the Palette is visible, you can select cells from the report table. The Palette allows you to Add, Edit, Duplicate or Remove the selected assignment. This will launch the Assignment Dialog, shown below:

You can edit all of the different characteristics from the selected action override and create new overrides of any type. When some menu items are selected, other menu items will change to make sure that your action override will validate when deployed. When you are done editing, click the OK button and return to the main report interface. Now you will see the Deploy button, available below:

Click the Deploy button to launch the Smart Deploy interface. This interface will show the Metadata Assets that you have changed in the Create Job List. These assets might include Custom Tabs, Custom Objects, and Custom Applications. You can make any changes that are needed to the Create Job List or other deployment parameters. The last tab of the Smart Deploy interface allows you to Deploy these changes to the destination org.

Smart Deployment

Action overrides can be difficult to deploy between orgs. There can be hundreds of action overrides stored in a single Custom Application. When you deploy the Custom Application, all of the action overrides on the destination are overwritten. Unlike Profiles, Custom Applications do not have child types. The new Smart Deploy interface in the Lightning Security Report makes the deployment of action overrides much simpler. You can view the entire configuration of action overrides on a single screen, make changes quickly, and deploy the changes without leaving the report or editing XML.

Conclusion

The new Lightning Security Report is another great way to understand the complexity in your Salesforce org and make changes when needed. Keep reducing that technical debt and your org will be more agile than ever! Let us know how the Lightning Security Report is working for you.

 

Bill Appleton
CTO Metazoa


License Expense Management with Snapshot

Snapshot has a new report that covers Organization, Feature, Package, and Permission Set Licenses. The actual usage and cost of each license type can be compared. This information will give you a bird’s eye view of all your license expenditures and areas where usage is high, medium, or low. You can also discover Users that have the Permission to do something but not the License, or the License to do something but not the Permission. This information can help identify permission problems and also pinpoint licenses that are not being used or cannot be used. Let’s dive into the new License Expense Management report!

Selecting Users

The first tab greatly simplifies the visualization and assignment of User Licenses. First, you will need to select a group of Users that you are interested in. The Select Users button at upper left will bring up the familiar User Selection dialog. There are many different ways to select Users. Note that there is an ability to search by Last Login Date. This option is useful to find Users that have not logged in recently and evaluate their license usage.

License Assignment

Back on the main screen, you can select any one of the Users at left. You will see all of their License Assignments in the four lists at right. These lists show Organization, Package, Feature, and Permission Set Licenses. The Organization Licenses are just the regular Salesforce licenses that you have purchased for Standard Users, Chatter Users, etc. The Organization Licenses must be associated with a Profile. Essentially a Profile is an instance of a Salesforce License. Assigning an Organization License is the same thing as assigning a Profile to the User. Because of this, the Organization Licenses show both the License Name and the Profile Name.

A Feature License entitles a User to access an additional feature that is not included with their User License, such as Marketing or Work.com. Users can be assigned any number of Feature Licenses. Permission Set Licenses grant more granular control because the User needs both a Permission Set License which gives them the license and a Permission Set which gives them the permission to use a certain feature. There’s validation involved when assigning a Permission Set to a User.

The Package Licenses are related to the Salesforce License Management Application and control access to various Managed Packages. Partner products (such as our beloved Snapshot application) are available on the AppExchange and carry a Package License. We also display the Package Namespace to simplify the identification of the Managed Package.

The License Assignment interface is mainly used to check and change User Licenses. You can also click one of the five radio buttons on the screen to focus that license type. For example, if you click the Feature Licenses radio button, and then select a specific Feature License, you will see all of the Users that have access to that Feature. You can check and uncheck the checkboxes next to each User and click the Save Assignments button to change multiple Users at the same time

.

Managing Expenses

You can right click any report and select the option to Manage Expense. There is also a button to Manage Expenses on the Display Report Tab. This dialog allows you to edit information about the expense and usage of each license. There is a picture of the Manage Expenses interface, below.

The cost and usage information in this dialog works in a hierarchy. When you set the cost or usage information for any item that will also define the cost or usage information for all of the child items as well. For example, in the picture below, the Snapshot Package License is being edited. Because this is a leaf node, these settings will override all parent settings. The items that define cost or usage information are shown in bold.

The default settings for this dialog define User Logins as the default usage statistic for All Licenses, and $1800 as the default cost for all Organization Licenses. These choices are there to get you started with some cost and usage information.

The Manage Expense interface allows you to select a proxy value as a usage statistic. The number of Salesforce logins is a basic statistic for usage. In some cases, you can select a more detailed proxy for usage based on the number of created or modified records. In the expense reports, usage is color coded as red, yellow, and green. The Usage Limits button above will calculate low and high usage numbers that attempt to evenly divide the report into red, yellow, and green areas. These numbers are inserted into the text fields above the button and can be edited thereafter.

License Reporting

Things get interesting when you start looking at the License Reports. The first four reports are:

  • Feature Licenses
  • Package Licenses
  • Organization Licenses
  • Permission Set Licenses

These reports show a rollup of all the information about the given license type for the current Org. This information is useful for compliance and security. License changes can be documented over time. The next five reports include information about license expense and usage:

  • Feature Expenses
  • Package Expenses
  • Organization Expenses
  • Permission Set Expenses
  • Combined License Expenses

The first four expense reports have detailed information on each license type. The Combined License Expenses report combines all of this information into a single table. The Combined License Expenses report is a complete view of all the expense and usage for a given set of Salesforce Users. This report displays the selected Users down the left-hand side and each of the available license types across the top. Each column displays the expense and usage for a particular license. The last row is especially important. This row shows the total cost of licenses that have low usage.

Permissions Vs. Licenses

The next three reports compare the assigned licenses and the assigned permissions for the selected group of Users. The Combined Licenses report shows which Users have which Permission Set Licenses in a nice array. The Combined Permissions report shows which Users have which Permission Sets. Lastly, the Combined Differences report compares these two arrays. This report shows which Users that have the permission to do something but not the license, or the license to do something but not the permission. This is vital information because if a User has the license but not the permission, then they cannot actually use the license, and this could point to wasted money. Likewise, Users with the permission but not the license may have an extraneous permission, or need a license assigned to them. For example, in the picture below, [email protected] has the license for CampaignPermission2 but not the matching permission.

In order to troubleshoot problems like this, select any affected Username from the final section of reports. You will see their base Profile at the top of the report followed by all of their subsequently assigned Permission Set Licenses below that. At the bottom of the report, you will see their base Profile followed by all of their assigned Permission Sets above that. In the middle of the report, you will see rows for the Combined Licenses and Combined Permissions with the Combined Differences in the middle. This report makes it easy to spot where the imbalance between licenses and permissions is happening.

In this case, the User was assigned the CRMUserPsl license but not the corresponding permission. They will not be able to actually use the license until this situation is corrected. You could give this User a new license on the first tab. If you want to change User permissions, Snapshot offers a variety of ways to do that. The User Permission Assignment dialog has easy to use tools for changing User permissions.

Conclusion

There you have it. The new License Expense Management report can rapidly assign Organization, Package, Feature and Permission Set Licenses to any group of Users. This report can provide a bird’s eye rollup of all your license expense and usage information. Lastly, the report can identify imbalances between the permissions and licenses for your Salesforce Users. This information can potentially save you thousands of dollars managing your Salesforce implementation!


Introducing Metazoa Smart Deploy

DevOps is focused on moving lots of assets quickly between different Salesforce Orgs. DevOps does a good job of supporting Salesforce Developers working in continuous integration environments. Developers work with a handful of Metadata Types such as Apex Classes, Lightning Pages, and Custom Objects. Developers tend to be focused on the application they are building or the code they are writing.

But Salesforce Administrators have a broader responsibility to manage the release cycle, customize the Salesforce CRM product, and implement corporate policies. They need to look at all 250 Metadata Types in order to manage the Org effectively. Administrators focus on the content being deployed rather than the number of deployments. Technical debt, compliance, optimization, cleanup, and security are major parts of an Administrators job.

The Metazoa Snapshot product has always had great deployment tools on the one hand, and dozens of vital reports on the other, but these capabilities were somewhat isolated. Now with the introduction of Smart Deploy, busy Administrators can see problems in a report and then instantly launch deployment tools to resolve the issue. This is a game-changing capability because now the visualization of Org content is directly coupled with state-of-the-art deployment tools. This is a new kind of deployment technology focused on Org Management.

Field Usage Report

Take for example the Field Usage Report. This report will analyze any selected Object and generate comprehensive information on Usage for each field. The number of empty values, the number of distinct values, the field creation and modification dates, and all compliance information are displayed. Now with Smart Deploy you can edit compliance information for any field. Simply click the “Edit Compliance” button to get started. For example, you might mark an unused field as a Candidate for Deprecation.

On the Display Report tab, you will see all of the selected Objects and the used and unused fields. If you click the ‘Smart Deploy” button at upper right the deployment interface is launched with all of the unused fields already added to the Delete Job List. You can select which Custom Fields to delete from the current Org, or just save the Job List for later deployment.

The ability to generate the Delete Job List is available for the Field Usage Report, Last Activity Date Report, and the Forgotten Assets Report. Here is a link to a movie of Smart Deploy in action for the Field Usage Report:

https://www.youtube.com/watch?v=RsNssFC5dN8

Forgotten Assets Report

The Forgotten Assets Report now contains an extremely powerful Smart Deploy interface. This report allows you to select dozens of Metadata Types and discover which ones are inactive, disconnected, no longer in use, or not properly configured. The Display Report tab shows all of these forgotten assets. Click the “Smart Deploy” button, and you are instantly in the deployment interface with these assets added to the Delete Job List. You can decide which assets actually need to be deleted at that point.

Here is a link to a movie of Smart Deploy in action for the Forgotten Assets Report:

https://www.youtube.com/watch?v=qKJFMCWKxUE&t=31s

View Profiles

Let’s take a look at one more place where Smart Deploy comes in handy. The View Profiles and Permission Sets interface has powerful tools that enable the bulk editing of Profiles and Permission Sets. You can edit the permissions for Apex Class Accesses, Apex Page Accesses, Application Visibility, Custom Metadata, External Data Sources, Field Permissions, Flow Accesses, Layout Assignments, Object Permissions, Record Type Visibility, Tab Visibility, User Permissions and Custom Permissions. Now with Smart Deploy, you can instantly deploy these changes as well. After editing various permissions, click the “Smart Deploy” button and the permissions that you changed will be automatically added to a Create Job List when the deployment interface is launched.

Here is a movie that shows how Profile Editing is enhanced with Smart Deploy:

https://www.youtube.com/watch?v=62TfQWf1tLs

Smart Deploy works a little differently when you are creating new assets instead of just deleting them. When you click the “Smart Deploy” button, a new Snapshot is created behind the scenes with all of your edits. That Snapshot will become the source for your deployment. The destination Snapshot will be the original one that you started with. This is very convenient because all of your edits are saved in the new current snapshot, but the original unchanged snapshot is still available as a backup. If you open the Manage Time Series interface you can target the backup or delete the new Snapshot, etc.

The first tab in the View Assets interface also supports Smart Deploy. You can navigate to this tab and create, delete, or otherwise modify any of the assets in the Snapshot. When you click the “Smart Deploy” button all of your edits are saved to the newly created Snapshot as discussed above. Look for more and more Metazoa Reports and interfaces to feature Smart Deployment capabilities in the future! We are working hard to make Org Management as easy and powerful as possible.

 

Bill Appleton
CTO Metazoa


Metazoa Cascade DevOps and User Stories

Metazoa has released a great new FREE application on the AppExchange! Cascade can deploy metadata between any Salesforce Org, local project folder, or remote Git Repository. Cascade also supports collaborative Sprints and User Stories that streamline Org Management for teams of developers and administrators. Unlike other solutions, that can take months to configure and implement, Cascade works immediately with any Org, there is no package to install. Cascade is a desktop application that communicates directly from your personal computer to your Salesforce account, and because of this, Cascade is an inherently secure solution that never compromises your Salesforce credentials, data, or metadata. Let’s take a look at Cascade!

Main Interface

When you launch Cascade the first time you will need to select an Installation Org. This is where Cascade finds administrators and developers for your team. Any user with administrative credentials will be available as a team member. Cascade also uses this Org to store deployment history, user stories, and other information. The first time you login, some custom objects with the prefix “metazoa” will be added to the Org. These objects are easy to remove if you ever need to uninstall Cascade.

Next, Cascade presents options to select a metadata source and a metadata destination. The source and the destination can be any Salesforce Org, local project folder, or Git Repository. Simple click the buttons to select a Salesforce Org or Developer Project.

Salesforce Orgs

If you select a Salesforce Org, Cascade prompts you for the org credentials. You can authenticate with any kind of Salesforce Org. Cascade has full support for Scratch Orgs, OAuth, Custom Domains, Developer Orgs, and Sandboxes. The next screen prompts you to Take a Snapshot. This will download all Org metadata to your client computer. Cascade can retrieve all of the metadata from even the largest Salesforce Orgs! If necessary, Cascade will conduct multiple retrieve operations simultaneously and reassemble all of the metadata at the client. Don’t worry, all of this is handled automatically for you!

Developer Projects

The other option is to retrieve source or destination metadata from a Developer Project. This option allows you to select either a local project folder or a remote repository. The folder or repository can be in either Metadata API, Eclipse, or Salesforce DX format. For example, if you have a Salesforce DX project on your computer, all you have to do is select that folder. If you have a Git Repository that you want to use, then enter the credentials to the repo and test the connection with the Clone Repository button. The next tab allows you to Take a Snapshot, and this will create a new metadata source or destination.

By the way, on the main screen there are some handy menu options under the source and destination buttons. These include:

  • Open Salesforce Browser
  • Open Local Project Folder
  • Open SFDX Command Line
  • Open Repository Browser
  • Import Metadata Snapshot
  • Export Metadata Snapshot

All of these options operate on the current source or destination. Cascade makes it super easy to work with your selected metadata! Once the source and destination are set up, you can start performing the main Cascade Activities listed in the center of the Welcome Screen. The first two options provide powerful capabilities for DevOps: deploying and comparing metadata.

Deploy Metadata

The Deploy Metadata interface allows you to move any number of metadata assets from the source to the destination. All 200 metadata types are supported. The basic idea here is to build up a Create and Delete Job List of the assets that you want to move. The user interface is very rich, and lots of information is displayed. For example, you can see if each asset is the same or different on the current source and destination. The XML or textual differences are also displayed. The picture below shows the process of building a Create Job List.

When you have all of your selected assets ready to go, the last screen presents state of the art deployment tools for moving the assets from source to destination. You can manage your job lists, import and export package.xml files, test code quality, run code coverage, transform XML data, remove bad references, and fix problems with sparse profiles. Remember, the source and destination can be a Salesforce Org, local project folder, or Git Repository in any file format. Because of this, the metadata deployment interface is an incredibly flexible way to move any type of metadata to any destination!

Compare Metadata

Cascade also provides an amazing XML comparison report. This is useful any time you want to be sure what the metadata source or destination look like, or what the differences are. You can select any metadata type and see what assets are the same, somewhat different, very different, missing on the source, or missing on the destination. You can click the checkbox to the left of any metadata type and it will be included in the Display Report which is available on the last tab. The Display Report can be exported as HTML, CSV, PDF, or in Native Excel format for distribution to team members. You can also archive these reports for compliance.

Sprints and User Stories

Cascade provides three additional capabilities that streamline Org Management for administrators and developers. These options are visible on the main screen: Manage Sprints, Update User Story, and Deploy User Stories.

Project managers can use the Manage Sprints interface to create new Sprints and assign out User Stories to team members. Team members such as Salesforce administrators and developers can use the Update User Stories interface to add manual setup tasks and metadata assets to their stories. Lastly, release managers can choose the Deploy User Stories interface to select multiple stories for deployment, merge the stories, compare them against the destination, and then begin the deployment process. Let’s take a look at the Sprint and User Story management capabilities in Cascade!

Manage Sprints

Project managers can use the Manage Sprints interface to create new Sprints and assign out User Stories to team members. There are options to create, manage, duplicate, and delete Sprints and User Stories. Every story is associated with a Kanban Stage. These stages help organize the sprint from beginning to end. The available Kanban Stages are:

  • Draft
  • Backlog
  • Requested
  • Developing
  • Waiting
  • Testing
  • Deployment
  • Finished
  • Archived

Every Sprint has options to broadcast email notifications to the owner of the User Story when things change. These notification help provide a collaborative backbone for the Sprint. The available notifications are:

  • Sprints Created or Deleted
  • User Story Created or Deleted
  • Sprint and User Story Deployment
  • User Story Kanban Stage Changes
  • Pull Request for User Story

Sprints can be managed in multiple ways. On the main screen, you can work with the selected Sprint as a linear To-Do list and filter by the current Kanban Stage.

Sprints can also be managed with a Kanban Board. The board shows team members down the left-hand side and Kanban Stages across the top. User Stories are shown as cards on the board. The total number of items in any column is displayed to prevent bottlenecks. You can drag and drop any User Story to reassign the story owner or change the Kanban Stage.

Sprints can also be managed as interactive Gantt Charts. This is a good option for longer and more complex projects where some User Stories depend on the completion of others. Each User Story is displayed as a task bar on the chart. The Gantt Chart will display the projected start and finish dates of all of the User Stories, as well as the percent complete.

Lastly, the Manage Sprints interface will automatically calculate a Burndown Graph for the entire Sprint. This shows the actual progress of the Sprint compared to linear progress. If the red area appears to the upper right of the diagonal line, that is a sign that the project is behind schedule. If the red area appears to the lower left of the diagonal line, that is a sign that the project is ahead of schedule.

Update User Story

Individual team members such as administrators and developers can use the Update User Story interface to add manual setup tasks and metadata assets to their User Stories. Metadata assets can be selected and added to the story with a Create or Delete Job List. Manual setup tasks represent various activities that need to be completed before or after the actual deployment. Examples include manual operations with the Setup Menu, user acceptance testing, corporate notifications, and institutional security requirements.

The first screen shows each team member the Sprints and User Stories that they have been assigned to. They can select any story, and the next tab allows them to select manual tasks that need to be completed before deployment. They can create task templates for common manual tasks that are used over and over again. After that, the selected task can be customized if necessary. Each task has the ability to execute a command line argument, or launch a Salesforce browser window for the source or destination Org.

The next tab allows the team member to build a Create Job List for metadata assets that should be captured and included in the User Story. All of the interfaces show Overlap Awareness. If any other team member is using the same asset in their story, then these assets are shown in red. The tool tip will display the story name, story owner and email address so that the two team members can coordinate their development activities.

After this, the team member can select assets to add to the Delete Job List. After that, manual tasks that need to be completed after deployment can be added to the story. The final tab allows the team member to save their User Story. When this happens, all of the before tasks, created assets, deleted assets, and after tasks are saved to the User Story. This information is stored in one of the custom objects in the Cascade Installation org.

The team member can choose at this point to return to the first screen and click the Pull Request button. This moves their User Story into the Kanban Stage for Deployment and sends an email to the Sprint Owner that work on the story has been completed.

Deploy User Stories

Release managers can choose the Deploy User Stories interface to select multiple stories for deployment, merge any conflicts, compare against the destination, and then start the deployment process. First the release manager will need to select a Sprint. Then they can select any number of User Stories for deployment.

The next tab presents powerful tools to resolve any merge conflicts in the selected User Stories. All overlaps and conflicts are presented side by side and the release manager can choose to accept one of the assets or merge multiple XML documents as necessary. If there is uncertainty about the correct way to resolve a conflict, then the release manager can choose to send an email to the developers with the relevant metadata assets as an attachment. There is also a button to Accept Everything, and in this case, Cascade merges the stories in the most intelligent manner.

After all conflicts are merged, the selected User Stories form a new metadata source that can be deployed to any destination. The next tab allows the merged stories to be compared against the metadata destination. If everything looks correct, the deployment process can begin.

First, the merged before task checklist must be completed. These tasks can run command lines and launch Salesforce browsers as needed. When the tasks are completed, the actual merged metadata from the User Stories must be deployed to the destination. Lastly, the after deployment task checklist must be completed.

Each metadata deployment will be added to the Sprint’s deployment history. This information is available in the Manage Sprints interface. By the way, there is also a checkbox to Archive the Sprint. An archived Sprint and the related User Stories cannot be altered in any way, but archived Sprints can still be deployed.

Cascade Complete

That’s a quick overview of what Cascade can do! Let us know about your challenges and how we can help. Also, please take a look at our Snapshot for Org Management product available on the AppExchange. Snapshot provides powerful tools that help Salesforce Administrators manage the change and release process, visualize and reduce complexity, improve security and compliance, and lower the total cost of org ownership.


Building The New Org Dictionary Report

Our Snapshot product has over 40 special purpose reports that detail the complexity found in almost any Salesforce Org. The Data Dictionary Report documents Custom Objects and Fields with all of the information available from the Data, Metadata, and Tooling API. The Profiles and Permission Sets Report shows the inner structure of Org permissions and helps identify security problems. But what about the other 200 Metadata Types? Some of these asset types are both very complex and quite important: Custom Applications, Flows, Reports, Dashboards, and Workflows all come to mind. We have specific reports that cover certain use cases, but we need general purpose reports for everything else. The new Org Dictionary report was designed to tackle this challenge. This blog post outlines some of the trials and tribulations we encountered building this technology.

In order to create the report, we needed the ability to generate example XML envelopes for each Metadata Type. We couldn’t just look at the documentation, there are too many errors there. And we couldn’t just look at the actual XML retrieved from an Org, either. Depending on the shape of the Org, the downloaded XML might or might not contain all of the information we needed. The sheer size of the job was also daunting. There are roughly 200 different Metadata Types with over 6000 different named elements. We had to understand the generic structure of each Metadata Type before an Admin could assemble any type of report

The best approach was to study the Metadata WSDL. This information can be downloaded from the Setup Menu. Unfortunately, if you have ever looked at WSDL before, these documents are difficult to understand, and this one was over one gigabyte in size. But by generating example XML envelopes from the WSDL and comparing that against known Metadata and the documentation we were able to generate example XML for almost all of the different Metadata Types. We were also able to identify the data type and enumerated values for each element.

Another problem was that the Metadata WSDL depends on the Org shape to some extent. Because we could not find Orgs with certain features, there are a handful of Metadata Types that we were unable to define. For example, there are some new Metadata Types associated with Salesforce C360, but this product is not yet widely available. If a Metadata Type that you need is not available in the Org Dictionary report then please send us your Metadata WSDL and we can add that asset type to the list. As new Salesforce products become available over time we will add new Metadata Types to the Org Dictionary report.

Once we were able to generate the XML structure of all the different Metadata Types, an interesting picture emerged. By looking at the number of elements per asset type we were able to visualize the overall complexity of all Salesforce Metadata. The chart below shows all of the Metadata Types that have more than 20 XML elements. The chart is shown with the simpler types at the top and the most complex types at the bottom. This chart provides an incredibly high-level view of Metadata complexity.

As you can see, Object Translations, Workflows, Page Layouts, Dashboards, Reports, Custom Objects and Flows are among the most complex Metadata Types. Flows are by far the most complex, with over 2000 different element names in that one structure. Flows are more than twice as complex as Custom Objects. Because of this, we truncated the horizontal axis of the graph above for the sake of presentation. Since some of these elements define arrays, any given instance of an asset could be very large in size. For example, Profiles only have about 100 different element names, but because they contain so many arrays, they can be among the largest assets in any Org.

Another challenge was how to display complex assets with a hierarchical XML structure in a two-dimensional report table. For example, here is the Role asset type. There are no arrays, so all the Roles stack right up in a linear fashion.

But if you have an asset like Custom Objects, there are sub arrays like Fields, and sub-sub arrays like Picklist Values, and so on. There are over 40 different types of arrays just in Custom Objects alone. If you display everything then you will probably be overwhelmed by the complexity of the resulting report. Because of this, the Org Dictionary allows the user to pick the elements to display and adjust the column order.

Here is the report table below that was generated for a Custom Object with Fields and Picklist Values. Note the labels at the top of the picture that show the structure of the nested arrays. This naming convention is similar to XPath, for you developers out there. This example was chosen to illustrate how hierarchical structures are rendered. Obviously, the more specialized Data Dictionary report could provide more information on Custom Objects.

After the WSDL work was completed, the new Org Dictionary report was pretty easy to for the engineering team to build. The interface is similar to other Snapshot reports. The different Asset Types are listed in the first column. When one is selected, the WSDL structure is displayed in the center column. Administrators can select these elements and move them into the report column on the right. We gave each asset type a nice default report to help get things started. All you need to do is select any additional fields and preview the results to be sure the report is what you wanted. Now you can document any Metadata Type in your Org!

There is an option to hide and show the camel case names in the report labels. These names match the Metadata API documentation, but might be difficult for someone like an auditor to understand. There is also an option to hide and show the element data types and enumerated values. This information helps explain the purpose behind each field so the report can be assembled quickly. There are some other nifty advantages to having this information that are discussed below.

There have been some interesting side effects to our research effort exploring the Metadata WSDL. Now Snapshot has the ability to really start dealing with interior characteristics of different Metadata Types in a more comprehensive manner. Developers often like to edit XML by hand. Now Snapshot can validate these documents and highlight any errors. Salesforce Administrators sometimes make changes to the Snapshot file before deployment. Now Snapshot can suggest the proper data type and enumerated values. This inside information simplifies editing operations and reduces errors.

 

For example, in the View Snapshot interface, an Admin might need to edit an Apex Class. If they edit the apiVersion, Snapshot will make sure that the new value is a floating-point number. If they edit the status, they are automatically prompted to choose a new enumerated value: Active, Inactive, or Deleted. This level of intelligence is possible because of the new WSDL database we developed for the Org Dictionary report.

I wrote this blog to show some of the thinking and research that goes into building new reports for Snapshot. We are trying to make very powerful capabilities very easy to use. Let us know if you like the new Org Dictionary report! If this is the type of report that you would like to run on your Org then please check out Snapshot on the AppExchange.


Create Sprints and User Stories in Snapshot

Snapshot can now create Sprints and User Stories that help Salesforce administrators manage complex Orgs with collaborative projects and teamwork automation. Project managers can create Sprints and assign User Stories to individual developers and administrators. These individual contributors can add metadata assets and deployment tasks to their User Stories. Lastly, a release manager can deploy the User Stories into any destination Salesforce Org, developer project, or online repository. Let’s take a quick look at what you can do with Sprints and User Stories!

Manage Sprints

The Manage Sprints interface is where project managers can create, edit, and delete Sprints and assign User Stories to individual contributors. Sprints can be managed with a simple to do list, a Kanban Chart, or a Gantt Chart. A simple Sprint might be managed with a to do list. A more complex Sprint might be managed with a Kanban Board. A long running Sprint might need to be managed like a project with Gantt Charts. You can use all of these visualizations at the same time or focus on one in particular. Here is the first tab of the Manage Sprints interface.

Teamwork Automation

You can turn on awesome teamwork automation features for any Sprint and send email notifications when a Sprint is created, deleted, deployed, or assigned a new owner. You can also notify individual contributors when a User Story is created, deleted, assigned, or when the Kanban Stage changes. Sprints make collaborative Org Management a breeze!

Kanban Boards

You can view your Sprints with a Kanban Board in both Snapshot and Salesforce Lightning. The Kanban Stages are shown across the top of the board with team members displayed down the left-hand side. Kanban Boards help manage work by showing which team members are available and the current stage of all the Stories. User Stories can be reassigned to different Stages or team members by dragging and dropping the cards on the board.

Gantt Charts

Every Sprint also has an optional Gantt Chart interface. Each User Story is displayed as a task bar on the chart. Gantt Charts are useful for long running projects where some Stories must be completed before other Stories can begin. The Gantt Chart will calculate the amount of time for the entire project if some stories are delayed or contingencies change.

Burndown Graphs

Keep track of Sprint progress with a Burndown Graph. This graph shows when a Sprint is behind or ahead of schedule.

Update Story

The Update Story interface is for individual contributors that have been assigned User Stories by a project manager. The Story Owner can add metadata assets and manual deployment tasks to their User Story. This information is saved with the Story for collaborative use by other team members. The metadata assets attached to the User Story can be tested in context and saved.

Pull Request

When the User Story is finished, the Story Owner can make a pull request that moves the Story into deployment. The Sprint Owner is notified via email about the pull request.

Global Templates

Some deployment tasks will be part of any Sprint. For this reason, we have added the ability to create global task templates that can be reused. Each task contains the option to launch the source or destination Salesforce browser and to run a command line. The browser can be used to start a manual operation with the setup menu. The command line results will be included in the Sprint History when the task is completed.

Overlap Awareness

All of the Sprint and User Story interfaces provide instant Overlap Awareness if other team members are using the same metadata assets that you are. Item on the Job List will appear in red if they are overlapped with other User Stories in the same Sprint. The tooltip for the field will show who else is using the object and the name of their User Story.

The final tab looks similar to the regular deployment interface. The Update Story button will save all of the selected deployment tasks and metadata assets into the current User Story. The Test Deployment button will conduct a validating deployment using the selected assets against the destination Org.

Deploy User Stories

The Deploy User Stories interface is designed for release managers who need to deploy a Sprint or User Story into an Org, a developer project, or an online repository. This interface could also be used by an individual contributor who wants to move a User Story into a Scratch Org or some other destination.

Merge Conflicts

If the two developers are working on the same piece of metadata, then there is a conflict and the merge panel will be displayed. In the picture below we can see that one object has a description and the other does not. Since adding a description is nondestructive, the left hand Accept button is automatically highlighted indicating the best way to resolve the conflict. Every aspect of the merge process is guided by the user interface. This is way better than trying to resolve conflicts in a text editor!

At any point the release manager can select the Automatic Merge option and look at the conflict map. This shows all of the component parts of the metadata asset across the top of the screen and all of the conflicting User Stories down the left-hand side of the screen. The conflict map can be exported for review by the engineering team.

In some cases, a release manager might be tempted to send the conflicting User Stories back to the individual contributors and let them figure it out. The Notify Owners button will send an email to the Story Owners, along with an attachment that contains the conflicting asset data and the conflict map.

Metadata Comparison

Some Sprints will not have any asset overlaps, and in other cases the Merge Conflicts interface must be used to resolve problems. When all of the selected User Stories have been merged into a single metadata source, then the release manager can compare the assets against the destination Org.

The next tab begins the deployment process for the merged User Stories. First, the before deployment tasks are completed and checked off one by one. In some cases, a deployment task might launch a browser to begin setup menu changes or to run a command line test. When all of the before deployment tasks are completed the release manager can proceed to the next tab and deploy the assets.

The Deploy Assets tab is very similar to the regular deployment interface. You can deploy User Stories into a destination Org, developer Project, or online repository. When you are finished, complete the deployment by completing the after tasks checklist on the last tab.

Sprint History

Every time you complete a User Story deployment the full history of what happened is added to the Sprint History object. This records the list of deployed User Stories, the before and after tasks that were successfully completed, and the results of the deployment.

Conclusion

There you have it! This Blog has covered some of the cool new features behind Sprints and User Stories. You will need the latest Managed Package for Snapshot version 3.5 to start using the new User Story capability. Let us know if you need help. If you want more information, please check out the User Story whitepaper below:

https://www.metazoa.com/best-practices-sprints-and-user-stories/