Metazoa Cascade DevOps and User Stories

Metazoa has released a great new FREE application on the AppExchange! Cascade can deploy metadata between any Salesforce Org, local project folder, or remote Git Repository. Cascade also supports collaborative Sprints and User Stories that streamline Org Management for teams of developers and administrators. Unlike other solutions, that can take months to configure and implement, Cascade works immediately with any Org, there is no package to install. Cascade is a desktop application that communicates directly from your personal computer to your Salesforce account, and because of this, Cascade is an inherently secure solution that never compromises your Salesforce credentials, data, or metadata. Let’s take a look at Cascade!

Main Interface

When you launch Cascade the first time you will need to select an Installation Org. This is where Cascade finds administrators and developers for your team. Any user with administrative credentials will be available as a team member. Cascade also uses this Org to store deployment history, user stories, and other information. The first time you login, some custom objects with the prefix “metazoa” will be added to the Org. These objects are easy to remove if you ever need to uninstall Cascade.

Next, Cascade presents options to select a metadata source and a metadata destination. The source and the destination can be any Salesforce Org, local project folder, or Git Repository. Simple click the buttons to select a Salesforce Org or Developer Project.

Salesforce Orgs

If you select a Salesforce Org, Cascade prompts you for the org credentials. You can authenticate with any kind of Salesforce Org. Cascade has full support for Scratch Orgs, OAuth, Custom Domains, Developer Orgs, and Sandboxes. The next screen prompts you to Take a Snapshot. This will download all Org metadata to your client computer. Cascade can retrieve all of the metadata from even the largest Salesforce Orgs! If necessary, Cascade will conduct multiple retrieve operations simultaneously and reassemble all of the metadata at the client. Don’t worry, all of this is handled automatically for you!

Developer Projects

The other option is to retrieve source or destination metadata from a Developer Project. This option allows you to select either a local project folder or a remote repository. The folder or repository can be in either Metadata API, Eclipse, or Salesforce DX format. For example, if you have a Salesforce DX project on your computer, all you have to do is select that folder. If you have a Git Repository that you want to use, then enter the credentials to the repo and test the connection with the Clone Repository button. The next tab allows you to Take a Snapshot, and this will create a new metadata source or destination.

By the way, on the main screen there are some handy menu options under the source and destination buttons. These include:

  • Open Salesforce Browser
  • Open Local Project Folder
  • Open SFDX Command Line
  • Open Repository Browser
  • Import Metadata Snapshot
  • Export Metadata Snapshot

All of these options operate on the current source or destination. Cascade makes it super easy to work with your selected metadata! Once the source and destination are set up, you can start performing the main Cascade Activities listed in the center of the Welcome Screen. The first two options provide powerful capabilities for DevOps: deploying and comparing metadata.

Deploy Metadata

The Deploy Metadata interface allows you to move any number of metadata assets from the source to the destination. All 200 metadata types are supported. The basic idea here is to build up a Create and Delete Job List of the assets that you want to move. The user interface is very rich, and lots of information is displayed. For example, you can see if each asset is the same or different on the current source and destination. The XML or textual differences are also displayed. The picture below shows the process of building a Create Job List.

When you have all of your selected assets ready to go, the last screen presents state of the art deployment tools for moving the assets from source to destination. You can manage your job lists, import and export package.xml files, test code quality, run code coverage, transform XML data, remove bad references, and fix problems with sparse profiles. Remember, the source and destination can be a Salesforce Org, local project folder, or Git Repository in any file format. Because of this, the metadata deployment interface is an incredibly flexible way to move any type of metadata to any destination!

Compare Metadata

Cascade also provides an amazing XML comparison report. This is useful any time you want to be sure what the metadata source or destination look like, or what the differences are. You can select any metadata type and see what assets are the same, somewhat different, very different, missing on the source, or missing on the destination. You can click the checkbox to the left of any metadata type and it will be included in the Display Report which is available on the last tab. The Display Report can be exported as HTML, CSV, PDF, or in Native Excel format for distribution to team members. You can also archive these reports for compliance.

Sprints and User Stories

Cascade provides three additional capabilities that streamline Org Management for administrators and developers. These options are visible on the main screen: Manage Sprints, Update User Story, and Deploy User Stories.

Project managers can use the Manage Sprints interface to create new Sprints and assign out User Stories to team members. Team members such as Salesforce administrators and developers can use the Update User Stories interface to add manual setup tasks and metadata assets to their stories. Lastly, release managers can choose the Deploy User Stories interface to select multiple stories for deployment, merge the stories, compare them against the destination, and then begin the deployment process. Let’s take a look at the Sprint and User Story management capabilities in Cascade!

Manage Sprints

Project managers can use the Manage Sprints interface to create new Sprints and assign out User Stories to team members. There are options to create, manage, duplicate, and delete Sprints and User Stories. Every story is associated with a Kanban Stage. These stages help organize the sprint from beginning to end. The available Kanban Stages are:

  • Draft
  • Backlog
  • Requested
  • Developing
  • Waiting
  • Testing
  • Deployment
  • Finished
  • Archived

Every Sprint has options to broadcast email notifications to the owner of the User Story when things change. These notification help provide a collaborative backbone for the Sprint. The available notifications are:

  • Sprints Created or Deleted
  • User Story Created or Deleted
  • Sprint and User Story Deployment
  • User Story Kanban Stage Changes
  • Pull Request for User Story

Sprints can be managed in multiple ways. On the main screen, you can work with the selected Sprint as a linear To-Do list and filter by the current Kanban Stage.

Sprints can also be managed with a Kanban Board. The board shows team members down the left-hand side and Kanban Stages across the top. User Stories are shown as cards on the board. The total number of items in any column is displayed to prevent bottlenecks. You can drag and drop any User Story to reassign the story owner or change the Kanban Stage.

Sprints can also be managed as interactive Gantt Charts. This is a good option for longer and more complex projects where some User Stories depend on the completion of others. Each User Story is displayed as a task bar on the chart. The Gantt Chart will display the projected start and finish dates of all of the User Stories, as well as the percent complete.

Lastly, the Manage Sprints interface will automatically calculate a Burndown Graph for the entire Sprint. This shows the actual progress of the Sprint compared to linear progress. If the red area appears to the upper right of the diagonal line, that is a sign that the project is behind schedule. If the red area appears to the lower left of the diagonal line, that is a sign that the project is ahead of schedule.

Update User Story

Individual team members such as administrators and developers can use the Update User Story interface to add manual setup tasks and metadata assets to their User Stories. Metadata assets can be selected and added to the story with a Create or Delete Job List. Manual setup tasks represent various activities that need to be completed before or after the actual deployment. Examples include manual operations with the Setup Menu, user acceptance testing, corporate notifications, and institutional security requirements.

The first screen shows each team member the Sprints and User Stories that they have been assigned to. They can select any story, and the next tab allows them to select manual tasks that need to be completed before deployment. They can create task templates for common manual tasks that are used over and over again. After that, the selected task can be customized if necessary. Each task has the ability to execute a command line argument, or launch a Salesforce browser window for the source or destination Org.

The next tab allows the team member to build a Create Job List for metadata assets that should be captured and included in the User Story. All of the interfaces show Overlap Awareness. If any other team member is using the same asset in their story, then these assets are shown in red. The tool tip will display the story name, story owner and email address so that the two team members can coordinate their development activities.

After this, the team member can select assets to add to the Delete Job List. After that, manual tasks that need to be completed after deployment can be added to the story. The final tab allows the team member to save their User Story. When this happens, all of the before tasks, created assets, deleted assets, and after tasks are saved to the User Story. This information is stored in one of the custom objects in the Cascade Installation org.

The team member can choose at this point to return to the first screen and click the Pull Request button. This moves their User Story into the Kanban Stage for Deployment and sends an email to the Sprint Owner that work on the story has been completed.

Deploy User Stories

Release managers can choose the Deploy User Stories interface to select multiple stories for deployment, merge any conflicts, compare against the destination, and then start the deployment process. First the release manager will need to select a Sprint. Then they can select any number of User Stories for deployment.

The next tab presents powerful tools to resolve any merge conflicts in the selected User Stories. All overlaps and conflicts are presented side by side and the release manager can choose to accept one of the assets or merge multiple XML documents as necessary. If there is uncertainty about the correct way to resolve a conflict, then the release manager can choose to send an email to the developers with the relevant metadata assets as an attachment. There is also a button to Accept Everything, and in this case, Cascade merges the stories in the most intelligent manner.

After all conflicts are merged, the selected User Stories form a new metadata source that can be deployed to any destination. The next tab allows the merged stories to be compared against the metadata destination. If everything looks correct, the deployment process can begin.

First, the merged before task checklist must be completed. These tasks can run command lines and launch Salesforce browsers as needed. When the tasks are completed, the actual merged metadata from the User Stories must be deployed to the destination. Lastly, the after deployment task checklist must be completed.

Each metadata deployment will be added to the Sprint’s deployment history. This information is available in the Manage Sprints interface. By the way, there is also a checkbox to Archive the Sprint. An archived Sprint and the related User Stories cannot be altered in any way, but archived Sprints can still be deployed.

Cascade Complete

That’s a quick overview of what Cascade can do! Let us know about your challenges and how we can help. Also, please take a look at our Snapshot for Org Management product available on the AppExchange. Snapshot provides powerful tools that help Salesforce Administrators manage the change and release process, visualize and reduce complexity, improve security and compliance, and lower the total cost of org ownership.


Building The New Org Dictionary Report

Our Snapshot product has over 40 special purpose reports that detail the complexity found in almost any Salesforce Org. The Data Dictionary Report documents Custom Objects and Fields with all of the information available from the Data, Metadata, and Tooling API. The Profiles and Permission Sets Report shows the inner structure of Org permissions and helps identify security problems. But what about the other 200 Metadata Types? Some of these asset types are both very complex and quite important: Custom Applications, Flows, Reports, Dashboards, and Workflows all come to mind. We have specific reports that cover certain use cases, but we need general purpose reports for everything else. The new Org Dictionary report was designed to tackle this challenge. This blog post outlines some of the trials and tribulations we encountered building this technology.

In order to create the report, we needed the ability to generate example XML envelopes for each Metadata Type. We couldn’t just look at the documentation, there are too many errors there. And we couldn’t just look at the actual XML retrieved from an Org, either. Depending on the shape of the Org, the downloaded XML might or might not contain all of the information we needed. The sheer size of the job was also daunting. There are roughly 200 different Metadata Types with over 6000 different named elements. We had to understand the generic structure of each Metadata Type before an Admin could assemble any type of report

The best approach was to study the Metadata WSDL. This information can be downloaded from the Setup Menu. Unfortunately, if you have ever looked at WSDL before, these documents are difficult to understand, and this one was over one gigabyte in size. But by generating example XML envelopes from the WSDL and comparing that against known Metadata and the documentation we were able to generate example XML for almost all of the different Metadata Types. We were also able to identify the data type and enumerated values for each element.

Another problem was that the Metadata WSDL depends on the Org shape to some extent. Because we could not find Orgs with certain features, there are a handful of Metadata Types that we were unable to define. For example, there are some new Metadata Types associated with Salesforce C360, but this product is not yet widely available. If a Metadata Type that you need is not available in the Org Dictionary report then please send us your Metadata WSDL and we can add that asset type to the list. As new Salesforce products become available over time we will add new Metadata Types to the Org Dictionary report.

Once we were able to generate the XML structure of all the different Metadata Types, an interesting picture emerged. By looking at the number of elements per asset type we were able to visualize the overall complexity of all Salesforce Metadata. The chart below shows all of the Metadata Types that have more than 20 XML elements. The chart is shown with the simpler types at the top and the most complex types at the bottom. This chart provides an incredibly high-level view of Metadata complexity.

As you can see, Object Translations, Workflows, Page Layouts, Dashboards, Reports, Custom Objects and Flows are among the most complex Metadata Types. Flows are by far the most complex, with over 2000 different element names in that one structure. Flows are more than twice as complex as Custom Objects. Because of this, we truncated the horizontal axis of the graph above for the sake of presentation. Since some of these elements define arrays, any given instance of an asset could be very large in size. For example, Profiles only have about 100 different element names, but because they contain so many arrays, they can be among the largest assets in any Org.

Another challenge was how to display complex assets with a hierarchical XML structure in a two-dimensional report table. For example, here is the Role asset type. There are no arrays, so all the Roles stack right up in a linear fashion.

But if you have an asset like Custom Objects, there are sub arrays like Fields, and sub-sub arrays like Picklist Values, and so on. There are over 40 different types of arrays just in Custom Objects alone. If you display everything then you will probably be overwhelmed by the complexity of the resulting report. Because of this, the Org Dictionary allows the user to pick the elements to display and adjust the column order.

Here is the report table below that was generated for a Custom Object with Fields and Picklist Values. Note the labels at the top of the picture that show the structure of the nested arrays. This naming convention is similar to XPath, for you developers out there. This example was chosen to illustrate how hierarchical structures are rendered. Obviously, the more specialized Data Dictionary report could provide more information on Custom Objects.

After the WSDL work was completed, the new Org Dictionary report was pretty easy to for the engineering team to build. The interface is similar to other Snapshot reports. The different Asset Types are listed in the first column. When one is selected, the WSDL structure is displayed in the center column. Administrators can select these elements and move them into the report column on the right. We gave each asset type a nice default report to help get things started. All you need to do is select any additional fields and preview the results to be sure the report is what you wanted. Now you can document any Metadata Type in your Org!

There is an option to hide and show the camel case names in the report labels. These names match the Metadata API documentation, but might be difficult for someone like an auditor to understand. There is also an option to hide and show the element data types and enumerated values. This information helps explain the purpose behind each field so the report can be assembled quickly. There are some other nifty advantages to having this information that are discussed below.

There have been some interesting side effects to our research effort exploring the Metadata WSDL. Now Snapshot has the ability to really start dealing with interior characteristics of different Metadata Types in a more comprehensive manner. Developers often like to edit XML by hand. Now Snapshot can validate these documents and highlight any errors. Salesforce Administrators sometimes make changes to the Snapshot file before deployment. Now Snapshot can suggest the proper data type and enumerated values. This inside information simplifies editing operations and reduces errors.

 

For example, in the View Snapshot interface, an Admin might need to edit an Apex Class. If they edit the apiVersion, Snapshot will make sure that the new value is a floating-point number. If they edit the status, they are automatically prompted to choose a new enumerated value: Active, Inactive, or Deleted. This level of intelligence is possible because of the new WSDL database we developed for the Org Dictionary report.

I wrote this blog to show some of the thinking and research that goes into building new reports for Snapshot. We are trying to make very powerful capabilities very easy to use. Let us know if you like the new Org Dictionary report! If this is the type of report that you would like to run on your Org then please check out Snapshot on the AppExchange.


Create Sprints and User Stories in Snapshot

Snapshot can now create Sprints and User Stories that help Salesforce administrators manage complex Orgs with collaborative projects and teamwork automation. Project managers can create Sprints and assign User Stories to individual developers and administrators. These individual contributors can add metadata assets and deployment tasks to their User Stories. Lastly, a release manager can deploy the User Stories into any destination Salesforce Org, developer project, or online repository. Let’s take a quick look at what you can do with Sprints and User Stories!

Manage Sprints

The Manage Sprints interface is where project managers can create, edit, and delete Sprints and assign User Stories to individual contributors. Sprints can be managed with a simple to do list, a Kanban Chart, or a Gantt Chart. A simple Sprint might be managed with a to do list. A more complex Sprint might be managed with a Kanban Board. A long running Sprint might need to be managed like a project with Gantt Charts. You can use all of these visualizations at the same time or focus on one in particular. Here is the first tab of the Manage Sprints interface.

Teamwork Automation

You can turn on awesome teamwork automation features for any Sprint and send email notifications when a Sprint is created, deleted, deployed, or assigned a new owner. You can also notify individual contributors when a User Story is created, deleted, assigned, or when the Kanban Stage changes. Sprints make collaborative Org Management a breeze!

Kanban Boards

You can view your Sprints with a Kanban Board in both Snapshot and Salesforce Lightning. The Kanban Stages are shown across the top of the board with team members displayed down the left-hand side. Kanban Boards help manage work by showing which team members are available and the current stage of all the Stories. User Stories can be reassigned to different Stages or team members by dragging and dropping the cards on the board.

Gantt Charts

Every Sprint also has an optional Gantt Chart interface. Each User Story is displayed as a task bar on the chart. Gantt Charts are useful for long running projects where some Stories must be completed before other Stories can begin. The Gantt Chart will calculate the amount of time for the entire project if some stories are delayed or contingencies change.

Burndown Graphs

Keep track of Sprint progress with a Burndown Graph. This graph shows when a Sprint is behind or ahead of schedule.

Update Story

The Update Story interface is for individual contributors that have been assigned User Stories by a project manager. The Story Owner can add metadata assets and manual deployment tasks to their User Story. This information is saved with the Story for collaborative use by other team members. The metadata assets attached to the User Story can be tested in context and saved.

Pull Request

When the User Story is finished, the Story Owner can make a pull request that moves the Story into deployment. The Sprint Owner is notified via email about the pull request.

Global Templates

Some deployment tasks will be part of any Sprint. For this reason, we have added the ability to create global task templates that can be reused. Each task contains the option to launch the source or destination Salesforce browser and to run a command line. The browser can be used to start a manual operation with the setup menu. The command line results will be included in the Sprint History when the task is completed.

Overlap Awareness

All of the Sprint and User Story interfaces provide instant Overlap Awareness if other team members are using the same metadata assets that you are. Item on the Job List will appear in red if they are overlapped with other User Stories in the same Sprint. The tooltip for the field will show who else is using the object and the name of their User Story.

The final tab looks similar to the regular deployment interface. The Update Story button will save all of the selected deployment tasks and metadata assets into the current User Story. The Test Deployment button will conduct a validating deployment using the selected assets against the destination Org.

Deploy User Stories

The Deploy User Stories interface is designed for release managers who need to deploy a Sprint or User Story into an Org, a developer project, or an online repository. This interface could also be used by an individual contributor who wants to move a User Story into a Scratch Org or some other destination.

Merge Conflicts

If the two developers are working on the same piece of metadata, then there is a conflict and the merge panel will be displayed. In the picture below we can see that one object has a description and the other does not. Since adding a description is nondestructive, the left hand Accept button is automatically highlighted indicating the best way to resolve the conflict. Every aspect of the merge process is guided by the user interface. This is way better than trying to resolve conflicts in a text editor!

At any point the release manager can select the Automatic Merge option and look at the conflict map. This shows all of the component parts of the metadata asset across the top of the screen and all of the conflicting User Stories down the left-hand side of the screen. The conflict map can be exported for review by the engineering team.

In some cases, a release manager might be tempted to send the conflicting User Stories back to the individual contributors and let them figure it out. The Notify Owners button will send an email to the Story Owners, along with an attachment that contains the conflicting asset data and the conflict map.

Metadata Comparison

Some Sprints will not have any asset overlaps, and in other cases the Merge Conflicts interface must be used to resolve problems. When all of the selected User Stories have been merged into a single metadata source, then the release manager can compare the assets against the destination Org.

The next tab begins the deployment process for the merged User Stories. First, the before deployment tasks are completed and checked off one by one. In some cases, a deployment task might launch a browser to begin setup menu changes or to run a command line test. When all of the before deployment tasks are completed the release manager can proceed to the next tab and deploy the assets.

The Deploy Assets tab is very similar to the regular deployment interface. You can deploy User Stories into a destination Org, developer Project, or online repository. When you are finished, complete the deployment by completing the after tasks checklist on the last tab.

Sprint History

Every time you complete a User Story deployment the full history of what happened is added to the Sprint History object. This records the list of deployed User Stories, the before and after tasks that were successfully completed, and the results of the deployment.

Conclusion

There you have it! This Blog has covered some of the cool new features behind Sprints and User Stories. You will need the latest Managed Package for Snapshot version 3.5 to start using the new User Story capability. Let us know if you need help. If you want more information, please check out the User Story whitepaper below:

https://www.metazoa.com/best-practices-sprints-and-user-stories/


The New Code Quality Report

Snapshot has an amazing new Code Quality Report that can identify many common programming flaws in any Apex Class. You can schedule the Code Quality Report to run after any Snapshot and send a PDF, CSV, or HTML file to admins or developers with all of the various problems that were found. Super admins can even set up a Code Quality Gateway to prevent selected programming problems from being deployed to any org! Let’s dig in and take a look at this new capability.

Installing PMD

The Code Quality Report uses the PMD open source static code analyzer. Nobody is really sure what PMD stands for, but this is the gold standard for analyzing Apex code and finding common programming flaws. The open source PMD project is the foundation for most of the code quality systems available on the Salesforce platform. You will need to be sure that PMD is installed on your local machine. Snapshot uses PMD through the command line interface. PMD is a Java application, so you will also need to be sure that Java is installed, if it is not already. You can learn more about PMD here:

https://pmd.github.io/

We have simplified the PMD installation process as much as possible. The first tab of the Apex Code Quality dialog has a button at upper right called “PMD Connection.” Click that button, and there will be various links and information about how to install PMD. When you are ready, click the “Test Installation” button, and Snapshot will tell you if PMD is installed or not. You can also return to this screen and update the version of PMD that you are using. The path to the current version is in the pathname above the button.

Apex Code Quality

Once PMD is installed, you can right-click any Snapshot item and select “Apex Code Quality” from the popup menu. You can also use the Options Menu at the top of the screen. The dialog below will appear, and you can see all of your Apex Classes at left. Click on any class name and the middle section will show you various problems with the code. You will see the Rule Set and Rule that was violated. Scrolling to the right, the report details the seriousness of the problem, the impacted line numbers, and a detailed explanation of how to fix the issue. Click on any rule and the lines of code that are causing the problem will be highlighted at right.

This first tab provides an easy way to quickly preview all of your Apex Classes and review any code problems. You can click down the list of issues and see all of the affected lines of code at right. Right-click the center table to export all of the problems as a CSV, HTML, or PDF file. If you check multiple Apex Classes at left, then all of the selected items will be included in the final report under the middle tab.

Understanding Quality Results

PMD divides code quality problems into seven categories:

  • Best Practices
  • Code Style
  • Design
  • Documentation
  • Error Prone
  • Performance
  • Security

Each category contains different rules. There are about 45 different rules that PMD uses to analyze Apex code. A full description of each rule is documented in the Snapshot interface. Each rule is assigned a number between 1 (highest priority) and 5 (lowest priority). The priority numbers for each rule can be assigned in the Snapshot interface. In this manner, you can decide what rules are the most critical or least important. Rules can also be removed from consideration completely if desired.

  • Priority 1) Change Absolutely Required
  • Priority 2) Change Highly Recommended
  • Priority 3) Change Recommended
  • Priority 4) Change Optional
  • Priority 5) Change Highly Optional

Managing Code Quality

Right-click the center report on the first tab or click the “Manage Quality” button on the second tab to assign each of the available rules a priority. In the example below, we have assigned rules regarding Code Style and Best Practices to be optional, while rules regarding Security and Performance are absolutely required. If there are some rules that you would like to ignore entirely then uncheck the “Include Rule in Report” checkbox.

Automating Code Quality

Just like other reports in Snapshot, you can schedule and automate the Code Quality Report. In the last tab, select the event to trigger the report, and then send a PDF, CSV, or HTML file to an email list or Chatter group. Alternatively, the report could be saved to Salesforce Content or a local folder. Code Quality Reports can be conditional, and only sent out if changes are required in the code. In this manner, any org can be monitored for code quality, and a full report is generated and delivered if there are problems.

Conclusion

There you have it. Snapshot now has a state-of-the-art code quality analysis and monitoring system. Be sure to give this new feature a try and let us know how it works for you.