Release 6.3.1

Release 6.3.1 is here! We are pleased to announce several new features, including Feedback Prompts and Certification Levels, a new Plugin for Domo, significant improvements to our existing Plugins, especially to SharePoint and SAP Cloud Analytics, a completely new approach to search index update, a Compact Mode for External Reports, and many visible and invisible changes that make Metric Insights more powerful and extend it in new directions. This is our fourth and final release of 2021.

Legacy Reports are disabled in 6.3.1. Their metadata is still available, but Reports cannot be saved and end-users will no longer be able to view them. Dataset Reports may be used as a replacement.

Major features in 6.3.1 (released December 6, 2021)

  1. Feedback Prompts are a new way for users to assess content and provide specific feedback on their portal experience. This can help answer questions like: "How do users feel about our Sales reports?".
  2. Publishing Workflows have been further improved and polished to be more flexible and user-friendly, whether interacting/editing via the Publishing Board, or configuring the Workflow itself.
  3. A new, more granular method for certifying elements by creating various Levels of Certification has been introduced. Think: Gold, Silver, Bronze, where Gold is the most vetted and Bronze is the least.
  4. Privileges can now be grouped into Privilege Sets that can be added to Groups or to individual Users. 
  5. Microsoft SharePoint Plugin integration capabilities have expanded exponentially.
  6. SAP Analytic Cloud Plugin has matured significantly and is now on par with our other BI tool integrations.
  7. MicroStrategy Plugin includes a few new Optional Parameters.
  8. A new Domo Plugin has been introduced. Its capabilities include getting data and Card embedding, with more to come in 6.3.2.
  9. A new incremental approach to the search index update has been implemented; i.e.,  in many cases, new content will appear in search results immediately.
  10. Bulk Add External Reports now works for all Plugins that have existing Data Sources in MI.
  11. A new Compact Mode setting has need added for External Reports (available to turn on/off via External Report Types).

The primary Docker images used for standard deployments are:

  1. web - docker.metricinsights.com/metric-insights/release_6_3/167
  2. dataprocessor - docker.metricinsights.com/dataprocessor/release_6_3/10
  3. seed - docker.metricinsights.com/seed/release_6_3/5
  4. data-analyzer - docker.metricinsights.com/data-analyzer/release_6_3/37
  5. monitoring - docker.metricinsights.com/monitoring/release_6_3/39

Optional images:

  1. logger  - docker.metricinsights.com/logger/release_6_3/38
  2. mysql - docker.metricinsights.com/mysql:release_6_3

Feedback Prompts

Feedback Prompts are designed to help you understand sentiment about your Reports and Dashboards across one or many Categories and/or User Groups, or to gather feedback about your BI portal initiative as a whole; e.g., "How likely are you recommend the Enterprise BI Portal to a colleague?"

  • Configure these under Content > Feedback Prompts. You are free to pose any questions, depending on what aspects you want to discover about the content. 
  • Users can rate content from 1 to 10 or/and leave specific comments.
  • Feedback and some other statistics are available on the Engagement tab of Element Editors.
  • Global limits for how often a user should be prompted can be set via the following Config Variables:
    • FEEDBACK_PROMPT_MAX_LIMIT
    • FEEDBACK_PROMPT_MAX_LIMIT_INTERVAL

See Set Up Feedback Prompts and Understanding Engagement and Feedback on Elements.

Publishing Workflows

Publishing Workflows have improved significantly in 6.3.1.

Expanded capabilities for those configuring Workflows include:

  • Publishing Workflows can be duplicated.
  • There are two types of Categories available in Publishing Workflow Editor:
    • Staging Categories
    • Publishing Categories
  • When Power User adds Categories to the Workflow, contents of the Category drop-down are limited to the Categories to which the Power User has View or Edit Access.
  • Content that already exists in Category can be added to Workflow when Category has been assigned.
  • A Folder can be selected as an Element Field that is involved in a Stage.
  • When associating Categories to a Publishing Workflow, the contents of that Category can automatically become involved in the New or In Progress stages of the Workflow, based on the user's choice.
  • Categories can be set to change automatically at the final Stage of review.
  • When assigning a Group to a Workflow Stage, users are required to check either of the options for Edit/Assign.
  • Certification for Elements can be set when editing any Stage. If Stage involves Certification, the Certification icon indicating the level is displayed next to it on the grid.

See Create a 2-Party Publishing Workflow.

Some of the improvements for users are as follows:

  • Elements can be bulk-moved between Stages.
  • Changes in the History can be filtered by type and keywords.
  • Short Editors include more details on Elements' Sources and Data Collection Triggers.
  • From the Category and Group Editors, the list of associate Publishing Workflows have active Name listings to the related Editor.

See Execute Publishing Workflow in Content Center.

Certification Levels

  • The Certification Level Editor is accessible under Admin > System > Certification Setup.
  • Default Certification Levels include Bronze, Silver, and Gold.
  • New Certification Levels can be added and the default ones can be modified. 
  • You can set Certification for Elements on the Info tab.
  • While editing any Stage of the Publishing Workflow, you can choose Certification for Elements.
  • Certification Levels can be auto-synced. 
  • You can find Certification Levels displaying on: 
    • Homepage Tiles
    • Tile preview
    • Element Viewers
    • Lineage Diagram
  • Certification Levels are available for Filtering via:
    • Homepage Filters
    • Global Search
    • Element List

See Certification Levels for Elements (6.3.1) and Certification Level Auto Synchronization.

Privilege Sets

  • Available under Users & Groups > Privilege Sets.
  • Selected Privileges can be grouped into Sets that you can add to Groups or to individual Users. 
  • There are two default Privilege Sets, designed based on the User type: "Power User" and "Regular User".  
  • Admins can modify these default sets or create new ones by adding any type of Privileges to the Set.
  • Changes made to Sets can be reviewed from the History tab of the Privilege Set Editor.

See Create a Privilege Set.

Plugin Improvements

Microsoft SharePoint

  • SharePoint plugin supports multiple authentication types:
    • Client credential
    • OAuth
    • Password
    • Identity Profile
  • The team has implemented User's OAuth token for Delegated Permissions.
  • We have added the possibility to download SharePoint External Content from Bursts.
  • SharePoint Plugin now supports image capture for .xlsx files. You can see the Excel file previews on Tiles, Tile Previews, Bursts, Digests, and when sharing External Report via email.
  • Naming for SharePoint references has been improved to exclude special characters.
  • We have implemented a full Object hierarchy, and it now represents a better and more transparent logic.
  • The new Enable Custom Field Data Synchronization feature allows for mapping custom-added Columns in SharePoint to Custom Fields in MI. The Include columns parameter must be enabled to collect these custom columns to MI. 
  • New Document Library filter to fetch Objects from specific libraries has been added.
  • You can now specify a folder list to fetch by using Folder filter
  • From now on, SharePoint Plugin supports Proxy.
  • We have added more logging for SharePoint to be able to see a bigger picture: 
    • Which auth type was used during the connection test
    • Whether or not proxy was used during the connection test
    • What SharePoint sites were parsed during the connection test
    • Some additional logging for errors

Tableau

  • We have simplified the Tableau Optional Parameters Grid by removing such unused Parameters as Image resolution, Key Store Path, Key Store Password, Trust Store Path, and Trust Store Password.
  • Metric Insights supports using Proxy for Tableau Plugin.

SAP Analytics Cloud

  • The Plugin supports Content Sync, collecting Name, Description, Tags, and Category that exist in SAP Analytic Cloud. 
  • Implemented full Object hierarchy for SAP Analytics Cloud. Metric Insights reflects the hierarchy presented in SAP Analytics Cloud.
  • By enabling external filters for SAP Analytics Cloud, we allow users to pre-filter their External Report data.

MicroStrategy

  • Optional Parameters have been expanded to have new Exclude Sub-Folders and Folder Exclusion List Parameters for more flexible experience with folders. 
  • Object Types that can be configured to collect by the Plugin now include shortcuts too.

Other Plugin Improvements

  • We have designed a new Plugin, Domo. Its capabilities include getting data and card embedding.
  • Our Salesforce Plugins’ UI has become much cleaner since the Token parameter is no longer needed. 
  • Qlik Sense Plugin Profile Editor has been improved to include two server fields, Qlik Sense Server and API Server.
  • Viewer for Other External Content from Local FileSystem Plugin now also supports showing a real preview of .doc, .docx, .ppt, and .pptx files.
  • External Reports in iFrame are now loading faster. This improvement applies to all External Report Viewers regardless of the Plugin.  

Lineage

  • Users can see the entire Lineage diagram with all Elements and objects displayed on it, regardless of their Permissions. To fully view or edit an object, users must have the necessary Element permissions.

Search

  • Incremental update of index is the main change in this release. New, updated, and deleted objects are searchable immediately without the need to update indexes manually or wait for the system to do that.

Categories

  • List Page has Active Name Links to child Category Editors for Admins and Power User's with Edit Access.
  • When selecting a Group for Permissions, a search/chosen control facilitates locating the Group.
  • The options on the Name Menu > Settings > Catalog Display > Default Folder setting are updated to limit choices displayed to a Regular or Power Users in accord with Category Permissions.

User and Group Editors

  • On the Info tab:
    • Provided a grid that contains any Privilege Sets assigned and a button to add other Sets as needed;
    • The last Privileges grid, the Source link shows Directly Assigned, Group(s) (User Editor Only) and/or Privilege set from which the Privilege was acquired;  
    • Included the ability to Search for Privileges in the grids.
  • On the Other Access and Power Users tabs, objects are listed in alphabetic order.

Bulk Change

  • Provided additional Fields that can be bulk-changed:
    • External Report Type
    • URL Template
  • When Publishing Workflows is the Bulk Change operation,  the user selects a Publishing Workflow and, optionally, is prompted to select a Category as part of the same operation.

Bulk Add External Reports

  • You can add multiple reports to Metric Insights at once by using Bulk Add External Reports. This is now available not only for Tableau. The feature works for Data Sources that have already been added to the system (but not based on External Report Templates).

External Reports

  • We have added a new Compact mode setting to External Report Type Editor to create a more streamlined External Report. Once set, all existing reports using this Report Type will display in Compact Mode.
  • Metric Insight has expanded the Element View options for Embed Codes to include "Live Dashboard". This will allow users to embed the full dashboard for an Element.

Portal Pages

  • Some of the events can be tracked for MI Portal Pages by using third-party APM tools such as New Relic. We can track which Page was accessed, how long it took to open, and some other details.

Migration Tool

  • We have implemented the migration of Portal Page's internal Assets (those on the Assets Tab of the Editor) and Portal Page Template Variables. 
  • Portal Page Entities can be migrated along with the Objects associated with them too. Data Source Entities are migrating with linkage to selected Data Source if it exists on the target system.

Tags

  • We have fixed the inconsistencies with how Tags look across the system.
  • Governance Definitions popups look less busy in this implementation. 
  • Governance Definitions can now be auto-synced.

Data Processor

  • Datasets can be moved between Data Storages from Dataset Editors. 
  • You can set Auth Type to “Identity Profile” for Data Storages, instead of standard Username and Password. 
  • "Download ZIP File" has been renamed to "Download RDP Installer".
  • Metric Insights supports the Active Directory Service Principal auth for MS SQL Server.

Data Storage

  • SQL Server Bulk Load option is no longer present in Data Storage Editor.
  • Dataset Storage Editors now support using Identity Profiles as Auth Type.

Other Improvements

  • External Content attachments in a pdf format can be sent in Bursts and Digests if External Content Type is configured to do so.
  • Burst List Page has columns with counts of Recipients and Tiles for each entry.
  • Metric Insights supports MS Graph API for email delivery.
  • If the DASHBOARD_EXPAND_SUBFOLDERS System Variable is set to "N", sub-folders do not expand when clicking on Show All.
  • Added the Live Dashboard option for Embed Code.

Security

Publishing Workflows

  • On the Content Center, the Power User only sees tabs for Categories
  • Added an Extended Security Privilege for this object

Privilege Sets

  • Allow an Admin to define a Privilege Set via its Editor

Categories

  • Going forward, the Privilege "Assign Category with View Access to Element" is no longer available for use by default on new installs; the Privilege can be made available to an instance by an Admin on the Privilege List Page using the Gear edit icon
  • New Privilege, "Allow user to display Categories on the Homepage" allows both Regular and Power Users to show Categories in the Left Sidebar and turn on/off on My Settings page

DevOps & Installation

All dependencies on Python 2.7 have been removed and code has been re-written to be compatible with Python 3.

  • Installer:
    • Ubuntu 16.04 is no longer supported.
    • Added support for Debian 11.
    • The --use-prior-command parameter can be used without specifying the --path option when MI is installed to a non-default folder, the installation path is detected automatically.
    • Added new parameters:
      • --hostname specifying an additional hostname for the Web Service.
      • Simple Install: --show-prior-command displaying the last used installation command.
  • MySQL: added gtid-mode parameter to the config files (especially relevant for AWS RDS).
  • Patcher: Added the Pre-action field that executes a script before applying a patch.
  • Custom Components: Added ability to create custom cron jobs by adding them to the /opt/mi/external_config/crontab directory.
  • Kubernetes, OpenShift:
    • Renamed the installer option’s value for OpenShift 3 from openshift to openshift3. Example command: ./installer.py kubernetes --type openshift3
    • Wizard options (--secrets-folder, -y, --db-hostname, --db-port, --db-user, --db-password) are now available from the installer with ability to generate the manifest file and secrets without user input to the command line.
    • Added a more detailed description  for the --registry option to the help section (-h) of the ./installer.py wizard.
  • Monitoring Service: All resources are loaded and updated asynchronously without full page refresh.
  • Added the latest stable releases of boto3 and botocore Python libraries to the web container.

API

  • Added API endpoint <MI hostname>/home/index/refresh?all_folders=Y to support display of folders hidden from the homepage.
  • Added API endpoints for getting all Report Types and changing Report Type (via API Toolkit):
    • /api/report_type returns all External Report Types.
    • api/external_report allows changing External Report Type via PUT request with all the required parameters and a report_type parameter with new value.
  • Added API endpoints to access and edit user Favorites (via API Toolkit): 
    • api/favorite_element returns id of ‘My Favorites’ category for specific user.
    • /api/favorite supports GET, POST and DELETE methods and allows retrieving, adding and removing favorite elements from all folders or a specific folder.
  • Added API endpoint <MI hostname>/api/search?q= which provides the same functionality as Global Search.

Issues Fixed

  • Tableau Filter Values that include “+” are now correctly transferred in URLs for Bursts. This was causing problems with the visualization display.  
  • Logs from Status Monitor and RDP can be successfully downloaded. The empty insight.zip archive issue does no longer exist.
  • In v6.3.0 there was an issue with Power BI Plugin not collecting objects on Refresh Object List. Now Report objects can be successfully retrieved.
  • Some of the Tableau Filters were not collecting Values. The system will be parsing the .hyper files when Filter Values are not found in XML files, like it was in this case.
  • Metric Insights no longer shows multiple results for the same user in the Users tab. This was a minor UI bug.
  • The issue with slow response and processing times in Content Center have been addressed by optimizing SQL queries, splitting the preparation of data for the initial display of the page into separate requests, and improving JS code for the Usage Chart.
  • Users now are not experiencing any random problems with not getting daily Bursts on schedule.
  • Embedded folders in Portal Pages do not show discoverable Elements to non-Admins, as our Security Model implies.
  • Datasets built using Dataset (SQL) as a Data Source can be successfully imported from one instance to another.
  • Inability to filter by an integer value in Power BI External Reports Viewers has been fixed in this release. 
  • Auto-update of ModSecurity which was preventing the web interface from working correctly has been disabled.