Release 6.2.5

Metric Insights 6.2.5 is our second release of 2021. Its major focus is data governance—it includes a brand new lineage diagram, extensions to tagging capabilities, and an additional structural piece of our vision for Custom Fields, aptly called Custom Field Sections. Of those, by far the most notable to your users will be the wide introduction of the lineage diagram throughout the user interface—you'll notice it on Element Viewers, Editors, on list pages and in search. Keep in mind that that new functionality must be granted via new Privilege.

Major features in 6.2.5 (released May 12, 2021)

  1. Lineage for all core Metric Insights objects
  2. Tags can be optionally extended into full Governance Definitions (for use in representing your business glossary terms)
  3. Custom Field Sections function as containers for Custom Fields and provide a visual grouping throughout the UI. All Custom Fields in 6.2.5 must reside in a section.
  4. Global Search now indexes Portal Pages, includes new filtering, better performance, and more.
  5. Elements: descriptions now support markdown and provide further options for mobile display
  6. Filters & Dimensions now provide several additional option for sorting
  7. Datasets now include additional metadata; that metadata is now also displayed in the Dataset Viewer
  8. Burst engagement can now be tracked directly in the Burst Editor's "Run History" tab
  9. Portal Pages: A variety of changes have been introduced to both the UI and overall functionality—the two biggest changes being the introduction of tighter security and an all new "List" variable type.
  10. Our Tableau integration now supports automatic fetch for Filters and their values.
  11. Several new Plugins have been introduced
  12. Plugin and Data Storage enhancements have been introduced
  13. Security: With new objects come additions and changes to our Security model via new Privileges and Permissions
  14. Additions to the Status Monitor and System Configuration have arrived to list applied patches via a new tab on the Status Monitor, improve understanding of MySQL configuration, and set global system behavior via some new Config Variables.
  15. Our Import/Export tool becomes more robust with each release

Features in 6.2.5a (released July 1, 2021)

This minor release introduces several small improvements and bug fixes:

  1. Power Users can synchronize Portal Pages with Git. Before the release of 6.2.5a, this option had been available for Admins only. 
  2. Metric Insights can now generate PDFs of different sizes rather than just A4, as it used to. Users can configure the PDF page format on the Advanced tab in External Report Editors.
  3. The system can recognize extra white space in the collected image and automatically remove it (for Tableau only)
  4. From now on, if the “Collect additional files: PDF” option is enabled in External Report Editor a native PDF is available to us via the BI tool, then the system returns a native PDF for that External Report. Otherwise a PDF will be generated with a raster image/screenshot of the visualization.
  5. Plugins that support Setup Image can now be configured to collect images up to 20,000 pixels high (helpful if you have built very tall reports in your BI tool)
  6. With the new Optional Parameter, App Filter, plugin connection profiles can be configured to collect objects by either Application or Workspace. From now on users can create External Content linked to Applications from Microsoft Power BI Cloud.
  7. With improved security in mind, we have implemented the ability to use a service account and associated token to connect to Google BigQuery via Proxy.
  8. We have implemented the possibility to configure explicit and implicit FTPS connections. Explicit is set by default and implicit uses port 990. 
  9. Element Editors now have a slightly improved Object list display – less clutter thanks to the removal of extra App references.
  10. To avoid confusion, the updated Data Source Editor for MI Portal Page plugin includes a clearer message that specifies that a group must be set in order to get an object list. 
  11. We have developed a new plugin, Zendesk Explore, with standard image capture functionality. 

Issues fixed:

  • Issues with the display of External Reports in iframe after customizing Tableau toolbar and tabs have been fixed.
  • URLs from Bursts that were redirecting users to the wrong Category after passing SSO have been fixed.
  • In situations where where the database timezone is different than the server, measurement_time in dataset_instance is now properly handled.

The primary Docker images used for standard deployments are:

  1. seed - docker.metricinsights.com/seed/release_6_2/41
  2. monitoring - docker.metricinsights.com/monitoring/release_6_2/42
  3. data-analyzer - docker.metricinsights.com/data-analyzer/release_6_2/121
  4. dataprocessor - docker.metricinsights.com/dataprocessor/release_6_2/105
  5. web - docker.metricinsights.com/metric-insights/release_6_2/507

Optional images:

  1. mysql - docker.metricinsights.com/mysql:release_6_2
  2. logger - docker.metricinsights.com/logger/release_6_2/58

Support for Metric Insights v5.x is ending with the release of v6.3.0 (tentatively scheduled for Summer 2021).

Lineage

The new Lineage Diagram is available on all key objects:

  • Elements, both Viewers and Editors, and the Elements List
  • Datasets, both Viewer and Editor, and the Datasets List
  • Data Source Editor and Data Sources list
  • Burst Editor

Lineage allows anyone with the privilege (called "Show Lineage Icon in Search and on Viewers") the ability to navigate upstream and downstream objects within Metric Insights. Starting in v6.3.0, objects upstream of Metric insights will be supported.

Tags as Governance Definitions

Tags can now be transformed into Governance Definitions—effectively making a tag more meaningful and metadata rich. A Tag that's been deemed "Governance Definition" includes the follow additional fields:

  • Tag Type (example: Enterprise KPI)
  • Definition (example: Key enterprise performance indicators)
  • Description (example: These KPIs are regularly reviewed by executives and below and are reviewed most rigorously by the Enterprise Data Governance team)
  • Business Owner
  • Technical Owner
  • Data Steward

Custom Field Sections

Custom Field Sections and their associated privilege allow Custom Fields to be grouped together, and to managed by multiple Power Users, as a grouping of fields.

Additionally, the display of Custom Fields in the Element Viewer has been revised. They are no longer displayed in the footer, and have instead be pushed up into the actions bar in the upper right, next to the Lineage icon, the Favorites star, and other key functionality.

Starting v6.2.5, Global Search is available by default with no need for manual configuration. Global Search has become faster, more sophisticated, and more flexible, compared to previous versions.

Search results now list matching Tags at the top of the page so that users can easily apply them as filters.

Portal Pages have become searchable, with the possibility for users to decide whether or not a page will appear in search results. The search for Portal Pages supports such queries as names, keywords, descriptions, Tags, text appearing on a page, and iframe URLs. Query Language filters are also implemented.

Not only Portal Pages can be configured to be visible in Search, but other Objects too. Nearly all objects, via their Editor, now include a setting determine whether that object can be found via Global Search. Here's the list of them:

  • Category Editor (toggles the setting for all Elements & Datasets in that Category)
  • Dataset Editor
  • Metric Editor
  • Multi-Metric Editor
  • Report Editor
  • External Report Editor
  • External Content Editor

We have enhanced External Tool Search to support opening the tool in a separate tab by default instead of embedding it in Metric insights. Both of the options are available and the behavior can be configured according to the needs of your organization.

Global Search capabilities have been expanded to support searching by Custom Fields and their values using the Query Language logic. For example, if there is Custom Field "color" with values "red", "blue", and "green", users are able to construct the following query color: blue.

See Configure Intelligent Global Search and Utilizing Intelligent Global Search for more details.

Elements

  1. Markdown: Metric Insights now recognizes Markdown in the Element's Description field. Users can add text links and provide other formatting like bold and italic. Lists, headings and other more advanced Markdown features are NOT supported.
  2. External Reports: Users have the ability to hide filters from External Reports in the mobile app by selecting the Hide Filters on Mobile option on the External Report Editor's Advanced tab.

Filters & Dimensions

  1. The order of manually entered Filter values for External Reports can be manually changed via a drag and drop interface.
  2. Users can set up sorting for Filter Values, regardless of the value source, with the following options:
    • Inherit from source
    • Ascending
    • Descending
  3. Dimension dropdowns now use the Display Value rather than the key value

Datasets

As another effort in data governance optimization, we have added a better representation and visibility to Dataset details and origin. In this version, Dataset Viewers include an info control that displays the following information:

  • Description
  • Certification information
  • Ownership
  • Category
  • Tags
  • Data last loaded

Dataset Editors now display Business Owner, Data Steward, and Technical Owner names on the Info tab.

Bursting

Burst publishers can now track Burst opens and link engagement (links clicked in an email) in the same way that tools like Hubspot and Mailchimp are able to. Bursts have to be configured for tracking individually.

Portal Pages

A variety of changes have been introduced to both the UI and overall functionality — the two biggest changes being the introduction of tighter security and an all new "List" variable type.

  1. We've introduced a couple UI enhancements to Portal Pages:
    1. they have been moved to the Content drop-down menu
    2. the Portal Page Editor and Portal Page Template Editor boast a cleaner UI.
  2. Several changes have been introduced to lock down security for Portal Pages by default.
  3. Two new Portal Page Variables have been added:
    1. There is a new List variable type that allows users to manually create a list of items without JSON. Users can drag-and-drop easily reorder these items.
    2. With the new HOSTNAME Global Variable, we have alleviated the need to hardcode the hostname in URLs. This would significantly simplify the migration process by reducing the amount of manual input.
  4. JSON Web Tokens (JWT) can now be retrieved for React applications (only applicable to Advanced Portal Pages)

Tableau Integration

A few key quality-of-life improvements have been made to our Tableau integration:

  1. Automatically retrieve filter names and values for Elements, Datasets, and Dimensions
  2. Customize Tableau toolbar and tabs display and pass these parameter settings to URLs in External Report Editors
  3. Implemented a new license request when embedding Tableau Dashboard in iframe on Portal Page
  4. Pull in and apply Technical Owner from Tableau when using Content Auto Sync
  5. Display the last Tableau refresh time in real-time in External Report Viewer

New Plugins

Several new plugins have been added in 6.2.5:

  1. Google Data Studio Plugin that captures data and images. Create Datasets, Reports, and External Reports.
  2. Superset Plugin with image capture functionality for now.
  3. Google Search Console Plugin for creating Datasets and Reports.
  4. Sumo Logic Plugin with the ability to create Datasets and Reports. The plugin utilizes our native MIQL to construct queries.
  5. Apple App Store Connect Plugin that now allows your organization to pull data from the iTunes store, particularly about the performance of different apps.

Other Plugin Enhancements

  1. With improved user experience in mind, we are starting to add more selectable lists/dropdowns instead of manually entered text strings in all plugins, just like we have done for SharePoint Plugin.
  2. For Data Storage, there is now a new option for Data Upload Type that allows to upload data directly via Local Folder in a container.
  3. Disabled Datasets created from SQL Data Sources now have a better representation on the Datasets tab with the corresponding legend.
  4. Every Plugin Data Source Editor now displays its plugin version and JAR file name on the Info tab.

Security

For current information (up to and including 6.2.5) on all Privileges, see Privileges List: Rel. 6.1.3 and beyond

  1. Tags
    • Two new Power User Privileges
      • Create Tags:
        • Allows PU's to access the Tags list page and create/manage Tags that a user creates
      • Create Governed Tags: This child of Create Tags allows PU's to:
        • Access the Tag Type list page and create/manage Tag Types that this user creates
        • Change whether or not a Tag that the user created is governed, substitute Tag Types and change the designated Business Owner, Data Scientist and Technical Owner
        • Use "Bulk change" on Tag LIst Page to replace existing Tag Type of selected Tags
    • Without these Privileges, a PU:
      • Cannot create or change Tags or Tag Types or make a Tag "governed"
      • Can use any available Tag when Editing an element or Dataset to which the PU has Edit Access
  2. FTP Connection
    • The revised Create FTP Connections Privilege allows PU's to:
      • Access the FTP list page and manage Connections that this user creates
      • Share the Connection with Groups that the PU has Permission to Edit or to which User belongs as well as to individual PU and RU Members of those Groups.  
      • The Group's members and/or designated  users with whom a Connection has been shared may select it for results storage when creating or editing a Burst.
  3. Custom Field Sections
    • Two new Power User Privileges:
      • Create Custom Field Sections
        • Allows PU's to add new Custom Field Sections, maintain settings/content and grant Edit Access via the Permissions button to Groups of which User is a member and with other PU's in those Groups
      • Allow Power Users to grant Custom Field Section Edit Access to any Power User or Group
        • Expands Power User's authority to allow granting Custom Field Section access to any Power User or Group
    • Only Admins can create Custom Fields
  4. Lineage
    • The new Show Lineage Icon in Search and on Viewers Privilege allows a PU or RU to access a  Lineage diagram from Search as well as Element and Dataset/User Map Viewers
  5. KPI Alerts
    • Two new Power User Privileges:
      • Create KPI Alerts: Allows:
        • PU's and RU's to add new KPI Alerts and maintain settings/content
        • PU's to use Permissions button to grant Edit Access to Groups of which User is a member and to other Power Users in those Groups
      • Allow Power Users to grant KPI Alert Rule Edit Access to any Power User or Group
        • Permits PU's to  grant KPI Alert Rule access to any Power User or Group
    • KPI Alert capabilities are now uncoupled from Create Bursts and Modify My Settings so that these Privileges are no longer required
  6. Sort Categories
    • The new Sort Categories Privilege allows PU's to sort Categories on the List page as well sort a Category's children (Subcategories) on its Editor.
      • In addition to this privilege, the PU must also have Edit Access to the Category to perform these functions.
  7. Datasets
    • Provides the ability to Bulk Change User and Group access from the Dataset /User Map List pages
      • Includes both Grant and Revoke options
      • Allows Dataset Edit Access to be granted to either Groups or Power Users
        • If Edit Access is selected, also grants Use Access of the Dataset's Data Source
  8. Global Documents
    • Provides an ability to designate a Document as as "Global Document"
    • Displays a new "Visible To" setting that allows sharing with one of more types of Users
    • Provides a special menu on the upper toolbar that only contains Global Documents if any have been shared with the user
      • If only "Groups" is selected, an Admin must be a member of Groups to which a Global Document is visible
  9. Portal Page Security
    1. A new toggle ensures that Portal Pages are only accessible to the Groups members and individual Users with whom the page is shared.
    2. A new setting that  Sharing to be limited to one to three User Types.  These additions are explained below.
  10. Bulk Change of  Access to Datasets/User Maps
    1. Provides two new options to grant or revoke Edit/View Access to Datasets and User Maps

Status Monitor & System Configuration

Status Monitor:

  1. Enhanced to include a section for Datasets with Errors on the Errors & Logs tab.
  2. Includes a new tab that displays the history and details of applied patches.
  3. Shows MySQL settings with highlighted values in case they significantly differ from the recommended ones. Settings are displayed in the priority order, from highest to lowest.
  4. If MySQL is running on localhost, users will see a corresponding note so that they have more control over their production environments.

System Variables:

  1. A new System Variable, WARN_IF_NO_ACCESS_DENIED_MESSAGE, allows users to configure showing a notice that no custom access request message has been set for Element or Category after its been made discoverable.
  2. A new System Variable, DATASET_DEFAULT_MAX_INSTANCES_NUMBER, allows users to manually specify how many Snapshot Instances they want to keep.
  3. Users are able to track views of Public Elements to reflect more accurate statistics and have better understanding on Element exposure.
  4. Two new options, Group Access and User Access, have been added to Dataset List's Bulk change option, allowing users to select multiple Users or Groups to grant or revoke access to selected Datasets.

Import/Export

  1. Email Templates can now be migrated.
  2. Metric Insights supports migration of Custom Field Sections, along with all Custom Fields and all Elements.

Miscellaneous

  1. Category Hierarchy Limits have been asked about, and although no specific changes have been implemented in the scope of 6.2.5, we now have documentation on the subject. In short:
    1. A parent Category at any level may only have 10 Subcategories (horizontal limit)
    2. The number of offsprings of a Category/Subcategory is unlimited (vertical limit)
  2. FTP Connections can now be given custom names for better representation and convenience.
  3. Access Requests: An Access Request ID column is now included in the Access Request page for reference.

API

A new API endpoint, user_preference_catalog, with two supported methods GET and PUT, is added to control default start folder position in Catalog for all users that are members of a specified group.

Issues Fixed

  1. External Filter in Element Viewer shows Dimension Value Names instead of Dimension Value Keys, as it used to.

Additional Notes & Recommendations

  1. For those using Data Storage, we recommend Microsoft SQL Server 2019.

  2. For troubleshooting TLS issues, use a new parameter tlsVersion. The default value is TLSv1.2.
    The value needs to match the TLS version on the server.