Release 6.3.3

Our second significant release of 2022 is out! This time, we've focused our efforts around strengthening data governance by introducing a business Glossary, made up of Terms and Sections. Terms can be applied to Metric Insights content (both manually and via an auto-sync process) and should be used to increase data literacy.

We've also introduced a new Business Unit object, to which Groups can be assigned to represent the real structure of your business, with department/LoB/division/unit as the parent of user Groups.

Also: Microsoft SharePoint is a new destination for Bursting, External Filters now support free text entries and dates as Filter Types. And last but not least, Metric Insights is available on AWS Marketplace (for the second release in a row, and for all releases to come). There are dozens of additional enhancements and changes, so continue on below.

The RDP installer has been migrated to Java 11. Version 8 is no longer supported.

Noteworthy changes to plugins and JDBC drivers:

Added to MI Removed from MI
Denodo JDBC Driver Beckon Plugin

Re:dash Plugin

Major features in 6.3.3 (Released May 20, 2022)

  • Glossary Terms have been added as a new data governance object, replacing "Governed Tags"
  • Business Units can help to organize Groups so that they better reflect the organizational structure of your business. Think of them like groupings of Groups.
  • Text and Date Filter types provide for more flexibility while working with External Report Filters.
  • Distribution to Microsoft SharePoint Site is now possible, complementing the list of existing distribution channels.
  • Restoring deleted Elements is now easy with the new Trash page, where users can bring objects back in a couple of clicks.
  • Both Folder and Category Editors now include a History tab that keeps a record of all sharing and permission related activity.
  • Metric Insights is available on AWS Marketplace.
  • Our Google Looker Plugin supports passing External Filters to Metric Insights.
  • In addition to Element and Constant Prompts, MicroStrategy Plugin now supports Expression Prompts as External Filters.

The primary Docker images used for standard deployments are:

  1. web – docker.metricinsights.com/metric-insights/release_6_3/185
  2. dataprocessor – docker.metricinsights.com/dataprocessor/release_6_3/21
  3. seed – docker.metricinsights.com/seed/release_6_3/11
  4. data-analyzer – docker.metricinsights.com/data-analyzer/release_6_3/44
  5. monitoring – docker.metricinsights.com/monitoring/release_6_3/42

Optional images:

  1. logger  – docker.metricinsights.com/logger/release_6_3/42
  2. mysql – docker.metricinsights.com/mysql:release_6_3

Business Glossary

In this version, we have replaced Governed Tags with a more robust Governance object, Glossary Terms, which can be used to construct a business glossary. Users can apply Terms to Metric Insights content to identify the data that resides within. For example, a Global Sales Analysis Report will likely have "Global Sales" applied to it as a specific business definition (e.g. "Total sales across all geographic regions and all markets"). This new feature makes it possible to establish a well-rounded data governance program. 

Glossary Terms can be applied to all Element types, but not Data Sources, Datasets, Portal Pages, or Bursts. For user convenience, we have added the new Glossary Terms Filter to the Elements List Page.

Business Units

A new Business Unit object provides for a more accurate representation of your business, functioning as a grouping of Groups that can be used to represent lines of business or departments in your organization. A Business Unit can be associated with one or more Groups. A Group may be assigned to only one Business Unit. Any Admin may create a Business Unit, but a Power User requires a Privilege, Create a Business Unit, to do so.

See Create a Business Unit.

Text and Date External Filters

External Filters now support free text entries as Filter Values. Users can set Filter defaults from Editors by simply entering a text value and these text Filter Values will also be supported from Element Viewers. Text Filters support only one value at once. The condition that still applies to External Filters and text Filter in particular is that their Values must exist in a BI tool in order to work on the Metric Insights side. This feature works with Filters regardless of the option that is being used to bring them to the system. 

The new Date Picker Filter type brings a calendar to filter data display by a particular date. Date Format selected in Metric Insights must correspond to the one configured in a given BI tool. Date Picker works only with External Reports.  

See Filtering in External Reports.

Distribution via Microsoft SharePoint

The list of distribution options for bursting has been expanded to include SharePoint Site. For channel configuration, access Admin > Distribution > Microsoft SharePoint.

See Deliver a Burst via FTP, Shared Drive, SharePoint Site.

Trash Page

From the new Trash Page available under the Content menu, you can easily restore deleted Elements.

Audit Logs for Folders and Categories

The new History tab has been added to Folder Editor. This tab lists all sharing change records, with the recent ones appearing at the top. Similarly to Folder Editor, Category Editor now includes the History tab too, logging Category assignments between Users and Groups.

Other External Content

  1. Similar to External Report Type, there is now a standalone Editor for External Content Type. When creating or editing Other External Content, users can access it directly from the External Content Editor.
  2. The new System Variable, MAX_FILE_SIZE_FOR_PREVIEW_AND_VIEWER, allows users to set the max file size for documents (Excel, Doc, PPT, XML, etc.) in KB to display in External Content Viewer and Tile Preview.
  3. The option “Include files as email attachments” for Other External Content is now included in an Editor by default. ATTACH_EXTERNAL_CONTENT_FILES_DEFAULT controls whether this option defaults to “Yes” or “No” in the Editor.
  4. VIEWER_MAX_FILE_TO_IMAGE_CONVERSION_TIME_SEC sets the maximum time limit to generate file preview for a Viewer in seconds. If the application cannot generate previews within this time, the file will be removed from the processing queue and Users will be presented with a “download file” option, skipping the preview.
  5. EDITOR_MAX_FILE_TO_IMAGE_TIME_SEC sets the maximum time limit to generate a preview image for a file in seconds.
  6. FILE_TO_PDF_PAGE_MAXIMUM defines the maximum number of pages to process when generating a PDF from an Excel, Doc, or PPT. For large Excel files, Metric Insights will only generate the specified number of pages.

Release on AWS Marketplace

Metric Insight's AWS Marketplace listing is live at this link. Follow the instructions on how to deploy the application in AWS Marketplace by using an AMI.

Plugin Improvements

Microsoft Power BI Cloud

  1. The EXTERNAL_REPORT_NO_TOKEN_MESSAGE System Variable determines what message will users see when an External Report’s token has not been generated or the token is expired.
  2. There is a new grid “User Tokens” on the Objects tab in the Plugin Data Source Editor which shows the token issuer and expiry date and time. A token can be deleted by clicking on the trashcan icon.
  3. For Power BI External Reports that have been generated from a Data Source with OAuth enabled, Metric Insights now shows the “Preparing preview image. This could take a while.” message when opening a Tile Preview for a particular user on demand.
  4. The new Optional Microsoft Power BI Cloud Parameter, OleDB Data, can be used for getting data from OleDB instead of DAX API.

MicroStrategy

Aside from Element and Constant Prompts, MicroStrategy Plugin also supports Expression Prompts as External Filters.

Google Looker

  1. Looker Plugin has been updated to work with API v4.0.
  2. The Plugin now also supports passing External Filters to Metric Insights.

Other Plugin Improvements

  1. Optional Parameter “Folder Filter” for SAP Business Objects has been renamed to “Folder ID Filter”.
  2. In IBM Cognos Data Source Editors, “Cognos URL” has been renamed to “Dispatch Endpoint” and “Server” has become “Server URL”.
  3. Local Filesystem Plugin now provides a 2-tier structure of references (path and file).
  4. Beginning in this version, Domo Plugin supports working via proxy.
  5. Instead of displaying nothing like before, Metric Insights now passes "Fetched 0 [Plugin Name] objects, confirming that Filter Parameters on Info tab are correct." This indicates that the attempt to get objects has been successful, but there are no objects that match the parameters.

External Reports

  1. Aside from hiding drill-through links from External Report Viewers, users can now set custom text for the links. This possibility is controlled via an External Report Type Editor.
  2. Metric Insights will notify users if External Filters are used by other objects in the system and restrict changing Filter types from Editors of these objects. This will help make informed decisions about changing Filter Types.

Datasets

Now there is an easy way to find out if a given Dataset is used as a Portal Page Entity. The Editor lists such associations on the View & Elements tab.

Custom Fields

  1. In the Custom Field Autosync Editor, when switching to another Dataset via “Dataset & View” drop-down, the column choice will persist in the mapping if that Dataset contains one or more columns that are present in the original Dataset.
  2. There is now a possibility to duplicate a Custom Field Autosync profile, along with all the Fields.
  3. In Custom Field Editor, from now on the Advanced section will be available only if “Enable Custom Field Data Synchronization” is enabled for at least one SharePoint Data Source. 
  4. Power Users and Admins who have permissions to edit Custom Fields can now access a Custom Field Editor directly from Elements, Datasets, and User Maps.

Data Analyzer, the Metric Insights component that is responsible for search performance, has been further enhanced to deliver improved robustness, accuracy, and promptness. Some additional improvements include:

  1. We have changed the searchability logic for Bursts. This allows the Owner filter to properly filter results.
  2. The logic for capturing the last index update has been improved to take into account all index update triggers: manual, automatic, and programmatic.

Bursts

  1. By enabling the “Append date/time to files” option for Bursts, you can configure the system to append the date when a file has been sent to attachments of any type regardless of the distribution method.
  2. There is now a message in a Burst Editor that notifies how many Users don’t have a Power BI token and, as a result, will not see images in their Burst.

Content Center

On the Behavior tab, if more than 100 Tiles display they will be grouped by a Category. Instead of displaying individual Tiles, Metric Insights will display the Category they are included in for a more compact view.

Feedback

  1. In an Element Viewers, the manual Feedback option will now allow users to trigger Feedback submission form only once per view. 
  2. Users can purge existing Feedback to remove historical scoring. Purged Feedback won’t be deleted but is not going to be calculated in scores.

File Groups

Now users have the ability to download arbitrary files that are not identified by SharePoint Plugin as supported if they are selected within a File Group.

Security

Business Unit:  A new Privilege, "Create Business Unit" allows a Power User to define and maintain a Business Unit object.  A Power User with "Create Groups" Privilege and Edit Access to a Group may assign any available Business Unit to a Group.  Change history is tracked as Groups are added and removed from a Business Unit.

Tags: The "Create Tags" Privilege remains as a standalone Privilege, but the "Create Governed Tags" child Privilege and all of its related functionality has been removed.

Glossary Sections and Terms: Two new Privileges added:

  • (Parent) "Create Glossary Sections and Terms" allows a Power User to define and manage Sections and Terms.  These are customer-defined objects used to covey information within the organization of about its data in MI and how MI features and functions in relation to this data.  The Glossary is not maintained by MI and is delivered unpopulated.
  • (Child) "Allow Power Users to grant Glossary Section or related Term access to any User or Group" expands a Power User's authority to allow granting Glossary Section or related Term access to any User or Group

Configurable Data Source:

  • A config variable MODIFY_UNPRIVILEGED_DATA_SOURCES_VIA_API is available to allow organizations to bypass the restriction that requires a PU to have the Create Data Source  Privilege to modify a Data Sources by Power Users with Edit Access. Its Default Value is "N"; i.e., the restriction remains in place.

 Group/Power User Editors:

  • Added Permissions grids to "Other Access" tabs on both Group and User editors to show objects to which the Group's users or individual users have been granted access:  
    • Folders
    • FTP Connections

System Variables

  1. If set to “Y”, HIDE_DISCOVERABLE_CONTENT_FROM_HOMEPAGE hides discoverable content from Homepage as Tiles. Those discoverable Tiles can only be searchable.
  2. SAML_LOADING_SCREEN_OPTION allows for selecting which option to display when users are being synced with an SSO/SAML provider  a text message or a loading indicator.

Other Improvements

  1. Metric Insights now allows for bullet-pointed lists in Announcements.
  2. We have added a possibility to turn branded rows on and off for Dataset Report Tables. Under the Table settings, there is now a checkbox “Display banded rows” that controls this possibility.
  3. New User and Administrator Defaults provide a possibility to set system language for new Users.
  4. Usernames can now include up to 255 chars.
  5. Migration Utility supports import/export operations for Glossary Terms. 
  6. Vocabulary that previously had been available only at <hostname>/admin/vocabulary, is now added to the Admin menu  Admin > System > Vocabulary
  7. The Status Monitor page now displays the license key expiration date. Users can find it in the right-most column, below the “Installed” date.
  8. The new logic that is effective beginning in this release is that the theme defined under the System (default) brand is applied to the login page.

DevOps & Installation

  1. Orchestrated Deployments: Added a new database dashboard_analyzer for the data-analyzer service to increase the performance of searching process and execute multiple search queries simultaneously. Before updating to 6.3.3, update the secrets for the web and data-analyzer services:
    • Add a new secret MYSQL_DA_PASSWORD to web.env
    • Change the user for the data-analyzer service in data-analyzer.env:
MYSQL_PASSWORD=<MYSQL_DA_PASSWORD>
MYSQL_USER=mi_data_analyzer
  1. Metric Insights is available to be deployed into own AWS account from AWS Marketplace.
  2. Added ability to sync user groups from Office 365 using mi-o365-usersync script.
  3. Updated Metric Insights docker images to use Debian 11 as basic Operating System.
  4. Security scan results are added to readme.md file of installation packages.
  5. Added status subcommand for service command to display current status of services.
  6. Simple Install:
    • Beta version identification and installation status are displayed in mi-version output.
    • Updated docker-compose to V2.
    • Added --setup option that installs and configures system packages without application installation.
    • Removed support for CentOS 8.
  7. Patcher: Updated the mechanism of applying patches, the Patcher checks whether the patches are fully downloaded before applying them.
  8. ECS Terraform: Added --enable-logger option that stores logs in /opt/mi/log/ directory of each service.

Issues Fixed

  1. The issue with the subscribed User's email address is missing in the "Send To" column has been fixed. It was a minor UI problem with the grid that did not affect sending or receiving Alerts.
  2. The connection issue with Snowflake has been resolved. 
  3. An issue was that prevented Power BI Report Server PDF downloads from being collected at native/full quality has been fixed.
  4. We have also fixed cropping behavior for Power BI Report Server images for proper display in Metric Insights.
  5. For Metrics created from a Dataset, the Count Distinct aggregation had a bug that was fixed.
  6. The Favorites page under Content is successfully loading for Admin users after applying the favorite.py script.
  7. The link on the New Feedback Prompt popup is now leading to the configuration document.
  8. Metric Insights can trim extra spaces in the Server fields of Data Source Editors to ensure proper URL building.
  9. Power Users can successfully create a Feedback Prompt and assign it to a Group.
  10. Dataset Reports appearing as disabled when the source Datasets were enabled have been fixed. It was a minor UI problem that did not affect the actual status of Reports.
  11. "Unexpected Error" when attempting to upload a PDF file as Expert Analysis and as "New Document" has been fixed by changing the default Ingress Controller settings for K8s deployments.
  12. The “Invalid date” error occurring when :measurement_time is used in SQL command has been addressed.

Known Issues

Tableau Plugin will display java.lang.NullPointerException upon Object Refresh if users have Workbooks in Personal Spaces. Contact our support to assist with a patch.