Help & Documentation
Open Support Ticket

Configure Search to Include Table and Column Metadata from Collibra

With the introduction of Analytics Marketplace in v7.1.0, discovery and search capabilities across multiple data catalog platforms has been significantly expanded.

You can search for the associated database table assets without leaving the Metric Insights UI and without recreating them as objects in Metric Insights. Global Search will return the assets with the list of the available metadata and links to proceed to the asset in the data catalog tool.

The process consists of the following steps:

  1. Create a Dataset to retrieve all Table Assets metadata into Metric Insights.
  2. Create a derived Dataset from the Table Assets Dataset to generate JSON objects for metadata fields available in the data catalog tool and include any additional custom fields specific to your environment. 
    NOTE: The Columns and Tags fields are already formatted in JSON.
  3. Add the Dataset to Search and map the Dataset Columns.
  4. Add Metadata Fields to Search — include the fields from Step 2 so they are indexed and displayed in the search results for each object, as shown in the final screen.

PREREQUISITES:

1. Create Dataset from Tables

For more details on how to create Datasets, refer to Understanding Datasets.

For more details on how to create a Dataset form Collibra, refer to Create Dataset from Collibra.

  1. Create a Dataset from the Tables object under All Data Assets.
  2. Note the Dataset ID to be used in Step 2.

2. Create Dataset from Tables Dataset

  1. Data Source: select the Data Source used as Data Storage when creating Dataset from the Collibra Tables object.
  2. Query: the query is intended to:
    • Set Object Type Column to "Table";
    • Transform standard Collibra fields, such as Status, Created By, Last Modified By, etc., into JSON objects.
    • Transform user-added fields into JSON objects.
      • This step will ensure that metadata is searchable and appears in search properly formatted.
      • The query must include the ID of Dataset created from the Collibra All Table Assets object in "FROM dataset_datasetID".
  3. Make sure the Dataset Column Length for "columns" is set high enough. This may be necessary if your table contains many columns or very detailed descriptions.
3. Example Query 
SELECT display_name
, asset_url
, domain
, "Table" as "type"
, columns
, JSON_OBJECT('Status',   status  ) AS "Status"
, JSON_OBJECT('Created By',created_by
, 'Created On',created_on
) AS "Created"
, JSON_OBJECT('Last Modified By',last_modified_by
, 'Last Modified On', last_modified_on
) AS "Last Modified" 
from   dataset_XXX

Acceptable JSON Formats:

{"Pii Data": ""}
{"Data Steward": "Clyde, Curt", "Business Owner": "Clint", "Technical Owner": "Kyle"}
[{"label":"Catalog","value":"Gifts"}]
[{"name":"Product Type","description":"High level grouping of our products."}]

5. Map Dataset Columns

Use the following mapping example with Allow click through to external tool enabled.

FieldDataset Column
Title Columndisplay_name
External Catalog TypeCollibra
External Catalog Asset URLasset_url
Content Type Columndomain
Object Type Columntype

For more details on configuring search across Datasets and Access, refer to Configure Search Across Specific Dataset (v7.1.0+).

6. Add Dataset Columns for Search

  1. Under Dataset Columns for Search, [+Add Dataset Columns].
  2. Dataset Column: choose a custom field column from the Dataset.
  3. Column data is formatted in JSON: enable. Must be enabled for all added columns, if multiple.
  4. Icon: optionally, select an icon.
  5. [Save] and add as many as needed.
  6. [Back to Search Setup] and [Update All Search Indexes]at the upper right.

7. Verify Result