Configure Data Dependencies for a Data Collection Trigger

Data Dependencies are typically associated with a Data Collection Trigger in order to ensure that data is available and current in the source system through the collection cycle specified in that Dependency.

PREREQUISITES:

Examples of Data Dependency Checks:

  1. Check that data in a Warehouse Customer Order table contains records post midnight of the prior day (which indicates that all of the prior day's orders have been loaded).
  2. Verify that an ETL control table specifies that ETL processing has been completed since the last data collection cycle.
  3. Verify that volume in a data table is at least X% of the volume of data in a prior day as a data qualify check before loading data.

Video Tutorial

1. Add a New Data Dependency

  1. Click [+ New Data Dependency] in the Data Collection Trigger Editor Dependencies tab
  2. The Add Data Dependency pop-up opens
  3. From the drop-down list select the Add New Data Dependency option.

2. Define the basics for the New Data Dependency

Define the basics for the New Data Dependency

After the new Data Dependency is created, the Data Dependency Editor opens.

3. Enter Data Dependency Specifications

  1. Make sure your Data Dependency is 'enabled'
  2. Specify the Data Source for the Data Dependency
  3. Provide a fetch command that will return a positive number when the dependency is satisfied (you can use :last_dependency_satisfied as a bind variable)
  4. Validate your fetch command

3.1. Specify Last Dependency substitution

Specify Last Dependency substitution

If :last_dependency_satisfied was included in the fetch command, provide a value for testing the data dependency logic

4. Below the statement specify when and how often to perform the Data Dependency check

Below the statement specify when and how often to perform the Data Dependency check
  1. Specify how often the dependency check fetch command should be run.
  2. Provide a time window during which checks should be performed.
  3. Specify a time period after which the dependency check should be expired.

Enable Dependency at the top right corner of the screen.

This Data Dependency can now be added to the Data Collection Trigger

NOTE:

  • A given dependency can be shared among multiple Data Collection Triggers
  • Multiple dependencies can be specified for a Data Collection Trigger and all associated dependencies must be satisfied before the data collection trigger starts

Control when Data Dependency of Data Collection Trigger is Satisfied

If your Data Collection Trigger's interval is an hour or longer, you can control the circumstances under which an associated Data Dependency must be satisfied by accessing the Data Collection Trigger Editor and editing an existing dependency.

0 Comments

Add your comment

E-Mail me when someone replies to this comment