Collect Metric Data Incrementally
Data collection speed can be increased and load on source systems can be decreased by collecting Metric data incrementally. Incremental data collection provides the ability to fetch either new or recent records from the source system and process only those records in Metric Insights (rather than the full historical Dataset). This approach works well when data is only changing incrementally.
This approach can also be used when historical data is changing in the source system but you only need to worry about changes to recent history. For example, only changes in the last 7 days need to be processed and changes to older data can be ignored.
1. Using ":last_measurement_time" Command
The :last_measurement_time command can be used in fetch command of Query field to specify incremental data collection.
Include a :last_measurement_time bind variable in your Metric Data Fetch Command (Query field). It will be substituted at run-time with the last data point collected for the Metric.
2. Re-Fetching Recent History
If you want to re-fetch data going back a certain number of days before the last measurement time collected, you can do so by setting an offset in the Data tab of the Metric Editor. This offset will be applied to the last measurement time associated with the metric to determine the :last_measurement_time substitution variable.
NOTE: The On data collection also re-run last field shows up only if the data is collected from the Dataset, that Dataset is snapshotted and Calculate Metric Values for field is set to "Only new data".