Collect Metric Data Incrementally
Data collection speed can be increased and load on source systems can be decreased by collecting metric data incrementally. Incremental data collection provides the ability to fetch either new or recent records from the source system and process only those records in Metric Insights (rather than the full historical dataset). This approach works well when data is only changing incrementally.
This approach can also be used when historical data is changing in the source system but you only need to worry about changes to recent history (for example, only changes in the last 7 days need to be processed and changes to older data can be ignored).
1. Using :last_measurement_time in fetch command to specify incremental data collection
In your Metric Data Fetch Command (SQLStatement text box above), include a :last_measurement_time bind variable which will be substituted at run-time with the last data point collected for the metric.
2. Re-fetching recent History for a Metric
If you want to re-fetch data going back a certain number of days before the last measurement time collected, you can do so by setting an offset in the Data Selection tab of the Metric Editor. This offset will be applied to the last measurement time associated with the metric to determine the :last_measurement_time substitution variable.