Skip to content

HISTORICAL SHIFT CONSOLIDATED DATA VALIDATOR

Description

The HISTORICAL SHIFT validator is designed to check if a value that is usually the same has changed by a certain percentage within a specified number of days. This validator is particularly useful for monitoring changes in values over time, ensuring that significant shifts are flagged for review. It supports all numeric data types, making it versatile for various data validation needs.

Config Location

To configure the HISTORICAL SHIFT validator, you need to define it within the consDPValidators section of your data point configuration within the schema. This involves specifying the validator name and entity.

Supported Data Types

  • Double
  • Decimal
  • Integer

Config Requirements

Config ({}) options are required for the HISTORICAL SHIFT validator. If no configuration is provided or if the configuration is improperly set up, a block violation will be thrown. For more information on Apiro violations and their appearance in the Apiro UI or logs, refer to the violations section.

Example Config

Below is an example of how to configure the HISTORICAL SHIFT validator in XML format. This example demonstrates a validator that checks if the "PERCENTAGE_SHARE" data point has changed by more than 5.2% over the last 5 days. If this condition is met, a violation is triggered.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
<dataPoint name="PERCENTAGE_SHARE" dataType="DOUBLE">
    <consDPValidators>
        <consDPValidator name="CHECK_PERCENTAGE_SHIFT" entity="HISTORICAL_SHIFT">
            <config>
                <![CDATA[
                    {
                        "priorValues" : 5,
                        "percent" : 5.2,
                        "comparisonMore" : true
                    }
                ]]>
            </config>
        </consDPValidator>
    </consDPValidators>
</dataPoint>

Example Result

  • the HISTORICAL SHIFT validator will compare the current "PERCENTAGE_SHARE" value against its historical values over the specified number of days
  • If the change exceeds the specified percentage then
  • a violation will be triggered, indicating a significant shift in the value.

Config Parameters

name acceptable values comment
priorValues An integer representing the number of days to look back for historical values. This defines the time frame for the historical comparison.
percent A double representing the percentage change threshold. This defines the threshold for triggering a violation.
comparisonMore A boolean value indicating whether the change should be greater than the specified percentage. If set to true, the validator checks for changes greater than the specified percentage.

Common Mistakes

  • Ensure that the validator name and entity are correctly defined in the configuration to ensure that the validator is correctly identified and applied during the data processing pipeline.
  • Verify that the configuration parameters within the <config> tag are correctly written and match the expected format and values.
  • Remember that the configuration for the HISTORICAL SHIFT validator is required. If no configuration, or improper configuration is provided then a block violation will be thrown.