Skip to content

Best practices

  • Every entity descriptor is pushed to GIT. The files can be copied in a flat or unlimited nested directory structure.
    However, it is strongly recommended to follow the following structure.
  • create a directory /config/ and use it as the root directory for all your configuration files. Then the following paths should be used:
  • schemas: /config/schemas
  • dataSources: /config/dataSources
  • dataSinks: /config/dataSinks
  • dataFeeds: /config/dataFeeds
  • distributions: /config/distributions
  • dataBlockCollections: /config/datablockCollections
  • scriptables: /config/scriptables
  • rest Extensions: /config/restExtensions
  • api extensions: /config/apiExtensions
  • orchestrations: config/orchestrations
  • dataPointTemplates: dataPointTemplates
  • sysProps: /config/sysProps

API tokens and secrets

Apiro has support for enterprise vaults and secret managers like AWS KMS, Azure KeyVault or Cyberark but it also supports local system properties and secrets. It is strongly advised to always encode any keys that are placed in system properties as explained in the Authentication Manager page

Pipeline Data Processors

  • When creating data processors you should modify values only within the data point descriptor
  • Note: This point will be clearer once you complete section 8 of the getting started guide.
  • We can see two data point descriptions below, TFN and TFN_MASKED.
  • TFN_MASKED is a derived field because it masks the TFN value. We can see the processor TFN_HASH_MASKING is included in the TFN_MASKED data point descriptor and not in the TFN. This is a best practice to ensure that all processors that modify a specific data point are included in the data point itself and not update other data points. This ensures that it is easier to troubleshoot and understand what modifies a specific data point. 
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
          <dataPoint name="TFN" dataType="STRING"/>

      <dataPoint name="TFN_MASKED" dataType="STRING">
          <consDPProcessors>
              <consDPProcessor name="TFN_HASH_MASKING" entity="HASH_MASK">
                  <config>
                      <![CDATA[
                      {
                          "inputValue":"#GRV{ CTX['TFN'] }",
                          "maskingSalt":"akjahsdfkljhadsflkjhasldkfhakls"
                      }
                      ]]>
                  </config>
              </consDPProcessor>
          </consDPProcessors>
      </dataPoint>