Skip to content

REST - DATA SOURCE

Description

The REST Data Source allows for the integration of external RESTful APIs into your application, enabling the retrieval of data from various services over the internet. This method is particularly useful for accessing data that is not stored locally but is available through a RESTful API endpoint.

Config

REQUIRED

Config Parameters

name description
url The URL of the RESTful API endpoint. This is a required parameter.
method The HTTP method to use for the request (e.g., GET, POST). This is a required parameter.
headers An object containing any headers required by the API. This is optional but may be necessary for authentication or specifying content types.
requestParams An object containing any query parameters required by the API. This is optional and depends on the API's requirements.
substitutions An object containing values to be substituted into the URL or request parameters. This is optional and can be used for dynamic values.
cascadeSub The name of the substitution to cascade through multiple pages. This is optional and used for pagination.
cascadeMax The maximum number of pages to cascade through. This is optional and used in conjunction with cascadeSub.

Config Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
<?xml version="1.0" encoding="UTF-8"?>
<apiroConf version="1" xmlns="http://apiro.com/apiro/v1/root">
    <loadOrder>10</loadOrder>
    <dataSources>
        <dataSource definition="REST" execPriority="10" name="TEST_REST_SOURCE">
            <config>
                <![CDATA[
                      {
                          "url" : "https://sxvk0vpxh4.execute-api.ap-southeast-2.amazonaws.com/cache/apis.esganalytics.io/${SYS:ESGNL_PRODUCT_ID_500k_limit}/securities/",
                          "method" : "GET",
                          "headers" : {
                            "X-BLOBR-KEY" : "${SYS:ESGNL_API_KEY}"
                          },
                          "requestParams" : {
                            "page" : "[PAGE]"
                          },
                          "substitutions" : {
                            "PAGE" : "1"
                          },
                          "cascadeSub" : "PAGE",
                          "cascadeMax" : 2
                      }
                ]]>
            </config>
        </dataSource>
    </dataSources>
</apiroConf>

Inline Feed Source

A rest feed can also be using inline within a dataFeed. This inline version uses the same parameters. The payload for each data point is accessed using Json list path operators. An example is given below:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
<dataFeeds> 
    <dataFeed definition="EXPR_JSON_FEED2" name="ESG_ANALYTICS_PULSE_REST"> 
      <execPriority>10</execPriority>  
      <enabled>true</enabled>  
      <push>true</push>  
      <pull>true</pull>  
      <schema>ESG_ANALYTICS_PULSE</schema>  
      <config><![CDATA[{
  "dataSource" : {
    "config" : {
      "url" : "https://apis.esganalytics.io/${SYS:ESGNL_PRODUCT_ID_500k_limit}/company/",
      "method" : "GET",
      "headers" : {
        "X-BLOBR-KEY" : "${SYS:ESGNL_API_KEY}"
      }
    },
    "entity" : "REST"
  },
  "itemLimit" : 200,
  "listPath" : "$.results",
  "explicitMappings" : [ {
    "dictionary" : "ID",
    "value" : "#GRV{PAYLOAD.resolve('id')}"
  }, {
    "dictionary" : "ROOT_DOC",
    "value" : "#GRV{PAYLOAD.resolve('$')}"
  }, {
    "dictionary" : "COMPANY_PULSE",
    "value" : "#GRV{PAYLOAD.resolve('$.company_pulse')}"
  }, {
    "dictionary" : "COMPANY_NAME",
    "value" : "#GRV{PAYLOAD.resolve('$.company_name')}"
  }, {
    "dictionary" : "GICINDUSTRY",
    "value" : "#GRV{PAYLOAD.resolve('$.gicIndustry')}"
  }
  ]
}]]></config> 
    </dataFeed> 
  </dataFeeds>

Common Mistakes

  • Incorrect URL: Ensure that the url parameter is correctly specified and points to a valid RESTful API endpoint.
  • Incorrect HTTP Method: Verify that the method parameter matches the API's requirements (e.g., GET, POST).
  • Missing Headers or Parameters: If the API requires specific headers or query parameters, ensure they are correctly specified in the headers and requestParams objects.
  • Incorrect Substitutions: If using substitutions, ensure that the substitutions object correctly maps to the placeholders in the URL or request parameters.
  • Cascade Configuration: If using cascading for pagination, ensure that cascadeSub and cascadeMax are correctly configured to avoid excessive requests.