Skip to content

S3 - DATA SOURCE

Description

The S3 data source is designed to retrieve data from Amazon Simple Storage Service (S3), a scalable object storage service offered by AWS. S3 is widely used for storing and retrieving any amount of data at any time, making it an ideal choice for applications that require high durability, availability, and scalability. The S3 data source supports various configuration options to customize the data retrieval process, including specifying the AWS access key, secret key, bucket name, region, and the key of the file within the S3 bucket.

Config

REQUIRED

Config Parameters

Name Description
accessKey The AWS access key ID used for authentication. This is a required parameter.
secretKey The AWS secret access key used for authentication. This is a required parameter.
bucket The name of the S3 bucket where the data is stored. This is a required parameter.
region The AWS region where the S3 bucket is located. This is a required parameter.
key The key of the file within the S3 bucket to retrieve. This is a required parameter.

Config Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
<?xml version="1.0" encoding="UTF-8"?>
<apiroConf version="1" xmlns="http://apiro.com/apiro/v1/root">
    <loadOrder>10</loadOrder>
    <dataSources>
        <dataSource definition="S3" execPriority="10" name="TEST_S3_SOURCE">
            <config>
                <![CDATA[
                    {
                          "accessKey" : "xxxxx",
                          "secretKey" : "yyyyy",
                          "bucket" : "mybucket",
                          "region" : "ap-southeast-2",
                          "key" : "myfile"
                    }
                ]]>
            </config>
        </dataSource>
    </dataSources>
</apiroConf>

Common Mistakes

  • Incorrect AWS Credentials: Ensure that the accessKey and secretKey parameters are correct and have the necessary permissions to access the specified S3 bucket.
  • Bucket or Region Issues: Verify that the bucket and region parameters are correctly specified and correspond to an existing S3 bucket in the specified AWS region.
  • File Key Errors: If you encounter issues retrieving the file, ensure that the key parameter correctly points to the file within the S3 bucket.