Skip to content

SFTP - DATA SOURCE

Description

The SFTP data source is designed to retrieve data from a remote server using the Secure File Transfer Protocol (SFTP). SFTP is a network protocol that provides file access, file transfer, and file management functionalities over any reliable data stream. It is widely used for securely transferring files between a client and a server, ensuring data integrity and confidentiality. The SFTP data source supports various configuration options to customize the data retrieval process, including specifying the SFTP server host, username, private key, base directory on the server, and the file prefix to match files for retrieval.

Config

REQUIRED

Config Parameters

Name Description
host The hostname or IP address of the SFTP server. This is a required parameter.
username The username for authentication with the SFTP server. This is a required parameter.
key The private key or password for authentication with the SFTP server. This is a required parameter.
baseDirectory The base directory on the SFTP server where the files are located. This is a required parameter.
filePrefix The prefix of the files to be retrieved from the SFTP server. This is a required parameter.
discriminator The sheetname of the file the retrieved .xlsx file. This parameter should only be used when sourcing an excel file.

Config Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
<?xml version="1.0" encoding="UTF-8"?>
<apiroConf version="1" xmlns="http://apiro.com/apiro/v1/root">
    <loadOrder>10</loadOrder>
    <dataSources>
        <dataSource definition="SFTP" execPriority="10" name="TEST_SFTP_SOURCE">
            <config>
                <![CDATA[
                    {
                        "host": "${SYS:APIRO_SFTP_HOST}",
                        "username": "${SYS:APIRO_SFTP_USERNAME}",
                        "key": "${SYS:APIRO_SFTP_KEY}",
                        "baseDirectory": "/home/apirodrop/feeds_DEMO7/sandbox",
                        "filePrefix": "m1_poc_data"
                    }
                ]]>
            </config>
        </dataSource>
    </dataSources>
</apiroConf>

Common Mistakes

  • Incorrect Host or Credentials: Ensure that the host, username, and key parameters are correct and have the necessary permissions to access the specified directory on the SFTP server.
  • Base Directory Issues: Verify that the baseDirectory parameter correctly points to the directory on the SFTP server where the files are located.
  • File Prefix Errors: If you encounter issues retrieving files, ensure that the filePrefix parameter correctly matches the prefix of the files you intend to retrieve.