S3

Amazon Simple Storage Service (Amazon S3) is a highly scalable object storage solution provided by Amazon Web Services (AWS). By integrating S3 with Upriver, you can automate data governance and maintain data quality directly from your cloud storage. This integration ensures reliable data monitoring, traceability, and consistency across your data pipeline.

Configure AWS

Before connecting S3 to Upriver, make sure your AWS account is properly set up. Follow the guidelines provided in this page to ensure correct setup.

Configure the Data Source in Upriver platform

Once your AWS account is set up, configure S3 in the Upriver platform by providing the following details:

  1. Configure a new Data Source - Data Source Configuration.

  2. Fill in the connection details:

    • Bucket: Enter the name of the S3 bucket where the data is stored.

    • Prefix in Bucket: Specify the prefix path within the bucket to point to the relevant data folder. Notice that you can put wildcards (*) as part of the prefix

    • Region: Select the region where the S3 bucket is located.

    • Data Format: Choose the data format (e.g., JSON, Parquet, CSV).

Monitor and Manage Your Data

After configuring an S3 datasource, Upriver will automatically monitor it for you. You can track data issues and enforce governance policies, ensuring your data is consistently accurate and trustworthy throughout the pipeline.

PreviousMWAANextRedshift

Last updated