Google Cloud Storage (GCS)
Google Cloud Storage (GCS) is a scalable object storage service provided by Google Cloud that allows users to store and access large amounts of data. By integrating GCS with Upriver, you can ensure automated data governance and maintain data quality directly from your cloud storage, enabling reliable data monitoring, traceability, and consistency across your pipeline.
Configure GCP
Before connecting GCS to Upriver, ensure that your Google Cloud Platform (GCP) account is properly configured. Follow the guidelines provided in this page to ensure correct setup.
Configure the Data Source in Upriver platform
Once your GCP account is set up, configure GCS in the Upriver app by providing the following details:
Configure a new Data Source - Data Source Configuration.
Fill in the connection details:
Bucket: Enter the name of the GCS bucket where the data is stored.
Prefix in Bucket: Specify the prefix path within the bucket to point to the relevant data folder. Notice that you can put wildcards (
*) as part of the prefixRegion: Select the region where the GCS bucket is located.
Data Format: Choose the data format (e.g., JSON, Parquet, CSV).
Monitor and Manage Your Data
After configuring GCS as a datasource, Upriver will automatically monitor the data quality and completeness. You can track data issues and enforce governance policies, ensuring your data is consistently accurate and trustworthy throughout the pipeline.
Last updated