AWS S3 Storage
Amazon S3 or Amazon Simple Storage Service is a service offered by Amazon Web Services (AWS) that provides object storage through a web service interface. Amazon S3 uses the same scalable storage infrastructure that Amazon.com uses to run its e-commerce network. Amazon S3 can store any type of object, which allows uses like storage for Internet applications, backups, disaster recovery, data archives, data lakes for analytics, and hybrid cloud storage.
Our AWS S3 Storage DataLakeHouse integration:
- replicates AWS S3 Storage data to your Cloud Data Warehouse target
- synchronizes to your target destination at a scheduled frequency
It allows you to replicate/synchronize your S3 data files, including capturing snapshots of data at any point int time, and keep it up-to-date with little to no configuration efforts. You don’t even need to prepare the target schema — DataLakeHouse will automatically handle all the heavy lifting for you.
All you need is to specify the connection to your S3, point to your target system, or use a DataLakeHouse.io managed Data Warehouse and DataLakeHouse.io does the rest. Our support team can even help you set it up for you during a short technical on-boarding session.
DataLakeHouse.io securely connects to your AWS S3 Storage. Using the form in the DataLakeHouse.io portal please complete the following basic steps.
- Enter a Name or Alias for this connection, in the 'Name/Alias' field, that is unique from other connectors
- Enter a 'Target Schema Prefix', which will be the prefix for the schema at the target you will sync your data files into
- Enter a 'Bucket' name, where your files are stored
- Typically starts with s3:// or https://, so enter just the name without the prefix.
- Select your 'Region'
- Enter your 'Access Key', credentials to access the bucket
- Enter your 'Secret Key', credentials to access the bucket
- Enter any other optional details in the available fields (See the setup video if you need help or contact support)
- Folder Path, is a path on the root bucket from where desired files will be retrieved
- File Pattern, is a regular expression (RegEx) used to isolated only certain files to be retrieved
- File Type, allows for a pre-determined type of file extension to be retreived
- Click the Save & Test button. Once your credentials are accepted you should be able to see a successful connection.
If any issues occur with the authorization simply return to the sources page in DataLakeHouse.io, edit the source details and click the 'Save & Test' button to confirm connectivity. If any issues persist please contact our support team via the DataLakeHouse Support Portal.
