Learn the Basic Concepts

FAQ : DataLakeHouse.io

9min

Here are answers to some common questions users typically have about DataLakeHouse:

Do I need to know how to write code to use the DataLakeHouse.io?

No! Follow our simple instructions to guide you and with just a few clicks, you can easily load Data from any Source into Snowflake at the frequency of your choosing, and watch your Data flow 🙂

What Data Sources can I Synchronize to a Target Destination?

DataLakeHouse.io currently offers Connectorsďťż (aka connections or integrations) for many popular solutce systems, for example:

  • Aloha POS
  • Ceridian
  • DoorDash For Work
  • Facebook / Meta Ads
  • Harvest
  • JIRA
  • McLeod Transportation
  • MongoDB
  • MongoDB Sharded
  • MySQL
  • NetSuite (Oracle NetSuite)
  • Optimum HRIS
  • PostgreSQL
  • QuickBooks
  • Salesforce
  • Shopify
  • Snowflake
  • Snowflake Data Marketplace
  • SQL Server
  • Square
  • Square Marketplace
  • Stripe
  • Xero
  • always check our main connectors page for the latest list of connectors

+ New Sources Added Frequently. You can request a connector anytime if the one you are looking for is not listed.

Is Snowflake or a Cloud Data Warehouse the only Target that I can land my Data into?

No. DLH.io focuses on a number of cloud data warehouse and storage technology vendors for you to synchronize and replicate your data. Our Business Critical plan offers both tailored Sources and Targets to meet your organization's needs, if needed. We boast turnaround times of less than four weeks for most connector integrations.

How frequently can my source data be Synchronized / Replicated to a target destination like Snowflake?

This depends on the Data Source it's coming from, but we currently offer Sync Bridges that bring data in as frequently as every 15 minutes (faster if on our Business Critical plan). Additional intervals offered include every 30 minutes, 1 hour, 2, 3, 6, 8, 12, and 24 hours. Depending on your organization's specific needs, you might want to consider our Business Critical plan, which offers additional synchronization frequencies.

Can I receive Notifications?

Yes! You can setup Alerts & Notifications for email and Slack that let you know thinks like:

  • if a connector fails to connect to a source
  • if a sync bridge does not complete within a certain time threshold
  • if an issue is encountered

For more information, review Alerts & Notificationsďťż

Frequently Asked Questions about DataLakeHouse.io will always be available here. Also, join our Slack community for more insights and to collaborate with other DataLakeHouse users and developers.

What types of changes or metadata changes from source to target are handled and tracked by DLH.io?

DLH.io handles the following types of changes from source systems:

  • Change Data Capture (CDC)
  • Change Tracking (CT)
  • Metadata Additions
    • ex: columns/attributes added to source system objects/tables
  • New Tables/Entities

What types of changes or metadata changes are NOT handled by DLH.io?

DLH.io does not handle deletions for structural source system metadata. An example is that of if a customer has a source, for example a PostgreSQL database and sychronizes a set of database tables to a target successfully, and then at the source an action is conducted by the customer to delete/drop a colum, DLH.io will not in the next subsequent data load of that table remove the column from the target system. This would be a direct violation of data integrity as DLH.io would be uninformed if the change made was by accident by the customer, temporary, etc.

It might not be obvious to some customers or developers but the downstream impact potential could be severe in a production setting where if DLH.io removed a target column that previously contained data that was used for production reporting or development for exmple in the data warehouse that column and its data would be removed, thus impacting any systems or applications depending on said column or structure. Therefor, customers should take note that if they choose to delete a database source system structure by dropping a column they must themselves handle the target system impact. If a column is dropped in the source system the target column will remain but it will no longer be updated with data by DLH.io as DLH.io understands that that column no longer exists in the source.

What do the __DLH_ columns mean?

DataLakeHouse.io has a number of system generated columns to each table in the Target Connection. Below is the definition of each of these columns:

  • __DLH_IS_DELETED
    •  Indicates where this record has been deleted in the source system.
  • __DLH_SYNC_TS
    • Indicates when this record was synced into the target database. Depending on the volume of records and any queuing in the source system, this value may be the same as the __DLH_START_TS and __DLH_FINISH_TS columns.
  • __DLH_START_TS
    • Indicates when this record started to be synced into the target database. Depending on the volume of records and any queuing in the source system, this value may be the same as the __DLH_SYNC_TS and __DLH_FINISH_TS columns.
  • __DLH_FINISH_TS
    • Indicates when this record finished being synced into the target database. Depending on the volume of records and any queuing in the source system, this value may be the same as the __DLH_SYNC_TS and __DLH_START_TS columns.
  • __DLH_IS_ACTIVE
    • Indicates where this record is the active record. When TRUE, this is the most current record available from the source system, i.e. Ceridian Dayforce. When FALSE, this indicates that a more recent record is available and the source system has Change Data Capture (CDC) enabled. 

ďťż

What type of Support is offered by DLH.io for Troubleshooting issues?

You can communicate with our support team using our support portal, or by sending us an email. More on support is found here in our Customer Supportďťż page.