website logo
⌘K
Getting Started 🚀
What is DataLakeHouse.io?
Our Business-Value Focus
Learn the Basic Concepts
Connectors
Operations Applications
Asana
Aloha POS
Baremetrics
Beans Route
BILL
Bloom Growth
Bullhorn
Calendly
Ceridian Dayforce
ClinicalTrials.gov
ConnectWise
DBT Cloud
DBT Cloud Log Stream
Facebook Ads
Food Delivery Service Connector
Google Analytics 4
Google Play
Harvest
Hubspot
Jira
MailChimp
McLeod Transportation
Microsoft Teams
NetSuite (Oracle NetSuite)
NetSuite SuiteAnalytics
Optimum HRIS
QuickBooks Online
Salesforce
Salesloft
Shift4 Payments
Shopify
Square
Square Marketplace
Stripe
Toast
TriNet
Verizon Wireless Business
Workday HCM
Xero
Zendesk Sell
Zoom
Databases
Files & Object Storage
SQL Data Query
SSH Tunnel Setup for Hosted Database Systems
SQL Playground Editor
SQL Transformations
DBT Cloud Transformations
Terraform: Reverse Terraforming
Sync Bridge (Data Pipelines)
Create a Sync Bridge
Manually Run a Sync Bridge
Deleting a Sync Bridge
Historical Re-sync
Analytics
Access Analytics
Snowflake Usage Analytics
Data Catalog
Create the Catalog
Populate the Catalog
Access the Catalog
Data Warehouse Clouds
❄️Snowflake
Open Source DW Models
Alerts & Notifications
Slack Notifications
Logs & Monitoring
Security
Callback Links
Service Level Agreement (SLA)
Release Notes
July 2023
June 2023
May 2023
April 2023
Q3 2022
Q4 2022
Community Overview
Contributor Agreements
Code Contribution Guide
About
Customer Support
License
Viewpoint
Credit Consumption Breakdown
Docs powered by
Archbee
website logo
Connectors

SQL Data Query

4min

Many of our business customers solve their one off data sharing needs by using a simple sql data query connection.

A SQL Data Query connection allows a SQL query to query data from their source system, such as a Cloud Data Warehouse database and send the results of that data set somewhere else such as via SFTP as part of their integration workflow.

It is as easy as it sounds. Create SQL query against a known data source such as your database table(s), joins are allowed, etc. Then take the proven query and enter it in the SQL Query field of the SQL Data Query.

This now becomes a SQL data query connector that is based one of our existing cloud data warehouse connectors (other application systems such as NetSuite SuiteQL are possible), and it can then be added as a source in your Sync Bridge to load to your target destination of choice, i.e.: Cloud Data Warehouse, SFTP.

Prerequisites

You need to make sure that:

  • You have a source connector already created for your Cloud Data Warehouse
    • SQL data Query connector needs a source from which to query. This is done by selecting the source in question in the new SQL data Query source connection form. Without an existing Cloud Data Warehouse connection (e.g.: Snowflake Source Conector) you will not be able to use this connector.
  • You have an existing working query that you've validated returns results/records.
  • You understand that SQL Data Query results can be limited based on your current subscription plan. The lower plan(s) are limited and controlled to allow a maximum of 100,000 results/records based on any one query.
  • Only one SQL query per SQL Data Query connector are allowed otherwise your connector is likely to result in failure. CTE's are permissable.

Setup Instructions

Using the source connector form please complete the following basic steps:

  1. Enter a Name or Alias for this connection, in the Name/Alias field, that is unique from other connectors.
  2. Enter in the Target Schema Prefix field, a name that will be unique in your data cloud destination where you wish to land the data.
  3. Select from the Source Cloud Database dropdown, a source connection you've previously created from which this query will be executed.
    • If you do not yet have a source connector to a cloud data warehouse you must create one before this connector can be create.
    • Document image
      
  4. Enter the Output File Name for what the target result of this query will be referrenced as.
    • If ultimately using an SFTP target/destination in a Sync Bridge for this source connector, this field will be the file name generated for the output file containing the result set (also see Jinja style templating below) from your SQL query.
  5. Enter in the SQL Query field the SQL code which you have previously validated
  6. Click Save & Test to save the connector



Updated 22 Jul 2023
Did this page help you?
PREVIOUS
What is the file size limit for flat file uploads?
NEXT
SSH Tunnel Setup for Hosted Database Systems
Docs powered by
Archbee
TABLE OF CONTENTS
Prerequisites
Setup Instructions
Docs powered by
Archbee