Skip to main content
The SQL database connector runs your query against Microsoft SQL Server or Snowflake and imports the results as CSV. Fetch on demand or once a day.

Before you begin

Prepare your query and choose the event types to create in Mangrove. You’ll also need connection details for your database: For direct connections, check that your database’s network rules allow access from Mangrove. Contact support for the current IP allowlist.

Add a SQL source

1

Choose your database

In your project, go to Data Inputs > Data Feeds. Click Add Data Feed and select SQL Database, or choose Add source on an existing data feed. For a new feed, name it on Configure Data Feed, then continue to the source configuration.Choose your database under SQL Database. For Microsoft SQL Server, select MSFT SQL Server.
2

Connect

Enter the connection details for your database type, then click Test connection to verify access.
3

Choose what each run fetches

Under What each run fetches, choose:
  • Everything, every run to import the full query result each time. Existing records are not replaced or deduplicated.
  • Only what’s new to pick up where the previous run stopped. Your query’s variables decide how: {{cursor_value}} continues after the last value received, and {{window_start}} with {{window_finish}} fetches the next time window. Fill in the cursor fields that appear once the query uses them.
See Choosing what each run fetches for help choosing a cursor and window size.
4

Write and test your query

Enter the Data query with the variables for your choice, then click Test query to see up to 100 rows. Check the columns and values; a sync runs the query without the 100-row limit.Click Add a pre-query to look up values the query needs, such as a table name. Pre-queries cannot be combined with time windows. With time windows, Settle lag (seconds) and Maximum window (seconds) appear under Settings.
5

Review the schedule

Before leaving the source configuration, check Schedule. New sources set to Everything, every run start manual. A new source set to Only what’s new is scheduled by default if no other source on the feed is. Turn off Fetch automatically if you’d like to check your first import with a manual sync.
6

Finish setup

For a new feed, upload an example CSV or spreadsheet matching your query’s columns on the Schema step. On Configure Result, select the event types to create, then save the feed. For an existing feed, save the source and check that its columns match the schema and transformations.
7

Run your first sync

Click Sync beside the source in the Sources table, then check its Last run, the run in Data Inputs > Data Feed Runs, and the resulting events. If the import isn’t what you expected, click Revert on the run to remove its events; see Checking progress.To fetch daily, edit the source and turn on Fetch automatically. See Scheduling.

Writing SQL queries

Use double curly braces to insert variables into your data query: Variables are inserted as plain text. Quote string and timestamp values using your database’s SQL syntax. Use either {{cursor_value}} or both window variables, never both, and neither with Everything, every run.

Continue from the last value

For an increasing integer ID, set Cursor column to reading_id and use:
Set Initial cursor to the value before the first record you want to fetch, such as 0 for IDs starting at 1. The cursor column must be returned by the query, in ascending order; see Continue from the last value for choosing a safe cursor.

Fetch the next time window

Include the start with >= and exclude the end with <:
Return every row in the window. Do not add a row limit that truncates the result: the cursor advances to the window’s end after collection. Test query uses the window the next sync would fetch: from Initial cursor for a new source, or from the saved cursor (or the Start the next window at value you entered) for an existing one. Window bounds are UTC. If your column stores local times without a timezone, convert them in the query. For an epoch column, choose the matching seconds or milliseconds cursor type and compare numeric bounds without quotes.

Using a pre-query

A pre-query’s first row supplies variables to the data query, useful for table names that change over time. For a pre-query returning table_name and company_id:
This example uses an integer cursor and a text company ID. Adjust quoting to match your column types. When continuing from the last value, a changed pre-query result restarts from Initial cursor, for example when selecting a new monthly table. A pre-query cannot be combined with time windows.

Troubleshooting

  • Check the connection details and credentials for the selected database type.
  • Check that network rules allow Mangrove to reach the database.
  • For Secure Share, check that the shared database and schema are accessible to Mangrove.
  • Confirm that the table contains rows matching your query’s filters and initial cursor or window.
  • Check that the database user has permission to read the queried tables.
  • If you use a pre-query, test it and check the values supplied to the data query.
  • Check the highlighted fields and match the query variables to What each run fetches.
  • For time windows, use both window variables, remove any pre-query, and enter an initial cursor with the selected timestamp or epoch format.
  • When switching an existing source to windows, choose an explicit starting boundary. See Changing fetch modes.
For duplicate or missing records, or a feed that is falling behind, see shared troubleshooting.

Managing the connector

Go to Data Inputs > Data Feeds, open the data feed, and click Edit beside the SQL source to change connection details, credentials or queries. If you point the connection at a different server or Snowflake account, enter the credentials again. Only an account admin can move saved credentials to a new destination without re-entering them. Before changing what each run fetches or the cursor column, read Changing fetch modes.