> ## Documentation Index > Fetch the complete documentation index at: https://docs.mangrovesystems.com/llms.txt > Use this file to discover all available pages before exploring further. # Data connectors > Fetch data from your databases and APIs into your Mangrove project, daily or on demand. Data connectors bring records from your database or API into a data feed, where transformations turn them into events. Add a connector as a **source**, then click **Sync** to fetch on demand or turn on daily fetching. Testing a connection or query and previewing a response read from your system without creating events or moving the saved cursor. They do send real requests, so point them at endpoints where reading has no side effects. ## Choose a connector | Your system | Setup guide | | - | - | | Microsoft SQL Server | [Connect a SQL database](/data-collection/activate-data-connector) | | Snowflake (with credentials or Secure Share) | [Connect a SQL database](/data-collection/activate-data-connector) | | A REST API that returns CSV or JSON | [Connect an HTTP API](/data-collection/activate-http-api-connector) | Don't see your system? [Contact the Mangrove team](mailto:support@mangrovesystems.com) to discuss additional connector options. ## Choosing what each run fetches Under **What each run fetches**, choose one of two options: * **Everything, every run** imports the full result each time, adding every record again; earlier imports are not updated or deduplicated. * **Only what's new** picks up where the previous run stopped. Use it whenever your database or API can select records after a value or within a time range. With **Only what's new**, the variables in your SQL query or HTTP request decide how Mangrove keeps its place. | You use | Each run fetches | Learn more | | - | - | - | | `{{cursor_value}}` | Records after the last value it received, such as an increasing ID | [Continue from the last value](#continue-from-the-last-value) | | `{{window_start}}` and `{{window_finish}}` | The next time window | [Fetch the next time window](#fetch-the-next-time-window) | Use one approach or the other, not both. **Everything, every run** uses neither. If fetching fails, the saved cursor stays where it was: fix the cause and click **Sync** again. If a later step fails, such as a transformation, Mangrove has already stored the fetched data and moved the cursor. Open **Data Inputs** > **Data Feed Runs** and click **Retry** on that run instead of **Sync**, so its records are processed rather than skipped. ### Continue from the last value A **cursor** marks where the previous fetch stopped. Set **Cursor column** to the returned column holding this value, and **Initial cursor** to the value before the first record you want. HTTP API sources also need a **Cursor type**. Request records after the cursor and return them in ascending cursor order. After a successful fetch, Mangrove saves the cursor column's value from the last row. An empty result leaves the cursor unchanged. Use a unique, stable value, such as an increasing record ID. If a result ends partway through records sharing a timestamp, the next request can skip the rest of that group. Records added later at or below the saved cursor are also missed. ### Fetch the next time window Set **Cursor type** to a timestamp or Unix epoch type, and **Initial cursor** to the inclusive start of the first window. For timestamps, include `Z` or an explicit offset, for example `2026-01-01T00:00:00Z`. Each run starts at the saved cursor and ends at the earlier of: * The current time minus **Settle lag (seconds)**. The default is 300 seconds: a run at noon fetches records up to 11:55. * The window start plus **Maximum window (seconds)**, if set. Leave it empty for no limit. Your query or API must include the start, exclude the end and return every record in the window. Mangrove supplies UTC bounds but cannot check how your source filters them. After a successful fetch, the cursor moves to the window's end, even if the window had no records. **Maximum window** caps how much each run fetches, not how far a scheduled source gets: when a run of a scheduled source is cut short by the maximum and completes, Mangrove fetches the next window straight away and repeats until it catches up. Each window is transformed and published before the next starts, so catching up is no faster than one uncapped run. A failed run pauses the catch-up; retrying it successfully, the next scheduled fetch, or a **Sync** starts it again. A manual source fetches one window per **Sync**. Allow enough settle lag for records to become available. Records arriving after their window was fetched are not picked up automatically. ### Cursor types | Cursor type | Example | | - | - | | Timestamp (ISO 8601 with timezone) | `2026-05-31T23:59:00Z` | | Unix epoch seconds | `1780271940` | | Unix epoch milliseconds | `1780271940000` | | Integer identifier | `48213` | | Opaque text | `reading_0048213` | Use the representation your source expects. Time windows offer only the timestamp and epoch types; epoch values must be whole numbers in the selected unit. Opaque text is a value from a returned row, not a pagination token from a response envelope. ## Scheduling Turn on **Fetch automatically** in the source's **Schedule** section to fetch daily. Automatic fetches run at 11:59 pm Eastern Time; the section shows that time, and the next run, in your account's time zone. Click **Sync** in the **Sources** table to fetch on demand at any time. A data feed can have at most one scheduled source. If another source holds the schedule, the section names it. Turning on **Fetch automatically** here and saving moves the schedule to this source, and the other source becomes manual. New HTTP API sources and sources set to **Everything, every run** start manual. A new SQL source set to **Only what's new** takes the schedule if no other source on the feed has it. Check **Schedule** before saving. ## Changing fetch modes Editing **Initial cursor** does not reset an existing source's saved progress. When you switch an existing source to time windows, or change its window cursor type, the form asks for **Start the next window at (inclusive)** and shows the current cursor. Reusing a last-row cursor as an inclusive start can import those records again; a later start can skip records. Enter the boundary in the selected cursor type. Other changes keep or reset progress without asking: * Switching from time windows back to `{{cursor_value}}` restarts a SQL source from **Initial cursor**, which can import history again. An HTTP source continues from the last window's end, as a timestamp or epoch value. * Changing **Cursor column** keeps the saved cursor, which may not suit the new column. Before syncing after a change like this, check **Initial cursor** and the **Cursor** column in the **Sources** table. ## Checking progress The **Sources** table shows each source's **Last run** and its **Cursor**, the point the next fetch starts from. Open **Data Inputs** > **Data Feed Runs** for each run's details and events. If an import went wrong, click **Revert** on the run in **Data Feed Runs** to remove its events. Reverting doesn't move the cursor back, so with **Only what's new** the next sync won't fetch those records again. With **Everything, every run**, the next sync imports the full result again, so fix the query or request first. Mangrove keeps the fetched data, though: once you've fixed the cause, click **Restart** on the reverted run to process it again. If a run's events are already used in a model run, **Revert** is unavailable and the run says why. ## Cloning a project Cloned sources copy the configuration without credentials, saved cursors or schedules. Add any required credentials, review the initial cursor, and run **Sync** before turning on the schedule. The first fetch starts from the initial cursor, so it may import historical data again. ## Troubleshooting * With **Everything, every run**, each run imports the full result again. Switch to **Only what's new** if your source supports it. * When continuing from the last value, check that rows are returned in ascending order with a unique, stable cursor value. * Check whether two sources fetch the same records, or whether a new window start overlaps data already imported. * To remove a duplicate import, click **Revert** on its run in **Data Feed Runs**. * For SQL sources with a pre-query, a changed pre-query result restarts fetching from the initial cursor. * When continuing from the last value, check the row order and whether several records share a cursor value. Records that arrive behind the saved cursor are not fetched. * With time windows, include the start and exclude the end. Check for row limits or pagination that stop the full window from being returned. * Check for records that arrived after their window was fetched. Increase the settle lag to allow for delays in future windows. * With time windows, the next window opens only once the settle lag has passed its start. The **Sources** table shows when it opens. Nothing is wrong; sync again later. * Check that the source is scheduled and look at recent runs in **Data Feed Runs**. If fetching failed, the cursor stayed in place; if a later step failed, click **Retry** on that run. * A manual source with a **Maximum window** advances one window per **Sync**. Turn on **Fetch automatically** to let it catch up on its own. * For HTTP APIs, check whether each response returns only one page. See [One response per run](/data-collection/activate-http-api-connector#one-response-per-run). This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.