​
Importing data

1

Prepare your data

Prepare the raw data CSV that you need to bring into your project in Mangrove.

  • Ensure you’re using the right import template
  • Populate all required columns in the template
    To streamline imports, ensure that data in the import templates reflect the correct formatting (e.g., date format or datetime format) and units (e.g., values in (%) should be between 0 to 100). A best practice is to “paste as value only” into the import templates to avoid copying external links and formulae into the template.
2

Upload the file

Upload the CSV file into the corresponding Data Source in the Mangrove platform

  • Orders recorded for this customer will have the currency set as the default
  • Save contacts, and important information about the customer’s registry accounts
3

Review the loaded events

Events are transformed from your bulk import and populate the Analytics charts and Events Feed.

  • View event data: by selecting each event on the feed
  • Attach evidence files to relevant events: with Upload Evidence on each event
  • Filtering events: You can filter for specific event types and time ranges in the feed and on the Analytics charts

To check the status of ongoing bulk imports, visit the Data Inputs > Bulk Jobs section. If you encounter an error that you need assistance with, please reach out to Mangrove through your shared channel.

  • Reviewing issues with imports: Issues with bulk imports would be reflected as errored bulk jobs. Review the error message for more detail on the specific issue transforming the data in your bulk import file into Mangrove.
  • Cancel or Reverse an import: Mistakenly import wrong data? A completed bulk job can be Reversed. A bulk job that has already started can be Canceled - any existing data transformed from your import file will be deleted.
  • Retry an import

​
Data transformations

Transformations can be written by Admin users to be applied to every new file bulk imported from that data source.

Transformations can be applied to bulk import files to generate events and evidences from them.

​
Editing transformations on each Data Source

Admin users can define the event types that are generated from every Data Source through the Transformations Editor.

  • In Data Inputs > Input Settings, select an existing Data Source
  • Add transformation
  • Select an event type for data from the Data Source
  • Edit the Python transformation in the editor

Transformations are written in Python, and need to return a results array of objects representing the events to generate from the import data. An example is shown below:

Transformation
from datetime import datetime

results = []
for row in rows:
	event_object = {
		"event_type": "<event slug>",
		"notes": row["<notes column>"],
		"start_time": parse_datetime(row["<start time column>"] row["<timezone column>"]).isoformat(),
		"end_time": parse_datetime(row["<end time column>"] row["<timezone column>"]).isoformat(),
		"locations": [
			{
				"name": row["<location name column>"],
				"lat": 43.6499286,
				"long": -79.3858228
			}
		],
		"evidences": [
			{
				"name": "<name for evidence file>",
				"type": "json",
				"content": {
					"some_key": "some_content"
				}
			}
		],
		"data_points": [
			{
				"slug": "<datapoint 1 slug>",
				"value": row["<datapoint1 column>"]
			},
			{
				"slug": "<datapoint 2 slug>",
				"value": row["<datapoint2 column>"]
			}
		]
	}
	results.append(event_object)

Here’s an example of what the results array might look like following execution of the transformation above:

results
[
	{
		'event_type': '<event slug>', 
		'notes': 'This is an example event note',
		'start_time': '2024-02-01 12:00:00-04:00',
		'end_time': '2024-02-01 15:30:00-04:00',
		'locations': [
			{
				"name": "<location name>",
				"lat": 43.6499286,
				"long": -79.3858228
			}
		],
		'evidences': [
			{
				'name': '<name for evidence file>',
				'type': 'json',
				'content': {
					'some_key': 'some_content'
				}
			}
		],
		'data_points': [
			{
				'slug': '<datapoint 1 slug>',
				'value': '<datapoint 1 value>'
			},
			{
				'slug': '<datapoint 2 slug>',
				'value': '<datapoint2 value>'
			}
		]
	}
	{
	  # another event object
	  ...
	}
]

Mangrove will process this result to create events in the Data Inputs > Events feed that can be used to run production models.

​
Transformed Event Attributes

Below is the full list of fields that can be defined for each event object transformed from the import data:

​
Using external packages in transformations

Apart from the Python standard library, Mangrove also supports a curated list of external packages that you can import and use in transformations:

  • mapbox
  • boto3
  • geopy
  • pandas
  • numpy