Data Mapper

The Data Mapper enables you to configure data mappings.

Categories: helpers

Type: dataMapper/v1

Actions

Map Objects to Array

Name: mapObjectsToArray

Transform an object or array of objects into an array of key-value pairs.

Properties

Name	Label	Type	Control Type	Description	Required
inputType	Input Type	STRING Options OBJECT, ARRAY	SELECT	Type of the input. Cam be an object or an array of objects.	true
input	Input	OBJECT Properties {}	OBJECT_BUILDER	An input object containing one or more properties.	true
input	Input	ARRAY Items [{}]	ARRAY_BUILDER	An input array containing one or more objects.	true
fieldKey	Field Key	STRING	TEXT	Property key of each newly created object in the array. Its property value will be a property key from the input.	true
valueKey	Value Key	STRING	TEXT	Property key of each newly created object in the array. Its property value will be a property value from the input.	true

JSON Example

{
  "label" : "Map Objects to Array",
  "name" : "mapObjectsToArray",
  "parameters" : {
    "inputType" : "",
    "input" : [ { } ],
    "fieldKey" : "",
    "valueKey" : ""
  },
  "type" : "dataMapper/v1/mapObjectsToArray"
}

Map Objects to Object

Name: mapObjectsToObject

Creates a new object with the chosen input properties. You can also rename the property keys.

Properties

Name	Label	Type	Control Type	Description	Required
inputType	Input Type	STRING Options OBJECT, ARRAY	SELECT	The input type.	true
input	Input	OBJECT Properties {}	OBJECT_BUILDER	An object containing one or more properties.	true
input	Input	ARRAY Items [{}]	ARRAY_BUILDER	An array containing one or more objects.	true
mappings	Mapping	ARRAY Items [{STRING(from), STRING(to), BOOLEAN(requiredField)}]	ARRAY_BUILDER	An array of objects that contains properties ‘from’, ‘to’ and ‘requiredField’. For nested keys, it supports dot notation, where the new mapped path can be used for nested mapping.	true
includeUnmapped	Include Unmapped	BOOLEAN Options true, false	SELECT	Should fields from the original object that do not have mappings be included in the new object?	null
includeNulls	Include Nulls	BOOLEAN Options true, false	SELECT	Should fields that have null values be included in the new object?	null
includeEmptyStrings	Include Empty strings	BOOLEAN Options true, false	SELECT	Should fields with empty string values be included in the new object?	null

JSON Example

{
  "label" : "Map Objects to Object",
  "name" : "mapObjectsToObject",
  "parameters" : {
    "inputType" : "",
    "input" : [ { } ],
    "mappings" : [ {
      "from" : "",
      "to" : "",
      "requiredField" : false
    } ],
    "includeUnmapped" : false,
    "includeNulls" : false,
    "includeEmptyStrings" : false
  },
  "type" : "dataMapper/v1/mapObjectsToObject"
}

Merge and Pivot Properties by Key

Name: mergeAndPivotPropertiesByKey

Creates a new object out of all objects that have the same key as the specified field kay and an object as value. That value of the new object contains values of all properties that share the specified field key as keys and the they all have the specified field value as a value.

Properties

Name	Label	Type	Control Type	Description	Required
input	Input	ARRAY Items [{}]	ARRAY_BUILDER	An array that contains objects with key-value properties that need do be merged.	true
fieldKey	Field Key	STRING	TEXT	The key of the newly created object.	true
fieldValue	Field Value	STRING	TEXT	The value of each property in the newly created objects value.	true

JSON Example

{
  "label" : "Merge and Pivot Properties by Key",
  "name" : "mergeAndPivotPropertiesByKey",
  "parameters" : {
    "input" : [ { } ],
    "fieldKey" : "",
    "fieldValue" : ""
  },
  "type" : "dataMapper/v1/mergeAndPivotPropertiesByKey"
}

Rename Keys

Name: renameKeys

The action renames keys of an input object defined by mappings.

Properties

Name	Label	Type	Control Type	Description	Required
input	Input	OBJECT Properties {}	OBJECT_BUILDER	The input object that contains property keys and values.	true
mappings	Mappings	ARRAY Items [{STRING(from), STRING(to)}]	ARRAY_BUILDER	An array of objects that contains properties ‘From Path’ and ‘To’.	true

JSON Example

{
  "label" : "Rename Keys",
  "name" : "renameKeys",
  "parameters" : {
    "input" : { },
    "mappings" : [ {
      "from" : "",
      "to" : ""
    } ]
  },
  "type" : "dataMapper/v1/renameKeys"
}

Replace All Specified Values

Name: replaceAllSpecifiedValues

Goes through all object parameters and replaces all specified input parameter values.

Properties

Name	Label	Type	Control Type	Description	Required
inputType	Input Type	STRING Options OBJECT, ARRAY	SELECT	The input type.	true
input	Input	OBJECT Properties {}	OBJECT_BUILDER	An object containing one or more properties.	true
input	Input	ARRAY Items [{}]	ARRAY_BUILDER	An array containing one or more objects.	true
type	Value Type	STRING Options ARRAY, BOOLEAN, DATE, DATE_TIME, INTEGER, NUMBER, OBJECT, STRING, TIME	SELECT	The value type of ‘from’ and ‘to’ property values.	true
mappings	Mappings	ARRAY Items [{[](from), [](to)}]	ARRAY_BUILDER	An array of objects that contains properties ‘from’ and ‘to’.	null
mappings	Mappings	ARRAY Items [{BOOLEAN(from), BOOLEAN(to)}]	ARRAY_BUILDER	An array of objects that contains properties ‘from’ and ‘to’.	null
mappings	Mappings	ARRAY Items [{DATE(from), DATE(to)}]	ARRAY_BUILDER	An array of objects that contains properties ‘from’ and ‘to’.	null
mappings	Mappings	ARRAY Items [{DATE_TIME(from), DATE_TIME(to)}]	ARRAY_BUILDER	An array of objects that contains properties ‘from’ and ‘to’.	null
mappings	Mappings	ARRAY Items [{INTEGER(from), INTEGER(to)}]	ARRAY_BUILDER	An array of objects that contains properties ‘from’ and ‘to’.	null
mappings	Mappings	ARRAY Items [{NUMBER(from), NUMBER(to)}]	ARRAY_BUILDER	An array of objects that contains properties ‘from’ and ‘to’.	null
mappings	Mappings	ARRAY Items [{{}(from), {}(to)}]	ARRAY_BUILDER	An array of objects that contains properties ‘from’ and ‘to’.	null
mappings	Mappings	ARRAY Items [{STRING(from), STRING(to)}]	ARRAY_BUILDER	An array of objects that contains properties ‘from’ and ‘to’.	null
mappings	Mappings	ARRAY Items [{TIME(from), TIME(to)}]	ARRAY_BUILDER	An array of objects that contains properties ‘from’ and ‘to’.	null

JSON Example

{
  "label" : "Replace All Specified Values",
  "name" : "replaceAllSpecifiedValues",
  "parameters" : {
    "inputType" : "",
    "input" : [ { } ],
    "type" : "",
    "mappings" : [ {
      "from" : "00:00:00",
      "to" : "00:00:00"
    } ]
  },
  "type" : "dataMapper/v1/replaceAllSpecifiedValues"
}

Replace Multiple Values by Key

Name: replaceMultipleValuesByKey

Replaces all values specified by the keys in the input object with the values specified by keys in the output object.

Properties

Name	Label	Type	Control Type	Description	Required
input	Input	OBJECT Properties {}	OBJECT_BUILDER	An object containing one or more properties.	true
output	Output	OBJECT Properties {}	OBJECT_BUILDER	An object containing one or more properties.	true
mappings	Mappings	ARRAY Items [{STRING(from), STRING(to)}]	ARRAY_BUILDER	Object that contains properties ‘from’ and ‘to’.	true

JSON Example

{
  "label" : "Replace Multiple Values by Key",
  "name" : "replaceMultipleValuesByKey",
  "parameters" : {
    "input" : { },
    "output" : { },
    "mappings" : [ {
      "from" : "",
      "to" : ""
    } ]
  },
  "type" : "dataMapper/v1/replaceMultipleValuesByKey"
}

Replace Value

Name: replaceValue

Replaces a given value with the specified value defined in mappings. In case there is no mapping specified for the value, it returns the default value, and if there is no default defined, it returns null. You can also change a string value with regex.

Properties

Name	Label	Type	Control Type	Description	Required
type	Value Type	STRING Options ARRAY, BOOLEAN, DATE, DATE_TIME, INTEGER, NUMBER, OBJECT, STRING, TIME	SELECT	The value type.	true
value	Value	ARRAY Items []	ARRAY_BUILDER	The value you want to replace.	true
value	Value	BOOLEAN Options true, false	SELECT	The value you want to replace.	true
value	Value	DATE	DATE	The value you want to replace.	true
value	Value	DATE_TIME	DATE_TIME	The value you want to replace.	true
value	Value	INTEGER	INTEGER	The value you want to replace.	true
value	Value	NUMBER	NUMBER	The value you want to replace.	true
value	Value	OBJECT Properties {}	OBJECT_BUILDER	The value you want to replace.	true
value	Value	STRING	TEXT	The value you want to replace.	true
value	Value	TIME	TIME	The value you want to replace.	true
defaultValue	Default Value	ARRAY Items []	ARRAY_BUILDER	If there is no existing mapping, assign this value as default.	true
defaultValue	Default Value	BOOLEAN Options true, false	SELECT	If there is no existing mapping, assign this value as default.	true
defaultValue	Default Value	DATE	DATE	If there is no existing mapping, assign this value as default.	true
defaultValue	Default Value	DATE_TIME	DATE_TIME	If there is no existing mapping, assign this value as default.	true
defaultValue	Default Value	INTEGER	INTEGER	If there is no existing mapping, assign this value as default.	true
defaultValue	Default Value	NULL	NULL	If there is no existing mapping, assign this value as default.	true
defaultValue	Default Value	NUMBER	NUMBER	If there is no existing mapping, assign this value as default.	true
defaultValue	Default Value	OBJECT Properties {}	OBJECT_BUILDER	If there is no existing mapping, assign this value as default.	true
defaultValue	Default Value	STRING	TEXT	If there is no existing mapping, assign this value as default.	true
defaultValue	Default Value	TIME	TIME	If there is no existing mapping, assign this value as default.	true
mappings	Mappings	ARRAY Items [{[](from), [](to)}]	ARRAY_BUILDER	An array of objects that contains properties ‘from’ and ‘to’.	null
mappings	Mappings	ARRAY Items [{BOOLEAN(from), BOOLEAN(to)}]	ARRAY_BUILDER	An array of objects that contains properties ‘from’ and ‘to’.	null
mappings	Mappings	ARRAY Items [{DATE(from), DATE(to)}]	ARRAY_BUILDER	An array of objects that contains properties ‘from’ and ‘to’.	null
mappings	Mappings	ARRAY Items [{DATE_TIME(from), DATE_TIME(to)}]	ARRAY_BUILDER	An array of objects that contains properties ‘from’ and ‘to’.	null
mappings	Mappings	ARRAY Items [{INTEGER(from), INTEGER(to)}]	ARRAY_BUILDER	An array of objects that contains properties ‘from’ and ‘to’.	null
mappings	Mappings	ARRAY Items [{NUMBER(from), NUMBER(to)}]	ARRAY_BUILDER	An array of objects that contains properties ‘from’ and ‘to’.	null
mappings	Mappings	ARRAY Items [{{}(from), {}(to)}]	ARRAY_BUILDER	An array of objects that contains properties ‘from’ and ‘to’.	null
mappings	Mappings	ARRAY Items [{STRING(from), STRING(to)}]	ARRAY_BUILDER	An array of objects that contains properties ‘from’ and ‘to’.	null
mappings	Mappings	ARRAY Items [{TIME(from), TIME(to)}]	ARRAY_BUILDER	An array of objects that contains properties ‘from’ and ‘to’.	null

JSON Example

{
  "label" : "Replace Value",
  "name" : "replaceValue",
  "parameters" : {
    "type" : "",
    "value" : "00:00:00",
    "defaultValue" : "00:00:00",
    "mappings" : [ {
      "from" : "00:00:00",
      "to" : "00:00:00"
    } ]
  },
  "type" : "dataMapper/v1/replaceValue"
}

Additional instructions

Overview

Data Mapper is a component or a tool designed to facilitate and configure the transformation of data from one format to another. Data Mapper typically allows you to map fields from the source data (typically a JSON output from another component) to the fields of the target data format or structure (e.g., the data model of another application). This component is particularly useful when dealing with diverse data sources and targets, or when data structures do not align directly and require transformation, such as renaming fields, renaming keys or reformatting values.

Rename Keys

Overview

This action allows you to specify mappings that direct the renaming of keys in an input object to new names according to defined rules.

Property Description

The process involves two main components:

Input: This defines your input value. It can be inputted manually or using data pills, which is its intended use case.
Mappings: Each mapping requires two parameters:
- From Path: This specifies the location of the key within the structured data that needs to be renamed.
  - Dot Notation: The path is defined in a dot notation (for JSON), which allows the action to precisely identify and access the key in a nested or hierarchical data structure.
- To: This specifies the name that the original key should be changed to. This renaming can be necessary for several reasons such as making key names more descriptive, aligning data from different sources with a common schema, or complying with the naming conventions of a target system.

Use Cases:

By defining these mappings, the Rename Keys action effectively helps in normalizing data, making it easier to integrate and process across various systems and applications. This capability is particularly valuable in data integration, migration projects, or when interfacing with external systems that require a specific data schema.

Example

Replace Value

Overview:

This action enables the substitution of specified values, based on predefined mappings. This action strictly adheres to type consistency, meaning that the values being replaced and their replacements must be of the same data type.

Property Description

Value Type: Initially, you must define the data type of the values you intend to replace. This ensures type consistency, preventing errors that could arise from type mismatches (e.g., replacing a string with an integer).
Value: This defines your input value. It can be inputted manually or using data pills, which is its intended use case.
Default Value: If the replacement action encounters a value for which no explicit mapping is defined, it refers to a predefined default value. If no default value is specified either, the action will return null or a similar placeholder to denote the absence of a replacement.
Mappings: You need to establish a set of mappings that dictate how each value should be replaced. A mapping consists of a pair - the original value (From) and its intended replacement (To).
- Regex Support for Strings: In special cases where the values to be replaced are strings, you have the flexibility to define replacements using regular expressions (regex). This allows for more dynamic and complex matching and replacing scenarios, such as pattern-based text substitutions.

Use Cases:

This action is particularly useful in data cleansing and transformation tasks where consistency and correctness of data types are crucial. It is useful when you need to transform a single data pill output or a value in the output in a multiple iteration process.

Replace All Specified Values

Overview:

This action automates the batch replacement of specified data within objects or arrays of objects, based on predefined mappings of values. Once initiated, the action iterates through each parameter of the input objects. If a parameter’s value matches a key in the mappings, it is replaced by the mapped value. Parameters not specified in the mappings remain unchanged. Additionally, the action provides the capability to modify string values through the use of regular expressions.

Property Description

Input Type: This action accepts either a single object or an array of objects.
Input: Depends on Input Type. Each object is treated individually, undergoing a comprehensive scan through all of its parameters for potential replacements.
Value Type: Before performing replacements, it’s crucial to ensure that all mappings maintain type consistency. This means that both the original value (From) and the new value (To) within each mapping pair must share the same data type.
Mappings: You need to establish a set of mappings that dictate how each value should be replaced. A mapping consists of a pair - the original value (From) and its intended replacement (To).
- Regex Support for Strings: In special cases where the values to be replaced are strings, you have the flexibility to define replacements using regular expressions (regex). This allows for more dynamic and complex matching and replacing scenarios, such as pattern-based text substitutions.

Use Cases:

This action can be used when you want to replace multiple values, but don’t know where they are located or don’t have the time to locate all of them. For example:

Data Sanitization: Replacing or anonymizing sensitive information from objects within a dataset.
Data Standardization: Ensuring that all data entries adhere to a uniform format or set of terminologies.

Example

Replace Multiple Values by Key

Overview:

This action facilitates targeted replacements within an object based on a set of predefined mappings between keys. These mappings dictate which values from specified keys (‘From Path’) should be replaced by values from other keys (‘To Path’).

Property Description

Input: This defines your input value. It can be inputted manually or using data pills, which is its intended use case.
Output: This defines your output value.
Mappings: Each mapping requires two parameters:
- From Path: Indicates the key path in the Input object where the current value is located.
- To Path: Points to the key path in the Output object whose value will replace the ‘From Path’ value.
  - Dot Notation: Paths to both keys within are specified in dot notation. This allows for precise targeting and modification of entries, even within deeply nested structures.

Use Cases:

This method is especially useful for restructuring or transforming the structure of complex data objects without altering their inherent data format. For example:

Data Integration: Useful in integrating systems where data from one system needs to be mapped to the schema of another system, hence facilitating smoother data interoperability.
Configuration Overrides: Allows dynamic adjustments of configurations within software systems where properties from one part of a configuration object need to be replaced with those from another based on varying operational conditions or business rules.

Example

Map Objects to Object

Overview:

This action is a transformation technique designed to selectively project and potentially rename properties from input data, creating a new, simplified object. This action is highly configurable, allowing decisions on inclusion based on mappings, and handling of unmapped fields, nulls, and empty strings.

Property Description

Input Type: Indicates whether the provided data is a single object or an array of objects.
Input: An object that contains various properties to be mapped and filtered based on specified conditions.
Mapping: An array of objects where each object includes properties ‘From Path’, ‘To’, and ‘Required Field’. This directs how each property in the input is processed and renamed in the output.
- From Path: Specifies the current location of the property using dot notation, which is useful for locating nested values.
- To: The parameter allows users to rename properties in the output object, providing flexibility in structuring the output.
- Required Field: The parameter can indicate which fields are mandatory for the function to process. If the field doesn’t have a value, an exception is thrown. Default value is false.
Include Unmapped: Specifies whether properties not included in mappings should nevertheless appear in the output object. Default value is false.
Include Nulls: Determines if properties with null values should be included in the output. Default value is true.
Include Empty Strings: Indicates whether to include properties that have empty strings as values in the output object. Default value is true.

Use Cases:

Data Cleaning and Structuring: Perfect for restructuring incoming data streams to fit the schema expected by downstream systems or databases.
API Data Preparation: Useful in preparing data received from external APIs, where only specific information is needed, or names need standardization.
Configurations and Settings Management: Can be used to selectively extract and rename settings from complex nested configuration objects, making it easier to manage application settings.

Map Objects to Array

Overview:

This action is designed to convert a given single object or an array of objects into an array composed of key-value pairs. This transformation facilitates easier manipulation, aggregation, or visualization of nested or complex object data by flattening it into a more accessible, tabular format.

Property Description

Input Type: Defines whether the input provided is a single object or an array of objects.
Input: An input object containing one or more properties that will be transformed into an array of key-value pairs.
Field Key: Specifies the property key in each newly created object within the output array. The value of this key in the new object will be the name (key) of a property from the input object.
Value Key: Specifies the property key in each newly created object within the output array that holds the property value from the input object.

Use Cases:

Data Transformation: Useful for data processing tasks where object structures need to be simplified or normalized for further processing, such as in analytics or reporting tools that require flat data structures.
API Response Transformation: Ideal for transforming complex JSON structures received from API calls into a format that is easier to manage or display in user interfaces.
Database Loading: Assists in the preparation of data for loading into databases that are optimized for handling flat structures, such as relational databases or certain types of NoSQL databases.

Example

Merge and Pivot Properties by Key

Overview:

This action transforms an array of objects into a new, consolidated object based on a specified key. This action enables the aggregation and reorganization of data where each unique key becomes a single property in the resultant object, and the associated values are formed into sub-properties based on other shared properties in the input objects.

Property Description

Input: This property takes an array of objects. Each object should have key-value pairs that need to be evaluated and merged based on shared keys.
Field Key: This is the key based on which the input objects are analyzed and merged. For each unique value of this key found across objects, a new property is created in the resultant object.
Field Value: This is the value that every property in each sub-object of the resultant main object will have. Essentially, it defines the static value assigned to each key derived during the merging process.

Use Cases:

Data Aggregation: Useful in scenarios where there is a need to aggregate information that shares common identifiers across multiple records, such as combining different attributes of products listed by the same identifier across various datasets.
Analytical Reporting: Facilitates the creation of pivoted data structures that are often required in analytical reporting and data visualization to summarize data effectively.
Configuration Management: In systems configurations, merging different configuration objects based on a common identifier can simplify the management and deployment of configuration settings.