The Data Mapper at a glance

It’s the end of the 16th week of the internship and this post explains the progress of the JavaScript based data mapper so far.

After the ‘backbone’ of the app was set out, focus was given to adding more UI functionalities – where the user can customize the input, output and the operations.

The front-end

So far, the data mapper initializes with an input container, output container and a tool palette(integrated from the NEL tooling platform).

Each tool in the tool palette can be dragged and dropped on the canvas, where the corresponding operator is drawn. Each operator has its defined set of input/output labels and valid types and each operator has the option to be deleted.

Additionally, the direct connectors (Connecting Input container to output container) will be corresponding to a “Direct operator” without visible components.

The “Input” tool in the “Tree containers” is used to drag and drop extra input tree containers.

Each tree container has a total of 3 options which may appear or not based on the content in it. Empty containers have the option “Add new root” in place of the “Clear container” option in non-empty containers and the abstract functionalities of the options are as follows:

1. Load schema: Select the structure based on a file of one of the types of XML, JSON, CSV, XSD, JSON Schema.
2. Add new root: Build the structure manually by defining the root and adding nodes.
3. Clear container: Empties the container

The extra containers added by the user have the additional option to be deleted.

When reading from a file or building the structure in a tree container, the containers create “Node”s in a tree-structure. The parent nodes are of type Object or Array and others are leaf nodes and each leaf node has an “Anchor”. Input type anchors can be dragged and dropped on a node of opposite type which will create “Connector”s. A hidden svg:polyline of higher width is used to improve clicking on the connector.

Each node has 3 options as follows:

1. Add node: Add a new node to the structure.
2. Edit node: Edit the title and type of node
3. Delete node: Delete the node from the app.

When adding a new node, it is added as a sibling by default, but if the triggered node is a parent node, there is an option to add the new node as a child. When adding a child node, the child is added as the last child of the triggered node and when adding a sibling node, the sibling is added as the immediate next sibling of the triggered node.

The back-end at a glance

The tree structure of each tree container is maintained as a JSON Schema. When reading a file of a different type such as XML, JSON, CSV and XSD, the structure is converted to JSON Schema type and is manipulated throughout the app.

As a whole, the app creates and maintains a global JS object “Diagram” which includes the objects of the diagram as Backbone models or collections.

1 2 3 4 5 6 7 8 9

var Diagram = { Canvas: {}, //the editor = svg TreeContainers: {}, /* Backbone collection corresponding to Input/Output container the model contains the loaded schema as a JS object and a collection of nodes */ Connectors: {}, //all the 'lines' drawn in the application ToolList: {}, //The collection of tools PalleteView: {}, Operators: {} //a Backbone collection of all the operators in the canvas };

The result

The generated graph of mappings is calculated by means of the input/output adjacency lists of each operator. The result includes:

1. A variable list: All the nodes
2. Operators: A list of all operators including direct operators
3. Input adjacency list: Input nodes of the operators in the list
4. Output adjacency list: Output nodes of the operators in the list