say hello to sdTF – the structured data transfer format for computational design

Say hello to sdTF – an extensible/hackable data transfer format inspired by glTF and Grasshopper DataTrees. It’s optimized for storing and transferring structured geometry/BIM data – but can be extended pretty much in any way.

This is a product of the ShapeDiver / Swarm merger that was announced a while ago. We needed an efficient way to transfer data structures between sever and client in such way it could easily be parsed processed on either ends. Additionally, chaining applications based on parametric 3D modelling requires an open, efficient, and easily extensible data format for exchange of complex data structures. sdTF aims to close this gap.

We’ve been working hard on this for the past couple of months, so it’s super excited to see it’s finally finding its way out of the cave! 🙂 So far the specification has been published on this github repo, but we’re planning to release the JavaScript and .NET implementation we’ve worked on soon. Check out the demo below:

Demo of sdTF saving/loading from Grasshopper. The enire DataTree can be stored in the sdTF along with the path/branch information. Courtesy of Alexander Schiftner

Without digging too deep into the technical details (see the spec in the github repo), sdTF makes it possible to separate the meta data from the binary payload data to enable lazy loading. That means you can describe the enire content of a DataTree without actually having to access/parse/fetch the data, which is useful when dealing with large geometic objectcs.

The metadata is stored in JSON format. A flat hierarchy is used, using numeric indices for refering between child items of top level attributes. The properties of the JSON contains:

  • Asset – Meta information about the asset itself.
  • Nodes – analog to branches in Grasshopper
  • Chunks – analog to data trees in Grasshopper
  • Data items – analog to branch items in Grasshopper
  • Buffers – aggregated binary data
  • Bufferviews – a pieace of the buffer representing a file (can be Rhino3dm files, images and more)
  • Accessors – A link between the Data items and BufferViews
  • Attributes – A mechanism for associating attribute data with data items.
  • Type hints – Contains information about types (for instance text, numbers, meshes, curves etc)

Here’s what an example JSON can look like:

{
  "attributes": [{
      "Name": {
        "value": "Mesh sphere",
        "typeHint": 3
      },
      "Color": {
        "value": "126, 156, 255",
        "typeHint": 4
      },
      "Layer": {
        "value": "Some layer",
        "typeHint": 3
      },
      "Preview": {
        "accessor": 3,
        "typeHint": 1
      }
    }, {
      "Id": {
        "value": "57e60008-ee18-4864-8711-1cbc8adfc821",
        "typeHint": 5
      },
      "Name": {
        "value": "Mesh",
        "typeHint": 3
      },
      "Type": {
        "value": "Mesh",
        "typeHint": 3
      }
    }, {
      "Id": {
        "value": "c641e0f8-ecfd-4607-936f-01ed06ac7dbd",
        "typeHint": 5
      },
      "Name": {
        "value": "Images",
        "typeHint": 3
      },
      "Type": {
        "value": "GrasshopperBitmap",
        "typeHint": 3
      }
    }, {
      "Id": {
        "value": "fc5cedb5-42c0-4238-ae01-9cf94d194130",
        "typeHint": 5
      },
      "Name": {
        "value": "Numbers",
        "typeHint": 3
      },
      "Type": {
        "value": "Number",
        "typeHint": 3
      }
    }
  ],
  "typeHints": [{
      "name": "rhino.mesh"
    }, {
      "name": "image"
    }, {
      "name": "double"
    }, {
      "name": "string"
    }, {
      "name": "color"
    }, {
      "name": "guid"
    }
  ],
  "chunks": [{
      "name": "57e60008-ee18-4864-8711-1cbc8adfc821",
      "nodes": [0, 1],
      "typeHint": 0,
      "attributes": 1
    }, {
      "name": "c641e0f8-ecfd-4607-936f-01ed06ac7dbd",
      "nodes": [2],
      "typeHint": 1,
      "attributes": 2
    }, {
      "name": "fc5cedb5-42c0-4238-ae01-9cf94d194130",
      "nodes": [3, 4],
      "typeHint": 2,
      "attributes": 3
    }
  ],
  "nodes": [{
      "name": "[0,0]",
      "items": [0],
      "typeHint": 0
    }, {
      "name": "[0,1]",
      "items": [1],
      "typeHint": 0
    }, {
      "name": "[0]",
      "items": [2],
      "typeHint": 1
    }, {
      "name": "[0,0]",
      "items": [3, 4, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14],
      "typeHint": 2
    }, {
      "name": "[0,1]",
      "items": [4, 5, 6, 7, 15, 16, 9, 17, 18, 19, 20],
      "typeHint": 2
    }
  ],
  "items": [{
      "accessor": 0,
      "typeHint": 0,
      "attributes": 0
    }, {
      "accessor": 1,
      "typeHint": 0
    }, {
      "accessor": 2,
      "typeHint": 1
    }, {
      "value": 0.0,
      "typeHint": 2
    }, {
      "value": 1.0,
      "typeHint": 2
    }, {
      "value": 2.0,
      "typeHint": 2
    }, {
      "value": 3.0,
      "typeHint": 2
    }, {
      "value": 5.0,
      "typeHint": 2
    }, {
      "value": 8.0,
      "typeHint": 2
    }, {
      "value": 13.0,
      "typeHint": 2
    }, {
      "value": 21.0,
      "typeHint": 2
    }, {
      "value": 34.0,
      "typeHint": 2
    }, {
      "value": 55.0,
      "typeHint": 2
    }, {
      "value": 89.0,
      "typeHint": 2
    }, {
      "value": 144.0,
      "typeHint": 2
    }, {
      "value": 7.0,
      "typeHint": 2
    }, {
      "value": 11.0,
      "typeHint": 2
    }, {
      "value": 17.0,
      "typeHint": 2
    }, {
      "value": 19.0,
      "typeHint": 2
    }, {
      "value": 23.0,
      "typeHint": 2
    }, {
      "value": 27.0,
      "typeHint": 2
    }
  ],
  "accessors": [{
      "bufferView": 0,
      "id": "ae2b6cef-ce2c-4a44-8fc7-e0eb23588a89"
    }, {
      "bufferView": 0,
      "id": "01b482a5-115a-4327-b557-79e3f40ad397"
    }, {
      "bufferView": 1
    }, {
      "bufferView": 2
    }
  ],
  "bufferViews": [{
      "buffer": 0,
      "byteOffset": 0,
      "byteLength": 12477,
      "contentType": "model/vnd.3dm",
      "contentEncoding": "gzip"
    }, {
      "buffer": 0,
      "byteOffset": 12480,
      "byteLength": 2172131,
      "contentType": "image/png"
    }, {
      "buffer": 0,
      "byteOffset": 2184612,
      "byteLength": 173507,
      "contentType": "image/png"
    }
  ],
  "buffers": [{
      "byteLength": 2358119
    }
  ],
  "asset": {
    "generator": "ShapeDiverSdtfWriter",
    "version": "1.0"
  }
}

I recommend reading the full specification for more info.

Stay tuned for the JavaScript and .NET SDKs! Also worth emphasising is that all implementations will be open sourced, so anyone who’s interested in contributing is very welcome to open a pull request or reach out to us for a chat!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s