Data Visualization Authoring Tool

[JBatUN]

Greetings all,

I'm new to Babylon and this forum (first post) but wanted to share a project I've been working on in case anyone is interested.  I'm attaching a few screen shots of a data visualization authoring tool that provides views for planar and spherical maps, graphs, trees, and most conventional charts (cartesian and polar).

The platform is based on Electron.js (to enable file system support), Vue.js (for property management), Polymer (to componentize the interface), and D3.js which is used for the data manipulation, e.g. parsing, scales, layouts, geo projections, etc.  Of course Babylon is used for all the rendering.

Once I clean up all the code I'll share the work on github with hopes others will be interested and possibly contribute.   My goal is to have a blender-like authoring tool for data visualization.  As you can see from the screen shots, I'm trying to expose all the properties needed to configure a custom view with all the benefits of Babylon.  As part of my learning process I've tried to implement just about every feature available from materials, textures, post processing, geometry, etc.

Eventually I have a few questions on some challenges I've faced.  One of the key features we need is to be able to export a scene for use in an authoring tool such as Blender or Maya.  This is to develop camera fly-throughs and other data oriented storyboards.  I've been trying to use the GLB serializer but it appears there are some issues with lines, etc.  I also struggle a lot with text rendering.  Looking forward to posting some questions and benefitting from all the expertise of the community.  Although this is my first post, I've consulted the forum for support at least a thousand times!

Anyway, hope some find this interesting and eager to hear your feedback.

Cheers,

JB

[Sebavan]

This is an amazing project ! and something we have been discussing a few  times. Do you have an online deployed version we could take a look at ?

About the issues with glb serialiazition let me add @bghgary to the thread.

Concerning the text, could you detail a bit more your issues in a new thread in the Questions and Answers questions ? I bet @Deltakosh will fix it in less time I am writing my answers as Gui is its baby pet ?

[JBatUN]

Sebavan,

Thanks for the words of encouragement!!!  Unfortunately, I don't have an online deployed version yet and now the app is significantly coded for working in the Electron environment.  This was necessary to enable a user to select the csv files used for the visualization, and eventually save files.  I'm moving at a pretty fast pace and should have the code in Github within a week or two.

In the meantime, a few more details for those that might be interested.  The approach provides for virtually any type of data visualization.  Currently the following models are available: planar point map, link map, time map, time path map (e.g. to see flights over time); sphere based map (globe) for all the same; force directed and sankey graphs, several tree layouts (e.g. cartesian, radial, treemap, circle packing, etc.), and finally cartesian and polar based charts (using any geometry).

The integration with D3.js and Babylon.js makes all the above fairly easy.  D3 loads the csv, parses it, and generates the layout, scales (e.g. Linear, Log, Pow, Sqrt, or geospatial).  

function loadData(url) {

      d3.csv(url, function (error, data) {
         data.forEach(function (d) {
            d.lfield = d.lfield;
            d.lat = +d.lat;
            d.long = +d.long;
            d.coords = [+d.long, +d.lat];
            d.zfield = +d.zfield;
            d.sfield = +d.zfield;
            d.cfield = d.zfield;
         });
         renderData(data);

      });
   }

Example of a sizing scale:

var zExtent = d3.extent(data, function (d) { return d.zfield });

var zScale = d3.scaleLinear()
                .domain(zExtent)
                .range([0, 50]);

Once D3 is done, all the coordinates and properties are available for Babylon to consume and render based on the view model.  Depending on the view model, a coordinate transform function generates the sphere, polar, radial, etc. coordinates.

The Babylon particle system then does all the magic:

sps.initParticles = function () {
         var p = 0;
         for (var i = 0; i < data.length; i++) {

            // data variables
            var obj = data;
            var n = obj.lfield;
            var v = obj.zfield;
            var z = zScale(obj.zfield);
            var r = rScale(obj.rfield);
            var s = sScale(obj.sfield);
            var c = cScale(obj.cfield);
            var coords = (obj.coords);

            // scale by z
            sps.particles[p].scale.y = z;
            // reset to account for center origin of geometry
            var offset = sps.particles[p].scale.y;

            // set position by coords
            var position = gcoordsph(coords, offset); // geospatial coordinate transform based on projection
            sps.particles[p].position.x = position.x;
            sps.particles[p].position.y = position.y;
            sps.particles[p].position.z = position.z;

            // set color by variable
            sps.particles[p].color = new BABYLON.Color3.FromHexString(c);

            // Used for tooltips
            sps.particles[p].metadata = [n,v]; // adds name of each object

            p++;
         }
      } // end init function

So far the performance for 1000+ data points has been excellent.  The only exception is for cases where labels are used like in the example attached (World Country Tree with Population).  It takes several seconds to render.  I'm sure a lot of it has to do with my own errors in coding.  However, for a few reasons, the dynamic texture approach is not ideal for this purpose - hoping there is a possibility to implement a vector, svg like approach for text rendering.

One key note is that I named it Cambro (Camera in Esperanto) to strictly limiting the scope to visualization and not data manipulation.  Each view expects a specific data model as a csv file. If the file fits the model, you can map the visual properties for all of the fields.  I'm working towards exposing virtually all the properties available to create a fully customized scene.

Will definitely be posting some questions soon to get the needed expert advice.

Thanks again!  Cheers,

JB

[bghgary]

On ‎5‎/‎7‎/‎2018 at 6:01 AM, JBatUN said:

I've been trying to use the GLB serializer but it appears there are some issues with lines, etc.

@JBatUN Can you explain more about what kind of issues you are having with the serializer?

[JBatUN]

I've only run a few quick tests and am not sure yet how the serializer will work with all the visualization types, but in my initial tests I found two issues:

1.) It didn't seem to work on anything that had lines (the vector maps, or axis in a chart)

2.) There were unpredictable behaviors when loading the file and rendering dynamic texture labels.  From what I can tell (if I use the glTF), these are treated as raster texture images.  Sometimes they show fine, other times the backing plane is black, etc.

At this point I recognize that there are very good chances any issues I experience are because of my own coding and limited experience with Babylon.  I'm also testing the import with Blender, Godot, Babylon Editor, and ClayJS ... each often renders differently.  Once I spend more time on the export capability I'll understand more and will share the results.

Thanks,

JB

[JohnK]

 

Very impressed with what you are doing.

On 5/9/2018 at 7:23 PM, JBatUN said:

The only exception is for cases where labels are used like in the example attached (World Country Tree with Population).  It takes several seconds to render. ....................... However, for a few reasons, the dynamic texture approach is not ideal for this purpose

Even though you said that the dynamic texture approach is not ideal for you I could not help having a play around. What I have attempted is to use SPS for the labels by creating one dynamicTexture and then adding planes to the SPS of appropriate size. In the example I use a single array to give position, rotation and text for each label but obviously can be adapted depending on the data source . The first PG has about 70 labels, the second about 90 labels and the third about 120. After not may more labels my browser starts to ask if it should wait or stop. So I am probably not doing any better than your system. More of a proof of concept than anything else. Anyway just ignore if it is of no help.

https://www.babylonjs-playground.com/#A9K31I

https://www.babylonjs-playground.com/#A9K31I#1

https://www.babylonjs-playground.com/#A9K31I#2

 

Alternative - have you seen this

though I guess the number of vertices needed for lots of labels would slow everything down even more.

Looking forward to seeing your progress.

[JBatUN]

JohnK,

Thanks for the test and playground examples!  It was very useful ... although I'm using SPS for all the geometry, I was not using it for the labels.  Will take a close look at your examples as every little bit may help in performance.

Also, thanks for the 3D text link - was not aware.  I think you're right though that the 3d geometry could add even more overhead.

Ideally, 2d vector text would be the best option.  Something similar to SVG text rendering.  Basically line and curve paths.

Everything else is working so perfectly so I'm hoping a solution will prevail...

Will keep sharing progress and thanks again for the interest,

JB

[kcoley]

Hi @JBatUN,  Awesome stuff you are doing!  I am working on the glTF serializer in Babylon.  I assume that you are using MeshBuilder.CreateLines ?  I currently have only implemented line behavior for geometry fill modes, but I will add this to the list of features to add support for.  Do you happen to know of other things that are currently not working on export? 

[JBatUN]

Wow - this community is awesome!  

To draw the topojson vector maps I use:

var vmap = BABYLON.MeshBuilder.CreateLineSystem("lineSystem", {lines: maplines}, scene);
vmap.color = color;

In other cases, like the flows among countries I use curves such as:

var links = getQuadraticBezierCurve(segments[0], segments[1], segments[2], 16);

(BABYLON.Curve3.CreateQuadraticBezier).

Basically, I'm using just about every form of geometry that Babylon has to offer.

As for the serializer, the lines and curves are the key right now.  For visuals that don't have lines the output has been excellent.  Great work!  It seems the bigger challenge is that tools have exporters (e.g. Maya) but don't have the same capabilities for import.  So far each tool I use to view the GLB has a slightly different result.

The ClayGL viewer has been among the best, but unfortunately not useful for my purpose.

https://pissang.github.io/clay-viewer/editor/

My ultimate goal is to use Blender, Maya or Cinema 4D to produce narrative stories with the visualization.  I am experimenting with a timeline / keyframe editor from within the application using https://github.com/zz85/timeliner.

Thanks for your support!

JB

[The Leftover]

JohnK, dang!  That's an eye-opener.  I didn't know we could do that. 

I have a longstanding tendency to do things the hard way but my little adventure with text this week takes the cake.

[JBatUN]

Quote

little adventure with text...

Leftover - I was very interested in the text work you posted.  I could be wrong but it seems like these libs are similar to what you were doing and could be a potential approach to having 2d vector text.  Based on your experience, curious what you think about these ... worth looking into further?

https://github.com/mikolalysenko/vectorize-text

https://github.com/xeolabs/xeogl/blob/81b1469b38c698d22b17a73895d2de18d6423c5d/examples/js/geometry/vectorTextGeometry.js

Eager to hear your thoughts.

JB

[JBatUN]

Hi all,

Following up on my last with some progress on creating a vector text capability based on the xeogl work.  Attached is a simple test with about 200 labels.  For our purpose, we're happy with the results.  The text has a CAD like appearance and is rendered with a line system.  It does also work with Tube for a 3D appearance but the performance was significantly effected as expected.  If anyone is interested and needs a simple vector text approach we'd be happy to share the code.  Kcoley - I think this should work well with the glTF serializer once you have the time to include line geometry.

Cheers,

JB

[kcoley]

@JBatUN nice! Sounds good, I’ll add line support to the list of features to implement for the exporter. 

[JBatUN]

@kcoley not sure if this is helpful for your glTF serializer work, but aside from the export of lines, I'm also getting the following error message:

BJS - [02:58:48]: Material type ShaderMaterial for material colorShader is not yet implemented in glTF serializer.

Also, if you need some extensive testing for your updates, would be happy to help.

Cheers,

JB

[Spankied]

what UI library are u using?

[JBatUN]

@Spankied  I'm not using a UI Library per se if you mean frameworks like Bootstrap.  Not sure if this answers your question, but here are a few additional details...

- Most of the UI components are based on Polymer and custom CSS.  I never used Polymer before but found it so easy, it served the purpose to have reusable UI elements for each view.  All the elements are pretty much straight html as I'm trying to avoid dependencies.  I'm not event using Polymer components...just the framework for creating elements.

- These components are linked to a Vue.js model [component -> Vue model -> Babylon]

- Also, I tend to use D3.js (for all it's features) as a JQuery like event handler.  E.g,

var saveImage = d3.select("#saveImageFile");
saveImage.on("click", function() {
    BABYLON.Tools.CreateScreenshotUsingRenderTarget(engine, camera, 2048);
});

Lastly, for now I'm borrowing the Blender icons (as svg) or creating my own.

Let me know if you have any questions.

Cheers,

JB

[Spankied]

So must of the styling is custom? Very nice.

[kcoley]

@JBatUN, that is very useful information. I will have to handle the LinesMesh with the exporter and the ShaderMaterial. Thanks for sharing, and I would definitely appreciate any testing you can provide!

[kcoley]

Hi @JBatUN.  I plan on having a fix for the lines issue by Friday at the latest.  I'm tracking progress on the lines issue here: https://github.com/BabylonJS/Babylon.js/issues/4196  .  I'll let you know when I make the next update.

[kcoley]

Hi @JBatUN, my update for lines support was recently merged to master.  Feel free to give it a try and let me know if you hit any other issues.

[JBatUN]

@kcoley  Wow - thanks!!!  I just tried the update with an example that is probably a good stress test (world map line system, label lines, etc.).  It's hard for me to assess the update fully with the variances among the different GLB importers.  I've tried an exported file in the Babylon editor, ClayGL viewer, and Godot ... all have a different result.

There's a chance some of the issues I'm facing are my own error, but here's the error message I receive when running the following code:

var glb = BABYLON.GLTF2Export.GLBAsync(scene, "sceneFile").then((glb) => {
    glb.downloadFiles();
    });

I'm also including the GLB file in case you want to give it a try yourself.

Thanks again for your great work!!!

JB

sceneFile.glb

[The Leftover]

JBatUN, hello!  We are both working on data visualization, as you noticed.

I have focused on public web-based application of civic data.  The target is community, leaders and so on.  Good luck to you!

[JBatUN]

@kcoley I've done a some more experimenting and wanted to confirm that your update to include lines is working perfectly.  The map vectors and labels are rendering exactly as expected.  Awesome job!!!

The issue I'm facing appears to relate to the coloring of the sps geometry:

sps.particles[p].color = new BABYLON.Color3.FromHexString(c);  Where c is just a scale of hex values.  For all the editors I've tried these values are not included in the export.

@The Leftover I'm following your work closely especially your recent work with text rendering.  Very impressive.  We're focused on humanitarian and development data for an international organization and hoping that this platform will enable us to create data driven stories about complex subjects.  Making quick progress on our authoring tool and will share on github as soon as possible.  Hoping others will find it useful and possibly contribute.  And good luck to you too!

[jerome]

sps particles expect a color4, not a color3... because they can handle the transparency

[JBatUN]

@jerome  Thanks for the tip.  I made the change to:

var rgb = new BABYLON.Color3.FromHexString(c);
sps.particles[p].color = new BABYLON.Color4(rgb.r, rgb.g, rgb.b, 1);

c is a data generated value in hex so I'm stuck with the conversion, but the change is working well.

Still getting the error message on export:  babylon.js:4 BJS - [10:55:50]: Unsupported material type: colorShader

In addition to the color shader a material is added with textures ... but this is also not coming through in the export.

I'm sure lot's of user error here so I'll keep experimenting.