Drag and drop imported meshes

[aaronmck]

Hello all,

I have a quick question if anyone could help me with.

I've set up a scene and imported multiple meshes that were made in 3ds Max and have them positioned where I want them on the scene. It consists of a PC case and all the PC's components beside it.

Now I'm trying to implement the drag and drop function so I can click on an imported mesh and be able to drag it into the PC case and position it where it's supposed to be.

I tried adding the code from the drag and drop demo from the playground into my code but when I run it, it doesn't do anything. 

Can I use this drag and drop function on imported meshes or is it only for meshes that are created from scratch in the code?

Thanks!

PS, I would add a playground with my models and code but I'm unsure how to set it up to show my models.

[Wingnut]

Hi @aaronmck   First, yes, you can certainly do drag'n'drop on imported mesh.  You might have to set their isPickable = true, but no problems.  You might need to add an actionManager to each, but still, easy.  But building drag-handlers that will allow you to align nicely, and rotate nicely... on all axes... that is a challenge.  You are starting to venture into writing a scene layout editor.

Most folks would not bother doing "scene layout" with dragging.  They would just set mesh.position(x, y, z) and mesh.rotation(x, y, z)... and see how it looked.  If not good, change the numbers, RUN again.  Letting the engine place the objects... is actually easier than writing drag'n'drop code WITH possible "snap" system.  But don't let me stop you from building an editor... if you have that much gumption and patience. 

The drag'n'drop demo in the playground... absolutely NEEDS the ground to be there.  It needs it for its line 64 getGroundPosition function.  So if you removed or replaced the ground during your testing, that would make things fail (watch the JS console for error reports)

But let's put the drag'n'drop on-hold, for now.  (ar ar ar  drag'n'drop... on "hold"... good comedy there) 

Do you have a free github account, or some other place where you could publish your .babylon file?  Or are you using .obj imports?  We need to find a place to publish your files, so we can access them from the playground.

Here's an example...  http://www.babylonjs-playground.com/#25Z6LR#2

In line 79... I load-in https://cdn.rawgit.com/Wingnutt/misc/master/tree.babylon

I actually drag'n'drop'd that file... from my Windows Desktop, onto my misc folder... on my free github account.   (Then hit the "commit" button at the bottom of the page.)  It was really easy... and really free.    If you can learn to do that... first... we can REALLY advance your situation... nicely.  Let's get that darned .babylon file of yours... even if its only a testing file... into the playground.  Then we'll play with drag'n'drop.  Sound like a plan?

You don't need to use github to publish your .babylon files.  Other places are within reach of the playground, too.  I chose github... cuz it was easy.    Take notice that even though tree.babylon is in my github misc folder... it's address to retrieve it into the playground... is slightly different... https://cdn.rawgit.com/Wingnutt/misc/master/

Party on, we'll talk again soon... if ya want.

[aaronmck]

Hey @Wingnut thanks for your response.

I followed your guide and created the git repository and successfully added a few of my models to the playground for testing purposes. Here it is.

http://www.babylonjs-playground.com/#BUERM#0

I added the drag and drop code from the example on playground and it works, but it doesn't work when I use it in the rest of my code on Visual Studio for some reason???

A problem I found was that if my model has more than one mesh making it up I am able to move all these individual meshes rather than the full model itself as seen on the motherboard model. Any ideas on how that can be fixed so that I can click anywhere on the motherboard and be able to move the entire model?

A couple of other things I would like to be able to do are;

1. Move the models up and down as well as left and right (not worried about rotating it)

2. Be able to click on a model and a description of it appears on the bottom right of the screen if that is possible?

I look forward to any of your suggestions. 

Many thanks!

[GameMonetize]

What is the problem when using inside your visual studio?

[aaronmck]

The drag and drop function works in the playground but when I add the code into my visual studio code and run it, the models show up but I am unable to drag them. 

There are no errors reported in visual studio.

I've attached a .txt file with my code if you get a chance to look at it and see if there is any issues that you can see.

my_code.txt

[GameMonetize]

Ok I think you need to add a reference to jQuery PEP or hand.js to get pointer events support

[aaronmck]

Sorry for being a noob but how do I reference hand.js?

Is that just adding <script src="hand.js"></script> to the top of the html?

[Wingnut]

Well done on the github and pg!  Holy crap.  You've come a long way in a short time!  Nice models, too.

I did a little playing...

http://www.babylonjs-playground.com/#BUERM#4

Just the motherboard, now.  Moved the camera around a bit, now facing +z... just because I like +z aimed cameras.     Rotated the motherboard around, too, so it seemed correct per the camera.  When I rotated everything around, the drag'n'drop went goofy. It was dragging objects in the opposite direction than what was wanted.  Because of this... I added line 113.  It's the first time I have ever used a Vector3.Cross() in my life, and, well, I'm kind of proud that it worked.  Dragging is normal again. 

Now let's go to lines 29-31.  Here, I iterate thru all 42 meshes of the motherboard, and set eachmesh.isPickable = false.  I was hoping to turn-OFF pick-ability for all motherboard mesh, of course.  It didn't work!  What the heck?  I'm still puzzled by that one.  So I haven't done ANYTHING helpful in that area, yet.  Still thinking/testing.

Now for your numbered questions: 
  #1.  Can do, but there is an interesting thing to think about.  Do you know what role the ground... and the getGroundPosition play, in this drag'n'drop code?  They allow us to establish a X/Z "diff"... which is the amount of change... per each onPointerMove run.  But if you want to do Y changing, you should actually have another ground... sitting upright (head-on to the camera)... BEHIND the motherboard.  It would act as your Y diff determiner, and would also work for X positioning... but would not work for Z (moving a mesh forward/backward).

This is why some modelers use tri-plane grids.  If you think about this...
    - if "diff" is established from a BACK grid-plane... you can drag on X and Y, but not Z.
    - if "diff" is established from a SIDE grid-plane... you can drag on Y and Z, but not X.
    - if "diff" is established from a BOTTOM/FLOOR grid-plane... you can drag on X and Z, but not Y.

But this is the "official" way of thinking.  You can do it MUCH easier... with something like checking for the control/shift/alt button being pressed.  Inside the onPointerMove function, if control button held while dragging, just FORCE the mesh to do UP/DOWN (Y) moving instead-of X and Z.  Let's see if I can do it...

http://www.babylonjs-playground.com/#BUERM#5

I changed line 115.  I force-fed diff.x into the Y value of a new vector3 I made.  (Apparently X is now forward and backward, because of my poorly-thought-out motherboard rotation).  All in all, I wanted to show you how easy it was to change the "binding" of what drag-change... is mapped to what mesh position change.  I think, with a little playing, you can learn to check for control key and adjust your "drag mode" in any way you wish.  The same can be done for other drag "constraints", too.  Maybe you want to disallow Z movement when the ALT key is held.  Same deal... just check for alt key, and if held, do a DIFFerent type of diff-binding to mesh... adjusting our beloved addInPlace values to get what you want.

With me so far?  I hope so.  This type of kludge-forcing removes the necessity for extra ground planes (tri-planes).  Ok... continuing on...

  #2 - this is best done with canvas2 panels... often a Group2D, with a single Rectangle2D as a child, with a Text2D as a child.  Those are all good playground search terms... to find examples.  Canvas2D operations always begin with a screenSpaceCanvas or a worldSpaceCanvas.  In your case, I suggest a worldSpaceCanvas (wsc).  A wsc has a mesh ( a node) associated with it... and that mesh can be parented to the camera... and thus... the wsc mesh can be placed into the corner of the camera view, no matter where the camera is aimed.

Conversely, screenSpaceCanvas is mapped across the entire screen, and they can be transparent-filled, so the scene can be viewed "through" the ssc.  BUT, we have seen SOME issues in getting picks-on-mesh to work "through" the ssc.  Excellent docs exist for Canvas2D system, but, unfortunately some/all worldSpaceCanvas demos are currently failing.  I've now seen two... fail... because of this.parent problem.  My lovely 4-wsc super-gui demo (which acts like a single screenspace but allows mesh-picking in the middle)... is currently broken for the same reason.  Maybe @Nockawa will visit.

So, wsc will work great for you, with its mesh (its node) parented to the camera, positioned in a lower corner of the screen, displaying mesh.description, upon mouseover perhaps... quite doable.  Maybe even worldspace.node.setEnabled(true/false) can be used to toggle its displaying.  *shrug*

I hope all this helps.  Sorry to flop this down in the middle of your chat with Deltakosh.  You can ignore all this till later, of course.  Party on!

[Nockawa]

@Wingnut the bug just got solved by raanan and merged, the PG should be updated soon. I wasn't aware of this bug. Did you ping me before about it?

Another point to mention, ScreenSpaceCanvas don't have to span the whole screen, you can set a position and/or a size. There are issues with Pointer Event which I'm resolving right now.

Do you have a PG about the Mesh Picking issue due to a SSC ? I think I could solve this issue pretty quickly if I have a Use Case, thanks.

Edit: PG Updated and the bug is fixed

[Wingnut]

Hi @Nockawa, thanks for the comment/caring.  Nope, this is the first time, I think.  It all came about from a thread.  At first, I misunderstood @masterK when he said "above".  I thought he meant button panel on bottom of screen, mesh-picking scene above.  But then I realized he really meant "atop"... layered over a scene... he was having problems getting mesh-clicks to happen "thru" it, as I understood it.

That's when I made that ugly 4-wsc demo.  But yeah, I forgot about ssc's having size and origin.  Yep, good.  And also good news about the parent problem fix... coming soon.

And good news about picking events, too.  You're FULL of good news, today, noxy!  Thanks! 

Yeah, even WITH origin-settable ssc's, you still need 4 of them, in order to put GUI on all 4 edges of the scene (unless/until that pointer-events issue is worked).  I don't envy your task.    oooh, what about picking mesh THRU transparent fills of rectangles?  Wow!  That would REALLY be hard, eh?  Seems like it to me.    Sorry I don't have any definitive test playgrounds.... other than what you might find in that thread.  Be well, thx agn.

Back to aaronmck's pending question, when we have a chance.  How to ref-up hand.js in typescript project?  I don't have that answer.  Others will. 

 

[aaronmck]

Thanks for all your responses, a great help!

I referenced PEP in the document head using <script src="https://code.jquery.com/pep/0.4.1/pep.js"></script> and now I am able to interact with the meshes which is great.

I went back into 3ds Max and used ProBoolean to connect all the pieces of the motherboard together so I am able to move the motherboard along the x axis by clicking anywhere on the model which rocks!

I just need to figure out how to implement clicking the alt/ctrl key to move the models up and down on the y axis and utilising worldSpaceCanvas for a description to appear somewhere on screen.

Slowly but surely I'm getting there.

Thanks!

[Wingnut]

Hi again @aaronmck

Hey, I just finished a post showing a new friend how to do that control key thing.  It might be helpful for you, too.  Have a read, if you like. 

Be sure to use our playground search, forum search, and Nockawa's seriously-fantastic Canvas2D docs... for many examples and much talk about the new Canvas2D system... for your corner-of-screen description panel.  And in the Announcements subForum... is the Canvas2d "howdy-doo" thread... which is a large but delicious read.

Nockawa visited and reminded us that screenSpaceCanvas has a size and origin (location on screen space) settings, so I think screenSpaceCanvas is best for your info-panel.  It doesn't need a plane parented to the camera, or any parenting whatsoever.  It just sticks to the back-side of your monitor screen, and stays there.    That's why it's called SCREENspace.    Conversely, a worldSpaceCanvas can fly all over the world.

Still, try to keep the panel reasonably small, for now.  Perhaps less-than 25% of the screen, which is already quite big.  You'll be able to know more, once you activate the panel and play with fonts.  I hope it word-wraps... in case you get as talkative as ol' Wingnut... in that panel.

If it gets too big, and you need to click a mesh that is behind that panel (esp true if your panel fill is transparent)... the click might not reach the mesh.  It might be blocked by the corner canvas2d.  But that is being worked-on, too, according to Nockawa.  Your future looks bright, AA! 

Talk soon.