DGL 3 DITs presentation By Andrew A Ray

DGL / 3 DITs presentation By Andrew A. Ray

Schedule n n n n 9: 00 -10: 00 - DGL – Overview and programming guide 10: 00 -10: 10 - Break 10: 10 -10: 30 - Discussion on potential usages of DGL with your software projects 10: 30 -11: 00 - Overview of available VR hardware 11: 00 -11: 30 - Introduction to 3 D Interaction techniques 11: 30 -11: 40 - Break 11: 40 -12: 00 - Discussion and final wrap-up 12: 00 - Lunch

DGL n n n Capabilities Case studies Output n n Input n n Open. GL, Open Scene. Graph, Coin, VTK Mouse, keyboard Communication with other processes

Capabilities of DGL n n n Extends DIVERSE’s reach into Open. GL based rendering and toolkits built to render to Open. GL Primary thrust is to provide VR capabilities to Open Scene Graph, while allowing more flexibility than DPF did with Open. GL Performer Improves upon some parts of the DPF API

Case study 1 n Road crossing experiment using VRML n n n Elderly citizens crossing the street in a city Controlled environment with no unreasonable risk for the participant VRML animation for cars / streetlights Coin used to render the project Coinfly example was all it took to run the project

Case study 2 n Benefits of immersion study n n 3 D mathematical equations Different levels of immersion (walls on / off) Sometimes head tracker used / sometimes not Used to help understand 3 D math and what it looks like in its natural surroundings

Case study 3 n n n AMADEUS – Underground mining visualization tool Designed to view statistically generated fractures to simulate what the rock looks like before it is tunneled through Somewhat simple novel computer graphics algorithm to cut different types of the tunnels out of the rock and be able to see them in true 3 D

Screenshot

Result of graphics algorithm

DGL Output capabilities n n n Based around a draw callback that is added to a dtk. Augment Allows for Open. GL to be drawn before and after navigation and has an overlay capability to ensure that it is drawn after all the other Open. GL How does this model work with Open Scene. Graph / Coin / VTK?

DGL Augment Interface n n n Same exact process as making your own dtk. Augment Derive off the draw callback Pass in the type of dgl. Augment you want it to be (i. e. where it is drawn) Load the DSO into your favorite application Done

Porting existing programs to DGL n n n DGL is designed to ease porting old Open. GL programs. There are C based wrappers that offer the same functionality as the C++ interface dgl. Display. Callback dgl. Init. GLCallback dgl. Postframe. Callback dgl. Set/Get. Data

Problems with porting old GL programs n n Timers aren’t implemented in DGL Input probably is going to have to be redone Depending on the application, it may have to be re-architected a bit to handle cluster development May also have to worry about multicontext issues

Open Scene Graph usage n n Step 1 – Develop OSG application Step 2 – Initialize DOSG (DOSG: : init()) Step 3 – Insert top level OSG root node into DOSG: : get. World()>add. Child(here); Done

Open Scene Graph n n DGL handles structuring Open Scene Graphs internals so that all you have to do is add a child to a node in a scenegraph that is drawn by DGL Structure of scenegraph is scene nav ether world

Coin n n Very similar to osg, except there is no nav node scene world ether DCoin: : init – Make the system DCoin: : get. World()->add. Child(here) DCoin: : load. File(“model. iv/wrl”);

VTK n n n 4 step process Create a render window (DVtk. Render. Window: : New()) Create a renderer (DVtk. Renderer: : New()) Add the renderer to the window (Add. Renderer(pointer)) Add your actors to the scene Flip the scene to Open. GL coordinates if desired (DVtk. Renderer->Rotate. Scene. X(90))

Input for DGL n n n Now that we have graphical development taken care of, we need to get how the user develops applications! Same as DPF for wand / joystick / buttons / head tracker (dtk. Shared. Memory) Has a different mouse / keyboard interface due to a different input system (Producer)

Mouse input n n DGLMouse and DGLMouse. Button DGLMouse allows for setting the mouse position, and reading the mouse’s current position or the queued position of the mouse (set. Mouse(x, y); read. Mouse(float* x, float* y); get. Mouse(float* x, float* y);

Mouse Input n n DGLMouse. Button – Tells you what happened to the mouse button, and the position of the mouse when the button action happened Based on top of shared memory Concept of Actions, state for each mouse button int read. Mouse. Button(x, y, action) n n n Returns 0 if it doesn’t have data, otherwise it tells you the position it was changed at and what the change is Actions are press / release / unknown Allows for the ability to fake mouse events if there is a need

Keyboard Input n n It uses the same actions concept as mouse input did DGLKeyboard: : get. State(DGLKeyboard: : get. Key. Character(“a”)); Pass an action pointer (to be written to) into read. Key. Character and it will return the action / character that was pressed Do have to switch between char and Key. Character due to Producer syntax

The raw deal n n If you don’t like the helper classes created then there is always shared memory. It is what the system is based off of. dgl/mouse dgl/keyboard dgl/mousebutton are the segments The structure is usually an integer for the data and an integer for the action

Nuts and bolts n n n Building dgl programs is the same as with dpf programs, dgl-config is your friend dgl-config --include and --libs dgl-config --osg-include --osg-libs dgl-config --coin-include --coin-libs dgl-config --vtk-include --vtk-libs Example makefiles in the examples directory on how they are used

Design strategies for DGL programs n n Decide the type program that you are developing – OSG , Open. GL, etc… Decide the level of interactivity Often having a DSO that reads / displays data is the best approach because you can reuse DGL shell programs (i. e. dosg-fly) Full applications give the most flexibility and opportunity

DGL design strategies n Often I break the problem into these specific parts n n n Reading input data Transforming input data Displaying data Interacting with data Often need to be careful about structure of data storage / accessibility because these decisions often interactivity

DGL design strategies n n Often interactivity places the most demands on applications The earlier in the development process that you consider it, usually the best Aim for several quick iterations that can test different features / abilities Get users involved as soon as possible in the development cycle

Questions / Comments n n Capabilities? Design decisions? Application design? Building applications?

10 minute Break

VR Hardware n n Often what you do in VE creation is heavily influenced by the input devices you have available Very large range of devices, some very well supported, some not so much. In CAVE environments usually light weight one handed devices are preferable Often have to consider how equipment will affect others in the CAVE environment (for demos)

Outline for VR hardware n n n Text entry Positional trackers Pointing devices (other than wands) Niche areas Haptics

One handed text entry A one handed text entry method through chording Press multiple buttons to obtain different letters of the alphabet Advantages: Only needs 16 keys to operate Can be used as a cheap input device that has 16 different buttons Can be combined with a velcro’d tracker Disadvantages: Requires training Will not be as fast as regular keyboard text entry

Frog pad

Software text entry On screen keyboards are often used for text entry. Basically make a rectangle with all of the different keys you want. Stick a ray on the end of the wand, put the ray into the letter you want. Press a button on the wand to select the letter.

TULIP n n Tulip menus / text entry Imagine wearing pinch gloves and having a floating keyboard in front of you in the world Reach out and pinch the letter in front of you 11 -12 WPM for expert users

Optical tracking Different way of tracking Put markers (little balls) on a set of glasses or a hat. The camera then recognizes them and generates a 3 D position based on the Info. Advantages: Cheaper than Intersense Can easily make any object trackable Disadvantages: Requires users to wear something Marker has to be visible to camera

Magnetic tracking n n Tracking through electromagnetic fields Doesn’t work well with metal objects in the environment Higher latency than with Intersense trackers More portable though

Acoustical tracking Low latency Not sensitive to metal in the environment Not cheap or portable (has to be calibrated for each environment. Very good for fixed setups such as CAVEs

Computer vision tracking n n n Through the use of markers and a standard webcam you can get 6 DOF tracking Software process Useful and cheap for passive haptics

Gyroscopic mice The gyroscopic mice provide 2 D in air movement and generally do not require drivers, just plug in and the OS / USB does the rest (even on Linux) Advantages: -Allows for a cheaper ($70) more restricted wand movement (only desktop X, Y) -Is possible to have multiple mice influencing the VE at the same time -Scenario (All data on front wall, multiple clickable options). Disadvantages: -Does not have absolute tracking -Has to be used with care, a flick of the wrist during a demo could be problematic

Pinch Gloves Pinch gloves allow for fingers to be pinched together to produce different combinations for input into the VE. Combine with a 6 DOF tracker to know where the users hands are. Somewhat cheap: $2, 000 device Advantage: More natural style of interaction than with a wand Disadvantage: Does not track finger movement

Data gloves n n n Allows for true finger positioning to be reported to the VE Can do sign language type gestures Has calibration issues More expensive Doesn’t always work right Very expensive

Shape tape n n A piece of material that can be shaped and have this shape reported to the computer Useful for art, engineering (measuring / curve placement) Can be used for interfaces when combined with trackers Also can be used to imitate snakes

Omni Directional Treadmills Imagine being able to walk around in an environment naturally and not have to worry about the running into the CAVE walls. Advantages: Natural movement in a VE Perfect for types of training Disadvantages: Expensive Noisy May be harder than regular walking and may not react well to running

Haptic devices n n Provide force feedback Sometimes in a pen form Sometimes a skeleton framework over a glove Often easier to use passive haptics (pit experiment)

Discussion on Input devices n n Would adding a different input device be useful for your application? Make for interesting demos Can even go so far as hang-gliders with fans / tilt recognition or walking in place harnesses / floor tiles that move around Questions / Comments

3 DIT n n n Interacting with VEs makes them not only more usable, but more useful Helps increase the sense of presence Many different types that provide very interesting abilities.

3 DIT Input n n n Require input devices to work Does not require a 3 D input device to do a 3 D task (mouse / virtual sphere) Input devices do enable / limit different types of input though

3 DIT Realism n n Natural versus Magic techniques Think of having a virtual cup of water on the table in front of you. n n Reach out and grab it Now imagine it being 15 feet in front of you Both are possible in 3 DITs Can be mixed to have some of both

3 D Selection and Manipulation n Tasks are mainly composed of selection, positioning, and rotation Selection is composed of indicating which object is desired, confirming that it is the object that is desired to be selected, and then providing feedback to the user. Can be done through a couple of different methods (ego / exo centric)

Different types of selection techniques n n n Simple infinite (or finite) ray to select an item Two handed pointing Flashlight / aperture addition to raycasting Flexible ray casting Image plane techniques Fishing reel technique

Selection and Manipulation n n Once you select an object, what do you normally want to do with it? Manipulate it. Think back on the water cup example Imagine being able to stick your hand out 50 feet and grab objects Take a hand tracker, and use a non-linear scaling function on the object it is attached to

World in Minature n n Imagine being in a room and then having a top down view that is slightly bigger than your hand. Combine with a tracked tablet and can be used with a wand to pick and move objects Can be modified to provide an overview of the world on the floor of a cave and then warping based on that Can be used to replicate maps and then travel based on that

Voodoo Dolls n n n Select an object in the world Create a miniature copy of the object in front of you Use tracked pinch gloves scale, rotate, and position the object

Travel techniques n n n Exploration, search, maneuvering Scaling movement from slow to fast to cover long distances Velocity Location guides Reset to start usually essential Users get lost in VEs all the time, plan for it

Different types of travel techniques n n n n Point and fly based travel Gaze directed FPS style Button versus joystick Camera in hand technique Maps to place self in world Set of guided points

System Control n n There are whole hosts of options that can be configured with VEs How do you do this inside of a VE? n n 2 D in 3 D (Ring / XWand / VEWL type interactions / Pen + Tablet) 3 D menus (Tulip, cone trees, Command control cube)

Different ways to combine techniques n n 3 DITs are often best used when combining with other 3 DITs Example HOMER n n Select an object with ray casting, then manipulate with Go. Allows for selecting a far away object and moving it even farther. Flavors n n Adding small nuances to improve usability of technique. I. e. having the ray in ray casting snap to the nearest object in the world. Adding scaling and scrolling to the WIM.

Implementation strategies n n n Where possible 3 DITs should be created as DIVERSE DSOs Plugs into different applications Static interfaces are your friend n Need to get a handle to the world (easier if the world is loaded as a DSO)

Implementation strategies n n Make sure they can be turned on and off Think about creating a master 3 DIT DSO that turns sub parts on and off Don’t be afraid to try different options You can only do so much with a simulator on the desktop, you need to use the real device in the CAVE to make sure that it works well

DGLMessage structure n n n Think of having the visualization component as only interacting through messages sent to it You need a protocol for 3 DITs and the Viz core to talk to each other One way is to have messages designed to be sent to a specific class to do something particular.

Structure of message usage User Action 3 DIT Message Center Delivers message Output shown to user Vis 1 -3 register with the Message Center at starting time Vis 2 Vis 3

Message wrapup n n Advantage – Works through shared memory so it can be used on cluster systems rather easily Disadvantages – Not meant for large amounts of data transfer, mainly just directions on what to do. Takes planning, usage of multiple inheritance, and has overhead in implementation. Can pay off in the long term though.