HCI Programing in a Professional Environment with Unity

HCI Programing in a Professional Environment with Unity 3 D Yingcai Xiao

Under the Hood • Key components of HCI programming. • How a professional programming environment can support HCI programming. We use Unity 3 D as an example. • How to program for HCI in a professional IDE.

HCI Programming Essentials EDP: Event Driving Programming 1. 2. 3. 4. 5. 6. 7. 8. Event generators (input devices) Events Event loop Event mapping (compile time) Event dispatching (run time) Event handlers Computation Output

HCI Programming with Unity 1. Event generators (input devices) Unity takes input from almost all input devices: CLI: keyboard GUI: mouse, touch screen NUI: Kinect, Leap. Motion, Smart Phones

HCI Programming with Unity 2. Events 2. A. Virtual Events: Horizontal/Vertical Movements 2. B. Software Events: Collision 2. C. Idle Event: for background activities 2. D. User defined events: at any key points of computation.

HCI Programming in Unity 3. Event loop : Unity takes care of it. 3. A. At coding time: visual programming. Unity IDE takes all events. 3. B. At application run time: Application takes all events. API libraries built into the application contains the event loop. 3. C. When debugging, Unity IDE shares events with the application and the debugger. 3 -way splits.

HCI Programming in Unity 4. Event mapping (compile time) 4. A. At coding time: visual programming 4. B. At application run time: can be done, but no API yet. 4. C. At debugging time, unity IDE maps and distributes events accordingly at runtime.

HCI Programming in Unity 5. Event dispatching (run time) At application run time: done by unity built-in runtime libraries. Visual Programming: Events distributed by the Unity IDE to A. IDE components, when coding; B. IDE and application, when simulating (play); C. the IDE, the application and the debugger when in debugging mode.

HCI Programming in Unity 6. Event handlers 6. A. Written by the application programmer. 6. B. Built-in by the Game Engine (Physics Engine, AI Engine, …) 6. C. Imported as part of the prefabricates.

HCI Programming in Unity 7. Computation 7. A. Written by the application programmer. 7. B. Built-in by the Game Engine (Physics Engine, AI Engine, …) 7. C. Imported as part of prefabricates.

HCI Programming in Unity 8. Output 8. A. Audio: sound bytes 8. B. Video: animated graphics 8. C. Haptic: force feedback

HCI Programming in Unity 8. B. Graphics Output: all taken care by Unity IDE at coding and debugging time and by unity libraries at application runtime. 8. B. 1. Local: Open. GL 8. B. 2. Remote: Web. GL 8. B. 3. Others: XR: VR/AR/MR

HCI Programming Essentials EDP: Event Driving Programming 1. 2. 3. 4. 5. 6. 7. 8. Event generators (input devices) Events Event loop Event mapping (compile time) Event dispatching (run time) Event handlers Computation Output

HCI Programming Essentials EDP: Event Driving Programming Project: directory and files for a specific game project. C: UsersxiaoDocumentsNew Unity Project 1 Assets (anything you can reuse) Library (binary files)

Unity 3 D IDE: Integrated Development Environment Project: directory and files for a specific game project. C: UsersxiaoDocumentsNew Unity Project 1 Assets (anything you can reuse) Library (binary files)

Unity 3 D: Assets C: UsersxiaoDocumentsNew Unity Project 1Assets (anything you can reuse) Standard Assets Open. NI Scripts _Scenes Materials Artwork

Unity 3 D: Standard Assets C: UsersxiaoDocumentsNew Unity Project 1AssetsStandard Assets Objects: (Look) Tree Terrain Charater Lights: (Look) Light Flares Light Cookies Code: (Feel: control, interaction, animation, …) Scripts

Unity 3 D: Objects C: UsersxiaoDocumentsNew Unity Project 1AssetsStandard AssetsCharater: Prefab: (Predefined Objects) First Person, 3 rd Person Source: Prototype (Look) Constructor. FBX Materials (properties) Textures (images) Scripts (Feel: actions) Java Scripts: Third. Person. Controller. js C#: Mouse. Look. cs

Unity 3 D: Scripts Languages: Interpreted : Java Script Compiled: C# Usages: General: under ProjectScripts Exit. On. Escape. cs Objects: attached to objects Third. Person. Controller. js

Unity 3 D: Library cashe: for speeding up processing metadata: data that describes data previews: for previewing scenes Script. Assemblies: compiled object assemblies for scripts