Author
Divya Udayan J
Keywords
Hand Gesture; Leap Motion Controller; VR Environment
Abstract
Recent development in virtual reality (VR) interaction with 3D camera and sensors like kinect, range camera, leap motion controller etc., has enabled opportunity in development of human computer interaction (HCI) application. Hand gesture is one of the popular ways that people use to interact with the computer. Even automatic hand gesture recognition appears as a suitable means for interacting with virtual reality systems. This paper focuses on the study and analysis of the application based on gesture interaction technology in virtual reality. Customizing gestures for pointing, grabbing, zoom in/out, swap were defined and implemented in unity 3D with leap motion SDK. The effectiveness of the hand gesture was analyzed through recording user experience and questionnaire.
References
[1] Jocelyn S McGee, Cheryl Van der Zaag, J Galen Buckwalter, Marcus Thiébaux, Andre Van Rooyen, Ulrich Neumann, D Sisemore and Albert A Rizzo.: Issues for the assessment of visuo-spatial skills in older adults using virtual environment technology. CyberPsychology& Behavior, vol. 3, no. 3, pp 469–482, (2000).
[2] FDavidRose,BarbaraMBrooksandAlbertARizzo.:Virtualrealityinbrain damage rehabilitation: review. CyberPsychology & Behavior, vol. 8, no. 3, pp. 241–262, (2005).
[3] Patrice L Weiss, Rachel Kizony, Uri Feintuch and Noomi Katz.: Virtual reality in neurorehabilitation. Textbook of neural repair and neurorehabilitation, vol. 2, pp. 182–197,( 2006).
[4] H. Joo, T. Simon, and Y. Sheikh: Total capture: A 3d deformation model for tracking faces, hands, and bodies, in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), ser. CVPR ’18, vol. abs/1801.01615. Washington, DC, USA: IEEE Computer Society, pp. 8320–8329, (2018).
[5] M. Kim, J. Lee, C. Jeon, and J. Kim: A study on interaction of gaze pointer-based user interface in mobile virtual reality environment, Symmetry, vol.9, no.9, pp.189, (2017).
[6] S. Marwecki, M. Brehm, L. Wagner, L.-P. Cheng, F. F. Mueller, and P. Baudisch: Virtual space – over loading physical space with multiple virtual reality users, in Proceedings of CHI Conference on Human Factors in Computing Systems, ser. CHI ’18. New York, NY, (2018).
[7] Leap Motion Homepage,https://www.leapmotion.com/.
[8] P.LindemannandG.Rigoll : A diminished reality simulation for driver-car interaction with transparent cockpits, in IEEE Virtual Reality (VR). IEEE, pp. 305–30618-22, (2017).
[9] T. Pfeiffer, Understanding Multimodal Deixis with Gaze and Gesture in Conversational Interfaces. Aachen, Germany: Shaker Verlag GmbH, (2011).
[10] T. Pfeiffer : Using virtual reality technology in linguistic research, IEEE Virtual Reality Workshops (VRW). IEEE, pp. 83–84,(2012).
[11] Oprea, S., Martinez-Gonzalez, P., Garcia-Garcia, A., Castro-Vargas, J. A., Orts-Escolano, S., & Garcia-Rodriguez, J.: A Visually Plausible Grasping System for Object Manipulation and Interaction in Virtual Reality Environments. arXiv preprint arXiv:1903.05238.
[12] A. Henrysson, M. Billinghurst, and M. Ollila: Virtual object manipulation using a mobile phone, in Proceedings of the International Conference on Augmented Tele existence, ser. ICAT ’05. New York, NY, USA pp. 164–171,(2005).
[13] S. Han and J. Kim: A study on immersion of hand interaction for mobile platform virtual reality contents, Symmetry, vol. 9, no. 2, p. 22, (2017).
[14] W. Zhao, J. Chai, and Y.-Q. Xu: Combining markerbased mocap and rgb-d camera for acquiring high-fidelity hand motion data, in Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, ser. SCA ’12. Aire-la-Ville, Switzerland, pp. 33–42, (2012).
[15] M. Kim, C. Jeon, and J. Kim :A study on immersion and presence of a portable hand haptic system for immersive virtual reality,” Sensors, vol. 17, no. 5, p. 1141, (2017).
[2] FDavidRose,BarbaraMBrooksandAlbertARizzo.:Virtualrealityinbrain damage rehabilitation: review. CyberPsychology & Behavior, vol. 8, no. 3, pp. 241–262, (2005).
[3] Patrice L Weiss, Rachel Kizony, Uri Feintuch and Noomi Katz.: Virtual reality in neurorehabilitation. Textbook of neural repair and neurorehabilitation, vol. 2, pp. 182–197,( 2006).
[4] H. Joo, T. Simon, and Y. Sheikh: Total capture: A 3d deformation model for tracking faces, hands, and bodies, in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), ser. CVPR ’18, vol. abs/1801.01615. Washington, DC, USA: IEEE Computer Society, pp. 8320–8329, (2018).
[5] M. Kim, J. Lee, C. Jeon, and J. Kim: A study on interaction of gaze pointer-based user interface in mobile virtual reality environment, Symmetry, vol.9, no.9, pp.189, (2017).
[6] S. Marwecki, M. Brehm, L. Wagner, L.-P. Cheng, F. F. Mueller, and P. Baudisch: Virtual space – over loading physical space with multiple virtual reality users, in Proceedings of CHI Conference on Human Factors in Computing Systems, ser. CHI ’18. New York, NY, (2018).
[7] Leap Motion Homepage,https://www.leapmotion.com/.
[8] P.LindemannandG.Rigoll : A diminished reality simulation for driver-car interaction with transparent cockpits, in IEEE Virtual Reality (VR). IEEE, pp. 305–30618-22, (2017).
[9] T. Pfeiffer, Understanding Multimodal Deixis with Gaze and Gesture in Conversational Interfaces. Aachen, Germany: Shaker Verlag GmbH, (2011).
[10] T. Pfeiffer : Using virtual reality technology in linguistic research, IEEE Virtual Reality Workshops (VRW). IEEE, pp. 83–84,(2012).
[11] Oprea, S., Martinez-Gonzalez, P., Garcia-Garcia, A., Castro-Vargas, J. A., Orts-Escolano, S., & Garcia-Rodriguez, J.: A Visually Plausible Grasping System for Object Manipulation and Interaction in Virtual Reality Environments. arXiv preprint arXiv:1903.05238.
[12] A. Henrysson, M. Billinghurst, and M. Ollila: Virtual object manipulation using a mobile phone, in Proceedings of the International Conference on Augmented Tele existence, ser. ICAT ’05. New York, NY, USA pp. 164–171,(2005).
[13] S. Han and J. Kim: A study on immersion of hand interaction for mobile platform virtual reality contents, Symmetry, vol. 9, no. 2, p. 22, (2017).
[14] W. Zhao, J. Chai, and Y.-Q. Xu: Combining markerbased mocap and rgb-d camera for acquiring high-fidelity hand motion data, in Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation, ser. SCA ’12. Aire-la-Ville, Switzerland, pp. 33–42, (2012).
[15] M. Kim, C. Jeon, and J. Kim :A study on immersion and presence of a portable hand haptic system for immersive virtual reality,” Sensors, vol. 17, no. 5, p. 1141, (2017).
Received : 06 February 2020
Accepted : 20 May 2020
Published : 04 June 2020
DOI: 10.30726/esij/v7.i2.2020.72011