Gesture Recognition in Virtual reality
DOI:
https://doi.org/10.59388/pm00336Keywords:
gesture recognition, machine leaning algorithms, sensor technologies, VRAbstract
Virtual Reality (VR) provides users with a sensory experience that is close to reality, creating a sense of interaction. It is widely used, and the gesture recognition in VR also has a great effect. Gesture recognition enriches VR using experience and promotes a more direct and natural interaction. Gesture recognition usually employs sensors to collect data from users and machine leaning algorithms to interpret and respond to human activities. Complex gestures need more complex algorithms and more rigorous operations. The reason is that complex gestures mean larger quantity of data. If data is larger, the harder to get robust and effective datasets. Then, features can also become difficult to extract, contributing to misrecognition or unrecognizable. Though machine leaning algorithms are widely used in gesture recognition, there are still some important challenges need to be addressed, like lack of standardization and limitations of availability of diverse and large datasets. However, VR, gesture recognition and machine leaning algorithms all have excellent prospect, because they are in line with the development of the Times and show the progress of science and technology. This paper not only focuses on their advantages but also does not ignore their shortcomings, and looks at them comprehensively.
References
Angad, B., Khaldoun, A. M. M., Hussam, A. H., Shayan, Q., & Patrick, O. (2023). The Integration of Artificial Intelligence Into Patient Care: A Case of Atrial Fibrillation Caught by a Smartwatch. Cureus, 15(3), e35941-e35941. https://doi.org/10.7759/cureus.35941
Berk, C., Bahadir, A. M., Tolga, K. C., & Ufuk, C. (2024). Gaze-directed and saliency-guided approaches of stereo camera control in interactive virtual reality. Computers & Graphics, 118, 23-32. https://doi.org/10.1016/j.cag.2023.10.012
Brian, C., Johnathan, H., Chris, H., Anthony, C., Amir, I., Emmanuel, B., Jo, S. A., Taylor, S., Wayne, S., Michael, T., Michael, B., & Ivona, C. (2024). Assessing the utility of high spectral resolution lidar for measuring particulate backscatter in the ocean and evaluating satellite ocean color retrievals. Remote Sensing of Environment, 300. https://doi.org/10.1016/J.RSE.2023.113898
Caglar, Y. (2023). Point and Select: Effects of Multimodal Feedback on Text Entry Performance in Virtual Reality. International Journal of Human–Computer Interaction, 39(19), 3815-3829. https://doi.org/10.1080/10447318.2022.2107330
Cantone, A. A., Esposito, M., Perillo, F. P., Romano, M., Sebillo, M., & Vitiello, G. (2023). Enhancing Elderly Health Monitoring: Achieving Autonomous and Secure Living through the Integration of Artificial Intelligence, Autonomous Robots, and Sensors. Electronics, 12(18). https://doi.org/10.3390/ELECTRONICS12183918
Caon, A. J., Sícoli, S. J. C., Vanessa, L. G., Gomes, d. S. E., Guilherme, F. L., Elena, d. S. S. E., Mendes, d. O. I., & Cipresso, P. P. H. (2023). High-resolution optical remote sensing geomorphological mapping of coral reef: Supporting conservation and management of marine protected áreas. Journal of Sea Research, 196. https://doi.org/10.1016/J.SEARES.2023.102453
Constanza, M., Fernando, A., Ignacio, V., & Julián, G. (2020). Developing an Innovative Medical Training Simulation Device for Peripheral Venous Access: A User-Centered Design Approach. Healthcare (Basel, Switzerland), 8(4). https://doi.org/10.3390/HEALTHCARE8040420
Cun-jiang, Y., Guo-bao, Z., Cheng-wei, Y., xiao-ying, D., & Cheng-shuo, L. (2022). An Improved Gesture Recognition Model Based on Mini-Xception. Journal of Physics: Conference Series, 2400(1). https://doi.org/10.1088/1742-6596/2400/1/012020
D.A.Sanaguano-Moreno, J.F.Lucio-Naranjo, R.A.Tenenbaum, & G.B.Sampaio-Regattieri. (2024). Real-time impulse response: a methodology based on Machine Learning approaches for a rapid impulse response generation for real-time Acoustic Virtual Reality systems. Intelligent Systems with Applications, 21. https://doi.org/10.1016/J.ISWA.2023.200306
Dongyun, K., Hoyoung, Y., & Youngjoo, L. (2021). Ultralow-Latency Successive Cancellation Polar Decoding Architecture Using Tree-Level Parallelism. IEEE TRANSACTIONS ON VERY LARGE SCALE INTEGRATION (VLSI) SYSTEMS, 29(6), 1083-1094. https://doi.org/10.1109/TVLSI.2021.3068965
Elster, L., Staab, J. P., & Peters, S. (2023). Making Automotive Radar Sensor Validation Measurements Comparable. Applied Sciences, 13(20). https://doi.org/10.3390/APP132011405
Eva, C., & Helder, A. (2022). An Experimental Assessment of Depth Estimation in Transparent and Translucent Scenes for Intel RealSense D415, SR305 and L515. Sensors, 22(19), 7378-7378. https://doi.org/10.3390/s22197378
Fan, Z., Manman, P., Yuanyuan, S., & Qiang, W. (2024). Hierarchical features extraction and data reorganization for code search. The Journal of Systems & Software, 208. https://doi.org/10.1016/J.JSS.2023.111896
Guoliang, C., & Lin, W. (2022). A Novel Automatic Tracking Method of Moving Image Sequence Marker Points Uses Kinect and Wireless Network Technology. Wireless Communications and Mobile Computing, 2022. https://doi.org/10.1155/2022/7033711
Hongjiao, G., & Fan, L. (2024). The application of virtual reality technology in the teaching of clarinet music art under the mobile wireless network learning environment. Entertainment Computing, 49. https://doi.org/10.1016/J.ENTCOM.2023.100619
Huashi, Z., Zhichao, W., Yubin, H., Qiujia, F., Shouyu, L., Guang, M., Wenchao, L., & Qun, Y. (2023). Combination optimization method of grid sections based on deep reinforcement learning with accelerated convergence speed. Frontiers in Energy Research, 11. https://doi.org/10.3389/FENRG.2023.1269854
HyungIl, K., Boram, Y., Young, O. S., & Woontack, W. (2023). Visualizing Hand Force With Wearable Muscle Sensing for Enhanced Mixed Reality Remote Collaboration. IEEE transactions on visualization and computer graphics, PP. https://doi.org/10.1109/TVCG.2023.3320210
Jella, P. (2023). From E-Commerce to Virtual Commerce: The Exciting Opportunities of Virtual Shopping. NIM Marketing Intelligence Review, 15(2), 10-17. https://doi.org/10.2478/nimmir-2023-0011
Jestine, P., Yeling, J., & Mesut, A. (2023). Virtual reality technology for workplace training: The case for developing cultural intelligence. International Journal of Cross Cultural Management, 23(3), 557-583. https://doi.org/10.1177/14705958231208297
Jing, Q., Yinuo, Z., Weizhong, T., Wenming, C., Yu, Z., & Lingguo, B. (2023). Developing a virtual reality healthcare product based on data-driven concepts: A case study. Advanced Engineering Informatics, 57. https://doi.org/10.1016/J.AEI.2023.102118
Jonathan, W. K. (2023). Distance Perception in Virtual Reality: A Meta-Analysis of the Effect of Head-Mounted Display Characteristics. IEEE transactions on visualization and computer graphics, 29(12), 4978-4989. https://doi.org/10.1109/TVCG.2022.3196606
Josefine, S., Tonia, M., Danny, S., Maximilian, T., Jazan, O., Maciej, P., & Christian, H. (2022). A multimodal user interface for touchless control of robotic ultrasound. International journal of computer assisted radiology and surgery, 18(8), 1429-1436. https://doi.org/10.1007/s11548-022-02810-0
Jung-Wook, H., Jeongkyun, P., Rae-Hong, P., & Hyung-Min, P. (2023). Audio-visual speech recognition based on joint training with audio-visual speech enhancement for robust speech recognition. Applied Acoustics, 211. https://doi.org/10.1016/J.APACOUST.2023.109478
Kamil, K., Thomas, C., & Marek, S. (2023). Visual programming simulator for producing realistic labeled point clouds from digital infrastructure models. Automation in Construction, 156. https://doi.org/10.1016/J.AUTCON.2023.105126
Liang, L. (2023). Motion detail feature extraction of sports injury based on three-dimensional sensor tracking. Physical Communication, 61. https://doi.org/10.1016/J.PHYCOM.2023.102210
Lingjun, Z., Bo, L., Lijie, D., & Guanghui, G. (2023). Tough, recyclable and biocompatible carrageenan-modified polyvinyl alcohol ionic hydrogel with physical cross-linked for multimodal sensing. International Journal of Biological Macromolecules, 253(P4). https://doi.org/10.1016/J.IJBIOMAC.2023.126954
Lurdes, C. M. d., & Soumodip, S. (2024). A systematic review of virtual reality in tourism and hospitality: The known and the paths to follow. International Journal of Hospitality Management, 116. https://doi.org/10.1016/J.IJHM.2023.103623
Mahajan, R., & Padha, D. (2018). A Review On Various Techniques To Recognize Gesture Based Facial Expressions. Research Cell: An International Journal of Engineering Sciences, 27(1), 78-82.
Marc, L. W., K., M. J., & Madhumita, D. (2024). Augmented and virtual reality in hotels: Impact on tourist satisfaction and intention to stay and return. International Journal of Hospitality Management, 116. https://doi.org/10.1016/J.IJHM.2023.103631
Nikola, H., Tomislav, M., Ivan, U., & Stanko, Š. (2024). Use it early: The effect of immersion on spatial and design space aspects in team-based mechanical design reviews. Advanced Engineering Informatics, 59. https://doi.org/10.1016/J.AEI.2023.102270
Panella, M., & Altilio, R. (2019). A Smartphone-Based Application Using Machine Learning for Gesture Recognition: Using Feature Extraction and Template Matching via Hu Image Moments to Recognize Gestures. IEEE Consumer Electronics Magazine, 8(1), 25-29. https://doi.org/10.1109/MCE.2018.2868109
Patricia, L., Diego, I., Alejandro, M., Fernando, O. J., Pilar, P. E., & Rocío, B. (2022). Machine Learning-Based Processing of Multispectral and RGB UAV Imagery for the Multitemporal Monitoring of Vineyard Water Status. Agronomy, 12(9), 2122-2122. https://doi.org/10.3390/agronomy12092122
Patrick, S. (2023). Unbalanced visual cues do not affect search precision at the nest in desert ants (Cataglyphis nodus). Learning & behavior. https://doi.org/10.3758/S13420-023-00613-0
Perry, A. . B. S. (2023). Book Review: Rituals for Virtual Meetings: Creative Ways to Engage People and Strengthen Relationships. Adult Learning, 34(4), 260-261. https://doi.org/10.1177/10451595221099569
Proffitt, R., Ma, M., & Skubic, M. (2023). Development and Testing of a Daily Activity Recognition System for Post-Stroke Rehabilitation. Sensors, 23(18). https://doi.org/10.3390/S23187872
Salihbegovic, F. (2020). The Encounter with the Real: What Can Complicite’s Theatre Performance 'The Encounter' Teach Us about the Future of VR Narratives? Body, Space & Technology, 19(1), 125-152. https://doi.org/10.16995/bst.336
Seda, A., Xiaolong, L., Qiyuan, W., Paige, M., YueHin, L., Jed, J., Joey, H., Laura, O., Narutoshi, H., & Axel, K. (2023). Virtual Planning and Patient-Specific Graft Design for Aortic Repairs. Cardiovascular engineering and technology. https://doi.org/10.1007/S13239-023-00701-2
Serena, S., Marco, A., Milena, M., & Marco, R. G. (2023). Experimental validation and uncertainty analysis of an innovative IoT infrared sensor for wall thermal transmittance measurement. Measurement Science and Technology, 34(12). https://doi.org/10.1088/1361-6501/ACF064
Shengcai, D., Le, W., Aiping, L., & Xun, C. (2023). Alignment-Enhanced Interactive Fusion Model for Complete and Incomplete Multimodal Hand Gesture Recognition. IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society, PP. https://doi.org/10.1109/TNSRE.2023.3335101
Shinji, S., Miho, A., Toru, S., Yuta, T., Jun, H., Taishi, W., Yuki, M., Sukchan, K., Saeri, K., Yuki, S., Kensuke, S., Yoshiaki, A., & Shigenori, K. (2023). Evaluation of electrical activity at the carpal tunnel area in response to median nerve elbow stimulation using magnetoneurography. Clinical Neurophysiology, 156, 89-97. https://doi.org/10.1016/j.clinph.2023.09.016
Shui’er, H., Sujin, K., & Hyun, J. J. (2023). The effect of visual rivalry in peripheral head-mounted displays on mobility. Scientific Reports, 13(1), 20199-20199. https://doi.org/10.1038/s41598-023-47427-8
Stanley, M., Krisha, G. V., Christine, C., Matteo, Z., Rezia, M., & Damien, C. (2022). Improving Haptic Response for Contextual Human Robot Interaction. Sensors, 22(5), 2040-2040. https://doi.org/10.3390/s22052040
Takeshi, K., Xiao, M., Santhosh, S., Thije, B., Gerard, d. H., Bart, P., Razvan, P., Daniel, T., Roy, V., B., A. H., Moreno, H. L., Florian, d. R., Itai, L., Fujito, Y., J., J. R. A., A., M. E., Jisk, K. A., & H., G. G. (2023). A touchless user interface based on a near-infrared-sensitive transparent optical imager. Nature Electronics, 6(6), 451-461. https://doi.org/10.1038/s41928-023-00970-8
Tamanna, N., & Waqar, N. (2022). A research protocol of an observational study on efficacy of microsoft kinect azure in evaluation of static posture in normal healthy population. Journal of Datta Meghe Institute of Medical Sciences University, 17(1), 30-33. https://doi.org/10.4103/jdmimsu.jdmimsu_176_21
Tanmay, D. S. (2023). Augmented reality applications and the future library. Library Hi Tech News, 40(9), 7-11.
Tarkov, M. S., & Chiglintsev, E. A. (2012). Reducing the dimensionality of the data in the problem of diagnosing thyroid disease. Optical memory & neural networks, 21(2), 119-125. https://doi.org/10.1108/LHTN-07-2023-0129
Tianchen, X., Xiaohua, R., Jiale, Y., Bin, S., & Enhua, W. (2023). Efficient Binocular Rendering of Volumetric Density Fields With Coupled Adaptive Cube-Map Ray Marching for Virtual Reality. IEEE transactions on visualization and computer graphics, PP. https://doi.org/10.1109/TVCG.2023.3322416
Un, L. S., Jinwook, K., & Jeongmi, L. (2023). Effects of Reward Schedule and Avatar Visibility on Joint Agency during VR Collaboration. IEEE transactions on visualization and computer graphics, PP. https://doi.org/10.1109/TVCG.2023.3320221
Viet, L., & Luca, C. (2022). Practical Approach to Digitally Simulate Nonsynoptic Wind Velocity Profiles and Its Implications on the Response of Monopole Towers. Journal of Structural Engineering, 148(1). https://doi.org/10.1061/(ASCE)ST.1943-541X.0003228
Wang, J. (2021). A Review of Deep Learning on Medical Image Analysis. Mobile Networks & Applications, 26(1), 351-380. https://doi.org/10.1007/s11036-020-01672-7
Wang, S. (2015). Pathological Brain Detection by a Novel Image Feature—Fractional Fourier Entropy. Entropy, 17(12), 8278-8296. https://www.mdpi.com/1099-4300/17/12/7877
Wang, S. (2021). Advances in data preprocessing for biomedical data fusion: an overview of the methods, challenges, and prospects. Information Fusion, 76, 376-421. https://doi.org/10.1016/j.inffus.2021.07.001
Wang, S.-H. (2021a). COVID-19 classification by CCSHNet with deep fusion using transfer learning and discriminant correlation analysis. Information Fusion, 68, 131-148. https://doi.org/10.1016/j.inffus.2020.11.005
Wang, S.-H. (2021b). Covid-19 classification by FGCNet with deep feature fusion from graph convolutional network and convolutional neural network. Information Fusion, 67, 208-229. https://doi.org/10.1016/j.inffus.2020.10.004
William, E., Dominique, B., & Mohsen, Z. (2023). Will visual cues help alleviating motion sickness in automated cars? A review article. Ergonomics, 31-34.
Yongyi, L., Ghaffar, A., & Abdul, R. A. (2023). Advances in geothermal energy prospectivity mapping research based on machine learning in the age of big data. Sustainable Energy Technologies and Assessments, 60. https://doi.org/10.1016/J.SETA.2023.103550
Zhang, Y. (2015). Pathological brain detection in MRI scanning by wavelet packet Tsallis entropy and fuzzy support vector machine. SpringerPlus, 4(1), Article 716. http://www.springerplus.com/content/4/1/716
Zhang, Y. (2016). Smart detection on abnormal breasts in digital mammography based on contrast-limited adaptive histogram equalization and chaotic adaptive real-coded biogeography-based optimization. Simulation, 92(9), 873-885. https://doi.org/10.1177/0037549716667834
Zhang, Y. (2017). Pathological brain detection in MRI scanning via Hu moment invariants and machine learning. Journal of Experimental & Theoretical Artificial Intelligence, 29(2), 299-312. https://doi.org/10.1080/0952813X.2015.1132274
Zhang, Y.-D. (2016). Computer-aided diagnosis of abnormal breasts in mammogram images by weighted-type fractional Fourier transform [Article]. Advances in Mechanical Engineering, 8(2), Article 11. https://doi.org/10.1177/1687814016634243
Zhang, Y.-D. (2018). Twelve-layer deep convolutional neural network with stochastic pooling for tea category classification on GPU platform. Multimedia Tools and Applications, 77(17), 22821-22839. https://doi.org/10.1007/s11042-018-5765-3
Zhanming, C., Huawei, T., & Huiyue, W. (2023). User-Defined Foot Gestures for Eyes-Free Interaction in Smart Shower Rooms. International Journal of Human–Computer Interaction, 39(20), 4139-4161. https://doi.org/10.1080/10447318.2022.2109260
