Identifying users of immersive virtual-reality serious games through machine-learning techniques

Abstract

User identification is currently an open issue in immersive Virtual Reality (iVR) environments. Three main goals are usually associated with the use of tracking-data and Machine-Learning (ML) techniques: safeguarding privacy, user authentication, and user-experience customization. However, research to date has only involved very limited recordings of user data (e.g., on a single session and for low-interactive situations), rare in real iVR environments. So, the research gap between real iVR data and ML techniques for user identification is addressed in this paper. To do so, a 3-session iVR experience of operating a bridge crane is considered. In this simple yet highly interactive learning action, the dataset records of user performance show rapid changes between one experience and another. Eye, head, and hand movements of 64 users of similar age and with comparable previous experience were all recorded while engaged with the experience. The final raw dataset had a size of approximately 50M data points with 25 attributes that were mainly temporal series values. Secondly, different ML algorithms were used for user identification: Decision Tree, Random Forest, XGBoost, k-Nearest Neighbors, Support Vector Machines, and Multilayer Perceptron. The results showed that ML ensemble learning techniques, particularly Random Forest, were the most suitable solutions on the basis of different measures for the prediction of user identity. Additionally, the inclusion of stress and no-stress conditions significantly enhanced model performance, highlighting the importance of data diversity. Temporal segmentation revealed that user identification during later phases of the exercise was slightly more effective, due to increased individual variability. Finally, a minimum duration of the iVR experience was identified as a requirement to assure high identification rates.