Goal: To bring together professionals from various fields in academia, military, and industry to work on identifying research areas that will help produce the next generation of personalized and adaptive VR/AR systems and content. The meeting focuses on two areas, described in detail below: Perceptual systems: How visual systems, auditory systems, and VR/AR content can adapt to optimize an individual’s experience; and, Social, behavioral and cognitive patterns: How VR/AR systems can assess an individual’s knowledge and adapt to ease interaction, navigation, locomotion, and learning; as well as how VR/AR systems can adapt to and influence individuals’ social behavior.
1.Perceptual Systems: This research theme investigates how adaptive VR/AR systems may accommodate a greater range of individuals and how the interplay of adaptive display and audio systems may play a role in mitigating discomfort or enhancing the perception of presence
A. Traditional visual display systems have been designed using the average statistics of the human visual system. VR and AR head-mounted display systems on the other hand are uniquely personal devices whose optimal functioning requires adaptation to each user because of the significant effect of factors such as individual IPD, acuity, motion sensitivity, and prior model. The objective under this area will be to identify the hard challenges that need to be addressed to realize adaptive visual displays.
B. Likewise, a range of factors influence human auditory perception. Adaptive audio systems may be designed to better accommodate individual psychoacoustic response. Spatial audio perception is strongly influenced by the shape of our ears as well as the ear canal resonances that are individual to every human. Adaptive systems that extract and apply physical characteristics such as Head Related Transfer Functions (HRTFs) to audio signals and that are trained on individual differences in perception (including hearing disorders) may enhance aural discrimination, localization, and spatialization. They may also mitigate discomfort or enhance the perception of presence. The objective under this area will be to identify the hard challenges that need to be addressed to realize adaptive auditory systems.
C. Traditional content design either delivers a “one size fits all” solution, or enables limited user selection of content or UI. Using real-time sensors or feedback mechanisms, content designers can create VR experiences that are automatically personalized and adaptive. Such systems can dynamically adjust the emotional or psychological saliency of content. Adaptive VR experiences that consider physiological measures can also be used to reduce the onset of cybersickness or increase presence based on real-time data. The objective under this area will be to identify the hard challenges that need to be addressed to realize adaptive VR/AR content.
2. Social, behavioral and cognitive patterns: This research theme investigates how VR/AR systems can assess an individual’s knowledge and adapt to personalize interaction, navigation, locomotion, and learning; as well as how VR/AR systems can adapt to and influence a user’s social behavior.
A. Spatial Cognition: How do cognitive models of 3D space influence user experience of augmented and virtual environments? How can we assess wayfinding strategies and ability in virtual and augmented worlds, and develop profiles that better match user experience? How can we better understand the impact of experiences in virtual and augmented space on the real-world perception of physical spaces? What strategies best scaffold virtual and augmented training curricula for real-world spatial tasks? How does distance estimation affect transfer of skills or knowledge in gained virtual environments to real world experience? What effect do perceptual tricks (e.g., redirected walking or redirected haptics) have on virtual world spatial awareness and navigation and transfer to the real world? The objective under this area is to better understand how to personalize VR/AR experiences to match the diversity of strategies and skills that individuals use to build spatial and navigational awareness.
B. Social Cognition: Current virtual environments rely on developers, users, and groups to structure and mediate social interactions. For example, users can customize avatars to represent themselves or rely on reputation systems to establish norms, and ban or block undesired behaviors. How can adaptive systems support these functions by gauging an individual’s social intent and behaviors to deliver experiences that are more inclusive, safe, and comfortable? Will the uncanny valley, driven by atypical features on humanlike characters, disrupt social cognition, particularly for children? What role will eye tracking technologies play in supporting social cognition? How might individuals with ASD or behavior disorders benefit from such adaptive, personalized social applications? How might personalized and adaptive VR/AR social systems improve the transfer of pro-social behaviors and skills to a user’s real world social interactions? The objective under this area is to better understand how to personalize VR/AR experiences to support healthy social interaction.