Sony Patents System To Enhance Players' Confidence Using AR

Rundown:

Sony Interactive Entertainment has recently published a patent for a system to enhance the way players perceive themselves in augment reality (AR) environments by modifying their physical features in real-time.
The system analyses the field of view from the augmented reality (AR) headset to detect images of the player’s physical features and modify them by creating a virtual mesh that changes in size and shape relative to the corresponding physical features.
The system could also capture one or more measurements of the player during interaction with the augmented reality (AR) scene and adjust the intensity of the augmented reality (AR) scene based on the measurements.
With this system, the company intends to improve players’ self-perception and confidence in their appearance and encourage them to overcome certain challenges or obstacles during gameplay.

Sony Interactive Entertainment was recently awarded a patent titled “AUGMENTED REALITY ARTIFICIAL INTELLIGENCE ENHANCE WAYS USER PERCEIVE THEMSELVES,” filed in July 2022 under SONY INTERACTIVE ENTERTAINMENT INC. [JP]/[JP]. The patent, published earlier this week, describes a system that uses augmented reality (AR) to enhance the way players perceive themselves in a virtual environment by modifying their physical features, such as hands, arms, shoulders, height, and weight, in real-time to make them appear stronger, taller, faster, or healthier when viewed through an augmented reality (AR) headset. With this system, Sony Interactive Entertainment intends to improve the player’s self-perception and confidence in their appearance and encourage them to “overcome challenging tasks or obstacles during the gameplay.”

“Methods and systems are provided for generating augmented reality (AR) scenes where the AR scenes can be adjusted to modify at least part of an image of the physical features of a user to produce a virtual mesh of the physical features. The method includes generating an augmented reality (AR) scene for rendering on a display for a user wearing AR glasses, the AR scene includes a real-world space and virtual objects overlaid in the real-world space,” reads the abstract for the patent. “The method includes analyzing a field of view into the AR scene from the AR glasses; the analyzing is configured to detect images of physical features of the user when the field of view is directed toward at least part of said physical features of the user. The method includes adjusting the AR scene, in substantial real-time, to modify at least part of the images of the physical features of the user when the physical features of the user are detected to be in the AR scene as viewed from the field of view of the AR glasses, wherein said modifying includes detecting depth data and original texture data from said physical features to produce a virtual mesh of said physical features; the virtual mesh is changed in size and shape and rendered using modified texture data that blends with said original texture data.”

It further explains, “In one embodiment, the modified physical features of the user appear to the user when viewed via the AR glasses as existing in the real- world space. In this way, when the physical features of a user are detected to be in the AR scene, the physical features are augment in the AR scene which can result in the self-perception of the user improving which in turn can provide the user with confidence to overcome challenging tasks or obstacles during the gameplay of the user.” As the patent elaborates, the purpose behind this system is to encourage players in an augmented reality (AR) environment to continue with their gameplay if they encounter particular challenges or obstacles that may discourage them by enhancing their self-perception through augmented modification of their physical features within the augmented reality (AR) environment. Hence, when the players perceive themselves as stronger or faster, they’re likely to feel more confident in overcoming the challenges or obstacles.

Sony — Embodiment of a user wearing AR glasses while operating interface objects to interact with an AR scene and an adjusted AR scene that is generated in substantial real-time to include a modified image of the body of the user, in accordance with implementations of the disclosure. | Source: PATENTSCOPE

According to the claims made by the patent, the system generates an augmented reality (AR) scene, which includes “a real-world space and virtual objects overlaid in the real-world space,” that is rendered on display for a player wearing an augmented reality (AR) headset. The system analyses the field of view from the augmented reality (AR) headset to detect images of the player’s physical features and adjusts the augmented reality (AR) scene in real-time to modify the images of the physical features by creating a virtual mesh of said physical features, which is changed in size and shape relative to the corresponding physical features, as preferred by the player, and rendered using modified texture data that blends with the original texture data. Likewise, the modified physical features of the player would appear to them as existing in real-world space. The virtual mesh, in this case, refers to a three-dimensional representation of the player’s physical features used for rendering in the augmented reality (AR) scene, representing at least part of the player’s avatar.

Additionally, the system could also detect images of the physical features of other players and modify their images, too. “The computer-implemented method of claim 1 , wherein the analyzing is configured to further detect images of physical features of one or more additional users when the field of view of the user is directed toward at least part of said physical features of the one or more additional users to modify the images of physical features of the one or more additional users,” explains the patent. The patent also claims that the system could capture one or more measurements of the player during interaction with the augmented reality (AR) scene and adjust the intensity of the augmented reality (AR) scene based on the measurements. “The computer-implemented method of claim 7, wherein the adjustment to the depicted intensity of the AR scene increases or decreases a perceived difficulty associated with the AR scene by the user,” it further claims.

Furthermore, the system may also be able to use sensors to monitor the player’s heart rate, blood pressure, oxygen saturation, respiration, body temperature, eye gaze, facial expression, voice, controller movement, body movement, controller input, or a combination of two or more of these, which can be adjusted to produce virtual measurements that correspond to the player’s avatar and give them a perceived greater athletic endurance. Hence, the perceived difficulty of the augmented reality (AR) scene may increase or decrease, but the player’s physical intensity would remain the same. In addition to this, the system may also capture sounds produced by the player during the interaction with the augmented reality (AR) scene and adjust it based on pitch, tone, volume, intensity, speed, or a combination of two or more of these to produce a virtual sound that represents the sound produced by the player’s avatar.

The patent application from Sony Interactive Entertainment states that the system can be used in various industries, such as video games, shopping and retail, education, entertainment, healthcare, and virtual assistance. It’s important to note, however, that a patent application doesn’t mean certainty. While it remains to be seen exactly how (and if) Sony Interactive Entertainment intends to implement the system, it has the potential to revolutionise the way players interact with augmented reality (AR) environments. By allowing players to perceive themselves in a more positive light, it could make augmented reality (AR) experiences more pleasurable and accessible to a wider audience.

What do you think about this? Do tell us your opinions in the comments below!