Creating more realistic hand movements in VR

How can you create a more realistic Virtual Reality experience? Well, it might help if your hand couldn’t move straight through solid objects like tables and walls. Phillip Phoelich and Óscar Losada set out to create a software solution to this problem in their thesis project from the Games programme at the IT University of Copenhagen.

Digital Design DepartmentEducationcomputer gamesITU thesis

Written October 26, 2017 9:52 AM by Vibeke Arildsen

Why did you decide to work with hand movements in VR?

These days, VR devices have controllers that track the position and rotation of your hands so that the movements of your avatar’s hand corresponds to your movements in the real world.

But the problem is that virtual objects don’t stop your virtual hand. For instance, if you move your hand into a virtual table, the hand will go right through it.

Finding a solution to this problem is quite difficult, so there is a good reason why many VR products work like this. We wanted to see if we could make a realistic VR hand that doesn’t enter objects, without losing the player’s experience of being present inside the virtual world.

Philip and Óscar used the VR game Job Simulator as an example in their thesis. The image shows the avatar’s hands going through a table in the game.

How did you approach the problem?

There are hardware solutions in existence– such as an actual glove that can stop your fingers from moving when you touch an object in the virtual world. But we wanted to create a software-based prototype that would be usable without people having to buy extra equipment.

This is a very important aspect for game companies, because if people have to buy extra hardware in order to play a game, you decrease its market potential.

How does your prototype work?

We took tracking data from the VR controller about the position of the physical hand, its rotation and finger placement. With this information, we could detect collisions between the virtual hand and virtual objects and calculate ways to allow the virtual hand to deviate from the physical hand when these collisions happened. Essentially, we made sure that the hand is ‘pushed back’ when these positions overlap, so it looks like the hand is just touching the object, not penetrating it.

When the prototype was built, we tested it on users and presented it to games companies at an industry event. We have gotten really positive feedback, even though the prototype was still unpolished.

As we were working on the project, we found out that many companies are actually working on implementing similar software in their VR products.

What would be the benefits of more realistic hand movements?

The VR experience works best when it is realistic and the avatar’s movements mirror those of the physical body. You simply feel more immersed in the VR experience if the rules of physics are respected. Imagine that you wanted to do a psychological or behavioural experiment in VR.

If the subjects find that things don’t work like they do in the real world, you can hardly expect them to behave normally. The same thing goes for VR gaming experience – the more realistic it is, the more fun it is for the player.

Further information

Vibeke Arildsen, Press Officer, phone 2555 0447, email viar@itu.dk