Text Input In Virtual Reality

Overview

In this project, the team aimed to improve user experience in virtual reality by optimising learnability and efficiency using text input methods that are currently accessible to consumers. The team approached the issue by conducting research and developing s text input methods concurrently gathering feedback iteratively. Synthesising our findings from the first 2 designs, the team developed a final design based on the 2 designs.

Premise

The premise of the project lies within exploring text input in VR gaming. As a mode of communication, text input is easily the most reliable and basic kind. When we think of sending a message to another player, it is usually in text. the forms of text input may change but standard text inputs will still be around..

Persona

• Gamers
• Expert OS users
• Has to multitask (gaming and typing)
• Does not require full functionality of a keyboard

Persona

• Gamers
• Expert OS users
• Has to multitask (gaming and typing)
• Does not require full functionality of a keyboard

Use Cases

• Character naming/login
• Inputting values
• Chatting with other players

Measuring Success

The three sentences used for testing are as follows:
“Hello there!”
“Where can I meet you?”
“The quick brown fox jumps over the lazy dog.”

For the data collected from each of the keyboard, we calculated for 3 attempts per user:
• Mean time taken per letter
• Mean number of errors made per letter

Usability Concepts

Efficiency

      • Maximise typing efficiency in VR game
      • Unhindered VR experience
Measured by: Mean time taken per letter
Technique: Laboratory testing

The overall mean time taken per letter gave us a metric for comparing efficiency. Higher efficiency should mean that the user is able to type within a shorter time per letter.

Learnability

      • Easy to pick up
Measured by: Initial efficiency and rate of improvement
Technique: Laboratory testing & Interview

The overall mean errors per letter gave us a metric for comparing error rate. Lower error rates should mean that the user makes lesser errors per letter. 

Error Rate

      • Increase efficiency by reducing error
Measured by: Mean errors per letter
Technique: Laboratory testing

 The mean time taken per letter for the first attempt be a metric for comparing learnability. Being able to type efficiently on the first attempt should mean that the user is able to learn the system quickly. The rate of improvement over the three attempts is another variable that helps us compare learnability. The greater the improvement in typing speed over the three attempts, the faster the user is learning about the system. 

Baseline

For our baseline, we used SteamVR’s default VR keyboard. The reason why we chose a keyboard used in SteamVR is because Steam is one of the most, if not the most, popular VR gaming platforms currently. Many games are published on this platform and their support for the OpenVR framework allows more accessibility to games as it is not proprietary to any hardware.

This keyboard consists of two input methods:

1. Pointer

2. Touchpad

Analysis

Errors due to jitter
From the user gathered feedback as well as observations during laboratory tests, we realized that the jitter caused by the limitations of current VR technologies resulted in many unintentional errors. There were multiple occasions for both the Touchpad and Pointer keyboards where the user attempted to select a specific key, but failed to do so as the cursor was unstable.

This led us to consider the importance of mitigating this technical fault through several design considerations, such as having larger areas for each key, providing informative feedback of the current hand or finger position as well as easy reversal of action through easily accessible DELETE buttons.

Minimizing movements
The evaluation results of user’s efficiency revealed the user was slightly faster at typing with the Touchpad in comparison to the Pointer keyboard. A hypothesis for such results is that the user was able to type faster as there are smaller physical movements required when typing with the Touchpad. This led us to experiment with the concept of the T9 keyboard, where we proposed to maximize efficiency with a touchpad based keyboard.

Familiarity of interactions
The evaluation results also shows that the user produced less errors initially when typing with the Pointer keyboard as compared to the Touchpad keyboard. A possibility for this lower error rate could be that the user is more familiar with the pointing interaction, as most of the VR environment employs this form of input. This consistent form of interaction makes the Pointer keyboard more learnable for the user.

The error rate of the Touchpad keyboard falls over the next two tests, which shows that there is potential for the user to familiarize with this form of input over time.

Expectations of layout
During the test, we observed that many users struggled to find the punctuation marks desired. Although both SteamVR’s Touchpad and Pointer keyboard featured a QWERTY layout for the alphabets, the punctuation layout was different from that of a usual keyboard. For example, the position of a question mark in a standard keyboard is on the bottom right hand side, while the SteamVR keyboards placed it on the bottom left hand side instead.

Most users understood that they could access the special punctuation menu through the SHIFT button. This may be because the standard keyboard also requires user to hold on to the SHIFT button to type the special characters. This translated well to the SteamVR keyboard, but the differing layout still caused confusion for the user. We understood from this that the user’s expectations of a layout is an important aspect of learnability.

Designs

T9 Keyboard

For this concept, we aimed to explore a keyboard that primarily emphasizes efficiency.

The idea behind this design is reminiscent of the keypad interfaces of old Nokia phones from the early 2000s. A keypad interface will be displayed on the controllers that the user will be holding, specifically on the touchpad region. The touchpads are split into various regions as shown in the above figure, each with a set of characters mapped to it. Words will be formed by a single keypress of a region of the touchpad, and a built-in dictionary will predict the words which you intended to type. You will be allowed to choose which word is the correct one, from a list of available words that is shown right below the typing area. If the word is not currently known by the dictionary, it will be possible to add the word to the dictionary, such that the dictionary learns it and suggests it to the user the next time round.

Estimated Metrics
Given that the dictionary function helps to minimize key presses as much as possible, hence reducing the time spent on spelling the word out, this design is intended to emphasize efficiency, which is one of the usability concepts we chose to focus on. An expected trade-off would be lower learnability as this is not a familiar interface to VR Gamers and they might have to take some time learning how to use it well.

PunchyKeys

Punchykeys was modelled to test how having high learnability impacts typing efficiency using certain key design guidelines.
This keyboard design features an user-centered approach to predict a keyboard that would easy for the user to learn.

As part of the design, problems noticed from baseline results were also considered. For example, users experienced tracking issues using the pointer input style. This introduced a lot of user-driven errors where the user often clicked on the wrong key and had to reverse their action. In addition, the backspace button was on the keyboard too. On top of the previous two points, spacing in between the keys were inconsistent, keyboard layout was slightly different and only one controller could be used to point. Punchykeys was designed based on these problems to be a possible solution.

Estimated Metrics
This keyboard’s aim was mainly to focus on learnability. The excess movements required by the user to press the button may impact the efficiency. Error handling of this keyboard should be better as using colliders are less susceptible to controller jittering.

CorrectiveKeys

Aims
This keyboard was developed after the T9 keyboard and PunchyKeys were developed. After some user evaluation, insights were drawn to find a way to synthesize these two keyboards together to create a keyboard that is both efficient and learnable. We took the results that we learned from the punchy keys and merged it with an already existing keyboard type, the pointer based keyboard.

Similar to the punchy keys, we created the keyboard with a similar mapping to the QWERTY keyboard for it to be more learnable (using affordance of the shape and arrangement to facilitate learnability). We added another set of keys that provided an error correction feature to allow users to easily correct typos caused by the instability of the pointer tracking.

Estimated Metrics
We assume that the jitteriness of the pointer based controller will result in a lower efficiency due to having to correct more mistakes. This keyboard also takes a penalty in learnability because the corrective keys that are displayed close to the users is something new that most other keyboards do not have and thus may be a challenge for users to figure out how to correct themselves while typing (or they might even miss out on the corrective nature of the keys). However, error handling of this keyboard will definitely be the best amongst the other 3 keyboards as it affords the user to correct any mistake he makes while typing.

Evaluation

Results

Analysis

Efficiency takes time
Although we hypothesized that the T9 keyboard would perform better in terms of efficiency, our evaluation results show otherwise. The T9 keyboard performs worse than PunchyKeys and CorrectiveKeys in terms of efficiency. User responses tell us that the T9 keyboard was difficult to use, as the access to functions were unclear and the placement of alphabets were unfamiliar.

However, the gradient of improvement over the attempts was steeper for the T9 keyboard as compared to the other two. This could possibly mean that the user has the potential to eventually be more efficient at this keyboard after familiarizing with it over time. The tradeoff here being that a user will have to endure a period of inefficiency before becoming proficient at it.

Faults of prioritizing efficiency
From the comparison with baseline, we see that the T9 keyboard performed worse than the baseline keyboards. Although we aimed to design a keyboard that focuses on efficiency, it loses in terms of efficiency as well as the rate of improvement. By attempting to make everything a shortcut that can be accessed with minimal thumb movements, the user has too many things to learn before becoming efficient. This shows that efficiency cannot be achieved without considering learnability.

Importance of familiarity
We learnt from our baseline that familiarity of interactions is important to users. Comparing our results above strengthens this claim even further. Keyboards with familiar layouts, such as PunchyKeys and CorrectiveKeys that use the QWERTY layout, perform significantly better than T9, which was unfamiliar to most users despite being an adaptation on older mobile phone keypads.

Confidence with lower error rates
With larger areas for each key in the PunchyKeys and CorrectiveKeys design, the errors associated with the jitter is reduced. Although the user will have to make larger physical movements, the confidence that comes with better error handling allows the user to type much faster whilst retaining accuracy. This could mean that efficiency of a task is not only dependent the click distance, but also with low error rates.

Initial learnability
Although PunchyKeys and CorrectiveKeys have a lower rate of improvement than the baseline keyboards, they perform significantly better in terms of efficiency and error avoidance. We discovered that PunchyKeys and CorrectiveKeys has a high initial learnability,  which allows the user to type quickly from the beginning without much struggle.

Hence, rate of improvement is insufficient as a metric for learnability when there is a high initial learnability. The rate of improvement could be a metric for the cumulative learnability, showing the potential for a user to learn a system over time.