User Testing

Due to being unable to attend our User Testing Days, this post will incorporate alot of information from my teammate's blog reports on these days of tests.  3 were carried out in total and following each user's experience of our project, questions were asked regarding their understanding, feelings and advice for the project as a whole.  This feedback was noted down and was then used by each of us to improve various areas of the project, whether in the area of animation, effects, textures, modelling, lighting or narrative.

Below are results and notes taken from our March User Test by our Producer, Lorna -

Our First Survey group consisted of 10 people, a mixture from our own year group, as well as people from other courses in their final year, so the majority of the age group where in their early twenties. For this stage we had our environment, blocked animation in with basic posing, as well as some holographic effects in the scene. After the test we asked a series of question focused on:

• If the story made sense – To almost all they understood the story, only one stated they where getting use to the VR experience and needed more time to adjust to it. However we need to consider that class mates know what our piece is meant to be.

• If they felt ill or dizzy while watching the piece – the story can make sense, but if the viewer is too dizzy to watch it, then there is a serious issue. Luckily nobody did, which considering one of our participants suffers from vertigo, we felt this was a win and keep the viewer seated and fixed from one spot was the right call.

• How did they fine the experience as a passive observer – Quite a few people have said to us ‘You should add controllers to this piece’. We have always felt that there was no story reason to add this though. So we put it to our survey. The majority said no, that they could understand the story better or knew that they didn’t really need to move. Others said yes they felt hindered, but it added to the discomfort. The odd person did feel they should have more control. We have decided to add more design features that explain why you as a character are unable to move to add to the story. We feel that if we gave people controllers, they would not read the story and only throw objects which doesn’t add to it, or emulate the emotions we want our viewers to have.

• What did you feel while watching it – We asked this to see what emotions our viewers most felt. The top emotions felt where nervous, interested, helpless and confused (confused not in the sense of the story line having checked that in a previous question). Overall we where happy at how people where reacting to the piece.

• The viewers eyeline needs checked. An issue we are having is that as everyone is a different height, they either feel too tall or too small within the scene. We need to get as close to average as possible.

• Make the glitches snapper.

• Add the menu.

• Alter more of the surroundings more.

• Ensure it is engaging throughout the piece, the middle section pulled someone out of the immersion.

Priorities from this User Test (many of which we had yet to add or knew about):

Our second user test took place in April and featured an updated and more smoothed animation which was preparing to see lip syncs introduced to the characters.  The build of the scene looked like this -

Below are the notes taken by our Director, Cassie, after this second user test of our project.  This user test featured some new participants (so we could gain new eyed opinions on the readability and face value of the project) and some of the previous participants from our testing in March (so they could make us aware of what had changed and if the project is progressing well) -

The feedback was overall constructive with positive and helpful elements that we can use to take our project further. The main things discussed were comments such as “you should include a loading screen” or “you should glitch the screen” which are things we have yet to do. However they are encouraging comments as it means we are aware of the improvements we need to make.

Positives from the user testing was participants we had tested the scene before has seen improvement and everyone overall understood the story. Everyone also liked the environment and understood where they were, meaning we are in the refinement stage where it’s about altering tiny details to improve the scene. Asides from the obvious issues such as missing items from the story and the eyes glitches (unintentionally!!) we feel good with what we need to do going forward.

Things to focus on for the final deadline:

• Skinning issues with the animation such as the eyes (even though some users found it funny!)
• our loading screen and crashed screen to give users context
• Importing glitches so users can look around them.

Finally, we had user testing with just one user.  We asked our tutors to try out our headset to set what feedback they could give us as the time for submitting draws closer.  Below are the notes from our user test with our tutor, Alec.

This feedback from Alec was especially helpful as it impacts each one of us in some way and gives us something to work on as we stretch towards finalising various aspects of our narrative.  For me, the area of the male's almost floating appearance will need to be my main focus as at the beginning, we need to portray an atmosphere of clear and clean animation.  The floating can distract the user and take away from the impact we want our project to have for each person who puts on the headset.

Headset and User Experience (UX)

We had a lot of fun as a team trying out various VR technologies in the run up to the production of our project.  It was in these moments we discovered our excitement and passion for the topic of VR and were intrigued and fired up to produce some VR content of our own.  Moving from Google Cardboard, to Oculus, to Vive has been a long journey, especially for our Director, Cassie, but each stage, as stressful and troublesome as it has been, has led us to where we are now.  On the brink of our own VR creation.  The Happy Place.