FairyLand VR

FairyLand was my research project during graduate school. After a chance encounters with an occupational therapy center and discovering how children with autism learn in class, I developed a strong interest in the topic and tried to think about what changes I could bring with my power.

‍

‍

When we were a kid, we were always able to convey various meanings through body movements, facial expressions, and metaphorical words. We were always able to interact with our peers in every way.

However, we found that children with autism are unable to establish long-term, positive social relationships with their peers in school.

‍

So, what caused the autistic children unable to maintain long-term social relationships?

‍

At first, we read some journal articles, found that children with autism have difficulty with non-verbal communication and lack of imagination, which made peers didn't want to engage in closer communication with autism.

Early Insight

‍

Field Observations & Interviews with Stakeholders

To gain a practical understanding of the difficulties faced by children with autism in group interactions, we observed them in close in a daycare center focused on occupational therapy. We interviewed elementary school teachers, parents, and occupational therapists. We aimed to understand the problems experienced by the users from stakeholders.

‍

‍

We have summarized some issues:

1. Some traditional teaching models always make them less focused and less effective.
2. Teaching training is lengthy, ranging from half-year to several years.
3. They can't understand the meaning of abstract words.
4. They are afraid of interacting with real people, so they are so hard to build social relationships with others.

‍

Overall, current teaching strategies for children with autism do not provide a positive way to improve social skills, and some autistic kids are afraid of facing real people, which will reduce their learning outcomes.

‍

‍

Deeper Insight

‍

‍

To gain a more comprehensive understanding of the difficulties faced by children with autism, I have spent a lot of time reading the literature. In the last ten years of literature, we found that:

‍

• Our target users are unable to understand abstract meanings because of a lack of graphical and imaginative connections, called "symbolic play".
• There are currently no applied technologies (e.g., virtual reality) to improve the symbolic playability of children with autism.
• "Social Story" is the most common training strategy, but it takes a too long time.

‍

So, this is a potential opportunity we can take, we will reorganize the problem.

‍

Through field observations, literature research, and stakeholder presentations, we understood the needs and redefined the problem.

‍

The following is our line of thinking:

‍

‍

‍

After defining the problem from the first stage, since our target users are children, we can understand the difficulties intuitively from the stakeholders and finally decide the direction of our solution.

‍

Ｗhy do we choose VR?

Because children with autism lack immersion in traditional teaching strategies and take a long time to learn. However, VR can easily bring them into a new world to learn and explore, reduce the fear of facing real people.

Our goal is to develop a VR product for user testing to observe feedback from users. The engineer and I spent several days sprinting to define the project goals, hypotheses, product functionality, and user testing methods.

‍

‍

After confirming the problem and the goal, I always used to draw sketches on paper. It helped me to quickly visualize the ideas in my brain, which allowed me to think about the function, level, and structure of each interface more intuitively. Also, it helped me to communicate with the engineer more efficiently, which allowed us to define the functionality and determine the feasibility of development after the information architecture was produced.

‍

‍

‍

To ensure that the project progressed, I worked with the engineer on system development and checked that the perspective in the game engine was suitable for children with autism.

‍

What is special here is that we incorporated the social situation of the theater-based game by dividing the story into 5 scripts. Users can experience a different social story (script) each time. Also, this IA gives me a more complete view of the usage process from the user's perspective.

‍

‍

The interface of a VR game is different from the web or APP interface, the purpose of opening the product is to participate in the game. Therefore, it has a clear task to guide the user through each step. After interviewing with occupational therapists, we planed the product experience into a course format to engage users in the experience, with a full theater framework for each engagement.

‍

💡 A Richer Immersive Experience: Theater-Based VR Game

The MVP version was launched to make sure that our target users would be able to interact and talk smoothly in each script and 3D scene. We call that a "Richer Immersive Experience". Using theater-based games in VR instead of traditional methods, we created a social interaction that is safe and repeatable and enhances children with autism in understanding abstract words. The MVP features are as follows:

‍

Vivid character voices and social interaction to enhance the sense of interactivity and realism

To allow target users to interact with the characters in the game, we designed five-story characters (with voices) to create a richer sense of immersion and interaction.

‍

Graphical thinking to enhance understanding of abstract words for children with autism

Children with autism are unable to understand abstract words because of their weak imagination (lack of symbolic playability). Therefore, we will use task-based drama interaction to enable them to link abstract words to symbolic patterns through pictorial thinking.

‍

Integrating abstract words into scripted dialogues

Using theater-based games in VR, we added abstract words to daily conversations so that they could deduce the meaning of the abstract words through the back-and-forth dialogue of the script.

‍

After completing the MVP, we conducted a simple product test with peer users and interviewed occupational therapists for their opinions, and we began iteration of the design.

Here are the key changes and features that we have made in the design iteration:

‍

Complete social Script Interaction

Based on the teaching strategies of occupational therapists and special education teachers, we designed the game tasks into a week-long course. Therefore, it is necessary to create more scripted dialogues with different story situations. We designed a 100+ dialogues interface so that the user can think and practice different social situations every time.

‍

Integrating abstract words into scripted dialogues

Using theater-based games in VR, we added abstract words to daily conversations so that they could deduce the meaning of the abstract words through the back-and-forth dialogue of the script.

‍

‍

‍

‍

‍

In response to the fact that social training takes a certain amount of time, we designed the VR game within the same story framework to develop a different social dialogue.

‍

‍

‍

‍

Guiding users to select symbolic objects to train abstract thinking skills

Due to their weak imagination, children with autism are less able to translate abstract words and understand each other's expressions and gestures, which reduces their communication with peers. So, that's why we used 3D objects and let them associate with words they don't understand.

Gradually, they can communicate more naturally by understanding and familiarizing themselves with social interaction patterns.

‍

‍

‍

‍

‍

Improve product usability for our target users

After entering the VR world, the user had a first-person perspective to talk to the 3D character and practice various social situations. And according to user feedback, we adjusted the height of the viewpoint to meet the vision of children aged 6-9.

‍

‍

‍

‍

Practice responding to various social reciprocities

In order for children with autism to naturally understand the meaning of social responsibility in their lives, we included social response actions at the end of the dialogue, where the 3D character performs social actions and then allows our users to respond. The social actions were designed based on literature and interviews with our stakeholders.

‍

‍

‍

‍

A teaching experience that is easy for instructors to use

To make FairyLand VR easy to use for people who have not been exposed to it, we designed a simple login and task selection process so that they can easily evaluate the progress of each case using the data we recorded during the project.

‍

‍

‍

‍

Highly Consistent Component Library

I planned a consistent user interface and hierarchy for this project. All components were made according to information architecture and helped me to collaborate more efficiently with the engineer.

‍

‍

Once the iterations were completed, we began looking for users to participate in our user testing:

Recruiting Participants

30+ groups of parents were recruited for interviews through the Facebook group.

Considering that our users are a special group, we wanted to get data by recruiting actual cases. Since Because our target user, autism, has high heterogeneity, we had to do one-on-one interviews and an easy test with parents and children to understand the situation of each case.

‍

‍

After recruiting participants to experience the course, we obtained the following data:

‍

‍

‍

After several weeks of the experiment, we collected a lot of data and the results showed that our iterative version was a huge success. Our target users have significantly improved their social behavior, understanding of abstract words, and even fluency in spoken expression with our teaching intervention strategies.

We have also heard many positive comments from special education professionals and parents of children.

‍

‍

Let's check the final result:

‍

It was an exciting and fun project to try out all kinds of creative ideas and to discuss with the engineers to make the project complete. It was a very busy year due to time constraints and the need to read a lot of literature for user research, but we are very glad to have achieved such a great result.

In fact, there were many trade-offs and challenges during the process, such as: How to convince people in the special education field about our project? How to make the product more effective? What to do if the results are not as good as expected? The technical limitations of implementation, the acceptance of autistic children ...... And so on. In the end, what I realized is: if you have an idea and a plan, then just try it out!

‍

Because of this project, I also got to know more teachers, doctors, psychologists, and parents of children in the field of special education and was able to understand different views and questions by being close to different stakeholder roles, which also allowed me to think more completely when defining the design solution and make the product experience closer to the real users. During the user testing, I was able to observe the participants from the sidelines and get feedback from parents, teachers, and children, which was one of the best takeaways.

‍

After going through the 0 to 1 project, it's a path that doesn't stop. I've learned a lot of things from the process and I'll keep working on them, and the most important things I've learned from it are:

‍

• Always think from the perspective of the target user.
• Listen to the stakeholders in the project.
• Release the MVP and iterations according to plan in a limited time.