The Fanbot was a chatbot commissioned by Mastercard and Tennis Australia to bring people closer to the Australian Open 2018 (AO). By interacting with the chatbot, users were able to get live information on game scheduling, players' profiles, fun facts and more. Digital Arts Network Sydney partnered with Wizeline (my team) to design and develop this chatbot.
1. My role:
Project duration: 2 Months.
What I did: Proto-personas, Job Stories, Object-Oriented UX, Conversation Flows, Prototyping, Usability Testing.
2. Beginning: where did we start?
First things first, we had a meeting with DAN to understand Mastercard’s needs and goals. From there we gathered the following:
As one of the main sponsors of the AO, Mastercard was looking for innovative ways to increase their brand visibility throughout the tournament.
Mastercard wanted to show special offers to their clients and promote their Served App.
Tennis Australia wanted to try new communication channels to reach out to their audience.
Both wanted an experience that was good for fans attending the AO in Melbourne, as well as for those watching from home in the rest of Australia.
We had a strict deadline. The Fanbot had to be ready before the AO started and fully available until the final match.
Once we defined the client's needs, we dived into the users' needs. Because of our tight deadline, I didn't have enough time to interview our target users. I still wanted to know DAN’s assumptions about potential users. I sent out a questionnaire to the DAN team, analyzed their responses and grouped them to create three proto-personas. This was the result.
Finally, I created job stories for each persona. I’ve found that job stories are quicker to consult during design, they force you to think about the user’s tasks’ context and, also, they are a good tool to communicate with project managers.
3. Getting into it:
Object Oriented UX
Based on his previous experience working with bots, one of my colleagues suggested to me to do an exercise called Object Oriented UX before starting the conversation flows. I created a map of objects and how they related to each other (i.e. there are matches, matches have players, each player has a score, etc). I also outlined each object's properties.
OOUX - Fanbot Sample
For some projects this could be an overkill but, for chatbots, it is really useful as it helps the team understand how users may jump from one flow to another. Additionally, engineers can use these objects as a base for their own programming objects (or at least that was my case).
In total we had 14 different flows, divided in 3 groups:
Conversations triggered by users: whenever a user asked a question to the chatbot or selected a quick reply.
Conversations triggered by DAN: this included polls, Mastercard moments, Priceless facts (fun facts) or giveaways. These only happened once per day.
Subscription messages: updates sent to users who subscribed to topics or players.
This is an example of a DAN-triggered conversation. Users could win prizes by participating in different activities like sending a picture or answering a question:
As you can see, this conversation has two flows. Ideally, users would receive a notification about a giveaway happening and immediately participate. However, what if the user wasn't ready? What if between them taking the picture and uploading it they wanted to check information about a match? That would take the bot into a different conversation thread. With this in mind, we let users know how to trigger the "giveaway" conversation at any moment and created a shorter version of the flow (they didn't need to read the whole thing twice after all). Most flows have many entry points, but we didn't need to create a different flow for each of them, as you can see in the example, conversations are sort of modular.
Flows had to cover all these cases, they were the conversation's blueprint. Now, initial flows didn't have final copy, as you see them in here. For each flow, the first step was to create what we call "intent copy". Intent copy indicates what the bot is saying as oppose to how the bot is saying it; in other words, it is the chatbots equivalent of wireframes. Based on the intent copy, DAN developed the "voice" and final copy.
A conversation is more than text; is tempo, context and language. Because of that, I wanted to use a prototype that actually resembled a chatbot. I used Framer and a framer module to create a prototype which had quick replies, carrousels and key words that triggered specific responses. At this point, we didn’t have the final copy, so I created dummy dialogues that were easier to read than intent copy.
Testing and iterations
We had two rounds of testing, one in the middle of the project (where we used the prototype) and one more when the bot was live on Facebook but not public yet. In the first round, I conducted five interviews with Wizeline employees that were not related to the project in any way. At the same time, we shared the prototype and usability testing script with DAN so they could test with people in Australia.
We discovered a lot of things from testing. For instance, the on-boarding process was too long. Users were overwhelmed by the amount of continuous text.
To solve it, I added questions with quick replies in between the dialogues of the chatbot. This way we avoided having a chatbot monologue and gave users control of when the bot continued talking. I also learned about how users talked to the chatbot which was crucial to train the bot Natural Language Processing.
4. Final results
The Fanbot was available on Facebook messenger and ran throughout the tournament's duration from January 15th to January 28th. It had over 8700 unique users and registered around 350,000 interactions. 57 percent of the fans that used it, returned daily and 64 percent signed up to receive updates on different AO topics.
Or, as summarized by DAN:
5. Challenges and learnings
The prototype was effective but took too much time to build. For my next similar project, I actually tried something different. I adapted a technique called “pluralistic walkthrough” to test voice user interfaces. You can read more in the "Other Projects" section.
The first conversation flow I designed was the On-boarding. In hindsight, I should have done this once everything else was ready. Is like designing a videogame, you design the game first and then the tutorial, not the other way around.
By the end of the project, the natural language processing of the bot was very comprehensive. However, the continuous presence of quick replies distracted users from trying to input their own commands. The experience could have been more complete by giving users better guidance on how to take advantage of the NLP.