User Experience Sampling with Twitter

Collecting user experience data with Twitter

We design ‘User Experience’ by trying to understand what experience people have with a product (or a service.) To get a full understanding, we need to determine which expected values people have before using a product and whether those values are adequate for their needs. That is, by investigating how profoundly ‘useful, convenient, fun, and satisfying’ the product is, we can get to the core of user experience.

To understand their overall experience, it is acceptable to conduct a standard usability test in a controlled lab environment or simply ask people what they think about a product. However, when we allow people to remain in their real life context when observing and asking how they interact with a product, we can gain a better understanding of their experience.

To gather experiential data from an actual real life environment, we usually apply some typical methodology such as a diary study or qualitative field research. A diary study enables us to collect and record various forms of experience data over a long period of time. However, it is hard to guarantee the freshness of collected data since the gap between the time of doing something and recording that experience may be prolonged, possibly leaving participants without a vivid memory of their experience. On the other hand, qualitative field research can greatly increase the assurance of collecting and recording fresh experience data by enabling us to directly observe people’s behavior and attitudes in real life situations. But most UX consultants know that it is difficult to conduct large‐scale field research due to time and budget limitations. Therefore, if you want to collect a broad range of vivid experience data, it is unavoidable to consider using an alternative experience sampling methodology which takes good traits from typical methodologies and overcomes their weaknesses.

To do this, we contemplated using Twitter as a possible tool for experience sampling and decided to see if it could be used effectively for actual research projects. Twitter, currently the most popular micro‐blogging service, is based on one simple question: ‘What are you doing?’ Once users leave brief updates (called ‘tweets’) to this question describing personal situations, emotions, interests, or random thoughts on their Twitter homepage, their followers can read these messages in real time. It allows users not only to write their prompt thoughts and feelings in a simple way but also to check others’ current life stories on their own homepage. Such simplicity has played an important role in Twitter’s success. Moreover, by accepting messages from SMS, web, mobile web, instant message, or other third party API projects, Twitter makes it easy for users to stay connected all of the time from any location.

We brought those traits of Twitter into focus to overcome weaknesses found with typical experience sampling methodologies and adapted them to an actual research project to see if Twitter could be utilized effectively.
How we used Twitter as an experience sampling tool
The research process

We recruited 15 participants who were in their twenties or thirties and using a smartphone. They were asked to install the Twitter mobile application to their smartphone and follow each other so that they could all communicate with one another.

The research was conducted for 7 days. During this period, two kinds of data were collected: participants’ individual smartphone experiences and opinions from several discussions. The individual experience data was reported by the participants as soon as they received a SMS sent by the researcher. They left the answers to the following 4 questions 8 times per day during the week via Direct Message, with only the researcher having exclusive access to these responses

We checked these direct messages on the web in real time and if there was something that needed to be clarified for better understanding, we sent out additional questions.

From the third day, we collated an abundance of individual experience data and started to select top‐ranked smartphone design issues. Then we openly posted one issue per day as the theme for daily discussion in order to dig more deeply and widely into certain issues.

Throughout the research period, we were able to collect over one thousand rich responses of experience data. Among these responses, 70{579fc5914157f01252f22c89e9ec6be3358cbf8c9d19695844f6687910d137e4} of the participants’ personal experiences were recorded via Direct Message (587 responses). Considering the participants had to send answers more than 50 times per day, from early morning to late night, for 7 days, such a high response rate seemed quite impressive.

► Discussion among the participants on Twitter

What are you doing with your smartphone now? (If you aren’t doing anything now, what was the most recent activity you did with your smartphone? When?)
What was the specific purpose of that activity?
Please describe the situation around you at that time.
What inconvenience did you have? To improve your smartphone experience, what do you expect?

Everyday, the participants also freely posted brief ordinary thoughts unrelated to smartphones about social issues, weather, or personal stories on Twitter as well. They communicated with each other, especially office workers, since it is easy to access to Twitter website with a PC or laptop during working hours. In the process of such interaction, a strong bond occurred between participants and such feeling encouraged them to actively participate until the end of the research period. Contrary to the decrease in participation of typical long‐term diary studies, this experience sampling method with Twitter lead to spontaneous and constant participation by giving the users a sense of group identity.

The most brisk interaction among the participants was achieved when they discussed the major design issues we had suggested. These discussions were carried out only by the users themselves without any intervention from the researcher. We were able to gather 60 opinions per theme, with each theme being discussed on average for 5 hours. Generally. the participants expressed their own viewpoints, agreed or disagreed with one another, and asked questions or provided answers based on their previous experiences. Through such interaction among the participants, we were able to gain a better understanding of common inconveniences and expectations users have. Moreover, for the researcher, it was a quite effective way to promptly obtain users’ thoughts whenever they wanted since such discussions do not require any preparation steps or time limitation.
Conclusion & Suggestions

Since it is very simple and easy to for users to leave their personal thoughts on Twitter anywhere and anytime, we were able to collect their ‘fresh’ experience data, including emotion, attitude, and behavior, in real life context. Plus, Twitter enables users to share their experiences with each other without much effort and it results in an abundance of real‐time interaction.

The most impressive finding is that the traits of Twitter keep participation rates high by promoting strong bonds within a group, as well as leading to spontaneous responses to a researcher’s inquiry. In addition, today’s current online features allow a researcher to overcome time or spatial limitations, conduct open discussions with minimal effort, and easily encourage participants to join in discussions of interest by themselves.

In conclusion, we maintained the intrinsic attributes of a typical experience sampling method and obtained the strengths of Focused Group Interviews by investigating opinions in a group setting. Thus, if a research situation requires the collection of vivid experience data for the long term or group opinions on various themes in a simple and fast way, such methodology using Twitter seems be more effective.

This research was conducted with smartphone users whose device allowed them to install third party applications. In order to get immediate responses from participants, it is necessary to use mobile device on which users can freely set up applications. Moreover, there is a 140 character limitation to what a participant can record per posting. In addition to using Twitter to collect and understand user experiences, researchers

may be required to ask participants to send contextual photos, videos, and any other form of multimedia via email or messenger to gain a more profound understanding of their subjects.