Diary studies with WhatsApp - a critical review

Last year Facit Digital was commissioned to find out how children and young people use digital media and electronic devices. Another client wanted to generate ideas for a new product and to learn about the usage habits and problems of his target group regarding audio usage.

In both cases we decided to carry out a diary study. In this way, we were able to gain numerous in-depth insights into the usage behavior and pain points of the investigated target groups. We used WhatsApp as the medium for communicating with the test subjects.

In addition to some of the limitations of this type of implementation compared to special diary study tools, WhatsApp has many strong advantages.

Everyone can use WhatsApp

WhatsApp is very widely used. 75% of the German population aged 14 and older use WhatsApp regularly, 63% daily. In contrast to special tools, no new application needs to be downloaded and "learned". The study participants can document and send their self-observations intuitively from the usage situation. Nowadays, even the smartphone is always and everywhere at hand. Situations thus remain as authentic as possible. A large amount of valid contributions can be generated. In addition, WhatsApp offers a simple way to use images, videos and voice messages for documentation purposes in addition to classic text contributions.

Researching with WhatsApp: How does it work?

First of all, you need a smartphone with a sim card. This can be used to create a WhatsApp account. You need the "WhatsApp Business" app to create so-called broadcasts. With this app, all study participants can be added to a list and the research team can send a message to all contacts simultaneously. The participants only need to save the test mobile phone number in their mobile phone. The participants themselves cannot see who else is on the broadcast list.

Afterwards, the daily tasks and instructions can be sent to the test subjects via the broadcast. In order to praise individual participants or to motivate those who are less committed, you should also respond to them separately and send individual messages. By using WhatsApp as you are used to, you usually get relatively quick and unfiltered feedback.

What is the catch?

As soon as more than one researcher is involved in the communication with the test subjects, things get a bit complicated. Since the communication is linked to a mobile phone and a sim card, it is hardly possible to collaboratively manage the study. You always have to have your mobile phone with you when you want to communicate with the participants. Even WhatsApp on the desktop can never be run by more than one person at a time.

Further obstacles are added to the documentation and evaluation. For us it made sense to transfer the text messages into an Excel document and sort all contributions by task and participant. To do this, the text messages first had to be exported from WhatsApp or copied individually to Excel. Images and videos had to be downloaded and stored (but were then decoupled from the associated text messages). In order to evaluate the voice messages, we recommend that you first select those that help answer the research questions and then transcribe only those.

To WhatsApp or not to WhatsApp

There are numerous special tools for the execution, documentation and evaluation of diary studies (e.g. of core value, dscout or Indeemo). In a dashboard, participants can be managed collaboratively and tasks can be assigned in a targeted manner. Depending on the scope of services offered, such tools can also handle the transcription of voice and video contributions. In contrast to WhatsApp, however, the special tools incur costs for their implementation. In addition, both the test persons and the researchers first have to get used to operating a program. Such tools often contain many functions that are not needed for smaller studies.

As with most methods and tools, it is important to weigh the pros and cons of conducting diary studies with WhatsApp. For example, if studies are conducted over a longer period of time, involve more participants, or require an above-average number of video and voice messages, the use of specific tools could be more efficient.

For less extensive studies, it may make sense to have only one researcher supervise the subjects during the field time anyway.  With a number of participants of 10 or less, the effort for manual documentation and transcription remains manageable. In this case, the benefits of low-threshold and familiar use of WhatsApp would outweigh the costs.