Field User Test – testing in real life conditions

While UX testing is a proven method to identify usage problems of an interface, it also has some limitations. These tests usually take place once in the research lab, where real usage is only simulated. In this controlled environment, certain aspects can only be considered to a limited extent, such as the speed of the Internet connection or the display size of the output device. In the case of apps or technical devices that are subject to frequent and repeated use, the UX test may not be sufficient. However, if serious problems with a technical device are not noticed until too late, it can lead to the need to restart serial production - which is associated with high costs. To avoid this, we carry out a Field User Test (FUT).

The Field User Test in theory and practice

The FUT is an effective method of testing regularly used products in the run-up to the full launch in detail, on a long-term basis and under real conditions with the target group. Only in this way can weaknesses and problems be uncovered that would not be revealed by a time-related test. As part of a project Facit Digital conducted a field user test for Vodafone Kabel Deutschland. Facit Digital accompanied the development of the GigaTV 4K Box.

FUT in use: For one year test participants tested the GigaTV 4K Box at home.

During the one-year test phase we built up a test community with over 100 participants. We equipped this community with functional beta boxes. In regular test sessions, the participants were asked to report their experiences from everyday use of the box. For this purpose they conducted typical use cases on the features of the box and the Companion App.  They documented their experiences in an online questionnaire. The technical input for the test sessions was provided by developers and product managers at Vodafone Kabel Deutschland. As a result, complex user problems were uncovered, which were subsequently eliminated.

Adaptation of the GigaTV 4K Box to user needs

Through the quantitative analysis of the use cases, frequently occurring bugs were identified, which were deepened and described by the qualitative results. For example, the button assignment of the remote control and the wordings of the interface were further developed and adapted to user needs. The results of the test sessions were fed back to the developers on the customer side so that they could be considered and implemented without delay in the next development phase. The regular feedback from users helped Vodafone Kabel Deutschland to further optimize the GigaTV 4K Box before the launch and to ensure a smooth market launch.