Talk the talk, walk the walk: A case study on systematic evaluation in Participatory Design

Participatory Design (PD) has become a versatile, often-used design practice. However, its legitimacy is often criticised due to a lack of empirically backed benefits or impact. This impedes PD's ability to convey its benefits to its stakeholders, other disciplines, and society at large, making progress in the field more difficult. One way to address these criticisms is using systematic evaluations in PD. While evaluation frameworks for PD exist, they are rarely used or built upon, and results from evaluations are seldom disseminated. Thus, this paper presents a systematic evaluation of the first year of a participatory project for developing a Virtual Reality training system as a case study. We discuss the impacts of systematic, explicit evaluations on PD practitioners, the PD community, and beyond. The work, therefore, contributes to establishing and developing evaluation as a practice in PD to enhance knowledge-building and accountability.