Moving past the lexical affiliate with a frame-based analysis of gesture meaning

Interpreting the meaning of co-speech gesture often involves identifying a gesture's 'lexical affiliate', the word or phrase to which it most closely relates (Schegloff 1984). Though there is work within gesture studies that resists this simplex mapping of meaning from speech to gesture (e.g. de Ruiter 2000; Kendon 2014; Parrill 2008), including an evolving body of literature on recurrent gesture and gesture families (e.g. Fricke et al. 2014; Müller 2017), it is still the lexical affiliate model that is most apparent in formal linguistic models of multimodal meaning(e.g. Alahverdzhieva et al. 2017; Lascarides and Stone 2009; Pustejovsky and Krishnaswamy 2021; Schlenker 2020). In this work, I argue that the lexical affiliate should be carefully reconsidered in the further development of such models. In place of the lexical affiliate, I suggest a further shift toward a frame-based, action schematic approach to gestural meaning in line with that proposed in, for example, Parrill and Sweetser (2004) and Müller (2017). To demonstrate the utility of this approach I present three types of compositional gesture sequences which I call spatial contrast, spatial embedding, and cooperative abstract deixis. All three rely on gestural context, rather than gesture-speech alignment, to convey interactive (i.e. pragmatic) meaning. The centrality of gestural context to gesture meaning in these examples demonstrates the necessity of developing a model of gestural meaning independent of its integration with speech.