Overview

Consider this simple scene to the right and suppose you want to point one of these persons to an addressee.

There are many ways in which you could do this, but most speakers have no difficulty in quickly deciding which information to include and how to realize this information verbally and non-verbally via referring expressions like “the man in a suit,” “the woman” and “the younger-looking man”. How do speakers do this?

Both in psycholinguistics and computational linguistics, researchers have addressed aspects of this question, but we still have an incomplete understanding of how the human production of referring expressions works. This project rests on the assumption that for a better, more complete understanding of this process, it is important to bridge the gap between the two disciplines. Such a bridge has both theoretical and methodological advantages. Psycholinguistics has important insights to offer in the human production of referring expressions and a methodology involving carefully constructed and controlled experiments; computational linguistics a well-established approach involving corpus analysis and computational modeling. Together they can be harnessed to achieve new scientific insights in the human production of referring expressions which in turn may result in a better understanding of speech production in general, especially where the interactions between speaker and addressee, between planning and realization, between verbal and non-verbal communication, and between scene perception and speech production are concerned.

Specific questions that are addressed in this project include: How does scene perception influence the production of referring expressions? Why and how do speakers overspecify their references? What is the impact of audience design and adaptation on the production of referring expressions? How do speech and gesture relate to each other? How are visual cues timed and produced during referring expression production? To what extent do speakers adapt their non-verbal realizations of referring expressions?

For more information, see the full project proposal [PDF].