People interact with one another using words, intonation, gestures, facial expressions, and eye gaze.
All these communication channels are intertwined and used simultaneously. That is why the process of ordinary
human communication can be described as having a multichannel character. We take part in this kind
of communication throughout our lives. Nevertheless, it remains an understudied phenomenon. There are at least
two reasons for such a state of affairs. First of all, the process of multichannel communication is ephemeral,
and it takes place without leaving any tangible trace. Secondly, research of the contents of various communicative
channels traditionally belongs to the realms of different disciplines which have rather loose connections with
one another. For instance, the verbal component is studied by linguists, while research on gestures and eye
gaze is mostly conducted by psychologists. The research team at the
Institute of Linguistics of the Russian Academy of Sciences has set the goal
to conduct a comprehensive study of multichannel discourse based on a unified theoretical and methodological approach.
Within the framework of the given approach, we have created a resource called “Russian Pear Chats and Stories” which includes
a series of recordings of natural communication among several participants along with
their vocal and kinetic annotations. The process of making a recording was carried out with the help of innovative
technical solutions such as high-quality audio and video recordings, as well as the use of eye
tracking devices. Vocal annotation is based on the principles previously developed for oral speech;
see spokencorpora.ru. The principles of annotation for kinetic behavior
were developed specifically for the objectives of the given project. You will find the results of the research
based on the corpus “Russian Pear Chats and Stories” under the tab “Publications and talks”.
If you want to contact us and ask questions about the project and the corpus, feel free to use this e-mail address.
A scheme of multichannel communication
Acknowledgement: The project team is grateful to the translator Maria S. Sapozhnikova and the editor Kelly Callahan for their assistance in preparing the English version of this web site.