Coordinating Gesture, Word, and Diagram: Explanations for Experts and Novices

Successful explanations are a symphony of gesture, language, and props. Here, we show how they are orchestrated in an experiment in which students explained complex systems to imagined novices and experts. Visual-spatial communication - diagram and gesture - was key; it represents thought more directly than language. The real or virtual diagrams created from gestures served as the stage for explanations, enriched by language and enlivened by deictic gestures to convey structure and iconic gestures to enact the behavior and functionality of the systems. Explanations to novices packed in more information than explanations to experts, emphasizing the information about action that is difficult for novices, and expressing information in multiple ways, using both virtual models created by gestures and visible ones.