CALLAS Project

From Gretadoc

Interactive Storyteller is done in collaboration with the BBC and Reading University. In this scenario a computer application presents a story content and displays a sequence of still images (with a high emotional impact) related to the story. A web site at BBC is dedicated to public presentations of news items. Photos accompanied by explicative captions relate an emotional content. A particular story reports images on the Schezuan earthquake.

In the Interactive Storyteller application the interaction takes the form of a guided conversation between the virtual agent and the user. The ECA is a storyteller. Its role is to interact with the user. It first asks the user to comment each displayed image, then provides some explanation. To enhance the emotional experience of the user, the agent shows affective empathy toward the user showing its engagement in the story. The user expresses its opinion about the images. In the background its speech and gestures are analysed by a system to detect user's emotional states. For this purpose the MKS Keyword Spotting and Gesture Expressivity Recognition software provided by CALLAS partners are used. Next, the agent explains what the image is about. It uses various nonverbal signals like emotional facial expressions and gestures to emphasize the message and guide the emotional reactions of the user. Detected information about user’s affective state is used by the system to influence the agent’s affective behaviour. When the ECA begins conveying the scene si+1 it expresses the same affective state as measured from the user in the previous segment of the story si, and then, throughout the duration of the scene, it gradually changes its expressed emotional state to the si+1 target affective state.