Nonlinear Storytelling

Das Erzählen von Geschichten ist eine Form der Datenstrukturierung, welche in der natürlichen Umgebung des Menschen sehr oft zur Kommunikation zwischen Menschen eingesetzt wird. Aus diesem Grund wird das Geschichtenerzählen auch in der Informatik als Datenstrukturierung und Methode der Inhaltspräsentation genutzt. Leider gehen in diesem Zusammenhang die interaktiven Komponenten des Geschichtenerzählens zumeist verloren - der Zuhörer kann den Erzähler und damit die Erzählung nur noch unzureichend beeinflussen. Um den oben benannten Mißstand zu überwinden wird das Konzept der interaktiven Narration von kontinuierlichen Medien vorgeschlagen. Dem Konzept liegt die Idee einer Modellierung der Geschichte basierend auf generativen (animierten) Komponenten und selektiven Komponenten (Video Clips) zugrunde. Diese Komponenten werden zur Grundlage der interaktiven Narration, dies durch Interaktionsformen zur Unterstützung der direkten Handlung des Publikums auf den Video Clips, sowie durch Interaktionsformen zur konversationalen Beeinflussung des Erzählers, und einer Einbindung der Interaktion in die Geschichtenmodellierung. Aufbauend auf einer Diskussion der Anwendungsgebiete sowie der grundlegenden Mechanismen des interaktiven Geschichtenerzählens werden in der direkten Handlung des Publikums und in der konversationalen Beinflussung des Erzählers die wichtigsten Komponenten des interaktiven Geschichtenerzählens erkannt. Mit diesem Wissen stellt diese Arbeit ein kombiniertes Konzept des Geschichtenerzählens sowie ein kombiniertes Konzept der Interaktion auf den Geschichten auf. Das in dieser Arbeit erstellte Konzept wird über eine Nonlinear Storytelling API für die Anwendungsprogrammierung zugänglich gemacht und anhand von verschiedenen Projekten getestet und bewertet. Die Bewertung erfolgt dabei über einen systematisch aufgestellten Anforderungskatalog, mit Hilfe dessen die grundlegenden Annahmen bezüglich der nichtlinearen Narration mit den Ergebnissen abgeglichen werden können.

Introduction: The telling of stories is a form of information presentation commonly used by people all over the world, within every period of time. For this reason, it seems to be a matter of course to use this form of presentation and data structuring in computer science. Unfortunately, the interactive components of telling a story get lost, if one tries to tell stories automatically. The audience is not able to manipulate the narrator or the story in an adequate way. To overcome this situation, an approach to interactive narration of continuous media (like video clips, animation or VR) is developed The general idea is focused on the abstract modelling of a story, with the story based on selective fragments (video clips) and generative fragments (animation), combined in scenes. The concrete formulation of the story is enhanced with direct manipulative interaction possibilities on the fragments as well as with conversational interaction possibilities on the fragments and the narrator, the concrete fragments and interaction possibilities are interpreted on the abstract level of story forming and give therefore the basis for interactive narration of stories. The concepts of this work are accessible for application programmers via the so called Nonlinear Storytelling API, the concepts are evaluated and verified in several project applications, namely the MAP, EMBASSI, DIVA, IzA and GEIST projects. The verification was done, based on a systematically compiled catalogue of requirements. With the help of this catalogue the projects were evaluated. In result, the basic assumptions of nonlinear narration were able to be verified in this work. The following paragraphs give some details on the story processing, conversational interaction and direct manipulation (video-hyperlinks) parts of this work. Story Processing: Propp, a Russian Structuralist, wrote the book ‘Morphology of the folk tale’, introducing a semiotic approach for story understanding and story generation. This days (1928) his work was ground-breaking – and still in our days, his work is the basement of many approaches for automated storytelling. The basics of Propp’s approach are quickly explained: He identifies the smallest functions of story, with every function is a basic unit of abstract action, done by abstract actors. He showed that every action, done by any actor, has a different meaning in regard to its (temporal) position in a story. (Example: Given an opponent, a hero, a princess, a father (the king). Those abstract roles could be used to identify any real actors –have a look at ‘Star Wars IV’: Darth Vader, Luke Skywalker, Princess Lea, Obi Wan. Typically, a story starts with an initial action – the princess getting abducted by the opponent. Another look at ‘Star Wars IV’: As a start, Lea gets abducted by Darth Vader.) More than that, the composition of the functions depends on specific rules, these rules are universal for tales. (As one can see, these rules are used for most of the stories you enjoy in TV or at the movies.) With these rules it is possible to generate new stories, based on given story fragments. Until this point, there is no interaction involved. We have expanded the approach to abstract interactions – introducing the concept that actions may or may not fail. With this concept, the functions are expanded to a polymorphism – e.g., the function ‘reaction of the hero’ (so called function F) is expanded to the two possibilities of ‘right reaction of the hero’ (so called function F+) and ‘wrong reaction of the hero’ (so called Function F-). As one can see, this kind of interaction is very abstract – just like the semiotic approach, for this reason, it fits so well into that approach. Of course, the right or wrong result of scene interaction is simplified for this example – the approach offers possibilities to define all kind of abstract interaction results. The story fragments (morphologic functions), are adapted in each case to the needs of the several projects. We use the expanded approach to concatenate scenes to a story – the scenes are of course predefined, with a given set of actors and actions, annotated with semiotic knowledge. We use the abstract semiotic knowledge to process the story line, this without any knowledge about the ‘real’ actions and interactions within a scene. The only thing we get out of a scene is the abstract interaction knowledge – therefore, the polymorphic function (e.g. F) identified with the scene, morphed to one of its specializations (e.g. F+ or F-). Scene Interaction: Scene interaction is done in a conversational way via virtual actors and/or virtual narrators, or direct manipulative via video hyperlinks within video clips. The video hyperlink approach to video interaction uses an advanced version of the common hypermedia metaphor – the temporal video hyperlink. The hypermedia metaphor is expanded to the properties of the medium – it gets s start, a duration and an end. This characteristic is visible for the user while the hyperlink is active – therefore, the user always has the knowledge about the start, the current position, the time remaining until the end and the end of a hyperlink. The presentation of the hyperlink is intramedial (within the same medium) – an annotation of the video’s graphics is done within the video graphics, an annotation of the video’s acoustic is done within the video’s audio channel. Even the interaction of the user happens intramedial – graphical hyperlinks are activated via graphic interaction, acoustic hyperlinks are activated via the voice of the user. Integrated approach: The approach, presented in this work, integrates several aspects of virtual storytelling. Within the scene level, presentation and interaction is done using (advanced) approaches of Hyperstories and Emergent Narrative. Within the story forming level, presentation and interaction is organized in a semiotic way, as so called Controlled Storytelling. Conclusion: This work presents an interactive storytelling approach, based on adapted literary concepts for an interactive high level story forming algorithm, as well as on abstract and concrete new interaction concepts for the concrete formulation of the story in scenes. To the audience, the approach gives back the possibilities of manipulating a story while experiencing the story – and closes the circle from the beginning of verbal storytellig to our times of multimedia-enhanced storytelling in the information society.