Contents:
- 1. Introduction
- 2. Related Work
- 3. Text-Video Alignment
- 4. Story-based search
- 5. Evaluation
- 6. Conclusion
- 7. References
This paper present a novel approach to search for plots in the story- line of structured videos such as TV series. In section 3 the researchers of the Karlsruhe Institute of Technology describe how they achieve this. They align natural language descriptions of the videos, such as plot synopses obtained for instance from Wikipedia or fan-sites, with the corresponding shots in the video. Furthermore, the alignment is enhanced by using timestamps of both the subtitles and the appearance of the characters in the episodes extracted automatically from text and video. After the alignment between shots and sentences is achieved, story-based video search becomes a text search in the plot synopses. The proposed alignment between plot synopses and shots of a video and the story-based retrieval is evaluated on the complete season 5 of the TV series Buffy the Vampire Slayer (22 episodes).The evaluation of the alignment (based on automatic indexing and video analysis) shows that 49% of the shots are assigned to the correct sentence. According to the research team the story-based retrieval, evaluated on 15+ hours of video, shows promising performance. In the future, they intend to improve the alignment by using additional cues such as object detection and scene recognition.
This paper presents a possible solution for searching for plots in the story-line of structured videos such as TV series or episodes. Though still in the research phase, this could become an appraised solution for concept searching via search interfaces to large collections of structured video.