Annotating images through adaptation: an integrated text authoring and illustration framework

Abstract

This paper presents concepts to support authors illustrating their texts. Our approach incorporates content- and feature-based retrieval techniques in multimedia databases containing 2D images and 3D models. Moreover, we provide tools (i) to adapt the retrieval results to contextual requirements and (ii) to ease their integration into target documents. For 3D models the adaptation comprises aspects of the image composition (i. e., the selection of an appropriate view and the spatial arrangement of visual elements) and the selection of appropriate parameters for the rendering process. In addition, secondary elements (e. g., textual annotations or associated visualizations) are smoothly integrated into adapted 2D or 3D illustrations. These secondary elements reveal details about the semantic content of illustrations and author’s communicative intentions. They can ease the retrieval, reuse, and adaptation of illustrations in multimedia databases and are explicitly stored in conjunction with the adapted illustrations. Moreover, we developed a novel technique to support the mental reconstruction of complex spatial configurations by shape icons. With this illustration technique, shape properties of salient objects can be conveyed using abstractshaped models. We present retrieval techniques to determine appropriate 3D models to be displayed for shape icons. These shape icons along with the other secondary elements are smoothly integrated into the illustration that can be interactively explored by the user.