Collascope Supporting Serendipitous Asset Exploration for Collage-Based Storytelling

Jiayi Zhou1, Longji Huang2, Lvmin Zhang3, Yun Wang4, Zeyu Wang1,2, Maneesh Agrawala3, Huamin Qu1, Anyi Rao1

1 HKUST 2 HKUST (GZ) 3 Stanford University 4 Microsoft Research

Comparison of conventional keyword and image search with Collascope, which structures a story into Scene Parts and supports concept and visual associations while composing a collage.
Collascope keeps story intent, asset exploration, and collage composition connected.

Story collaging involves continual negotiation among evolving narrative intent, available assets, and visual composition. Unlike conventional search workflows, which require creators to fragment stories into disconnected queries, Collascope structures evolving intent through Scene Parts and expands exploration through Concept Association and Cutout Association. Unexpected and imperfect results can then serve as materials for revising the story and enriching the collage.

Method

Collascope workflow from photo collection preprocessing and element-group retrieval, through concept and cutout association, to collage composition and an aligned story description.
The Collascope workflow connects preprocessing, element-group retrieval, asset exploration, and collage composition.

Collascope turns an evolving story description into structured Scene Parts and visual elements, then supports context-aware Concept Association and attribute-aware Cutout Association. Selected assets flow directly into collage composition, while the aligned story description evolves with the scene.

Collage Results

Collage stories created with Collascope in our user study.

User Study Findings

We evaluated Collascope in a within-subject study against a conventional search baseline. Beyond stronger perceived support for exploration and expression, Scene Parts helped participants externalize emerging story structures and keep track of what they had already explored.

Study cases comparing Baseline and Collascope collages, with annotations showing satisfying search strategies and serendipitous discoveries for participants P2, P4, P5, P7, P9, and P11.
Working with unexpected and unavailable assets. When desired assets were unavailable, participants used approximation, substitution, attribute composition, and semantic cues to continue developing their collages. Associated assets also prompted narrative revision: a faceless hat led P2 to replace a human head with a fish head; a street-corner setting redirected P11 beyond the original plant-box motif; and P7 retained a floating boat and butterflies to enrich the scene.
Normalized session timelines for P2, P4, and P5 showing query editing, asset exploration, concept and cutout association, and collage composition across Baseline and Collascope conditions.

Three recurrent ways of using Association

Participants incorporated Association differently depending on whether exploration was guided by a clear goal, an evolving story, or visual intuition.

Goal-oriented P5 P6 P9
Used Association selectively while following a clear target.
Story-driven P3 P4 P8 P10 P11 P12
Alternated manual concepts and Association as the narrative changed.
Visual-driven P1 P2 P7
Repeatedly followed related cutouts beyond the initial search target.

Demo Video