Architectural Dramaturgy: A total and endless theatre with multimodal artificial intelligence

Referencing Gropius' 'Total Theatre' (1927) and Kiesler's 'Endless Theatre' (1926), the theatre is a fertile ground for incorporating the latest technologies to prototype a ‘totally’ multimodal and ‘endlessly’ generative form of architecture. Complemented by today’s rapidly foregrounding of Large Language Models and Large Multimodal Models in artificial intelligence within the architecture discipline, the project posits that it is timely to revisit the conceptual cross-fertilisation of theatre and architecture – architectural dramaturgy.

A human-in-the-loop interpretive framework is formulated based on a single source of video data of an actor’s performance of a monologue – the poignant “To be or not to be” soliloquy from Shakespeare’s Hamlet, performed by actor Andrew Scott. In the soliloquy, Hamlet reflects on the existential dilemma of life versus death. Analogous to today’s discussions on creative AI, the soliloquy becomes a metaphor for the ethical and existential considerations surrounding AI’s role in creative processes. Like Hamlet, architects and theatre-makers alike grapple with the impact of AI on their professions, raising timeless questions about innovation, tradition, and the evolving relationship between human creativity and AI. A combination of a text-image-to-image model, a text-image-to-video model, an image inpainting model, and an image-to-3D model, is used on a video recording and the performed text. A dataset of Scott’s Hamlet performance is created, combining the soliloquy’s text with extracted video frames. The text is analyzed for narrative and emotion, then broken into lines with spatial qualities assigned. Each frame serves as an image prompt, generating images of Scott’s postures within imagined spaces, which then generate 4-second videos compiled into a 1:59 film with Scott’s original audio. 3D spaces are generated from the images, then an actor performs the same soliloquy in VR. The actor adapts delivery to the environment, revealing shifts in interpretation with every VR environment. With the deliberate bracketing of site, programs and other specificities typical of an architecture project, the research demonstrates how concepts borrowed from theatre when layered with multimodal AI could extend the discipline’s longstanding conception of a total and endless architecture.