About a month ago, I uploaded my first extension for automatic1111 (which is a web front-end for Stable Diffusion) titled “Corridor Crawler Outpainting” on GitHub. It uses Stable Diffusion’s depth-to-image model to create long, AI-generated hallways based on the user’s prompt settings and selected images.
Since then, I’ve added and improved a feature that animates these hallways as GIFs. In the future I plan to add more features for animations, such as changing the zoom direction or saving the animations in different file formats (WEBM, MP4, etc.).
How it Works
- User selects a “Starting Image” to give Stable Diffusion some context for the first few corridor steps. (After enough corridor steps, you won’t be able to see the original Starting Image at all.)
- User selects a “Prefab Image” which Stable Diffusion will use as the blueprint for each step in the hallway. Because this extension should be used with the depth-to-image model, the most important part of the prefab image is how its depth-maps look according to MiDaS.
- User adds other prompt and extension settings (including the “mask” image which separates the prefab image from the starting image) and clicks “Generate”.
- The extension scales down the Starting Image, pastes it into the Prefab Image, and then runs that through Stable Diffusion to modify the Prefab Image.
- Step 4 is repeated for the given number of Corridor Steps, except the Starting Image is replaced with the output of the previous image generation. With enough Corridor Steps, this creates the illusion of an endlessly long hallway.
How the Animations Work
At first, I implemented animations as a simple slideshow of all generated steps (like the art museum example above).
But after trying other methods, I settled on the current implementation which creates smooth animations (like the animation at the very beginning of this post):
- Start with Image1 and Image2, which are two corridor step images.
- Double the scale of Image1 and paste Image2 in the middle of it (which we will call Image3).
- Create animation frames by cropping Image3 and then resizing it.
- Repeat all previous steps for each pair of corridor step images, then save all the frames as a single animation.
Use Cases
Aside from the simple amusement of generating infinite hallways, there are some practical use cases for this extension in game development:
- Level generator for an old-school dungeon crawler RPG (this was my original vision for the project).
- Scrolling background for an endless-runner-style game like Subway Surfers.
- Moving background for a first-person rail shooter game.