How Does Figma Help Codex Build Motion?

Marvin Schwaibold uses Figma as a scratch pad so a broad sketch can carry motion he cannot put into words, then he gives the idea to Codex. He thinks Codex interprets the image the way he interpreted it. Codex understood what he was doing and how he was thinking about the motion.

On a side project over a weekend, he wanted a two by three overview of grass studies and videos he took. Opening any video and dragging down closes it, and the interaction works the same in the simulator. He described it to AI while working in Codex, and Codex built it, but the close was not interruptible. Every time he clicked an asset and pulled it down, he had to wait until the asset was all the way back in position and sat down before he could tap another image. The pattern has to stay very responsive very quickly, or the interface gets annoying very fast. In Figma he drew rudimentary simple boxes. Click the box and red comes up, ready to swipe away. While the swipe is still moving, a click on blue brings blue up, and green works the same way.

The scratch pad makes decompiling an idea easier for him, and Codex decompiles it easily when Codex sees the picture. Articulating the motion by text or voice failed. Drawing it out worked. Codex and current AI software are extremely capable of creating anything, as long as he knows the work, takes a harder concept such as a shader, breaks it into small components, understands each component, and puts the pieces back together.