Back to Stop building AI wrappers. Start building agent harnesses., with Jon Noronha.
Why Can Slowing Users Down Raise Paid Conversion?
Jon Noronha says slowing people down raised paid conversion. Gamma tested a fast mode that skipped the outline and the upfront questions, on the idea that less friction would get people to a useful result sooner. He wanted that test to win. It lost. Seeing the outline, choosing the high-level arc of the story, and answering questions led to much higher satisfaction ratings. People who got through the slower path felt more invested in the output, did more with it, and converted to paying at a higher rate. A test is not only one answer. Gamma still shipped a version of fast mode for brand-new users on mobile, where typing a stack of answers is a poor fit.
The same testing habit is constant. Gamma probably has 10 or 20 experiments running at once, and many last less than a day. On the way to the recording, the morning after a new small model came out, he started eight tests for cheap jobs such as picking a template, writing an outline, making a small edit on a slide, or revising an outline. He planned to check them afterward and expected to cancel about four. A common result is that a model at half the cost matches the more expensive one. Sometimes the large model is still required.
Stop building AI wrappers. Start building agent harnesses. ·
From the transcript
we're very lucky to have a big user base where I can start a test in an hour and get data two hours later on how it's doing. So one of the things I learned, because I'm a co-founder of a design agency called LCA, and we actually worked on a project together at Gamma. And one of the things I learned working with you guys was you guys run a lot of experiments.
Yeah. I'm curious about experiment, you know, maybe one experiment that the results surprised you because you guys are just, it feels like you guys are running a lot, a lot at a time. We run a huge number of experiments. At any given time, we probably have 10 or 20 running. And we're cycling through... Many of the experiments last less than a day.