Alex Zhang

Alex Zhang says a recursive language model, an RLM, is a harness whose only tool is code. He wishes that definition had been in the original paper. The code can call the model again as a tool, and the other tools are functions in that same code. The context stays in memory inside the code environment, which can be a file system. In Prime Agent, the trajectory stays on disk even after compaction, so the model can still read the original context, and the tools run in a Python REPL or a Bash REPL. He calls that a primitive abstraction.

Most harnesses differ on that point. Context offloading is not done that way, so the working context does not stay in the code environment the way an RLM keeps

Alex Zhang says one person who knows a problem can erase a huge token search. The people who can look at a problem and see how to solve it are also the people who can use AI on it, because they act as a strong verifier. Shawn Wang's example is diagramming the code and refusing to continue until every part is understood. Agent swarms, given enough compute, can explore a lot of solutions. The burn might be a hundred billion tokens, or a trillion. Someone who already understands the problem can uncover a step for the model that wipes out that spend.

What that trend will be is not clear. There are still many problems to solve, and the work cannot afford to throw the maximum compute at every one

Alex Zhang says academia is for big bets that an industry lab will not take. Doing that work in academia is hard. Quiet-STaR, ReAct, recursive language models, and SWE-bench are not a GPT-6 or Astra release, the kind of launch where everyone says they will use it now. Academia cannot afford that, at least right now. A PhD student can work on almost anything. If that freedom goes to problems nobody cares about, or to an idea someone already thought of and does not use, the research will not be that interesting.

The alternative he gives is an industry lab, which has the resources and the talent. A PhD program has fewer resources and fewer people around. A lot of the bets fail, and that is nor

Alex Zhang says a puzzled reaction is a good sign for a paper, because it means people are not thinking about what the work is for. Someone may work on a harness because a clear goal is already shaped around the models that exist today. The recursive language model paper is a very simple idea. When it came out, people asked what the purpose was, called it too cool, and said it was just sub-agents.

He treats that response as a sign the point was missed. SWE-bench had the same shape. Ofir tells it as a story: nobody cared, and people called the task impossible, until Devin came out and the field decided it was something to hill-climb. A simple paper can look pointless and still name a goal th

Alex Zhang chooses which research idea to chase from about 10 or 15 different ideas he wants to do, and most of them are bad. He allows that an idea from someone who reaches out might actually be bad, yet it looks interesting. Some time goes into looking at it. If there is actually something there, the next few weeks go to really pursuing it.

He loves the PhD because there are times when he is just thinking about problems on a run or while playing tennis. He guesses he is not working then, and the time is the most fun. When he is really convicted about something, he drops everything and spends all his time thinking and working on the problem. Once he can just run experiments, the work becom

Alex Zhang answers a cold email when the writer shows up with a strong opinion, and he treats “I really like RLMs, I wanna work together” as a bad reach-out. He has nothing in particular he feels he needs to work on with someone, unless it is with a company for compute or he wants to talk an idea through with other people. He is never opposed to working on ideas with other people.

Undergrads and other students reach out often. The worst version of the email is “can I pick your brain?” He tells them to read the paper. People write back that they read the paper, then name recursive language models or prime agent, a self-improving RLM harness.

He likes people who are opinionated even when the

Alex Zhang agrees, in some sense, that the capability overhang is real. Even if progress paused, the current state of models still leaves a lot of impact available. The case he is presenting is the easiest form of the idea. Models have jagged intelligence on a lot of things. They are disproportionately good at coding and math, and the abilities can be translated to many other things.

Someone who was an IMO gold can be applied to a lot of different problem-solving domains and figure them out. He does not know whether the pattern applies to models. In GPU code optimization, one question is whether a model as good as Astra is now, with all of the GPU programming data taken out, would still opt

Alex Zhang says Sakana is different from OpenAI because it is a research lab, closer to a PhD group than a company with a model everyone uses. The team is very smart and branched off from an earlier group at GDM that worked on open-ended evolutionary research. While he was there, Sakana did not have a big competitor model in wide use, and it does not operate in the same space as OpenAI or Anthropic.

He still likes the lab after the GPU kernel mishap that people cannot forgive. Criticism of the AI scientist work sits outside his lane, so he offers no take on it. David Ha is really smart. The market in Japan is a little different, and who Sakana targets is slightly different from what people

Shawn Wang says the amount OpenAI has spent on agents is semi-public. He puts the run at 10,000 agents over 88 hours, using 130 billion output tokens in that run. His estimate for that output is about 40 million dollars in public pricing, and he calls the total not that much.

Alex Zhang says the figure was less than he expected. Vibhu Sapra says those 130 billion tokens are the output for the final result, while a lot of context was still being passed around. The total agent messages came to more than double the 130 billion output tokens, so the traffic between agents sat above the finished output.

Alex Zhang describes Prime Agent as a harness on top of Pi Mono, the minimalist harness he uses as the reference for other coding harnesses. Prime Agent is Pi with one hard limit: IPython is the only tool the model can call directly.

Every other tool loads in as a Python module or a bash script the model can run. The setup puts the core code RLM abstraction on top of Pi. A continual harness also lives inside the IPython kernel as a tool. The principle is to decide which parts of the harness the harness itself may change: its own skills, the subagents available to it, and the system prompt sent to the model. Everything else in Prime Agent is standard.

Newer frontier models work well inside

Alex Zhang says the Harness Tax paper, which he thinks is by Arena, confirms a prior he already held: most harness choices do not matter because the harnesses are the same.

He thinks Grokbot is quite different, from his understanding, from how some other harnesses were designed, and he likes the difference. He is pretty sure Anthropic or OpenAI train exclusively on their own harnesses, and he assumes they do not train on a competitor harness, because he does not know why they would. Open models show the same requirement. Old Gemmas were notoriously bad at harness use, so a strong model still has to be trained in a harness. As models get smarter, the distinction becomes less important. A mod

Alex Zhang treats a harness as the choices that let a language model solve a task, and raises whether a language model can just do the task on its own. A looped transformer can be modeled as a harness by looping over the model, with the difference that the loop is not decoding. The same model architecture choice has stayed in place for a long time, while training now runs around harness rollouts.

Training a language model to act inside a harness means a really long rollout, and whether that rollout uses multiple agents is left open. The methods in use are awkward and hacky. The blog asks whether harness design can do something more meaningful than tool calls alone, with a way to grep throug

Alex Zhang's benchmark on language models playing video games asks whether vision language models are good enough at plugging into games with the latency constraint included. The benchmark came out right after Claude Plays Pokemon. About two years later, he calls the release ancient.

The numbers are very outdated. A lot of the models are very different, and some folks are running newer models on the games. The fascination behind the work is whether an intelligent system can play new games. The game examples also sit with a separate point of excitement: a unique way to capture language and understanding in a very, very fast model.

The premise keeps the latency constraint inside the test of

Alex Zhang says matrix multiplication makes it very easy to compute a speed of light estimate of the fastest GPU kernel, while the same kind of estimate is not easy once the problem is more complex.

The estimate moves with where the data lives at the start. All the data might start on the CPU, or it might start in DRAM on the GPU, and the transfers change the numbers slightly. The estimate assumes perfect overlapping and transfer of data. In a lot of cases it is not clear whether the theoretical number can even be hit, because some bottleneck may be impossible to get around. Kernels people write are often not even close. They are not nearly close enough to the speed of light number to be me

Alex Zhang says almost all recent solutions on the GPU Mode leaderboard are generated by AI, including work from people with no background in kernel writing. The only kernel in the top 10 that stayed stable in an actual end-to-end system came from a regular GPU Mode member who steered the model.

On a lot of the recent GPU Mode problems, almost all the solutions are AI generated. A regular member, known for a long time as a very strong GPU kernel writer, used AI to help with the solutions. For the most part he prompted the model and moved it in certain directions. The kernel was basically the only one in the top 10 that stayed stable in actual end-to-end systems. GPU kernels have a verificat

Anthony Zhang says he loves building Vinovest in Los Angeles for the life as much as the hiring pool. He names the beach, the weather, Hollywood, fashion, and real estate, and he says the talent is not one industry deep. Founding teammates had worked at Snapchat, at ByteDance, the parent company of TikTok, and at Headspace. He thinks people in the city are generally happier. His picture of a day is twelve or thirteen hours of hard work and then a separate life with friends, which he says is available there in a way that keeps the job from becoming the whole day.