Anastasios Angelopoulos
Related people
Episodes
Notes
Grit · The AI Race Has a Leaderboard | Arena CEO Anastasios Angelopoulos
-
Why Is a Chinese Model Number One on Arena's Web Leaderboard?
Anastasios Angelopoulos says a Chinese model is number one on Arena's web development leaderboard because Chinese labs are doing more than distilling American models. If they are number one and America is number two, distillation cannot be the whole story. Something else is pushing the frontier past the American labs.
Web development is where most software developers work, and that is where most of the economic value is coming from. The result is a phase shift between closed and open models that the industry has not understood. Chinese scientists are showing intelligence and creativity that Americans are not fully appreciating, and more frontier models are on the way. The shift is the start
-
What Does Arena's Leaderboard Measure?
Anastasios Angelopoulos says Arena's leaderboard measures the utility models give real users, not a benchmark a researcher wrote down. People make strange benchmarks because benchmarks attract attention, and because nobody knows how to rank models or which one performs best on a real task.
A pelican-riding-a-bicycle image test took over Twitter for a year. It is saturated now, because generating those pictures became training data. Toy tests still do not speak to real user value. Coding, and finance work such as spreadsheets about which companies to fund, need a score that comes from a person in the middle of the job marking what is good and what is bad.
Arena measures utility by providing
-
What Does Grit Mean When the Mission Comes First?
Anastasios Angelopoulos thinks of grit as people who never give up. Day in and day out they come in with their nose to the grindstone. Even if terrible things are happening around them, or something else is pulling them in another direction, they never lose focus. They always pursue the goal. The work is never about themselves, and they never have any ego about it. Grit means somebody who is always putting the greater mission ahead of themselves and is willing to do anything to get the job done.
Joubin Mirzadegan had been going back over the company for the last couple of weeks. His fiancé wrote the full Teddy Roosevelt man in the arena quote in marker across the entire bathroom mirror. The
-
Did Arena Favor Closed Models Over Open Ones?
Anastasios Angelopoulos says Arena has always been fair to everybody. A coalition of MIT and Stanford looked like a reputable source. Digging into the analyses about Arena at the time showed the claim that the company was favoring closed source models over open source models.
The complaint came from Cohere, and Cohere was about number 70 at the time. People can say what they want about the incentives. It felt like a stab in the back from people who could have been future colleagues if he ever decided to go to Stanford.
It was another difficult experience on top of the personal challenges at the time. It was very hard for him and his wife, and it happened a few months before their wedding.
-
How Long Did Anastasios Angelopoulos's Wife Cry After Her Father Died?
Anastasios Angelopoulos says his wife cried every day for many months after her father died suddenly in Serbia, then every two days maybe six months later, and every four days three months after that. She now cries every so often. She says she remembers her dad and is starting to forget him, including the sound of his voice. Angelopoulos guesses the death was a heart attack. His wife is Serbian. The death came in the early days of getting the company off the ground, during a seed round he had no idea how to run, at the term-sheets-coming stage.
He dropped everything and went to Serbia. His father-in-law was her person. They were very close and similar, and Angelopoulos loved him too. He ask
-
Is One Model Provider Enough for Arena?
Anastasios Angelopoulos says Arena sucks with only one model provider. With two providers, Arena is barely making it. He wants 10 or 50, and three or four would probably do. A slowdown from regulation would be the worst thing for Arena.
American open source is absolutely needed. Regulation might accelerate American open source, and the hope is that regulation does not also slow things down. Few providers is bad for everyone, regardless of Arena's incentives. Nobody wants an oligopoly. The point of a capitalist system is to try to encourage competition.
Winners going to Washington and convincing politicians to create regulations that only incumbents can satisfy is the most anti-competitive
-
What Happens to Money When Labor Splits From Capital?
Joubin Mirzadegan says the real question is what happens to money once labor splits from capital. People have earned money through labor, and that is no longer going to happen. Capitalism stops working for people when capital and labor come apart, so the next problem is what happens to people.
Stock value for companies used to be tied to labor, because labor was the input. As labor detaches from the creation of value, stock-market capital diverges from labor input. The U.S. stock market is worth about $100 trillion, roughly in line with the GDP of the world. Twenty or thirty years ago, the overall value of the stock market was way lower than the GDP of the world.
The decoupling is the crux
-
Why Do People Problems Take More Time at Arena?
Anastasios Angelopoulos loves the chief executive role at Arena because it puts him with people from every kind of work, and because the job is to deal with their problems.
As a PhD student he mostly met technical experts: honest, brilliant people with little sense of how to turn an idea into something people use. He still loves that group, and calls it a tiny slice of the world. The circle now includes people who treat everything as a negotiation, plus lawyers, investors, engineers who turn an idea into a product people love to use, and marketing. The people at Arena are best in class, and they have built generational companies before. People problems take more of the time as time goes on,
-
How Fast Did Arena Reach a Hundred Million in Revenue?
Anastasios Angelopoulos says the evaluation product Arena sells to labs scaled to over a hundred million in revenue in eight months. The public leaderboard never touches money.
Arena runs the public leaderboard as a charity and values every model on it for free, which also works as marketing. The paid work happens while models are still in development. A lab can show up with about 50 checkpoints in a week and ask whether they are good at front end work, full stack work, or tool calls, and where they need to improve. Arena evaluates the checkpoints on data that represents real usage and hands the results back as an insights dashboard. The company had about 15 to 20 people eight months ago an
-
Why Did American Companies Skip Free Frontier Models?
Anastasios Angelopoulos says American companies largely skipped giving frontier models away, because spending $5 billion to train one and then releasing it free sounds like a stupid idea. Chinese companies can make a release like that because corporate dynamics in China are different. In the US the open problem was how to earn back the money spent on training.
People are trying two approaches. One is a license that is partly open source, with revenue thresholds. If a product using the model clears $20 million in revenue, the lab takes a revenue share, and an inference provider that wants to serve the model shares revenue as well. The other approach, becoming more common at Reflection and Mi