Parallel’s Parag Agrawal: Building a New Web for AI Agents

0:00

Our view at Parallel is that human click data is a bug and agent doing work with search should rely on agent feedback, not human feedback. We believe that these models are really good at compressing information and we can benefit from a lot of the research that have gone into building models and apply it to search indexing and ranking. And so you can now make many, many arguments, and that's the arguments we made back then, that actually now it's way more tractable as a problem because of the existence of agents, not just as a technology, but as a distinct customer.

0:58

Parag, thank you so much for joining us today. We're delighted to have you on the show. For those who don't know, Parag of Twitter CEO fame was the CEO of Twitter before selling it to Elon and is now back on the founder arc. You founded a company called Parallel Web Systems, which is scaling up agentic search for the agentic web. We're very excited to have you here today to talk about the future of search and the future of the internet.

1:24

So let's get started. What's Parallel? Thank you, Sonia, for having me. Thanks, Andrew, for joining us. I'm really excited about this conversation. At Parallels, we're building a bunch of technology in order to allow agents to search and use the web. So just like humans forever have figured out how to use browsers and search engines and clicking and browsing on the web to access information, agents need to do the same things.

1:50

We started Parallels with the bet that agents would do it 1,000x more than humans ever have. And as a result, we need to reinvent the technology that can power search for agents, but also the business models that go alongside it. And that's what we're doing at Parallel. Okay, I want to go back into what makes human and agent search so different.

2:12

But before we get there, you've told us that you're unlearning a lot of the lessons that you learned from running Twitter as you build Parallel. Why is that? Listen, when I was at Twitter in leadership roles, Twitter was a post product market fit, extraordinarily skilled business where your feedback loops were from the hundreds of millions of customers using the product for 30 plus minutes every day.

2:43

Right. In that world, you operate differently than a pre-product market fit company based on the premise that in a few years, a new customer is going to show up on the internet and let's go build technology for the not yet here customer that we are all learning every day and every week. Fantastic. Okay, let's talk about Parallel. I have a question, actually.

3:12

Can I jump in? No, you cannot, Andrew. I'm Andrew. I didn't get an introduction, but I'm also happy to be an inaugural guest on the Training Data podcast. Andrew is making his podcast debut on Training Data. We are honored to have you finally here. Thank you. I feel like all of us in Silicon Valley have a very surface level idea of search.

3:32

Like what is search? Like the job to be done is getting the answer makes sense. But we all know there's crawling and there's an index and there's ranking. But maybe let's take a big step back and explain what is the problem of web search, be it for a human or an agent. And then we can dive into the differences. The problem of web search, and we all know it and experience it, when we want to find something and we do not know where it is on the web, we go to a search engine like Google, and the search engine then hopefully surfaces the answer to us in the most convenient of locations.

4:05

That's the base problem. Now, in order to do this, what is the search engine doing? The search engine is going and crawling the web, which is finding every URL that's out there. trying to read it, trying to organize that information in what might be called an index. So that by having all of this stuff in memory, in one location, you don't have to, once the query comes in, you don't have to spend time loading up pages because you already have them.

4:35

You've already done pre-processing to organize them. And then when the query does come in, You quickly understand the query. You find the most relevant results. And then there are many, many, because you're essentially taking hundreds of billions of pages and narrowing it down to, what, five or 10 or ideally one in terms of what someone is looking for right now.

4:55

So you go through these multiple stages of retrieval and ranking. in order to surface the most relevant result. So that's the broad problem. One way to think about it is it's a billion to billion matching problem, right? So you have hundreds of billions of pages and hundreds of billions of queries over time. And you need to figure out how to match make across these two.

5:18

That sounds like an enormously expensive infrastructure challenge. And I think for the longest time, basically only Google and Bing had, you know, done full web scale crawls and indices. Why did you think about, you know, you as a young company could go off and tackle that problem? It seems like a problem of the giants. So it is expensive in the long term.

5:40

So what's actually interesting is that when I was starting the company three years ago,

5:46

you could imagine a world where one, some of the reasons it was difficult for others to compete, which was not having access to great feedback in terms of, is this a better search result than this?

6:01

To collect that feedback at scale, there was a problem around human ratings. There was a problem around click data. You need to access those. Now, it is, of course, expensive to crawl the entire web and index it. But as you think about what agents and the large models that we now have access to have enabled, it's The ratings data can be now created by experts way more cheaply.

6:27

Our view at Parallel is that human click data is a bug and agent doing work with search should rely on agent feedback, not human feedback. We believe that these models are really good at

6:46

compressing information and we can benefit from a lot of the research that have gone into building models and apply it to search indexing and ranking. And so you can now make many, many arguments. And that's the arguments we made back then that actually now it's way more tractable as a problem because of the existence of agents, not just as in technology, but as a distinct customer.

7:11

And then we figured out a way to go about building this business, which did not require us to go spend all of the money on infrastructure upfront before we could service a customer, right? So if you can paint a path of incrementally being able to build an increasingly larger and more sophisticated index over time as you solve problems for more and more customers.

7:38

That's the insight that actually convinced me that this was a viable problem for us to take on. And how do you make that happen? Because I imagine this is one of those things where customers want full coverage day one. So how do you go about making that happen? When we first launched the product, we did not launch a search product first. We launched a search agent product first.

8:02

Our search agent could go essentially crawl the web after a query arrived. So if you're doing deep research, you have patience to the extent of a minute. And we've had products which sometimes take 10 minutes of research. That's a lot of time to be able to crawl a lot of pages if only you have enough of a map to know how to prioritize crawling, right? So you can make up for shortcomings.

8:27

Like indexes, oftentimes you can think of it as a latency optimization. So if you give up on that dimension, if you're competing with humans, that's why our search agents were competing with the alternative being outsourcing to humans to curate amazing data, right? So we said, it seems like humans sitting on search engines are far way easier to compete with than a search engine on day zero.

8:54

So by building a product that was a search agent to do real work on top of web data, We were able to incrementally go build our index. Well, there's some examples of what people built with your search agents. In the very early days, there were search agents being built with us for doing

9:14

some kinds of insurance underwriting workflows and claims processing workflows. People in sales were doing all kinds of sales data enrichment work. People in finance who would previously and historically go to build a model, collect data by sending it overnight to a set of humans who would curate data that would feed into a modeling exercise they would do, would start using us to do that instead.

9:46

And so we were going initially after replacing where there was outsourced human work on top of web data in order essentially to collect evals, to run agents, to figure out what search for agents should look like in the first place with empirical use cases instead of theoretical evals. I see. You're trading off the crawl for inference time compute effectively.

10:16

Yeah. You guys were started before the term like Neo Lab came into existence and you have, you know, web systems product, but you also have like a bunch of AI people. From the outside, before we met for the first time, like it wasn't obvious to me how much of a web systems company or an AI company that you've built. The more we spent together, the more time we spent together, the more clear it's like By some definitions, this is a new lab.

10:41

Do you want to talk about like where the research side of what you're doing comes in, not just the like infrastructure and system side of what you're doing? So we don't categorize ourselves as a new lab. Well, of course you would not. We, no, because I think, I don't think our output is a model. Like I somehow, maybe my definition is broken.

11:03

My definition of a new lab is an output is a model. I think our output is a complement to a model. What we build is something that multiplies on top of a model. in order to give either, you can call it the model gets better or the agent built with the model gets better and has superpowers, right? So we've always want to be in a place where whenever someone ships a better model, a new lab, for example, if somebody else ships a better model, they have a higher hill to climb for us.

11:33

Somebody ships a better model. They have now unlocked four more use cases where we can be valuable. Now, whether we need to do work that might be framed as research that remains so in that sense we do have to do research but we're also not uh pre-training extraordinary large models in fact our job is to compress things down into tiny ranking models, right? Like I think if you go back to my framing around like this 100 billion to 100 billion matching problem, every query is essentially give me a thousand tokens from

12:14

trillion web pages on the web and make sure that the right thousand tokens like that's the query that we are getting effectively from to our search engine and so now what you have to do and until last week we would give ourselves three seconds to throw a bunch of compute at read time to do it this week last week we shipped a product which now does it in 200 milliseconds right so you have now that much time to go figure out how to pick those right so you will do We're trying to organize information in memory across the memory hierarchy in a way that we can access it fast.

12:47

We're trying to figure out how to train what model

12:51

to select the best thousand tokens. And so there's a large amount of research that goes into figuring out how to take a pre-trained model of some kind, adapt its architecture for this new problem, and allocate effectively compute in a way that produces the best output with the limited compute slash latency budget. So a lot of these use cases seem like the deep research kind of shaped use case.

13:19

And when I think deep research, at least in the earlier incarnations, it was effectively like an agentic loop with the model reasoning and then basically just calling a search engine, calling Google or some proxy to Google. Why is that insufficient in your eyes versus what I'm hearing from you is agentic search is kind of a net new capability?

13:41

Calling Google for every query in a deep research thing, it gets you somewhere, right? Okay. If you use parallel search, you will, for the most part, use under half the tokens in your agent. It will become more accurate and be faster into it.

14:05

And so you can think of it as, but every time, if you use only half the tokens, if your model is context limited or memory limited, you can now do

14:18

More problems, you can do the same problems cheaper or faster. So everything to me, when you have a infinite appetite for information and relevant information for all kinds of work, it's at its core an optimization problem around quality, cost and latency. Every model advancement is about how do you squeeze out more intelligence and then how do you distill it down into keep most of it at a tenth of the cost, right? And that applies to search as well.

14:53

And every time you can produce all of the signal with less noise in your search results to give to a model,

15:01

You now give the model the ability to do more. And so a lot of the bet here is intelligent compute allocation across the model layer and the agent layer down to the search layer. The other interesting thing to observe is the interface actually changes when you think about serving agents versus humans. Humans rely on keyword search. With agents, we've had to innovate quite a lot on the, what does an agent tell our search engine? The more we know, the better we can do.

15:34

Interesting. So humans can only hold a few words in memory, basically, versus an agent's. No, I think we can. We're just lazy.

15:44

We can have a conversation. We can hold a lot of words. But when you start typing, we want to tell Google two incomplete words with a typo in there and hope for the best. Or rely on some form of a drop-down autocomplete thing. to avoid typing those three words in the first place. So we're fundamentally lazy. Turns out like models less so.

16:08

Fewer typos, better specified queries, perhaps longer queries, less for the search engine to guess what the agent might want. So you get to solve a different class of problems. One other thing about humans is I feel like we know how to decipher the, you know, pre AI slop that populates many of the top ranking Google results, you know, for best product for X, Y, and Z.

16:36

And you get all the affiliate advertising things.

16:39

and yet obviously they rank highly. I'm very susceptible to it. Are you? Yeah, I'm like, I feel like I have this incredible, I can just see it coming a mile away. How do agents deal with things like that, right? Where these are, you know, well-trafficked pages, they have seemingly good answers, but you just know they're not real. Perhaps with parallel agents, you have to deal a little bit less with that.

17:00

Let me tell you why those pages exist in the first place, right? Let's work through a simple example. If you ask for a public company's most recent financials, like just the headline revenue number, we can sit here and know that there exists an authoritative filing with the SEC, which has that number, perhaps on page 73 of a PDF. But that is the authoritative number, right? Now imagine Google decided, like, I care about authority.

17:33

And whenever you ask that query, that's the first result you see, right? And then you click that. This PDF takes what, like three and a half seconds to load instead of the second that another page does. You're already frustrated. And then you see a earnings page. That's the alternative. But here you have to now grab your way to page 77 to find your answer.

17:58

we're lazy, we're not gonna do the work, right? So now there exists an entire class of content on the web, which is like, okay, this information is needed by a lot of people. It is worth putting it on a page that loads fast, where this information is above the fold, All of you can go there. It's 99.99% right. So you're not like so skeptical when you go there that this is 100% wrong.

18:30

And it's added real value in the process because it identified out of the 300 page earnings report to NT bits of information that should be above the fold. And so you can call it pre-AI human slop. Or you can call it catering to a lazy human and being successful at SEO, right? The good news is with agents, we're not making the agent click around and fumble around and grab a PDF, right? We're taking a excerpt from the most authoritative place on the web and trying to bring it to the agent's context window.

19:10

And so we aren't forced into this situation weird trade-off and this trade-off existed with humans in the first place because like with with standard browsers and protocols and everything we just didn't figure out how to have us point exactly consistently across everywhere on the web to the exact right highlighted tiny paragraph right with agents we get to bring that to the agent's context and then let it figure out what's next Can you walk us through what actually happens when one of your developers sends a query off to your agentic search API? It's, in some sense, pretty standard.

19:48

So we run some models to figure out what this query is and enrich it to figure out how it will flow into the rest of our system. We have a bunch of indexes which organize different subsets of the web in different ways. And so the first layer will essentially craft queries for each of these different systems.

20:08

One of these systems perhaps is our sort of big index. One of these is perhaps our fresh index. One of these is perhaps like you can, some people will describe it like a knowledge graph. Some people will describe it like a structured index. There are a bunch of these, right? So you're now deciding which ones this query needs to go to. You're figuring out what is the query rewrite for each of these.

20:31

Then each of these has like a big retrieval layer. And then a ranking layer and then more ranking layers. So you're going to try to boil down tens, hundreds of billions of URLs or documents

20:45

down into thousands, tens of thousands down into specific excerpts and paragraphs in those tens of thousands. with more and more bigger models running at each stage with different architectures, pulling more features to ultimately get down to here are the thousand tokens I want to bring back to this AI, which has the highest signal. And now if you look at our various versions of our search API, They just throw for different latency and cost constraints, different amounts of compute at various points in this journey to hit those limits.

21:27

So in the abstract, it's simple, right? What's interesting is what each of these models that I described, how you curate and collect the training data for them, Right. How you build those models and optimize them, the index itself, how you use the the memory hierarchy to store it, to be able to hit certain cost quality latency thresholds.

21:55

What's the North Star from a quality perspective? Like, you know, on Google, there's that, you know, did you get the result you wanted in the first three answers or something? What is the equivalent North Star for you? So I think of, I don't know if I'm right on this, but like my take is that a billion to billion matching problem is a forever problem.

22:12

And so the real question is at what point incremental optimization isn't worth the squeeze.

22:21

So I don't think there is a thing as like, okay, we're done on improving this thing. The question is at some point it's going to get harder and harder to improve this thing

22:32

and it just won't be worth it. Right. But I'm hoping that we don't get that actually, because like, if you think of what we're doing with AI, we will have more intelligence that gets cheaper every few months.

22:50

And as a result of it, it will come down to.

22:54

having great models which are cheap, having great information, matchmaking across a need and all of the information available to you, whether it's your own or on the web, and doing something unique and differentiated with it to produce more knowledge, right? And anytime you can do something 20% better than somebody else, that might give you an edge.

23:18

So why wouldn't you? So in the super AGI build worldview, it feels like if you can push on quality across web search and in the model layer, why wouldn't you? I've heard a point of view that this is so fundamental to the model companies that they're just going to own it. And in part because as they're collecting data for pre-training, that's a very, very expensive infrastructure exercise.

23:47

That kind of is your source of truth for the crawl. What do you think of that? I don't see empiric data on the ground to support that view. To build a fresh web index,

24:00

I don't think that crawl is particularly useful.

24:04

And let me frame why. So if you think of what we are building, we're building a complement to models. We like to crawl things that

24:16

people don't like to crawl for pre-training. Because if the model already was trained on it, it's not useful. Model companies for the training aren't patient enough to go in a completionist way, try to wait for really slow random JavaScript to load. Because the number of tokens you get per amount of compute you throw at it is like one order of magnitude.

24:42

So two order of magnitude, too bad. And so like, is it worth the extra effort to get these tokens? For us, it is because we're completionist, right? For them, it's like, I'll take X trillion tokens. So that's one. Now, I do think this is a core part of every agent, right? My worldview is that if you're buying LLMs for doing work, for nine out of 10 use cases, you will want them to have access to the web and great search infra optimizing for agents.

25:22

So it is a real adjacency for all kinds of LLM inference. And that supports your view that model companies should have the best in class infra for it. So now they can build it or they can buy it. And that's the conversation. And we will see who builds and who buys and who partners and how things evolve. Are you partnering with any of the model companies that you can share? We want to.

25:53

We will. I can't share anything on that. We did announce, and I don't know your definitions of model companies. We announced today, actually, that we are working with Google Cloud to be a

26:10

search and grounding provider for their enterprise agent APIs. So if you think of grounding Gemini models or other models available on GCP when you build agents on GCP or chat apps on GCP or do any other inference with LLMs on GCP. When you attach web search to it,

26:31

your options are Google search or parallel search. And parallel search is product integrated. The integration is optimized. We've spent time with technical teams and training teams and product teams and commercial teams to make sure that when people use Gemini models with parallel, they get exceptional and great results. So yes, there are these partnerships now emerging.

27:02

I bet that there'll be several of these. They will all look somewhat unique. That's a big deal from the search king. Congratulations. Google's the original Neo Lab. I don't know how to frame Google as a model lab versus a hyperscaler. And I don't know what precise lessons to learn from this one, whether it applies to other labs or to... hyperscalers.

27:27

And so we will see. The way that I use voice agents and I need to get a reservation for dinner at night, what restaurant should I go to? And then that request gets fulfilled by the agent. When we talk to a lot of the parallel customers, there is this you know, background agent, whether the monitor product, these agents that sort of are always watching the world or watching the web.

27:49

And when something happens, then they go off and take actions and do something with it. It might be worth, because if we think about what does a thousand X more mean, there's like the depth of research. And then there's just like the, you know, what is actually initiating the tasks? Is it a human initiating the search or is it an agent itself? You want to talk about that dynamic a little bit? Yeah, so there's a bunch of dimensions here.

28:10

Let's go back to for a moment on search agents, right? If you run a typical search agent, and even without doing deep research, it'll do somewhere between five to 20 searches, even if it answers within a few seconds, which is why not, right? So already, if you transition from using a, chat gpt medium or i don't know all the categories but like somewhere not on the high tier like instant low medium every time you write a prompt to it in chat gpt it will do five to ten searches as you dial it up it'll do hundreds and thousands of searches so one interesting thing to observe is like a human action to a multiplier on number of searches that happened right so just by

29:02

using a AI app, you're kind of multiplying your way to perhaps one order of magnitude more searches. Now, a lot of our initial takes on the product and the market were to go after bigger multipliers than even that. So we were much more interested when you said, I have a portfolio of 10,000 small businesses where I have given out credit to.

29:34

For all of them, every month, I have this human process that runs to feel out how my risk is going up or down. Now can we, and it relies on a bunch of web data. We're trying to use agents for doing this. So here a developer is effectively, the multiplier there is hundreds of thousands or a million in terms of the number of web searches that happen because a human goes and programs that instead of now doing this process every month, We can do it every week, right? So doing a lot of searches when this agent runs every week to create a dashboard on the portfolio and a collection of action items that somebody needs to look at, right?

30:18

Now you can go to another example, which is even more interesting. I know, do you use something to do meeting prep documents for you all that's an agent? There's a Sequoia one. There's a Sequoia agent. Yeah, yeah. I was going to say James Flynn, but he's one of our great young guys. So I use

30:39

Notions agent, and you can build custom agents which look into all the internal data that

30:48

we have at Parallel, plus all of the web data using Parallel's APIs to... Create meeting prep docs. One time I went and created one prompt to build this custom agent. Now it does tens and hundreds of web searches for every meeting I have. Every time I build a new agent for a new use case, that keeps multiplying. So I think the path to these background agents doing more and more and more work all the time for us, it's only going to be bounded by value versus spend.

31:23

You know, like I don't think it's rational right now. Like I'm probably spending more on it than I should be, but it's not too much. So I don't care yet. So there'll be some rationalization in all of these agents. But I think we will deploy background agents to the extent that there is incremental value in doing that compute. And the same thing applies to searches.

31:44

All of these background agents would do a bunch of searches. And so our first set of products were really obsessively focused on these, in part because we were building, growing the index, and we decided that our company was based on three dimensions, quality, cost, latency. And for the first couple of years, we said, let's focus, let's ignore latency, and let's just nail the other two.

32:10

because optimizing systems, distilling to smaller models is much more known art than unknown research, right? So once we achieved the best quality search and search agent products at every price point, We started working on latency, and that's what we shipped with a product we call TurboNow. It is the fastest, highest quality agentic web search on the market by a lot.

32:43

Do you think there are more agentic queries than human queries on the web now? I don't think yet. Just given some of the multipliers you mentioned and given the background agents, it seems... I don't have to remind you, but we are early. We are very early in agent adoption. You go step outside of our bubble, people haven't heard the word fable.

33:06

So we are very, very early. I think there are now people like me who are probably operating at the

33:16

thousand X level. I don't know. What do you think? How many Google searches a day did you do three years ago before ChatGPT? 20 or 30? Yeah, I would have guessed something like 20, 30. I think today, if you just look across at all of my agents, I

33:34

bet they're doing 1,000x more than that.

33:39

maybe 100 to 1,000. If you count some of the things that happen at my company, which isn't like assigned to a human,

33:50

it might easily be more than 1,000x. But I think we are the outliers rather than the norm. So I think we're very, very early on this journey. I do think I recently saw, I think it was Cloudflare that said that In their monitoring of web traffic, the AI traffic is about the same as human traffic in terms of page reads, which is slightly different from searches because it includes perhaps all crawlers that are out there and a bunch of other stuff.

34:24

But I think it's going to happen. So maybe that's a good segue to talk about a topic that I know you are passionate about, which is the economics of the Internet as we know them. Some of the kind of fundamental assumptions there, you know, human eyeballs, scarcity of attention, seem to be falling right in front of us right now. Are the economics of the Internet broken now and what's going to happen? Yeah, no, I think this was perhaps part of the thing that totally convinced me that this needed to be done by me and build parallel.

34:58

I know most people hate ads. I used to do ads and build systems for ads. I love ads. Oh, wow. I love shopping. Okay. Yeah. So you get good ads. I don't know if I love ads, but I intellectually love ads because ads is the reason that so much amazing content and technology is available for free to all of us, right? Like Google search wouldn't be free without ads.

35:26

Twitter wouldn't be free without ads. And these are truly useful pieces of technology. A lot of content on the web out there that we can access for free wouldn't be free if not for ads. And so ads is a very efficient monetization scheme. It's a very efficient monetization scheme. because it is exceptional at differential pricing, right? So most queries, Google loses money on.

35:56

Some of them make it up and it's an extraordinary business with extraordinary margins. Same with Twitter. Like most users... Exactly. You're welcome. I'm subsidizing the free information you guys are getting. You are subsidizing all of us. Thank you. But I think ads is extraordinarily efficient at differential pricing and monetization on the web, which is why it has been a dominant business model.

36:20

Now, the core assumptions, as you noted, going into it around limited human attention to translate into outcomes. If humans don't show up and their agents show up on the web, like what does this mean? How does the business work? And yeah, I think this was the, if we don't figure out a new business model, we're seeing it already, right? Like people are going to say, okay, I don't want my content to be accessed by an agent.

36:51

Because I have a business model for humans. So I actually want to go pay someone to SEO, optimize myself so more humans show up. But then if their agent shows up, I'm going to cut it off. Which seems confusing and discontinent. Like, okay, ultimately this agent is acting on behalf of a human, but we haven't found business model alignment in a way.

37:16

You can't monetize that visit. You can't monetize that visit. So let's say you are in the business subscriptions, right? So you get a bunch of, you get a thousand humans and you convert 20 of them into monthly subscriptions, right? You don't know how that loop works, right? with agents you can't distinguish you don't have statistics you don't know if these will lead to subscriptions or they'll just keep stealing your content as nameless agents and so there are real challenges around the old business models breaking and us not figuring out real scalable new business models right so if you own high quality content and you currently put it on the web

37:57

Your one available business model pre-parallel was to be in the head and be able to transact with the lab on some fixed fee contract, which includes some amount of training and liability and then

38:14

inference time access. That was your one option. That option is not available to most content on the web. It's a very head phenomena. And even for the head, it is a broken business model because when AIs or inference grows, let's say 7x this year and another 7x the next year.

38:42

On this 50X, their deal size is not growing 50X. Like none of them after signing a two year deal believes that their share isn't going to decline materially at Renewal. And so these are like fixed price constructs in the world of AI inference. And so which doesn't drive for sustainability for, for all of these businesses. Now, our solution is trying to learn all the lessons from my work at ads on having transacted, like I sold Twitter data to OpenAI, having transacted on that side

39:21

to figure out what actually might work and be incentive aligned. So what might work? Efficient differential pricing.

39:30

Paying for differentially a lot for extraordinarily high value content. Paying a lot for high value work accessing the same content. So it's differential in both dimensions, quality and value of work being done with it.

39:47

and a way of doing this scalably and not just with bespoke deals. So those are the properties needed for any reasonable solution. And the biggest property of it all is incentive alignment, right? Like at what point do

40:05

people want to collaborate into this enterprise? Why do the model companies need to pay anything at all? You should ask them. My understanding is

40:16

is one, you want fresh data during inference time to be able to display it in products like ChatGPT or Claude. Two, you want training data. And three, you want some liability protection for training that you already did.

40:35

So the payments are some combination of these three things. And I can't be sure of like how they value each of these three. You said, you know, even a very simple query will go do 10 searches. And so how do you do attribution between all the different sources that, you know, boil down to one paragraph response? Yeah, at Parallel, we like to build models.

41:01

I heard. So, no, I think it goes back to my point around incentive alignment. So what are you, let's, before we try to build a model, let's try to figure out, like, if you were going to try to do this intellectually, theoretically, how would you go to it? so you have to ask the question of okay how much incremental value did somebody's content add right so you can you can run all of these simulation exercises you take one piece of content out of the corpus and then you say let's run the agent let's see if the quality of our output declined how much to claw back that quality perhaps if i threw a little bit more compute or a better model in some way Could I claw back that

41:45

quality? Oh, I could. It cost me a cent.

41:50

Huh. I could get the quality I lost by not having this source. My alternative was to throw more compute.

42:00

a cent worth of compute to get that same quality. It might be a different result, but as far as the end customer value is concerned, it's the same quality. And so you're like, okay, this source is worth close to a cent, feels like, right? That's intuitive. Now, the formalization of this kind of an intuition is

42:20

the core framework we use. It's called Shapley values. Yes. Let's go. What's a Shapley value? Music's my ears. It's a game theoretic. You're a game theoretic? A bunch of game theory, yeah. Yes. Shapley value is this... very theoretical mathematical concept. It is used in sharp values in feature importances for those who are ML folks here.

42:42

Let's say the three of us collaborate on something and the whole is bigger than the sum of parts in that moment. The theoretical question is, okay, how do I divide up this sort of bigger pie that we created by collaboration. So all three of us have incentive to collaborate, right? And Shapley values is a mathematical way of effectively answering this question, right? Now that sounds amazing, right? Like you could, if you're co-founders, you could figure out how to divide equity.

43:13

Unfortunately, it's not that useful. It's not that useful because in order to compute Shapley value, you need to simulate all words where some subset of the two of us collaborated, but the third one doesn't and play out those realities to then impute back to today in terms of how we should divide the pie, which in practice you can't do most places, right? In ML models, when you do feature importances, you kind of can.

43:39

You can hold a feature back, run your model and see how well it did. In web search, we can run

43:47

simulations of if we did not have access to this URL or this domain or this collection of them, how would the agent perform? We can, if you're good at evals, if you're good at assessing quality, you can build that data by running various scenarios. Collect a bunch of this data, and then you can train models to... Your favorite thing. Favorite thing.

44:09

So the challenge with Shapley values is computing Shapley values in our context. is way more expensive than the amount of dollars we spend on the agent. Forget the amount of dollars we want to pay a publisher, right? So to compute a content owner, to compute that a content owner gets a dollar, if I decide to do the full Shapley value computation, that might take several dollars.

44:33

So it doesn't make any sense, except you can really well estimate them if you build the right kind of data and models around it. But we have confidence that our estimations are good and that this is sound theoretically that if there was perfect information symmetry, people would want to collaborate. So the same way in the old ads days, people did second price auctions and believed, for better or worse, that people would reveal their true bids in an ad auction and end up paying less than that.

45:13

Once feedback loops get established in the market, like today, everything is like an auto bid, right? Like most people are measuring ROI when they're doing advertising and running on auto bid instead of making up bids. I think a solid foundation based on incentive alignment that Shapley math drives will ultimately maximize participation of content owners into this.

45:39

as well as content seekers via AI in an optimized system. And we have some positive evidence to support it. We've had some interesting partnerships that we've been able to do and announce.

45:54

And yeah, imagine sitting with content owners and explaining Shapley math. It takes a moment. But ultimately, once you pull out the properties that you participate in the value, if you have unique differentiated data, you get paid more. If

46:14

a banker in an expensive job reads your data versus my retired dad reads your data, the banker ends up paying more for that read, right? Because it's a part of high value work. And the macro math also seems to work. Like if you're going to spend a lot of money on inference on LLMs for knowledge work,

46:38

If we allocated like what, two to 10% of it to data on the web, that's way bigger than all web data business models today outside of like walled gardens like Facebook and LinkedIn. So the macro math supports it. It is a scalable approach and as agents on the web grow an order of magnitude year on year, it's like by my calculations, like we're 12 to 24 months from this math being able to give meaningful dollars for a very wide range of content owners on the web.

47:27

That's very exciting, especially at a time when you see the traffic data, the stack overflow plummeting. You see a lot of the human internet as we know it going away because of incentives. That's very exciting to see how you're thinking about incentive alignment for people to keep publishing. Yeah, it's why we started the company. Why is the company called Parallel?

47:51

The company's original name was Shapley Inc. Really? It was. Do you know this? I knew that, yeah. When I incorporated... I'm telling you, I was obsessing about everything to do with the problem space. So while knowing that the first set of also one Shapley Inc is a terrible name for a B2B product. It was not going to be the long-term name.

48:16

So we, we, but I incorporated as Shapley Inc for lack of a better word and Shapley.ai happened to be available. Shapley.com is a parked condominium. So.

48:32

It was not going to be a name. I went around not talking about my company as Shapley and my badges at all events used to call it Nuco or Stealth Company. So, and it took us almost six, eight months to figure out what the real name of the company was. will be we end up at parallel in part because it's like one at the time we're doing a lot more in parallel and two we started visualizing this sort of a parallel web built for ais and how its properties are different and this metaphor that when you publish you're now thinking of as we all are now of two audiences okay i'm going to create a page

49:17

I know humans will read it. What should it look like to them? And then how should I make sure that agents can read it too? So it feels like you're dual publishing to two audiences. And so we had this like this parallel web for agents will emerge. And so that's why we started liking parallel. Love it. I was just thinking about this recently with the things like earning transcripts, right? I feel very confident that more people are consuming earnings transcripts through agents than are actually certainly listening to the audio and honestly probably been reading the transcript itself.

49:53

So there are these very high value portions of the web that have flipped already to being agent first. And obviously the communication has not yet flipped. But I think if I were a public company CEO today, and I was doing an earnings report, I would make very clear that the thing that I'm saying will be transcribed and interpreted correctly by the agents, not just by the people listening, theoretically.

50:18

Same for us when we're publishing docs for our APIs.

50:23

It's like our customers are building AI solutions. They are using AI to do it. It's their agents reading our docs.

50:35

encode in our FDKs it's not humans fumbling around docs pages for the most part so in fact for us the primary audience is a agent and that's how we test our docs a parallel web for agents that's very cool maybe close us out tell us just give us a snapshot of where where parallel is today and if everything that you hope to build comes true what does parallel look like what does the world look like what's your role in it I think of three levels in this journey level one is people are building simple agents that use the web more as like a tool like a web search tool because it's familiar because the first set of agents we built to your point was just like model and give it

51:21

the same tools because these models have been trained to use all the tools human have been humans have been used to and let them do work

51:29

By and large, if you think of most work, it's there today, or influence, it's there today. We have now started seeing some subset of customers who are in this world where they're building more sophisticated

51:44

multi-agent systems which use sub-agents and have agents wake each other up or orchestrate in interesting ways. People are seeing that as sub-agents. Most familiar one is sub-agents with encoding agent harnesses. But we see a lot more of that. Like if you're building an AI scientist, Some of those systems are very interesting, very sophisticated, very long-running, and just throw large amounts of compute and data at a really hard problem.

52:17

And I think the third layer for the web specifically is the web goes from pull to push. So today, by and large, across the first two modalities I described, either an agent is calling a tool or a sub-agent. But it is telling it a request, saying, go find this for me right now. And I think where we will end up in a couple of years is a variety of use cases will be

52:47

the web parallel Call me if this happens so that my agent can do some work or a human can do some work. So I use this line with my team all the time, which is like, imagine like agents are everywhere and they can do a lot of things, right? If there is something you can do today and it's worth doing

53:12

and we still have GPUs available, We'll just go do it, right? Like we won't be like, oh, let's just do this tomorrow for the sake of it. If it can be done today, do it. So what will we do tomorrow? We will do tomorrow work in response to something that circles through either another agent's work or something changes in the world as visible in satellite imagery or some customer commentary that happens, some agent finishing some compute, a human having a new insight to trigger work.

53:46

But there are going to be a few feeds like this, which will drive new agentic work tomorrow. And one of those feeds is going to be everything that changed on the web, which is really exciting for me because then you're framing not a point in time need, But a long term,

54:08

here is what is actionable for me. If something like this happens as evidenced by all of the information on the web that is changing all the time, call me.

54:20

And then I'll run my agent on it. And so we get to then allocate compute onto everything. the entire web all the time on behalf of all the customers. And that's really exciting. That's awesome. Really ambitious vision. You're clearly extremely passionate about this and exciting to see you building. Thank you so much for joining us today, Parag.

54:42

And thank you for joining us, Andrew. You're very welcome, Sonya. Thanks, Parag. Thank you, Andrew, for your debut.