Evan Conrad
Related people
Episodes
Notes
Latent Space: The AI Engineer Podcast · The Programmable Data Center — Evan Conrad, SF Compute
-
What Is a Programmable Data Center?
Evan Conrad says a programmable data center is a data center SF Compute has virtualized, so a customer gets the control that used to come with owning the hardware. A feature already shipping lets that customer subdivide an InfiniBand fabric without asking anyone to redraw the lines. Many clouds hand over a single fabric and then require a call when the boundaries need to move. Redrawing them on the fly fits a market where clusters scale up and down.
The same release is meant to make the capacity usable for platform-as-a-service companies. The goal is to virtualize the rest of what ownership used to include. The last shift like that was the arrival of cloud computing.
-
Why Is Live GPU Inference Harder to Interrupt Than a Batch Job?
Evan Conrad says live GPU inference is harder to interrupt than a batch job because live customers expect the service to stay online, while a batch job can stop at any point. A batch workload can be cut with about an hour of warning, or even a minute or two. Live capacity can be taken back a month, two months, or three months out, while the operator adapts, and it cannot be pulled in a minute.
The two are priced differently because the cost of the interruption is different. A live job that goes down means paying a service-level penalty to that customer. The market is a market for breaking leases, not only a sublet of spare capacity. Power markets already split this way, with a day-ahead mar
-
How Does GPU Compute Trading Deliver the Hardware?
Evan Conrad says GPU compute trading delivers the machines, which futures markets call physical settlement. SF Compute began by trying to sublease a cluster. Selling that capacity was like offering a hotel room, then being asked to clean it, then to run room service, then security, until the seller was running the whole hotel. At that point the supplier looks more like a fund or a cloud that already has the data center. The company spent several years building the operating capacity of a cloud so the compute actually arrives and stays reliable.
In oil or soybeans, physical settlement means the goods show up. Oil needs pipelines. During COVID, oil went negative: a holder stuck with barrels i
-
How Does a Healthy Compute Price Curve Work?
Evan Conrad says a healthy compute price curve is a hump, and it can point either way depending on the buyer. Capacity that is about to reach the present cannot be sold in the past, so unsold time has to drop in price to clear, the way milk about to expire is marked down. The price then rises toward where it wants to sit. Compute further out is worth less, because buyers pay more for a shorter lead time.
A B300 today breaks that smooth picture. Running inference on one can bring in about $40 an hour when the chip is fully used, and that figure is for basically any model. A rough case at about half utilization is near $25 an hour. Buying the same B300s costs about $5 to $6 an hour. One of th
-
How Long Are GPU Clusters Financed?
Evan Conrad says GPU clusters in the broader market are financed for three to five years, and the payments cover that period. On the SF Compute order book, a block of capacity can run for any length of time. A buyer can take the next hour, the stretch from right now to the next hour, or the stretch from right now out to three years, and any span in between. Five years is available technically.
A block is paid entirely upfront. A longer-term lease is not. The cash flow on those longer blocks still needs more work. The priority is capacity further out, because the goal is a forwards curve with a lot of liquidity.
-
Why Is Next Month's Compute Capacity Easier to Sell?
Evan Conrad says capacity that starts a month from now can be bought and sold all the time, so liquidity shows up easily. Capacity right now sells only by interrupting the workload already running on it, and the seller has to decide whether that interruption is acceptable. Worrying about capacity next month is a different problem from cutting a job that is already running.
The way an interruption lands depends on storage. A Weka or Vast system, and how large it is, changes how a job handles being preempted or pulled off the cluster. If data can move quickly from one cluster to another, he leaves open whether a job spreads across several clusters, drops on one, and starts again on another. T