Google's AI Infrastructure Chief, Amin Vahdat, on the Physics & Economics of Frontier AI

In hardware, again, as you know, there is this opportunity where the more you specialize to a particular workload, the less flexible it is, the faster, the more power efficient the hardware is going to be.

So it is this art and it's this projection of what are you designing to and how persistent is that workload.

In other words, if it's going to go away after a month or two months or three months, even if it's big for those three months, you got a really narrow window to intercept

So it has to be somewhat durable, and you have to be able to project exactly what win you can get for specializing to it.

Thrilled to welcome Amin Vadat to the show. Amin, thank you for joining us for today. It's really exciting to be here. I'm really looking forward to it.

I am very excited for today's topic because we are in the middle of the biggest CapEx buildout in human history.

You are in the middle of it. Google alone is expected to spend more than $200 billion on CapEx this year, most of which is going into building data centers.

You're at the center of it all. You were named the head of Google's AI infra at the end of last year.

So you are the man spearheading the efforts of one of the most capital intensive buildouts in human history.

And so I am really excited to get into it with you today. It's a huge year for sure.

Of course, across the industry, including at Google, really, I don't think we've ever seen anything like this, honestly.

Certainly not at Google, but as you said, I think in the history of humanity in terms of the buildouts and the pace of transformation, really unbelievable.

Before we get into it, maybe just crash course for the audience.

What is an AI data center and how is it different from a non-AI data center?

It's a very good question in that AI data center and a non-AI data center do have a lot of similarities, actually, and they're not radically different.

I mean, it consists of concrete, an enclosure. It consists of electrical yards, mechanical yards, cooling.

You have row after row of power being distributed. There's a huge amount of network infrastructure there.

In other words, we connect large amounts of computes to one another. A fair amount of storage infrastructure goes in there.

I think that the big difference that we're seeing with AI infrastructure is specialization.

In the past, when we're building a data center, it really is a 20, 25, 30-year building investment.

And we're thinking about how it's going to be evolving over that 20, 25, 30-year period. You might have servers go into it.

You might have networking storage. you might have some accelerators, GPUs, TPUs, whatever they might be going into it.

But it's a 25, 30-year planning horizon. And the lifetime of the hardware might be six years.

So many generations that we have to plan for. An AI data center oftentimes is going to be more purpose-built.

So in other words, we are oftentimes co-designing the building with the hardware that might go into it.

we might be saying, you know what, actually, we're not going to put a lot of storage into this building. Why?

Because a storage rack might have 10, 20, 30, 40 kilowatts of power.

You put that next to a TP rack or a GP rack that easily is hitting hundreds of kilowatts today, might hit megawatts.

People are talking about that in the next few years. Designing a building that might take 30 storage racks in a row versus one or two AI racks in that same row.

Very, very different design. Just think of it in terms of the size. Think of it in terms of the power and how that would be distributed across the building, et cetera.

Networking would be the same. The amount of networking that you would need for a storage rack, tiny, I mean, especially if it's hard drives,

compared to an AI rack. So if you're trying to make it fully fungible, you'll probably make it too big and too overbuilt, fungible over certainly a 30-year period.

An AI data center likely is going to be much more purpose-built, co-designed with the hardware, even to the point of cooling and power distribution, et cetera.

So a lot more co-optimization. You all delivered a big Verorubin cluster to one of my portfolio companies, Ineffable Intelligence, and I saw the photographs of it as it went out.

That is a thing of beauty. It is. It is.

I mean, it's one of those things where, you know, similar to looking at the biggest construction projects in human history, you look at that data center, it's like, wow, that is a monument to what mankind can do.

No, we got a nice picture of it. And this is just, you know, a couple of racks, you know, the cabling, the fiber distribution for it is really beautiful.

I mean, beauty is not the beholder, but for folks like me and probably yourself and others in the audience as well,