NVIDIA UCIe Interview Transcript

Allyson Klein: Welcome to the TechArena. My name is Allyson Klein, and today I'm delighted to be joined by Durgesh Srivastava, Senior Director and Data Center Architect at NVIDIA Durgesh, welcome to the program.

Durgesh Srivastava: Hey, thank you, Allyson. So glad to be here.

Allyson Klein: So Durgesh, you, you work on a lot of exciting technologies for NVIDIA, but why don't we just start with an introduction of your role and how it relates to our topic today, UCIe.

Durgesh Srivastava: Great. So I'm senior director in hardware engineering at NVIDIA. I'm driving the product architecture for several server products. So my focus has been to look at the silicon and systems to address the growing need for the compute, and we'll touch on a little bit more. And I represent NVIDIA in UCIe consortium as a board member, and that's how you and I connected.

Prior to joining NVIDIA, a year and a half ago, I was at Intel for 24 years. A little over 24. I worked on various projects starting from Itanium and Xeon, and I worked on chipset Atom SOC and client products. And I also spent some time on designing the autonomous driving solution for data centers specifically last couple of years at Intel, I was very active in server memory pooling and, and we'll touch a little bit on disaggregation to solve the issues related to memory or polishing. So overall, to summarize, I come from systems and silicon background and that's what I am passionate about and trying to solve the problems and continue to do that at NVIDIA.

Allyson Klein: That's fantastic. You know a lot has been written about the slowing of Moore's Law and the industry looking for innovative ways to continue to scale performance. We've seen things like multi-core processors and heterogeneous solutions come to the table, but it seems like we need more. What methods are being considered and how does UCIe enter that picture?

Durgesh Srivastava: That's a very, very good question, Allyson, and it's in like everyone's mind in the industry and trying to figure out what do we do. So like as a background, as you're saying, computer requirements are growing at expenditure rate due to AI, deep learning usages, and we have seen the chat GPT these days and which amazing. And how it has been trained and developed. So overall, just to give you the data, the size of data and AI market is on track to break the 500 billion mark by 2024. So as you can see, it's pretty big. And current computing solutions, which we have, are not really catching up at the same rate. So, the requirements are going much faster, computing solutions are growing, but not that fast. Like as an example, memory in terms of the gigabyte per dollar is saturating is not getting cheaper. CPU performance causes irregular cadence by 10, 15, 20%, power efficiency is not improving. So overall, we have a big gap in requirement and solutions.

What needs to be done even before we go in the UCIe. And first thing is we have to do redefine the way we do silicon, the way we do compute, we do infrastructure and systems for data center. So there'll be a lot of customization instead of using general purpose just to address this, get more compute or getting more power efficient and, and this will help the address, the slowdown of Moore's Law.

What I believe is that we must move toward disaggregation and accelerators to address the gap, and accelerators will help with application specific usages. We must build servers and racks and data centers infrastructure a little bit differently. What we have been doing in which we have been doing for multiple decades. And last but not least, we also have to make sure we are designing for sustainability because the power consumption keeps going on high, more and more in dissipation and, and how do we keep a balance on the requirements, but we keep a greener planet as well. So what we thought is chiplets is one way of providing a solution to this increasing compute requirements, which is break into smaller pieces and try to address it.

 We can talk more about it in the sense, but the usages, like you can see video, transcoding usages are already there. Compression usage are there. So if these chiplets exist, people can plug and play or use it and define and address the compute requirements as well as try to solve that slowness of Moore's Law

Allyson Klein: So let's take a step back. It was five years ago that DARPA started its chiplet group, and the industry started to get serious about chiplets. Let's just get a definition of what are chiplets and what does the commercial delivery look like for chiplet based architectures?

Durgesh Srivastava: That's again, important point to clarify and define as even within companies there are different ologies. Sometimes people use chiplet and dilets and, um, microchips. I've also heard from some people, so yeah, chiplet architecture is really involves a small and modular chips, which we call as chiplets. , and, and they are like smaller blocks.

You put them together and create larger and more complex ecosystem. So the goal really is to provide the flexibility in design and manufacturing process. What it means is that you can have one chiplet that is small building block, could be on a different process, technology, or even coming from a different company or organization.

And you can plug it to another one, which is coming on a totally different environment. So, this reduced the cost as well because you can use the same chiplet for a different product and, and plug and play really? Mm-hmm. So putting another way, chiplets are one way of providing more compute with specific application and flexibility, which we are seeing, and more and more as we generate more product and product combinations and the usages are growing.

So as I was touching previously, let me elaborate on that a little bit. Like say examples are video transcoding, compression, encryption, memory tracking, or tracing. All these things are needed, so some applications require it, some don't. , if we come up with a chiplet solution for these kind of accelerators, we can combine it, use it where they're needed and not use it where they're not needed.

So you can see that we have accelerated, we are trying to provide the solution for exceeding compute requirements as well as we don't have the silicon sitting, which is not being utilized. So that it helps in all sort of ways. And one other thing which you asked is what do you think of the whole ecosystem?

So the chiplet ecosystem is going to grow, and I'm already seeing that it is, there's a lot of interest and there are a lot of things which are happening. So the way it'll evolve where there'll be chiplet based accelerators, as I mentioned, where there'll be companies providing the packaging solution like 2D, 2.5D, 3D, and then there'll be companies which will be integrating for some of these companies which need these solutions. So chiplet make that multi-vendor implementation possible. I want to see this Lego plug in play that you want something, you go in a portfolio, you look at things and churn out a chip, which can be deployed very quick and come out of the solution very fast in the industry.

Allyson Klein: Now that Lego plug and play is something that made me really excited to see the announcement of the UCIe consortium. You're one of the lead architects on it. It's a new standard interconnect for chiplets. Why is this specification so important for the industry and that vision of Lego plug and play?

Durgesh Srivastava: I'm very excited about UCIe. I'm really glad that we are doing it for industry and bringing a whole ecosystem together, so, What is UCIe? UCIe provides a complete die to die interconnect or like what we are talking about the chiplets in this sense.

And it is not just focused on specific layers. And generally these protocols are layered protocols. So it has the physical interconnect, which are the electricals, it goes to the protocol stack, that how it talks to the back end of a specific chip. It has a software model, the whole manageability and compliance testing.

So the way we are doing it, It provides this, universal chiplet express interface and providing all the complete solution. So now as you can see, independent companies can, come and develop these solutions and they can talk to each other on a package itself. So that's the interest driven that now make it more necessary standard so that people can do the innovation and develop it independently, and we have leveraged, as you have seen in the announcement, PCI Express Compute Express Link as the backbone of it, but they're not just tied to it. There are other, like whole ARM ecosystem also is very excited about it because you can plug it in an streaming protocol, which is part of the UCI Express.

So it's like really in, in the end, is a multi-vendor ecosystem for system on a chip, which is like a full motherboard coming on the chip and customization. It is going to be extremely useful for this. The main thing is the open standard, bringing everyone together. So we have like a hundred plus now members and, and just bringing the goodness of various experts, various people.

That's going to go a long way. The last thing I will just add is that we also have eyes on the future. So we not just defining the problems of today that how we will scale in the future. So we have like modular design within the chiplet architecture and if you have high bandwidth you can have more modules and so on so forth.

So we kept that in mind that how we will keep scaling as a computer requirements kept going up and up.

Allyson Klein: So you've created this vision of Lego plug and play. Choose your best of breed chiplets for your solution and utilize UCIe to connect them. But what we've seen from the industry thus far is more use of proprietary interconnect for chiplet designs and chiplets provided by a single vendor, and there's also been a lot of industry attention on CXL, the industry standard that offers chip to chip interconnects on a motherboard.

How do we get to the future vision from where we are today, and is there a role for all of these solutions as we move forward? How do they fit between multi-chip and chiplet architectures?

Durgesh Srivastava: Excellent point and excellent question just the basis of UCIe is that how do we standardize rather than going the "every company has their own", and I can call it loosely proprietary. In some sense it is because companies have to address their requirements. so that was your first, so let me, let me deep dive what you asked the first question is extremely important. So all the big circuit providers like us, like NVIDIA, we have our own chip to chip.

The reason is because as we talked, Moore's Law is slowing down and we have computing requirements. We have come up with our interconnectivity, which is very tightly coupled, meets our requirements. And same thing, other big silicon vendors have the solutions which are providing the way to address the problems. That's the main reason to come up with this industry standard so that we don't diverse too much in terms of the chiplet, in terms of the packaging technology in terms of the substrate so that we can come up all together and put our minds together and we have a standard.

So I do still believe that in some usages the silicon providers like us will continue in certain usages having a tightly coupled our proprietary just for that is specific usages, which is internal in inward looking, but anything which is going outward, which is where we can have accelerators or having another chip talking to a third party provider that will definitely go through UCIe and UCIe is the best solution to address that. And the second part of the question was that you were talking about how this whole thing ties in PCIe, CXL and then ARM has its own standard, which is a pretty big ecosystem. So the goal is UCIe is providing the interconnect between the chiplets.

What it means is that whatever IP you have or whatever internal protocol, whether it's ARM base, CXL, PCIe, you're building on top of it. So you're trying to compliment it. So that's the beauty of UCIe, that UCIe as a interconnect between chiplets, it will bring all of these protocols, which are running in the background on the chip into one altogether so that they can talk to each other. They can interact with each other and does not have to be exactly the same. And that's the beauty of the open ecosystem, leveraging the industry and putting it together. And as you can see, as you see in industry, it's not just the silicon profile. There are packaging companies there are, TSMC is a board member. So all, all those things, Samsung is there. So all those things, all putting it together as a complete solution helps to build that ecosystem. Not having just the proprietary solutions, not just going with the one specific solution. Leverage CXL, PCIe or any of the ecosystem protocols which exist.

Allyson Klein: It's an exciting future, and you can see how it really puts the power in the hands of the customer to dial in exactly what they need. Yeah. Now, I have to ask the question. Nvidia obviously has a great Silicon portfolio, and you're investing in this space because it's part of your strategy. What are NVIDIA's plans for integration of UCIe into your lineup.

Durgesh Srivastava: We are very excited and, and that's why as you can see, we are the promoter and board members and I'm really proud to represent Nvidia in the board. And as we mentioned so far, chiplet interconnects are more important than ever. And we are seeing that to ensure big workloads and we can scale, right? We are leaders in the GPU market for training and AI workloads, and we definitely have seen the heterogeneous compute engines can execute at their full potential and homogeneity is no longer helping or helping as a scale. So as workloads continue to grow, we need move past some of the today's platform level limitation and enable innovation and system integration.

We are going to fully support UCIe across all our product range as we go further. And we are also helping in promoting UCIe adoption with our partners. So wherever there's a need and where there is a chiplet concept with industry and external interaction, we are fully supportive of it. We are very excited about it.

And we have our NV link C2C, which is chip to chip does something similar, but any inward looking or very tightly coupled applications from customers, which are a lot of bandwidth and working with us. So we will, we'll have that. But this whole chiplet ecosystem and UCIe is extremely important for us, and we are really driving some of the innovation in that area.

Allyson Klein: I think that you've laid out so much interesting information for the audience that folks are going to want to continue to engage and ask questions. Where can folks go to find out more information about NVIDIA's technology in this space and engage with your team further, and where can they find out more information about UCIe?

Durgesh Srivastava: Yes. So let me start with UCIe. So best place is to find the information about is our website, UCI Express consortium website. I really encourage people, if you're not members, please become members. Contributing members joined the technical work groups. We have five of those technical work groups talking from center protocol software all the way down to electrical and compatibility and help to bring the ecosystem and address the slow down of Moore's Law and where the computer requirements are going.

Regarding NVIDA. Where we do the best is our GTC, which happens twice a year, and we bring our new information related to products and our mindset and where we are going with it. And of course regarding me I can be contacted on LinkedIn if there are any specific question interest anything you want to know, but I do encourage everyone to join the UCIe consortium and help us grow this ecosystem and make it a successful solution to address the computer requirements

Allyson Klein: Durgesh. I was so excited to do this episode. I had named UCIe one of the most important technologies to follow in 2023, along the lines of Chat GPT.

So maybe I'm a geek, but maybe I know the importance of the continuation of delivering semiconductor performance. What you and the UCIe team are doing is really exciting. I can't wait to see more. And thank you so much for being on the show today.

Durgesh Srivastava: Thank you Allyson and appreciate your time and thank you for inviting me and giving an opportunity to talk about where we are going and what we want to do.