Overview:
In this episode of the Tech Sessions podcast, host Art Jannicelli welcomes Matt Baran, Senior Channel Systems Engineer at Nutanix. The discussion focuses on the current scenarios of artificial intelligence (AI) and how organizations can effectively integrate AI into their operations using Nutanix's GPT in a Box solution. The episode covers key challenges in AI adoption, the benefits of Nutanix’s scalable and secure AI platform, and practical insights on deploying AI with reduced risk and cost.
Watch the Episode:
Key Topics Covered:
- AI Adoption Challenges: Understanding why up to 50% of AI projects fail and strategies to mitigate these risks.
- Nutanix GPT in a Box 2.0: An overview of Nutanix’s turnkey solution for deploying AI with partnerships from NVIDIA and Hugging Face.
- Data Security and Sovereignty: How Nutanix ensures data protection and compliance in AI deployments.
Key Takeaways:
- Simplified AI Deployment: Nutanix GPT in a Box provides an easy-to-implement, scalable AI solution that leverages existing infrastructure and reduces initial costs.
- Data Security: Nutanix emphasizes data sovereignty, ensuring that sensitive information remains protected and within the organization’s control.
- Scalability and Flexibility: The platform supports growth from small-scale deployments to extensive AI operations, enabling organizations to start small and scale as needed.
Read the Transcript:
Ep. 5 MI: Nutanix GPT in a Box: The Resilient AI Data Platform
[00:00:00] Art Jannicelli: Hello, welcome to the Tech session. Today, Matt Baran, a senior channel systems engineer with Nutanix who I've worked with for years, and I are going to talk about the latest trends in IT, particularly around AI, and what you can do to quickly get involved with developing AI in your organization.
AI is top of mind for everybody. It's something that every organization is looking at right now. But if you look into it, Gartner's saying that up to 50 percent of AI projects are failing right now. Matt and I got talking recently about what organizations can do to make this transition easier and have less risk. We feel like there could be a great discussion around this, and I think Matt's a great person for us to dive into this with.
[00:00:55] Matt Baran: I really appreciate the opportunity to come here and speak on behalf of Nutanix, especially with one of our trusted partners, e360. I'm happy to be here and talk about GenAI and GPT.
[00:01:26] Art Jannicelli: Thanks, Matt. The thing behind AI is that we've been talking for 10 years about how IT is changing faster and faster all the time. But in my 25-year career, as cliché as it sounds, it's changing like never before. There are new things coming out all the time with AI. We are at such the beginning of this whole revolution, even though it's been around for a year now. I think a lot of organizations are struggling, not only with where to start but how to start without painting themselves into a corner where a year from now, they're really kicking themselves.
I personally have had vendors tell me, "We have a 16 million dollar solution and it's okay if it doesn't work out because the customer can then just tear it apart and use it for something else." What's been your experience, Matt?
[00:02:18] Matt Baran: My experience has been that organizations are really interested in jumping into GenAI, but they don't have the resources or time commitment to do so. Traditionally, with GenAI, you had a few options. You could build it yourself, which is mostly open-source components that can be built on pretty traditional hardware. But that takes a really high level of expertise and specialty to be able to do that. You have platform engineering, DevOps, Kubernetes, and a lot of moving pieces.
The other option was the cloud, where GenAI was born. That's super easy, but the challenge is that it can be costly. Also, organizations have concerns about data sovereignty. You're taking your very protected data and pushing it out to the cloud where you lose control over it. Those have been some of the challenges I've seen.
[00:03:18] Art Jannicelli: When we talk about cloud and options there, are customers discussing with you where it's best to use the cloud and where it's best to use it on-premises? The elephant in the room is that these are probably seven-figure solutions for a lot of this. Circling back to risk, what are you talking to customers about in terms of getting into this without having to make a gigantic long-term commitment, or where to start?
[00:03:50] Matt Baran: I have a lot of conversations with customers about this. It's about what they're excited about and what's bringing them to this. It can be a lot of different things, from the novelty of jumping into ChatGPT and asking it a question to getting an answer that seems like a human wrote it. That really hooks your attention, and then you start to think about what you can do with this to make your business better.
For example, let's say you're a small law firm with thousands of case documents specific to your industry or area of practice. Today, to go through that documentation when preparing a new case and referencing old data, you have employees who search the library for previous cases and pull information snippets. What if you could take all that data, augment a GenAI model or GPT, and ask it a question about a previous case or piece of law? You'd get that answer delivered to you with the context in which it was found, speeding up your time to business and ability to move more quickly.
I think that's the case everywhere. We create data now at a rate that has never been faster, and every day we break that record. So what could we do to harness that data? It's not just about being able to search or index it, but how can we have a human-like conversation with that data and get that answer back out?
[00:05:33] Art Jannicelli: One thing that just crossed my mind with you mentioning law is, on top of the change with this greater access to our data through indexing and the interface, how do we keep the wrong people from looking at the wrong things or accessing the wrong cases? You could have a bad actor inside a firm who's stealing stuff for another firm or just having a personal interest in a case.
[00:06:02] Matt Baran: That's where it becomes really important with GenAI. When you hook something like GenAI into your organization, think of Microsoft Copilot. What we've seen is that it immediately grabs its hooks into every piece of data that you have. It becomes really trivial for an employee to log into Copilot, for example, and type in a query like, "Show me people that got bonuses last year." Someone somewhere might have had that document saved in their drive with incorrect permissions, and it was able to index that.
One of the things that Nutanix is really focused on, beyond simplifying the adoption of these GenAI platforms, is data security and data sovereignty. We want to protect the AI from leakage of data or poisoning of data. We really want to make sure that the data you're providing remains on-premises. That's a big deal because with ChatGPT, depending on the tier you're using, they make it very clear that the data you feed into it gets added to the model and becomes data for other people. As an organization, you have to ensure that your data does not get mixed into the general pool of data being leveraged.
[00:07:13] Art Jannicelli: That really sounds like a great reason to think about the cloud and what you would do with Nutanix, in comparison to being able to have greater access and control of your data. When we look at all these things, and we've talked about a lot of components here - we've mentioned LLMs, there's obviously storage involved, there's compute, there's GPUs - and of course, you can do those things in the cloud. How are you seeing customers approach the complexity of this?
[00:07:48] Matt Baran: The way we'd like to see customers approach the complexity of this is to look at a platform like Nutanix's GPT in a Box. What Nutanix has done with GPT in a Box, and more specifically GPT in a Box 2.0, which I think you saw announced at our annual event .NEXT in Barcelona, is we've taken our industry-standard Nutanix cloud platform and our Nutanix unified storage, which are pre-existing products. These already have leverages into Kubernetes, which is the foundation for these GenAI solutions.
We went out and partnered with NVIDIA and Hugging Face. NVIDIA is providing what they call NIM, which is their pre-built stack of tooling to build GenAI on-premises or wherever you're building it. Hugging Face provides those large language models that you're able to augment with your own data.
Basically, what we've put together at Nutanix in our GPT in a Box 2.0 is really a turnkey GPT solution that is quick to build, easy to support, and has very low TCO. The benefit of a lot of these things is that Nutanix was built around the idea of scaling web-scale technology. Start small, scale out.
You mentioned earlier that there are some GPT or AI solutions you can deploy within your own confines that are seven, eight, or nine-figure solutions. The beauty of Nutanix in this space is we scale the same way we scale for any other workload. We're workload agnostic. Start with a three-node cluster and see how it goes. As that GPT or GenAI model picks up in popularity and becomes more ingrained in what you're doing as a business, scale by adding another node or a couple of nodes. That's the beauty of reducing the over-provisioning and initial waste of buying a solution that's just too large for getting started.
[00:09:53] Art Jannicelli: That sounds like a great plan. When we talk about Nutanix, of course, Nutanix is a software-only solution. They're definitely famous for that. So who's Nutanix partnering with? What does the GPT in a Box look like when we talk about the underlying hypervisor and hardware components? I know you've mentioned NVIDIA; who else are you working with?
[00:10:18] Matt Baran: Obviously, NVIDIA is the big one there. They're the 10,000-pound elephant in the room right now. They're sort of running the show. That may change; there are other players in the space that may come and introduce a solution that can compete, but for now, it's definitely NVIDIA.
We are working with Intel and their new Intel platform, AMX, which is supposed to be really powerful for GenAI workloads. From a hardware perspective, again, Nutanix is hardware agnostic, and we really believe in the power of choice. We've got support with our Nutanix NX platform, but we partner with the rest of the ecosystem. Cisco, HPE, Dell - those are all platforms that are validated to run our GPT in a Box offering.
We support a broad list of NVIDIA GPUs at the moment. Depending on the size of the models you're working with, whether or not you're doing your own inferencing or model building, we have a variety of different GPUs that you can install to meet the correct use case. Again, this is to reduce that over-provisioning and waste of potentially having more than you need.
[00:11:27] Art Jannicelli: One thing that's interesting, I didn't hear you mention was storage. How does Nutanix provide storage for this important high-demand workload?
[00:11:36] Matt Baran: Super great question. Nutanix early on had a Kubernetes platform, now known as NKP (Nutanix Kubernetes Platform). What we have underneath that is Nutanix data services for Kubernetes. Nutanix data services for Kubernetes is a Container Storage Interface (CSI) that provides the storage attachment for Kubernetes containers. The beauty of that is it's built into our platform in our storage stack.
You still have all of the same benefits like multi-cloud snapshot technology, backup, and replication. We're not strapping an additional software layer on there. We don't have extra pieces. It's all part of that NCI, that Nutanix core infrastructure, and our Nutanix unified storage platform. We're leveraging all those investments that we've been making over the last 10 years to bring that same level of support and resiliency to Kubernetes and therefore, GenAI and GPT.
[00:12:39] Art Jannicelli: Is that bundled into Nutanix at large in terms of licensing? Is that something completely different, or how does that work for you guys?
[00:12:50] Matt Baran: The overwhelming number of Nutanix deployments that we sell have the correct licensing in place to run Kubernetes workloads. I think we've done that because we believe that not too far into the future, maybe just a couple of years, a vast number of our enterprises will have incorporated both AI and Kubernetes.
When you talk about Nutanix, it's "any app anywhere, any platform, your choice." So it's not just limited to virtual machines, but containers and storage workloads as well. We're definitely focused far beyond just virtual machines and virtualization.
[00:13:29] Art Jannicelli: That's very fair of Nutanix. I've been seeing a lot of things out at conferences that many AI plays and vendors are looking at a pricing model of charging per transaction or data set size. It's good to see that Nutanix is taking a more agnostic approach and including the ability to do all these things.
As you've been discussing this, it really sounds like Nutanix has this turnkey platform. We've talked about security, and I understand that the data storage needs on these can be extreme. How is Nutanix going to protect that data if we're going to move it on-site, and is Nutanix prepared for that?
[00:14:17] Matt Baran: Absolutely. As we've been making a push into our unified storage platform, which is our ability to serve up files, object, and block storage all from the same software layer, we've been increasing the node density that we support. We've continued to support things like SSD and NVMe, high-speed storage, and dense storage.
In fact, in our PNP 2.0 pricing model, which is all current Nutanix pricing, we're not even charging based on the amount of flash capacity anymore as we used to. We are simply core-based now. So we encourage customers to build those nodes with all NVMe, the fastest possible storage, the densest possible storage. There's no additional upcharge for that; that's just part of purchasing cores of Nutanix now.
Beyond that, obviously, Nutanix was really well founded on data resiliency. You have different resiliency and redundancy factors. We have rack and block awareness, depending on how you've deployed. We support snapshots to other Nutanix clusters and clouds. There are a ton of options in how you want to manage your data, but we really focus on data performance and data safety as part of our heritage.
[00:15:33] Art Jannicelli: That's impressive. Thinking about security and resiliency, is Nutanix with their container platform able to provide resiliency for customers? I was just thinking about customer service chatbots - that's a communication that can never go down. Can Nutanix host that on-prem and in the cloud and automate the recovery and things like that?
[00:15:56] Matt Baran: Absolutely. That's our bread and butter; that's our core competency. An on-premises cluster connected to an NC2 cluster, which is a Nutanix cloud cluster, would operate in lockstep with resiliency plans, the ability to fail over in the event of an emergency disaster. We take that type of stuff very seriously, and we have many customers with very high requirements on uptime.
[00:16:27] Art Jannicelli: Wow. So that would make this even more turnkey than going to the cloud, especially if you're trying to move between different hyperscalers. You could definitely run into some problems there.
[00:16:38] Matt Baran: I actually think that's a really good point that you just brought up and something that we haven't discussed. There's a concept of lock-in that I'm sure you're familiar with. Lock-in being, "Hey, I'm using a cloud-provided model. Maybe it's Azure Machine Learning. Maybe it's ChatGPT." But my data becomes sort of beholden to that model and that product.
To your point, let's say I'm a Microsoft customer. I'm all in on Azure Machine Learning. And then something changes, and now we're looking at Google. We want to use Google's cloud inferencing APIs. My data sets, my data structures, the work that I've done may not be compatible immediately with that because we're not using open-source toolsets. We're not using standardized toolsets like NIM. Rather, we're using a vendor-specific solution.
One of the other pieces of Nutanix is we're really trying to help you avoid lock-in. If you wanted to make a change and say, "Hey, I want to switch the model that I'm using," or "I want to flip from open source to NIM," you have the ability to do all of that on the fly, and you're using all open-source tooling.
[00:17:47] Art Jannicelli: Wow. I'm just hearing there's a whole lot of choice here. There's so much to consider, and it's great to hear that Nutanix is a place where, like we started the conversation, you can deploy this with your three nodes, get started here, and like you said, grow this out, try it out, get it running, figure out what your load is, and then scale it as needed without painting yourself into a corner. This really sounds like a great resilient AI-ready platform. I'm looking forward to working with you more on this.
[00:18:21] Matt Baran: We're really looking forward to taking this to market further. We've got an entire team of AI specialists, and they're out there to talk with partners, customers, and really help share our vision on how we think this looks. Our goal is really to build a resilient AI-ready platform, and I think we've done that with GPT 2.0. I'm really excited to see what customers do with that.
[00:18:52] Art Jannicelli: I think that's one of the hardest things about AI. There was a law of computer science I learned way back in college: whatever's easy for machines is hard for humans, and whatever's hard for humans is easy for machines.
When we jump into AI, it's great to hear Nutanix is ready to have these conversations. Especially with the high failure rate of AI projects throughout the industry, having that sounding board to work with the organization, look at your business case, and figure out how we can ask a question that AI can answer is crucial. How can we build a solution around this that we can get started with? We all learn through failure, and this is a very fast-paced thing. I'd be curious, what kind of use cases is Nutanix seeing, and what has been the experience so far working through these kinds of problems?
[00:19:53] Matt Baran: From a use cases perspective, it's really about analyzing, operating, and modernizing data. The first step is to analyze your data. You have all this data you've created, so what is it? What's in the data? Is it useful data? From there, you can operate that data, operate your business with that data. You can say, "I want my business to run off this data. I want to intelligently process documents and data and get answers that help me make my business better."
The last piece is really about modernizing your business. This goes hand in hand with GPT, though it's not necessarily the same. Organizations, even taking a step back from AI, are struggling with containerization. Containerization is a really tricky technology, and while adoption is ticking up, there are definitely some people who post on different blogs and LinkedIn that Kubernetes doesn't always save time or make them more efficient.
[00:21:06] If we can change the mindset on things like Kubernetes and say, "You don't have to do it all yourself," that's helpful. There is an enterprise platform that supports Kubernetes, our Nutanix NKP. We have that data platform underneath it, and we have all those things that typically require teams to build for you.
If you could just modernize your business by containerizing, that gets your business cloud-ready. We've all seen how costly it is to take your current workload infrastructure and move it to the cloud - it's not the right way to do it. But if you could modernize that workload, make it containerized, then if you needed to move that workload to the cloud, you can do it natively. You're not just lifting a VM and shifting it up to the cloud. So I think this entire thing helps organizations be more modern, and GPT may just be the catalyst that forces them to make that modernization push.
[00:22:05] Art Jannicelli: 100%. I lead a team of architects, and I've told my guys that with AI, containerization is no longer optional. This is something we have to embrace. One thing I've been surprised customers don't know is that you can actually break up your existing legacy applications that run in VMs and convert them to services that run inside containers. There are professionals who can help you do that. Now, not everything's going to fit perfectly, but there is a substantial amount that you can move in there.
Along with AI, it's something we need to consider. I've been comfortable with virtualization since 2005, but when I talk to customers now, they need to start thinking about data locality. What are you going to run in the cloud? When are you going to run on-prem? To your point, what's going to be a container, what's going to be a microservice, what's going to be a VM, and then also where does SaaS fit into your play? How are you going to integrate that and security?
It's a very large conversation, but circling back to GPT in a Box, if you can have all of these components in one platform that you can scale in a linear fashion, it makes your life a lot easier. And as you were saying earlier, if you're running NC2, you can move these things back and forth, giving you a way to ease yourself into this.
Even with microservices, you might be thinking, "Oh, that's cloud-only." Actually, I have clients who run containers to run the basis of their microservices and run their microservices as extensions of their container environment. So you can set up something very similar to what you might run in a hyperscaler inside your own GPT in a Box. Then you'll be able to scale that back and forth with NC2.
Don't think of this as containers being passed by microservices. You can still do a lot with them in conjunction with each other. I think the real challenge, though, is the Broadcom acquisition. Customers are looking at containerizing or moving to the cloud or microservices, but then they find out their application is only certified to run on VMware. This move is going to be something where you're going to have to ease yourself into it and test things out. Back in the early 2000s when I was doing virtualization, I would hide the VMware icon and then call the vendor to ask if it worked. They'd say, "Oh yeah, this one's great!" Then I'd reveal it was virtualized.
[00:24:40] Matt Baran: I remember those days. It was unthought of to have virtualized hardware and a virtualized operating system. Vendor support was tricky. To your Broadcom comment, I think a differentiator on the Nutanix side, as far as our Kubernetes platform is concerned, is that we're really based on vanilla Kubernetes. So again, no lock-in. VMware has a containerized platform, or I should say Broadcom at this point, Tanzu, but it's got some proprietary components to it. That makes it challenging and you're locked in. You have some proprietary components that make it difficult to portabilize some of those things.
Namely, their kernel is still challenging to work with sometimes, from a security perspective or observability. So definitely, the fact that we're leveraging vanilla Kubernetes and have that portability again just further emphasizes our stance on choice and no lock-in. We want you to be successful, hopefully by using our software platform, but we're going to make that as easy as we can for you.
[00:25:43] Art Jannicelli: You've mentioned the open Kubernetes platform that you guys are in, and I'm assuming that you've got published APIs and you're ready for integration with these partners that you're offering choice through.
[00:25:58] Matt Baran: Absolutely. We could have a whole second podcast and go really deep and gritty on Kubernetes. It would probably take us four and a half hours just to sort of get through the first layer, but yes, absolutely. It's really well ingrained in our product. All of our vendors that we work with, all of our OEMs, we're really tightly integrated with what they're doing and making sure that everything is successful. We really pride ourselves on that, and we definitely feel like Kubernetes containerization is a path towards the future, so we continue to innovate there.
[00:26:32] Art Jannicelli: One thing that I really genuinely appreciate about Nutanix, having worked in the startup space like you have, is that if you're a customer or someone new to Nutanix, you can just Google "Nutanix Bible." Nutanix publishes all of their documentation. GPT in a Box, even as new as this, exists in the Bible. I'm so impressed because, having worked with the startup community, there's a lot of faking it 'til you make it. Nutanix publishes everything; it's right there. Nutanix doesn't hide things. If you have questions, I've always had great success talking to people like Matt in getting help. So if you want more details, besides reaching out to Matt, you can just Google the Nutanix Bible or GPT in a Box, Nutanix Bible, and you'll see all the in-depth detailed information you're going to need to fully understand this.
[00:27:25] Matt Baran: To even go a step beyond the Bible, which is a fantastic resource that we're truly blessed to have, we've actually published a technical paper on our GPT in a Box 1.0 release. It's several hundred pages long and walks you through the entire solution and exactly how we built it. There are no secrets, no gatekeeping. It shows you every single command, every version of every piece of product that goes on there, every open source component. So if you have the hardware in your lab, you could build our GPT in a Box 1.0 with that validated design that we've published and made available.
[00:28:02] Art Jannicelli: Wow, that's excellent. It sounds like Nutanix is giving you all this choice. They're here to support you. I'm really impressed and looking forward to working with you a lot more on this. Are there any new Nutanix promotions to help customers get started, whether NC2 or on-prem?
[00:28:30] Matt Baran: We're always running different promotions. I urge everyone who has interest to reach out to your account executive at e360 and take a look. We're definitely offering promotions all the time for different configurations and spaces.
[00:28:44] Art Jannicelli: That's great to hear. Matt and I are here for the bigger conversations, whether you want to talk business cases or how this fits into your infrastructure as it is. As you dive into the documentation from Nutanix, questions are bound to come up about how this fits into your network or what else you can use it for. As I've worked with Nutanix over the years, I've always been continually impressed. Like Matt mentioned, the storage can do file, block, and object. I mean, how many places can do that? And besides the resiliency through that, Nutanix even has backup capabilities as well.
[00:29:30] Matt Baran: Yes, we have inbuilt backup capabilities and replication capabilities, but we also work with our industry partners to extend that support. So wherever our initial support drops off, we have industry vendors that help pick up after that. There's a ton of platforms out there that we very closely integrate with.
[00:29:50] Art Jannicelli: That's great to hear. Matt, I really appreciate you taking the time with me today and covering what a great solution GPT in a Box is. We're looking forward to helping any organization jumpstart with a turnkey AI-ready, resilient data platform. Is there any closing advice you'd have for customers?
[00:30:16] Matt Baran: I would advise customers, prospects, people interested to genuinely take a look at the field of solutions out there. I think after looking at the various options, you'll probably come to the same conclusion, which is that Nutanix is the best platform for Enterprise AI and modern apps. It's scalable, secure, and cost-effective. The data services are built in. I think once you look at it, you won't look back.
Our NPS score represents that very well. NPS score being a measure of how customers value your product, ranges from negative 100 to positive 100, and ours hovers in the 95-ish percent range for the last eight or nine years. Customers who come to Nutanix love Nutanix. So if you're looking at modern apps, enterprise AI, absolutely please give us a look. I think we can change your outlook on the world.
[00:31:18] Art Jannicelli: Absolutely. That NPS score is so impressive on top of the fact that you're in several Gartner magic quadrant leaders. If you go to Nutanix, you're not taking a step back or making a compromise. You are getting best of breed, cutting-edge technology, and we're here to help you. In closing, thank you again, Matt.
[00:31:38] Matt Baran: Thanks for having me.
[00:31:39] Art Jannicelli: I encourage all viewers to make sure you subscribe to our tech session podcast, and we look forward to seeing you again in the future.
[00:31:45] Matt Baran: I look forward to being on the next one. Thanks. Thank you.
