Overview:
In this episode of the Tech Sessions podcast, e360 Kevin Kohn, VP of Cloud Services, Jeff Dickman, Senior Director of Cloud Architecture, and Josh Thornes, Senior Cloud Application Architect, discuss the latest advancements revealed at Google Cloud Next '24. They explore significant updates in AI-driven code assistance and the integration of sophisticated cloud architectures, assessing their impact on enterprise IT strategies and developer workflows.
The panelists discuss how these innovations can streamline processes, enhance security, and drive efficiency across cloud environments. They provide insights that are crucial for IT professionals and tech leaders aiming to harness Google Cloud's capabilities to foster innovation and robustness in their technology stacks.
Listen to the Episode:
Watch the Episode:
Key Topics Covered:
- AI-Driven Code Assistance: Discussion on new tools that automate and enhance coding processes, making development faster and more efficient.
- Cloud Architecture Innovations: Insights into the latest infrastructure technologies and their impact on building more robust and scalable cloud environments.
- Application Development: Examination of advancements in application frameworks that support greater agility and security.
- Integration of Cloud Infrastructures: Analysis of how new integrations can facilitate advanced security and operational efficiency.
- Impact on IT Infrastructure: Consideration of how these innovations influence overall enterprise IT strategies.
Key Takeaways:
- Enhanced Efficiency: AI-driven code assistance tools are set to significantly reduce the dependency on traditional coding methods, allowing for quicker and more efficient development cycles.
- Improved Security: Advances in cloud architecture and integrated infrastructures offer better security measures, crucial for protecting enterprise data and applications.
- Increased Scalability: New technologies in cloud architecture provide the groundwork for more scalable and flexible IT environments, supporting growth without compromising performance.
- Strategic IT Innovation: The insights discussed provide a roadmap for IT leaders and professionals to leverage Google Cloud technologies to drive innovation and maintain competitive advantage in their respective industries.
Read the Transcript:
[00:00:11] Kevin Kohn: Welcome to the cloud technology session summit today.
[00:00:13] Kevin Kohn: I'm Kevin, your host today. We're going to do something special. We're going to recap the incredible announcements that we had from Google Next 24 that we just attended. We've got the perfect guests to help us unpack all of the, we learned and all of the announcements that were brought to everyone's attention in that session.
[00:00:33] Kevin Kohn: Jeff, our resident Google architect and Josh, our AI wizard. Guys, you ready to dive in?
[00:00:40] Jeff Dickman: Yeah, for sure, Kevin. Next is always a highlight for Google. So, it was a pleasure to be there and it definitely didn't disappoint.
[00:00:47] Kevin Kohn: It was pretty crazy, wasn't it, Jeff? I mean, a lot of announcements, a lot of fascinating people to talk to, a lot of engaging architects, and just the, the excitement was palpable.
[00:00:57] Kevin Kohn: That's, that's for sure.
[00:00:58] Kevin Kohn: Yeah, for sure. And I logged about 30 miles, so that was great too. That's great.
[00:01:05] Deep Dive into AI Announcements and Code Assist Features
[00:01:05] Josh Thornes: I was really excited also about all the exciting, AI announcements. So there's a, there's a lot there. So I'm really excited to be able to talk about that today. That's great. Well, look, why don't we go ahead and get started, Josh?
[00:01:18] Josh Thornes: And what are the biggest announcements or updates that you found interesting as you paid attention to those?
[00:01:25] Josh Thornes: Oh, man, just kind of top of mind, there is a few really exciting ones. There is a great demo of, you know, some of, you know, how to build an AI agent for your website or some of the code assistants, or just some of the AI building.
[00:01:38] Josh Thornes: So there's, there's definitely some really cool ones, and hopefully we'll in later sessions be able to dive more into them, but, those are the ones that are top of mind for me.
[00:01:47] Josh Thornes: Well, Josh, you know, it's really fascinating because we heard a lot about Gemini Code Assist. During the week, and can you tell us a little bit of what that has to do with helping you [00:02:00] encode and, and, and talk, talk a little bit about what that announcement gave you.
[00:02:05] Josh Thornes: And, you know, typically when you're going through and you're building things, as a builder, you're looking at, you know, having to look at, you know, stack overflow and find things through search. And you don't have to do that any longer. So that was really exciting to see just kind of that integration into the, into the coding, your IDE, your coding environment, or looking at, you know, like multi line completing of code, or being able to prompt things like instead of having to go out to, ChatGPT and put in a prompt and then bring it back.
[00:02:34] Josh Thornes: You can go through and just do that right in line in your code editor, or just looking at, you know, even smart actions and pulling in, you know, some of those things that you want to like create a unit test or create something like a block of comment. And so there's some really cool things that you can do that will help to expedite some of your coding process.
[00:02:56] Kevin Kohn: That's awesome. Jeff, what did, what was stand out to you as you listened to the code assist and the things that Josh has brought up to us?
[00:03:04] Jeff Dickman: Yeah, you know, there's, there's so much that happens when you're coding and, you know, we all have bad habits and, we also have creative workarounds, that come back to bite us later.
[00:03:13] Jeff Dickman: I've had a few customers where, they've leveraged what you could call a bug. Within their code to to get something done and then when that bug gets patched suddenly Their code doesn't work anymore. And so some of those things can be avoided with code assist, which I think is really fantastic because you're able to prevent those future headaches from from even happening
[00:03:32] Josh Thornes: I also liked on code assist too.
[00:03:34] Josh Thornes: I don't know if you saw this Jeff, but inside of some of the sessions, you can kind of tailor some of your code to have guardrails, so that way you're not having like loss of intelligence around, you know, some of your code, but also you put some of those guardrails in where you're like saying like, Hey, you know what, I really want to see some of this code style come through in our code assist.
[00:03:53] Josh Thornes: And so. I like some of those kind of like, kind of like either soft or hard guardrails that they introduced in some of [00:04:00] their announcements.
[00:04:01] Kevin Kohn: Yeah, those are really nice because what they do is they, they ensure that your, your coding standards are consistent across the board. And you're, you're not really going to be looking at a mishmash of different styles.
[00:04:10] Kevin Kohn: As you, as you read through code. So it makes it reading easier and, easier to update to down the road when you need to, because you don't have to keep changing context on what you're thinking is happening.
[00:04:20] Josh Thornes: Yeah, I agree.
[00:04:21] Josh Thornes: Well, Jeff, as an architect, but you know, bugs are one thing, but are you thinking about broader code patterns as part of this,
[00:04:27] Jeff Dickman: Yeah, there are, there are broader code patterns here.
[00:04:29] Jeff Dickman: I mean, and it's. Some of this also speeds up, right? If you're talking about deploying, infrastructure into Google and you're, you're writing Terraform being able to quickly get to, let's just say it's a VM that you need to spin up with a certain config, Code Assist can help you with that. So it's able to quickly get you going as far as like different coding styles, or if you're using a Python CDK or something like that, Code Assist will, will be able to jump in and accelerate your development time, so that you can, you can get to deploying faster and work on your applications.
[00:04:59] Josh Thornes: One of the things, one of the things I also liked that was kind of it's kind of like a smaller nuance announcement, but being able to interpret some of your code. And so you can ask questions of the code assist and it will interpret some of that code and tell you about it. So it's a great way of kind of understanding your code base or even just one page of code.
[00:05:20] Josh Thornes: So kind of a cool, follow along with what Jeff had brought up.
[00:05:24] Kevin Kohn: So it's not just about fixing typos anymore. It's upping the game all over across the spectrum. Wow. Wow. That's pretty powerful. Jeff, what are your thoughts about that?
[00:05:35] Kevin Kohn: I think it's definitely really powerful because what it does, it allows when you hire resource, it allows you to get them up to speed faster, right? It's not just looking at your code and saying, hey, you've got a bug here, you know, running just a basic debugger in there, but it's really like getting people up to speed faster.
[00:05:57] Kevin Kohn: So if you have like a junior developer or something like [00:06:00] that, you can have them be productive much more quickly because they'll be learning best practices as they work in real time.
[00:06:08] Josh Thornes: What's fascinating about that. And I'm having to deal with this now where, you know, youth are coming out of school, out of, out of college.
[00:06:16] Josh Thornes: They want to get their first gig. They want to be involved in some interesting projects. Using tools like this, help them ramp faster into a company so that they can get more experience and be more effective. And that's, that's a win, win, I think, for both the employer who's kind of putting themselves out there and the student who's coming in looking for their next career.
[00:06:39] Josh Thornes: I think that's pretty impressive.
[00:06:42] Jeff Dickman: Yeah, I think it's pretty impressive too. It's also helpful because a lot of Developers, they, they work interesting hours. So, it's one of those things where at 2 AM you have this great idea, but you're not entirely sure how to implement it. You now have, you know, effectively and somebody that you can talk to who you can bounce that idea off of and potentially, you know, come to a conclusion on what you're working on instead of laying in bed, thinking about it for the next six hours.
[00:07:06] Kevin Kohn: This is a kind of a sea change, I think in the industry. I mean, we, I think we've all been around long enough to see things move and change and gradually morph, but this is exciting. This is something that moves the needle in a meaningful way, I think.
[00:07:20] Jeff Dickman: Yeah, for sure. I
[00:07:21] Jeff Dickman: agree.
[00:07:25] Kevin Kohn: So Josh, from a dev standpoint, how are you feeling about the, the pain that you have to go through on the infrasight and, and going through your projects? And, and, you know, what, what are all those things that you have to deal with that you think that this might adjust?
[00:07:41] Gemini Cloud Assist[00:07:41] Exploring Cloud Assist and Architectural Insights
[00:07:41] Josh Thornes: You know , I think kind of what you're talking about is that the Gemini Cloud Assist, is that right?
[00:07:46] Kevin Kohn: Yeah.
[00:07:47] Josh Thornes: Yeah. I was really excited about that announcement because, you know, there's always like emerging changes as far as, you know, infrastructure or new patterns and practices emerging. And what I was really excited about that is, is [00:08:00] that, you know, like it could be something simple, like, you know, building out a cloud function, or how to like Change or, you know, created a trigger off of, cloud storage.
[00:08:11] Josh Thornes: There's always different patterns and practices to be able to do that from a developer's perspective or a builder's perspective and with cloud assist, I was really excited because, I'm assuming that some of this, I don't know that I saw any announcements around it as far as like how frequently it will be updated, but I'm really excited about being able to, as you're building, being able to ask questions about, you know, what reference architecture I should be using.
[00:08:36] Josh Thornes: And so being able to just ask a question and it be able to deliver on not just like a single line of code, but like a reference architecture for patterns and practices for like. A mobile application and how best to roll that out. And so I was really excited about that announcement and just how some of that reference architecture will or knowledge base instead of having to dig through Google searches or Google Docs , being able to just pull that knowledge out of the agent and be able to use that reference architecture, which helps to expedite the development process.
[00:09:08] Josh Thornes: So I was really excited about that. The cloud assist.
[00:09:12] Josh Thornes: Yeah, and I think that it also, it also sort of democratizes cloud. Um, because what it allows you to do is have developers who may not know exactly what infrastructure they need to run an application and they can in plain language ask cloud assist kind of what they're thinking of they want to do and how they want to do it and they can put together Something that can go to architecture as, you know, more than just a first pass.
[00:09:36] Josh Thornes: It can actually be, you know, a little bit more solid so it can speed up the time of going from, you know, concept through prototype to production. So there, there's a lot of advantage there for developers to step in and sort of help define what the architecture is going to look like for their applications.
[00:09:52] Josh Thornes: Having been an architect my whole life. I mean, there's so much that goes into first learning the platforms, understanding the nuances, getting into all the trainings, and [00:10:00] then seeing how it interacts with whatever workload you're throwing at it from a performance standpoint, throughput, you know, all the things that go into the back, the CPU, the storage, everything has to come together.
[00:10:11] Josh Thornes: This seems like it's kind of going down that path of kind of taking the complexities out of all that activity that you would normally have to address and learn and, and spend time building skills in so that you can just get right to the workload. I mean, is that kind of how you saw it, Josh, or did I overrepresent that?
[00:10:28] Josh Thornes: No, I definitely agree. And, Jeff, you probably can appreciate this from an architecture perspective, you know, locking down the different layers, right. I'm, assuming that would expedite that area too. Right.
[00:10:41] Josh Thornes: Yeah, I mean, when you when you talk about it, you describe it. You can sort of say I need to provision this infrastructure.
[00:10:46] Josh Thornes: I needed to connect to this other service or I needed to have this capability and you can overlay your security context into that conversation, and so it's able then to say, okay, you need these security groups and you need these VMS and maybe you're trying to connect to a database. So here's the policy you need to to be able to make that happen as well.
[00:11:04] Josh Thornes: I agree. With, you know, like building out a full mobile application, for instance, there's a lot of layers that you have to consider, right? And that architecture can sprawl fairly wide, and what I love about the cloud is you can easily spin up those services, but there's a lot of things to consider, right?
[00:11:20] Josh Thornes: And of that checklist or reference architecture, you may forget about something. And so, and Kevin, you kind of brought up earlier, you know, some of those developers that are coming out of school, they may not be aware of all those different layers, you know, whether it's a database layer or all the way out into like prompt or, you know, SQL injection.
[00:11:38] Josh Thornes: Right. And so just kind of locking down all those layers. I'm excited about the possibility of being able to spin some of that up even quicker.
[00:11:46] Vertex AI Updates and the Power of Prompt Management
[00:11:46] Kevin Kohn: Well, if we can bounce into the next item, which came up Vertex AI goodies. So a lot was announced there, a lot, a huge push. What stood out to you, Josh?
[00:11:56] Josh Thornes: Yeah, you know, two areas that were really exciting to me.
[00:11:59] Josh Thornes: One [00:12:00] was prompt management or prompt sharing and versioning. The other was just kind of looking at the possibility of quickly building out an agent. So those are two areas that I was really excited about. And what about demystifying the AI development? So, Vertex AI has always had my head spinning with all the possibilities that are available to you.
[00:12:22] Josh Thornes: What about prompt management? Let's start there.
[00:12:25] Josh Thornes: Yeah, you know, with each model, you're going to have different prompts that you need to worry about, right? And so like, you know, temperature or, you know, just different settings or APIs and so on. And so there's a lot of knowledge that has to go into some of that as far as building out the prompts.
[00:12:40] Josh Thornes: And, as we're going through, you know, like we can stand on the shoulders of some of what Google has already done as far as on some of the models. And be more effective at looking at some of those, models and how to leverage them, looking at what they've already dealt with prompts.
[00:12:55] Josh Thornes: But then also looking at within organizations, like maybe there's prompts that we've built that we want to share with each other. And so those are two areas that usually you spend a lot of time in that people under appreciate, and so I was really excited about that announcement.
[00:13:11] Kevin Kohn: That's great. So with regards to your day to day work effort, you know, how does it change using this, this capability?
[00:13:19] Kevin Kohn: You know, models are always getting built out new, right? And there's always new versions coming out. Just with this conference, Gemini 1. 5 came out, right?
[00:13:30] Kevin Kohn: And recently, Claude 3 came out with a few different models. And so for me, I'm really excited about, you know, just being more effective as far as like, either sharing prompts or how to best leverage those code bases. So for instance, that model, we can look at and say, Hey, you know what, I want to look at, instead of having to build, you know, document intelligence and pull that data out, I can use these models with these prompts and say, Hey, you know what, I'm going to pass in this list of JSON and being able to pull all [00:14:00] that data out of just using that awesome model. It can take care of me having to hand code a lot of these things. And so through prompts and this prompt engineering, me and other developers within our organization or within other organizations can be a lot more effective at using these awesome models. That's great. With regards to the next item, GKE preloading, you know, for a smoother AI deployments, faster rollouts, things like that.
[00:14:25] Kevin Kohn: What do you think about this so called superpowers of GKE's preloading capability? You guys have built some pretty complex systems in the past. You know, how does this matter to you?
[00:14:37] Kevin Kohn: This is, this is actually really important when it comes to, you know, using models and, and doing machine learning because when you're running in containers, the containers aren't necessarily up and running when the application starts or when the user makes a request of the environment.
[00:14:50] Kevin Kohn: And so that container has to be loaded, then it has to be started, and then depending on how you're doing your loads, you may have to then bring in your models into that, into that container, which can be a lengthy time process. For for customers to be sitting there waiting for an answer, and we have seen some occasions where, you know, there might be 10 to 20 seconds where a question is asked of the model, and there's just isn't an answer.
[00:15:12] Kevin Kohn: And depending on how your applications architected, you could see errors or just user frustration because, wow, this is really taking a long time to get to an answer for. So, you know, when you're able to say, I want this container running. And I want it ready to go. So it's, it's effectively there. When the customer asks the question, it can go to the container.
[00:15:30] Kevin Kohn: It can go to the model. It can get the answers it needs and bring that back quickly. That's a massive performance boost for, for the models, as well as something to improve customer satisfaction or end user satisfaction when they're using your, your particular tool or product.
[00:15:45] Kevin Kohn: Excellent.
[00:15:46] Advancements in Data Management and Vector Search
[00:15:46] Kevin Kohn: Well, the next topic is that we want to get into is data.
[00:15:49] Kevin Kohn: So they announced this enterprise truth or the the panacea of the end of data disputes is how they kind of thinking about this.
[00:15:59] Kevin Kohn: You know, [00:16:00] when we start talking about enterprise truth, you know, it sounds like a big vision or a goal.
[00:16:05] Kevin Kohn: We've heard of data lakes. We've heard of all this, ability to congregate your data and to centralize it.
[00:16:10] Kevin Kohn: What is the goal of, of enterprise truth and where is it taking us?
[00:16:15] Jeff Dickman: So it's, it's a pretty broad definition. But it's working to solve something that, as you said, most organizations have experienced. Your data is all over the place.
[00:16:24] Jeff Dickman: You might have the same data in more than one location and it might be slightly different or one may be a little bit more outdated than the other.
[00:16:31] Jeff Dickman: And so you and I run a report on, let's say it's sales numbers and your numbers say we did really fantastic and my numbers say we did okay. Who's right? Whose numbers are correct? How do you know who's correct? And so that's part of what Enterprise Truth is working to, to solve. It's about putting the data in one place.
[00:16:49] Jeff Dickman: It's about verifying it. It's about putting governance on top of that data, so that you know that what's being updated from all the other systems is going to be there. It's going to be current. And then if you run your report and I run my report, we're pulling on the same data, so we should have consistency in numbers coming back.
[00:17:06] Kevin Kohn: So Josh, what does this mean from the process side? And we talk about truth and how do we define it and whose data wins over who?
[00:17:14] Kevin Kohn: Walk us through what it means in that perspective.
[00:17:16] Josh Thornes: And, you know, there's a few different challenges, you know, AI is delivering on the promise of big data.
[00:17:22] Josh Thornes: And so like, there's always things that you have to worry about as far as data prep, right and data engineering and bringing the data forward and, It is like Jeff had brought up, it's really important to understand whose data wins.
[00:17:35] Josh Thornes: But when I was really excited about with this enterprise truth also was just the ability to have, Who does win? But then also be able to quickly index some of that data.
[00:17:46] Josh Thornes: And so that, with sharing who actually won on some of the data, but then also quickly being able to index it, the indexing of data is actually a big challenge when you're trying to bring, forth some of this data into [00:18:00] AI. And so I don't know that that was necessarily like the intent of, you know, kind of the announcement, but it did seem like that was kind of like the, underlying benefit of doing some of these investments.
[00:18:12] Josh Thornes: Excellent. Well, thank you. The next thought I was having is around vector search, you know, this whole idea of finding the needle in the haystack. I think you bring it up. You know, it sounds a little bit like, oh yeah, it's a grand promise. You know, we keep hearing about this. How close are they really to delivering on this?
[00:18:34] Josh Thornes: You know, vector search, we all love how Google works, right? And how you can just find things, right? And you kind of marry these awesome AI agents, you know, with search and then with all of their databases, they started adding all of these vector indexing and then out on Firebase and Firestore, adding some of this really cool vector searching to it.
[00:18:56] Josh Thornes: Man, it's just. They keep on adding these really cool components that makes app dev even easier. And so being able to, you know, either for just search within a website or a search within an agent or indexing data within an agent. I was really excited about this. I don't know how big it was as far as for other people, but for me, you know, coming from a builder perspective, you know, being able to have all these databases, having these capabilities and being able to deliver those on the front end. It was really cool.
[00:19:23] Josh Thornes: So I was really excited about this one. It'll be fascinating to see how much it lives up to the hype, right? And, and get into this. Can't wait to get our hands dirty with this capability for sure. Definitely.
[00:19:37] Innovations in Applications, Infrastructure, and Security
[00:19:37] Kevin Kohn: Well, on the applications and infrastructure side, what cuts, what was the thing that caught your eye the most?
[00:19:44] Kevin Kohn: I don't know for others. But I always like to be really structured around my data and making sure that, you know, we're bringing all the insights around an application, you know, monitoring and so on. And so to me, App Hub was a really exciting one. [00:20:00] Just being able to, you know, invest in that area and make sure that, you know, App Hub was kind of taking care of our app application portfolio and application insights.
[00:20:08] Kevin Kohn: And so, that was a big announcement for me. I don't know, Jeff, what do you think?
[00:20:12] Jeff Dickman: Yeah, I think that one was pretty cool. I like the BigQuery continuous queries. I think there's just so much potential around automation when you're able to do continuous queries and you know, maybe even things as far as getting alerted when there's changes to data.
[00:20:26] Jeff Dickman: So you're constantly able to check your data and see if there's updates for things happening and then trigger immediate workflows based on that. So, I think there's a lot with the BigQuery continuous queries.
[00:20:37] Jeff Dickman: The other one that I really liked was the Confidential VMs. You know, getting deeper data security was just something to me that, you know, was, it's needed.
[00:20:45] Jeff Dickman: It's needed within the environments. And so I think that it's something that's gonna be really huge going forward.
[00:20:50] Josh Thornes: And another exciting one too was,... we saw just recently some of the NVIDIA announcements come out, right, and some of their exciting new GPU chips and, and so on.
[00:21:00] Josh Thornes: I really liked how Google is trying to integrate that natively into some of their compute components. And so a lot of those little things I'm looking forward to seeing and being able to leverage as far as like we talked about earlier, you know, just, you know, model model inference, which is kind of like, you know, the response times, but just all these little lightweight components that we can leverage through the Google platform and being able to pull that through and use these new, awesome GPUs.
[00:21:28] Josh Thornes: And so that was another area that I was really excited about.
[00:21:31] Kevin Kohn: Excellent.
[00:21:34] Wrapping Up and Looking Forward
[00:21:34] Kevin Kohn: Well guys, that was really insightful about all the changes that we were able to identify here in Google Next. I think we just, we covered some of the highlighted ones. There's obviously more that was there. I mean, Gemini AI, we had Vertex AI, talking about the AI capabilities from the buzzword into something that we can actually use.
[00:21:52] Kevin Kohn: The data never sleeps components. Google's tackling all those challenges head on with enterprise truth and there's [00:22:00] plenty of others that came up. Thanks a lot for your insights, Josh and Jeff. Appreciate you taking the time out today and looking forward to doing this again soon.
[00:22:08] Jeff Dickman: Pleasure. Thanks, Kevin.
[00:22:10] Josh Thornes: Thanks.
