Red Hat Tool Lets Developers Explore AI Applications Locally
Developers, frontend or otherwise, need a way to tinker with AI models. A new offering by Red Hat allows frontend developers to do just that by working locally with AI models.
Podman AI Lab lets developers explore models on their desktops in order to create applications that are more than chatbots, said Stevan Le Meur, the principal product manager for developer tools at Red Hat.
Much of the focus has been on how AI will revolutionize how applications are built, but it will also revolutionize the applications themselves, Le Meur said, adding that it’s already becoming the new norm.
“We enable the developers to work locally, but we also give them best practices on how they could deploy their application onto production, so it’s a tool,” Le Meur told The New Stack. “We want to enable them to discover what they can do with AI, and we also want them to be able to experiment quickly with the model.”
How AI Applications Are Different
It also allows developers to focus on the organization’s own data, rather than just using generically trained models.
“The data [is] the fuel for the AI application,” he said. “It’s not only about the APIs themselves. It’s also about how you are going to build the entire pipeline.”
Developers will find that AI applications are different to build because when working with large language models (LLMs) or other GenAI models, it’s necessary to spend a lot of time building blocks that will control the behavior of the model, he said.
“There is always this indeterministic aspect of the model that will give you some very tricky results that you want to be able to handle because you don’t want to expose that to your end user,” Le Meur said.
That’s one reason why, until now, data scientists have owned the process of building machine learning applications.
“There’s a little bit of this dissonance between data scientists on one side, who were building a lot of things already, and the application developer, who may not be familiar with the technology, who may be a little bit afraid about what AI means in general, and how you work with AI,” he said. “With the rise of GenAI, you can see that it’s coming much more into the mindset of the application developers, and now application developers are open to modernizing their application and to start thinking about how AI can be helping.”
Key to AI success will be setting up the right processes between data scientists and developers, ensuring developers aren’t “stuck into a corner,” he added.
For developers of all stripes, that will mean becoming more involved with data.
Working Locally with Data and Models
Podman AI Labs, which became available in May, is an open source extension for Podman Desktop, a tool that helps application developers containerize an application and work with Kubernetes on their local developer environment, he said.
Podman AI Labs can also support preprocessing, RAG (retrieval augmentation generation) or post-processing of data, and then transitioning the application to platforms for managing containerized applications, such as Kubernetes or OpenShift, which is Red Hat’s cloud native application platform built on Kubernetes, he added.
“As an application developer, working locally is very important, and I want to be able to test things, quickly iterate and get the benefits of all the different tools that I am using already in my developer environment,” Le Meur explained.
It’s also designed to address the issue of cost when it comes to using APIs to connect to AI models.
“Obviously, when you run things locally, you just leverage the compute that you have on your local environment, and that is making things a little bit cheaper than calling an API that is somewhere on the cloud, which can be very costly as well,” he said.
Podman AI Labs allows developers to have the AI model on cloud services while still keeping data in-house. Running data locally also helps ensure the data is secure and not leaked out to third-party vendors, who might use the data to retrain their own models, he added.
Frontend Development Use Cases
Chatbots were the first stage of AI application development, but AI is quickly moving into other use cases, Le Meur said.
Developers who want to build with AI can start by looking at ways to optimize existing applications and make them more efficient, he said. They might also consider how AI could add new experiences for customers or business users.
“Then you need to look at, what [is] the data that I have that [is] unique to my business, that [is] unique to my knowledge, that can be very thoughtful for building an experience, that is going to differentiate myself and that is going to help my business,” he said.
To help developers jump-start the creative process, Podman AI includes a Recipes Catalog that offers open source “recipes” to help developers identify common AI use cases and solutions. Frontend developers will find use cases for their work there, he said. They can simply take the model and run it locally.
“They will then have an inference server with APIs, and those APIs are the one that we see more or less becoming standards, which are the OpenAI APIs, so we are compatible with them,” he said. “The developers can build their application, connect to those APIs … and they will also have a way to debug their application. They will be able to see how they can package their application as well to deploy them onto the cloud.”
Le Meur says the intersection of the user interface, user experience and AI is key.
“It’s being smart about how you can best utilize your data and how you can also create this competitive advantage by going looking at how AI can help your application, help your business, and looking at the intersect between UI, UX and AI,” he said.
