Red Hat Previews OpenShift Platform for AI Models

At the Ray Summit this week, Red Hat previewed a platform for running artificial intelligence (AI) workloads based on the Red Hat OpenShift platform the company built atop Kubernetes.

Taneem Ibrahim, manager of software engineering at Red Hat, said Red Hat OpenShift AI will provide IT teams with a consistent distributed computing framework based on Ray clusters, which are designed to run workloads in parallel, combined with CodeFlare, a framework for training AI models.

Step 1 of 7

14%

Which of the following best describes how you currently monitor and manage systems across your hybrid cloud environment?(Required)

Multiple separate tools for different environments

Multiple separate tools for different monitoring and management functions

Single consolidated dashboard

Custom built monitoring and management solution

No full monitoring and management solution

Don’t know

How valuable would it be for your monitoring and management solution to incorporate predictive analytics to help you identify and address system issues that are critical and not waste time or effort working through all the data and “noise” that is not critical?(Required)

Extremely valuable

Moderately valuable

Somewhat valuable

Slightly valuable

Not valuable

How do you most typically address potential system issues BEFORE they impact your operations?(Required)

Automated workflows and playbooks to remediate potential issues as soon as they are detected

An orchestrated response process that leverages automation, AI, and human expertise to rapidly respond to potential issues

Advanced monitoring tools such as predictive analytics and anomaly detection to proactively identify and address potential issues

Respond to issues as they arise, using manual processes and troubleshooting techniques

A combination of automated remediation, orchestrated response, and/or proactive monitoring to address potential system issues

Don’t know

Other

What methods does your organization use to control and monitor cloud/IT spending?(Required)

Centralized procurement processes

Pre-approved budget thresholds

Automated cost monitoring tools

Department-level spending controls

Regular cost audits and reviews

Cloud cost optimization platform

Approval workflows for resource provisioning

Native financial operations tools provided by the cloud provider

No formal controls in place

Don’t know

Other

Please specify:

Which methods do you rely on most to detect security vulnerabilities in your systems?(Required)

Automated scanning tools

Manual assessments

Vendor notifications

Threat modeling (for early detection)

None of these

Don’t know

Other

Please specify:

What are the TWO most common ways you prioritize vulnerabilities to address within your environment?(Required)

Based on potential impact

Based on exploitability

Regulatory compliance requirements

Based on CVSS scores

Risk-based analysis of multiple factors

First-in-first-out approach

No formal prioritization process

Don’t know

How do you typically handle system updates and security patches?(Required)

Automated updates and patches

Manually review and apply updates and patches on a regular schedule

A combination of automation and manual review

Apply updates and patches on an as-needed basis.

Outsourced update and patch management

No defined process

Don’t know

Other

Δ

That approach will enable IT teams to invoke AI models more easily in real-time or process workloads in batch mode using scheduling capabilities that enable them to optimize usage of IT infrastructure, such as graphical processor units (GPUs), as required, said Ibrahim.

The Red Hat OpenShift AI platform provides access to KubeRay, a Kubernetes operator for deploying and managing Ray clusters, and CodeFlare, a Kubernetes operator that deploys and manages the life cycle of the three components of the framework used to manage jobs and dynamically aggregate and scale resources as needed.

In addition, there is KKerve, a Kubernetes Custom Resource Definition for deploying and serving models, a Text Generation Inference Server (TGIS) for loading inference engines, and Caikit, an AI toolkit/runtime that handles the life cycle of the TGIS processes.

There is also OpenShift Serverless, a framework for driving event-driven applications based on the open source Knative project and OpenShift Service Mesh, an instance of the open source Istio service mesh.

Kubernetes platforms such as Red Hat OpenShift are generally preferred when running greenfield AI applications where technical debt is much less of an issue, noted Ibrahim. However, as more organizations begin to embed AI workloads within applications, there will be a more pressing need to integrate DevOps and machine learning operations (MLOps) workflows. Most MLOps workflows today are managed by data scientists, but when it comes to deploying them, an AI model is treated as just another type of software artifact invoked via an application programming interface (API). The Red Hat OpenShift AI platform will make it simpler for those teams to collaborate both before and after AI models are deployed and continuously updated, noted Ibrahim.

It’s still early days as far as operationalizing AI models is concerned, but it’s now only a matter of time before organizations will need to find some way to meld DevOps and MLOps workflows to enable the level of observability that will be required to succeed, said Ibrahim. To rise to that challenge, organizations will need to address a range of technical and cultural issues that inevitably arise when individuals with diverse backgrounds need to collaborate. While data scientists tend to have a lot of experience building AI models, their knowledge of deploying them in production environments is often much less extensive.

In the meantime, IT teams would be well-advised to become a lot more familiar with the nuances of building and deploying AI models, as they will soon be pervasively embedded within every application imaginable.