Stop Managing Infrastructure: A New Way to Deploy AI Agents and Models

Standing up an agent as a production service on Kubernetes means five YAML files, a few hundred lines between them, and (in most enterprises) a ticket in someone else’s queue. On the Workload API it means one spec file, one command, and about five minutes to a live URL. No manifests, no kubectl, no namespace, nobody else in the loop.

AI workloads increasingly look like long-running services, not request/response models. Agents are the clearest example: they hold state, call tools, wait on LLM responses, and run for minutes or hours at a time. The same is true of inference servers, RAG pipelines, and the frontends that sit on top of them. In most enterprises, turning any of these into a production service means Kubernetes: namespaces, Deployments, Ingress rules, autoscaling policies, health probes, and a platform team in the loop for every change.

Today we’re announcing the general availability of the Workload API: a single layer for deploying and governing AI services on DataRobot. Bring a container image that serves HTTP; you get a stable URL with autoscaling, monitoring, and sharing, with a lifecycle you drive end to end through one API. For you, that means no Kubernetes manifests, no kubectl, and no platform ticket. The governance underneath is what buys you that: because every workload is a governed object by default, your platform team never has to choose between reviewing your deployment and handing you cluster credentials.

What stands between a working service and a production one

Shipping a long-running AI service on self-managed infrastructure typically requires:

• A cluster, a namespace, and permissions to use them
• Deployment manifests, Services, and Ingress configuration
• Autoscaler tuning and node pool planning for GPUs
• Liveness and readiness probes, wired up correctly
• Log aggregation, metrics, and tracing, assembled from separate tools
• A platform engineer involved in every version rollout

None of this is the service itself, and every item lands on someone. Either the AI developer learns Kubernetes, or a platform team fields the ticket. At enterprise scale, IT ends up choosing between two bad options: become the bottleneck for every AI deployment in the organization, or hand out cluster permissions to teams whose job is building agents, not operating infrastructure.

Generic serverless container platforms remove part of the setup, but they stop at the URL. What they don’t hand you is an identity: a governed object that carries sharing, monitoring, and an immutable production version, and that survives the trip from the thing you were iterating on to the thing your company depends on. They also don’t give you AI-native observability, an answer when a compliance team asks who can invoke a service and what it has been doing, or an autoscaler that understands KV-cache pressure instead of CPU. The Workload API keeps the one-command experience and adds the part that makes a service shippable inside a company.

Artifacts, workloads, and protons

A deployment layer is only useful if its model is small enough to hold in your head. The Workload API reduces the infrastructure surface to three objects:

• Artifact   → what to run (image, port, entrypoint, env vars, probes)
• Workload   → the governed identity (stable URL, sharing, monitoring)
• Protons    → the running instance(s) backing the workload

The artifact describes what to run. The workload is the governed identity you hand to consumers. Protons are the execution. Scaling is a replica count. GPU selection is a bundle name rather than node pools and taints. The Workload concepts and Artifact concepts pages cover the full model.

The API is container-shaped by design. Agent services built on LangGraph, CrewAI, or custom orchestration run alongside model inference servers (NVIDIA NIM, vLLM), RAG pipelines, MCP servers, vector databases, and Streamlit or Gradio frontends. Any service that listens on HTTP fits, so an application and the services it depends on can run on one platform with one lifecycle.

Deploy in one command

Describe the workload in a spec file (YAML or JSON), then create it with the DataRobot CLI. One command creates the workload, schedules the container, and returns a stable endpoint URL.

# spec.yaml
name: support-agent
artifact:
  name: support-agent-artifact
  type: service
  spec:
    containerGroups:
      - name: default
        containers:
          - name: agent
            imageUri: your-registry/support-agent:1.0.0
            port: 8080
            primary: true
            readinessProbe: {path: /health, port: 8080, initialDelaySeconds: 5}
            environmentVars:
              - name: LOG_LEVEL
                value: info
              - source: dr-credential      # injected from the DataRobot credential store
                name: OPENAI_API_KEY
                drCredentialId: <credential-id>
                key: apiToken
runtime:
  containerGroups:
    - name: default
      replicaCount: 1
      containers:
        - name: agent
          resourceAllocation: {cpu: 1, memory: "512MB"}

dr workload create --spec-file spec.yaml

The spec has two halves. The artifact half carries everything that travels with the image: port, entrypoint, environment variables, probes. The runtime half carries what varies per deployment: replicas, CPU, memory. Note the environmentVars block: plain values are passed as-is, and secrets are injected by reference from the DataRobot credential store. The API key never appears in the spec, the image, or version control. Check progress and grab the URL:

dr workload status ${WORKLOAD_ID}     # submitted → launching → running
dr workload endpoint ${WORKLOAD_ID}   # the stable URL
dr workload logs ${WORKLOAD_ID}       # container logs

Once the status reaches running, the service is live on a stable URL. What you’ve created is a draft workload: a real endpoint with full monitoring, free to iterate on, and cleaned up automatically after 8 hours of inactivity. Production is one call away and this is the part with no equivalent on a generic container platform: promoting doesn’t redeploy anything. The workload ID, the endpoint URL, and everyone you shared it with all stay exactly as they are, and the artifact locks so production runs the bytes you tested. The thing you iterate on and the thing your company depends on are the same object.

There are other ways to run the same flow. Everything the CLI does maps to REST calls, so plain curl works. The DataRobot Pulumi provider and Terraform provider expose artifacts and workloads as native resources, so workloads can be managed as code: diffable, reviewable, and reproducible across environments. Code-to-Workload builds the container from source, with no Dockerfile or registry push. And the DataRobot Agent Skills plugin lets you create, scale, and debug workloads conversationally from Claude Code and Cowork.

Serve the models behind your agents, too

An agent is only as good as the model endpoint it calls. The Workload API runs generative AI models alongside your agent through two primary paths: seamless integration with NVIDIA NIM, and deploying open models directly from Hugging Face.

For NVIDIA NIM, microservices deploy as a first-class artifact type using the NIM Operator (currently available on self-managed DataRobot on OpenShift). Any model in the NGC catalog—such as Nemotron 3 Nano Omni—can be served with optimized GPU performance and managed weight caching. Alternatively, you can host open-source models directly from Hugging Face using inference servers like vLLM. In both cases, model weights are cached efficiently on persistent volumes, credentials are injected securely from the DataRobot store, and models run on identical GPU bundles with full autoscaling.

The result: your agent and the models powering it run side by side as governed, independent endpoints with unified observability and security.

Day-two operations through the same API

Deployment is one command. The operations that usually require Kubernetes expertise go through the same API::

• Promoting to production. The agent starts as a draft: iterate freely while it behaves like a real service. When it’s ready, one call promotes it: the artifact locks (immutable and versioned, so production runs exactly what you tested), the draft TTL is removed, and the workload ID, endpoint, and sharing all stay the same. No redeployment, no environment migration.
• Diagnosing a workload that won’t start. Every workload exposes a lifecycle event log and per-replica status, including container readiness, restart counts, and a log tail. Image pull failures and crash loops are visible through the API and CLI.
• Observing what the service is doing. Container logs are collected out of the box, with no instrumentation required. For traces and metrics, instrument the container with OpenTelemetry: standard OTel instrumentation ships traces, metrics, and structured logs to DataRobot. For an agent, that means seeing individual LLM calls and tool invocations inside each request. 
• Monitoring health and utilization. Service health, resource utilization, and quota consumption are tracked per workload with no instrumentation, in the same panes as the rest of the platform. In practice: you can see whether a replica is saturated or idle, whether a restart count is climbing, and whether you are about to hit an org-level scaling cap — before any of it becomes an incident.
• Moderating traffic in real time. Guards from the DataRobot evaluation and moderation library attach to a workload and run in the request path, scoring quality, tracking token cost per call, and blocking unsafe or non-compliant responses before they reach a user. Same configuration surface as the guards on a DataRobot deployment, so an agent running as a workload is governed the same way a model is.
• Shipping a new version. Replacing the artifact in a running workload rolls out the new container without dropping the endpoint. The URL stays the same.
• Controlling access. Sharing is a property of the workload. Services deployed through the Workload API appear in the same governance and monitoring plane as an organization’s models and applications, and the platform runs wherever DataRobot runs, including VPC and on-premise environments. That’s the trade the Workload API makes possible: IT gets one governed surface for every AI service in the organization, and developers never touch a namespace.

Get started

A first workload takes about five minutes: one spec file, one command, and your container is live. Start with Tutorial: Hello, Workload!, then take a real service to production with sharing and monitoring.