See exactly what every agent is doing. Act the moment it matters.
A single view of your entire agent workforce. Not model monitoring on a single pipeline. A unified view of every agent you’re running: behavior, output quality, data inputs, system interactions, and performance metrics, in one place. That includes anomaly detection with configurable alerting thresholds, performance degradation signals before they become production failures, and data input monitoring to catch upstream changes before they cascade downstream.
One pane of glass, even for agents you didn’t build here. Most enterprises run agents from more than one origin: built on the platform, built by vendors, built by teams before a monitoring standard existed. OpenTelemetry traces, logs, and metrics from external agents flow directly in, so the whole estate is visible and governable in one place, regardless of where an agent was built.
Monitoring designed for agents that don’t sit still. Production agents are no longer single request-response pipelines. They run for hours or days, adapt to failures, and dynamically spawn sub-agents to get work done. Monitoring on the Agent Workforce Platform is built for that reality: full visibility into long-running trajectories, sub-agent trees, and tool interactions, not just per-request metrics.
Customize and track what matters most. Gain critical insights with built-in or custom metrics like GPU cost, ROI, or user feedback – from one place. Automate retraining, risk mitigation, alerts, and more with configurable policies tailored to your needs. Use the built-in value tracker for project-level ROI and informed decision-making.
Track, version, and organize use cases. Improve transparency and streamline navigation by organizing AI projects into structured use cases. Automatically document and version all key assets, including agents, vector databases, LLMs, AI apps, and notebooks, with built-in role-based access controls for secure collaboration and traceability.
Capture end-to-end lineage. Track the full lifecycle of your agents, and the AI strategies behind them, from experimentation to production. Everything is packaged within the Agent Workforce Platform: agents, data transformations, feature discovery, LLMs, vector databases, and prompting strategies, so you can audit and track every deployment end-to-end.
Interpret complex relationships. Visualize complex data structures across generative and predictive use cases, like explaining geospatial models or analyzing vector database coverage.
Trace and assess prompts. Gain prompt and response insights with automatically generated tracing tables to improve your vector databases and generative AI responses. Monitor production prompts for response quality, user feedback, PII flags, rouge, or custom metrics to better manage risk.
Monitor prompts and responses in production
Custom metrics to uncover negative feedback, documentation gaps, and areas of improvement
Generate tracing tables for vector databases
Filter, search, and sort tracing tables for keywords, and blocked prompt patterns
Dive into drift. Identify and investigate drift patterns, including embedding drift for agentic AI and text or location drift for predictive models. Visually compare scoring data segments or scoring vs. training segments across any feature and time window. Access contextual insights like prediction value over time to pinpoint root causes faster.
Maintain high-quality vector databases. Keep your vector databases accurate and aligned. Monitor embedding drift, and track, version, and manage key components – including metadata, benchmarks, and validation results – with tools to quickly surface gaps and update vector databases without disrupting deployments.
Track vector database versions and documents
Manage vector databases built in DataRobot or bring your own
Visual vector database insights for quality assessment
Automatically update deployments with new vector database version
When something goes wrong, you need to act, not wait. Pause a specific agent, redirect it, retrain it, or roll back a version, without taking down the surrounding system or waiting for a release cycle. In practice: stop an agent from executing in a specific workflow without a system outage, without pulling in an engineering team after hours, and without affecting the other agents running in parallel.
# Ensure the thread has finished
fit_thread.join()
print("Model fitting completed.")
# Save the model to filesystem
with open(model_output_path, 'wb') as pkl:
pickle.dump(rf, pkl)
print(f'Saved model to {model_output_path}')
retrained_custom_model_version = dr.CustomModelVersion.create_clean(
custom_model_id=custom_model.id,
folder_path=model_output_path,
base_environment_id=execution_environment_id,
training_dataset_id=dataset_id
)
print(f'Retrained model version_id = {retrained_custom_model_version.id}')
“With open-source software, optimizing a model could take anywhere from two weeks to a month. And we needed to be able to explain the dynamics underlying the model build and its predictions.”
“DataRobot’s rich machine learning blueprints, feature engineering methods and explainability features amongst others make it a cornerstone in BMW Group’s AI Platform to scale AI adoption.”