Check out the newest way to compare different models for a task/agent harness: AutoEvals

API

Data Retention

Understand how Inference.net handles request data, observability records, and retention controls.

Inference.net is designed to support production workloads without treating captured request data casually.

Core principles

  • Request data is not used for model training by default
  • Secrets and similar sensitive values are stripped where possible
  • Platform data is encrypted in transit and at rest
  • Retention should match the operational need of the workflow

Direct API vs Gateway

The direct API and Gateway are different product paths, but the same general rule applies: only keep what is operationally useful, and use project-level controls and data curation intentionally.

For the workflow-first entry point into traffic capture, start with Integrate with Your LLM Provider.

  • Use environments and task IDs to segment traffic
  • Create long-lived datasets only for the examples you want to preserve
  • Review retention expectations before broad production rollout

Retention and asynchronous inference

The Asynchronous and Batch APIs store requests and results until they are delivered, so they are unavailable while the Zero Data Retention (ZDR) team setting is on. Teams with ZDR on receive a 403 Forbidden on /v1/slow, /v1/batches, and /v1/files upload requests, and should use the synchronous /v1 endpoints or turn ZDR off in team settings.

Zero Data Retention

New teams start with the Zero Data Retention (ZDR) setting on, so Inference.net does not retain request and response content. You can change it under Settings → General in the dashboard. Turn it off to keep inference results or to use the Asynchronous and Batch APIs. See the Zero Data Retention Policy for what ZDR covers.

Need a specific retention policy?

If you need a specific policy, a data processing agreement, or help mapping the platform into your internal compliance requirements, meet with our team.

On this page