API & Data Access Services

Artificial Analysis provides access to data and also bespoke analysis services. Please fill out the form below to express interest in our data access services or with other enquiries, or alternatively email [email protected].

API Coverage details:

Metrics:

• Pricing: Input/output per token pricing
• Latency: Time to First Token
• Output Speed: Tokens/s
• Total Response Time: For 100 output tokens
• Intelligence Index

Test Workloads (6 combinations):

• Prompt/context lengths:
  • ~100 tokens
  • ~1k tokens
  • ~10k tokens
• Parallel Queries:
  • 1 query
  • 10 parallel queries

Time Basis:

• 14 Day Rolling Averages: P5, P25, Median, P75, P95
• Per Day Basis: Median of daily measurements over the preceding 31 days

Access Options:

• HTTP API: JSON output, API key for authentication. Generally preferred.
• CSV Files: Accessible via Amazon S3 bucket or Google Drive folder

Update Frequency:

• All API metrics are updated in real-time from our latest measurements, taken every 3 hours (for short prompts, varied for longer prompts).