A fair, category-by-category map of the tools teams compare when they evaluate Portkey — gateways, observability platforms, and where an in-process, no-proxy approach fits. No FUD, just architecture and trade-offs.
The landscape
Most of these aren't direct substitutes — they sit in different parts of the stack. Here's a neutral rundown, grouped by what they actually do.
Gateways & routers
Portkey
AI gateway / proxy unifying 1600+ models behind one API, with routing, fallbacks, caching, spend caps, prompt management and multi-team governance. Mature and well-resourced; acquired by Palo Alto Networks and folding into Prisma AIRS.
LiteLLM
Open-source proxy and SDK that unifies many LLM provider APIs behind a single OpenAI-style interface. Self-hostable.
OpenRouter
Hosted API aggregator and router to many models through one endpoint; adds a usage surcharge on top of provider pricing.
Observability & eval
LangSmith
LangChain's tracing, evaluation and prompt-management platform — strong if your stack is already built on LangChain.
Helicone
LLM observability via a proxy or async logging — request logs, cost and latency dashboards.
Arize / Phoenix
ML and LLM observability with evaluation tooling; Phoenix is the open-source, self-hostable side.
In-process observability + compliance
Peekr
In-process (no proxy) observability, in-process regulatory compliance, and claim-level hallucination detection — for teams that can't or won't route traffic through a third party. MIT-licensed; self-host in your own VPC.
Where Peekr fits
Peekr isn't a gateway. It's a library that runs inside your process — deep visibility into your LLM calls, without putting a third party in your request path.
In-process, no proxy
Patches OpenAI, Anthropic, Gemini and Bedrock at the class level and observes calls inside your own process. Nothing sits in your request path — zero added network latency, no shared-cloud single point of failure.
Compliance that isn't Enterprise-gated
17 regulatory packs — HIPAA, FDCPA, FINRA, GDPR, EU AI Act and more — enforced in-process on the $99 Pro plan. MIT-licensed, self-host in your own VPC. Data and provider keys never leave your stack.
Claim-level hallucination detection
Every sentence of every response scored supported, contradicted or unsupported — the exact claim that was wrong, not a single opaque score.
At a glance
✓ = present · ✗ = not offered · — = partial. Read it as trade-offs, not a scoreboard — a gateway and an in-process SDK are built for different jobs.
| Capability | Portkey (gateway) | Proxy-based observ. | Peekr (in-process) |
|---|---|---|---|
| Multi-model routing & fallbacks | ✗ | ✗ | |
| Centralized spend caps for many teams | ✗ | ✗ | |
| Prompt management / governance (RBAC, SSO) | — | ✗ | |
| Traffic stays in your process (no proxy hop) | ✗ | ✗ | |
| Provider keys never leave your stack | ✗ | ✗ | |
| Zero added network latency | ✗ | ✗ | |
| Claim-level hallucination scoring | ✗ | — | |
| Regulatory compliance packs (not Enterprise-gated) | ✗ | ✗ | |
| Open-source / self-host in your VPC | — | — |
Proxy latency figure: independent benchmarks have measured roughly 20–40ms added per request for a proxy hop. Portkey's compliance certifications and VPC/private deployment are Enterprise-only.
Honest take
Peekr is not a multi-model gateway. If routing across providers is your problem, Portkey is purpose-built for it — and it's mature and well-resourced. Reach for the gateway when any of these are true.
How to choose
Pick a gateway if…
You need multi-model routing across many providers, fallbacks and load-balancing, caching, or a single spend-control and governance plane for multiple teams. A gateway like Portkey is built for exactly this.
Pick in-process if…
Data residency, latency, lock-in or compliance are the binding constraints — and routing traffic and provider keys through a third party isn't acceptable. An in-process SDK keeps data in your stack with zero added latency.
FAQ
What is the best Portkey alternative?
It depends on the job. If you need multi-model routing, fallbacks and a unified spend-control plane across many teams, Portkey's gateway is purpose-built for that and remains a strong choice. If your priority is observability and regulatory compliance without routing traffic through a third party, an in-process tool like Peekr fits — it runs inside your own process, so data and provider keys never leave your stack.
Is Peekr a drop-in replacement for Portkey?
No. Peekr is not a multi-model gateway. It does not do centralized routing, fallbacks, load-balancing or a shared spend-control plane across teams — Portkey does those and Peekr does not. Peekr is an in-process observability and compliance SDK: tracing, claim-level hallucination detection, and 17 regulatory packs enforced inside your process with no proxy. Many teams run a gateway for routing and Peekr for in-process observability and compliance.
What is the difference between a gateway and in-process observability?
A gateway (or proxy) sits in your request path — your LLM traffic routes through the vendor's cloud, which enables routing, fallbacks and caching but adds a network hop and means the vendor holds your provider keys. Independent benchmarks have measured roughly 20–40ms of added latency per request for a proxy hop. In-process observability runs as a library inside your own process: it observes calls without a proxy, so there is no added network latency and data never leaves your stack.
Which option is better for HIPAA or other regulated workloads?
If data residency and compliance are the binding constraints, an in-process approach avoids sending traffic and keys to a third party at all. With Portkey, compliance certifications (SOC 2, GDPR, HIPAA/BAA) and VPC or private deployment are Enterprise-only. Peekr enforces 17 compliance packs — including HIPAA, FDCPA, FINRA and GDPR — in-process on its $99 Pro plan, not gated behind Enterprise, and is MIT-licensed for self-hosting in your own VPC.
In-process observability, compliance and hallucination scoring — no proxy, no data leaving your stack. Free up to 10k spans/month. MIT license.