Systems Evaluation
Decide on evidence,
not on decks.
Panaptico runs vendor evaluations against your live stack — real workloads, real identity, real topology. Every decision ships with a signed record procurement can defend and engineering can build from.
Snowflake
Composite score
78
/ 100
Cost
$0.042 / GB-scan
+18% vs baseline
P95 query latency
2.4s
on 1.2B row join
Integration fit
Partial
No native Iceberg write — Tabular bridge required
Identity
Okta SSO · SCIM ok
Databricks
Composite score
86
/ 100
Cost
$0.031 / GB-scan
baseline
P95 query latency
1.7s
on 1.2B row join
Integration fit
Full
All connectors resolved
Identity
Okta SSO · SCIM ok
The gap
Vendor decks lie.
Your environment doesn't.
Most evaluations happen in slide decks and sandbox tenants. You pick between Databricks and Snowflake by reading Gartner. You choose an observability platform from a call with a sales engineer. You guess whether Cloudflare One will play nice with Tailscale by skimming docs. Nobody runs the comparison against your workloads, your identity setup, or your topology — so the tool that looked great in the demo breaks against your reality three months in.
Sandbox ≠ production
Demo tenants run 50k rows. You run 1.2B. The vendor's p95 was measured on their data, not yours.
Reference architectures hide cost
Integration work — identity, egress, bridge connectors — is 40% of TCO and lives nowhere in the deck.
Procurement loses the why
Six months later nobody can reconstruct the criteria. The next evaluation starts from zero.
Real-environment benchmarking
Your workloads. Your data. Real numbers.
Panaptico replays a sampled slice of your production traffic against each candidate. Costs, latencies, and failure modes come back in the shape of your actual stack.
| Query | Rows | Snowflake | Databricks | Confidence |
|---|---|---|---|---|
| Q1 · Revenue attribution join | 1.24B × 48M | 2.41s | 1.67s | 98% |
| Q2 · Cohort funnel scan | 840M × 2.1M | 3.82s | 4.14s | 96% |
| Q3 · Daily rollup materialization | 2.8B | 11.2s | 7.9s | 99% |
| Q4 · Ad-hoc BI query · 95p | mixed | 1.12s | 1.38s | 92% |
| Q5 · Iceberg write · 500M rows | 500M | fail · bridge required | 41s | 100% |
Integration feasibility
Prove they talk
before you sign.
Most integration failures hide in the boundary — identity handoff, DNS, policy evaluation, egress routing. Panaptico spins up a shadow pairing, runs the four boundary checks, and returns an evidence-backed pass/fail.
Boundary gates
Identity handoff · Okta → CF → Tailscale
OIDC token exchange · 840 probes
Split DNS resolution
Tail-net records propagate · 280ms p95
Policy evaluation · CF Access + TS ACL
No conflicts across 12 policies
Egress routing · stable IP to SaaS
No flapping over 48h
MTU / fragmentation
No reassembly drops
Connector failover
Regional cutover in 4.1s
Audit log interleave
Correlated via request-id
Device posture passthrough
CF posture not forwarded · workaround available
Cross-vendor scorecard
Score on what matters to you,
not the category report.
Dimension weights come from your criteria. Every score traces back to a measured probe in your environment — no vendor-reported numbers, no analyst-derived scores.
Databricks
86
/ 100
Recommended · Iceberg native · workload cost-optimal
Snowflake
78
/ 100
Strong BI path · Iceberg via Tabular bridge
BigQuery
74
/ 100
Lowest cost · weakest on identity parity with Okta
Decision record
Every evaluation exits as
a signed artifact.
Procurement gets evidence they can defend. Engineering gets a baseline they can build from. Six months in, when someone asks why you chose X — the record is still there, still reconcilable to the state of the stack.
Decision
Adopt Databricks as primary warehouse. Retain Snowflake for BI-facing marts through 2026-Q3.
Criteria weight
Cost 30 · Coverage 25 · Performance 25 · Fit 20 — set by Finance + Data platform lead.
Decision owner
D. Park · VP Data Platform · ratified by CTO council 2026-04-18
Evidence
BENCH-2091 · INT-0074 · 1,820 query traces · 3,180 boundary probes · 48h replay window
Reversal trigger
Rebench if p95 regresses > 15% or cost delta > 20% at any monthly reconciliation.
Open risks
Device posture passthrough (CF → TS) unresolved — workaround documented in INT-0074-R2.
Close the tab on vendor decks.
Run the next evaluation against your real stack. Ship a decision you can defend — and a baseline Panaptico will keep reconciled after go-live.