CSL-Meter

CSL-Meter is an independent observatory that checks whether the model you pay for is the model you actually receive. Key capabilities: • Sixteen real upstream API probes per measurement round • Response distribution fingerprinting with JS divergence scoring • Model substitution risk detection • Consistency and drift tracking across 14:00–18:00 peak hours • Live latency and throughput measurement • Raw distributions published for independent review • Open source under Apache-2.0 with a pure Python standard library Rather than relying on vendor claims, CSL-Meter sends genuine requests to subscription and pay-as-you-go plans, then compares each served model's output fingerprint against its claimed baseline. The resulting CSL index quantifies how closely a plan matches its advertised model, helping users spot silent downgrades or substitutions. Only reproducible readings are published, and credentials are held in memory and never stored. Because the tool is self-hostable and dependency-free, teams can run their own probes and verify published readings independently. Ideal for developers, engineering teams, procurement analysts, and researchers who buy model access through subscriptions or APIs and need evidence rather than marketing claims. Use it to compare providers, monitor plan quality over time, and validate performance before committing budget.