Our approach · Published rubric
PQC solutions receive scores from 0 to 100 across 10 published dimensions, weights, and thresholds. Domain-expert review controls are independent of the automated evaluation system. No vendor pays for inclusion, ranking, or evaluation. Scoring methodology is published in full, and published score records carry provenance labels before release. Scores derive from executed and preserved evidence artifacts — the Intelligence Model weighs evidence, it never generates it. Score records are ML-DSA-65 signed; public proof manifests are Ed25519-signed against the published Lab keyset when proof health is green and backed by our fidelity commitment.
Process
The methodology is public because the scores are meant to be defensible. Buyers can inspect the 10 scoring dimensions, their weights, the publication thresholds, and the provenance state behind each published result.
Evidence is gathered from public implementation artifacts and reviewable technical materials:
Scores only publish when the public rubric, provenance state, and release controls are satisfied:
Verified evidence: When the required evidence is present and the publication controls pass, the score is published with a Verified provenance tag.
Contested evidence: If the available evidence does not support a clean publication state, the score is published conservatively and tagged Contested so buyers can see that the record needs closer review.
Missing evidence: If a dimension lacks the artifacts needed for a high-confidence result, it is marked Inferred or Limited instead of being presented as verified truth.
Rubric · 10 axes
Each solution is evaluated across these 10 dimensions with published weights that sum to 100%. Scores reflect real evidence — source code analysis, benchmark data, compliance certificates, and published security evaluations. The five highest-stakes dimensions require dual-agent consensus before they can carry a Verified tag.
Does the implementation produce correct output for all standard test vectors and parameter sets?
Does it conform to FIPS 203/204/205, CNSA 2.0, and other regulatory requirements?
Are builds reproducible? Are releases signed? Is the dependency tree audited?
How fast are key generation, encapsulation, and signing operations across target platforms?
Is the implementation protected against timing, power, and cache-based side channel attacks?
Is the underlying hard problem proven resistant to known quantum algorithms?
Does the solution work with other PQC implementations and standard protocols?
Are key and ciphertext sizes practical for real-world protocols like TLS and X.509?
How long has this solution been in production? What is its release and maintenance track record?
Are API references, migration guides, security guidance, and examples available?
Adversarial process
A score that nobody tried to attack is an assertion, not an evaluation. High-stakes dimension scores pass through an adversarial debate before they can publish as Verified.
An independent external Intelligence Model acts as a hostile auditor and attacks any dimension score of 8.0 or higher, looking for evidence gaps, overclaims, and rubric misapplication.
A separate Intelligence Model defends the proposed score using only the collected evidence artifacts — never new assertions. Evidence that cannot be produced cannot be argued.
An independent judge turn issues the final verdict. Every turn is recorded with per-turn model provenance, so who argued what is part of the record.
When the judge’s verdict disagrees with the proposed score by more than 0.5 points, the disagreement is preserved as a fidelity record in the score’s provenance. Contested dimensions publish with a Contested tag so buyers can see exactly where the debate did not resolve cleanly.
Separately, five dimensions — Algorithm Correctness, Standards Compliance, Supply Chain, Side Channel Resistance, and Quantum Resistance — require dual-agent consensus: two independent scorers must agree within 2 points before the dimension can carry a Verified tag.
Public criteria
Quantitative criteria for every dimension at every score tier. This is the public rubric used to publish scores. No hidden criteria, no paid placement, and no undisclosed overrides.
Want to see how this rubric applies to a real solution? View the liboqs scored walkthrough →
Rubric source: Automated test harness (NIST KAT vectors, ACVP test vectors, cross-implementation differential testing)
No test vectors run. Implementation produces incorrect output or fails to compile.
Passes fewer than 50% of NIST Known Answer Test (KAT) vectors. Some parameter sets untested.
Passes all NIST KAT vectors for primary parameter set. Other parameter sets partially tested.
Passes all KAT vectors across all parameter sets. Cross-implementation differential testing confirms agreement with at least 2 other implementations.
Passes all KAT and ACVP vectors. Cross-tested against 3+ implementations. Edge cases (zero-length inputs, maximum-size parameters) explicitly tested and documented.
Rubric source: Automated benchmark harness (100 iterations per algorithm, p99 latency, memory profiling)
No published benchmarks or performance characterization.
Basic benchmarks exist but lack methodology disclosure (iteration count, hardware, compiler flags).
Published benchmarks with methodology. Key generation, encapsulation/signing, and decapsulation/verification each measured. Performance within 10x of reference implementation.
Benchmarks across multiple platforms (x86-64, ARM). Performance within 2x of reference. Memory usage profiled. No unexpected allocations.
Comprehensive benchmarks across 3+ platforms with statistical methodology (confidence intervals, p99 tails). Performance at or below reference. Hardware-accelerated paths (AVX-512, NEON) benchmarked separately.
Rubric source: Automated measurement (byte-exact sizes for all parameter sets, protocol overhead analysis)
Sizes not documented. Cannot determine protocol compatibility.
Sizes documented for one parameter set only. No protocol overhead analysis.
All parameter set sizes documented. Public key fits in standard TLS ClientHello. Total handshake overhead calculated.
Sizes documented with protocol-specific analysis (TLS 1.3, X.509, IKEv2). Hybrid sizes calculated. Fits in single MTU for at least one parameter set.
Complete size analysis across all protocols. Compression options evaluated. Bandwidth impact quantified for real-world traffic patterns. Comparison table against all NIST finalists.
Rubric source: Expert review (constant-time pattern analysis, Valgrind, published audit reports)
No side-channel protections documented. Secret-dependent branches or memory access visible in source.
Claims constant-time but no tooling verification. No formal analysis of secret-dependent operations.
Constant-time patterns verified by static analysis or equivalent. Secret-dependent branches eliminated in core operations. No published audit.
Constant-time verified. Published side-channel analysis or audit by independent party. Power/EM analysis considered in threat model.
Constant-time verified with multiple tools. Independent side-channel audit published. Fault injection countermeasures documented. Hardware-specific mitigations (DOIT, speculative execution barriers) implemented where applicable.
Rubric source: Automated test harness (cross-implementation key exchange, protocol integration testing)
Standalone implementation with no demonstrated interoperability.
Interoperates with 1 other implementation. No protocol-level integration tested.
Interoperates with 2+ implementations. At least one protocol integration demonstrated (TLS, SSH, X.509).
Interoperates with 3+ implementations across multiple protocols. Hybrid key exchange with classical algorithms demonstrated. OID registration complete.
Full interoperability matrix published. Works across TLS, SSH, X.509, CMS, and IKEv2. Hybrid and composite modes tested. Backward compatibility with classical-only peers verified.
Rubric source: Expert review (FIPS validation status, CNSA 2.0 alignment, regulatory requirement mapping)
No alignment with any published standard.
Implements a NIST-selected algorithm but has not entered FIPS validation. No CNSA 2.0 analysis.
FIPS validation in progress (CMVP queue) or uses FIPS-validated module as dependency. Partial CNSA 2.0 compliance documented.
FIPS 140-3 validated (or uses validated module). CNSA 2.0 compliant. NSM-10 timeline alignment documented.
FIPS 140-3 validated with algorithm certificates. Full CNSA 2.0 compliance. EO 14028 requirements met. PCI DSS 4.0.1 and HIPAA cryptographic requirements addressed. Compliance matrix published.
Rubric source: Automated analysis (release history, CVE history, adoption metrics, maintenance cadence)
Pre-release or proof-of-concept. No production deployments known.
Initial release available. Fewer than 3 releases. No documented production use.
Stable release with 5+ versions. At least 1 known production deployment. Maintenance commits within last 90 days.
Mature project with 2+ years of releases. Multiple production deployments. CVEs (if any) patched within 30 days. Semantic versioning followed.
Battle-tested with 3+ years of production use. Widely adopted. Established CVE response process with published security policy. LTS or guaranteed support commitment.
Rubric source: Expert review (API docs, migration guides, security guidance, example completeness)
No documentation beyond source code.
README with basic usage examples. No API reference or migration guidance.
API reference generated from code. Basic usage guide. Security considerations mentioned but not comprehensive.
Complete API reference. Migration guide from classical algorithms. Security guidance with recommended parameter sets. Code examples for major use cases.
Comprehensive documentation suite: API reference, migration guide, security guidance, threat model, integration examples for TLS/SSH/X.509, FAQ, and troubleshooting. Documentation versioned alongside releases.
Rubric source: Automated analysis (reproducible builds, dependency audit, SBOM, release signing)
No build reproducibility. Dependencies unaudited. No release signing.
Dependencies listed but not pinned. Builds not reproducible. Releases exist but unsigned.
Dependencies pinned with lockfile. Releases signed with PGP or Sigstore. SBOM not published.
Reproducible builds verified. Dependencies audited with no known critical CVEs. Signed releases. SBOM published in CycloneDX or SPDX format.
Fully reproducible builds with CI verification. Zero-dependency or all dependencies audited. Signed releases + SBOM + provenance attestation (SLSA Level 3+). Dependency update policy documented.
Rubric source: Expert review (underlying hard problem analysis, known quantum attacks, security parameter evaluation)
Not based on a problem believed to be quantum-resistant, or uses deprecated parameters.
Based on a quantum-resistant problem family (lattice, hash, code) but uses non-standard parameters or non-NIST-selected variant.
Implements a NIST-selected algorithm (ML-KEM, ML-DSA, SLH-DSA) with standard parameters. NIST security level 1 or higher.
NIST-selected algorithm at security level 3+. No known quantum or classical attacks reducing effective security. Parameter choices justified against latest cryptanalysis.
NIST-selected algorithm at security level 5 (or multiple levels supported). Hybrid mode available combining PQC with classical (X25519/P-384). Tracks latest cryptanalysis with published response to new attack papers.
Point-in-time honesty
A score is a statement about a solution at a point in time, under a specific methodology version — not a permanent property. The Lab shows each score’s date and expiry alongside it.
Every score record carries its scoring timestamp, the methodology version it was scored under, and the weight vector applied. When the methodology changes, previously published scores keep the version they earned.
Each dimension has a published time-to-live: 14 days for fast-moving signals like performance and supply chain, up to 180 days for standards compliance. When a newly disclosed CVE affects a vendor, the time-to-live on every dimension drops to 24 hours pending re-evaluation.
Expiry is derived from the scoring timestamp and the published TTL policy, and every score record exposes its scoring date and expiry through the Lab API, so an expired score is never presented as current.
Discipline
Scores carry observable evidence and provenance labels — source code, test results, published benchmarks, compliance certificates, or disclosed evidence gaps. The Intelligence Model weighs evidence; it never generates it. Marketing claims require corroborating evidence before they can improve a score.
Domain-expert review controls are independent of the automated evaluation system. Publication controls surface contested evidence conservatively.
Every dimension score carries a provenance tag (Verified, Contested, Inferred, or Limited) so you know exactly how confident the assessment is.
Every published score includes a full audit trail. Score records carry the Lab signature trail, and public proof manifests are Ed25519-signed against the published Lab keyset.
Every score is dated and carries the methodology version it was scored under. Scores expire on a published per-dimension TTL, and every score shows its scoring date and expiry, so it is never presented as current past that date.
No vendor pays for placement or influence. Scoring criteria are applied uniformly. The only way to improve a score is to improve the solution.
Any vendor or maintainer can request a methodology retest at any time. New versions, patched vulnerabilities, and updated certifications trigger re-evaluation through the same published methodology.
No paid inclusion
Qtonic Quantum Lab is funded by Qtonic Quantum’s consulting and platform revenue. No vendor pays for inclusion, evaluation, or a specific score. Findings publish exactly as evidenced — scores that decrease are published with the same transparency as scores that increase.
Qtonic Quantum sells remediation through QSolve and assessment through QScout and QStrike. The Lab scores the same vendors Qtonic Quantum could commercially engage — that conflict is disclosed, not denied, and it is contained structurally: commercial relationships do not affect score publication or ranking.
Independence is enforced mechanically. A continuous integration gate runs a static import-boundary audit over the scoring codebase and proves that no commercial, billing, or QSolve-coupled module is reachable from the scoring path. Any new import that crosses the boundary fails the build before it can ship.
Any vendor or maintainer can request a methodology retest at any time. Email lab@qtonicquantum.com to request one. Retest results are published regardless of score direction.
ML-DSA-65-backed signing covers every public score: each record is signed with a context-bound payload, so a published score cannot be altered after release without breaking its signature. Separately, public proof manifests use Ed25519 against the published Lab keyset. These are complementary layers — not interchangeable claims for the same object.
Scores that decrease are published with the same transparency as scores that increase. See our correction history.
Request QScout assessment for a verified website snapshot, then use governed QScout scope when infrastructure-level PQC evidence is needed.
Every published Qtonic Quantum Score follows a 10-dimension published rubric with expert oversight and full audit trail. Score records are ML-DSA-65 signed; public proof manifests are Ed25519-signed against the published Lab keyset.
Each published score includes: solution_id, score_total, scored_at, breakdown_json. This covers the composite score, all 10 dimension scores, provenance tags, and the scoring timestamp. The payload is ML-DSA-65 signed under a domain-separated signing context, and the signature and verification key are exposed with each published score.
Score revisions are cryptographically chained. Each revision’s hash covers the score, reason, release record, and previous hash. This forms a tamper-evident chain — any modification to historical entries breaks the chain and is detectable. View the public changelog for every score revision.
Lab Journey
Overview
Evidence-linked registry and queue entry point
Leaderboard
Public leaderboard
Our Approach
Scoring and evidence model
Coverage
Evidence-linked registry vs research queue
Standards
Compliance tracking and deadlines
Fidelity
Proof, signing, and retest policy
Changelog
Public revisions and retests
Compare
Side-by-side PQC evaluation
Qtonic Quantum Lab scores are generated through automated evaluation using publicly available evidence. Scores reflect conditions at the time of evaluation and may change as solutions evolve. Vendors may request a retest at any time. See our fidelity commitment for full details.
Some infrastructure assurance belongs to the underlying provider. It supports buyer diligence, but it is not a Qtonic Quantum-held certification or attestation.
We evaluate PQC solutions on request. Submit a vendor for review and we will route it through the Qtonic Quantum Lab intake queue.
Qtonic Quantum Lab scores are independent research opinions based on a published rubric and publicly available evidence. Provenance labels disclose the review state. No vendor pays for inclusion, ranking, or evaluation. Scores are not endorsements, certifications, warranties, or legal advice. Read the published methodology or the full research disclaimer.