Standards and Regulation: US CAISI
Standards and Regulation · US CAISI. NIST's in-house Center for AI Standards and Innovation, which runs independent evaluations. This source has 3 entries admitted under this category's review policy; each links directly to the original.
Authority tiers in this category: A1.
NIST AI 800-2 IPD: practices for automated benchmark evaluation of language models
Proposes preliminary best practices for automated benchmark evaluation, covering task validity, implementation, execution and reporting.
Evidence use: Supports the evaluation practices and terminology described in the draft.
Limitations: An Initial Public Draft, not final guidance. The public comment period has closed; the document may be revised.
Related concepts: Model Evaluation and Benchmarks · LLM Application Evaluation
Link last checked: 2026-09-24
NIST AI 800-3: expanding the AI evaluation toolbox with statistical models
Distinguishes fixed-benchmark accuracy from generalized accuracy and uses statistical models to improve uncertainty quantification and interpretation of results.
Evidence use: Supports NIST AI 800-3's measurement goals, models and experimental results.
Limitations: A methodology report, not a mandatory standard. Model assumptions must match the specific evaluation design.
Related concepts: Model Evaluation and Benchmarks · Uncertainty Calibration and Selective Prediction
Link last checked: 2026-09-24
CAISI evaluation of DeepSeek V4 Pro
Publishes CAISI's multi-domain evaluation results for DeepSeek V4 Pro, with comparison methods and appendices.
Evidence use: Supports CAISI results for this model version on the specified benchmarks, configuration and date.
Limitations: Model-specific and time-bound. Cannot be extrapolated into safety conclusions for every deployment.
Related concepts: Model Evaluation and Benchmarks · Reasoning Models
Link last checked: 2026-09-24