Knowledge Bases and Evaluation Organizations: OpenCompass (Sinan)
Knowledge Bases and Evaluation Organizations · OpenCompass (Sinan). Open-source evaluation system and monthly leaderboard. This source has 6 entries admitted under this category's review policy; each links directly to the original.
Authority tiers in this category: A2–B. Admission policy: All core resources are included. Each secondary source is admitted according to its own database, methodology, versions and leaderboard structure; there is no cross-organization scoring scheme.
OpenCompass public academic leaderboard for large models
Evaluates mainstream open-source and commercial LLMs side by side across many datasets and publishes detailed scores.
Evidence use: Supports side-by-side comparison of models in the Chinese ecosystem at a specified time and configuration.
Limitations: Public test sets may be contaminated. Models and datasets are replaced on a rolling basis.
Related concepts: Model Evaluation and Benchmarks
Link last checked: 2026-09-24
Guide to reproducing CompassAcademic leaderboard results
Explains the academic leaderboard's update principles, live configuration and how to reproduce its results.
Evidence use: Supports leaderboard configuration, update frequency and the limits of reproducing results.
Limitations: Configurations evolve across versions. Reproduction still depends on model access.
Related concepts: Model Evaluation and Benchmarks
Link last checked: 2026-09-24
OpenCompass evaluation framework overview
Explains the standardized pipeline of data, model, inference, evaluation and summary modules.
Evidence use: Supports understanding how OpenCompass results are produced and reproduced.
Limitations: Configurations, datasets and model versions are continuously updated. Results on public datasets must account for contamination and prompt sensitivity.
Related concepts: Model Evaluation and Benchmarks
Link last checked: 2026-09-24
OpenCompass dataset catalog
Lists supported datasets, task types, sample sizes and configurations in one place.
Evidence use: Supports checking evaluation coverage and data scope.
Limitations: Configurations, datasets and model versions are continuously updated. Results on public datasets must account for contamination and prompt sensitivity.
Related concepts: Model Evaluation and Benchmarks
Link last checked: 2026-09-24
OpenCompass metric computation notes
Explains how objective, generative and custom tasks are scored and aggregated.
Evidence use: Supports correct interpretation of how scores are defined for different tasks.
Limitations: Configurations, datasets and model versions are continuously updated. Results on public datasets must account for contamination and prompt sensitivity.
Related concepts: Model Evaluation and Benchmarks
Link last checked: 2026-09-24
CompassArena multimodal arena
Evaluates image-text understanding and generation models by users' pairwise preferences.
Evidence use: Supports human preference comparisons of multimodal models in the Chinese ecosystem.
Limitations: Configurations, datasets and model versions are continuously updated. Results on public datasets must account for contamination and prompt sensitivity.
Related concepts: Model Evaluation and Benchmarks
Link last checked: 2026-09-24