Academic Publications: OpenReview
Academic Publications · OpenReview. Open submissions, peer review and conference forums. This source has 5 entries admitted under this category's review policy; each links directly to the original.
Authority tiers in this category: A2–R.
Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors
Compares long-sequence models trained from scratch with those given data-driven pretraining, and shows that random initialization exaggerates architectural differences between Transformers and state space models.
Evidence use: For checking training priors, pretraining settings and their scoped experimental results in comparisons of long-sequence architectures; importance is based on the ICLR 2024 Outstanding Paper committee statement.
Limitations: The findings are constrained by the models compared, data priors and training settings; they do not imply that architectural differences can be ignored in all long-sequence tasks. This site links to the final ICLR record and summarizes the award rationale; experiments were not independently reproduced.
Related concepts: Transformer · State Space Models and Mamba · Model Evaluation and Benchmarks
Link last checked: 2026-09-24
Vision Transformers Need Registers
Identifies high-norm artifacts in the feature maps of vision Transformers and adds register tokens to absorb internal computation, improving feature and attention maps.
Evidence use: For checking the diagnosis of high-norm tokens in vision Transformers, the register token approach and the downstream results reported in the paper; importance is based on the ICLR 2024 Outstanding Paper committee statement.
Limitations: The evidence supports the diagnosis and fix in vision Transformers; it does not mean register tokens improve every Transformer architecture. This site links to the final ICLR record and summarizes the award rationale; experiments were not independently reproduced.
Related concepts: Transformer · Attention · Convolutional Neural Network (CNN)
Link last checked: 2026-09-24
Learning Interactive Real-World Simulators
Trains UniSim by combining heterogeneous data from images, robotics and navigation; it responds to natural language instructions or low-level controls with visual outcomes and is used to train planners and policies.
Evidence use: For checking UniSim's orchestration of heterogeneous data, its interactive generation interface and the reported sim-to-real transfer results; importance is based on the ICLR 2024 Outstanding Paper committee statement.
Limitations: The paper shows simulation and transfer results for specific data and tasks; it should not be described as achieving universal, accurate real-world simulation. This site links to the final ICLR record and summarizes the award rationale; experiments were not independently reproduced.
Related concepts: World Models and 3D Generation · Multimodal Models · Reinforcement Learning
Link last checked: 2026-09-24
Generalization in diffusion models arises from geometry-adaptive harmonic representations
Analyzes, through score functions and denoiser representations, why image diffusion models generalize beyond their training samples, linking the inductive bias to geometry-adaptive harmonic bases.
Evidence use: For checking the generalization phenomena, score function comparisons and geometry-adaptive harmonic representation analysis in the image diffusion settings studied; importance is based on the ICLR 2024 Outstanding Paper committee statement.
Limitations: The evidence is limited to the image diffusion and denoising settings studied in the paper and does not generalize to the claim that no diffusion model memorizes training data. This site links to the final ICLR record and summarizes the award rationale; experiments were not independently reproduced.
Related concepts: Diffusion Models · Image Generation · Overfitting
Link last checked: 2026-09-24
LLMs Get Lost In Multi-Turn Conversation
Compares single-turn and multi-turn tasks in large-scale simulation, reports that the LLMs tested degrade on underspecified multi-turn instructions, and attributes the main problem to reliability rather than capability.
Evidence use: For checking the single-turn and multi-turn evaluation design, the decomposition of performance loss and error recovery behavior; importance is based on the ICLR 2026 Outstanding Paper committee statement.
Limitations: Results are constrained by the models, generation tasks and interaction simulation design tested; the committee also noted that the model versions were somewhat dated. This site links to the final ICLR record and summarizes the award rationale; experiments were not independently reproduced.
Related concepts: Large Language Model (LLM) · Model Evaluation and Benchmarks · Context Window
Link last checked: 2026-09-24