F pre 8kv 8k 14k SWA
F_pre_8kv_8k_14k_SWA is an AllenAI OlmPool architectural-variation model for studying long-context extension. It is a 7B to 8B class language model trained to 150B tokens total, including 140B pretraining and 10B context extension, and is meant for research on architecture choices rather than polished instruction-following use.
- Type
- Text
- Weights
- Open
- Downloads
- 6
- License
- apache-2.0
- Library
- transformers
Model hub
allenai/F_pre_8kv_8k_14k_SWA
- Repo
- allenai/F_pre_8kv_8k_14k_SWA
- Access
- Public
Technical tags and languages
More from Ai2
11
-
Molmo 2
Text
Molmo 2 is AI2’s open 4B/7B/8B multimodal model for images and video, delivering state-of-the-art grounded vi…
-
MolmoAct 2
Multimodal
MolmoAct 2 is Ai2’s open robotics foundation model for real-world robot manipulation. It is an Action Reasoni…
-
313 DL
MolmoMotion 4B H1 F32
Multimodal
4B vision-language model that forecasts 3D point trajectories from a single RGB observation frame, 2D query p…
-
OLMo
Text
OLMo is Ai2's fully open language model, released in February 2024 in 1B and 7B sizes. Unlike most open-weigh…
-
Olmo 3
Text
Olmo 3 is Ai2's fully open model family in 7B and 32B sizes, shipped as Base, Instruct, Think and RL Zero var…
-
olmOCR
Image
olmOCR is AllenAI’s open-source document recognition pipeline and model family that converts PDFs and images…
-
54k DL
OLMoEarth
Multimodal
OlmoEarth is Ai2/AllenAI’s foundation-model collection for remote-sensing tasks using satellite and Earth-obs…
-
152 DL
SERA 32B
Text
SERA-32B is Ai2’s flagship 32B Open Coding Agent, a Qwen 3-32B based repository-level coding model trained wi…
About Ai2
1
Ai2 is a 501(c)(3) non-profit AI research institute founded in 2014 by the late Paul Allen (Microsoft co-founder), dedicated to conducting high-impact, open AI research and engineering for the common good, including open language models (OLMo), scientific AI tools (Semantic Scholar, Asta), environmental AI platforms, and embodied robotics research.