Apertus v1.1 4B
Apertus-v1.1-4B is a 4B parameter open-source base language model created via pre-training distillation from the Apertus-8B teacher model. It uses a dense transformer with grouped-query attention and xIELU activations, natively supports 1,811 languages, and is trained on 1.7T high-quality tokens. Designed for constrained hardware and further fine-tuning.
- Type
- Text
- Weights
- Open
- Downloads
- 574
- Likes
- 4
- Pipeline
- text generation
- License
- apache-2.0
Model hub
swiss-ai/Apertus-v1.1-4B
- Repo
- swiss-ai/Apertus-v1.1-4B
- Library
- transformers
- Access
- Public
Technical tags and languages
Papers
2
More from Swiss AI Initiative
10
-
22k DL
Apertus 70B Instruct 2509
Text
Apertus 70B Instruct is a fully open 70B-parameter decoder-only language model trained from scratch on 15T to…
-
11k DL
Apertus 8B 2509
Text
Apertus 8B 2509 is a fully open 8B parameter language model by Swiss AI Initiative, pretrained on 15T tokens.…
-
Apertus 8B Instruct 2509
Text
Apertus 8B Instruct 2509 is an 8B parameter open-source instruction-tuned language model pretrained on 15T to…
-
805 DL
Apertus v1.1 0.5B Instruct
Text
Apertus-v1.1-0.5B-Instruct is a 0.5B parameter multilingual instruction-tuned language model built via pre-tr…
-
561 DL
Apertus v1.1 1.5B
Text
Apertus-v1.1-1.5B is a 1.5 billion parameter base language model trained via distillation from an 8B teacher…
-
803 DL
Apertus v1.1 1.5B Instruct
Text
Apertus-v1.1-1.5B-Instruct is a 1.5B parameter instruction-tuned language model built on a dense transformer…
-
1.2k DL
Apertus v1.1 4B Instruct
Text
A 4B parameter instruction-tuned language model created via pre-training distillation from the Apertus-8B tea…
About Swiss AI Initiative
1
The Swiss AI Initiative is the world's largest open science/open source effort for AI foundation models, started in December 2023. Seeded with over 10M GPU hours on the Alps supercomputer and a 20M CHF grant from the ETH Domain, it is the first initiative of the Swiss National AI Institute—a partnership between the ETH AI Center and the EPFL AI Center—leveraging 800+ researchers (70 AI-focused professors) from 10+ Swiss academic institutions.