1.58-bit large language model

Large language model with ternary weights A 1.58-bit large language model (also known as a ternary LLM) is a type of large language model (LLM) designed to be computationally efficient. It achieves this by using weights that are restricted to only three values: −1, 0, and +1. This restriction significantly reduces the model's memory footprint and allows for faster processing, as computationally expensive multiplication operations can be replaced with lower-cost additions. This contrasts with traditional models that use 16-bit floating-point numbers (FP16 or BF16) for their weights. Studies have shown that for models up to several billion parameters, the performance of 1.58-bit LLMs on various tasks is comparable to their full-precision counterparts.Ma Wang Ma Wang 2024 1Hutson 2024 This approach could enable powerful AI to run on less specialized and lower-power hardware.Friha Amine Ferrag Kantarci Cakmak 2024 5822 The name "1.58-bit" comes from the fact that a value with three states contains  bits of information. These models are sometimes also referred to as 1-bit LLMs in research papers, although this term can also refer to true binary models (with weights of −1 and +1).Ma Wang Ma Wang 2024 1Morales 2025 BitNet BitNet a computer network BITNET In 2024, Ma et al., researchers at Microsoft, declared that their 1.58-bit model BitNet b1.58 is comparable in performance to the 16-bit Llama 2 and opens the era of 1-bit LLM.Huyen 2024 330 BitNet creators did not use the post-training quantization of weights but instead relied on the new BitLinear transform that replaced the nn.Linear layer of the traditional transformer design.Wang Ma Dong Huang 2023 1 In 2025, Microsoft researchers had released an open-weights and open inference code model BitNet b1.58 2B4T demonstrating performance competitive with the full-precision models at 2B parameters and 4T training tokens.Ma Wang Huang Zhang 2025 Post-training quantization BitNet derives its performance from being trained natively in 1.58 bit instead of being quantized from a full-precision model after training. Still, training is an expensive process, and it would be desirable to be able to somehow convert an existing model to 1.58 bits. In 2024, HuggingFace reported a way to gradually ramp up the 1.58-bit quantization in fine-tuning an existing model down to 1.58 bits.Fine-tuning LLMs to 1.58bit: extreme quantization made easy 20 April 2025 https://huggingface.co/blog/1_58_llm_extreme_quantization#pre-training-results-in-158b Critique Some researchersOuyang Ge Hartvigsen Zhang 2024 point out that the scaling lawsKumar Ankner Spector Bordelon 2024 of large language models favor the low-bit weights only in case of undertrained models. As the number of training tokens increases, the deficiencies of low-bit quantization surface. References Sources Ma Shuming Wang Hongyu Ma Lingxiao Wang Lei Wang Wenhui Huang Shaohan Dong Li Wang Ruiping Xue Jilong Wei Furu The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits 2402.17764 2024-02-27 cs.CL 2504.12285 Ma Shuming Wang Hongyu Huang Shaohan Zhang Xingxing Hu Ying Song Ting Xia Yan Wei Furu BitNet b1.58 2B4T Technical Report 2025 cs.CL Friha Othmane Amine Ferrag Mohamed Kantarci Burak Cakmak Burak Ozgun Arda Ghoualmi-Zine Nassira LLM-Based Edge Intelligence: A Comprehensive Survey on Architectures, Applications, Security and Trustworthiness IEEE Open Journal of the Communications Society 5 2024 2644-125X 10.1109/OJCOMS.2024.3456549 free 5799–5856 2024IOJCS...5.5799F 1-bit LLMs Could Solve AI's Energy Demands IEEE Spectrum 2024-05-30 https://spectrum.ieee.org/1-bit-llm Matthew Hutson 2025-04-22 Huyen Chip AI Engineering O'Reilly Media, Inc. 2024-12-04 978-1-0981-6627-4 https://books.google.com/books?id=S7M1EQAAQBAJ&pg=PA330 2025-04-22 2411.04330 Kumar Tanishq Ankner Zachary Spector Benjamin F. Bordelon Blake Muennighoff Niklas Paul Mansheej Pehlevan Cengiz Ré Christopher Raghunathan Aditi Scaling Laws for Precision 2024 cs.LG Morales Jowi Microsoft researchers build 1-bit AI LLM with 2B parameters Tom's Hardware 2025-04-17 https://www.tomshardware.com/tech-industry/artificial-intelligence/microsoft-researchers-build-1-bit-ai-llm-with-2b-parameters-model-small-enough-to-run-on-some-cpus 2025-04-21 2411.17691 Ouyang Xu Ge Tao Hartvigsen Thomas Zhang Zhisong Mi Haitao Yu Dong Low-Bit Quantization Favors Undertrained LLMS: Scaling Laws for Quantized LLMS with 100T Training Tokens 2024 cs.LG 2310.11453 Wang Hongyu Ma Shuming Dong Li Huang Shaohan Wang Huaijie Ma Lingxiao Yang Fan Wang Ruiping Wu Yi Wei Furu BitNet: Scaling 1-bit Transformers for Large Language Models 2023 cs.CL Category:Large language models