BLOOM LLM：BLOOM (language model)

From Wikipedia, the free encyclopedia

Open-access multilingual language model

BigScience Large Open-science Open-access Multilingual Language Model (BLOOM)[1][2] is a 176-billion-parameter transformer-based autoregressive large language model (LLM). The model, as well as the code base and the data used to train it, are distributed under free licences.[3] BLOOM was trained on approximately 366 billion (1.6TB) tokens from March to July 2022.[4][5]

BLOOM is the main outcome of the BigScience collaborative initiative,[6] a one-year-long research workshop that took place between May 2021 and May 2022. BigScience was led by HuggingFace and involved several hundreds of researchers and engineers from France and abroad representing both the academia and the private sector. BigScience was supported by a large-scale public compute grant on the French public supercomputer Jean Zay, managed by GENCI and IDRIS (CNRS), on which it was trained.

BLOOMs...