StableLM and is designed to efficiently generate text and code
The news was released that Stability AI, the company behind the Stable Diffusion imaging AI model, has announced the first of its set of StableLM language models.
With that Stability hopes to replicate the effects of its open source image synthesis model stable diffusion, released in 2022. With refinement, StableLM could be used to build an open source alternative to ChatGPT.
For those unfamiliar with Stability AI, you should know that this is a London-based company that positions itself as an open source rival to OpenAI, a company that develops powerful but proprietary artificial language models such as ChatGPT.
About StableLM
StableLM is the name of the family of artificial language models created by Stability AI, which are available as open source on GitHub under the Creative Commons BY-SA-4.0 license. StableLM is a text generation model that can compose human text and write programs by predicting the next word in a sequence. It uses a technique called “chip prediction” which involves guessing the next word fragment from the context provided by a human in the form of a “hint”.
Like other "small" LLMs StableLM claims to achieve similar performance to the GPT-3 reference model of OpenAI while using far fewer general parameters (7 billion for StableLM vs. 175 billion for GPT-3).
The release of StableLM builds on our experience with previous open source language models with EleutherAI, a non-profit research center. These language models include GPT-J, GPT-NeoX, and the Pythia suite, which were trained on the open source dataset The Pile.
StableLM claims to have similar performance to GPT-3, the language model that powers ChatGPT, while using far fewer parameters (7 billion vs. 175 billion). Parameters are variables that the model uses to learn from the training data. Having fewer parameters makes the model smaller and more efficient, which can make it easier to run on local devices like smartphones and laptops.
StableLM trained on a new dataset based on The Pile, containing 1,5 trillion tokens, which is about 3 times the size of The Pile. The Pile is a high-quality and diverse dataset for training language models.
Stability AI mentions that the templates are already available in the GitHub repository and that a full white paper is coming soon, and looks forward to continuing to collaborate with developers and researchers as it rolls out the StableLM suite.
In addition, they mention launching the RLHF open collaboration program and working with community efforts like Open Assistant to create an open source dataset for AI assistants.
Last but not least, Speaking of Stability AI releases, we can also highlight that it announced the beta release of SDXL (which stands for Stable Diffusion Extra Large), a new artificial intelligence model capable of generating images from textual descriptions. SDXL is the latest addition to the Stable Diffusion suite, which also includes SD, SDT, and SDC models.
SDXL differs from other models in its size and capabilities. With 2300 billion parameters, SDXL is more than 2,5 times larger than the original SD model, which had only 890 million. These additional parameters allow SDXL to generate images that better adhere to complex patterns. For example, SDXL can produce readable text on images or create strikingly realistic portraits of fictional characters.
SDXL is currently in beta in DreamStudio and other popular imaging applications such as NightCafe Creator. Like all Stability AI models, SDXL will soon be released as open source for optimal accessibility. Stability AI announces that SDXL is permissively licensed for commercial and non-commercial use, as long as you follow ethical and legal guidelines.
Finally, if you are interested in knowing more about it, you can consult the details In the following link.