top of page

Llama 2 70B foundation Model is now available

Llama 2 is a collection of pretrained and fine-tuned generative text models developed by Meta AI, with models ranging in scale from 7 billion to 70 billion parameters [1][3][5]. The Llama 2 70B model is part of this collection and is one of the largest models in the family [1][3][5].


The Llama 2 70B model is an auto-regressive language model that uses an optimized transformer architecture [1][3]. It was trained on a new mix of publicly available online data, with a total of two trillion tokens used for pretraining [1][3]. The model uses Grouped-Query Attention (GQA) for improved inference scalability [1][3].


The Llama 2 70B model was trained between January 2023 and July 2023[1][3]. It is a static model trained on an offline dataset, and future versions of the tuned models will be released as improvements are made in model safety with community feedback [1][3].


The Llama 2 70B model is specifically designed for dialogue use cases and is optimized to generate human-like responses to natural language input, making it suitable for chatbot and conversational AI applications [4]. It was fine-tuned for chat using a specific structure for prompts, relying on special tokens to structure the conversation [4].


The Llama 2 70B model is a 70 billion parameter model pretrained on a large corpus of text that includes conversational data [4]. It is comparable in quality to closed proprietary models such as OpenAI's ChatGPT and Google's PaLM[6].

The Llama 2 70B model is licensed under the Llama 2 Community License, and a custom commercial license is available [1][3]. It is freely available for research and commercial use, enabling developers to build on top of it and leverage it for commercial use cases [6].



Citations:



53 views0 comments

Recent Posts

See All

Comments

Couldn’t Load Comments
It looks like there was a technical problem. Try reconnecting or refreshing the page.
bottom of page