Llama 2 is a collection of pretrained and fine-tuned generative text models developed by Meta AI, with models ranging in scale from 7 billion to 70 billion parameters [1][3][5]. The Llama 2 70B model is part of this collection and is one of the largest models in the family [1][3][5].
The Llama 2 70B model is an auto-regressive language model that uses an optimized transformer architecture [1][3]. It was trained on a new mix of publicly available online data, with a total of two trillion tokens used for pretraining [1][3]. The model uses Grouped-Query Attention (GQA) for improved inference scalability [1][3].
The Llama 2 70B model was trained between January 2023 and July 2023[1][3]. It is a static model trained on an offline dataset, and future versions of the tuned models will be released as improvements are made in model safety with community feedback [1][3].
The Llama 2 70B model is specifically designed for dialogue use cases and is optimized to generate human-like responses to natural language input, making it suitable for chatbot and conversational AI applications [4]. It was fine-tuned for chat using a specific structure for prompts, relying on special tokens to structure the conversation [4].
The Llama 2 70B model is a 70 billion parameter model pretrained on a large corpus of text that includes conversational data [4]. It is comparable in quality to closed proprietary models such as OpenAI's ChatGPT and Google's PaLM[6].
The Llama 2 70B model is licensed under the Llama 2 Community License, and a custom commercial license is available [1][3]. It is freely available for research and commercial use, enabling developers to build on top of it and leverage it for commercial use cases [6].
Citations:
Comments