Meta Unveils Llama 2: An Open Source, Commercially Licensed AI Language Model

Meta announced on Tuesday its latest development in artificial intelligence: Llama 2. The second iteration of its Llama AI language models, Llama 2 is notable for its commercial license, a feature missing from its predecessor. This significant change means that the models can now be integrated into commercial products, offering a significant shift in the landscape of AI language models.

Introducing Llama 2: The New Generation of AI Models

The Llama 2 models range from 7 to 70 billion parameters and have reportedly outperformed open-source chat models on most benchmarks tested, according to Meta. This reveals an impressive evolution in the model’s capability, in spite of not being able to match the performance of OpenAI's GPT-4.

The Chief AI Scientist of Meta, Yann LeCun, remarked on Twitter that the launch of Llama 2 is set to "change the landscape of the LLM market". For developers and data scientists, the new models are accessible via Microsoft Azure and will soon be available on AWS, Hugging Face, and other providers.

The Technical Side: Pre Trained and Fine-tuned Models

The Llama 2 pretrained models (the bare-bones models) are trained on 2 trillion tokens and have a context window of 4,096 tokens. The context window is critical as it determines the length of the content the model can process at once. Moreover, Meta stated that its Llama 2 fine-tuned models, specifically developed for chat applications like ChatGPT, have been trained on over 1 million human annotations.

Jim Fan, a senior AI scientist at Nvidia, evaluated Llama 2's performance as being close to GPT-3.5 on reasoning tasks. Although there remains a significant gap on coding benchmarks, it performs on par or even better than PaLM-540B on most benchmarks. However, it's still noticeably behind GPT-4 and PaLM-2-L. For an in-depth understanding of Llama 2's performance, benchmarks, and construction, interested readers can refer to a research paper Meta released on Tuesday.

A Brief Look at the Origins

Before Llama 2, Meta launched the original LLaMA in February. It was released as an open-source model but with a non-commercial license. Though it was officially only accessible to certain credentialed academics, the model’s weights were quickly leaked onto torrent sites and became widely disseminated within the AI community. This leak led to the creation of fine-tuned variations of LLaMA, such as Alpaca, which sparked a rapidly growing underground LLM development scene.

What Llama 2 Means for the AI Community

The introduction of Llama 2, with its allowance for commercial use, brings this vibrant activity to the forefront of the AI community. Yet, it's worth noting that entities with "greater than 700 million monthly active users in the preceding calendar month" need to request special permission from Meta for its use. This provision might preclude giants like Amazon or Google from freely utilizing it.

The launch of Llama 2 undoubtedly brings a new dawn in the world of AI language models. As it becomes integrated into various applications and platforms, the potential for advancements in machine learning and artificial intelligence is tremendous. With this development, Meta to assert its place as a leader in AI innovation.

Get updates directly in your mailbox by signing up for our newsletter. Signup Now