The Dawn of Poro: Silo AI releases a new AI model for Europe

Are you fascinated by the ever-evolving world of AI and language technology? If so, let's dive into the exciting journey of Poro, an innovative project by Silo AI that is changing the landscape of multilingual language models.

The Birth of Poro

Silo AI, Europe’s largest private AI lab, along with its generative AI arm, SiloGen, and in collaboration with the University of Turku's TurkuNLP Group and the HPLT project, embarked on an ambitious journey to develop Poro. This open-source large language model (LLM) aims to cover all official European languages and programming code, thereby democratizing access to LLMs and ensuring European digital sovereignty.

The Poro 34B Model: A Multilingual Maverick

The heart of Poro lies in its 34 billion parameter LLM, named after the Finnish word for reindeer. Designed primarily for English and Finnish, Poro also excels in various programming languages. With its innovative BLOOM architecture and ALiBi embeddings, Poro 34B boasts advanced capabilities, including basic translation between English and Finnish, and a context window extrapolation feature. What's more, the model is trained with an impressive dataset of 1 trillion tokens and uses 512 AMD MI250X GPUs on the LUMI supercomputer in Finland.

Addressing the Challenge of Low-Resource Languages

A remarkable aspect of Poro is how it tackles the challenge posed by low-resource languages like Finnish. Typically, training LLMs requires vast amounts of data, often unavailable for such languages. Poro ingeniously addresses this by cross-training low-resource languages with high-resource ones, thereby enhancing performance and teaching basic translation capabilities. Notably, after 30% of its training phase, Poro has already surpassed the state-of-the-art performance on the Finnish language benchmark FIN-bench.

For Researchers and Beyond

Poro isn't just a technological marvel; it's a beacon of knowledge sharing. Through the Poro Research Checkpoints program, external researchers gain unprecedented access to the model's training process, fostering greater visibility and understanding of language model training. However, it's important to note that these checkpoints are intended for academic and industry research and are not yet ready for deployment in production without additional training and testing.

Poro: A Collaboration Triumph

This project exemplifies the power of collaboration. Combining Silo AI's industry expertise with the University of Turku's research prowess, Poro stands as a testament to the synergy between academic research and practical AI applications. The TurkuNLP Group's contribution, with its extensive experience in open-source NLP resources, has been invaluable in this endeavor.

SiloGen: The Driving Force

SiloGen, integral to Poro's development, is a testament to Europe's commitment to advancing generative AI technology. This initiative leverages Europe’s leading generative AI and LLM expertise, powerful computational resources, and rich data sources to train, run, and operate LLMs. Operational since late 2022, SiloGen is working with renowned clients to provide accurate, trustworthy, and robust AI applications.

The Future is Bright

Poro is more than just a language model; it's a beacon of innovation, collaboration, and progress in the realm of AI and language technology. As it continues to grow and evolve, Poro promises to be a key player in the advancement of multilingual language processing, bringing a world of languages and cultures closer together through the power of AI.