Introducing pansophic-1-preview 🦉

We are proud to present pansophic-1-preview, our pioneering effort to train a Large Language Model (LLM) in Romanian. This model represents our first attempt to bring advanced language AI capabilities to our native language.

Due to limited resources, pansophic-1-preview underwent only training and alignment phases, without pre-training. This means we've leveraged the knowledge base of Meta's LLaMA model, focusing our efforts on teaching the model to follow instructions in Romanian.

To achieve this, we created a diverse and high-quality dataset, which we're releasing alongside the model under the Apache-2.0 license. We believe this open approach will enable others to continue our work, create more diverse and higher quality examples, and ultimately move everything forward. Our hope is that the broader Romanian tech community will build upon our foundation, enhancing and expanding the capabilities of Romanian Large Language Models.

Our goal is twofold: to demonstrate that we, as Romanians, are capable of training sophisticated language models, and to actively participate in the ongoing global AI transformation. We invite everyone to help us move this project forward, showcasing Romania's ability to innovate and contribute to the cutting edge of AI development.

Key Features of pansophic-1-preview 🔑

• Advanced Tool Integration
  Seamlessly interacts with various external tools and APIs, enhancing problem-solving capabilities and expanding its knowledge base.
• Transparent Context Handling
  Provides clear indications of context usage and interpretation, improving understanding of its decision-making process.
• High Adherence to System Prompts
  Demonstrates exceptional ability to follow complex instructions and maintain consistent behavior as specified.

These features combine to create a powerful and flexible language model that pushes the boundaries of AI capabilities in Romanian. For a deeper dive into pansophic-1-preview's capabilities and to explore practical use cases, please visit our HuggingFace page.

Download pansophic-1-preview

Benchmarks 📊

pansophic-1-preview has demonstrated exceptional performance across various benchmarks, showcasing its advanced capabilities. This represents a significant step forward in our ongoing efforts, building upon the valuable work of previous models and respected peers in the field.

Limitations ⛔️

When working with AI models, it's important to remember that they might not always align perfectly with human values. This misalignment can sometimes lead to the generation of harmful content. As users, we are responsible for the content we create with these tools.

Additionally, the model can sometimes produce inaccurate results, especially in languages it wasn't extensively trained on, like Romanian (base model pretraining). This might lead to some unusual or incorrect outputs. The model might also occasionally produce sentences that don't quite follow the rules of our language. It's worth noting that this was a preliminary effort to demonstrate the model's capabilities, and limitations in resources were a factor.

As we continue to refine and improve pansophic-1-preview, we encourage users to provide feedback and report any issues they encounter. Your input is invaluable in helping us enhance the model and ensure its accuracy and reliability.

The hidden gem 💎

We think that the most valuable part of our work is the dataset we've created. We're releasing it under the Apache-2.0 license, so that others can build upon it and create more diverse and higher quality examples. We believe that this open approach will enable the Romanian tech community to continue our work, enhancing and expanding the capabilities of Romanian Large Language Models.

We are excited to see what the community will create with this dataset, and we hope that it will inspire others to contribute to the development of Romanian AI models.

Our previous work 🚀

Before pansophic-1-preview, we made significant strides in the field of AI language models with our pioneering project, rocket-3B.

Launched in October 2023, rocket-3B is our first model—a compact yet powerful English language model derived from StableLM-3B. Despite the rapid advancements in AI, rocket-3B continues to be a benchmark in the realm of small-scale language models, showcasing our commitment to efficient and accessible AI solutions.

Key features of rocket-3B include:

• Compact size optimized for on-device deployment
• High proficiency in English language tasks
• Excellent performance-to-size ratio
• Versatility across various NLP applications

Since its release, rocket-3B has achieved a significant milestone of over 30,000 downloads, a number that continues to grow steadily. This impressive adoption rate underscores the model's enduring relevance and the trust placed in it by the AI community.

If you're seeking a lightweight, efficient model capable of running on edge devices while maintaining high performance in English language tasks, we highly recommend exploring rocket-3B. It's an excellent choice for projects where computational resources are limited but quality output is paramount.

The success and insights gained from rocket-3B have been instrumental in our journey towards developing pansophic-1-preview, demonstrating our ongoing commitment to advancing AI technology.

Connect with us 🔗

If there is anyone who would like to collaborate with us or support our work to further develop models for the Romanian language, please reach out to us. We are always looking for new opportunities to expand our reach and impact.

Experience pansophic-1-preview ⚡️

If you are interested in exploring the capabilities of pansophic-1-preview, we invite you to try the model. We encourage you to experiment with the model and provide feedback on your experience. Your input is invaluable in helping us refine and improve the model, ensuring its accuracy and reliability.

Try pansophic-1-preview