Microsoft debuts Phi-3 model

Pro Search

3 steps completed
Answer
Microsoft has recently introduced Phi-3, a new family of small language models (SLMs) that offer impressive performance in a compact size. Here are the key details about Phi-3:
  • Phi-3 Mini is the first model released, with 3.8 billion parameters. Despite its small size, it rivals the performance of much larger models like GPT-3.5 on various benchmarks testing language understanding, reasoning, math and coding abilities.
  • The innovation behind Phi-3's performance lies in its training dataset, which uses heavily filtered web data and synthetic data generated by larger models to teach reasoning skills, rather than just crawling huge amounts of web data.
  • Phi-3 Mini's small size makes it suitable for resource-constrained environments like smartphones and enables fast, low-latency responses. It can run locally on devices without requiring an internet connection.
  • Microsoft achieved the compact yet capable Phi-3 model by using a "curriculum" inspired by how children learn from simple books. They had a larger model generate synthetic children's book data to train Phi-3.
  • Phi-3 Mini is available now on Microsoft's Azure platform, Hugging Face, and the Ollama framework for local deployment.
  • Microsoft plans to release two larger Phi-3 models soon - Phi-3 Small with 7B parameters and Phi-3 Medium with 14B parameters, to provide more options across cost and quality.
In summary, Phi-3 represents a breakthrough in making highly capable AI models that can run efficiently on a wider range of devices, unlocking new possibilities for businesses and developers to incorporate AI into applications where a massive cloud-based model would be impractical. The strong performance of Phi-3 Mini demonstrates the potential for small language models to punch above their weight.
Related
what are the differences between phi-3 mini, small, and large models
how does phi-3 compare to other ai models in terms of performance
what are the potential applications of phi-3