The next phase of artificial intelligence may not depend only on building ever larger systems in remote data centers. A new wave of smaller AI models, including Microsoft’s Phi-3-mini, suggests that useful assistants can run directly on everyday devices such as laptops and smartphones.
That matters because local AI changes the tradeoff. Instead of sending every request to the cloud, a device could handle more work itself, reducing delays, avoiding some outages, and keeping data on the device.
Why Phi-3-mini matters
When ChatGPT was released in November 2023, the model behind it was too large to run directly on personal hardware. Access depended on the cloud. The source article describes a very different experience today: a similarly capable AI program running on a Macbook Air without pushing the machine hard.
The model is Phi-3-mini, part of a family of smaller AI models released by researchers at Microsoft. It is compact enough to run on a smartphone, though the article describes testing it on a laptop and accessing it from an iPhone through Enchanted, an app that offers a chat interface similar to the official ChatGPT app.
Microsoft researchers say in a paper describing the Phi-3 family that Phi-3-mini compares favorably with GPT-3.5, the OpenAI model behind the first release of ChatGPT. That comparison is based on standard AI benchmarks intended to test common sense and reasoning. The author’s own testing also found the model to appear similarly capable.
The shift from bigger to better trained
The usual story of modern AI has been scale. Large language models such as OpenAI’s GPT-4 and Google’s Gemini are trained on very large collections of text from books, websites, and other accessible sources. The article notes that this approach has raised legal questions, but it has also helped unlock new capabilities as models and training compute increased.
Phi-3 points to another path. Sébastien Bubeck, a Microsoft researcher involved with the project, said the team wanted to test whether careful selection of training material could shape a model’s abilities without simply expanding the training data.
Last September, Bubeck’s team trained a model roughly one-17th the size of OpenAI’s GPT-3.5 on "textbook quality" synthetic data generated by a larger AI model. The data included factoids from specific domains, including programming. The resulting model showed unexpected strength for its size.
Lo and behold, what we observed is that we were able to beat GPT-3.5 at coding using this technique.
That was really surprising to us.
The broader lesson is not that small models can replace every large model. It is that the quality and focus of training data can matter deeply. A smaller system can become useful if it is trained on material that teaches the right patterns and skills.
Local AI could change everyday apps
Running AI models locally means the model operates on a smartphone, laptop, or PC instead of relying on a cloud service for every interaction. The source article identifies several direct advantages: lower latency, fewer problems when cloud access fails, and stronger privacy because the data stays on the device.
Those advantages could change what developers build. If an AI system can work close to the operating system, it may support features that are hard to deliver through a cloud-only design. Microsoft’s Recall feature is one example mentioned in the source: it uses AI to make everything a person did on a PC searchable, and offline algorithms are described as a key part of it.
Local AI also makes the user experience feel different. A cloud chatbot depends on a connection, a remote service, and the path between the user and that service. A device-based model can respond from the hardware already in front of the user. That does not make cloud AI irrelevant, but it expands the range of practical designs.
- Responsiveness: local processing can reduce the delay that comes from sending queries to the cloud.
- Reliability: device-based AI can avoid some outages tied to remote services.
- Privacy: keeping data on a phone, laptop, or PC can prevent it from leaving the device.
- Integration: smaller AI models may support apps built more deeply into an operating system.
Multimodal models are arriving on two tracks
The movement toward smaller models is happening alongside a push toward multimodal AI. Microsoft announced a new multimodal Phi-3 model capable of handling audio, video, and text at its annual developer conference, Build. That announcement came days after OpenAI and Google promoted new AI assistants built on multimodal models accessed through the cloud.
This contrast is important. One track emphasizes powerful cloud systems. The other explores what can happen when capable models become small enough to fit on personal devices. Both approaches can advance at the same time, but they support different kinds of products.
For cloud-based assistants, the central question is how much capability can be delivered through remote infrastructure. For local AI, the question is how much intelligence can be brought directly onto the machine a person already uses. Phi-3-mini makes that second question more concrete.
What this means for the future of computing
Bubeck says the results suggest future AI systems will need more than simple scale to become smarter. The Phi work supports the idea that carefully chosen training material can make smaller systems far more useful than their size might imply.
That could make scaled-down models a major part of future computing. If AI can run on phones, laptops, and PCs, it can become less like a remote website and more like a built-in layer of the device. The source article also notes that Apple is widely expected to unveil its long-awaited AI strategy at its WWDC conference next month, and that Apple has previously emphasized local machine learning on its devices.
The central point is clear: the future of AI will not be defined only by enormous models in the cloud. Smaller AI models such as Phi-3-mini show that capability, efficiency, and careful training can combine in ways that bring useful intelligence closer to the user.