Bloomberg built BloombergGPT to handle work in the financial sector, combining the company’s financial data with general online text. The project shows how a model trained for a particular industry can serve a range of tasks, from interpreting financial news to helping users retrieve database information.
Training on financial and general text
Bloomberg’s AI teams assembled an English-language dataset with 363 billion financial-specific tokens from the company’s own data assets. They added 345 billion generic tokens from online text datasets The Pile, C4, and Wikipedia.
From that dataset, the team used 569 billion tokens to train BloombergGPT, a 50-billion-parameter decoder-only language model optimized for financial tasks. The model uses the open source Bloom language model as its base architecture.
The mix reflects the model’s intended role. Financial material gives it exposure to language and information specific to the sector, while general text broadens the kinds of language tasks it can address. Bloomberg’s project therefore offers an example of how industry data can shape a language model without limiting it to a single specialized task.
Performance across specialized and general tasks
Bloomberg reports that BloombergGPT outperforms open-source models GPT-NeoX, OPT, and Bloom on finance-specific tasks. On generic language tasks, including summaries, it also exceeds those models in some cases, and Bloomberg says its results are almost on par with GPT-3 according to the company’s benchmarks.
Those comparisons point to a central goal: a financial language model that can perform specialized work while remaining useful for broader language tasks. The reported results are based on Bloomberg’s benchmarks, as described in the source article.
Gideon Mann, Head of Bloomberg’s ML Product and Research team, linked model quality to the data used to train it. That emphasis is especially relevant in finance, where a model’s usefulness depends in part on whether it can work with the kinds of material and terminology found in the sector.
Potential uses in financial technology
Bloomberg describes several possible applications for language models in financial technology. These include sentiment analysis of articles about individual companies, automatic entity recognition, and answering financial questions. Bloomberg’s news division could also use the model to generate headlines for newsletters.
Another described use connects everyday requests to Bloomberg’s own query language, BQL. With only a few examples, the model could formulate BQL queries to extract data from a database. A user can describe the data they need in natural language, and the model can generate the corresponding query.
This kind of translation could make data retrieval easier for people who know what information they want but do not want to write the query language themselves. It also illustrates how a domain-specific language model can support multiple activities within one industry.
One model for a range of applications
Bloomberg CTO Shawn Edwards said the company sees value in applying generative language models to few-shot learning, text generation, and conversational systems. He said a model focused on finance could help Bloomberg develop new kinds of applications and achieve higher performance than custom models built for each application, while reaching the market faster.
The broader proposition is that a shared, finance-focused model could provide a foundation for different tools instead of requiring a separate custom model for every use. BloombergGPT’s described capabilities span text analysis, question answering, headline generation, and database queries. How useful those applications prove in practice depends on how well the model handles each task and the data it is given.
BloombergGPT thus presents an industry-focused approach to language AI: train on substantial domain material, combine it with general text, and apply the resulting model across related workflows. Bloomberg’s reported benchmarks and examples make the case for this approach within financial technology.