OpenChatKit offers developers an open-source starting point for building chatbots. Released by the Together community, it combines a chat-tuned language model with tools for customization, retrieval, moderation, and user feedback. The project aims to make chatbot development more accessible, while its early performance shows that an open alternative still has ground to cover.
A model and a toolkit for developers
The central model, GPT-NeoXT-Chat-Base-20B, is based on EleutherAI’s 20 billion parameter GPT-NeoX language model. It was tuned for chat use with 43 million instructions. In the HELM benchmark, the chat-tuned model outperformed its base model.
OpenChatKit is available to developers on GitHub under the Apache 2.0 license. The release includes more than the language model itself: it also provides customization recipes intended to help developers fine-tune the model for particular tasks.
Another component is an extensible retrieval system. It can bring information into a response from a document repository, an API, or another source that updates live. That gives developers a way to connect the chatbot with information beyond what the model can supply on its own.
The kit also includes a moderation model fine-tuned from GPT-JT-6B, designed to filter which questions the bot responds to. Users can provide feedback on answers and contribute new datasets, making improvement part of the project’s workflow.
Where the chatbot performs best
Developers identify summarization, answering questions with context, information extraction, and text classification as OpenChatKit’s stronger tasks. These uses give the model a clear input to work from: a passage to summarize, context for a question, or text to classify.
That distinction matters when choosing where to use it. A system built around supplied documents or a defined classification task may suit the model better than one expected to handle any question or create polished prose without supporting context. The retrieval tools can also help connect answers to a repository or live information source.
The source reports that performance improved considerably when OpenChatKit was fine-tuned for specific use cases. Together was developing its own chatbots for learning, financial advice, and support requests. Those examples point to a use pattern centered on adapting the model to a defined purpose.
Early limits in conversation and writing
OpenChatKit was less convincing on questions without context, coding, and creative writing. Those are areas associated with ChatGPT’s popularity, though the source also notes that OpenAI’s chatbot hallucinates regularly. OpenChatKit has its own conversational weaknesses: it can struggle when a discussion changes subject and may repeat an answer.
In a short test, its replies were less eloquent than ChatGPT’s. The article connects that partly to a response limit of 256 tokens, compared with around 500 for ChatGPT. Replies were shorter, but OpenChatKit generated them faster.
The test also found that changing languages did not seem to trouble the bot. It could format output as a list or a table. These abilities offer useful flexibility, although they do not erase the reported difficulties with open-ended questions, coding, or creative writing.
Open development and what comes next
Together is relying on user feedback to improve OpenChatKit. The project also uses a decentralized training approach: instead of keeping the required computing power in one central data center, the developers distribute it across many computers. The article presents that process as a possible direction for large-scale open-source projects.
OpenChatKit was described as the first open-source project to emulate ChatGPT, but not as the last. The source points to Meta’s LLaMa models, which had leaked earlier that month, and says the largest had three times as many parameters as GPT-NeoX-20B. It suggests a chatbot based on those models could follow.
For now, OpenChatKit is best understood as a developer toolkit with a capable but limited chat model. Its open license, customization options, retrieval system, and feedback mechanisms provide ways to shape a chatbot for a particular use. Its weaker performance on broader conversational tasks makes that tailoring especially relevant. The kit could be tried for free on Hugging Face.