The race to create AI assistants that help humans write computer code is heating up. TabbyML, built by two ex-Googlers, has secured $3.2 million in seed funding to work on its open source code generator.
In contrast to GitHub’s Copilot, a self-hosted coding assistant like TabbyML has the advantage of being highly customizable, suggested the startup’s co-founder Meng Zhang. “We believe in a future where all companies will have some sort of customization demand in software development,” he told TechCrunch in an interview.
“There are probably more mature and complete products in the proprietary software space, but if we compare an open source solution with GitHub’s OpenAI-powered tool, there are more limitations to the latter,” he added.
Open source software particularly meets the needs of bigger enterprises, suggested Lucy Gao, Zhang’s co-founder. While independent developers might incorporate open source code in their projects, engineers within enterprises are often pulling code that is proprietary to the organizations and hence out of reach for Copilot.
“For example, if my colleague just wrote a line of code, I can quote it immediately [by using TabbyML],” Gao explained.
Code generators, like other genres of AI pilots, are not always dependable as they can be riddled with bugs. Gao reckoned the challenge is “relatively easy to address” in the case of a self-hosted solution. Every time users choose not to incorporate TabbyML’s suggestions or make edits to its auto-filled code, the AI model finetunes based on that information.
The intent of code generators is to assist human programmers rather than replace them, and there have been promising outcomes. In June, GitHub released a survey showing that Copilot users accepted 30% of the suggestions generated by the coding assistant. Zhang cited another figure that he found more revealing: at a recent developer event, Google announced that 24% of its software engineers experienced more than five “assistive moments” a day using its AI-augmented internal code editor Cider.
Decision-makers might be tempted to cut engineers after implementing a code generator, but Zhang argued “it’s not that simple. Coding isn’t a production line.”
TabbyML, which launched in April, has been starred some 11,000 times on GitHub as of writing. The two investors that participated in its latest round are Yunqi Partners and ZooCap.
When asked about its competition with Copilot the Goliath, Zhang argued that OpenAI’s advantage will taper off as other AI models become more powerful and the costs of computing power decrease over time.
The advantage of GitHub and OpenAI, said Zhang, stems from their capability to deploy AI models with tens of billions of parameters through the cloud. Though the serving cost of such large models is higher, Copilot has so far managed to mitigate expenses to some extent by request batching.
However, the strategy has demonstrated its limitations: In the first few months of this year, Microsoft was losing on average more than $20 a month per GitHub Copilot user, according to a report by the Wall Street Journal.
In contrast, Tabby aims to lower the deployment barrier by recommending models trained on 1-3 billion parameters, an approach that inevitably results in lower quality in the short term.
“However, as the cost of computing power goes down over time and the quality of open source models continues to improve, the competitive edge of GitHub and OpenAI will eventually diminish,” said Zhang.
