Databricks expands Mosaic AI to help enterprises build with LLMs

Comment
A year ago, Databricks acquired MosaicML for $1.3 billion. Now rebranded as Mosaic AI, the platform has become integral to Databricks’ AI solutions. Today, at the company’s Data + AI Summit, it is launching a number of new features for the service. Ahead of the announcements, I spoke to Databricks co-founders CEO Ali Ghodsi and CTO Matei Zaharia.
Databricks is launching five new Mosaic AI tools at its conference: Mosaic AI Agent Framework, Mosaic AI Agent Evaluation, Mosaic AI Tools Catalog, Mosaic AI Model Training and Mosaic AI Gateway.
“It’s been an awesome year — huge developments in GenAI. Everybody’s excited about it,” Ghodsi told me. “But the things everybody cares about are still the same three things: How do we make the quality or reliability of these models go up? Number two, how do we make sure that it’s cost-efficient? And there’s a huge variance in cost between models here — a gigantic, orders-of-magnitude difference in price. And third, how do we do that in a way that we keep the privacy of our data?”
Today’s launches aim to cover the majority of these concerns for Databricks’ customers.
Zaharia also noted that the enterprises that are now deploying large language models (LLMs) into production are using systems that have multiple components. That often means they make multiple calls to a model (or maybe multiple models, too), and use a variety of external tools for accessing databases or doing retrieval augmented generation (RAG). These compound systems speed up LLM-based applications, save money by using cheaper models for specific queries or caching results and, maybe most importantly, make the results more trustworthy and relevant by augmenting the foundation models with proprietary data.
“We think that is the future of really high-impact, mission-critical AI applications,” he explained. “Because if you think about it, if you’re doing something really mission critical, you’ll want engineers to be able to control all aspects of it — and you do that with a modular system. So we’re developing a lot of basic research on what’s the best way to create these [systems] for a specific task so developers can easily work with them and hook up all the bits, trace everything through and see what’s happening.”
As for actually building these systems, Databricks is launching two services this week: the Mosaic AI Agent Framework and the Mosaic AI Tools Catalog. The AI Agent Framework takes the company’s serverless vector search functionality, which became generally available last month and provides developers with the tools to build their own RAG-based applications on top of that.
Ghodsi and Zaharia emphasized that the Databricks vector search system uses a hybrid approach, combining classic keyword-based search with embedding search. All of this is integrated deeply with the Databricks data lake and the data on both platforms is always automatically kept in sync. This includes the governance features of the overall Databricks platform — and specifically the Databricks Unity Catalog governance layer — to ensure, for example, that personal information doesn’t leak into the vector search service.
Talking about the Unity Catalog (which the company is now also slowly open sourcing), it’s worth noting that Databricks is now extending this system to let enterprises govern which AI tools and functions these LLMs can call upon when generating answers. This catalog, Databricks says, will also make these services more discoverable across a company.
Ghodsi also highlighted that developers can now take all of these tools to build their own agents by chaining together models and functions using Langchain or LlamaIndex, for example. And indeed, Zaharia tells me that a lot of Databricks customers are already using these tools today.
“There are a lot of companies using these things, even the agent-like workflows. I think people are often surprised by how many there are, but it seems to be the direction things are going. And we’ve also found in our internal AI applications, like the assistant applications for our platform, that this is the way to build them,” he said.
To evaluate these new applications Databricks is also launching the Mosaic AI Agent Evaluation, an AI-assisted evaluation tool that combines LLM-based judges to test how well the AI does in production, but also allows enterprises to quickly get feedback from users (and let them label some initial datasets, too). The Agent Evaluation includes a UI component based on Databricks’ acquisition of Lilac earlier this year, which lets users visualize and search massive text datasets.
“Every customer we have is saying: I do need to do some labeling internally, I’m going to have some employees do it. I just need maybe 100 answers, or maybe 500 answers — and then we can feed that into the LLM judges,” Ghodsi explained.
Another way to improve results is by using fine-tuned models. For this, Databricks now offers the Mosaic AI Model Training service, which — you guessed it — allows its users to fine-tune models with their organization’s private data to help them perform better on specific tasks.
The last new tool is the Mosaic AI Gateway, which the company describes as a “unified interface to query, manage, and deploy any open source or proprietary model.” The idea here is to allow users to query any LLM in a governed way, using a centralized credentials store. No enterprise, after all, wants its engineers to send random data to third-party services.
In times of shrinking budgets, the AI Gateway also allows IT to set rate limits for different vendors to keep costs manageable. Additionally, these enterprises then also get usage tracking and tracing for debugging these systems.
As Ghodsi told me, all of these new features are a reaction to how Databricks’ users are now working with LLMs. “We saw a big shift happen in the market in the last quarter and a half. Beginning of last year, anyone you talk to, they’d say: we’re pro open source, open source is awesome. But when you really pushed people, they were using Open AI. Everybody, no matter what they said, no matter how much they were touting how open source is awesome, behind the scenes, they were using Open AI.” Now, these customers have become far more sophisticated and are using open models (very few are really open source, of course), which in turn requires them to adopt an entirely new set of tools to tackle the problems — and opportunities — that come with that.
Every weekday and Sunday, you can get the best of TechCrunch’s coverage.
Startups are the core of TechCrunch, so get our best coverage delivered weekly.
The latest Fintech news and analysis, delivered every Tuesday.
TechCrunch Mobility is your destination for transportation news and insight.
By submitting your email, you agree to our Terms and Privacy Notice.
Hiya, folks, and welcome to TechCrunch’s regular AI newsletter. This week in AI, Apple stole the spotlight. At the company’s Worldwide Developers Conference (WWDC) in Cupertino, Apple unveiled Apple Intelligence,…
360 One WAM, India’s largest wealth manager focused on ultra-high-net-worth individuals, has agreed to acquire popular Indian mutual fund investment app ET Money for about $44 million. 360 One disclosed…
Helen Toner, a former OpenAI board member and the director of strategy at Georgetown’s Center for Security and Emerging Technology, is worried Congress might react in a “knee-jerk” way where…
Layoffs are tough. This year alone, we’ve already seen 60,000 job cuts across 254 companies according to layoffs.fyi. Looking for ways to grow your network can be even harder during…
YouTube announced this week the rollout of “Thumbnail Test & Compare,” a new tool for creators to see which thumbnail performs the best. The feature first launched to select creators…
Waymo has voluntarily issued a software recall to all 672 of its Jaguar I-Pace robotaxis after one of them collided with a telephone pole. This is Waymo’s second recall. The…
The hotel guest management technology company’s platform digitizes the hotel guest journey from post-booking through checkout.
The TechCrunch team runs down all of the biggest news from the Apple WWDC 2024 keynote in an easy-to-skim digest.
InScope leverages machine learning and large language models to provide financial reporting and auditing processes for mid-market and enterprises.
Venture fundraising has been a slog over the last few years, even for firms with a strong track record. That’s Foresite Capital’s experience. Despite having 47 IPOs, 28 M&As and…
A year ago, Databricks acquired MosaicML for $1.3 billion. Now rebranded as Mosaic AI, the platform has become integral to Databricks’ AI solutions. Today, at the company’s Data + AI…
RetailReady targets the $40 billion compliance market to help reduce the number of retail compliance losses that shippers incur annually due to incorrectly shipped packages.
Since its launch in 2013, Databricks has relied on its ecosystem of partners, such as Fivetran, Rudderstack, and dbt, to provide tools for data preparation and loading. But now, at…
A big shoutout to the early-stage founders who missed the application window for the Startup Battlefield 200 (SB 200) at TechCrunch Disrupt. We have exciting news just for you! You…
When one of the co-creators of the popular open source stream-processing framework Apache Flink launches a new startup, it’s worth paying attention. Stephan Ewen was among the founding team of…
With most residential solar panels installed by smaller companies, customer experience can be a mixed bag. To try to address the quality and consistency problem, Civic Renewables is buying small…
Small VC firms require deep trust, mutual support, and long-term commitment among the partners —a kinship that, in many ways, resembles a family dynamic. Colin Anderson (Palantir’s ex-CFO and former…
Fisker is issuing the first recall for its all-electric Ocean SUV because of problems with the warning lights, according to new information published by the National Highway Traffic Safety Administration.…
Gorilla, a Belgian company that serves the energy sector with real-time data and analytics for pricing and forecasting, has raised €23 million ($25 million) in a Series B round led…
South Korea’s fabless AI chip industry saw a slew of fundraising events over the last couple of years as demand for hardware to power AI applications skyrocketed, and it seems…
Here’s a list of third-party apps that were Sherlocked by Apple at this year’s WWDC.
Black Semiconductor, which is developing a chip-connecting technology based on graphene, has raised $273M in a combination of private and public funding. 
Featured Article
It’s not the sexiest of subject matters, but someone needs to talk about it: The CFO tech stack — software used by the chief financial officers of the world — is ripe for disruption. That’s according to Jonathan Sanders, CEO and co-founder of fledgling Danish startup Light, which exits stealth…
Fresh off the success of its first mission, satellite manufacturer Apex has closed $95 million in new capital to scale its operations.  The Los Angeles-based startup successfully launched and commissioned…
After educating the D.C. market, YC aims to leverage its influence, particularly in areas like competition policy.
Lina Khan says the FTC wants to be effective in its enforcement strategy, which is why it has been taking on lawsuits that “go up against some of the big…
With dozens of antitrust cases and close to a hundred on the consumer protection side, the agency is now turning to innovative tactics to help it fight fraud, particularly in…
The ability to pause your activity rings is a minor feature update for most, but for those of us who obsess about such things to an unhealthy degree, it’s the…
Featured Article
It’s a very Apple approach in the sense that it prioritizes a frictionless user experience above all.
When generative AI tools started making waves in late 2022 after the launch of ChatGPT, the finance industry was one of the first to recognize these tools’ potential for speeding…
Powered by WordPress VIP

source
Sponsor:News technical sponsor
Sponsor:News AI sponsor
Sponsor: AI sponsor
Sponsor: AI sponsor