Databricks today announced that it has acquired Okera, a data governance platform with a focus on AI. The two companies did not disclose the purchase price. According to Crunchbase, Okera previously raised just under $30 million. Investors include Felicis, Bessemer Venture Partners, Cyber Mentor Fund, ClearSky and Emergent Ventures.

Data governance was already a hot topic, but the recent focus on AI has highlighted some of the shortcomings of the previous approach to it, Databricks notes in today’s announcement. “Historically, data governance technologies, regardless of sophistication, rely on enforcing control at some narrow waist layer and require workloads to fit into the ‘walled garden” at this layer,’ the company explains in a blog post. That approach doesn’t work anymore in the age of large language models (LLMs) because the number of assets is growing too quickly (in part because so much of it is machine-generated) and because the overall AI landscape is changing so quickly, standard access controls aren’t able to capture these changes quickly enough.

Okera then uses an AI-powered system that can automatically discover and classify personally identifiable information, tag it, and apply rules to this (with a focus on the metadata), using a no-code interface.

As the Databricks team stressed, that’s one of the reasons the company was interested in acquiring Okera, but the other is the service’s isolation technology, which can enforce governance control on arbitrary workloads without any major overhead. This technology is still in private preview but was likely one of the major reasons why Databricks acquired the company.

Databricks, which launched its own LLM a few weeks ago, plans to integrate Okera’s technology into its Unity Catalog, its existing governance solution of data and AI assets. The company also noted that the acquisition will enable Databricks to expose additional APIs that its own data governance partners will be able to use to provide solutions to their customers.

With this acquisition, Databricks is also bringing Okera co-founder and CEO Nong Li on board. Li created the Apache Parquet data storage format and was actually briefly an engineer at Databricks between working at Cloudera and before starting Okera, where he was the founding CTO and became the CEO in February 2022.

“As data continues to grow in volume, velocity, and variety across different applications, CIOs, CDOs, and CEOs across the board have to balance those two often conflicting initiatives – not to mention that historically, managing access policies across multiple clouds has been painful and time-consuming,” writes Li in today’s announcement. “Many organizations don’t have enough technical talent to manage access policies at scale, especially with the explosion of LLMs. What they need is a modern, AI-centric governance solution. We could not be more excited to join the Databricks team and to bring our expertise in building secure, scalable and simple governance solutions for some of the world’s most forward-thinking enterprises.”

If you know more about this acquisition, you can contact Frederic on Signal at (860) 208-3416 or by email (frederic@techcrunch.com). You can also reach us via SecureDrop.

Databricks acquires AI-centric data governance platform Okera by Frederic Lardinois originally published on TechCrunch

https://techcrunch.com/2023/05/03/databricks-acquires-ai-centric-data-governance-platform-okera/