Data has emerged as one of the world’s greatest resources, underpinning everything from video-recommendation engines and digital banking, to the burgeoning AI revolution. But in a world where data has become increasingly distributed across locations, from databases to data warehouses to data lakes and beyond, combining it all into a compatible format for use in real-time scenarios can be a mammoth undertaking.
For context, applications that don’t require instant, real-time data access can simply combine and process data in batches at fixed intervals. This so-called “batch data processing” can be useful for things like processing monthly sales data. But often, a company will need real-time access to data as it’s created, and this might be pivotal for customer support software that relies on current information about each and every sale, for example. Elsewhere, ride-hail apps also need to process all manner of data points in order to connect a rider with a driver — this isn’t something that can wait a few days. These kinds of scenarios require what is known as “stream data processing,” where data is collected and combined for real-time access — something that is far more complex to configure.
And this is something that Dozer is setting out to address, by powering fast, read-only APIs directly from any source via a plug-and-play data infrastructure backend.
Dozer in the handiwork of Vivek Gudapuri and Matteo Pelati, who founded the company from their base in Singapore nearly a year ago. The duo have built a distributed team of 10 across Asia and Eastern Europe as they gear up to expand beyond the product’s current source available (i.e. not-quite open source) incarnation and into a fully monetizable product.
Dozer has been testing its product with a handful of undisclosed design partners, and today it’s emerging from stealth for any developer to access. The company also revealed it has raised $3 million in seed funding from Sequoia Capital India, Google’s Gradient Ventures, Surge, and January Capital.
Dozer co-founders Matteo Pelati and Vivek Gudapuri Image Credits: Dozer
Distributed
There are already countless tools out there designed to transform, integrate, and harness distributed data, including streaming databases and ETL (extract, transform, load) tools such as Apache Flink, Airbyte and Fivetran; caching layers for transient data storage such as Redis; and instant APIs powered by the likes of Hasura or Supabase to funnel data between systems.
Dozer, for its part, works across all these various categories, adopting what it deems to be the best parts and removing the friction that goes with building the infrastructure and plumbing that underpin real-time data apps.
Users plug Dozer into their existing data stack, which may include databases, data warehouses, and data lakes, and Dozer takes care of real-time data extraction, caching, and indexing, and surfacing it through low-latency APIs. So while something like Airbyte or Fivetran helps with getting data into a data warehouse, Dozer focuses on the other side — “making this data accessible in the most efficient way,” Gudapuri explained to TechCrunch.
Gudapuri said that Dozer “takes an opinionated approach,” one that tackles very specific problems and no more. For instance, incumbent streaming databases solve many problems far beyond what Dozer offers, which is all about serving real-time data updates and APIs in a single product.
“We solve just the right amount of problems in each of these categories to offer a fast building experience for developers, as well as ready-to-go performance,” Gudapuri said. “Developers (currently) have to integrate several tools to achieve the same.”
By way of example, an existing streaming database will probably try to present the entire database experience to the user, replete with query engine, data exploration, OLAP (online analytical processing), and so on. Dozer deliberately doesn’t offer these things, instead focusing on what Pelati calls “pre-computed views” using SQL, Python, and JavaScript, and all accessible via low-latency gRPC and REST APIs.
And it’s for this reason, Pelati says, Dozer can promise better data-query latency.
“Because of these design choices, Dozer offers a far superior query latency which is necessary for customer-facing applications,” Pelati said. “A single developer can spin-up entire data apps in minutes, that would typically take months of effort. A team doesn’t have to build and maintain several integrations saving time and money.”
The (not-quite) open source factor
While Dozer is touted as an “open source” platform, a quick peek at its license on GitHub reveals that it uses an Elastic license 2.0 (ELv2), the very same license enterprise search company Elastic adopted two years ago as part of its transition away from true open source. Indeed, the Elastic license is not recognized as open source, as it prevents third-parties from taking the software and offering it themselves as a hosted or managed service.
More accurately, ELv2 can be called a “source available” license, which effectively means that it does offer many of the benefits of a more permissive open source license such as MIT, including codebase transparency, the ability to extend Dozer’s capabilities, or fine-tune features and fix bugs. This alone will likely be enough to win the hearts and minds of businesses of all sizes, so long as it’s not AWS or some other cloud giant looking to monetize directly on top of Dozer.
However, the company said that it does intend to switch to a dual-license “very soon,” where everything in the core Dozer project will be MIT-licensed except for “one core module.” Moreover, the company is quick to stress that all of its client libraries are already MIT-licensed, including Python, React, and JavaScript.
It’s worth noting that some companies have created internal tooling themselves to solve a similar problem to what Dozer is tackling, including Netflix which built Bulldozer several years back. Notably, one of the main creators behind Bulldozer, Ioannis Papapanagiotou, now works as an advisor to Dozer.
It’s still early days for Dozer, but with $3 million in the bank from a host of high-profile backers, the company is fairly well-financed as it pushes through to commercialization, which will include introducing a hosted SaaS version replete with a bunch of add-on features. Gudapuri said it expects this to go live in the coming months.
“The hosted service will take care of auto-scaling, instant deployments, security, compliance, rate-limiting and some additional features,” Gudapuri said.
Dozer exits stealth to help any developer build real-time data apps ‘in minutes’ by Paul Sawers originally published on TechCrunch
