Data has emerged as one of the world’s greatest resources, underpinning everything from video-recommendation engines and digital banking, to the burgeoning AI revolution. But in a world where data has become increasingly distributed across locations, from databases to data warehouses to data lakes and beyond, combining it all into a compatible format for use in real-time scenarios can be a mammoth undertaking.
For context, applications that don’t require instant, real-time data access can simply combine and process data in batches at fixed intervals. This so-called “batch data processing” can be useful for things like processing monthly sales data. But often, a company will need real-time access to data as it’s created, and this might be pivotal for customer support software that relies on current information about each and every sale, for example.
Elsewhere, ride-hail apps also need to process all manner of data points in order to connect a rider with a driver — this isn’t something that can wait a few days. These kinds of scenarios require what is known as “stream data processing,” where data is collected and combined for real-time access, which is far more complex to configure.
This is what Dozer is setting out to address, by powering fast, read-only APIs directly from any source via a plug-and-play data infrastructure back end.
Dozer is the handiwork of Vivek Gudapuri and Matteo Pelati, who founded the company from their base in Singapore nearly a year ago. The duo has built a distributed team of 10 across Asia and Eastern Europe as it gears up to expand beyond the product’s current source available (i.e. not-quite open source) incarnation into a fully monetizable product.
Dozer has been testing its product with a handful of undisclosed design partners, and today it’s emerging from stealth for any developer to access. The company also revealed it has raised $3 million in seed funding from Sequoia Capital’s Indian arm (via its Surge program), Google’s Gradient Ventures, and January Capital.
Distributed
There are already countless tools out there designed to transform, integrate and harness distributed data, including streaming databases and ETL (extract, transform, load) tools such as Apache Flink, Airbyte and Fivetran; caching layers for transient data storage such as Redis; and instant APIs powered by the likes of Hasura or Supabase to funnel data between systems.
Dozer, for its part, works across all these various categories, adopting what it deems to be the best parts and removing the friction that goes with building the infrastructure and plumbing that underpin real-time data apps.
Users plug Dozer into their existing data stack, which may include databases, data warehouses and data lakes, and Dozer takes care of real-time data extraction, caching and indexing, and surfacing it through low-latency APIs. So while something like Airbyte or Fivetran helps with getting data into a data warehouse, Dozer focuses on the other side: “making this data accessible in the most efficient way,” Gudapuri explained to TechCrunch.
Gudapuri said that Dozer “takes an opinionated approach,” that tackles very specific problems and no more. For instance, incumbent streaming databases solve many problems far beyond what Dozer offers, which is all about serving real-time data updates and APIs in a single product.
“We solve just the right amount of problems in each of these categories to offer a fast building experience for developers, as well as ready-to-go performance,” Gudapuri said. “Developers (currently) have to integrate several tools to achieve the same.”
By way of example, an existing streaming database will probably try to present the entire database experience to the user, replete with query engine, data exploration, OLAP (online analytical processing) and so on. Dozer deliberately doesn’t offer these things, instead focusing on what Pelati calls “pre-computed views” using SQL, Python and JavaScript, all accessible via low-latency gRPC and REST APIs.
It’s for this reason, Pelati says, that Dozer can promise better data-query latency.
“Because of these design choices, Dozer offers a far superior query latency, which is necessary for customer-facing applications,” Pelati said. “A single developer can spin-up entire data apps in minutes; that would typically take months of effort. A team doesn’t have to build and maintain several integrations, saving time and money.”
The (not-quite) open source factor
While Dozer is touted as an “open source” platform, a quick peek at its license on GitHub reveals that it uses an Elastic license 2.0 (ELv2), the same license that enterprise search company Elastic adopted two years ago as part of its transition away from true open source. Indeed, the Elastic license is not recognized as open source, as it prevents third-parties from taking the software and offering it themselves as a hosted or managed service.
More accurately, ELv2 can be called a “source available” license, which effectively means that it offers many of the benefits of a more permissive open source license such as MIT, including codebase transparency, the ability to extend Dozer’s capabilities, or fine-tune features and fix bugs. This alone will likely be enough to win the hearts and minds of businesses of all sizes so long as it’s not AWS or some other cloud giant looking to monetize directly on top of Dozer.
However, the company said that it does intend to switch to a dual license “very soon,” where everything in the core Dozer project will be MIT-licensed except for “one core module.” Moreover, the company is quick to stress that all of its client libraries are already MIT-licensed, including Python, React and JavaScript.
It’s worth noting that some companies have created internal tooling to solve a similar problem to what Dozer is tackling, including Netflix, which built Bulldozer several years back. Notably, one of the main creators behind Bulldozer, Ioannis Papapanagiotou, now works as an advisor to Dozer.
It’s still early days for Dozer, but with $3 million in the bank from a host of high-profile backers, the company is fairly well-financed as it pushes through to commercialization, which will include introducing a hosted SaaS version replete with a bunch of add-on features. Gudapuri said it expects this to go live in the coming months.
“The hosted service will take care of auto-scaling, instant deployments, security, compliance, rate-limiting and some additional features,” Gudapuri said.
Comment