The rise of streaming architectures — frameworks of software components built to ingest and process large volumes of data from multiple sources — is driving the demand for better reliability and performance. Engineering teams often encode data to improve app performance by using what are known as “message envelopes.” But these add complexity — and tend to be difficult and costly to debug.
Daniel Selans and Ustin Zarubin — engineers by trade, having worked at New Relic, InVision, DigitalOcean and Community.com — thought what was needed is a way to detect anomalous behavior in encoded data streams. After running into problems with streaming data frameworks, they co-founded Streamdal, which not only alerts users to streaming issues but can also transform in-flight data and reprocess broken data on the fly.
“We saw the need for more actionable insights for streaming data in distributed systems,” Selans told TechCrunch in an email interview. “Alternative approaches can’t introspect streaming data and instead rely on metadata-driven metrics. Additionally, given most companies using streaming also utilize some form of data encoding, there are no tools that can read that encoded data.”
Beyond monitoring for critical data issues, Streamdal uses AI, including natural language processing algorithms, to detect personally identifiable information in streams and take action on it (e.g., redact it). The company also maintains an open source package, Plumber, that can be used to dig into data streams and connect disparate streaming systems together.
Image Credits: Streamdal
Future capabilities might include providing a more detailed lineage across data streams and analyzing in-flight data for schema changes, Selans says.
Selans sees Streamdal competing mostly against in-house engineering teams who’ve strung together purpose-built, custom solutions for their employers. He wasn’t at liberty to name many clients for “contractual reasons,” but revealed that Recharge and ParkMobile are among Streamdal’s higher-profile paying customers. Meanwhile, Plumber has been downloaded over 150,000 times, Selans claims.
“We are helping enterprises monitor and semantically analyze billions of events in their event-driven architectures for data issues such as real-time schema changes that otherwise might lead to potential customer outages,” Selans said.
As for the current economic headwinds and whether they might impact business, Selans doesn’t believe they will. “We believe that even with widespread layoffs, companies still need to maintain their event-driven architecture powering their distributed systems, and may even require additional support to manage these complex systems,” he added.
Streamdal itself — a Y Combinator graduate — appears to be well positioned to weather the storm, having raised $5.4 million in a seed round led by Work-Bench with participation by Crosscut, Verissimo, Data Council and unnamed angel investors. To date, the company has raised $7.2 million in venture funding, which Selans says is being put toward strategic hires (Streamdal has a ten-person team), product and go-to-market initiatives.
Kelley Mak, a partner at Work-Bench, added in an emailed statement: “Thanks to the proliferation of modern data architectures and the sheer volume of data that is being processed across distributed systems, implementing the right data performance guardrails for distributed systems is a challenge for many. From financial services to highly regulated industries, it is mission critical for organizations to react proactively to ‘bad data’ in order to prevent any outage on the customer end. The founders have lived this pain in their past lives as engineers … we couldn’t agree more with their mission to be the data performance standard for event-driven systems for engineering teams.”
Streamdal wants to bring greater visibility to streaming data architectures by Kyle Wiggers originally published on TechCrunch
