The Data Race to Revenue and How to Keep Up

Can’t keep up with your data processing needs?  The race seemingly never ends.

This seems to be a big concern – so we reached out to an industry expert – DataStax Vice President of Strategy Bryan Kirschner – to learn his point of view on what technology leaders are doing to help their developers keep up… and deliver more revenue-generating applications.

Based on a recent survey, he found that companies are figuring out how to tie real-time data to revenue generation.  But how?  Here’s what we learned from Bryan.

Q1: Thank you for taking the time to answer our questions. What exactly is real-time data and how is it used?
BK: Real-time data powers in-the-moment use cases, like recommendations and personalization, or always up-to-date inventory and logistics. Fine-grained details aside, there are two big things that distinguish what we mean by real-time. The first is about how fast the data needs to move, which is really up to the customer, the business process, or the systems that are interacting with each other. The speed needs to be seamless, whether that’s seconds, sub-seconds, or milliseconds. The second is that “real-time” is fundamentally about action. It’s about a system being able to trigger something without needing a human to be in the loop. That’s what changes the nature of interactions, and by extension, the business as organizations pursues a real-time journey.

Q2: According to the Data Race Report, implementing real-time data is key for success. Could you share with us some of the findings the report unveiled regarding this connection?
BK: The big, flashing neon sign about why real-time data is key to success is the association between real-time and revenue. 71% of organizations can tie real-time data to revenue, and among organizations with a strategic focus on real-time, 42% have seen a transformative effect on revenue (18% more than those that lack this focus). At the end of the day, this should feel intuitive when thinking about the consumer experience. A retailer who has gone digital and is now activating data in real-time to do on-the-fly segmentation, smart recommendations, and algorithmic pricing will ultimately have more success. It is all about transforming the operating model from human-driven and glitchy towards in-the-moment, always up-to-date, and system-driven. But it may go even further in the B2B and industrial space. I think in two years we’ll find it bizarre that any users of vehicles or equipment waited around to be surprised when something broke. Predictive maintenance, data-driven, real-time efficiency, and digital twins will transform what competitiveness means in those areas as well.

71% of organizations can tie real-time data to revenue, and among organizations with a strategic focus on real-time, 42% have seen a transformative effect on revenue.

Q3: Why does real-time data enhance developer productivity? And can companies of all sizes and scope benefit from it?
BK: I think the relationship we’re seeing between increased developer productivity and real-time data (our research shows that 66% of real-time-data-focused organizations agree that developer productivity has improved) is a function of two forces converging. First, organizations have been working on increasing developer productivity with DevOps practices and cloud tools for some time. In a way, this movement was already in motion. Second, leading organizations have really figured out that agile practices and intimacy with customer experiences and business processes are key to unlocking the power of real-time data. They are leaning into establishing business unit accountability for results and having developers work in cross functional teams that include what might, in the old days of IT, have been siloed business, development, and data science teams. Business teams are now able to ask “how might we get a revenue uplift” or “reduce cost and risk” using modern technology and the data we’re generating while doing business. A lot of real-time data isn’t rocket science: it’s taking advantage of today’s technology to rapidly count and correlate. The result is a significant improvement across many use cases (think: recommendation engines or credit card fraud detection).

Q4: How is the current developer shortage affecting production? And in your opinion, what should companies be doing in order to maintain developer talent and find new recruits?
BK: Skills have been an ongoing challenge in the tech space. We need to look at an important lesson from the last few years about how enterprises and IT responded to the pandemic. Essentially, everyone was under-utilizing technology and talent pre-pandemic. With the increased focus on new technologies, enterprise IT got more done, faster, and under unprecedented conditions than anyone would have expected. 86% of developers at organizations with a strategic focus on real-time data say that “technology is more exciting than ever” — 24 percentage points more than organizations with no real-time deployments. Give developers the technology they want, don’t get in their —or your own — way.

Leading organizations have really figured out that agile practices and intimacy with customer experiences and business processes are key to unlocking the power of real-time data.

Q5: What should be in every developer’s toolkit when it comes to real-time data? What open-source technologies should they be using for optimal data management?
BK: There are three key elements required to deliver a real-time data experience. First, it’s critical to have a data store that’s optimized for customer context and instant access. It’s how you take advantage of your real-time data at rest. NoSQL databases are optimized for modern data applications that require large data volume, low latency, and flexible data models. Apache Cassandra™ is an obvious choice, with its high throughput and ability to support applications that are globally distributed and always-on.

Secondly, developers need streaming technology. Organizational behaviors and actions need to be visible and available to all applications across an organization; a best-of-breed streaming system should not only pass events along from one service to another but store them in a way that keeps their value for future usage. The system needs to scale to manage tons of data and millions of messages per second—the kind of performance that real-time apps demand. Apache Pulsar™ is an advanced, open-source streaming and messaging technology that’s ideal for handling real-time data. It was built for the high throughput and scalability that many data-driven applications require.

Finally, it’s critical to empower developers to make the most of real-time data—quickly and easily.  A good example is Azul Platform Prime, which can improve customer experience and increase performance for real-time data processing architectures – and it’s based on Open JDK that developers are used to working with.  Another key thing is the API layer that lets them build applications with freedom of choice and without operational distractions. Stargate, for example is an open-source data API layer that sits between applications and the database, offering a variety of endpoints for developers to build with.

Q6: This is the second year that the Data Race Report was conducted. What changes have you seen year-over-year? Did any of the changes surprise you?
BK: The biggest surprise this year was that many organizations have really grasped the optimal developer experience. In this year’s data, we’re seeing that optimism bears fruit in organizations that have focused on the developer experience and incorporated real-time data. Our research found that 35% of all developers polled described working with real-time data as “very important” – making it the second most important job feature after flexible and remote work opportunities.

In this year’s data, we’re seeing that optimism bears fruit in organizations that have focused on the developer experience and incorporated real-time data. Our research found that 35% of all developers polled described working with real-time data as “very important” – making it the second most important job feature after flexible and remote work opportunities.

Q7: Now that we’ve heard all about the positives, are there any downsides to real-time data?
BK: In general, speed and smarts win — real-time is always better. However, as organizations go down the real-time path to enable systems to act and effectively make decisions that humans might once have made, they need to be mindful that they are re-wiring their business. Businesses need to pay attention to observability, auditability, and properly equipping humans to hold the reins. One of the powerful and amazing things about activating data in real-time is that it is not magical artificial intelligence. A lot of it is entirely doable math and statistics made possible by modern technology. But those systems are not artificial intelligence. Humans need to be able to look end-to-end at what they are driving toward and why.

Q8: And finally, what’s next? In your opinion, where do you see real-time data going in the next five years?
BK: In five years, I don’t believe any organization using equipment or vehicles is going to accept being surprised by something breaking unexpectedly. Predictive maintenance will become the norm. No consumer will be content with a lack of (or off-base) personalized recommendations. Real-time data will become pervasive, because in many cases, it is delightful, and in many others. it reduces cost and risk.

At Azul, we’ve helped many companies improve speed to get to more smarts faster – including ad-tech leader Taboola, who delivers content to over 1.4B people a month.  We’ve designed Azul Platform Prime to optimize Apache Cassandra – reducing security issues, improving customer experience, and increasing performance for real-time data processing architectures. 

Need more help? Get more details from DataStax’s The State of the Data Race. Together with our support, Azul offers an optimized JVM to improve performance for your business-critical applications – download Azul Platform Prime for free.