Media Query Source: Part 56 - Reworked (US digital magazine); Can real-time data processing be a viable alternative to AI in workplaces where speed and adaptability matter most?

• Reworked (US digital magazine)
• Can real-time data processing be a viable AI alternative?
• Availability of real-time data in itself isn't an AI alternative
• What insights are needed, and when? What operations will fulfill these needs?

The query responses I provided to Reworked on July 8, 2025:

Reworked: In the era of instant information and accelerated decision-making, real-time data processing is emerging as a transformative force across industries. No longer confined to specialized sectors like finance or defense, this capability is now reshaping how businesses operate—powering agile workflows, enhancing responsiveness, and reducing latency between data and action. As organizations strive for greater adaptability in unpredictable markets, real-time systems offer a practical, scalable alternative—or strategic complement—to AI. Unlike traditional AI models that often rely on historical data and batch processing, real-time data infrastructure delivers up-to-the-second insights, fueling more dynamic and context-aware decisions.

This shift prompts a fundamental question: Could real-time data be the "smarter" solution in environments where speed, relevance, and adaptability outweigh predictive sophistication? In this exploration, we uncover how real-time processing is influencing digital strategies, reshaping enterprise architectures, and challenging the AI-dominant narrative in certain business contexts.

Gfesser: My short answer is yes - real-time data processing can be a viable alternative to AI. However, it's important to understand that availability of real-time data in itself isn't going to provide insights: regardless of data timeliness, the data needs to be analyzed. Data analysis, however, doesn't necessarily need to involve AI models: depending on the goal, traditional analysis methods can be used instead, or traditional analysis methods can be used alongside AI models.

The key use case in which real-time data analysis is going to provide benefit is when decision making needs to be especially timely. For example, if a potential customer is interacting with a business's website, and the business would like the customer to take action in some way during their visit, the data associated with this visit likely has a short shelf life, and as such, this data needs to be acted upon quickly: waiting to execute batch processes on this data long after the customer's departure likely isn't going to take advantage of the value this data had during their visit.

What are the biggest technical or cultural challenges companies face when shifting to real-time processing environments?

The biggest culture challenge that I've seen across the many businesses I've consulted is coming to terms with what the business means when it uses the term "real-time", because not everyone means the same thing. For example, the first time I implemented a real-time solution early in my career for Tyson Foods, "real-time" meant "hard" real-time, meaning that decisions need to be made with respect to reacting to the availability of physical objects in space. In this case, it was programming machinery set up alongside factory conveyor belts to react to the presence of product passing by. This type of real-time processing is critical, because there is no second opportunity once the physical product is no longer available. At the opposite end of "hard" real-time is "soft" real-time, which means there's still a window of time within which data needs to be processed, but the tolerance is greater because the physical world isn't a factor.

Most business stakeholders likely mean "soft" real-time. However, the window of time within which data needs to be processed can vary widely when this term is initially used. Some businesses I've consulted, for example, just mean that they want data to be processed during the day and not just during a once nightly batch, because this is all that they've seen from their industry experience. However, if a business were to process throughout each day, say, four times evenly distributed throughout each day, is this real-time? Unless there is criticality to the chosen time windows, the most likely answer is no, this isn't real-time. There's also a third category that I've needed to handle, for example while building a global data platform for Deloitte. I used the term "on demand" with stakeholders to differentiate it from batch in the sense that, similar to real-time processing, data isn't processed at scheduled times, but the data being processed also doesn't have a short shelf life. Alternatively, "on demand" is synonymous with "unscheduled batch", but I came up with a different term to help ensure that stakeholders understood we prioritized their need to process historical data anytime they wanted to do so.

Can real-time data infrastructures serve as an alternative to AI in industries where historical training data is limited or unreliable?

The availability of real-time data infrastructures in itself arguably shouldn't be viewed as an alternative to AI. What matters is the insights needed from the data, what operations need to be performed to gather these insights, and the timeliness of when these insights need to be made available.