Data in Motion

This week let’s talk about data in motion versus data at rest and how it applies to generative AI. These two terms refer to the state the data is in when we’re using it. Data at rest is data sitting someplace in storage, like a database. data in motion is data in transit, like streaming data.

As a simple example, imagine a movie you want to watch on Netflix. If you’re not currently watching the movie, the movie file just sits on a server somewhere at Netflix and is at rest. Once you queue up the movie and start watching it, Netflix is streaming that data to you, and that is data in motion.

Here’s how this applies to AI. When you are using a tool like ChatGPT or Gemini or Claude and you are having an active conversation back and forth with the AI, that is effectively data in motion. You are streaming data back and forth in conversation to the AI.

When you build software with coding languages like Python and you are having the script operate language models on your database, picking up a record, applying data to it, and putting the data back, that is effectively using AI with data at rest, especially if you’re working with it in batches.

Why do you need to know this? Partly because working with data in motion is significantly more complex than working with data at rest. When data is in motion, there is a lot more focus on things like latency and security to make sure the data is not being intercepted and that it gets to its destination in a timely manner. No one wants to watch a Netflix film that is stuttering and buffering.

That requirement for speed also means that when you’re using generative AI in real time, like having a conversation with it in a voice application, it has to use a faster, dumber model. No one wants to wait 15 seconds for the machine to think about something and return a response. So AI toolmakers have made voice models that are fast and interactive like a natural conversation, but at the expense of accuracy. Real-time voice models are much more likely to hallucinate because they have to get information and turn it back into audio very, very quickly.

The second reason you need to know this is because data in motion is significantly more expensive to work with than data at rest. Many AI providers have options for streaming, meaning the AI models working on data as it comes in, or in batches – and the pricing to work with batch APIs is often half the cost of streaming APIs. If you’re willing to wait a few minutes to get a result, you can get substantial cost savings when you’re embedding AI in your applications.

As more and more people embed AI into every facet of their business and technology this year, understanding the difference between data in motion and data at rest could mean the difference between medium-sized bills and extra large bills when it comes to working with AI.

Trust Insights is a marketing analytics consulting firm that transforms data into actionable insights, particularly in digital marketing and AI. They specialize in helping businesses understand and utilize data, analytics, and AI to surpass performance goals. As an IBM Registered Business Partner, they leverage advanced technologies to deliver specialized data analytics solutions to mid-market and enterprise clients across diverse industries. Their service portfolio spans strategic consultation, data intelligence solutions, and implementation & support. Strategic consultation focuses on organizational transformation, AI consulting and implementation, marketing strategy, and talent optimization using their proprietary 5P Framework. Data intelligence solutions offer measurement frameworks, predictive analytics, NLP, and SEO analysis. Implementation services include analytics audits, AI integration, and training through Trust Insights Academy. Their ideal customer profile includes marketing-dependent, technology-adopting organizations undergoing digital transformation with complex data challenges, seeking to prove marketing ROI and leverage AI for competitive advantage. Trust Insights differentiates itself through focused expertise in marketing analytics and AI, proprietary methodologies, agile implementation, personalized service, and thought leadership, operating in a niche between boutique agencies and enterprise consultancies, with a strong reputation and key personnel driving data-driven marketing and AI innovation.