AI budgets often focus on the cost of processors and the power they use. But when a model answers prompts, it can also generate logs, cached context, outputs, backups, and safety records that remain after the response is delivered. As inference becomes a regular, ongoing workload, the storage needed for those records can become a significant part of infrastructure planning.
Inference leaves data behind
Training a model is typically organized as a finite run: it begins, uses computing resources, and ends. Inference happens each time a model responds, so its data footprint can build continuously. That difference makes it harder to estimate storage needs from a single project or launch.
AI-generated video illustrates the gap between what a user receives and what a system may need to retain behind the scenes. WD says an 8-sec ond clip returned as a 3.97MB 720p file left more than 500MB of associated back-end data, including logs, cached context, immediate renders, backups, and safety records. The company describes that as a 130x increase over the output file size. Longer, higher-resolution video at enterprise scale can push the back-end footprint into the hundreds of gigabytes per clip.
The example shows why the visible output is not a reliable guide to the storage required to produce and manage it. Each generation can leave supporting data that may be kept for operational or future use. When similar work happens repeatedly, those records accumulate even after the individual task is finished.
Demand is shifting toward ongoing workloads
The source cites McKinsey projections that AI inference demand will grow at a 35% CAGR through 2030, compared with 22% for training, and account for more than 40% of total data center demand by the end of the decade. It also points to a WD-sponsored IDC survey of 763 IT and business decision makers: nearly 95% reported an increase in data volume due to generative AI over the past 12 months, and roughly 61% reported growth of 25% or more.
These figures point to a planning challenge: compute activity may rise and fall with training cycles, launches, or hardware refreshes, while retained data does not necessarily shrink when activity slows. Model logs, synthetic outputs, and inference results can keep collecting. That can make storage costs less visible at first, then harder to ignore as retention grows.
Storage also affects how efficiently computing resources are used. A senior product manager of a hyperscale cloud service provider quoted in the report said that more storage can help keep GPUs busy, because GPU idle time while waiting for data becomes expensive. If data is slow to arrive, a system can have ample compute capacity yet still fail to use it effectively.
Archives can become working data
Some retained information that once sat in an archive may now be useful as input to later AI workloads. When organizations retrieve and reuse older data for inference or retrieval-augmented generation (RAG), storage has to support more than low-cost retention: it also needs to make selected information available when the workload calls for it.
In the same WD-IDC survey, 96% of respondents anticipated needing faster archive retrieval to support inference or RAG. That expectation highlights a tension in storage design. Enterprises need capacity for large volumes of retained data, but some of that data also needs to be accessible quickly enough to support active work.
Different storage technologies can serve different needs across the data life cycle. The article describes HDDs as a cost-effective, high-capacity foundation for large volumes that need to be retained and revisited, with a significant cost-per-terabyte advantage. The broader planning question is how to balance access speed and capacity as data moves between stages of use.
Bring storage into the AI budget
For finance and technology leaders, an AI infrastructure estimate centered on compute may miss a cost that compounds as work continues. A retailer producing product videos or an insurer processing claims documents through a RAG pipeline may have a clear per-generation or per-query compute cost. Storage is different: every video or processed claim can add back-end data that remains after the task.
The source warns that organizations treating storage as a rounding error could find it becomes a significant fraction of a program’s total cost in 12 to 18 months, around the time leaders consider whether to expand. The cost may appear in operating expenses or retrieval delays instead of alongside accelerators in a capital spending plan.
Planning for economically scalable capacity means accounting for the full data life cycle, including performance, capacity, power, reliability, and total cost of ownership. It also means deciding what to retain and for how long, considering whether archive data may later serve inference or RAG, and evaluating storage decisions across fleets. Those choices can help organizations keep retained data useful while managing the cost of capacity as AI workloads grow.