A well-executed big data strategy can streamline operational costs, reduce time to market and enable new products. But enterprises face a variety of big data challenges in moving initiatives from board room discussions to practices that work.

Enterprises can waste a lot of money storing big data if they don’t have a strategy for how they want to use it. Organizations need to understand that big data analytics starts at the ingestion of data, said George Kobakhidze, head of enterprise solutions at ZL Tech, a business analytics firm. Curating enterprise repositories also requires consistent retention policies to cycle out old information, especially now because pre-COVID data is rarely accurate in today’s market. Thus, teams should plan out the types, schemas and uses of data. This is easier said than done, said Travis Rehl, vice president of product at CloudCheckr, a cloud management platform.

“Oftentimes you start from one data model and expand out but quickly realize the model doesn’t fit your new data points and you suddenly have technical debt you need to resolve,” he said.

A generic data lake with the appropriate data structure can make it easier to reuse data efficiently and cost effectively. For example, Parquet files often provide a better performance-to-cost ratio than CSV dumps within a data lake.

Continue Reading 10 big data challenges and how to address them in TechTarget