At least half of all organizations surveyed by data firms Talend and TDWI say that their main cloud data management challenge is preserving the integrity, availability, and security of their data.
Timothy King filed this report in Data Management Solutions Review:
The study, which is entitled Cloud Data Warehouse Trends for 2019 and took place in September and October of 2018, features feedback from more than 200 data architects and senior IT and analytics professionals. The study’s aim was to uncover key challenges with cloud data warehousing.
The overall theme was that cloud data warehousing enabled users to achieve improved performance and position them to better take advantage of new cloud technologies. However, implementation troubles remain prevalent among those that don’t utilize all the best practices at their disposal. Complexity too remains a major issue, as more than 35 percent of those polled expressed the need for in-memory processing to support both structured and unstructured data. There’s also a growing need for better integration with third-party analytics and BI software.
Data access is the other major pain point for data and analytics leaders. Respondents complained that integrating data across multiple sources and “getting into the data warehouse” were also atop their list of frustrations. These needs can often be mitigated by deploying data integration middleware, but also bring to light the fact that cloud data warehouses must accommodate a variety of data types and serve a range of use cases.
All of the professionals polled by Talend and TDWI had interest in data quality, metadata management, and processing and transforming data before and after loading into a cloud data warehouse. The survey correctly points out that CDW technologies cannot be solely responsible for these tasks, however, and some caution is warranted. The best approach is hybrid in nature, and data and analytics leaders should deploy solutions that complement their existing environments.
