Data integration rises to the top of the cloud service stack

David Linthicum, sage of sages of the IT world, has some advice for those hoping their vendors will come to the rescue to untangle their ever-growing hairballs of big data and services: don't hold your breath. "Those who think they can sit on the sidelines and wait for their data integration technology provider to create the solutions they require will be very disappointed," he cautions.

Instead of waiting on new releases of products, look to a rising breed of cloud-based services, David continues. As he explains in a new report (Sponsor: SnapLogic.), "everything you currently understand about data integration is changing. Assume that your traditional technology provider will soon be dead. Assume that the strategic use of technology and data will begin to provide even more value to most enterprises, and hopefully create a sense of urgency that things need to quickly change. Data integration is getting a reboot, and new players are likely to replace older players."

For years, data integration has been accomplished with tools and techniques such as extract, load and transform (ETL), data warehousing, and even enterprise service buses (ESBs). These traditional approaches cannot handle the waves of data and demand for services hitting enterprises, which David calls "a data-related technology crisis." Several converging forces are making data integration more complex -- the exploding use of cloud-based resources, and database technology built for specific purposes, the proliferation of devices that now produce and consume information at gigabytes per second, and the changing nature of IT itself.

Cloud-based services are increasingly seen as new vehicles for achieving data integration at the massive scales that will define it, David writes.

"The rise of services, and, now, microservices, changes the game, in terms of how we leverage and manage data. These services are the new dial tone for cloud computing, and are appearing within the enterprise as well. Data services are services married with data, and they will be the most common mechanism for accessing data as we move forward. Therefore, data integration technology must layer in service directories, service governance, and service identity-based security."

The intent of these services should be to enable access to all manner of big data, be it structured or unstructured, David continues. The good news is that enterprises can keep their existing infrastructures and legacy systems in place, and just build or deploy services around them. "Data services are able to define structure within the services themselves, and thus read and write unstructured data without requiring that a structure exist within the source or target databases."