The Stars are Aligning for Big Data, Cloud and Integration
Posted by Margaret Dawson on July 14, 2011
It’s Moore’s Law applied to data. The stats on growing data volumes show exponential growth that is unprecedented (a word I hate to use casually, but it really does apply in the case of Big Data).
IDC’s Digital Universe study predicts the amount of data being stored will more than double every two years, and could grow by 50 times by the year 2020. The survey also found that the amount of data created and replicated is expected to top 1.8 zettabytes, or 1.8 billion TBs, in 2011, up from just over 1 zettabyte in 2010. And it’s not just the analysts who are pontificating on the subject. At every cloud conference and cloud camp I’ve been to, big data has been a huge topic.
We have also seen this trend first-hand, with a massive increase in not only transactional volume but message size from our community of some 20,000 B2B connections.
The intersection of cloud and big data makes sense, particularly from a scalability perspective. The ability to scale on-demand solves the problem of physically adding more servers in the data center to accommodate increases in storage and processing volumes. This is especially important for industries like e-commerce that experience spikes in transactions or cyclical increases. The alternative, in the physical world, is costly and time-consuming data center build outs in terms of hardware, software, and labor, not to mention bandwidth constraints. In the long-run, cloud-based data storage is also more cost-effective. I know this since our primarily data center uses a major brand of SANs for storage – and it ain’t cheap 
.
Which leads me to my next point, integration. It’s great that cloud vendors are giving customers the agility to accommodate increases in data volume on the fly. But, what about taking into consideration how to access that data or share that data when needed? This is where integration comes in.
Whether it’s leveraging cloud service brokerage as Gartner calls it or just thinking about integration before you move the data to the cloud. Too often, companies start using data storage or applications in the cloud without taking integration or the exchange of that data across other systems or companies into account.
I think we will see some real innovation in this area, and it offers a huge opportunity. Just last month, Informatica announced integration with Hadoop. This is a good move for Informatica and builds nicely on its data management and integration story within the enterprise. We hope and expect to see more announcements like this from other vendors in the near future.
These are exciting times from our vantage point in the front row seat, as the interdependencies between big data, cloud and integration take center stage in the enterprise.
No Comments »
No comments yet.