There’s a rule: evolution is constant, revolution is occasional. While it is popular to discuss the revolutionary aspects of data and analytics (relational data model, real-time, or hadoop anyone?), understanding the evolution of analytics ensures success. For firms who want to remain ahead of the curve, individuals who want to position themselves advantageously, or entrepreneurs who want to change the world, it is critical to understand the dynamics of the current environment.
Strategy, Technology, People
Relentless competition for consumer dollars and increased efficiency, enabled by technological advances, has driven rapid progress in the business world since the global financial crisis. Banks and telcos now use network data for modeling, cost reduction through digital channels, and real-time fraud detection. Meanwhile, retailers implement dynamic pricing and smarter recommendation engines. Finally, machine learning drastically fast-cycles and improves the existing value chain of modeling.
However, big data has not fully integrated with these more traditional domains yet. These businesses are still dominated by data warehouses, human driven analytical modeling, and multi-channel selling and servicing.
Still, advancements in data and analytics are driving some new business models in this domain. For example, new companies like kabbage.com and similar firms use vast amounts of internet data to assess credit risk and extend credit primarily to SMEs, offering these businesses banking services without the banks. Or take GE, which has made “predictive maintenance” a hot phrase. Projected savings from just a few applications based on connected and smart devices could fund GE's next big leap forward.
In a recent article, Michael Porter and James Heppelmann of the Harvard Business Review discuss three waves of business driven by IT. The first was the automation of tasks via computerization in the 1960s and 70s. Second was the internet driven transformation of the 80s and 90s. The final shift is happening now and might best be called “the internet of things” or “industrial internet.” The authors argue that the first two waves have created greater efficiency and impacted the value chain, but have not changed the product itself. Alternatively, the third wave is shaping the product directly. A simple example is to compare the business models of Amazon or Netflix to that of Nest.
The data and analytics world has followed a similar evolution. Database marketing represents the first wave in which customer-level data inspired new marketing and analytics strategies focused on increasing return on marketing dollars. Here, mathematical and statistical modeling found mainstream applications predicting customer responses. This evolution paralleled operational applications such as stock management for retailers and electronic data interchange for more fluid business processes. This wave created a new ecosystem comprised of computer hardware giants, marketing agencies, and mail fulfillers, among others. It also drove huge demand for mainframe programmers, statistical modelers, and IT sales skills.
The second wave saw enterprise data warehouses and data-marts. This new level of sophistication involved route optimization for delivery trucks and supply chain integration between manufacturers and retailers. Customer churn prediction drove real-time offering at call centers and click-stream analysis provided better user interface design. These advancements were enabled by relational database technology, T1 lines, and sophisticated system integration. Predictive analytics replaced descriptive and was in turn replaced by prescriptive. Buzzword bingo was clearly in full-speed. It was CMO vs CIO until everyone agreed it was both. Data analysts co-habited with their peers in marketing, operations, and channels. System integrators were in high demand to tie the various systems and processes together. During the war for end to end service provision, smaller businesses, from data storage to human resources, were swallowed by larger companies. There were even new, hot careers with exotic names like “java programmer” or “UX designer.”
The Third Wave in Data Today
The third wave is upon us, but remains unclear. Companies like Google and Amazon have developed new technologies that enable a much larger application base, resulting in a new technology stack made up of many moving parts. Many developments are open-source and collaborative. However, this new technology requires significant skill in platforms and tools like Phyton or R, and talent is the immediate bottleneck for mainstream big data usage. “Data scientist” has been named the hottest job title, but there is little agreement on the ideal role for this position.
In short, big data is revolutionizing storage, processing, and typical IT limitations, thereby making possible a whole slew of applications. For now, concentrate on use cases, find people qualified for new technologies, and profit.