Commerce 2.0 Data Collection

The Big Data challenge is twofold, one is the collection and storage of the data and the second is utilizing this data.  Both of these aspects are at the same time dependent and promoting each other.  They are dependent because the data collected should from one perspective be driven by the use of that data.  They are promoting each other because the needs and capabilities of one aspect feed the other aspect.  The challenge here however is a classic ‘chicken or the egg’ question; does the utilization drive the collection, or the collection drive the utilization?  I think that the challenge is to collect the data based on a future potential and not on a current need. It is important to build the collection capabilities based on the future potential because you want to ensure that you have historical information for queries and analysis when you identify the need.  

At one point in time this concept was almost overwhelming from a cost perspective and an analysis perspective.  The cost of storage was expensive and the tools and computing horsepower were in their infancy in the ability to support search and comparison requirements of large amounts of data crunching.  These limitations drove frustration from the business and technology staff at the same time.  From the technology perspective there was a great deal of difficulty and effort required to justify the cost of the of equipment to support the objective.  From the business perspective there was a great deal of frustration because it seemed that any time one analysis determined the need for additional data, the technology either did not have the data or required a great deal of effort to create the analysis.  A third technical frustration was the time involved in collecting and analyzing the data.

Current capabilities have overcome the technology hurdles; storage and computing power is cheap.  These capabilities are also changing and improving on a continuous basis.  This has allowed the potential of Big Data concepts to at least start to be realized.  With this realization comes a new realization, though, of the real, or additional, potential achievable.  This realization is driving new desires and needs in technology and capabilities.  We have seen the changes realized in the consumer market and shopping capabilities encouraged and driven by improvements in mobile technology.  There are similar capabilities encouraged and driving in the research and analysis of data by improvements in Big Data technology.  My advice is to not cut yourself short, or hold yourself back, by limiting the amount of data that you collect.  My advice is to collect everything from every aspect of the consumer market so that your analysis can go where it needs as your analysis and questions expand.

And now for the audience participation portion of the show…

ECommerce will have wide ranging impacts on both the retail and manufacturing sectors.  How can you focus these abilities to improve the consumers’ experience?  Improving the consumers experience will require a re-evaluation of the sales channels, the manufacturing channels and practices and the supply chain channels and practices from the raw materials to the consumers’ homes.  In order to ensure and maintain success in this new reality you must harness the tools and capabilities in many new areas.  How can you support these continuously changing requirements?