Articles everywhere keep hailing the end of Big Data and criticizing the hype. They are right, to some extent. The revolutionary world where data answers all our questions before we even ask them and where correlation has overcome rational causality making everything statistically predictable has gone out of fashion.That does not mean that big data is dead, and far from it.
Big Data is Dead, Long Live Big Data in Production!
Big data today is no longer a concept; it’s a real thing. Companies everywhere have accepted that they can gain significantly in their core business by investing in technologies and skills to extract something new and valuable from their data . In fact, it has become so widely generalized that companies have come to terms with the fact that if they are not doing it, their competitor surely is. If they don’t step up and catch up, he will be the one taking home the biggest piece of the cake.
This brings big data back to what it was always meant to be: a tool for businesses. An assembly line that extracts and manipulates the data to produce actionable information from it.
Articles, talks at conferences, and companies specialized in building technologies and sharing best practices with other companies have gotten their point across: businesses are now transitioning to big data. They are adopting the ever growing number of tools and technologies available to build their advanced analytics projects efficiently. And more and more of them have constituted the teams to create these projects. Hence the Age of the Superstar Data Scientist.
Why Companies Fail to Deploy Data Science Projects Into Production
However, it seems that big data is failing at that as well. Plenty of surveys keep telling us year after year, that if companies are convinced that their data is valuable, as little as 4% of those companies actually extract full value from their information.
Companies everywhere with sophisticated data teams and business analyst involving operational teams still don’t get the value they were expecting. Their efforts just don’t seem to be enough. They build prototypes for projects but have trouble seeing them go into production and become a part of existing processes.
This is something we hear about over and over, and we don’t have all the elements to give an answer as to why that is. We are beginning to see tendencies and working on fixing little bits of the problem, but we are still attempting to capture the bigger picture.
Take the Global Survey on Data Science in Production
So we are asking you what you think. You? Yes you. Everybody. Any company that has decided to put efforts in data has to face bringing these projects from the design and development phase to the production phase at some point. So tell us how you do it. And we will tell you what we learned from you. All of you. On how people and organizations are putting data science into production today.
PS: We were lucky enough to have this article published by our friends at KDnuggets, you can find proof over here.