Failure rate of Data Science projects:
Failure rate of Data Science projects is astonishing. Big Data technology is still in infancy, processes are not matured and more importantly the kind of talent(mathematicians/statisticians) requires for Big Data lacks IT experience or interaction with IT and business world. One needs to do the hands dirty and sometimes theoretical experience is hindrance rather than an advantage. In the hindsight, e-commerce in late 90s and early 2000 also attracted people who lacks IT experience. E-commerce was hyped, but to some extent the prediction was right, only in the long-term. In early 2000 neither technology nor ecosystem was ready for e-commerce. Jeff Bezos was among few handful of entrepreneurs who understood the trajectory. Nowadays, almost every company is embracing Data Science. CIO and senior executives lacks experience in creating structure, formulating vision and they believe that just throw the tools--many Big Data tools are free--- and hire rookies, and they will build something awesome.
There are many reasons but fundamentally over exuberance, lack of experience and missing SMART(Specific, Measurable, Achievable, Realistic, Time-bound) goals are main. Developing Big Data projects mimic the process of Product Development and it is critical to structure the project on agile methodology.
Failure rate of Data Science projects is astonishing. Big Data technology is still in infancy, processes are not matured and more importantly the kind of talent(mathematicians/statisticians) requires for Big Data lacks IT experience or interaction with IT and business world. One needs to do the hands dirty and sometimes theoretical experience is hindrance rather than an advantage. In the hindsight, e-commerce in late 90s and early 2000 also attracted people who lacks IT experience. E-commerce was hyped, but to some extent the prediction was right, only in the long-term. In early 2000 neither technology nor ecosystem was ready for e-commerce. Jeff Bezos was among few handful of entrepreneurs who understood the trajectory. Nowadays, almost every company is embracing Data Science. CIO and senior executives lacks experience in creating structure, formulating vision and they believe that just throw the tools--many Big Data tools are free--- and hire rookies, and they will build something awesome.
There are many reasons but fundamentally over exuberance, lack of experience and missing SMART(Specific, Measurable, Achievable, Realistic, Time-bound) goals are main. Developing Big Data projects mimic the process of Product Development and it is critical to structure the project on agile methodology.