Unfortunately, in most organizations, there is a disconnect between development and production environments causing projects to either fail or to drag on for months beyond promised deadlines. But with the overwhelming plethora of new technologies and blooming skill sets, it doesn't have to be that way.
So why is it that companies are having so much trouble successfully building data products and then deploying them into production? Is it because teams are often isolated and playing by their own rules, which makes operationalization challenging? Is it because development and production environments are perceived as two separate worlds from the moment a project is conceived? Moreover, what can organizations do to bring development and production together?
By implementing the following nine steps, we believe that organizations can find the common ground needed to empower both the data science and IT Teams to work together for the benefit of the data projects as a whole:
- Consistent packaging and release: Supporting the reliable transport of code & data from one environment to the next.
- Continuous retraining of models: Establishing a strategy for efficient re-training, validation, and deployment of models.
- Multivariate optimization: A/B testing? Multi-armed bandit testing? Or optimized multi-armed bandit testing?
- Functional monitoring: Ensuring that business sponsors have the capability to detect early signs of drift.
- Rollback strategy: Making sure rolling back to a previous model version is just a few clicks away.
- IT environment consistency: Making sure Python, R, Spark, Scala, etc., H2o, scikit-learn, MLlib, etc., SQL, JAVA, .NET can work together.
- Failover strategy & robust scripts: Preparing for the worst with failover and validation procedures to maintain stability.
- Auditability & version control: Building in the ability to know what version of each output corresponds to the code used to create it.
- Performance & scalability: Creating an elastic architecture that can handle significant transitions.
All in all, the ultimate success of a data science project comes down to contributions from individual team members working together toward a common goal. That means effective contributions go beyond specialization in an individual skillset; team members must be aware of the bigger picture and embrace project level requirements, from diligently packaging both code and data to creating web-based dashboards for their project’s business owners.
Data science projects can be intimidating; after all, there are a lot of factors to consider. In today’s competitive environment, individual silos of knowledge will hinder team effectiveness. Best practices, model management, communications, and risk management are all areas that need to be mastered when bringing a project to life. In order to do this, team members need to bring adaptability, a collaborative spirit, and flexibility to the table. With these ingredients, data science projects can successfully make the transition from the planning room to actual implementation in a business environment