On Saturday, September 23rd, we took a trip to Toronto to join 1,500 data enthusiasts as they took their first journey into the world of data science. It was the fifth annual edition of the National Learn to Code Day, which brought together eager minds from more than 29 cities across Canada.
We were honored that the organizers at Ladies Learning Code chose Dataiku DSS as the platform on which participants would make their first steps in data science. It was the first time that so many users worked simultaneously in Dataiku across such a large territory, hence we were very excited but also a bit nervous. Our Lead Architect Joel did an amazing job, from setting up the instance to making sure everything went smoothly on the day of.
Training 1,000 ML Algorithms at the Same Time?!
You may wonder: what setup did we implement to enable our platform to train 1,000 machine learning algorithms at the same time? Here’s how the story goes: Joel wrote a new module for a bot, which implements Amazon Web Services (AWS) public APIs to build any analytics platform topology. This bot has the ability to create EC2 instances and define the whole network setup. Since AWS provides the ability to host multi region platform and orchestrate it via APIs, it was only a matter of writing these two additional modules to create the platform.
The day was off to a good start, as thousands of datasets were created and corresponding jobs were smoothly executed at the same time. The first test happened when an instructor asked 250 participants to train a model. The platform responded very well as it only slightly slowed down, which encouraged both teachers and students to do much more. Joel saw very complex and unexpected models being built simultaneously on the platform, and he couldn’t prevent a couple of jobs from failing. The good news is he was able to scale up the platform up by dynamically building new instances and adapting to the phenomenal development of the participants’ skills.
National Learn to Code Day aims to provide enthusiastic learners with a better understanding of the implications behind artificial intelligence (AI) and machine learning (ML). They strive to answer the question: what do AI and ML mean in practice? This year's participants heard a presentation of the main concepts and tools used in data science before diving into business use cases. And, because action speaks louder than words, the afternoon was spent trying to predict employee attrition using a dataset from IBM and working on a larger-scale projects (hello, new Kaggle competitors!).
Created in 2011, the nonprofit organization Ladies Learning Code launched and managed the event in every city thanks to its local chapters. At the forefront, they strive to increase digital literacy while equipping women with important 21st century digital skills. A particular emphasis is put on collaboration, as most of the workshops feature group projects to gain a mutual understanding of computer programming. The fruitful collaboration with Dataiku is no wonder!