When I was in high school, I lived in a part of Wisconsin where you could make good side-gig money trimming apple and cherry trees. There’s a strategy and art to it. If you trim too much, then yields drop. If you trim too little, then the organism ignores the change, becomes a cluttered mess, and yields drop. The goal is to make the fewest cuts necessary to boost performance.
Driving AI value is a lot like that. If you make too few changes, then the organization ignores it and the status quo prevails. If you make too many changes, then connectivity is lost and productivity plummets (e.g., AIG’s Science Group). What are the fewest changes needed to drastically boost AI value? Here are four examples: a moonshot, a hypothetical, and two practical ones.
A Moonshot Example From the U.S. Department of Defense
An interesting interview with Craig Martell dropped on YouTube on Oct. 26th, 2022. Craig is a former computer science professor, the former head of machine learning at Dropbox, LinkedIn, and Lyft, and the new head of data and AI in the U.S. Department of Defense (DoD). DoD is one of the largest organizations in the world with two million employees operating at 4,800 sites in 160 countries. Craig sees a high demand for AI but insufficient productivity. He wants to do 1,000 times more to keep up with China. How?
You can’t just let an RFP to boost throughput three orders of magnitude. The big prime contractors would win and three to five years later we’d be back in the same situation. Lockheed, Raytheon et al. could no more accomplish it than Kodak could make digital cameras (even though they invented them), Blockbuster could build a video streaming service, or AT&T can design an iPhone. Instead, Craig is making just a few changes to drastically change the economics of AI growth:
- Hire a bunch of data-as-a-product managers
- Manage the most valuable 15% of DoD data in a shared data fabric
- Add machine learning operations (MLOps) best practices services / platform / infrastructure (that’s a mouthful) to make it easy to test, operate, and monitor AI models that use the data
This should:
- Decouple data and AI, allowing them to innovate independently like Snowflake did with data and query processing.
- Lower the cost of entry so small startups can participate without subcontracting through Lockheed, Raytheon, or the usual suspects.
- Lower the skill level to participate so small startups without Stanford and MIT grads can participate.
- Create a competitive, liquid marketplace where the invisible hand of capitalism boosts yields.
It’s an ambitious approach that, if it takes hold, could generate billions of dollars in value quickly.
Central Data Team Auctions
Two days after watching Craig’s interview, Benn Stancil’s newsletter, whose title “Data’s Invisible Hand” is so good that I’m reusing it here, came across my desk. In his hypothetical thought piece, Benn describes using auctions to identify the projects that a central data team works on. Every worker, team, or department in a company is given some credits and they bid on projects each agile development sprint. If an interesting new supply chain project is being bid, for example, then all its downstream users could chip in credits to increase its chance of winning. Data consumers could even bid on specific data team scrum teams to let managers know who has high consumer satisfaction.
Two Everyday Examples
Professor Martell’s data fabric with MLOps is a funded moonshot under development. Benn’s auctions are a hypothetical. Next are two past examples that I was involved in that worked really well. In the early 2000s, the Amazon.com website development team was organized as a collection of mostly autonomous, small, two pizza teams (about seven people each) that constantly experimented, much like Wall Street trading desks where Amazon’s founder came from eight years earlier.
There were dozens of A/B tests running at all times and if one team’s algorithm for a website feature outperformed the incumbent team’s algorithm, then that team took over the website feature and the incumbent looked for new work or disbanded. It was like a game of thrones and drove hyper innovation. (During that time, the email optimization team took on keyword search and nearly won. The search team, ambiguously named Algorithm or A7, was so upset that they moved out of Seattle to Palo Alto.) Amazon made just a few changes that had a massive effect:
- Small, two pizza teams
- Excellent A/B testing services / platform / infrastructure
- Winner-take-all incentives
Sidebar: In the early 1990s, Intel designed their P6 processor in a similar way except that after each competition, the best people from the top two teams were combined into a new all-star team to move on to the next level.
Lastly, a more mundane example of small changes having big effects on AI/ML productivity. A big, old ad agency was moving from on-premise SAS to open source Python in the cloud. Many SAS developers had used SAS and only SAS for decades and weren’t keen to move. We made two changes:
- We separated data and AI/ML by creating a shared data lake with curated datasets and plenty of Hadoop and Spark computational power so that developers don't have to worry about storage or elastic computing.
- Set a quota of a minimum number of new models in production per developer per quarter.
SAS developers had to move to keep up with new hires.
All four of these examples of organizational change, like pruning fruit trees, aim to make minimal changes that maximize yield.