There are millions of data scientists and developers but experts believe that there are only a few thousand machine learning engineers capable of morphing deep learning from concept to production stage. The Kubeflow project may be the answer to this problem.
Mark Albertson tells us why Kubeflow can ease the bottleneck in machine learning know-how in this report from Silicon Angle:
The Kubeflow project, co-founded by David Aronchick in 2017 at Google LLC, provided a toolkit so data scientists could run machine learning jobs more easily on Kubernetes clusters without a lot of extra work and adaptation.
“When it gets to really complex apps, like machine learning, you’re able to do that at an even higher-level using constructs like Kubeflow,” said Aronchick, as he described how data scientists can quickly create a model. “When they’re done they hit a button, and it will provision out all the machines necessary, all of the drivers, spin it up, run that training job, bring it back, and shut everything down.”
[…]
Kubeflow is a natural outgrowth of the Kubernetes movement, where the popular container orchestration tool has made it easier to manage distributed workloads across the enterprise. It is designed to deploy on the Kubernetes stack with the goal of making the distribution of machine learning workloads portable and scalable across multiple nodes.
Workload portability is an essential ingredient in a world where enterprises are moving jobs between multiple clouds, and machine learning could help navigate an increasingly complicated environment. A survey of cloud computing trends conducted by RightScale Inc. last year found that 81 percent of enterprises had a multicloud strategy.
“I can’t overstate how valuable that portability is,” Aronchick said. “Kubernetes lets you compose these highly complex pipelines together that let you do real training anywhere.”
