Insights from the deep learning industry.
In this blog post we’ll look at which parts a machine learning platform consists of and compare building your own infrastructure from scratch to buying a ready-made service that does everything for you.
Running a local notebook is great for early data exploration and model tinkering, there’s no doubt about it. But eventually you’ll outgrow it and want to scale up and train the model in the cloud with easy parallel executions, full version control and robust deployment. (Letting you reproduce your experiments and share them with team members at any time.)
SwiftStack and Valohai, in joint partnership, announce the world’s first peta-scale ML solution that covers everything from computation to data management in a multi-cloud environment. The solution provides a global namespace removing silos and enabling universal access to all your data in all your machine learning use-cases. It has built-in support for Azure, Google Cloud, AWS and SwiftStack.
We may live in the era of “Big Data,” and yet the access to it is somewhat restricted; especially, when we talk about high-quality data. This blogpost will address the question of acquiring data for your Machine Learning projects from the perspective of EU and US copyright laws.
In part 1 we introduced Q-learning as a concept with a pen and paper example. In part 2 we implemented the example in code and demonstrated how to execute it in the cloud. In this third part, we will move our Q-learning approach from a Q-table to a deep neural net.
When we founded Valohai two years ago, we were lucky to make friends with team leads for Uber’s Michelangelo machine learning platform. Michelangelo has been an inspiration in building Valohai for the other 99.999...% of companies that aren’t Uber but still need to speed up their machine learning through automation.
In part 1, we looked at the theory behind Q-learning using a very simple dungeon game with two strategies: the accountant and the gambler. This second part takes these examples, turns them into Python code and trains them in the cloud, using the Valohai deep learning management platform.
By now you’ve surely heard about Kubeflow, the machine learning platform based out of Google. Kubeflow basically connects TensorFlow’s ML model building with Kubernetes’ scalable infrastructure (thus the name Kube and Flow) so that you can concentrate on building your predictive model logic, without having to worry about the underlying infrastructure. At least in theory.
There’s only one way to grow your deep learning team effectively: by adding new people to it! (We were just as shocked as you are by this revelation!) Filling your team can be done a couple ways: by recruitment, hiring freelancers, or outsourcing to consulting agencies. Finding talented people is hard enough already, so make sure your newly hired team members hit the ground running and don’t slow down the rest of the team.