Valohai blog

Insights from the deep learning industry.

All Posts

Random hyperparameter optimization

Valohai now supports random search for hyperparameter optimization (which we call the Tasks feature), which has been proven in the aptly named paper Random search for hyper-parameter optimization to be an efficient way to find “neighborhoods” of likely-to-be-optimal hyperparameter values, which can then be iterated further to find the really good values.

Random hyperparameter search for machine learning experiments

This is a valuable tool to add to the previously existing linear, logarithmic and multiple-value hyperparameter optimizers. Valohai uses a seeded Mersenne Twister random number generator to generate values in a given range and supports both an uniform distribution as well as a truncated normal distribution of values.

You can try this feature out today; simply create a Task for your parameter-enabled step and choose the Random option.

Aarni Koskela
Aarni Koskela
CTO and Founder of Valohai

Related Posts

Automatic Version Control Meets Jupyter Notebooks

Running a local notebook is great for early data exploration and model tinkering, there’s no doubt about it. But eventually you’ll outgrow it and want to scale up and train the model in the cloud with easy parallel executions, full version control and robust deployment. (Letting you reproduce your experiments and share them with team members at any time.)

Multi-Cloud Data & Infrastructure Solution for Machine Learning

SwiftStack and Valohai, in joint partnership, announce the world’s first peta-scale ML solution that covers everything from computation to data management in a multi-cloud environment. The solution provides a global namespace removing silos and enabling universal access to all your data in all your machine learning use-cases. It has built-in support for Azure, Google Cloud, AWS and SwiftStack.

Kubeflow as Your Machine Learning Infrastructure

By now you’ve surely heard about Kubeflow, the machine learning platform based out of Google. Kubeflow basically connects TensorFlow’s ML model building with Kubernetes’ scalable infrastructure (thus the name Kube and Flow) so that you can concentrate on building your predictive model logic, without having to worry about the underlying infrastructure. At least in theory.