Valohai blog

Insights from the deep learning industry.

All Posts

Valohai Joins Forces with Twitter and Facebook

Valohai, the MLOps platform company, is collaborating with Twitter and Facebook to launch a competition for the annual The Neural Information Processing Systems (NeurIPS) conference to advance the optimization of machine learning models towards more accurate AI solutions. The goal is to find better optimization algorithms for machine learning.

Our joint challenge is called Black-Box Optimization for Machine Learning where competitors are trying to find the best method for optimizing the black-box functions arising from tuning machine learning models. Competitors input the configuration of the search space for the algorithms, but everything else about the objectives remains hidden.

The competitors are encouraged to elaborate on the open-source optimization algorithm such as Bayesian optimization and evolutionary algorithms. One of the fundamental challenges in training a machine learning model is choosing the right hyperparameters. The estimation of hyperparameters is supposed to improve based on the knowledge gained during the several training iterations.

The competition runs on top of the Valohai API

Valohai, together with Twitter, built a dedicated website for participating and seeing how competitors fare against others throughout the competition. The platform is built on top of the Valohai Machine Learning platform to trigger the black box each time the submission is sent.

The Black-Box Optimization competition website is a custom-built lightweight layer on top of the Valohai platform and its API. While the user interface for competitors is simple and straightforward, what happens under the hood is a more complicated story. Every submission injects hundreds of jobs into an execution queue consumed by several cloud worker instances orchestrated by the Valohai auto-scaling system.

Each job runs a slightly different experiment to measure the optimizer's performance, and the results are read constantly back to the competition website for real-time scoring. Participants can stop the runs at any time midway. All the inputs, logs, metrics, and outputs for every new submission are automatically stored for the post-competition analysis, and each run is reproducible.

Do you want to participate?

The competition lasts from July 1st to October 15th. Individuals and teams can take part and send the submissions at any time between the competition dates. There will be a $10 000 total in prizes divided between the TOP 3 teams or individuals. The winner will receive a $5000 prize, second place is rewarded with $3000, and third place with $2000.

Go to the competition website bbochallenge.com

What is NeurIPS?

The Neural Information Processing Systems (NeurIPS) is a research-focused annual conference gathering together with the brightest minds in the machine learning (ML) industry to share the research on neural information processing systems in their biological, technological, mathematical, and theoretical aspects. The conference was first held in 1987 and, for the last few years, there has been a Competition Track where ML practitioners can join to solve the most pressing current challenges in ML development.

Joanna Purosto
Joanna Purosto
Technology oriented marketer training a model to recognize sequences in my golf swing.

Related Posts

Introducing Minihai – Easiest way to run notebooks remotely.

Imagine this; you are working on a notebook that takes ages to run, and it bogs down your computer, so it's even hard to multitask. We've seen this countless times, which is why we are introducing Minihai.

Why Are ML Engineers Becoming So Sought After?

For a long time, most machine learning initiatives have been stuck in a persistent state of proofs-of-concept. However, in the past year, we’ve seen a rapid acceleration of machine learning models getting real-world use. Consequently, machine learning engineers are increasingly sought after – nearly catching up to data scientists in posted jobs.

Valohai Joins Forces with Twitter and Facebook

Valohai, the MLOps platform company, is collaborating with Twitter and Facebook to launch a competition for the annual The Neural Information Processing Systems (NeurIPS) conference to advance the optimization of machine learning models towards more accurate AI solutions. The goal is to find better optimization algorithms for machine learning.