Valohai blog

Insights from the deep learning industry.

All Posts

Valohai Joins Forces with Twitter and Facebook

Valohai, the MLOps platform company, is collaborating with Twitter and Facebook to launch a competition for the annual The Neural Information Processing Systems (NeurIPS) conference to advance the optimization of machine learning models towards more accurate AI solutions. The goal is to find better optimization algorithms for machine learning.

Our joint challenge is called Black-Box Optimization for Machine Learning where competitors are trying to find the best method for optimizing the black-box functions arising from tuning machine learning models. Competitors input the configuration of the search space for the algorithms, but everything else about the objectives remains hidden.

The competitors are encouraged to elaborate on the open-source optimization algorithm such as Bayesian optimization and evolutionary algorithms. One of the fundamental challenges in training a machine learning model is choosing the right hyperparameters. The estimation of hyperparameters is supposed to improve based on the knowledge gained during the several training iterations.

The competition runs on top of the Valohai API

Valohai, together with Twitter, built a dedicated website for participating and seeing how competitors fare against others throughout the competition. The platform is built on top of the Valohai Machine Learning platform to trigger the black box each time the submission is sent.

The Black-Box Optimization competition website is a custom-built lightweight layer on top of the Valohai platform and its API. While the user interface for competitors is simple and straightforward, what happens under the hood is a more complicated story. Every submission injects hundreds of jobs into an execution queue consumed by several cloud worker instances orchestrated by the Valohai auto-scaling system.

Each job runs a slightly different experiment to measure the optimizer's performance, and the results are read constantly back to the competition website for real-time scoring. Participants can stop the runs at any time midway. All the inputs, logs, metrics, and outputs for every new submission are automatically stored for the post-competition analysis, and each run is reproducible.

Do you want to participate?

The competition lasts from July 1st to October 15th. Individuals and teams can take part and send the submissions at any time between the competition dates. There will be a $10 000 total in prizes divided between the TOP 3 teams or individuals. The winner will receive a $5000 prize, second place is rewarded with $3000, and third place with $2000.

Go to the competition website bbochallenge.com

What is NeurIPS?

The Neural Information Processing Systems (NeurIPS) is a research-focused annual conference gathering together with the brightest minds in the machine learning (ML) industry to share the research on neural information processing systems in their biological, technological, mathematical, and theoretical aspects. The conference was first held in 1987 and, for the last few years, there has been a Competition Track where ML practitioners can join to solve the most pressing current challenges in ML development.

Joanna Purosto
Joanna Purosto
Technology oriented marketer training a model to recognize sequences in my golf swing.

Related Posts

Valohai Joins Forces with Twitter and Facebook

Valohai, the MLOps platform company, is collaborating with Twitter and Facebook to launch a competition for the annual The Neural Information Processing Systems (NeurIPS) conference to advance the optimization of machine learning models towards more accurate AI solutions. The goal is to find better optimization algorithms for machine learning.

Why colabel adopted Valohai instead of hiring their first MLOps engineer

Colabel decided to adopt Valohai to manage their machine learning infrastructure & model serving through Kubernetes instead of hiring an MLOps engineer. TL;DR colabel enables companies to automate workflows specific to their business, from recognizing objects in microscopic images to automatically categorizing incoming documents for different internal workflows. They needed a solution that automatically manages their machine learning infrastructure and model serving through their Kubernetes cluster on Google Cloud Platform. colabel started by building their own MLOps solution with Kubeflow but quickly realized that the costs associated with building, maintaining, and hiring the right talent to build your own MLOps solution were not financially sustainable. Today colabel can focus on building their platform and its integrations, leaving the hassle of maintaining the machine learning infrastructure and Kubernetes to Valohai. colabel helps businesses build custom models for automating image and document processing Every business has workflows that can be automated and every business is different. Using out of the box solutions, such as APIs to classify dogs and hot-dogs will take you only so far.

What did I Learn about CI/CD for Machine Learning

Most software development teams have adopted continuous integration and delivery (CI/CD) to iterate faster. However, a machine learning model depends not only on the code but also the data and hyperparameters. Releasing a new machine learning model in production is more complex than traditional software development.