Gantry Blog


Subscribe to our newsletter
How to teach an old model new tricks

How to teach an old model new tricks

If you want a single machine learning model that can solve a variety of image classification tasks, you might look to an open-vocabulary…
Josh Tobin
Josh TobinMarch 14, 2023
Putting Responsible AI into Practice

Putting Responsible AI into Practice

Microsoft’s challenges rolling out Bing Chat put responsible use of AI in the news last week. As AI systems get more capable, ML…
Josh Tobin
Josh TobinFebruary 23, 2023
How to measure language model performance

How to measure language model performance

You probably feel like Language Models are advancing at a stunning pace. But how do we know they really are? And how can we quantify how…
Josh Tobin
Josh TobinFebruary 16, 2023
Why do ML Projects Fail?

Why do ML Projects Fail?

MLOps is a field devoted to studying how infrastructure, tools, and other technology can solve ML problems. But a new paper asserts that…
Josh Tobin
Josh TobinJanuary 26, 2023
Monolith: The Recommendation System Behind TikTok

Monolith: The Recommendation System Behind TikTok

Among all of the social networks, TikTok may be the one that is best known for its recommendations. It’s downright addictive. However, until…
Josh Tobin
Josh TobinJanuary 10, 2023
MLOps at Industrial-Scale: Lessons from Google

MLOps at Industrial-Scale: Lessons from Google

Have you ever wondered what it's like to do MLOps at Google-scale? A new paper shines a light on how Google deploys, maintains, and improves…
Josh Tobin
Josh TobinDecember 23, 2022
From prompt magic to prompt engineering?

From prompt magic to prompt engineering?

Large language models (LLMs) like GPT-3 and ChatGPT have captured the heart of techno-twitter and were one of the catalysts of the current…
Josh Tobin
Josh TobinDecember 14, 2022
How do people actually operationalize ML in 2022?

How do people actually operationalize ML in 2022?

You’ve probably seen countless blogs and courses on how to do MLOps. While a great starting point, at best they represent the views of one…
Josh Tobin
Josh TobinDecember 07, 2022
Do You Really Need a Feature Store?

Do You Really Need a Feature Store?

If you’re building out your ML stack, you’ve probably considered implementing a feature store. Used by companies like Uber, Netflix, Airbnb…
Josh Tobin
Josh TobinNovember 30, 2022
Test-Time Adaptation: update your model using only unlabeled test data

Test-Time Adaptation: update your model using only unlabeled test data

If you’re reading this post, it’s probably not news to you that machine learning models can perform poorly on out-of-distribution data…
Josh Tobin
Josh TobinNovember 22, 2022
Active Surrogate Estimators: How many labels do you really need to approximate model performance?

Active Surrogate Estimators: How many labels do you really need to approximate model performance?

Say you deploy your model in a new setting and want to measure how accurate it is in the new domain. Naively, you could randomly sample data…
Josh Tobin
Josh TobinNovember 10, 2022
MLDemon: cheaper monitoring of production models

MLDemon: cheaper monitoring of production models

Say you’ve deployed a model and want to see how it’s performing in production. As we’ve discussed in the past, the commonly recommended…
Josh Tobin
Josh TobinOctober 27, 2022
How personalized can products get with deep learning?

How personalized can products get with deep learning?

When you and I look at a software application, we see more-or-less the same thing. Maybe the order of the items presented to you in a…
Josh Tobin
Josh TobinOctober 21, 2022
Better learning by learning about learning

Better learning by learning about learning

As machine learning practitioners, we spend a lot of time staring at loss curves. Over the years, by looking at thousands of them, we build…
Josh Tobin
Josh TobinOctober 14, 2022
Responsible machine learning is like security (and maybe like product management, too)

Responsible machine learning is like security (and maybe like product management, too)

Many machine learning practitioners advocate for the importance of ethical AI. But in practice, few ML teams put even basic fairness / bias…
Josh Tobin
Josh TobinOctober 06, 2022
Do ML-powered products need to be designed differently than all other products?

Do ML-powered products need to be designed differently than all other products?

So you built a great ML model. All that’s left is to do is deploy it into your product, right? Not so fast. A model is just one part of a…
Josh Tobin
Josh TobinSeptember 29, 2022
What can Data-Centric AI Learn from Data and ML Engineering?

What can Data-Centric AI Learn from Data and ML Engineering?

Normally, we think of the ML process as model-centric: we iterate on the model until it performs well on a given dataset. Data-centric AI…
Josh Tobin
Josh TobinSeptember 22, 2022
Can we do better than "drift detection"?

Can we do better than "drift detection"?

As we've discussed in the past, "detecting drift" is usually the wrong framing for monitoring ML models in production. Placing too much…
Josh Tobin
Josh TobinSeptember 15, 2022
Fixes that Fail: Self-Defeating Improvements in Machine Learning Systems

Fixes that Fail: Self-Defeating Improvements in Machine Learning Systems

Production ML Papers to Know This is a continuation of Production ML Papers to Know, a series from Gantry highlighting papers we think have…
Josh Tobin
Josh TobinSeptember 08, 2022
Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learnt

Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learnt

Production ML Papers to Know This is a continuation of Production ML Papers to Know, a series from Gantry highlighting papers we think have…
Josh Tobin
Josh TobinSeptember 01, 2022
What to do if you have *too much* labeled data

What to do if you have *too much* labeled data

This week, let's talk about what you can do if you have more data and labels than you know what to do with 🤑 Your favorite AI breakthrough…
Josh Tobin
Josh TobinAugust 26, 2022
You're probably monitoring your models wrong

You're probably monitoring your models wrong

You shipped your machine learning model, and it’s starting to interact with real users. Congratulations on not being part of a (possibly…
Josh Tobin
Josh TobinJuly 20, 2022
Introducing Gantry: The tool to iterate on machine learning-powered products

Introducing Gantry: The tool to iterate on machine learning-powered products

TL/DR. As easy as it has become to train models, getting them to work well in real products with real users is still a mess. These ML…
Josh Tobin
Josh TobinJune 07, 2022
Toward continual learning systems

Toward continual learning systems

One of the biggest misconceptions I hear from laypeople about AI is the belief that machine learning models get smarter as they interact…
Josh Tobin
Josh TobinJuly 08, 2021