Cooking With Chef Watson: AI in the Kitchen

14 min read

After years of mastering the world’s most challenging games, predicting failures on crucial industrial equipment, and defending against destructive cyber attacks, artificial intelligence (AI) has picked up a surprising new challenge: cooking.

It may seem improbable that AI could be applied to something as personal as taste, but there may a logic to flavor combinations.

Ahn et al. (2011) proposed a food pairing hypothesis, which suggests that successful food pairings share similar molecular components — for example, caviar and white chocolate have several of the same chemical compounds, and would therefore taste good together.

Interestingly, this correlation appears more strongly in Western cuisine, while many Eastern recipes rely on contrasting molecular components.

Thus, an algorithm can be programmed with rules to follow the food pairing hypothesis and trained on a large corpus of data: online recipes.

So what insights can AI add to the world of food?

A Machine Palette

The IBM Watson group, the team behind AI’s “Jeopardy!” victory, ventured into culinary territory soon after it beat the trivia game. Leadership proposed that the system could be used to generate ideas, and cooking was suggested as a starting point.

The group believed that a cognitive computing system could help chefs explore new flavor combinations to create never-before-seen dishes. This certainly proved to be the case, for better or worse.

Watson was “fed” hundreds of thousands of recipes as a base point for flavor pairings. Lav Varshney, an IBM scientist, partnered with Florian Pinel, a computer scientist who also had a culinary degree, to train the system.

The pair classified dishes according to surprise (how novel the ingredients were), pleasantness (human perception of odor and flavor molecules), and synergy (the amount of shared chemical compounds).

Pleasantness, in particular, can occasionally be at odds with novelty — for example, eel and cheesecake is certainly a novel combination, but it likely would not be pleasant.

Creations from the Cognitive Kitchen

Watson can generate bold new ideas for flavor combinations.

However, initially its outputs were simply ingredients, making chefs essential to conceptualizing the dish (including determining the amounts of each ingredient), executing it, and seeing if Watson’s suggestions actually taste good.

The chefs in the research group had fun with this, often having two chefs come up with recipes from the same ingredients and having them go head-to-head in taste tests.

In 2014, Chef Watson attracted attention at South by Southwest (SXSW) in Austin, with a food truck serving dishes like an Austrian chocolate burrito. There was also a formal dinner with a tasting menu of dishes like roast duck with cherry, olive, and cinnamon and Kenyan Brussels sprouts with sweet potato puree and almond gremolata.

Even cocktails were added to Watson’s repertoire. Mixologist Anthony Caporale took the unconventional step of asking Watson to incorporate a protein into the recipe, meaning drinks had ingredients like bacon or chicken broth (though in this case we can all agree shrimp cocktail should not be literal).

A Galaxy of Food

Across the Pacific, Dr. Yoshiki Ishikawa, a public health researcher from Japan, wanted to tap into the personalization abilities of AI to benefit the food world.

“Modern dietary styles have been more inclined toward unhealthy eating patterns,” he explained. “While there is compelling research in nutrition sciences on what makes a recipe healthy, this does not necessarily mean that such a recipe is matched to one’s unique food preferences.”

He continued, “Personalized information systems that can transform a recipe into any selected dietary style that a user might prefer would help food companies and professional chefs create new recipes.”

For example, a family moving from Japan to the United States may want to find equivalent local recipes that match their palate.

To tackle this problem, he proposed two steps:

  1. Identifying the dietary influences of any selected recipe and
  2. developing an algorithm that shifts a recipe into any selected dietary style.

He created a team to research these issues using a corpus of data from Yummly recipes. The team included two nutrition scientists to categorize world recipe data, a computer scientist to build the algorithm, a designer to visualize the project, the chef of a Michelin-starred restaurant to create the recipes, and finally, Ishikawa and Varshney, now at the University of Illinois, to oversee the project.

A Recipe for Success

First, the recipes were categorized into 20 different cuisines (like Thai, Russian, and Southern) based on the ingredients. Then algorithms were developed using the food pairing hypothesis to find appropriate substitutions from other cuisines.

Given that certain cuisines are considered more beneficial from a health perspective (for example, the Mediterranean diet), this program could allow users to shift their tastes to healthier alternatives.

The product of this research was unveiled at SXSW in 2017. The result is The Galaxy of Food, which can transform a recipe from one style (say, Japanese) into another style (say, French) by suggesting ingredient substitutions.

The algorithm recreated sukiyaki, a kind of Japanese hot pot with meat cooked with vegetables, soy sauce, and mirin, with French flavors by substituting calvados (French apple brandy) for the mirin and bouquet garni (French herbs) for the soy sauce.

“What we want to ultimately solve through this eort is ‘100,000 meals problem,” explained Ishikawa, referring to the 100,000 meals that a person eats in their lifetime. “I, as a public health researcher, really want to make each meal enjoyable, diversified, and healthy as much as possible, which might even create conditions for world peace.”

Adding Flavor Without Taste Buds

AI has made great strides in the kitchen. Recipes now include ingredient amounts and instructions, and the emphasis on novel flavor combinations can be adjusted. However, it may not have quite earned its culinary degree yet.

Over at the New Yorker, Alexandra Kleeman appreciated the AI’s ability to help her use leftover ingredients in her fridge, but was less successful when she tried to throw a dinner party using Watson-recommended recipes. Her guests were ready to mutiny by the time the program suggested ice cream with curry powder.

However, like any algorithm, AI for recipes learns from feedback. As more adventurous chefs tell the systems what does and does not work, it will adjust its ingredient outputs to become more palatable to the rest of the population. Though cognitive recipes may be a bit too “creative” for now, an AI kitchen assistant may not be far off.

Art or Artifact: Music in the AI EraPrevious ArticleArt or Artifact: Music in the AI Era The New Reality of Cyber ThreatsNext ArticleThe New Reality of Cyber Threats