Part 4: Transforming 20,000 Recipes with ChatGPT

Introduction

In the previous three parts, I covered the classical machine learning side of this project: data cleansing (Part 1), cosine similarity for nutritional search (Part 2), and LSTM for temporal menu prediction (Part 3).

All of that work was done years ago, before the current AI revolution. The system worked, but there was one problem I could never solve with traditional ML: the recipes themselves were too complicated for everyday cooking.

Then large language models arrived, and everything changed.

---

The Problem Traditional ML Could Not Solve

The recipe database contained nearly 20,000 dishes, many of them from professional cookbooks and culinary databases. These recipes were detailed, precise, and often impractical for a typical weeknight dinner.

A recipe might call for:

• 15+ ingredients, some of which are hard to find
• Elaborate multi-step preparation (marinate overnight, prepare a stock from scratch)
• Specialized equipment or techniques

The cosine similarity engine could find nutritionally equivalent alternatives, and the LSTM could suggest varied weekly menus. But if every suggestion was a 90-minute, 20-ingredient production, the system was not actually solving the daily cooking problem.

What I needed was a way to transform recipes — keeping the essence and approximate nutrition while making them realistic for someone cooking after a long day.

This is not a mathematical operation. It requires understanding what makes a dish work, which ingredients are essential versus optional, and how to simplify techniques without losing flavor. In other words, it requires language understanding.

---

Enter ChatGPT

When the ChatGPT API became available, I immediately saw the application. An LLM could read a complex recipe and rewrite it as a simpler version — something that no amount of cosine similarity or LSTM training could accomplish.

The API Call

Here is the core function for sending a recipe to ChatGPT for simplification:

import openai
import json

openai.api_key = "your-api-key"

def simplify_recipe(recipe_text, recipe_title):
    """
    Send a recipe to ChatGPT and get back a simplified version
    with a catchy name and structured ingredient list.
    """
    prompt = f"""以下のレシピを、忙しい日でも作れる簡単バージョンに変換してください。

ルール:
1. 材料は10個以内に減らす（代替可能なものは省略）
2. 工程は5ステップ以内にまとめる
3. 調理時間30分以内を目標にする
4. キャッチーな料理名をつける
5. 元の栄養バランスはできるだけ維持する

出力はJSON形式で:
{{
  "original_title": "元の料理名",
  "simplified_title": "新しいキャッチーな料理名",
  "ingredients": ["材料1", "材料2", ...],
  "steps": ["手順1", "手順2", ...],
  "estimated_time": "調理時間（分）",
  "nutrition_notes": "栄養面での変更点"
}}

レシピ:
タイトル: {recipe_title}
{recipe_text}
"""

    response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=[
            {"role": "system", "content": "あなたは家庭料理の専門家です。"},
            {"role": "user", "content": prompt}
        ],
        temperature=0.7,
        max_tokens=1000
    )

    result_text = response.choices[0].message.content

    # Parse JSON from response
    try:
        result = json.loads(result_text)
        return result
    except json.JSONDecodeError:
        # Sometimes the model wraps JSON in markdown code blocks
        if "```json" in result_text:
            json_str = result_text.split("```json")[1].split("```")[0].strip()
            return json.loads(json_str)
        raise

Prompt Engineering

The prompt design went through several iterations. Key lessons learned:

1. Explicit constraints work: Telling the model “10 ingredients or fewer” and “5 steps or fewer” produced consistently concise results. Vague instructions like “make it simpler” led to inconsistent output.

2. JSON output format in the prompt: Specifying the exact JSON structure in the prompt dramatically reduced parsing errors. The model tends to follow the template closely when you provide a concrete example.

3. Nutrition preservation note: Adding “maintain the original nutritional balance as much as possible” helped the model keep the protein source and vegetable components even when simplifying.

4. Temperature 0.7: Lower temperatures produced boring, formulaic simplifications. Higher temperatures occasionally invented ingredients. 0.7 was the sweet spot for creative but grounded results.

---

Batch Processing 20,000 Recipes

Processing the entire recipe database required careful batch handling — both for API rate limits and cost management:

import time
import csv

def batch_simplify(input_csv, output_csv, batch_size=10, delay=2):
    """
    Process recipes in batches with rate limiting and error handling.
    """
    df = pd.read_csv(input_csv)
    results = []
    errors = []

    for i, row in df.iterrows():
        try:
            result = simplify_recipe(
                recipe_text=row["instructions"],
                recipe_title=row["recipe_title"]
            )
            result["original_recipe_id"] = row["recipe_id"]
            results.append(result)

            if (i + 1) % batch_size == 0:
                print(f"Processed {i + 1}/{len(df)} recipes")
                time.sleep(delay)  # Rate limiting

        except Exception as e:
            errors.append({
                "recipe_id": row["recipe_id"],
                "title": row["recipe_title"],
                "error": str(e)
            })
            print(f"Error on recipe {row['recipe_id']}: {e}")
            time.sleep(5)  # Back off on errors

    # Save results
    with open(output_csv, "w", encoding="utf-8", newline="") as f:
        writer = csv.DictWriter(f, fieldnames=results[0].keys())
        writer.writeheader()
        writer.writerows(results)

    print(f"\nCompleted: {len(results)} simplified, {len(errors)} errors")
    return results, errors

Cost Considerations

At the time of processing, GPT-3.5 Turbo cost approximately $0.002 per 1,000 tokens. Each recipe simplification used roughly 500-800 tokens for input and 300-500 tokens for output. For 20,000 recipes:

• Estimated total tokens: ~20 million
• Estimated cost: ~$40

For an individual developer, this was a reasonable one-time investment. Using GPT-4 would have cost roughly 20 times more — which is why I opted for GPT-3.5 Turbo for the batch processing.

---

Before and After

Here is a concrete example of what the simplification looks like:

Before (Original Recipe)

Title: Braised Mackerel with Miso Glaze and Root Vegetables
Ingredients (16 items):
  - Fresh mackerel fillet (2 pieces)
  - White miso paste (3 tbsp)
  - Red miso paste (1 tbsp)
  - Sake (3 tbsp)
  - Mirin (2 tbsp)
  - Sugar (1 tbsp)
  - Fresh ginger (1 knob, sliced)
  - Daikon radish (1/3, cut into rounds)
  - Burdock root (1, shaved)
  - Lotus root (small, sliced)
  - Dried shiitake (4, reconstituted)
  - Konbu dashi (200ml)
  - Soy sauce (1 tsp)
  - Green onion (for garnish)
  - Shiso leaves (for garnish)
  - Togarashi (for garnish)

Steps (8 steps):
  1. Reconstitute dried shiitake in water for 30 minutes
  2. Prepare dashi by simmering konbu...
  3. Pre-boil daikon rounds for 10 minutes...
  4. Score mackerel skin and salt for 15 minutes...
  ...

After (Simplified by ChatGPT)

{
  "original_title": "Braised Mackerel with Miso Glaze and Root Vegetables",
  "simplified_title": "Quick Miso Mackerel Bowl",
  "ingredients": [
    "Mackerel fillet (2 pieces)",
    "Miso paste (3 tbsp, any type)",
    "Sake or white wine (2 tbsp)",
    "Sugar (1 tbsp)",
    "Ginger (1 tsp, grated)",
    "Daikon or turnip (1 cup, diced)",
    "Soy sauce (1 tsp)",
    "Green onion (for topping)"
  ],
  "steps": [
    "Mix miso, sake, sugar, and ginger to make the glaze",
    "Pan-sear mackerel skin-side down for 3 minutes",
    "Add diced daikon and 100ml water, simmer 10 minutes",
    "Pour miso glaze over fish, cook 5 more minutes until glossy",
    "Serve over rice, top with green onion"
  ],
  "estimated_time": "25 minutes",
  "nutrition_notes": "Protein and omega-3 preserved. Reduced garnishes lower negligible nutrients. Simplified root vegetables maintain fiber content."
}

The 16-ingredient, 8-step recipe became an 8-ingredient, 5-step dish that can be made in 25 minutes. The core nutritional value — protein from the mackerel, fiber from the root vegetables — is preserved. The specialized ingredients (lotus root, burdock, dried shiitake, konbu) are removed or substituted with more accessible alternatives.

---

A Decade in Reflection

This project spans more than ten years, and looking back, it traces the evolution of applied AI itself:

Phase 1: Classical Data Science (2010s)

• Tool: pandas, scikit-learn
• Approach: Represent recipes as numerical vectors, compute mathematical similarity
• Strength: Precise, explainable, no API costs
• Limitation: Cannot understand recipes as language

Phase 2: Deep Learning (Mid-2010s)

• Tool: Keras, TensorFlow, LSTM
• Approach: Learn temporal patterns from meal sequences
• Strength: Captures time-based variety, generates novel sequences
• Limitation: Needs substantial training data, no semantic understanding

Phase 3: Large Language Models (2020s)

• Tool: OpenAI API, ChatGPT
• Approach: Natural language understanding and generation
• Strength: Can read, understand, and rewrite recipes; handles nuance and creativity
• Limitation: API costs, latency, occasional hallucination

Each phase did not replace the previous one — they complement each other. Cosine similarity still provides the fastest, most precise nutritional matching. LSTM still handles temporal sequencing better than throwing meal history at an LLM. And ChatGPT does something neither of the earlier approaches could: understand and transform natural language.

What I Would Build Today

If I were starting this project from scratch in 2024, I would likely use an LLM for all three tasks — nutritional comparison, menu planning, and recipe simplification — in a single unified pipeline. The embeddings from a modern LLM already encode nutritional relationships, temporal patterns, and semantic meaning simultaneously.

But there is something valuable about having built it the hard way first. Understanding why cosine similarity works for nutritional matching, or why temperature sampling produces diverse outputs, gives you a foundation that makes you a better engineer even when working with LLMs. You know what the model is doing under the hood because you have built simpler versions of the same mechanisms.

---

Conclusion

What started as a personal project to solve a daily cooking struggle became a decade-long exploration of applied machine learning — from hand-crafted feature vectors to neural sequence prediction to large language models.

The cooking problem itself remains unsolved at scale. No app has truly cracked the daily meal planning challenge in a way that accounts for nutrition, variety, preferences, available ingredients, cooking time, and budget simultaneously. But the tools to build such a system are now accessible to any developer with an API key and some Python knowledge.

If there is one takeaway from this series, it is this: the best time to start building with AI was ten years ago. The second best time is now. The problems worth solving have not changed. The tools have just gotten dramatically better.

---

Series Navigation

• Part 1: Solving the Daily Meal Planning Problem with Data
• Part 2: Finding “Same Nutrition, Different Meal” with Cosine Similarity
• Part 3: Predicting “Non-Boring” Menus with LSTM Time Series
• Part 4: Transforming 20,000 Recipes with ChatGPT (this article)