<sub><sup>05/01/2025</sup></sub>

#### Don't Use JSON Schemas in Your LLM Prompts

Use them in the `response_schema` instead, if your LLM supports it of course. :^)

After a long and costly battle with Gemini, I’m calling it quits. I’ve been trying to get a JSON response from Gemini's `flash` model for a while now. Most of the time, the JSON response was malformed (often beyond repair), and I had to manually fix it. This required extensive testing to identify where it broke and whether the breakage was consistent enough to address.

Thanks to my team lead, I found out that providing a JSON schema in the prompt was only one way to get a JSON response. The better approach was to create a custom class that represents the JSON response and use that class with the `response_schema` parameter in the `GenerationConfig` in Python. I guess tunnel vision got the best of me, and I hadn’t even considered this solution.

Here’s a simplified example:

```python
class ComplexObject:
    color: str
    is_cool: bool

class JSONResponse:
    attributes: List[ComplexObject]

generation_config = genai.GenerationConfig(response_mime_type="application/json", 
                                           response_schema=JSONResponse)

gemini_model = genai.GenerativeModel("gemini-1.5-flash-latest",
                                     generation_config=generation_config)

payload =   """
            You are a helpful assistant that outputs 
            random cool colors and numbers based on user input.
            """

response = gemini_model.generate_content(payload)
```

With this very simplified example, we can get a JSON response from Gemini without any errors—at least so far in my testing.

#### Nice Tip

If the order of attributes in the JSON response is important, you can add a prefix to the attribute names. For example, to switch the order of the `colors` and `is_cool` attributes in the JSON response, you can modify the attribute names like this:

```python
class ComplexObject:
    a_is_cool: bool
    b_color: str
```

This may not seem like a huge deal, but if the LLM generates a response and is affected by its own prior output, this can be a lifesaver.