LLM-generated JSON output

Sometimes with LLM-based AI, we would like to receive output in a format other than unstructured plain text: a structured format such as JSON (JavaScript object notation) enables machines (and even humans) to act upon with greater ease!

Code

To do this, we'll need to follow this page (have Python installed + install the openai library + get an api key + connect to the LLM)

Connecting to the API

If we're all setup, then we can just connect to the API:

# app.py

import os
from openai import OpenAI

client = OpenAI(
  api_key=os.environ.get("GROQAPIKEY"),
  base_url="https://api.groq.com/openai/v1"
)

Chat completion function

To get our output in JSON, we will have to modify our chat completion function by:

• including a system_prompt in addition the user's prompt
• using the system_prompt as an argument to the client.chat.completions.create's messages parameter
• add an argument to the client.chat.completions.create response_format parameter to emphasize that the output will appear in JSON format

# app.py (continued from last snippet)

def get_json_completion(prompt):
  system_prompt = "You are a digital assistant designed to output JSON for machine readability"
  response = client.chat.completions.create(
    model="llama-3.3-70b-versatile",
    messages=[      
      {"role": "system", "content": system_prompt},
      {"role": "user", "content": prompt}
    ],
    response_format={"type": "json_object"}
  )
  return response.choices[0].message.content

Calling the chat completion function

We would also call this function with a prompt that specifies the LLM to respond in JSON:

# app.py (continued from last snippet)

response = get_json_completion('What is the currency of Guatemala? Please respond in JSON, with "currency" as the key and the the curency as the value?')
print(response)

Output

With that, we can run the script with python3 app.py to get a machine-readable:

{
  "currency": "Quetzal"
}

Now, to retrieve only the value of currency we would need to use the json library and its loads method like such:

# app.py (continued from the last app.py snippet)

import json
json_response = json.loads(response)
print(json_response['currency'])

so that we can get the human-readable:

Quetzal

Implications

Of course, in most real world applications:

• we could have JSON output with multiple (and nested) properties
• values could have various data types, such as numbers, booleans, lists, etc.

Having the output in a structured JSON format helps keep the output simple enough for a machine to use!