Generating AI Model Responses in JSON Format Using Ollama and Llama 3.2

In the rapidly evolving field of artificial intelligence, generating accurate and contextually relevant responses is crucial. Ollama, a lightweight and extensible framework, combined with the powerful Llama 3.2 model, provides a robust solution for generating AI model responses in JSON format. This article explores how to leverage these tools to create efficient and effective AI responses.

In case, if you are interested in knowing every single bit then here is my video recording:

Setting Up Ollama and Llama 3.2

Before diving into the specifics of generating responses, it's essential to set up Ollama and Llama 3.2 on your local machine. Ollama offers a straightforward installation process, and you can download the necessary models from the Ollama library. 

Import required packages

In order to get started with code, first we need to import the required packages:

from ollama import chat

from pydantic import BaseModel

Generating Responses in JSON Format

JSON format is a structured, human-readable data format with a more compact syntax. To generate responses in SON format, you need to define the structure of the response and ensure that the model outputs data in this format.

Here is the sample structure:

class AnimalInfo(BaseModel):

    animal_name:str

    height:int

    notable_feature:str

class AnimalsList(BaseModel):

    animals:list[AnimalInfo]

Using below line, you can take a quick look at the generated schema:

print(AnimalsList.model_json_schema())

and this is how the schema looks like:

Once the schema is ready, next step is to make a call to LLM and that can be done by using below code:

response = chat(

    model='llama3.2',

    messages=[{'role':'user', 'content':'Tell me about 2 tallest animals.'}],

    format=AnimalsList.model_json_schema(),

    options={'temperature':0.1}

)

And finally the output part:

animals_response = AnimalsList.model_validate_json(response.message.content)

print(animals_response)

For the sample question asked by user, here is the output in desired form:

Conclusion

Generating AI model responses in JSON format using Ollama and Llama 3.2 provides a structured and efficient way to handle AI-generated content. By leveraging these tools, developers can create more accurate and contextually relevant responses, enhancing the overall user experience.