Use Your Own Data To Get Response From GPT like ChatGPT | Python

In this article, I’ll show you how you can use your locally stored text files to get response using GPT-3. You can ask questions and get response like ChatGPT.

On technology front, we will be using:

• OpenAI 
• Langchain
• Python

Input files

You can take bunch of text files and store them in a directory on your local machine. I’ve grabbed input data from here and created 5 text files. My files are all about ‘Cause And Effect Of Homelessness’ and are placed in a directory named Store.

Import Required Packages

As we are using Python, let’s go ahead and import the required packages.

If you do not have above packages installed on your machine, then please go ahead and install these packages before importing.

Once required packages are imported, we need to get OpenAI API key.

Get OpenAI API Key

To get the OpenAI key, you need to go to https://openai.com/, login and then grab the keys using highlighted way:

Once you got the key, set that inside an environment variable(I’m using Windows).

Load Input Data

In order to load our text files, we need to instantiate DirectoryLoader and that can be done as shown below:

In above code, glob needs to be mentioned so that it will pick only the text files. This is particularly useful, when your input directory contains mix of different-different types of files.

Split Data

As input data could be very long, we need to split our data into small chunks and here I’m taking chunk size as 1000.

After splitting, this is how the text looks like:

Checkout my complete article here