The basic thing I had to understand about this is how the structure of the messages list is a list of dictionaries. In this example an exchange between the user and the AI is modeled, but how to get actual user input while the program is running? Read on to get into it step by step, or go straight to the Github repository.
Before we get into it let me say that I'm using Mu Editor to write and run code. This is what I'll use to teach students but for more professional purposes you would use something like VSCode. To get the required library installed for Mu I simply added 'openai' to the third party packages tab and it was installed.
Incorporating User Input Into a Single Query
Here is the bare minimum you need to send a single question to ChatGPT and get a response (Sorry this code is not syntax highlighted):
from openai import OpenAI
client = OpenAI(api_key = "<your-API-key here")
# give AI a specialization
ai_experience = input("What am I? ")
# ask user question
user_input = input("You: ")
# send request to API
response = client.chat.completions.create(
model = "gpt-4o-mini",
messages = [
{"role": "system", "content": "You are a " + ai_experience},
{"role": "user", "content": user_input}
]
)
# Get just the response text
response = response.choices[0].message.content.strip()
# display response
print("Chatbot: ", response)
Some things to understand here:
- You need your own API key so OpenAI can track your API calls and start charging you if you go over the free allotment of tokens (about 4 characters = a token). However, just playing around with it for a while will cost very little. I put $5 on my account and over 2 days of API calls I used 3 cents. Cost really only becomes a factor when you have a publicly available app that thousands of people are using daily.
- The API key can be stored in different ways. Hard coding it into your program would not be wise by professional standards and it is usually tucked away in a hidden environment variable. But for the purpose of learning yourself and teaching beginners, this is fine.
Having a Conversation
Next we would like to loop this so we can keep asking questions either to get more detail on a point or refine previous answers. This will involve some steps:
Loop the Queries and Responses
from openai import OpenAI
client = OpenAI(api_key = "your-API-key here")
print("Welcome to ChatGPT! Type 'quit', 'exit', or 'bye' to end the conversation.")
# include the AI's area of expertise, will be the first item in every API call
ai_experience = input("What am I? ")
while True:
# ask user question
user_input = input("You: ")
if user_input.lower() in ["quit", "exit", "bye"]:
print("Chatbot: Goodbye!")
break
# send request to API
response = client.chat.completions.create(
model = "gpt-4o-mini",
messages = [
{"role": "system", "content": "You are a " + ai_experience},
{"role": "user", "content": user_input}
]
)
response = response.choices[0].message.content.strip()
# display response
print("Chatbot: ", response)
With the above code, the user input and chatbot response are repeated in an infinite loop (while True). The user can break out of the loop and end the conversation by typing one of the stated quit words. The problem with this code, which we will have to address—and which sent me searching all over for answers—is that the AI has no memory of the conversation you've been having. It's starting a fresh exchange each time the user submits a new prompt. I read this referred to as "goldfishing" somewhere. So if you first say, "Explain the steps to make an egg salad sandwich," it answers a list of steps, and you then say, "Elaborate on the first step," it will ask you for more context because it doesn't remember the first step it gave you in its previous response. The solution I found in my searching was that the developer is responsible for figuring out a way of saving the conversation history and re-submitting all of it each time a new query is sent, so the AI has the context of your session to use in its answers.
The developer's problem here is that with each API call your tokens are being used, which is costing you money. So you could keep re-submitting the entire chat history each time, but you're paying for the same content multiple times, which will really add up, especially if the chatbot's responses are verbose (you can tell it to answer in fewer than a certain number of characters to keep it succinct). Here are some solutions I came across:
- truncate the saved chat history you include each time, dropping the oldest prompts
- somehow have another service summarize the conversation so far and include that summary in your prompts
- some others I can't remember, so I'll just say I went with the first option which was the only one I could wrap my head around
Maintaining Chat History
The basic idea is to add each user prompt and chatbot response to a list of previous prompts and responses. Like the messages list above, it should be a list of dictionaries, which you can then drop the oldest elements from with "chat_history = chat_history[-10:]" (this keeps only the last 10). The messages list always starts with the system prompt you initially gave it—what type of expertise you want it to have—and then the history is added on to be submitted each time. So here is the final code that does all that:
from openai import OpenAI
client = OpenAI(api_key="your-API-key here")
print("Welcome to ChatGPT! Type 'quit', 'exit', or 'bye' to end the conversation.")
# include the AI's area of expertise, will be the first item in every API call
ai_specialization = input("What is my area of expertise? ")
system_prompt = {'role': 'system', 'content': 'You are a '+ ai_specialization}
# keep track of conversation to include with each API call
chat_history = []
while True:
# ask user question
user_input = input("You: ")
if user_input.lower() in ["quit", "exit", "bye"]:
print("Chatbot: Goodbye!")
break
# keep responses succinct to limit token usage
user_input = user_input + "Please keep your response to 50 words or fewer."
# append to chat history
chat_history.append({'role': 'user', 'content': user_input })
# start messages dict list with area of expertise and add recent chat history
messages = [system_prompt]
messages.extend(chat_history)
# send request to API
completion = client.chat.completions.create(
model="gpt-4o-mini",
messages=messages,
)
# add response to chat history and remove oldest items
response = completion.choices[0].message.content.strip()
chat_history.append({'role':'assistant','content': response})
chat_history = chat_history[-10:]
# for message in messages:
# print(message)
# display response
print(completion.choices[0].message.content.strip())
One problem that dogged me for a while was getting the dictionaries in proper format as I kept making them into strings by including too many quotes. Another general problem was any tutorial over like 6 months old was outdated because OpenAI made some big changes in the API between the release of ChatGPT 3.5 and 4, so using older examples was just a lot more work to update the language.
But finally I got this basic ChatGPT conversation app working and now you can, too!
No comments :
Post a Comment