Thanks: the reason why I thought it was token limited is that if you prototype prompts in the OpenAI playground, you can run out tokens quickly: OpenAI tells you you've used too many in the question and answer. So I guess I'm not really understanding this.
can anyone here give an example of a single prompt that will yield over 750 words?
Google doesnt detect that texts from ChatGPT as plagiarism if you dont pass them through a text spinner??
As warriorsam53 has implied, I'm not sure I understand the way tokens work, so I thought the only way to find out would be to conduct experiments in the OpenAI playground. I created a prompt based on what was initially suggested by the OP, with input gleaned by SEO Minion . I then changed variables one by one and did three runs every time and took the average word count.
1. I know you didnt talk about this, but i have read a few articles saying ChatGPT contains plagiarism. Is this True?
Python
I did like that, but I don't understand where output.csv should be saved?
input.csv, output.csv, openai_scraper_v1.exe and openai_key.txt all in the same folder
# Importuj biblioteki
import openai
import pandas as pd
import pickle
from tqdm import tqdm
# Wczytaj plik CSV
df = pd.read_csv("input.csv")
# Ustaw klucz API OpenAI
openai.api_key = "apikey"
# Utwórz pustą listę na wyniki
results = []
# Próbuj wczytać wyniki z pliku cache, jeśli istnieje
try:
with open("results_cache.pkl", "rb") as f:
results = pickle.load(f)
print("Wczytano dane z cache.")
except (FileNotFoundError, EOFError):
print("Brak danych w cache.")
pass
# Pobierz indeks ostatnio pobranego wiersza
last_index = -1
if len(results) > 0:
last_index = results[-1][0]
# Implementacja pętli for i paska postępu
for index, row in tqdm(df[last_index+1:].iterrows(), total=len(df[last_index+1:])):
try:
# Określ tekst zapytania dla OpenAI
prompt = f"Please write a detailed article about {row['topic']} Use this information to write the article: {row['context']}."
# Wywołaj usługę OpenAI
response = openai.Completion.create(engine="text-davinci-003", prompt=prompt, temperature=1, max_tokens=2048, top_p=1)
# Dodaj odpowiedź do listy wyników
results.append([index, row['topic'], response.choices[0].text])
# Zapisz wyniki do pliku cache co kilka wierszy
if index % 5 == 0:
with open("results_cache.pkl", "wb") as f:
pickle.dump(results, f)
except Exception as e:
print("Wystąpił błąd:", e)
break
# Utwórz DataFrame z wynikami
output_df = pd.DataFrame(results, columns = ['Index', 'Topic', 'Generated Text'])
# Zapisz wyniki do pliku CSV
output_df.to_csv('output.csv', index=False)
Nice tips... The reason I asked was that Mr. Warrior implied that it was possible in a single prompt. I have tried so many prompts, not even 1 post with over 800 words.As warriorsam53 has implied, I'm not sure I understand the way tokens work, so I thought the only way to find out would be to conduct experiments in the OpenAI playground. I created a prompt based on what was initially suggested by the OP, with input gleaned by SEO Minion . I then changed variables one by one and did three runs every time and took the average word count.
What I discovered was:
That's as far as I've got. The point of this post is to help people understand how to adjust the prompt. I think this is by far the most difficult aspect of the technique because the OpenAI interface is a bit non-intuitive, and if you Google this, you'll find all kinds of nonsense alongside the good stuff. I asked ChatGPT to help, but it hasn't produced much in the way of insight so far!
- Reducing Top P reduces the word count - in all my tests after this discovery, I set it to 1.
- Increasing T increasing variability and also average word count. In subsequent experiments I set it at 1. Note that this may impact the quality of the answer but this wasn't my interest - I wanted to understand the efffect of settings on word count.
- Surprisingly (for me but confirmed by something the OP said in the thread), increasing the max token parameter beyond a threshold did not increase the word count - and on average sometimes led to decreased word count. (But this may be due to sampling error: just taking 3 samples per setting. If this were a proper science project, I'd be taking 10 samples per setting.)
- By threshold I mean that if the max token setting was too low, OpenAI did not complete the answer: it stalled in mid sentence. A way to fix this is to type "continue", but I felt this would make the script too complicated, so preferred to find settings that produced no stalls.
- When the prompt included wording to the effect that the answer must include over 400 words, the word count in the answer increased for all settings and often, but not always, over 400 words. The threshold in this instance was around 500 tokens, or around 1.4 tokens per word.
- When the prompt was changed to say the answer must include over 800 words, the number of words in the output increased, but not to 800.
++++++++++++++++++++++++++++
Edit: I think one way forward is to specify the structure of the answer: start with an executive summary, finish with a comprehensive conclusion - that kind of thing. Hope that helps
Yeah, I did like that and nothing happened.
Hello,I did it with trial and error method but it generates only one article. Maybe server is busy but no errors at all. I don't know how to repair this code.Code:# Importuj biblioteki import openai import pandas as pd import pickle from tqdm import tqdm # Wczytaj plik CSV df = pd.read_csv("input.csv") # Ustaw klucz API OpenAI openai.api_key = "apikey" # Utwórz pustą listę na wyniki results = [] # Próbuj wczytać wyniki z pliku cache, jeśli istnieje try: with open("results_cache.pkl", "rb") as f: results = pickle.load(f) print("Wczytano dane z cache.") except (FileNotFoundError, EOFError): print("Brak danych w cache.") pass # Pobierz indeks ostatnio pobranego wiersza last_index = -1 if len(results) > 0: last_index = results[-1][0] # Implementacja pętli for i paska postępu for index, row in tqdm(df[last_index+1:].iterrows(), total=len(df[last_index+1:])): try: # Określ tekst zapytania dla OpenAI prompt = f"Please write a detailed article about {row['topic']} Use this information to write the article: {row['context']}." # Wywołaj usługę OpenAI response = openai.Completion.create(engine="text-davinci-003", prompt=prompt, temperature=1, max_tokens=2048, top_p=1) # Dodaj odpowiedź do listy wyników results.append([index, row['topic'], response.choices[0].text]) # Zapisz wyniki do pliku cache co kilka wierszy if index % 5 == 0: with open("results_cache.pkl", "wb") as f: pickle.dump(results, f) except Exception as e: print("Wystąpił błąd:", e) break # Utwórz DataFrame z wynikami output_df = pd.DataFrame(results, columns = ['Index', 'Topic', 'Generated Text']) # Zapisz wyniki do pliku CSV output_df.to_csv('output.csv', index=False)
/edit it generated two articles. so maybe its about overloaded server. I'll try in the evening to use it because progress bar may be very useful.
# Import necessary libraries
import openai
import pandas as pd
import pickle
import requests
import json
import base64
import re
from tqdm import tqdm
# Load CSV file
df = pd.read_csv("input.csv")
# Set API key for OpenAI
openai.api_key = "API KEY"
# Initialize a list to store the results
results = []
# Try to load the results from cache file
try:
with open("results_cache.pkl", "rb") as f:
results = pickle.load(f)
print("Data loaded from cache.")
except (FileNotFoundError, EOFError):
print("No data in cache.")
pass
# Get the index of the last row
last_index = -1
if len(results) > 0:
last_index = results[-1][0]
# Loop through the rows and generate text
for index, row in tqdm(df[last_index+1:].iterrows(), total=len(df[last_index+1:])):
try:
# Define the query text for OpenAI
headers = {
"Content-Type": "text/html",
}
prompt = f"<!-- Create detailed article with html tags and headings and bolded keywords on title {row['topic']}.--!>"
# Call the OpenAI API
response = openai.Completion.create(engine="text-davinci-003", prompt=prompt, temperature=0.7, max_tokens=3600, top_p=0.5, headers=headers)
# Add the response to the results list
results.append([index, row['topic'], response.choices[0].text])
# Save the results to the cache file
if index % 5 == 0:
with open("results_cache.pkl", "wb") as f:
pickle.dump(results, f)
except Exception as e:
print("Error:", e)
break
# Generate HTML from generated text
def parse_and_generate_html(text):
text = text.strip()
lines = text.split("\n")
html = ""
for line in lines:
if not line.strip(): # Skip empty lines
continue
if line.startswith("#"):
if line.startswith("##"):
line = line[2:]
html += f"<h2>{line}</h2>\n"
else:
line = line[1:]
html += f"<h1>{line}</h1>\n"
else:
html += f"<p>{line}</p>\n"
return html
# Create a DataFrame from the results
output_df = pd.DataFrame(results, columns = ['Index', 'Topic', 'Generated Text'])
# Generate HTML and add it to the data frame
output_df['Generated HTML'] = output_df['Generated Text'].apply(parse_and_generate_html)
# Loop through the rows and post the HTML content to WordPress
for index, row in output_df.iterrows():
try:
# Prepare the data for the WordPress API
data = {
"title": row['Topic'],
"content": row['Generated HTML'],
"status": "draft"
}
# Add the required credentials to the headers
headers = {
"Authorization": "Basic " + base64.b64encode(b"wordpress_python:password").decode("utf-8"),
"Content-Type": "application/json"
}
url = "domain.com/wp-json/wp/v2/posts"
# Post the generated text to WordPress as a draft post
response = requests.post(url, headers=headers, data=json.dumps(data))
except Exception as e:
print(e)
# Save the results to a CSV file
output_df.to_csv('output.csv', index=False)import openai
import pandas as pd
from tqdm import tqdm
df = pd.read_csv("input.csv")
# Set OpenAI API key
openai.api_key = "APIKEY"
results = []
errors = []
for index, row in tqdm(df.iterrows(), total=len(df)):
prompt = f"Please write a detailed article about {row['topic']}. Use this information to write the article: {row['context']}"
try:
response = openai.Completion.create(engine="text-davinci-003", prompt=prompt, temperature=0.7, max_tokens=2048, top_p=0.5)
results.append([row['topic'], response.choices[0].text])
except Exception as e:
errors.append([row['topic'], row['context'], str(e)])
continue
output_df = pd.DataFrame(results, columns=['Topic', 'Generated Text'])
# Save output to CSV file
output_df.to_csv('output.csv', index=False)
if len(errors) > 0:
# Log any errors that occurred during the script
error_df = pd.DataFrame(errors, columns=['Topic', 'Context', 'Error'])
error_df.to_csv('errors.csv', index=False)
wouldn't be easier to upload it with XML-RPC?, to def it and call it in the for loop so it's uploaded on the fly?
Hi, nice one. can you make it so that it does not try to pull content again from the ones already processed? Just in case you have to start over again to prevent duplicates.
