GPT-3 Fine tuning - Find any good tutorials?
- Thread starter Jeff Vh
- Start date
Cognitive
Regular Member
- Mar 11, 2015
- 401
- 222
If you want to fine-tune an Open AI GPT-3 model, you can just upload your dataset and OpenAI will take care of the rest...you don't need any tutorial for this.
If you want to fine-tune a similar model to GPT-3 (like those from Eluther AI) because you don't want to deal with all the limits imposed by OpenAI, here it is a tutorial that might help you a bit:
This is extremely helpful when you want to fine-tune on a Google TPU (recomended for their speed during training/ fine-tuning): https://github.com/kingoflolz/mesh-transformer-jax/blob/master/howto_finetune.md
If you want to fine-tune a similar model to GPT-3 (like those from Eluther AI) because you don't want to deal with all the limits imposed by OpenAI, here it is a tutorial that might help you a bit:
This is extremely helpful when you want to fine-tune on a Google TPU (recomended for their speed during training/ fine-tuning): https://github.com/kingoflolz/mesh-transformer-jax/blob/master/howto_finetune.md
Jeff Vh
Power Member
- Jun 19, 2019
- 557
- 786
@Cognitive Thank you for the clear response and the links. In regards to the quote above, how does one accomplish this? From following their documentation I believe I'd have to install Node .js, python etc in order to train the model?
basically you just upload a jsonl file to the finetuning endpoint, you could do that in postman if you don't want to write any code
Jeff Vh
Power Member
- Jun 19, 2019
- 557
- 786
Oh really?
Thanks for the heads up. I'll try that now. Anything I need to know to 'do' that? I've zero programming skills/postman experience.
- Apr 23, 2012
- 2,845
- 5,858
Oh really?
Thanks for the heads up. I'll try that now. Anything I need to know to 'do' that? I've zero programming skills/postman experience.
You'll need at bear minimum a knowledge of the Deep-learning model you're looking to fine-tune.
Based on that, you'll require a training dataset. At this point, domain knowledge is important. Prompt Engineering expertise is recommended.
The format for your dataset is typically
Code:
{"prompt": "<preferred prompt>", "completion": "<preferred generated text>"}
{"prompt": "<preferred prompt>", "completion": "<preferred generated text>"}Your domain knowledge comes at play here because the AI model will understand your input prompts and try to present you with data of varying output.
This same model can also be utilised to "update" the AI on what happened post 2019 (where it stops for instance)
Depending on your use case, you can implement dashboarding for your models using a third party service like Weights & Biases. - https://wandb.ai/
OpenAI W&B integrations works really well and you can use this Colab to run the fine-tune through them - https://colab.research.google.com/g...i/Fine_tune_GPT_3_with_Weights_&_Biases.ipynb
As for the dataset, here's a sample dataset - It's Wikipedia where the prompt is the title of the page and the preferred output is the first line - https://wandb.ai/borisd13/GPT-3/art...et/v0/files/wiki_title_description.table.json
Once you have the dataset, the actual fine tuning is really easy. And working your trainer model is even easier (both in your code + the OpenAI playground).
If you have a valid dataset, the actual training is not that hard and requires some basic python knowledge.
P.S - Don't forget to validate your model and its output.
Jeff Vh
Power Member
- Jun 19, 2019
- 557
- 786
Thank for the reply. W&B looks insane, but I'm currently not in need of a dashboard representation - thank you anyway!
by the way, do you happen to have some postmark experience to accomplish this with their website?
I've the domain knowledge, now I only need to know how to apply it.
I wish Openai had just a simple "upload json file here" input. - for those that know python, maybe this can be your new little side project? Lots of folks joining the nocode world without programming skills.
- Apr 23, 2012
- 2,845
- 5,858
Do you mean Postman?
If so, then do a couple things
1. Watch this video -
2. Keep this page open, so you know what details to fetch and where - https://beta.openai.com/docs/api-reference/fine-tunes
You will need to create 2 GET requests and 2 post requests in Postman -
POST - Upload a file + Create a Fine Tune
GET - List all fine tunes in your accounts + Get info on specific finetunes.
A POST request is where you send data to a server/API - while a GET request is where you ask for data from said API/server.
I wish Openai had just a simple "upload json file here" input. - for those that know python, maybe this can be your new little side project? Lots of folks joining the nocode world without programming skills.
It's easy enough to build something like that, if you have a dataset. Might consider it if there are enough takers.
Jeff Vh
Power Member
- Jun 19, 2019
- 557
- 786
This is the video I was looking for, thanks a lot for sharing!
Jeff Vh
Power Member
- Jun 19, 2019
- 557
- 786
So in this case I'd just fine-tune "davinci" then?
saad01
Junior Member
- Jan 20, 2013
- 108
- 90
Yes. davini-001 and any other models. 002 is already extremely fine tuned and capable.
- Apr 23, 2012
- 2,845
- 5,858
You have to understand the difference between models and API engines.
OpenAI offers their own fine tuned models for text completion and their codex API.
So in this case
text-davinci-002 is their own fine-tuned text completion model which has been trained until June 2021 (the base models have training data until 2019).When fine-tuning a model, you're asked to define a base model and a suffix. The base model can be either of
ada, babbage, curie or davinci.
- Apr 23, 2012
- 2,845
- 5,858
No - you cannot fine-tune an already fine-tuned model.
The approach in such cases would be to enhance your existing dataset and then fine-tune a new model.
Jeff Vh
Power Member
- Jun 19, 2019
- 557
- 786
Thanks for the clarification!
Jinxer
Junior Member
- Dec 7, 2021
- 129
- 68
Had some experience with GPT-3 a couple of weeks ago, encountered some interesting articles:
atlachinolli
Newbie
- Aug 14, 2022
- 2
- 0
Hey sorry if this is considered necroing but i was wondering if you wouldn't mind answering a couple questions i have, i'd greatly appreciate it
How can you "teach" a model true information without making it output similar text but just use it to "know" its true? I've seen people use regular informative text datasets to "teach" concepts to the model and then use dialog querry input vs dialog output datasets to fine tune it to talk like that, and thus the bot "knows" information that is true and will not make up information that is false while conversing, but it doesn't make text like the informative text, it still only talks like a dialog in short and concise replies to the querry instead of writing an article based on the prompt. Is there a name for this that I can read in the Open AI docu? The only examples i've seen are through OpenAI's own website but if it's possible I would prefer to be able to code it myself for future use and to avoid using their API, which takes me to the second question:
I've been looking for ways to fine tune with my 3080 but the docu seems to be only about using their paid service/hardware throught the API. I've done it in the past just copying other people's code but i'd be greatful if you could point me to a good resource to understand how to properly fine tune a pre trained model with transformers/torch etc for my application, mostly because I'd like to be able to modify existing code or write my own for different purposes/applications in the future. I'm a begginer- intermediate coder, i know python and c++ up to stuff like pointers, don't know a lot about data structures but I'm learning
My objective is conversational generation through saving in memory the last 5-10 messages and using them as context for predictions, i already have a code that works like that but i was wondering if there is any useful guide I could've missed by going directly to a niche use case solution.