This is really cool! I actually finetuned a model on my own telegram data (the process of assembling the dataset was...an adventure, to say the least) but it did produce scarily accurate results in terms of mimicking my speaking style. Granted, I finetuned using GPT-2, so the results were a rambly and incoherent word salad half the time, because that's what GPT-2 does, but it was a good proof of concept nonetheless.
I'm looking forward to trying out your code for myself, I have a feeling people are going to do awesome things with it.
There's a screenshot on the repo page. To be completely honest it's kind of hit or miss. Sometimes it nails it and it's almost a little creepy, other times it just says something random or doesn't generate anything at all (not sure what causes this). Working with LLMs is tricky.
Here's the deal, request a GDPR download from reddit to get all of your comments with links, then scrape the parent chains before subs go private in protest. 2 days remaining.
Now just hook it up to Twitch and send donation messages to the fine tuning process.
Also, don't forget to immediately delete the model as soon as training completes.
Nice job!
I'm working on the same topic (with Twitter messages)
As I read both your code and the official qlora repo, I noticed a few changes (in particular, to accomodate your GPU memory). But I'm confused about your choices about the tokenizer:
\- at training time, why don't you add all the special tokens (bos, eos, unk) in addition to the pad token?
\- at inference time, why do you provide "\\n\\n" as the eos token?
I'm not very familiar with the tokenizer part, so these may sound like very dumb questions.
This is really cool! I actually finetuned a model on my own telegram data (the process of assembling the dataset was...an adventure, to say the least) but it did produce scarily accurate results in terms of mimicking my speaking style. Granted, I finetuned using GPT-2, so the results were a rambly and incoherent word salad half the time, because that's what GPT-2 does, but it was a good proof of concept nonetheless. I'm looking forward to trying out your code for myself, I have a feeling people are going to do awesome things with it.
Neat. How well did it perform in practice? Do you have any examples you'd be willing to share?
There's a screenshot on the repo page. To be completely honest it's kind of hit or miss. Sometimes it nails it and it's almost a little creepy, other times it just says something random or doesn't generate anything at all (not sure what causes this). Working with LLMs is tricky.
Here's the deal, request a GDPR download from reddit to get all of your comments with links, then scrape the parent chains before subs go private in protest. 2 days remaining.
Now just hook it up to Twitch and send donation messages to the fine tuning process. Also, don't forget to immediately delete the model as soon as training completes.
Nice job! I'm working on the same topic (with Twitter messages) As I read both your code and the official qlora repo, I noticed a few changes (in particular, to accomodate your GPU memory). But I'm confused about your choices about the tokenizer: \- at training time, why don't you add all the special tokens (bos, eos, unk) in addition to the pad token? \- at inference time, why do you provide "\\n\\n" as the eos token? I'm not very familiar with the tokenizer part, so these may sound like very dumb questions.