Have your own LLM ai model like chat gpt, run locally on your pc, and write NSFW content

"Listen up, partner! Y'all been talkin' 'bout all them fancy models like ChatGPT, Bing, and GPT-4, but it seems y'all forgot about the open-source models you can run on your own dang PC! Yep, you heard it right, amigo. These models are free as a bird, and you can download 'em to your heart's content.

Now, let me tell ya, some of these models ain't afraid to tackle the wild stuff. They're uncensored, which means you can ask 'em anything, even about NSFW and adult content. And that ain't all, partner! They even got models with a limit of 65k tokens so you can go all out!

You wanna know the best part? You can fine-tune these models or make your very own. The possibilities are endless, my friend. You can unleash your creativity and train 'em to do all sorts of tricks.

Now, I reckon you're curious 'bout how to run these models. Well, I'll tell ya, it's as simple as can be. Just follow a straightforward method, and even though it might not have all the fancy features like in oobabooga WebUi, it gets the job done, partner." this Part is written by A.I.

GPT4All is the easiest to install and min resource requirement since this one run on CPU, but it lacks support for the newer model.
To run a newer model go with WebUi; it's more complicated but worth it.
download from here
https://github.com/nomic-ai/gpt4all

Install it


Choose your model



Enjoy

thanks for sharing!

does it have API? Or can I host it on my own website?

jadecoder said:

does it have API? Or can I host it on my own website?

Click to expand...

for gpt4all? it have api, you can run it on vps, than use api for your website to interact with it
but if you need cuztomized one i suggest you use web ui, to run it on server.

Is that slow? I treid but it's painstaking slow, dropping one word by second. I have a decent laptop, but wonder if it's this way...

How we can use this content?

tex42 said:

Is that slow? I treid but it's painstaking slow, dropping one word by second. I have a decent laptop, but wonder if it's this way...

Click to expand...

Depend on your spec, this one is using CPU so yeah kinda slow, but it allow most PC to run it,
it will be faster with GPU but you need good GPU with min 8gb vram the more the better.
i do remind you that this is running the ai on your PC, even fine tuning it will require 40gb vram can be less or more depending on the model, so it have diff decent standard than average use.

Deleted member 832639 said:

How we can use this content?

Click to expand...

i cannot answer that, since it all depend on your need and circumstance.
example you need to generate a adult story, than this option enable you to do that, it will never be allowed on subcription based service. there are ton more example, but it all basically return to you

Nice, but these CPU tied models will always kinda suck. Although if you're using them to generate bulk content then even if it's only doing 1 word a second, that's still like 84, 1000 word articles a day you could make. Another downside is those articles still won't be that good but could play a role in generating some content to mix-in with GPT generated content to throw off ML generated text fingerprinting.

If you want to run larger, more capable models, just use a cloud GPU processing service like https://www.runpod.io, you can get access to GPUs with 80GB of VRAM for a pretty reasonable price.

what gpt model would this be comparable to? and would a 3060 be able to generate a good amount of content in a short period of time?

BlackHatSoda said:

Nice, but these CPU tied models will always kinda suck. Although if you're using them to generate bulk content then even if it's only doing 1 word a second, that's still like 84, 1000 word articles a day you could make. Another downside is those articles still won't be that good but could play a role in generating some content to mix-in with GPT generated content to throw off ML generated text fingerprinting.

If you want to run larger, more capable models, just use a cloud GPU processing service like https://www.runpod.io, you can get access to GPUs with 80GB of VRAM for a pretty reasonable price.

Click to expand...

what would be the purpose of making all those articles? just curious.

BlackHatSoda said:

Nice, but these CPU tied models will always kinda suck. Although if you're using them to generate bulk content then even if it's only doing 1 word a second, that's still like 84, 1000 word articles a day you could make. Another downside is those articles still won't be that good but could play a role in generating some content to mix-in with GPT generated content to throw off ML generated text fingerprinting.

If you want to run larger, more capable models, just use a cloud GPU processing service like https://www.runpod.io, you can get access to GPUs with 80GB of VRAM for a pretty reasonable price.

Click to expand...

yes run pod io is good suggestion it's also not that expensive, and there is also google collab for a free solution. i agree GPU is better, but i share the CPU one , cause this the most easiest to run, and can run in a lot of cheap budget PC too , albeit slow. still a good way to entry into open source AI. more people use open source = more growth for it. we need more open source ai model

with a good GPU you can use storytellers model that prove 63k token per generate, compared to gpt 4 around 8k token. and yes as always it will require tuning the article more, same like chatgpt and other premium ai in my opinion,
a copy and paste article from any ai atm, will not be good enough to rank.

mkosag08 said:

what gpt model would this be comparable to? and would a 3060 be able to generate a good amount of content in a short period of time?

Click to expand...

a 3060 is decent enough to run most model, as for how much content and how comparable it to gpt model. it return back to what ai model you use.
there are ton of model you can choose some even have 90% capability of chatgpt. some focus on making story, roleplay, guide, other for uncensored and no morale content. you can easily google this model and pick choose the one suitable to your need.btw the good model will require you run llama directly, or oobabooga WebUi

Elvlin said:

a copy and paste article from any ai atm, will not be good enough to rank.

Click to expand...

That's what's I've found playing with GPT4All. Even the top models available are just so-so and they're trained on GPT4 output so they're likely to still be fingerprinted as ChatGPT if you're trying to avoid that.

But it is good for avoiding OpenAI's draconian censorship which is always a good thing.

with a good GPU you can use storytellers model that prove 63k token per generate

Click to expand...

Where is that an option? I didn't see that in GPT4All.

there are ton of model you can choose some even have 90% capability of chatgpt

Yes, but keep in mind that those scores are based on benchmarks so don't really tell you how well one model compares to another in the real world as the benchmarks only sample a limited amount of the capabilities and also models can be trained to do better on the benchmarks.

A smaller, highly optimized model trained on the best, hand picked training content can produce better quality output than a much larger model but a much larger model, especially on the scale of GPT4, is going to be able to cover a lot more topics.

So if you're looking for a specialized model, like you mentioned, focused on making a story or a role playing game then something like this can be as good as GPT4. However, if you want a general purpose model that can handle a much wider range of prompts, these small models just don't contain enough data to compare with a model that's 1000 times larger.

But the near term future of LLMs is to have a lot of smaller, better trained, specialized models working together anyhow. Making massive single models has already reached the limits of what they can do.

It won't take long before the multimodal approach reaches the same limits. Then we will need a fundamentally different approach to the architecture. Trying to keep thousands of highly optimized, specialized models up to date will end up taking more work than training one large model, even if the results are better.

But multimodal systems will have the ability to run on lower spec hardware as they can swap out the models needed in and out of VRAM on demand, as opposed to a single large model where the entire thing has to be loaded at all times.

Similar to how large open world games can stream in assets as you move around so they don't have to be limited by the VRAM.

BlackHatSoda said:

That's what's I've found playing with GPT4All. Even the top models available are just so-so and they're trained on GPT4 output so they're likely to still be fingerprinted as ChatGPT if you're trying to avoid that.

But it is good for avoiding OpenAI's draconian censorship which is always a good thing.



Where is that an option? I didn't see that in GPT4All.

there are ton of model you can choose some even have 90% capability of chatgpt

Yes, but keep in mind that those scores are based on benchmarks so don't really tell you how well one model compares to another in the real world as the benchmarks only sample a limited amount of the capabilities and also models can be trained to do better on the benchmarks.

A smaller, highly optimized model trained on the best, hand picked training content can produce better quality output than a much larger model but a much larger model, especially on the scale of GPT4, is going to be able to cover a lot more topics.

So if you're looking for a specialized model, like you mentioned, focused on making a story or a role playing game then something like this can be as good as GPT4. However, if you want a general purpose model that can handle a much wider range of prompts, these small models just don't contain enough data to compare with a model that's 1000 times larger.

But the near term future of LLMs is to have a lot of smaller, better trained, specialized models working together anyhow. Making massive single models has already reached the limits of what they can do.

It won't take long before the multimodal approach reaches the same limits. Then we will need a fundamentally different approach to the architecture. Trying to keep thousands of highly optimized, specialized models up to date will end up taking more work than training one large model, even if the results are better.

But multimodal systems will have the ability to run on lower spec hardware as they can swap out the models needed in and out of VRAM on demand, as opposed to a single large model where the entire thing has to be loaded at all times.

Similar to how large open world games can stream in assets as you move around so they don't have to be limited by the VRAM.

Click to expand...

yup for 63k token, you need to run oobabooga web ui, i suggest you rent some vps on runpod and install it there, it need a lot of vram, like hundred or more
you can find the model here https://huggingface.co/spaces/mosaicml/mpt-7b-storywriter.

well keeping up thousand model won't be a problem as we are talking open source model, where every model have diff team handle it, and everyone in the world can contribute into it.
no need to worry, the model will become more efficient ,at the same time tech will keep growing, it won't take long till even phone can run this locally ,and used everywhere.
just see in the past a few year ago 16gb ram is high end., now aday 16gb is like average minimum.

Elvlin said:

yup for 63k token, you need to run oobabooga web ui, i suggest you rent some vps on runpod and install it there, it need a lot of vram, like hundred or more
you can find the model here https://huggingface.co/spaces/mosaicml/mpt-7b-storywriter.

well keeping up thousand model won't be a problem as we are talking open source model, where every model have diff team handle it, and everyone in the world can contribute into it.
no need to worry, the model will become more efficient ,at the same time tech will keep growing, it won't take long till even phone can run this locally ,and used everywhere.
just see in the past a few year ago 16gb ram is high end., now aday 16gb is like average minimum.

Click to expand...

I guess being a smaller model it can handle a larger token context so that makes sense.

Yes, hardware will get cheaper but for the foreseeable future, you'll still need qualified human labor to currate optimal content for training and do good feedback training. OpenAI uses cheap labor for that and that's why their models are sub-optimal.

I wouldn't rely on open source guaranteeing quality. Just go on huggingface and see how nearly all the models on there suck lol. Or the countless open source projects in general that someone makes then gets bored of and abandons.

It's one thing to make the model but as new info is always emerging, they have to be constantly retrained on new data which has to be collected by people with expertise in the specialty the model is trained for.

Basically we need a fundamentally different approach to the whole thing. GPT is old tech, based on outdated NLP and neural network techniques developed last century. We need neural networks which accurately simulate biological neurons and synapses and can dynamically change how neurons operate, which synaptic pathways they have to other neurons, and strengthen or weaken those pathways.

Basically, we need real AI and not ML pretending to be AI. ML is just an automated way to do a lot of statistical computation on lots of data and package it into a vector database or digital neural network (which doesn't operate at all like a biological one).

That's why I think GPT is way overhyped. It's like when VR first came out and everyone was like, "This is going to change everything", but it didn't. Then it came back when the hardware got better and everyone was like, "This is going to change everything", but it didn't.

We need a lot more hardware advancements and breakthroughs in understanding how human neural networks operate so they can be reverse engineered.

Don't get me wrong, ML will lead to a lot of jobs disappearing, but it won't lead to Skynet.

ML is like a different evolutionary branch. It's like how humans and monkeys share an evolutionary ancestor but monkeys will never evolve into humans.

Thanks for sharing

Would love to hear more about your views on ML and the opportunities it can avail to us in the bhw community.

BlackHatSoda said:

I guess being a smaller model it can handle a larger token context so that makes sense.

Yes, hardware will get cheaper but for the foreseeable future, you'll still need qualified human labor to currate optimal content for training and do good feedback training. OpenAI uses cheap labor for that and that's why their models are sub-optimal.

I wouldn't rely on open source guaranteeing quality. Just go on huggingface and see how nearly all the models on there suck lol. Or the countless open source projects in general that someone makes then gets bored of and abandons.

It's one thing to make the model but as new info is always emerging, they have to be constantly retrained on new data which has to be collected by people with expertise in the specialty the model is trained for.

Basically we need a fundamentally different approach to the whole thing. GPT is old tech, based on outdated NLP and neural network techniques developed last century. We need neural networks which accurately simulate biological neurons and synapses and can dynamically change how neurons operate, which synaptic pathways they have to other neurons, and strengthen or weaken those pathways.

Basically, we need real AI and not ML pretending to be AI. ML is just an automated way to do a lot of statistical computation on lots of data and package it into a vector database or digital neural network (which doesn't operate at all like a biological one).

That's why I think GPT is way overhyped. It's like when VR first came out and everyone was like, "This is going to change everything", but it didn't. Then it came back when the hardware got better and everyone was like, "This is going to change everything", but it didn't.

We need a lot more hardware advancements and breakthroughs in understanding how human neural networks operate so they can be reverse engineered.

Don't get me wrong, ML will lead to a lot of jobs disappearing, but it won't lead to Skynet.

ML is like a different evolutionary branch. It's like how humans and monkeys share an evolutionary ancestor but monkeys will never evolve into humans.

Click to expand...

Yup, ai is just a tool; most people watch too many movies thinking this is like ai in Terminator LOL
the fear is overhyped as a traffic source. Yes, some even earn money making crowdfunding movements to reject AI.
Well,, i don't know which model you try, as the one i have tried so far is good. Also they share it for free, but it costs them money to make.
We certainly have diff views on Open source and community projects. There are open project hence the need to test and find out which one is good, compared to one made by big company that is RnD, before release, but it comes with its cons, censorship, limited feature, priority to specific group, list goes on and don't ever hope to see their source code, their product, their Rules.

pj777777 said:

Thanks for sharing

Would love to hear more about your views on ML and the opportunities it can avail to us in the bhw community.

Click to expand...

you're welcome, well ML certainly can be used in any workflow to cut the time required and make it more effective and efficient.

Elvlin said:

Yup, ai is just a tool; most people watch too many movies thinking this is like ai in Terminator LOL
the fear is overhyped as a traffic source. Yes, some even earn money making crowdfunding movements to reject AI.
Well,, i don't know which model you try, as the one i have tried so far is good. Also they share it for free, but it costs them money to make.
We certainly have diff views on Open source and community projects. There are open project hence the need to test and find out which one is good, compared to one made by big company that is RnD, before release, but it comes with its cons, censorship, limited feature, priority to specific group, list goes on and don't ever hope to see their source code, their product, their Rules.

you're welcome, well ML certainly can be used in any workflow to cut the time required and make it more effective and efficient.

Click to expand...

Thanks and yep, I use ML extensively atm, but feel I'm missing a trick with regards to automation.

One area that I've been playing with, but still needs manual triggers, is to scan text based niche updates and convert these to video snippets.

pj777777 said:

Thanks and yep, I use ML extensively atm, but feel I'm missing a trick with regards to automation.

One area that I've been playing with, but still needs manual triggers, is to scan text based niche updates and convert these to video snippets.

Click to expand...

if you need the video to be original, than probably use Stable difussion to generate image based on prompt than make it into video.
but honestly current tool is justs not good enough yet for making video, maybe it's better to just use python to make bot for scarping people video and mix it up

Elvlin said:

if you need the video to be original, than probably use Stable difussion to generate image based on prompt than make it into video.
but honestly current tool is justs not good enough yet for making video, maybe it's better to just use python to make bot for scarping people video and mix it up

Click to expand...

Thanks,

I'm using a combo of gpt4 and a lumen 5 type product (name escapes me atm).

Just about acceptable as a mvp, but not sustainable imo.

I'll put some feelers out for bot creators.

Elvlin said:

Yup, ai is just a tool; most people watch too many movies thinking this is like ai in Terminator LOL
the fear is overhyped as a traffic source. Yes, some even earn money making crowdfunding movements to reject AI.
Well,, i don't know which model you try, as the one i have tried so far is good. Also they share it for free, but it costs them money to make.
We certainly have diff views on Open source and community projects. There are open project hence the need to test and find out which one is good, compared to one made by big company that is RnD, before release, but it comes with its cons, censorship, limited feature, priority to specific group, list goes on and don't ever hope to see their source code, their product, their Rules.

Click to expand...

I think open source projects will play a big role at first. Like right now there's so much hype behind generative ML models that there's a lot of interest in making open source models and tools around them.

But the hype will die down as it always does, the best tech will be locked up in for-profit companies or for use by government agencies. You'll get a random research paper here and there that demonstrates a jump forward and people will step up to make some open source versions of it but I still see the best stuff being locked away. Companies don't like to spend billions on tech then just give it away unless it's the weak sauce version or some kind of loss leader to lure people into paying for their paid options.

But real AGI is going to take fundamentally different hardware, possibly a hybrid quantum/digital computer and that's just not something your average Joe can make at home. I don't think even field programable gate arrays will get flexible, fast, and cheap enough to mimic simulating biological nerual networks at a speed that's usable.

There have been attempts at simulating simplified models of biological neural networks at large scale in the past and they ran so slow that even on massive super computers, they ran at 100-1000 times slower than a biological network. And these were very simplified models based on a primitive understanding of how real neural networks operate.

So it will be interesting to see how this progresses. I think there's countless uses for advanced, specialized ML models, and they'll lead to a lot more lost jobs than they create. Merged with cheap, advanced robotics and you've got yourself the ability to replace blue collar workers at scale as well as create armies and police forces that won't question orders.

Honestly, you could have the robotics now if we had the technology for sufficient energy storage solutions. Even the best battery tech still in the lab (which will probably never leave the lab) doesn't have the energy density you'd need, which is why Boston Dynamics uses a gas powered engine to run a generator for their robots designed for sustained use.

But once someone figures out how to get the energy density of at least gasoline into a rechargeable storage device which doesn't require some middle man engine or turbine, things will get a lot like Terminator.

Except it won't be some singularity AGI controlling the robots, it'll be shitty CEOs and dictators.