Facebook's Llama got leaked

B

BroTherm

Junior Member
Joined
Sep 29, 2022
Messages
112
Reaction score
45
Facebook's new "ground breaking" Llama family models got leaked on ***** and a magnet link got added to the git repo by somebody not affiliated to Facebook.

HuggingFace support is already available.

Llama info from fb: https://ai.facebook.com/blog/large-language-model-llama-meta-ai/

I am still doing some testing so can't really comment on the quality of the text. Currently at 13b and going up untill I run OOM.
 
Advertise on BHW
The other two options work.
Seems like they took down the magnet in the mean time.
 
I think it started in ***** and spread.
Hype? maybe... No reason to do it though as this was not supposed to go into mass public.... maybe
No idea what their intentions were, but as it is considerably smaller than let's say OPT-175B, it is easier to run at home.
 
kalpana007 said:
KoWho leaked it? Are they creating hype for it?
Click to expand...
Some Facebook employee put an internal torrent link on the internet and then the internet did it's thing.

V said:
Do you have any idea how to run this model locally? :confused: I am not so experienced with this stuff.
Click to expand...
You will need a hefty good machine to run the models. I am renting a 3090 on vast with 24gb VRAM and 32gb of Ram with a 50gb swap to run the 13B model. I am running the inference with int8 quantization from a fork here.
https://github.com/tloen/llama-int8Just download the models, setup an environment, clone that repo and run
python example.py --ckpt_dir [TARGET_DIR]/13B --tokenizer_path [TARGET_DIR]/tokenizer.model --max_batch_size=1

Simpler instructions here https://rentry.org/llama-tard-v2
 
  • Like
Reactions: V
The one putting it in the git repo is not a Facebook employee.

I am running it locally on a not so hefty machine.


georgetoons said:
What is this? How can this help sorry never heard of this before?
Click to expand...

AI model for text generation like GPT-3. Not trained for instructions like davinci or ChatGPT though.
 
BroTherm said:
The one putting it in the git repo is not a Facebook employee.
Click to expand...
The initial model torrent got leaked on chan and then they started distributing it. There are plenty of forks now.


BroTherm said:
I am running it locally on a not so hefty machine.
Click to expand...
Depends on what you mean by "hefty" ig :D. I am running the 13B model on 8bit quantization on a 3090 and 32gigs ram. Maybe that's not "very hefty" these days :^)

The model seems to be optimized asf. I am seeing people who are running the 66B model on a single A100 card with some parameter tuning and it's giving results close to gpt-3-davinci-166b.
 
Yep :) 3090 ... no so hefty nowadays. 32GBs of RAM is the new norm on a desktop.
On 13b I got a readable on topic generation. Just a bit low on length.
 
I've playing around with this the last couple of days and none of the generated text comes as AI in any of the AI detecting sites online.
I know those are not very accurate, but most of the copy/paste simple prompt ChatGPT texts still come out positive.
 
How good is it compared to the current local text generation ais?
 
I only tested up to 30b as this is the most I can run at the moment, but it is quite better. Ofc this is subjectively my opinion.
It is not completely production ready as it needs some cleaning after a generation and it is not an instruct model like ChatGPT.
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top