【新智元导读】一经发布,地表最强开源模型Falcon 180B直接霸榜HF。3.5万亿token训练,性能直接碾压Llama 2。
世界最强开源大模型
Falcon 180B上手可用
Prompt 格式
System: Add an optional system prompt here
User: This is the user input
Falcon: This is what the model generates
User: This might be a second turn input
Falcon: and so on
Transformers
pip install --upgrade transformers
huggingface-cli login
from transformers import AutoTokenizer, AutoModelForCausalLM
import transformers
import torch
model_id = "tiiuae/falcon-180B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
prompt = "My name is Pedro, I live in"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
output = model.generate(
input_ids=inputs["input_ids"],
attention_mask=inputs["attention_mask"],
do_sample=True,
temperature=0.6,
top_p=0.9,
max_new_tokens=50,
)
output = output[0].to("cpu")
print(tokenizer.decode(output)
My name is Pedro, I live in Portugal and I am 25 years old. I am a graphic designer, but I am also passionate about photography and video.
I love to travel and I am always looking for new adventures. I love to meet new people and explore new places.
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
**load_in_8bit=True,**
device_map="auto",
)
def format_prompt(message, history, system_prompt):
prompt = ""
if system_prompt:
prompt += f"System: {system_prompt}n"
for user_prompt, bot_response in history:
prompt += f"User: {user_prompt}n"
prompt += f"Falcon: {bot_response}n"
prompt += f"User: {message}nFalcon:"
return prompt
网友热评
https://twitter.com/TIIuae/status/1699380904404103245
https://twitter.com/DrJimFan/status/1699459647592403236
https://huggingface.co/blog/zh/falcon-180b
https://huggingface.co/tiiuae/falcon-180B