1800亿参数，世界顶级开源大模型Falcon官宣！碾压LLaMA 2，性能直逼GPT-4

System: Add an optional system prompt hereUser: This is the user inputFalcon: This is what the model generatesUser: This might be a second turn inputFalcon: and so on

pip install --upgrade transformershuggingface-cli login

from transformers import AutoTokenizer, AutoModelForCausalLMimport transformersimport torch
model_id = "tiiuae/falcon-180B"
tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(    model_id,    torch_dtype=torch.bfloat16,    device_map="auto",)
prompt = "My name is Pedro, I live in"inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
output = model.generate(    input_ids=inputs["input_ids"],    attention_mask=inputs["attention_mask"],    do_sample=True,    temperature=0.6,    top_p=0.9,    max_new_tokens=50,)output = output[0].to("cpu")print(tokenizer.decode(output)

My name is Pedro, I live in Portugal and I am 25 years old. I am a graphic designer, but I am also passionate about photography and video.I love to travel and I am always looking for new adventures. I love to meet new people and explore new places.

model = AutoModelForCausalLM.from_pretrained(    model_id,    torch_dtype=torch.bfloat16,    **load_in_8bit=True,**    device_map="auto",)

def format_prompt(message, history, system_prompt):    prompt = ""    if system_prompt:        prompt += f"System: {system_prompt}n"    for user_prompt, bot_response in history:        prompt += f"User: {user_prompt}n"        prompt += f"Falcon: {bot_response}n"        prompt += f"User: {message}nFalcon:"    return prompt