1 Star 1 Fork 1

李思远/Llama3.1-70B-Chinese-Chat

加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
该仓库未声明开源许可证文件(LICENSE),使用请关注具体项目描述及其代码上游依赖。
克隆/下载
config.json 921 Bytes
一键复制 编辑 原始数据 按行查看 历史
Shenzhi Wang 提交于 2024-07-25 04:10 . Upload folder using huggingface_hub
{
"_name_or_path": "shenzhi-wang/Llama3.1-70B-Chinese-Chat",
"architectures": [
"LlamaForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"bos_token_id": 128000,
"eos_token_id": [
128001,
128008,
128009
],
"hidden_act": "silu",
"hidden_size": 8192,
"initializer_range": 0.02,
"intermediate_size": 28672,
"max_position_embeddings": 131072,
"mlp_bias": false,
"model_type": "llama",
"num_attention_heads": 64,
"num_hidden_layers": 80,
"num_key_value_heads": 8,
"pretraining_tp": 1,
"rms_norm_eps": 1e-05,
"rope_scaling": {
"factor": 8.0,
"high_freq_factor": 4.0,
"low_freq_factor": 1.0,
"original_max_position_embeddings": 8192,
"rope_type": "llama3"
},
"rope_theta": 500000.0,
"tie_word_embeddings": false,
"torch_dtype": "bfloat16",
"transformers_version": "4.44.0.dev0",
"use_cache": true,
"vocab_size": 128256
}
马建仓 AI 助手
尝试更多
代码解读
代码找茬
代码优化
1
https://gitee.com/lsy1213/Llama3.1-70B-Chinese-Chat.git
git@gitee.com:lsy1213/Llama3.1-70B-Chinese-Chat.git
lsy1213
Llama3.1-70B-Chinese-Chat
Llama3.1-70B-Chinese-Chat
main

搜索帮助