2 Star 0 Fork 1

RudonOpenSource/llama

加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
文件
克隆/下载
download.sh 1.29 KB
一键复制 编辑 原始数据 按行查看 历史
Timothee Lacroix 提交于 2023-02-24 05:20 . Initial commit
# Copyright (c) Meta Platforms, Inc. and affiliates.
# This software may be used and distributed according to the terms of the GNU General Public License version 3.
PRESIGNED_URL="" # replace with presigned url from email
MODEL_SIZE="7B,13B,30B,65B" # edit this list with the model sizes you wish to download
TARGET_FOLDER="" # where all files should end up
declare -A N_SHARD_DICT
N_SHARD_DICT["7B"]="0"
N_SHARD_DICT["13B"]="1"
N_SHARD_DICT["30B"]="3"
N_SHARD_DICT["65B"]="7"
echo "Downloading tokenizer"
wget ${PRESIGNED_URL/'*'/"tokenizer.model"} -O ${TARGET_FOLDER}"/tokenizer.model"
wget ${PRESIGNED_URL/'*'/"tokenizer_checklist.chk"} -O ${TARGET_FOLDER}"/tokenizer_checklist.chk"
(cd ${TARGET_FOLDER} && md5sum -c tokenizer_checklist.chk)
for i in ${MODEL_SIZE//,/ }
do
echo "Downloading ${i}"
mkdir -p ${TARGET_FOLDER}"/${i}"
for s in $(seq -f "0%g" 0 ${N_SHARD_DICT[$i]})
do
wget ${PRESIGNED_URL/'*'/"${i}/consolidated.${s}.pth"} -O ${TARGET_FOLDER}"/${i}/consolidated.${s}.pth"
done
wget ${PRESIGNED_URL/'*'/"${i}/params.json"} -O ${TARGET_FOLDER}"/${i}/params.json"
wget ${PRESIGNED_URL/'*'/"${i}/checklist.chk"} -O ${TARGET_FOLDER}"/${i}/checklist.chk"
echo "Checking checksums"
(cd ${TARGET_FOLDER}"/${i}" && md5sum -c checklist.chk)
done
马建仓 AI 助手
尝试更多
代码解读
代码找茬
代码优化
1
https://gitee.com/RudonOpenSource/llama.git
git@gitee.com:RudonOpenSource/llama.git
RudonOpenSource
llama
llama
llama_v1

搜索帮助

0d507c66 1850385 C8b1a773 1850385