1 Star 0 Fork 0

ZTMZ Club/ztmz_strange_scripts

加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
该仓库未声明开源许可证文件(LICENSE),使用请关注具体项目描述及其代码上游依赖。
克隆/下载
convert_pacenote_csv_to_db.py 1.55 KB
一键复制 编辑 原始数据 按行查看 历史
strawhat 提交于 2024-06-06 10:51 . emmm
import sqlite3
import pandas as pd
from argparse import ArgumentParser
from tqdm.rich import tqdm
from loguru import logger
def short_id(id: str):
new_id = id.removeprefix('detail_')
new_id = new_id.removeprefix('corner_')
new_id = new_id.removeprefix('number_')
return new_id
def main():
p = ArgumentParser()
p.add_argument('--path', type=str, help='path to pacenotes.csv')
p.add_argument('--db', type=str, help='path to db')
args = vars(p.parse_args())
df = pd.read_csv(args['path']) # pacenotes.csv
df.sort_values(by='Id', inplace=True)
df.reset_index(drop=True, inplace=True)
data_pacenote = []
data_pacenote_filenames_primary = []
data_pacenote_filenames = []
rows = list(df.iterrows())
for i, row in tqdm(rows):
data_pacenote.append((i, row['Description']))
data_pacenote_filenames_primary.append((i, short_id(row['Id']), True))
data_pacenote_filenames.append((i, row['Id'], False))
logger.info(f'got {len(data_pacenote)} pacenotes and {len(data_pacenote_filenames)} filenames')
conn = sqlite3.connect(args['db'])
c = conn.cursor()
c.executemany('INSERT INTO pacenote(`id`, `description`) VALUES (?, ?)', data_pacenote)
c.executemany('INSERT INTO pacenote_filenames(`id`, `filename`, `is_primary`) VALUES (?, ?, ?)', data_pacenote_filenames_primary)
c.executemany('INSERT INTO pacenote_filenames(`id`, `filename`, `is_primary`) VALUES (?, ?, ?)', data_pacenote_filenames)
conn.commit()
logger.info('done')
if __name__ == '__main__':
main()
马建仓 AI 助手
尝试更多
代码解读
代码找茬
代码优化
1
https://gitee.com/ztmz/ztmz_strange_scripts.git
git@gitee.com:ztmz/ztmz_strange_scripts.git
ztmz
ztmz_strange_scripts
ztmz_strange_scripts
master

搜索帮助

23e8dbc6 1850385 7e0993f3 1850385