1 Star 2 Fork 0

Yinqi Bai/ crawler GDP

加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
文件
该仓库未声明开源许可证文件(LICENSE),使用请关注具体项目描述及其代码上游依赖。
克隆/下载
utils.py 20.79 KB
一键复制 编辑 原始数据 按行查看 历史
Yinqi Bai 提交于 2022-10-15 23:38 . 函数文件
import requests
from lxml import etree
country = {#ASIA
'中国': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/chn.html',
'香港': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/hkg.html',
'澳门': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/mac.html',
'新加坡': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/sgp.html',
'日本': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/jpn.html',
'泰国': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/tha.html',
'菲律宾': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/phl.html',
'韩国': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/kor.html',
'马来西亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/mys.html',
'越南': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/vnm.html',
'印尼': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/ina.html',
'柬埔寨': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/khm.html',
'老挝': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/lao.html',
'缅甸': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/mmr.html',
'巴基斯坦': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/pak.html',
'斯里兰卡': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/lka.html',
'尼泊尔': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/npl.html',
'印度': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/ind.html',
'马尔代夫': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/mdv.html',
'蒙古': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/mng.html',
'孟加拉': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/bgd.html',
'约旦': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/jor.html',
'巴勒斯坦': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/sau.html',
'沙特阿拉伯': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/pse.html',
'黎巴嫩': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/lbn.html',
'也门': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/yem.html',
'叙利亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/syr.html',
'哈萨克斯坦': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/kaz.html',
'土库曼斯坦': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/tkm.html',
'格鲁吉亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/geo.html',
'阿富汗': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/afg.html',
'塔吉克斯坦': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/tjk.html',
'不丹': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/btn.html',
'以色列': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/isr.html',
'东帝汶': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/tls.html',
'阿曼': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/omn.html',
'文莱': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/brn.html',
'卡塔尔': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/qat.html',
'阿塞拜疆': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/aze.html',
'乌兹别克斯坦': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/uzb.html',
'科威特': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/kwt.html',
'伊拉克': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/irq.html',
'伊朗': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/irn.html',
'吉尔吉斯斯坦': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/kgz.html',
'阿联酋': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/are.html',
'巴林': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/bhr.html',
'亚美尼亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/arm.html',
#Europe
'英国': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/gbr.html',
'德国': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/deu.html',
'法国': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/fra.html',
'比利时': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/bel.html',
'意大利': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/ita.html',
'西班牙': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/esp.html',
'葡萄牙': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/prt.html',
'瑞士': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/che.html',
'冰岛': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/isl.html',
'奥地利': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/aut.html',
'捷克': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/cze.html',
'匈牙利': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/hun.html',
'波兰': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/pol.html',
'希腊': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/grc.html',
'俄罗斯': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/rus.html',
'土耳其': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/tur.html',
'保加利亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/bgr.html',
'罗马尼亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/rou.html',
'斯洛文尼亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/svn.html',
'克罗地亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/hrv.html',
'圣马力诺': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/smr.html',
'芬兰': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/fin.html',
'列支敦士登': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/lie.html',
'马其他': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/mlt.html',
'荷兰': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/nld.html',
'黑山': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/mne.html',
'白俄罗斯': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/blr.html',
'安道尔': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/and.html',
'拉脱维亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/lva.html',
'挪威': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/nor.html',
'卢森堡': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/lux.html',
'马其顿': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/mkd.html',
'摩尔多亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/mda.html',
'立陶宛': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/ltu.html',
'法罗群岛': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/fro.html',
'丹麦': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/dnk.html',
'马恩岛': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/imn.html',
'阿尔巴尼亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/alb.html',
'塞尔维亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/srb.html',
'乌克兰': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/ukr.html',
'斯洛伐克': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/svk.html',
'塞浦路斯': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/cyp.html',
'波黑': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/bih.html',
'爱沙尼亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/est.html',
'爱尔兰': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/irl.html',
'摩纳哥': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/mco.html',
'瑞典': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/swe.html',
#Americas
'美国': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/usa.html',
'加拿大': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/can.html',
'委内瑞拉': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/ven.html',
'玻利维亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/bol.html',
'波多黎各': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/pri.html',
'荷属圣马丁': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/xsm.html',
'尼加拉瓜': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/nic.html',
'巴拉圭': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/pry.html',
'智利': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/chl.html',
'圣基茨和尼维斯': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/kna.html',
'牙买加': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/jam.html',
'乌拉圭': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/ury.html',
'法属圣马丁': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/maf.html',
'科斯群岛': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/tca.html',
'多米尼克': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/dma.html',
'圭亚那': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/guy.html',
'哥斯达黎加': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/cri.html',
'圣文森特和格林纳丁斯': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/vct.html',
'多巴哥': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/tto.html',
'百慕大': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/bmu.html',
'伯利兹': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/blz.html',
'格林纳达': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/grd.html',
'格陵兰': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/grl.html',
'海地': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/hti.html',
'墨西哥': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/mex.html',
'圣卢西亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/ica.html',
'安提瓜和巴布达': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/atg.html',
'洪都拉斯': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/hnd.html',
'多米尼加': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/dom.html',
'秘鲁': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/per.html',
'苏里南': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/sur.html',
'哥伦比亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/col.html',
'厄瓜多尔': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/ecu.html',
'库拉索': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/cuw.html',
'危地马拉': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/gtm.html',
'巴巴多斯': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/brb.html',
'巴西': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/bra.html',
'开曼群岛': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/cym.html',
'萨尔瓦多': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/slv.html',
'阿鲁巴': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/abw.html',
'巴拿马': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/pan.html',
'阿根廷': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/arg.html',
'巴哈马': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/bhs.html',
'古巴': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/cub.html',
#oceania
'澳大利亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/aus.html',
'新西兰': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/nzl.html',
'基里巴斯': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/kir.html',
'法属波利尼西亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/pyf.html',
'关岛': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/gum.html',
'北马里亚纳群岛': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/mnp.html',
'马绍尔群岛': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/mhl.html',
'斐济群岛': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/fji.html',
'密克罗尼西亚联邦': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/fsm.html',
'瓦努阿图': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/vut.html',
'美属萨摩亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/asm.html',
'帕劳': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/plw.html',
'萨摩亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/wsm.html',
'图瓦卢': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/tuv.html',
'汤加': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/ton.html',
'巴布亚新几内亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/png.html',
'所罗门群岛': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/slb.html',
'新喀里多尼亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/ncl.html',
'瑙鲁': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/nru.html',
#african
'南非': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/zaf.html',
'埃及': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/egy.html',
'安哥拉': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/ago.html',
'阿尔及利亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/dza.html',
'尼日尔': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/ner.html',
'纳米比亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/nam.html',
'加纳': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/gha.html',
'佛得角': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/cpv.html',
'利比里亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/lbr.html',
'毛里求斯': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/mus.html',
'利比亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/lby.html',
'坦桑尼亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/tza.html',
'博茨瓦纳': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/bwa.html',
'布基纳法索': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/bfa.html',
'毛里塔尼亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/mrt.html',
'塞拉利昂': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/sle.html',
'吉布提': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/djl.html',
'几内亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/gin.html',
'塞舌尔': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/syc.html',
'刚果(金)': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/cod.html',
'贝宁': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/ben.html',
'尼日利亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/nga.html',
'多哥': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/tgo.html',
'喀麦隆': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/cmr.html',
'摩洛哥': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/mar.html',
'乍得': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/tcd.html',
'布隆迪': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/bdi.html',
'莱索托': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/iso.html',
'莫桑比克': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/moz.html',
'津巴布韦': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/zwe.html',
'刚果(布)': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/cog.html',
'埃塞俄比亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/eth.html',
'科摩罗': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/com.html',
'圣多美和普林西比': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/stp.html',
'马达加斯加': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/mdg.html',
'塞内加尔': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/sen.html',
'卢旺达': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/nwa.html',
'赞比亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/zmb.html',
'冈比亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/gmb.html',
'南苏丹': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/ssd.html',
'肯尼亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/ken.html',
'突尼斯': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/tun.html',
'科特迪瓦': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/clv.html',
'赤道几内亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/gnq.html',
'几内亚比绍': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/gnb.html',
'斯威士兰': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/swz.html',
'中非': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/cat.html',
'索马里': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/som.html',
'厄立特里亚': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/eri.html',
'加蓬': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/gab.html',
'马拉维': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/mwi.html',
'马里': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/mli.html',
'苏丹': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/sdn.html',
'乌干达': 'https://www.kylc.com/stats/global/yearly_per_country/g_gdp/uga.html',
}
class parse(object):
def __init__(self, html, data_list):
self.html = html
self.data_list = data_list
def parse_data(self):
tr_list = self.html.xpath('//div[@class="table-responsive"]/table/tbody/tr')
for tr in tr_list:
data = tr.xpath('./td/text()')
if len(data) == 3:
year = data[0]
gdp = data[1]
rate = data[2]
item = {
'year': year,
'GDP': gdp,
'rate': rate
}
self.data_list.append(item)
马建仓 AI 助手
尝试更多
代码解读
代码找茬
代码优化
1
https://gitee.com/yinqi-bai/crawler-gdp.git
git@gitee.com:yinqi-bai/crawler-gdp.git
yinqi-bai
crawler-gdp
crawler GDP
master

搜索帮助

0d507c66 1850385 C8b1a773 1850385