简介由于工作需要，做了一个公司前端的小工具，使用python语言，爬取搜狗微信的微信文章，附上搜狗微信官网搜狗微信：https://weixin.sogou.com/从流行圈到时尚圈，加上每栏目下的loadmorecontent选项，一共需要爬取3000+篇文章，获取每篇文章的标题和右边的图片，将爬取的图片按照规定的命名输出method进入指定文件夹，将文章标题和图片名称对应输出到Excel和txt。完整代码如下PackageVersion----------------------------------altgraph0.17certifi2020.6.20chardet3.0。4future0.18.2idna2.10lxml4.5.2pefile2019.4.18pip19.0.3pyinstaller4.0pyinstaller-hooks-contrib2020.8pywin32-ctypes0.2.0requests0.2.0requests.0urllib31.25.10XlsxWriterr/bin3.3.3.31.3.3#-*-coding:UTF-8-*-importosimportrequestsimportxlsxwriterfromlxmlimportetree#请求微信文章头部信息headers={'accept':'text/html,application/xhtml+xml,aapplication/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding':'gzip,deflate,br','Accept-Language':'zh-CN,zh;q=0.9','Host':'weixin.sogou.com','User-Agent':'Mozilla/5.0(WindowsNT10.0;Win64;x64)AppleWebKit/537.36(KHTML,likeGecko)Chrome/85.0.4183.102Safari/537.36'}#下载图片的头部信息headers_images={'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding':'gzip,deflate','Accept-Language':'zh-CN,zh;q=0.9','Host':'img01.sogoucdn.com','User-Agent':'Mozilla/5.0(WindowsNT10.0;Win64);x64)AppleWebKit/537.36(KHTML,likeGecko)Chrome/85.0.4183.102Safari/537.36'}a=0all=[]#创建根目录save_path='./WeChatarticles'folder=os.path.exists(save_path)ifnotfolder:os.makedirs(save_path)#创建图片文件夹images_path='%s/pictures'%save_pathfolder=os.path.exists(images_path)ifnotfolder:os.makedirs(images_path)foriinrange(0,20):forjinrange(1,10):url="https://weixin.sogou.com/pcindex/pc/pc_%d/%d.html"%(i,j)#请求搜狗文章的url地址response=requests.get(url=url,headers=headers).text.encode('iso-8859-1').decode('utf-8')#构造一个XPath解析对象并自动修正HTML文本html=etree.HTML(response)#XPath使用路径表达式选择用户名xpath=html.xpath('/html/body/li')forcontentinxpath:#counta=a+1#articletitletitle=content.xpath('./div[@class="txt-box"]/h3//text()')[0]author_=content.xpath('./div[@class="txt-box"]/div/a//text()')article={}#authorexistsifauthor_:article['title']=titlearticle['id']='%d.jpg'%一篇文章['author']=author_all.append(article)#图片路径path='http:'+content.xpath('./div[@class="img-box"]//img/@src')[0]#下载文章图片images=requests.get(url=path,headers=headers_images).contenttry:withopen('%s/%d.jpg'%(images_path,a),"wb")asf:print('正在下载文章图片%d'%a)f.write(images)exceptExceptionase:print('未能下载文章图片%s'%e)#信息存入excel#创建工作簿xworkbook=xlsxwriter.Workbook('%s/Excelformat.xlsx'%save_path)#创建工作表worksheet=workbook.add_worksheet()print('Excelisbeinggenerated...')try:foriinrange(0,len(all)+1):#第一行用来写表头ifi==0:worksheet.write(i,0,'title')worksheet.write(i,1,'id')worksheet.write(i,2,'author')continueauthor_=all[i-1]['author']#作者存在如果author_:worksheet.write(i,2,author_[0])worksheet.write(i,0,all[i-1]['title'])worksheet.write(i,1,all[i-1]['id'])workbook.close()print("Excelgeneratedsuccessfully")exceptExceptionase:print('生成失败Excel%s'%e)print('Generatingtxt...')try:withopen('%s/arrayformat.txt'%save_path,"w")作为f:f.write(str(all))print('生成txt成功')exceptExceptionase:print('生成txt失败%s'%e)print('总共爬取了%d篇文章'%a)最后将程序打包成exe文件运行它在windows系统上可以直接运行程序。我用pyinstaller//安装pyinstallerpipinstallpyinstaller//打包成一个exe文件pyinstaller-Fweixin.py执行exe文件点赞，收藏，关注，你们的支持是我最大的动力！
