鏈枃鐢卞浘闆€绀惧尯鎴愬憳鐏胯嫢鏄熺┖鎾板啓銆傛杩庡姞鍏ュ浘闆€绀惧尯锛屾墦閫犵簿褰╃殑鍏嶈垂鎶€鏈暀绋嬶紝涓虹紪绋嬭涓氱殑鍙戝睍璧嬭兘銆傝寰楁垜浠啓鐨勮繕涓嶉敊璁板緱鐐硅禐+鍏虫敞+璇勮涓夐亶馃グ馃グ馃グ锛岄紦鍔辨垜浠啓鍑烘洿濂界殑鏁欑▼馃挭涔嬪墠鐨勬儏鍐靛洖椤惧湪涓婁竴绡囨暀绋嬬埇铏吇鎴愮瑪璁?绗竴姝ヨ繘鍏ヨ繖涓簿褰╀腑涓栫晫锛堝コ鐢熷畾鍒讹級锛屾垜浠凡缁忓彲浠ユ媿鍒版墍鏈夊皬鍝ュ摜鐨勫皝闈㈠浘浜嗭紝浣嗘槸鍏夐潬灏侀潰鍥剧殑璐ㄩ噺鍜屾暟閲忔€庝箞鑳芥弧瓒冲皬浠欏コ浠殑瑕佹眰鍛紵鍦ㄦ湰鏁欑▼涓紝鎴戜滑涓茶捣涓€鏉″濮烩€滅嚎鈥濓紝灏嗘湰绯诲垪鐨勬墍鏈夊皬鍏勫紵閮藉泭鎷湪鍐呫€傚嚭闂ㄥ厛鍖栧灏忎粰濂充滑绾︿細鎬绘槸鈥滄贰濡嗘€绘槸鍚堥€傜殑鈥濓紝鑰岀埇铏嚭闂ㄧ埇鏁版嵁涔熷緱鎵撴壆涓€鐣紝涓嶇劧鎬庝箞鑳借鎵€鏈夊ぇ绁為兘涔栦箹璺熷湪鍚庨潰鍛紵鐖櫕鐨勨€滃寲濡嗏€濅笉鏄€滃簳濡?->绮夊簳-->閬憰-->鏁ｇ矇-->鐢荤湁-->鍙ｇ孩鈥濈瓑姝ラ锛岀洰鐨勬槸璁╁鏂圭綉绔欐洿鐩镐俊璁块棶鑰呮槸涓嶆槸鐖櫕绋嬪簭锛岃€屾槸娲讳汉銆備汉浠€氳繃鎿嶇旱娴忚鍣ㄦ潵璁块棶缃戠珯锛屽洜姝ょ埇铏彧闇€瑕佹ā浠挎祻瑙堝櫒鍗冲彲銆傞偅涔堟垜浠啀鏉ョ湅鐪嬫祻瑙堝櫒鍦ㄦ墦寮€缃戦〉鏃剁粯鍒剁殑鈥滃瀹光€濄€傛墦寮€Chrome鎵撳紑璋冭瘯鍣紝鍒囨崲鍒癗etWork閫夐」鍗★紝姝ゆ椂璁块棶https://www.nanrentu.cc/sgtp/锛屼綘浼氬湪璋冭瘯鍣ㄤ腑鐪嬪埌寰堝閾炬帴淇℃伅锛岃闂槸鍝釜閾炬帴鍛紵鎴戜滑闇€瑕佷粈涔堬紵鍥炲繂涓€涓嬩笂绡囨枃绔犵殑鍐呭锛岄鍏堟槸鑾峰彇HTML鏂囨。锛岀劧鍚庝粠杩欎釜鏂囨。涓彁鍙栧浘鐗囩殑閾炬帴锛屾墍浠ョ洰鏍囧氨鏄壘鍒版祻瑙堝櫒鑾峰彇鍒拌繖涓狧TML鏂囨。鐨勯摼鎺ャ€侰hrome鐭ラ亾杩欎箞澶氱殑閾炬帴淇℃伅鑲畾浼氳寮€鍙戣€呬竴澶撮浘姘达紝鎵€浠ュ畠瀵归摼鎺ヨ繘琛屼簡鍒嗙被锛岀偣鍑讳笂闈㈢殑Doc绫伙紝鍐嶇偣鍑诲敮涓€鐨勯摼鎺ワ紝灏变細鐪嬪埌鑾峰彇杩欎釜HTML鏂囨。閾炬帴鐨勮缁嗕俊鎭€傝嚦姝ゆ垜浠噸鐐瑰叧娉ㄤ富瑕丷equestHeaders鐨勫唴瀹广€傛祻瑙堝櫒閫氳繃http鍗忚涓庢湇鍔″櫒浜や簰鑾峰彇淇℃伅锛岀埇铏€氳繃妯′豢娴忚鍣ㄤ笅鍙慼ttp鍗忚鏉ヨ幏鍙栦俊鎭紝鍏朵腑鏈€閲嶈鐨勬ā浠跨偣涔嬩竴灏辨槸RequestHeaders銆俬ttp鍗忚涓殑鈥滅摱鐡剁綈缃愨€濓紝璁╃敺鐢熺湅鐫€濂崇敓鍖栧鐢ㄧ殑鐡剁摱缃愮綈锛屼及璁′細闄峰叆娌夋€濄€傝繖鏄疊B闇滐紝灏辨槸绮夊簳娑诧紝杩樻湁鏁ｇ矇銆佺溂褰便€侀伄鐟曡啅锛屾洿涓嶇敤璇村悇绉嶆繁娴呬笉涓€鐨勫彛绾簡銆傚瀛愮湅鍒拌繖涓猦ttp閲岄潰鐨勫悇绉嶅唴瀹逛及璁′細涓€澶撮浘姘淬€傚叾瀹炶繖姣斿寲濡嗗搧绠€鍗曞浜嗐€傛垜浠彧闇€瑕佺畝鍗曠殑浜嗚В锛屽氨鍙互鐢诲嚭绮剧編鐨勭埇琛屽姩鐗╁瀹广€?authority:www.nanrentu.cc:method:GET//鑷畾涔夎姹傚ご璇锋眰鏂瑰紡锛歱ath:/sgtp///鑷畾涔夎姹傚ご璇锋眰璺緞锛歴cheme:https//鑷畾涔夎姹傚ご璇锋眰鏂瑰紡//Accepted鍐呭鏍煎紡accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9//Acceptedencodingaccept-encoding:gzip,deflate,br//Acceptedlanguageaccept-language:zh-CN,zh;q=0.9//缂撳瓨鎺у埗锛氬憡璇夋湇鍔″櫒瀹㈡埛绔兂瑕佽幏鍙栨渶鏂扮殑璧勬簮cache-control:鏈€澶у勾榫?0cookie锛歎M_distinctid=170a5a00fa25bf-075185606c88b7-396d7407-100200-170a5a00fa3507锛沨m_lvt_45e50d2aec057f43a3112beaf7f00179=1583326696,1583756661;CNZZDATA1274895726=1196969733-1583323670-%7C1583752625锛沨m_lpvt_45e50d2aec057f43a3112beaf7f00179=1583756721sec-fetch-dest:documentsec-fetch-mode:navigatesec-fetch-site:nonesec-fetch-user:?1//涓篐TTP璇锋眰璀︽姤灞忚斀HTTPS椤甸潰upgrade-insecure-requests:1user-agent:Mozilla/5.0(Macintosh;IntelMacOSX10_14_2)AppleWebKit/537.36(KHTML,likeGecko)Chrome/80.0.3987.132Safari/537.36鐖櫕娌″繀瑕佸姞閭ｄ箞澶氫俊鎭紝鍥犱负杩欎釜缃戠珯鍙嶇埇鎺柦涓嶉珮-绾э紝鏆傛椂鍙敤涓や釜鍏抽敭鐨勫氨澶熶簡銆俢ookie锛氳繖鏄瓨鍌ㄥ湪娴忚鍣ㄤ腑鐨勪竴娈垫枃鏈紝鏈夋椂鍖呮嫭璁よ瘉淇℃伅鍜屼竴浜涚壒娈婄殑璇锋眰淇℃伅user-agent锛氱敤浜庤瘑鍒姹傛槸浠庝粈涔堝伐鍏峰彂鍑虹殑銆傚叧浜嶶ser-Agent鐨勬洿澶氫俊鎭彲浠ュ弬鑰冭繖绡囧崥鏂囪鍒癠serAgent瀛楃涓茬殑瑙勫垯鍜屼吉閫犳柟娉曪紝浣嗘槸鍦ㄧ埇鍏朵粬缃戠珯鐨勬椂鍊欏彲鑳介渶瑕併€備箣鎵€浠ュ湪杩欓噷閲嶅杩欎箞澶氾紝鏄笇鏈涘ぇ瀹惰兘澶熶簡瑙ｄ吉瑁呯埇铏殑閲嶈鎬э紝浠ュ強濡備綍鑾峰彇杩欎簺浼淇℃伅銆?/鍒涘缓涓€涓悕涓篽eaders鐨勫瓧鍏竓eaders={'User-Agent':'Mozilla/5.0(Macintosh;IntelMacOSX10_14_2)AppleWebKit/537.36(KHTML,likeGecko)Chrome/80.0.3987.132Safari/537.36','cookie':'UM_distinctid=170a5a00fa25bf-075185606c88b7-396d7407-100200-170a5a00fa3507;CNZZDATA1274895726=1196969733-1583323670-%7C1583925652锛沨m_lvt_45e50d2aec057f43a3112beaf7f00179=1583326696,1583756661,1583926583;hm_lpvt_45e50d2aec057f43a3112beaf7f00179=1583926583'}//鍙戦€佽姹傛椂甯︿笂璇锋眰澶磖esponse=requests.get(baseUrl,headers=headers)鎸夌収绾跨储銆備竴涓綉绔欑敱澶氫釜缃戦〉缁勬垚銆傝缃戦〉鍏呮弧浜嗗悇绉嶈秴閾炬帴銆備粠杩欎釜缃戦〉閾炬帴鍒伴偅涓綉椤点€傚鏋滄垜浠兂瑕佹洿澶氱殑灏忓厔寮燂紝閭ｄ綘灏卞緱鍏堝垎鏋愯繛鎺ヤ粬浠殑瓒呴摼鎺ワ紝鐒跺悗浣犲氨鍙互椤鸿棨鎽哥摐浜嗐€傚綋榧犳爣鏀惧湪鏍囬涓婃椂锛屾爣棰樼殑棰滆壊鍙戠敓浜嗗彉鍖栵紝璇佹槑杩欎釜鍏冪礌鏄竴涓秴閾炬帴銆傜偣鍑绘爣棰樻祻瑙堝櫒浼氳嚜鍔ㄦ墦寮€涓€涓爣绛鹃〉鏄剧ず缃戦〉銆傝娉ㄦ剰锛屼笅闈㈢殑椤电爜鏍囩鏄繖浜涘厓绱犺繛鎺ヤ簡鏁翠釜闆嗗悎銆傚崟鍑烩€滄渶鍚庝竴椤碘€濅互瑙傚療url鍙戠敓浜嗕粈涔堛€傛渶鍚庝竴椤电殑url锛歨ttps://www.nanrentu.cc/sgtp/...棣栭〉鐨剈rl锛歨ttps://www.nanrentu.cc/sgtp/...鐪嬭捣鏉ユ湁鐐规剰鎬濓紝鏈€鍚庝竴椤电殑url姣旂涓€椤电殑url澶氫簡鈥淿7鈥濓紝鐒跺悗鍒嗗埆鐐瑰嚮杩涘叆绗簩椤靛拰绗笁椤碘€︹€﹁瀵焨rl鐨勫彉鍖栵紝鍙互寰楀埌涓嬭〃銆傞〉闈rl棣栭〉https://www.nanrentu.cc/sgtp/...绗?椤礹ttps://www.nanrentu.cc/sgtp/...绗?椤礹ttps://www.nanrentu.cc/sgtp/銆?.绗?椤礹ttps://www.nanrentu.cc/sgtp/...绗?椤礹ttps://www.nanrentu.cc/sgtp/...绗?椤礹ttps://www.nanrentu.cc/sgtp/...Page7https://www.nanrentu.cc/sgtp/...澶氱偣鍑犲紶鍥撅紝浣犱細鍙戠幇鍚屾牱鐨勫浘妗堛€傝繖浣垮畠鏇村姞娓呮櫚銆傛垜浠凡缁忓垎鏋愬嚭浜嗏€滆棨钄撯€濈殑棣栧熬锛屾帴涓嬫潵灏卞彲浠ユ暡鍑轰唬鐮佽鐖櫕寮€濮嬧€滄懜鐡溾€濅簡銆傜涓€姝ワ細鎻愬彇鏍囬閾炬帴銆傝繖涓搷浣滃拰涓婁竴绡囧崥鏂囦粙缁嶇殑涓€鏍枫€傛墦寮€璋冭瘯骞冲彴骞跺垏鎹㈠埌Elements閫夐」鍗′互寮€濮嬫帰绱㈠拰鎻愬彇銆傜浜屾锛氭彁鍙栨渶鍚庝竴椤电殑閾炬帴锛屽緱鍒扮粍鍥句腑鐨勯〉鏁伴€氳繃瑙傚療HTML鍏冪礌缁撴瀯锛屽彲浠ュ彂鐜板寘鍚渶鍚庝竴椤电殑
- 锛岃繖鏍峰氨鍙互寰楀埌鍒涘缓涓嬮潰鐨刢ss閫夋嫨鍣ㄣ€俻age>ul>li:nth-last-child(2)>a绗笁姝ワ細鏍规嵁璧锋閾炬帴鏋勯€爑rl涓轰簡鏇存柟渚跨殑鏋勯€爑rl锛屾垜浠彲浠ユ妸棣栭〉https://www.nanrentu.cc/sgtp/...鍙樻垚https://www.nanrentu.cc/sgtp/...鍦ㄦ祻瑙堝櫒涓墦寮€杩欎釜鍚庣紑涓篣RL锛屼粛鐒跺彲浠ユ垚鍔熻闂椤点€備笉瑕侀棶鎴戜负浠€涔堬紵杩欏彲鑳芥槸绋嬪簭鍛樹箣闂寸殑涓€绉嶉粯濂戝惂~鎽哥摐绗洓姝ワ細瀛樺偍鍥剧墖锛屽畬鏁翠唬鐮佸涓嬶細importrequestsfrompyqueryimportPyQueryaspqimportuuidheaders={'User-Agent':'Mozilla/5.0(Macintosh;IntelMacOSX10_14_2)AppleWebKit/537.36(KHTML,likeGecko)Chrome/80.0.3987.132Safari/537.36','cookie':'UM_distinctid=170a5a00fa25bf-075185606c88b7-396d7407-1003005a-17;faCNZZDATA1274895726=1196969733-1583323670-%7C1583925652;hm_lvt_45e50d2aec057f43a3112beaf7f00179=1583326696,1583756661,1583926583;hm_lpvt_45e50d2aec057f43a3112beaf7f00179=1583926583'}defsaveImage(imgUrl,name):imgResponse=requests.get(imgUrl)fileName="瀛︿範鏂囦欢/%s.jpg"%nameifimgResponse.status_code==200:withopen(fileName,'wb')asf:f.write(imgResponse.content)f.close()defgetPic(urlArray):forurlinurlArray:res=requests.get(url,headers=headers)ifres.status_code==200:doc=pq(res.text)imgSrc=doc('.info-pic-list>a>img').attr('src')鎵撳嵃(imgSrc)saveImage(imgSrc,uuid.uuid1().hex)defcreateUrl(indexUrl,allPage):baseUrl=indexUrl.split('.html')[0]urlArray=[]foriinrange(1,allPage):tempUrl=baseUrl+"_"+str(i)+".html"urlArray.append(tempUrl)returnurlArraydefgetBoys(link):#鎽哥摐绗竴姝ワ細鑾峰彇棣栭〉杩炴帴picIndex=link.attr('href')#鎽哥摐绗簩姝ワ細鎵撳紑棣栭〉锛屾彁鍙栨湯椤甸摼鎺ワ紝鑾峰彇缁勫浘椤电爜res=requests.get(picIndex,headers=headers)print("褰撳墠姝ｅ湪鎶撳彇鐨刾icIndex锛?+picIndex)ifres.status_code==200:withopen("picIndex.html",'w',encoding="utf-8")asf:f.write(res.text)doc=pq(res.text)lastLink=doc('.page>ul>li:nth-last-child(2)>a').attr('href')#鎷嗗垎瀛楃涓插緱鍒版€婚〉鏁癷f(lastLinkisNone):return#灏?html浣滀负瑕佹媶鍒嗙殑鍒嗛殧绗︼紝鍙栫粨鏋滄暟缁勪腑鐨勭涓€椤箃emp=lastLink.split('.html')[0]#鐒跺悗鐢ㄤ笅鍒掔嚎_鎷嗗垎锛屽彇缁撴灉鏁扮粍涓殑绗簩椤癸紝骞跺皢鍏惰浆鎹负鏁板€糰llPage=int(temp.split('_')[1])#鎵剧摐绗笁姝ワ細鏍规嵁绗竴涓拰鏈€鍚庝竴涓摼鎺ユ瀯閫爑rlurlArray=createUrl(picIndex,allPage)#鎵剧摐绗洓姝ワ細瀛樺偍鍥剧墖锛屽苟鎵惧埌鐡滄垚鍔?00:withopen("index.html",'w',encoding="utf-8")asf:f.write(response.text)doc=pq(response.text)#鑾峰彇鎵€鏈夊浘闆嗙殑鏍囬閾炬帴titleLinks=doc('.h-piclist>li>a').items()#閬嶅巻杩欎簺閾炬帴浠ヨ幏寰梩itleLinks涓殑閾炬帴锛歡etBoys(link)if__name__=="__main__":main()杩愯缁撴灉锛氬洖澶寸湅鏁翠釜鐖櫕绋嬪簭锛屾槸涓€涓繛缁殑娴佹按绾胯繍琛岋紝姣忎竴姝ラ兘鏄幆鐜浉鎵ｇ殑锛屾墍浠ュ湪鍐欑▼搴忎箣鍓嶏紝涓€瀹氳鎯虫竻妤氭暣涓祦姘寸嚎鐨勬瘡涓€涓幆鑺傦紝杞畠浠箣闂寸殑椤哄簭渚濊禆鍏崇郴鍙樻垚浜嗕竴涓畝鍗曠殑娴佺▼鍥俱€傛湁浜嗙▼搴忓浘鍐嶅啓绋嬪簭浼氭竻妤氬緢澶氥€傛垜浠彲浠ユ妸姣忎釜妯″潡閮藉啓鎴愪竴涓嚱鏁帮紝鍏堝鍑芥暟鍋氬崟鍏冩祴璇曪紝鐒跺悗渚濇缁勫悎杩欎簺鍑芥暟銆傚垎鑰屾不涔嬶紝鏈夋満缁撳悎杩欏氨鏄紪绋嬬殑绮鹃珦銆傚啀澶嶆潅鐨勯」鐩紝閮芥槸鎸夋ā鍧楁惌寤鸿捣鏉ョ殑锛屽氨鍍忔惌绉湪涓€鏍枫€傝繖涓祦绋嬪浘鍙敤涓€涓澶寸敾鍑轰簡鑾峰彇鍥剧墖鐨勬暣涓繃绋嬶紝鐩稿綋浜庝竴涓獁orker鍦ㄥ伐浣溿€傛垜浠殑鐢佃剳鏄竴涓ぇ鍘傦紝鏈夊嚑鍗冨伐浜猴紝鍙厑璁镐竴涓伐浜哄伐浣溿€傚叾浠栦汉鍦ㄤ负浠栧姞娌瑰悧锛熼偅澶笉鍚堢悊浜嗐€傚湪涓嬩竴绡囨枃绔犱腑锛屾垜浠皢鐢诲嚭瀹屾暣鐨勬祦绋嬪浘锛屽垎鏋愬叾浠栧伐浜轰笉宸ヤ綔鐨勫師鍥狅紝杩涜€屽厖鍒嗚皟鍔ㄨ绠楁満鐨勮绠楄兘鍔涳紝鎻愰珮绋嬪簭鐨勮繍琛屾晥鐜囥€傚鏋滆寰楁垜浠啓鐨勮繕涓嶉敊锛岃寰楃偣璧?鍏虫敞+璇勮涓夎仈馃グ馃グ馃グ锛岄紦鍔辨垜浠啓鍑烘洿濂界殑鏁欑▼馃挭鎯冲涔犳洿澶氱簿褰╁疄鐢ㄧ殑鎶€鏈暀绋嬪悧锛熷揩鏉ュ弬瑙傚浘鍏嬬ぞ鍖恒€?/p>