当前位置: 首页 > 后端技术 > Python

女人、男人、相机、电视:如何做一个完整的深度学习应用

时间:2023-03-25 22:21:15 Python

浣滆€咃細LeanCloud濮滅孩鍓嶆鏃堕棿锛屽綋鐨勯噰璁挎垚涓虹ぞ浜ゅ獟浣撶殑鐒︾偣鏃讹紝姝eソ鍦ㄥ涔犱竴浜涚缁忕綉缁滅殑璧勬枡锛屼簬鏄垜鎴戞兂鎴戝彲浠ュ皾璇曚娇鐢ㄤ竴浜涙柊鐨勫紑婧愬伐鍏锋潵鍒朵綔涓€涓彲浠ヨ瘑鍒コ浜恒€佺敺浜恒€佺浉鏈哄拰鐢佃鐨勫畬鏁村簲鐢ㄧ▼搴忋€傝繖涓緥瀛愬緢灏忥紝鍙互鍦ㄥ緢鐭殑鏃堕棿鍐呭畬鎴愶紝闈炲父閫傚悎璇存槑濡備綍鍋氫竴涓畬鏁寸殑娣卞害瀛︿範搴旂敤銆傚畬鎴愮殑搴旂敤绋嬪簭閮ㄧ讲鍦╤ttps://trump-sim.jishuq.com涓婏紙鍦↙eanCloud鐨勪簯寮曟搸瀹炰緥涓婏級銆傚仛杩欎釜搴旂敤鍒嗕笁姝ワ細鍏堢敤涓€浜涘浘鐗囧畬鎴愭ā鍨嬬殑璁粌锛岀劧鍚庡鍑烘ā鍨嬶紝鍋氬悗绔疉PI璇嗗埆鍥剧墖锛屽啀鍋氬墠绔笂浼犲浘鐗囧苟鏄剧ず缁撴灉.鍑嗗璁粌鏁版嵁Jupyternotebook鏄竴绉嶉潪甯告祦琛岀殑鏁版嵁鍒嗘瀽鍜屾満鍣ㄥ涔犱氦浜掑紡鐜銆傚彲浠ユ妸Markdown鏂囨。鍜孭ython浠g爜鏀惧湪鈥嬧€嬬瑪璁版湰閲岋紝杩樺彲浠ラ€氳繃鍥捐〃銆佸浘鐗囩瓑鍙嬪ソ鐨勬柟寮忓睍绀轰唬鐮佺殑杩愯鎯呭喌銆傜粨鏋溿€傝繖閲屼篃浼氱敤鍒癋astAI锛屽畠鏄竴涓熀浜嶱yTorch鐨勫紑婧愬簱锛屼负缃戠粶鍜屾枃浠舵壒閲忔搷浣滄彁渚涗簡寰堝鏂逛究鐨勬帴鍙c€傛湰鏂囨槸鐢↗upyternotebook鍐欑殑锛屾墍浠ヤ綘鍙互鐩存帴clone杩欎釜repo锛屽畨瑁呬緷璧栵紝鍚姩Jupyternotebook銆俫itclonehttps://github.com/hjiang/trump-sim-notebookpipinstall-rrequirements.txtjupyternotebook鎴戜滑涔熶細浣跨敤Bing鍥剧墖鎼滅储API鑾峰彇璁粌鍥剧墖锛岄渶瑕佽嚜宸辨敞鍐屽苟鍏嶈垂鐢宠API瀵嗛挜銆傚綋鐒讹紝鍥犱负鎼滅储鍒扮殑鍥剧墖鍦ㄥ緢澶氱涓夋柟缃戠珯涓婏紝鎵€浠ラ渶瑕佽兘澶熸棤闅滅璁块棶鍥藉鐨勭綉绔欍€傪煠封€嶁檪锔忓皢浣犵殑Bing鍥剧墖鎼滅储API瀵嗛挜鏀惧湪椤圭洰鐩綍涓嬬殑.env涓紝浠ュ厤鍦ㄤ唬鐮佷腑娉勯湶锛欱ING_SEARCH_API_KEY=XXXXXXXX....鐒跺悗鍦≒ython涓鍙杋mportosfromdotenvimportload_dotenvload_dotenv()key=os.getenv('BING_SEARCH_API_KEY')缂栧啓鎼滅储鍥惧儚鐨勫嚱鏁帮細'https://api.cognitive.microsoft.com',CognitiveServicesCredentials(key))returnL(client.images.search(query=term,count=150,min_height=min_sz,min_width=min_sz).value)瀹為檯璁よ瘉涓轰竴浼氬効锛屾悳绱rtemis鐨勫浘鐗囷細fromtorchvision.datasets.utilsimportdownload_urlfromPILimportImageimportfastai2.vision.widgetsresults=search_images_bing(key,'Artemis')urls=results.attrgot('content_url')download_url(urls[0],'images/','artemis.jpg')image=Image.open('images/artemis.jpg')image.to_thumb(128,128)纭鍥剧墖涓嬭浇娌¢棶棰樺悗锛屾垜浠細娉ㄦ剰浠巉astai2.vision.utilsimportdownload_imagesfrompathlibimportPathobject_types='woman','man','camera','TV'path=Path('objects')if涓嬭浇/objects涓嬪洓涓洰褰曚笅鐨勫洓绉嶅浘鐗囦笉鏄痯ath.exists():path.mkdir()foroinobject_types:dest=(path/o)dest.mkdir(exist_ok=True)results=search_images_bing(key,o)download_images(dest,urls=results.attrgot('content_url'))鍙兘浼氱湅鍒颁竴浜涘浘鐗囦笅杞藉け璐ョ殑淇℃伅锛屽彧瑕佷笉鏄お澶氾紝鍙互蹇界暐銆傜綉涓婃湁浜涘浘鐗囨崯鍧忥紝鎴栬€呮槸Python鍥剧墖搴撲笉鏀寔鐨勬牸寮忥紝闇€瑕佸垹闄ゃ€備粠fastai2.vision.utils瀵煎叆get_image_files浠巉astai2.vision.utils瀵煎叆verify_imagesfns=get_image_files(path)failed=verify_images(fns)failed.map(Path.unlink);棰勫鐞嗗湪寮€濮嬭缁冧箣鍓嶏紝浣犻渶瑕佸憡璇塅astAI濡備綍鏍囪鍥剧墖锛屽苟鍔犺浇鍒板畠鐨勬暟鎹粨鏋勪腑銆備笅闈㈢殑浠g爜鎵ц浠ヤ笅鎿嶄綔锛氱敤鐖剁洰褰曠殑鍚嶇О(parent_label)鏍囪姣忎釜鍥惧儚銆備繚鐣?0%鐨勫浘鐗囦綔涓洪獙璇侀泦锛屽叾浣欑殑浣滀负璁粌闆嗐€傝缁冮泦鏄敤鏉ヨ缁冪缁忕綉缁滅殑鏁版嵁锛岄獙璇侀泦鐢ㄦ潵琛¢噺璁粌妯″瀷閬囧埌鏂版暟鎹椂鐨勫噯纭巼銆傝繖涓や釜闆嗗悎涓嶈兘閲嶅彔銆傜缉灏忓浘鍍忎互鎻愰珮鏁堢巼鏈€鍚庝竴琛屼唬鐮佹樉绀洪獙璇侀泦涓殑鍓嶄笁寮犲浘鍍忋€備粠fastai2.data.block瀵煎叆DataBlock锛屼粠fastai2.vision.data瀵煎叆ImageBlock浠巉astai2.data.transforms瀵煎叆RandomSplitter锛宲arent_label浠巉astai2.vision.augment瀵煎叆Resizeobjects=DataBlock锛堝潡=锛圛mageBlock锛孋ategoryBlock锛夛紝get_items=get锛屽浘鍍忔枃浠禦andomSplitter(valid_pct=0.2,seed=42),get_y=parent_label,item_tfms=Resize(128))dls=objects.dataloaders(path)dls.valid.show_batch(max_n=3,nrows=1)鍦ㄥ仛鍥惧儚璇嗗埆鐨勬椂鍊欑粡甯革紝瀵瑰浘鐗囪繘琛屼竴浜涢殢鏈虹缉鏀俱€佽鍓瓑鍙樻崲锛屼互浜х敓瓒冲鐨勬暟鎹潵鎻愰珮璁粌鏁堟灉銆傛偍鍙互浠庝笅闈唬鐮佺殑缁撴灉涓湅鍒板鍚屼竴寮犲浘鍍忚繘琛屼笉鍚屽彉鎹㈢殑缁撴灉銆備粠fastai2.vision.augment瀵煎叆aug_transforms锛孯andomResizedCropobjects=objects.new锛坕tem_tfms=RandomResizedCrop锛?24锛宮in_scale=0.5锛夛紝batch_tfms=aug_transforms锛堬級锛塪ls=objects.dataloaders锛堣矾寰勶級dls.train.show_batch锛坢ax_n=6锛宯rows=2,unique=True)璁粌鏁版嵁缁堜簬鍙互寮€濮嬭缁冧簡銆傚浜庡浘鍍忚瘑鍒瓑搴旂敤鍦烘櫙锛屽線寰€涓嶅彲鑳戒粠澶村紑濮嬭缁冧竴涓柊妯″瀷锛屽洜涓哄瓨鍦ㄥぇ閲忓嚑涔庢墍鏈夊簲鐢ㄩ兘闇€瑕佽瘑鍒殑鐗瑰緛锛屼緥濡傜墿浣撶殑杈圭紭銆侀槾褰便€佺敱鐗╀綋褰㈡垚鐨勫浘妗堢瓑銆備笉鍚岀殑棰滆壊銆傞€氬父鐨勫仛娉曟槸鏍规嵁棰勮缁冩ā鍨嬶紙渚嬪姝ゅ鐨剅esnet18锛夊湪鎮ㄨ嚜宸辩殑鏂版暟鎹紙绉颁负寰皟锛変笂璁粌鏈€鍚庡嚑灞傘€傚湪澶氬眰绁炵粡缃戠粶涓紝杈冩棭锛堟洿闈犺繎杈撳叆锛夌殑灞傝礋璐h瘑鍒洿鍏蜂綋鐨勭壒寰侊紝鑰岃緝鍚庣殑灞傝瘑鍒壒寰佹洿鎶借薄锛屾洿鎺ヨ繎鐩殑銆備笅闈㈡渶鍚庝竴琛屼唬鐮佹寚鐨勬槸4涓猠poch鐨勮缁冦€傚鏋滀綘鏈塏vidia鏄惧崱锛屽湪Linux涓嬶紝骞朵笖瀹夎浜嗙浉搴旂殑椹卞姩锛屼笅闈㈢殑浠g爜鍙渶瑕佸嚑绉掑埌鍗佸嚑绉掞紝鍚﹀垯灏遍渶瑕佸嚑鍒嗛挓銆備粠fastai2.vision.learner浠巘orchvision.models.resnet瀵煎叆resnet18浠巉astai2.metrics瀵煎叆error_rateimportfastai2.vision.allasfa_visionlearner=cnn_learner(dls,resnet18,metrics=error_rate)learner.fine_tuner_timer_loseserlos01.9280010.602813:11630.163epochtrain_lossvalid_losserror_ratetime00.5507570.4118350.12069001:4210.4639250.3639450.10344801:4620.3725510.3361220.09482801:4430.3145970.3213490.09482801:44鏈€鍚庤緭鍑虹殑琛ㄦ牸閲屾槸姣忎竴杞噷璁粌Thelossoftheset,楠岃瘉闆嗙殑鎹熷け锛屼互鍙婇敊璇巼锛坋rrorrate锛夈€傞敊璇巼鏄垜浠叧蹇冪殑鎸囨爣锛宭oss鏄帶鍒惰缁冭繃绋嬬殑鎸囨爣锛堣缁冪殑鐩爣鏄loss鏇存帴杩戜簬0锛夈€傞渶瑕佽繖涓や釜涓嶅悓鐨勬寚鏍囨槸鍥犱负loss闇€瑕佹弧瓒充竴浜沞rrorrate涓嶄竴瀹氭弧瓒崇殑鏉′欢锛屾瘮濡傝兘澶熷紩瀵兼墍鏈夊弬鏁帮紝errorrate涓嶆槸杩炵画鍑芥暟銆俵oss瓒婁綆锛岄敊璇巼瓒婁綆锛屼絾瀹冧滑涔嬮棿娌℃湁绾挎€у叧绯汇€傝繖閲岀殑閿欒鐜囧嚑涔庢槸10%锛屼篃灏辨槸璇村噯纭巼澶х害鏄?0%銆傛帴涓嬫潵锛屾垜浠渶瑕佺湅鐪嬮獙璇侀泦涓湁鍝簺鍥剧墖琚璇嗗埆浜嗐€備互涓嬩唬鐮佸皢鎵撳嵃鍑烘贩娣嗙煩闃点€傚湪杩欎釜鐭╅樀涓紝瀵硅绾夸笂鐨勬暟瀛楁槸姝g‘璇嗗埆鍥剧墖鐨勬暟閲忥紝鍏朵粬鍦版柟鐨勬暟瀛楁槸閿欒璇嗗埆鍥剧墖鐨勬暟閲忋€俧romfastai2.interpretimportClassificationInterpretationinterp=ClassificationInterpretation.from_learner(learner)interp.plot_confusion_matrix()浠庤緭鍑虹煩闃靛彲浠ョ湅鍑轰竴鍏辨湁11澶勯敊璇紝鍏朵腑鐢峰コ鎬у埆鍚勬湁4澶勯敊璇紝杩樻湁鐢佃鍜屽叾浠栫被鍒篃鏈夊緢澶氭贩娣嗐€傪煠旇鎴戜滑鏄剧ず鎹熷け鏈€楂樼殑鍥剧墖锛岀湅鐪嬪摢閲屽嚭浜嗛棶棰樸€俰nterp.plot_top_losses(12,nrows=4)鐨勮緭鍑哄弽鏄犱簡浠庝簰鑱旂綉涓婃姄鍙栨暟鎹殑涓€涓吀鍨嬮棶棰橈細鍣煶澶ぇ銆備緥濡傦紝鐢佃鐨勬悳绱㈢粨鏋滃寘鎷數瑙嗛仴鎺у櫒銆佺數瑙嗙洅瀛愩€佺數瑙嗗墽娴锋姤锛屼互鍙婁竴浜涘畬鍏ㄤ笉鐩稿叧鐨勭粨鏋溿€侳astAI鎻愪緵浜嗕竴绉嶆竻娲佸櫒锛屽彲浠ュ府鍔╂垜浠墜鍔ㄦ竻娲佽緝灏忕殑鏁版嵁闆嗐€傚畠鍙互鍒楀嚭鏁翠釜鏁版嵁闆嗕腑鎹熷け鏈€楂樼殑鍥剧墖锛屼互渚跨敤鎴锋墜鍔ㄤ慨鏀规爣绛炬垨鍒犻櫎瀹冧滑銆俧romfastai2.vision.widgetsimportImageClassifierCleanercleaner=ImageClassifierCleaner(learner)cleaner闇€瑕佹敞鎰忕殑鏄痗leaner鍙槸鍋氭爣璁扮殑锛屽疄闄呭鐞嗛渶瑕佺敤Python浠g爜銆傛垜閫氬父鍙槸灏嗘湁闂鐨勫浘鍍忔爣璁颁负鍒犻櫎骞跺垹闄ゅ畠銆俧oridxincleaner.delete():cleaner.fns[idx].unlink()娓呯悊鍚庨噸澶嶈缁冭繃绋嬨€俹bjects=DataBlock(blocks=(ImageBlock,CategoryBlock),get_items=get_image_files,splitter=RandomSplitter(valid_pct=0.2,seed=42),get_y=parent_label,item_tfms=Resize(128))objects=objects.new(item_tfms=Crop(Resized224,min_scale=0.5),batch_tfms=aug_transforms())dls=objects.dataloaders(path)learner=cnn_learner(dls,resnet18,metrics=error_rate)learner.fine_tune(3)epochtrain_lossvalid_losserror_ratetime0501.6635559011epochtrain_lossvalid_losserror_ratetime00.4582120.2268660.09174301:3210.3583640.1452860.03669701:3120.2815170.1464770.03669701:32濡傛灉浣犳敞鎰忓埌error_rate鍦ㄥ悗闈㈢殑epoch鏈変笂鍗囩殑璇濓紝鍙互闄嶄綆fine_tune鐨勫弬鏁颁互杈惧埌鏈€濂界殑Effect.Becauseiftherearetoomanytrainingrounds,themodelwilloverfitthetrainingset,andtheerrorratewillbecomehigherwhenencounteringnewdata.浠庝笂闈㈢殑杈撳嚭鍙互鐪嬪嚭锛屽噯纭巼宸茬粡鎻愰珮鍒?6%浠ヤ笂銆傝揪鍒版弧鎰忕殑鍑嗙‘鐜囧悗锛屽嵆鍙皢妯″瀷瀵煎嚭骞跺湪绾夸娇鐢ㄣ€備互涓嬩唬鐮佽浼氬皢妯″瀷淇濆瓨鍒癳xport.pkl銆俵earner.export()鍚庣API鍚庣API鏄繖涓」鐩腑鏈€绠€鍗曠殑閮ㄥ垎锛屽彧鏈変竴涓鐐广€傚姞杞戒箣鍓嶅鍑虹殑妯″瀷锛屽苟鍦ㄦ敹鍒版柊鍥剧墖鏃朵娇鐢ㄨ妯″瀷棰勬祴鍒嗙被銆倀rump=load_learner('model.pkl')@app.route('/api/1.0/classify-image',methods=['POST'])defclassify():image=request.files['image']res=trump.predict(image.read())response=jsonify({'result':res[0]})response.status_code=200returnresponse瀹屾暣浠g爜鍦℅itHub涓娿€傛寜鐓ф枃妗i儴缃插埌LeanCloud浜戝紩鎿庛€傚墠绔綉绔欏墠绔篃姣旇緝绠€鍗曘€傚畠鍙渶瑕佷竴涓〉闈緵鐢ㄦ埛涓婁紶鐓х墖锛屽湪娴忚鍣ㄤ腑缂╁皬鐓х墖骞跺彂閫佸埌鍚庣API銆傚畬鏁寸殑React椤圭洰鍦℅itHub涓婏紝涓昏浠g爜鍦ˋpp.js涓€傞檺浜庣瘒骞咃紝鎴戝氨涓嶇粏璇翠簡锛屽彧闄勪笂杩愯鎴浘锛氬ぇ瀹跺彲鑳藉凡缁忔敞鎰忓埌锛屼笂闈㈢殑鍚庣API鏈嶅姟鏄棤鐘舵€佺殑锛屼笉瀛樺偍浠讳綍鏁版嵁锛屾墍浠ュ叾瀹炶瘑鍒繃绋嬪彲浠ュ湪鍓嶇Finish瀹屾垚銆傛湁鍏磋叮鐨勫彲浠ョ爺绌朵竴涓嬪浣曞皢PyTorch妯″瀷杞崲涓篔avaScript鍙敤鐨勬ā鍨嬶紝骞跺皾璇曠洿鎺ュ湪娴忚鍣ㄤ腑璇嗗埆鐓х墖銆傚湪瀹為檯搴旂敤涓紝杩欑鏂规硶涓嶉渶瑕佸悜鏈嶅姟鍣ㄤ紶杈撲换浣曟暟鎹紝鍙互寰堝ソ鍦颁繚鎶ょ敤鎴烽殣绉併€傝繖涔熸槸Apple姝e湪鎺ㄥ姩鐨刼n-devicemachinelearning鐨勬柟鍚戙€傚浘鍍忚瘑鍒槸鏈哄櫒瀛︿範鍙互瑙e喅鐨勬渶绠€鍗曠殑涓€绫婚棶棰橈紝鍥犱负鏈夊緢澶氱幇鎴愮殑缁撴灉鍙互閲嶅浣跨敤锛屾柊鐨勫簲鐢ㄥ嵆浣垮彧闇€瑕佸皯閲忕殑璁粌鏁版嵁涔熻兘鍙栧緱鏇村ソ鐨勬晥鏋溿€傝繕鏈夎澶氬叾浠栫被鍨嬬殑闂涓嶅鏄撳緱鍒颁护浜烘弧鎰忕殑缁撴灉銆侺eanCloud鐩墠姝e湪寮€鍙戞満鍣ㄥ涔犳柟闈㈢殑鏂颁骇鍝侊紝甯姪寮€鍙戣€呮洿瀹规槗鍦板彂鐜版暟鎹殑浠峰€笺€傚鏋滀綘瀵规鎰熷叴瓒o紝鍙互鍏虫敞鎴戜滑鐨勫井鍗氥€佸井淇″叕浼楀彿銆佹帹鐗癸紝鎴栬€呮敞鍐屾垚涓篖eanCloud鐢ㄦ埛銆傛垜浠皢鍦ㄨ繎鏈熷叕甯冩洿澶氫俊鎭紝骞堕個璇烽儴鍒嗙敤鎴疯瘯鐢ㄦ柊浜у搧銆傛爣棰樺浘CharlesDeluvio