多线程、多进程、多协程（转载）哔哩哔哩蚂蚁学习PythonUP主说的很好。上班看视频不方便。关键部分截取改成文字版，方便复制。原文地址：https://www.bilibili.com/video/BV1bK411A7tV多线程定义一个函数defmy_func(a,b):do_something(a,b)创建一个线程importthreadingt=threading.Thread(target=my_func,args=(a,b))启动一个线程t.start()等待结束t.join()queuequeue.Queue是线程安全的importqueueq=queue.Queue()#添加并获取q.put(time)item=q.get()#查看状态q.qsize()q.empty()q.full()线程安全（锁）try-finallyimportthreadinglock=threading.Lock()lock.acquire()try:#dosomethingfinally:lock.release()withimportthreadinglock=threading.Lock()withlock:#dosomethingthreadpoolmapfunction,result预先输入的参数序列对应于fromconcurrent.futuresimportThreadPoolExecutorarg_list=[]withThreadPoolExecutor()aspool:results=pool.map(my_func,arg_list)forresultinresults:print(results)提交函数，as_completed顺序可以按顺序完成fromconcurrent.futuresimportThreadPoolExecutor,as_completedarg_list=[]withThreadPoolExecutor()aspool:futures=[pool.submit(my_func,arg)forarginarg_list]#根据future中的输入顺序utures:print(future.result())#在as_completed(futures)中完成future的顺序:print(future.result())Flask中使用线程池importtimefromconcurrent.futuresimportThreadPoolExecutorfromflaskimportFlaskapp=Flask(__name__)pool=ThreadPoolExecutor()defdo_1():time.sleep(1)return'do_1'defdo_2():time.sleep(1)return'do_2'defdo_3():time.sleep(1)return'do_3'@app.route("/")defindex():result_1=pool.submit(do_1)result_2=pool.submit(do_2)result_3=pool.submit(do_3)return{'1':result_1.result(),'2':result_2.result(),'3':result_3.result(),}if__name__=="__main__":app.run()多进程图片截图自蚂蚁学PythonBilibili03:00Flask使用多进程importtimefromconcurrent.futuresimportProcessPoolExecutorfromflaskimportFlaskapp=Flask(__name__)defdo_1():time.sleep(1)return'do_1'defdo_2():time.sleep(1)return'do_2'defdo_3():时间.sleep(1)返回'do_3'@app.route("/")defindex():result_1=pool.submit(do_1)result_2=pool.submit(do_2)result_3=pool.submit(do_3)return{'1':result_1.result(),'2':result_2.result(),'3':result_3.result(),}if__name__=="__main__":pool=ProcessPoolExecutor()app.run()协议：asyncio、awaitimportasyncioimportaiohttploop=asyncio.get_event_loop()asyncdefget_url(url):asyncwithaiohttp.ClientSession()assession:asyncwithsession.get(url)asresp:result=awaitresp.text()print(f"url:{url},{len(result)}")urls=[f"https://www.cnblogs.com/#p{page}"forpageinrange(1,50+1)]tasks=[loop.create_task(get_url(url))forurlinurls]loop.run_until_complete(asyncio.wait(tasks))控制异步并发数asynciosem=asyncio.Semaphore(10)asyncwithsem:#dosomething举例importasyncioimportaiohttploop=asyncio.get_event_loop()#限制10个并发semaphore=asyncio.Semaphore(10)asyncdefget_url(url):asyncwithsemaphore:asyncwithaiohttp.ClientSession()assession:asyncwithsession.get(url)asresp:result=awaitresp.text()print(f"url:{url},{len(result)}")urls=[f"https://www.cnblogs.com/#p{page}"forpageinrange(1,50+1)]tasks=[loop.create_task(get_url(url))forurlinurls]loop.run_until_complete(asyncio.wait(tasks))
