LinuxShell提供了两个非常有用的抓取网页的命令，它们是curl和wgetcurl，wget使用代理curl支持http,https,socks4,socks5wget支持http,https代理示例：#!/bin/bash##curl支持http,https,socks4,socks5#wget支持http,https##Mimvp代理示例：#http://proxy.mimvp.com/demo2.php##Mimvp代理购买：#http://proxy.mimvp。com##mimvp.com#2015-11-09#http代理格式http_proxy=http://IP:Port#https代理格式https_proxy=http://IP:Port{'http':'http://120.77.176.179:8888'}curl-m30--retry3-xhttp://120.77.176.179:8888http://proxy.mimvp.com/exist.php#http_proxywget-T30--tries3-e"http_proxy=http://120.77.176.179:8888"http://proxy.mimvp.com/exist.php#http_proxy{'https':'http://46.105.214.133:3128'}curl-m30--retry3--proxy-insecure-xhttp://46.105.214.133:3128-khttps://proxy.mimvp.com/exist.php#https_proxywget-T30--tries3--no-check-certificate-e"https_proxy=http://46.105.214.133:3128"https://proxy.mimvp.com/exist.php#https_proxy#curl支持socks{'socks4':'101.255.17.145:1080'}curl-m30--retry3--socks4101.255.17.145:1080http://proxy.mimvp.com/exist.php{'socks5':'82.164.233.227:45454'}curl-m30--retry3--socks582.164.233.227:45454http://proxy.mimvp.com/exist.php#wget不支持袜子
wget配置文件设置代理vim~/.wgetrchttp_proxy=http://120.77.176.179:8888:8080https_proxy=http://12.7.17.17:8888:8080use_proxy=onwait=30wget-T30--tries3http://proxy.mimvp.com
Shellsettemporarylocalproxy#proxynoauthexporthttp_proxy=http://120.77.176.179:8888:8080exporthttps_proxy=http://12.7.17.17:8888:8080#proxyauthexporthttp_proxy=http://username:password@120.77.176.179:8888:8080exporthttps_proxy=http://username:password@12.7.17.17:8888:8080#Unsetunsethttp_proxyunsethttps_proxy
Shellsetsystemglobalproxy#Modify/etc/配置文件，保存并重启服务器sudovim/etc/profile#everyoneisvalidorsudovim~/.bashrc#everyoneisvalidorvim~/.bash_profile#personalisvalid#proxynoauthexporthttp_proxy=http://120.77.176.179:8888:8080exporthttps_proxy=http://12.7.17.17:8888:8080#代理authexporthttp_proxy=http://username:password@120.77.176.179:8888:8080exporthttps_proxy=http://username:password@12.7.17.17:8888:8080source/etc/profileorsource~/.bashrcorsource~/.bash_profilesudoreboot
米扑代理示例米扑代理示例，包括Python、Java、PHP、C#、Go、Perl、Ruby、Shell、NodeJS、PhantomJS、Groovy、Delphi、EasyLanguage等十余种编程语言或脚本，通过大量的操作实例，详细讲解了代理的正确使用方法IP、便捷的网页爬取、数据采集、自动化测试等领域。米扑代理示例，测试中使用的代理IP均来自米扑代理，覆盖120多个国家，中国34个省市，支持http、https、socks4、socks5等。米扑代理示例官网：http://proxy.mimvp.com/demo2.php
推荐参考：mimvp代理使用示例（推荐）mimvp-proxy-demo（GitHub）Linux网络爬虫示例（shell+awk）LinuxIP代理筛选系统（shell+代理人）
