当前位置: 首页 > 科技观察

lvs100000+并发优化实例

时间:2023-03-12 07:25:29 科技观察

1原因是在实际项目中,lvs只能承载很低的容量,需要对lvs和linuxkernel的参数进行优化。2为什么要使用lvs+keepalived架构(1)LVS可以实现负载均衡,但是不能进行健康检查。比如某个RS失效了,LVS还是会把请求转发给失效的RS服务器,这会导致请求失效;keepalived软件可以执行健康检查。(2)使用keepalived可以同时实现LVS的高可用,解决LVS的单点故障问题。3lvs+keepalived部署3.1部署图注:(1)lvs+keepalived至少需要2台服务器。(2)需要VIP。(3)RS服务器不能与LVS和Keepalived复用。4.2lvs+keepalived部署(2台DS服务器)安装ipvs和keepalivedyuinstallipvsadmkeepalived-ykeepalived配置注意:(1)keepalived是否需要竞争主IP,如果不需要,需要将状态改为BACKUP并配置nopreempt。(2)persistence_timeout的作用是:在一定时间内,将来自同一个Client的所有TCP请求加载到同一个RealServer,查看ipvsadm-S-nmasternode#masternode(MASTER)配置文件cat>/etc/keepalived/keepalived.conf<<'EOF'!ConfigurationFileforkeepalivedglobal_defs{router_idLVS_DEVEL}vrrp_instanceVI_1{stateBACKUP!主为master,不争抢模式改为BACKUPnopreempt!不争抢模式添加interfaceeth0virtual_router_id51priority100advert_int1authentication{auth_typePASSauth_pass1111}virtual_ipaddress{192.168.112.10}}virtual_server192.168.112.1080{delay_loop6lb_algorrlb_kindDR!persistence_timeout0protocolTCPreal_server192.168.112.1380{weight1TCP_CHECK{connect_timeout10retry3delay_before_retry3connect_port80}}real_server192.168.112.1480{weight1TCP_CHECK{connect_timeout10retry3delay_before_retry3connect_port80}}real_server192.168.112.1580{weight1TCP_CHECK{connect_timeout10retry3delay_before_retry3connect_port80}}}EOF从节点cat>/etc/keepalived/keepalived.conf<<'EOF'!ConfigurationFileforkeepalivedglobal_defs{router_idLVS_DEVEL}vrrp_instanceVI_1{stateBACKUPnopreempt!不争抢模式添加interfaceeth0virtual_router_id51priority90advert_int1authentication{auth_typePASSauth_pass1111}virtual_ipaddress{192.168.112.10}}virtual_server192.168.112.1080{delay_loop6lb_algorrlb_kindDR!persistence_timeout0protocolTCPreal_server192.168.112.1380{weight1TCP_CHECK{connect_timeout10retry3delay_before_retry3connect_port80}}real_server192.168.112.1480{weight1TCP_CHECK{connect_timeout10retry3delay_before_retry3connect_port80}}real_server192.168.112.1580{weight1TCP_CHECK{connect_timeout10retry3delay_before_retry3connect_port80}}}EOFkernelparametersecho1>/proc/sys/net/ipv4/ip_forwardsysctl-wnet.ipv4.ip_forward=1startsystemctlenablekeepalivedsystemctlstartkeepaliveds4.3realserverconfiguration脚本不需要需要配置在所有真实服务器上。注意vip必须和keepalived上的vip一样。#vimlvs_dr_rs.shSNS_VIP=192.168.112.10/etc/rc.d/init.d/functionscase"$1"instart)ifconfiglo:0$SNS_VIPnetmask255.255.255.255broadcast$SNS_VIP/sbin/routeadd-host$SNS_VIPdevlo:0echo"1">/proc/sys/net/ipv4/conf/lo/arp_ignoreecho"2">/proc/sys/net/ipv4/conf/lo/arp_announcecho"1">/proc/sys/net/ipv4/conf/all/arp_ignoreecho"2">/proc/sys/net/ipv4/conf/all/arp_announcesysctl-p>/dev/null2>&1echo"RealServerStartOK";;stop)ifconfiglo:0downroutedel$SNS_VIP>/dev/null2>&1echo"0">/proc/sys/net/ipv4/conf/lo/arp_ignoreecho"0">/proc/sys/net/ipv4/conf/lo/arp_announcecho"0">/proc/sys/net/ipv4/conf/all/arp_ignoreecho"0">/proc/sys/net/ipv4/conf/all/arp_announceecho"RealServerStoped";;*)echo"Usage:$0{start|stop}"exit1esacexit0执行生效#在所有RS节点上执行脚本:chmod+xlvs_dr_rs.sh./lvs_dr_rs.shstart5参数优化5.1LVS参数增加ipvs模块有h表的大小ipvs模块的哈希表默认值为2^12=4096。如果改为2^20=1048576,可以使用ipvsadm-l命令查询当前哈希表的大小。IPVirtualServerversion1.2.1(size=4096)修改方法:在/etc/modprobe.d/目录下添加文件ip_vs.conf,内容为:optionsip_vsconn_tab_bits=20重新加载ipvs模块。ipvirtualServerversion1.2.1(尺寸=1048576)修改lvs表表表表timeoutipvsadm--set90060300ipvsadm-ln-pimetimeouttimeout(tcptcppfinudp):900603005.21048576net.ipv4.ip_forward=1net.core.wmem_default=8388608net.core.wmem_max=16777216net.core.rmem_default=8388608net.core.rmem_max=16777216net.core.somaxconn=65535net.core.optmem_max=81920net.core.backlognet1.netdev_max2ipv4.route.gc_timeout=20net.ipv4.tcp_syncookies=1net.ipv4.tcp_abort_on_overflow=1net.ipv4.tcp_max_tw_buckets=6000net.ipv4.tcp_sack=1net.ipv4.tcp_window_scaling=1net.ipv4.tcp_no_metrics_save=1net.ipv4.tcp_rmem=3276813107216777216net.ipv4.tcp_wmem=819213107216777216net.ipv4.tcp_mem=94500000915000000927000000net.ipv4.tcp_max_syn_backlog=262144net.ipv4.tcp_max_orphans=3276800net.ipv4.tcp_timestamps=0net.ipv4.tcp_synack_retries=1net.ipv4.tcp_syn_retries=1net.ipv4.tcp_tw_recycle=1net.ipv4.tcp_tw_reuse=1net.ipv4.tcp_fin_timeout=10net.ipv4.tcp_keepalive_time=120net.ipv4.tcp_keepalive_probes=3net.ipv4.tcp_keepalive_intvl=15net.ipv4.tcp_retries2=5net.ipv4.ip_local_port_range=102465000net.ipv4.tcp_keepalive_intvl=15net.ipv4.ip_local_port_range=102465000net.ipv4.tcp_keepalive_intvl=15net.ipv4.ip_local_port_range=102465000net.ipv4.tcp_keepalive_intvl.=0net.ipv4.conf.all.arp_ignore=1net.ipv4.conf.all.arp_announce=2#modprobeip_conntracknet.netfilter.nf_conntrack_tcp_timeout_established=180net.netfilter.nf_conntrack_max=1048576net.nf_conntrack_max=1048576kernel.sysrq=0内核。core_uses_pid=1kernel.msgmnb=65536kernel.msgmax=65536kernel.shmmax=68719476736kernel.shmall=4294967296注意:net.ipv4.tcp_tw_recycle=1有坑,nat环境下慎用越并发越要注意net.ipv4.tcp_max_tw的值不能太大。