LVS (Linux Virtual Server)구축 : ldirectord + heartbeat
L4 스위치의 부하 분산 기능을 리눅스 서버를 통해 진행하고자 한다.
서버의 구성은 아래와 같이 하면 될 것 같다.
VIP : 121.254.175.70 LVS1 : 121.254.175.77 LVS2 : 121.254.175.78 WEB1 : 121.254.175.75 WEB2 : 121.254.175.74 |
임의 설정한 호스트네임 정보를 각 LVS 서버에 등록을 해야 한다.
/etc/hosts /etc/sysconfig/network [Hostname] LVS Active: lvsactive.gunnm.xyz LVS Standby: lvsstandby.gunnm.xyz |
1. LVS Active 서버에서의 설정 : 121.254.175.77
# echo 'lvsactive.gunnm.xyz' > /proc/sys/kernel/hostname |
# vi /etc/sysconfig/network NETWORKING=yes HOSTNAME=lvsactive.gunnm.xyz |
# vi /etc/hosts 121.254.175.77 lvsactive.gunnm.xyz 121.254.175.78 lvsstandby.gunnm.xyz |
# vi /etc/sysctl.conf net.ipv4.ip_forward = 1 # 하단에 추가 net.ipv4.vs.secure_tcp=3 net.ipv4.vs.drop_packet=0 net.ipv4.vs.drop_entry=0 net.ipv4.vs.am_droprate=10 net.ipv4.vs.amemthresh=1024 # sysctl -p |
heartbeat + ldirectord 설치 # rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm # yum install heartbeat* -y # yum -y install ipvsadm # yum -y install perl-IO-Socket-INET6 # yum -y install perl-Email-Send # yum -y install perl-MailTools # yum -y install perl-Net-SSLeay # yum -y install perl-libwww-perl # yum install ldirect* 빨간색은 실패하는 설치 패키지이다. 아래의 링크를 통해 ldirectord 파일 다운로드 및 설치를 진행해야 한다. - 32bit # wget ftp://ftp.pbone.net/mirror/ftp5.gwdg.de/pub/opensuse/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/i686/ldirectord-3.9.5-3.1.i686.rpm - 64bit # wget ftp://ftp.pbone.net/mirror/ftp5.gwdg.de/pub/opensuse/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/x86_64/ldirectord-3.9.5-3.1.x86_64.rpm # rpm -ivh ldirectord-3.9.5-3.1.x86_64.rpm |
HA 설정 (Standby 의 상태 체크) /usr/share/doc/heartbeat-3.0.4/ha.cf 를 복사해와도 되지만 그냥 새로 만들고 아래 내용을 입력한다. # vi /etc/ha.d/ha.cf debugfile /var/log/ha-debug.log logfile /var/log/ha.log keepalive 2 deadtime 10 udpport 694 ucast eth0 121.254.175.78 node lvsactive.gunnm.xyz node lvsstandby.gunnm.xyz ping 121.254.175.65 auto_failback on respawn hacluster /usr/lib64/heartbeat/ipfail apiauth ipfail gid=haclient uid=hacluster |
가상IP 를 설정한다. # vi /etc/ha.d/haresources lvsactive.gunnm.xyz \ldirectord::ldirectord.cf \ IPaddr2::121.254.175.70/32/eth0 |
authkey 를 생성한다. # vi /etc/ha.d/authkeys auth 2 2 sha1 gunnmlvs root 만 읽을수 있도록 권한을 설정한다. # chmod 600 /etc/ha.d/authkeys -rw-------. 1 root root 23 2018-08-08 13:07 /etc/ha.d/authkeys |
분배 설정을 진행한다. # vi /etc/ha.d/ldirectord.cf checktimeout=10 checkinterval=10 autoreload=no logfile="/var/log/ldirectord.log" quiescent=no # Web Server LVS virtual=121.254.175.70:80 real=121.254.175.75:80 gate 10 real=121.254.175.74:80 gate 10 service=http request="gunnm.lvs" receive="gunnm" scheduler=wlc persistent=3600 protocol=tcp checktype=negotiate |
세션 동기화를 위한 방화벽 룰셋 추가 # iptables -t nat -A OUTPUT -d 224.0.0.81 -p udp --dport 8848 -j DNAT --to 121.254.175.78:8848 # iptables -t nat -A PREROUTING -s 121.254.175.78 -p udp --dport 8848 -j DNAT --to 224.0.0.81:8848 # iptables -t mangle -A PREROUTING -p tcp -s 0.0.0.0/0 -d 121.254.175.70/32 --dport 80 -j MARK --set-mark 1 # iptables -t mangle -A PREROUTING -p tcp -s 0.0.0.0/0 -d 121.254.175.70/32 --dport 443 -j MARK --set-mark 1 # service iptables save # service iptables restart |
2. LVS Standby 서버에서의 설정 : 121.254.175.78
# echo 'lvsstandby.gunnm.xyz' > /proc/sys/kernel/hostname |
# vi /etc/sysconfig/network NETWORKING=yes HOSTNAME=lvsstandby.gunnm.xyz |
# vi /etc/hosts 121.254.175.77 lvsactive.gunnm.xyz 121.254.175.78 lvsstandby.gunnm.xyz |
# vi /etc/sysctl.conf net.ipv4.ip_forward = 1 # 하단에 추가 net.ipv4.vs.secure_tcp=3 net.ipv4.vs.drop_packet=0 net.ipv4.vs.drop_entry=0 net.ipv4.vs.am_droprate=10 net.ipv4.vs.amemthresh=1024 # sysctl -p |
heartbeat + ldirectord 설치 # rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm # yum install heartbeat* -y # yum -y install ipvsadm # yum -y install perl-IO-Socket-INET6 # yum -y install perl-Email-Send # yum -y install perl-MailTools # yum -y install perl-Net-SSLeay # yum -y install perl-libwww-perl # yum install ldirect* 빨간색은 실패하는 설치 패키지이다. 아래의 링크를 통해 ldirectord 파일 다운로드 및 설치를 진행해야 한다. - 32bit # wget ftp://ftp.pbone.net/mirror/ftp5.gwdg.de/pub/opensuse/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/i686/ldirectord-3.9.5-3.1.i686.rpm - 64bit # wget ftp://ftp.pbone.net/mirror/ftp5.gwdg.de/pub/opensuse/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-6/x86_64/ldirectord-3.9.5-3.1.x86_64.rpm # rpm -ivh ldirectord-3.9.5-3.1.x86_64.rpm |
HA 설정 (Active 의 상태 체크) /usr/share/doc/heartbeat-3.0.4/ha.cf 를 복사해와도 되지만 그냥 새로 만들고 아래 내용을 입력한다. # vi /etc/ha.d/ha.cf debugfile /var/log/ha-debug.log logfile /var/log/ha.log keepalive 2 deadtime 10 udpport 694 ucast eth0 121.254.175.77 node lvsactive.gunnm.xyz node lvsstandby.gunnm.xyz ping 121.254.175.65 auto_failback on respawn hacluster /usr/lib64/heartbeat/ipfail apiauth ipfail gid=haclient uid=hacluster |
가상IP 를 설정한다. # vi /etc/ha.d/haresources lvsstandby.gunnm.xyz \ldirectord::ldirectord.cf \ IPaddr2::121.254.175.70/32/eth0 |
authkey 를 생성한다. # vi /etc/ha.d/authkeys auth 2 2 sha1 gunnmlvs root 만 읽을수 있도록 권한을 설정한다. # chmod 600 /etc/ha.d/authkeys -rw-------. 1 root root 23 2018-08-08 14:12 /etc/ha.d/authkeys |
분배 설정을 진행한다. # vi /etc/ha.d/ldirectord.cf checktimeout=10 checkinterval=10 autoreload=no logfile="/var/log/ldirectord.log" quiescent=no # Web Server LVS virtual=121.254.175.70:80 real=121.254.175.74:80 gate 10 real=121.254.175.75:80 gate 10 service=http request="gunnm.lvs" receive="gunnm" scheduler=wlc persistent=3600 protocol=tcp checktype=negotiate |
세션 동기화를 위한 방화벽 룰셋 추가 # iptables -t nat -A OUTPUT -d 224.0.0.81 -p udp --dport 8848 -j DNAT --to 121.254.175.77:8848 # iptables -t nat -A PREROUTING -s 121.254.175.77 -p udp --dport 8848 -j DNAT --to 224.0.0.81:8848 # iptables -t mangle -A PREROUTING -p tcp -s 0.0.0.0/0 -d 121.254.175.70/32 --dport 80 -j MARK --set-mark 1 # iptables -t mangle -A PREROUTING -p tcp -s 0.0.0.0/0 -d 121.254.175.70/32 --dport 443 -j MARK --set-mark 1 # service iptables save # service iptables restart |
3. LVS Active 및 Standby 서버에서 Heartbeat 구동
# service heartbeat start Starting High-Availability services: ERROR: Cannot locate resource script \ldirectord /usr/share/heartbeat/ResourceManager: line 209: ldirectord.cf: command not found Done. 서비스 구동이 실패한다! 실패 원인을 확인해 본다. /etc/ha.d/ldirectord.cf 의 구문 띄어쓰기에 문제가 있었음이 확인됐다. (별 말도 안되는 걸로 오류가 남) checktimeout=10 checkinterval=10 autoreload=no logfile="/var/log/ldirectord.log" quiescent=no # Web Server LVS virtual=121.254.175.70:80 real=121.254.175.75:80 gate 10 real=121.254.175.74:80 gate 10 service=http request="gunnm.lvs" receive="gunnm" scheduler=wlc persistent=3600 protocol=tcp checktype=negotiate 이렇게 수정하고 나니깐 정상적으로 작동 된다. LVS Active 에서 # ip addr 을 통해서 아이피 두 개가 활성화 된 것을 확인 할 수 있다. 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 5e:df:dd:9f:3b:e0 brd ff:ff:ff:ff:ff:ff inet 121.254.175.77/28 brd 121.254.175.79 scope global eth0 inet 121.254.175.70/32 brd 121.254.175.79 scope global eth0 inet6 fe80::5cdf:ddff:fe9f:3be0/64 scope link valid_lft forever preferred_lft forever LVS Standby 에서는 아래와 같이 확인 된다. # ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 2a:8b:f9:ad:f0:30 brd ff:ff:ff:ff:ff:ff inet 121.254.175.78/28 brd 121.254.175.79 scope global eth0 inet6 fe80::288b:f9ff:fead:f030/64 scope link valid_lft forever preferred_lft forever |
4. 강제 failover 발생시 구동 상태 확인
LVS Active 에서 heartbeat 를 중지한다. # service heartbeat stop Stopping High-Availability services: Done. # ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 5e:df:dd:9f:3b:e0 brd ff:ff:ff:ff:ff:ff inet 121.254.175.77/28 brd 121.254.175.79 scope global eth0 inet6 fe80::5cdf:ddff:fe9f:3be0/64 scope link valid_lft forever preferred_lft forever # vi /var/log/ha.log ResourceManager(default)[10507]: 2018/08/08_16:33:14 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf stop Aug 08 16:33:15 lvsactive.gunnm.xyz heartbeat: [10494]: info: All HA resources relinquished. Aug 08 16:33:16 lvsactive.gunnm.xyz heartbeat: [9790]: info: killing /usr/lib64/heartbeat/ipfail process group 9816 with signal 15 Aug 08 16:33:17 lvsactive.gunnm.xyz heartbeat: [9790]: WARN: 1 lost packet(s) for [lvsstandby.gunnm.xyz] [542:544] Aug 08 16:33:17 lvsactive.gunnm.xyz heartbeat: [9790]: info: No pkts missing from lvsstandby.gunnm.xyz! Aug 08 16:33:18 lvsactive.gunnm.xyz heartbeat: [9790]: info: killing HBFIFO process 9793 with signal 15 Aug 08 16:33:18 lvsactive.gunnm.xyz heartbeat: [9790]: info: killing HBWRITE process 9794 with signal 15 Aug 08 16:33:18 lvsactive.gunnm.xyz heartbeat: [9790]: info: killing HBREAD process 9795 with signal 15 Aug 08 16:33:18 lvsactive.gunnm.xyz heartbeat: [9790]: info: killing HBWRITE process 9796 with signal 15 Aug 08 16:33:18 lvsactive.gunnm.xyz heartbeat: [9790]: info: killing HBREAD process 9797 with signal 15 Aug 08 16:33:18 lvsactive.gunnm.xyz heartbeat: [9790]: info: Core process 9797 exited. 5 remaining Aug 08 16:33:18 lvsactive.gunnm.xyz heartbeat: [9790]: info: Core process 9794 exited. 4 remaining Aug 08 16:33:18 lvsactive.gunnm.xyz heartbeat: [9790]: info: Core process 9795 exited. 3 remaining Aug 08 16:33:18 lvsactive.gunnm.xyz heartbeat: [9790]: info: Core process 9793 exited. 2 remaining Aug 08 16:33:18 lvsactive.gunnm.xyz heartbeat: [9790]: info: Core process 9796 exited. 1 remaining Aug 08 16:33:18 lvsactive.gunnm.xyz heartbeat: [9790]: info: lvsactive.gunnm.xyz Heartbeat shutdown complete. LVS Standby 에서 상태 확인 한다. # ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 2a:8b:f9:ad:f0:30 brd ff:ff:ff:ff:ff:ff inet 121.254.175.78/28 brd 121.254.175.79 scope global eth0 inet 121.254.175.70/32 brd 121.254.175.79 scope global eth0 inet6 fe80::288b:f9ff:fead:f030/64 scope link valid_lft forever preferred_lft forever # vi /var/log/ha.log Aug 08 16:33:16 lvsstandby.gunnm.xyz heartbeat: [1415]: info: Received shutdown notice from 'lvsactive.gunnm.xyz'. Aug 08 16:33:16 lvsstandby.gunnm.xyz heartbeat: [1415]: info: Resources being acquired from lvsactive.gunnm.xyz. Aug 08 16:33:16 lvsstandby.gunnm.xyz heartbeat: [3108]: info: acquire all HA resources (standby). ResourceManager(default)[3135]: 2018/08/08_16:33:16 info: Acquiring resource group: lvsstandby.gunnm.xyz ldirectord::ldirectord.cf IPaddr2::121.254.175.70/32/eth0 Aug 08 16:33:16 lvsstandby.gunnm.xyz heartbeat: [3109]: info: Local Resource acquisition completed. ResourceManager(default)[3135]: 2018/08/08_16:33:16 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf start /usr/lib/ocf/resource.d//heartbeat/IPaddr2(IPaddr2_121.254.175.70)[3224]: 2018/08/08_16:33:17 INFO: Resource is stopped ResourceManager(default)[3135]: 2018/08/08_16:33:17 info: Running /etc/ha.d/resource.d/IPaddr2 121.254.175.70/32/eth0 start IPaddr2(IPaddr2_121.254.175.70)[3356]: 2018/08/08_16:33:17 INFO: Adding inet address 121.254.175.70/32 with broadcast address 121.254.175.79 to device eth0 IPaddr2(IPaddr2_121.254.175.70)[3356]: 2018/08/08_16:33:17 INFO: Bringing device eth0 up Active 다운시 자동으로 Standby 가 역할 전담하는 것을 확인했다. 다시 Active 활성화시의 상태를 확인한다. Active log Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10859]: info: Pacemaker support: false Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10859]: WARN: Logging daemon is disabled --enabling logging daemon is recommended Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10859]: info: ************************** Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10859]: info: Configuration validated. Starting heartbeat 3.0.4 Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: heartbeat: version 3.0.4 Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: Heartbeat generation: 1533705889 Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: glib: ucast: write socket priority set to IPTOS_LOWDELAY on eth0 Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: glib: ucast: bound send socket to device: eth0 Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: glib: ucast: set SO_REUSEPORT(w) Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: glib: ucast: bound receive socket to device: eth0 Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: glib: ucast: set SO_REUSEPORT(w) Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: glib: ucast: started on port 694 interface eth0 to 121.254.175.78 Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: glib: ping heartbeat started. Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: G_main_add_TriggerHandler: Added signal manual handler Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: G_main_add_TriggerHandler: Added signal manual handler Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: G_main_add_SignalHandler: Added signal handler for signal 17 Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: Local status now set to: 'up' Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: Link 121.254.175.65:121.254.175.65 up. Aug 08 16:37:54 lvsactive.gunnm.xyz heartbeat: [10860]: info: Status update for node 121.254.175.65: status ping Aug 08 16:37:56 lvsactive.gunnm.xyz heartbeat: [10860]: info: Link lvsstandby.gunnm.xyz:eth0 up. Aug 08 16:37:56 lvsactive.gunnm.xyz heartbeat: [10860]: info: Status update for node lvsstandby.gunnm.xyz: status active harc(default)[10869]: 2018/08/08_16:37:56 info: Running /etc/ha.d//rc.d/status status Aug 08 16:37:56 lvsactive.gunnm.xyz heartbeat: [10860]: info: Comm_now_up(): updating status to active Aug 08 16:37:56 lvsactive.gunnm.xyz heartbeat: [10860]: info: Local status now set to: 'active' Aug 08 16:37:56 lvsactive.gunnm.xyz heartbeat: [10860]: info: Starting child client "/usr/lib64/heartbeat/ipfail" (496,495) Aug 08 16:37:56 lvsactive.gunnm.xyz heartbeat: [10886]: info: Starting "/usr/lib64/heartbeat/ipfail" as uid 496 gid 495 (pid 10886) Aug 08 16:37:57 lvsactive.gunnm.xyz heartbeat: [10860]: info: remote resource transition completed. Aug 08 16:37:57 lvsactive.gunnm.xyz heartbeat: [10860]: info: remote resource transition completed. Aug 08 16:37:57 lvsactive.gunnm.xyz heartbeat: [10860]: info: Local Resource acquisition completed. (none) Aug 08 16:37:57 lvsactive.gunnm.xyz heartbeat: [10860]: info: lvsstandby.gunnm.xyz wants to go standby [foreign] Aug 08 16:37:58 lvsactive.gunnm.xyz heartbeat: [10860]: info: standby: acquire [foreign] resources from lvsstandby.gunnm.xyz Aug 08 16:37:58 lvsactive.gunnm.xyz heartbeat: [10890]: info: acquire local HA resources (standby). ResourceManager(default)[10903]: 2018/08/08_16:37:58 info: Acquiring resource group: lvsactive.gunnm.xyz ldirectord::ldirectord.cf IPaddr2::121.254.175.70/32/eth0 ResourceManager(default)[10903]: 2018/08/08_16:37:58 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf start /usr/lib/ocf/resource.d//heartbeat/IPaddr2(IPaddr2_121.254.175.70)[10967]: 2018/08/08_16:37:59 INFO: Resource is stopped ResourceManager(default)[10903]: 2018/08/08_16:37:59 info: Running /etc/ha.d/resource.d/IPaddr2 121.254.175.70/32/eth0 start IPaddr2(IPaddr2_121.254.175.70)[11098]: 2018/08/08_16:37:59 INFO: Adding inet address 121.254.175.70/32 with broadcast address 121.254.175.79 to device eth0 IPaddr2(IPaddr2_121.254.175.70)[11098]: 2018/08/08_16:37:59 INFO: Bringing device eth0 up IPaddr2(IPaddr2_121.254.175.70)[11098]: 2018/08/08_16:37:59 INFO: /usr/libexec/heartbeat/send_arp -i 200 -r 5 -p /var/run/resource-agents/send_arp-121.254.175.70 eth0 121.254.175.70 auto not_used not_used /usr/lib/ocf/resource.d//heartbeat/IPaddr2(IPaddr2_121.254.175.70)[11072]: 2018/08/08_16:37:59 INFO: Success Aug 08 16:37:59 lvsactive.gunnm.xyz heartbeat: [10890]: info: local HA resource acquisition completed (standby). Aug 08 16:37:59 lvsactive.gunnm.xyz heartbeat: [10860]: info: Standby resource acquisition done [foreign]. Aug 08 16:37:59 lvsactive.gunnm.xyz heartbeat: [10860]: info: Initial resource acquisition complete (auto_failback) Aug 08 16:38:00 lvsactive.gunnm.xyz heartbeat: [10860]: info: remote resource transition completed. Aug 08 16:38:06 lvsactive.gunnm.xyz ipfail: [10886]: info: Telling other node that we have more visible ping nodes. Aug 08 16:38:12 lvsactive.gunnm.xyz heartbeat: [10860]: info: lvsstandby.gunnm.xyz wants to go standby [all] Aug 08 16:38:13 lvsactive.gunnm.xyz heartbeat: [10860]: info: standby: acquire [all] resources from lvsstandby.gunnm.xyz Aug 08 16:38:13 lvsactive.gunnm.xyz heartbeat: [11183]: info: acquire all HA resources (standby). ResourceManager(default)[11196]: 2018/08/08_16:38:13 info: Acquiring resource group: lvsactive.gunnm.xyz ldirectord::ldirectord.cf IPaddr2::121.254.175.70/32/eth0 ResourceManager(default)[11196]: 2018/08/08_16:38:14 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf start /usr/lib/ocf/resource.d//heartbeat/IPaddr2(IPaddr2_121.254.175.70)[11254]: 2018/08/08_16:38:14 INFO: Running OK Aug 08 16:38:14 lvsactive.gunnm.xyz heartbeat: [11183]: info: all HA resource acquisition completed (standby). Aug 08 16:38:14 lvsactive.gunnm.xyz heartbeat: [10860]: info: Standby resource acquisition done [all]. Aug 08 16:38:15 lvsactive.gunnm.xyz heartbeat: [10860]: info: remote resource transition completed. Standby log Aug 08 16:37:55 lvsstandby.gunnm.xyz heartbeat: [1415]: info: Heartbeat restart on node lvsactive.gunnm.xyz Aug 08 16:37:55 lvsstandby.gunnm.xyz heartbeat: [1415]: info: Link lvsactive.gunnm.xyz:eth0 up. Aug 08 16:37:55 lvsstandby.gunnm.xyz heartbeat: [1415]: info: Status update for node lvsactive.gunnm.xyz: status init Aug 08 16:37:55 lvsstandby.gunnm.xyz heartbeat: [1415]: info: Status update for node lvsactive.gunnm.xyz: status up harc(default)[3761]: 2018/08/08_16:37:55 info: Running /etc/ha.d//rc.d/status status harc(default)[3777]: 2018/08/08_16:37:55 info: Running /etc/ha.d//rc.d/status status Aug 08 16:37:57 lvsstandby.gunnm.xyz heartbeat: [1415]: info: Status update for node lvsactive.gunnm.xyz: status active harc(default)[3793]: 2018/08/08_16:37:57 info: Running /etc/ha.d//rc.d/status status Aug 08 16:37:58 lvsstandby.gunnm.xyz heartbeat: [1415]: info: remote resource transition completed. Aug 08 16:37:58 lvsstandby.gunnm.xyz heartbeat: [1415]: info: lvsstandby.gunnm.xyz wants to go standby [foreign] Aug 08 16:37:58 lvsstandby.gunnm.xyz heartbeat: [1415]: info: standby: lvsactive.gunnm.xyz can take our foreign resources Aug 08 16:37:58 lvsstandby.gunnm.xyz heartbeat: [3809]: info: give up foreign HA resources (standby). Aug 08 16:37:58 lvsstandby.gunnm.xyz heartbeat: [3809]: info: foreign HA resource release completed (standby). Aug 08 16:37:58 lvsstandby.gunnm.xyz heartbeat: [1415]: info: Local standby process completed [foreign]. Aug 08 16:38:00 lvsstandby.gunnm.xyz heartbeat: [1415]: WARN: 1 lost packet(s) for [lvsactive.gunnm.xyz] [12:14] Aug 08 16:38:00 lvsstandby.gunnm.xyz heartbeat: [1415]: info: remote resource transition completed. Aug 08 16:38:00 lvsstandby.gunnm.xyz heartbeat: [1415]: info: No pkts missing from lvsactive.gunnm.xyz! Aug 08 16:38:00 lvsstandby.gunnm.xyz heartbeat: [1415]: info: Other node completed standby takeover of foreign resources. Aug 08 16:38:12 lvsstandby.gunnm.xyz heartbeat: [1415]: info: lvsstandby.gunnm.xyz wants to go standby [all] Aug 08 16:38:13 lvsstandby.gunnm.xyz heartbeat: [1415]: info: standby: lvsactive.gunnm.xyz can take our all resources Aug 08 16:38:13 lvsstandby.gunnm.xyz heartbeat: [3849]: info: give up all HA resources (standby). ResourceManager(default)[3862]: 2018/08/08_16:38:13 info: Releasing resource group: lvsstandby.gunnm.xyz ldirectord::ldirectord.cf IPaddr2::121.254.175.70/32/eth0 ResourceManager(default)[3862]: 2018/08/08_16:38:13 info: Running /etc/ha.d/resource.d/IPaddr2 121.254.175.70/32/eth0 stop IPaddr2(IPaddr2_121.254.175.70)[3923]: 2018/08/08_16:38:13 INFO: IP status = ok, IP_CIP= /usr/lib/ocf/resource.d//heartbeat/IPaddr2(IPaddr2_121.254.175.70)[3897]: 2018/08/08_16:38:13 INFO: Success ResourceManager(default)[3862]: 2018/08/08_16:38:13 info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf stop Aug 08 16:38:13 lvsstandby.gunnm.xyz heartbeat: [3849]: info: all HA resource release completed (standby). Aug 08 16:38:13 lvsstandby.gunnm.xyz heartbeat: [1415]: info: Local standby process completed [all]. Aug 08 16:38:15 lvsstandby.gunnm.xyz heartbeat: [1415]: WARN: 1 lost packet(s) for [lvsactive.gunnm.xyz] [26:28] Aug 08 16:38:15 lvsstandby.gunnm.xyz heartbeat: [1415]: info: remote resource transition completed. Aug 08 16:38:15 lvsstandby.gunnm.xyz heartbeat: [1415]: info: No pkts missing from lvsactive.gunnm.xyz! Aug 08 16:38:15 lvsstandby.gunnm.xyz heartbeat: [1415]: info: Other node completed standby takeover of all resources. Active 와 Standby 가 알아서 역할을 잘 설정한다. |
5. 실시간 모니터링 및 ldirectord 설정 변경시 적용 파일 생성
# vi /root/lvs.sh #!/bin/sh watch -n1 -t "echo ; ip addr ; echo ; ipvsadm -Ln ; ipvsadm -Ln --rate ; ipvsadm -Lnc" # vi /root/ldirect.sh (/etc/ha.d/ldirectord.cf 파일 내용 수정 후, reload 하는 파일) #!/bin/sh /etc/ha.d/resource.d/ldirectord /etc/ha.d/ldirectord.cf reload Active 에서 # sh lvs.sh 입력시 접속 세션 확인 가능 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 5e:df:dd:9f:3b:e0 brd ff:ff:ff:ff:ff:ff inet 121.254.175.77/28 brd 121.254.175.79 scope global eth0 inet 121.254.175.70/32 brd 121.254.175.79 scope global eth0 inet6 fe80::5cdf:ddff:fe9f:3be0/64 scope link valid_lft forever preferred_lft forever IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 121.254.175.70:80 wlc persistent 3600 -> 121.254.175.74:80 Route 10 0 1 -> 121.254.175.75:80 Route 10 5 2 IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port CPS InPPS OutPPS InBPS OutBPS -> RemoteAddress:Port TCP 121.254.175.70:80 0 1 0 85 0 -> 121.254.175.74:80 0 0 0 38 0 -> 121.254.175.75:80 0 0 0 47 0 IPVS connection entries pro expire state source virtual destination TCP 48:09 NONE 92.31.1.6:0 121.254.175.70:80 121.254.175.75:80 TCP 14:37 ESTABLISHED 222.231.27.246:6060 121.254.175.70:80 121.254.175.75:80 TCP 05:11 ESTABLISHED 222.231.27.246:6005 121.254.175.70:80 121.254.175.75:80 TCP 14:36 ESTABLISHED 222.231.27.246:6064 121.254.175.70:80 121.254.175.75:80 TCP 01:08 FIN_WAIT 222.231.27.246:6061 121.254.175.70:80 121.254.175.75:80 TCP 01:36 FIN_WAIT 222.231.27.246:6062 121.254.175.70:80 121.254.175.75:80 TCP 05:11 ESTABLISHED 222.231.27.246:6020 121.254.175.70:80 121.254.175.75:80 TCP 00:00 CLOSE 118.69.170.35:60980 121.254.175.70:80 121.254.175.74:80 TCP 14:47 ESTABLISHED 121.254.171.226:52493 121.254.175.70:80 121.254.175.75:80 TCP 59:49 NONE 118.69.170.35:0 121.254.175.70:80 121.254.175.74:80 TCP 59:17 NONE 121.254.171.226:0 121.254.175.70:80 121.254.175.75:80 TCP 59:36 NONE 222.231.27.246:0 121.254.175.70:80 121.254.175.75:80 Standby 에서는 아무것도 연결이 되어 있지 않다. 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 2a:8b:f9:ad:f0:30 brd ff:ff:ff:ff:ff:ff inet 121.254.175.78/28 brd 121.254.175.79 scope global eth0 inet 121.254.175.70/32 brd 121.254.175.79 scope global eth0 inet6 fe80::288b:f9ff:fead:f030/64 scope link valid_lft forever preferred_lft forever IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 121.254.175.70:80 wlc persistent 3600 -> 121.254.175.75:80 Route 10 3 0 IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port CPS InPPS OutPPS InBPS OutBPS -> RemoteAddress:Port TCP 121.254.175.70:80 0 0 0 0 0 -> 121.254.175.75:80 0 0 0 0 0 IPVS connection entries pro expire state source virtual destination TCP 11:27 ESTABLISHED 222.231.27.246:5992 121.254.175.70:80 121.254.175.75:80 TCP 56:27 NONE 222.231.27.246:0 121.254.175.70:80 121.254.175.75:80 TCP 11:27 ESTABLISHED 222.231.27.246:5991 121.254.175.70:80 121.254.175.75:80 TCP 11:27 ESTABLISHED 222.231.27.246:5993 121.254.175.70:80 121.254.175.75:80 |
이번 테스트는 동일 아이피 대역의 서버들끼로 진행을 했으니깐 간단하게 설정이 되었으나, 대역이 다른 서버들간의 구성시에는 라우트 추가가 진행되어야 한다.
해당 방법은 차후에 다시 해보기로 한다.
'리눅스 공부' 카테고리의 다른 글
| 리눅스 / 웹사이트 트래픽 제한 : cband 모듈 설치 (0) | 2018.04.04 |
|---|---|
| 리눅스 / 현재 시간 동기화 (0) | 2018.03.29 |
| 리눅스 / DNS 서버 구축 (1) | 2018.03.23 |
| 리눅스 / 워드프레스 설치 (0) | 2018.03.12 |
| resolv.conf 초기화 해결 (0) | 2018.03.09 |
리눅스 / 웹사이트 트래픽 제한 : cband 모듈 설치
웹호스팅 서비스를 제공하기 위해서는 필수적으로 각 웹사이트에 대한 트래픽 제한이 필요하다.
apache 에 적용 가능한 트래픽 및 대역폭 제한 모듈인 cband 설치 테스트를 진행 하였다.
테스트 환경 :
OS : CentOS release 6.9 (Final)
Apache : Apache/2.2.15 (Unix)
1. cbnad 모듈 설치.
설치 경로로 이동
# cd /usr/local |
wget 을 이용하여 설치 파일 다운로드
# wget http://cband.linux.pl/download/mod-cband-0.9.7.4.tgz |
하였는데, 다운로드가 실패 한다.
개발 웹사이트가 폐쇄되어 파일 다운로드 불가하다.
다른 사이트에서 다운로드 받은 후에 설치 경로에 파일 업로드 한다.
http://linux.softpedia.com/get/Internet/HTTP-WWW-/mod-cband-4737.shtml#download
압축 해제 및 경로 진입
# tar xvfpz mod-cband-0.9.7.5.tgz # cd mod-cband-0.9.7.5 |
configure 실행
# ./configure --with-apxs=/usr/sbin/apxs |
생성 실패한다.
로그를 확인해 본다.
configure:1340: checking for gcc configure:1369: result: no configure:1420: checking for cc configure:1449: result: no configure:1462: checking for cc configure:1508: result: no configure:1561: checking for cl configure:1590: result: no configure:1604: error: no acceptable C compiler found in $PATH |
gcc 가 설치가 되어 있지 않아서 발생하는 오류 이므로 설치를 진행 한다.
# yum install gcc -y |
gcc 설치도 했으니 다시 configure 실행한다.
# ./configure --with-apxs=/usr/sbin/apxs |
다른 에러가 나온다.
configure: error: Invalid location for apxs: '/usr/sbin/apxs |
원인 : apsx 경로 확인 불가
apsx 경로를 찾기 위해 명령어를 입력한다.
# find / -name "apxs" -print |
아무런 결과도 출력되지 않는다.
yum 을 이용해 apache 를 설치 했기 때문에, apsx 미등록 상태 였다.
httpd-devel 패키지 설치를 진행한다.
# yum install httpd-devel -y |
경로 재확인
# find / -name "apxs" -print /usr/sbin/apxs |
configure 및 make 실행
# ./configure --with-apxs=/usr/sbin/apxs # make # make install |
정상적으로 설치가 되었음을 확인했다.
chmod 755 /usr/lib64/httpd/modules/mod_cband.so [activating module `cband' in /etc/httpd/conf/httpd.conf] |
2. cbnad 설정 파일 생성.
httpd.conf 에 직접 설정을 추가해도 되지만, 관리의 용이성을 위하여 개별 설정 파일을 만든다.
# vi /etc/httpd/conf.d/cband.conf <IfModule mod_cband.c> <Location /cband-status> SetHandler cband-status </Location> <Location /cband-status-me> SetHandler cband-status-me </Location> <Location /~*/cband-status-me> SetHandler cband-status-me </Location> <Location /cband-status> Order deny,allow Deny from all Allow from 192.168.1.100/32 // 관리 페이지 접근 허용 아이피 </Location> </IfModule> |
3. vhost 설정
모듈 적용이 필요한 웹사이트의 vhost 를 수정한다.
(일일 트래픽 용량 5G 로 제한)
# vi /etc/httpd/conf/httpd.conf <VirtualHost *:80> ServerName gunnm.xyz ServerAlias www.gunnm.xyz Documentroot /var/www/html CBandLimit 5000M CBandPeriod 1D </VirtualHost> |
서비스 재시작
# service httpd restart |
웹브라우저에서 설정 확인
gunnm.xyz/cband-status
추가 옵션값은 아래의 링크에서 확인이 가능하다. 설명이 상세히 잘 되어 있다.
https://www.linux.co.kr/home/lecture/?leccode=10588
'리눅스 공부' 카테고리의 다른 글
| LVS (Linux Virtual Server)구축 : ldirectord + heartbeat (0) | 2018.08.08 |
|---|---|
| 리눅스 / 현재 시간 동기화 (0) | 2018.03.29 |
| 리눅스 / DNS 서버 구축 (1) | 2018.03.23 |
| 리눅스 / 워드프레스 설치 (0) | 2018.03.12 |
| resolv.conf 초기화 해결 (0) | 2018.03.09 |
리눅스 / 현재 시간 동기화
서버에서 생성 한 파일의 생성 시간과 데스크톱 PC 에서 확인 된 시간하고 상이하다.
시간 동기화 진행을 한다.
# rdate -s time.bora.net
명령어를 입력했는데, 실행 불가가 나온다.
-bash: rdate: command not found
# yum install rdate -y
설치 후 동기화 재실행 하니, 정상적으로 완료 되었다.
'리눅스 공부' 카테고리의 다른 글
| LVS (Linux Virtual Server)구축 : ldirectord + heartbeat (0) | 2018.08.08 |
|---|---|
| 리눅스 / 웹사이트 트래픽 제한 : cband 모듈 설치 (0) | 2018.04.04 |
| 리눅스 / DNS 서버 구축 (1) | 2018.03.23 |
| 리눅스 / 워드프레스 설치 (0) | 2018.03.12 |
| resolv.conf 초기화 해결 (0) | 2018.03.09 |
