主要实现的是一个高可用负载均衡web服务器集群，适合lamp架构。
前端使用两台服务器做lvs+keepalived负载调度器，中间可以用N台做apache+php应用服务器，后面使用两台做mysql高可用双机，最后面用了一台虚拟机做文件服务器。
一共开了7个虚拟机。
系统环境：RHEL6.0 kvm虚拟机
lvs调度服务器：192.168.0.1 192.168.0.2
apache服务器:192.168.0.3 192.168.0.4
mysql服务器:192.168.0.7 192.168.0.8
文件服务器:192.168.0.10
———————————————————————————————————–
一、lvs调度服务器
———————————————————————————————————–
Keepalived+lvs
服务器环境:
系统均是:RHEL6.0 内核:2.6.32-71.el6.i686)
Virtual IP:192.168.0.50
Load Balancer:192.168.0.1
Backup:192.168.0.2
Real Server 1:192.168.0.3
Real Server 2:192.168.0.4
主备机上的软件包安装与配置
yum install ipvsadm kernel–devel -y
wget http://www.keepalived.org/software/keepalived-1.1.20.tar.gz
tar zxf keepalived-1.1.20.tar.gz
cd keepalived-1.1.20
./configure –prefix=/usr/local/keepalived –with-kernel-dir=/usr/src/kernels/2.6.32-71.el6.i686/
86/
Keepalived configuration
———————-Keepalived version : 1.1.17
Compiler: gcc
Compiler flags: -g -O2
Extra Lib: -lpopt -lssl -lcrypto
Use IPVS Framework: Yes;注意编译时一定要支持 lvs
IPVS sync daemon support: Yes
Use VRRP Framework: Yes
Use LinkWatch: No
Use Debug flags: No
make
make install
mkdir /etc/keepalived
ln –s /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
ln –s /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
ln –s /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
ln -s /usr/local/keepalived/bin/* /bin/
ln -s /usr/local/keepalived/sbin/* /sbin/
vi /etc/rc.local
modprobe ip_vs
vi /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@example.com#接收警报的 email 地址,可以添加多个
}
notification_email_from root@localhost
smtp_server 127.0.0.1#使用本机转发 email
smtp_connect_timeout 30
router_id LVS_DEVEL#load balancer 的标识 ID,用于 email 警报
}
vrrp_instance VI_1 {
state MASTER#备机改为 BACKUP,此状态是由 priority 的值来决定的,当前
priority 的值小于备机的值,那么将会失去 MASTER 状态
interface eth0#HA 监测网络接口
virtual_router_id 50#主、备机的 virtual_router_id 必须相同
priority 150#主机的优先级,备份机改为 50
advert_int 1#主备之间的通告间隔秒数
authentication {
auth_type PASS#主备切换时的验证
auth_pass 1111
}
virtual_ipaddress {
192.168.0.111#HA 虚拟 ip,可加多个
}
}
virtual_server 192.168.0.111 80 {
delay_loop 6#每隔 6 秒查询 realserver状态
lb_algo rr#lvs 调度算法,这里使用轮叫
lb_kind DR#lvs 负载均衡机制,这里使用直连路由
# persistence_timeout 50 #同一 IP 的连接 60 秒内被分配到同一台 realserver
protocol TCP #用 TCP 协议检查 realserver 状态
real_server 192.168.0.1 80 {
weight 1
TCP_CHECK {
connect_timeout 3 #故障重试秒数
nb_get_retry 3 #重试延迟
delay_before_retry 3
}
}
real_server 192.168.0.2 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
分别在 realserver 上执行以下命令
vi /etc/sysctl.conf
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.lo.arp_announce = 2
sysctl -p
ifconfig eth0:0 192.168.0.200 netmask 255.255.255.255 up
route add -host 192.168.0.200 dev eth0:0
vi /etc/rc.local
ifconfig eth0:0 192.168.0.200 netmask 255.255.255.255 up
route add -host 192.168.0.200 dev eth0:0
echo `hostname` > /var/www/html/index.html
service httpd start
测试:
访问 http://192.168.0.50,看到页面在两个 realserver 上切换表示成功!
你也可以通过 ipvsadm -Lnc 查看详细连接情况!
——————————————————————————————————————-
二、mysql服务器
——————————————————————————————————————-
mysql双主高可用
————————————-
系统环境：
RHEL6.0_I386
VIP 192.168.0.51
real server1 192.167.0.7
real server2 192.168.0.8
————————————-
1.server1和server2安装好mysql并修改配置文件：
yum install mysql-server
vi /etc/my.cnf:
[mysqld]
log-bin=MySQL-bin
server-id=1 ##在server2上配置为server-id=2
————————————-
2.server1和server2相互设置为主从同步，（双主）.
server1:
mysql> grant replication slave on *.* to ‘cluster’@’%’ identified by ‘cluster’;
mysql> show master status;
————————-
MySQL-bin.000001 236
————————-
————————————
server2:
mysql> change master to
-> master_host=’192.168.0.7′,
-> master_user=’cluster’,
-> master_password=’cluster’,
-> master_log_file=’MySQL-bin.000001′,
-> master_log_pos=236;
mysql> start slave;
mysql> show slave status;
在server2和server1上执行相反的操作，使其互为主从.
————————————–
3.安装软件包
yum install gcc popt-devel kernel-devel openssl-devel ipvsadm make
tar xf keepalived-***.tar.gz
cd keepalived-*
./configure –prefix=/usr/local/keepalived –with-kernel-dir=/usr/src/kernels/2.6.32-71.el6.i686
make && make install
modprobe ip_vs #此处如果系统没有自动加载此模块将导致keepalived启动后无法找到lvs负载均衡协议
mkdir /etc/keepalived/
ln -s /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
ln -s /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
ln -s /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
ln -s /usr/local/keepalived/bin/* /bin/
ln -s /usr/local/keepalived/sbin/* /sbin/
—————————————————
4.修改server1/server2配置文件
server1:
vi /etc/keepalived/keepalived.conf:
! Configuration File for keepalived
global_defs {
notification_email {
root@example.com
}
notification_email_from root@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id MYSQL-HA ##确保和server2相同
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51 ##确保和server2相同，同网内不同集群此项必须不同,否则发生冲突
priority 100 ##此处server2上设置为50
advert_int 1
nopreempt ##不抢占，只在priority高的server1上设置，server2上此项注释掉
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.0.51
}
}
virtual_server 192.168.0.51 3306 {
delay_loop 2
lb_algo wrr
lb_kind DR
persistence_timeout 60
protocol TCP
real_server 192.168.0.7 3306 {
##server2上此处改为192.168.0.8(即server2本机ip)
weight 3
notify_down /usr/local/keepalived/bin/mysql.sh
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
}
——————————–
server1和server2上都添加此检测脚本，作用是当mysql停止工作时自动关闭本机的keepalived
从而实现将故障机器踢出（因每台机器上keepalived只添加了本机为realserver）.
vi /usr/local/keepalived/bin/mysql.sh:
#!/bin/sh
pkill keepalived
——————————–
vi /etc/rc.local:
modprobe ip_vs ##此模块如果无法自动加载则需手动加载
——————————–
server1和server2启动keepalived守护进程.
/etc/init.d/keepalived start
————————————————————————————————————
三、文件服务器
————————————————————————————————————
data server:192.168.0.10
data client1:192.168.0.3
data client2:192.168.0.4
—————————–
data server:
yum install luci -y
/etc/init.d/luci start
访问https://192.168.0.10/:8084/ 使用系统帐号密码登录进入创建一个集群并将client1/2加为节点
然后添加一个虚拟fence设备并加入client1 client2
yum install scsi-target-utils -y
chkconfig tgtd on
/etc/init.d/tgtd start
tgtadm –lld iscsi –op new –mode target –tid 1 -T webdata
tgtadm –lld iscsi –op new –mode logicalunit –tid 1 –lun 1 -b /dev/sda
tgtadm –lld iscsi –op bind –mode target –tid 1 -I ALL
tgtadm –lld iscsi –op show –mode target #验证是否ok
vi /etc/rc.local 将上面的配置命令写入rc.local
tgtadm –lld iscsi –op new –mode target –tid 1 -T webdata
tgtadm –lld iscsi –op new –mode logicalunit –tid 1 –lun 1 -b /dev/sda
tgtadm –lld iscsi –op bind –mode target –tid 1 -I ALL
data client:
yum install ricci -y
/etc/init.d/ricci start
lvmconf –enable-cluster
yum install iscsi-initiator-utils -y
iscsiadm -m discovery -t sendtargets -p 192.168.0.10
iscsiadm -m node -T webdata -p 192.168.0.10 -l
信息自动保存至配置文件/var/lib/iscsi/nodes/webdata/192.168.0.10,3260,1/default
fdisk -l 查看可看到多了一块/dev/sda硬盘
pvcreate /dev/sda
vgcreate datavg /dev/sda
lvcreate -L 1020M -n lv1 datavg
cman_tool status|grep Name #查看cluster name
Cluster Name: web_cluster
创建gfs文件系统
mkfs.gfs2 -p lock_dlm -t web_cluster:gfs -j 2 /dev/datavg/lv1
#此处-j 2 提供两台client主机连接
mount -t gfs2 /dev/datavg/lv1 /mnt
vi /etc/fstab
/dev/datavg/lv1 /mnt gfs2 defaults 0 0
/etc/init.d/gfs2 start #文件系统自动挂在到/mnt
chkconfig cman on
chkconfig rgmanager on
chkconfig ricci on
chkconfig modclusterd on
chkconfig clvmd on
chkconfig gfs2 on
如果gfs开机时无法识别到而lvscan命令又遇到lv状态为inactive：
lvchange -ay /dev/gfsvg/gfs