gfs

环境说明：

        centos5.2+iscsi+gfs

1、安装centos5.2：

          最小化安装系统三台

2、初始化三台系统：

          1、关闭不必要的服务

turnseroff( )
        {
        for ser in `chkconfig --list   |grep 3:on |awk '{print $1}'`
        do
                echo $ser
                case $ser in
                  crond | irqbalance | microcode_ctl | network | random | sendmail \
                        | sshd | syslog | messagebus | haldaemon | readahead_early \
                        | apmd | readahead_later | readahead | iptables | lvm2-monitor | xinetd | auditd | cpuspeed )
                echo "Base services, Skip"
             ;;
             *)
                 echo "change $ser to off"
                 chkconfig --level 3 $ser off
                 service $ser stop
 
             ;;
 
                esac
        done
}
turnseroff

          2、修改yum 源

#change the yum source
\cp -f /etc/yum.repos.d/CentOS-Base.repo ~/
cat </etc/yum.repos.d/CentOS-Base.repo
[lan]
name=LAN
baseurl=ftp://10.20.2.11/centos52
gpgcheck=1
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-5

EOF

          3、添加vi 语法高亮

aliasvi ()
{
        sed -i "8 s/^/alias vi='vim'/" /root/.bashrc

        source /root/.bashrc
}
aliasvi

          4、关闭防火墙

service iptables stop
chkconfig iptables off

 

3、安装需要的软件包

  yum install -y cman gfs-utils kmod-gfs kmod-dlm  cluster-snmp iscsi-initiator-utils lvm2-cluster  rgmanager

如查有报错，直接导入KEY 参考：

4、在每个节点上持载iscsi共享存储

有关iscis 相关请参考：

service iscsi restart
iscsiadm --mode discovery --type sendtargets --portal 10.20.15.249
service iscsi restart

fdisk -l 会多一个20.3GB 的sdb

5、在其中一个节点上上创建一个名为gfscg的 LVM 卷组

pvcreate -ff  /dev/sdb
vgcreate  gfsvg /dev/sdb

列出VG大小：

vgdisplay  gfsvg |grep "Total PE "

  Total PE              4847

创建lv

lvcreate -l 4847  -n gfs gfsvg

5、在其中一个节点上上创建一个GFS卷

        1、确认cluster name

    cman_tool status |grep "Cluster Name"
   Cluster Name: gfs_cluster

       2、创建GFS 文件系统

gfs_mkfs -j 5 -p lock_dlm -t gfs_cluster:gfs /dev/gfsvg/gfs

相关参数请参考：

6、配置节点做集群

    1、分别在三台机器上添加主机名到ip的映设

echo "10.20.15.151 clustera"  >>/etc/hosts
echo "10.20.15.152 clusterb"  >>/etc/hosts
echo "10.20.15.153 clusterc"  >>/etc/hosts

  2、创建集群配置文件,并分别复制到每个节点上

 cat <<EOF  >/etc/cluster/cluster.conf
<?xml version="1.0"?>
<cluster alias="gfs_cluster" config_version="2" name="gfs_cluster">
        <fence_daemon post_fail_delay="0" post_join_delay="3"/>
        <clusternodes>
                <clusternode name="clustera" nodeid="1" votes="1">
                        <fence>
                                <method name="1"/>
                        </fence>
                </clusternode>
                <clusternode name="clusterb" nodeid="2" votes="1">
                        <fence>
                                <method name="1"/>
                        </fence>
                </clusternode>
                <clusternode name="clusterc" nodeid="3" votes="1">
                        <fence/>
                </clusternode>
        </clusternodes>
        <cman/>
        <fencedevices>
                <fencedevice agent="fence_manual" name="manual"/>
        </fencedevices>
        <rm>
                <failoverdomains>
                        <failoverdomain name="failover" ordered="0" restricted="0"/>
                </failoverdomains>
                <resources>
                        <clusterfs device="/dev/gfsvg/gfs" force_unmount="0" fsid="32101" fstype="gfs" mountpoint="/opt/pgfsql" name="gfsmount" options="-t gfs"/>
                </resources>
        </rm>
</cluster>
 
EOF

7、在每一个节点上启动相关服务

以下命令在每个节点上执行

 for   ser in iscis gfs gfs2 cman clvmd
        do
             chkconfig $ser on
 done

for ser in  gfs gfs2 cman clvmd
do
service $ser restat
done

8、确认集群各节点的状态

    1、clustat 如下图:

[root@pgsqla scripts]# clustat
Cluster Status for gfs_cluster @ Wed Sep  2 23:13:20 2009
Member Status: Quorate
 
 Member Name                                                     ID   Status
 ------ ----                                                     ---- ------
 clustera                                                            1 Online, Local
 clusterb                                                            2 Online
 clusterc                                                            3 Online
 
[root@pgsqla scripts]#

Online 说明节点正常,反之为offlice  ,可以通过cat /var/log/messages 得到相关信息

9、命令行集群管理工具

 1、cman_tool

Usage:
 
cman_tool <join|leave|kill|expected|votes|version|wait|status|nodes|services|debug> [options]

 2、ccs_tool

Usage::
  ccs_tool [options] <command>
 
Options:
  -h                  Print this usage and exit.
  -V                  Print version information and exit.
 
Commands:
  help                Print this usage and exit.
  update <xml file>   Tells ccsd to upgrade to new config file.
  upgrade <location>  Upgrade old CCS format to new xml format.
  addnode <node>      Add a node
  delnode <node>      Delete a node
  lsnode              List nodes
  lsfence             List fence devices
  addfence <fencedev> Add a new fence device
  delfence <fencedev> Delete a fence device
  create              Create a skeleton config file
  addnodeids          Assign node ID numbers to all nodes