顯示具有 Cluster 標籤的文章。 顯示所有文章
顯示具有 Cluster 標籤的文章。 顯示所有文章

VCS MySQL Agent Configuration

太久沒有碰到客戶用,又忘的差不多了,這次筆記下來吧


MySQL Agent Function
The operations or functions that the Symantec High Availability agent for MySQL
can perform are as follows:
online
$ BaseDir/bin/mysqld_safe --defaults-file=MyCnf --datadir=DataDir --user=MySQLUser
monitor
$ BaseDir/bin/mysqladmin --user=MySQLAdmin --password=MySQLAdminPasswd status
offline
$ BaseDir/bin/mysqladmin --user=MySQLAdmin --password=MySQLAdminPasswd shutdown
clean
$ BaseDir/bin/mysqladmin --user=MySQLAdmin --password=MySQLAdminPasswd shutdown


Create a MySQLAdmin user with shutdown privileages only:
# mysql -uroot -p{PASSWORD}
mysql> use mysql;
mysql> select user,host,password from user;
mysql> create user 'MySQLAdmin'@'localhost' identified by 'passw0rd' ;
mysql> create user 'MySQLAdmin'@'127.0.0.1' identified by 'passw0rd' ;
mysql> grant shutdown on *.* to 'MySQLAdmin'@'localhost' ;
mysql> grant shutdown on *.* to 'MySQLAdmin'@'l27.0.0.1' ;
mysql> select user,host,password from user;
mysql> flush privileges;

Test the user can shutdown the MySQL database
# $BaseDir/bin/mysqladmin --user=MySQLAdmin --password=passw0rd status
# $BaseDir/bin/mysqladmin --user=MySQLAdmin --password=passw0rd shutdown


Installation
# rpm -Uvh VRTSacclib-6.2.0.0-GENERIC.noarch.rpm
# rpm -Uvh VRTSmysql-6.2.0.0-GENERIC.noarch.rpm


Import MySQL resource type: 
/etc/VRTSagents/ha/conf/MySQL/MySQLTypes.cmd


Encrypt MySQLadmin password
# vcsencrypt -agent
Enter Password:
Enter Again:
GUGsHUjUJuNSvUIsK


VCS MySQL Resource Attribute
group MySQL (
        SystemList = { node1 = 0, node2 = 1 }
        )

        MySQL MySQLdb (
                Critical = 1
                MySQLAdmin = MySQLAdmin
                MySQLAdminPasswd = GUGsHUjUJuNSvUIsK   <=== encrypt password
                BaseDir = "/usr"
                DataDir = "/var/lib/mysql"
                MyCnf = "/etc/my.cnf"
                )


Running multiple instances of MySQL on a single node
VCS ENV file
#!/bin/ksh
MYSQL_UNIX_PORT=/tmp/mysql.sock; export MYSQL_UNIX_PORT
MYSQL_TCP_PORT=3307; export MYSQL_TCP_PORT


How to install Symantec HA guest components from vCenter


偶爾會碰到在 VMware guest os 安裝 VCS 的案子
因為目前 VMware 尚無法針對 guest os 中的 application 去做偵測
VCS 還是有其必要性

以前都要分別在 Windows guest OS 中安裝 VCS
6.01 起可以由 vCenter 中來 deploy 安裝了
Ref: http://www.symantec.com/connect/articles/achieving-high-availability-oracle-and-vmware-vcs-602

Installation Task:

1. Install SHA Console: 省事兒起見就裝在 vCenter 上吧
裝完後可以看到 vCenter 多了 Symantec High Availability 選項
這時 Install Guest Components 還看不到東西,因為我們 source 還沒裝
 
 
2. 安裝軟體 source 至 vCenter 中,才會有東西可以 deploy 到 guest os
指令 CopyInstaller.bat

現在 Install Guest Components 可以看到東西了

選擇要裝到哪幾台 guest os
 


3. Single Sign-on 可不設,用傳統的 VCS 帳號密碼登入方式


4.5. 就跟原本設定 VCS 的方式一樣
我還是覺另外單獨裝傳統的 VCS GUI Java Console 來設定比較快速方便


參考文件:
仔細看一下文件了解一下 VMware 中的限制,尤其是對 vMotion, DRS 等...

For Windows 2008
Symantec High Availability 6.0.1 Solutions Guide for VMware (Windows)
http://www.symantec.com/docs/DOC6131
For Windows 2012
Symantec High Availability 6.0.2 Windows Solutions Guide for VMware
http://www.symantec.com/docs/DOC6369

.


SCSI3 PGR operations on a VxDMP dmpnode result in dmp path disablement with RHEL5U8 and later kernels


RHEL 5.8 + SFCFS 一台重開另一台就 hang 住不動到 reboot ...

被這個搞死了,總算找到原因了
Linux Kernel 跑太快,竟然還加了個新的 scsi error type
VxVM 還不認識這個新 error type,導致使用 vxdmp 的 vxfen 出狀況



SCSI3 PGR operations on a VxDMP dmpnode result in dmp path disablement with RHEL5U8 and later kernels
http://www.symantec.com/docs/TECH192940

Problem
SCSI3 PGR (Persistent Group Registrations) operations on a VxDMP (VERITAS Dynamic Multi-Pathing) dmpnode result in VxDMP path disablement.

Side effects of the issue include the following:
• Reservation conflict immediately followed by VxDMP error V-5-0-112, whenever SCSI3 PGR operations are executed on a dmpnode.
         Operations that will issue SCSI3 PGR operation on a dmpnode include:
          - Stopping the cluster with hastop.
          - Issuing `/etc/init.d/vxfen start | stop`.  This command is automatically executed at server boot and shutdown.
          - Deporting a diskgroup that contain disk(s) with registrations or importing a diskgroup with groupreserve option.
• When a node is rebooted. The surviving node(s) may see diskgroups go into dgdisabled state and file system(s) get disabled. If VCS is managing resources you will see resource faults. This occurs because dmp paths and dmpnodes are being disabled as a result of multiple SCSI3 PGR operations.
• When a node is rebooted. The surviving node(s) may get paniced. The panic will be initiated by VxFEN as it is trying to avoid a split brain condition. This occurs because dmp paths and dmpnodes are being disabled as a result of multiple SCSI3 PGR operations, then reenabled by the VxDMP recovery daemon.

Error
kernel: sd 2:0:0:14: reservation conflict
kernel: VxVM vxdmp V-5-0-112 disabled path 66/0x0 belonging to the dmpnode 201/0xd0 due to path failure


Environment
RHEL5U8 running kernel 2.6.18-308.el5 and later
Storage Foundation 5.1 and later or Storage Foundation 6.0, 6.0RP1
VxFEN is configured and enabled with SCSI3 disk based fencing in either raw or dmp mode.
VxDMP is configured.


Cause
RHEL5U8 kernel SCSI layer error handling routine introduced a new error type : ID_NEXUS_FAILURE
This new error type is not handled properly by VxDMP, resulting in dmp paths getting disabled during SCSI3 PGR operations.


Solution
If you are planning to upgrade to RHEL5U8 or are currently running RHEL5U8 kernel 2.6.18-308.el5 or later kernels and are running VxVM 5.1 and later. Symantec recommends installing VxVM5.1SP1RP2P3HF5If you are planning to upgrade to RHEL5U8 or are currently running RHEL5U8 kernel 2.6.18-308.el5 or later kernels and are running VxVM 6.0 or 6.0RP1. Symantec recommends installing VxVM6.0RP1HF1

Workarounds:
Downgrade the kernel to pre RHEL5U8.
Configure vxfenmode in disabled mode.

Note: VERITAS Storage Foundation 6.0.1 will contain the fix and is slated for public release in early September. Rolling patch 5.1SP1RP3 will contain the fix and is slated for public release in early October. Rolling Patch 5.1SP1RP3 will also contain the vxfen patch noted in related articles.
.

Sun Cluster 3.3 + MySQL 5.5.x problem

Sun Cluster 3.3 文件寫到,似乎只支援到 MySQL 4.X
我用到 MySQL 5.X 上果然出了些問題
似乎是 MySQL create table 的 key 換了

於是乎 Sun Cluster MySQL Agent 在 create test table 時就出現了此訊息
Feb 17 12:56:27 TPDBDSS2 SC[SUNWscmys.probe_mysql]:BDSS:mysql-rs: [ID 986150 daemon.error] check_mysql - Sql-command USE sc3_test_database; CREATE TABLE sc3_test_table ( Number int(5) NOT NULL, PRIMARY KEY (Number)) TYPE=MyISAM;COMMIT returned error (ERROR 1064 (42000) at line 1: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'TYPE=MyISAM' at line 1)

debug 一番,找到 SUNWscmys/function
修改這行
for cmd in 'USE sc3_test_database; CREATE TABLE sc3_test_table ( Number int(5) NOT NULL, PRIMARY KEY (Number)) TYPE=MyISAM;COMMIT' \

把 TYPE 換成 ENGINE
for cmd in 'USE sc3_test_database; CREATE TABLE sc3_test_table ( Number int(5) NOT NULL, PRIMARY KEY (Number)) ENGINE=MyISAM;COMMIT' \


ok, work fine now

--
update: a patch release

Patch-ID# 145646-03
Keywords: oracle ha-mysql patch
Synopsis: Oracle Solaris Cluster 3.3: HA-MySQL Patch for Oracle Solaris 10
Date: Jul/11/2011

Add support for MySQL 5.X

---------------
    if (( ${MYSQL_VERSION} >= 5 ))
    then
        ENGINE="ENGINE=MyISAM"
    else
        ENGINE="TYPE=MyISAM"
    fi
---------------

續 - Solaris 8 Brand Zone with VCS in Share Storage

前幾天提到 Solaris 8 Brand Zone with VCS in Share Storage

昨天 Veritas 有 patch 出來了
VERITAS Cluster Server 5.0 MP1 and MP3 Hot Fix Zone Agent Patch for Solaris 10 SPARC

實際測試的結果....非常慢... :(
從 initinal onlie 到 online 花了約 150 sec
有點讓人無法接受....

因為只是是將手動的流程,轉由 zone agent online 來處理
把 switch over to other node 當作 migration to new host 來處理而已
每次的 switchover/failover 就是一次 detach/attach/migration to new host .... *_*
導致 zone online 要花上不少時間
除此之外,還有一些對 vcs permission 的檢查
不過那花不了一秒鍾....
每次 migration to new host 實在是太耗時了

顯然 Sun 的做法並不是這樣子
Patch 126020-03 for HA_Container
126020-03/SUNWsczone/reloc/SUNWsczone/sczbt/bin/functions
brand zone on share disk 可以用 override hostid file 的做法
也印證了不必每次切換都 s8_p2v migration to new host

若在 VCS zone agent 使用 Sun Cluster 的方式
只要十秒內 Zone 就 online 了
.

Solaris 8 Brand Zone with VCS in Share Storage

繼上次測試 Oracle in Solaris 8 Container with VCS 成功後
這次測試可否將 zone root 放在 share storage 上
文件的部份只有 solaris 10 native zone 寫說可以
不幸滴,測試 solaris 8 zone 的結果是失敗的
-------------------------------------------------------
zoneadm: zone 's8': This zone has been migrated to a new host.
zoneadm: zone 's8':
zoneadm: zone 's8': Run "/usr/lib/brand/solaris8/s8_p2v s8" before booting.
zoneadm: zone 's8':
zoneadm: zone 's8': exec /usr/lib/brand/solaris8/s8_boot s8 /zone/clarify failed
zoneadm: zone 's8': call to zoneadmd failed

--------------------------------------------------------
切換到另一個 node 後,認為轉移到了新的 host
需要跑個 migration 程序,s8_p2v
之後就可以順利將 s8 zone 由 vcs zone agent 帶起來了

可是 cluster failover 總不能這樣子啊,要可以自動帶起來才行
找找找,看有沒有法子 bypass 這個檢查步驟....

原來在 s8_boot 這程序中,會去檢查主機的 hostid
比對 zone root/.host.orig 這個檔案的 hostid
只要不一樣,就認為是轉移主機了
需要執行 s8_p2v 這個 migration 程序後,才可以 boot

那那那... Sun Cluster 怎麼處理這個問題呢?
嘿,給我找到了... Patch 126020-03 for HA_Containers
畢竟 S8/S9 Container 是後來出來的玩意兒,不過 Sun 手腳也很快
上個 Patch 就解決了這個問題
將 126020-03 解開看看裡面到底改了什麼玩意兒?
SUNWsczone/sczbt/bin/functions 裡面加了些東西
一些判斷 zone type 是 solaris8/solaris9/lx
重點來了,執行 hostid > $zonepath/root/.host.orig

嗯嗯,我如法炮製
去修改 VCS Zone agent 的 online script
測試一下,嘿嘿﹋﹋ it works!!

經詢問的結果,聽說這幾天 VCS patch 會針對此狀況做處理
到時再補個 patch 吧
.

Sun Cluster 3.2 New Feature

Sun Cluster 3.2 FAQ
What are the new features?
  • Ease of Use
    • New Command Line Interfaces
    • Oracle 10g improved integration and administration
    • Agent configuration wizards
    • Flexible IP address scheme
  • Higher Availability
    • Cluster support for SMF services
    • Quorum server
    • Extended flexibility for fencing protocol
  • Greater Flexibility
    • Expanded support for Solaris Containers
    • HA ZFS - agent support for Sun's new file system
    • Extended support for Veritas software components
  • Better Operations and Administration
    • Dual-partition software update
    • Live upgrade
    • Optional GUI installation
With Solaris Cluster Geographic Edition, new features include:
  • Support for x64 platforms
  • Support for EMC SRDF replication software
完整介紹
What's New in the Sun Cluster 3.2 Software

VCS installation script 不認識 e1000g 網路卡

Discovering NICs on node1 .................................discovered e1000g0 e1000g1 e1000g2 e1000g3 e1000g4 e1000g5

Enter the NIC for the first private heartbeat link on node1: [b,?] e1000g1
e1000g1 is not a valid NIC name
Enter the NIC for the first private heartbeat link on node1: [b,?] e1000g1
e1000g1 is not a valid NIC name

好吧,只好安裝完在自己手動設定 heartbeat 了
修改 /etc/llttab 自己手動加上
link e1000g1 /dev/e1000g:1 - ether - -


2008.04.15 補
Veritas Cluster Server 5.0+e805061 fix for the e1000g NICs on Solaris
http://support.veritas.com/docs/301904


VCS with Solaris 8 Zone

畢竟 Solaris 8 zone in Solaris 10 (S8MA) 這玩意兒出來的 VCS Zone support 晚,所以目前 VCS 文件中寫到的 zone,並沒有特別說明是哪種 zone。
現在看來,當然指的都是 Solaris 10 native zone

由文件中看來,zone agent 對 local zone 的動作很單純,online 就是 zoneadm boot,offline 就是 zoneadm halt,對 Solaris 8 Zone 的 online/offline 應該沒什麼問題,但是其他要進去 local zone monitor 的 agent 呢?

在安裝 VCS 或建立 Solaris 10 native zone 時
有部份 VRTS 檔案是會自動從 global zone 複製到 native zone 的
就像 pkgadd or patchadd 時,若不加 -G 參數(Global only)
預設是會 apply 到所有的 running native zone
這兒我看來最重要的是那些 VCS agent 了
因為 global/local zone 都是 solaris 10 沒問題
但在 Solaris 8 zone 就有問題了...

check engin_A.log :
2008/01/13 18:56:16 VCS INFO V-16-2-13001 (node-2) Resource(ora8): Output of the completed operation (online) sh: /opt/VRTSvcs/bin/Oracle/online: not found
2008/01/13 19:33:35 VCS INFO V-16-2-13001 (node-2) Resource(ora8): Output of the completed operation (offline) sh: /opt/VRTSvcs/bin/Oracle/offline: not found

為什麼 not found ?原來是去用 local zone 中的 agent 了
因為不是 native zone,所以安裝時(pkgadd)不會複製這些檔案
導致 agent 無法正確的 monitor zone 中的 AP
我的偷吃步就是看有哪些 VRTS 檔案會被安裝到 native zone 中
就照樣複製這些檔案到 Solaris 8 zone 中

還有另一個問題
VCS WARNING V-16-1-52529 Login Incorrect, Invalid username/password
Agnet 在 global/local zone 中怎麼溝通?這兒需要設定一個 cluster user 讓他有 Group Administor 權限就ok了

在 native zone 中沒有這些問題,安裝時都會自動設好
但在 p2v zone 中,我們只好手動來把這些設定做完了

ok! It works !!

參考文件:
Veritas Cluster White Paper
Implementing Solaris™ Zones with Veritas™ Cluster Server
by Symantec
July 2006
Eric Hennessey
Group Technical Product Manager
Unix Clustering Solutions

[轉載] Two-node Cluster How-To Guide

健忘做個筆記,Sun Cluster 3.2 還沒裝過... :(

Solaris Operating System
Two-node Cluster How-To Guide


How to Install and Configure a Two-node Cluster

This Sun Cluster Two-Node Installation and Configuration How-To guide is intended to help a new or experienced Solaris user quickly and easily install and configure Sun Cluster software for two nodes, including the configuration of aquorum device. Users are guided step-by-step through the process, with examples and screenshots to simplify the process. This document will not cover the configuration of highly available Services; that topic will be covered in a separate guide.

For more details about how to install and configure other Sun Cluster software configurations, see the Sun Cluster Software Installation Guide at docs.sun.com/app/docs/doc/819-0420.

..... 以上為節錄

Sun Cluster with Solaris 8 Migration Assistant

from: New Open HA Cluster Agent source code and Solaris Cluster Express 10/07 released
文中提到 Sun Cluster 3.1 的 Patch 120590-06 已加強對 zone 的支援,可監控 Solaris 8 zone 了,但 Sun Cluster 3.2 尚未支援,不過應該也快了
Status: RELEASED
Patch Id: 120590-06
Keywords: sunwsczone
Summary: Sun Cluster 3.1/3.1_x86: HA for Solaris Containers
Date:  Oct/19/2007

Problem Description:
6604747 HA Container agent should support solaris8 zones on sparc
(from 120590-05)
6488338 HA Container agent should support BrandZ lx zones with Solaris 10 Update 4
6554262 validate() for sczbt should check ip-type=exclusive and then only allow SC_NETWORK=false

VCS 呢?下回分解