标签归档:迁移

0racle 11.2.0.3 RAC在线迁移OCR&VOTE ASM磁盘组

最近一个土豪客户要更换存储(原存储IBM DS8300,新存储EMC VMAX 20K,这都是高端存储),因此涉及到系统和数据库的迁移,这个项目很大,据说三年前就已经开始策划。其中要迁移的核心数据库是一套DB2,数据量达30T+,还有一套11.2.0.2 Oracle RAC,数据量500G+,DB2迁移暂先不说,因为不是专门搞那个的。由于要负责这套Oracle RAC的迁移,采用的迁移方案是,先将OCR&VOTE ASM磁盘组在线从原存储IBM DS8300迁移到新存储EMC VMAX 20K的ASM磁盘组上,然后将集群全部关闭,通过存储间的拷贝方式将原IBM DS8300存储上的DATA和ARCH ASM磁盘组中的数据拷贝到新存储EMC VMAX 20K上,拷贝完成后,扫描磁盘然后启动集群。这里我们不讨论这个迁移方案是否合理,因为这其中夹杂着很多政治因素,我们只关心迁移方案中的第一步,即OCR&VOTE ASM磁盘组在线迁移问题。其实,这里我们并不用非常关心OCR&VOTE ASM磁盘组到底是从IBM DS8300迁移到EMC VMAX 20K上还是从EMC VMAX 20K迁移到IBM DS8300上,因为从本质上讲只是将OCR&VOTE换一个位置而已。下面是我在自己虚拟机上进行的一个测试,特记录一下,留作备用。

一、测试环境

----软件版本
Oracle VirtualBox 4.3 + Oracle Linux 5.8 + UDEV + 11.2.0.3 Grid + 11.2.0.3 DB

----IP地址
# cat /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1               localhost.localdomain localhost
::1             localhost6.localdomain6 localhost6
# Public IP
192.168.56.111 11grac1.localdomain 11grac1
192.168.56.112 11grac2.localdomain 11grac2
# Private IP
192.168.10.1 11grac1-priv.localdomain 11grac1-priv
192.168.10.2 11grac2-priv.localdomain 11grac2-priv
# Virtual IP
192.168.56.113 11grac1-vip.localdomain 11grac1-vip
192.168.56.114 11grac2-vip.localdomain 11grac2-vip
# SCAN IP
192.168.56.115 SCAN

----共享存储磁盘
# ls -l /dev/sd*
brw-rw---- 1 root root 8, 16 Dec 25  2013 sdb
brw-rw---- 1 root root 8, 32 Dec 25  2013 sdc
brw-rw---- 1 root root 8, 48 Dec 25  2013 sdd

----UDEV绑定ASM磁盘
# ls -l /dev/asm*
brw-rw---- 1 grid asmadmin 8, 16 Dec 25 05:22 /dev/asm-diskb
brw-rw---- 1 grid asmadmin 8, 32 Dec 25 03:45 /dev/asm-diskc
brw-rw---- 1 grid asmadmin 8, 48 Dec 25 05:22 /dev/asm-diskd

----ASM实例
SQL> select instance_name,status from gv$instance;
  
INSTANCE_NAME    STATUS
---------------- ------------
+ASM1            STARTED
+ASM2            STARTED

----ASM实例spfile
SYS@+ASM1> show parameter spfile;
  
NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
spfile                               string      +OCRVOTE/rac11g-cluster/asmpar
                                                 ameterfile/registry.253.835086289
SYS@+ASM2> show parameter spfile;
  
NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
spfile                               string      +OCRVOTE/rac11g-cluster/asmpar
                                                 ameterfile/registry.253.835086289

----ASM DISKGROUP
SYS@+ASM1> select GROUP_NUMBER,NAME,STATE,type,TOTAL_MB,free_mb,VOTING_FILES,COMPATIBILITY from v$asm_diskgroup;
  
GROUP_NUMBER NAME            STATE       TYPE     TOTAL_MB    FREE_MB V COMPATIBILITY
------------ --------------- ----------- ------ ---------- ---------- - ---------------
           1 DATA            MOUNTED     EXTERN       5120       2792 N 11.2.0.0.0
           3 OCRVOTE         MOUNTED     EXTERN       2048       1652 Y 11.2.0.0.0
SYS@+ASM2> select GROUP_NUMBER,NAME,STATE,type,TOTAL_MB,free_mb,VOTING_FILES,COMPATIBILITY from v$asm_diskgroup;
  
GROUP_NUMBER NAME            STATE       TYPE     TOTAL_MB    FREE_MB V COMPATIBILITY
------------ --------------- ----------- ------ ---------- ---------- - ---------------
           1 DATA            MOUNTED     EXTERN       5120       2792 N 11.2.0.0.0
           3 OCRVOTE         MOUNTED     EXTERN       2048       1652 Y 11.2.0.0.0

----ASM DISK
SYS@+ASM1> select GROUP_NUMBER,DISK_NUMBER,STATE,REDUNDANCY,TOTAL_MB,FREE_MB,name,path,failgroup from v$asm_disk order by GROUP_NUMBER;
  
GROUP_NUMBER DISK_NUMBER STATE    REDUNDA   TOTAL_MB    FREE_MB NAME            PATH            FAILGROUP
------------ ----------- -------- ------- ---------- ---------- --------------- --------------- ----------------
           0           0 NORMAL   UNKNOWN          0          0                 /dev/asm-diskd
           1           0 NORMAL   UNKNOWN       5120       2792 DATA_0000       /dev/asm-diskc  DATA_0000
           3           0 NORMAL   UNKNOWN       2048       1652 OCRVOTE_0000    /dev/asm-diskb  OCRVOTE_0000
SYS@+ASM2> select GROUP_NUMBER,DISK_NUMBER,STATE,REDUNDANCY,TOTAL_MB,FREE_MB,name,path,failgroup from v$asm_disk order by GROUP_NUMBER;  
  
GROUP_NUMBER DISK_NUMBER STATE    REDUNDA   TOTAL_MB    FREE_MB NAME              PATH            FAILGROUP
------------ ----------- -------- ------- ---------- ---------- --------------- --------------- -----------------
           0           0 NORMAL   UNKNOWN          0          0                 /dev/asm-diskd
           1           0 NORMAL   UNKNOWN       5120       2792 DATA_0000       /dev/asm-diskc  DATA_0000
           3           0 NORMAL   UNKNOWN       2048       1652 OCRVOTE_0000    /dev/asm-diskb  OCRVOTE_0000

----DB实例
SQL> select instance_name,status from gv$instance;
  
INSTANCE_NAME    STATUS
---------------- ------------
RAC11g2          OPEN
RAC11g1          OPEN

----DB实例spfile
SQL> show parameter spfile;
  
NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
spfile                               string      +DATA/rac11g/spfilerac11g.ora

---OCR&VOTE信息
[root@11grac1 ~]# ocrcheck && crsctl query css votedisk
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2960
         Available space (kbytes) :     259160
         ID                       :  331841301
         Device/File Name         :   +OCRVOTE
                                    Device/File integrity check succeeded
  
                                    Device/File not configured
  
                                    Device/File not configured
  
                                    Device/File not configured
  
                                    Device/File not configured
  
         Cluster registry integrity check succeeded
  
         Logical corruption check succeeded
  
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   5cc943d1e0544f49bf7a1f698de670c9 (/dev/asm-diskb) [OCRVOTE]
Located 1 voting disk(s).
[root@11grac2 ~]# ocrcheck && crsctl query css votedisk
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2960
         Available space (kbytes) :     259160
         ID                       :  331841301
         Device/File Name         :   +OCRVOTE
                                    Device/File integrity check succeeded
  
                                    Device/File not configured
  
                                    Device/File not configured
  
                                    Device/File not configured
  
                                    Device/File not configured
  
         Cluster registry integrity check succeeded
  
         Logical corruption check succeeded
  
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   5cc943d1e0544f49bf7a1f698de670c9 (/dev/asm-diskb) [OCRVOTE]
Located 1 voting disk(s).

二、测试目标
目前该RAC的OCR&VOTE存放在OCRVOTE ASM磁盘组,准备在线迁移到OCRVOTE_NEW ASM磁盘组上。 三、详细过程

1.查看OCR和VOTE状态与位置(在两个节点root执行)
[root@11grac1 ~]# ocrcheck && crsctl query css votedisk
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2960
         Available space (kbytes) :     259160
         ID                       :  331841301
         Device/File Name         :   +OCRVOTE
                                    Device/File integrity check succeeded
  
                                    Device/File not configured
  
                                    Device/File not configured
  
                                    Device/File not configured
  
                                    Device/File not configured
  
         Cluster registry integrity check succeeded
  
         Logical corruption check succeeded
  
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   5cc943d1e0544f49bf7a1f698de670c9 (/dev/asm-diskb) [OCRVOTE]
[root@11grac2 ~]# ocrcheck && crsctl query css votedisk
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2960
         Available space (kbytes) :     259160
         ID                       :  331841301
         Device/File Name         :   +OCRVOTE
                                    Device/File integrity check succeeded
  
                                    Device/File not configured
  
                                    Device/File not configured
  
                                    Device/File not configured
  
                                    Device/File not configured
  
         Cluster registry integrity check succeeded
  
         Logical corruption check succeeded
  
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   5cc943d1e0544f49bf7a1f698de670c9 (/dev/asm-diskb) [OCRVOTE]
Located 1 voting disk(s).

2.在两节点查看OCR备份情况(在两节点root执行)
[root@11grac1 ~]# ocrconfig -showbackup 
11grac2     2013/12/25 07:46:20     /grid/product/11.2.0/grid_1/cdata/rac11g-cluster/backup00.ocr 
11grac2     2013/12/25 07:46:20     /grid/product/11.2.0/grid_1/cdata/rac11g-cluster/day.ocr 
11grac2     2013/12/25 07:46:20     /grid/product/11.2.0/grid_1/cdata/rac11g-cluster/week.ocr 
11grac2     2013/12/24 20:54:01     /grid/product/11.2.0/grid_1/cdata/rac11g-cluster/backup_20131224_205401.ocr 
11grac2     2013/12/24 20:53:22     /grid/product/11.2.0/grid_1/cdata/rac11g-cluster/backup_20131224_205322.ocr 
11grac1     2013/12/24 20:21:31     /grid/product/11.2.0/grid_1/cdata/rac11g-cluster/backup_20131224_202131.ocr 
11grac1     2013/12/24 20:16:55     /grid/product/11.2.0/grid_1/cdata/rac11g-cluster/backup_20131224_201655.ocr 
11grac1     2013/12/24 18:09:31     /grid/product/11.2.0/grid_1/cdata/rac11g-cluster/backup_20131224_180931.ocr
[root@11grac2 ~]# ocrconfig -showbackup  
11grac2     2013/12/25 07:46:20     /grid/product/11.2.0/grid_1/cdata/rac11g-cluster/backup00.ocr  
11grac2     2013/12/25 07:46:20     /grid/product/11.2.0/grid_1/cdata/rac11g-cluster/day.ocr  
11grac2     2013/12/25 07:46:20     /grid/product/11.2.0/grid_1/cdata/rac11g-cluster/week.ocr 
11grac2     2013/12/24 20:54:01     /grid/product/11.2.0/grid_1/cdata/rac11g-cluster/backup_20131224_205401.ocr  
11grac2     2013/12/24 20:53:22     /grid/product/11.2.0/grid_1/cdata/rac11g-cluster/backup_20131224_205322.ocr  
11grac1     2013/12/24 20:21:31     /grid/product/11.2.0/grid_1/cdata/rac11g-cluster/backup_20131224_202131.ocr  
11grac1     2013/12/24 20:16:55     /grid/product/11.2.0/grid_1/cdata/rac11g-cluster/backup_20131224_201655.ocr  
11grac1     2013/12/24 18:09:31     /grid/product/11.2.0/grid_1/cdata/rac11g-cluster/backup_20131224_180931.ocr

3.手动执行物理备份---可选操作(在一个节点或两个节点root执行)
# ocrconfig -manualbackup

4.再次查看OCR备份情况---可选操作(在两个节点root执行)
# ocrconfig -showbackup

5.创建将要迁移OCR&VOTE到新的ASM磁盘组OCRVOTE_NEW(在一个节点grid用户登录ASM实例执行)
SYS@+ASM1> select GROUP_NUMBER,DISK_NUMBER,STATE,REDUNDANCY,TOTAL_MB,FREE_MB,name,path,failgroup from v$asm_disk order by GROUP_NUMBER;
  
GROUP_NUMBER DISK_NUMBER STATE    REDUNDA   TOTAL_MB    FREE_MB NAME            PATH            FAILGROUP
------------ ----------- -------- ------- ---------- ---------- --------------- --------------- ----------------
           0           0 NORMAL   UNKNOWN          0          0                 /dev/asm-diskd
           1           0 NORMAL   UNKNOWN       5120       2792 DATA_0000       /dev/asm-diskc  DATA_0000
           3           0 NORMAL   UNKNOWN       2048       1652 OCRVOTE_0000    /dev/asm-diskb  OCRVOTE_0000
  
SYS@+ASM1> select GROUP_NUMBER,NAME,STATE,type,TOTAL_MB,free_mb,VOTING_FILES,COMPATIBILITY from v$asm_diskgroup;
  
GROUP_NUMBER NAME            STATE       TYPE     TOTAL_MB    FREE_MB V COMPATIBILITY
------------ --------------- ----------- ------ ---------- ---------- - ---------------
           1 DATA            MOUNTED     EXTERN       5120       2792 N 11.2.0.0.0
           3 OCRVOTE         MOUNTED     EXTERN       2048       1652 Y 11.2.0.0.0
  
SYS@+ASM2> select GROUP_NUMBER,DISK_NUMBER,STATE,REDUNDANCY,TOTAL_MB,FREE_MB,name,path,failgroup from v$asm_disk order by GROUP_NUMBER;
  
GROUP_NUMBER DISK_NUMBER STATE    REDUNDA   TOTAL_MB    FREE_MB NAME              PATH            FAILGROUP
------------ ----------- -------- ------- ---------- ---------- ----------------- --------------- -----------------
           0           0 NORMAL   UNKNOWN          0          0                   /dev/asm-diskd
           1           0 NORMAL   UNKNOWN       5120       2792 DATA_0000         /dev/asm-diskc  DATA_0000
           3           0 NORMAL   UNKNOWN       2048       1652 OCRVOTE_0000      /dev/asm-diskb  OCRVOTE_0000
  
SYS@+ASM2> select GROUP_NUMBER,NAME,STATE,type,TOTAL_MB,free_mb,VOTING_FILES,COMPATIBILITY from v$asm_diskgroup;
  
GROUP_NUMBER NAME              STATE       TYPE     TOTAL_MB    FREE_MB V COMPATIBILITY
------------ ----------------- ----------- ------ ---------- ---------- - ---------------
           1 DATA              MOUNTED     EXTERN       5120       2792 N 11.2.0.0.0
           3 OCRVOTE           MOUNTED     EXTERN       2048       1652 Y 11.2.0.0.0
  
SYS@+ASM1> create diskgroup OCRVOTE_NEW EXTERNAL REDUNDANCY DISK '/dev/asm-diskd' attribute 'compatible.asm'='11.2';
  
Diskgroup created.
----创建OCRVOTE_NEW ASM磁盘组的节点ASM实例会自动mount该磁盘组,但是另一个节点不会mount上该磁盘组

6.查看新创建的ASM磁盘组(在两个节点grid用户登录ASM实例执行)
SYS@+ASM1> select GROUP_NUMBER,NAME,STATE,type,TOTAL_MB,free_mb,VOTING_FILES,COMPATIBILITY from v$asm_diskgroup;
  
GROUP_NUMBER NAME            STATE       TYPE     TOTAL_MB    FREE_MB V COMPATIBILITY
------------ --------------- ----------- ------ ---------- ---------- - ---------------
           1 DATA            MOUNTED     EXTERN       5120       2792 N 11.2.0.0.0
           3 OCRVOTE         MOUNTED     EXTERN       2048       1652 Y 11.2.0.0.0
           2 OCRVOTE_NEW     MOUNTED     EXTERN       2048       1996 N 11.2.0.0.0
  
SYS@+ASM1> select GROUP_NUMBER,DISK_NUMBER,STATE,REDUNDANCY,TOTAL_MB,FREE_MB,name,path,failgroup from v$asm_disk order by GROUP_NUMBER;
  
GROUP_NUMBER DISK_NUMBER STATE    REDUNDA   TOTAL_MB    FREE_MB NAME               PATH            FAILGROUP
------------ ----------- -------- ------- ---------- ---------- ------------------ --------------- ----------------
           1           0 NORMAL   UNKNOWN       5120       2792 DATA_0000          /dev/asm-diskc  DATA_0000
           2           0 NORMAL   UNKNOWN       2048       1996 OCRVOTE_NEW_0000   /dev/asm-diskd  OCRVOTE_NEW_0000
           3           0 NORMAL   UNKNOWN       2048       1652 OCRVOTE_0000       /dev/asm-diskb  OCRVOTE_0000
  
SYS@+ASM2> select GROUP_NUMBER,NAME,STATE,type,TOTAL_MB,free_mb,VOTING_FILES,COMPATIBILITY from v$asm_diskgroup;
  
GROUP_NUMBER NAME              STATE       TYPE     TOTAL_MB    FREE_MB V COMPATIBILITY
------------ ----------------- ----------- ------ ---------- ---------- - ---------------
           1 DATA              MOUNTED     EXTERN       5120       2792 N 11.2.0.0.0
           0 OCRVOTE_NEW       DISMOUNTED                  0          0 N 0.0.0.0.0
           3 OCRVOTE           MOUNTED     EXTERN       2048       1652 Y 11.2.0.0.0
  
SYS@+ASM2> select GROUP_NUMBER,DISK_NUMBER,STATE,REDUNDANCY,TOTAL_MB,FREE_MB,name,path,failgroup from v$asm_disk order by GROUP_NUMBER;
  
GROUP_NUMBER DISK_NUMBER STATE    REDUNDA   TOTAL_MB    FREE_MB NAME              PATH            FAILGROUP
------------ ----------- -------- ------- ---------- ---------- ----------------- --------------- -----------------
           0           0 NORMAL   UNKNOWN          0          0                   /dev/asm-diskd
           1           0 NORMAL   UNKNOWN       5120       2792 DATA_0000         /dev/asm-diskc  DATA_0000
           3           0 NORMAL   UNKNOWN       2048       1652 OCRVOTE_0000      /dev/asm-diskb  OCRVOTE_0000
----节点2 ASM实例没有mount上OCRVOTE_NEW磁盘组

7.挂载新创建的ASM磁盘组(在在没有mount上新创建的ASM磁盘组节点上以grid用户登录ASM实例执行)
SYS@+ASM2> alter diskgroup OCRVOTE_NEW mount;
  
Diskgroup altered.
  
SYS@+ASM2> select GROUP_NUMBER,DISK_NUMBER,STATE,REDUNDANCY,TOTAL_MB,FREE_MB,name,path,failgroup from v$asm_disk order by GROUP_NUMBER;
  
GROUP_NUMBER DISK_NUMBER STATE    REDUNDA   TOTAL_MB    FREE_MB NAME              PATH            FAILGROUP
------------ ----------- -------- ------- ---------- ---------- ----------------- --------------- -----------------
           1           0 NORMAL   UNKNOWN       5120       2792 DATA_0000         /dev/asm-diskc  DATA_0000
           2           0 NORMAL   UNKNOWN       2048       1953 OCRVOTE_NEW_0000  /dev/asm-diskd  OCRVOTE_NEW_0000
           3           0 NORMAL   UNKNOWN       2048       1652 OCRVOTE_0000      /dev/asm-diskb  OCRVOTE_0000
  
SYS@+ASM2> select GROUP_NUMBER,NAME,STATE,type,TOTAL_MB,free_mb,VOTING_FILES,COMPATIBILITY from v$asm_diskgroup;
  
GROUP_NUMBER NAME              STATE       TYPE     TOTAL_MB    FREE_MB V COMPATIBILITY
------------ ----------------- ----------- ------ ---------- ---------- - ---------------
           1 DATA              MOUNTED     EXTERN       5120       2792 N 11.2.0.0.0
           2 OCRVOTE_NEW       MOUNTED     EXTERN       2048       1953 N 11.2.0.0.0
           3 OCRVOTE           MOUNTED     EXTERN       2048       1652 Y 11.2.0.0.0

8.将新创建的ASM磁盘组OCRVOTE_NEW添加到OCR中(在一个节点root执行)
[root@11grac1 ~]# ocrconfig -add +OCRVOTE_NEW

9.查看OCR和VOTE状态与位置(在两个节点root执行)
[root@11grac1 ~]# ocrcheck && crsctl query css votedisk
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2976
         Available space (kbytes) :     259144
         ID                       :  331841301
         Device/File Name         :   +OCRVOTE
                                    Device/File integrity check succeeded
         Device/File Name         : +OCRVOTE_NEW
                                    Device/File integrity check succeeded
  
                                    Device/File not configured
  
                                    Device/File not configured
  
                                    Device/File not configured
  
         Cluster registry integrity check succeeded
  
         Logical corruption check succeeded
  
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   5cc943d1e0544f49bf7a1f698de670c9 (/dev/asm-diskb) [OCRVOTE]
Located 1 voting disk(s).
[root@11grac2 ~]# ocrcheck && crsctl query css votedisk
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2976
         Available space (kbytes) :     259144
         ID                       :  331841301
         Device/File Name         :   +OCRVOTE
                                    Device/File integrity check succeeded
         Device/File Name         : +OCRVOTE_NEW
                                    Device/File integrity check succeeded
  
                                    Device/File not configured
  
                                    Device/File not configured
  
                                    Device/File not configured
  
         Cluster registry integrity check succeeded
  
         Logical corruption check succeeded
  
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   5cc943d1e0544f49bf7a1f698de670c9 (/dev/asm-diskb) [OCRVOTE]
Located 1 voting disk(s).

SYS@+ASM1> select GROUP_NUMBER,NAME,STATE,type,TOTAL_MB,free_mb,VOTING_FILES,COMPATIBILITY from v$asm_diskgroup;
  
GROUP_NUMBER NAME             STATE       TYPE     TOTAL_MB    FREE_MB V COMPATIBILITY
------------ ---------------- ----------- ------ ---------- ---------- - ---------------
           1 DATA             MOUNTED     EXTERN       5120       2792 N 11.2.0.0.0
           2 OCRVOTE_NEW      MOUNTED     EXTERN       2048       1685 N 11.2.0.0.0
           3 OCRVOTE          MOUNTED     EXTERN       2048       1652 Y 11.2.0.0.0
  
SYS@+ASM1> select GROUP_NUMBER,DISK_NUMBER,STATE,REDUNDANCY,TOTAL_MB,FREE_MB,name,path,failgroup from v$asm_disk order by GROUP_NUMBER;
  
GROUP_NUMBER DISK_NUMBER STATE    REDUNDA   TOTAL_MB    FREE_MB NAME             PATH             FAILGROUP
------------ ----------- -------- ------- ---------- ---------- ---------------- ---------------- ----------------
           1           0 NORMAL   UNKNOWN       5120       2792 DATA_0000        /dev/asm-diskc   DATA_0000
           2           0 NORMAL   UNKNOWN       2048       1685 OCRVOTE_NEW_0000 /dev/asm-diskd   OCRVOTE_NEW_0000
           3           0 NORMAL   UNKNOWN       2048       1652 OCRVOTE_0000     /dev/asm-diskb   OCRVOTE_0000
  
SYS@+ASM2> select GROUP_NUMBER,NAME,STATE,type,TOTAL_MB,free_mb,VOTING_FILES,COMPATIBILITY from v$asm_diskgroup;
  
GROUP_NUMBER NAME             STATE       TYPE     TOTAL_MB    FREE_MB V COMPATIBILITY
------------ ---------------- ----------- ------ ---------- ---------- - ---------------
           1 DATA             MOUNTED     EXTERN       5120       2792 N 11.2.0.0.0
           2 OCRVOTE_NEW      MOUNTED     EXTERN       2048       1685 N 11.2.0.0.0
           3 OCRVOTE          MOUNTED     EXTERN       2048       1652 Y 11.2.0.0.0
  
SYS@+ASM2> select GROUP_NUMBER,DISK_NUMBER,STATE,REDUNDANCY,TOTAL_MB,FREE_MB,name,path,failgroup from v$asm_disk order by GROUP_NUMBER;
  
GROUP_NUMBER DISK_NUMBER STATE    REDUNDA   TOTAL_MB    FREE_MB NAME             PATH             FAILGROUP
------------ ----------- -------- ------- ---------- ---------- ---------------- ---------------- ----------------
           1           0 NORMAL   UNKNOWN       5120       2792 DATA_0000        /dev/asm-diskc   DATA_0000
           2           0 NORMAL   UNKNOWN       2048       1685 OCRVOTE_NEW_0000 /dev/asm-diskd   OCRVOTE_NEW_0000
           3           0 NORMAL   UNKNOWN       2048       1652 OCRVOTE_0000     /dev/asm-diskb   OCRVOTE_0000
----此时会看见OCR的位置出现在两处,其中+OCRVOTE_NEW为OCR的mirror

10.迁移VOTE DISK到新创建的ASM磁盘组OCRVOTE_NEW上(在一个节点root执行)
[root@11grac1 ~]# crsctl replace votedisk +OCRVOTE_NEW
Successful addition of voting disk 1bd1f2b28abc4ff6bf1a4113f2ce1067.
Successful deletion of voting disk 5cc943d1e0544f49bf7a1f698de670c9.
Successfully replaced voting disk group with +OCRVOTE_NEW.
CRS-4266: Voting file(s) successfully replaced

11.查看此时OCR&VOTE ASM磁盘组状态(在两个节点grid用户登录ASM实例执行)
SYS@+ASM1> select GROUP_NUMBER,NAME,STATE,type,TOTAL_MB,free_mb,VOTING_FILES,COMPATIBILITY from v$asm_diskgroup;
  
GROUP_NUMBER NAME               STATE       TYPE     TOTAL_MB    FREE_MB V COMPATIBILITY
------------ ------------------ ----------- ------ ---------- ---------- - ---------------
           1 DATA               MOUNTED     EXTERN       5120       2792 N 11.2.0.0.0
           3 OCRVOTE            MOUNTED     EXTERN       2048       1684 N 11.2.0.0.0
           2 OCRVOTE_NEW        MOUNTED     EXTERN       2048       1653 Y 11.2.0.0.0
----可以看见执行crsctl replace votedisk +OCRVOTE_NEW操作的节点上v$asm_diskgroup中已显示将voting file存放在新建的ASM磁盘组OCRVOTE_NEW上
----这里其实存在疑问,我测试过几次,得出的结果不同,1节点执行完crsctl replace votedisk +OCRVOTE_NEW操作后,v$asm_diskgroup中也会显示voting file同时存放在ASM磁盘组OCRVOTE和OCRVOTE_NEW上
SYS@+ASM2> select GROUP_NUMBER,NAME,STATE,type,TOTAL_MB,free_mb,VOTING_FILES,COMPATIBILITY from v$asm_diskgroup;
  
GROUP_NUMBER NAME              STATE       TYPE     TOTAL_MB    FREE_MB V COMPATIBILITY
------------ ----------------- ----------- ------ ---------- ---------- - ---------------
           1 DATA              MOUNTED     EXTERN       5120       2792 N 11.2.0.0.0
           2 OCRVOTE_NEW       MOUNTED     EXTERN       2048       1653 N 11.2.0.0.0
           3 OCRVOTE           MOUNTED     EXTERN       2048       1684 Y 11.2.0.0.0
----未执行crsctl replace votedisk +OCRVOTE_NEW操作的节点上v$asm_diskgroup中显示voting file仍然存放在ASM磁盘组上
----这里其实也存在疑问,我测试过几次,得出的结果不同,未执行crsctl replace votedisk +OCRVOTE_NEW操作的节点上v$asm_diskgroup中显示voting file既没有存放在OCRVOTE上,也没有存放在OCRVOTE_NEW上

12.查看OCR和VOTE状态与位置(在两个节点root执行)
[root@11grac1 ~]# ocrcheck && crsctl query css votedisk
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2976
         Available space (kbytes) :     259144
         ID                       :  331841301
         Device/File Name         :   +OCRVOTE
                                    Device/File integrity check succeeded
         Device/File Name         : +OCRVOTE_NEW
                                    Device/File integrity check succeeded
  
                                    Device/File not configured
  
                                    Device/File not configured
  
                                    Device/File not configured
  
         Cluster registry integrity check succeeded
  
         Logical corruption check succeeded
  
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   1bd1f2b28abc4ff6bf1a4113f2ce1067 (/dev/asm-diskd) [OCRVOTE_NEW]
[root@11grac2 ~]# ocrcheck && crsctl query css votedisk
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2976
         Available space (kbytes) :     259144
         ID                       :  331841301
         Device/File Name         :   +OCRVOTE
                                    Device/File integrity check succeeded
         Device/File Name         : +OCRVOTE_NEW
                                    Device/File integrity check succeeded
  
                                    Device/File not configured
  
                                    Device/File not configured
  
                                    Device/File not configured
  
         Cluster registry integrity check succeeded
  
         Logical corruption check succeeded
  
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   1bd1f2b28abc4ff6bf1a4113f2ce1067 (/dev/asm-diskd) [OCRVOTE_NEW]
Located 1 voting disk(s).

13.创建ASM实例spfile到新创建的OCRVOTE_NEW ASM磁盘组上(在一个节点grid用户登录ASM实例执行)
SYS@+ASM1> create pfile='/home/grid/asmpfile.ora' from spfile;
  
File created.
  
SYS@+ASM1> create spfile='+OCRVOTE_NEW' from pfile='/home/grid/asmpfile.ora';
  
File created.

14.删除OCR原位置ASM磁盘组OCRVOTE(在一个节点root执行)
[root@11grac1 ~]# ocrconfig -delete +OCRVOTE

15.查看OCR和VOTE新状态与位置(在两个节点root执行)
[root@11grac1 ~]# ocrcheck && crsctl query css votedisk
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2976
         Available space (kbytes) :     259144
         ID                       :  331841301
         Device/File Name         : +OCRVOTE_NEW
                                    Device/File integrity check succeeded
  
                                    Device/File not configured
  
                                    Device/File not configured
  
                                    Device/File not configured
  
                                    Device/File not configured
  
         Cluster registry integrity check succeeded
  
         Logical corruption check succeeded
  
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   1bd1f2b28abc4ff6bf1a4113f2ce1067 (/dev/asm-diskd) [OCRVOTE_NEW]
Located 1 voting disk(s).
[root@11grac2 ~]# ocrcheck && crsctl query css votedisk
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2976
         Available space (kbytes) :     259144
         ID                       :  331841301
         Device/File Name         : +OCRVOTE_NEW
                                    Device/File integrity check succeeded
  
                                    Device/File not configured
  
                                    Device/File not configured
  
                                    Device/File not configured
  
                                    Device/File not configured
  
         Cluster registry integrity check succeeded
  
         Logical corruption check succeeded
  
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   1bd1f2b28abc4ff6bf1a4113f2ce1067 (/dev/asm-diskd) [OCRVOTE_NEW]
Located 1 voting disk(s).

SYS@+ASM1> select GROUP_NUMBER,NAME,STATE,type,TOTAL_MB,free_mb,VOTING_FILES,COMPATIBILITY from v$asm_diskgroup;
  
GROUP_NUMBER NAME             STATE       TYPE     TOTAL_MB    FREE_MB V COMPATIBILITY
------------ ---------------- ----------- ------ ---------- ---------- - ---------------
           1 DATA             MOUNTED     EXTERN       5120       2792 N 11.2.0.0.0
           2 OCRVOTE_NEW      MOUNTED     EXTERN       2048       1652 Y 11.2.0.0.0
           3 OCRVOTE          MOUNTED     EXTERN       2048       1684 N 11.2.0.0.0
  
SYS@+ASM1> select GROUP_NUMBER,DISK_NUMBER,STATE,REDUNDANCY,TOTAL_MB,FREE_MB,name,path,failgroup from v$asm_disk order by GROUP_NUMBER;
  
GROUP_NUMBER DISK_NUMBER STATE    REDUNDA   TOTAL_MB    FREE_MB NAME             PATH             FAILGROUP
------------ ----------- -------- ------- ---------- ---------- ---------------- ---------------- ----------------
           1           0 NORMAL   UNKNOWN       5120       2792 DATA_0000        /dev/asm-diskc   DATA_0000
           2           0 NORMAL   UNKNOWN       2048       1652 OCRVOTE_NEW_0000 /dev/asm-diskd   OCRVOTE_NEW_0000
           3           0 NORMAL   UNKNOWN       2048       1684 OCRVOTE_0000     /dev/asm-diskb   OCRVOTE_0000
  
SYS@+ASM2> select GROUP_NUMBER,NAME,STATE,type,TOTAL_MB,free_mb,VOTING_FILES,COMPATIBILITY from v$asm_diskgroup;
  
GROUP_NUMBER NAME             STATE       TYPE     TOTAL_MB    FREE_MB V COMPATIBILITY
------------ ---------------- ----------- ------ ---------- ---------- - ---------------
           1 DATA             MOUNTED     EXTERN       5120       2792 N 11.2.0.0.0
           2 OCRVOTE_NEW      MOUNTED     EXTERN       2048       1652 N 11.2.0.0.0
           3 OCRVOTE          MOUNTED     EXTERN       2048       1684 Y 11.2.0.0.0
  
SYS@+ASM2> select GROUP_NUMBER,DISK_NUMBER,STATE,REDUNDANCY,TOTAL_MB,FREE_MB,name,path,failgroup from v$asm_disk order by GROUP_NUMBER;
  
GROUP_NUMBER DISK_NUMBER STATE    REDUNDA   TOTAL_MB    FREE_MB NAME             PATH             FAILGROUP
------------ ----------- -------- ------- ---------- ---------- ---------------- ---------------- ----------------
           1           0 NORMAL   UNKNOWN       5120       2792 DATA_0000        /dev/asm-diskc   DATA_0000
           2           0 NORMAL   UNKNOWN       2048       1652 OCRVOTE_NEW_0000 /dev/asm-diskd   OCRVOTE_NEW_0000
           3           0 NORMAL   UNKNOWN       2048       1684 OCRVOTE_0000     /dev/asm-diskb   OCRVOTE_0000

16.重启集群(在两个节点root执行)
[root@11grac1 ~]# crsctl stop crs && crsctl start crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on '11grac1'
CRS-2673: Attempting to stop 'ora.crsd' on '11grac1'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on '11grac1'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on '11grac1'
CRS-2673: Attempting to stop 'ora.cvu' on '11grac1'
CRS-2673: Attempting to stop 'ora.oc4j' on '11grac1'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on '11grac1'
CRS-2673: Attempting to stop 'ora.OCRVOTE.dg' on '11grac1'
CRS-2673: Attempting to stop 'ora.OCRVOTE_NEW.dg' on '11grac1'
CRS-2673: Attempting to stop 'ora.rac11g.db' on '11grac1'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on '11grac1' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on '11grac1'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on '11grac1' succeeded
CRS-2673: Attempting to stop 'ora.11grac1.vip' on '11grac1'
CRS-2677: Stop of 'ora.scan1.vip' on '11grac1' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on '11grac2'
CRS-2677: Stop of 'ora.11grac1.vip' on '11grac1' succeeded
CRS-2672: Attempting to start 'ora.11grac1.vip' on '11grac2'
CRS-2676: Start of 'ora.scan1.vip' on '11grac2' succeeded
CRS-2676: Start of 'ora.11grac1.vip' on '11grac2' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on '11grac2'
CRS-2677: Stop of 'ora.cvu' on '11grac1' succeeded
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on '11grac2' succeeded
CRS-2677: Stop of 'ora.rac11g.db' on '11grac1' succeeded
CRS-2673: Attempting to stop 'ora.DATA.dg' on '11grac1'
CRS-2677: Stop of 'ora.DATA.dg' on '11grac1' succeeded
CRS-2677: Stop of 'ora.oc4j' on '11grac1' succeeded
CRS-2677: Stop of 'ora.OCRVOTE.dg' on '11grac1' succeeded
CRS-2677: Stop of 'ora.OCRVOTE_NEW.dg' on '11grac1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on '11grac1'
CRS-2677: Stop of 'ora.asm' on '11grac1' succeeded
CRS-2673: Attempting to stop 'ora.ons' on '11grac1'
CRS-2677: Stop of 'ora.ons' on '11grac1' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on '11grac1'
CRS-2677: Stop of 'ora.net1.network' on '11grac1' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on '11grac1' has completed
CRS-2677: Stop of 'ora.crsd' on '11grac1' succeeded
CRS-2673: Attempting to stop 'ora.crf' on '11grac1'
CRS-2673: Attempting to stop 'ora.ctssd' on '11grac1'
CRS-2673: Attempting to stop 'ora.evmd' on '11grac1'
CRS-2673: Attempting to stop 'ora.asm' on '11grac1'
CRS-2673: Attempting to stop 'ora.mdnsd' on '11grac1'
CRS-2677: Stop of 'ora.crf' on '11grac1' succeeded
CRS-2677: Stop of 'ora.evmd' on '11grac1' succeeded
CRS-2677: Stop of 'ora.ctssd' on '11grac1' succeeded
CRS-2677: Stop of 'ora.mdnsd' on '11grac1' succeeded
CRS-2677: Stop of 'ora.asm' on '11grac1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on '11grac1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on '11grac1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on '11grac1'
CRS-2677: Stop of 'ora.cssd' on '11grac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on '11grac1'
CRS-2677: Stop of 'ora.gipcd' on '11grac1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on '11grac1'
CRS-2677: Stop of 'ora.gpnpd' on '11grac1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on '11grac1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
[root@11grac2 ~]# crsctl stop crs && crsctl start crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on '11grac2'
CRS-2673: Attempting to stop 'ora.crsd' on '11grac2'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on '11grac2'
CRS-2673: Attempting to stop 'ora.11grac2.vip' on '11grac2'
CRS-2673: Attempting to stop 'ora.OCRVOTE.dg' on '11grac2'
CRS-2673: Attempting to stop 'ora.OCRVOTE_NEW.dg' on '11grac2'
CRS-2673: Attempting to stop 'ora.rac11g.db' on '11grac2'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on '11grac2'
CRS-2677: Stop of 'ora.11grac2.vip' on '11grac2' succeeded
CRS-2677: Stop of 'ora.LISTENER.lsnr' on '11grac2' succeeded
CRS-2673: Attempting to stop 'ora.11grac1.vip' on '11grac2'
CRS-2677: Stop of 'ora.11grac1.vip' on '11grac2' succeeded
CRS-2677: Stop of 'ora.rac11g.db' on '11grac2' succeeded
CRS-2673: Attempting to stop 'ora.DATA.dg' on '11grac2'
CRS-2677: Stop of 'ora.DATA.dg' on '11grac2' succeeded
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on '11grac2'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on '11grac2' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on '11grac2'
CRS-2677: Stop of 'ora.scan1.vip' on '11grac2' succeeded
CRS-2677: Stop of 'ora.OCRVOTE.dg' on '11grac2' succeeded
CRS-2677: Stop of 'ora.OCRVOTE_NEW.dg' on '11grac2' succeeded
CRS-2673: Attempting to stop 'ora.asm' on '11grac2'
CRS-2677: Stop of 'ora.asm' on '11grac2' succeeded
CRS-2673: Attempting to stop 'ora.ons' on '11grac2'
CRS-2677: Stop of 'ora.ons' on '11grac2' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on '11grac2'
CRS-2677: Stop of 'ora.net1.network' on '11grac2' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on '11grac2' has completed
CRS-2677: Stop of 'ora.crsd' on '11grac2' succeeded
CRS-2673: Attempting to stop 'ora.crf' on '11grac2'
CRS-2673: Attempting to stop 'ora.ctssd' on '11grac2'
CRS-2673: Attempting to stop 'ora.evmd' on '11grac2'
CRS-2673: Attempting to stop 'ora.asm' on '11grac2'
CRS-2673: Attempting to stop 'ora.mdnsd' on '11grac2'
CRS-2677: Stop of 'ora.crf' on '11grac2' succeeded
CRS-2677: Stop of 'ora.mdnsd' on '11grac2' succeeded
CRS-2677: Stop of 'ora.evmd' on '11grac2' succeeded
CRS-2677: Stop of 'ora.ctssd' on '11grac2' succeeded
CRS-2677: Stop of 'ora.asm' on '11grac2' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on '11grac2'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on '11grac2' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on '11grac2'
CRS-2677: Stop of 'ora.cssd' on '11grac2' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on '11grac2'
CRS-2677: Stop of 'ora.gipcd' on '11grac2' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on '11grac2'
CRS-2677: Stop of 'ora.gpnpd' on '11grac2' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on '11grac2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.

17.查看OCR&VOTE位置及ASM实例spfile位置(在两个节点grid用户登录ASM实例执行)
[root@11grac1 ~]# ocrcheck && crsctl query css votedisk
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2976
         Available space (kbytes) :     259144
         ID                       :  331841301
         Device/File Name         : +OCRVOTE_NEW
                                    Device/File integrity check succeeded
  
                                    Device/File not configured
  
                                    Device/File not configured
  
                                    Device/File not configured
  
                                    Device/File not configured
  
         Cluster registry integrity check succeeded
  
         Logical corruption check succeeded
  
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   943d41e7dd624fd9bf8becbf18b434f5 (/dev/asm-diskd) [OCRVOTE_NEW]
Located 1 voting disk(s).
[root@11grac2 ~]# ocrcheck && crsctl query css votedisk
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2976
         Available space (kbytes) :     259144
         ID                       :  331841301
         Device/File Name         : +OCRVOTE_NEW
                                    Device/File integrity check succeeded
  
                                    Device/File not configured
  
                                    Device/File not configured
  
                                    Device/File not configured
  
                                    Device/File not configured
  
         Cluster registry integrity check succeeded
  
         Logical corruption check succeeded
  
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   943d41e7dd624fd9bf8becbf18b434f5 (/dev/asm-diskd) [OCRVOTE_NEW]
Located 1 voting disk(s).

SYS@+ASM1> select GROUP_NUMBER,NAME,STATE,type,TOTAL_MB,free_mb,VOTING_FILES,COMPATIBILITY from v$asm_diskgroup;
  
GROUP_NUMBER NAME             STATE       TYPE     TOTAL_MB    FREE_MB V COMPATIBILITY
------------ ---------------- ----------- ------ ---------- ---------- - ---------------
           1 DATA             MOUNTED     EXTERN       5120       2792 N 11.2.0.0.0
           2 OCRVOTE_NEW      MOUNTED     EXTERN       2048       1652 Y 11.2.0.0.0
           3 OCRVOTE          MOUNTED     EXTERN       2048       1684 N 11.2.0.0.0
  
SYS@+ASM1> select GROUP_NUMBER,DISK_NUMBER,STATE,REDUNDANCY,TOTAL_MB,FREE_MB,name,path,failgroup from v$asm_disk order by GROUP_NUMBER;
  
GROUP_NUMBER DISK_NUMBER STATE    REDUNDA   TOTAL_MB    FREE_MB NAME             PATH             FAILGROUP
------------ ----------- -------- ------- ---------- ---------- ---------------- ---------------- ----------------
           1           0 NORMAL   UNKNOWN       5120       2792 DATA_0000        /dev/asm-diskc   DATA_0000
           2           0 NORMAL   UNKNOWN       2048       1652 OCRVOTE_NEW_0000 /dev/asm-diskd   OCRVOTE_NEW_0000
           3           0 NORMAL   UNKNOWN       2048       1684 OCRVOTE_0000     /dev/asm-diskb   OCRVOTE_0000
  
SYS@+ASM2> select GROUP_NUMBER,NAME,STATE,type,TOTAL_MB,free_mb,VOTING_FILES,COMPATIBILITY from v$asm_diskgroup;
  
GROUP_NUMBER NAME             STATE       TYPE     TOTAL_MB    FREE_MB V COMPATIBILITY
------------ ---------------- ----------- ------ ---------- ---------- - ---------------
           1 DATA             MOUNTED     EXTERN       5120       2792 N 11.2.0.0.0
           2 OCRVOTE_NEW      MOUNTED     EXTERN       2048       1652 Y 11.2.0.0.0
           3 OCRVOTE          MOUNTED     EXTERN       2048       1684 N 11.2.0.0.0
  
SYS@+ASM2> select GROUP_NUMBER,DISK_NUMBER,STATE,REDUNDANCY,TOTAL_MB,FREE_MB,name,path,failgroup from v$asm_disk order by GROUP_NUMBER;
  
GROUP_NUMBER DISK_NUMBER STATE    REDUNDA   TOTAL_MB    FREE_MB NAME             PATH             FAILGROUP
------------ ----------- -------- ------- ---------- ---------- ---------------- ---------------- ----------------
           1           0 NORMAL   UNKNOWN       5120       2792 DATA_0000        /dev/asm-diskc   DATA_0000
           2           0 NORMAL   UNKNOWN       2048       1652 OCRVOTE_NEW_0000 /dev/asm-diskd   OCRVOTE_NEW_0000
           3           0 NORMAL   UNKNOWN       2048       1684 OCRVOTE_0000     /dev/asm-diskb   OCRVOTE_0000           
  
SYS@+ASM1> show parameter spfile;
  
NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
spfile                               string      +OCRVOTE_NEW/rac11g-cluster/as
                                                 mparameterfile/registry.253.83
                                                 5094929
  
SYS@+ASM2> show parameter spfile;
  
NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
spfile                               string      +OCRVOTE_NEW/rac11g-cluster/as
                                                 mparameterfile/registry.253.83
                                                 5094929

18.将OCR原位置ASM磁盘组OCRVOTE卸载掉(在一个节点grid用户登录ASM实例执行)
SYS@+ASM1> alter diskgroup OCRVOTE dismount;
  
Diskgroup altered.
  
SYS@+ASM1> select GROUP_NUMBER,NAME,STATE,type,TOTAL_MB,free_mb,VOTING_FILES,COMPATIBILITY from v$asm_diskgroup;
  
GROUP_NUMBER NAME             STATE       TYPE     TOTAL_MB    FREE_MB V COMPATIBILITY
------------ ---------------- ----------- ------ ---------- ---------- - ---------------
           1 DATA             MOUNTED     EXTERN       5120       2792 N 11.2.0.0.0
           2 OCRVOTE_NEW      MOUNTED     EXTERN       2048       1652 Y 11.2.0.0.0
           0 OCRVOTE          DISMOUNTED                  0          0 N 0.0.0.0.0
  
SYS@+ASM1> select GROUP_NUMBER,DISK_NUMBER,STATE,REDUNDANCY,TOTAL_MB,FREE_MB,name,path,failgroup from v$asm_disk order by GROUP_NUMBER;
  
GROUP_NUMBER DISK_NUMBER STATE    REDUNDA   TOTAL_MB    FREE_MB NAME             PATH             FAILGROUP
------------ ----------- -------- ------- ---------- ---------- ---------------- ---------------- ----------------
           0           0 NORMAL   UNKNOWN          0          0                  /dev/asm-diskb
           1           0 NORMAL   UNKNOWN       5120       2792 DATA_0000        /dev/asm-diskc   DATA_0000
           2           0 NORMAL   UNKNOWN       2048       1652 OCRVOTE_NEW_0000 /dev/asm-diskd   OCRVOTE_NEW_0000
  
SYS@+ASM2> select GROUP_NUMBER,NAME,STATE,type,TOTAL_MB,free_mb,VOTING_FILES,COMPATIBILITY from v$asm_diskgroup;
  
GROUP_NUMBER NAME             STATE       TYPE     TOTAL_MB    FREE_MB V COMPATIBILITY
------------ ---------------- ----------- ------ ---------- ---------- - ---------------
           1 DATA             MOUNTED     EXTERN       5120       2792 N 11.2.0.0.0
           2 OCRVOTE_NEW      MOUNTED     EXTERN       2048       1652 Y 11.2.0.0.0
           3 OCRVOTE          MOUNTED     EXTERN       2048       1684 N 11.2.0.0.0
  
SYS@+ASM2> select GROUP_NUMBER,DISK_NUMBER,STATE,REDUNDANCY,TOTAL_MB,FREE_MB,name,path,failgroup from v$asm_disk order by GROUP_NUMBER;
  
GROUP_NUMBER DISK_NUMBER STATE    REDUNDA   TOTAL_MB    FREE_MB NAME             PATH             FAILGROUP
------------ ----------- -------- ------- ---------- ---------- ---------------- ---------------- ----------------
           1           0 NORMAL   UNKNOWN       5120       2792 DATA_0000        /dev/asm-diskc   DATA_0000
           2           0 NORMAL   UNKNOWN       2048       1652 OCRVOTE_NEW_0000 /dev/asm-diskd   OCRVOTE_NEW_0000
           3           0 NORMAL   UNKNOWN       2048       1684 OCRVOTE_0000     /dev/asm-diskb   OCRVOTE_0000

19.删除OCR原位置ASM磁盘组OCRVOTE(在另一个节点grid用户登录ASM实例执行)
SYS@+ASM2> drop diskgroup OCRVOTE including contents;
  
Diskgroup dropped.
  
SYS@+ASM1> select GROUP_NUMBER,NAME,STATE,type,TOTAL_MB,free_mb,VOTING_FILES,COMPATIBILITY from v$asm_diskgroup;
  
GROUP_NUMBER NAME             STATE       TYPE     TOTAL_MB    FREE_MB V COMPATIBILITY
------------ ---------------- ----------- ------ ---------- ---------- - ---------------
           1 DATA             MOUNTED     EXTERN       5120       2792 N 11.2.0.0.0
           2 OCRVOTE_NEW      MOUNTED     EXTERN       2048       1652 Y 11.2.0.0.0
  
SYS@+ASM1> select GROUP_NUMBER,DISK_NUMBER,STATE,REDUNDANCY,TOTAL_MB,FREE_MB,name,path,failgroup from v$asm_disk order by GROUP_NUMBER;
  
GROUP_NUMBER DISK_NUMBER STATE    REDUNDA   TOTAL_MB    FREE_MB NAME             PATH             FAILGROUP
------------ ----------- -------- ------- ---------- ---------- ---------------- ---------------- ----------------
           0           0 NORMAL   UNKNOWN          0          0                  /dev/asm-diskb
           1           0 NORMAL   UNKNOWN       5120       2792 DATA_0000        /dev/asm-diskc   DATA_0000
           2           0 NORMAL   UNKNOWN       2048       1652 OCRVOTE_NEW_0000 /dev/asm-diskd   OCRVOTE_NEW_0000
  
SYS@+ASM2> select GROUP_NUMBER,NAME,STATE,type,TOTAL_MB,free_mb,VOTING_FILES,COMPATIBILITY from v$asm_diskgroup;
  
GROUP_NUMBER NAME             STATE       TYPE     TOTAL_MB    FREE_MB V COMPATIBILITY
------------ ---------------- ----------- ------ ---------- ---------- - ---------------
           1 DATA             MOUNTED     EXTERN       5120       2792 N 11.2.0.0.0
           2 OCRVOTE_NEW      MOUNTED     EXTERN       2048       1652 Y 11.2.0.0.0
  
SYS@+ASM2> select GROUP_NUMBER,DISK_NUMBER,STATE,REDUNDANCY,TOTAL_MB,FREE_MB,name,path,failgroup from v$asm_disk order by GROUP_NUMBER;
  
GROUP_NUMBER DISK_NUMBER STATE    REDUNDA   TOTAL_MB    FREE_MB NAME             PATH             FAILGROUP
------------ ----------- -------- ------- ---------- ---------- ---------------- ---------------- ----------------
           0           0 NORMAL   UNKNOWN          0          0                  /dev/asm-diskb
           1           0 NORMAL   UNKNOWN       5120       2792 DATA_0000        /dev/asm-diskc   DATA_0000
           2           0 NORMAL   UNKNOWN       2048       1652 OCRVOTE_NEW_0000 /dev/asm-diskd   OCRVOTE_NEW_0000 
----这里不执行19步无法删除该磁盘组

20.整个迁移过程结束

以上为Oracle 11gR2在线迁移OCR&VOTE ASM磁盘组过程,其中不涉及存储中所使用多路径软件问题,如果涉及到存储磁盘路径改变,则需要去修改ASM实例的disk_strings参数,具体操作可以参考Maclea的这篇博客在11gR2 RAC中修改ASM DISK Path磁盘路径,论坛贴链接[讨论]11G RAC更换存储,如有问题,欢迎讨论。