주 콘텐츠로 건너뛰기
📓 Tibero Online Manual Click here 📓

검색

Welcome to Tibero GTS!

ProSync 양방향 CM FailOver를 사용한 무중단 동기화 환경 구축 (4/4)

문서유형ㅣ기술정보

분야ㅣ설치

적용제품버전ㅣTibero7, ProSync 4.6.0

문서번호ㅣPINTI006

 

개요

총 4편에 걸쳐 ProSync 양방향 CM FailOver를 사용한 무중단 동기화 환경 구축` 가이드를 기술하였습니다.

본 문서에서는 4편 CM FailOver 동작을 기술하였습니다.

  • 1편: ProSync 양방향 설치
  • 2편: ProSync 양방향 동기화 확인 
  • 3편: CM FailOver 설정
  • 4편: CM FailOver 동작

 

 

방법

A, B Set ProSync CM FailOver 설정 확인

NodeAgent NameProSync Process
node1agent1_12_34ext1, llob
node2agent2_12_34ext2, llob(standby)
node3agent3_12_34apply
node4agent4_12_34apply(standby)
A Set ProSync: node1, node2 to node3, node4

 

NodeAgent NameProSync Process
node3agent1_34_12ext1, llob
node4agent2_34_12ext2, llob(standby)
node1agent3_34_12apply
node2agent4_34_12apply(standby)
B Set ProSync: node3, node4 to node1, node2

node1, node2 TAC

  • CM Group 및 Agent 확인
  • A Set CM ProSync Group: grp_src_12_34
  • B Set CM ProSync Group: grp_tar_34_12
$ cmrctl show all
Resource List of Node cm0
=====================================================================
  CLUSTER     TYPE        NAME       STATUS           DETAIL
----------- -------- -------------- -------- ------------------------
     COMMON  network           net0       UP (private) 10.10.10.65/29000
     COMMON  cluster           cls0       UP inc: net0, pub: N/A
       cls0     file         cls0:0       UP /share/tac/CMFILE/CMFILE
       cls0  service           db12       UP Database, Active Cluster (auto-restart: OFF)
       cls0       db          db120 UP(NRML) db12, /tibero/tibero7, failed retry cnt: 0
       cls0    group  grp_src_12_34       UP type: prosync (failover: ON)
       cls0    group  grp_tar_34_12       UP type: prosync (failover: ON)
       cls0    agent   agent1_12_34       UP /share/prosync4_12_34/bin/prs_agent1_12_34.sh, start retry cnt: 0
       cls0    agent   agent3_34_12       UP /share/prosync4_34_12/bin/prs_agent3_34_12.sh, start retry cnt: 0
=====================================================================
Resource List of Node cm1
=====================================================================
  CLUSTER     TYPE        NAME       STATUS           DETAIL
----------- -------- -------------- -------- ------------------------
     COMMON  network           net1       UP (private) 10.10.10.66/29000
     COMMON  cluster           cls0       UP inc: net1, pub: N/A
       cls0     file         cls0:0       UP /share/tac/CMFILE/CMFILE
       cls0  service           db12       UP Database, Active Cluster (auto-restart: OFF)
       cls0       db          db121 UP(NRML) db12, /tibero/tibero7, failed retry cnt: 0
       cls0    group  grp_src_12_34       UP type: prosync (failover: ON)
       cls0    group  grp_tar_34_12       UP type: prosync (failover: ON)
       cls0    agent   agent2_12_34       UP /share/prosync4_12_34/bin/prs_agent2_12_34.sh, start retry cnt: 0
       cls0    agent   agent4_34_12       UP /share/prosync4_34_12/bin/prs_agent4_34_12.sh, start retry cnt: 0
=====================================================================

$ prs12
$ prs_adm
Admin> status
prs_agent ID: agent1_12_34, HOST: 10.10.10.65, PORT: 7600, CM_GROUP: grp_src_12_34, CM_ID: 0 is running
prs_agent ID: agent2_12_34, HOST: 10.10.10.66, PORT: 7601, CM_GROUP: grp_src_12_34, CM_ID: 1 is running
prs_agent ID: agent3_12_34, HOST: 10.10.10.67, PORT: 7602, CM_GROUP: grp_tar_12_34, CM_ID: 0 is running
prs_agent ID: agent4_12_34, HOST: 10.10.10.68, PORT: 7603, CM_GROUP: grp_tar_12_34, CM_ID: 1 is running

Instance ID: [prs_12_34]
prs_12_34_ext1 (1) is running (prs_agent ID : agent1_12_34, HOST: 10.10.10.65, PORT: 7600)
prs_12_34_ext2 (2) is running (prs_agent ID : agent2_12_34, HOST: 10.10.10.66, PORT: 7601)
prs_12_34_apply1 (1) is running (prs_agent ID : agent3_12_34, HOST: 10.10.10.67, PORT: 7602)
prs_12_34_llob (1) is running (prs_agent ID : agent1_12_34, HOST: 10.10.10.65, PORT: 7600)

node3, node4 TAC

  • CM Group 및 Agent 확인
  • A Set CM ProSync Group: grp_tar_12_34
  • B Set CM ProSync Group: grp_src_34_12
$ cmrctl show all
Resource List of Node cm0
=====================================================================
  CLUSTER     TYPE        NAME       STATUS           DETAIL
----------- -------- -------------- -------- ------------------------
     COMMON  network           net0       UP (private) 10.10.10.67/29000
     COMMON  cluster           cls0       UP inc: net0, pub: N/A
       cls0     file         cls0:0       UP /share/tac/CMFILE/CMFILE
       cls0  service           db34       UP Database, Active Cluster (auto-restart: OFF)
       cls0       db          db340 UP(NRML) db34, /tibero/tibero7, failed retry cnt: 0
       cls0    group  grp_tar_12_34       UP type: prosync (failover: ON)
       cls0    group  grp_src_34_12       UP type: prosync (failover: ON)
       cls0    agent   agent3_12_34       UP /share/prosync4_12_34/bin/prs_agent3_12_34.sh, start retry cnt: 0
       cls0    agent   agent1_34_12       UP /share/prosync4_34_12/bin/prs_agent1_34_12.sh, start retry cnt: 0
=====================================================================
Resource List of Node cm1
=====================================================================
  CLUSTER     TYPE        NAME       STATUS           DETAIL
----------- -------- -------------- -------- ------------------------
     COMMON  network           net1       UP (private) 10.10.10.68/29000
     COMMON  cluster           cls0       UP inc: net1, pub: N/A
       cls0     file         cls0:0       UP /share/tac/CMFILE/CMFILE
       cls0  service           db34       UP Database, Active Cluster (auto-restart: OFF)
       cls0       db          db341 UP(NRML) db34, /tibero/tibero7, failed retry cnt: 0
       cls0    group  grp_tar_12_34       UP type: prosync (failover: ON)
       cls0    group  grp_src_34_12       UP type: prosync (failover: ON)
       cls0    agent   agent4_12_34       UP /share/prosync4_12_34/bin/prs_agent4_12_34.sh, start retry cnt: 0
       cls0    agent   agent2_34_12       UP /share/prosync4_34_12/bin/prs_agent2_34_12.sh, start retry cnt: 0
=====================================================================

$ prs34
$ prs_adm
Admin> status
prs_agent ID: agent1_34_12, HOST: 10.10.10.67, PORT: 6600, CM_GROUP: grp_src_34_12, CM_ID: 0 is running
prs_agent ID: agent2_34_12, HOST: 10.10.10.68, PORT: 6601, CM_GROUP: grp_src_34_12, CM_ID: 1 is running
prs_agent ID: agent3_34_12, HOST: 10.10.10.65, PORT: 6602, CM_GROUP: grp_tar_34_12, CM_ID: 0 is running
prs_agent ID: agent4_34_12, HOST: 10.10.10.66, PORT: 6603, CM_GROUP: grp_tar_34_12, CM_ID: 1 is running

Instance ID: [prs_34_12]
prs_34_12_ext1 (1) is running (prs_agent ID : agent1_34_12, HOST: 10.10.10.67, PORT: 6600)
prs_34_12_ext2 (2) is running (prs_agent ID : agent2_34_12, HOST: 10.10.10.68, PORT: 6601)
prs_34_12_apply1 (1) is running (prs_agent ID : agent3_34_12, HOST: 10.10.10.65, PORT: 6602)
prs_34_12_llob (1) is running (prs_agent ID : agent1_34_12, HOST: 10.10.10.67, PORT: 6600)

 

동작

ProSync CM FailOver 동작 (node1, node2)

node1 CM FailOver 수행

  • node1의 CM을 강제 종료합니다.
  • CM 종료 이후 프로세스 확인 시 ProSync Agent, Instance 프로세스 없음을 확인합니다.
$ cmrctl show
Resource List of Node cm0
=====================================================================
  CLUSTER     TYPE        NAME       STATUS           DETAIL
----------- -------- -------------- -------- ------------------------
... 중략 ...
       cls0       db          db120 UP(NRML) db12, /tibero/tibero7, failed retry cnt: 0
       cls0    group  grp_src_12_34       UP type: prosync (failover: ON)
       cls0    group  grp_tar_34_12       UP type: prosync (failover: ON)
       cls0    agent   agent1_12_34       UP /share/prosync4_12_34/bin/prs_12_34_agent1.sh, start retry cnt: 0
       cls0    agent   agent3_34_12       UP /share/prosync4_34_12/bin/prs_agent3_34_12.sh, start retry cnt: 0
=====================================================================

$ kill -9 ${CM PID}

$ ps -ef |grep prs |grep -v grep
$

node2 CM FailOver 확인

  • node2의 CM은 정상적으로 동작 중
  • node2의 ProSync Agent, Instance 프로세스를 확인합니다.
    • A Set ProSync 기준 llob 프로세스가 정상적으로 FailOver 된 것을 확인합니다.
    • B Set ProSync 기준 apply 프로세스가 정상적으로 FailOver 된 것을 확인합니다.
$ cmrctl show all
Resource List of Node cm1
=====================================================================
  CLUSTER     TYPE        NAME       STATUS           DETAIL
----------- -------- -------------- -------- ------------------------
... 중략 ...
       cls0       db          db121 UP(NRML) db12, /tibero/tibero7, failed retry cnt: 0
       cls0    group  grp_src_12_34       UP type: prosync (failover: ON)
       cls0    group  grp_tar_34_12       UP type: prosync (failover: ON)
       cls0    agent   agent2_12_34       UP /share/prosync4_12_34/bin/prs_12_34_agent2.sh, start retry cnt: 0
       cls0    agent   agent4_34_12       UP /share/prosync4_34_12/bin/prs_agent4_34_12.sh, start retry cnt: 0
=====================================================================

$ prs12
$ prs_adm
Admin> status
prs_agent ID: agent1_12_34, HOST: 10.10.10.65, PORT: 7600, CM_GROUP: grp_src_12_34, CM_ID: 0 is stopped
prs_agent ID: agent2_12_34, HOST: 10.10.10.66, PORT: 7601, CM_GROUP: grp_src_12_34, CM_ID: 1 is running
prs_agent ID: agent3_12_34, HOST: 10.10.10.67, PORT: 7602, CM_GROUP: grp_tar_12_34, CM_ID: 0 is running
prs_agent ID: agent4_12_34, HOST: 10.10.10.68, PORT: 7603, CM_GROUP: grp_tar_12_34, CM_ID: 1 is running

Instance ID: [prs_12_34]
Agent Process for proc[Extract], num[1] is not running.
prs_12_34_ext2 (2) is running (prs_agent ID : agent2_12_34, HOST: 10.10.10.66, PORT: 7601)
prs_12_34_apply1 (1) is running (prs_agent ID : agent3_12_34, HOST: 10.10.10.67, PORT: 7602)
prs_12_34_llob (1) is running (prs_agent ID : agent2_12_34, HOST: 10.10.10.66, PORT: 7600)

$ prs34
$ prs_adm
Admin> status
prs_agent ID: agent1_34_12, HOST: 10.10.10.67, PORT: 6600, CM_GROUP: grp_src_34_12, CM_ID: 0 is running
prs_agent ID: agent2_34_12, HOST: 10.10.10.68, PORT: 6601, CM_GROUP: grp_src_34_12, CM_ID: 1 is running
prs_agent ID: agent3_34_12, HOST: 10.10.10.65, PORT: 6602, CM_GROUP: grp_tar_34_12, CM_ID: 0 is stopped
prs_agent ID: agent4_34_12, HOST: 10.10.10.66, PORT: 6603, CM_GROUP: grp_tar_34_12, CM_ID: 1 is running

Instance ID: [prs_34_12]
prs_34_12_ext1 (1) is running (prs_agent ID : agent1_34_12, HOST: 10.10.10.67, PORT: 6600)
prs_34_12_ext2 (2) is running (prs_agent ID : agent2_34_12, HOST: 10.10.10.68, PORT: 6601)
prs_34_12_apply1 (1) is running (prs_agent ID : agent4_34_12, HOST: 10.10.10.66, PORT: 6602)
prs_34_12_llob (1) is running (prs_agent ID : agent1_34_12, HOST: 10.10.10.67, PORT: 6600)

$ ps -ef |grep prs |grep -v grep
tibero     1704500       1  0 03:26 ?        00:00:06 prs_agent agent2_12_34
tibero     1704556 1704500  0 03:26 ?        00:00:19 prs_ext prs_12_34 2 agent2_12_34
tibero     1737778       1  0 04:28 ?        00:00:04 prs_agent agent4_34_12
tibero     1853638 1737778  0 06:59 ?        00:00:00 prs_apply prs_34_12 1 agent4_34_12
tibero     1853643 1704500  0 06:59 ?        00:00:00 prs_llob prs_12_34 1 agent2_12_34

node1 CM 정상화

  • node1의 CM을 정상화 합니다.
$ tbcm -b
$ tbdown clean
$ tbboot
  • node1과 node2 CM 상태 확인 시 node1의 Agent가 모두 다운되어 있는 것을 확인합니다.
$ cmrctl show all
Resource List of Node cm0
=====================================================================
  CLUSTER     TYPE        NAME       STATUS           DETAIL
----------- -------- -------------- -------- ------------------------
... 중략 ...
       cls0       db          db120 UP(NRML) db12, /tibero/tibero7, failed retry cnt: 0
       cls0    group  grp_src_12_34       UP type: prosync (failover: ON)
       cls0    group  grp_tar_34_12       UP type: prosync (failover: ON)
       cls0    agent   agent1_12_34     DOWN /share/prosync4_12_34/bin/prs_12_34_agent1.sh, start retry cnt: 0
       cls0    agent   agent3_34_12     DOWN /share/prosync4_34_12/bin/prs_agent3_34_12.sh, start retry cnt: 0
=====================================================================
Resource List of Node cm1
=====================================================================
  CLUSTER     TYPE        NAME       STATUS           DETAIL
----------- -------- -------------- -------- ------------------------
... 중략 ...
       cls0       db          db121 UP(NRML) db12, /tibero/tibero7, failed retry cnt: 0
       cls0    group  grp_src_12_34       UP type: prosync (failover: ON)
       cls0    group  grp_tar_34_12       UP type: prosync (failover: ON)
       cls0    agent   agent2_12_34       UP /share/prosync4_12_34/bin/prs_12_34_agent2.sh, start retry cnt: 0
       cls0    agent   agent4_34_12       UP /share/prosync4_34_12/bin/prs_agent4_34_12.sh, start retry cnt: 0
=====================================================================
  • CM FailOver로 Agent가 상대 노드로 넘어가면, 수동으로 FailBack 해주어야합니다.
$ cmrctl start agent --name agent1_12_34
MSG SENDING SUCCESS!

$ cmrctl start agent --name agent3_34_12
MSG SENDING SUCCESS!
  • Agent 기동 후 정상적으로 기동 되었는지 확인합니다.
$ cmrctl show all
Resource List of Node cm0
=====================================================================
  CLUSTER     TYPE        NAME       STATUS           DETAIL
----------- -------- -------------- -------- ------------------------
... 중략 ...
       cls0       db          db120 UP(NRML) db12, /tibero/tibero7, failed retry cnt: 0
       cls0    group  grp_src_12_34       UP type: prosync (failover: ON)
       cls0    group  grp_tar_34_12       UP type: prosync (failover: ON)
       cls0    agent   agent1_12_34       UP /share/prosync4_12_34/bin/prs_12_34_agent1.sh, start retry cnt: 0
       cls0    agent   agent3_34_12       UP /share/prosync4_34_12/bin/prs_agent3_34_12.sh, start retry cnt: 0
=====================================================================
Resource List of Node cm1
=====================================================================
  CLUSTER     TYPE        NAME       STATUS           DETAIL
----------- -------- -------------- -------- ------------------------
... 중략 ...
       cls0       db          db121 UP(NRML) db12, /tibero/tibero7, failed retry cnt: 0
       cls0    group  grp_src_12_34       UP type: prosync (failover: ON)
       cls0    group  grp_tar_34_12       UP type: prosync (failover: ON)
       cls0    agent   agent2_12_34       UP /share/prosync4_12_34/bin/prs_12_34_agent2.sh, start retry cnt: 0
       cls0    agent   agent4_34_12       UP /share/prosync4_34_12/bin/prs_agent4_34_12.sh, start retry cnt: 0
=====================================================================
  • CM FailBack 수행 후 정상적으로 Agent에 맞는 프로세스가 분배 되었는지 확인합니다.
$ prs12
$ prs_adm
Admin> status
prs_agent ID: agent1_12_34, HOST: 10.10.10.65, PORT: 7600, CM_GROUP: grp_src_12_34, CM_ID: 0 is running
prs_agent ID: agent2_12_34, HOST: 10.10.10.66, PORT: 7601, CM_GROUP: grp_src_12_34, CM_ID: 1 is running
prs_agent ID: agent3_12_34, HOST: 10.10.10.67, PORT: 7602, CM_GROUP: grp_tar_12_34, CM_ID: 0 is running
prs_agent ID: agent4_12_34, HOST: 10.10.10.68, PORT: 7603, CM_GROUP: grp_tar_12_34, CM_ID: 1 is running

Instance ID: [prs_12_34]
prs_12_34_ext1 (1) is running (prs_agent ID : agent1_12_34, HOST: 10.10.10.65, PORT: 7600)
prs_12_34_ext2 (2) is running (prs_agent ID : agent2_12_34, HOST: 10.10.10.66, PORT: 7601)
prs_12_34_apply1 (1) is running (prs_agent ID : agent3_12_34, HOST: 10.10.10.67, PORT: 7602)
prs_12_34_llob (1) is running (prs_agent ID : agent1_12_34, HOST: 10.10.10.65, PORT: 7600)

$ prs34
$ prs_adm
Admin> status
prs_agent ID: agent1_34_12, HOST: 10.10.10.67, PORT: 6600, CM_GROUP: grp_src_34_12, CM_ID: 0 is running
prs_agent ID: agent2_34_12, HOST: 10.10.10.68, PORT: 6601, CM_GROUP: grp_src_34_12, CM_ID: 1 is running
prs_agent ID: agent3_34_12, HOST: 10.10.10.65, PORT: 6602, CM_GROUP: grp_tar_34_12, CM_ID: 0 is running
prs_agent ID: agent4_34_12, HOST: 10.10.10.66, PORT: 6603, CM_GROUP: grp_tar_34_12, CM_ID: 1 is running

Instance ID: [prs_34_12]
prs_34_12_ext1 (1) is running (prs_agent ID : agent1_34_12, HOST: 10.10.10.67, PORT: 6600)
prs_34_12_ext2 (2) is running (prs_agent ID : agent2_34_12, HOST: 10.10.10.68, PORT: 6601)
prs_34_12_apply1 (1) is running (prs_agent ID : agent3_34_12, HOST: 10.10.10.65, PORT: 6602)
prs_34_12_llob (1) is running (prs_agent ID : agent1_34_12, HOST: 10.10.10.67, PORT: 6600)

# node1
$ ps -ef |grep prs |grep -v grep
tibero     2091029       1  0 07:03 ?        00:00:00 prs_agent agent1_12_34
tibero     2091085 2091029  0 07:03 ?        00:00:00 prs_ext prs_12_34 1 agent1_12_34
tibero     2091089 2091029  0 07:03 ?        00:00:00 prs_llob prs_12_34 1 agent1_12_34
tibero     2091937       1  1 07:04 ?        00:00:00 prs_agent agent3_34_12
tibero     2092004 2091937  0 07:04 ?        00:00:00 prs_apply prs_34_12 1 agent3_34_12

# node2
$  ps -ef |grep prs |grep -v grep
tibero     1704500       1  0 03:26 ?        00:00:06 prs_agent agent2_12_34
tibero     1704556 1704500  0 03:26 ?        00:00:20 prs_ext prs_12_34 2 agent2_12_34
tibero     1737778       1  0 04:28 ?        00:00:04 prs_agent agent4_34_12

ProSync CM FailOver 동작 (node3, node4)

node3 CM FailOver 수행

$ cmrctl show
Resource List of Node cm0
=====================================================================
  CLUSTER     TYPE        NAME       STATUS           DETAIL
----------- -------- -------------- -------- ------------------------
... 중략 ...
       cls0       db          db340 UP(NRML) db34, /tibero/tibero7, failed retry cnt: 0
       cls0    group  grp_tar_12_34       UP type: prosync (failover: ON)
       cls0    group  grp_src_34_12       UP type: prosync (failover: ON)
       cls0    agent   agent3_12_34       UP /share/prosync4_12_34/bin/prs_agent3_12_34.sh, start retry cnt: 0
       cls0    agent   agent1_34_12       UP /share/prosync4_34_12/bin/prs_agent1_34_12.sh, start retry cnt: 0
=====================================================================

$ kill -9 ${CM PID}

$ ps -ef |grep prs |grep -v grep
$

node4 CM FailOver 확인

  • node4의 CM은 정상적으로 동작 중
  • node4의 ProSync Agent, Instance 프로세스를 확인합니다.
    • A Set ProSync 기준 apply 프로세스가 정상적으로 FailOver 된 것을 확인합니다.
    • B Set ProSync 기준 llob 프로세스가 정상적으로 FailOver 된 것을 확인합니다.
$ cmrctl show all
Node cm0 is not 'UP' Current status DOWN
If this node has been abnormally terminated, try again after the expire timeout(300 sec)
Resource List of Node cm1
=====================================================================
  CLUSTER     TYPE        NAME       STATUS           DETAIL
----------- -------- -------------- -------- ------------------------
... 중략 ...
       cls0       db          db341 UP(NRML) db34, /tibero/tibero7, failed retry cnt: 0
       cls0    group  grp_tar_12_34       UP type: prosync (failover: ON)
       cls0    group  grp_src_34_12       UP type: prosync (failover: ON)
       cls0    agent   agent4_12_34       UP /share/prosync4_12_34/bin/prs_12_34_agent4.sh, start retry cnt: 0
       cls0    agent   agent2_34_12       UP /share/prosync4_34_12/bin/prs_agent2_34_12.sh, start retry cnt: 0
=====================================================================

$ prs12
$ prs_adm
Admin> status
prs_agent ID: agent1_12_34, HOST: 10.10.10.65, PORT: 7600, CM_GROUP: grp_src_12_34, CM_ID: 0 is running
prs_agent ID: agent2_12_34, HOST: 10.10.10.66, PORT: 7601, CM_GROUP: grp_src_12_34, CM_ID: 1 is running
prs_agent ID: agent3_12_34, HOST: 10.10.10.67, PORT: 7602, CM_GROUP: grp_tar_12_34, CM_ID: 0 is stopped
prs_agent ID: agent4_12_34, HOST: 10.10.10.68, PORT: 7603, CM_GROUP: grp_tar_12_34, CM_ID: 1 is running

Instance ID: [prs_12_34]
prs_12_34_ext1 (1) is running (prs_agent ID : agent1_12_34, HOST: 10.10.10.65, PORT: 7600)
prs_12_34_ext2 (2) is running (prs_agent ID : agent2_12_34, HOST: 10.10.10.66, PORT: 7601)
prs_12_34_apply1 (1) is running (prs_agent ID : agent4_12_34, HOST: 10.10.10.68, PORT: 7602)
prs_12_34_llob (1) is running (prs_agent ID : agent1_12_34, HOST: 10.10.10.65, PORT: 7600)

$ prs34
$ prs_adm
Admin> status
prs_agent ID: agent1_34_12, HOST: 10.10.10.67, PORT: 6600, CM_GROUP: grp_src_34_12, CM_ID: 0 is stopped
prs_agent ID: agent2_34_12, HOST: 10.10.10.68, PORT: 6601, CM_GROUP: grp_src_34_12, CM_ID: 1 is running
prs_agent ID: agent3_34_12, HOST: 10.10.10.65, PORT: 6602, CM_GROUP: grp_tar_34_12, CM_ID: 0 is running
prs_agent ID: agent4_34_12, HOST: 10.10.10.66, PORT: 6603, CM_GROUP: grp_tar_34_12, CM_ID: 1 is running

Instance ID: [prs_34_12]
Agent Process for proc[Extract], num[1] is not running.
prs_34_12_ext2 (2) is running (prs_agent ID : agent2_34_12, HOST: 10.10.10.68, PORT: 6601)
prs_34_12_apply1 (1) is running (prs_agent ID : agent3_34_12, HOST: 10.10.10.65, PORT: 6602)
prs_34_12_llob (1) is running (prs_agent ID : agent2_34_12, HOST: 10.10.10.68, PORT: 6600)

$ ps -ef |grep prs |grep -v grep
tibero        3921     661  0 Mar19 pts/0    00:00:00 prs_adm
tibero     1717765       1  0 03:56 ?        00:00:05 prs_agent agent4_12_34
tibero     1733840       1  0 04:26 ?        00:00:05 prs_agent agent2_34_12
tibero     1733891 1733840  0 04:26 ?        00:00:17 prs_ext prs_34_12 2 agent2_34_12
tibero     1856977 1733840  0 07:10 ?        00:00:00 prs_llob prs_34_12 1 agent2_34_12
tibero     1856980 1717765  0 07:10 ?        00:00:00 prs_apply prs_12_34 1 agent4_12_34

node3 CM 정상화

  • node3의 CM을 정상화합니다.
$ tbcm -b
$ tbdown clean
$ tbboot
  • node3과 node4 CM 상태 확인 시 node3의 Agent가 모두 다운되어 있는 것을 확인합니다.
$ cmrctl show all
Resource List of Node cm0
=====================================================================
  CLUSTER     TYPE        NAME       STATUS           DETAIL
----------- -------- -------------- -------- ------------------------
... 중략 ...
       cls0       db          db340 UP(NRML) db34, /tibero/tibero7, failed retry cnt: 0
       cls0    group  grp_tar_12_34       UP type: prosync (failover: ON)
       cls0    group  grp_src_34_12       UP type: prosync (failover: ON)
       cls0    agent   agent3_12_34     DOWN /share/prosync4_12_34/bin/prs_agent3_12_34.sh, start retry cnt: 0
       cls0    agent   agent1_34_12     DOWN /share/prosync4_34_12/bin/prs_agent1_34_12.sh, start retry cnt: 0
=====================================================================
Resource List of Node cm1
=====================================================================
  CLUSTER     TYPE        NAME       STATUS           DETAIL
----------- -------- -------------- -------- ------------------------
... 중략 ...
       cls0       db          db341 UP(NRML) db34, /tibero/tibero7, failed retry cnt: 0
       cls0    group  grp_tar_12_34       UP type: prosync (failover: ON)
       cls0    group  grp_src_34_12       UP type: prosync (failover: ON)
       cls0    agent   agent4_12_34       UP /share/prosync4_12_34/bin/prs_12_34_agent4.sh, start retry cnt: 0
       cls0    agent   agent2_34_12       UP /share/prosync4_34_12/bin/prs_agent2_34_12.sh, start retry cnt: 0
=====================================================================
  • CM FailOver로 Agent가 상대 노드로 넘어가면, 수동으로 FailBack 해주어야합니다.
$ cmrctl start agent --name agent3_12_34
MSG SENDING SUCCESS!

$ cmrctl start agent --name agent1_34_12
MSG SENDING SUCCESS!
  • Agent 기동 후 정상적으로 기동 되었는지 확인합니다.
$ cmrctl show all
Resource List of Node cm0
=====================================================================
  CLUSTER     TYPE        NAME       STATUS           DETAIL
----------- -------- -------------- -------- ------------------------
... 중략 ...
       cls0       db          db340 UP(NRML) db34, /tibero/tibero7, failed retry cnt: 0
       cls0    group  grp_tar_12_34       UP type: prosync (failover: ON)
       cls0    group  grp_src_34_12       UP type: prosync (failover: ON)
       cls0    agent   agent3_12_34       UP /share/prosync4_12_34/bin/prs_agent3_12_34.sh, start retry cnt: 0
       cls0    agent   agent1_34_12       UP /share/prosync4_34_12/bin/prs_agent1_34_12.sh, start retry cnt: 0
=====================================================================
Resource List of Node cm1
=====================================================================
  CLUSTER     TYPE        NAME       STATUS           DETAIL
----------- -------- -------------- -------- ------------------------
... 중략 ...
       cls0       db          db341 UP(NRML) db34, /tibero/tibero7, failed retry cnt: 0
       cls0    group  grp_tar_12_34       UP type: prosync (failover: ON)
       cls0    group  grp_src_34_12       UP type: prosync (failover: ON)
       cls0    agent   agent4_12_34       UP /share/prosync4_12_34/bin/prs_12_34_agent4.sh, start retry cnt: 0
       cls0    agent   agent2_34_12       UP /share/prosync4_34_12/bin/prs_agent2_34_12.sh, start retry cnt: 0
=====================================================================
  • CM FailBack 수행 후 정상적으로 Agent에 맞는 프로세스가 분배 되었는지 확인합니다.
$ prs12
$ prs_adm
Admin> status
prs_agent ID: agent1_12_34, HOST: 10.10.10.65, PORT: 7600, CM_GROUP: grp_src_12_34, CM_ID: 0 is running
prs_agent ID: agent2_12_34, HOST: 10.10.10.66, PORT: 7601, CM_GROUP: grp_src_12_34, CM_ID: 1 is running
prs_agent ID: agent3_12_34, HOST: 10.10.10.67, PORT: 7602, CM_GROUP: grp_tar_12_34, CM_ID: 0 is running
prs_agent ID: agent4_12_34, HOST: 10.10.10.68, PORT: 7603, CM_GROUP: grp_tar_12_34, CM_ID: 1 is running

Instance ID: [prs_12_34]
prs_12_34_ext1 (1) is running (prs_agent ID : agent1_12_34, HOST: 10.10.10.65, PORT: 7600)
prs_12_34_ext2 (2) is running (prs_agent ID : agent2_12_34, HOST: 10.10.10.66, PORT: 7601)
prs_12_34_apply1 (1) is running (prs_agent ID : agent3_12_34, HOST: 10.10.10.67, PORT: 7602)
prs_12_34_llob (1) is running (prs_agent ID : agent1_12_34, HOST: 10.10.10.65, PORT: 7600)

$ prs34
$ prs_adm
Admin> status
prs_agent ID: agent1_34_12, HOST: 10.10.10.67, PORT: 6600, CM_GROUP: grp_src_34_12, CM_ID: 0 is running
prs_agent ID: agent2_34_12, HOST: 10.10.10.68, PORT: 6601, CM_GROUP: grp_src_34_12, CM_ID: 1 is running
prs_agent ID: agent3_34_12, HOST: 10.10.10.65, PORT: 6602, CM_GROUP: grp_tar_34_12, CM_ID: 0 is running
prs_agent ID: agent4_34_12, HOST: 10.10.10.66, PORT: 6603, CM_GROUP: grp_tar_34_12, CM_ID: 1 is running

Instance ID: [prs_34_12]
prs_34_12_ext1 (1) is running (prs_agent ID : agent1_34_12, HOST: 10.10.10.67, PORT: 6600)
prs_34_12_ext2 (2) is running (prs_agent ID : agent2_34_12, HOST: 10.10.10.68, PORT: 6601)
prs_34_12_apply1 (1) is running (prs_agent ID : agent3_34_12, HOST: 10.10.10.65, PORT: 6602)
prs_34_12_llob (1) is running (prs_agent ID : agent1_34_12, HOST: 10.10.10.67, PORT: 6600)

# node3
$ ps -ef |grep prs |grep -v grep
tibero     2064388       1  1 07:11 ?        00:00:00 prs_agent agent3_12_34
tibero     2064444 2064388  0 07:11 ?        00:00:00 prs_apply prs_12_34 1 agent3_12_34
tibero     2064558       1  1 07:12 ?        00:00:00 prs_agent agent1_34_12
tibero     2064617 2064558  0 07:12 ?        00:00:00 prs_ext prs_34_12 1 agent1_34_12
tibero     2064621 2064558  0 07:12 ?        00:00:00 prs_llob prs_34_12 1 agent1_34_12

# node4
$ ps -ef |grep prs |grep -v grep
tibero        3921     661  0 Mar19 pts/0    00:00:00 prs_adm
tibero     1717765       1  0 03:56 ?        00:00:05 prs_agent agent4_12_34
tibero     1733840       1  0 04:26 ?        00:00:05 prs_agent agent2_34_12
tibero     1733891 1733840  0 04:26 ?        00:00:17 prs_ext prs_34_12 2 agent2_34_12