Document Type | Technical Information
Category | Installation
Applicable Product Version | Tibero 7FS04PS
Document Number | PINTI010
Overview
- Part 1: Prosync Installation Preparation and Agent Installation
- Part 2: Instanct Installation
- Part 3: CM Failover Configuration
- Part 4: CM Failover Testing
Test Environment
| Category | OS | IP | DBMS Version | Prosync Version |
|---|---|---|---|---|
| Source_TAC0 | Rocky Linux release 8.10 | 10.10.10.61 | Tibero 7.2.4 (build 305455) | 4.6.0 (build 308985) |
| Source_TAC1 | Rocky Linux release 8.10 | 10.10.10.62 | Tibero 7.2.4 (build 305455) | 4.6.0 (build 308985) |
| Target_TAC0 | Rocky Linux release 8.10 | 10.10.10.63 | Tibero 7.2.4 (build 305455) | 4.6.0 (build 308985) |
| Target_TAC1 | Rocky Linux release 8.10 | 10.10.10.64 | Tibero 7.2.4 (build 305455) | 4.6.0 (build 308985) |
Note
The Prosync CM Failover feature is supported from Prosync version 4.3 and above.
Method
CM Failover Testing
1. Source_TAC0 DOWN
Check CM Failover operation when the DB & CM of node 0 on Source TAC is down.
1-1. Check Source_TAC prosync process status
-- Check prosync process on source node#0 $ ps -ef | grep prs tibero 965072 1 0 07:45 ? 00:00:00 prs_agent src_agent1 tibero 965151 965072 0 07:45 ? 00:00:00 prs_ext PRS_FAILOVER 1 src_agent1 tibero 965165 965072 0 07:45 ? 00:00:00 prs_llob PRS_FAILOVER 1 src_agent1 tibero 970098 846411 0 07:50 pts/2 00:00:00 grep --color=auto prs -- Check prosync process on source node#1 $ ps -ef | grep prs tibero 954562 1 0 07:45 ? 00:00:00 prs_agent src_agent2 tibero 954621 954562 0 07:45 ? 00:00:00 prs_ext PRS_FAILOVER 2 src_agent2 tibero 959694 471337 0 07:51 pts/0 00:00:00 grep --color=auto prs
1-2. Source_TAC0 DB & CM DOWN
-- source node#0 db down $ tbdown immediate Tibero instance terminated (IMMEDIATE mode). -- source node#0 cm down $ tbcm -d CM DOWN SUCCESS!
1-3. Check Source_TAC prosync process status
-- Check prosync process on source node#0 $ ps -ef | grep prs tibero 970966 846411 0 07:51 pts/2 00:00:00 grep --color=auto prs -- Check prosync process on source node#1 $ ps -ef | grep prs tibero 954562 1 0 07:45 ? 00:00:00 prs_agent src_agent2 tibero 954621 954562 0 07:45 ? 00:00:00 prs_ext PRS_FAILOVER 2 src_agent2 tibero 960366 954562 11 07:51 ? 00:00:02 prs_ext PRS_FAILOVER 1 src_agent2 tibero 960370 954562 0 07:51 ? 00:00:00 prs_llob PRS_FAILOVER 1 src_agent2 tibero 960746 471337 0 07:52 pts/0 00:00:00 grep --color=auto prs
1-4. Check prs_adm status
$ prs_adm ProSync 4 - Admin Utility TmaxData Corporation Copyright (c) 2024-. All rights reserved. Admin> status prs_agent ID: src_agent1, HOST: 10.10.10.61, PORT: 7600, CM_GROUP: SRC_CM, CM_ID: 0 is stopped prs_agent ID: src_agent2, HOST: 10.10.10.62, PORT: 7700, CM_GROUP: SRC_CM, CM_ID: 1 is running prs_agent ID: tar_agent1, HOST: 10.10.10.63, PORT: 7800, CM_GROUP: TAR_CM, CM_ID: 0 is running prs_agent ID: tar_agent2, HOST: 10.10.10.64, PORT: 7900, CM_GROUP: TAR_CM, CM_ID: 1 is running Instance ID: [PRS_FAILOVER] PRS_FAILOVER_ext1 (1) is running (prs_agent ID : src_agent2, HOST: 10.10.10.62, PORT: 7700) PRS_FAILOVER_ext2 (2) is running (prs_agent ID : src_agent2, HOST: 10.10.10.62, PORT: 7700) PRS_FAILOVER_apply1 (1) is running (prs_agent ID : tar_agent1, HOST: 10.10.10.63, PORT: 7800) PRS_FAILOVER_llob (1) is running (prs_agent ID : src_agent2, HOST: 10.10.10.62, PORT: 7700)
Note
After CM failover occurs, ext1 starts on the opposite node 2 server, but looking at the ext log, there are cases where synchronization does not occur as it searches for *.tlr files as below.
[03-09T00:26:39.209893] [REA-05] [I] txinfo found (/root/prosync4/var/PRS_FAILOVER/lr1/redo_data_13_0.tlr) [03-09T00:26:39.210079] [REA-05] [I] tli opened (flag:0x04, logblk_size:512, logblk_cnt:8) [03-09T00:26:39.210119] [REA-05] [E] tli_load_txinfo failed (rc:-3) [prs_ext_tread.c:1072]
- Cause: When using the CM Failover feature, the Prosync binary and TAC redo, Archivelog files must be located on a shared filesystem accessible by both nodes.
2. Source_TAC1 DOWN
Check CM Failover operation when the DB & CM of node 1 on Source TAC is down.
2-1. Check Source_TAC prosync process status
-- Check prosync process on source node#0 $ ps -ef | grep prs tibero 965072 1 0 07:45 ? 00:00:00 prs_agent src_agent1 tibero 965151 965072 0 07:45 ? 00:00:00 prs_ext PRS_FAILOVER 1 src_agent1 tibero 965165 965072 0 07:45 ? 00:00:00 prs_llob PRS_FAILOVER 1 src_agent1 tibero 970098 846411 0 07:50 pts/2 00:00:00 grep --color=auto prs -- Check prosync process on source node#1 $ ps -ef | grep prs tibero 954562 1 0 07:45 ? 00:00:00 prs_agent src_agent2 tibero 954621 954562 0 07:45 ? 00:00:00 prs_ext PRS_FAILOVER 2 src_agent2 tibero 959694 471337 0 07:51 pts/0 00:00:00 grep --color=auto prs
2-2. Source_TAC1 DB & CM DOWN
-- source node#1 db down $ tbdown immediate Tibero instance terminated (IMMEDIATE mode). -- source node#1 cm down $ tbcm -d CM DOWN SUCCESS!
2-3. Check Source_TAC prosync process status
-- Check prosync process on source node#0 $ ps -ef | grep prs tibero 978366 1 0 08:27 ? 00:00:00 prs_agent src_agent1 tibero 978476 978366 0 08:27 ? 00:00:01 prs_ext PRS_FAILOVER 1 src_agent1 tibero 978484 978366 0 08:27 ? 00:00:00 prs_llob PRS_FAILOVER 1 src_agent1 tibero 990974 978366 13 08:39 ? 00:00:00 prs_ext PRS_FAILOVER 2 src_agent1 tibero 991056 846411 0 08:40 pts/2 00:00:00 grep --color=auto prs -- Check prosync process on source node#1 $ ps -ef | grep prs tibero 970966 846411 0 07:51 pts/2 00:00:00 grep --color=auto prs
2-4. Check prs_adm status
$ prs_adm ProSync 4 - Admin Utility TmaxData Corporation Copyright (c) 2024-. All rights reserved. Admin> status prs_agent ID: src_agent1, HOST: 10.10.10.61, PORT: 7600, CM_GROUP: SRC_CM, CM_ID: 0 is running prs_agent ID: src_agent2, HOST: 10.10.10.62, PORT: 7700, CM_GROUP: SRC_CM, CM_ID: 1 is stopped prs_agent ID: tar_agent1, HOST: 10.10.10.63, PORT: 7800, CM_GROUP: TAR_CM, CM_ID: 0 is running prs_agent ID: tar_agent2, HOST: 10.10.10.64, PORT: 7900, CM_GROUP: TAR_CM, CM_ID: 1 is running Instance ID: [PRS_FAILOVER] PRS_FAILOVER_ext1 (1) is running (prs_agent ID : src_agent1, HOST: 10.10.10.61, PORT: 7600) PRS_FAILOVER_ext2 (2) is running (prs_agent ID : src_agent1, HOST: 10.10.10.61, PORT: 7600) PRS_FAILOVER_apply1 (1) is running (prs_agent ID : tar_agent1, HOST: 10.10.10.63, PORT: 7800) PRS_FAILOVER_llob (1) is running (prs_agent ID : src_agent1, HOST: 10.10.10.61, PORT: 7600)
3. Target_TAC0 DOWN
Check CM Failover operation when the DB & CM of node 0 on Target TAC is down.
3-1. Check Target_TAC prosync process status
-- Check prosync process on target node#0 $ ps -ef | grep prs tibero 925564 1 1 08:33 ? 00:00:00 prs_agent tar_agent1 tibero 925642 925564 0 08:33 ? 00:00:00 prs_apply PRS_FAILOVER 1 tar_agent1 tibero 925866 479606 0 08:33 pts/1 00:00:00 grep --color=auto prs -- Check prosync process on target node#1 $ ps -ef | grep prs tibero 921553 1 15 08:34 ? 00:00:00 prs_agent tar_agent2 tibero 921629 764786 0 08:34 pts/2 00:00:00 grep --color=auto prs
3-2. Target_TAC0 DB & CM DOWN
-- target node#1 db down $ tbdown immediate Tibero instance terminated (IMMEDIATE mode). -- target node#1 cm down $ tbcm -d CM DOWN SUCCESS!
3-3. Check Target_TAC prosync process status
-- Check prosync process on target node#0 $ ps -ef | grep prs tibero 924889 479606 0 08:31 pts/1 00:00:00 grep --color=auto prs -- Check prosync process on target node#1 $ ps -ef | grep prs tibero 900667 1 0 07:46 ? 00:00:00 prs_agent tar_agent2 tibero 919319 900667 0 08:28 ? 00:00:00 prs_apply PRS_FAILOVER 1 tar_agent2 tibero 920441 764786 0 08:31 pts/2 00:00:00 grep --color=auto prs
3-4. Check prs_adm status
$ prs_adm ProSync 4 - Admin Utility TmaxData Corporation Copyright (c) 2024-. All rights reserved. Admin> status prs_agent ID: src_agent1, HOST: 10.10.10.61, PORT: 7600, CM_GROUP: SRC_CM, CM_ID: 0 is running prs_agent ID: src_agent2, HOST: 10.10.10.62, PORT: 7700, CM_GROUP: SRC_CM, CM_ID: 1 is running prs_agent ID: tar_agent1, HOST: 10.10.10.63, PORT: 7800, CM_GROUP: TAR_CM, CM_ID: 0 is stopped prs_agent ID: tar_agent2, HOST: 10.10.10.64, PORT: 7900, CM_GROUP: TAR_CM, CM_ID: 1 is running Instance ID: [PRS_FAILOVER] PRS_FAILOVER_ext1 (1) is running (prs_agent ID : src_agent1, HOST: 10.10.10.61, PORT: 7600) PRS_FAILOVER_ext2 (2) is running (prs_agent ID : src_agent2, HOST: 10.10.10.62, PORT: 7700) PRS_FAILOVER_apply1 (1) is running (prs_agent ID : tar_agent2, HOST: 10.10.10.64, PORT: 7900) PRS_FAILOVER_llob (1) is running (prs_agent ID : src_agent1, HOST: 10.10.10.61, PORT: 7600)
4. Target_TAC1 DOWN
Check CM Failover operation when the DB & CM of node 1 on Target TAC is down.
4-1. Check Target_TAC prosync process status
-- Check prosync process on target node#0 $ ps -ef | grep prs tibero 925564 1 1 08:33 ? 00:00:00 prs_agent tar_agent1 tibero 925866 479606 0 08:33 pts/1 00:00:00 grep --color=auto prs -- Check prosync process on target node#1 $ ps -ef | grep prs tibero 900667 1 0 07:46 ? 00:00:00 prs_agent tar_agent2 tibero 919319 900667 0 08:28 ? 00:00:00 prs_apply PRS_FAILOVER 1 tar_agent2 tibero 920441 764786 0 08:31 pts/2 00:00:00 grep --color=auto prs
4-2. Target_TAC1 DB & CM DOWN
-- target node#1 db down $ tbdown immediate Tibero instance terminated (IMMEDIATE mode). -- target node#1 cm down $ tbcm -d CM DOWN SUCCESS!
4-3. Check Target_TAC prosync process status
-- Check prosync process on target node#0 $ ps -ef | grep prs tibero 925564 1 1 08:33 ? 00:00:00 prs_agent tar_agent1 tibero 925642 925564 0 08:33 ? 00:00:00 prs_apply PRS_FAILOVER 1 tar_agent1 tibero 925866 479606 0 08:33 pts/1 00:00:00 grep --color=auto prs -- Check prosync process on target node#1 $ ps -ef | grep prs tibero 924889 479606 0 08:31 pts/1 00:00:00 grep --color=auto prs
4-4. Check prs_adm status
$ prs_adm ProSync 4 - Admin Utility TmaxData Corporation Copyright (c) 2024-. All rights reserved. Admin> status prs_agent ID: src_agent1, HOST: 10.10.10.61, PORT: 7600, CM_GROUP: SRC_CM, CM_ID: 0 is running prs_agent ID: src_agent2, HOST: 10.10.10.62, PORT: 7700, CM_GROUP: SRC_CM, CM_ID: 1 is running prs_agent ID: tar_agent1, HOST: 10.10.10.63, PORT: 7800, CM_GROUP: TAR_CM, CM_ID: 0 is running prs_agent ID: tar_agent2, HOST: 10.10.10.64, PORT: 7900, CM_GROUP: TAR_CM, CM_ID: 1 is stopped Instance ID: [PRS_FAILOVER] PRS_FAILOVER_ext1 (1) is running (prs_agent ID : src_agent1, HOST: 10.10.10.61, PORT: 7600) PRS_FAILOVER_ext2 (2) is running (prs_agent ID : src_agent2, HOST: 10.10.10.62, PORT: 7700) PRS_FAILOVER_apply1 (1) is running (prs_agent ID : tar_agent1, HOST: 10.10.10.63, PORT: 7800) PRS_FAILOVER_llob (1) is running (prs_agent ID : src_agent1, HOST: 10.10.10.61, PORT: 7600)