Skip to main content

Search

Welcome to Tibero GTS!

RED ZONE Error and DB Down When Interconnect Load Occurs in TAC Environment

Document Type | Troubleshooting

Category | Patch/Upgrade

Applicable Product Versions | 6FS06, 6FS07, 6FS07PS, 7FS01, 7FS02

Document Number | TPATS019

 

Issue

After the INC REDZONE CHECK ERROR occurs, an Internal Error with condition: 'CINUSE(chunk)' (1 args) (shp_alloc.c:1131:shp_chunk_free) error follows.
 
1. REDZONE Error Occurrence
[74] *** 2025/01/14 16:57:05.221 ***
[74] callstack dump from tbsvr_ACSD for [CMPT023, 74, 56698/14617]
[74] SEE DUMP tbsvr.callstack.14617 (lwpid 56698) BY PSTACK
INC REDZONE CHECK ERROR.
REDZONE:
0000: 0D 1C EB 8F 0D C2 84 70 .......p 
----------------------------------------------------------------------
SHP Chunk Dump
----------------------------------------------------------------------
Chunk Struct Dump 0x9A7BC010
size: 0 cinuse: 0 pinuse: 0 footer: 0 prev-chunk size: 0 type: 0 alloc_idx: 0 recr: 0
---------------------------------------------------------------------- 
----------------------------------------------------------------------
Dbginfo Struct Dump 0x9A7BC020
mem ptr: 0x9A7BC040 mem size: -1431655766 loc: :-1073741823
----------------------------------------------------------------------
----------------------------------------------------------------------
Chunk Mem Hex Dump 0x9A7BC010 size 8
0000: 00 00 00 00 00 00 00 00 ........ 
----------------------------------------------------------------------
Previous Chunk Dump

2. Internal Error Occurrence
[74] Internal Error with condition: 'CINUSE(chunk)' (1 args) (shp_alloc.c:1131:shp_chunk_free) (pid=14617, sessid=74, tid=74)
chunk->head = 0 = 0 = 0x0
[74] client : ip [] process [] logon time : []
[74] prev sql : "NULL"
[74] current sql : "NULL"
[74] *** 2025/01/14 17:01:57.260 ***
[74] callstack dump from tbsvr_ACSD for [CMPT023, 74, 56698/14617]
[74] SEE DUMP tbsvr.callstack.14617 (lwpid 56698) BY PSTACK
Waiting for pstack process(1337) has timed out. kill_child=1
Waiting for pstack process(1337) has timed out. kill_child=1

3. Due to the above errors, important processes go down causing DB down
================================
Thread #74 (Session #74): tb_thread_exit: (signo:0, sigtype:15)
[39] 2025/01/14 17:07:01.476 Internal Error with condition '!"Background thread exited abnormally!"' (tbsvr_cthr.c:3054) (pid=14617, sessid=39, tid=39, os_thr_id=56649)
[39] client : ip [] process [] logon time : []
[39] prev sql : "NULL"
[39] current sql : "NULL"
[39] *** 2025/01/14 17:07:01.477 ***
[39] callstack dump from tbsvr_ACSD for [ACSD, 39, 56649/14617]
[39] SEE DUMP tbsvr.callstack.14617 (lwpid 56649) BY PSTACK
================================
[39] ***** FATAL ASSERT (PID = 14617, TID = 39, SESSID = 39):
***** tbsvr_tracedump.c:280 (tb_svr_assert_cmd): Assertion 'false' failed.
***** Program: tbsvr -startup -t NORMAL -SVR_SID kcip_idb1 
[0] ***** FATAL ASSERT (PID = 14537, TID = 0, SESSID = 0):
***** tbsvr_signal.c:1975 (sig_fatal_assert_handler): Assertion 'false' failed.
***** Program: tbsvr -startup -t NORMAL -SVR_SID kcip_idb1

 

Cause

The CMPT (Cluster Message Processor Thread), which is the thread that processes messages between nodes, requests other threads to wait for the message on its behalf when it cannot process the message.
 
This message forwarding process proceeds in the following two steps.
  1. Forward the message to another thread
  2. Increase the reference count so that the memory allocated for the message is not returned
 
During the above process, before increasing the reference count, the other threads that received the message may finish their work first, and the original CMPT may be busy with other tasks while a newly assigned CMPT finishes processing and returns the message first.
Subsequently, the original CMPT continues its remaining work and attempts to return the message again, which was already returned, resulting in an error due to duplicate return of the already returned message.

 

Solutions

Apply patch 274230c to resolve the issue.
Caution
Apply the patch through technical support provided by Tmax Tibero.
Note
You can work around the issue by applying the following parameters.
  • _CR_ASYNC_SERVICE: This parameter determines whether to request another thread to wait on behalf when the block being accessed during inter-node message processing is being read from disk by another session and needs to wait.
  • _BUF_REQ_ASYNC_SERVICE: This parameter determines whether to request another thread to perform the task on behalf when free blocks need to be secured in cache during inter-node message processing.
 
 
 

Related to