791
PostgreSQL 10.0 preview 功能增強 - WAL一致性校驗
標簽
PostgreSQL , 10.0 , WAL , wal_consistency_checking
背景
10.0 新增了一個DEBUG參數，用於檢測recovery過程中，由於wal replay BUG或者備庫的物理數據塊異常導致的wal replay回放出來的塊不正確的問題。
當產生髒頁時，在wal記錄中，可能有兩種信息：
1. 隻記錄了數據變更的部分。
2. FULL PAGE，記錄了整個數據塊。(發生時機：當開啟了full page write參數，checkpoint後第一次變更的塊)
在PostgreSQL進入恢複過程（或者standby）時，PostgreSQL startup進程會從WAL日誌中讀取wal record與數據文件的塊進行回放組合，生成變更後的塊。如果WAL是full page，則直接使用FULL PAGE。回放後的塊覆蓋老的數據塊，實現恢複的目的。
但是有可能因為各種原因，導致回放後的數據塊是不對的，比如前麵提到的原因：（由於wal replay BUG或者備庫的物理數據塊異常導致的wal replay回放出來的塊不正確）。
PostgreSQL 10.0新增的wal_consistency_checking參數，可以用於發現這種問題。
為什麼PostgreSQL 10.0要加這個參數呢？
因為PostgreSQL的擴展功能極強，已經支持了wal generic record，也就是說，用戶可以自定義往wal中寫數據，開放這樣的功能，有利於開發者調試自己擴展的wal generic writer的正確性。
Add WAL consistency checking facility.
author Robert Haas <rhaas@postgresql.org>
Thu, 9 Feb 2017 04:45:30 +0800 (15:45 -0500)
committer Robert Haas <rhaas@postgresql.org>
Thu, 9 Feb 2017 04:45:30 +0800 (15:45 -0500)
When the new GUC wal_consistency_checking is set to a non-empty value,
it triggers recording of additional full-page images, which are
compared on the standby against the results of applying the WAL record
(without regard to those full-page images). Allowable differences
such as hints are masked out, and the resulting pages are compared;
any difference results in a FATAL error on the standby.
Kuntal Ghosh, based on earlier patches by Michael Paquier and Heikki
Linnakangas. Extensively reviewed and revised by Michael Paquier and
by me, with additional reviews and comments from Amit Kapila, Álvaro
Herrera, Simon Riggs, and Peter Eisentraut.
wal_consistency_checking用法
wal_consistency_checking 參數可以設置為如下值：
all, heap, heap2, btree, hash, gin, gist, sequence, spgist, brin, and generic.
參數內容表示當主庫產生WAL對應的resource manger record時，自動將當時髒頁的FULL PAGE寫入WAL中。在startup進程回放日誌時，會比較 "這個FULL PAGE" 與 "wal partial record+data page replay出來的PAGE" 是否一致，如果不一致，說明WAL回放有問題。startup 進程將會fatal，停止恢複。
對於正常的差異（例如hint bit）是不會報錯的。
《為什麼PostgreSQL查詢語句也可能產生 xlog， 並且可能對buffer有write操作 ? hint bits》
wal_consistency_checking (string)
This parameter is intended to be used to check for bugs in the WAL redo routines.
When enabled, full-page images of any buffers modified in conjunction with the WAL record are added to the record.
If the record is subsequently replayed, the system will first apply each record and then test whether the buffers modified by the record match the stored images.
In certain cases (such as hint bits), minor variations are acceptable, and will be ignored.
Any unexpected differences will result in a fatal error, terminating recovery.
The default value of this setting is the empty string, which disables the feature.
It can be set to all to check all records, or to a comma-separated list of resource managers to check only records originating from those resource managers.
Currently, the supported resource managers are heap, heap2, btree, hash, gin, gist, sequence, spgist, brin, and generic. Only superusers can change this setting.
這個patch的討論，詳見郵件組，本文末尾URL。
PostgreSQL社區的作風非常嚴謹，一個patch可能在郵件組中討論幾個月甚至幾年，根據大家的意見反複的修正，patch合並到master已經非常成熟，所以PostgreSQL的穩定性也是遠近聞名的。
參考
https://www.postgresql.org/docs/devel/static/runtime-config-developer.html
最後更新：2017-04-21 00:30:37