黑塞矩陣

海森矩陣（德語：Hesse-Matrix；英語：Hessian matrix 或 Hessian），又譯作黑塞矩阵、海塞（赛）矩陣或海瑟矩陣等，是一個由多變量實值函數的所有二階偏導數組成的方陣，由德國數學家奧托·黑塞引入並以其命名。

定義

假設有一實值函數$f(x_{1},x_{2},\dots ,x_{n})\,$，如果 $f\,$的所有二階偏導數都存在並在定義域內連續，那麼函數$f\,$的黑塞矩陣為

$$\mathbf {H} ={\begin{bmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{n}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{n}}}\\\\\vdots &\vdots &\ddots &\vdots \\\\{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{n}^{2}}}\end{bmatrix}}\,$$

或使用下標記號表示為

$$\mathbf {H} _{ij}={\frac {\partial ^{2}f}{\partial x_{i}\partial x_{j}}}$$

顯然黑塞矩陣 $\mathbf {H} \,$是一個$n\times n\,$方陣。黑塞矩陣的行列式被稱爲黑塞式（英語：Hessian），而需注意的是英語環境下使用Hessian一詞時可能指上述矩陣也可能指上述矩陣的行列式。

性質

由高等數學知識可知，若一元函數$f(x)\,$在$x=x_{0}\,$點的某個鄰域內具有任意階導數，則函數$f(x)\,$在$x=x_{0}\,$點處的泰勒展開式為

$$f(x)=f(x_{0})+f'(x_{0})\Delta x+{\frac {f''(x_{0})}{2!}}\Delta x^{2}+\cdots \,$$

其中，$\Delta x=x-x_{0}\,$。

同理，二元函數$f(x_{1},x_{2})\,$在$x_{0}(x_{10},x_{20})\,$點處的泰勒展開式為

$$f(x_{1},x_{2})=f(x_{10},x_{20})+f_{x_{1}}(x_{10},x_{20})\Delta x_{1}+f_{x_{2}}(x_{10},x_{20})\Delta x_{2}+{\frac {1}{2}}[f_{x_{1}x_{1}}(x_{10},x_{20})\Delta x_{1}^{2}+2f_{x_{1}x_{2}}(x_{10},x_{20})\Delta x_{1}\Delta x_{2}+f_{x_{2}x_{2}}(x_{10},x_{20})\Delta x_{2}^{2}]+\cdots \,$$

其中，$\Delta x_{1}=x_{1}-x_{10}\,$，$\Delta x_{2}=x_{2}-x_{20}\,$，$f_{x_{1}}={\frac {\partial f}{\partial x_{1}}}\,$，$f_{x_{2}}={\frac {\partial f}{\partial x_{2}}}\,$，$f_{x_{1}x_{1}}={\frac {\partial ^{2}f}{\partial x_{1}^{2}}}\,$，$f_{x_{2}x_{2}}={\frac {\partial ^{2}f}{\partial x_{2}^{2}}}\,$，$f_{x_{1}x_{2}}={\frac {\partial ^{2}f}{\partial x_{1}\partial x_{2}}}={\frac {\partial ^{2}f}{\partial x_{2}\partial x_{1}}}\,$。

將上述展開式寫成矩陣形式，則有

$$f(x)=f(x_{0})+\nabla f(x_{0})^{\mathrm {T} }\Delta x+{\frac {1}{2}}\Delta x^{\mathrm {T} }G(x_{0})\Delta x+\cdots$$

其中，$\Delta x={\begin{bmatrix}\Delta x_{1}\\\\\Delta x_{2}\end{bmatrix}}\,$，$\Delta x^{\mathrm {T} }={\begin{bmatrix}\Delta x_{1}&\Delta x_{2}\end{bmatrix}}\,$是$\Delta x$的轉置，$\nabla f(x_{0})={\begin{bmatrix}{\frac {\partial f}{\partial x_{1}}}\\\\{\frac {\partial f}{\partial x_{2}}}\end{bmatrix}}\,$是函數$f(x_{1},x_{2})\,$在$x_{0}(x_{10},x_{20})\,$的梯度，矩陣

$$G(x_{0})={\begin{bmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}\end{bmatrix}}_{x_{0}}\,$$

即函數$f(x_{1},x_{2})\,$在$x_{0}(x_{10},x_{20})\,$點處的$2\times 2\,$黑塞矩阵。它是由函数$f(x_{1},x_{2})$在$x_{0}(x_{10},x_{20})$点处的所有二階偏導數所組成的方陣。

由函數的二次連續性，有

$${\frac {\partial ^{2}f}{\partial x_{1}\partial x_{2}}}={\frac {\partial ^{2}f}{\partial x_{2}\partial x_{1}}}$$

所以，黑塞矩陣$G(x_{0})\,$为對稱矩陣。

將二元函數的泰勒展開式推廣到多元函數，函數$f(x_{1},x_{2},\cdots ,x_{n})\,$在$x_{0}(x_{1},x_{2},\cdots ,x_{n})\,$點處的泰勒展開式為

$$f(x)=f(x_{0})+\nabla f(x_{0})^{\mathrm {T} }\Delta x+{\frac {1}{2}}\Delta x^{\mathrm {T} }G(x_{0})\Delta x+\cdots \,$$

其中，

$$\nabla f(x_{0})={\begin{bmatrix}{\frac {\partial f}{\partial x_{1}}}&{\frac {\partial f}{\partial x_{2}}}&\cdots &{\frac {\partial f}{\partial x_{n}}}\end{bmatrix}}_{x_{0}}^{T}\,$$

為函數$f(x)$在$x_{0}(x_{1},x_{2},\cdots ,x_{n})\,$點的梯度，

$$G(x_{0})={\begin{bmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{n}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{n}}}\\\\\vdots &\vdots &\ddots &\vdots \\\\{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{n}^{2}}}\end{bmatrix}}_{x_{0}}\,$$

為函數$f(x)\,$在$x_{0}(x_{1},x_{2},\cdots ,x_{n})\,$點的$n\times n\,$黑塞矩陣。若函數有$n\,$次連續性，則函數的$n\times n\,$黑塞矩陣是對稱矩陣。

說明：在優化設計領域中，黑塞矩陣常用$G\,$表示，且梯度有時用$g\,$表示。

函數$f\,$的黑塞矩陣和雅可比矩陣有如下關係：

$$\mathrm {H} (f)=\mathrm {J} (\nabla f)^{T}\,$$

即函數$f\,$的黑塞矩陣等於其梯度的雅可比矩陣。

應用

函數的極值條件

對於一元函数$f(x)\,$，在給定區間內某$x=x_{0}\,$點處可導，並在$x=x_{0}\,$點處取得極值，其必要條件是

$$f'(x_{0})=0\,$$

即函數$f(x)\,$的極值必定在駐點處取得，或者說可導函數$f(x)\,$的極值點必定是駐點；但反過來，函數的駐點不一定是極值點。檢驗駐點是否為極值點，可以採用二階導數的正負號來判斷。根據函數$f(x)\,$在$x=x_{0}\,$點處的泰勒展開式，考慮到上述極值必要條件，有

$$f(x)=f(x_{0})+{\frac {f''(x_{0})}{2!}}\Delta x^{2}+\cdots \,$$

若$f(x)\,$在$x=x_{0}\,$點處取得極小值，則要求在$x=x_{0}\,$某一鄰域內一切點$x\,$都必須滿足

$$f(x)-f(x_{0})>0\,$$

即要求

$${\frac {f''(x_{0})}{2!}}\Delta x^{2}>0\,$$

亦即要求

$$f''(x_{0})>0\,$$

$f(x)\,$在$x=x_{0}\,$點處取得極大值的討論與之類似。於是有極值充分條件：

設一元函数$f(x)\,$在$x=x_{0}\,$點處具有二階導數，且$f'(x_{0})=0\,$，$f''(x_{0})\neq 0\,$，則

1. 當$f''(x_{0})>0\,$時，函數$f(x)\,$在$x=x_{0}\,$處取得極小值；
2. 當$f''(x_{0})<0\,$時，函數$f(x)\,$在$x=x_{0}\,$處取得極大值。

而當$f''(x_{0})=0\,$時，無法直接判斷，還需要逐次檢驗其更高階導數的正負號。由此有一个規律：若其開始不為零的導數階數為偶數，則駐點是極值點；若為奇數，則為拐點，而不是極值點。

對於二元函数$f(x_{1},x_{2})\,$，在給定區域內某$x_{0}(x_{10},x_{20})\,$點處可導，並在$x_{0}(x_{10},x_{20})\,$點處取得極值，其必要條件是

$$f_{x_{1}}(x_{0})=f_{x_{2}}(x_{0})=0\,$$

即

$$\nabla f(x_{0})=0\,$$

同樣，這只是必要條件，要進一步判斷$x_{0}(x_{10},x_{20})\,$是否為極值點需要找到取得極值的充分條件。根據函數$f(x_{1},x_{2})\,$在$x_{0}(x_{10},x_{20})\,$點處的泰勒展開式，考慮到上述極值必要條件，有

$$f(x_{1},x_{2})=f(x_{10},x_{20})+{\frac {1}{2}}[f_{x_{1}x_{1}}(x_{0})\Delta x_{1}^{2}+2f_{x_{1}x_{2}}(x_{0})\Delta x_{1}\Delta x_{2}+f_{x_{2}x_{2}}(x_{0})\Delta x_{2}^{2}]+\cdots \,$$

設$A=f_{x_{1}x_{1}}(x_{0})\,$，$B=f_{x_{1}x_{2}}(x_{0})\,$，$C=f_{x_{2}x_{2}}(x_{0})\,$，則

$$f(x_{1},x_{2})=f(x_{10},x_{20})+{\frac {1}{2}}[A\Delta x_{1}^{2}+2B\Delta x_{1}\Delta x_{2}+C\Delta x_{2}^{2}]+\cdots \,$$

或

$$f(x_{1},x_{2})=f(x_{10},x_{20})+{\frac {1}{2A}}[(A\Delta x_{1}+B\Delta x_{2})^{2}+(AC-B^{2})\Delta x_{2}^{2}]+\cdots \,$$

若$f(x_{1},x_{2})\,$在$x_{0}(x_{10},x_{20})\,$點處取得極小值，則要求在$x_{0}(x_{10},x_{20})\,$某一鄰域內一切點$x\,$都必須滿足

$$f(x_{1},x_{2})-f(x_{10},x_{20})>0\,$$

即要求

$${\frac {1}{2A}}[(A\Delta x_{1}+B\Delta x_{2})^{2}+(AC-B^{2})\Delta x_{2}^{2}]>0\,$$

亦即要求$A>0\,$，$AC-B^{2}>0\,$

即

$$\left.{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}\right|_{x_{0}}>0\,$$

$${\begin{bmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}-({\frac {\partial ^{2}f}{\partial x_{1}\partial x_{2}}})^{2}\end{bmatrix}}_{x_{0}}>0\,$$

此條件反映了$f(x_{1},x_{2})\,$在$x_{0}(x_{10},x_{20})\,$點處的黑塞矩陣$G(x_{0})\,$的各階主子式都大於零，即對於

$$G(x_{0})={\begin{bmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}\end{bmatrix}}_{x_{0}}\,$$

要求

$$\left.{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}\right|_{x_{0}}>0\,$$

$$|G(x_{0})|={\begin{vmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}\end{vmatrix}}_{x_{0}}>0\,$$

$f((x_{1},x_{2})\,$在$x_{0}(x_{10},x_{20})\,$點處取得極大值的討論與之類似。於是有極值充分條件：

設二元函数$f(x_{1},x_{2})\,$在$x_{0}(x_{10},x_{20})\,$點的鄰域內連續且具有一階和二階連續偏導數，又有$f_{x_{1}}(x_{0})=f_{x_{2}}(x_{0})=0\,$，同時令$A=f_{x_{1}x_{1}}(x_{0})\,$，$B=f_{x_{1}x_{2}}(x_{0})\,$，$C=f_{x_{2}x_{2}}(x_{0})\,$，則

1. 當$A>0\,$，$AC-B^{2}>0\,$時，函數$f(x_{1},x_{2})\,$在$x_{0}(x_{10},x_{20})\,$處取得極小值；
2. 當$A<0\,$，$AC-B^{2}>0\,$時，函數$f(x_{1},x_{2})\,$在$x_{0}(x_{10},x_{20})\,$處取得極大值。

此外可以判斷，當$AC-B^{2}<0\,$時，函數$f(x_{1},x_{2})\,$在$x_{0}(x_{10},x_{20})\,$點處沒有極值，此點稱爲鞍點。而當$AC-B^{2}=0\,$時，無法直接判斷，對此，補充一個規律：當$AC-B^{2}=0\,$時，如果有$A\equiv 0\,$，那麼函數$f(x_{1},x_{2})\,$在$x_{0}(x_{10},x_{20})\,$有極值，且當$C>0\,$有極小值，當$C<0\,$有極大值。

由線性代數的知識可知，若矩陣$G(x_{0})\,$滿足

$$\left.{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}\right|_{x_{0}}>0\,$$

$${\begin{vmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}\end{vmatrix}}_{x_{0}}>0\,$$

則矩陣$G(x_{0})\,$是正定矩陣，或者說矩陣$G(x_{0})\,$正定。

若矩陣$G(x_{0})\,$滿足

$$\left.{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}\right|_{x_{0}}<0\,$$

$${\begin{vmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}\end{vmatrix}}_{x_{0}}>0\,$$

則矩陣$G(x_{0})\,$是負定矩陣，或者說矩陣$G(x_{0})\,$負定。

於是，二元函數$f(x_{1},x_{2})\,$在$x_{0}(x_{10},x_{20})\,$點處取得極值的條件表述為：二元函數$f(x_{1},x_{2})\,$在$x_{0}(x_{10},x_{20})\,$點處的黑塞矩陣正定，則取得極小值；在$x_{0}(x_{10},x_{20})\,$點處的黑塞矩陣負定，則取得極大值。

對於多元函數$f(x_{1},x_{2},\cdots ,x_{n})\,$，若在$x_{0}(x_{1},x_{2},\cdots ,x_{n})\,$點處取得極值，則極值存在的必要條件為

$$\nabla f(x_{0})={\begin{bmatrix}{\frac {\partial f}{\partial x_{1}}}&{\frac {\partial f}{\partial x_{2}}}&\cdots &{\frac {\partial f}{\partial x_{n}}}\end{bmatrix}}_{x_{0}}^{T}=0\,$$

取得極小值的充分條件為

$$G(x_{0})={\begin{bmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{n}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{n}}}\\\\\vdots &\vdots &\ddots &\vdots \\\\{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{n}^{2}}}\end{bmatrix}}_{x_{0}}\,$$

正定，即要求$G(x_{0})\,$的各階主子式都大於零，即

$$\left.{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}\right|_{x_{0}}>0\,$$

$${\begin{vmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}\end{vmatrix}}_{x_{0}}>0\,$$

$$\vdots$$

$$|G(x_{0})|>0\,$$

取得極大值的充分條件為

$$G(x_{0})={\begin{bmatrix}{\frac {\partial ^{2}f}{\partial x_{1}^{2}}}&{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{1}\,\partial x_{n}}}\\\\{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{2}^{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{2}\,\partial x_{n}}}\\\\\vdots &\vdots &\ddots &\vdots \\\\{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{1}}}&{\frac {\partial ^{2}f}{\partial x_{n}\,\partial x_{2}}}&\cdots &{\frac {\partial ^{2}f}{\partial x_{n}^{2}}}\end{bmatrix}}_{x_{0}}\,$$

負定。

拓展閱讀

• 雅可比矩陣
• 梯度