BoydC9pt4

---

Self-concordant

Definition and examples

Def 6.1.

• convex \(f:\mathbb{R}\to\mathbb{R}\) is self-concordant (s.c.) if \(|f^{\prime\prime\prime}(x)|< 2 |f^{\prime\prime}(x)|^{\frac{3}{2}}\) for \(x\in\mathbf{dom}~f\).
• \(f:\mathbb{R}^n\to\mathbb{R}\) is s.c. if \(g(t)=f(x+tv)\) is s.c. for \(x\in\mathbf{dom}~f\), \(v\in\mathbb{R}^n\).

Ex 6.1.

for \(f:\mathbb{R}\to\mathbb{R}\) ,

1. linear and quadratic functions
2. \(f(x)=-\log x\)
3. \(f(x)=x\log x-\log x\)
4. affine invariance: if \(f:\mathbb{R}\to\mathbb{R}\) is s.c., then \(\tilde{f}(x)=f(ax+b)\) is s.c.

Self-concordant calculus

Prop 6.1. properties of s.c.

• preserved under positive scaling \(\alpha\geq 1\) and sum.
• \(f:\mathbb{R}^n\to\mathbb{R}\) is s.c. \(\implies\) \(\tilde{f}(x)=f(Ax+b)\) is s.c.
• if \(g\) is convex with \(\mathbf{dom}~g=\mathbb{R}_{++}\) and \(|g^{\prime\prime\prime}(x)|< 3g^{\prime\prime}(x)/x\), then
  \[f(x) = \log(-g(x))-\log x \]
  is s.c.

Ex 6.2.

• \(f(x) = -\sum_{i=1}^m \log(b_i - a_i^T x) \text{ on } \{x \mid a_i^T x < b_i, \; i = 1, \ldots, m\}\)
• \(f(X) = -\log \det X \text{ on } \mathbb{S}^n_{++}\)
• \(f(x) = -\log(y^2 - x^T x) \text{ on } \{(x, y) \mid \|x\|_2 < y\}\)

Properties of self-concordant functions

Notation :

\[\lambda(x) := \left( (\nabla f(x))^T \nabla^2 f(x)^{-1} \nabla f(x) \right)^{1/2} \]

\(i.e.\) \(\lambda(x)=\|\nabla f(x)\|_{\nabla^2 f(x)^{-1}}\) ,
also

\[\lambda(x) = \sup_{v \neq 0} \frac{-v^T \nabla f(x)}{(v^T \nabla^2 f(x) v)^{1/2}} = \sup_{\|v\|_2=1} \frac{-v^T \nabla f(x)}{(v^T \nabla^2 f(x) v)^{1/2}} \]

which points out that

\[\lambda(x) \geq \frac{-v^T \nabla f(x)}{(v^T \nabla^2 f(x) v)^{1/2}} ~, \quad v \neq 0 \]

Prop 6.2. Upper and lower bounds on second derivatives : \(f:\mathbb{R}\to\mathbb{R}\) is a strictly convex self-concordant function, then

\[\left| \frac{d}{dt} \left( f^{\prime\prime}(t)^{-1/2} \right) \right| \leq 1 \]

thus

\[-t \leq \int_{0}^{t} \frac{d}{d\tau} \left( f^{\prime\prime}(\tau)^{-1/2} \right) d\tau \leq t \]

\(\implies\) \(-t \leq f^{\prime\prime}(t)^{-1/2} - f^{\prime\prime}(0)^{-1/2} \leq t\), then

\[\frac{f''(0)}{(1 + t f''(0)^{1/2})^2} \leq f''(t) \leq \frac{f''(0)}{(1 - t f''(0)^{1/2})^2} \]

Prop 6.3. Bound on suboptimality : \(f:\mathbb{R}\to\mathbb{R}^n\) is a strictly convex self-concordant function, then

\[\lambda (x)^2 \geq f(x) - p^* \]

Analysis of Newton’s method

Similar to the analysis in previous part, we can prove that there exist constants \(\eta\in(0,1/4],\gamma>0\) \(s.t.\)

1. if \(\lambda(x)>\eta\), then
   \[f(x^{(k+1)}) - f(x^{(k)}) \leq -\gamma \]

2. if \(\lambda(x)\leq\eta\), then
   \[2\lambda(x^{(k+1)}) \leq \left(2\lambda(x^{(k)})\right)^2 \]
   where \(\eta,\gamma\) only depend on backtracking parameters.