International Scholarly Research Notices

Research Article

Gradient-Type Methods: A Unified Perspective in Computer Science and Numerical Analysis

Algorithm 1

(a) Given { 𝛾 𝑘 } , { 𝜎 𝑘 } , 𝑘 = 0 , 1 … s.t.
i n f { 𝛾 𝑘 } > 0 , i n f { 𝜎 𝑘 } > 0
Let 𝐱 ( 0 ) ∈ 𝑅 𝑛 be a starting point
For a given vector 𝐬 ( 𝑘 ) ∈ 𝐷 ( 𝛾 𝑘 , 𝐱 ( 𝑘 ) ) , set
𝐱 ( 𝑘 + 1 ) = 𝐱 ( 𝑘 ) − 𝜆 𝑘 𝐬 ( 𝑘 ) , with 𝜆 𝑘 ∈ ( 0 , 𝜎 𝑘 ‖ ∇ 𝑓 ( 𝐱 ( 𝑘 ) ) ‖ ) :
𝑓 ( 𝐱 ( 𝑘 + 1 ) ) ≈ m i n 𝜇 { 𝑓 ( 𝐱 ( 𝑘 ) − 𝜇 𝐬 ( 𝑘 ) ) , 0 < 𝜇 ≤ 𝜎 𝑘 ‖ ∇ 𝑓 ( 𝐱 ( 𝑘 ) ) ‖ }