Since the sphere and Matyas functions contain only one global minimum, Gradient descent is effectively used to find that minimum at f(0,0) = 0.
The Matyas function also has the exact solution at f(0,0) = 0.
However for Rastringen's function, Gradient Descent was trapped by local minima as it can't escape from the closest local pit. this clearly shows the restriction of the method.
Most engineering design problems involve a complicated search landscape, i.e., irregular constraints. Since the Gradient Descent method utilizes function gradient to determine the search direction, it still can reach the minimum point even when obstructed by constraint for a particular path.
This might not be possible for the discontinuous objective function, and the solution might fail to reach such minimum point. Stochastic optimization algorithms can overcome such deficiencies and are commonly used for most practical engineering design problems.
