We have looked at the method behind steepest descent and explored two ways of calculating our step size. In this post we will look at the limitations and the best use practices for the method of steepest descent.
Overall the method of steepest descent is a reliable and standard test method. However, the sacrifice is speed. When the functions get more complicated and the evaluations more expensive the method of steepest descent begins to lag behind other methods. This tradeoff is an important one, as the desire to reach a minimum is counteracted by the waiting time.The method is steepest descent is generally used when a minimum must be found. This search method would not be appropriate for functions which need to be optimized quickly and roughly. This also brings us to our step finding method. Between the step finding methods we can drastically change the behavior of our search method. It also may require us to take derivatives of our function (which may or may not be applicable at our test points). A standard step size might be useful to minimize derivative evaluations, but at the cost of time. Nevertheless, the method of steepest is a reliable and consistent method. It is necessary to note that for any applicable function, the method of steepest descent will find a minimum. This guarentee must be noted when evaluating proper search methods, and makes steepest descent necessary for anyone’s optimization quiver.-Marcello