PD / Wikimedia
A mathematical technique of Isaac Newton widely used has been updated and can finally be applied to infinitely complex problems.
300 years ago, Isaac Newton It developed a very simple algorithm to find the minimum value of a function.
Centuries later, Newton’s method remains crucial for solving current problems in logistics, finance, computing.
Although extremely powerful, also It has a significant failure. As it points to, it does not work with all functions.
For this reason, mathematicians have continued to study the technique, discovering different ways to widen their scope without sacrificing efficiency.
A study recently in arXiv announced the last improvement of the Newton method.
Amir Ali Ahmadifrom the University of Princeton, along with its students ABRAAR CHAUDHRY E Jeffrey Zhangthey widened Newton’s method to work efficiently in the widest class of functions so far.
Method with centuries of existence
The functions can have dozens of high variables to high powers, challenging formula analysis. The graphs of their solutions form large landscapes that are impossible to explore from aerial view.
In the 1680s, Newton acknowledged that even when we are dealing with a very complicated function, we always have access to at least two information that helps us find its deepest valley.
It is possible to calculate the so -called first derivative of the function, or slope: the inclination of the function at a certain point. Then we can calculate the rate to which the slope is changing (the second derivative of the function).
Newton proved that if we repeat the process continuously, we eventually reached the minimum value of the more complicated original function. The method does not always work, especially if we start too far from the true minimum. But most of the time it works.
Newton’s method converges to the true minimum much faster: to a “quadratic” rate.
Newton could have written his method to converge to the true minimum value even faster if, instead of taking only the first and second derivatives at each point, he had also taken the third and fourth derivatives. This would give him approximations of more complicated Taylor, with exponents greater than 2.
But the goal of your strategy was to transform a complicated function into a simpler. These more complicated Taylor equations were more than Newton could handle mathematically.
Challenge: Extend Newton’s method
In the following centuries, mathematicians endeavored to widen Newton’s method to investigate the amount of information that can extract from the most complicated Taylor’s approaches from his duties.
As reported to Quanta Magazine, in 2021, Yurii Nesterov It demonstrated how to efficiently bring functions of any number of variables with cubic equations. But its method could not be extended to bring functions closer using quarter, cheese, etc., without losing its efficiency. However, the proof was a great advance in this domain.
Now the new study takes Nesterv’s theory further. The algorithm works for any number of variables and an arbitrary number of derivatives. In addition, it remains efficient in all these cases-something that so far was not possible.
As?
There is no quick and general method to find the minimum functions of high exponents to high exponents. This has always been the main limitation of Newton’s method.
But, as Quanta Magazine writes, there are certain types of functions that have characteristics that make them easy to minimize.
In the new study, Ahmadi, Chaudhry and Zhang prove that It is always possible to find approximation equations that have these characteristics.
What properties make an equation easy to minimize? Two things.
The first is that the equation must be in the form of a cup, or “Convex”. Instead of having many valleys, there is only one (which means that when we try to minimize it, we don’t have to worry about confusing an arbitrary valley with the lowest valley). The second is that the equation can be written as a sum of squares.
Ahmadi, Chaudhry and Zhang discovered how to use a technique called “Semi -defined programming” that fulfills the requirements; and managed to create a more powerful version of the Newton method – capable of reaching the true minimum value of a function in less iterations than previous techniques.
If, over time, the underlying computational technology needed to perform the Newton method becomes more efficient, then the algorithm developed by Ahmadi, Chaudhry and Zhang could eventually exceed the gradient descent. for all types of applications, including automatic learning.
“Our algorithm is, right now, proven faster, in theory,” said Ahmadi. Hope within 10 to 20 years Also in practice.
