There are a lot of useful functions that have no closed form solution, meaning we can't just do a computation and return the value. Instead, we need to use an iterative method to approximate the function value. We can use this to approximate sine (with Taylor series expansion), approximate square root (as we'll do in this lecture), or optimize a cost or error function (gradient descent in next lecture).
As with the previous uniform random variable lecture, we must translate a recurrence relation to Python. Instead of returning a single value in the recurrence series, we will look for **convergence of the series**. In other words, if we run the series out far enough, $x_{i+1}$ will be close to $x_i$ leaving $x_i$ as a very accurate approximation of square root. This will teach us the basics of iterative computing and prepare us for the more complicated function optimization material.
To approximate square root, the idea is to pick an initial estimate, $x_0$, and then iterate with better and better estimates, $x_i$, using the ([Babylonian method](https://en.wikipedia.org/wiki/Methods_of_computing_square_roots#Babylonian_method)) recurrence relation:
$x_{i+1} = \frac{1}{2}(x_i + \frac{n}{x_i})$
```
x{i+1} = 1/2 (xi + n/xi)
```
There’s a great deal on the web you can read to learn more about why this process works but it relies on the average (midpoint) of $x_i$ and $n/x_i$ getting us closer to the square root of n. The cool thing is that the iteration converges quickly.
Our goal is to write a function that takes a single number and returns it square root. What do we know about this function before even beginning to code? Well, we have a clear description of the problem per our function workplan, and we also have the function signature we want:
Because we are implementing a recurrence relation, we know that we will have a loop that computes $x_{i+1}$ from $x_{i}$.
### Convergence
因为我们正在实现递归关系，所以我们知道我们将有一个循环，从`xi`计算`x{i + 1}`。
The terminating condition of the loop is when we have reached convergence or close to it. Convergence just means that $x_{i+1}$ is pretty close to $x_i$. Because we can never compare to real numbers for equality, we have to check for the difference being smaller than some precision like 0.00000001.
Just as we have an outline for an analytics program, iterative methods all share the same basic outline. (I'm assuming here that $x_{i+1}$ depends only on a single previous value and that $i$ implicitly increments as the loop goes around.)
That is a fairly straightforward implementation of the recurrence relation, but you will notice that we don't actually need to keep all previous $x_i$ around except for the new value and the previous value. Here is a Python implementation that tracks only two values and follows the infinite loop pattern:
As you can see we can define a function within a function. It's not special in any way except that code outside of `test_sqrt()` cannot see function `check()`. On the other hand, `check()`**can** see the symbols outside of `test_sqrt()`, such as our `sqrt()`.
Type in (don't cut/paste) the `sqrt(n)` function and test with, for example, `sqrt(125348.0)`. Make sure you get the right answer (354.045195) and then add print statements so that you can see the sequence of $x_{i}$ values. I get:
Now that we know how to implement a recurrence relation that converges, let's take a look at function optimization. At first glance, it seems completely different, but uses the same abstraction of an iterative method.
