Before we start, Lemme tell you that this is the mso useless piece of information you will learn this course.
it is jsut smthing I found while doomscrolling and thought it was cool.
Automatic Differentiation is a technique used by computers to compute derivatives faster.
This technique utilizes smth called dual hypercomplex numbers which are writen in then form \(a + b \epsilon \) where \(a\)
and \(b\) are real numbers and epsilion (\(\epsilon\)) equals:
$$\begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix}$$
These numbers are very similar to imaginary numbers \(i\), execpt that their key propety isn't \(i^2 = -1\), it is \(\epsilon ^2 = 0\)
these numbers are used for automomatic differentiation using this formula:
$$f(x+ \epsilon) = f(x) + \epsilon f'(x)$$
for example:
$$x^2+2x+1$$
$$(x+ \epsilon)^2+2(x+ \epsilon)+1$$
$$x^2 + 2x\epsilon + \epsilon^2 + 2x + 2\epsilon + 1$$
and since \(\epsilon^2 = 0\)
$$x^2 + 2x + 1 + 2x\epsilon + 2\epsilon $$
notice anything?
the first 3 terms are the exact value of \(f(x)\) and the last 2 terms are \(\epsilon (2x + 2)\) which is \(\epsilon f'(x)\)