If $a,\,b$ and $c$ are the lengths of the sides of a right triangle with hypotenuse $c$, prove $\displaystyle \frac{(c − a)(c − b)}{(c + a)(c + b)}\le 17 − 12\sqrt{2}$.

Show that if $a,\,b$ and $c$ are the lengths of the sides of a right triangle with hypotenuse $c$, then
(a) $\displaystyle \frac{c}{\sqrt{ab}}\ge \sqrt{2}$
(b) $\displaystyle \frac{(c − a)(c − b)}{(c + a)(c + b)}\le 17 − 12\sqrt{2}$.

My solution:

(a)

$\begin{align*}c^2&=a²+b²\\&≥2ab \,\,\,\text{(by the AM-GM inequality)}\end{align*}$

$\dfrac{c^2}{ab}\ge 2$
Taking the square root on both sides completes the proof, i.e. $\dfrac{c}{\sqrt{ab}}\ge \sqrt{2}$, equality occurs when $a=b$.

(b)

We have $c^2−a^2=b^2 \implies c−a=\dfrac{b^2}{c+a}$.

By the similar token, we also have $c−b=\dfrac{a^2}{c+b}$, if we're going to replace these two into the original LHS of the inequality, we get:

$\displaystyle \begin{align*}\frac{(c − a)(c − b)}{(c + a)(c + b)}&=\frac{(b^2)(a^2)}{(c + a)^2(c + b)^2}\\&=\left(\frac{ab}{(c + a)(c + b)}\right)^2\\&=\left(\frac{ab}{c^2+c(a+b)+ab}\right)^2\\&=\left(\frac{1}{\dfrac{c^2}{ab}+\dfrac{c(a+b)}{ab}+1}\right)^2\\& \le \left(\frac{1}{\dfrac{c^2}{ab}+\dfrac{c(2\sqrt{ab})}{ab}+1}\right)^2\,\,\text{(by the AM-GM inequality)}\\& = \left(\frac{1}{\left(\dfrac{c}{\sqrt{ab}}\right)^2+\dfrac{2c}{\sqrt{ab}}+1}\right)^2\\& \le \left(\frac{1}{\left(\sqrt{2}\right)^2+2\sqrt{2}+1}\right)^2\\&= \left(3-2\sqrt{2}\right)^2\\&= 17-12\sqrt{2}\,\,\,\text{Q.E.D.}\end{align*}$

Equality occurs when $a=b$.