Answer:
x = √(a(a+b))
Step-by-step explanation:
We can also assume a > 0 and b > 0 without loss of generality. (If a and a+b have opposite signs, the maximum angle is 180° at x=0.)
We choose to define tan(α) = -(b+a)/x and tan(β) = -a/x. Then the tangent of ∠APB is ...
tan(∠APB) = (tan(α) -tan(β))/(1 +tan(α)tan(β))
= ((-(a+b)/x) -(-a/x))/(1 +(-(a+b)/x)(-a/x))
= (-bx)/(x^2 +ab +a^2)
This will be maximized when its derivative is zero.
d(tan(∠APB))/dx = ((x^2 +ab +a^2)(-b) -(-bx)(2x))/(x^2 +ab +a^2)^2
The derivative will be zero when the numerator is zero, so we want ...
bx^2 -ab^2 -a^2b = 0
b(x^2 -(a(a+b))) = 0
This has solutions ...
b = 0
x = √(a(a+b))
The former case is the degenerate case where ∠APB is 0, and the value of x can be anything.
The latter case is the one of interest:
x = √(a(a+b)) . . . . . . the geometric mean of A and B rotated to the x-axis.
_____
Comment on the result
This result is validated by experiments using a geometry program. The location of P can be constructed in a few simple steps: Construct a semicircle through the origin and B. Find the intersection point of that semicircle with a line through A parallel to the x-axis. The distance from the origin to that intersection point is x.
