Volume of the pyramid:
[tex]V=\dfrac{s^2h}3[/tex]
Perimeter of the cross-section:
[tex]40=\sqrt2\,s+2\sqrt{\dfrac{s^2}2+h^2}=\sqrt2\left(s+\sqrt{s^2+2h^2}\right)[/tex]
[tex]\implies h=\sqrt{\dfrac{(20\sqrt2-s)^2-s^2}2}=\sqrt{400-20\sqrt2\,s}[/tex]
Area of the cross-section:
[tex]P=\dfrac12(\sqrt2\,s)h=\dfrac{sh}{\sqrt2}[/tex]
[tex]\implies P=\dfrac{s\sqrt{400-20\sqrt2\,s}}{\sqrt2}=s\sqrt{200-10\sqrt2\,s}[/tex]
First derivative test:
[tex]\dfrac{\mathrm dP}{\mathrm ds}=\dfrac{20\sqrt2-3s}{\sqrt{4-\dfrac{\sqrt2}5s}}=0\implies s=\dfrac{20\sqrt2}3[/tex]
Then the height of the cross-section/pyramid is
[tex]h=\sqrt{400-20\sqrt2\,s}=\dfrac{20}{\sqrt3}[/tex]
The volume of the pyramid that maximizes the cross-sectional area [tex]P[/tex] is
[tex]V=\dfrac{\left(\frac{20\sqrt2}3\right)^2\frac{20}{\sqrt3}}3=\dfrac{16000}{27\sqrt3}[/tex]
