Parameterize the surface (I'll call it S) by
r(u, v) = (1 - u) (1 - v) i + u (1 - v) j + v k
with 0 ≤ u ≤ 1 and 0 ≤ v ≤ 1.
Take the normal vector to this surface to be
n = ∂r/∂u × ∂r/∂v = ((v - 1) i + (1 - v) j) × ((u - 1) i - u j + k) = (1 - v) (i + j + k)
with magnitude
||n|| = √3 (1 - v)
Then in the integral, we have
[tex]\displaystyle\iint_SG(x,y,z)\,\mathrm ds = \int_0^1\int_0^1 G((1-u)(1-v),u(1-v),v) \|\mathbf n\| \,\mathrm du\,\mathrm dv \\\\= \sqrt3 \int_0^1\int_0^1uv(1-v)^2\,\mathrm du\,\mathrm dv \\\\= \boxed{\frac1{8\sqrt3}}[/tex]
Alternatively, if you're not familiar with parameterizing surfaces, you can use the "projection" formula:
[tex]\displaystyle\iint_S G(x,y,z)\,\mathrm ds = \int_{S_{xy}}G(x,y,z)\sqrt{1+\left(\frac{\partial f}{\partial x}\right)^2+\left(\frac{\partial f}{\partial y}\right)^2}\,\mathrm dx\,\mathrm dy[/tex]
where I write [tex]S_{xy}[/tex] to mean the projection of the surface onto the (x, y)-plane, and z = f(x, y). We would then use
x + y + z = 1 ==> z = f(x, y) = 1 - x - y
and [tex]S_{xy}[/tex] is the triangle,
{(x, y) : 0 ≤ x ≤ 1, 0 ≤ y ≤ 1 - x}
Then the integral becomes
[tex]\displaystyle\int_0^1\int_0^{1-x}y(1-x-y)\sqrt{1+(-1)^2+(-1)^2}\,\mathrm dy\,\mathrm dx \\\\= \sqrt3\int_0^1\int_0^{1-x} y(1-x-y)\,\mathrm dy\,\mathrm dx \\\\= \frac{\sqrt3}{24} \\\\= \boxed{\frac1{8\sqrt3}}[/tex]
