Let
[tex]S_n = \displaystyle \sum_{k=0}^n r^k = 1 + r + r^2 + \cdots + r^n[/tex]
where we assume |r| < 1. Multiplying on both sides by r gives
[tex]r S_n = \displaystyle \sum_{k=0}^n r^{k+1} = r + r^2 + r^3 + \cdots + r^{n+1}[/tex]
and subtracting this from [tex]S_n[/tex] gives
[tex](1 - r) S_n = 1 - r^{n+1} \implies S_n = \dfrac{1 - r^{n+1}}{1 - r}[/tex]
As n → ∞, the exponential term will converge to 0, and the partial sums [tex]S_n[/tex] will converge to
[tex]\displaystyle \lim_{n\to\infty} S_n = \dfrac1{1-r}[/tex]
Now, we're given
[tex]a + ar + ar^2 + \cdots = 15 \implies 1 + r + r^2 + \cdots = \dfrac{15}a[/tex]
[tex]a^2 + a^2r^2 + a^2r^4 + \cdots = 150 \implies 1 + r^2 + r^4 + \cdots = \dfrac{150}{a^2}[/tex]
We must have |r| < 1 since both sums converge, so
[tex]\dfrac{15}a = \dfrac1{1-r}[/tex]
[tex]\dfrac{150}{a^2} = \dfrac1{1-r^2}[/tex]
Solving for r by substitution, we have
[tex]\dfrac{15}a = \dfrac1{1-r} \implies a = 15(1-r)[/tex]
[tex]\dfrac{150}{225(1-r)^2} = \dfrac1{1-r^2}[/tex]
Recalling the difference of squares identity, we have
[tex]\dfrac2{3(1-r)^2} = \dfrac1{(1-r)(1+r)}[/tex]
We've already confirmed r ≠ 1, so we can simplify this to
[tex]\dfrac2{3(1-r)} = \dfrac1{1+r} \implies \dfrac{1-r}{1+r} = \dfrac23 \implies r = \dfrac15[/tex]
It follows that
[tex]\dfrac a{1-r} = \dfrac a{1-\frac15} = 15 \implies a = 12[/tex]
and so the sum we want is
[tex]ar^3 + ar^4 + ar^6 + \cdots = 15 - a - ar - ar^2 = \boxed{\dfrac3{25}}[/tex]
which doesn't appear to be either of the given answer choices. Are you sure there isn't a typo somewhere?
