A source of information randomly generates symbols from a four letter alphabet {w, x, y, z }. The probability of each symbol is as follows: P(w) = 1 2 ; P(x) = 1 4 ; P(y) = 1 8 ; and P(z) = 1 8 . These symbols are now encoded into binary codes using the scheme shown below. Let the random variable L denote the length of the binary code and pL(l) denote the PMF of L.symbol codew 0x 10y 110z 111Find the expectation and variance of L.

where [tex]p_\alpha[/tex] is the probability of picking the letter [tex]\alpha[/tex], and [tex]\ell_\alpha[/tex] is the length of code needed to encode [tex]\alpha[/tex]. [tex]p_\alpha[/tex] is given to us, and we have

[tex]\begin{cases}\ell_w=1\\\ell_x=2\\\ell_y=\ell_z=3\end{cases}[/tex]

so that we expect a contribution of

[tex]\dfrac12+\dfrac24+\dfrac{2\cdot3}8=\dfrac{11}8=1.375[/tex]

bits to the code per encoded letter. For a string of length [tex]n[/tex], we would then expect [tex]E[L]=1.375n[/tex].

By definition of variance, we have

[tex]\mathrm{Var}[L]=E\left[(L-E[L])^2\right]=E[L^2]-E[L]^2[/tex]

For a string consisting of one letter, we have

[tex]\displaystyle\sum_{\alpha\in\{w,x,y,z\}}p_\alpha{\ell_\alpha}^2=\dfrac12+\dfrac{2^2}4+\dfrac{2\cdot3^2}8=\dfrac{15}4[/tex]

so that the variance for the length such a string is

[tex]\dfrac{15}4-\left(\dfrac{11}8\right)^2=\dfrac{119}{64}\approx1.859[/tex]

"squared" bits per encoded letter. For a string of length [tex]n[/tex], we would get [tex]\mathrm{Var}[L]=1.859n[/tex].