The lines in the poem "The Raven" by Edgar Allan Poe that clearly depict the speaker's grief over the loss of Lenore and his frustration with the raven are the following:
"Leave my loneliness unbroken! quit the bust above my door!
Take thy beak from out my heart, and take thy form from off my door!"
He says that he is lonely (because his love Lenore is dead) and he wants the bird to go away and not remind him of her death.