No, the jacket had nothing to do with the narrator’s bad years. (i read this story btw) The narrator doesn’t like the jacket and thinks as if it makes them look poor. Because of this, the narrator blames the jacket for everything. The speaker could have appreciated what they had more those years.