They argues because Mary was working in the garden when Colin told her to come see him, but she didn't; she kept on working in the garden. She went to see him later and he was upset that see didn't come and it broke out into a fight of who was selfish.