Agriculturalists tend to live in permanent villages that are larger and closer to other settlements than the semipermanent settlements of horticulturalists.
As the agriculture's are used to villages life and have to take care of the larger tracks of the lands and they thus live close by as compared to the semi-permanent horticulturalists.
As horticulture is not a permanent type of occupation and they don't need to bee close to the farms and required less maintenance.