Baldwin served as one of four Georgia delegates to the constitutional convention of 1787. Of the Georgia delegates, only he and William Few signed the constitution. At the convention, Baldwin served on the Committee of Postponed Matters, later called the "Grand Committee," which was tasked with determining how to apportion state representation in the national legislature. On the committee, Baldwin changed his vote on small-state representation to the national Senate, possibly because he feared the alienation of small states after befriending the Connecticut delegation. In so doing, he was instrumental in bringing about the "Great Compromise" that established representation in each house of Congress.