Margaret (Brisbane) Cary was born in 1788 to John and Elizabeth (Boyd) Brisbane in Philadelphia, Pennsylvania. Her father was an agent of the Holland Land Company in western New York and became wealthy from his associated real estate dealings. In 1802, John Brisbane was appointed postmaster of Batavia, New York. In June 1817, Margaret married Trumbull Cary, another agent of the Holland Land Company. Margaret had four children during her lifetime; only one, Walter Cary, survived to adulthood. Margaret and Frances Seward became close friends while their husbands served as New York senators. Frances Seward recounted attending to the illness and death of Margaret's son, Trumbull or "Trummy," in a letter to her sister, Lazette Worden, dated April 5, 1832. Margaret died June 22, 1863, in Batavia, New York.