Albert Haller Tracy was born in Norwich, Connecticut, on June 17, 1793. Albert was one of six children born to Abigail Trott and Philemon Tracy.

Albert began studying medicine, following in his father’s footsteps, but ultimately decided to pursue the study of law. In 1811, Albert moved to New York State. He was admitted to the bar in 1815 and began practicing in Buffalo the same year. From 1819 until 1825, Albert served as a New York representative in the House of Congress for three terms. When first elected to Congress, Albert was twenty-four years of age. He turned twenty-five before Congress convened in December 1819. Albert aligned with the Anti-Masons and, later, the Whig party. He worked closely with like-minded politicians such as William H. Seward, Thurlow Weed, Millard Filmore, and John C. Spencer. Albert served in the state senate for eight years (1830-1837). In 1839, he ran as a Whig candidate for the U.S. Senate. After an unsuccessful campaign, Albert did not pursue further appointments and retired from public life. In 1833-34, there was a significant falling out between Albert and William H. Seward. Albert made inappropriate advances on Frances Seward. William caught on and confronted Albert in a letter. William recounted this exchange for Frances in a letter addressed to her on December 29, 1834.   

On November 23, 1825, Albert married Harriet Foote Norton. Together, they had two children, Albert Haller (born in 1834) and Francis Walsingham Tracy (born in 1839).

Albert died on September 19, 1859, in Buffalo, New York, at the age of 66.