While he was stationed at Fort Monroe, he married Mary Anna Randolph Custis (1808–1873), great-granddaughter of Martha Washington by her first husband Daniel Parke Custis, and step-great-granddaughter of George Washington, the first president of the United States. They were married on June 30, 1831 at Arlington House, her parents' house just across from Washington, D.C. The 3rd U.S. Artillery served as honor guard at the marriage. They eventually had seven children, three boys and four girls:
The Lees married in the family parlor at Arlington House June 30, 1831. Their marriage produced seven children, six of them born in the dressing room adjoining the Lee's bedroom, according to family tradition. From all appearances, the Lees had a warm and loving family life.