Dillinger returned to Mooresville where he met Beryl Ethel Hovious (born August 6, 1906), the daughter of Stephen Hovious and Cara Vandeventer. The two were married in Martinsville on April 12, 1924. He attempted to settle down, but he had difficulty holding a job and preserving his marriage. The marriage ended in divorce on June 20, 1929. Beryl Dillinger remarried in July, 1929 to Harold McGowen, with the pair divorcing in July, 1931. In 1932, she married a third time to Charles Byrum and they had one child. Beryl Hovious Byrum died November 30, 1993 at Millers Merry Manor, Mooresville, Indiana and is buried at Mt. Pleasent Cemetery, Hall, Indiana.