I learned something new today. Andrew Jackson was the only President of the United States to have no deficit during his reign. Jackson, who served from 1829-37, was an ardent supporter of keeping banks and government separated. He believed that banks would have undue influence on government, and he was hell-bent on eliminating the debt.
Jackson oversaw the downscaling of the debt to zero in 1835, the plans were in place before he arrived, but zeroed during his term. Also, he used the proceeds from the closing of the Second Bank of the United States to speed the process. The debt remained close to zero the remainder of his term.