The words Masato gave you are by Joseph Hart, 1759 used for "The Grieved Soul" in the Sacred Harp. That tune is from 1859, so might not be completely appropriate.

Rural Tennessee in the time period you're dealing with used shape note music. There was even a book called Tennessee Harmony published in 1818. If you can track that down somehow, you'll have all the hymns you want.

Here are some folk hymns form the Southern Harmony that are probably good for the period you want. I've put in the date first published, if known. The melody for Southern Harmony is on middle staff if 3 parts & the tenor if 4.
Jefferson Published in Tennessee Harmony 1818.
Olney Published in Tennessee Harmony 1818.
Mear 1720, an 18th-19th cent. hymn standard
Ninety-Third Psalm 1812
Primrose 1812
Tribulation 1813
Detroit 1820
China 1801.
This is becoming a bigger project than I thought. The way I found these was to check the Chronological Index of the Sacred Harp beginning about 1811 then check the Southern Harmony to see if you can get the music there. I looked at 1810's especially because beginning then the tunes take on a definite folk tune feel. The earlier tunes I included are there because they are ones I know were almost ubiquitous at the time.