Jean Ritchie noted in her book Folk Songs of the Southern Appalachians:

Older menfolk in the Ritchie family, rather than the women for some unknown reason, sang this song: Balis, my Dad, and his cousins Jason and Isom Ritchie. In the sixties, banjo players were singing a more modern version, Handsome Molly. People ask us about “a bally boat” named in the last verse. Some guess it means “barley” boat, but my guess is that “bally” is a Celtic word meaning handsome or pretty, in other words, “a fine, pretty boat”. At any rate, Loving Hannah is a fine, pretty song, and one of my enduring favourites.

But see also the thread Origins: Loving Hannah - Bawley Boat?