My interpretation has always been that Jimmy Crack is the whip wielding overseer and 'corn'
is a mishearing of 'gone'
Jimmy Crack is gone and I don't care, the masters gone away.