I have had a Zoom H2 since March 2008 and have found it invaluable. Have recorded practice/sessions/concerts/my guitar playing/even a meeting with HR when I was working (LOL)

It sits about two foot away from me most of the time and is so simple to turn on and capture an idea. And it is always there when I play and practice with people so we can see whether things work or not.

Can record as mp3 (or 4 channel WAV at the other extreme). It has a Phones/line out so you could play stuff straight out. I tend to dump it into Reaper and process it a bit if I want to keep it. Rechargeable batteries (or plug into mains) mean cheap to run

Example of output from practice - that would be sticking the Zoom on the floor somewhere in the middle of us and pushing 'record'

Second hand you'd pick one up on ebay between £30 and £50 judging from Sold listings. There are newer versions but I have never needed to change it. If it broke I would immediately buy another.