All we know is that Pepys' uncle, John Perkin, cadged a fiddle for his son Frank whose mill had blown down, maybe recently. Pepys played the violin - he came fron a musical family. 'Old fiddle' suggests John knew that Pepys had or might have a spare. Perhaps John didn't word his request well if it vexed Pepys.

This_book relates a subsequent meeting between Pepys and Frank Perkin.

The comedown between being a miller and becoming an itinerant musician seems stark. Perhaps he was just a labourer in someone else's mill, and the owner couldn't affort to reconstruct it.

The comment about 'country girls' may have just been Pepys being dismissive. Or perhaps he knew Frank's character. I don't know how old Frank was in 1661 but he appears to have a wife and children two years later.