Linen sets

I just tried on a matching linen set. It was so comfortable and can imagine it wearing so well in that mid 20s weather. However, I had this niggling thought that it reminded me of pyjamas. Am I being dramatic and overthinking (not unusual 😅). Maybe it’s because I grew up on matching flannelette pjs? Whatever the reason, keen to hear others opinions