Neither of us could tell the difference between the lossless file and 320kbps mp3, except on Classical music.
You won't notice it as much on recordings that don't feature acoustic instruments, or recordings that aren't well mastered. Generally, any recording where you can hear "the room" will sound a hell of a lot better if you have the full spectrum of frequencies being reproduced than say, a Notorious BIG track where all of the signal was generated from line-in sources and there's no acoustic signature to even bother trying to reproduce accurately.
It's really all about what you are looking to get out of it.
I listen to a lot of jazz, and a lot of classical, and I notice the difference in those two genres. Classic rock, especially anything mixed by Eddie Kramer (Led Zeppelin, Jimi Hendrix, the Beatles, the Rolling Stones) sometimes benefits from this, but not always, because a lot of that shit was recorded by idiots.
If you really get into it, you'll spend as much time looking at who engineered a record as who is performing on it.
I've got two of
these on my desk, and they are fine for what I'm looking for, but some guys, like my father, go a lot further with active crossovers and custom printed circuit boards and shit.
For me, that's more than I need, but the difference between lossless file formats and everything else is clear if you know what you are listening for.