I see your thinking and I have given the point some (not that much) thought.
Being invisible would have all sorts of advantages such as
1) Invading the privacy of attractive females
2) Running on the pitch and kicking Scott Brown, Fat Boruc and their like square in the nuts without detection
3) you get the picture
however the following questions arise
1) When invading the privacy of such attractive females, how could we seem them?, how would we know they are attractive, even female or actually their!
2) When running on the pitch, how do we know who we are actually kicking square in the stones, or if we are kicking anyone square in the stones or if they are actually there!
3) We would always bump into each other, unless humans could somehow see each other in which case we wouldnt be invisible so we couldnt really do any of the fun stuff!
Now, I could have debated this with myself for much longer and Im sure you will be able to add some further positives/negatives but I still liked the original point.