Because that envelope is made up of fat (lipid) molecules that are very sensitive to environmental conditions. The envelope comes from the cell that the virus erupts out of, not from the virus. Naked virions do not have this lipid covering but are surrounded by proteins which are not as sensitive to the environment.
We actually don't need clothes; we used to have a coat of fur just like every other mammal.