It's always T-shirt weather here.

The opposite of T-shirt weather is more of a phrase "wrap up warm".

Make sure you wrap up warm when you go out.