If you have a room air conditioner, remove it for the winter or seal it up and insulate it. Water heater insulation jackets can work well for covering and insulating the air conditioner.