The South Pole is significantly colder than the North Pole due to its higher elevation and landmass, which is less influenced by ocean currents and temperatures.