Material Strength
Regardless of environmental conditions, some materials weather slower than others because of their inherent strength. A very strong mineral like quartz will weather very slowly. Loose, soft material like sandstone is easily broken down chemically and physically. Often, certain rocks are susceptible to one form of weathering but are resistant to another. The molecular structure of granite, for example, makes it very resistant to chemical weathering but susceptible to physical weathering.
Material Structure
Rocks, like shale, that form in layers crack when they reach the earth's surface. Many sedimentary rocks are formed in horizontal planes, as particles settle out of a river or ocean. These planes are often visible on a rock face. Grainy rocks, like sandstone, will crumble after long exposure to the elements. Rocks that have these built-in weaknesses will naturally weather faster than more solid materials. Granite monoliths like the cliffs of Yosemite Valley will retain their form for a long time.
Temperature
Extremes in temperature can cause weathering to proceed at different rates. Very cold conditions tend to slow all forms of weathering. High temperatures generally speed up weathering because the rocks try to expand. Weathering is fastest in places where temperatures move over a large range. When water goes from liquid to solid many times, frost wedging becomes a potent force. In deserts, cold nights and hot days put enormous stresses on rocks' physical structures.
Humidity
A wetter environment can speed both physical- and chemical-weathering processes. Water is frequently the means for chemicals to attack rocks. Water also allows for a greater diversity of lifeforms. In cold environments, water means that frost wedging can push rocks apart. In the tropics, high humidity accelerates plant growth and speeds up the cycling of chemicals through the environment. Weathering in the tropics is as fast as anywhere on earth, thanks to the heat and humidity.
Soil Environment
Certain soils retain moisture well, and provide breeding grounds for micro-organisms. The life processes of certain micro-organisms lead to chemical weathering of nearby rocks. Large plant life can push roots in between rocks, allowing for other forms of weathering to proceed faster. Plants in turn support animal life, which can degrade rocks in a variety of ways. The composition of soil also effects weathering rates. Certain chemicals present in soil can either speed or slow the breakdown of rock.