This thesis examines the contextual modulations at both the low and high stages of the visual processing hierarchy. Holistic processing, a high-level contextual modulation mechanism, is a hallmark of face processing, and is proposed to mark the recruitment of face-specialized regions. It can be easily disrupted by inversion, an image manipulation that largely preserves image properties processed at primary stages of vision. The preservation of low-level properties with inversion directed the traditional focus to high-level mechanisms to account for face holistic processing. Due to this focus, holistic processing has been considered independent from low-level mechanisms and its connection with the early regions was not adequately explored. In the present thesis, we investigated the link of high-level contextual modulations to low-level regions through behavioral and neuroimaging methods. Contextual modulations at both of these distinct stages demonstrate a reliance on local input intensity. This shared fundamental principle provided a refined lens for our combined studies. Inversion reduces the strength of face contextual modulations, decreases the response of face-specific brain regions, and leads to brain activation patterns more similar to non-face object categories. This makes inversion a valuable tool for exploring contextual modulations in advanced, non-specialized processing stages. Therefore, by testing inverted faces alongside upright faces, we were able to investigate the interactions between low level and non-specialized, high-level contextual mechanisms. The interactions between these mechanisms were investigated through an inter-individual differences framework. High-level, face-specialized contextual mechanisms seem to be working independently of the early regions. However, non face-specialized high-level contextual mechanisms seem to be influenced by the constraints presented by lower-levels of the processing hierarchy. These findings pose new questions in the literature such as whether contextual modulations at low-levels would influence the contextual modulations of other complex images that are not handled by dedicated mechanisms, and further adds to the large literature on whether faces are truly processed in a special manner or not.