One of the most important methods in cognitive neuroscience is fMRI. It comes from the early works of Ogawa and colleagues (1990) who first proposed the BOLD signal as a measure of neural activity. Since that moment, great progress has been made in fMRI in terms of hardware, acquisition protocols, and data analysis. At first, researchers were focused on the classification of brain areas responding to particular stimuli or tasks, and they strived to replicate findings from neuroanatomy and neuropsychology. Afterwards, new data analysis concepts and implementations shifted the focus on the content of those task-specific regions, the characterization of the representation of neuronal populations, and the communication between different areas. Mathematics, Statistics, and Computer science have been playing a big role in this revolution providing several methods that enhanced our understanding of the brain structures and functions. They introduced innovative tools that are expanding the possibilities in many field of cognitive sciences. In this essay, I will discuss some of those novel approaches.
Multi-variate pattern analysis
A core concept of Neuroscience is that neurons work as an ensemble to shape our inner and outer world in order to create significant meanings necessary to our survival. We refer to it as neural representation and every single mental state is associated with a different pattern of neural activity. “Representation links cognition to brain activity and enables us to build functional theories of brain information processing” (Kriegeskorte and Kievit, 2013). The variety of different possible activations is almost infinite. And, the representational space is very big and differentiated. Traditional analyses, known as activation-based or univariate analysis, aim to identify which region (or single voxels) becomes active during the execution of a specific task or group of voxels showing effects in the same direction and to infer involvement of the region in a specific mental function. To some extent, this type of analysis is able to decode the content of brain activity and are well suited to study, for instance, ensemble category selectivity in visual cortex (Kamitani and Tong, 2005). One of the main features of univariate analysis is that to increase sensitivity and to find voxels responding to a particular condition, it spatially averages the activity across voxels responding to that condition. This approach enhances the signal-to-noise ratio, yet it causes some loss of information: (i) statistically non-significant active voxels are discarded and they might carry some information; (ii) due to spatial averaging is not possible to look at fine-grained differences between spatial patterns. Furthermore, one other limitation is the fact that with this approach voxels or regions are assumed to be independent. Actually, brain areas are not necessarily discrete and it is important to consider and assess dependency between voxels. Moving from this idea recent analyses of neuronal recordings and functional imaging data have increasingly focused on patterns of activity within a functional region. To address this issue, one innovative tool is multi-variate pattern analysis (MVPA). “Pattern-information analysis aims to detect activity-pattern differences and to infer representational content” (Mur et al. 2009). One of the first prominent studies, exploiting this method, provided evidence that information about category is not only localized within areas with the strongest activation but is widely distributed (and overlapping) across the visual cortex (Haxby, 2001). MVPA does not involve spatial averaging, but it treats each voxel as a distinct source of information about the content of the neuronal code. When no difference in regional-average activity is present, only MVPA can detect fine-grained differences within the region. As shown in a study by Raizada et al. (2008), two sounds of the same length (‘la’ vs ‘ra’) elicit the same amount of activity in primary auditory cortex when analyzed with standard fMRI analysis. However, using multi-variate analysis they were able to probe the information content of the region of interest, and they found that the two sounds are represented by discriminable patterns of activation. Generally, the term MVPA refers to two distinct approaches that exploit different analysis methods: classifier-based and similarity-based MVPA.
To infer the representational content means that there might be a difference within local activation patterns so that we can distinguish which one pertains to one experimental condition or the other. In other words, we must be able to decode conditions from patterns of activity in a population of neurons that share the same neural code. There are plenty of statistical methods suitable for pattern-information analysis aimed to probe whether two patterns are significantly different. However, due to problems related to the assumptions of statistical tests (e.g. the distribution of residuals is assumed to be normal, and this might not be the case with fMRI data), the most used methods come from Computer science, in particular, machine learning field. The problem of assigning patterns to different conditions is approached as a classification problem, and has been referred to as “brain reading” (Cox and Savoy, 2003) or “decoding”. Thanks to classification algorithm is possible to create a correspondence map between experimental conditions and voxel activity patterns. Linear classifier are the most common and successful tool for this application. Many classification algorithms have been proposed (LDA, Carlson et al., 2003; SVM, Cox and Savoy 2003) and the field keeps on growing. Each algorithm is based on different assumptions on fMRI data, and better suited for different experimental design. Still, if applied properly they all provide a valid statistical test for pattern information analysis. The process is generally composed of two steps. First, the classifier learns how to discriminate between experimental conditions from the activity of each voxel and derives a linear decision boundary that separates classes. This is the training stage, in a subset of data, during which weights associated to single voxels are adjusted to maximize how well the boundary separates the conditions. The testing stage is carried out on the remain data where the weights are used to calculate a weighted sum of voxel activity, which is compared against the boundary to guess the class.
Thanks to high-resolution fMRI, and to the method described above, it is today possible to test for more complex predictions and to run very informative exploratory analysis that overcomes limitations of univariate analysis. On one hand, MVPA can be used to investigate representation across widely distributed areas (Haxby et al., 2001). In a prominent paper, Barany and colleagues (2014) used MVPA to disentangle the relationship between areas involved in sensorimotor transformation. In particular, they aimed to find where, in the fronto-parietal network, sensory inputs are transformed into motor commands. This network is made out of quite distributed area and includes parietal regions, premotor cortex and primary motor cortex. To look out for candidates of sensorimotor transformation, authors assumed that suitable loci where those containing both input and output features of the ongoing transformation. Operationally, they investigated which loci reflect interaction between pairs of features, or which one carried information about those pairs. This type of quest is not feasible with neurophysiological measurements (because recordings are circumscribed to specific small regions) or classical fMRI analysis (not sensitive to regional differences). Participants performed a set of wrist movements that differed along several aspects (target location, movement direction, movement amplitude, wrist orientation, and wrist angle). They hypothesized that the activity evoked by each feature’s neural representation would contribute differentially to the classifier output. Results provided evidence indicating superior parietal lobule as a locus in the transformation between target location and movement direction and showing that posture-dependent representations are widespread through the motor system. On the other hand, talking about exploratory analysis is worth to mention the multivariate searchlight approach (Kriegeskorte et al., 2006). This method allows to detect which regions contain information about experimental condition by mapping the entire brain. This is one of its most appealing aspects because it requires no a priori region specification. To build the whole-brain map of the most informative voxels, the idea is to run a series of multivariate “searchlight” analysis throughout the measured volume. A sphere of n-voxels radius is centered on each voxel, and for each of them, multivariate pattern classification is performed.
Eventually, it is important to note some limitations of classifiers. They can be defined as opportunistic since they exploit any discriminative variance. For instance, Todd et al. (2013) found that controlling for a variable (e.g. reaction time) that is confounded with a variable of interest, eliminates the MVPA results. “This raises the question of whether recently reported results truly reflect different representations, or rather the effects of confounds such as reaction time, difficulty, or other variables of no interest” (Todd et al., 2013). Also, MVPA analyses focus on often idiosyncratic patterns of response, they are typically conducted within individual subjects, and only relatively few observations are collected. Data analysis can suffer from the small number of observations per participant relative to the complexity of the data. Considering the representational space of which voxels are dimensions of variation, activity patterns (number of recordings) will fill it sparsely. This makes the estimation of decision boundary very difficult and impairs the classification performance. One way to overcome this issue is the so-called Shared Response Modeling (SRM) (Chen et al., 2015) or hyperalignment method (Haxby, 2011), which create a common low-dimensional space from fMRI responses of each participant. Combining data from multiple subjects into a low-dimensional space, that captures temporal variance shared across participants, improves MVPA both by increasing the number of observations and by aligning fine-grained spatial patterns within local functional regions.
The second MVPA form is similarity-based MVPA. This approach gained momentum in recent years because it provides system neuroscience with the opportunity to quantitatively assess the relationship between brain-activity measurement, behavioral measurement, and computational modeling (Kriegeskorte et al., 2008). As for classifier-based MVPA, the idea is to consider neuronal representations as a multidimensional space, where the number of dimensions corresponds to the number of voxels, and each point in that space corresponds to different activation patterns. The set of all possible mental content corresponds to a vast set of points in the space, where the distance between points indicates their similarity (Kiergeskorte and Kievit, 2013). But, the goal of pattern similarity analysis of fMRI data is to make inferences about the representational geometry of mental states based on the similarity of patterns elicited by those concepts. The dissimilarity (or similarity) of two patterns can be computed on brain data (i.e. fMRI, M/EEG, ECoG) with different metrics (e.g. decoding accuracy or 1 – Spearman’s R). And, having measured these distances, it is possible to construct a matrix called the empirical representational dissimilarity matrix (RDM). Each RDM cell contains the values of dissimilarity between pairs of representation. Another type of RDMs is named model RDM. It reflects specific theoretical hypotheses (e.g. computational or behavioral models). Thanks to this conceptualization is possible to compare representational geometries obtained from different type of data, such as brain activity, behavioral measurements, and computational models, by just computing the distance between the RDMs. This approach is called Representational similarity analysis (RSA). For example, Proklova and colleagues (2016), used RSA to disentangle the role in the animate/inanimate distinction of object shape and object category representations in human visual cortex. They mapped out brain regions in which the pairwise neural dissimilarity was successfully predicted by the objects’ pairwise visual dissimilarity (overall appearance, outline dissimilarity, and texture dissimilarity). These analyses revealed several clusters in which categorical dissimilarity, measured as reaction time in a visual search task, predicted neural dissimilarity, even when controlling for visual similarities. Results provide evidence that visual differences (shape or texture) of animate and inanimate objects do not fully account for the animate-inanimate organization of human visual cortex. RSA has been mostly applied to study perception and cognition. However, it might be an important tool for further applications such as biomarker detection for brain diseases that show different degrees of representational dissimilarity. Distinct representational models exist other than RSA, each of which has advantages and disadvantages, and better suited for different applications. More statistical methods remain to be developed to accommodate various analytic parameters, different definition of metrics, reliability and stability assessment, and different similarity measures from multivariate brain-activity data (Xue et al., 2013).
As described above, univariate analyses carry some limitations that multivariate analysis is able to overcome. One of the most important MVPA findings is that representations are not restricted to specific areas and they are characterized by idiosyncratic patterns related to distinct tasks (or stimuli). However, in order to distinguish and to describe different mental contents is also important to consider when and how neurons interact with each other (Vaadia et al., 1995). “Neural activity, and by extension neural codes, are constrained by connectivity” (Sporns, 2007). Nowadays, it is clear that brain regions do not work in isolation, and information processing depends on continuous local and long-range interactions. The most common approach to investigate how specific neurons communicate with each other over time is functional connectivity (FC). Generally, this term refers to the temporal covariance between distinct brain regions activity, which is assumed to be driven by their interaction. Traditional application of functional connectivity involves first the selection of “seed” regions of interest, based on their activity in specific tasks or coarse anatomical parcellation of the brain, and then the analysis of correlations between these regions and other voxels. This approach has provided many insights into the neural architecture, and identified several neural networks at rest or enabled during a specific task. Still, it has some limitations. First, seeds are often defined on the basis of different averaged activation, and this procedure has the same disadvantages of univariate analysis, that is to assume that regions with greater activation (or activation differences) are most interactive or that their interactions are most informative. But, voxels activation may differ between stimuli (or from baseline) in the absence of differences in the averaged time-locked amplitude. Some MVPA studies tried to address this issue by classifying patterns of correlations among multiple regions.
Anyway, regardless of which method is used to analyze data (univariate or multivariate), seed-based functional connectivity is affected by the small number of regions selected (“seeds”) with respect to the total number of voxels, resulting in loss of information. Doing so, only a small subset of possible interactions is considered. One might wonder why this method is used and why pairwise voxels analysis are not the most common way to compute temporal covariance. Despite its limitations, seed-based analysis is widely used because it allows to avoid statistical challenges related to big data and to test specific models with greater power. Comparing every voxel with one another is computationally demanding (50.000 voxels mean 1.249.975.000 unique voxel pairs to be tested). Notwithstanding, recent advances in computer engineering opened up new possibilities. One innovative method that takes into account the full set of voxels is the Full Correlation Matrix Analysis (FCMA) (Wang et al., 2015). To surmount the seed-based limitations, FCMA performs unbiased multivariate analysis of the whole-brain. It uses matrices made out of temporal correlations in BOLD activity of every voxel with every other voxel, separated for epochs of interest. These matrices represent the input of classifier. Instead of working with activity patterns, the processing units are correlation patterns. Thus, the classifier can now determine which correlations predict conditions. Therefore, it allows detecting regions with differential interaction patterns as a function of experimental task. Obviously, the main problem is the huge amount of data to analyze. FCMA exploits flexible parallelization and provided algorithm optimization that speed up the computational processing. Thanks to this method is possible to obtain new findings in brain connectivity. For example, in the same paper where they present FCMA, Wang et al. (2015) used it with a face/scene dataset. Results revealed the involvement of mPFC and precuneus in object category perception and suggested the role of mPFC in modulating face processing (but not scene) apparent in local and long-range correlations.
Another issue of functional connectivity analysis is related to the fact that temporal covariance between brain regions, locked to the processing of external stimuli, is caused by different sources of variation. The BOLD signal can be divided in stimulus-induced signal, intrinsic neural signal (fluctuations not related to the processing of external stimuli) and non-neuronal artifacts (such as heartbeat or breathing). To some extent, standard analysis can differentiate between these sources. As long as physiological changes are uncorrelated with the design, they can be controlled by comparing two or more experimental conditions. Also, stimulus-induced signal can be regressed out providing a measure of intrinsic neural signal (“background connectivity”; Norman-Haignere et al.,2012). Problems arise with stimulus-induced response. For example, Simony et al. (2015) found that stimulus induced covariance detected with standard FC analysis is biased by intrinsic signals. Along with that, they proposed a new statistical approach to better address connectivity patterns elicited during task, named inter-subject functional connectivity (ISFC). In ISFC, regional covariance is calculated across participants (region A/participant 1 with region B/participant 2). As a result, intrinsic neural responses and non-neuronal artifacts are ruled out since they vary across participants randomly and never align. The resulting covariance values are low and not statistically significant. At the same time, neural activity covariance locked to stimuli is shared across participants and it can be isolated. They tested this model to probe how the Default Mode Network (DMN) changes when subjects listen to an auditory narrative and different scrambled versions of the same. Results show that with FC analysis there is no difference in the DMN configuration with respect to the narrative version. Whereas, ISFC detected distinct network configurations changing together with conditions.
fMRI is a quite new tool and it proved to be promising to expand our knowledge about the brain. Limitations come with it but already several discoveries shed light on brain functional regions, their content, and how they communicate with each other. Computer science, Statistics, and Mathematics have been contributing to its development, providing new methods and new conceptual frameworks. And, in order to improve its potential, it is important to keep on integrating related fields, and a better understanding of the assumptions made by different techniques.