YouTube Transcript:
RSA Overview

Skip watching entire videos - get the full transcript, search for keywords, and copy with one click.

Video Transcript

in this video I'm going to provide a

brief overview of the application of

representational similarity analysis or

RSA to ERP data RSA is widely used in

fMRI but it can also be used with ERP is

to generate some amazing results it's a

general-purpose method for assessing

links among different kinds of neural

measures computational models and

behavior each of these sources of data

has a different format which makes them

difficult to compare directly for

example we have bold activation across

the set of voxels in fMRI voltage over

time at a bunch of electrodes in EEG and

a pattern of activation across the units

of a neural network model how can we

compare these different data formats RSA

solves this problem by converting each

source of data into a representational

similarity matrix or RSM we can then

look at how well the are SMS from the

different data sources are correlated

with each other

okay let's unpack all of this to get a

representational similarity matrix you

need to look at the pattern of activity

produced by several different inputs for

example imagine that we showed

participants a set of twenty scenes and

for each scene we obtain the pattern of

bold activation across visual cortex we

could then ask how similar to the

pattern of activation is for scene one

versus scene two there are many ways of

quantifying similarity but we could

simply calculate the Pearson R

correlation coefficient between the two

patterns of bold activation we might get

a correlation of 0.84 between the fMRI

patterns for scene 1 and scene 2 and we

might find a correlation of 0.09 between

the fMRI pattern for scene 1 and the

fMRI pattern for scene 3 we have a total

of 20 scenes so we'd end up with a 20 by

20 matrix of correlations the result is

called a representational similarity

matrix or RSM because it expresses the

pattern of similarity in the neural

representations of the 20 scenes we'd

get a separate RSM for each subject the

correlation between a given scene and

itself is always 1 so we ignore the

diagonal and the upper and lower

triangles are mirror images so we ignore

the upper triangle

some researchers prefer to use a

representational dissimilarity matrix

which is just one minus the correlation

in the end you get exactly the same

results either way we could also get a

representational similarity matrix for a

neural network model that's trained on

scene recognition we could feed each of

the 20 scenes into the network and note

the pattern of activation across the

units for each scene we could then

examine the correlation between the

active pattern for each pair of scenes

this would give us a 20 by 20

representational similarity matrix for

the neural network we could then ask

whether the RSM for the model is similar

to the RSM for the fMRI data we would do

this by just looking at the correlation

between the lower triangles of the two

representational similarity matrices we

use a rank order correlation because we

don't want to assume anything about the

scaling of these two matrices if the

matrices are correlated with each other

this indicates that the representational

geometry of the model is predictive of

the representational geometry of the

fMRI data and vice-versa we could also

have subjects view these twenty scenes

while we record the EEG we could then

make an average ERP for each scene and

calculate the similarity between each

pair of scenes in terms of the ERP data

to take advantage of the millisecond

level temporal resolution of the ERP

data we could do this separately for

each time point that is for each time

point in an average we can compute a

scalp distribution we can then get a 20

by 20 representational similarity matrix

for that time point by computing the

correlation between the scalp

distributions for each pair of scenes we

actually have a separate scalp

distribution for each time point in the

ERP waveform so we have a separate RSM

for each time point for example if we

have a sampling rate of 250 Hertz one

sample every 4 milliseconds we'd get

something like this you have an RSM at

time 0 which is the onset of the

stimulus then we have 1 at 4

milliseconds after stimulus onset 8

milliseconds after stimulus onset etc

this would go on for several hundred

milliseconds depending on the length of

the epoch that was used during averaging

of course the RSM stirring the period

immediately following stimulus onset

will be noise because information about

the stimulus hasn't reached the cortex

yet for a visual stimulus the RSM will

start being structured around

50 to 70 milliseconds after stimulus

onset remember each cell in these our

SMS is just the correlation between the

ERP scalp distributions for a pair of

scenes at that time point we can then

correlate the RSM at each time point

with the RSM for the fMRI data or the

neural network model this is just the

rank order correlation between the our

SMS we'd expect pretty low correlations

at time zero because the ERP RSM should

just be noise at that time we then

compute the correlation between the ERP

RSM at four milliseconds with the fMRI

and model our SMS and then we'd repeat

this for every time point in the ERP

waveform each Spearman Rho correlation

value indicates the representational

similarity between the ERP data at a

given time and either the fMRI data or

the pattern of activation in the model

we can plot these values as waveforms

much the way we plot ERP waveforms

except now the y-axis is the correlation

between the ERP RSM at a given time

point and either the fMRI RSM or the

model RSM so by abstracting away from

the original units of measure and

creating representational similarity

matrices we can quantify the extent to

which the representational geometry of

one source of data matches the

representational geometry of the other

sources up to this point I've been

showing artificial data but here's a

real study they used event related

magnetic fields rather than ERPs but the

principle is the same you tend to get

stronger effects with M eg than with EEG

but given how much cheaper EEG is I'm

not tempted to move to M eg subjects in

this study viewed a sequence of natural

images and performed an orthogonal

vigilant's task to keep them alert and

attentive each subject was tested in

separate M eg and fMRI sessions with the

same images the goal was to link the

temporal resolution of the event related

magnetic fields with the neuroanatomical

specificity of the ERP data to

accomplish this they calculated a

representational dissimilarity matrix

for the M eg data at each time point and

a representational dissimilarity matrix

for the fMRI data for each brain area

remember a representational

dissimilarity matrix is just one - the

representational similarity matrix

the next step was to calculate the

spearmen row rank order correlation

between the m EG matrix at each time

point with the fmri matrix for each

region this resulted in one Spearman row

correlation value for each combination

of M eg time point and fMRI brain region

here's what they found the intensity at

each voxel is the representational

similarity between that brain region in

the fMRI data and the magnetic field

distribution at a given time point in

the m EG data this is completely

different from most source localization

methods there making no assumptions

about the physics of EEG e or m EG

instead these results are obtained by

using a variety of visual stimuli to

probe the brain and comparing the

representational geometries of the fMRI

Click on any text or timestamp to jump to that moment in the video

Most transcripts ready in under 5 seconds

One-Click Copy125+ LanguagesSearch ContentJump to Timestamps

Paste YouTube URL

Enter any YouTube video link to get the full transcript

Most transcripts ready in under 5 seconds

Get Our Chrome Extension

Get transcripts instantly without leaving YouTube. Install our Chrome extension for one-click access to any video's transcript directly on the watch page.

Add to Chrome — Free

Works with YouTube, Coursera, Udemy and more educational platforms

Get Instant Transcripts: Just Edit the Domain in Your Address Bar!

YouTube

←

→

↻

https://www.youtube.com/watch?v=UF8uR6Z6KLc

YoutubeToText

←

→

↻

https://youtubetotext.net/watch?v=UF8uR6Z6KLc

YouTube Transcript:RSA Overview

Video Transcript

Paste YouTube URL

Transcript Extraction Form

Get Our Chrome Extension

Get Instant Transcripts: Just Edit the Domain in Your Address Bar!

YouTube Transcript:
RSA Overview