That New Brain and Sex Differences Study: "Deep learning models reveal replicable, generalizable, and behaviorally relevant sex differences in human functional brain organization"

All's not what it seems on the Twitter sphere -- neither is the presentation of a new paper.

Recently, Twitter showed me an article on sex differences in the brain that was reposted from DatePsych — an account I highly like on Twitter. However, I believe people are interpreting this paper rather incorrectly, primarily because comments regarding the paper seem to think there is literally a male and female brain.

Why are people thinking there’s no overlap between male and females in the brain? Well, I think it’s primarily because the original tweet posted a plot in reference to something else.

Let’s take a loot at the original paper, Ryali et al. (2023), which can be downloaded here. Figure 3 refers to the discreet clusters that arise when using statistical methods (in this case, tSNE) to visualize “clusters.” These clusters can arise based on the parameters set by they researcher, so they should not be thought of as data being inputted, the computer is just asked to sort them, and these clusters arise naturally. Rather, they use markers that groups might differ in, and these markers are used to separate people into their own respective groups.

Ryali et al. used stDNN — space-time delay neural networks — for their tSNE to organize individuals into their respective sex’s group. What are stDNNs? I won’t pretend to know, so I asked ChatGPT to define it for me. As it said:

In short, space-time delay neural networks are like smart observers that can track changes over time and space simultaneously, helping them make sense of complex data and predict future events.

So, researchers take brain scans of individual males and females, they extract relevant features that distinguish male brains from female brains, and then train their model to detect patterns and features that are typically associated with male brains and those that are typically associated with female brains. From there, they get a sample of people, do brain scans on them, let the computer separate them and your clusters are conjured.

In this context, researchers got brain waves scans and found that male waves could be grouped with other male waves. Wait no. Researchers actually got…nothing? The authors do not make clear what specific fingerprints they used, instead leaving it vague:

We then used XAI-based approaches to identify the brain features underlying the classification of female and male brains. We identified individual fingerprints of predictive brain features in each participant using an IG procedure (58) (SI Appendix, Fig. S5). Briefly, a “fingerprint” of an individual refers to the unique whole brain pattern of an IG-derived stDNN model feature importance that classifies that individual as either female or male. We evaluated the validity of brain features distinguishing females and males by measuring the similarity between IG-derived dynamic brain features. Based on their fingerprints, individuals of the same sex were clearly grouped into the same cluster (Fig. 3A).

What fingerprints specifically did the AI use to separate participants? The authors do not say, so we can not say for sure what features are sex specific or due to environment.

Put it shortly, the plot doesn’t tell us anything on if sex differences exist within the brain or not. And it can’t measure that. We do not even what specifically its even looking at. However, the same paper did find variation in overlap across different regions of the brain. So, some areas of the brain have larger overlap between the sexes, others smaller.

Other studies using different models have found contrasting results (see: Eliot et al. 2021) when it comes to sex differences in brain, finding much smaller differences. At some point, it becomes an argument on which model is more accurate, what parameters are best and appropriate, why, blah blah. Others have brought other criticisms to the table. For example, @iris_IGB on Twitter notes that some of the regions found that do have differences between the sexes are not shown to be consistent.

Another user (@SentientPotato6) also notes the freedom researchers have with visualization tools used in that study.

Interested to see if this paper replicates. Some final thoughts:

People got to stop citing this study to say that there is literally a male brain and female brain. The study did not find this, and the plot people are sticking to is not equipped to answer that and does not offer any information on that. Also, some dubious stuff going on.