User:Chillpilled/Evaluating transgender-related citations

Page for evaluating the quality of studies and other works that are sometimes or frequently cited by either side of transgender-related controversies.

These will be my personal judgements largely based on criteria among the considered factors section.

Sources may be categorized as "anti-trans" or "pro-trans" less because of the source itself, and more because of who it is more commonly cited by.

Considered factors

Each entry should consider, where applicable:

• Author(s). Their reputation(s) and credentials can be relevant.
• Journal (or equivalent). Not necessarily an indication of value. Could be a red flag, e.g. if the paper's topic matter doesn't fit under the journal's focus or expertise.
  • Peer review. Never an intrinsic indication of value. Having peer review is the total bare minimum in research. But if a study lacks it, or there is some problem with the peer review, it could be a red flag.
  • Reputation and impact factor.
  • Editor(s).
• Publisher(s). Not necessarily an indication of value.
• Affiliations (formal or informal) and/or ideological stance(s), if any, of any of the above.
  • Tone in language can hint at the biases of the author(s).
  • Funding could be relevant.
  • There could even be a geographical or cultural bias. Researchers in Saudi Arabia or Russia are going to tend to have a different view of transgender people than researchers in Spain or New Zealand. A lot of our evaluated research will be from the Anglosphere, where opinions don't differ too radically from each other, so this will probably not be very relevant very often.
  • If the author(s) have an ideological stance, it could be relevant whether they developed that stance in response to their own research or if they instead held it prior to their published work. This might not be easy to determine if they aren't a high-profile figure.
• Recency. Sometimes does not matter very much. In the case of medical research, it probably does. A cultural bias could apply. Psychiatrists from the 1970s tended to have a different view of transgender people than the ones practicing today.

Most importantly, of course, the substance should be evaluated.

• Was it written as research or opinion? Some papers are published in academic journals, but as a "letter to the editor", OP/ED, or otherwise opinionated work. The arguments can still be considered, but not as original, empirical research findings if none actually exist in the work.
• Research question(s) and design(s).
• Underlying assumptions made on the part of the author(s). Their assumptions could be wrong.
• Acknowledgement of previous research, or a lack of such acknowledgement.
• Acknowledgement of study limitations, or a lack of such acknowledgement.
• Sample. Representativeness, generalizability. Although individual studies on transgender people often suffer from lower-than-ideal samples, the studies can be considered alongside each other if and when they yield very similar, consistently significant, results.
• Control of confounding variables. If not done properly, this can actually be ripe for abuse.
• Ethical standards.
• Criteria.
• Transparency and clarity in methodology. This is important for replicability. If the research includes a survey, the exact questions provided to respondents should be available to readers; loaded questions can influence responses.
• Data analysis.
• Discussion and conclusions. Did the author(s) reach a conclusion that their study was not fit to provide evidence for?
• Quality of the paper's references and citations. Whether the author(s) often cite themselves, or otherwise cite very selectively, could be relevant. They may also make errors about the information found in their referenced works, or paraphrase things.
• Sound logic, or lack thereof.

Some of these aspects are more important than others. The research question(s) and design(s), for instance, are probably one of the more important factors to consider.

A work can contain marginal, minor, or moderate errors that don't necessarily impact other aspects of the same work. However, major errors can make the work mostly or entirely unreliable, e.g. mistakes impacting the research question, design, or key findings.

Pro-trans citations

Colizzi et al. (2013)

Work: Colizzi et al. (2013) "Hormonal Treatment Reduces Psychobiological Distress in Gender Identity Disorder, Independently of the Attachment Style"

Source

The authors are from the Department of Neuroscience and Sense Organs at the University of Bari in southern Italy. The institution is reputable for medical research. U.S. News & World Report ranks it at #135 globally in its "Neuroscience and Behavior" category. The paper was published in The Journal of Sexual Medicine, (JSM) which was published by Wiley at the time. JSM's topic matters include urology and sexology. Wikipedia says: "the journal has a 2020 impact factor of 3.802, ranking it 22nd out of 85 journals in the category 'Urology & Nephrology'."

The study

The sample is 70 transsexual patients, with 45 male-to-females (MtFs) and 25 female-to-males (FtMs). This differs from other studies examining the effectiveness of hormone replacement therapy because it's a study that examines biological stress indicators rather than self-reported measures, psychiatric treatment access, etc. So it's been of particular interest to myself. While untreated, the patients' cortisol levels were out of wack. The study mentions that samples were taken at 8:00 AM, so I can compare the results against what a normal cortisol range for that time of day is. The post-treatment group had a cortisol range that was about within normal ranges (albeit on the higher end of normal).

The sample size may appear as a limitation, however, it is adequate when compared against the treatment's effect size (which was significant) and p-values (see statistical significance). Because there were no significant differences between MtF and FtM patients in the results (especially in post-treatment where they had almost identical cortisol levels), they can be taken together.

Patients that had "any neurologic or psychiatric pathology" or "any metabolic or intersexual pathology" were ruled out from participation. None of the subjects had a surgical intervention. The participants also self-reported measures of psychological stress, which gave the usual positive outcome. This was of less interest to me than the biomarkers, which are more objective, but is worth mentioning.

Limitations

Some possible improvements to this research type include adding other objective markers of stress, such as blood pressure. A larger sample size is obviously preferable, which may also allow for a more diverse group. Long-term follow-up would also have been useful. Ideally, observations could be made of patients using different types of hormone replacement therapy regimens as well.

Anti-trans citations

Dhejne et al. (2011)

For more information, see: TERF glossary § Swedish study

Work: Dhejne et al. (2011) "Long-Term Follow-Up of Transsexual Persons Undergoing Sex Reassignment Surgery: Cohort Study in Sweden"

Sometimes colloquially referred to as the Swedish study in reference to the country that it was conducted in. It is frequently cited by anti-trans activists to bolster claims about transgender suicidality and, less commonly, transgender criminality. Basically, it's used as purported evidence that gender transition (specifically sex reassignment surgery) increases suicidality, and that transgender women are as prone to criminality as cisgender men.

The authors are all associated with various departments at the Karolinska Institute, a top medical university in Stockholm. It is published in PLOS One, an American-based journal that publishes a large amount of content (a mega journal; this means that the journal has a broad focus and so is not very selective with what topics it publishes on) and has a relatively low impact factor. The journal as an institution is reputable, but its "mega journal" status means that quality may be inherently lower than journals with a narrower focus — if everyone who's an editor at a journal has expertise in a specific topic area, they're all more familiar with it, so more opportunity to catch bullshit, right?

The samples on the paper are good (and involved long-term follow-up). Information was pulled from Swedish national registers, which can be considered a very reliable source. Indeed, as trans people are a small population, this sort of register may be very helpful to create more representative samples. The paper notes that it "included only post-operative transsexual persons that also legally changed sex." Some limitations of the study that were acknowledged by the paper:

“”Other facets to consider are first that this study reflects the outcome of psychiatric and somatic treatment for transsexualism provided in Sweden during the 1970s and 1980s. Since then, treatment has evolved with improved sex reassignment surgery, refined hormonal treatment, and more attention to psychosocial care that might have improved the outcome. Second, transsexualism is a rare condition and Sweden is a small country (9.2 million inhabitants in 2008). Hence, despite being based on a comparatively large national cohort and long-term follow-up, the statistical power was limited.

Overview

The paper's conclusion states that sex reassignment (which is recognized in the introduction as "the internationally recognized treatment to ease gender dysphoria in transsexual persons") indeed alleviates gender dysphoria, but may not suffice by itself and could be paired with "improved psychiatric and somatic care". Honestly, who exactly couldn't benefit from "improved psychiatric and somatic care" in addition to whatever treatments they're currently receiving, though? I have to ask.

Overall, the study just says that post-op transsexuals continue to suffer from various health issues, one of which happens to be suicidality. I have little reason to doubt that post-op transsexuals have lasting health issues; trauma and mental health problems are not something that goes away instantly and gender transition is not a magical cure for them. If they faced discrimination pre-op, that could have had a lasting impact on their mental health. If you only provided depressed people with SSRIs and no other psychiatric care, you should not be surprised to find that they have lasting mental health problems ten years later, even if the SSRIs helped. However, Dhejne et al. state in their discussion:

“”Notably, however, in this study the increased risk for psychiatric hospitalisation persisted even after adjusting for psychiatric hospitalisation prior to sex reassignment. This suggests that even though sex reassignment alleviates gender dysphoria, there is a need to identify and treat co-occurring psychiatric morbidity in transsexual persons not only before but also after sex reassignment.

Suicidality

When examining this paper, it's worth realizing that the sex-reassigned transgender people were compared against cisgender controls, not non-sex-reassigned transgender controls, for suicidality rates. This is an important distinction because trans people who underwent sex reassignment may have lower suicidality than those who didn't, but higher suicidality than the cisgender population.

The paper isn't, on the other hand, designed at all to compare rates of suicidality to those of non-transitioned gender-dysphoric people, so why is it being cited for that? And despite what is mentioned above, the introduction to the paper makes clear that it isn't seeking to answer "whether sex reassignment is an effective treatment or not." So that isn't relevant to the research questions or design. Here is a quote from the paper itself:

“”[It is important] to note that the current study is only informative with respect to transsexuals persons health after sex reassignment; no inferences can be drawn as to the effectiveness of sex reassignment as a treatment for transsexualism. In other words, the results should not be interpreted such as sex reassignment per se increases morbidity and mortality. Things might have been even worse without sex reassignment. As an analogy, similar studies have found increased somatic morbidity, suicide rate, and overall mortality for patients treated for bipolar disorder and schizophrenia. This is important information, but it does not follow that mood stabilizing treatment or antipsychotic treatment is the culprit.

The discussion section adds that the overall mortality rate was only significantly increased for pre-1989 group. They note that this might "be explained by improved health care for transsexual persons during 1990s, along with altered societal attitudes towards persons with different gender expressions." In a later interview, Mikael Landén (one of the paper's authors) added:

“”The risk of suicide was increased 19 times compared to the general population, but that is because gender dysphoria is a distressing condition in itself. Our study does not inform us whether sex reassignment decreases (which is likely) or increases (which is unlikely) that risk. [...] Sex reassignment is the preferred treatment and outcome studies suggest that gender dysphoria (the main symptom) decreases. But it goes without saying that the procedure is a stressful life event. And that the surgery and medical treatment is not perfect. It is thus not surprising that this group of patients will continue to suffer from stress-related psychiatric disorders. There might be lingering professional and relational problems. It is also possible (but unproven) that gender dysphoria is somehow etiologically related to depression. In that case, fixing the first with a cure would not automatically fix the latter.

—

In later interviews, Dhejne herself aimed to dispel what she viewed as misinterpretations of the study (regarding both criminality and suicidality). She added in her TransAdvocate interview:

“”The aim of trans medical interventions is to bring a trans person’s body more in line with their gender identity, resulting in the measurable diminishment of their gender dysphoria. However trans people as a group also experience significant social oppression in the form of bullying, abuse, rape and hate crimes. Medical transition alone won't resolve the effects of crushing social oppression: social anxiety, depression and posttraumatic stress. What we've found is that treatment models which ignore the effect of cultural oppression and outright hate aren't enough. We need to understand that our treatment models must be responsive to not only gender dysphoria, but the effects of anti-trans hate as well. That's what improved care means.

—

Suicide death criteria

I have to question the paper's criteria for deaths by suicide. Deaths registered under ICD-9 codes E980 through E989, and ICD-10 codes Y10 through Y34, are included. These codes specifically describe deaths that were (respectively) "undetermined whether accidentally or purposely inflicted" and that were of "undetermined intent".

Some of the codes listed under ICD-10 codes Y10 through Y34 in particular look like they have a serious potential to be confounding because they include possibly-unintentional recreational drug overdoses, deaths from antidepressants, and from hormonal supplements. These very much sound to me like deaths which trans people would be more prone to, without suicidal intent. Other included codes are "crashing of motor vehicle, undetermined intent" (ICD-10 Y32), and even "unspecified event, undetermined intent" (ICD-10 Y34).

An NHS Scotland dictionary notes that "codes Y10-Y34 are only to be used where available information is insufficient to enable a legal authority to make a distinction between accident, self-harm and assault." A 2017 paper notes that these ICD-10 codes are used in other research to determine suicide deaths, and can be utilized accurately. They do, however, note that researchers cutting ICD-10 codes Y10-Y34 from their criteria could help reduce false positives. That may have been a good idea if the research design here involved figuring out whether sex reassignment reduced deaths by suicide, which again it wasn't specifically aimed at doing.

Criminality

Was only significantly different for the pre-1989 group. For the 1989–2003 group, transsexuals had similar crime rates to cisgender controls. Though, that's after adjustment based on immigration status and past severe psychiatric morbidity (somewhat strangely, it doesn't appear to control for economic status). The study adds in its discussion:

“”In this study, male-to-female individuals had a higher risk for criminal convictions compared to female controls but not compared to male controls. This suggests that the sex reassignment procedure neither increased nor decreased the risk for criminal offending in male-to-females. By contrast, female-to-males were at a higher risk for criminal convictions compared to female controls and did not differ from male controls, which suggests increased crime proneness in female-to-males after sex reassignment.

In her interview with TransAdvocate, Dhejne she states:

“”The individual [...] who is making claims about trans criminality, specifically rape likelihood, is misrepresenting the study findings. The study as a whole covers the period between 1973 and 2003. If one divides the cohort into two groups, 1973 to 1988 and 1989 to 2003, one observes that for the latter group (1989 – 2003), differences in mortality, suicide attempts, and crime disappear. This means that for the 1989 to 2003 group, we did not find a male pattern of criminality. As to the criminality metric itself, we were measuring and comparing the total number of convictions, not conviction type. We were not saying that cisgender males are convicted of crimes associated with marginalization and poverty. We didn't control for that and we were certainly not saying that we found that trans women were a rape risk. What we were saying was that for the 1973 to 1988 cohort group and the cisgender male group, both experienced similar rates of convictions. As I said, this pattern is not observed in the 1989 to 2003 cohort group.

—

I would have to note that prostitution (in the sense of selling sex) was illegal in Sweden until 1999 (so for a very large swath of the studied period), and that trans people have been found to have higher prostitution rates. (See e.g. the transgender sex workers article on Wikipedia). This could even have been confounding in the study's measure of criminality and especially violent crime, which it defined as: "homicide and attempted homicide, aggravated assault and assault, robbery, threatening behaviour, harassment, arson, or any sexual offense". Many legal constituencies that ban prostitution have regarded prostitution (including selling sex) as a sexual offense, and prostitutes have even been put onto sex offender registries for this reason.

One writer at Freethought Blogs shared my observations some years ago:

“”The study never at any point compared the types of crimes for which the two groups were arrested, meaning being arrested for prostitution contributes to the statistic in the same way that an arrest for sexual assault would. And on top of that, this is only true of the older cohort–1973 to 1988–and that this pattern disappeared in the later cohort, corresponding with better healthcare as well as improved legal and social climates.

—

I could speculate that lessening police hounding of prostitutes in Sweden may also have contributed. For a sliver of the later period, again, selling sex was decriminalized.

Littman (2018)

See the main article on this topic: Littman 2018 ROGD study

Work: Littman (2018) "Parent reports of adolescents and young adults perceived to show signs of a rapid onset of gender dysphoria"

Written by physician-scientist Lisa Littman, and published in PLOS One. Proposes that transgender status is can be a "social contagion". Conclusion is supported by... parents (not their transgender children) surveyed after recruitment at anti-trans websites. That's basically it. According to our page about the study, it "simplified and dumbed-down the DSM criteria for gender dysphoria so that parents could more easily use those criteria".

Judgement: It ain't good, that's for sure. At least, it's not good as solid evidence that Littman's ROGD hypothesis is valid. The main RationalWiki article on this paper looks good enough that I probably have little to add. I could say that the study at least doesn't misrepresent what it is — an exploratory study, based on a survey of parents recruited on online (anti-trans) forums, supportive of a mere hypothesis that is described as not yet clinically validated.

Background:

“”I think as humans, we are all susceptible to seeking out information that confirms what we want to believe as true and believing that research that confirms our own view is of higher quality than research that challenges it.

—"Straight from the horse's mouth".

In a later presentation, Littman reflects that she began researching gender dysphoria because she noticed more teenagers in her own town identifying as transgender in recent times. This suggests that she formed her hypothesis before conducting the research (which is not necessarily an issue whatsoever but worth mentioning). She also says that she found it curious that a shift in sex ratio had occurred among transitioning patients over the years (from more male to more female); she is skeptical that this is purely due to differing stigma towards transgender males and females (although that's a perfectly reasonable explanation for a difference in sex ratio). Additionally, she notes that adolescents are prone to being impressionable and that this can have clinical implications (e.g. eating disorders).

Her assumption that there has been a shift in sex ratio among transgender patients may not hold up consistently. Studies in the past have varied wildly on what the transgender sex ratio is. That includes research that has examined the transgender sex ratio among adolescents.