A CNN headline reads "Kids of lesbians have fewer behavioral problems, study suggests".

Interesting, but wait...

The study, published Monday in the journal Pediatrics, followed 78 lesbian couples who conceived through sperm donations...

The involvement of mothers may be a contributing factor [to having fewer behavioral problems], in addition to the fact that the pregnancies were planned, Gartrell said.

The children "didn't arrive by accident," she said. "The mothers were older... they were waiting for an opportunity to have children and age brings maturity and better parenting."

What? Wasn't there a control group of heterosexual couples who also used artificial insemination?

Unfortunately, there was not. This is an important point, because the investigators had to compare their lesbian households against some other group of households. It isn't really fair to compare them against a group from another study because there is no way to guarantee that the study design doesn't have inadvertent differences that created the effect that the researcher is reporting.  In psychology studies, these differences could be as subtle as the personality of the investigator and their ability to get the subjects to open-up in an interview. (In molecular biology, it could be slight impurities in the water at a facility). This is a fundamental point of doing research -- all researchers have to set up their own controls, and make sure  that all groups are given the exact same treatment, as much as is humanly possible.

I have discussed the issue of controls before -- ironically, it was in the context of the pseudoscientific theory of homosexuality being an illness.

This new study is available for free online, so I took a closer look at the methods. I only skimmed it (and did word searches), but it looks like the situation is worse than I had thought. The households in this study were compared to a previously collected national sample, and I saw no attempt to match households according to income, ethnicity, geographic location (Methods: Analyses). The samples are not very similar to each other demographically (Table 1), neither of the samples are typical of America in terms of income, and the lesbian sample is overwhelmingly "white". Also the lesbian samples were all recruited from San Francisco, Boston, and Washington D.C., hardly typical American communities, not even typical cities.

Otherwise, it turns out that my concerns about investigators influencing the results were unfounded -- but only because the evaluation of the children was not done by the researchers themselves. It was done by the mothers.

Let me repeat that: the evaluation of the child's psychological adjustment was done by their mothers. To top it off, these women probably knew what the study was about. If they weren't told outright, they may have caught a hint when they were "recruited via announcements that were distributed at lesbian events, in women’s bookstores, and in lesbian newspapers" (Methods: sampling).

To be fair, the researchers asserted that this survey methodology (self reporting) "is known for its reliability, internal consistency, and factor structure", and I'm in no position to judge these claims. I work in a very different field where it is much easier to collect data, so this seems very sloppy to me. However, it also goes against everything I've heard about doing studies on humans, such as making sure that the evaluations are done by blinded experts. That's not the only way that they deviated from standard procedures-- the researchers noted that they only implemented one of the three components of this survey: they questioned the mothers but not the children or teachers.

The CNN article is full of silly statements by a representative of the anti-lesbian group "Concerned Women for America", where she asserts that the study is untrustworthy because it was commissioned by gay advocacy groups, and it goes against her own prejudices. She doesn't have to resort to questioning the integrity researchers or refer to her own divinely revealed knowledge: she only needs to read the study to see that it is meaningless.

Finally, we come to the issue of why this study doesn't have proper controls. My suspicion is that it is simply an issue of insufficient funding. Perhaps this study is meant to start a debate, to put an idea out there (with some data) and challenge other researchers to look into it more thoroughly. Perhaps the first salvo in this debate HAS to make highly controversial statements because of the political sensitivity of the topic. Last time I checked, many conservative politicians considered human sexuality to be an inappropriate topic for federally funded research. Well, now they are probably going to be clamoring for that research.