The New York Post recently published an article about some guy who says he has decided to stop dating hot women and the internet is really angry about it. I can’t tell you what his reasoning was because I refuse to waste even a couple minutes of my life reading an article about a guy who treats all women as a monolith. However, Amanda Marcotte pointed out on twitter that part of his reasoning was a scientific study that supposedly proves that the sexier a person is, the worse they are at relationships.

The study in question was entitled “Attractiveness and relationship longevity: Beauty is not what it is cracked up to be” by Harvard researchers Ma-Kellams, Wang and Cardiel. Although the title seems to suggest the study was looking at relationship duration, the authors make it clear early on in their paper that their real goal is to determine how a person’s attractiveness or lack thereof predicts their relationship satisfaction. In fact, they specifically state this as their goal, writing “we focus primarily on actor effects, or the effects of a person’s [personal attractiveness] on his/her own relationship satisfaction.”  Throughout their study they will often conflate shorter relationship durations with lower overall relationship satisfaction but they will never bother to try to prove that people who are in longer relationships are actually happier in their relationships. This is one of many issues I have with this study, but I’ll get to that in more detail later.

In order to prove their hypothesis that more attractive individuals have lower relationship satisfaction, they actually did 4 very different studies that the authors contend together proves this hypothesis is true.

Study 1

The researchers downloaded copies of High School yearbooks from classmates.com for two schools (one a public school in a rich Silicon Valley suburb and one a private Catholic school in a  working class midwest town) from 30 years ago. They then had two people, which they called “independent female coders” rate the photos of all the senior boys on a one to ten attractiveness scale. They used ancestry.com to look up publicly available marriage and divorce data that matched the yearbook students in the towns the high schools were located. They included only students for which there was at least one marriage on file in the analysis.

They ended up with a dataset of 238 men who had all been married in their hometowns. Of those men, the researchers were able to find divorces filed for 39 of them. They found that the average attractiveness rating of the 39 divorced men was higher than that of the 199 married but not divorced men. The more attractive men were more likely to have gotten a divorce.

So many problems:

It’s kind of ingenious to be able to use yearbook photos from 30 years ago to predict what happened to the people in those photos, but in this case this particular study is so flawed that I have trouble seeing how the researchers are able to make a strong claim about the entire human race (more attractive people have worse relationships) based on this very, very limited dataset.

First of all, looking at two US high schools is hardly representative. The researchers did address this issue, but proclaimed that because the Silicon Valley high school was in a rich, white city and the midwestern private Catholic high school was in a working class, ethnically diverse town, so many diverse people were represented that the results can be applied across demographics. I don’t think I can fully stress how much this is not the case. Comparing a public school in a rich white area to a private religious school in a less rich area is hardly including the breadth of the human experience. For all we know, the private religious school might be located in a working class town but may still have mostly white affluent students. Even if these schools turn out to have tons of diversity, are we really going to make a claim about all people based on 39 divorcees? How much diversity can you even get with only 39 people? Not to mention that the only data they have is based on people who married in their hometowns, so this is not counting anyone who had moved away from their hometown after high school.

Really though, my biggest issue with this particular study is the fact that they had two adults rating high schoolers on their relative attractiveness. Is it just me or is that pretty fucking creepy? If I volunteered for a research study and they gave me photos of underage boys and asked me to rate how sexy I found them on a one to ten scale, I would drop out of that study and maybe report them to the FBI. I get that these were photos from 30 years ago and the boys in the photos are adults now, but the creepy factor is still there. I certainly don’t want to imagine that my high school yearbook photos were taken without my consent and my teenage attractiveness was rated by adults, even if it was for science.

Besides, they had only two women do the ratings. So, when they say some men were more physically attractive than others, what they really mean is two anonymous adults looked at photos of some teenagers and happened to be more into some of them than others. The researchers don’t ever address the fact that attractiveness may not actually be an objective factor that you can determine by having a person or two look at some photos, but instead something that is fluid and changes by individual preference. Even if they are trying to get a measure of physical attractiveness that fits within a general cultural norms standard, they should probably have had way, way more people rate the photos and perhaps cared about the diversity of the photo raters.

What the researchers think they discovered:

People who are more physically attractive are more likely to have had a divorce and therefore have less relationship satisfaction.

What the researchers actually discovered:

Men at two high schools who got married in their hometown and were deemed attractive by two adult women were slightly more likely to be divorced.

Study 2

The same two coders from study 1 rated celebrities on a one to ten scale as to their physical attractiveness. Using publicly available data, the researchers tallied up the number of marriages each celebrity has had and determined that the sexier a celebrity is, the more marriages they have had throughout their lifetime.

So many problems:

First of all, we are talking about celebrities and only celebrities. And not only that, but we are talking about the Thems. The Themiest of the Thems. We are talking about the Beyonces and the Rihannas and the Britney Spearses and then we’re having two random people rate the relative hotness of Beyonce versus Rihanna versus Britney Spears and making broad claims about how that relates to the lives of all humans. It’s really hard for me to take seriously something discovered about top celebrities being then applied to the rest of humankind as if we’re all the same. Yes, celebrities are humans too, but their lives are so incredibly different than that of the rest of us, it seems bananas to make a claim about us based on the lives of some of the most famous people alive today.

It’s also hard to know how well this claim holds up even among celebrities. The study cares only about physical attractiveness, not overall attractiveness. Obviously, there are many things that make a person attractive that have nothing to do with physical attractiveness and in general I would bet that celebrities tend to have a lot of those non-physical attractive aspects to them. For one, they are super rich and whether we want to admit it or not, money is attractive in that it clearly helps people attract sexual and relationship partners. The top celebrities also tend to be charismatic and smart and entrepreneurial and incredibly talented, all very attractive traits.

They did ask the two independent coders to rate the celebrities on physical attractiveness, but I doubt these women were unfeeling robots (no offense to any empathetic robots out there). They surely know who all these celebrities are and their feelings about how physically attractive they find them are affected by the things they know about them. For example, if someone asked me today to rate the physical attractiveness of Johnny Depp I’d be all “ew, no. Is there a number below one?” because what I know about him in terms of being a (alleged) woman batterer changes what I see when I look at him. I know intellectually that there was a time before I knew this about him that I did find him attractive, but now I can’t even stand to look at his face. How is it possible to put what we know about Johnny Depp aside and rate his physical appearance objectively? Did these coders really rate the celebrities’ attractiveness only on their physical appearance and nothing else? Were they able to put their hates and fandoms aside and give a rating as if these were photos of anonymous teenagers rather than super celebrities? And what did they do for famous older people? Was Paul McCartney rated based on photos from his Beatles days or a recent photo? And if a recent photo was used, did the raters base their attractiveness rating on that or what they knew he looked like when he was younger or only what they saw in the photo? Also, how does any of this relate to a universal trait about humankind again?

What the researchers thought they discovered:

People who are more physically attractive are more likely to have lots of a marriages and therefore lower relationship satisfaction.

What the researchers actually discovered:

Super rich celebrities who are relatively more attractive than other super rich celebrities (based on the opinions of two non-celebrities) have had more marriages.

Study 3

The researchers surveyed a group of 134 people (combination of undergrads and paid participants in Boston) about their past relationships and relationship satisfaction, 41% of whom were currently in an exclusive romantic relationship. They were shown a photo of an opposite sex person and asked to rate that person’s attractiveness. Meanwhile, they were surreptitiously being rated themselves on their own physical attractiveness by two of the experimenters who were blinded to the answers the participants gave on their surveys. They found that participants who were in relationships and had higher physical attractiveness rated the photos of the opposite-sex person as more attractive than did the participants who were currently in a relationship but deemed to be less attractive by the experimenters.

So many problems:

Ugh, stop it with the creepy non-consensual rating people’s attractiveness. I get that a study on attractiveness needs to somehow be able to rank people by attractiveness, but the only method creepier than having adults rate how sexy they find photos of teenagers is experiments where a study participant believes they are there merely to take a survey but then is secretly being sized up as to how sexy the researchers find that participant compared to the other participants. Just hearing about this study makes me never want to ever participate in a scientific study ever again, lest I be secretly ranked on an attractiveness scale by the experimenters.

Like the other studies, we are getting into really low numbers of participants that they are basing their results on. There were 134 participants, but only 55 of them were currently in a relationship, so their final results were essentially only based on these 55 people. Plus, the researchers mentioned that anyone who left a survey question blank was removed from the analysis, though they never mention how many people were removed. So, we’re actually looking at a maximum of 55 people but possibly even less. Even more worryingly, although the researchers asked participants for their sexual orientation, they didn’t release the results in the paper. All participants got a photo of someone of the opposite sex to rate, whether or not they are attracted to people of the opposite sex.

Even with the researchers constantly conflating relationship duration with relationship satisfaction, it’s hard to see how this study has to do with any of that considering their results have nothing to do with either of those things. The way the researchers explain it is that people in relationships who are more likely to see people outside their relationship as attractive are potentially more likely to cheat on their partner or otherwise end their current relationship, thereby having shorter relationship durations and less relationship satisfaction. This seems like a big stretch to get that out of this study, but the researchers don’t appear bothered by the disconnect.

What the researchers think they discovered:

People who are more attractive themselves are more likely to be seeing potential new partners as attractive when they are in an exclusive relationship. This will cause them to have lower relationship duration and in turn lower relationship satisfaction. Therefore attractive people have lower relationship satisfaction.

What the researchers actually discovered:

Survey participants who were currently in a relationship and that the experimenters thought were hot ended up rating a photo of an opposite sex person as sexier than did participants the experimenters were not attracted to.

Study 4

Other studies have shown that when people look at photos of same-sex attractive people they tend to feel worse about their own perceived attractiveness. The researchers used this tendency to manipulate how a person feels about their own attractiveness in a survey of 139 participants (all currently in an exclusive relationship) they recruited through Amazon’s Mechanical Turk. Each participant saw 5 photos of same-sex individuals. These photos came from a google image search for either “attractive female,” “unattractive female,” “attractive male” or “unattractive male,” with a portion of the participants seeing only the attractive photos and another portion seeing the unattractive photos. They were then asked to rate their own attractiveness. They then looked at three images of opposite sex people and were asked to rate their attractiveness. Finally they were asked a series of questions about their own relationship satisfaction. The researchers found that participants made to feel attractive (by looking at photos of unattractive same-sex people) were more likely to rate the photos of the opposite sex people as more attractive than did the people made to feel less attractive (by looking at photos of attractive same-sex people).

So many problems:

Can we talk about using google searches to determine attractiveness? A photo of Steve Buscemi is one of the top results when I google both “attractive male” and “unattractive male.” When I google “unattractive male” the first photo that comes up is of Leonardo DiCaprio.

photo of Leonardo DiCaprio

So obviously unattractive! How can anyone stand to look at him?

When I google “attractive female” I get mostly photos of beautiful blond women while googling “unattractive female” I end up with mostly photos of beautiful black women BECAUSE THE INTERNET IS A FUCKING TERRIBLE AND INCREDIBLY RACIST PLACE. Sorry, for the all-caps but I just cannot stress enough what a bad idea it is to determine whether photos of humans are “attractive” or not based on a google search, not to mention how difficult and fraught trying to get an “objective” measure of attractiveness is. It’s clear that there is no way the researchers just took the top photos for those google searches. They had to have used a lot of discretion in choose which photos they themselves thought were most attractive or unattractive and there is little to no visibility in how they made those decisions.

As for manipulating people to feel more or less attractive themselves, it does seem like an ingenious way to determine how a person’s feelings about themselves affects how they answer surveys, though it seems a bit wasted on these particular researchers because they used it in such an odd way. What I assumed the researchers were going to do was determine whether people made to feel attractive later reported less relationship satisfaction than people made to feel unattractive. Since this entire paper was presumably about how physical attractiveness affects relationship satisfaction, this would have made sense, but this isn’t what the researchers did. Instead, they found that people who were made to feel attractive in turn rated photos of opposite-sex people as more attractive (again presuming that everyone is attracted to people of the opposite sex) and that this implies that attractive people have lower relationship satisfaction.

I also want to put forward another theory that might explain their results. If you show a person photos of unattractive people (of any gender) and then show them photos of attractive people (again of any gender), are they more likely to rate the photos of attractive people as more attractive than if they hadn’t looked at the unattractive photos first? In other words, if you just show someone three photos of attractive people and ask them to rate them, maybe the scale they are rating them on (in terms of what constitutes a one versus a five versus a ten) is different than if they also just looked at photos of unattractive people. In fact, this cognitive bias is so well-known that it has a name: the anchoring effect.  There is a lot of evidence that the anchoring effect exists, so this seems a far more likely explanation to explain what the researchers found than does their theory that it has to do with people looking for new relationship partners.

What the researchers thought they discovered:

People in relationships that are made to feel more attractive are more likely to think people of the opposite sex are more attractive, thereby implying they are considering them as a candidate for a relationship, which will likely shorten the duration of their current relationship and therefore they will have less overall relationship satisfaction. And also, people who are attractive all know they are attractive because otherwise the entire theory falls apart.

What they actually discovered:

People who looked at photos of same-sex unattractive people rated photos of opposite sex people as more attractive.

So where does this leave us?

According to the researchers all these studies together constitutes evidence that attractive people are more likely to have less relationship satisfaction. I’m not so sure they managed to get even remotely close to proving this. Four garbage studies don’t suddenly become a great, solid study when combined. We just end up with a lot more garbage.

I have to say that the statistics the researchers used were on point. They did the proper testing before using models in order to ensure that the assumptions of that model were met and they always used the correct statistical test or model types for the situation. It’s clear these are talented researchers which is why it’s so baffling that they managed to produce such a questionable study. They had the math down pat, but the problem was that none of their studies were measuring what they claimed to be measuring.

None of these studies measured relationship satisfaction, so it’s honestly bewildering to me that the researchers were able to conclude that physical attractiveness affects relationship satisfaction. Their conflation in the first two studies of relationship duration with relationship satisfaction is a giant red flag. At no point do they attempt to prove that people who have one long relationship have more relationship satisfaction than people who have had many shorter relationships. Many people are unable to get a divorce due to cultural shame or lack of financial independence or abuse or even just lack of initiative or staying together “for the children.” This hardly implies that they then must have more relationship satisfaction than people who are more easily able to get out of marriages they are unhappy in. There is nowhere in the study where the researchers grappled with or even acknowledged this conflation. They just assumed longer relationship duration = better relationship satisfaction and plowed ahead on that assumption.

The connections between what they studied in their final two studies and what they were attempting to measure seem particularly questionable. They measured some people rating photos of opposite-sex people as more attractive then jumped from that to an assumption of lower relationship satisfaction. It’s highly questionable to assume that someone who finds a person of the opposite sex more attractive must be on the prowl for new partners and therefore will soon end their current relationship and therefore must be unhappy in their relationships. There are so many jumps from one assumption to another in order to get from the things the researchers were studying to their conclusions, and they need to do a lot more work showing data connecting each of these steps before it can be considered a plausible connection, but they don’t ever do this work except in the most shallow sense.

In the end, this study is unimpressive, but the researchers still take their small, limited studies and then use them to make a broad and universal claim which then gets reported on in the media as fact and used by New York Post bros to justify not dating attractive women. Really though, if Dan Rochkind, the NYPost bro in question, truly believes the results of this study he shouldn’t change what type of women he dates. After all, the study was about how much relationship satisfaction attractive people have, not their partners. If Rochkind wants to apply the results of this study to his life, he could get the most relationship satisfaction by making himself less attractive. After all, according to this study, the less attractive a person is, the better they are at finding true love, so stop showering and pick out the oldest, rattiest clothes in your closet to wear on your next Tindr date. If you think I’m wrong about this dating strategy, take it up with the Harvard researchers who wrote this study.

Jamie Bernstein

Jamie Bernstein

Jamie is a data, stats, policy and economics nerd who sometimes pretends she is a photographer. You can usually find her at skeptic events in Chicago or on Twitter or Flickr. She also blogs about music at Notes From Chicago Music Underground.

Previous post

Facebook in Talks with Pakistan to Help Catch Blasphemers

Next post

Ivanka Trump Booed while Dad Destroys Office of Global Women's Issues

2 Comments

  1. April 25, 2017 at 3:46 pm —

    Funny, but when you have two people rating attractiveness, not only is it creepy, it does not take into account the fact that people have different “types”; That is, a person whom one person finds attractive may not be so attractive to somebody else – you hear about arguments debating people’s “hotness” in a lot of places. So it’s bad science andbad manners.

  2. April 28, 2017 at 7:28 am —

    Another glaring bit of nonsense from the study: people can change dramatically in their physical appearance between high school and their marriage years. I was a pretty awkward teenager, and my high school photo shows a rather ungainly, skinny person with bad hair, dental braces, and bad skin. I was shy and had zero self esteem. Half way through college I changed – I was more athletic, dressed and groomed differently, had clear skin and had even gone through a bit of a late growth spurt. The opposite can happen too, of course – people who are deemed attractive in high school might look rather different a few years later. People can also have plastic surgery, which I guess can wok in either direction, beauty being in the eye of the beholder and all that.
    Furthermore, the photographs were not verified as being linked to the names of the married / divorced adults. it is perfectly possible for kids to have moved away after high school, and for new people with similar names to have moved into that town, or for there being more than one person in a town with the same name to begin with.
    So, when you say that the outcome of study 1 was that “Men at two high schools who got married in their hometown and were deemed attractive by two adult women were slightly more likely to be divorced”, to be more precise, the outcome is that “A small group of adult men who had attended two high schools and who may have got married in their hometown, and whose physical appearance when they were teenagers was deemed attractive by two adult women, based on one unverified photograph of each male, may be slightly more likely to be divorced”. So, yeah, hogwash.

Leave a reply