For a range of time now there has been a bit of a disagreement in experimental and research psychology over narrative versus meta-analysis. Now, I have found a meta-analysis on violence in video games that was released recently. So here I go trying to show why a person has to take these studies with a lot of hesitation (Barbara you can stop reading now lol). But first off before getting to that I’m going to mention something about the way the results are stated.
“Playing violent video games is a causal risk factor for long-term harmful outcomes” is the way that the author of the article generalises the results. However, a paragraph or two later its stated that “And it’s a risk factor that’s easy for an individual parent to deal with — at least, easier than changing most other known risk factors for aggression and violence, such as poverty or one’s genetic structure.” These two statements conflict with eachother.
Topics like poverty and genetic structure are not by themselves risk factors. For example, poverty and low socio-economic status are only risk factors if they effect the parenting that can be given to a child. Reduced parenting is what is a risk factor, not the poverty by itself. To include video games as a “causal” risk factor seems like a leap. If I remember correctly my first post was about how about video games are not causal, they are simply a risk factor. People who have more inherent violent tendencies tend to play violent video games. Who would have guessed… What geniuses.
So whats wrong with this study besides the results seems a little off. Well here we go. A meta-analysis is the process of doing a literature review where you take many studies that have been previously done and combining them to get a “definitive” result. The old way f doing literature reviews is a narrative approach. Where you go through the research and pull what you think is the best and interpret the results together. This can be peer-reviewed and evaluated.
So at a point such as this you might ask what’s wrong with meta-analysis if researchers have accepted it. Well many haven’t and the ones who have I don’t exactly have much faith in. First off, the biggest problem to me is the choosing of studies. The current one on violent video games that I’m looking at took 130 or so studies into account. Now who gets to say what studies are included. To think that a person does not have some bias when choosing studies would be to neglect human nature. Secondly, if you aren’t picking ones that are biased towards your pre-held viewpoint you cannot guarentee quality. Different studies have different quality. Some are better then others. How does a researcher choose which studies are high quality or low quality. I bet different researchers would have differing viewpoints on what are low and high quality ones. With the inability to decide upon which articles to include seems, to me anyways (and Chow :)), to be a major drawback of meta-analysis. One must think that the chance of biases coming into account are important since meta-analysis is usually used to sway policy makers. Do we want to start using biased research to sway policy holders (cancer screening tests, high fats being bad, or the current high salt issues anyone???). But biases in choosing studies isn’t the only drawback to meta-analysis literature reviews.
When you conduct a meta-analysis you are looking at studies as if they were the same as an individual in a normal study. Treating a study in this way does not make sense. You need to be able to assume that these studies are identical just like the individuals in a study are. The individuals in a study are all being analysed to test a specific part of a theory. Different experiments are testing different aspects of a theory. How can they be combined then? They were looking at different topics. Therefore, there are theoretical, methodological, and empirical differences between them.
On that same topic there are mathematical methods used by researchers to control for the differences between individuals. No similar methods exist for controlling for the individual differences between experiments.
Also there is an issue called the nonindependence problem. It is researchers wanting to believe that the experiments they choose are a random sample. However, that can’t be, it does not work like that. But meta-analysis supporters have gone the route of accepting the problem and saying that it doesn’t matter. One then gets taken back to the issue of why a person is doing the research in the first place. If you are accepting a problem to let you keep on doing the research it really seems like you might have a point to make (say if you were trying to affect public policy?).
The reason people keep on defending meta-analysis is because they believe you should have a numerical/ mathematical way to do a literature review. It supposedly has rigor, replicability, and objectivity. But really when you can’t solve the problems without simply accepting one of them as practical are you really achieving any of those goals.
The author of the study itself has claimed that he may no longer do research in this area because his meta-analysis is so conclusive. Personally, please don’t keep on doing research in this area. It seems almost embarassing that a researcher would find a meta-analysis that conclusive. But in the end, everyone please don’t just blatantly believe meta-analysis because the person has a title and a degree. Curiously, the researcher is from the same institution that the other experiment I attempted to beat up on is from (remember the helpless pencils being left on the floor?). Really having issues with Iowa State University after these two experiments.
**The meta-analysis research that I used in this entry is from Chow, S. L. (1987). Meta-analysis of pragmatic and theoretical research: A critique. The Journal of Psychology, 121, 259-271.**
#bp
