Many speech analyses
This research was completed in collaboration with Benjamin V. Tucker, Scott James Perry, and Matthew C. Kelley. But not just with them – the purpose of the project was to involve a large number of different researchers individually or in small teams (like our four-member team) in an effort to answer the same research question using a shared dataset so the number of contributors to the project is very large. Here, I will only mention the three leading researchers, originators of the idea for this project: Stefano Corretta, Joseph V. Casillas, and Timo B. Roettger.
Our team, like everyone else, received the same dataset from an experiment in which some speakers produced some sentences. We were all asked to use that dataset to answer the same research question, but were given liberty in choosing our own approach to answering it. I won’t go into more detail (the study is much better described here anyway) because the purpose of the project was not answer this specific research question. The actual goal, at the time known only to the three leading researchers on this project, was rather to investigate the variety of analyses that will be present in the submissions of different teams or individual researchers and to test whether these ‘‘researcher degrees of freedom’’ will have an impact on the final conclusions made. A high degree of such analytic flexibility, partly depicted in the image on this page, is a hallmark of speech science and other areas of research where complex phenomena are investigated and where there are many decisions a researcher has to make regarding data collection and analysis.
The results showed substantial variety in the 46 submitted analyses, as the researchers used different ways of measuring something or different statistical analyses to make inferences about these measurements. In some cases, this variety even lead to differing study outcomes. I would say that this project is yet another agrument showing why scientists should favor open science, wherever possible. Sharing details about one’s work is a staple of any empirical science. We now have the means to make the datasets and the procedures we applied available to others, going beyond mere verbal description of what we did. Of course, much more information and a better phrased discussion of this issue can be found in the published paper and on the OSF project page.
