I've thought for a while whether I would blog about this topic. But I guess in the end, I thought that I would rather write it down than just let it simmer in my head. Anyway, what I wanted to write about today has something to do with the relationship linguists have with their data. It's an interesting, and by no means, straight-forward one.
See, for the longest time, linguists were content with native speaker intuitions. Formalists would go around and ask native speakers whether Sentence X is accepted or not, or whether Sentence X is better than Sentence Y. Looking back, it is quite funny that most linguists would claim that they are practicing cognitive science, when their methods are not cognitive at all. In my opinion, linguists who work this way are simply interested in modeling language, without considering the psychological reality of language. There is nothing wrong with that, it's just for me, I don't find that interesting enough.
I happen to belong to the experimental camp. I conduct experiments in the lab, observing human behavior with respect to language. Simply put, I am interested in the psychological and behavioral correlates of language. It is not my goal to describe what grammar consists of. Rather, I would like to investigate what linguistic behavior consists of.
And then there is the middle type, the theorist who is not content with simply sitting in an armchair and positing stars on ungrammatical sentences and hash signs on infelicitous ones. Rather, there is the experimental theorist who would like to be armed with experimental evidence to further support their intuition that such a star or such a hash sign should be there.
However, I think this is where problems arise. The thing is, once you start running experiments, you have to worry about many other things, things that as a theorist, you might not care about. As a theorist modeling language, you might not be interested in temporarily ambiguous sentences, like The horse raced past the barn fell, because after all, for you, the sentence is the basic unit of language, and you can definitely provide a structure to that sentence. But for psycholinguists, it is important to see why this sentence is a garden path sentence, while the sentence The bomb buried in the sand exploded is less so, even though they have the exact same linguistic structure.
What concerns me sometimes is this idea that theorists ought to be able to distinguish between experimental effects that are caused by elements in the grammar (which presumably they care about) and experimental effects that are caused by things that are not in the grammar (like frequency, for example, which presumably they don't care about). I feel like this is a very thin line to walk, as one can presumably extend this reasoning to give one the choice which experiment one should believe in. Because I can imagine that if an experimental result comes up, and the theorist doesn't agree with the results, then it is easy for the theorist to say that this experiment is all about the non-grammar effects that the theorist doesn't care about, and therefore it doesn't falsify the theory that the theorist supports.
I told a colleague once, that I call this the Marie Antoinette Principle of Experimental Linguistics. Once one decides to conduct experiments, one needs to deal with plenty of other non-linguistic and supposedly uninteresting factors that might affect experimental results. There's frequency, there's fatigue, there's strategy, there's plausibility, there's a zillion other things that might affect an experimental response. And somehow, I find it bizarre that some theorists are going ahead and conducting these experiments where the responses to these experiments are clearly affected by these factors, and yet they simply sweep it under the rug because for them, these factors are uninteresting.