19 January 2013

Guardians of Grammar

I came across a link before Christmas that I found quite interesting.  It was a Guardian guest article by an applied linguistics lecturer, Dr Catherine Walter.  The headline was bold and clear: "Time to stop avoiding grammar rules", so of course I was interested, being firmly in the pro-grammar camp myself.  The subhead went for the jugular: "The evidence is now in: the explicit teaching of grammar rules leads to better learning"

Excellent, I thought, everything I've ever said has been vindicated.  Does this mean the tide will now turn in the teaching world?  Sadly not.

The article was something of a disappointment.  Despite its mention of meta-analyses, it provided precious little evidence and instead went down the road of discussion views and hypotheses... views and hypotheses which I personally agree with, but do nothing to convince those currently holding the opposite view.

That said, I still find it difficult to see how this sort of logic fails to convince people:
most English language learning takes place in countries where English is not the predominant language: a foreign language situation. Much of the thinking leading to strictures against grammar teaching has taken place in countries where English is the predominant language: a second language situation. The enormous difference in exposure to the target language makes arguments based on exposure or emergence much less plausible in the foreign language situation.
ie the leading lights of the ESOL/EFL world are working in an atypical environment (and of course Walter one of them) -- their students have plenty of opportunities for practice outside of class.  Why are so many teachers so quick to accept the pronouncements of people whose teaching environment and student base is so radically different from theirs?

(This argument alone goes beyond the article's boundaries of exposure vs training and drills into the fundamental identity crisis of the Communicative Approach: in a class with a shared native language, isn't English inherently a barrier to communication, not a facilitator of it?  Surely the communicative imperative is broken if the language gets in the way?)

She also talks about "chunking" -- a central pillar of the lexical approach.  She says:
But the best estimate is that there are hundreds of thousands of chunks in English; learning enough of these to have an appropriate chunk to hand in a given situation is not a quick or trivial job. With much less time and effort, learners can acquire grammar for putting together comprehensible phrases and sentences that can serve them on the long journey towards more native-like proficiency.
...which is perfectly correct.  If a chunk is composed of regular grammatical features, then the chunk can be understood as a construction until the learner has seen it enough times to identify it as a chunk.  Learning some of the underlying grammar rules of a language makes the learner capable of dealing with a lot larger a subset of the language than learning the same number of chunks.

But Walter doesn't go far enough, because she doesn't point out the inconsistency of those who espouse both "learn by induction" and "learn by chunks".  Learning by induction is supposed to be the "natural" way -- it's how babies do it.  But it is readily demonstrable that babies learn grammar before they learn chunks -- they cannot repeat a large chunk until and unless they have internalised the component language.  OK, you will find babies and older children who have incorrectly generalised two frequently co-occurring words as units, but even if you consider that chunking (the alternative interpretation is that they've mistakenly identified the two words as being single ones), the vast majority of early language appears to be unchunked.

If the justifications and theoretical underpinnings of the approach are inconsistent, why have any faith in it?

Just what is applied Linguistics?
A lot of teachers will be likely to reject Walter's views because she's an academic, a linguist.  Sadly, the term "applied linguistics" is somewhat opaque, because applied linguistics could involve so many things -- from forensic linguistics ("was this confession written by the accused?") right through to producing better instruction manuals for washing machines.  But no, applied linguistics is used almost exclusively to mean one thing and one thing only: language teaching.  Most "applied linguists" are genuine, honest-to-goodness teachers, but ones that do research as well (many universities make a lot of money by running summer EFL courses out of their Applied Linguistics departments).

Applied linguistics as a field is so far from ivory towers that one author named his textbook "An Introduction to Applied Linguistics: from Practice to Theory" (Alan Davies, Edinburgh University Press), in order to highlight that all language teaching theory starts in the classroom, and all theory is an attempt to describe and understand successful teaching.

So Walter isn't an outsider -- she isn't just someone who doesn't understand the reality of teaching, which is one of the criticisms most often aimed at academics who attempt to give advice on how to teach.  It didn't pop up in the comments thread explicitly, but there was a rumbling...  One commenter, Espoolainen, noted the lack of "chalkface examples".

The collective noun for "anecdotes" is "an internet"
We all know that the plural of "anecdote" is not "data", but yet anecdote is what the internet thrives on.  We don't want figures, we don't want proof, we want one or two stories with real protagonists.  It doesn't matter to many of the commenters on the article that the uncontrolled variables in a single example make it meaningless -- that's what people want to hear, but they don't call it "anecdote", they call it "chalkface examples" and kid themselves on that it's not really anecdote.  (On the other hand, I'd bet good money that if Walter had given any genuine examples, 101 commenters would have jumped on her for using anecdotes in her article.)

One of the other commenters linked to a paper by the National Research and Development Centre for adult literacy and numeracy(NRDC) which was proclaimed by the very next commenter as "more useful than the original article".

The report is full of "chalkface" examples.  One chapter is called "take 40 teachers: ESOL teachers' working lives", and the next "take 40 classrooms: teaching and learning strategies in the classrooms observed", followed by "Telling cases: ten classroom case studies".

That's a plurality of anecdotes, isn't it?

Meta-analyses and literature reviews
The problem with arguing any case in terms of figures, evidence or science, is that for every published paper you can cite to support your case, your opponent will be able to cite one that states exactly the opposite.

Good academic practice relies on a thorough and complete overview of all available data, which is nigh-on impossible for any one person to do, and a humungous waste of time for everyone to do.  This is where meta-analyses and literature reviews come in.  Someone (more often a group of people) sits down and sifts through all the published papers they can get their hands on and try to work out what they all mean when taken together.

This means eliminating studies with unreliable methodologies, checking whether the conclusions were truly warranted and balancing the volume of evidence on both sides.

This is the sort of material that Walter claims backs her up, and when pressed she quoted the following sources:
Norris, J. M. & L. Ortega. 2000. Effectiveness of L2 instruction: a research synthesis and quantitative meta-analysis. Language Learning 50/3: 417-528.
Gass, S. & L. Selinker. 2008. Second Language Acquisition: an Introductory Course (Third Edition). New York: Routledge/Taylor.
Spada, N. & Y. Tomita. 2010. Interactions between type of instruction and type of
language feature: a meta-analysis. Language Learning 60/2: 1-46.
Spada, N. & P. M. Lightbown. 2008. Form-focused instruction: isolated or integrated?
TESOL Quarterly 42: 181-207

Now let's look at the NRDC study.  In the introduction they say:
The five NRDC Effective Practice Studies explore teaching and learning in reading, writing, numeracy, ESOL and ICT, and they set out to answer two questions:
  1. How can teaching, learning and assessing literacy, numeracy, ESOL and ICT be improved?
  2. Which factors contribute to successful learning?
Even before NRDC was set up it was apparent from reviews of the field (Brooks et al, 2001; Kruidenier, 2002) that there was little reliable research-based evidence to answer these questions. Various NRDC reviews showed that progress in amassing such evidence, though welcome where it was occurring, was slow (Coben et al, 2003; Barton and Pitt, 2003; Torgerson et al, 2003, 2004, 2005). Four preliminary studies on reading, writing, ESOL and ICT were undertaken between 2002 and 2004 (Besser et al, 2004; Kelly et al, 2004; Roberts et al, 2004; Mellar et al, 2004). However, we recognised the urgent need to build on these in order greatly to increase the research base for the practice of teaching these subjects.
So what we've got is a group saying that they were essentially starting from scratch; that they were incapable of performing a meta-analysis with the existing research.  That makes this a single, solitary study, with 500 students (not a lot if you're building a national strategy on it).  And there's absolutely no mention of Norris and Ortega's meta-analysis, either to rule it in or rule it out of their work.  There are two possible interpretations of its absence: either they didn't know about it or they didn't like its conclusions.  If it's the latter, in my book that's just dishonesty.  If you disagree with something, don't ignore it: explain why you are disregarding it, so that readers can judge for themselves.

The NRDC report is pretty much what Walter complained of in the article:
Each approach has been defended with carefully structured arguments, and some approaches have been embraced enthusiastically by ministries of education around the world.
Arguments, not evidence, define policy.  That's what she's saying, and it's all too often true -- just Google the term "policy-based evidence"!

Taking off the blinkers of English
Dr Walter came so close to making an important point when she said:
There is a problem with English: it is a morphologically light language. It doesn't have many different verb endings, and its nouns only inflect for plural. If the language under discussion were Polish, with its three noun genders and seven cases, the idea that teaching grammar rules wasn't necessary would probably not even occur.
Why did she not expand on this?  I came across a masters thesis on the net a while ago on the teaching of Finnish and it took as a given that conscious grammar study was required, because decades of figures showed conclusively that immigrants really never quite "got" the language without a bit of explanation.

The problem with Finnish is often claimed to be its complexity, but the author of the thesis was more interested in the difference between Finnish and the immigrants' native languages.  Just look at the very first sentence of the abstract:

Research has shown that explicit instruction with a focus on forms is needed in learning a very different language.
Difference: that's what we should be looking at.

Consider that one of France's top-selling language products is Assimil, a course that makes a big deal out of learning by assimilation.  We'll leave aside the fact that there's still a fair bit of explicit grammar explanation in the books and instead look at the situation in terms of differences.  French is anything but a language isolate, being part of a broad continuum of Romance languages.  When you consider that Italian and Spanish are two of the most popular languages for study in France, it's easy to argue that Assimil got its reputation by teaching languages with a very low "difference" from the learner's native tongue.  Even German, the other big neighbour, isn't too different from French in many respects.  (French may not mark case in nouns, but its pronoun system still makes a distinction between subject, direct object, indirect object and possessive that maps closely enough to German's nominative, accusative, dative and genitive to give the learner a head-start.)

ESOL has justified itself generally by dismissing any learner errors as unimportant, but it can be readily seen that the most common errors are caused by fundamental differences between languages, and the most obvious example is the use of articles, because most learners have some kind of problem with them.  Speakers of article-less languages either leave them out or add them in an arbitrary and meaningless way (NB: arbitrary, not random: most are consistent in their misuse) and speakers of Romance languages tend not to be able to chose between an indefinite article and the number one.


Even if English isn't very complex, it is still different in several fundamental ways from any given language, and if that's a problem for Finnish, it must be a problem for us too...


...and it is, because these problems never go away.  There's no magic tipping point where the Spanish speaker has had enough exposure to English to tell the difference between "a car" and "one car", or where a Polish speaker suddenly can distinguish between "cars" and "the cars".  It just doesn't happen.

Walter's cardinal sin
But Walter's biggest mistake was trying to make it sound as though her conclusions were based on some piece of brand-new research.  It grabs the attention, but it undermined her argument when she got called on it.

Why not simply point out that the evidence has been available for years to those who were willing to listen?

I know it's a hard argument to sell, but it's the only intellectually honest one.  Giving an explanation of why orthodoxy is so slow to change won't necessarily have an immediate effect, but in the long term it should open peoples critical faculties.

Why orthodoxy is so slow to change
The human lifecycle is a fairly predictable thing.  We are born. We go to school.  If we do well at school, we'll probably end up in university.  After university, we walk out into the world looking to do things differently and better than those who came before us.  But nobody listens to us and we end up as the least important employee in our companies.

Over the next decade or two, the genuinely gifted start to rise to prominence in their careers.  It's only now that they can start to impose "their way" on the world, but their information is over a decade out of date by this point.  Their view of the mainstream is in reality a river that ran its course a long time past.

This leads them to reject new research that supports the old orthodoxy that they have already "proven" wrong in their heads.

Would any of us in our 30s or 40s pass our university exams if we went back now?  Probably not -- our information is hopelessly dated (unless you're a mathematician, perhaps).  So who are we to tell others in our field what to do?

Don't they remember what it was like to go into that first job and have everything you knew about your field rubbished by people who just didn't understand the new stuff?  Why do we visit this same humiliation upon the next generation?

9 comments:

random review said...

Interesting post and I certainly don't disagree with your general thrust, but there are some aspects of language where (if we're honest) we simply don't understand the rules well enough to explain them accurately. The example of articles falls into this category (as I know from great frustration trying and to find a good rule for the omission of the Spanish indefinite article.*). Of course we can explain what we do know and some rough and ready rules that work most of the time, but do you not think that ultimately these things can only be learned by THE RIGHT KIND OF MEANINGFUL EXPOSURE supplemented by these rules? I write "the right kind" because of course you are right that, "There's no magic tipping point where the Spanish speaker has had enough exposure to English to tell the difference between "a car" and "one car", or where a Polish speaker suddenly can distinguish between "cars" and "the cars"." and because I wouldn't know how to define what that would be more precisely. Of course, there's more to grammar instruction than explicit rules.
Your most recent post on headwords was fascinating b.t.w.

* I know you speak Spanish, so I'll just quickly explain what I mean: the textbooks give some seemingly good rules, then you get more exposure and realise that it is simply not true that you can't say "tengo un coche" or "soy un profesor" but that these (rarer) structures have a subtly different meaning though equivalent English translation (I'm not including examples where un/una mean "one", which are easy to explain). It is sometimes awe-inspiring to see native speakers (to take two real examples from TV) switching back and forth between (forms of) "tener móbil" "tener un móbil" or "ser bruja" and "ser una bruja" in the same conversation, but meaningfully and definitely not at random. I've never seen a good rule that captures this 100% (though I've seen a few that are fairly decent rules of thumb).

Titch said...

You're right, of course. You can only get so far with rules, but I reckon they take you pretty damn far, and I hear too many people dismissing rules because they're imperfect.

But I do fall into the trap of not being clear on that, and it is important not to make people believe that the rules do everything.

Titch said...

As for your Spanish example, I hadn't given much thought to the "ser" example, and I wasn't even aware that you could do that with "tener"...

But the "ser" I always took as literal vs figurative. But of course using the literal form figuratively makes it a stronger metaphor... which means that "eres una bruja" isn't actually a metaphor at all because it doesn't claim to be literally true, but it's not really a simile either. Interesting....

But the tener one's really caught my attention, cos a quick Google shows me examples that prove that I knew some this without ever having consciously studying it, meaning I'm partly wrong.

I saw the example "no tengo coche", and I know I've said that myself, but I'm sure I at least "noticed" the form written somewhere before starting to use it, so there was some conscious process behind it.

But it still makes me stop and think that maybe I've been a bit too hard-line up till now....

random review said...

I don't think you're being too hard line. You are right that input, even comprehensible input (even all this i+1 stuff) as it is described does not work. I think that what is happening (if you don't mind me sticking my two cents worth in) is that people who successfully learn through input are doing more than they realize they're doing and if they actually did only what they think they were doing they wouldn't be successful. It's quite valuable that you point out flaws in their stated methods (as opposed to their actual methods). That's speaking as someone who thinks there's something important to be learned from what these people are doing. An interesting blog in that respect is "Keith's voice on extreme language learning" by an extremely likeable and honest young man who used the TV method and documented his progress. The results (he wasn't trying to impress anyone) showed exactly where the strengths (his pronunciation) and weaknesses (pretty much everything else- even comprehension!) of trying to learn with comprehensible input lie. To be fair to him, his experiment is not over and more interesting than I have described. 2000 hours of TV was actually only phase one of his method, but it does document what would happen if you did follow that school of thought rigorously.

Titch said...

" people who successfully learn through input are doing more than they realize they're doing and if they actually did only what they think they were doing they wouldn't be successful "

Exactly -- which is exactly why I've never liked he-who-will-not-be-named. He was never willing to get into a discussion about what he actually did, and any attempt to start a discussion was taken as an insult, as though I was ignoring his great truth.

Moving on...
I remember when I first read about Keith's blog, and I'm sure I've got a post somewhere where I talked about how the oversimplistic blurb of most language courses is harming the independent learner's approach to learning, but I can't find it (maybe it was on a forum rather than here...)

I can't say I'd followed the blog since, though... I'll have a nose and see what he's got to say for himself.

Titch said...

Aha... this must be the post I was talking about.

random review said...

Oh dear, I know I'll regret it and it does make me look like a bit of a p***k, but it's so current I have to say this. I wish I had a way to PM you but since you left HTLAL I don't. Benny recently completed a 3 month mission learning Egyptian Arabic in Brazil (to prove you don't need to be in the country). It was extremely impressive. At roughly the same time, a guest blogger, Brian Kwong, completed a 3 month mission to learn the much easier language of German, with the considerable advantage of living in Austria, and using a method mostly inspired by what Benny says he does. You can find it here: http://www.fluentin3months.com/papa-chat/. There is a video of the final result. It is billed as a great success. I have to be careful what I say here, so let me put it this way: having seen that video, I personally would never, ever recommend to anyone I know who wanted to learn German that they follow this method. That's putting it as politely as I can. I only discovered Keith's blog this summer (when looking for stuff on ALG), which explains how I missed your post. He seems a really nice guy and is quite openly going back to studying with Assimil. He believes that in the long term that 2000 hours of nothing but input will help his Chinese. That remains to be seen, but the general consensus is that it has helped his accent.

Titch said...

I can delete your post and all references to it if you want. My comments below will probably be expanded on into a full blog post soon anyway.

If you want to contact me directly in future, you can drop an email to info@after-words.co.uk .

Don't worry about sounding harsh. What the guy did is above average for someone who doesn't normally learn languages, but he could clearly have done a lot better -- I could follow a fair bit of the conversation, and I even noticed a glaring error: mistranslation of "will" as "will" instead of "werde" (or rather "wird", as it was a third person singular).

Now, when I finished my first long-distance challenge ride on my bike (80+ miles through the Scottish Highlands), I was very proud of myself. I took about three times as long as the first finishers, so I was by no means a "good cyclist", but I still had every reason to be proud of myself.

But I wasn't qualified to set myself up as a personal trainer on a website called "80milesin100days.com", because I wasn't an expert, and I wasn't very good.

Brian has conquered a personal obstacle and reached an admirable goal, and has every reason to be proud of himself. But he is now trying to claim his personal success as a public success, and use it to market a piece of software.

As Brian's making a commercial product, he has no right to expect not to be criticised for his results.

Also, given that he's all he's selling is an SRS system and he essentially claims that they don't exist on iOS, he's left himself right open for this.

Titch said...

Back to the Spanish, cos I started remembering things during the week.

My process of reasoning was as follows:

Es irlandes. (He's Irish.)
Es alto. (He's tall.)

These are adjectives, right? So what about this:
Es profesor. (He's [a] teacher.)

It uses the exact same structure, and if you think about it, we're describing a fundamental defining property of the person in question. (Whereas in the Celtic languages, interestingly enough, your profession is not considered fundamental, but I digress....)

So for a while I accepted it as a semi-sort-of adjective... until I noticed the use of "es muy amigo" for "he's a very good friend" and I decided it was definitely an adjective, because "muy" is an adverb.

And why not? The boundary between adjectives and nouns in Spanish is pretty thin -- you use adjectives pronominally all the time where in English we would normally add in one (Quiero un verde -- I want a green one).

It's a bit more complicated in other Romance languages, right enough -- I don't recall having heard anything like the "muy amigo" thing in others (but that may be down to the fact that I've spent more time in Spanish than any of the others....)