Pig Sty Avenue

[Not proofed yet, and will likely want some editing before submission. Still undecided whether or not to give a few lines each to Weir, 2005, and Chapelle, 2012. Will finish it on Monday. Allotment tomorrow.] This research is concerned with the validity of language tests used by the immigration authorities to measure the fitness of people resident in Scotland to become citizens of the United Kingdom. The first issue is to establish what is meant by validity in the context of assessing abilities in English as a Second Language (L2) for people who have a different first language, (L1). I refer to ‘citizenship tests’ meaning for the purposes of this submission the language element (actually a test in the speaking and listening domains at the B1 level of the Common European Framework of Reference). But the literature reviewed could equally apply to all tests of language ability connected with migration to the UK, and in particular to Scotland. These include the A1 level tests for spousal...

[This blogpost was formerly entitled "Testing Validity Paradigm(s) because that is what was being researched at the time. But it has taken on a life as its own as a database of Journals regularly publishing articles on Language Testing, (all of which need to be therefore regularly consulted). I have amended it today because I have expanded the scope of research beyond straightforward validity and into Critical Language Testing, and wish to include journals discovered in connected literature searches, e.g. (5) Educational Research and Evaluation. 22/03/2016.] Today's the day when I stop flitting around the garden of academia like an unfussy butterfly, and get my oomska together. 1st order of business is a list of journals which regularly publish articles on language testing, so that I can be aware of the current paradigms, focusing today on Test Validity, (so as not to be distracted, - I've got a bloody deadline approaching). American Journal of Speech-Language Patho...

  Goodman, 2012, notes that the novel introduction of language and cultural knowledge tests in Europe has interested scholars, but there is general disinterest in the variety of such tests from state to state. The historical shift from having to pass some kind of assessment, (and perhaps swear an oath) is Europe wide. It contrasts with the previous requirement of simply "being there", (Spiro, 2008).  Goodman notes at p663 that historically the rights of citizenship were counterbalanced by its obligations, particularly in times of war, "but the updated version is a bit more ambiguous regarding the types of duties or obligations the citizen owes the state."  [She does not spell put these ambiguous duties, but what could they be? Think of both rights and obligations: Consular protection and advice if/when abroad; voting; jury service; theoretical liability to be obliged to serve in the armed forces...  [Are any of these particularly meaningful or onerous in the ...

"Mastering the language of a country of immigration is a skill acquired over time through residence and social interaction and is therefore one of the most prevailing markers of an applicant’s personal level of integration," (Goodman, 2010). "Mastery" has a specific meaning in CEFR terms, referring to C2 level. Most EU states' language requirements are in the A2/B1 region. Many language learners will reach A2/B1 levels in the L1 of a country which they have never visited, or at any rate never resided in or had social interaction with L1 speakers, for example [get hard data] school learners of French in the UK, or English in China or the Netherlands.  At p14 Goodman (2010) refers to the Austrian requirement that learners do 300 hours of instruction. Is this an alternative to the A2 referred to in that paper's Table 4, below?  So that 300 hours of instruction can be sufficient regardless of test result? That would be a rather honest acknowledgement of the fa...

I'm researching the PM David Cameron's  recent remarks on the Today Programme about spousal visas. Here's my transcript of the interview, and the Downing Street press release is here , but it refers to an article in The Times which is behind a pay-wall. Fortunately, the text can be found (eventually) on Cameron's Facebook page, here , (Cameron, 2016).  ' Ed Husain put it brilliantly last week when he said that our political correctness stops us from identifying this separatist mentality – terming it “the racism of low expectations”.' (Cameron, 2016). 'Think about the young boy growing up in Bradford. His parents came from a village in Pakistan. His mum can’t speak English and rarely leaves the home, so he finds it hard to communicate with her, and she doesn’t understand what is happening in his life. At the same time, as a teenager he is struggling to identify with Western culture.' (Cameron, 2016). But doesn't he speak Urdu with her? Are we ...

Almost finished a submission on a major theme of my Lit Rev: current paradigms in Language Test Validity. I'm looking at what Lyle, (Bachman & Palmer, 2010) has to say about "unintended detrimental consequences". I've got to reframe that for the language test score = indefinite leave to remain discourse. A language test like this is unlike any other.  For example, say a student needs IELTS 7.5 to get her place at a UK University. Let's also agree, just for now, that IELTS scores have been fully justified/validated, and that the University admissions department knows full well, using past experience, that anyone with less than 7.5 will be unable to keep up, submit essays of the required standard, and will not graduate. Maybe they know that because until a few years ago the bar was set at 7.0, and they had a whole load of mushkila situations with L2 students.  So our student gets 7.0. This looks like a "detrimental consequence" for her. She re...

What with this morning's battery charging , and the postgrads' Xmas party in an hour or two, (hopefully a rather tame affair, by my former party standards) and a seminar this evening on collecting qualitative data, I'm a bit short on time today.  But yesterday I got back through Language Testing to the October of 2013, and it's clear that since then no big rows about validity have broken out, and that any serious researchers are using "argument based validity" or "validity arguments" without taking any pains to explain why they're doing so. All of which suggests that this is the current paradigm. I'll continue the trawl backwards through LT later, (I felt that just reading the titles of the papers the last couple of years was helping get me up to speed: some things I must get back to when I have more time, like, "A testlet response theory modelling approach", whatever that is when it's at home).  Kane (2013) is 64 pages long...

I'm still going back in time through Language Testing to get a handle on the current paradigms relating to Test Validity. Chapelle et al (2015) examine two EAP automated diagnostic assessments for writing by means of validity argument. All assessments, they suggest, "need to be evaluated in view of the validity of their intended interpretations, uses and consequences", (p385-6). The question is, "how to frame a validity argument by identifying five types of inferences that one might wish to make on the basis of results from an assessment".  I need to see Chapelle et al (2008) for Domain Definition. Also important are the warrants, inferences, assumptions and ramifications, (consequences?) of an assessment. From these we can build a framework for a validity argument. [This has echoes of Bachman and consequential validity]. Resist temptation to go sideways from this ref, but just bear in mind the concepts for now and come back later. Youn (2015) is also intere...

It's a fact which seems to leave academics scoobied . When a person finishes their MA, and gets a job outside of the University, the keys to the Ivory Tower are taken away without ceremony. This means you can't keep up with published research and thought on your discipline, (or on anything else academic). My keys were taken from me in 2010, I only got them back two months ago, and all paradigm shifts during that time have passed my by. I'm having to work hard to catch up. For example, Bonfiglio's (2010) castigation of the "native speaker" construct, and Garcia's (2009) taxonomical "emergent bilingual", are, I would suggest, significant shifts which I needed to get to grips with, and which have, furthermore, had time to mature. Today and tomorrow I need to see what test validity has been up to since 2010. My starting point will be Language Testing, (the assessment person's trade journal) starting with 2015 and working backwards, noting al...

As Lyle Bachman didn't say to me, that evening we drank too much whisky in Boston, "It's all about validity, kid. The rest is just conversation." But it's a sentiment he might approve, if we mean consequential validity. We need to be sure about test fairness and reliability, too, but validity (construct as well as consequential) is The Father in that Trinity. See, test validity is something people in education bang on about without really understanding it. They want valid tests, naturally, but don't want to invest in the time and independent research that's needed to validate a test. They just don't get it. On top of that, it's a complicated thing, not quite quantum mechanics, but not straightforward, either. Even people who specialise in Assessment, (I don't exclude myself) don't always know this essential concept back to front. Yet it's vital. The "people in education", (for example, managers in a private English langua...