Assessment in Practice

| Action-oriented Approach|Plurilingualism|Tech-mediated

Assessment as a cyclical process 🔄️

Assessment is best seen as a cyclical process that guides learners towards success, as shown in the figure below. Assessment is not just a final phase after teaching. In this section we look at the way in which the CEFR can help in implementing assessment as a cyclical process, providing a source for setting learning goals and success criteria and for designing effective assessment instruments that can be used for teacher, peer and self-assessment. In this way, CEFR descriptors offer a way to integrate planning, teaching, and assessment in what has been described as a dynamic curriculum [9].

How CEFR descriptors can help 🆘

CEFR descriptors can help directly in relation to several of the characteristics of good assessment listed in the previous section

Good Assessment Practice	CEFR Connection
· involves sharing learning goals with pupils; · aims to help learners to know and recognize the standards they are aiming for;	The two types of CEFR descriptors help here: · setting goals in terms of communicative language activities (WHAT learners can do) · adding to those practical goals aims relevant descriptors for language competences that define the degree of quality expected (HOW they can do it)
· provides feedback which leads to students recognizing their next steps and how to take them;	Feedback in terms of communicative aims (WHAT) and qualitative factors (HOW) can help learners focus their efforts, with the addition of language details related to the quality of performance
· involves learners in self-assessment;	Adolescents and adults are well capable of using simple checklists and assessment grids for self- and peer assessment, which can be an effective way of raising linguistic awareness.
· is underpinned by confidence that every student can improve	The CEFR values positively what learners can do. The descriptors take a proficiency perspective (‘can do’) as opposed to the traditional deficiency perspective

The CEFR offers a wealth of descriptors for different areas—with an Excel tool to select the ones you want (See
AALE infosheet Exploring and Selecting Descriptors). As suggested above, there are two main types of CEFR descriptors: those for communicative activities (WHAT) and those for aspects of competence (HOW).

A major function of CEFR descriptors for communicative activities is to inspire learning tasks and the design of assessment tasks as shown in the figure below. These descriptors can also provide ‘Task fulfillment’ criteria. The CEFR 2001 puts it like this: Descriptors for communicative activities “are very suitable for teacher- or self-assessment with regard to real-world tasks. […] They are attractive because they can help to focus both learners and teachers on an action-oriented approach” (p 180).

Here are some descriptors at Level B1 from the scale Informal Discussion with Friends, a subcategory of Oral Interaction.:

INFORMAL DISCUSSION B1

Can give or seek personal views and opinions in discussing topics of interest.

Can make their opinions and reactions understood as regards solutions to problems or practical questions of where to go, what to do, or how to organize an event (e.g., an outing). Can express belief, opinion, agreement, and disagreement politely.

Can express belief, opinion, agreement, and disagreement politely.

Criteria for aspects of quality (competence descriptors) can be added in order to help one turn impressions into more objective, considered judgments—as well as summarize strengths and weaknesses in a learner’s performance style. The CEFR descriptors for competence aspects appear under the following headings:

Linguistic Competence	General Linguistic Range Vocabulary Range Grammatical Accuracy Vocabulary Control Phonological Control
Sociolinguistic Competence	Sociolinguistic Appropriateness
Pragmatic Competence	Flexibility Turntaking Thematic Development Cohesion and Coherence Propositional Precision Spoken Fluency

And here are some of the competence descriptors at B1 (HOW this is done) that could be seen as most relevant to informal discussion:

SOCIOLINGUISTIC/ PRAGMATIC	Can perform and respond to a wide range of language functions, using their most common exponents in a neutral register.
LINGUISTIC RANGE	Has a sufficient language to express him/herself with some circumlocutions on most topics pertinent to his everyday life such as family, hobbies and interests, work, travel, and current events.
FLUENCY	Can keep going comprehensibly, even though pausing for grammatical and lexical planning and repair is very evident, especially in longer stretches of free production.
LINGUISTIC ACCURACY	Uses reasonably accurately a repertoire of frequently used “routines” and patterns associated with more predictable situations.

The discussion above is summarized in the figure below:

Formats for assessment instruments ✔️✖️

Descriptors of both types (communicative activities/competences) can be used either separately or together in two types of assessment instruments: checklists and grids.

Below is an example of a checklist. Notice that the answer is not Yes or No. In this case, the columns are:

can do it by themselves / can do it with help / they cannot do it yet.

Among other possible column titles are:

what I can do now / my priorities
I can do this / it’s better than before / more work please
with a lot of help / with a little help / without help
my assessment / a peer’s assessment / the teacher’s assessment

The advantages of the checklist approach is that it is simple and transparent, there are clear link between descriptors used as objectives (in the curriculum, in course planning) and assessment, and such checklists are good for reporting results, for example to parents.

The alternative is the assessment grid. These are usually used as the rubric to judge performance in a particular assessment task, but they can also be used for a general self- or peer assessment. The advantage here is that, because there are different levels, there is an element of comparison, of choosing the best fit to learner performance for different qualitative aspects. The judgements are evidence-based and considered, and both teachers and students (in peer or self-assessment) enjoy the approach.

The example give below has been created by merging CEFR descriptors of both types (activities / competences) in order to add qualitative elements to the criteria for Production & fluency and for Interaction, in addition to a third category, ‘Pragmatic/discourse competence.’ Three or four categories and three or four levels are most practical in a grid (imagine trying to work with 9 levels and 9 categories = 81 squares; a grid like that was once published!!!). This grid has 12 boxes, which is manageable.

	A2	B1	B2	B2+
Production (oral) and fluency	Can produce a series of isolated phrases. Can express him/herself with pauses and frequent repetition. Needs help in order to be understood	Can express him/herself in an understandable way, provided he/she has the time to prepare and rehearse, despite hesitations and frequent repetitions	Can convey the message effectively on topics of general interest. Can express him/herself with a degree of fluency; pauses and repetition do not hinder comprehension	Can produce clear and organized oral texts that fully convey the message. The flow of speech does not put strain on the audience.
Interaction (oral)	Can manage simple routine exchanges on familiar topics provided that the interlocutor is collaborative and supportive	Can interact with reasonable ease in short conversations on familiar topics, asking for help from the interlocutor if necessary.	Can interact reasonably fluently even in extended exchanges. Can react to the interlocutor appropriately without appearing to compromise on the message	Can interact fluently and effectively in extended conversations and discussions without having to compromise the message. Can link what he/she says to what other people say
Pragmatic/discourse competence	Can get attention and exploit learnt phrases to initiate simple exchanges	Can initiate, maintain and close simple conversations on a familiar topic using simple language flexibly to express him/herself	Can intervene effectively in a discussion on familiar topics adapting his/her expressions appropriately when necessary	Can adjust to the direction of a discussion and intervene appropriately to take the floor, bring in other speakers or make a closure.

A model of the process of teacher assessment ✏️

A very clear model of what should be involved when a teacher assesses their students has four steps as follows:

Planning (what and how you are assessing and what most appropriate assessment activities are for the specific assessment purpose)
Implementing (assessment processes like introducing the assessment activity to the learners and later providing feedback to them)
Monitoring (both students’ achievement and their development over time by recording and interpreting the evidence collected and revising teaching plans)
Recording and disseminating (students’ achievement and progress in reports etc.)

A word of warning 📢

With regard to creating assessment grids, it is unfortunately a common practice in applied linguistics to separate the levels with qualifiers like ‘a few / some / many’ or ‘limited / some / considerable / thorough – even using the same qualifiers for all the different categories in the grid as in this example below:

This practice was satirized in applied linguistics as long ago as 1991 as ‘spreading the category across the levels like butter’ and has been condemned as invalid in applied psychology since 1941!!

Finally, the same research group[1] that identified the characteristics of good assessment, referred to in the ‘bite-sized’ section also pointed out factors that inhibit good assessment:

a tendency for teachers to assess quantity of work rather than the quality of learning;
greater attention to marking and grading than to providing advice for improvement, with a risk of lowering learners’ self-esteem;
emphasis on comparing learners with each other, with a risk of demoralization;
teachers’ feedback to learners for managerial purposes rather than for more effective learning;
teachers not knowing enough about their students’ learning needs.

To round off this session, watch this 8-minute video from Cambridge Assessment. It explains how CEFR-based assessment instruments can be exploited for learning-oriented assessment (Cambridge Assessment).

References 📝

Assessment Reform Group. 1999. Assessment for learning: Beyond the black box. Cambridge: University of Cambridge, School of Education. https://www.nuffieldfoundation.org/sites/default/files/files/beyond_blackbox.pd

Piccardo, Enrica; Berchoud, Marie; Cignatta, Tinziana; Memntz, Olivier & Pamula, Malgorzata (2011): Pathways Through Assessment, Learning and Teaching in the CEFR. Graz: European Centre for Modern Languages. https://www.ecml.at/en/ECML-Programme/Programme-2008-2011/Pathways-through-assessing-learning-and-teaching-in-the-CEFR

Piccardo, E. & North, B. (2019). The action-oriented approach: A dynamic vision of language education. Bristol: Multilingual Matters

CEFR Expert Group (2023). A guide to action-oriented, plurilingual and intercultural education. Strasbourg: Council of Europe.https://rm.coe.int/1680af9496

Weir, C. (2005). Language testing and validation: An evidence-based approach. Basingstoke: Palgrave Macmillan.

Messick, S. (1989). Validity. In R.L. Linn (Ed.),Educational Measurement. (pp.13–103) London: Collier Macmillan Publishers.

Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1-73.https://doi.org/10.1111/jedm.12000

Bachman, L. F. (2005). Building and supporting a case for test use. Language Assessment Quarterly: An International Journal, 2(1), 1-34.https://doi.org/10.1207/s15434311laq0201_1

Purpura, J. E. (2021). A rationale for using a scenario-based assessment to measure competency-based, situated second and foreign language proficiency. In M. Masperi M., C. Cervini & Y. Bardière (Eds.), Évaluation des acquisitions langagières : Du formatif au certificatif. MediAzioni, 32, A54-A96. http://www.mediazioni.sitlec.unibo.it

Graves, K. (2008). The language curriculum: A social contextual perspective. Language Teaching, 41(2), 147-181.

AALE Toolkit Home | Action-oriented Approach | Plurilingualism | Tech-mediated | AALE Infosheets Exploration