Title :

“Advantages and limitations of digital technologies for scenario-based assessment: the case of the French Language Certificates for Higher Education (Certificats de compétences en langues de l’enseignement supérieur – CLES) ”

Author(s) :

Fries, Marie-Hélène [Auteur]
Institut des Langues et Cultures d'Europe, Amérique, Afrique, Asie et Australie [ILCEA4]
Rivens, Annick [Auteur]
Savoirs, Textes, Langage (STL) - UMR 8163 [STL]
Université de Lille
Rouveyrol, Laurent [Auteur]
PRISMES - Langues, Textes, Arts et Cultures du Monde Anglophone - EA 4398 [PRISMES]

Conference title :

Languages for Specific Purposes and digital technologies (LSP-num)

Conference organizers(s) :

Sara Alvarez-Martinez, Sophie Belan, Marie-Hélène Fries, AudeLabetoullle, Evgueniya Lyu

City :

St Martin d'Hères (université Grenoble Alpes)

Country :

France

Start date of the conference :

2024-10-16

English keyword(s) :

CLES (French higher education language certificates)
digital technologies
language testing
scenario-based language assessment

HAL domain(s) :

Sciences de l'Homme et Société/Linguistique

English abstract : [en]

CLES is a language certification based on scenarios, i.e. realistic, integrative themes in which candidates are given a mission to fulfil and a set of tasks to perform (Rouveyrol, 2012). CLES exams are based on levels B1, ...
Show more >CLES is a language certification based on scenarios, i.e. realistic, integrative themes in which candidates are given a mission to fulfil and a set of tasks to perform (Rouveyrol, 2012). CLES exams are based on levels B1, B2 or C1 of the Common European Framework of Reference for Languages (CEFR, 2001). They are situated on a continuum ranging from language for general communication to language for specific purposes (LSP), via the language of communication for professional use, depending on the skills targeted (Fries, 2022). At B1 and B2 levels, the themes of the scenarios are deliberately situated within a broad perimeter of contexts (life abroad for B1 and societal debates for B2), but the skills assessed are useful for professional use (e-mails and voice messages at B1 level, mini-reports and negotiations aiming at finding a compromise at B2 level). The scenarios in the CLES C1 papers on the other hand, focus more on socio-scientific controversies, within broad macro-domains: law and economics, health and life sciences, engineering sciences, social sciences and humanities. They therefore span somehow the interval between communicative language for professional use and LSP. The development of a digital version for the CLES exams has been approached differently for level C1 and for levels B1 and B2. From the outset, CLES C1 exams were designed as a certification rooted in academic or professional life, using computers in a way that is similar to the professional environments it targets, so as to make the tasks more realistic (Zabala & Rouveyrol, 2022). Candidates draft their slides for the oral part on computers, for instance, in the new version of the CLES C1 exams, which is very similar to a professional panel session. B2 and B1 exams, on the other hand, were originally designed in paper form, but the idea of a digital version was already expressed in a conference over 10 years ago (Tardieu, Reichert & Rivens Mompéan, 2013) and has gained momentum since the COVID 19 pandemic, as for other language tests (see for example Clark, Spiby & Tasviri, 2021 or Papageorgiou & Manna, 2021). The current digital version of the CLES exams (devised by Annick Rivens and Laurent Rouveyrol and implemented by Mehdi El Heddaoui for the B1 and B2 levels) is based on two principles: keeping the added-value of the scenario-oriented tasks for the candidates, and using the Moodle platform, which is widespread in French universities (Rouveyrol & Rivens, 2024). The digital version of CLES B1 and B2 exams has a number of practical advantages. The ergonomic design of the tests helps to improve the conditions in which the CLES is taken, since candidates have individual headphones for listening comprehension, instead of having to listen to videos in a lecture theatre. Comprehension questions, both written and oral, are automatically assessed by computer, which reduces the assessment workload for teachers and allows them to concentrate on the production and oral interaction parts (Rivens & Rouveyrol, 2024). The results of these tests are then transferred to the Apogée or Pégase school management systems (via Moodle) without manual input, which simplifies the administrative management of the CLES exams (Fries, Brouttier, Pradat-Paz et Tinchant, 2023). Their digital version, however, does not include the oral part yet. In fact, the assessment of CLES exams takes into account pragmatic criteria as well as linguistic ones, for example successfully completing the requested ask, and pragmatics seems to have been insufficiently taken into account by AI tools so far, as well as equity issues concerning automatic decision-making processes. The digital version of the CLES exams being developed therefore seems to exemplify both the current advantages and limitations of digital tools applied to language assessment. ReferencesClark, T., Spiby, R., & Tasviri, R. (2021). Crisis, collaboration, recovery: IELTS and COVID-19. Language Assessment Quarterly, 18(1), 17-25. https://doi.org/10.1080/15434303.2021.1892153Council of Europe. (2001). Common framework of reference for languages: Learning, teaching, assessment. Retrieved September 19, 2024, from https://rm.coe.int/common-european-framework-of-reference-for-languages-learning-teaching/16809ea0.Fries, M.-H., Brouttier, J.-F., Pradat-Paz, I., & Tinchant, S. (2023). Un environnement numérique pour le certificat de compétences en langues de l’enseignement supérieur (CLES)? Défis et opportunités. Paper presented at the RANACLES Conference, Lyon, France.Fries, M.-H. (2022). De l’utilité du Certificat de compétences en langues de l’enseignement supérieur (CLES) pour les langues au travail. Études en didactique des langues, 38, 25-40.Papageorgiou, S., & Manna, V. F. (2021). Maintaining access to a large-scale test of academic language proficiency during the pandemic: The launch of TOEFL iBT Home Edition. Language Assessment Quarterly, 18(1), 26-41. https://doi.org/10.1080/15434303.2021.1892154Rivens Mompean, A., & Rouveyrol, L. (2024). Numérisation de la certification CLES: Impact et questionnement didactique. Paper presented at the Colloque de l’Association pour la recherche en didactique et acquisition de l’anglais (ARDAA), Paris, France.Rouveyrol, L., & Rivens, A. (2024). Digitalising multilingual certification systems: What are the challenges? For what benefits? Insights from the French CLES. Paper presented at the CERCLES Conference on Multilingualism and the Anglosphere, Durham, UK.Rouveyrol, L. (2012). À propos de la représentation de la langue dans le cadre du CLES. Interaction verbale, interaction sociale? Recherche et pratiques pédagogiques en langues, 31(3), 15-30.Tardieu, C., Reichert, M., & Rivens-Mompéan, A. (2013). The e-CLES project: How to make a scenario-based certificate valid, reliable and fair? In Language Testing in Europe: Time for a new framework? Antwerp, Belgium.Zabala, J., & Rouveyrol, L. (2022). Assessing verbal interaction: Towards European harmonization. Insights from the cooperation between Spanish and French language exams for Higher Education (CertAcles/CLES). Language Learning in Higher Education, 12(1), 129-157. https://doi.org/10.1515/cercles-2024-0010Show less >

Language :

Anglais

Peer reviewed article :

Oui

Audience :

Internationale

Popular science :

Non

Collections :

• Savoirs, Textes, Langage (STL) - UMR 8163

Source :

Harvested from HAL