Rubric design and self-assessment of EFL writing: A case of Russian undergraduate linguistics students

Full Text

Introduction Teaching writing in English as a Foreign Language (EFL) presents both challenges and opportunities, making it a key area of focus in language education (O’Brien, 2004). Strong writing skills are crucial not just for effective communication © Rodomanchenko A.S., Sokolov O.A., 2025 This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License https://creativecommons.org/licenses/by-nc/4.0/legalcode but also for academic success and career advancement. In an increasingly digital world, the ability to write clearly and persuasively influences everything from professional interactions to personal expression (Hedgcock, 2006). Moreover, writing serves as a powerful tool for cultivating critical thinking, encouraging self-reflection, and deepening student engagement with learning- competencies that are especially valuable in higher education (Yanning, 2017). However, traditional teaching methods, including those common in the Russian context, often emphasise a product-oriented approach, where students rely heavily on model texts to guide their writing (Andreeva, 2022; Abubakarova et al., 2021; Shadzhe, 2021). This can limit their ability to develop original thought and adapt their writing to various contexts. This paper explores the implementation of such an approach, emphasising self-assessment and reflection while prioritising the development of transferable writing strategies and fostering student creativity. Literature Review Types and Role of Self-Assessment in Language Education Self-assessment plays a crucial role in effective language instruction, particularly in EFL education. By reflecting on their skills, identifying areas for improvement, and setting personal learning goals, students assume greater ownership of their progress. Research indicates that self-assessment can enhance students’ metacognitive awareness, leading to improved writing performance and greater motivation (Falchikov & Boud, 1989). In higher-level EFL contexts, where learners prepare for academic or professional communication, the ability to critically evaluate one’s own writing becomes especially important (Barnawi, 2011; Nadri & Azhar, 2017). Moreover, self-assessment fosters a deeper understanding of language conventions and writing standards (Zhang & Zhang, 2022). Through interaction with assessment criteria, learners become more sensitive to genre expectations, which fosters more coherent, contextually appropriate, and higher-quality writing. Research on the subject has focused on various aspects of self-assessment, investigating its similarities and differences with teacher assessment and focusing on the functions it performs, as well as exploring its impact on student performance, the development of study skills, and student motivation and well-being (Barmuta, 2023; Vekkesser, 2020). It has been argued that, as with teacher assessment, selfassessment can serve two major purposes: assessment of learning (summative) and assessment for learning (formative) (Butler, 2023). Summative assessment occurs at the end of an instructional period and aims to evaluate student learning against predetermined standards or benchmarks (Gardner, 2012). Common examples include final exams, standardised tests, and end-of-term projects. While summative assessment is useful for evaluating overall achievement and program effectiveness, it often lacks the immediacy required for timely feedback. Given that summative assessment occurs less frequently and aims to track student progress across a broader range of material, it is unable to provide the level of detail to be diagnostic and inform targeted teaching decisions. As a result, formative use of summative data through reinterpretation is unlikely to yield consistent and practical results, calling for “maintaining a clear distinction between formative and summative in terms of the use made of the evidence” (Harlen, 2012, p. 116). In contrast, formative assessment is characterised by its ongoing nature, occurring throughout the learning process rather than at its conclusion (Bell & Cowie, 2001). Its primary aim is to monitor student learning and provide continuous feedback that can inform both teaching practices and students’ learning strategies. This type of assessment is not merely about measuring performance; it emphasises the learning process itself. Incorporating formative self-assessment tools, such as checklists, rubrics, exit tickets and learning journals, in the learning process allows educators to make real-time adjustments to their instructional methods based on students’ needs (Black & William, 2012). The ultimate goal of formative assessment is to promote student engagement and self-regulation, enabling learners to reflect on their progress and identify areas for improvement (Andrade & Valtcheva, 2009). In higher-level EFL contexts, where learners are often preparing for academic or professional communication, the ability to critically evaluate one’s own writing represents a key outcome of such formative practices. In the domain of writing instruction, self-assessment tools such as rubrics have proven effective in providing learners with clear benchmarks for evaluating their work. Rubrics serve as structured frameworks that delineate specific criteria, helping students understand expectations and measure their progress. The research underscores the value of rubrics in guiding learners toward better writing practices by fostering metacognitive awareness and encouraging reflection on their writing processes (Panadero & Johnsson, 2013). The Impact of Rubric Design on Self-Regulated Learning Self-regulated learning (SRL) has emerged as a central concept ineducational psychology, encompassing cognitive, metacognitive, behavioural, motivational, and emotional dimensions of learning (Panadero, 2017; Pemberton & Cooker, 2012; Zimmerman, 2002). Defined as “learning that results from students’ self-generated thoughts and behaviours that are systematically oriented toward the attainment of their learning goals” (Schunk & Zimmerman, 2013, p. 45), self-regulated learning emphasises the proactive role of students in their own learning processes. This approach views learning as an activity that students undertake for themselves, rather than as a passive reaction to teaching. Its significance in education lies in its capacity to foster lifelong learning skills, which are essential for success beyond formal academic settings and formal schooling (Council of Europe, 2020). For language learners, self-regulated learning is particularly relevant as it provides a framework for managing the complex and ongoing process of language acquisition. The emphasis on goal-setting, self-monitoring, and adaptation aligns well with the needs of language learners who must navigate various linguistic and cultural challenges. Within the broader framework of self-regulated learning, self-assessment plays a crucial role in enabling students critically evaluate and refine their work. Through this process, learners identify relevant standards, assess their own performance, and make adjustments accordingly. Studies indicate that training in self-assessment can significantly enhance student outcomes, including performance summative assessments. As educators seek effective ways to implement self-assessment, rubrics have gained prominence as a powerful instructional tool (Wang, 2017). Serving as structured frameworks that articulate clear performance criteria, rubrics help learners understand expectations and provide explicit benchmarks against which they can evaluate their work. In formative assessment contexts, rubrics are particularly valuable because they both guide students in their writing and facilitate constructive feedback from instructors (Andrade & Du, 2005). However, the effectiveness of self-assessment is closely linked to the design and wording of rubric descriptors. One of the factors that impact the accuracy of self-assessment and, as a result, the subsequent learner behaviour is the wording of descriptors (Wang, 2017). However, the impact of descriptor wording extends beyond the self-assessment procedure. In the context of language learning, where self-assessment is employed formatively as part of the teaching process, the wording of descriptors plays a pivotal role in guiding students’ efforts and focusing their attention on different aspects of their communicative competence (Tierney & Simon, 2004; Vekkesser, 2020). The same task, especially one requiring language production, can emphasise different learning outcomes depending on how the rubric is framed. By adjusting the focus - whether on linguistic accuracy, communicative effectiveness, or broader learning strategies - educators can steer students toward specific developmental goals. As Jonsson & Svingby (2007) note, rubrics assess performance and signal to learners and instructors what matters most in a given task, ultimately shaping their approach to learning. Overall, within the context of language learning, the complexity of communicative competence has two key implications. First, it encourages learners to take a self-directed approach, shaping their own learning paths based on teacher guidance or self-obtained insights. On the other hand, it calls for the use of specific, goal-oriented strategies and tools, such as rubrics, to navigate the learning process and achieve better results (Griffiths & Inceçay, 2016). However, this issue seems to be largely underexplored. Much of the existing research has focused on the importance of learners’ awareness of assessment criteria for improving the accuracy of self-assessment (Andrade & Valtcheva, 2009; Panadero & Romero, 2014), or on examining rubric-based self-assessment as an in-class procedure, its objectives, and its impact. These studies consistently show that self-assessment conducted without structured tools, such as rubrics, “is not as effective” in fostering “positive self-regulatory strategies oriented to learning” (Panadero & Romero, p.142). While these analyses provide valuable insights, they offer little discussion of how rubric design itself functions as a pedagogical mechanism-specifically, how particular rubric items guide learners toward distinct developmental outcomes, whether linguistic, metacognitive, or communicative. What is more, the use of rubrics as pedagogical tools is particularly underexplored in Russian language education, with few empirical studies examining their role in improving writing outcomes (Barmuta, 2023). Additionally, research shows that the use of rubrics increases learners’ awareness of the different aspects of “specific tasks” (Panadero & Romero, p.142), which results in better performance, as general and holistic descriptors have been found to decrease accuracy, while descriptors consistent with learners’ experiences tend to increase it (Butler, 2023). Nonetheless, limited attention has been paid to the impact of the wording of rubric items on different components of student performance, i.e. activation of prior knowledge, specific strategies, or subskills involved in doing the task. At the same time, the results of a study of self-regulated learning by Kitsantas and Zimmerman (2007) highlight the significance of “finegrained standards” in enhancing learners’ awareness of subtle improvements in skill acquisition, which is particularly relevant for language learners. However, it is unlikely that students can attain this level of awareness without the support of a teacher, as highlighting the specific areas to work on and articulating them in the form of a rubric is a challenging task requiring an in-depth understanding of the language and language learning. In a study by Wang (2017), an analysis of the factors impacting a rubric’s effectiveness indicated the significance of item wording, suggesting that the employment of unambiguous, objective, and specific performance quality descriptors contributes to enhanced accuracy in self-assessment and may also impact students’ motivation to evaluate their own work against established criteria. Moreover, the inclusion of evaluative adjectives in rubrics was found to elicit subjective and unhelpful responses, with participants in the study expressing confusion and sсepticism towards descriptors such as “excellent,” “good,” and “poor,” which they deemed “misleading” (Wang, 2017, p.1287). According to Dawson (2017), the degree of complexity demanded by the rubric when evaluating performance plays a critical role in its effectiveness. Rubrics that demand advanced-level judgment - requiring students to make expert connections between quality criteria and complex components - can sometimes be counterproductive. Learners may struggle to differentiate between relevant and irrelevant aspects of their performance, which can make the process of selfassessment more cognitively demanding and ultimately less accurate. For this reason, developing the skills needed for rubric-based assessment is essential and should come before using rubrics for formative self-assessment (Brown et al., 2015). As highlighted by Panadero and Romero (2014), enhancing such skills plays a significant role in improving the accuracy of self-assessment and elevating the overall quality of students’ decision-making that follows this process. Similar findings had previously been reported by Blanche and Merino, who argued that “the self-test items that seem to have yielded the most accurate answers contain descriptions of concrete linguistic situations that the learner can size up in behavioural terms” (Blanche & Merino, 1989, p. 324). However, particular examples of rubric items and their links with student performance seemed, again, to be overlooked. Arguably, this is one of the reasons why Panadero and Jonsson (2013) included the need to investigate how different rubric designs impact learning outcomes in the list of future research directions. As item wording is an integral part of rubric design and, as a matter of fact, the key determinant of a rubric’s content, the connection between the phrasing of performance quality descriptors and students’ performance deserves closer investigation. This study addresses a gap in the research by examining how specific rubric elements correlate with student performance, offering insights into improving rubric effectiveness in language education. The study investigates whether emphasising pragmatics leads to more substantial writing improvements than a traditional product-oriented approach common in EFL settings. Through this comparative analysis, the study seeks to determine which instructional approach- one grounded in pragmatic awareness or one focused on formal textual organisation- more effectively fosters students’ writing development and supports the transfer and retention of acquired writing skills over time. Methods This study took place at the Higher School of Economics in Moscow, Russia, involving a group of 15 senior undergraduate linguistics students aged 21-22, all at the C1 proficiency level of English in accordance with the CEFR. There were 13 female and 2 male students; however, since the participants shared L1 (Russian), and their gender was not relevant to this research, the data below are presented in aggregate form. The study adopted an exploratory study approach to investigate the role of rubric design in shaping self- and peer-assessment practices and learning outcomes in English as a Foreign Language (EFL) writing. A fully experimental design was not feasible due to practical limitations, such as not having enough participants for a separate control group. Instead, we worked with a single cohort of 15 students, which was divided into two treatment subgroups (TS1 and TS2) of 7 and 8 students each through random assignment, with the condition that each subgroup included an equal representation of the three specialisations. The students were tasked with creating and developing blogs centred on topics relevant to their fields of professional interest. The study compared the influence of two different rubric types within these groups, looking at how each rubric shaped student writing and assessment processes and outcomes. The key research question of this study was whether and how the design of self-assessment rubrics shapes the development of specific EFL writing skills, as well as informs writing decisions of university students. Specifically, students using a pragmatics-oriented rubric, which emphasises audience awareness, engagement strategies, and communicative purpose, were expected to show greater improvement in these areas compared to a baseline and to their counterparts using a genre-oriented rubric, which focused on structural and stylistic conventions such as organisation and clarity, as they were anticipated to make more significant gains in the aspects of writing targeted by their rubric. By comparing the effects of these two rubric types, the study aimed to clarify how rubric design related to students’ self-assessment focus and resulting writing skills development. Data collection employed both quantitative and qualitative methods. Quantitative data comprised assessment scores from self-, peer-, and teacher evaluations, recorded on a three-point scale for each blog post. Quantitative data were analyzed using basic descriptive statistics, including means. Qualitative data were derived from students’ reflective paragraph submissions. A thematic analysis was performed to identify recurring themes related to self-assessment accuracy, engagement with the audience, and awareness of genre conventions. The study was conducted within the framework of the course Discourse for Professional Communication (DPC), designed to enhance students’ linguistic and extralinguistic competencies, enabling them to become effective communicators in their specialised fields. The DPC course is subdivided into three modules, each targeting skills relevant to professional communication required in the student’s future careers: (1) Discourse Analysis (Module 1, 24 hours); (2) Multimodal Content Creation (Module 2, 28 hours); and (3) Planning and Conducting an Interactive Event (Module 3, 20 hours). The research was carried out specifically during Module 2, which focused on developing students’ writing skills through the creation of blog posts. Students were involved in the creation of engaging multimodal content, such as text posts, image-nuclear stories, newsbites, podcasts, etc., published on their Telegram channels to engage with the professional community. To examine the relationship between audience awareness and writing skills development, the study employed two distinct rubrics. One rubric concentrated on the specific characteristics of the blog post as a genre, while the other emphasized the potential pragmatic effects on readers. Both rubrics were used by learners for self-assessment and were intended to inform their decisions regarding the content, language, and structure of their blog posts. The assessment rubrics in this study were designed to evaluate students’ blog writing from both a pragmatic and a genre-specific perspective. Each took a distinct angle on core writing skills but shared some common ground on purpose. For example, the pragmatics-oriented rubric’s criterion “Tailoring content” and the genre-oriented rubric’s “Organisation” both addressed effective structuring to meet audience expectations, though their focus reflected the overarching purpose of each rubric. Similarly, “Engagement strategies” and “Clarity and conciseness” reflected different aspects of readability, while “Purpose clarity” and “Personal voice” assessed consistency in discourse. Criteria such as “Use of multimedia” and “Searchability and referencing” highlighted the role of digital tools in enhancing communication. The integrated focus on multimedia elements also addresses modern digital literacy requirements, aligning with Kress’s multimodal literacy theory, which argues that effective modern communication blends various forms of expression - like text, images, and hyperlinks - into a cohesive message (Kress, 2003). Overall, the rubrics were designed in such a way as to enable course participants to build a deeper understanding of the connection between theoretical frameworks of discourse analysis previously covered in Module 1 and their practical implications. The conscious understanding of the communicative process, paired with the focus on writing skills, is believed to have provided a solid foundation for assessing student progress against the competencies outlined in the Common European Framework of Reference for Languages (CEFR). According to the CEFR, learners at the C1 proficiency level are expected to produce “clear, detailed, well-structured texts in an assured, personal, natural style appropriate to the reader in mind” (Council of Europe, 2001, p. 62). Furthermore, the overall written production descriptors mention that examples given in texts should be relevant, and the CEFR Companion Volume (Council of Europe, 2020, p. 66) states that the learner is expected to be able to vary “the tone, style, and register according to the addressee, text type, and theme,” which also hints at the learners’ expected pragmatic awareness. This requirement reflects a deeper pragmatic awareness expected from learners, suggesting that effective communication involves adapting one’s writing to suit different contexts and audiences. Pragmatics-oriented rubric. To make the rubrics accessible and avoid overwhelming students with complex pragmatic terminology, the design focused on the practical textual implications of pragmatics within the blog post-genre. The pragmatics-oriented rubric prioritised clear, real-world examples and transparent descriptors, directly addressing the nuances of writing with a specific audience in mind. This approach aimed to bridge the gap between theoretical concepts and practical application, enabling students to readily understand and apply the principles of effective audience engagement in their writing. By providing concrete guidance and minimizing technical language, the rubric ensured both usability and relevance for the learners. Based on Scott’s analysis of the pragmatic properties of successful blog posts, several key features were identified as the foundation for the pragmatics-oriented rubric. These were the effective use of elements that actively encourage reader interaction, such as questions or calls to action; the incorporation of relatable experiences and examples; the strategic use of multimedia to enhance understanding and engagement; and the clear articulation, either explicitly stated or easily inferred from the content, of the blog post’s overarching purpose. These features, considered crucial for engaging and informing the target audience, formed the core criteria for evaluating students’ blog posts using the pragmatics-focused rubric (Scott, 2022, p.15-17). The first criterion in the pragmatics-oriented rubric was intentionally crafted to be broader in scope than the other three criteria, serving two significant purposes. Firstly, this design choice aimed to enhance students’ understanding of the overall direction the rubric intended to guide them towards. Secondly, this broader criterion serves as a foundational tool for familiarising students with the rubric’s functionality. It offers initial training in distinguishing between the various sub-criteria within a single criterion and helps students connect the rubric’s content to specific texts they are analysing (Table 1). Genre-oriented rubric. In a similar vein, the genre-oriented rubric featured “Organisation” as its first criterion, strategically designed to give learners a foundational understanding of the expected structure typical of a blog post. This initial focus not only clarified the expectations for students but also served as a practical tool for them to engage in rubric-based assessment and hone the skills necessary for further incorporation of self-directed learning strategies in their work. The subsequent criteria within the rubric were derived from an analysis of common features of blog posts, as articulated by Scott (2022). These were the adoption of a more personal and conversational tone, the incorporation of features more associated with spoken language, which was reflected in shorter sentences and paragraph length, searchability enhanced through the strategic use of keywords and headlines, and selfrepresentation through consistent voice and writing style (Table 2). Table 1 Genreoriented rubric. Criterion 1 Criterion Excellent Good Needs improvement Organisation My headline uses strong action words, numbers or emotional triggers effectively, capturing attention while accurately reflecting the content in a compelling way. Clear subheadings with key words make my post easy for the reader to navigate. Example: «Unlock Your Potential: 5 Steps to Master Time Management» draws interest while indicating what readers will learn. My headline includes action words but may lack impact or creativity because emotional triggers are missing; the headline is relevant but could be more engaging. My post is mostly organised with subheadings present, but some ideas feel jumbled together due to ineffective use of keywords, linkers, or paragraphing. Example: «Time Management Tips» describes the content but doesn’t entice curiosity about what those tips are. My headline doesn’t use strong action words or doesn’t relate to the content; it is somewhat relevant but does not effectively draw readers in or engage them fully. Some organisation exists, but subheadings are unclear or missing; transitions between sections are weak, and paragraphing is illogical or absent. Example: «Managing Time» feels dull and does little to spark interest. Table 2 Pragmaticsoriented rubric. Criterion 1 Criterion Excellent Good Needs improvement Tailoring Content I consistently identify and engage with my audience’s needs, making the content relatable and engaging throughout. My language and tone are tailored to fit my imagined audience’s content consumption habits, and I effectively utilise both linguistic means and multimedia elements to meet my imagined audience’s expectations. Example: If my audience is young professionals, I might reference current trends in their industry to connect with them. I generally address my audience’s needs, although I am not fully aware of them and might occasionally have to rely on my subjective judgement. My language and tone are mostly appropriate; linguistic means and multimedia elements connect well with the audience but may lack full relevance in some areas. Example: My tone is friendly, but some technical terms may confuse readers who are not familiar with them. My understanding of my imagined audience’s needs is limited; some content may not resonate effectively with them, and I make some attempts to tailor content, but there is a lack of consistency in my writing caused by my fragmented understanding of my imagined audience. Example: My post might include general advice that does not consider the specific professional challenges faced by my audience. The study procedure was conducted over four weekly in-class sessions, each designed to progressively enhance students’ blog-post writing skills through structured self-assessment using two different rubrics. The sessions began with a brief group discussion on the characteristics of a high-quality blog post, after which students were introduced to the objective of improving their writing skills across multiple sessions. Participants were organised into groups of three to five members based on their respective specialisations. This grouping was essential for fostering collaboration and ensuring diverse perspectives within each group. Within these groups, students created and developed blogs focused on topics relevant to their fields of study. As part of the coursework, they were required to design content plans for their blogs and participate in brainstorming sessions to generate ideas for upcoming posts. Prior to the start of the study, students received instruction on the fundamentals of genre analysis and the essential components that define various genres as part of the in-class work in Module 1. This foundational knowledge was crucial as it equipped students with the analytical tools necessary to understand the specific conventions and expectations associated with different writing forms, such as narrative, persuasive, or expository texts. Session 1 (Week 1). During the first session, participants were divided into two treatment subgroups that included representatives from different specialisations. This division was necessary solely for the distribution of the self-assessment rubrics. Treatment Subgroup 1 (TS1) used a pragmatics-oriented rubric, while Treatment Subgroup 2 (TS2) employed a genre-oriented rubric. Each rubric was organised into four sections, with one section representing a specific criterion the students were supposed to assess their writing against at each stage of the learning process. This division allowed for the generation of more focused feedback. It was projected to increase the efficacy of the rubrics as each criterion encompassed several assessed elements, thus forming a basis for comprehensive yet structured feedback. To initiate the process, students were provided with selected blog posts for skimming, allowing them to notice key features relevant to effective blog writing. Subsequently, they received PDF versions of the first sections of the two rubrics, highlighting different aspects of blog post writing: “Tailoring Content” for TS1 and “Organisation” for TS2. All participants were informed that although the rubrics differed in content, they served the same purpose in guiding their assessments. Students were instructed to carefully read these rubrics and apply them in assessing the previously skimmed blog posts by highlighting relevant elements using different colours. This collaborative exercise allowed them to compare and discuss their findings within their designated groups. Following this analysis, each student drafted their individual blog posts and conducted a self-assessment using the sections of the rubrics provided earlier. Based on this self-evaluation, students made revisions before publishing their posts on Telegram. Additionally, they wrote a brief reflective paragraph, not exceeding 150 words, summarising their progress and outlining goals for further improvement. During Session 1, a notable difference emerged between the two treatment groups based on the type of rubric they were given. Students who worked with the pragmatics-oriented rubric posed significantly more questions than those who utilised the genre-oriented rubric. This difference can be attributed to the latter group’s familiarity with product-oriented writing principles, commonly emphasised in traditional EFL instruction. A closer examination of the questions posed by students who worked with the pragmatics-oriented rubric reveals a heightened awareness of various aspects of their writing. For instance, TS1 expressed concerns regarding the need to tailor content to the preferences of the target audience and sought clarification on methods for conveying purpose clarity. This line of inquiry suggests that students were actively considering the potential impact of their writing on the intended audience, demonstrating an increased awareness of the complex relationships between writer, text, and reader. Conversely, TS2, who worked with genre-oriented rubrics, raised a few questions regarding the use of jargon and profession-specific terminology in their blog posts aimed at professional audiences, demonstrating that they recognised the importance of making informed language choices in influencing audience perception. This observation suggests that using genre-oriented rubrics does not necessarily preclude students from engaging with the pragmatic dimensions of their writing and may even facilitate a more nuanced understanding of the complex interplay between genre conventions and audience needs. Session 2 (Week 2). The same treatment subgroups were maintained throughout the experiment to ensure consistency in the learning experience. During this 80-minute session, students followed a process similar to that in Session 1 but concentrated on the remaining three sections of the rubrics. Using these new criteria, they reassessed the same blog posts, allowing for a more thorough and nuanced evaluation of their work. This iterative approach was designed to enhance students’ comprehension of rubric-based self-assessment and provide practice in applying new skills. The students were also tasked with writing a new blog post to reinforce learning, conducting a self-assessment, and submitting a brief reflective paragraph. In this reflection, they were encouraged to identify their perceived strengths and specify areas for improvement in future writing. Session 3 (Week 3). During this session, students received complete rubrics in PDF format and used them as guidelines for writing a new blog post. However, this time, the task complexity was increased by requiring students to organically incorporate two obligatory elements into their blog posts: a summary of an article relevant to their academic major and an extended metaphor based on a thesaurus, which each student had prepared as part of their home assignment. After drafting their posts, the students repeated the assessment process from previous sessions but now evaluated their writing against all criteria represented in the rubrics. This comprehensive assessment allowed them to gain insights into various aspects of their writing performance. Session 4 (Week 4). The final session focused on peer assessment, which involved the students critically evaluating a blog post previously written by their peers. The evaluation process was guided by the same rubric that had been used by the author during the original writing of the posts, ensuring consistency in the assessment criteria. After completing the peer assessment, the students were required to provide detailed and constructive feedback, highlighting strengths, identifying areas for improvement, and suggesting actionable recommendations. Following this, students were tasked with creating a new blog post incorporating several components. First, they summarised the content and key points of the previously assessed post. Next, they addressed the issues raised in it. This task was designed to achieve several pedagogical objectives, including fostering more profound engagement with the subject matter, enhancing critical thinking skills, and facilitating the internalisation of the writing techniques and strategies practised during the experiment. After each session, students’ posts were assessed by the researchers against all criteria from the respective rubrics, while the students were made aware only of the grade for the criterion they had been working on during that session. This approach ensured that feedback remained focused and manageable while still providing valuable insights into overall performance. Results and Discussion Figure 1 presents the mean scores (from self-, peer-, and teacher evaluations) assigned to student blog posts across the criteria outlined in the pragmatics-oriented rubric. Scores range from 1 (“Needs Improvement”) to 3 (“Excellent”). It illustrates several key findings regarding students’ writing performance across various tasks. Figure 1. Mean Scores by Criterion in the Pragmatics Rubric Source: created by Aida S. Rodomanchenko and Oleg A. Sokolov using MC Excel Students consistently improved their ability to tailor content, as demonstrated by their use of diverse content formats. This adaptability was further reflected in adjustments to writing tone, aligning with the conventions of blog post composition and online communication. Conversely, clarity of purpose exhibited a steady decline as task complexity increased. Each successive task introduced additional challenges, such as the requirement to integrate a summary of an article or to employ an extended metaphor, thereby complicating the writing process. A possible explanation for the decline in purpose clarity lies in the students’ overemphasis on task specifics, which prioritised structural writing aspects over audience consideration. Consequently, many posts lacked direction or originality, primarily presenting summaries without adequate contextualisation or connections to the imagined audience’s experiences. The application of engagement strategies showed improvement in Posts 2 and 3 compared to Post 1, likely due to the explicit instruction for students to compose a blog post later to be published online. However, a notable decline occurred in Post 4, where students were instructed to comment on their peers’ posts. This shift resulted in a lack of recognition regarding the necessity to stimulate reader engagement, leading students to revert to basic commenting rather than employing more interactive strategies. Similarly, the effectiveness of multimedia elements varied according to the prescribed blog format. No data is presented for Post 3, as it was produced by hand on paper, thus limiting the analysis of multimedia integration in that instance, even though some students drew pictures and underlined some words, adding “link” in brackets. The analysis of student performance in TS2, as illustrated by Figure 2, reveals the consistent focus on organisation from the outset of the study. The data also indicates a positive trend in the clarity of student writing, with Post 2 and Post 3 achieving the highest mean scores. The iterative nature of the assessment process, where students received and applied feedback over multiple sessions, likely contributed to this improvement. Interestingly, Post 4, which required students to integrate a partner’s summary into their writing, saw an increase in wordiness and a corresponding decline in clarity. This suggests that when students perceived a task as cognitively demanding, they tended to adopt a more elaborate, less reader-friendly academic style. Figure 2. Mean Scores by Criterion in the Genre Rubric Source: created by Aida S. Rodomanchenko and Oleg A. Sokolov using MC Excel The most notable gains occurred in the “Personal Voice” criterion, likely because the rubric encouraged students to develop their own stylistic preferences. This led to a more individualised writing style and greater incorporation of personal experiences in their work. Interestingly, these enhancements in personal voice did not depend on the complexity of the task at hand. This suggests that the rubric’s role in promoting personal expression and reflection may be a fundamental aspect of effective writing instruction, regardless of the task’s difficulty. The “Searchability and Referencing” criterion also shows strong performance, suggesting that the use of the rubric encouraged the students to acknowledge the significance of citing sources and integrating them into their posts, as well as raised students’ awareness of the impact the strategic use of keywords has on the searchability of a blog post. While the pragmatics-oriented rubric fostered audience awareness and engagement, it sometimes sacrificed clarity and structure. Conversely, the genreoriented rubric promoted coherence and adherence to genre conventions. Still, it may have stifled personal expression, as the overall mean score for “Personal Voice” in TS2 was lower than the other criteria. Qualitative analysis also suggests that the pragmatics rubric’s focus on communicative goals may have been more accessible to students than the genre rubric’s emphasis on formal conventions. Notably, the genre-oriented rubric demonstrated more consistent and steady improvement across its criteria compared to the pragmatics-oriented rubric. This trend is evident in the gradual increase in scores for “Clarity and Conciseness,” “Organisation,” and “Searchability and Referencing” throughout the four blog posts. By comparison, the pragmatics rubric produced more variable results, likely due to its broader emphasis on audience engagement and strategic language use - skills that may require more time to develop. These findings underscore the significant role of rubric design in shaping students’ writing development, influencing both their self-assessment focus and overall outcomes. Each TS’s writing was also evaluated using the opposing rubric, which helped identify the following trends (Table 3). The pragmatics rubric group displayed inconsistent trends when assessed with the use of the genre-oriented rubric, with scores often declining by Post 4 (e.g., Organisation fell from 1.83 to 1.17). Conversely, the genre rubric group consistently achieved higher mean scores in their own categories, such as a 2.58 average in Clarity and Conciseness, alongside steady or upward trends in other areas, like Personal Voice rising from 1.25 to 2.5. Overall, both groups demonstrated sustained, though differentiated, improvement in the areas targeted by their respective rubrics, supporting the argument that carefully constructed, criterion-specific interventions using self-generated feedback can help learners achieve measurable gains in distinct aspects of L2 blog post writing. Table 3 CrossAssessment Results for Pragmatics Group (PG) and Genre Group (GG) by Pragmatics Rubric (PR) and Genre Rubric (GR) Criterion Mean for Post 1 Mean for Post 2 Mean for Post 3 Mean for Post 4 PG GG PG GG PG GG PG GG 1 PR:Tailoring Content 2.26 1.67 2.80 1.50 2.71 1.20 2.83 1.50 GR: Organisation 1.83 2.25 2.00 2.20 1.86 2.25 1.17 2.25 2 PR: Engagement Strategies 2.00 1.67 2.40 1.25 2.28 1.20 1.00 1.00 GR: Clarity and Conciseness 1.67 2.25 1.80 2.80 2.14 2.75 1.83 2.50 3 PR: Purpose Clarity 2.80 1.33 2.80 2.00 2.71 2.00 2.50 2.25 GR: Personal Voice 1.83 1.25 1.20 1.40 1.29 2.00 1.50 2.50 4 PR: Use of Multimedia 1.60 1.33 2.80 1.25 NA 1.20 1.67 1.00 GR: Searchability and Referencing 1.67 2.00 2.20 2.60 2.00 2.25 1.17 2.25 Observations derived from cross-assessment offer broader insights into the relationship between rubric design and student outcomes, supported by both quantitative trends and qualitative analysis. Overall, rubric-driven progress is often confined to the specific competencies emphasised by the assessment criteria. While rubrics are valuable for shaping targeted skill growth, their specificity may also limit adaptability. However, the cross-assessment data suggest there may be opportunities for blending pragmatic and genre-based approaches. In several instances, pragmatic gains were achieved from genre-based instruction, and vice versa. For example, TS2’s high scores in “Clarity and Conciseness” (mean 2.75 in Post 3, Table 3) sometimes contributed to better audience engagement when assessed from a pragmatic perspective. Reflective comments noted how “clear ideas discussed” made the content more readable for hypothetical readers. Similarly, TS1’s focus on “Purpose Clarity” (mean 2.71 in Post 3, Table 3) seemed to overlap with TS2’s “Organisation,” hinting that a clear communicative goal could support stronger structural coherence. This overlap aligns with Hyland’s (2007) view that genre mastery involves an understanding of how texts relate to their contexts. Limitations. Despite the valuable insights gained from this research, several limitations must be acknowledged. First, the study’s sample size was relatively small, consisting of only 15 undergraduate students from a single institution, which may limit the generalisability of the findings. Also, although the focus was on comparing two types of rubric design, other important factors that can influence how accurately students assess their own work - like their individual learning styles, background knowledge, or motivation - weren’t explored in depth. Finally, the results indicated a notable disparity in improvement across the evaluated criteria. Some aspects clearly benefited from the use of the pragmaticsbased rubric, but others didn’t improve to the same extent. This inconsistency raises important questions about the specific components of writing that are influenced by different rubric designs and how these may interact with students’ existing competencies. Conclusions Overall, the preliminary findings suggest that by emphasising pragmaticsoriented approaches in rubric design, educators can enhance students’ metacognitive awareness and ultimately improve their writing performance. As this study has shown, when it comes to formative self-assessment and the feedback it generates for learners, rubric design isn’t simply a technical detail or background feature of instruction. Rather, it plays an active role in shaping how students approach their writing. The way criteria are framed can guide learners’ attention, influence the strategies they use, and highlight particular aspects of writing over others. The pragmatics-oriented rubric, for example, seemed to support greater awareness of the audience, while the genre-oriented rubric encouraged progress in more formal dimensions of writing, leading to different results within the same task. The crossassessment results support this view: students tended to perform better in areas outlined by the rubric they were trained on. The results align with previous research that has underscored the importance of clear, specific, and actionable feedback in fostering metacognitive awareness and self-regulated learning skills among foreign language learners. From a teaching perspective, the study also highlights the importance of designing rubrics that are responsive to both course aims and student needs. Rather than relying on generic, one-size-fits-all performance quality descriptors, instructors may find it more effective to tailor rubrics to specific writing goals and stages of development. For example, as this study has shown, when teaching interactive or real-world genres like blogs, emails, or professional communication, pragmaticsoriented rubrics can help students think more critically about how tone, clarity, and engagement affect their message. The findings of this study also contribute to the growing body of research on self-assessment and writing instruction by providing empirical evidence on how rubric design influences learner outcomes. By highlighting the role of descriptor specificity, the study offers practical insights for educators seeking to enhance the effectiveness of self-assessment tools. Furthermore, the rubrics designed for the research might be used by other educators while developing students’ writing skills. For instance, with minor adjustments, they have already been implemented in a master’s-level course, English for the Sociocultural Agenda, at the Higher School of Economics, where students used them to support the development of news report writing skills.

About the authors

Aida S. Rodomanchenko

HSE University; National University of Science and Technology MISIS

Author for correspondence.
Email: a.rodomanchenko@gmail.com
ORCID iD: 0000-0001-6545-8256
SPIN-code: 7665-6062
ResearcherId: ABD-2284-2021

Ph.D. in Pedagogics, Associate Professor, School of Foreign Languages, HSE University; Associate Professor, National University of Science and Technology MISIS, Department of Foreign Languages and Communication Technologies

20 Myasnitskaya St, Moscow, 101000, Russian Federation; 4/1 Leninsky Ave, Moscow, 119049, Russian Federation

Oleg A. Sokolov

Moscow City Pedagogical University

Email: olegsokolov0501@gmail.com
ORCID iD: 0009-0007-6973-2049
SPIN-code: 7329-8435

postgraduate researcher, Institute of Foreign Languages

4 Vtoroy Selskohoziajstvenny proezd, Moscow, 129226, Russian Federation

References

Supplementary files