Assessing Academic Texts Authenticity in EFL Classes

Full Text

Introduction Academic writing as a new research area in language and teaching research has become a ‘common ground’ for discussing efficient methods to shape a successful researcher in any field (Frumina, 2018; Guzikova, Akoev, 2018). Gradually, integrative nature of academic writing brought to life methods for researchers preparation ‘within the umbrella framework of academic literacy’ (Korotkina, 2018). However, extensive analysis of the process and product of academic writing in English (Voevoda, 2018; Bazanova & Sokolova, 2017; Bogolepova, 2016) does not include sufficient information on the results assessment. Assessment of any course outcomes should be an objective procedure of measuring exactly the competencies a teacher sets long before the course implementation. Focus on traditional criteria of assessing written texts - structure, content or language proficiency alone - may not provide a complex vision of the whole text. Our students may follow laws of structuring, but provide a reader with a dull content, or elaborate on content without structuring the ideas properly. To avoid a one-sided effect on a text production, it is essential to introduce into studies an idea that would unite all the aspects of the text and would explain the necessity to mind all the aspects of text writing at a time, no matter how numerous, strict and complicated the requirements may first seem. Considering academic writing, such a category could be named “authenticity”. By authenticity of an academic text, I mean three essential requirements: 1. providing a reader with an original content/ideas, which is derived from understanding a primary function of an academic text as critical evaluation of the given facts and building new knowledge; 2. following conventions accepted in an academic context, which forms our academic writing competency in general; 3. adopting to culturally determined requirements in the academic context while switching from a native language into a foreign one. While blending native language conventions with newly adopted rhetorical norms of a second language (L2) writing, a writer produces a text within a new cultural context. L2 writing as a specimen of ‘intercultural rhetoric’, since ‘no rhetorical tradition is pure but everything exists between cultures’ (Connor, 2008, p. 312). This paper further narrows Connor’s idea and provides examples of mixing two rhetoric cultures - Russian and English academic writing culture. As research results are supposed to be published or presented for a large professional audience, academic writing should follow generally accepted strict requirements. The most convenient way to categorize them is by context, structure and academic style requirements. Nowadays the ideas of contrastive rhetoric (Robert Kaplan, Ulla Connor, Michael Clyne, John Swales, Natalie Reid) have shifted the focus in academic writing teaching from traditional characteristics to sociocultural ones by introducing cross-cultural analysis of the way all requirements are interpreted by non-native writers. Consequently, the modern course structure should also incorporate sociocultural elements and their description. In order to bring non-native writers closer to authentic patterns, it seems logical to specify the term ‘academic text authenticity’ and introduce the descriptors to assess the category. Statement of the Problem While practicing academic writing, students create text that should a) follow universal academic interaction rules and b) include culturally determined elements of writing in English. New academic environment is also ‘terra incognita’ in terms of native speakers’ expectations. This should be borne in mind while defining ‘academic text authenticity’ as the quality that characterizes how original the text is (content authenticity), how competent the writer and L2 learner is (academic competence) in delivering his/her ideas to the audience, writing in another language (cultural competence). The problem of authenticity has been tackled by researchers from different perspectives. Some scholars focus on the real-life usage perspective or/and the content originality, while others assess formal parameters of an academic text, e.g. analyzing the way the language or/and the sources are used in it. According to Hirvela, Hyland, and Manchón (2016), the main aim of teaching academic writing is to develop students’ research skills. They perceive writing as an opportunity to form the ability to create an original text, and argue that ‘…we should assist our learners in becoming researchers of the texts they will need rather than adopt a single formula for teaching writing’ (p. 51). Synthesis of writing practice and research could be achieved by introducing real-life tasks into the learning process. Meanwhile, Akerblom and Lindahl (2017) assume that ‘there are problems with authentic tasks’ (p. 205). For instance, the definition of “an authentic task” may vary from teachers’ and students’ perspective. Authentic tasks may provide relevance, i.e. make learning motivating because they ‘include ideas, words, phrases and expressions that are heard and read in real-life situations’ (Ciornei & Dina, 2015, p. 275). Authentic tasks are believed to help students become aware of the relevancy and meaningfulness of what they are learning because ‘the tasks mirror real-life experiences’ (Nicaise, Gibney, & Crane, 2000, p. 80). Though authenticity is an important quality, some contradictions still exist in teaching practice. Thus, the authors emphasize the fact that ‘most of the research on authentic classrooms has described the processes teachers have used to develop the classroom environment (learning activities, resources, etc.); however, few have examined authentic classrooms from the students’ perspective’ (Nicaise, Gibney & Crane, 2000, p. 79). Ultimately, Weigle & Parker (2012) percieve authenticity as the quality of an original text that is characterised by correct referencing and respectable use of any material sources with their acknowledgement, stating that ‘source-based writing is becoming more common in tests of academic English, in part to make tests more reflective of authentic academic writing’ (p. 118). A similar veiw might be found in other spheres where authorship is crucially important. Thus, Dr Patricia Akester (2004), analysing authenticity of works in cyberspace, eaquals ‘authenticity’ to ‘academic honesty’. No wonder, further research in academic reading-and-writing interaction indicates that thorough and thoughtful use of sources enables students to display high level of ‘intertextual awareness’, i.e. high level of authenticity (McCulloch, 2013). Additionally, some scholars believe that it is possible to think of authentic texts as of authentic teaching materials, rather than of authentic students’ works. The researchers explain that no matter how well L2 students imitate the models, they still behave the way they do in the same context in their native language culture. ‘Learners can mimic the behavioural patterns of that community derived from the authentic text to a certain extent since the first goal is to communicate and not to behave like someone else which means somehow losing one’s social and linguistic identity’ (Ciornei & Dina, 2015, p. 275). Following this approach, while teaching L2 writing, we accept the existence of some ‘metacommunication’ (Ciornei & Dina, 2015) or texts with ‘intercultural rhetoric’ (Connor, 2008) that would never be 100 % authentic. Logically, a new additional assessment system of students’ works is required to indicate to what extent the initial norms of authentic academic writing are modified due to culture differences. Method The objective of the paper is to track what parameters of authentic texts, while being in teaching focus, may facilitate students’ academic performance. To meet the objective, it is necessary to: 1. define the parameters for assessing students’ texts authenticity; 2. organize a course of academic writing around these text parameters; 3. develop the procedure of measuring students’ academic texts authenticity; 4. measure texts by the elaborated parameters before and after the course; 5. compare the results and report findings. The starting point of a new assessment procedure is the review of the existing ones. Thus, the procedure of Perceived Authenticity in Writing (PAW) is a well-developed tool for investigating students’ perceptions/expectations of text authenticity (Behizadeh & Engelhard, 2014, 2015). However, the objectives of this research include developing assessment criteria for teachers’ perception of text authenticity, since the teachers remain ‘critical text analysts’ who shape L2 writing practice (Gebhard, Demers & CastilloRosenthal, 2008). There really exist some assessment systems with certain descriptors of assessing academic texts. For instance, Academic module of International English Language Testing System (IELTS) provides a system of band descriptors, such as Task Achievement, Coherence and Cohesion, Lexical Resource, Grammatical Range and Accuracy. Meanwhile, the assessment system of IELTS provides little information for cross-cultural comparison of writing norms or conventions in academic communication. Following international experience, the Russian Writing Centers Consortium has also adapted summative assessment scale1. Unfortunately, as well as the international assessment scale, it is not valid for assessing texts authenticity within cross-cultural paradigm. The next step in this research should be to define the descriptors of students’ academic texts authenticity. To understand, what intercultural elements an academic text possesses or/and how the requirements change in a new cultural writing context, it is necessary to define three main criteria. Academic text authenticity includes such elements as 1. creating authentic content, (2) shaping the content according to academic writing context, common for any culture and (3) making mistakes caused by mismatches in English/Russian writing cultures. The first two criteria increase the level of text authenticity, the third one dramatically decreases the level of text authenticity. Consequently, these three criteria, content authenticity, academic literacy and cultural literacy in academic writing context, may be described further by a number of parameters. Content authenticity starts with topicality, one of the essential qualities of an academic text. Additionally, the research done on an urgent problem should be original to make a contribution into the field. Originality might be reflected in different text constituents. Their complex description forms the idea of what content authenticity should be characterized by. A list of content authenticity descriptors may look the following way: o topicality; o original thesis statement; o original arguments that support the facts; o original arguments that beat the facts; o original/catchy title; o original/promising introduction; o original/promising conclusion; 1 Science Essay Competition 2017. Retrieved from http://awuc.misis.ru/science-essaycompetition-2017/ o dialogue of different viewpoints in the content that indicates author’s professional outlook; o dialogue of old and new theories/approaches; o perspectives for further research. The list may be extended by other teachers with some new aspects they would like to focus on in their original writing course. Academic literacy that assists in achieving context authenticity is also a heterogeneous criterion. Definition of its components may assist in assessing student’s autonomy, structuring skills, style of writing and manner of presenting final results after self-editing: 1. Student’s autonomous work: o choosing a topic; o material selection; o formulating a thesis statement; o formulating main arguments that develop a thesis statement; o structuring the text; o self-editing; o submitting the assignment in time; o presentation of work; o working with the tutor’s correction and re-editing; o self-reflection on final results. 1. Academic text structure: o text structure fits the genre; o the universal requirements to academic text are met (topic sentence, framing); o logical arguments organization. 1. Academic style: o academic vocabulary; o professional vocabulary; o high lexical density; o active grammar forms; o variety in syntax structures; o hedging; o avoidance of non-qualified statements. 1. Work presentation: o language competence: use of grammar and vocabulary; o correct title page, referencing and bibliography, etc. Cultural literacy is the criterion that is difficult to limit as culture is everywhere, at any level of the text production. However, it is possible to single out some dimensions to teach and to assess within cultural literacy. The attempts to describe them, for instance, could be found in ‘culturally relevant writing instructions’ (Winn & Johnson, 2011) and genre-focused instructions (Kostrova & Kulinich, 2015). The criterion is revealed through cross-cultural mismatches. Consequently, their number eliminates the quality of text authenticity. In students works there may be frequently found sociocultural mismatches at: o the task interpretation level(e.g. task/thesis/genre mismatches); o the idea selection level (e.g. material should be credible and easy to perceive for potential audience of another culture); o the idea organization level (e.g. linear English writing vs. non-linear Russian writing); o the language choice level (e.g. academic words collocations, active grammar vs. passive constructions, precise syntax vs. complex syntax). Since the descriptors for various aspect of authenticity are defined, a new mechanism to assess them in students’ academic texts should be suggested. The calculation of “academic text authenticity” coefficient (Сata) crucially simplifies the process of cross cultural comparison between a model and a student’s text. To calculate Сata, 50 students taking a course in academic writing were assessed. In every student’s text every variable/ criterion is calculated in an analytical table where every descriptor is specified. Afterwards, the criteria are used to calculate Сata. Formula of text authenticity calculation: . (1) content authenticity + academic literacy = Cata sociocultural mismatches in written communication Formula (1) indicates correlation between all three criteria. Content authenticity (max 60 scores) weighs more than academic literacy (max 40 scores), since the ideas seem to be more important than the way they are shaped in the academic context. The mistakes in a culturally determined written text reduce the value of coefficient Сata. As the formula is used to assess L2 writing, it should be assumed that sociocultural mismatches could not be avoided, and the parameter is never equal to 0. The practice shows that the minimal value of the variable ‘sociocultural mismatches’ is 5, thus, the maximal value of the coefficient Сata is equal to 20 (100:5 in an ideal L2 text). max Сata = 20. To illustrate the way the Сata formula works, examples of paragraphs assessment are given below. Student’s texts are presented in the original. The first and the final drafts are compared to illustrate the quality of a written work in the initial and final stage of learning to write a paragraph. Paragraph: First Draft My objectives of taking academic writing course There are several professional reasons to study an academic writing course. This course is especially useful for such professions as a teacher, a scientist. It helps me as a future teacher to learn how to write and to draw up my lectures, articles, reports and business letters. It also develops my writing skills and helps me to booster my knowledge. This course seems to give me an opportunity to become more successful in my future career. Paragraph: Final Draft Academic Writing Course as a Way to Professional Success One could suggest several professional reasons to study an academic writing course. The course is especially useful for such professions as a teacher and a scientist. It provides a future teacher with experience to write and present essays, articles, reports and to learn more about professional correspondence in educational sphere. Additionally, the organized practice boosters writing skills in researching, evaluating information, logical structuring, arguing and expressing ideas clearly in a paper. These skills are primarily valued by employers (Hughes, etc., 2014) as well as by students focused on professional studies (Bazanova & Sokolova, 2017). Academic writing course seems to give me an opportunity to enhance my employability skills. Works Cited Bazanova, E.M., & Sokolova, E.E. (2017). Massive Open Online Course on Academic Writing: Management of Students’ Motivation to Study. Vysshee obrazovanie v Rossii [Higher Education in Russia]. No. 2 (209), pp. 99-109. Hughes, T., Birkin, G., Durazzi, N. & Brennan, J. (2014). Research and Analysis of the Benefits of International Education Opportunities. A literature review on UK competitiveness and skills (needs) British Council, 35 p. (by Kulagina P., 3rd-year student) Paragraphs Assessment Table 1 Criterion Criterion specification Score (max score) First draft Final draft 1. Content authenticity Score 6 out of 6 2 out of 6 1 out of 6 0 out of 6 2 out of 6 0 out of 6 3 out of 6 23 (max 60) 6 out of 6 2 out of 6 5 out of 6 0 out of 6 5 out of 6 3 out of 6 6 out of 6 5 out of 6 2 out of 6 5 out o f 6 39 (max 60) 2. Academic A. Student’s autonomous work: 0 out of 1 0 out of 1 literacy - choice of topic - material selection 1 out of 1 1 out of 1 - formulating a thesis statement 1 out of 1 1 out of 1 - choice of arguments 0 out of 1 1 out of 1 - structuring the text 0 out of 1 1 out of 1 - self-editing 0 out of 1 0 out of 1 - submitting the assignment in time 1 out of 1 1 out of 1 - presentation of work 0 out of 1 1 out of 1 - work with the tutor’s correction and re-editing 1 out of 1 1 out of 1 - self-reflection on final results 1 out of 1 1 out of 1 5 (max 10) 8 (max 10) B. Academic text structure: 1 out of 3 4 (max 10) 3 out of 3 3 out of 4 3 out of 3 9 (max 10) § topicality § original thesis statement § original arguments that support the facts § original arguments that beat the facts § original/catchy title § original/promising introduction § original/promising conclusion § dialogue of different viewpoints in the content that indicates author’s professional outlook § dialogue of old and new theories/ approaches § perspectives for further research 11. out of 6 12. out of 6 13. out of 6 § text structure fits the genre § common requirements to academic text are met: (topic sentence, framing) § logical arguments organization 17. out of 3 18. out of 4 End of table 1 Criterion Criterion specification Score (max score) First draft Final draft 2. Academic literacy C. Academic style: 1 out of 2 0 out of 2 0 out of 1 1 out of 1 1 out of 2 0 out of 1 0 out of 1 3 (max 10) 2 out of 2 2 out of 2 0 out of 1 1 out of 1 1 out of 1 9 (max 10) D. Work presentation: Score 2 out of 5 1 out of 5 3 (max 10) 15 (max 40) 5 out of 5 5 out of 5 10 (max 10) 36 (max 40) 3. Sociocultural mismatches Score 5 out of 25 (direct, semi-formal tone) 20 out of 25 (no well-grounded argumentation) 0 out of 25 10 out of 25 (General English instead of Academic English, simple syntax) 35 (5-100) 0 out of 25 5 out of 25 (part of the arguments is well grounded) 5 out of 25 (no framing structure) 5 out of 25 (The work has undergone a significant correction after tutorials and consultations) 15 (5-100) Total Score Total: 23 15 35 Total: 39 36 20 § academic vocabulary § professional vocabulary § high lexical density § active grammar forms § variety in syntax structures § hedging § avoidance of non-qualified statements 26. out of 1 27. out of 2 § language competence: use of grammar and vocabulary § correct title page, referencing and bibliography 30. Motivation level (e.g. task/thesis/ genre mismatches) 31. Content level (e.g. material is credible and easy to perceive for potential audience of another culture) 32. Organisation level (e.g. linear English writing vs. non-linear Russian writing) 33. Linguistic level (e.g. academic words collocations, active grammar vs. passive constructions, precise syntax vs. complex syntax) Having formula (1), we may put the variables and calculate the coefficient of the text authenticity in its initial and final versions. 23 +15 C = = 1.08 (Cf. max C = 20). ata1 35 аta 39 + 36 C = = 3.75 (Cf. max C = 20). ata2 20 аta The first text is characterized by low Cаta coefficient caused by insufficient content authenticity. The second text is more stylized. However, it is not a typical indicator of a student’s level; it is a mark of thorough correction and edition within tutorials and consultations. Findings and Results Cаta coefficient was calculated for the texts of the students at different levels of higher education. The parameters vary from 0,2 to 3,7 for bachelor degree students and from 3,8 to 7 for master degree students. Postgraduate students and scientists who start their career create their texts with the “academic text authenticity” coefficient ranges from 7 to 14. Twenty-eight bachelor students specialized in linguistics and twenty-eight bachelor students in foreign language (FL) teaching took an academic writing course in 2016/2017 and 2017/2018 academic years correspondingly. The control group took a traditional academic writing course with the focus on genre conventions and academic style requirements. The experimental group was taught with the focus on sociocultural peculiar features and, additionally, learned cultural patterns of writing in English comparing them to Russian writing style (Chuikova, 2017). Dynamics of Сata coefficient in Academic Writing course Table 2 Groups of bachelor degree students Pre-course mean Post-course mean Linguists 2016/2017 Control Group (14 students) 1.83 2.73 Experimental Group (14 students) 1.93 3.25 Future FL teachers 2017/2018 Control Group (14 students) 1.83 2.73 Experimental Group (14 students) 1.93 3.25 As a result the experimental groups are characterized with strongly marked changes in students’ texts authenticity. It was also estimated that students with high level of language proficiency display higher results due to their overall language competency. However, switching onto another language, they face problems of lexical and grammar density and academic vocabulary use/register similarly to students with lower level of language proficiency. Regardless language competency level, very few students refer to literature review, choosing well-known facts as the most frequent type of argumentation. Autonomous students are initially in a favourable position as they have developed self-editing strategies. The results of the pre-course test highlight the direct link between text originality and the result of academic text authenticity coefficient. The results of the post-course test also reveal correlation between cultural awareness and text authenticity: the better students’ understanding of culturally-determined written elements is, the less sociocultural lapses may be found in their texts, and the higher parameters of their texts are. Conclusions and Recommendations In the paper academic texts authenticity is defined as the quality of text originality and its congruity with the English academic context. The paper provides the algorithm of assessing academic texts authenticity that has been tested in the Russian higher education system. For further implementations, researchers may try to change parameters in variables, namely they may adapt sociocultural mistakes typical for their cultural context. Additionally, the parameters of Сata formula assist a teacher in choosing the material to focus on while teaching Academic writing. The more precisely the researcher defines each criterion and its descriptors, the more focused teaching might be. To sum up, the author agrees with the view that the rise in authenticity parameter ‘cannot be explained through commonly accepted theories of effective cross-cultural adaptation, or adaptation to a host culture. One-way adaptation is not the same as authentic intercultural interaction’ (Fox, 1997, p. 85). Consequently, L2 writing courses can make students’ texts resemble native speakers’ ones. Still this is one of the ways to reach standards of intercultural written communication.

About the authors

Elina S Chuikova

Moscow City University (Samara Branch)

Email: chuikova_elina@mail.ru
Ph.D. in Education, Assosiate Professor, Samara Branch of Moscow City University (Samara, Russia). 76 Stara Zagora St., Samara, 443081, Russian Federation

References