Beyond words in evaluation: Formulaic language in critical reviews of research articles across disciplines

Full Text

1. Introduction

In today’s academic landscape, apprentice second-language scholars face considerable pressure to publish their research in prestigious international journals. The research publication field is highly competitive, and gaining recognition within the disciplinary community requires achieving wide readership and visibility through publication (Paltridge 2019). The visibility of research articles is commonly indicated by an index known as ‘impact factor’, which is calculated based on the number of citations and reader access they receive.

The peer review process plays a crucial role in ensuring the quality of academic publications. Journal editors recruit expert reviewers in the field to act as quality assurance representatives and evaluate submitted manuscripts. Reviewers carefully assess the papers and provide recommendations to the editors, including acceptance, rejection, or suggestions for major or minor revisions. In the case of manuscripts requiring revisions, the reviewers’ feedback is  communicated to the authors, guiding them on the necessary changes that need to be made to their manuscripts. Authors routinely address the reviewers’ comments in order to enhance the quality of their work. Additionally, editors rely on the reviewers’ reports to determine the suitability of a manuscript for publication in the journal (Mungra & Webber 2010). Given the reviewers’ decisive role in determining the publication status of a manuscript (Paltridge 2017), their feedback serves as an important evaluative tool to assess the content and express potentially opposing yet constructive views that align with the standards of disciplinary communities. In essence, feedback from reviewers works as a means of verifying scientific norms and upholding the disciplinary distinctiveness recognized by the members of research communities.

Despite their importance, reviews remain challenging to understand compared to other written genres due to their ambivalent (Samraj 2016) and occluded (Swales 1996) nature. Moreover, despite often including many criticisms and negative comments (Curzon & Cleaton-Jones 2011, Paltridge 2020), their constructive role may be unknown to many novice and nonnative writers. This lack of understanding can be partly attributed to the failure to control the subjectivity of review feedback. In other words, recommendations are sometimes influenced by a reviewer’s expertise or tendencies rather than objective investigations. As a result, comprehending the rhetorical norms of this evaluative genre can be challenging for early-career and nonnative researchers who are required to address disciplinary criticisms in an effective way (Paltridge 2019). This difficulty is exacerbated by the fact that reputable journals in each discipline typically publish research outputs in English. Consequently, writers whose native language is not English may find themselves marginalized and disadvantaged when competing with native-speaker writers.

Another feasible reason could be the unfamiliarity of novice researchers with the linguistic choices commonly employed by expert reviewers, as well as with the discoursal patterns found in the review reports they receive from English-medium journals (Belcher 2007, Loonen et al. 2005, Swales 1990, 1996). It is widely acknowledged that each field of academic language is recognized by a set of linguistic conventions, and writers often make discipline-specific linguistic choices to demonstrate their membership in disciplinary communities (Becher 1994, Breeze 2013, Cortes 2004, 2006, Hyland 2008, Reppen & Olson 2020). One significant linguistic choice in constructing discourse is the degree of formulaicity employed by writers and speakers through the use of formulaic bundles. These are “words which occur together more frequently than expected by chance, helping to shape text meanings and contributing to our sense of distinctiveness in a register” (Hyland 2008: 5).

Formulaic or lexical bundles are widely recognized as the most frequently used lexico-grammatical devices within a specific register, selected based on certain criteria. Their appropriate and adequate use is considered not only an indicator of language proficiency (Wei & Lei 2011) but also a marker of membership in a particular discourse or disciplinary community (Salazar 2014). Therefore, examining reviewers’ discourse choices associated with and realized through formulaic bundles is of paramount importance for those engaging in English for academic and publication purposes as it can shed light on the discoursal and organizational patterns of this genre of criticism. Furthermore, identifying potential similarities and variations in the use of formulaic language by reviewers from different disciplines is beneficial for novice researchers in those fields. It helps them understand how reviewers rely on prefabricated and familiar linguistic clusters to examine and criticize research articles in accordance with disciplinary conventions.

Numerous studies have provided evidence that formulaic language sequences differ across genres (Biber & Barbieri 2007, Biber & Conrad 1999, Biber et al. 2004, Breeze 2013), across disciplines (Cortes 2004, Hyland 2008, Kashiha & Chan 2013, Kashiha & Chan 2014a, 2014b, Reppen & Olson 2020), and between native and nonnative English speakers (Adel & Erman 2012, Chen & Baker 2010, Karabacak & Qin 2013, Kashiha 2015, Kashiha & Chan 2014c, 2015, Nekrasova 2009, Pan & Liu 2019, Shin 2019). However, upon reviewing previous studies, it is evident that no study to date has specifically examined how these bundles are perceived and employed in the genre of criticism, as represented by reviewers’ feedback on submissions of L2 early-career researchers to internationally reputable journals across different disciplines. Besides, existing body of research into review feedback has mainly concentrated on analyzing linguistic features other than formulaic bundles, such as stance and engagement (Paltridge 2017, 2019, 2020), or the distinctions between review reports in different cultural contexts, such as English and Russian (Larina & Ponton 2020, 2022). Thus, this study aims to address this oversight by examining how reviewers in the fields of AL, engineering, and business employ formulaic bundles to construct their evaluative discourse, both in similar and distinct ways. The study also seeks to confirm that research articles should be organized and written in a manner that fits within the disciplinary norms and practices of the respective field, as indicated by reviewers’ feedback. The following research questions are formulated to guide the study:

1. To what extent do reviewers in the fields of AL, engineering, and business use formulaic bundles to construct their evaluative discourse?
2. How do the structures and functions of these bundles compare or differ in the reviews of the three disciplines?

2. Literature review

2.1. Formulaic bundles and their importance

Words in isolation have been reported to fit within larger multiword combinations in which their frequency and function vary from those of individual parts that make them up (Howarth 1998). The fact that formulaic sequences appear in everyday language use and constitute a substantial portion of discourse in all genres has led to an increasing interest in investigating the formulaic nature of language over the past two decades. The formulaicity of language is determined by the use of lexical bundles, which are defined by Biber et al. (1999) as extended collocations or combinations of three or more words that occur in discourse more frequently than expected by chance. These bundles are identified through corpus-informed analysis of language and they emphasize idiomatic, lexicalized, and memory-based language acquisition. Although Altenberg (1993) was a pioneer in establishing a specific method for identifying formulaic bundles, many scholars have subsequently investigated different aspects of formulaic language use by various users and in different contexts. The majority of studies in this area have indicated that formulaic bundles play a crucial role in psychology and socio-functional learning (Pawley & Syder 1983, Sinclair 1991). It is also believed that formulaic bundles are often stored in memory holistically as a cluster, aiding faster processing during retrieval (Schmitt et al. 2004). Tremblay et al. (2011) focused on self-study reading materials to examine the pace of reading and learning formulaic bundles and found that students process and read sentences containing these bundles faster than those containing non-formulaic units. When analyzing the structures of formulaic bundles, Jeong and Jiang (2019) suggested that, in addition to having a high density, a complete grammatical structure plays a role in the speed of bundle processing.

The socio-functional significance of formulaic bundles, which serves as the theoretical framework of this study, is informed by the fact that using these bundles can display a writer’s membership in a particular academic and disciplinary community (Wray 2013). Academic communities often have their own ways, including linguistic choices like formulaic bundles, of presenting and negotiating propositional information that is scientifically grounded in the discursive patterns and knowledge transfers of their disciplines. Through these discipline-specific tendencies, writers not only demonstrate uniformity with other members of their community (Cortes 2006) but also express their ‘voice’ and offer a credible representation of themselves (Hyland 2008). Such shared approaches to using language may help writers to move beyond simple tendencies by focusing on more complex qualities of their disciplines like gate-keeping standards which, if not adhered to, can lead to exclusion and marginalization.

2.2.  Formulaic bundles and their variation

While formulaic bundles are closely connected to the natural and idiomatic use of language, there have been several reports as to their variations across different registers and communicative events (Howarth 1998, Meunier & Granger 2008). For example, Breeze (2013) found that specialized bundles constitute a significant portion of the discourse in legislative contexts. Similar results have been reported in spoken contexts, such as medical (Grabowski 2015) and mathematics (Herbel-Eisenmann et al. 2010) classrooms. In one of the studies on the distinct registeral preferences for the use of formulaic bundles, Biber et al. (1999) found that the frequency and structure of these bundles differed in the discourse of conversation and academic prose, with conversation showing a greater tendency to such prefabricated sequences. Biber et al. (2004), in a follow-up study, reported that formulaic bundles were more predominantly used in the classroom teaching registrar than in textbooks, daily conversation and academic prose. The four registers also showed substantial variations in terms of the structures and functions of the bundles used.

Another line of research has concentrated on variations between disciplines in the use of formulaic bundles. Hyland (2008) examined the deployment of these bundles in four university disciplines: AL, business, biology, and electrical engineering, representing hard and soft science disciplines. He found that electrical engineering writers deployed the largest share of bundles compared with other disciplines. While most of the bundles used in AL and business were prepositional phrases, those in electrical engineering and biology were predominantly passive structures. In terms of bundle functions, the soft disciplines made greater use of stance and text-oriented bundles, whereas the hard disciplines exhibited relatively greater use of research-oriented bundles. Cortes (2004) compared and contrasted biology and history and found that the function of hedge or mitigation was more dominant in biology texts than in history texts. Another variation observed was that while biology writers employed a wide array of bundle structures, history writers primarily used noun and prepositional phrase bundles. In a broader perspective, comparing nine disciplines, Reppen and Olson (2020) found only nine shared bundles across all disciplines, with almost 85% of the retrieved bundles being exclusively used in only two disciplines. Furthermore, the shared bundles primarily functioned as textual indicators or frame markers, whereas the unique bundles served discipline-specific functions.

Another factor that has been reported to impact the use of formulas is the writer’s knowledge and competency. Pan and Liu (2019) looked at the use of bundles in published research articles and MA theses written by native and nonnative writers in AL and found that both groups of writers employed more bundles when writing theses compared to research articles. Moreover, the most and least frequently used functions of bundles in the two datasets were text-oriented and stance, respectively. Regarding bundle structures, theses contained more phrases and fewer clauses than published articles. In a similar study by Wei and Lei (2011), it was found that Chinese PhD students were more inclined to use formulaic bundles in their doctoral dissertations compared to expert writers in writing research articles. Participant-oriented and passive bundles were also more prevalent in PhD dissertations than in articles. In contrast, the study by Chen and Baker (2010) reported a higher frequency of bundle use in research articles written by experts compared to the writing of native and nonnative students.

3. Data and methodology

3.1. Corpus compilation

The corpus used in this study consists of 120 reviewers’ reports on the manuscripts submitted by Iranian junior researchers in AL, engineering, and business. They were 40 reports from each discipline. The manuscripts that were analyzed were full-length “research articles” following Swales’ (1990: 134) established empirical article format (AIMRD) and were submitted to reputable ISI- and/or Scopus-indexed journals that publish in English (refer to the Appendix for the list of journals in each discipline and the number of reviews obtained from each journal). The manuscripts represented the researchers’ initial attempts to target prestigious journals and were written by either one writer or two to three writers, with some business and engineering manuscripts even involving up to five writers. In some cases, the same researcher or group of researchers authored multiple articles that were submitted to and reviewed by different journals (4 cases in AL, 3 cases in engineering, and 5 cases in business). Additionally, all manuscripts underwent review by at least two anonymous reviewers, and in some cases, three or four reviewers, due to conflicting decisions received by the journal editors from the initial two reviewers. All journals followed a double-blind review procedure, ensuring that the identities and affiliations of both the authors and the reviewers were kept confidential to avoid potential conflicts of interest.

To ensure the relevance and focus of the corpus, only reviews recommending major or minor revisions were selected. This decision was made because review reports in these categories tend to be longer than those recommending rejection or acceptance, thus providing more examples of naturally occurring language constructions that could offer a clearer account of how formulaic language is employed by reviewers. Moreover, only reports from the first round of reviews were included in the analysis due to limited access to a sufficient number of second-round reviews and in accordance with Belcher’s (2007) agreement that first-round reviews typically contain more criticisms and judgmental arguments for authors to consider. It was also observed that all collected reports were independently written by reviewers without following any checklist or template.

To confirm that the selected journals were relevant to the fields under investigation, expert university scholars in each field were consulted, and they confirmed that the chosen journals are among the leading publications in the respective fields. The selection of these three fields for analysis and comparison was motivated by their diverse nature according to Becher’s (1994) classification of fields, aiming to capture a broad cross-disciplinary perspective on evaluative academic practices. Since these fields represent different scientific and disciplinary domains, with AL belonging to the humanities, business to the social sciences, and engineering to the hard/applied sciences, they exhibit distinct understandings, values, and reader expectations within their disciplinary communities. Therefore, exploring potential variations in the type of lexico-grammatical features used by reviewers, who are expert representatives in each field, would provide valuable insights into knowledge transfer for apprentice researchers in the respective fields. Additionally, conducting comparative studies of this nature contributes to a deeper understanding of disciplinary variations in English for Academic Purposes (EAP), shedding light on the specific language practices within different academic domains. Table 1 presents detailed information about the corpus.

Table 1. Description of the corpus 

To ensure a parallel comparison across the subcorpora with varying word counts, the overall frequency counts in Table 2 were normalized to occurrences per 1,000 words. This normalization was achieved by dividing the raw frequency count by the number of words in the text and then multiplying by 1,000, following the approach proposed by Biber et al.’s (1998). This approach further helps to control for frequency discrepancies and facilitates comparison across the subcorpora of different sizes. The researcher ensured that all relevant details regarding the writers and their co-authors, such as their first language background, the structure of their submitted manuscripts, and the type of recommendation they received on their submissions, were collected by directly obtaining the review reports from the first authors or corresponding authors. The researcher assured the authors that their credentials would remain concealed when seeking permission to include the reviews for analysis.

3.2. Identification and classification of formulaic bundles

Several selection criteria were employed to identify and classify formulaic bundles in the corpus. Firstly, only four-word strings were considered formulaic in this study, as “they are far more common than 5-word strings and offer a clearer range of structures and functions than 3-word bundles” (Hyland 2008: 8). More specifically, there is a consensus among researchers to use four-word bundles because they already include three-word bundles, and longer strings (5 to 9 words) are relatively less frequent in the corpus (Breeze 2013, Cortes 2004). The frequency and dispersion of bundles were also taken into account as selection criteria. Adapting Hyland’s (2008) threshold, only strings that occurred at least 10 times per hundred thousand words and appeared in at least 10% of the corpus were considered formulaic and included for analysis. These criteria were applied to avoid any idiosyncratic effects imposed by individual reviewers or groups of reviewers.

The concordance software WordSmith Tools 4 (Scott 2008) was utilized to generate a list of bundles based on the specified selection criteria. The final list of bundles underwent manual analysis to determine their structural and functional categories. Biber et al.’s (1999: 1014–1024) taxonomy was used for structural categorization, which identified eight grammatical forms for formulaic bundles:  1) noun phrase + of, 2) other noun phrases, 3) prepositional phrase + of,  4) other prepositional phrases, 5) passive + prepositional phrase, 6) anticipatory  it + verb/adjective, 7) be + noun/adjective phrase, and 8) others.

Among the existing functional taxonomies for formulaic bundles, the one proposed by Hyland (2008) was drawn on because it encompasses a wider range of discourse functions that are relevant to written academic discourse. Hyland’s taxonomy assigns formulas to three main functional categories: research-oriented, text-oriented, and participant-oriented categories.

1. Research-oriented bundles: These bundles allow writers to emphasize their research endeavors and their experience in the natural world. There are five subcategories within this category:

• Location: Indicates both time and place (e.g., ‘at the beginning of’, at the end of’).
• Procedure: Describes the employment of a particular approach (e.g., ‘the employment of the’).
• Quantification: Involves expressing numerical or quantitative measures (e.g., ‘a number of the’, ‘a wide variety of’).
• Description: Provides information about the characteristics of something (e.g., ‘the nature of the’).
• Topic: Focuses on the main issue for topic being discussed (e.g., ‘the main issue with’).

2. Text-oriented bundles: These bundles reflect the writer’s efforts to organize their discourse, frame their text structure, and convey meaning effectively. There are four subfunctions within this category:

• Transition devices: Establish additive or contrastive connections between ideas (e.g., ‘in addition to this’, ‘on the other hand’).
• Resultative devices: Construct rational and causal links between arguments (e.g., ‘is due to the’, ‘as a result of’).
• Structuring devices: Refer to specific parts of the discourse and guide readers through the text (e.g., ‘on the page X’, ‘in the following sections’).
• Framing devices: Position propositions within the discourse by indicating restrictive circumstances (e.g., ‘with regard to the’, ‘in terms of the’).

3. Participant-oriented bundles: These bundles indicate the writer’s attitude and evaluation toward propositions and the audience. There are two subcategories within this category:

• Stance markers: Signal the writer’s attitude and evaluation toward propositions (e.g., ‘I don’t know if’, ‘is likely to be’)
• Engagement markers: Interact with the audience and draw them along with the arguments (e.g., ‘as you are aware’, ‘you may know that’).

During the discourse analysis phase of the study, each token of the identified bundle types was examined carefully within its contextual meaning to determine the specific subfunction it served. This analysis involved studying the concordance lines in the three datasets to understand how each bundle contributed to the disciplinary evaluative practices.

4. Results and discussion

The analysis found 76 different types of formulaic bundles that met the identification criteria in the corpus. Additionally, there were 2065 individual instances of these bundles, accounting for 2.8% of the total words in the corpus. The top 5 most frequent bundles, which constituted nearly 8% of all bundles in the corpus, included: ‘in terms of the’, ‘in the case of’, ‘that there is a’, ‘is one of the’, and ‘the results of the’. The frequency analysis also indicated that the top 20 bundles comprised 18.3% of the total bundles and 0.7% of the total running words. Table 2 illustrates the distribution of bundles across the three disciplines. As can be seen, AL had the greatest variety of bundles, with 72 different bundles occurring 9.04 times per 1,000 words. Engineering had the fewest number of different bundles, while business had the smallest range of individual bundle cases (tokens). Furthermore, there were some bundles exclusively used in AL reviews that were not found in the other fields. These results suggest that AL reviews tended to be more formulaic, with AL reviewers displaying a stronger proclivity for employing prefabricated patterns of language when communicating their comments to authors, compared to their counterparts in other fields. It is challenging to discuss the potential reasons behind such variations from a discipline/research point of view because this study does not focus on analyzing the original research papers in these fields. However, one possible reason could be the distinct ways in which review reports are written and organized in each field.

Table 2. Frequency distribution of bundles in the three disciplines

To gain a better understanding of the possible similarities and differences among the three review datasets in terms of formulaic language usage, the study also calculated the number of bundles that were unique to each discipline as well as those shared by two or all three disciplines. Out of the 76 identified bundles, 34 (45%) were found to be shared across all three disciplines. Moreover, there were 11 unique bundles in AL, 8 in business, and 6 in engineering. The remaining bundles were shared by two of the three disciplines, with business and engineering having more shared bundles than any other pairs.

Table 3 presents the top 20 bundles in each discipline, with unique bundles highlighted in bold and those shared by two disciplines shown in italics. The fact that nearly half of the bundles were used in all reviews, regardless of discipline, suggests that reviewers rely on similar and familiar prefabricated expressions when making research-related arguments. For instance, they used ‘the results of the’ to connect their comment to the paper’s findings or ‘as shown in the’ to draw attention to a specific point in the manuscript. Engaging with authors during the review process and referencing different sections of their manuscripts, therefore, seem to be common practices when providing evaluative feedback. The following subsections provide a detailed analysis of the structures and functions of formulaic bundles in disciplinary peer reviews, accompanied by examples extracted from the corpus.

Table 3. The 20 most frequent formulaic bundles in the three disciplines

* (bold = unique bundles, italics = shared by two disciplines)

4.1. Grammatical structures of formulaic bundles

The analysis of bundle structures revealed that, similar to previous studies on academic prose (Chan et al. 2014, Hyland 2008, Kashiha & Chan 2013, Kashiha & Chan 2014b), the majority of bundles used in the reviews of the three disciplines were in the form of a noun or prepositional phrase, accounting for more than 80% of the total bundles in each subcorpus. Table 4 illustrates the distribution of bundle structures in the three disciplines.

As for variations, AL reviewers exhibited a greater tendency to use the prepositional phrase + of bundles. The analysis of bundles that begin with prepositional phrases suggests that these bundles, often incorporating of-phrase, are predominantly employed to establish links between reviewers’ propositions (1&2) or to guide authors in connecting their propositions to other relevant aspects of their research (3).

(1)  understanding the way that academic discourse is organized is of great value especially in the case of spoken discourse (AL)

(2)  … and test interventions targeting those moderators in the context of retail apps. (Business)

(3)  I suggest to cite these and discuss new data in relation to the previous studies. (Engineering)

Table 4. Structures of bundles in the three disciplines (%)

The most frequent bundle form in business and engineering was noun phrase + of (31% and 29% respectively), while it was the second most common structure in AL (25%), although the actual proportion (raw frequency) was somewhat similar across the disciplines. This indicates the significance of this grammatical structure in establishing formulaic relations between reviewers and writers. By utilizing concordance software to examine the context in which such bundles contributed to feedback, it was observed that reviewers typically used them to associate their feedback with various research-related elements such as quantity (4), quality (5), or existence (6). The research characteristics of the business and engineering fields, which involve more quantitative measurements and size considerations, could explain the higher occurrence of noun phrase + of bundles in the reviews of these disciplines. These lexico-grammatical choices by the reviewers reflect their intention to present and negotiate their comments in ways that align with the ideologies of their respective disciplines. These choices are also characterized by adopting varied commenting styles. As such, comments in AL were found to follow a point-by-point approach, which necessitated a higher usage of prepositional phrase bundles to establish connections between points. On the other hand, business and engineering reviewers adopted a section-by-section approach for commenting, which required more transitions between sections using relevant headings.

(4)  There are different types of microbial fuel cell, not least of which is the size of the anodic chamber. (Engineering)

(5)  Please provide an elaboration here, as the role of the context has not been discussed before this. (AL)

(6)  We see a large number of major points arising from the paper that are of greatest relevance to policymakers (Business)

Although bundles with a passive structure followed by a prepositional phrase fragment were infrequent in the corpus, their purpose was to direct the author’s attention to a specific part of their manuscript, such as a figure (7) or table (8), or to justify a disciplinary argument and provide a basis for it (9).

(7)  Authors stated the HTMT values are below 0.85 but as is indicated in Figure 2, one of the numbers showed more than 0.85. (Business)

(8)  As is shown in Table 2, the terms "Raw No." and "Norm." are used. (AL)

(9)  The results you get with Pseudomonas can be due to the soluble mediators. (Engineering)

Using a passive structure preceded by a modal auxiliary (as in Example 9) to convey unfolding arguments is a sign of mitigating language, which is a common practice in disciplinary knowledge transfer. It has been reported that the level of such mitigation, which involves toning down claims, tends to increase as we move from hard knowledge fields to soft knowledge fields in the writing of disciplinary research articles (Hyland 2008). However, the limited use of bundles featuring mitigation in all review texts, regardless of discipline, can be justified by the focus of this study, which primarily analyzed reviewers’ feedback rather than the writers’ original research articles. Therefore, reviewers might have employed other language structures to express their important disciplinary concerns that are not adequately addressed in the manuscript. This conjecture can help differentiate evaluative and critical academic genres from other types of academic genres.

Another way of presenting claims through prefabricated language was by using anticipatory-it bundles, which were more pervasive in the reviews of business and engineering compared to AL. This can be partly explained by the nature of the topics under review in business and engineering, which often require stronger claim-based criticisms from their referees. While AL reviewers employed two identified anticipatory-it bundles (‘it is recommended that’, ‘it is important to’) to draw the author’s attention to the comment, business and engineering reviewers used other bundles with this structure to signal their certainty and authorial stance toward the topic (‘it is clear that’, ‘it is a fact’). These variations in the use of a particular structure to fulfill a discipline-specific function may be attributed to the different ways in which arguments and comments are presented and categorized in the reviews of the three disciplines. Such varied evaluative linguistic choices can raise awareness among novice reviewers, especially those whose native language is not English, so that they can provide a plausible basis for their claims and produce more convincing feedback through effective formulaic language structures.

4.2. Discourse functions of formulaic bundles

Upon analyzing the discourse functions of bundles, it became evident that there were connections between the structures and functions of bundles, that is, reviewers often relied on a specific grammatical form to serve a particular communicative function. As such, prepositional phrase patterns were predominantly used for participant-oriented functions, while anticipatory-it patterns were commonly employed for text-oriented functions. The research-oriented function showed a nearly equal distribution between noun and prepositional phrases. These findings align with previous research on various types of academic texts, including research articles, dissertations, theses, and textbooks, across different disciplines (Hyland 2008, Shirazizadeh & Amirfazlian 2021).

Table 5. Discourse functions of bundles in the three disciplines (%)

Table 5 illustrates the frequency distributions of bundle functions in the three disciplines. It is evident that text-oriented functions were most prevalent in AL, constituting nearly half of the bundles in this subcorpus, while ranking second in the other two disciplines. On the other hand, research-oriented functions were the leading category in business and engineering reviews, accounting for 48% and 52% respectively, but were the least frequently employed in AL, comprising only 12% of the total bundles. Identifying these disciplinary variations in evaluative linguistic choices provides insights into the field-specific evaluation and criticism standards of the disciplines by shedding light on how members of disciplinary communities organize their evaluative discourse and establish connections with their intended target audiences through diverse discourse decisions.

4.2.1. Research-oriented bundles

One notable aspect of the frequency counts of bundle functions was the higher degree of formulaicity observed in research-oriented expressions in engineering and business reviews compared to AL reviews. This can be justified by the intrinsic scientific and technical characteristics of these disciplines, leading their reviewers to rely more on prefabricated research-related expressions. Reviewers typically used research-oriented bundles to critique various research aspects of the reviewed study, particularly its methods and results (10–12), in order to make an empirical impact on the writer as the primary reader of their report. Therefore, engineering and business reviewers may have relied on a higher proportion of these bundles to establish their authoritative voice, disciplinary persona, and showcase their expertise as academics immersed in the practices of their respective fields. On the other hand, AL reviewers might have employed different linguistic features to convey a similar message or achieve similar evaluative functions.

(10) It is not true to use Johansen test procedure for this application because this test can be used to show that all the series have the same integration order (Business).

(11) It is not clearly used to isolate Pseudomonas-like species, to show the presence of the inoculum within the system (Engineering)

(12) They are said to be used to structure discourse, but structuring discourse is a broad topic and .... (AL)

Among the subfunctions of research-oriented bundles, the procedure subfunction was the most frequent in all three datasets. This reflects the significant and prevalent research-related issue faced by Iranian writers in the investigated disciplines, highlighting the need for criticism regarding the procedural aspects of their work, such as research methods, experimental design, and analysis procedures.

Reviewers in engineering and business found the research description of manuscripts to be more contentious compared to other research-oriented aspects, as evidenced by its second-ranking position in these disciplines. It occurred nearly three times more frequently in their reviews than in AL reviews. This discrepancy can again be accounted for by the distinct characteristics of the disciplinary papers under review or the varying levels of proficiency or credibility of the authors. Comments that included description bundles aimed to draw the authors’ attention to issues related to the scope of their research or to instances of excessive or inadequate language use when describing their research design. Examples include:

(13) It is important to note that the nature of the test itself plays a significant role in our understanding of …, which is not discussed in the paper. (AL)

(14) The business model used must move beyond a primary focus on … because this forms the basis for the research conducted. (Business)

(15) It's been barely attempted to discuss the implementation of the model in any great seriousness which made the results unclear. (Engineering)

The data analysis also revealed that engineering reviewers utilized slightly more quantification bundles (16) compared to their counterparts in other disciplines. The remaining subfunctions of research-oriented bundles, namely location and topic, were marginally more common in business than in other disciplines. However, it is important to note that these subfunctions were sparingly used in the corpus and had almost similar raw numbers across the three disciplines. The few location bundles, such as ‘at the end of’ or ‘at the beginning of’, worked to direct the author’s attention to specific parts of their manuscript that required further consideration or improvement. Topic bundles were used to refer to discipline-specific concepts, such as ‘of foreign language acquisition’ in AL or ‘antarctic sea ice ecosystems’ in engineering.  

(16) This is a weakness as we see a large number of unique advantages of mechanical systems compared to JJ technology. (Engineering)

4.2.2. Text-oriented bundles

In contrast to the frequency results of research-oriented bundles, AL reviews were dominated by text-oriented bundles, accounting for 45.8% of all bundles. This occurrence was significantly higher compared to the relatively fewer and almost equal occurrences in business and engineering reviews respectively. In a study comparing the use of bundles in research articles of various disciplines, Hyland (2008) also reported a higher occurrence of text-oriented bundles in AL, comprising two-thirds of all four-word strings. As he notes, this disciplinary preference may mirror the more elaborate and discursive nature of communicating propositions in a soft knowledge field like AL, where both positive and negative arguments are more persuasively expressed and interpreted.

Similarly, the heavy deployment of text-oriented bundles in AL reviews serves to introduce familiar and prefabricated linguistic ways that connect ambivalent elaborations and a means to scaffold the review text. This aids in guiding authors through the review process, creating more textually cohesive and persuasive review feedback. Such preference can partially explain the far more frequent utilization of framing signal bundles, among other text-oriented subfunctions, in AL reviews compared to those in the other two disciplines (see Table 5). Framing signal bundles were generally used in peer reviews to organize propositions by establishing links between arguments or postulating complementary elaborations, as in:

(17) In case of the isolator and circulator and having a detailed and matching theory… (Engineering)

(18) The quality of this paper in terms of the research and writing cannot satisfy the requirement of the journal. (AL)

(19) I have some concerns with regard to the incremental contribution of this work (Business)

The analysis revealed further disciplinary dispositions in the use of bundles and their associated discourse functions. Engineering reviewers tended to utilize more bundles featuring transitions, indicating their preference for providing smooth transitions between different sections or ideas both in their feedback and in the reviewed manuscript. On the other hand, business reviewers dominated in using resultative signaling bundles when commenting on the results of papers.

Another distinct commenting style between AL and the other two disciplines was observed in the use of structuring signal bundles. While these bundles were collectively employed in all three subcorpora and were more favored by AL reviewers, the way they were used differed among reviewers. In AL reviews, the majority of these bundles worked to criticize the general research components of a manuscript, without referring to specific sections, such as using bundles like ‘in the present study’ or ‘in the current study’ (20). These helped shape the arguments in a clear and organized manner. On the other hand, in business and engineering reviews, a considerable portion of structuring signal bundles helped provide a frame for suggesting (21) or criticizing the lack of a particular research aspect (22), by referring to the respective section of the manuscript. This suggests that review reports in these two disciplines followed a section-by-section approach. In general, these variations in associating bundles with discourse functions can again confirm the diverse commenting and criticizing styles in different disciplines and reflect the reviewers’ awareness of the discursive norms in presenting their disciplinary arguments and criticisms, as well as their community-specific rhetorical expectations.

(20) However, how it is operationalized and implemented in the present study is not clear. (AL)

(21) In the results section the outcome of the accuracy and precision investigation can be presented with the help of tables and figures (Engineering)

(22) Several statements in the introduction section are left without proper citation (Business)

4.2.3. Participant-oriented bundles

Participant-oriented bundles comprised the lowest proportion of bundles in business and engineering reviews, while ranking second in AL reviews. This mirrors previous studies on written academic genres (Hyland 2008, Shirazizadeh & Amirfazlian 2021, Wei & Lei 2011), suggesting that written discourses tend to have fewer instances of establishing writer-reader interactions compared to spoken discourses. In the corpus of peer reviews in this study, the double-blind feature of peer reviews, where the identities of both the reviewer and the writer are undisclosed, could contribute to the lower occurrences of reviewer-writer interactions. Another feasible reason is that reviewer stance and writer engagement might have been expressed and conveyed through means other than four-word bundles.

The majority of participant-oriented bundles in the three review sets served as engagement devices, reflecting the reviewers’ attempts to engage with the authors and acknowledge their presence throughout the review text. This is in contrast to previous studies on research articles, textbooks, theses, and dissertations (Hyland 2008, Shirazizadeh & Amirfazlian 2021), as well as the writing of advanced students (Wei & Lei 2011), where participant-oriented bundles mainly functioned as stance-taking devices. This divergence in function can be related to the different nature of the discourses under investigation. In peer review, the fundamental purpose is to improve the quality of a manuscript and ensure that writers can construe and address disciplinary comments. This highlights a greater need for effective engagement strategies through familiar and prefabricated phraseological patterns.

Despite being used sparingly in the corpus, AL reviews made use of participant-oriented bundles almost twice as often as business and engineering reviews. These bundles often functioned as directives, addressing the writers using audience pronouns like ‘you’ or ‘the author’ to pull them along with the comment and capture their attention, as in:

(23) I suggest you to move to the introduction the explanation of "adopter" vs. "switcher". (Business)

(24) I suggest the author to insert in table 1 an extra column to report the sources for each item (Engineering)

(25) …that you need to revisit the entire paper and start thinking … (AL)

One attention-grabbing strategy typical of AL reviews was the use of bundles containing judgmental adjectives to emphasize certain points or actions within the manuscript, such as ‘it is important to’ or ‘it is necessary to’. Reviewers occasionally softened the degree of obligation by inserting a modal auxiliary in an engagement bundle, aiming to open up a discursive avenue for potential rebuttals from writers, especially in the results section (26). This approach was intended to make writers feel that their knowledge of the obtained findings was acknowledged, even amidst criticism.

(26) You may want to prove this by doing some quantitative analysis. (AL)

However, unlike in other research writing genres like research articles and theses, where writers often leave room for dialogue and alternative views from readers, no instances were found in the corpus where a reviewer explicitly provided evident avenues for anticipating and acknowledging absolute disputes with the submitting writers, especially when commenting on the literature review and methodology sections. This can be justified by the judgmental nature of peer reviews and the authoritative role of reviewers in validating and defending the theoretical and methodological aspects of their discipline when evaluating these sections.

It is also noteworthy that some business and engineering reviewers showed a greater disposition for using bundles with WH-questions to indirectly prompt the writer to notice an issue in their manuscript, thereby addressing it, as in:

(27) What do you mean by "different consumers?" (Business)

(28) Why did you not analyze the probe measurements statistically? (Engineering)

The other subfunction of participant-oriented bundles, stance features, displayed the reviewers’ endeavor to explicitly or implicitly appraise propositional information and balance their claims through the use of personal or impersonal stance bundles. When providing feedback on the results section, reviewers collectively demonstrated a tendency to withhold full commitment to what they were saying, by employing strategies such as mitigation and face-saving to avoid making potentially incorrect or exaggerated arguments. This approach of minimizing personal involvement when discussing the results, often achieved through impersonal or hedging bundles, can be explained by the fact that in scientific manuscripts, authors typically possess a higher level of expertise regarding their research findings than the reviewers, no matter their inexperience as academic writers. As a result, reviewers are less likely to adopt judgmental stances or present subjective and biased claims using personal evaluative stance bundles. This explains why the majority of the bundle realizations of reviewers’ stance across the three disciplines were impersonal, incorporating either an epistemic adverb (29) or a modal (30).

(29) the geometric mean of chl-a concentrations is more likely to result from optical measurement differences (Engineering)

(30) This finding may be due to a lot of factors beyond control. (Business)

In contrast, the usage of personal stance bundles was minimal and served to emphasize the reviewers’ certainty by reinforcing the strength of their arguments (31). The limited presence of personal stance-taking bundles can again be due to the possibility that reviewers may have relied on alternative lexico-grammatical means, aside from four-word clusters, to convey such behaviors.

(31) I definitely think that this is so because the authors have only considered % figures. (AL)

5. Conclusion and implications

This study aimed to examine the role of formulaicity in shaping the evaluative discourse of reviewers belonging to different disciplines. By analyzing the frequency, structure, and function of the most frequent four-word formulaic bundles used in the reviews of AL, business, and engineering manuscripts authored by early-career Iranian writers, the study found results that comparatively align with previous cross-disciplinary and cross-generic research on formulaic expressions, with some variations in the ways they were employed. The analysis in this study revealed that reviewers rely on a wide variety of formulaic and familiar bundles to articulate their criticisms, engage with writers, and invite them to address the necessary changes. Some bundles were exclusive to specific fields, while others were shared between two or more fields. These findings shed light on the interplay between commonalities and uniqueness in evaluative written discourse, as bundles used exclusively within a specific discipline or commonly employed across disciplines can demonstrate both discipline-specific tendencies and commonalities. However, the study emphasized the prevalence of commonalities due to its focus on a similar genre, and the fact that all review reports, regardless of discipline, serve the same purpose of enhancing the quality of submitted manuscripts, leaving limited room for flexible formulaic choices made by reviewers.

It is important to note that this study did not analyze the original research papers in the three fields, and therefore, any generalizations about the writing and publication needs of researchers in these fields should be avoided. The relatively small corpus size also warrants caution in interpreting the study’s results. Further examinations encompassing different fields and language backgrounds of writers are needed to provide novice researchers with generalizable insights into making informed discourse decisions that are rhetorically grounded in the writing patterns and research practices of their respective disciplines. Additionally, while the identified bundles in this descriptive study may not directly inform pedagogical implications, future research into EAP can explore their practicality by seeking feedback from EAP practitioners, tailoring their content to meet the evolving needs of EAP audiences and syllabi.

Despite the interpretational limitations, the findings of this study contribute significantly to the field of English for academic and publication purposes by challenging the conventional assumption that disciplinary discourse is primarily characterized by a single specialized lexicon rather than formulaic chunks. EAP syllabus designers and teachers, when considering the contextualized target students as a starting point for instruction, need to recognize that formulaic language may serve different functions in various contexts and fields. They can rely on corpus-driven lists of frequently used and meaningful bundles extracted from the registers that students need to learn, along with the specific contexts in which these bundles are employed, to develop pertinent learning materials for their courses.

This study demonstrated how four-word bundles served as fundamental linguistic units and their frequency reflected the linguistic choices made by peer reviewers from different disciplines in constructing their evaluative discourse in a formulaic manner. Familiarity with these discipline- and genre-specific conventions can also benefit novice reviewers in the fields under investigation, as well as scholars aiming to enter the peer review community. The study highlights the connection between rhetorical norms within each field, including the use of familiar multi-word patterns, and the level of criticality that should be applied to reviewers’ claims. Mastering a new discourse or genre requires an understanding of writers’ or speakers’ tendencies towards established and prefabricated linguistic choices. Therefore, preparing reviewers for providing peer feedback goes beyond simply providing them with review guidelines and an overview of the task; it requires imparting knowledge about the linguistic features and rhetorical norms specific to the evaluative genre in line with their disciplinary practices. Formulaic language use emerges as a common linguistic feature, which this study disclosed to be fairly discipline-sensitive.

About the authors

Hadi Kashiha

Author for correspondence.
Email: hkashiha@su.edu.om
ORCID iD: 0000-0001-8907-4804

Assistant Professor of Applied Linguistics at Sohar University in Oman and a former postdoctoral researcher at Alzahra University in Iran. His research interests focus on Corpus and Applied Linguistics, with particular emphasis on Discourse Analysis, Genre studies, English for Academic Purposes, and Pragmatics. He has authored several research articles published in prestigious journals, including Journal of Pragmatics, Australian Journal of Linguistics, European Journal of Applied Linguistics, Southern African Linguistics and Applied Language Studies, and a few other international journals.

References