Моделирование атрибуции авторства коротких текстов в условиях деградации обучающего корпуса

Полный текст

Introduction Research into automatic authorship attribution dates back to the mid-20th century. Over this period, a wide range of methods has been developed - from statistical author profiles to modern large language models. Today, the task remains highly relevant due to the proliferation of digital communication channels (messengers, email, forums, blogs), which often lack robust user verification and are susceptible to identity forgery. Contemporary studies in authorship attribution increasingly focus on more challenging variants of the classical problem. These include the analysis of short texts [1], attribution under conditions of textual distortion [2], and scenarios with a large number of candidate authors [3]. Each of these complications has been shown to significantly reduce attribution accuracy. However, the combined effect of two simultaneous challenges-reduced training corpus size and textual distortions within those corpora-remains underexplored. This paper addresses this gap by investigating authorship attribution for short texts under such conditions of training data degradation. We focus on random character-level distortions (specifically, symbol substitutions) and consider six standard classifiers: SVM, Logistic Regression, Naive Bayes, Random Forest, Decision Tree, and kNN. As stylistic features, we employ token -grams ( = 1, 2) and TF-IDF values computed on various combinations of -grams-features commonly used in authorship attribution studies. The aim of this work is to analyze how the accuracy of these classifiers degrades when the training data is subject to random distortions and/or when the volume of the training corpus is reduced. Negative effects arising from text distortions Text distortions can arise from various sources, including typos, optical character recognition (OCR) errors [4], speech recognition errors [5], machine translation inaccuracies [6], and imperfections from decrypting certain ciphers [7]. Such distortions can significantly impair both the human readability of a text and the performance of automated processing systems. A review by [8] examines how traditional text analysis methods-such as information retrieval, topic identification, summarization, and named entity extraction-are affected by distorted input. More recently, the impact of distortions on large language model (LLM) performance has been explored in [9]. The specific challenge of summarizing documents containing OCR errors was addressed in [10]. The study demonstrated that while the “Cut-and-Paste Text Summarization” method [11] performs well on clean text, its effectiveness drops sharply when applied to distorted text. The quality of the generated summary degrades rapidly with even low levels of noise, with tokenization errors and incorrect sentence boundary detection causing the most significant harm. Errors introduced at the character level can also propagate through processing pipelines. In [4], the authors trace how OCR errors (character replacements, insertions, and deletions) lead to incorrect tokenization. This, in turn, causes faulty sentence segmentation and ultimately results in errors during part-of-speech tagging. Thus, in a pipeline-based approach, even minor initial distortions can amplify at each subsequent stage. Interestingly, the impact of distortions varies by task. While [12] reports that OCR errors and synthetic character-level distortions have only a minor effect on document clustering accuracy, they significantly reduce the performance of topic modeling with Latent Dirichlet Allocation. 162 Computer science DCM&ACS. 2026, 34 (2), 160-174 On the attribution of authorship of distorted texts A number of studies have investigated how distortions affect authorship attribution under various conditions. In [13], the problem is considered in a scenario where the training and test texts differ in genre and subject matter-a difference that negatively impacts identification accuracy. The study demonstrates that stylistic features based on character -grams (particularly for = 3) are more robust to such cross-genre and cross-topic variations than word-based features. The robustness of character -grams to textual noise itself was explored in [14] using the “Burrows’ delta” method [15]. The authors employed frequency vectors of the most frequent words (MFW) and character -grams as stylistic features, introducing random character-level distortions at noise levels ranging from 1 to 100% (in 1% increments). Experiments on relatively long historical texts in several European languages revealed a trade-off: higher attribution accuracy at low noise levels comes at the cost of reduced noise immunity, and vice versa. Specifically, increasing the length of MFW vectors improves accuracy when noise is minimal, while shorter vectors perform better as distortion levels rise. The study also found that vectors of the most frequent character 3-grams serve as reliable stylistic markers even under heavy distortion-in some corpora, authorial signal remained traceable up to a 60% noise level. A different approach to distortion-optical character recognition (OCR) of noisy images-is examined in a series of studies [16-18]. The authors constructed a corpus of 25 English texts (approximately 850 words each) written by five authors. Printed pages were saved as high-resolution JPG images, then artificially degraded using MATLAB’s “Salt and Pepper noise” filter [19] at levels from 1 to 7% (in 1% increments). The noisy images were subsequently processed by an OCR system, and authorship attribution experiments were conducted on the resulting texts. Four classifiers were tested, using words and character -grams ( = 1-4) as stylistic features. Unsurprisingly, attribution accuracy decreased as noise increased, with classifiers based on 4-gram characters achieving the highest overall performance. Finally, a real-world application of authorship attribution under distortion is presented in [2], which analyzes letters by the Brothers Grimm-Jacob (1785-1863) and Wilhelm (1786-1859). The corpus of 85 letters was digitized using the Transkribus system [20] for printed and handwritten text recognition. The proportion of words correctly recognized (matching expert manual transcription) was 88% for printed texts and 81% for handwritten ones. The authors argue that this level of accuracy is sufficient for practical authorship attribution, at least in this binary classification task. Using a support vector machine with a linear kernel (implemented in Scikit-learn [21]), they achieved better-than-chance attribution accuracy (above 50%) on texts where at least 20% of characters were undistorted. Description of experiments Corpus Description The experiments were conducted on the English-language Twitter corpus introduced in [22]. This dataset has been widely used for benchmarking authorship attribution models (see, e.g., [23, 24]). It comprises tweets collected in 2009-2010 from 7,000 globally popular accounts; each account is considered a distinct author. From this pool of 7,000 authors, 50 were randomly selected for the present study, and 1,000 messages were retained for each. Representative examples from the corpus are shown as follows. The first three texts are from the same author, texts 4 and 5 from another, texts 6 and 7 from a third, and texts 8, 9, and 10 each have different authors. V. M. Khvostenko et al. Modeling authorship attribution 163 Now playing: Modern Talking - Angie’s Heart. Tune in: http://stream.laut.fm/eurodance.m3u. Now playing: A-ha - Take on me. Tune in: http://stream.laut.fm/eurodance.m3u. Now playing: Mozzart - Malice And Vice. Tune in: http://stream.laut.fm/eurodance.m3u. Build your Sales Marketing for Free. Utilize this permission emailing tool to successfully communicate with clients. http://ow.ly/pROT. Build your corporate brand or a new brand through a Free marketing and sending permission based email solution. http://ow.ly/zZopC. New blog post: Stop Smoking by Hypnosis at Self Hypnosis Stop Smoking http://bit.ly/2bRcbm. New blog post: Stop Smoking Habitrol Gum | Little Green Papoose http://bit.ly/8a1JmE. Obama says to reduce role of U.S. nuclear weapons (Reuters) http://bit.ly/9XIGY8. The time in Hollywood, CA, is 6:05:03 am. @julio_zin *smirks and smiles slyly* are you now? Data Preparation and Experimental Design The corpus was split into training and test sets using an 80/20 ratio. For each of the 50 authors, 800 messages were allocated as potential training data and 200 messages as test data. All texts underwent preliminary preprocessing using the NLTK and Scikit-learn libraries, which included removal of hyperlinks and hashtags, word tokenization, stopword removal, and lemmatization. The experimental workflow comprised eight stages, illustrated in Figure 1. After preprocessing, features were extracted from the original (clean) test texts. These features served as the reference for evaluation. For the training data, two types of degradation were introduced. First, synthetic random distortions of the “symbol substitution” type were applied using the method described in [25]. For a given distortion probability of ∈ (0, 1), the text was scanned character by character. Each character was independently subjected to replacement with probability p; if replaced, it was substituted by a randomly selected Latin character. The distortion parameter p was chosen such that 0, 1, 3, 5, 10, and 20% of the text characters were distorted. Thus, for each original training text, six versions with different distortion levels were generated. The second type of degradation involved varying the number of training texts per author from 50 to 800 in increments of 50, yielding 16 distinct training set sizes. These two factors were combined factorially, producing a set of training set variants. For each such variant, stylistic features were extracted. The following feature types were used: Bag of Words (BOW)-token counts; -grams-word tokens and bigrams; TF-IDF-calculated on five different -gram combinations: tokens; tokens + bigrams; tokens + bigrams + trigrams; bigrams + trigrams; and tokens + bigrams + trigrams + 4-grams. Six classifiers from the Scikit-learn library were employed (six classifier variants): Support Vector Machines (SVM), Logistic Regression, Naive Bayes, Random Forest, Decision Tree, and k-Nearest Neighbors (kNN). Default parameters were used for all classifiers, except kNN where = 1 was set. For each combination of distortion level, corpus size, feature type, and classifier, the model was trained on the degraded training data and subsequently tested on the clean test set. Accuracy was used as the primary evaluation metric, defined as the ratio of correctly attributed authors to the total number of predictions. Testing the authorship attribution models was carried out in eight stages, as shown in Figure 1. 164 Computer science DCM&ACS. 2026, 34 (2), 160-174 Figure 1. Stages of authorship attribution model testing Results The experimental results are presented in Figure 2 to 13. The figures show attribution accuracy for six different feature sets, color-coded as follows: Blue: Bag of Words (BOW); Orange: TF-IDF on tokens; Green: -grams (tokens and token bigrams); Red: TF-IDF on tokens, bigrams, and trigrams; Purple: TF-IDF on bigrams and trigrams; Brown: TF-IDF on tokens, bigrams, trigrams, and 4-grams. For each classifier, two figures are provided: one showing performance on clean training data as a function of training set size, and another showing performance under increasing distortion levels (1, 3, 5, 10, and 20%). All classifiers were implemented using the Scikit-learn library with default parameters, unless otherwise specified. The kNN classifier required additional parameter tuning, described in Section 3.6. Support vector machine (linear kernel) The linear SVM classifier was implemented using the LinearSVC package from Scikit-learn with default parameters. Logistic regression The logistic regression classifier was implemented using the LogisticRegression package from Scikit- learn with default parameters. V. M. Khvostenko et al. Modeling authorship attribution 165 Figure 2. Accuracy of the linear SVM classifier as a function of training set size (clean training data) Figure 3. Accuracy of the linear SVM classifier as a function of training set size under different distortion levels (1, 3, 5, 10, and 20%) Figure 4. Accuracy of the logistic regression classifier as a function of training set size (clean training data) 166 Computer science DCM&ACS. 2026, 34 (2), 160-174 Figure 5. Accuracy of the logistic regression classifier as a function of training set size under different distortion levels (1, 3, 5, 10, and 20%) Figure 6. Accuracy of the Naive Bayes classifier as a function of training set size (clean training data) Naive bayes The Naive Bayes classifier was implemented using the MultinomialNB package from Scikit-learn with default parameters. Random forest The Random Forest classifier was implemented using the RandomForestClassifier package from Scikit-learn with default parameters. Decision tree The Decision Tree classifier was implemented using the DecisionTreeClassifier package from Scikit- learn with default parameters. V. M. Khvostenko et al. Modeling authorship attribution 167 Figure 7. Accuracy of the Naive Bayes classifier as a function of training set size under different distortion levels (1, 3, 5, 10, and 20%) Figure 8. Accuracy of the Random Forest classifier as a function of training set size (clean training data) -Nearest Neighbors (kNN) The kNN classifier was implemented using the KNeighborsClassifier package from Scikit-learn with the Euclidean distance metric. Since the performance of kNN depends critically on the choice of , preliminary experiments were conducted to select an appropriate value. Table 2 presents the accuracy of the kNN classifier for different values of (from 1 to 10) across all six distortion levels. These results were obtained using TF-IDF features on tokens, which proved to be the best-performing feature set for the maximum training corpus size (see Table 2 below). The highest accuracy for each distortion level is highlighted in bold. Based on these results, = 1 was selected for all subsequent experiments. Figure 12 and 13 show the accuracy of the kNN classifier (with = 1) as a function of training set size for clean and distorted training data, respectively. 168 Computer science DCM&ACS. 2026, 34 (2), 160-174 Figure 9. Accuracy of the Random Forest classifier as a function of training set size under different distortion levels (1, 3, 5, 10, and 20%) Figure 10. Accuracy of the Decision Tree classifier as a function of training set size (clean training data) Table 1 Accuracy of the kNN classifier for the maximum training corpus size (800 texts per author) with 1 ⩽ ⩽ 10 k 1 2 3 4 5 6 7 8 9 10 Err=0% 0.83 0.76 0.74 0.73 0.72 0.71 0.71 0.71 0.72 0.72 Err=1% 0.82 0.74 0.72 0.70 0.70 0.69 0.69 0.69 0.69 0.69 Err=3% 0.79 0.71 0.68 0.66 0.65 0.64 0.64 0.64 0.64 0.64 Err=5% 0.77 0.68 0.65 0.63 0.61 0.6 0.6 0.6 0.6 0.6 Err=10% 0.72 0.62 0.59 0.56 0.54 0.52 0.52 0.52 0.51 0.51 Err=20% 0.66 0.54 0.49 0.45 0.42 0.41 0.4 0.39 0.39 0.39 V. M. Khvostenko et al. Modeling authorship attribution 169 Figure 11. Accuracy of the Decision Tree classifier as a function of training set size under different distortion levels (1, 3, 5, 10, and 20%) Figure 12. Accuracy of the kNN classifier ( = 1) as a function of training set size (clean training data) Analysis and discussion of results Table 2 summarizes the best attribution accuracy achieved by each classifier for three distortion levels (0%, 5%, and 20%) and five training set sizes (800, 600, 400, 200, and 50 texts per author). The color coding indicates which stylistic feature set yielded the best performance for each classifier under each condition. The feature sets are: Bag of Words (BOW), TF-IDF on tokens, -grams (1,2), TF-IDF on tokens+bigrams+trigrams (denoted TF-IDF(1,3)), and TF-IDF on tokens+bigrams+trigrams+4-grams (denoted TF-IDF(1,4)). For the baseline scenario (clean training data with full corpus size), our results closely match those reported in [24], confirming the validity of our experimental setup. A notable pattern emerges from Figure 2-13: the TF-IDF(2,3) feature set-comprising only bigrams and trigrams of tokens-consistently underperforms compared to all other feature sets. This can be attributed to two factors. First, all other feature sets include unigrams (individual tokens), which appear to be highly informative for authorship attribution. Second, token -grams are particularly 170 Computer science DCM&ACS. 2026, 34 (2), 160-174 Figure 13. Accuracy of the kNN classifier ( = 1) as a function of training set size under different distortion levels (1, 3, 5, 10, and 20%) vulnerable to character-level distortions, as random symbol substitutions can easily disrupt word boundaries and token sequences. Support Vector Machine (SVM) demonstrated the highest overall accuracy across nearly all parameter combinations tested. This finding aligns with the well-established suitability of SVM for high-dimensional, sparse data typical of text classification tasks. Logistic Regression consistently ranked second, delivering comparable performance to SVM in many scenarios, particularly when using Bag of Words features (as noted in Section 3.2). Naive Bayes, Random Forest, and Decision Tree showed intermediate performance, with accuracy declining predictably as distortion levels increased and training set sizes decreased. -Nearest Neighbors (kNN) proved to be the least effective classifier in our experiments, exhibiting both low accuracy and high variability. This poor performance is likely due to the well-known sensitivity of distance-based methods to the high dimensionality of feature spaces-a phenomenon often referred to as the “curse of dimensionality.” An interesting observation is that = 1 consistently yielded the best results for the kNN classifier (see Table 2). While this might appear counterintuitive-since smaller values typically increase sensitivity to noise-the structure of the Twitter corpus provides an explanation. As illustrated in Table 1, many authors in the dataset exhibit highly formulaic writing patterns (e.g., automated “Now playing...” posts or templated promotional messages). This creates a situation where each author’s texts have many close neighbors within their own collection, making the single nearest neighbor a surprisingly effective choice. This phenomenon bears some resemblance to the “overfitting” effects discussed in [26], though in this case the simplicity of the 1- rule proves advantageous given the repetitive nature of the data. Conclusion This study investigated authorship attribution for short texts under training data degradation, combining two challenging conditions: random character-level distortions and reduction in corpus size. Using a Twitter corpus of 50 authors with 1,000 texts each, training data were varied from 800 to V. M. Khvostenko et al. Modeling authorship attribution 171 Table 2 Single division error with leading monomial weight of 95% 800 600 400 200 50 Err=0% SVM 0.92 0.91 0.88 0.83 0.65 LR 0.9 0.88 0.85 0.78 0.58 NB 0.89 0.87 0.83 0.76 0.53 RF 0.88 0.86 0.82 0.75 0.57 DT 0.84 0.82 0.78 0.71 0.53 kNN 0.72 0.7 0.67 0.63 0.51 Err=5% SVM 0.89 0.87 0.83 0.77 0.58 LR 0.87 0.84 0.8 0.72 0.53 NB 0.86 0.84 0.79 0.71 0.49 RF 0.84 0.81 0.76 0.7 0.51 DT 0.81 0.78 0.73 0.65 0.47 kNN 0.65 0.63 0.6 0.56 0.46 Err=20% SVM 0.75 0.71 0.64 0.54 0.37 LR 0.74 0.7 0.62 0.52 0.32 NB 0.74 0.7 0.63 0.52 0.33 RF 0.71 0.67 0.59 0.48 0.3 DT 0.7 0.65 0.57 0.46 0.28 kNN 0.42 0.39 0.37 0.32 0.25 BOW TFIDF Ngram (1,2) TFIDF (1,3) TFIDF (1,4) 50 texts per author across six distortion levels (0-20%), while test texts remained unchanged. Six classifiers-SVM, Logistic Regression, Naive Bayes, Random Forest, Decision Tree, and kNN-were evaluated using BOW and TF-IDF features on various -gram combinations. With the exception of kNN, all classifiers exhibited comparable accuracy degradation patterns, and those performing well on complete data maintained their advantage under degraded conditions. SVM with TF-IDF features proved most robust across all training set sizes and distortion levels, achieving accuracies of 0.92, 0.89, and 0.75 at 0, 5, and 20% distortion respectively, followed closely by Logistic Regression with BOW (0.90, 0.87, and 0.74). Decision Tree and Random Forest showed greater vulnerability to data reduction and distortion, while kNN yielded the poorest and most unstable results, likely due to the high dimensionality of the feature space. Feature sets including unigrams consistently outperformed those based solely on higher-order -grams, underscoring the importance of token-level information 172 Computer science DCM&ACS. 2026, 34 (2), 160-174 for attribution of short, noisy texts. These findings quantify the robustness of common attribution methods and identify SVM with TF-IDF as the most reliable choice when training data quality cannot be guaranteed.

Об авторах

В. М. Хвостенко

Национальный исследовательский университет «Высшая школа экономики»

Email: vkhvostenko@hse.ru
ORCID iD: 0009-0000-1985-1401

Researcher of Cybersecurity CPS Department of HSE University

Покровский бульвар, д. 11, Москва, 109028, Российская Федерация

А. П. Коваленко

Академия криптографии Российской Федерации

Email: kovalenko.ap@cryptoacademy.gov.ru
ORCID iD: 0009-0007-8777-8622

Professor, Doctor of Technical Sciences, Full member of Academy of Cryptography of the Russian

а/я 100, Москва, 119331, Российская Федерация

С. Ю. Мельников

Национальный исследовательский университет «Высшая школа экономики»; Российский университет дружбы народов

Email: melnikov-syu@rudn.ru
ORCID iD: 0000-0002-9023-9896

Doctor of Physical and Mathematical Sciences, Professor of Department of Probability Theory and Cyber Security of RUDN University; Chief Researcher of Cybersecurity CPS Department of HSE University

Покровский бульвар, д. 11, Москва, 109028, Российская Федерация; ул. Миклухо-Маклая, д. 6, Москва, 117198, Российская Федерация

Р. В. Мещеряков

Национальный исследовательский университет «Высшая школа экономики»; Институт проблем управления им. В. А. Трапезникова Российской академии наук

Автор, ответственный за переписку.
Email: mrv@ieee.org
ORCID iD: 0000-0002-1129-8434

Professor, Doctor of Technical Sciences, Chief Researcher of Cybersecurity CPS Department of HSE University; Head of the Laboratory, Institute of Control Science of RAS

Покровский бульвар, д. 11, Москва, 109028, Российская Федерация; ул. Профсоюзная, д. 65, стр. 2, Москва, 117342, Российская Федерация

Список литературы

Дополнительные файлы