Automating Public Feedback: A Big Data Study of Deliberation in Social Media

Full Text

Introduction

The work of government authorities in social media in Russia is an important area of information policy at the federal level and in the regions. Recent Russian research in the field of e-governance and social media focuses on the phenomenon of feedback and the effectiveness of public accounts of politicians. The article analyzes content strategies, strategies for working with comments and appeals, the overall effective-ness of accounts in various social media (mainly VKontakte and Telegram) and audience expectations from political content. In the theoretical field, the most pressing issue remains the possibility of using the social media space as a public field for deliberation and political communication. Nevertheless, the search continues for approaches to how to analyze communication patterns in social media, how to evaluate the effectiveness of specific politicians by quantitative and qualitative parameters, and how to look for a correlation between the general sentiments of citizens and their behavior in the media as an audience.

According to private data^[1] almost 7.4% of Russians surveyed are subscribed to the governor’s social media accounts. No general statistics on the involvement of Russians in the accounts of Russian governors were found at the time of the study’s preparation. Obviously, the number of accounts and followers does not correlate in any way with the actual effectiveness of governors on social media. Most of the studies are descriptive in nature and are devoted to individual regions of Russia.

Most researchers positively assess the overall growth of channels and subscribers in government publications. For example, Kupriyanova (2024) writes that “in recent years, the number of state-owned websites in Russia has increased significantly – from 47 million subscribers in 2022 to 146 million in 2024. This allows the authorities to convey important information to a wide audience and respond promptly to citizens’ questions and concerns”. There is no confirmation of such data in the scientific literature yet. Moreover, feedback effectiveness estimates also vary depending on the way the data is collected and evaluated. For example, Bolshakov (2024) points out that “in practice, a very small number of citizens leave applications describing problems on the resources that the Regional Management Centers (RMC) use for monitoring and analysis or contact government representatives directly. For this reason, theoretically useful developed mechanisms turn out to be incomplete”. Regional management centers are an important tool in providing feedback between government and citizens on social media platforms, but their activities are not based on the theoretical foundation of political communication and cannot be evaluated from an academic perspective. Moreover, detailed RMC statistics by region are closed to external auditors, including the scientific community. Research in individual regions shows that there are problems with the motivation of citizens to engage in feedback through social media. So, Runaev (2024, p. 74) writes that in the South of Russia, “the audience does not participate in the creation and discussion of content and tacitly accepts the content broadcast on the accounts of regional leaders”.

This study is devoted to the development of a procedure for comparative analysis of the effectiveness of governors of Russian regions in the social network VKontakte.

Theoretical premises

The public sphere has changed significantly in the era of digital communications. A sufficient number of studies consider the transformation of political communication and the digital transformation of media as interrelated processes leading to common effects. Indeed, in the modern world, the line between the electorate and the media audience is almost blurred.

The new conditions for the implementation of deliberative processes imply a theoretical adaptation of the classical concept of the public sphere. The main issue is the relationship between possible consensus and dissonance in public opinion, which arise as media effects. According to Bodrunova (2024), we understand cumulative deliberation as: 1) the process of accumulation, redistribution and dispersion of public opinion (opinions) created by the participation of Internet users of various institutional statuses in online discursive activities, and 2) the influence of accumulated opinions on the positions of institutional actors and discourses, including media work and policymaking.

From the point of view of their use in regional political strategies, social media is perceived as an important resource for monitoring public sentiment, promoting socially significant policies, and neutralizing possible discontent among the population (Mousavi, Gu, 2024). According to Itiuridze (2024), “the model of deliberative democracy as a part of political culture can already serve as a basis for informing the country’s leader-ship about the real state of affairs and pluralism of opinions in different regions”. According to foreign researchers, the role of social media may even be underestimated in relation to political competition and conflict. According to Lane, Moxley and McLeod (2023), social networks may bear a greater share of the blame for political competition and conflicts on a group basis than is supported by evidence.

There is no clear answer to the question of the parallelism of online sentiments and offline deliberation, as there are no accurate comparative statistics on the representation of the electorate of Russian regions on individual social media platforms.

Polyakov and Uvarova (2024) write that “today we can safely say that the social network has acquired the characteristics of a parallel reality in which virtualized formats of political representation and political governance operate, where the extensive multimedia capabilities of digital platforms generate new forms of inter-action between the political class and civil society”. The criteria for this parallelism are the strategies of working with information from both politicians and citizens. Both entities both produce and consume information of a political nature, leaving digital traces expressed in quantitative parameters. It is important to remember that digital footprints are a visible part of activity accessible to researchers and political strategists, while communications in private accounts, personal chats and not in-tended for the public sphere become hidden. Also important is the problem of falsification of information in social media, the development of deepfakes and new strategies for the unfair struggle of political competitors, which has remained outside the scope of this study.

It is also important to characterize the voluntary nature of social media communications on the part of the audience and partial coercion on the part of politicians, especially in terms of responses to comments. In this case, the parallelism becomes distorted and implies a different degree of interest in the dialogue. Moreover, the researchers note the lack of a dialogic orientation in social media and study the manifestations of communicative deviations such as aggression, trolling, and the dissemination of false information.

The governors’ activity on social media depends on the will of the politicians themselves and the quality of the PR services. Researchers note that more and more regional heads consider social media activity to be an important part of their work, including for the tasks of effective regional management. E.I. Birucheva, A.A. Gnedash and N.A. Ryabchenko (2024) note “the governors’ attitude to the possi-bilities of the digital space. If at first only the governors of the ‘new wave’ considered representation in the digital space to be really important, then at the moment most of the regional leaders are already concerned about this issue”.

Obviously, in the future, the saturation of the social media space with political content will depend on the introduction of neural network technologies into its creation, as well as audience research, but this challenge remains beyond the scope of the current project.

The idea (Filatova, 2024) that “political discourse on the Web shapes the position and will of ordinary citizens, and with the help of new technologies it is possible to explore it more deeply and in detail so that decision makers can rely on the results of these studies and manage the state more effectively” is challenged by defining criteria for investigating the effectiveness of the presence of politicians on social media. On the one hand, the possibilities and limitations of various electronic platforms for online interaction are being studied. On the other hand, universal quantitative indicators of the effectiveness of social media promotion do not take into account the factors and features of political communication.

Researchers recognize that the image of politicians on social media is an important part of election strategies and building confidence in the current government. As Kotlyarova (2024, p. 13) writes, “citizens’ perception of the political image of government bodies on social networks is one of the aspects of society’s assessment of government and the state, which allows virtual users to form their own political picture of the world in conditions of modern information instability”. It is unlikely that a political image on social media can lead to stability and certainty in the political picture of the world, but without an established image, such a picture will be far from complete. Surprisingly, even though there is a commitment to maintain ac-counts, not all the governors in our sample are still present on social media.

A comparison of the activity of politicians and official media on social media has already been the subject of research by the authors. It was found that state-owned media demonstrate higher engagement than local state-owned media and even than unofficial local news media. This is confirmed by other research (Bryantsev, Bryantseva, 2024): “Account management by the heads of regions is a standard and a requirement of the time. In many cases, they are more popular than groups of official publications. Posts are gaining more views, comments, and coverage is almost like that of the largest regional media outlets. We can say that the heads of the regions personify the regional brand”. The high level of engagement can easily be explained by the fact that journalists have lost their role as intermediaries on platforms where direct communication between citizens and authorities is organized.

The research (Maksimenko et al., 2024) also supports the idea that the electorate and the audience highly value the activity of politicians on social media platforms and the ability to quick-ly establish communication about problematic issues. “The results obtained make it possible to classify the governor’s involvement in modern online communication and his ability to establish constant and effective feedback with the public as one of the most important competencies. The analysis shows that these qualities are still falling out of the personal priorities of some regional leaders”.

Thus, a set of external and internal characteristics becomes the criteria for the effectiveness of governors: the awareness of the politician about the capabilities of platforms and the intention to develop accounts, the presence and balance of an in-formation strategy in social media (for example, openness of comments), engagement indicators and audience statistics, the speed and quality of responses to comments, a positive assessment of the activity in the account recorded offline-polls.

Materials and Methods

As a meta-study of publications analyzing comments on social networks has shown, in recent years, little attention has been paid to platforms such as VKontakte, the topic of local government and the specifics of regions in foreign literature (Alafwan et al., 2023). As part of a comprehensive study of social networks, audiences, and media communications, we present an updated rating of the effectiveness of interaction between the heads of Russia’s regions and the audience on the VKontakte platform. Unlike existing methods focused on the implementation of federal guidelines, the generation of information guides or the frequency of mentions in the media, this approach focuses on the quality of communication between regional leaders and their target audience, analyzing the perception of content and the level of feedback.

Data parsing and processing are carried out according to the author’s methodology using Python and the HuggingFace library.

At the first stage, a pilot analysis of accounts was conducted based on criteria of audience size, activity, engagement in content, general approval of publications, coverage of posted materials and types of reactions. 83 active accounts of governors with a total audience of 5902 thousand people were studied (at the time of information collection, the corresponding pages were not opened in all regions). The share of active users among subscribers who visit the VKontakte site daily is 68.8%, which indicates its high success as a two-way communication channel. The number of users who have not been online for more than 1 year is at 14%. The female audience largely prevails among the subscribers of the heads of subjects – 63.8%. The main target group is middle–aged citizens who are most involved in public life. The proportion of people aged 35 to 44 years is 32.2%, and from 45 to 54 years – 19.1%.

Based on the results of the first iteration, the evaluation criteria were adjusted. The final index was formed based on the aggregation of account positions in seven independent ratings, each of which considers specific aspects of interaction.

1. The size of the community. To eliminate the imbalance caused by differences in the demographic indicators of the regions, the estimate was transferred from the absolute number of subscribers to a relative indicator – the ratio of audience size to population (based on Rosstat data). This makes it possible to offset the advantages of large regions: for example, 90 000 subscribers in the Krasnodar Territory with a population of 5.8 million people are initially incomparable with an audience of 85 000 in the Orel region, where fewer than 700 000 residents live.
2. The index of audience activity. It is calculated as the percentage of subscribers who are regularly active on the platform, i.e. going online. The metric is aimed at eliminating the advantage of artificially inflated accounts: Thus, a page with 50 000 active users is considered more relevant than an account with 100 000 subscribers, of which over 50% have not visited the platform for a year.
3. The average coefficient of visibility of publications (VR post active). It is de-fined as the ratio of post views to the number of active subscribers. The indicator reflects the effectiveness of content distribution in terms of algorithmic ranking of the VKontakte smart feed, demonstrating the ability of materials to attract the attention of the target audience.
4. Active Audience Engagement Index (ER day active). It is calculated based on the percentage of active users interacting with posts (likes, reposts, comments) during the day. Only users with confirmed activity are counted, which increases the reliability of the assessment.
5. The average number of reposts per publication. Normalization by publication frequency makes it possible to assess the quality of content, eliminating distortions associated with mass posting of materials or single viral outbreaks. Yandex.Metrica ensures comparability of accounts with different publication activity.
6. The volume of comments. It reflects the audience’s willingness to engage in dialogue, which is interpreted as an indicator of the depth of engagement. An active discussion indicates that the content is provocative or socially significant, contributing to the formation of a community.
7. Approval rate (likes/views). Replacing the absolute number of likes with a relative indicator minimizes the impact of audience size and identifies content that evokes an emotional response rather than passive consumption.

The algorithm of rating formation. The final position of the account is determined by the sum of the places occupied in each of the seven ratings. The advantage is given to pages that show consistently high results in all areas.

Technological base and objectivity. Automated data collection through a specialized software package eliminates the subjective factor and ensures reproducibility of the results. The combination of absolute and normalized indicators, adjusted for demographic and behavioral characteristics of the audience, increases the validity of the comparison. The proposed methodology sets a new standard for assessing media communications of regional authorities, shifting the focus from quantitative metrics to the quality of dialogue with the audience. The integration of relative indicators and algorithmic data processing helps to increase transparency and reliability of results, which is relevant for research in the field of digital policy and public administration.

Below is the final rating based on an analysis of 85 accounts. At the time of its compilation, the heads of 4 regions (Volgograd, Tambov, Tver and Sverdlovsk regions) had not created or stopped maintaining pages on the VKontakte social network.

To solve the problem of identifying the main types of user comments (general messages, insults, complaints and criticism, gratitude, questions to other users), we have developed an automated text processing procedure based on the RuBERT transformer architecture (DeepPavlov/rubert-base-cased). The choice of the model was due to the possibility of minimal preprocessing, since it independently performs tokenization, and the removal of stop words and lemmatization are undesirable, since the model was initially trained on raw texts from social networks. The main libraries for working with the neural network were transformers and torch. Pandas and numpy were used to work with tables and arrays. To evaluate the qualities, as well as dividing the sample into training and test – scikit-learn. The methodology included several stages.

At the first stage, a body of comments was gathered from official accounts for further training of the model, after which manual marking was carried out in five target categories (with indexes from 1 to 5). The markup was carried out by several experts, discrepancies were discussed, which ensured consistency of interpretations and reliability of the training sample.

The pre-processing of texts included the removal of technical elements (links, repeated spaces, special characters and emojis) and the normalization of the utterance structure.

Next, the RuBERT model was further trained on a marked-up corpus, which made it possible to adapt it to the task of classifying communicative types of messages. During the training process, the model formed distinctions between neutral remarks, expressed criticism, appreciative statements, inter-user questions and aggressive forms of communication.

The quality of the model was evaluated on a deferred sample using standard metrics (accuracy, completeness, F1-measure), which made it possible to check the stability of the classification and the correctness of recognition of critical classes such as complaints and insults. After validation, the model was applied to the full body of comments, as a result of which each text was automatically assigned to the appropriate category.

The classification obtained made it possible to quantify the feedback structure, identify the distribution of reactions, the frequency of interpersonal interactions, and the nature of the neutral background. Automation of the process has made it possible to process large arrays of comments, which makes the method an effective tool for researching the digital public sphere.

The thematic breakdown was carried out manually only for those comments that were marked by the neural network as a Complaint. 10 topics were compiled for coding: Education, Healthcare, Social benefits and benefits, Work and salary, Construction and infrastructure, Housing and communal services and landscaping, Transport and roads, Sports, Culture. Comments that did not fit any of these topics were marked with a Different marker.

Results and Discussion

So far, no answer has been found to the question of how much the audience of social networks and offline voters coincide. Pisareva’s research (2023) proved that the portrait of an average user of the VKontakte social network is based on the key characteristics that match the portrait of the average voter. However, the author admits that it is wrong to say that the sentiments of Russians are reliably reproduced on social networks due to the uneven spread of the Internet and social networks among residents of different regions of the country. Moreover, research shows that not all possible groups of the public participate in discussions on social networks in principle (Kalogeropoulos et al., 2017). Based on the assumption that this correlation exists for the nuclear audience of VKontakte aged 25–45, we believe that the data obtained demonstrate the real interest of the population in the political life and activity of the governors (Figure 1, 2).

In our sample, we took into account the ratio of subscribers to the population of the region to see how much citizens are involved in the media activity of the head of the region. Thus, by the end of February 2025, the average share of subscribers of the heads of constituent entities of the Russian Federation on the VKontakte social network in relation to the number of citizens living in the region was 5.14%. The spread between the maximum and minimum values is quite significant – from 0.16% to 59.05%. At the same time, it is important to understand that a high proportion of subscribers does not always indicate the real interest of the audience. Often, for formal reasons, a significant part of it consists of civil servants and public sector employees, as well as users connected using massfollowing mechanisms. It is possible to talk about the serious interest of subscribers in the information agenda and the activities of the heads of subjects only with the organic growth of the audience and its sufficient activity. The head of the Chechen Republic remains the leader by this criterion. 

Figure 1. The topic of comments in the accounts of the governors
Source: created by Kamilla R. Nigmatullina, Renat M. Kasymov, Haya Yu.J. Ashour.

Figure 2. The dynamics of the main indicators of engagement for three months
Source: created by Kamilla R. Nigmatullina, Renat M. Kasymov, Haya Yu.J. Ashour.

The leader in terms of audience engagement is the head of the Komi Republic. The index reflects the share of subscribers with regular online access on the VKontakte site. This metric is critically important and gives an idea of the “liveliness” of the community and the potential for a quick response to information events. The average value of the indicator for all analyzed accounts of the heads of subjects is 69.6%, the maximum is 95.9%, and the minimum is 30.7%. Publics with an index above 70% demonstrate good activity and high involvement in the agenda.

The head of the Lugansk People’s Republic is in the lead by a wide margin in terms of the visibility of publications. The metric shows what proportion of the active audience sees the publications of the heads of subjects in their feed. It takes into account, first of all, the principles of algorithmic issuance of VKontakte, the time and frequency of posting, as well as additional coverage earned through reposts. The average value of the indicator is at the level of 45.09%. The maximum index is 407.18%, the minimum is 0.41%.

The leader of the rating in terms of daytime audience engagement remains the head of the Karachay-Cherkess Republic. The indicator reflects the percentage of active subscribers who not only see posts, but also interact with them. The metric indicates the “quality” of the audience and its direct involvement in the content. However, it is important to make allowances for the presence of competing sites in the regions. When the head is running, for example, a Telegram channel, some of the active users may shift their activity there. In addition, in the political sphere, a relatively small proportion of subscribers are usually involved in direct discussion and reaction, so an index of 3–5% can be considered quite good. The average engagement among the 85 accounts of the heads of subjects in February is 2.71%. The maximum value is 11.10%, the minimum value is 0.08%.

The head of the Kemerovo region became the most “viral” governor. The metric of the average number of reposts per post shows the level of “virality” of content distribution. The more often the audience shares posts, the wider the reach, especially beyond the initial circle of subscribers. As a rule, high peaks of reposts are often associated with high-profile events when the content is really important or causes a strong emotional response. On average, one post in the account of the heads of regions in February gained 24 reposts. The maximum value is 120.98, and the mini-mum value is 0.47.

In February 2025, the head of the Samara Region was in the lead in terms of the number of comments. The indicator indicates the level of discussion in the communities. Together with the tone of the comments and their “quality” (whether they contain questions, suggestions, complaints), it evaluates feedback. It is important to keep in mind that “more is not always better”. Sometimes a significant number of comments arise around problematic, conflicting topics or due to spam attacks. The average number of comments left under the posts of the governors in February is 2037. The maximum value is 19 921, and the minimum value is 38. At the same time, the possibility of commenting is still disabled in 7 accounts.

According to the criterion of “content approval”, the head of the Republic of Sakha (Yakutia) is in the lead. The metric reflects the proportion of users who have seen the post and at the same time put a “like”. If the indicator is constantly maintained at a certain level or slightly changed, it can act as a signal of a normal routine reaction, when a certain proportion of subscribers leave a reaction automatically (like everything in a row). The average content approval rate for February is at 2.65%. The maximum value is 6.81%, the minimum value is 0.65%.

Conclusion

Political communication on social media has generally been studied in sufficient detail. However, before the phenomenon of governors’ accounts appeared, political sentiments were studied on the entire array of social media posts, and it was quite difficult to identify regional specifics of sentiments.

The effectiveness of power on social media was of interest not only to scientists, but also to officials. The methodology for studying accounts has developed in parallel in academic and administrative circles. Globally, it was based on a comprehensive study of quantitative metrics of social networks, universal for any account, regardless of orientation, and qualitative metrics, mainly related to comments. Regional management centers have established a complaint collection system, and commercial monitoring services (such as Brand Analytics) have improved sentiment analysis systems. As a result, the former encountered the fact that, among all the variations of moods, they focused only on negative ones, while the latter presented the mood field as generally neutral (up to 95% of messages could be marked by the automatic system as not containing emotions).

At the same time, the analysis of the effectiveness of an official, and even more so the head of the region, on a social network cannot be limited only to the number of responses to complaints or the number of posts per month. An integral index that takes into account the size of the region’s population, external factors (for example, the events in the Kursk region in 2024 and the change of governor attracted in-creased interest in Alexander Khinstein’s account), and the quality of the audience itself (how long they stay online and whether they see published posts) can become a more objective indicator of the effectiveness of governors’ accounts.

At this stage of the study, manual encoding was used for the thematic analysis of complaints. At the next level, the manually marked-up dataset will be used to train the neural network and then automatically analyze the comments.

 

 

^{1 Polylog PR Agency (2023). A year without Meta: What has changed in the internet communication of Russian governors? (2023). https://clck.ru/3QXyjE Meta is banned in Russia as an extremist organization from March 21, 2022.}

About the authors

Kamilla R. Nigmatullina

St. Petersburg State University

Author for correspondence.
Email: k.nigmatullina@spbu.ru
ORCID iD: 0000-0002-9146-1712
SPIN-code: 4373-4174

Grand PhD in Political Sciences, Professor at the Department of Digital Media Communications

7-9 Universitetskaya naberezhnaya, Saint Petersburg, 199034, Russian Federation

Renat M. Kasymov

St. Petersburg State University

Email: renat.kasymov@spbu.ru
ORCID iD: 0009-0003-5223-0385
SPIN-code: 3436-2041

Assistant Lecturer at the Department of Digital Media Communications

7-9 Universitetskaya naberezhnaya, Saint Petersburg, 199034, Russian Federation

Haya Y.J. Ashour

American University in the Emirates

Email: haya.ashour@aue.ae
ORCID iD: 0000-0002-5532-5924
SPIN-code: 3809-2897

PhD in Philology, Assistant Professor Mass Media, College of Media and Mass Communication

Dubai Intl. Academic City, Dubai, 503000, United Arab Emirates

References

Supplementary files