Application of Machine Learning for Adaptive Trajectory Control of UAVs Under Uncertainty

Full Text

Introduction Unmanned aerial vehicles (UAVs) are an integral part of many areas, including military, agriculture, environmental monitoring (EM), and logistics. Effective UAV flight control under EM uncertainty is one of the key challenges facing researchers and engineers. The main challenge is the need for real-time trajectory adaptation, which requires a high degree of autonomy. Machine learning (ML) provides new opportunities for solving adaptive control problems. ML algorithms are able to analyze large amounts of data and extract patterns from them, which allows predicting changes in EM conditions and adjusting the flight trajectory. The purpose of this paper is to explore the theoretical foundations of ML application for adaptive UAV control under EM uncertainty. The main principles of ML are considered, and existing theoretical approaches and models are analyzed. 1. Methods The study used both empirical and theoretical methods of data analysis and processing. The work is based on modeling, which allows describing the trajectory of objects based on mathematical relationships and ML algorithms. The work used an experimental method, including simulation calculations and testing the proposed algorithms on simulated data. Among the theoretical research methods, analysis and synthesis were used, aimed at systematizing existing UAV trajectory control models, as well as abstraction and analogy, which allow identifying general patterns and adapting known methods to new conditions. In addition, a classification was used to structure approaches to ML in this area. For a quantitative assessment of the effectiveness of the developed methods, statistical methods and quantitative analysis were used, allowing to identify patterns and evaluate the accuracy of model predictions. This approach ensures the reliability and objectivity of the conclusions, which makes the research results applicable in the adaptive UAV control. 2. Results 2.1. ML Algorithm Concepts Machine learning (ML) is a field of artificial intelligence (AI) that focuses on developing algorithms and models that can learn and make decisions based on data. Researchers estimate that the global ML market will exceed $150 billion in 2023. ML is based on the principles of statistics, probability theory, and optimization, which allows you to create systems that can improve their performance with experience. The main difference between ML and traditional programming is the ability to independently identify patterns and adapt to new data without the need for explicit programming. ML is based on several key concepts: Supervised learning (SL) on labeled data containing input and corresponding output values (Figure 1). Figure 1. Supervised learning scheme S o u r c e: made by A.S. Ermilov, O.A. Saltykova The external environment provides input data, such as information about the current flight con-ditions and surrounding environment. The information is passed to the teacher, who generates the desired response of the SL system, for example, the optimal trajectory of the UAV [1]. The system takes the input data and tries to reproduce the desired response. The actual UAV response is then com-pared with the desired one, and the difference between them (error) is used to adjust the ML algorithms. In the context of UAV operation, this concept uses data about the current flight conditions and surrounding environment to generate and adjust the optimal flight trajectory. Unsupervised learning (UL) is performed on unlabeled data, where the model must independently identify hidden structures (Figure 2). Figure 2. Unsupervised learning scheme S o u r c e: made by A.S. Ermilov, O.A. Saltykova The external environment provides stimuli that enter the UL system. It analyzes these stimuli, identifying structures and patterns in the data without predetermined responses [2]. Based on this, the system generates a response that affects the external environment, creating a new stimulus. UL allows the UAV to function effectively in com-plex and dynamic conditions without the need to predetermine all possible scenarios. Reinforcement learning (RL) is the agent interacts with the OS, receiving rewards or penalties for its actions (Figure 3). Figure 3. Reinforcement learning scheme S o u r c e: made by A.S. Ermilov, O.A. Saltykova Reward is a numerical value that reflects the success or failure of an agent’s actions in a given situation. In the RL process, an agent representing a UAV control system selects an action (at) based on the current state of the environment (st). It changes the state of the environment, which is recorded [3]. After that, the agent receives rein-forcement, which is understood as a feedback signal. It can be positive (reward) or negative (penalty) and depends on how effectively the action is performed. The implementation of ML algorithm concepts ensures a high degree of adaptation and accuracy in UAV control [4]. This allows analyzing and processing data in real time, optimizing flight trajectories and making autonomous decisions in complex and changing conditions. To understand the specifics of ML application, it is necessary to study the classification of UAVs by their capabilities, design features and technological characteristics. 2.2. UAVs and Their Classification Unmanned aerial vehicles (UAVs), or drones, are aircraft systems that operate without the direct participation of a pilot on board. These vehicles are controlled remotely by an operator or autonomously using onboard computers and sensors. The global UAV market was valued at US$ 37.46 billion in 2023 and is expected to grow to US$ 148.19 billion by 2032, with a compound annual growth rate of 16.5% from 2024 to 2032. Figure 4 presents the distribution of UAV applications across various industries in 2023 based on current market data. Figure 4. Distribution of UAVs by industries worldwide in 2023, % S o u r c e: made by A.S. Ermilov, O.A. Saltykova UAVs are classified by purpose into three main categories: military, civil and scientific. Military drones are used for reconnaissance, surveillance, target designation and combat operations. Civil drones are used for commercial and private purposes, such as aerial photography, delivery of goods and monitoring of agricultural lands. Scientific and research drones are designed to monitor the environment, study atmospheric pheno-mena and collect data [5]. UAVs are classified by design into three main categories. Multi-rotors (multicopters) are equipped with several rotors, providing high maneuver-ability and flight stability, which makes them ideal for aerial photography and video filming [6]. Airplanes have fixed wings, allowing them to fly efficiently over long distances at high speed, and are used for long-term surveillance and monitoring missions. Hybrid devices combine elements of multicopters and aircraft, providing high maneuverability and the ability to fly long distances. Based on size and weight, UAVs can be classified into several categories (Table 1). Table 1 Classification of UAVs by size and weight UAV type Weight Main areas of application Micro UAV Up to 250 g Research in limited spaces, observation Very small UAVs 250 g - 2 kg Aerial photography, light commercial use Small UAVs 2-25 kg Agricultural monitoring, infrastructure inspections, commercial applications Medium sized UAVs 25-150 kg Military and commercial purposes, long-term surveillance and data collection missions Large UAVs More than 150 kg Large-scale military operations, high-load commercial missions S o u r c e: made by A.S. Ermilov, O.A. Saltykova The operation of the UAV includes three main phases: § Navigation is the process of collecting and analyzing information about the environment, which is necessary for constructing routes and avoiding obstacles [7]. § Control includes solving flight problems and ensuring the execution of planned routes. § Tracking is the assessment of the current location of the UAV and adjusting the route as necessary. The classification of UAVs demonstrates that these aviation systems can be effectively used in various industries due to their ability to perform a wide range of tasks. Adaptive control of the UAV trajectory, using ML methods, plays an important role in their ability to function effectively at all phases of operation. 2.3. Adaptive Control of the UAV Trajectory To ensure autonomous flight in conditions of uncertainty of the environment, it is necessary for the system to be able to respond to changes and adjust the trajectory in real time. Control of UAVs in conditions of uncertainty is associated with many factors (Table 2). Table 2 Factors associated with UAV control under uncertainty Factor Description Impact on UAV control Changing weather conditions Wind speed and direction, temperature, humidity, pressure May affect flight stability, require trajectory correction Presence of obstacles Static and dynamic obstacles such as buildings, trees, birds Constant monitoring is required to avoid collisions OS Dynamics Changes in landscape, moving objects, other UAVs Requires flexibility in route planning and adaptation Communication instability Communication with the operator is disrupted, interference in data transmission channels May lead to loss of control and management Limited computing resources Insufficient data processing capabilities of onboard computers Limits the complexity of algorithms and their speed of operation Low energy reserves Short flight times, need to conserve energy Requires route optimization and energy management Data uncertainty Inaccuracies and errors in data received from sensors Can lead to errors in decision making and management Route complexity Includes the need to navigate routes in complex and unfamiliar areas Requires high precision and reliability in planning and execution S o u r c e: made by A.S. Ermilov, O.A. Saltykova [8; 9] Effective UAV control requires a system that can analyze sensor data and predict potential changes. Sensors collect information about wind speed, temperature, pressure, and the presence of obstacles, and machine learning algorithms process this data to adjust the trajectory. The use of such technologies allows us to identify complex dependencies and predict changes more accurately. 2.4. Theoretical Approaches to the Application of ML for UAV Control The main tasks solved by ML in the UAV control system are: § Pattern recognition: using ML algorithms to analyze visual data, which helps the UAV to identify objects and obstacles in the environment [10]. § Route optimization: using ML to find the most efficient and safe flight paths, which minimizes time and energy costs. § Threat prediction and prevention: using ML models to predict potential risks and develop strategies to prevent them, ensuring flight safety. § Improving interaction with operators: implementing ML to improve the quality of communication and coordination between UAVs and their operators, which contributes to more effective control and monitoring [11]. The implementation of ML requires comprehensive strategies. Various theoretical approaches to the application of ML, such as different learning options, neural networks, ensemble learning and probabilistic models, play a significant role in the development and improvement of UAV control technologies (Table 3). The application of ML theoretical approaches to UAV control provides many benefits, such as improving the adaptability, accuracy, and efficiency of these systems. The use of methods including reinforcement learning, neural networks, ensemble learning, and probabilistic models allows for the creation of more autonomous and robust systems that can cope with uncertainty and complex con-ditions [14]. These approaches not only improve the functionality and safety of UAVs, but also open up new prospects for their application in various fields, from military operations to civil and commercial tasks. Table 3 Theoretical approaches to the use of ML for UAV control Approach Application Advantages Disadvantages Neural networks capable of nonlinear data processing Object recognition, image processing High accuracy, big data capability The need for large amounts of data for training Ensemble learning (multiple models) Improving classification accuracy Increased stability High computational costs Probabilistic models for accounting for uncertainty in data Forecasting and risk assessment, data processing Accounting for uncertainty, improved interpretability Difficulty in building and setting up models S o u r c e: made by A.S. Ermilov, O.A. Saltykova from [12; 13] 3. Results and discussions 3.1. Application of ML for Forecasting and Optimization of Mathematical Models of UAV Trajectory Control One of the main models used to describe the UAV flight dynamics is the system of differential equations (1): x = f (x, u, w, t), (1) where x is the state vector, including the coordinates, speeds and orientation of the UAV; u is the vector of control actions; w is the vector of dis-turbances, such as wind and turbulence; t is time. The function f describes the dynamic behavior of the system. Using a system of differential equations to describe the UAV flight dynamics allows us to accurately model and predict their behavior under various conditions, taking into account the coordinates, speeds, orientation, control actions and external disturbances, such as wind and turbulence [15]. ML methods, such as neural networks, are used to approximate the function f, allowing us to model complex dependencies in the data [16]. For example, recurrent neural networks (RNN) can model the temporal dynamics of the system using formula (2): ht = σh (Wh xt + Uh ht-1 + bh); yt = σy (Wy ht+by), (2) where ht is hidden state of the network at a given time t; xt - input data at time t; Wh, Wy - weight matrices; bh , by - response; σh, σy - activation functions such as sigmoid or ReLU. Sigmoid and ReLU (Rectified Linear Unit) are activation functions used in neural networks to introduce nonlinearity into the model. They allow the network to learn and model complex nonlinear dependencies in the data. For UAVs, this means that ML methods model complex dependencies between the data and the temporal dynamics of the system [17]. Long Short-Term Memory (LSTM) networks, owing to built-in mechanisms for forgetting and remembering information, are able to retain important data for long periods and ignore irrelevant ones. This allows them to effectively predict future system behavior based on past data, which is necessary for adjusting the UAV flight path in changing conditions [18]. The networks process sequential data, such as weather conditions, and help the system adapt to changes in real time. The use of ML in UAV trajectory control significantly increases their autonomy and efficiency. The use of differential equation systems, recurrent neural networks and networks with long short-term memory allows not only to improve control accuracy, but also to significantly reduce the risks associated with the uncertainty of the OS [19]. This opens up new opportunities for the creation of highly efficient and safe unmanned systems capable of performing complex tasks. 3.2. Practice in the Application of ML for Forecasting and Optimizing UAV Trajectories Examples of successful application of ML in UAV control can be found among both Russian and foreign companies. The Russian company Kronstadt specializes in the development and pro-duction of high-tech solutions in the field of UAVs, shipbuilding, robotics and simulators. It offers complex control systems for military and civil aviation, maritime transport and other areas. Kronstadt actively participates in innovative projects, introducing advanced technologies and ensuring high reliability and efficiency of its products. The share of innovative developments in the manu-factured products is 90%. The American company Northrop Grumman uses advanced AI and ML solutions to create complex systems that support the execution of important tasks in the field of national security. These technologies improve the quality of decisionmaking, providing faster and more accurate data processing at the tactical level [20]. Northrop Grumman is implementing AI algorithms for vertical takeoff and landing, which allows them to be used in expeditionary conditions with minimal logistics and maintenance requirements. These systems can be quickly deployed and operated by a small team, which increases their tactical flexi-bility and effectiveness in combat conditions. The American company General Atomics Aeronautical Systems, Inc. (GA-ASI) is one of the leading American manufacturers of military UAVs. Well-known models produced by this company are the MQ-9 Reaper, Gray Eagle and Predator C Avenger. These drones are widely used to perform reconnaissance tasks, as well as to carry out combat missions. ML is used to analyze large amounts of data in real time, which improves target recognition and decision making [21]. Adaptive control provides a high degree of autonomy for drones, allowing them to independently adjust their actions depending on changing conditions on the battlefield. Despite the fact that legal regulation of the use of UAVs is becoming more stringent in various countries, this direction remains promising. Strengthening requirements for flight safety and data protection stimulates the development of more advanced UAV control technologies, including the use of ML. Regulatory changes are aimed at ensuring the safe integration of UAVs into national airspace, which opens up new opportunities for developers and users of these technologies. Conclusion The use of ML for adaptive control of UAVs in uncertain conditions represents a significant step forward in ensuring the accuracy and safety of flights. The use of advanced ML algorithms allows for the efficient processing and analysis of data in real time, which facilitates timely correction of trajectories. This is especially important for complex missions in various fields, such as military affairs, civil and scientific research. The introduction of ML technologies opens up new opportunities for increasing the autonomy and efficiency of UAVs. Despite the tightening of legal regulation, innovative developments in the field of ML continue to evolve, offering improved algorithms and models for data processing and decision-making. Prospects for the development of ML in UAVs include the creation of more complex systems for collective control, which allows for the coordination of actions of several UAVs in a single network. This facilitates the implementation of complex tasks, such as joint survey of territories and synchronized response to emergency situations, significantly expanding the functionality of UAVs and increasing their operational flexibility.

About the authors

Alexander S. Ermilov

RUDN University

Email: eemilov-sasha@yandex.ru
ORCID iD: 0009-0007-4549-172X
SPIN-code: 8696-5057

Postgraduate student of the Department of Mechanics and Control Processes, Academy of Engineering

6 Miklukho-Maklaya St, Moscow, 117198, Russian Federation

Olga A. Saltykova

RUDN University

Author for correspondence.
Email: saltykova-oa@rudn.ru
ORCID iD: 0000-0002-3880-6662
SPIN-code: 3969-6707

PhD in Physical and Mathematical Sciences, Associate Professor of the Department of Mechanics and Control Processes, Academy of Engineering

6 Miklukho-Maklaya St, Moscow, 117198, Russian Federation

References

Supplementary files