Overview

My research has explored advanced control mechanisms for quadrotor and multi-robot systems, emphasizing model-free approaches and reinforcement learning to address nonlinear dynamics and under-actuation. The findings are detailed in two publications, highlighting the development of innovative methodologies for trajectory tracking and formation control.

These contributions are encapsulated in the following peer-reviewed publications:

Publications

Title	Journal/Conference (peer-reviewed)
Robot Trajectory Tracking using Combined Stochastic Model-Free Position and DDPG-based Attitude Control	ISA Transactions
Formation Control of Multi-Robots Based on Deep Q-learning	International Conference on Robotics and Mechatronics

Abstracts

Publication	Abstract
Robot Trajectory Tracking using Combined Stochastic Model-Free Position and DDPG-based Attitude Control	This article presents a cascade controller for the quadrotor to track the desired trajectory effectively. Unlike previous approaches, this method avoids simplification and linearization assumptions, making it applicable in a wider range of scenarios. A novel linear quadratic tracking method is utilized, which takes into account both process noise and measurement noise while maintaining a model-free nature. Furthermore, the stability analysis of this stochastic method is thoroughly investigated. In terms of attitude control, a model-free approach is adopted. The Deep Deterministic Policy Gradient (DDPG) algorithm is implemented, leveraging an actor-critic network to handle the nonlinearities associated with attitude control. This model-free approach eliminates the need for an accurate model of the quadrotor’s dynamics. Simulations are conducted to evaluate the performance of the proposed controller, and the results demonstrate its ability to effectively control the quadrotor, ensuring accurate trajectory tracking and stability.
Formation Control of Multi-Robots Based on Deep Q-learning	The purpose of this study is to address a model-free formation problem for a team of quadrotors. A cascade controller, including a tracking controller and an attitude controller, is developed. The assumptions preserve the nonlinearity and the under-actuation of the model. The tracking controller uses reinforcement learning to develop a model-free online controller. Moreover, the attitude controller is equipped with an actor-critic neural network to solve the nonlinearity issue. The whole formation leads with a virtual leader in the center of the predesigned formation. Simulation results of multi-aerial vehicles, including four heterogeneous quadrotors, demonstrate the effectiveness of the proposed controller.

Comparison

Both publications advance the field of control strategies for quadrotors and multi-robot systems, addressing key challenges in dynamic and uncertain environments:

Aspect	Stochastic Model-Free Position & DDPG-based Control	Deep Q-learning Formation Control
Core Design	Utilizes a stochastic model-free position controller and DDPG-based attitude control to ensure trajectory tracking.	Employs Deep Q-learning for online reinforcement learning to achieve model-free formation control.
Strengths	Handles process and measurement noise effectively, offering stability in noisy environments.	Adapts dynamically to nonlinearities and under-actuation in formation scenarios.
Challenges	Performance in highly nonlinear dynamics may require further refinement.	Computational scalability remains a concern for larger formations.

Conclusion

The studies provide complementary advancements in trajectory tracking and formation control, leveraging stochastic methods and reinforcement learning. The Stochastic Model-Free Position and DDPG-based Control method excels in addressing noise and ensuring trajectory stability but faces limitations in nonlinear settings. On the other hand, the Deep Q-learning Formation Control approach proves highly effective for coordination tasks but encounters scalability issues. Together, these works pave the way for integrated solutions that combine adaptability, scalability, and computational efficiency.