Comparison of LQG and LQR Controllers for Uncertain Satellite Model
1. Motivation and Goals
John C. Doyle famously wrote in the abstract to his landmark paper, “Guaranteed Margins for LQG Regulators,” that “there are none.” While the concept of optimality translates well into robustness for the linear-quadratic regulator (LQR) - the regulator achieves infinite gain margin and a phase margin that exceeds 60 degrees - the addition of a Kalman filter kills these assurances for LQG control.
The objective of this project is to demonstrate the importance of robustness in controller design through the comparison of an LQG and an LQR controller. By varying the value of a parameter in the dynamic system without altering the controller, we attempt to reveal the vulnerability of the LQG controllers when faced with uncertainties, showcasing the superior robustness of the LQR controller in maintaining stability and performance across a range of operating conditions.
2. Dynamic Model
The model chosen to make this comparison was modified from “Design Study C: Satellite Attitude Control,” an exercise in Randal W. Beard’s Introduction to Feedback Control.
The satellite and solar panels are only able to rotate about the center; there is no translational movement allowed. The satellite body has a moment of inertia of \(J_s\), and the panels are assumed to be rigid, with a moment of inertia of \(J_p\). The angle between the horizontal reference and the orientation of the satellite is \(\theta\), and the angle between the horizontal reference and the angle of the panel is \(\varphi\). The satellite body and the panels are connected by a spring \(k\) and damper \(b\). Additionally, there exist thrusters about the axis of the satellite that command an input torque \(\tau\).
The default specifications of these parameters are \(J_s\) = 5 kg m², \(J_p\) = 1 kg m², \(k\) = 0.1 N m, and \(b\) = 0.05 N ms. After initial design, the value of moment of inertia \(J_s\) will be varied by up to 600% (5 kg m² \(< J_s <\) 30 kg m²) of its original value to gauge the robustness of the controllers to uncertainty in the model.
The equations of motion are straightforward to solve for using the Euler-Lagrange equations:
\[\frac{d}{dt} \left( \frac{\partial L}{\partial \dot{q}_i} \right) = \frac{\partial L}{\partial q_i} + Q_i \]
where the generalized coordinates \(\textbf{q}\) are defined as:
\[ \textbf{q} = \left[ \theta \qquad \varphi \right]^T \]
\[ \begin{bmatrix} J_s \ddot{\theta} \\ J_p \ddot{\varphi} \end{bmatrix} = \begin{bmatrix} k (\varphi - \theta) \\ -k (\varphi - \theta) \end{bmatrix} + \begin{bmatrix} \tau + b (\dot{\phi} - \dot{\theta}) \\ -b (\dot{\phi} - \dot{\theta}) \end{bmatrix} \]
This system is linear, so no linearization is necessary to assemble the system equations:
\[ \dot{\textbf{x}} = \textbf{A} \textbf{x} + \textbf{B} \textbf{u} \to \begin{bmatrix} \dot{\theta}\\ \dot{\varphi}\\ \ddot{\theta}\\ \ddot{\varphi}\\ \end{bmatrix} = \begin{bmatrix} 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 1\\ -\frac{k}{J_s} & \frac{k}{J_s} & -\frac{b}{J_s} & \frac{b}{J_s}\\ \frac{k}{J_p} & -\frac{k}{J_p} & \frac{b}{J_p} & -\frac{b}{J_p}\\ \end{bmatrix} \begin{bmatrix} \theta\\ \varphi\\ \dot{\theta}\\ \dot{\varphi}\\ \end{bmatrix} + \begin{bmatrix} 0\\ 0\\ \frac{1}{J_s}\\ 0 \\ \end{bmatrix} \tau \]
As the implementation of LQR requires full-state feedback, the output equation is written as follows:
\[ \textbf{y} = \textbf{C} \textbf{x} + \textbf{D} \textbf{u} \to \textbf{y} = \textbf{I}_4 \begin{bmatrix} \theta\\ \varphi\\ \dot{\theta}\\ \dot{\varphi}\\ \end{bmatrix}\]
The rank of the controllability matrix of the system is 4, which is equal to the number of states. This criterion means that any state can be reached through the implementation of control. The poles of the system are \(\{0.4585, -0.2597 + 0.3699i, -0.2597 - 0.3699i, 0\}\); the poles with nonnegative real parts, at 0.4585 and 0, indicate that the system is unstable.
3. Controller Design Parameters
The following controllers were designed with all nominal parameters specified in section 2.
We use Bryson’s rule to choose viable \(\textbf{Q}\) and \(\textbf{R}\) matrices for the LQR design - for the purpose of this design, let’s say the maximum acceptable value of the error for \(\theta\) and \(\varphi\) is \(\frac{\pi}{6}\), the maximum acceptable value of the error for \(\theta\) and \(\varphi\) is \(\frac{1}{4}\), and the maximum acceptable value of the error for \(\tau\) is 1.
\[ \textbf{Q} = \text{diag} \left[ \left( \frac{\pi}{6}\right)^{-2} \quad \left( \frac{\pi}{6}\right)^{-2} \quad \left( \frac{1}{4}\right)^{-2} \quad \left( \frac{1}{4}\right)^{-2} \right] \qquad \textbf{R} = 1 \]
The gains that minimize the value of the cost functional are:
\[ \textbf{K}_{\text{LQR}} = \left[ -10.0047 \quad 7.3038 \quad -26.3253 \quad -48.2365 \right] \]
and the corresponding controller input \(\textbf{u}\) is:
\[ \textbf{u} = -\textbf{K}_{\text{LQR}} \textbf{x} \]
The Kalman filter for the LQG controller is provided with estimated noise covariances \(\textbf{Q}_k = (0.01)^2 \, \textbf{I}_4\) and \(\textbf{R}_k = (0.5)^2 \, \textbf{I}_4\).
4. Plant Simulation
Measurement noise was simulated using Simulink’s “Random Source” block, which outputs a random signal with Gaussian distribution using the ziggurat algorithm. Four blocks were configured to output noise with zero mean and variance \(0.5^2\), then vectorized and added to the system output.
Process noise was not simulated, but the Kalman filter was still configured with a \(\textbf{Q}\) matrix to satisfy the requirement for a valid and well-defined mathematical formulation.
Controller performance was gauged by setting initial conditions at:
\[ \textbf{x}_0 = \left[ \frac{\pi}{12} \quad \frac{\pi}{6} \quad 0 \quad 0 \right]^\text{T} \]
and gauging the initial response.
5. Comparisons and Conclusions
The controllers as designed above were used to regulate the plant, with four different satellite moments of inertia: \(J_s\) = 5 kg m², \(J_s\) = 10 kg m², \(J_s\) = 15 kg m², and \(J_s\) = 30 kg m². The time-domain performance of each controller was gauged using the aforementioned initial response. Each simulation was run for 40 seconds.
The LQG controller outperforms the LQR controller for the plant, when fitted with nominal parameters. The addition of the Kalman filter results in less deviation from 0 over time - this is especially apparent when comparing \(\phi\) for each controller (blue).
The implementation of the Kalman filter significantly influences the control input \(\textbf{u}\) - this is rationalized by recognizing that \(\textbf{u} = -\textbf{K}_\text{LQR} \textbf{x}\), and as the controller receives a state estimate with reduced noise from the Kalman filter, it results in a more subdued and less abrupt control input, while still yielding better results.
Next, we hike up the satellite moment of inertia to \(J_s\) = 10 kg m². The plant \(\textbf{A}\) and \(\textbf{B}\) matrices are recalculated, but the LQR gain is not adjusted to the new parameter.
The LQR and LQG controllers currently demonstrate similar behavior, and neither controller is failing to regulate yet. However, the overall system performance is not promising, indicating that further variation of \(J_s\) will lead to controller failure.
We proceed to \(J_s\) = 15 kg m², and things start to go haywire. Both controllers exhibit initial signs of failure, partially in the angular displacement states \(\theta\) and \(\phi\), where oscillations with a growing exponential envelope are emerging. Interestingly, the LQG controller, for the first time, showcases a larger deviation from 0 compared to the LQR controller. This divergence in behavior raises concerns about the stability and effectiveness of the controllers under the updated system parameters.
Despite the escalating instability and oscillations in the controlled system, it’s noteworthy that the average magnitude of the controller input has become comparable for both controllers.
Finally, just for fun, let’s look at \(J_s\) = 30 kg m². While both controllers seem to be badly failing at regulating the plants, the LQG controller is doing far worse than the LQR controller. \(\max{|\theta|}\) for the LQR-controlled system is 42 radians, whereas \(\max{|\theta|}\) for the LQG-controlled system is a whopping 133 radians!
This is a wildly unrealistic scenario (because there is no way a satellite company would send a satellite with moment of inertia six times its intended value into space) but it does work as a valuable illustration of the delicate balances and tradeoffs in control system design. While an LQG controller may outperform an LQR controller for a perfect plant, deviations from the assumed model parameters can lead to unexpected and counterintuitive outcomes. This highlights the importance of considering robustness and adaptability in control system design, especially when dealing with real-world systems that are subject to uncertainties and variations.
Taking a look at the controller inputs reveals a consistent pattern: the LQG controller consistently requires larger inputs compared to the LQR controller, despite yielding significantly poorer results.
The introduction of noise in the system is a game-changer for LQG compared to LQR. While LQR assumes perfect knowledge of the system state, LQG acknowledges and accounts for the imperfections in state information through a Kalman filter. This extra layer of complexity in LQG, aimed at handling uncertainties, introduces the potential for errors in the state estimation process. If the estimation is inaccurate, the controller’s decisions may be based on flawed information. Consequently, the control actions taken by LQG might not be as optimal as those of LQR in situations where noise is minimal or absent.
Essentially, LQG’s performance can be influenced by the accuracy of the state estimation, and in scenarios with low noise, the additional complexity of the estimator might lead to suboptimal results compared to the simpler, noise-agnostic LQR controller. It’s a trade-off between robustness to uncertainties and potential performance degradation in scenarios where uncertainties are minimal.
6. Future Work
Additionally, in the case we return to this model in the future, we would like to generalize the problem to incorporate more controllers than optimal LQR and LQG controllers, such as H-infinity synthesis. In addition, representing the variation of the model parameters using MATLAB’s integrated uss command and other tools in the Robust Control Toolbox instead of manually testing certain values would allow for a more comprehensive analysis of robust stability and performance.
We would also like to learn how to set an RNG seed in Simulink to create reproducible results; while this is trivially simple in Python for libraries like NumPy, there does not exist a simple way to do it using the Digital Signal Processing toolbox’s Random Source block.
A. Code
clear, clc, clf
Js = 5 % kg m^2
Jp = 1 % kg m^2
k = 0.1 % N m
b = 0.05 % N m s
A = [0 0 1 0;
-k/Js k/Js -b/Js b/Js;
0 0 0 1;
k/Jp -k/Jp b/Jp -b/Jp];
B = [0 1/Js 0 0].';
C = eye(4);
D = [0;0;0;0];
sys = ss(A, B, C, D);
sys.StateName = {'theta [rad]'; 'phi [rad]'; 'theta dot [rad/s]'; 'phi dot [rad/s]'};
sys.InputName = {'input torque [N m]'};
sys.OutputName = {'theta [rad]'; 'phi [rad]'; 'theta dot [rad/s]'; 'phi dot [rad/s]'};
rank(ctrb(sys))
pole(sys)
Q = diag([(pi/6)^-2, (pi/6)^-2, (1/4)^-2, (1/4)^-2]);
% Bryson's rule - max error of angles is pi/6, max error of rates is 1/4
R = 1;
K_lqr = lqr(A, B, Q, R)
% Js = 5 % kg m^2
Js = 15 % kg m^2
% Js = 30 % kg m^2
A = [0 0 1 0;
-k/Js k/Js -b/Js b/Js;
0 0 0 1;
k/Jp -k/Jp b/Jp -b/Jp];
B = [0 1/Js 0 0].';B. References
- R. Beard, T. McLain, C. Peterson, and M. Killpack, “Introduction to Feedback Control using Design Studies,” 2016.
- J. C. Doyle, “Guaranteed margins for LQG regulators,” IEEE Transactions on Automatic Control, vol. 23, no. 4, pp. 756-757, 1978.
- R. M. Murray et al., “Optimization-based control,” California Institute of Technology, CA, 2023.