Consider some kind of ellipsoid where your data is constrained:
\begin{align} \mathcal{A} = \left\{\bar{A} + u_1 A_1 + \dots + u_{p} A_{p} \mid \norm{u}_{2} \leq 1\right\} \end{align}
You can form the “worst-case robust least squares”:
\begin{align} \min \text{sup}_{A \in \mathcal{A}} \norm{A x - b}_{2}^{2} = \min \text{sup}_{\norm{u}_{2} \leq } \norm{P\left(x\right) u + q\left(x\right)}_{2}^{2} \end{align}
This is usually a minimax problem, but taking the dual of the inner maximize thing turns out has zero duality gap.