City Research Online

Abstract

Generalised Linear Models are widely used for analysing multivariate data with non-normal responses for which the Iteratively Reweighted Least Squares algorithm is the main device to solve small and medium scaled problems. The popularity of this algorithm is due to the fact that is reduced to solving a series of weighted least square instances that are computationally less expensive than the general purpose optimisation algorithms that could solve the underlying maximum likelihood problem. Deploying the Iteratively Reweighted Least Squares algorithm may be affected by convergence issues, sensitivity to starting values, and more importantly, by significant parameter estimation error. A recent paper has shown how effective shrinkage estimation could be to improve the estimation error of the ordinary least square estimator without increasing the computational cost. The efficiency of these novel shrinkage estimators is explained by their design to reduce the theoretical Mean Square Error that is achieved by introducing a small bias with a sizable reduction in the new estimators’ variance. We show in this paper that the Iteratively Reweighted Least Squares algorithm could significantly benefit from replacing in each iteration the least square estimators with our shrinkage estimators. In addition, we introduce an optimisation method to obtain more reliable starting values, further enhancing convergence. Simulation studies and real-data applications demonstrate that our proposed methods improve convergence speed, stability, and overall performance compared to standard Generalised Linear Model non-penalised implementations.

Publication Type:	Other (Preprint)
Additional Information:	Copyright the authors, 2025.
Publisher Keywords:	Generalised Linear Model, Shrinkage Estimation, Iteratively Reweighted Least Square
Subjects:	H Social Sciences > HF Commerce Q Science > QA Mathematics
Departments:	Bayes Business School Bayes Business School > Actuarial Science & Insurance
SWORD Depositor:	Symplectic Administrator

Export

Downloads