Project of Computational Mathematics for Learning and Data Analysis course @ Department of Computer Science of University of Pisa

(M) is a so-called extreme learning, i.e., a neural network with one hidden layer, y=W2σ(W1x), where the weight matrix for the hidden layer W1 is a fixed random matrix, σ(⋅) is an elementwise activation function of your choice, and the output weight matrix W2 is chosen by solving a linear least-squares problem (with L_2 regularization).

(A1) is an algorithm of the class of accelerated gradient methods, cf. also here, applied to (M1).

(A2) is a closed-form solution with the normal equations and your own implementation of Cholesky (or LDL) factorization.