Publications

Initializing ReLU networks in an expressive subspace of weights

Dayal Singh and G J Sreejith, Under review

We analyze signal propagation in ReLU networks with anti-correlated weights and demonstrate that it has an order-to-chaos phase transition, unlike the uncorrelated case. Furthermore, we demonstrate that ReLU networks initialized at this criticality train faster.

Inferring long memory using extreme events

Dayal Singh Kalra and M. S. Santhanam, Chaos 31, 113131 (2021)

We study extreme events in time series with long-range correlations. We find that it is possible to infer the long memory of time series just by extreme events.