Distributed statistical learning problems arise commonly when dealing with large datasets. In this setup, datasets are partitioned over machines, which compute locally, and communicate short messages. Communication is often the bottleneck. paper, we study one-step iterative weighted parameter averaging in linear models under data parallelism. We do regression on each machine, send results to a ...

Abstract Economics and social science research often require analyzing datasets of sensitive personal information at fine granularity, with models fit to small subsets the data. Unfortunately, such fine-grained analysis can easily reveal individual information. We study regression algorithms that satisfy differential privacy , a constraint which guarantees an algorithm’s output reveals little a...

We study random design linear regression with no assumptions on the distribution of covariates and a heavy-tailed response variable. In this distribution-free setting, we show that boundedness conditional second moment given is necessary sufficient condition for achieving nontrivial guarantees. As starting point, prove an optimal version classical in-expectation bound truncated least squares es...