2026-09-19T00:40:09Zhttps://dspace.mit.edu/server/oai/requestoai:dspace.mit.edu:1721.1/1432222022-06-16T03:24:03Zcom_1721.1_7582com_1721.1_7581col_1721.1_131022
Pentland, Alex P.
Dubey, Abhimanyu
Program in Media Arts and Sciences (Massachusetts Institute of Technology)
2022-06-15T13:04:38Z
2022-06-15T13:04:38Z
2022-02
2022-02-27T16:55:37.195Z
https://hdl.handle.net/1721.1/143222
In this thesis, we study sequential multi-armed bandit and reinforcement learning in the federated setting, where a group of agents collaborates to improve their collective reward by communicating over a network.
We first study the multi-armed bandit problem in a decentralized environment. We study federated bandit learning under several real-world environmental constraints, such as differentially private communication, heavy-tailed perturbations, and the presence of adversarial corruptions. For each of these constraints, we present algorithms with near-optimal regret guarantees and maintain competitive experimental performance on real-world networks. We characterize the asymptotic and minimax rates for these problems via network-dependent lower bounds as well. These algorithms provide substantial improvements over existing work in a variety of real-world and synthetic network topologies.
Next, we study the contextual bandit problem in a federated learning setting with differential privacy. In this setting, we propose algorithms that match the optimal rate (up to poly-logarithmic terms) with only a logarithmic communication budget. We extend our approach to heterogeneous federated learning via a kernel-based approach, and also provide a no-regret algorithm for private Gaussian process bandit optimization.
Finally, we study reinforcement learning in both the multi-agent and federated setting with linear function approximation. We propose variants of least-squares value iteration algorithms that are provably no-regret with only a constant communication budget.
We believe that the future of machine learning entails large-scale cooperation between various data-driven entities, and this work will be beneficial to the development of reliable, scalable, and secure decision-making systems.
Ph.D.
Massachusetts Institute of Technology
In Copyright - Educational Use Permitted
Copyright MIT
http://rightsstatements.org/page/InC-EDU/1.0/
Private and Provably Efficient Federated Decision-Making
Thesis
application/pdf
Doctoral
Doctor of Philosophy
Publication
unknown
unknown
<Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="a19a0980-27d2-46ea-88bd-f1d4f86f456c">
<Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type>
<Title>Private and Provably Efficient Federated Decision-Making</Title>
<PublishedIn>
<Publication>
</Publication>
</PublishedIn>
<PublicationDate>2022-02</PublicationDate>
<Authors>
<Author>
<DisplayName>Dubey, Abhimanyu</DisplayName>
<Affiliation>
<OrgUnit>
</OrgUnit>
</Affiliation>
</Author>
</Authors>
<Editors>
</Editors>
<Publishers>
<Publisher>
<DisplayName>Massachusetts Institute of Technology</DisplayName>
<OrgUnit />
</Publisher>
</Publishers>
<License>http://rightsstatements.org/page/InC-EDU/1.0/</License>
<Abstract>In this thesis, we study sequential multi-armed bandit and reinforcement learning in the federated setting, where a group of agents collaborates to improve their collective reward by communicating over a network.
We first study the multi-armed bandit problem in a decentralized environment. We study federated bandit learning under several real-world environmental constraints, such as differentially private communication, heavy-tailed perturbations, and the presence of adversarial corruptions. For each of these constraints, we present algorithms with near-optimal regret guarantees and maintain competitive experimental performance on real-world networks. We characterize the asymptotic and minimax rates for these problems via network-dependent lower bounds as well. These algorithms provide substantial improvements over existing work in a variety of real-world and synthetic network topologies.
Next, we study the contextual bandit problem in a federated learning setting with differential privacy. In this setting, we propose algorithms that match the optimal rate (up to poly-logarithmic terms) with only a logarithmic communication budget. We extend our approach to heterogeneous federated learning via a kernel-based approach, and also provide a no-regret algorithm for private Gaussian process bandit optimization.
Finally, we study reinforcement learning in both the multi-agent and federated setting with linear function approximation. We propose variants of least-squares value iteration algorithms that are provably no-regret with only a constant communication budget.
We believe that the future of machine learning entails large-scale cooperation between various data-driven entities, and this work will be beneficial to the development of reliable, scalable, and secure decision-making systems.</Abstract>
<Access xmlns="http://purl.org/coar/access_right"
>
</Access>
</Publication>