2026-09-18T22:32:07Zhttps://dspace.mit.edu/server/oai/request

oai:dspace.mit.edu:1721.1/1036712022-01-13T07:54:01Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023

Suvrit Sra and Leslie Pack Kaelbling. Mariet, Zelda Elaine Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science. Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science 2016-07-18T19:11:30Z 2016-07-18T19:11:30Z 2016 2016 http://hdl.handle.net/1721.1/103671 953457802 Thesis: S.M., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2016. This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections. Cataloged from student-submitted PDF version of thesis. Includes bibliographical references (pages 63-66). As machine-learning techniques continue to require more data and become increasingly memory-heavy, being able to choose a subset of relevant, high-quality and diverse elements among large amounts of redundant or noisy data and parameters has become an important concern. Here, we approach this problem using Determinantal Point Processes (DPPs), probabilistic models that provide an intuitive and powerful way of balancing quality and diversity in sets of items. We introduce a novel, fixed-point algorithm for estimating the maximum likelihood parameters of a DPP, provide proof of convergence and discuss generalizations of this technique. We then apply DPPs to the difficult problem of detecting and eliminating redundancy in fully-connected layers of neural networks. By placing a DPP over a layer, we are able to sample a subset of neurons that perform non-overlapping computations and merge all other neurons of the layer into the previous diverse subset. This allows us to significantly reduce the size of the neural network while simultaneously maintaining a good performance. by Zelda Elaine Mariet. S.M. 66 pages eng Massachusetts Institute of Technology M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission. http://dspace.mit.edu/handle/1721.1/7582 Electrical Engineering and Computer Science. Learning and enforcing diversity with Determinantal Point Processes Thesis application/pdf false Publication unknown unknown <Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="34b31c9a-f996-48e2-b9b3-6fe1845577c7"> <Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type> <Language>eng</Language> <Title>Learning and enforcing diversity with Determinantal Point Processes</Title> <PublishedIn> <Publication> </Publication> </PublishedIn> <PublicationDate>2016</PublicationDate> <Authors> <Author> <DisplayName>Mariet, Zelda Elaine</DisplayName> <Affiliation> <OrgUnit> </OrgUnit> </Affiliation> </Author> </Authors> <Editors> </Editors> <Publishers> <Publisher> <DisplayName>Massachusetts Institute of Technology</DisplayName> <OrgUnit /> </Publisher> </Publishers> <License>http://dspace.mit.edu/handle/1721.1/7582</License> <Keyword>Electrical Engineering and Computer Science.</Keyword> <Abstract>As machine-learning techniques continue to require more data and become increasingly memory-heavy, being able to choose a subset of relevant, high-quality and diverse elements among large amounts of redundant or noisy data and parameters has become an important concern. Here, we approach this problem using Determinantal Point Processes (DPPs), probabilistic models that provide an intuitive and powerful way of balancing quality and diversity in sets of items. We introduce a novel, fixed-point algorithm for estimating the maximum likelihood parameters of a DPP, provide proof of convergence and discuss generalizations of this technique. We then apply DPPs to the difficult problem of detecting and eliminating redundancy in fully-connected layers of neural networks. By placing a DPP over a layer, we are able to sample a subset of neurons that perform non-overlapping computations and merge all other neurons of the layer into the previous diverse subset. This allows us to significantly reduce the size of the neural network while simultaneously maintaining a good performance.</Abstract> <Access xmlns="http://purl.org/coar/access_right" > </Access> </Publication>