2026-09-19T06:54:55Zhttps://dspace.mit.edu/server/oai/requestoai:dspace.mit.edu:1721.1/1195692026-06-06T00:55:00Zcom_1721.1_7582com_1721.1_7581col_1721.1_131023
Iyad Rahwan.
Mir, Remi
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science.
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
2018-12-11T20:40:31Z
2018-12-11T20:40:31Z
2018
2018
http://hdl.handle.net/1721.1/119569
1076275047
Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2018.
This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.
Cataloged from student-submitted PDF version of thesis.
Includes bibliographical references (pages 39-41).
In this thesis, we identify best practices for evaluating style modification, or style transfer, for text. Research of style transfer is bottlenecked by a lack of standard evaluation practices. We define three key aspects of interest (style transfer intensity, content preservation, and naturalness) and show how to obtain more reliable measures of them from human evaluation than in previous work. We also demonstrate stronger correlation between human judgment and a new set of automated metrics: the Wasserstein distance, word mover's distance on texts with style masked out, and adversarial classification for the respective aspects. Lastly, we illustrate aspect tradeoff curves for three state-of-the-art style transfer models to highlight the importance of evaluating style transfer models at specific points on the curves. This can enable direct comparison of the models, facilitating future research in style transfer.
by Remi Mir.
M.Eng.
41 pages
eng
Massachusetts Institute of Technology
MIT theses are protected by copyright. They may be viewed, downloaded, or printed from this source but further reproduction or distribution in any format is prohibited without written permission.
http://dspace.mit.edu/handle/1721.1/7582
Electrical Engineering and Computer Science.
Evaluating style modification in text
Thesis
application/pdf
false
Publication
unknown
unknown
<Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="8927e557-07e1-41c4-87fb-36ed71d735a8">
<Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type>
<Language>eng</Language>
<Title>Evaluating style modification in text</Title>
<PublishedIn>
<Publication>
</Publication>
</PublishedIn>
<PublicationDate>2018</PublicationDate>
<Authors>
<Author>
<DisplayName>Mir, Remi</DisplayName>
<Affiliation>
<OrgUnit>
</OrgUnit>
</Affiliation>
</Author>
</Authors>
<Editors>
</Editors>
<Publishers>
<Publisher>
<DisplayName>Massachusetts Institute of Technology</DisplayName>
<OrgUnit />
</Publisher>
</Publishers>
<License>http://dspace.mit.edu/handle/1721.1/7582</License>
<Keyword>Electrical Engineering and Computer Science.</Keyword>
<Abstract>In this thesis, we identify best practices for evaluating style modification, or style transfer, for text. Research of style transfer is bottlenecked by a lack of standard evaluation practices. We define three key aspects of interest (style transfer intensity, content preservation, and naturalness) and show how to obtain more reliable measures of them from human evaluation than in previous work. We also demonstrate stronger correlation between human judgment and a new set of automated metrics: the Wasserstein distance, word mover's distance on texts with style masked out, and adversarial classification for the respective aspects. Lastly, we illustrate aspect tradeoff curves for three state-of-the-art style transfer models to highlight the importance of evaluating style transfer models at specific points on the curves. This can enable direct comparison of the models, facilitating future research in style transfer.</Abstract>
<Access xmlns="http://purl.org/coar/access_right"
>
</Access>
</Publication>