2026-09-19T10:14:53Zhttps://dspace.mit.edu/server/oai/requestoai:dspace.mit.edu:1721.1/756532022-01-13T07:54:29Zcom_1721.1_7582com_1721.1_7581col_1721.1_131022
Boris Katz.
Mohammad, Ali (Ali H.)
Massachusetts Institute of Technology. Dept. of Electrical Engineering and Computer Science.
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
2012-12-13T18:49:30Z
2012-12-13T18:49:30Z
2012
2012
http://hdl.handle.net/1721.1/75653
818354322
Thesis (Sc. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2012.
Cataloged from PDF version of thesis.
Includes bibliographical references (p. 133-139).
We introduce CLAIRE, a mathematically principled model for inferring ranks and scores for arbitrary items based on forced-choice binary comparisons, and show how to apply this technique to statistical models to take advantage of problem-specific assistance from non-experts. We apply this technique to two language processing problems: parsing and machine translation. This leads to an analysis which casts doubts on modern evaluation methods for machine translation systems, and an application of CLAIRE as a new technique for evaluating machine translation systems which is inexpensive, has theoretical guarantees, and correlates strongly in practice with more expensive human judgments of system quality. Our analysis reverses several major tenants of the mainstream machine translation research agenda, suggesting in particular that the use of linguistic models should be reexamined.
by Ali Mohammad.
Sc.D.
139 p.
eng
Massachusetts Institute of Technology
M.I.T. theses are protected by
copyright. They may be viewed from this source for any purpose, but
reproduction or distribution in any format is prohibited without written
permission. See provided URL for inquiries about permission.
http://dspace.mit.edu/handle/1721.1/7582
Electrical Engineering and Computer Science.
CLAIRE makes machine translation BLEU no more
Thesis
application/pdf
false
Publication
unknown
unknown
<Publication xmlns="https://www.openaire.eu/cerif-profile/1.1/" id="772b4d37-f8e6-4b6e-8d4c-14f8e6b9a158">
<Type xmlns="https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types">http://purl.org/coar/resource_type/c_1843</Type>
<Language>eng</Language>
<Title>CLAIRE makes machine translation BLEU no more</Title>
<PublishedIn>
<Publication>
</Publication>
</PublishedIn>
<PublicationDate>2012</PublicationDate>
<Authors>
<Author>
<DisplayName>Mohammad, Ali (Ali H.)</DisplayName>
<Affiliation>
<OrgUnit>
</OrgUnit>
</Affiliation>
</Author>
</Authors>
<Editors>
</Editors>
<Publishers>
<Publisher>
<DisplayName>Massachusetts Institute of Technology</DisplayName>
<OrgUnit />
</Publisher>
</Publishers>
<License>http://dspace.mit.edu/handle/1721.1/7582</License>
<Keyword>Electrical Engineering and Computer Science.</Keyword>
<Abstract>We introduce CLAIRE, a mathematically principled model for inferring ranks and scores for arbitrary items based on forced-choice binary comparisons, and show how to apply this technique to statistical models to take advantage of problem-specific assistance from non-experts. We apply this technique to two language processing problems: parsing and machine translation. This leads to an analysis which casts doubts on modern evaluation methods for machine translation systems, and an application of CLAIRE as a new technique for evaluating machine translation systems which is inexpensive, has theoretical guarantees, and correlates strongly in practice with more expensive human judgments of system quality. Our analysis reverses several major tenants of the mainstream machine translation research agenda, suggesting in particular that the use of linguistic models should be reexamined.</Abstract>
<Access xmlns="http://purl.org/coar/access_right"
>
</Access>
</Publication>