Show simple item record

dc.contributor.authorPainsky, Amichai
dc.contributor.authorWornell, Gregory W
dc.date.accessioned2021-10-27T20:30:05Z
dc.date.available2021-10-27T20:30:05Z
dc.date.issued2020
dc.identifier.urihttps://hdl.handle.net/1721.1/135947
dc.description.abstract© 1963-2012 IEEE. A loss function measures the discrepancy between the true values and their estimated fits, for a given instance of data. In classification problems, a loss function is said to be proper if a minimizer of the expected loss is the true underlying probability. We show that for binary classification, the divergence associated with smooth, proper, and convex loss functions is upper bounded by the Kullback-Leibler (KL) divergence, to within a normalization constant. This implies that by minimizing the logarithmic loss associated with the KL divergence, we minimize an upper bound to any choice of loss from this set. As such the logarithmic loss is universal in the sense of providing performance guarantees with respect to a broad class of accuracy measures. Importantly, this notion of universality is not problem-specific, enabling its use in diverse applications, including predictive modeling, data clustering and sample complexity analysis. Generalizations to arbitary finite alphabets are also developed. The derived inequalities extend several well-known f-divergence results.
dc.language.isoen
dc.publisherInstitute of Electrical and Electronics Engineers (IEEE)
dc.relation.isversionof10.1109/TIT.2019.2958705
dc.rightsCreative Commons Attribution-Noncommercial-Share Alike
dc.rights.urihttp://creativecommons.org/licenses/by-nc-sa/4.0/
dc.sourcearXiv
dc.titleBregman Divergence Bounds and Universality Properties of the Logarithmic Loss
dc.typeArticle
dc.relation.journalIEEE Transactions on Information Theory
dc.eprint.versionAuthor's final manuscript
dc.type.urihttp://purl.org/eprint/type/JournalArticle
eprint.statushttp://purl.org/eprint/status/PeerReviewed
dc.date.updated2021-01-25T18:36:56Z
dspace.orderedauthorsPainsky, A; Wornell, GW
dspace.date.submission2021-01-25T18:36:58Z
mit.journal.volume66
mit.journal.issue3
mit.licenseOPEN_ACCESS_POLICY
mit.metadata.statusAuthority Work and Publication Information Needed


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record