Wrapster : semi-automatic wrapper generation for semi-structured websites

Abstract

Many information sources on the Web are semi-structured; hence there is an opportunity for automatic tools to process and extract their information for easy access through a uniform interface language. Wrapper generation is the creation of wrappers which contains scripts that extract and integrate data from data sources, mostly from Web data sources due to the large amount of data available on the World Wide Web. Despite ongoing efforts to automate the process of wrapper generation, wrappers frequently break due to formatting and layout changes in data sources. This thesis presents Wrapster, a new system that semi-automatically generates wrappers for semi-structured Web sources, improves wrapper robustness, and eliminates the need for programming skills and, to a large extent, the process of script creation. Wrapster's novel component is the repairing module that constantly checks if any wrapper script has failed and repairs the failing wrapper's script using stored extracted instances. In addition, Wrapster provides an interactive Web user interface to control the wrapper generation process, edit the generated wrappers, and test their scripts. Wrapster is being tested on the START Question Answering system; however, it is a generic tool to be used by any QA system that uses the Web as its knowledge base.