Remote data access and analysis using SciDB
Author(s)Anderson, Alan M., M. Eng. Massachusetts Institute of Technology
Remote high resolution data access and calculation using SciDB
Massachusetts Institute of Technology. Dept. of Electrical Engineering and Computer Science.
Lewis Girod and Sam Madden.
MetadataShow full item record
SciDB is an innovative data analysis system that provides fast querying and manipulation of large amounts of time-series, scientific data. This thesis describes the design of a framework that provides a user interface to SciDB that facilitates interactive processing of large datasets and supports long-running batch jobs on a remote server or cluster. Using this interface, python user scripts access SciDB data, process it, and write new result arrays. The framework addresses problems such as garbage collection, data access permissions and maintenance of provenance. We present a case study in which we apply this framework to data from the WaterWiSe project, and analyze the runtime performance of the system.
Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2012.Cataloged from PDF version of thesis.Includes bibliographical references (p. 63).
DepartmentMassachusetts Institute of Technology. Dept. of Electrical Engineering and Computer Science.
Massachusetts Institute of Technology
Electrical Engineering and Computer Science.