Standards in the data storage industry : emergence, sustainability, and the battle for platform leadership

Abstract

In order to cope with the continuous increase of magnetically stored data and of mission critical dependence on this data, storage systems must similarly increase in their functionality offerings, and with that, their complexity. The efficient management of the heterogeneous and complex aggregation of these systems is becoming one of the major challenges to IT customers. At the same time, hardware is becoming commoditized, and the industry is looking towards software for additional revenue generation. This document examines proprietary as well as open-standards attempts at solving the interoperability problem. The first attempt was made by EMC when it developed WideSky, a middleware software layer that would be able to manage third party hardware. It is shown that the aim was to eventually transform this middleware into a de facto standard and with that establish platform leadership in the industry. The WideSky effort failed, and the analysis of this failure attributes it to a lack of industry support and inability at establishing a sustainable value chain. Meanwhile, the industry players rallied around the SNIA body and adopted the SMI specification (SMI-S) as a standard. SMI-S adoption is on the rise, but although it has the formal backing of most of the storage industry firms, it has not yet fulfilled its promise of enabling centralized management of heterogeneous systems. This is partially because of the fact that the functionality that it provides is still lagging behind the functionality that native APIs provide. Moreover, client adoption and the availability of client products that can be directly used by IT customers are still very limited.

(cont.) However, an examination of the dynamics surrounding this standard show how SMI-S will benefit greatly from learning effects and network externalities as it continues to grow, and although lagging in traditional functionality, it offers an ancillary functionality of interoperability that is missing from current non- standardized software interfaces. The adoption tipping point is highly dependant on whether or not the value chain can be established before vendors start dropping support for the specification. It is proposed that a positive tipping of the market will make SMI-S a disruptive technology that has the potential of becoming the dominant design for storage management interfaces.