Quantitative empirical trends in technical performance

Abstract

Technological improvement trends such as Moore’s law and experience curves have been widely used to understand how technologies change over time and to forecast the future through extrapolation. Such studies can also potentially provide a deeper understanding of R&D management and strategic issues associated with technical change. However, this requires that methodological approaches for these analyses be addressed and compared to more effectively interpret results. Our analysis of methodological issues recommends less ambiguous approaches to: 1) the unit of analysis; 2) choice of the metrics within a unit of analysis; 3) the relationships among possible independent variables; and 4) qualitative and quantitative data quality considerations. The paper then uses this methodology to analyze performance trends for 28 technological domains with the following findings: 1. Sahal’s relationship is tested for several effort variables (for patents and revenue in addition to cumulative production where it was first developed). 2. The relationship is quite accurate when all three relationships, ( a. an exponential between performance and time, b. an exponential of effort and time and c. a power law between performance and the effort variable) have good data fits (r2 >0.7) . 3. The power law and effort exponents determined are dependent upon the choice of effort variable but the time dependence exponential is not. 4. In domains where the quantity of patents do not increase exponentially with time, Sahal’s relationship gives poor estimates even though Moore’s law is followed even for these domains. 5. Good data quality for any of the relationships depends upon adequate screening involving not only r2 but also the confidence interval based upon two different statistical tests; by these measures, all 28 domains have high quality fits between the log of performance and time whereas less than ½ show this level of quality for power law fits with patents as the effort variable. Overall, the results are interpreted as indicating that Moore’s law is a better description of longer-term technological change when the performance data come from various designs whereas experience curves may be more relevant when a singular design in a given factory is considered.