This paper provides a framework to deal with large data samples which are difficult to oversee. When different stakeholders have different objectives, when different data sources could differ in quality, when model techniques could result in different outcomes, a uniform approach to assess performance is advised. A standardized model will make non-parametric assessments more reliable, more repeatable, and less costly.
We proposed a framework which consists of 6 interrelated phases: (1) Concepts and objectives, (2) On structuring data, (3) Operational models, (4) Performance comparison model, (5) Evaluation, and (6) Results and deployment. Abbreviated, we obtain the ‘COOPER-framework’. The framework provides both support and a step-by-step plan for the novice researcher, as well as a check-list for the experienced researcher. It is a tool which can be further adapted and modified along the specific needs of the researcher.
This paper also provides some interesting and promising lines for further research. Firstly, the Cooper-framework could benefit from the interaction with empirical applications. Indeed, a similar framework should never be finished and always be open for new developments. Potential applications of the framework consist of educational questions (e.g., the OECD Pisa dataset), business performance (e.g., World Economic Forum), consumer confidence, and the analysis of large statistical databases (e.g., on company performances). The practitioner applying the framework to a particular application may tailor the framework to his/her specific needs. Secondly, although extending the idea of the framework from the outlined DEA model to alternative methodologies (FDH, SFA and parametric models) is rather straightforward, not every phase and checklist item is applicable. We consider it as further research to create a similar framework for other methodologies. Finally, the framework will definitely benefit from new developments in the academic literature. As computing power grows and methodological advances are made, the phases will further evolve.
References
Allen, R., A. Athanassopoulos, R.G. Dyson and E. Thanassoulis (1997), Weights restrictions and value judgments in Data Envelopment Analysis: Evolution, development and future directions. Annals of Operations Research 73, 13-34.
Andersen, P. and N. Petersen (1993), A procedure for ranking efficient units in data envelopment analysis. Management Science, 39 (10), 1261-1264.
Avkiran, N. (1999), An application reference for data envelopment analysis in branch banking: helping the novice researcher. International Journal of Bank Marketing 17 (5), 206-220.
Banker, R. D. (1984), Estimating most productive scale size using data envelopment analysis. European Journal of Operational Research, 17(1), 35-44.
Banker, R. D., A. Charnes and W. W. Cooper (1984), Some models for estimating technical and scale inefficiencies in data envelopment analysis. Management Science 30.
Banker, R. D., A. Charnes, W. W. Cooper, J. Swarts and D. Thomas (1989), An introduction to data envelopment analysis with some of its models and their uses. Research in Governmental and Nonprofit Accounting 5, 125–163.
Banker, R. D., W. W. Cooper, E. Grifell-Tatjé, J. T. Pastor, P. W. Wilson, E. Ley and C. A. K. Lovell (1994), Validation and generalization of DEA and its uses. TOP 2 (2), 249-314.
Banker, R. D., V. M. Gadh and W. L. Gorr (1993), A Monte Carlo comparison of two production frontier estimation methods: Corrected ordinary least squares and data envelopment analysis, European Journal of Operational Research, 67(3), 332-343.
Banker, R. D. and R. Natarajan (2004), Statistical Tests Based on DEA Efficiency Scores, Chapter 11 in Handbook on Data Envelopment Analysis, W.W. Cooper, L. Seiford and J. Zhu (Eds.), Kluwer Academic Publishers, Norwell, MA, pp. 299-321.
Belton, V. and S. Vickers (1993), Demystifying DEA – A Visual Interactive Approach Based on Multiple Criteria Analysis. The Journal of the Operational Research Society 44 (9), 883-896.
Blanchard, B. S., and W. J. Fabrycky(2006), Systems engineering and analysis, 4th edition, New Jersey: Prentice Hall.
Bogetoft, P. (1997), DEA-based yardstick competition: The optimality of best practice regulation. Annals of Operations Research 73, 277-298.
Brown. R. (2006), Mismanagement or mismeasurement? Pitfalls and protocols for DEA studies in the financial services sector. European Journal of Operational Research 174, 1100-1116.
Camanho, A. S., M. C. Portela and C. B. Vaz (2009), Efficiency analysis accounting for internal and external non-discretionary factors, Computers & Operations Research, 36(5), 1591-1601.
Cazals C., J. Florens and L. Simar (2002), Nonparametric frontier estimation: a robust approach. Journal of Econometrics 106, 1-25.
Cerrito, P. B. (2007), Introduction to Data Mining Using SAS Enterprise Miner, SAS Publishing, p. 468.
Charnes, A., W. W. Cooper and E. Rhodes (1978), Measuring the efficiency of decision making units. European Journal of Operational Research 2, 429–444.
Charnes, A., W. W. Cooper, B. Golany, L. Seiford and J. Stutz (1985), Foundations of data envelopment analysis for Pareto-Koopmans efficient empirical production functions. Journal of Econometrics 30, 91-107.
Chen, C-M (2009), A network-DEA model with new efficiency measures to incorporate the dynamic effect in production networks, European Journal of Operational Research, 194 (3), 687-699.
Cherchye, L., K. De Witte, E. Ooghe and I. Nicaise (2009), Equity and Efficiency in Private and Public Education: a nonparametric comparison. CES Discussion Paper Series DPS 07.25. Forthcoming in European Journal of Operational Research.
Cook, W. D. and J. Zhu (2006), Rank order data in DEA: A general framework. European Journal of Operational Research 174 (2), 1021-1038.
Cook, W. D. and J. Zhu (2008), Classifying inputs and outputs in data envelopment analysis, European Journal of Operational Research, 180 (2), 692-699,
Cooper, W. W., K. S. Park and J. T. Pastor (1999), RAM: A range measure of inefficiency for use with additive models, and relations to other models and measures in DEA. Journal of Productivity analysis 11, 5-42.
Cooper, W.W., L. Seiford and J. Zhu (2004), Handbook of DEA. Kluwer Academic Publishers.
Daraio C. and L. Simar (2005), Introducing environmental variables in nonparametric frontier models: a probabilistic approach. Journal of Productivity Analysis 24, 93-121.
Daraio C. and L. Simar (2007), Advanced robust and nonparametric methods in efficiency analysis. Series: Studies in Productivity and Efficiency, Springer.
De Borger, B. and K. Kerstens (1996), Radial and nonradial measures of technical efficiency: An empirical illustration for Belgian local governments using an FDH reference technology. Journal of Productivity Analysis 7 (1), 41-62.
Deprins, D., L. Simar and H. Tulkens (1984), Measuring labor efficiency in post offices, The Performance of Public Enterprises: Concepts and Measurements, M. Marchand, P. Pestieau and H. Tulkens (eds.), Amsterdam, North-Holland, 243.267.
De Witte, K. and M. Kortelainen (2008), Blaming the exogenous environment? Conditional efficiency estimation with continuous and discrete environmental variables. CES Discussion Paper Series PS 08.33; MPRA Paper 14034.
De Witte, K. and R. Marques (2010), Designing incentives to local public utilities, an international comparison to the drinking water sector. Central European Journal of Operations Research, In Press.
De Witte, K. and R. C. Marques (2009), Influential observations in frontier models, a robust non-oriented approach to the water sector. mimeo.
Dula, J. H. and R. M. Thrall (2001), A Computational Framework for Accelerating DEA. Journal of Productivity Analysis 16 (1), 63-78.
Dyson, R. G., R. Allen, A. S. Camanho, V. V. Podinovski, C. S. Sarrico, E. A. Shale (2001), Pitfalls and protocols in DEA. European Journal of Operational Research 132 (2), 245-259.
Emrouznejad A. (2003), An alternative DEA measure: A case of OCED countries, Applied Economic Letters 10, 779–782.
Emrouznejad A. (2005), Measurement efficiency and productivity in SAS/OR, Computers and Operations Research, 32, 1665–1683.
Emrouznejad, A. and G. R. Amin (2009), DEA models for ratio data: Convexity consideration, Applied Mathematical Modelling, 33 (1), 486-498.
Emrouznejad A. and A. L. Anouze (2010), DEA/C&R: DEA with classification and regression tree: a case of banking efficiency, Expert Systems, in Press.
Emrouznejad, A., B. Parker and G. Tavares (2008), Evaluation of research in efficiency and productivity: A survey and analysis of the first 30 years of scholarly literature in DEA. Journal of Socio-Economic Planning Sciences, 42(3) 151-157.
Emrouznejad, A. and E. Thanassoulis (2005), A mathematical model for dynamic efficiency using data envelopment analysis, Applied Mathematics and Computation 160(2), 363-378.
Emrouznejad, A. and E. Thanassoulis (2010), Performance Improvement Management Software (PIMsoft): a user guide, www.DEAsoftware.co.uk.
Emrouznejad, A. and E. Thanassoulis (2010), Measurement of productivity index with dynamic DEA, International Journal of Operational Research 8(2) 247-260.
Emrouznejad, A., A. L. Anouze and E. Thanassoulis (2010a), A semi-oriented radial measure for measuring the efficiency of decision making units with negative data, using DEA, European Journal of Operational Research 200(1) 297-304.
Emrouznejad, A., G. R. Amin, E. Thanassoulis and A. L. Anouze (2010b), On the boundedness of the SORM DEA models with negative data, European Journal of Operational Research 206(1) 265-268.
Estache, A., M. Rossi and C. Ruzzier (2004), The case for international coordination of electricity regulation: evidence from the measurement of efficiency in South America. Journal of Regulatory Economics, 25(3), 271–295.
Färe R. And S. Grosskopf (1996) Intertemporal Production Frontiers: With Dynamic DEA, Boston: Kluwer Academic Publishers.
Färe, R. and S. Grosskopf (2000), Network DEA, Socio-Economic Planning Sciences 34(1), 35-49.
Färe, R. and C. A. K. Lovell (1978), Measuring the technical efficiency of production. Journal of Economic Theory 19 (1), 150-162.
Fried, H. O., C. A. K. Lovell and S. S. Schmidt (2008), The measurement of productive efficiency and productivity growth. Oxford University Press.
Greene, W. (2008), Econometric Analysis, 6th Edition. Prentice Hall.
Grifell-Tatjé, E. and C. A. K. Lovell (1999), Profits and productivity. Management Science 45 (9), 1177-1193.
Grinstein G., P. Hoffman and R. Pickett (2002), Benchmark Development for the Evaluation of Visualization for Data Mining. In Fayyad U., G. Grinstein and A. Wierse, Information Visualization in Data Mining and Knowledge Discovery.
Hollingsworth, B. (2008), The measurement of efficiency and productivity of health care delivery. Health Economics 17, 1007-1028.
Kao, C. and S. Liu (2000), Data envelopment analysis with missing data: an application to University libraries in Taiwan. Journal of the Operational Research Society 51, 897–905.
Kerstens, K. and P. Vanden Eeckaut (1999), Estimating returns to scale using non-parametric deterministic technologies: A new method based on goodness-of-fit. European Journal of Operational Research 113 (1), 206-214.
Kittelsen, S. (1993), Stepwise DEA. Choosing variables for measuring technical efficiency in Norwegian electricity distribution. Memorandum No. 6/93 from the Department of Economics, University of Oslo.
Kleine, A. (2004), A general model framework for DEA. Omega 32 (1), 17-23.
Kuosmanen, T. (2009) Data envelopment analysis with missing data, Journal of the Operational Research Society 60(12), 1767-1774.
Kuosmanen, T., and M. Kortelainen (2007) Stochastic Nonparametric Envelopment of Data: Cross-Sectional Frontier Estimation Subject to Shape Constraints, University of Joensuu, Economics Discussion Paper No. 46.
Kumbhakar S. C. and C.A.K. Lovell (2000), Stochastic Frontier Analysis. Cambridge University Press, Cambridge, UK.
Langford, I. and T. Lewis (1998), Outliers in Multilevel data. Journal of the Royal Statistical Society: Series A 161 (2), 121-160.
Marques, R. C. (2006), A yardstick competition model for Portuguese water and sewerage services regulation. Utilities Policy 14 (3), 175-184.
Meeusen, W. and J. van den Broeck (1977), Efficiency estimation from Cobb-Douglas production functions with composed error. International Economic Review 18 (2), 435-444.
OECD (2008), Handbook on Constructing Composite Indicators: Methodology and User Guide. OECD Publishing.
Olson D. L. and D. Delen (2008), Advanced Data Mining Techniques, Springer, p. 180
Pedraja-Chaparro, F., J. Salinas-Jimenez and P. Smith (1997), On the Role of Weight Restrictions in Data Envelopment Analysis. Journal of Productivity Analysis 8 (2), 215-230.
Pedraja-Chaparro, F., J. Salinas-Jimenez, P. Smith (1999), On the Quality of the Data Envelopment Analysis Model. The Journal of the Operational Research Society 50 (6), 636 – 644.
Podinovski, V. (1999), Side effects of absolute weight bounds in DEA models. European Journal of Operations Research 115 (3), 583-595.
Podinovski, V. (2004), Local and global returns to scale in performance measurement. Journal of the Operational Research Society 55, 170–178.
Portela, M. and E. Thanassoulis (2002), Profit efficiency in DEA. Aston Business School Research Paper RP 0206.
Portela, M., E. Thanassoulis and G. Simpson (2004), Negative data in DEA: a directional distance approach applied to bank branches. The journal of the Operational Research Society 55 (10), 1111-1121.
Ray, Subhash C. (2004) Data Envelopment Analysis: Theory and Techniques for Economics and Operations Research, Cambridge University Press, Cambridge, UK.
Ruggiero J. (2004) Data Envelopment Analysis with stochastic data, Journal of the Operational Research Society, 55(9),1008–12.
Schaffnit, C., D. Rosen and J. C. Paradi (1998), Best practice analysis of bank branches: An application of DEA in a large Canadian bank, European Journal of Operational Research 98(2), 269-289.
Sengupta , J. K. (1995) Dynamics of Data Envelopment Analysis: Theory of Systems Efficiency, Kluwer Academic Publishers, London
Sengupta J. K. (1998) Stochastic Data Envelopment Analysis: a new approach, Applied Economics Letters 5(5), 287.
Simar, L. (1996), Aspects of statistical analysis in DEA-type frontier models. Journal of Productivity Analysis 7 (2-3), 177-185.
Simar, L. (2003), Detecting outliers in frontier models: a simple approach. Journal of Productivity Analysis 20, 391-424.
Simar, L. and P. Wilson (1998), Sensitivity Analysis of Efficiency Scores: How to Bootstrap in Nonparametric Frontier Models. Management Science 44 (1), 49-61.
Simar, L. and P. Wilson (1999), Estimating and bootstrapping Malmquist indices. European Journal of Operational Research 115 (3), 459-471.
Simar, L. and P. Wilson (2002), Non-parametric tests of returns to scale. European Journal of Operational Research 139 (1), 115-132.
Simar, L. and P. Wilson (2007), Estimation and inference in two-stage, semi-parametric models of production processes. Journal of Econometrics 136 (1), 31-64.
Sousa, M and B. Stosic (2005), Technical efficiency of the Brazilian municipalities: correcting nonparametric frontier measurement of outliers. Journal of Productivity Analysis 24, 157-181.
Stolp, C. (1990), Strengths and Weaknesses of Data Envelopment Analysis: An Urban and Regional Perspective. Computer, Environment and Urban Systems 14 (2), 103-106.
Thanassoulis, E. (2001), Introduction to the theory and application of Data Envelopment Analysis. A foundation text with integrated software. Springer, p. 281.
Thompson, R., P. Dharmapala and R. Thrall (1993), Importance for DEA of zeros in data, multipliers, and solutions. Journal of Productivity Analysis 4 (4), 379-390.
Zhang, Y. and R. Bartels (1998), The Effect of Sample Size on the Mean Efficiency in DEA with an Application to Electricity Distribution in Australia, Sweden and New Zealand. Journal of Productivity Analysis 9 (3), 187-204.
Zhu, J. (2003), Quantitative Models for Performance Evaluation and Benchmarking: Data Envelopment Analysis with Spreadsheets and DEA Excel Solver. Springer, p. 297.
Zhu, J. and W. D. Cook (2007), Modeling Data Irregularities and Structural Complexities in Data Envelopment Analysis, Springer, p. 334.
