EXPERIMENTAL SOFTWARE ENGINEERING NETWORK (ESERNET), a Thematic Network Project, IST-2000-28754)

Publication list for the field, taken from the ESERNET site (web portal) and originally edited by Claes Wohlin.

This section includes references to a selection of relevant publications in experimental software engineering. The main focus has been on books and journal publications, although the lists below contain some conference publications too. The main reason for this focus is that books and journal publications are normally easier to find than conference publications. This section is divided into subsections to make it easier to navigate in the information. The references primarily include software engineering references relevant for empirical studies, but it does also include some material from other domains that are relevant for software engineers. In particular, several books on research methodology in general are included so that anyone who wants to know more about research methodology in general can find suitable references. Moreover, they provide an opportunity to put the empirical research in software engineering into a broader context.

Books on Empirical Methods in SE 
Articles on Empirical Methods in Software Engineering 
Books and Articles on Infrastructures Related to Empirical Studies in Software Engineering 
Books on Research Methodology in General or from Other Disciplines 
Books and Articles on Empirical Methods in General and Infrastructures from Other Disciplines 
Specific Empirical Studies

Books on Empirical Methods in Software Engineering

R. Conradi, A.I. Wang (eds), "Empirical methods and Studies in Software Engineering - Experiences from ESERNET", Lecture Notes in Computer Science LNCS 2765, Springer Verlag, Berlin Germany, 2003.

A. Endres, H.D. Rombach, "A Handbook of Software and Systems Engineering: Empirical Observations, Laws and Theories", Harlow: Pearson Addison Wesley 2003.

N. Juristo and A. Moreno, "Basics of Software Engineering Experimentation", Kluwer Academic Publishers, 2001.

H. D. Rombach, V. R. Basili, and Richard W. Selby (Eds.), "Experimental Software Engineering Issues: Critical Assessment and Future Directions", International Workshop, Dagstuhl Castle, Germany, Sept. 1992, Springer Verlag LNCS 706, 261 p.

C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell, and A. Wesslen,"Experimentation in Software Engineering: An Introduction", Kluwer Academic Publishers. 2000. ISBN 0-792-38682-5.

Articles on Empirical Methods in Software Engineering

W. R. Adrion, "Research Methodology in Software Engineering", Summary of the Dagstuhl Workshop on Future Directions in Software Engineering, ACM Software Engineering Notes, Vol. 18 (1993), No. 1, p. 36-37.

V. R. Basili, "Quantitative Evaluation of Software Engineering Methodology", Proceedings of the First Pan Pacific Computer Conference, Melbourne, Australia, Vol. 1, pp. 379-398, 1985.

V. R. Basili, R. W. Selby, and D. H. Hutchens, "Experimentation in Software Engineering", IEEE Trans. on Software Engineering, SE-12(7):733-743, July 1986.

V. R. Basili, "Software Development: A Paradigm for the Future", Proceedings of the 13th Annual International Computer Software & Applications Conference, COMPSAC´89, pp. 471-485, Orlando, Florida, USA, 1989.

V. R. Basili, "The Experimental Paradigm in Software Engineering", Experimental Software Engineering Issues: Critical Assessment and Future Directives, editors: H. D. Rombach, V. R. Basili and R. W. Selby, Springer-Verlag, #706, Lecture Notes in Computer Science, 1993.

V. R. Basili, "The Role of Experimentation in Software Engineering: Past, Current and Future", Proceedings 18th International Conference on Software Engineering, Berlin, Germany, pp. 442-449, 1996.

V. R. Basili, F. Shull, and F. Lanubile, "Building Knowledge through Families of Experiments", IEEE Transactions on Software Engineering, Vol. 25, No. 4, pp. 456-473, July 1999.

N. Fenton, S. L. Pfleeger, and R. L. Glass, "Science and Substance: A Challenge to Software Engineers", IEEE Software, Vol. 11, No. 4 (July 1994), p. 86-95.

R. L. Glass, "The Software Research Crisis", IEEE Software, Vol. 11, No. 6 (Nov. 1994), p. 42-47.

B. A. Kitchenham, L. Pickard, and S. L. Pfleeger, "Case Studies for Method and Tool Evaluation", IEEE Software, Vol. 12, No. 4 (July 1995), p. 52-62.

B. A. Kitchenham, "Evaluating Software Engineering Methods and Tools -- Part 1: The Evaluation Context and Evaluation Methods", ACM Software Engineering Notes, Vol. 21 (1996), No. 1, p. 11-15.

B. A. Kitchenham, S.L. Pfleeger, L.M. Pickard, "Preliminary Guidelines for Empirical Research in Software Engineering", in IEEE Transactions on Software Engineering, vol: 28 issue: 8, pp 721-735, 2002.

S. Linkman and H. D. Rombach, "Experimentation as a Vehicle for Software Technology Transfer - A Family of Software Reading Techniques", Journal of Information and Software Technology, 39(11), pp. 777-780, 1997.

J. Miller, J. Daly, M. Wood, M. Roper, and A. Brooks, "Statistical Power and Its Subcomponents -- Missing and Misunderstood Concepts in Empirical Software Engineering Research", University of Strathclyde, UK, 1996, Report no. ISERN-96-12.

D. L. Parnas, "The Limits of Empirical Studies of Software Engineering", in Proceedings of the 2003 International Symposium on Empirical Software Engineering, IEEE Computer Society Press, Los Alamitos CA, 2003.

D. E. Perry, A. A. Porter an L. G. Votta, "Empirical Studies of Software Engineering: A Roadmap, Proceedings of the International Conference on Software Engineering, pp. 345-355, 2000.

S. Pfleeger, "Experimental Design and Analysis in Software Engineering Part 1-5", ACM Sigsoft, Software Engineering Notes, Vol. 19, No. 4, pp. 16-20; Vol. 20, No. 1, pp. 22-26; Vol. 20, No. 2, pp. 14-16; Vol. 20, No. 3, pp. 13-15; and Vol. 20, No. 4, pp. 14-17, 1994-1995.

S. L. Pfleeger, "Albert Einstein and Empirical Software Engineering", in IEEE Computer, vol: 32 issue: 10, pp 32-38, 1999.

C. Potts, "Software Engineering Research Revisited", IEEE Software, pp. 19-28, Sept. 1993.

C. B. Seaman, "Qualitative Methods in Empirical Studies of Software Engineering", IEEE Transactions on Software Engineering, Vol. 25, No. 4, July/Aug. 1999, p. 557-572.

F. Shull, V. Basili, J. Carver, J. Maldonado, G. Travassos, M. Mendonca, and S. Fabbri, "Replicating Software Engineering Experiments: Addressing the Tacit Knowledge Problem", Proceedings of the First International Symposium on Empirical Software Engineering, Nara, Japan, October 2002.

J. Singer, N. G. Vinson, "Ethical Issues in Empirical Studies of Software Engineering", TSE 28(12): 1171-1180 (2002).

S. E. Sim, J. Singer, M-A D. Storey, "Beg, Borrow, or Steal: Using Multidisciplinary Approaches in Empirical Software Engineering Research", Empirical Software Engineering 6(1): 85-93 (2001).

J. Singer, N. G. Vinson, "Why and How Research Ethics Matters to You. Yes, You!", Empirical Software Engineering 6(4): 287-290 (2001).

L. S. Sørumgård, "Verification of Process Conformance in Empirical Studies of Software Development", Feb. 1997, dr.ing. thesis 1997:14 at NTNU, IDI-rapport 3/97, 252 p.

W. F. Tichy, P. Lukowicz, L. Prechelt, and E. A. Heinz, "Experimental Evaluation in Computer Science: A Quantitative Study", Journal of Systems and Software, 28(1):9-18, January 1995.

W. F. Tichy, "Should Computer Scientists Experiment More", IEEE Computer, 31(5):32-40, May 1998.

N. G. Vinson, J. Singer, "Getting to the Source of Ethical Issues", Empirical Software Engineering 6(4): 293-297 (2001).

M. V. Zelkowitz and D. R. Wallace, "Experimental Models for Validating Technology", IEEE Computer, Vol. 31, No. 5 (May 1998), p. 23-31.

A. Zendler, "A Preliminary Software Engineering Theory as Investigated by Published Experiments", in Empirical Software Engineering, vol: 6 issue: 2, pp 161-180, 2001.

Books and Articles on Infrastructures Related to Empirical Studies in Software Engineering

K-D. Althoff, A. Birk, S. Hartkopf, W. Müller, M. Nick, D. Surmann, C. Tautz, "Managing Software Engineering Experience for Comprehensive Reuse", In Proc. 11th Conf. on Software Engineering and Knowledge Engineering (SEKE'99), 16-19 June 1999, Kaiserslautern, p. 10-19, Knowledge Systems Institute, Skokie, IL, USA, June 1999.

V. R. Basili, G. Caldiera, H. D. Rombach, "Experience Factory", Encyclopedia of Software Engineering, ed. J. J. Marciniak, Vol. I, pp. 469-476, Wiley, 1994.

V. R. Basili, G. Caldiera and H. D. Rombach, "Goal Question Metrics Paradigm", Encyclopedia of Software Engineering, ed. J. J. Marciniak, Vol. I, pp. 528-532, Wiley, 1994.

V. R. Basili and S. Green, "Software Process Evaluation at the SEL", IEEE Software, pp. 58-66, July 1994.

L. C. Briand, K. El Emam, and S. Morasca, "On the Application of Measurement Theory in Software Engineering", Empirical Software Engineering, Vol. 1 (1996), No. 1, p. 61-68.

L. C. Briand, C. M. Differding and H. D. Rombach, "Practical Guidelines for Measurement-Based Process Improvement", Software Process Improvement and Practice, 2(4), pp. 253-280, 1996.

J. Carver, L. Jaccheri, S. Morasca, F. Shull, "Issues in Using Students in Empirical Studies in Software Engineering Education", in Proceedings of the Ninth International Software Metrics Symposium, IEEE Computer Society Press, Los Alamitos CA, 2003.

R. Conradi and T. Dingsøyr, "Software Experience Bases: A Consolidated Evaluation and Status Report". In Frank Bomarius (ed.): Proc. Second Int'l Conf. On Product Focused Software Process Improvement (PROFES'2000), Oulu, Finland, 20-22 June 2000. Springer Verlag LNCS 1840, p. 391-406.

R. Conradi, M. Lindvall, and C. Seaman, "Success Factors for Software Experience Bases: What We Need to Learn from Other Disciplines", pp. 113-119. In J. Singer, M-A. Storey, and S. Elliot Sim (Eds.): Proc. ICSE'2000 Workshop on "Beg, Borrow or Steal: Using Multidisciplinary Approaches in Empirical Software Engineering Research", Limerick, Ireland, 5 June 2000.

R. E. Courtney and D. A. Gustafson, "Shotgun Correlations in Software Measure", Software Engineering Journal, Vol. 8, No. 1 (Jan. 1993), pp. 5-13.

N. Fenton, "When a Software Measure is Not a Measure", Software Engineering Journal, Vol. 7, No. 5 (Sept. 1992), pp. 357-362.

N. Fenton, "Software Measurement: A Necessary Scientific Basis", IEEE Transactions on Software Engineering, Vol. 20, No. 3 (March 1994), pp. 199-206.

N. Fenton, and S. L. Pfleeger, "Software Metrics: A Rigorous & Practical Approach", 2nd edition, International Thomson Computer Press, 1996.

R. L. Glass, "Software Creativity", Prentice Hall, Englewood Cliffs, N.J., 1995.

R. B. Grady and D. L. Caswell, "Software Metrics: Establishing a Company-Wide Program", Prentice-Hall, 1994.

B. Hetzel, "Making Software Measurement Work: Building an Effective Measurement Program", John Wiley and Sons, 1993.

W. S. Humphrey, "A Discipline for Software Engineering", Addison Wesley, 1995.

W. S. Humphrey, "Introduction to the Personal Software Process", Addison Wesley, 1997.

M. Höst, B. Regnell, C. Wohlin, "Using Students as Subjects-A Comparative Study of Students and Professionals in Lead-Time Impact Assessment", in Empirical Software Engineering, vol: 5 issue: 3, pp 201-214, 2000.

IEEE, "IEEE Standard Glossary of Software Engineering Terminology", IEEE Std 610.12-1990, 1990.

M. Jørgensen, "Software Quality Measurement", Advances in Engineering Software, Vol. 30, No. 12 (Dec. 1999), p. 907-912.

B. A. Kitchenham, N. Fenton, and S. L. Pfleeger, "Towards a Framework for Software Measurement Validation", IEEE Transactions on Software Engineering, Vol. 21 (1995), No. 12, p. 929-944.

B. A. Kitchenham, "Software Metrics -- Measurement for Software Process Improvement", Blackwell Publishers Inc., 1996.

B. A. Kitchenham, S. G. MacDonell, L. Pickard, M. J. Shepperd, "What Accuracy Statistics Really Measure", IEE Proceedings - Software Engineering, vol. 148, pp. 81-85, 2001.

T. Punter, M. Ciolkowski, B. Freimut, I. John, "Conducting On-line Surveys in Software Engineering", in Proceedings of the 2003 International Symposium on Empirical Software Engineering, IEEE Computer Society Press, Los Alamitos CA, 2003.

S. Rifkin, "Discipline of Market Leaders and Other Accelerators to Measurement", Proc. 24th Annual NASA-SEL Software Engineering Workshop (on CD-ROM), NASA Goddard Space Flight Center, Greenbelt, MD 20771, USA, 1-2 Dec. 1999, 6 p., see http://sel.gsfc.nasa.gov/website/sew/1999/program.html .

M. Shepperd, "Foundations of Software Measurement", Prentice-Hall, 1995.

R. van Solingen and E. Berghout, "The Goal/Question/Metric Method: A Practical Guide for Quality Improvement and Software Development", McGraw-Hill International, 1999.

C. Wohlin, A. Gustavsson, M. Höst and C. Mattsson, "A Framework for Technology Introduction in Software Organizations", Proceedings Conference on Software Process Improvement, Brighton, United Kingdom, pp. 167-176, 1996.

C. Wohlin, "The Personal Software Process as a Context for Empirical Studies", Software Process Newsletter, No. 12, pp. 7-12, 1998.

Books on Research Methodology in General or from Other Disciplines

American Psychological Association, "Ethical Principles of Psychologists and Code of Conduct", American Psychologist, pp. 1597-1611, Dec. 1992.

C. Argyris and D. A. Schön, "Organizational Learning II: Theory, Method, and Practice", Reading, Massachusetts: Addison-Wesley, 1996.

E. Babbie, "Survey Research Methods", Wadsworth, ISBN 0-524-12672-3, 1990.

D. T. Campbell and J. C. Stanley, "Experimental and Quasi-Experimental Designs for Research", Houghton Mifflin Company, Boston, MA, USA, 1963.

D. Cook and D. T. Campbell, "Quasi-Experimentation - Design & Analysis Issues for Field Settings", Houghton Mifflin, USA. 1979. ISBN 0-395-30790-2.

J. W. Creswell, "Research Design, Qualitative and Quantitative Approaches", Sage Publications, 1994.

N. K. Denzin and Y. S. Lincoln, "Handbook of Qualitative Research", Sage Publications, London, UK, 1994.

F. J. Fowler, "Survey Research Methods", Sage Publications Ltd, UK, 2001.

J. F. Gilgun, "Definitions, Methodologies, and Methods in Qualitative Famility Research", Qualitative Methods in Famility Research, Thousand Oaks, Sage, 1992.

S. K. Kachigan, "Statistical Analysis: An Interdisciplinary Introduction to Univariate & Multivariate Methods", Radius Press, New York, 1986.

S. K. Kachigan, "Multivariate Statistical Analysis - A Conceptual Introduction", 2nd Edition, Radius Press, New York, 1991.

F. N. Kerlinger, "Foundation of Behavioral Research", New York, Holt Rinehart and Winston Inc., 1988.

P. Kline, "An Easy Guide to Factor Analysis", Routledge, New York NY, 1994.

K. L. Kraemer, "The Information Systems Research Challenge: Survey Research Methods", Boston, Harvard Busines School, 1993.

G. Kunda, "Engineering Culture: Control and Commitment in a High-Tech Corporation", Temple University Press, Philadelphia, 1992.

J. Lave and E. Wenger, "Situated Learning: Legitimate Peripheral Participation", Cambridge University Press, 1991.

M. Levin and D. J. Greenwood, "Introduction to Action Research -- Social Research for Social Change", Sage, 1998.

B. F. J. Manly, "Multivariate Statistical Methods - A Primer", Second Edition, Chapman & Hall, London, 1994.

L. A. Marascuilo and R. C. Serlin, "Statistical Methods for the Social and Behavioral Sciences", W. H. Freeman and Company, New York, USA, 1988.

D. C. Montgomery, "Design and Analysis of Experiments", 4th edition, John Wiley& Sons, 1997.

Nonaka and H. Takeuchi, "The Knowledge-Creating Company", Oxford University Press, 1995.

D. S. Pugh and E. M. Phillips, "How to Get a PhD: A Handbook for Students and their Supervisors", 3rd edition, Feb. 2000, Open University Press, ISBN 0-335-20550-X (paperback), 256 p.

K. Popper, "The Logic of Scientific Discovery", New York, Harper Torchbooks, 1968.

C. Robson, "Real World Research: A Resource for Social Scientists and Practitioners-Researchers", Blackwell, 1993.

C. Robson, "Experiment, Design and Statistics in Psychology", 3rd edition, Penguin Books, London, England, 1994.

P. M. Senge, "The Fifth Discipline: The Art and Practice of the Learning Organization", Currency/Doubleday, 1990.

S. Siegel and J. Castellan, "Nonparametric Statistics for the Behavioral Sciences", 2nd edition, McGraw-Hill International Editions, 1988.

R. E. Stake, "The Art of Case Study Research", SAGE Publications, 1995.

W. Trochim, "The Research Methods Knowledge Base", 2nd Edition, Cornell Custom Publishing, Cornell University, Ithaca, New York, 1999.

W. F. Whyte, "Participatory Action Research", Newbury Park, CA., Sage publications, 1991.

R. K. Yin, "Case Study Research Design and Methods", 2nd edition. Thousand Oaks, CA., Sage Publications, John Wiley and Sons, London, UK, 1994.

Books and Articles on Empirical Methods in General and Infrastructures from Other Disciplines

D. R. Anderson, D. J. Sweeney, T. A. Williams, "An Introduction to Management Science: Quantitative Approaches to Decision Making", South Western College Publishing, Cincinnati Ohio, 2000.

F. J. Barrett, "Creativity and Improvisation in Jazz and Organization: Implications for Organizational Learning", Organization Science, 1998, Vol. 9, No. 5, p. 605-622.

K. Braa and R. Vidgen, "Interpretation, Intervention, and Reduction in the Organizational Laboratory: A Framework for In-Context Information System Research", Accounting Management and Information Technologies, Vol. 9 (1999), No. 9, p. 25-47.

N. M. Bradburn, "Question-Wording Effects in Surveys", In N. M. Bradburn (Ed.): New Directions for Methodology of Social and Behavioral Science: Question Framing and Response Consistency, Jossey-Bass, San Francisco, 1982.

British Psychological Society, "Ethical Principles for Conducting Research with Human Participants", The Psychologist, pp. 33-35, Jan. 1993.

J. S. Brown and P. Duguid, "Organizational Learning and Communities of Practice: Toward a Unified View of Working, Learning, and Innovation", Organization Science, Vol. 2, No. 1 (Feb. 1991), p. 40-57.

J. B. Cunningham, "Case Study Principles for Different Types of Cases", Quality & Quantity, Vol. 31 (1997), No. 4, p. 401-423.

U. Fayyad, G. Piatetsky-Shapiro, and P. Smyth, Chapter on "From Data Mining to Knowledge Discovery: An overview", In Advances in Knowledge Discovery and Data Mining, AAAI/MIT Press, 1996

M. Frigge, D. C. Hoaglin, and B. Iglewicz, "Some Implementations of the Boxplot", The American Statistician, 43(1), pp. 50-54, Feb. 1989.

O. Helmer, N. Rescher, "On the Epistemology of the Inexact Sciences", in Management Science 6(1):25-52, 1959; reprinted in Leonard Krimerman (ed.), "The Nature and Scope of Social Science", Appleton-Century-Crofts, New York NY, pp. 181-203, 1969.

E. M. Hufnagel and C. Conca, "User Response Data: The Potential for Errors and Biases", Information Systems Research, Vol. 5 (1994), No. 1, p. 48-73.

A. S. Lee, "A Scientific Methodology for MIS Case Studies", MIS Quarterly, Vol. 13 (1989), No. 1, p. 33-50.

A. K. Romney, S.C. Weller, W.H. Batchelder, "Culture as Consensus: A Theory of Culture and Informant Accuracy." in American Anthropologist 88(2), 313-338, 1986.

T. L. Saaty, "The Analytic Hierarchy Process", McGraw Hill, Inc., New York NY, 1980.

T. L. Saaty, L.G. Vargas, "Models, Methods, Concepts & Applications of the Analytic Hierarchy Process", Kluwer Academic Publishers, Dordrecht, the Netherlands, 2001.

E. H. Schein, "Three Cultures of Management: The Key to Organizational Learning", Sloan Management Review, Vol. 38 (1996), No. 1, Fall, p. 9-20.

G. Walsham, "Interpretive Case Studies in IS Research: Nature and Method", European Journal of Information Systems, Vol. 4 (1995), No. 4, p. 74-81.

Specific Empirical Studies

Software Development

B. W. Boehm, T. E. Gray, T. Seewaldt, "Prototyping versus Specifying - A Multiproject Experiment", IEEE Transactions on Software Engineering, Vol. 10 (1984), No. 3, p. 290-302.

B. W. Boehm, C. Abts, E. B. Clark, "Empirical Observations on COTS Software Integration Effort Based on the Initial COCOTS Calibration Database?", In J. C. Dean, P. Oberndorf, and M. Vigder (Eds.):"Proc. Second Workshop on Commercial Off-The-Shelf Software" (in conjunction with ICSE'2000, 5-6 June 2000, Limerick, Ireland), p. 99-104.

R. E. Brooks, "Studying Programmer Behaviour Experimentally: The Problems of Proper Methodology", Human Aspects of Computing, 23, 207-213. 1980.

K. El Emam, S. Quintin, N. H. Madhavji, "User Participation in the Requirements Engineering Process - An Empirical Study", Requirements Engineering, Vol. 1 (1996), No. 1, p. 4-26.

J. Kiper, B. Auernheimer C. K. Ames, "Visual Depiction of Decision Statements: What is Best for Programmers and Non-Programmers?", Journal of Empirical Software Engineering, 2(4), pp. 361-379, 1997.

V. S. Lai, B.K. Wong, W. Cheung, "Group Decision Making in a Multiple Criteria Environment: A Case Using the AHP in Software Selection", in European Journal of Operational Research, vol: 137 issue: 1, pp 134-144, 2002.

C. M. Lott, "A Controlled Experiment to Evaluate On-Line Process Guidance", Journal of Empirical Software Engineering, 2(3), pp. 269-289, 1997.

H. Mili, E. Ah-Ki, R. Godin, H. Mcheick, "An Experiment in Software Component Retrieval", in Information and Software Technology, vol: 45 issue: 10, pp 633-649, 2003.

K. Molokken, M. Jorgensen, "A Review of Surveys on Software Effort Estimation", in Proceedings of the 2003 International Symposium on Empirical Software Engineering, IEEE Computer Society Press, Los Alamitos CA, 2003.

M. Morisio, "Applying the PSP in Industry", in IEEE Software, volume: 17 issue: 6, 2000.

M. C. Ohlsson, C. Wohlin, "A Project Effort Estimation Study", Journal of Software and Information Technology, 40(14), pp. 831-839, 1998.

U. Passing, M. Shepperd, "An Experiment on Software Project Size and Effort Estimation", in Proceedings of the 2003 International Symposium on Empirical Software Engineering, IEEE Computer Society Press, Los Alamitos CA, 2003.

S. L. Pfleeger, N. Fenton, and S. Page, "Evaluating Software Engineering Standards", IEEE Computer, Sept. 1994, p. 71-79.

L. Prechelt, B. Unger, "An Experiment Measuring the Effects of Personal Software Process (PSP) Training", in IEEE Transactions on Software Engineering, vol: 27 issue: 5, pp 465-473, 2001.

R. W. Selby, V. R. Basili, F. T. Baker, "Cleanroom Software Development: An Empirical Evaluation", IEEE Transactions on Software Engineering, 13(9), pp. 1027-1037, 1987.

Defect Reduction

C. Andersson, T. Thelin, P. Runeson, N. Dzamashvili, "An Experimental Evaluation of Inspection and Testing for Detection of Design Faults", in Proceedings of the 2003 International Symposium on Empirical Software Engineering, IEEE Computer Society Press, Los Alamitos CA, 2003.

M. P. Ardissone, M. Spolverini, M. Valentini, "Statistical Decision Support Method for In-Process Inspections", in Proceedings of the Fourth International Conference on Achieving Quality in Software, 1998.

V. R. Basili and R. W. Selby, "Comparing the Effectiveness of Software Testing Strategies", IEEE Transactions on Software Engineering, 13 (12), pp. 1278-1298, 1987.

V. R. Basili, S. Green, O. Laitenberger, F. Lanubile, F. Shull, S. Sørumgård and M. V. Zelkowitz, "The Empirical Investigation of Perspective-Based Reading", Journal of Empirical Software Engineering, 1(2), pp. 133-164, 1996.

S. Biffl, "Evaluating Defect Estimation Models with Major Defects", in Journal of Systems and Software, vol: 65 issue: 1, pp 13-29, 2003.

S. Biffl, M. Halling, "Investigating the Defect Detection Effectiveness and Cost Benefit of Nominal Inspection Teams", in IEEE Transactions on Software Engineering, vol: 29 issue: 5, pp 385-397, 2003.

L. Briand, Y. Labiche, Y. Wang, "Using Simulation to Empirically Investigate Test Coverage Criteria Based on statecharts" Technical Report SCE-02-09, Carleton University, October 2002.

M. Ciolkowski, F. Shull, S. Biffl, "A Family of Experiments to Investigate the Influence of Context on the Effect of Inspection Techniques", in Proceedings of the 6th International Conference on Empirical Assessment in Software Engineering, Keele University UK, 2002.

S. Elbaum, A.G. Malishevsky, G. Rothermel, "Test Case Prioritization: A family of Empirical Studies", IEEE Transactions on Software Engineering, vol: 28 issue: 2, pp. 159-183, 2002.

P. G. Frankl, G. Rothermel, K. Sayre, F.I. Vokolos, "An Empirical Comparison of Two Safe Regression Test Selection Techniques", in Proceedings of the 2003 International Symposium on Empirical Software Engineering, IEEE Computer Society Press, Los Alamitos CA, 2003.

P. Fusaro, F. Lanubile and G. Visaggio, "A Replicated Experiment to Assess Requirements Inspection Techniques", Journal of Empirical Software Engineering, 2(1), pp. 39-57, 1997.

M. Halling, S. Biffl, P. Grunbacher, "An Experiment Family to Investigate the Defect Detection Effect of Tool-Support for Requirements Inspection", in Proceedings of the Ninth International Software Metrics Symposium, IEEE Computer Society Press, Los Alamitos CA, 2003.

P. M. Johnson and D. Tjahjono, "Does Every Inspection Really Need a Meeting?", Journal of Empirical Software Engineering, 3(1), pp. 9-35, 1998.

A. Kamel, P.G. Sorenson, "The Application of Capture-Recapture Log-Linear Models to Software Inspections Data", in Proceedings of the 2003 International Symposium on Empirical Software Engineering, IEEE Computer Society Press, Los Alamitos CA, 2003.

C. Lott and H. D. Rombach, "Repeatable Software Engineering Experiments for Comparing Defect-Detection Techniques", Journal of Empirical Software Engineering, Vol. 1 (1996), No. 3, p. 241-278.

P. McCarthy, R. Porter, J. Riedl, "An Experiment to Assess Cost-Benefit of Inspection Meetings and Their Alternatives", Technical Report, Computer Science Department, University of Maryland, 1995.

J. Miller, M. Wood and M. Roper, "Further Experiences with Scenarios and Checklists", Journal of Empirical Software Engineering, 3(1), pp. 37-64, 1998.

A. A. Porter and L. G. Votta, "An Experiment to Assess Different Defect Detection Methods for Software Requirements Inspections", Proceedings 16th International Conference on Software Engineering, Sorrento, Italy, pp. 103-112, 1994.

A. A. Porter, L. G. Votta and V. R. Basili, "Comparing Detection Methods for Software Requirements Inspection: A Replicated Experiment", IEEE Transactions on Software Engineering, 21(6), pp.563-575, 1995.

A. A. Porter, H.P. Siy, L.G. Votta, "A Survey of Software Inspections", in Advances in Computers, Vol 42, pp 40-77, 1996.

A. A. Porter and P. M. Johnson, "Assessing Software Review Meetings: Results of a Comparative Analysis of Two Experimental Studies", IEEE Transactions on Software Engineering, 23(3), pp. 129-145, 1997.

A. A. Porter, H. P. Siy, C. A. Toman and L. G. Votta, "An Experiment to Assess the Cost-Benefits of Code Inspections in Large Scale Software Development", IEEE Transactions on Software Engineering, 23(6), pp. 329-346, 1997.

A. A. Porter, H. P. Siy, A. Mockus and L. G. Votta, "Understanding the Sources of Variation in Software Inspections", ACM Transactions on Software Engineering and Methodology, 7(1), pp. 41-79, 1998.

A. A. Porter, and L. G. Votta, "Comparing Detection Methods for Software Requirements Inspection: A Replication Using Professional Subjects", Journal of Empirical Software Engineering, 3(4), pp. 355-380, 1998.

L. Prechelt and W. F. Tichy, "A Controlled Experiment to Assess the Benefits of Procedure Argument Type Checking", IEEE Transactions on Software Engineering, 24(4), pp. 302-312, 1998.

B. Regnell, P. Runeson and T. Thelin, "Are the Perspectives Really Different? - Further Experimentation on Scenario-Based Reading of Requirements", Journal of Empirical Software Engineering, volume 5 issue 4, pp 331-356, 2000.

P. Runeson, C. Wohlin, "An Experimental Evaluation of an Experience-Based Capture-Recapture Method in Software Code Inspections", in Empirical Software Engineering: An International Journal, Vol. 3 No.4, pp 381-406, 1998.

P. Runeson, T. Thelin, "A Case Study Using Sampling to Improve Software Inspection Effectiveness", in Proceedings of the 2003 International Symposium on Empirical Software Engineering, IEEE Computer Society Press, Los Alamitos CA, 2003.

K. Sandahl, O. Blomkvist, J. Karlsson, C. Krysander, M. Lindvall, N. Ohlsson, "An Extended Replication of an Experiment for Assessing Methods for Software Requirements", Journal of Empirical Software Engineering, 3(4), pp. 381-406, 1998.

C. B. Seaman and V. R. Basili, "Communication and Organization: An Empirical Study of Discussion in Inspection Meeting", IEEE Transactions on Software Engineering, 24(7), pp. 559-572, 1998.

C. Smidts, X. Huang, J.C. Widmaier, "Producing Reliable Software: An Experiment", in The Journal of Systems and Software, vol: 61 issue: 3, pp 213-224, 2002.

T. Thelin, P. Runeson, C. Wohlin, "An Expirimental Comparison of Usage-Based and Checklist-Based Reading", in IEEE Transactions on Software Engineering, vol: 29 issue: 8, pp 687-704, 2003.

L. G. Votta, "Does Every Inspection Need a Meeting?", Proceedings of the ACM SIGSOFT 1993 Symposium on Foundations of Software Engineering, ACM Software Engineering Notes, 18(5), pp. 107-114, 1993.

Process Improvement

E. Arisholm, B. Anda, M. Jørgensen, and D. Sjøberg,"Guidelines on Conducting Software Process Improvement Studies in Industry", Proc. 22nd IRIS Conference (Information Systems Research Seminar In Scandinavia), Keuruu, Finland, 7-10 August 1999, p. 87-102.

E. Arisholm, J. Skandsen, K. Sagli, and D. I.K. Sjøberg, "Improving an Evolutionary Development Process -- A Case Study", Proc. European Software Process Improvement Conference 1999 (EuroSPI'99), Pori, Finland, pp. 9.40-9.50.

T. Dybå, "Enabling Software Process Improvement: An Investigation of the Importance of Organizational Issues", in Empirical Software Engineering, vol: 7 issue: 4, pp 387-390, 2002.

J. P. Kuilboer, N. Ashrafi, "Software Process and Product Improvement: An Empirical Assessment", in Information and Software Technology, vol: 42 issue: 1, pp 27-34, 2000.

S. Lauesen, O. Vinter, "Preventing Requirement Defects: An Experiment in Process Improvement", in Requirements Engineering, vol: 6 issue: 1, pp 37-50, 2001.

H. J. Lied and T. Stålhane, "Experience from Process Improvement in an SME", Proc. European Software Process Improvement Conference (EuroSPI'99), Pori, Finland, October 26--27, 1999. SU-report 24/99.

A. Rainer, T. Hall, "A Quantitative and Qualitative Analysis of Factors Affecting Software Processes", in The Journal of Systems and Software, vol: 66 issue: 1, pp 7-21, 2003.

M. A. Rothenberger, K.J. Dooley, U.R. Kulkarni, N. Nada, "Strategies for Software Reuse: a Principal Component Analysis of Reuse Practices", in IEEE Transactions on Software Engineering, 29(9):825-837, 2003.

T. Stålhane, T. Dybå, and R. Palmstrøm, "Experience of Introducing Goal-Oriented Measurement", Proc. 8th International Workshop on Software Technology and Engineering Practice (STEP'97), London, 14-18 July, 1997.

T. Stålhane and K. Juul Wedde, "SPI -- Why Isn't It More Used", Proc. European Software Process Improvement Conference (EuroSPI99), Pori, Finland, October 26--27, 1999. SU-report 25/99.

Object-Orientation

B. Anda, D.I.K. Sjoberg, "Applying Use Cases to Design Versus Validate Class Diagrams - a Controlled Experiment Using a Professional Modelling Tool", in Proceedings of the 2003 International Symposium on Empirical Software Engineering, IEEE Computer Society Press, Los Alamitos CA, 2003.

L. C. Briand, C. Bunse, J. W. Daly and C. M. Differding, "An Experimental Comparison of the Maintainability of Object-Oriented and Structured Design Documents", Journal of Empirical Software Engineering, 2(3), pp. 291-312, 1997.

L. C. Briand, C. Bunse, and J. W. Daly, "A Controlled Experiment for Evaluating Quality Guidelines on the Maintainability of Object-Oriented Designs", To appear in IEEE Transactions on Software Engineering, 1999.

L. Briand, E. Arisholm, S. Counsell, F. Houdek, P. Thévenod-Fosse, "Empirical Studies of Object-Oriented Artifacts, Methods, and Processes: State of The Art and Future Directions", in Empirical Software Engineering: An International Journal, 2000.

L. Briand, J. Wuest, H. Lounis, "Replicated Case Studies for Investigating Quality Factors in Object-Oriented Designs", in Empirical Software Engineering: An International Journal, 2001.

J. Daly, A. Brooks, J. Miller, M. Roper, and M. Wood, "Evaluating Inheritance Depth on the Maintainability of Object-Oriented Software", Empirical Software Engineering, Vol. 1 (1996), No. 2, p. 109-132.

A. Dunsmore, M. Roper, M. Wood, "Further Investigations Into the Development and Evaluation of Reading Techniques for Object-Oriented Code Inspection", in Proceedings of the 24th International Conference on Software Engineering, IEEE Computer Society Press, Los Alamitos CA, 2002.

R. Harrison, L. G. Samaraweera, M. R. Dobie and P. H. Lewis, "Comparing Programming Paradigms: An Evaluation of Functional and Object-Oriented Programs", Software Engineering Journal, pp. 247-254, July 1996.

R. Harrison, S. Counsell, and R. Nithi, "Experimental Assessment of the Effect of Inheritance on the Maintainability of Object-Oriented Systems", Proc. Empirical Assessment in Software Engineering (EASE'99), Keele, Staffordshire, 12-14 April 1999.

Software Maintenance

PO. Bengtsson, J. Bosch, "An Experiment on Creating Scenario Profiles for Software Change", in Annals of Software Engineering, vol: 9 issue: 1-4, pp 59-78, 2000.

K. Bennett, E. Burd, C. Kemerer, M. M. Lehman, M. Lee, R. Madachy, C. Mair, Dag Sjøberg, and S. Slaughter, "Empirical Studies of Evolving Systems", Empirical Software Engineering, 4(4):370-380, December 1999.

S. G. Eick, T. L. Graves, A. F. Karr, J. S. Marron, and A. Mockus, "Does Code Decay? Assessing the Evidence from Change Management Data", IEEE Transactions on Software Engineering, Vol. 27, No. 1 (Jan. 2001), pp. 1-12.

M. Jørgensen, "An Empirical Study of Software Maintenance Tasks", Software Maintenance: Research and Practice, 7:27-48, 1995.

C. F. Kemerer and S. Slaughter, "An Empirical Approach to Studying Software Evolution", IEEE Transactions on Software Engineering, Vol. 25 (1999), No. 4, pp. 493-509.

B. P. Lientz, E.B. Swanson, G.E. and Tompkins, "Characteristics of Application Software Maintenance", Communications of the ACM, Vol. 21, No. 6, pp. 466-471, 1978.

L. Prechelt, B. Unger, W.F. Tichy, "A Controlled Experiment in Maintenance Comparing Design Patterns to Simpler Solutions.", in IEEE Transactions on Software Engineering, vol: 27 issue: 12, pp 1134-1145, 2001.

D. J. Reifer, V.R. Basili, B.W. Boehm, B. Clark, "Eight Lessons Learned During COTS-Based Systems Maintenance", in IEEE Software, vol: 20 issue: 5, pp 94-96, 2003.

H. D. Rombach, "A Controlled Experiment on the Impact of Software Structure on Maintainability", IEEE Transactions on Software Engineering, 13(3), pp. 344-354, 1987.

S. R. Schach, B. Jin, L. Yu, G.Z. Heller, "Determining the Distribution of Maintenance Categories: Survey versus Measurement", in Empirical Software Engineering, vol: 8 issue: 4, pp 351-365, 2003.

Agile Software Development

R-J. Back, L. Milovanov, I. Porres, V. Preoteasa, "XP as a Framework for Practical Software Engineering Experiments", In Proceedings of the Third International Conference on eXtreme Programming and Agile Processes in Software Engineering - XP2002, 2002.

P. Baheti, L. Williams, E. Gehringer, D. Stotts, J. Smith, "Distributed Pair Programming: Empirical Studies and Supporting Environments", UNC-CH Technical Report TR02-010, University of North Carolina at Chapel Hill, 2002.

B. Hanks, "Empirical Studies of Pair Programming", in Proceedings of 2nd International Workshop on Empirical Evaluation of Agile Processes (EEAP 2003), 2003.

A. Janes, B. Russo, P. Zuliani, G. Succi, "An Empirical Analysis on the Discontinuous Use of Pair Programming", in Proceedings of XP2003 (Springer LNCS 2675), pages 205 - 214, 2003.

M. Lindvall, V. Basili, B. Boehm, P. Costa, K. Dangle, F. Shull, R. Tesoriero, L. Williams, M. Zelkowtiz, "Empirical Findings in Agile Methods", Proceedings of Extreme Programming and Agile Methods - XP/Agile Universe, 2002.

F. Macias, M. Holcombe, M. Gheorghe, "A Formal Experiment Comparing Extreme Programming with Traditional Software Construction", in Proceedings of the Fourth Mexican International Conference on Computer Science, IEEE Computer Society Press, Los Alamitos CA, 2003.

C. McDowell, B. Hanks, L. Werner, "Experimenting with Pair Programming in the Classroom", in Proceedings of the 8th Annual Conference on Innovation and Technology in Computer Science Education (ITiCSE 2003), 2003.

M.M. Muller, O. Hagner, "Experiment about Test-First Programming", in IEE Proceedings-Software, vol: 149 issue: 5, pp 131-136, 2002.

J. Nawrocki, A. Wojciechowski, "Experimental Evaluation of Pair Programming", in Proceedings of European Software Control and Metrics (ESCOM 2001), 2001.

J.T. Nosek, "The Cast for Collaborative Programming", in Communications of the ACM, vol. 41 No. 3, pp 105-108, 1998.

D. Stotts, L. Williams, N. Nagappan, P. Baheti, D. Jen, A. Jackson, "Virtual Teaming: Experiments and Experiences with Distributed Pair Programming", Extreme Programming/Agile Universe, 2003.

G. Succi, M. Stefanovic, M. Smith, R. Huntrods, "Design of an Experiment for Quantitative Assessment of Pair Programming Practices", in Proceedings of XP2001, 2001.

L. Williams, R. Kessler, W. Cunningham, R. Jeffries, "Strengthening the case for pair-programming", in IEEE Software, Vol. 17 No. 4, pp 19--25, July/August 2000.

L. Williams, R.R. Kessler, "Experimenting with Industry's 'Pair-Programming' Model in the Computer Science Classroom", in Computer Science Education, March 2001.

L. Williams, C. McDowell, N. Nagappan, J. Fernald, L. Werner, "Building Pair Programming Knowledge Through a Family of Experiments", in Proceedings of the 2003 International Symposium on Empirical Software Engineering, IEEE Computer Society Press, Los Alamitos CA, 2003.

Please send your comments to Claes Wohlin. This document was updated 25/11/2003.

This is a simplified .html file taken from the ESERNET site by Reidar Conradi on 18 Jan. 2004.