This file: www.idi.ntnu.no/grupper/su/publ/ebse/ebse-papers-13aug09.html

Reidar Conradi: EBSE paper overview per 13aug09

S-studies are one of the 20 [Kitchenham09] studies, named S1-S20.

1. Twelve R01-R12 papers from Simula/SINTEF/NTNU, w/ 4 S-studies:

  1. [R01=S18: Sjøberg05] D. I. K. Sjøberg, J. E. Hannay, O. Hansen, V. B. Kampenes, A. Karahasanovic, Nils-K. Liborg, A. C. Rekdal, A Survey of Controlled Experiments in Software Engineering, IEEE Transactions on Software Engineering 31(9):733-753, 2005 (R01 as .pdf -- 3117 KB).
  2. [R02: Jørgensen05] M. Jørgensen, Evidence-Based Guidelines for Assessment of Software Costs Uncertainty, IEEE Transactions on Software Engineering, 31(11):942--954 (2005), University of Oslo. (R02 as .pdf -- 762 KB).
    [R02xxx: Holt06] Nina E. Holt, A Systematic Review of Case Studies in Software Engineering, Masters thesis, University of Oslo, 2006. old R2 - omit.
  3. [R03=S2: Dybå06] Tore Dybå, V. B. Kampenes, D. I. K. Sjøberg, A systematic review of statistical power in software engineering experiments, Information and Software Technology 48(8):745-755, 2006. (R03 as .pdf -- 313 KB).
  4. [R04: Kampenes07] V. B. Kampenes, T. Dybå, J. E. Hannay, D. I. K. Sjøberg, A Systematic Review of Effect Size in Software Engineering Experiments, Information and Software Technology 49(11-12):1073-1086, 2007. (R04 as .pdf -- 281 KB).
  5. [R05=S9: Jørgensen07] M. Jørgensen, M. Shepperd, A Systematic Review of Software Development Cost Estimation Studies, IEEE Transactions on Software Engineering 33(1):33-53, 2007. (R05 as .pdf -- 857 KB).
  6. [R06=S6: Hannay07] J. E. Hannay, D. I. K. Sjøberg, T. Dybå, A Systematic Review of Theory Use in Software Engineering Experiments, IEEE Transactions on Software Engineering 33(2):87--107, 2007. (R06 as .pdf -- 4523 KB).
  7. [R07: Benestad08] H. C. Benestad, B. C. D. Anda, E. Arisholm, Understanding software maintenance and evolution by analyzing individual changes: A literature review, Accepted for Journal of Software Maintenance and Evolution: Research and Practice, 2008. (R07 as .pdf -- 122 KB).
  8. [R08: Dyba08] Tore Dybå, T. Dingsøyr, Empirical Studies of Agile Software Development: A Systematic Review. Information and Software Technology, 50 (9-10):833-859 (2008). (R08 as .pdf -- 392 KB).
  9. [R09: Kampenes09] V. B. Kampenes, T. Dybå, J. E. Hannay, D. I. K. Sjøberg, A systematic review of quasi-experiments in software engineering, Information and Software Technology 51(1):71-82, 2009. (R09 as .pdf -- 250 KB)
  10. [R10: Bjørnson08] Finn Olav Bjørnson, Torgeir Dingsøyr, "Knowledge Management in Software Engineering: A Systematic Review of Studied Concepts and Research Methods Used", Information and Software Technology, 50(11):1055-1068 (October 2008), doi:10.1016/j.infsof.2008.03.006. (R10 as .pdf -- 145 KB)
  11. [R11: Hannay09] J. E. Hannay, T. Dybå, E. Arisholm, D. Sjøberg, The Effectiveness of Pair-Programming: A Meta-Analysis, Information and Software Technology 55(7):1110-1122, 2009. (R11 as .pdf -- 558 KB).
  12. [R12: Hauge09] Øyvind Hauge, Claudia Ayala, Reidar Conradi, A systematic review of Industrial Adoption of Open Source Software, draft paper, NTNU, Aug. 2009, ca. 25 p. (R12 as .pdf -- xxx KB).

New X01-X08 systematic review papers from EBSE-site, w/ 3 S-studies:

  1. [X01: Bailey07] John Bailey, David Budgen, Mark Turner, Barbara Kitchenham, Pearl Brereton, Stephen Linkman, Evidence relating to Object-Oriented software design: A survey (2007), Proc. first International Symposium on Empirical Software Engineering and Measurement (ESEM 2007}, IEEE Computer Society Press, 3 p. (short paper). (X01 as .pdf -- 76 KB).
  2. [X02=S10b: Juristo04, 18] N. Juristo, A.M. Moreno, S. Vegas, Reviewing 25 years of testing technique experiments, Empirical Software Engineering Journal (1-2) (2004) 7-44. (X02 as .pdf -- 187 KB).
  3. [X03=S10: Juristo06: 19] N. Juristo, A.M. Moreno, S. Vegas, M. Solari, In search of what we experimentally know about unit testing, IEEE Software 23 (6) (2006) 72-80. (X03 as .pdf -- 137 KB).
  4. [X04=S11: Kitchenham07: 21] B. Kitchenham, E. Mendes, G.H. Travassos, A systematic review of cross- vs. within-company cost estimation studies, IEEE Transactions on SE 33 (5) (2007) 316-329. (X04 as .pdf -- 6863 KB).
    [X04b=S11b: Kitchenham06: 20] B. Kitchenham, E. Mendes, G.H. Travassos, A systematic review of cross-company vs. within-company cost estimation studies, Proc. EASE06, BSC (2006) 89-98 -- earlier version
  5. [X05=S17: Runeson06] Per Runeson, C. Andersson, T. Thelin, A. Andrews, T. Berling, What do we know about Defect Detection Methods?, IEEE Software, 23(3):82--86 (2006). (X05 as.pdf -- 163 KB).
  6. [X06: Whitley97] K. N. Whitley, Visual Programming Languages and the Empirical Evidence For and Against, Journal of Visual Languages and Computing. 1997(8):109--142. (X06 as .pdf -- 264 KB).
  7. [X07: Engstrom08] E. Engström, M. Skoglund, P. Runeson, Empirical Evaluations of Regression Test Selection Techniques: A Systematic Review, Proc. second ACM-IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM 2008), ACM Press, p. 22--31. (X07 as .pdf -- 257 KB).
  8. [X08: MacDonnel07] Stephen G. MacDonell, Martin J. Shepperd, Comparing Local and Global Software Effort Estimation Models - Reflections on a Systematic Review, Proc. first International Symposium on Empirical Software Engineering and Measurement (ESEM 2007}, IEEE Computer Society Press, p. 401--409. (X08 as .pdf -- 161 KB).

X09-X13: Other papers from OSS review paper work at IDI summer 2009:

  1. [X09: Beecham08] Sarah Beecham, Nathan Baddoo, Tracy Hall, Hugh Robinson, Helen Sharp, Motivation in Software Engineering: A systematic literature review (2008), Information and Software Technology, 50:9-10(860-878). (X09 as .pdf -- 289 KB).
  2. [X10: Staples08] Mark Staples, Mahmood Niazi, Systematic review of organizational motivations for adopting CMM-based SPI (2008), Information and Software Technology, 50:7-8(605-620). (X10 as .pdf -- 266 KB).
  3. [X11: Feller06] Joseph Feller, Patrick Finnegan, David Kelly, Maurice MacNamara, Developing Open Source Software: A Community-Based Analysis of Research, Proc. IFIP TC8 WG8.2 International Working Conference - Societal and Organizational Implications for Information Systems, Limerick, Ireland, July 12--15, pages 261--278, Springer LNCS ????, 2006. (X11 as .pdf -- 1031 KB).
  4. [X12: Afzal09] Wasif Afzal, Richard Torkar, Robert Feldt, A systematic review of search-based testing for non-functional system properties (2009), Information and Software Technology, 51:6(957-976). (X12 as .pdf -- 679 KB).
  5. [X13: Stol09] Klaas-Jan Stol, Muhammed Ali Babar, Reporting Empirical Research in Open Source Software: The State of Practice, Proc. 5th IFIP WG2.13 International Conference on Open Source Systems (OSS 2009) - Open Source Ecosystems: Diverse Communities, June 3-6, Skövde, Sweden, pages 156-169, Springer Verlag LNCS ????, 2009. (X13 as .pdf -- 133 KB).

Possible more Ynn syst. review. papers for our meta-studies:

  1. [YY01: Ali09] Ali, Briand et al.. On Evolutionary testing, Simula TR, 2009. (YY01 as .pdf -- xxx KB).
  2. Y01=S13 Web Engineering Research
  3. Y02 Requirements elicitation
  4. Y03=S12 Cost estimation analog/formal
  5. Y04 Component evaluation
  6. Y05 Francisco J. Pino, Félix García, Mario Piattini, Software process improvement in small and medium software enterprises: a systematic review, Software Quality Control, 16(2): 237-261 (June 2008), ISSN:0963-9314.
  7. Y06 Parastoo Mohagheghi and Reidar Conradi: "Quality, Productivity and Economic Benefits of Software Reuse: A Review of Industrial Studies", Journal of Empirical Software Engineering, 12(5):471-516, 2007, ISSN 1382-3256. downlaod
  8. Y07 P Tonella et al., Empirical studies in reverse engineering: state of the art and ...", Proc. The International Symposium on Empirical Software Engineering (ISESE 2007) ..... www.hitech-projects.com/euprojects/.../EmpiricalStudies.pdf - 2007.
  9. Y08 Muhammad Ali Babar, Barbara Kitchenham, Liming Zhu, Ian Gorton, Ross Jeffery, An empirical study of groupware support for distributed software architecture evaluation process, Proc. Int'l Conf. on Empirical Software Engineering (ESEM 2007), doi:10.1016/j.jss.2005.06.043, 2005 Elsevier Inc.,
    Y08.
  10. Y09 A. Powell, A. Vickers, Wing Lam, E. Williams, B. Cooke, Ëvaluating Tools to Support Component Based Software Engineering Assessment of Software Tools, Proc. 5th International Symposium on Assessment of Software Tools (SAST '97), 1997, pp. 0080. doi = http://doi.ieeecomputersociety.org/10.1109ÅST.1997.599914, ISBN: 0-8186-7940-9.
  11. Y10 Norman E. Fenton, Martin Neil. Software metrics: successes, failures and new directions. Journal of Systems and Software 47():149-157 (1999), www.dcs.qmul.ac.uk/ñorman/papers/jss_1999.pdf
  12. Y11 H. M. Kienle, H. A. Muller, Requirements of Software Visualization Tools: A Literature,...??, Proc. 4th IEEE International Workshop on Visualizing Software for Understanding and Analysis (VISSOFT 2007), 24-25 June 2007, pp. 2-9, IEEE Computer Society Press, ISBN: 1-4244-0600-5, INSPEC Accession Number: 9690880, DOI: 10.1109/VISSOF.2007.4290693.
  13. Y12 formal methods
  14. Y13 programming languages
  15. Y14 progam comprehension by dynamic analysis.
  16. Y15 risk in SE. Mark Keil, , a, Linda Wallace1, , b, Dan Turkc, Gayle Dixon-Randalld and Urban Nuldene, An investigation of risk perception and risk propensity on the decision to continue a software development project Journal of Systems and Software Volume 53, Issue 2, 31 August 2000, Pages 145-157.
  17. Y16 N-1 version testing
  18. Y17 Software process modelling for simulation.
  19. Y18 UML
  20. Y19 Aspect-oriented programming
  21. Y20 HCI (book)
  22. Y21...
Appendix A: Some files on ~epos/SU-grp/publ/ebse:

A.1 Some .pdf files from [Kitchenham09]:

  1. RK01...
  2. RK02...
  3. RK04-statpow-dyba-ist06.pdf -> R03-statpow-dyba-ist06.pdf/
  4. RK11-estimbabel-grimstad-jun06ist.pdf*
  5. RK14-expertformal-jorgensen-forecast07.pdf*
  6. RK15-reviewexpertestim-jorgensen-jss04.pdf*
  7. RK16-costestim-jorgensen-tse-jan07.pdf -> R05-costestim-jorgensen-tse-jan07.pdf/
  8. RK18-formalexper-sjoberg-sep05-pdf -> R01-formalexper-sjoberg-sep05-pdf/
  9. RK20-crossestim-kitchenham-ease06.pdf*
  10. RK27-regressionanalogycostestim-mair-isese05.pdf*
  11. RK31-surveycostestim-molokken-metrics04.pdf*

A.2 Systematic Reviews on files R0-R12, i.e. done locally at Simula, NTNU and SINTEF:

  1. R01-formalexper-sjoberg-sep05.pdf
  2. R02-evbasedguidelines-swdevcostuncert-jorgensen-nov05tse.pdf
  3. R03-statpow-dyba-jss06.pdf
  4. R04-effectsize-kampenes-ist-nov07.pdf
  5. R05-costestim-jorgensen-tse-jan07.pdf
  6. R06-theoryuse-hannay-tse-feb07.pdf
  7. R07-indivchanges-benestad-tr297-08.pdf
  8. R08-agile-dyba-dingsoyr-ist-aug08.pdf
  9. R09-quasiexper-kampenes-ist09.pdf
  10. R10-knowlmgmt-bjornson-istfeb08.pdf
  11. R11-pairprog-hannay-ist09.pdf
  12. R12??
  13. review-casestud-holt-masteruio-may06.pdf (R02xxx replaced by R02)

A.3 Similar paper files for systematic reviews X01-X13 done elsewhere:

  1. X01-evid-ooswdesign-bailey-esem07.pdf
  2. X02-25yeartesting-juristo-jemse04.pdf
  3. X03-search-unittest-juristo-ieeesw06.pdf
  4. X04-cross-within-costestim-kitchenham-tsemay07.pdf
  5. X05-know-defectdetectmeth-runeson-ieeeswmay06.pdf
  6. X06-visualproglang-whitley-jvislang97.pdf
  7. X07-regtestsel-engstrom-esem08.pdf
  8. X08-comp-locglobestimeff-macdonell-esem07.pdf
  9. X09-motivat-se-beecham-ist08.pdf
  10. X10-org-motiv-staples-ist08.pdf
  11. X11-oss-community-feller-oss06.pdf
  12. X12-searchbased-testing-afzal-ist09.pdf
  13. X13-oss-sotp-stol-oss09.pdf

A.4 From searching the net: more .pdf files:

  1. Y01-webwenginresearch-mendes-ieee07.pdf*
  2. Y02-rqmtselicit-davis-reconf06.pdf*
  3. Y03-costestim-regress-analog-mair-icse05.pdf*
  4. Y04-compeval-goulao-apsi04.pdf*
  5. Y05-spisme-pino--sqj08.pdf*
  6. Y06-swreuseadvantages-mohagheghi-jemse07.pdf*
  7. Y07-reveng-sota-tonella-ese07.pdf*
  8. Y08-groupware-distrswarcheval-babar-jss06.pdf*
  9. Y10-rqmtsguitools-kienle-ieeegui07.pdf*
  10. Y14-progcomprehension-cornelissen-tse-jan00.pdf*
  11. Y17-simmodelling-zhang-lncs08.pdf* Y01...
  12. Y02...
  13. Y03...
  14. Y04...
  15. Y05 Francisco J. Pino, Félix García, Mario Piattini, Software process improvement in small and medium software enterprises: a systematic review, Software Quality Control, 16(2): 237-261 (June 2008), ISSN:0963-9314.
  16. Y06 Parastoo Mohagheghi and Reidar Conradi: "Quality, Productivity and Economic Benefits of Software Reuse: A Review of Industrial Studies", Journal of Empirical Software Engineering, 12(5):471-516, 2007, ISSN 1382-3256.
  17. Y07 P Tonella et al., Empirical studies in reverse engineering: state of the art and ...", Proc. The International Symposium on Empirical Software Engineering (ISESE 2007) ..... www.hitech-projects.com/euprojects/.../EmpiricalStudies.pdf - 2007
  18. Y08 doi:10.1016/j.jss.2005.06.043 Copyright © 2005 Elsevier Inc. All rights reserved. An empirical study of groupware support for distributed software architecture evaluation process Muhammad Ali Babar, , Barbara Kitchenham, Liming Zhu, Ian Gorton and Ross Jeffery Empirical Software Engineering, National ICT Australia, University of New South Wales, Sydney, Australia Y08.
  19. Y09 A. Powell, A. Vickers, Wing Lam, E. Williams, B. Cooke, Ëvaluating Tools to Support Component Based Software Engineering,"Assessment of Software Tools, International Symposium on, pp. 0080, 5th International Symposium on Assessment of Software Tools (SAST '97), 1997. doi = http:°doi.ieeecomputersociety.org/10.1109ÅST.1997.599914 - 0-8186-7940-9
  20. Y10 Requirements of Software Visualization Tools: A Literature Survey Kienle, H.M. Muller, H.A. Victoria Univ., Victoria; This paper appears in: Visualizing Software for Understanding and Analysis, 2007. VISSOFT 2007. 4th IEEE International Workshop on Publication Date: 24-25 June 2007 On page(s): 2-9 ISBN: 1-4244-0600-5 INSPEC Accession Number: 9690880 Digital Object Identifier: 10.1109/VISSOF.2007.4290693 Current Version Published: 2007-08-20 ...
  21. Y11...
  22. Y12...
  23. Y13...
  24. Y14...
  25. Y15...
  26. Y16...
  27. Y17...
  28. Y18...
  29. Y19...
  30. Y20...
  31. Y21...

A.5 More general method papers with .pdf files:

  1. [Brereton07] O.P. Brereton, B.A. Kitchenham, D. Budgen, M. Turner, M.A. Khalil, Lessons from applying the Systematic Literature Review process within the Software Engineering domain, Journal of Systems and Software, 80(4):571--583, 2007. (paper as .pdf -- 208 KB).
  2. [Dyba08b] T. Dybå, T. Dingsøyr. Strength of Evidence in Systematic Reviews in Software Engineering, Proc. second ACM-IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM 2008), 2008, pp. 178--187, ACM Press. (paper as .pdf -- 293 KB)
  3. [Kitchenham07] B.A. Kitchenham, Stuart Charters, Guidelines for performing Systematic Literature Reviews in Software Engineering, v2.3, Keele University and Durham University Joint Report, EBSE Technical Report, EBSE-2007-001, 2007. (paper as .pdf -- 687 KB).
  4. [Kitchenham08] B. Kitchenham, D. Budgen, P. Brereton, M. Turner, S. Charters, S. Linkman. Large-scale software engineering questions - expert opinion or empirical evidence? Proc. EASE'08, British Computer Society, 2008, pp??. (paper as .pdf -- 127 KB).
  5. [Kitchenham08] Barbara Kitchenham, O. Pearl Brereton, David Budgen, Mark Turner, John Bailey, Stephen Linkman. Systematic literature reviews in software engineering - A systematic literature review, Information and Software Technology, 51(1):7-15 (January 2009) -- a meta-paper. (paper as .pdf -- 204 KB).
  6. See also in [Kitchenham09]'s references in A.1 above. +++++++++????

A.6 Literature references (42 such) from [Kitchenham90]'s "meta paper", IST, Jan., 2009 -- no DOIs:

This file: www.idi.ntnu.no/grupper/su/publ/ebse/ebse-papers-13aug09.html.
Reidar.Conradi@idi.ntnu.no
Last modified: Fri Aug 14 18:59:44 MEST 2009