Modeling Test Case Fault-detecting Capacity Based on Fault Detection Difficulty Level

doi:10.13190/j.jbupt.2016-284

JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM ›› 2017, Vol. 40 ›› Issue (5): 67-74.doi: 10.13190/j.jbupt.2016-284

• Papers • Previous Articles Next Articles

Modeling Test Case Fault-detecting Capacity Based on Fault Detection Difficulty Level

TAN Li-li¹, WANG Ya-wen¹, XING Ying², WAN Qian¹

1. State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China;
2. Automation School, Beijing University of Posts and Telecommunications, Beijing 100876, China

Received:2016-12-02 Online:2017-10-28 Published:2017-11-21

Abstract

Abstract: The credibility of mutation score is questioned because it is computed without regard to fault detection difficulty level. A fault-detecting capacity evaluation model about test suit considering fault detection difficulty level is suggested. Based on logistic regression, the quantitative relation between the fault recognition probability and fault detection difficulty level is expressed. With the area under this relation curve, the mutation score is redefined. The new defined mutation score is not affected by fault detection difficulty level and avoid inaccuracy of classical mutation score because of the appearance of equivalent mutants.

Key words: software testability, mutation testing, random test case generation, logistic model

CLC Number:

TP311.52

TAN Li-li, WANG Ya-wen, XING Ying, WAN Qian. Modeling Test Case Fault-detecting Capacity Based on Fault Detection Difficulty Level[J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM, 2017, 40(5): 67-74.

References

[1] Zhu Hong, Hall P A V, May J H R. Software unit test coverage and adequacy[J]. ACM Computing Survey, 1997, 29(4):366-427.
[2] Hamlet R G. Testing programs with the aid of a compiler[J]. IEEE Transactions on Software Engineering, 1977, 3(4):279-290.
[3] DeMillo R A, Lipton R J, Sayward F G. Hints on test data selection:help for the practicing programmer[J]. Computer, 1978, 11(4):34-41.
[4] Andrews J H, Briand L C, Labiche Y. Is mutation an appropriate tool for testing experiments?[C]//Proceedings of the 27^th International Conference on Software Engineering (ICSE'05). New York:ACM, 2005:402-411.
[5] Do H, Rothermel G. On the use of mutation faults in empirical assessments of test case prioritization techniques[J]. IEEE Transactions on Software Engineering, 2006, 32(9):733-752.
[6] Offutt A J, Lee A, Rothermel G, et al. An experimental determination of sufficient mutant operators[J]. ACM Transactions on Software Engineering and Methodology, 1996, 5(2):99-118.
[7] Jia Yue, Harman M. Constructing subtle faults using higher order mutation testing[C]//Proceedings of the 20088th IEEE International Working Conference on Source Code Analysis and Manipulation. Beijing:IEEE, 2008:249-258.
[8] DeMillo R A, Guindi D S, McCracken W M, et al. An extended overview of the Mothra software testing environment[C]//Proceedings of the second Workshop on Software Testing, Verification, and Analysis. Banff:IEEE, 1988.
[9] Delamaro M, Maldonad J C. Proteum-A tool for the assessment of test adequacy for C programs[C]//Proceedings of the Conference on Perform Ability in Computing Systems. Brunswick, NJ:[s.n.], 1996:79-95.
[10] Offutt A J, Hayes J H. A semantic model of program faults[C]//Proceedings of the 1996 ACM SIGSOFT International Symposium on Software Testing and Analysis. San Diego:ACM, 1996:195-200.
[11] DeMillo R A, Offutt A J. Constraint-based automatic test data generation[J]. IEEE Transactions on Software Engineering, 1991, 17(9):900-910.
[12] Voas J M. PIE:a dynamic failure-based technique[J]. IEEE Transactions on Software Engineering, 1992, 18(8):717-727.
[13] Chen W, Untch R H, Rothermel G, et al. Can fault-exposure-potential estimates improve the fault detection abilities of test suites?[J]. Software Testing, Verification and Reliability, 2002, 12(4):197-218.
[14] Tervonen T, van Valkenhoef G, Bastürk N, et al. Hit-and-run enables efficient weight generation for simulation-based multiple criteria decision analysis[J]. European Journal of Operational Research, 2013, 224(3):552-559.
[15] Valkenhoef G V, Tervonen T, Postmus D. Notes on Hit-and-run enables efficient weight generation for simulation-based multiple criteria decision analysis[J]. European Journal of Operational Research, 2014, 239(3):865-867.
[16] Harrell F E. Regression modeling strategies with applications to linear models, logistic regression and survival analysis[M]. New York:Springer, 2001.
[17] Cessie S L, Houwelingen J C V. Ridge estimators in logistic regression[J]. Applied Statistics, 1992, 41(1):191-201.
[18] Gelman A, Hill J L. Data analysis using regression and multilevel/hierarchical models[M]. Cambridge:Cambridge University Press, 2006.
[19] 金大海, 宫云战, 王雅文, 等. 软件代码测试技术[J]. 信息通信技术, 2015(3):34-39. Jin Dahai, Gong Yunzhan, Wang Yawen, et al. Software code test technology[J]. Information and Communications Technologies, 2015(3):34-39.
[20] 钱茛南, 宫云站, 王雅文, 等. 面向C语言的故障注入平台[J], 北京邮电大学学报, 2016, 39(3):95-99. Qian Gennan, Gong Yunzhan, Wang Yawen, et al. A C-language oriented fault injection platform[J]. Journal of Beijing University of Posts and Telecommunications, 2016, 39(3):95-99.
[21] Mirman D. Growth curve analysis and visualization using R[M]. Boca Raton, FL:Chapman & Hall/CRC Press, 2014.
[22] Friedman J, Hastie T, Tibshirani R. Regularization paths for generalized linear modelsvia coordinate descent[J]. J Stat Softw, 2010(33):1-22.

Modeling Test Case Fault-detecting Capacity Based on Fault Detection Difficulty Level

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 1

Recommended Articles

Metrics

Comments