Safety Score as an Evaluation Metric for Machine Learning Models of Security Applications

Tara Salman, Ali Ghubaish, Devrim Unal, Raj Jain, "Safety Score as an Evaluation Metric for Machine Learning Models of Security Applications," IEEE Networking Letters, Vol. 2, Issue 4, December 2020, pp. 207-211, ISSN:2576-3156 DOI: 10.1109/LNET.2020.3016583

ABSTRACT:

Machine learning studies have traditionally used accuracy, F1 score, etc. to measure the goodness of models. We show that these conventional metrics do not necessarily represent risks in security applications and may result in models that are not optimal. This letter proposes 'Safety score' as an evaluation metric that incorporates the cost associated with model predictions. The proposed metric is easy to explain to system administrators. We evaluate the new metric for two security applications: general intrusion detection and injection attack detection. Compared to other metrics, Safety score proves its efficiency in indicating the risk in using the model.

This is an open access paper and can be downloaded freely from IEEE Explore

Back to the List of Papers
Back to Raj Jain's home page