Spam over IP telephony (SPIT) is considered to be a great threat in future communications, and has the potential to become even more annoying than E-Mail Spam. At NEC Laboratories Europe, the VoIP SEAL project investigates how to prevent this future nuisance. Evaluating SPIT counter mechanisms is not straightforward. Various approaches are possible, and they depend on the kind of mechanism that is used. For example, evaluating a mechanism that imposes costs on a caller will focus on possible methods for an attacker to call without bearing these costs. We therefore focus on one class of SPIT prevention mechanisms, namely those that involve filtering of incoming calls.
The aim of this work is to define a framework for the evaluation of SPIT filtering mechanisms. This includes the investigation and selection of suitable metrics for the quality of a filtering mechanism. Such a metric could include false positives and false negatives - however, these are not applicable if calls are assigned to more than two classes (e.g. calls to be ignored, to be forwarded to a voice mailbox, or to be forwarded to the human user). The framework should focus on the use of distributed mechanisms for SPIT prevention. For example, a SIP service provider could tag outgoing calls, using locally available information. Therefore, it is insufficient to consider one instance of the filtering mechanism only. Instead, the behaviour of a large number of SIP service providers and an even larger number of their customers have to be modelled. An implementation of the framework and its application for the evaluation of a reputation-based SPIT prevention mechanism are expected as part of the thesis.