Stay Ahead With Linux Security News

spam filteringspam protectionbayesian filtering

Improving Spam Filtering Accuracy Using Bayesian Methods

Spam filtering is a subset of text classification, which is a well established field, but the first papers about Bayesian spam filtering per se seem to have been two given at the same conference in 1998, one by Pantel and Lin, and another by a group from Microsoft Research. . .. Spam filtering is a subset of text classification, which is a well established field, but the first papers about Bayesian spam filtering per se seem to have been two given at the same conference in 1998, one by Pantel and Lin, and another by a group from Microsoft Research . When I heard about this work I was a bit surprised. If people had been onto Bayesian filtering four years ago, why wasn't everyone using it? When I read the papers I found out why. Pantel and Lin's filter was the more effective of the two, but it only caught 92% of spam, with 1.16% false positives. When I tried writing a Bayesian spam filter, it caught 99.5% of spam with less than .03% false positives [4]. It's always alarming when two people trying the same experiment get widely divergent results. It's especially alarming here because those two sets of numbers might yield opposite conclusions. Different users have different requirements, but I think for many people a filtering rate of 92% with 1.16% false positives means that filtering is not an acceptable solution, whereas 99.5% with less than .03% false positives means that it is. . The evolution of spam filtering techniques highlights email security's importance amid rising unsolicited messages, evolving from basic keyword detection to Bayesian methods.. Bayesian Filtering, Email Classification, Text Filtering, Spam Protection. . LinuxSecurity.com Team

Jan 21, 2003 • LinuxSecurity.com Team Privacy