Introducing Crowdsec: A Modernized, Collaborative Massively Multiplayer Firewall for Linux

CrowdSec is a massively multiplayer firewall designed to protect Linux servers, services, containers, or virtual machines exposed on the Internet with a server-side agent. It was inspired by Fail2Ban and aims to be a modernized, collaborative version of that intrusion-prevention tool.

CrowdSec is free and open-source (under an MIT License), with the source code available on GitHub. It uses a behavior analysis system to qualify whether someone is trying to hack you, based on your logs. If your agent detects such aggression, the offending IP is then dealt with and sent for curation. If this signal passes the curation process, the IP is then redistributed to all users sharing a similar technological profile to “immunize” them against this IP.

The goal is to leverage the power of the crowd to create a real-time IP reputation database. As for the IP that aggressed your machine, you can choose to remedy the threat in any manner you feel appropriate. Ultimately, CrowdSec leverages the power of the community to create an extremely accurate IP reputation system that benefits all its users.

It was clear to the founders that Open Source was going to be one of the main pillars of CrowdSec. The project's founders have been working on open-source projects for decades - they didn’t just jump on the train. Rather, they are strong Open Source believers. They believe that the crowd is key to the mass hacking plague we are experiencing, and that Open Source is the best lever to create a community and have people contribute their knowledge to the project, ultimately make it better and more secure.

The solution recently turned 1.x, introducing a major architectural change: the introduction of a local REST API.

How CrowdSec Works

CrowdSec is written in Golang and was designed to run on modern, complex architectures such as clouds, lambdas, and containers. To achieve this, it's "decoupled," meaning you can "detect here" (e.g., in your database logs) and "remedy there" (e.g., in your firewall or rproxy).

The tool uses leaky buckets internally to allow for tight event control. Scenarios are written in YAML to make them as simple and readable as possible without sacrificing granularity. The inference engine lets you get insights from chain buckets or meta-buckets, meaning if several buckets (e.g., web scan, port scan, and login attempt failed) overflow into a "meta-bucket," you can trigger a "targeted attack" remediation.

Aggressive IPs are dealt with using bouncers. The CrowdSec Hub offers ready-to-use data connectors, bouncers (e.g., Nginx, PHP, Cloudflare, Netfilter), and scenarios to deter different attack classes. These bouncers can remedy threats in various ways.

Crowdsec works on bouncers such as Captcha, limiting applicative rights, multi-factor authentication, throttling queries, or activating Cloudflare attack mode just when needed. You can get a sense of what's happening locally (and where it's occurring) with a lightweight visualization interface and strong Prometheus observability.

Crowdsourcing Security

While the Crowdsec software currently looks like a spruced up Fail2Ban, the project's goal is to leverage the power of the crowd to create a highly accurate IP reputation database. When CrowdSec bounces a specific IP, the triggered scenario and the timestamp are sent to our API to be checked and integrated into the global consensus of bad IPs.

While we are already redistributing a blocklist to our community, we plan to really improve upon this aspect as soon as we have dealt with other prerequisite code lines. The network already has sightings of 130,000+ IPs (refreshed daily) and is able to redistribute ~10% (13,000) of those to our community members.

Our vision is that once the CrowdSec community is large enough, we will all generate, in real-time, the most accurate IP reputation database available. This global reputation engine, coupled with local behavior assessment and remediation, should allow many businesses to achieve tighter security at a very low cost.

Case Studies

Here are two examples of what CrowdSec does: 

Case #1

A company protecting its customers from DDoS attacks set up a DDoS mitigation strategy relying on Fail2Ban. When one of its customers was attacked by a 7,000-machine botnet, CrowdSec was able to ingest all the logs and successfully banned more than 95% of the botnet, efficiently mitigating the attack in less than five minutes. For the sake of comparison, to deal with this attack Fail2Ban would have needed to process several thousand logs per minute, which is quite challenging and would have taken nearly 50 minutes.

Case #2

An e-commerce business was going through a massive credit card stuffing attack. The attacker was spamming the payment gateway, testing thousands of different credit card details using a sole IP address. Instead of having to amend all of its apps to try to detect the attack, by installing CrowdSec, the company could scan all the logs and block the intrusion within minutes. 

Business model

A common stress among open-source projects is setting up a viable monetization model. So, in full transparency, we'll offer premium subscriptions to businesses that want to leverage our IP reputation database without contributing to it or sharing their banned IP data. This will allow anyone to query the IP reputation database upon receiving the first packet from an unknown IP before accepting it.

Getting Started and Getting Involved

CrowdSec's setup is quick and easy (taking just five minutes, tops). It's heavily assisted by a wizard to allow as many people and organizations as possible to use it. The project is production-grade and already runs in many places, including hosting companies (although it's still in beta).

Currently, community members come from 70+ countries across six different continents and have blocked 130,000+ malicious IPs. The Crowdsec team is looking for more users, contributors, and ambassadors to take the project to the next level.

The team would love to hear your feedback about this latest release. If you are interested in testing the software or would like to get in touch with the team, check the following links:

• Download CrowdSec v1.x
• The CrowdSecwebsite
• Their GitHub repository

Thank you to the Crowdsec project for contributing this article.