Linux Security
Linux Security
Linux Security

Get started with CrowdSec v.1.0.X

Date 01 Mar 2021
1699
Posted By Brittany Day
Crowd Is The Sol V1

Thank you to the Crowdsec project for contributing this article.

Introduction

The official release of CrowdSec v.1.0.X introduces several improvements to the previous version, including a major architectural change: the introduction of a local REST API.

This local API allows all components to communicate more efficiently to support more complex architectures, while keeping it simple for single-machines users. It also makes the creation of bouncers (the remediation component) much simpler and renders them more resilient to upcoming changes, which limits maintenance time.

In the new 1.0 release, the CrowdSec architecture has been deeply remodeled:

https://lh5.googleusercontent.com/S8nIWzLkbaVptPJzvjD6gtA-XUqCTg0r57L3juWWvUwCL9AqvAAjlWAyuV05hCbBiOcxSpZAacdFzR0tPaVnBpCZfKHWpN3jL8UGpHXzyzEzjs4OQuHR_90ar7oRf1vs9z9FwUOo

All CrowdSec components (the agent reading logs, cscli for humans, and bouncers to deter the bad guys) can now communicate via a REST API, instead of reading or writing directly in the database. With this new version, only the local API service will interact with the database (e.g. SQLite, PostgreSQL and MySQL).

In this tutorial, we are going to cover how to install and run CrowdSec on a Linux server:

  • CrowdSec setup 
  • Testing detection capabilities
  • Bouncer set up
  • Observability 

Set up the environment

The machine I used for this test is a Debian 10 Buster t2.medium EC2.

To make it more relevant, let’s start by installing nginx:

$ sudo apt-get update

$ sudo apt-get install nginx

Configure the security groups so that both secure shell (SSH) (tcp/22) and HTTP (tcp/80) can be reached from the outside world. This will be useful for simulating attacks later.

Install CrowdSec

Grab the latest version of CrowdSec:

$ curl -s https://api.github.com/repos/crowdsecurity/crowdsec/releases/latest | grep browser_download_url| cut -d '"' -f 4  | wget -i -

You can also download it from our GitHub page

Here is the installation process :

$ tar xvzf crowdsec-release.tgz 

$ cd crowdsec-v1.0.0/

$ sudo ./wizard.sh -i


The wizard helps guide installation and configuration.

https://lh6.googleusercontent.com/iCri1IP_vA7Ad9FdAsge-2IOLT0RzAUW29GXqym-uaEty4gEWti4hEJps_k41TGhmBxheVp4HkZijr5ejGroo1gyrbswBdFqZXiqSQQgK9uXQK0PLU5GRRm_yO8TyfHL_jXLu5XT

First, the wizard identifies services present on the machine 

https://lh5.googleusercontent.com/m6UcORJZBLxC3jax6_Hn94twupZgCqtcNgUnWgKf_UcRS9qZ86whU9NZP1e-P2UtHoeSQtHnAipOKRbSXreDTUwanJn4hoCXfUpTuSd5JvUK_01l3TzN6r0KPDWG3BkLyLR6Emz6

It allows you to choose which services to monitor. For this tutorial, go with the default option and monitor all three services: Nginx, SSHD, and the Linux system.

For each service, the wizard identifies the associated log files and asks you to confirm (use the defaults again):

https://lh6.googleusercontent.com/fH3hUEB93JG_XaLFTm5zDlt2D8DTFWbnTfabyIInuxCRXx1tSfFYxOwxpc9CwMPeqYB0QOFBXEcE-9d3X3jsjCKQu-X4uTc9QY-0wTHIj8DVyWE50JkqRfOVWEk8wPpeZ5mtHGKq

Once the services and associated log files have been identified correctly (which is crucial, as this is where CrowdSec will get its information), the wizard prompts you with suggested collections.

A collection is a set of configurations that aims to create a coherent ensemble to protect a technological stack. For example, the crowdsecurity/sshd collection contains a parser for SSHD logs and a scenario to detect SSH bruteforce and SSH user enumeration.

The suggested collections are based on the services that you choose to protect. 

https://lh4.googleusercontent.com/DOhJVMX5NUx01_47hlDTzWBFdXOnqizuIljEVKQV5Wpf_31oFuM5jVHTMuGdVQgoI2pKUqqm3LE-PxeBev7CXZndg3mdYgp-Of13s4I-IO8gvmfTWtFsYr3oFF3GWe0WCIJr0uyI

The wizard’s last step is to deploy generic whitelists to prevent banning private IP addresses. It also reminds you that CrowdSec will detect malevolent IP addresses but will not ban any of them. You need to download a bouncer to block attacks. 

This is essential to remember: CrowdSec detects attacks; bouncers block them.

https://lh5.googleusercontent.com/YZAmO3cnqSOe6-uhOznyml9e2tD3hNemqF6kkJWi6le6f06XPModVJCgkcZq8OjM7WTH17ZoY8uwcgJKA05J7WkFIbYk6ouTjBeqyQD3TyWkFS1LwirIq9L0N3wJNNaG_B6xuMp8

Now that the initial setup is done, CrowdSec should be up and running.

https://lh6.googleusercontent.com/lQ3s-lghCbEVAJ4pDZ7v0OM662Hwb1srxHlN-MItzpgpD3s6HjJzA0Atf2XebaRqsWmx8fGHHZxBxEX-AJgBsqiMlyOLMEGic58QDwpPNH8_NapSSe0OVckGjCHmmq-kKrpWEecy

Deter attacks with CrowdSec

By installing CrowdSec, you should already have coverage for common Internet background noise. Check it out!

Attacking a web server with wapiti

Simulate a web application vulnerability scan on your Nginx service using Wapiti, a web application vulnerability scanner. You need to do this from an external IP, and keep in mind that private IPs are whitelisted by default:

ATTACKER$ wapiti   -u https://34.248.33.108/

[*] Saving scan state, please wait...


 Note

========

This scan has been saved in the file

/home/admin/.wapiti/scans/34.248.33.108_folder_b753f4f6.db

...

On your freshly equipped machine, we can see the attacks in the logs :

https://lh6.googleusercontent.com/ZbcU3iN9u9N0YlqelredDQKoHSP8YOUD_DzcWR6Fc-lfJjQ32FBNg2pq0CHbHk0xoo8YJ-y2j0qjjsEUpVTScwFHYJ-rle1NaJeYckgwNx3WcGAAuAzFxS9o_C_4pmDutHh4eD4O

My IP triggered different scenarios :

Bear in mind that the website you attacked is an empty Nginx server. If this were a real website, the scanner would perform many other actions that would lead to more detections.

Checking results with cscli

Cscli is one of the main tools for interacting with the CrowdSec service, and one of its features is visualizing active decisions and past alerts.

https://lh5.googleusercontent.com/VzHqiq7KZdqvQCdE0nV5mdyY5nJ5CYgH3jPAEeaVTGsA5-lvocKfTQrIpv_5iaUQSVKzN6ko4b5LVOErqRERVXF1myE4xEsMTmpfq5CrPOxxyuBXhIcBCiQ0MMbA5MzYLvViTfP2

The cscli decisions list command displays active decisions at any time, while cscli alerts list shows past alerts (even if decisions are expired or the alert didn't lead to a decision).

You can also inspect a specific alert to get more details with cscli alerts inspect -d <ID> (using the ID displayed in the left-hand column of the alerts list).

https://lh3.googleusercontent.com/oCKY-9nc_z8GelHuErx4nCgxhYKup4jyRyrFk0CFH6shnk3kl6207JRzDDQhbSk3cqkycs-H74M8f6EbWSgTIN8saodGw9AO8dsSx5XcB4xXpbebo8LwtUgGf3-rqpq_Q7q_5FwZ

cscli offers other features, but one to look at now is to find out which parsers and scenarios are installed in the default setup.

https://lh6.googleusercontent.com/-2YtreUPNRe8mwO9KlDorns-Escfi9ErqY3aYAggvrP6B0QI24dD8hU7xserHAO9vUT1ecshKT6b1vnhmYcFU3AJM_dEP6I81HYIPMZirHyl88q3FtnXPJXx7ZFdIjoDVcPrdlr6

Observability

Observability (especially for software that might take defensive countermeasures) is always a key point for a security solution. Besides its "tail the logfile" capability, CrowdSec offers two ways to achieve this: Metabase dashboards, and Prometheus metrics.

Metabase dashboard

cscli allows you to deploy a new Metabase and Docker. Begin by installing Docker using its official documentation.

https://lh5.googleusercontent.com/KY_cbbBix-3cpbQNDhtotAWTYr6oTzKigje6XYOAdExU2LtZ0E5z3Qhixp6Ld4enATeqTu3tS1NWSgu5a6nqtHmV99OEGc0bMpV4X3Mryt85ePk-FmzIl6D7fifS2dV4XiEl6E5L

If you’re using an AWS EC2 instance, be sure to expose tcp/3000 to access your dashboard.

cscli dashboard setup enables you to deploy a new Metabase dashboard running on Docker with a random password.

https://lh4.googleusercontent.com/rUzdlbmVcJ6u1ljao4tF_fIgcjkuIE0LheBIRiscDR1xy36I-gEnj40EENzZMFUi0qRELSil-oMPBSs-gOZkT3xksARtGXwTPNN1LZU_0ggBM_ODohygSTbBDyDIamUY3LB9DzWm

 

https://lh5.googleusercontent.com/VaO-9KAqXNfld36rVtBWzA6YyEvZjzUiV8jXSqfbiQZV5pxPRZx0mvVEqRyzBCca7-hC2GwUPDbPvoykL4YD6gMh0gCOjcOXASqx5_AIRaODV0HaChnneQ9gEm_0RG_37Wd6T97_

 

https://lh4.googleusercontent.com/_2SsZcG8wtrad-_WuCeBdArKEevaB-W6QokGQ3wc-N9Dx72l9gqBucaoudUAvVce961LJ7VMBZBD_0OPbm60xYAbjweBLejtEEO-oRRBgk6IyfJU7Ai4E1fx-ABJRw7pRmAE031f

https://lh4.googleusercontent.com/uCc0hMw4ZzNLXxdpWqJBvqNjRyKxWBWAUpRo77r_nXfAjnIXKuvDyM9CDIJegG3VvmMVDtjKD6FxwhYivltaP62wJVEuzxOPGV7FmnxXCQk_2cJyde2yYCG6F3RukZD4Z5h-hRkJ

Prometheus metrics

While some people love visual dashboards, others prefer different kinds of metrics. This is where CrowdSec’s Prometheus integration comes into play.

One way to visualize these metrics is with cscli metrics:

https://lh5.googleusercontent.com/9zVYjYS5QCcueeyY9ai3033LwJz7nJPIWIUj4qtgEjUH4NuER2PcQGehP8a_HDlsMO6GpJEkfyVk9qx-6ppoxoanzPRW-1k9KgW633gRnePt3QO9Lpjz-o9tk__zZuGogqW0Y9oN

The cscli metrics command exposes only a subset of Prometheus metrics that are important for system administrators. You can find a detailed description of the metrics in the documentation. The metrics are split into various sections :

  • Buckets: How many buckets of each type were created, poured or have overflowed since the daemon startup?
  • Acquisition: How many lines or events were read from each of the specified sources, and were they parsed and/or poured to buckets later?
  • Parser: How many lines/events were delivered to each parser, and did the parser succeed in processing the mentioned events?
  • Local API: How many times was each route hit and so on?

Viewing Crowdsec’s Prometheus metrics via cscli metrics is more convenient but doesn’t do justice to Prometheus. It is out of scope for this article to deep dive into Prometheus, but these screenshots offer a quick look at what CrowdSec's Prometheus metrics look like in Grafana.

https://lh5.googleusercontent.com/TCkHxB8ktPGxKQN7Q-s2GBreUpeoKOSWJO-O_S6T9DqzxPhyW3b5bOJYhTKh-U1j3JmT6bSaWPqVdD0Qx2TScryZjFc0dGFqBE1WGIyUuz87jdz_OJSHpoqayCmdpkM0RG2OYH4G

https://lh5.googleusercontent.com/Ui3f54mbrpAxQcyT1g9H73_uGN7lb4cv-sgPj61AvDXSF0B8Jwg6FTRCDBMXyMG_mSlhn_KLHR1gwWEWxUofWPgfnf55PM7N5I4roXuTUPtXKF5OisiKbwibilsB8pRu7in2Upk4

Defend attacks with bouncers

CrowdSec's detection capabilities provide observability into what is going on. However, to protect yourself, you need to block attackers, which is where bouncers play a major part. Remember: CrowdSec detects, bouncers deter.

Bouncers work by querying CrowdSec’s API to know when to block an IP. You can download them bouncers directly from the CrowdSec Hub

https://lh5.googleusercontent.com/slW_sim5TzjHHOPkw9CtwuurRP-jJA44zLbMEtb7SHVUmV8kwi5oakR9mUllreKXG88sY0a53CIQctYPcVX-VgXAguxMwD96FSOkacYQwVkCHdzxyIPkgGN58bDweiDw8X8K372j

For this example, use cs-firewall-bouncer. It directly bans directly any malevolent IP at the firewall level using iptables or nftables

Note: if you used your IP to simulate attacks, unban your IP before going further:

sudo cscli decisions delete -i X.X.X.X

Install the bouncer

First, download the bouncer from the Hub:

$ wget https://github.com/crowdsecurity/cs-firewall-bouncer/releases/download/v0.0.5/cs-firewall-bouncer.tgz

$ tar xvzf cs-firewall-bouncer.tgz

$ cd cs-firewall-bouncer-v0.0.5/

The bouncer can be installed with a simple install script:

https://lh3.googleusercontent.com/szfFCssOAeDHx9_QAPuzSXLj2SA5FvIXL763HuJzDWVSHiyMSeenoxd1a5EruastsLsQJk4G6HykDV8NUM12dd-t_u1XcHqooN7aKTmDHd6eRDNGwBZCHttw1yy-t7tOMZjcS8Nh

The install script will check if you have iptables or nftables installed and prompt you to install if not.

Bouncers communicate with CrowdSec via a REST API, so check that the bouncer is registered on the API.

https://lh3.googleusercontent.com/YCNNHZVYtLepvdsUllvPJXj0i7k5T5n5FQ0XLZII0Ko0h-y66mhDrKRAb66tLLanac4q2WLI_Du4sqvz4pEWvukQL97dXB2Keo_E40rNv8kK0mzNAFZVFB7gtcPPA7vU_vK-F6Q-

The last command (sudo cscli bouncers list) shows our newly installed bouncer.

Test the bouncer

Warning: Before going further, ensure you have another IP available to access your machine and that you will not kick yourself out. Using your smartphone's internet connection will work.

Now that you have a bouncer to protect you, try the test again.https://lh4.googleusercontent.com/6Jaq5FC5OM1xgSxn7AGFXRh1hpfLH17n1XPnut7_nveJa8UcA_LVK6z47QcH934s3tU_Vg7D17lPn7d9WVKzdNtqDWet7VQ6p2z1ymAAyvVNUcxOA4L8t0W6uph_cgHp7pBdSyr8

Try to access the server at the end of the scan :

ATTACKER$ curl --connect-timeout 1 https://34.248.33.108/

curl: (28) Connection timed out after 1001 milliseconds

See how it turns out from the defender’s point of view. 

https://lh5.googleusercontent.com/JsyyCkhTqW5XMCroQV0E5ttGrFxdSmOPVKRDlNf0zZ0hvaB1zGdYwh6mlBxaWxwG2LZR5V3T_T5k3m8IZdqOihu4pYLp5knnl5sf2QJerv1duf-QE0ZrTg1aPv6N2q66LpNMLnzT

For the technically curious, cs-firewall-bouncer uses either nftables or iptables. Using nftables (used on Debian 10 by default) creates and maintains two tables named crowdsec and crowdsec6 (for IPv4 and IPv6 respectively).

$ sudo nft list ruleset

table ip crowdsec {

set crowdsec_blocklist {

type ipv4_addr

elements = { 3.22.63.25, 3.214.184.223,

    3.235.62.151, 3.236.112.98,

    13.66.209.11, 17.58.98.156, …

                        }

}


chain crowdsec_chain {

type filter hook input priority 0; policy accept;

ip saddr @crowdsec_blocklist drop

}

}

table ip6 crowdsec6 {

set crowdsec6_blocklist {

type ipv6_addr

}


chain crowdsec6_chain {

type filter hook input priority 0; policy accept;

ip6 saddr @crowdsec6_blocklist drop

}

}

You can change the firewall backend used by the bouncer in /etc/crowdsec/cs-firewall-bouncer/cs-firewall-bouncer.yaml by changing the mode from nftables to iptables  (ipset is required for iptables mode).

Get involved

We would love to hear your feedback about this latest release. If you are interested in testing the software or would like to get in touch with the team, check the following links:

Advisories

LinuxSecurity Poll

How frequently do you patch/update your system?

No answer selected. Please try again.
Please select either existing option or enter your own, however not both.
Please select minimum 0 answer(s) and maximum 3 answer(s).
/main-polls/52-how-frequently-do-you-patch-update-your-system?task=poll.vote&format=json
52
radio
[{"id":"179","title":"As soon as patches\/updates are released - I track advisories for my distro(s) diligently","votes":"67","type":"x","order":"1","pct":75.28,"resources":[]},{"id":"180","title":"Every so often, when I think of it","votes":"14","type":"x","order":"2","pct":15.73,"resources":[]},{"id":"181","title":"Hardly ever","votes":"8","type":"x","order":"3","pct":8.99,"resources":[]}] ["#ff5b00","#4ac0f2","#b80028","#eef66c","#60bb22","#b96a9a","#62c2cc"] ["rgba(255,91,0,0.7)","rgba(74,192,242,0.7)","rgba(184,0,40,0.7)","rgba(238,246,108,0.7)","rgba(96,187,34,0.7)","rgba(185,106,154,0.7)","rgba(98,194,204,0.7)"] 350

Please vote first in order to view vote results.

VOTE ON THE POLL PAGE


VIEW MORE POLLS

bottom 200

Please enable / Bitte aktiviere JavaScript!
Veuillez activer / Por favor activa el Javascript![ ? ]

We use cookies to provide and improve our services. By using our site, you consent to our Cookie Policy.