Linux Server Security: Essential Guide to Secure Your Server

Are your Linux servers secure? No machine connected to the internet is 100% secure, of course. In the words of security guru Bruce Schneier: “Security is a process, not a product.” However, this doesn't mean that you are helpless. Although cyber attacks, hacks and breaches are sometimes unavoidable, all system administrators and users can take definitive measures to mitigate their risk online.

A Linux system is like a house. It’s easy to keep the windows and doors always closed, but controlling when and how they get opened is an ongoing challenge. And the fact that poor administration and misconfigured servers are to blame for the majority of exploits on Linux systems is evidence that many system administrators are failing to meet this challenge.

This introductory guide will explore the fundamentals of information security and explain how these principles can be applied to evaluate and improve the security of your Linux servers.

Linux: An Increasingly Popular Target Among Cyber Criminals

Due to its growing popularity, Linux is increasingly being targeted in dangerous campaigns - despite its heralded security. Threat actors have come to view Linux servers as yet another viable target that often provides a valuable return on investment. Unfortunately, 2019 and the first half of 2020 have been filled with emerging malware variants targeting Linux servers - and a plethora of resulting security news headlines harping on how “insecure” Linux is. But if you investigate a bit, you’ll find that many of these stories are fake news, and that the underlying issue is incompetant system administrators. After all, regardless of the OS that you're running, if you’re failing at security, it doesn't matter how “secure” your OS is. For instance, the recent BootHole vulnerabilities that dominated security news coverage required admin access to exploit, making them very dangerous to an already hacked system. What was almost always overlooked in the coverage of BootHole was that if someone has root access to your system, you’re already in serious trouble!

The good news for Linux users is that in reality, despite the uptick in threats targeting Linux servers, Linux remains an exceptionally secure OS - especially when using a specialized secure Linux distro. The transparency of Linux source code and the constant scrutiny that this code undergoes by a vibrant global community results in the rapid detection and elimination of security vulnerabilities and potential exploits.

In comparison, security flaws generally remain undetected for significantly longer in closed-source proprietary code. Malicious actors recognize and exploit this, directing the majority of their attacks at proprietary software, platforms and operating systems.

That being said, the rise of Linux malware should serve as a much-needed wake up call for the security industry to invest additional resources in threat protection, detection and response.

What Is a Security Framework?

A security framework outlines the basic steps in the life cycle of securing a system. The following illustrates the framework needed to establish and maintain a secure system:

[ Risk Analysis ] [ Business Requirements ]

[ Security Policy ]

[ Security Service, Mechanisms, and Objects ]

[ Security Management, Monitoring, Detection and Response ]

‘Risk Analysis’ is the process of identifying and analyzing potential issues that could impact the security of a system, while ‘Business Requirements’ deals with the actual requirements for conducting business. These two components address the business aspects of s security framework.

The ‘Security Policy’ is the theoretical aspect of a security framework, and defines what it means to be “secure”. ‘Security Service, Mechanisms and Objects’ is the implementation aspect of security, and ‘Security Management, Monitoring, Detection and Response’ is the operational facet of security, which covers the specifics of identifying and reacting to a potential security breach.

When looking to secure your system, there are a selection of benefits associated with establishing a security framework:

Helps administrators understand the security status of their system
Communicates cybersecurity requirements with superusers and users
Identifies opportunities for new or revised standards
Assists in prioritizing improvement activities

As you can see, developing a security framework is a worthwhile investment for any system administrator looking to improve the security of his or her system.

Information Security Basics: Breaking It Down

Information security, or the process of protecting information against unauthorized use, can be broken down into two parts: (1) Physical security / host security and (2) Network security. Each of these components has three facets:

Protection: Slow down or stop intrusions or damage
Detection: Alert someone if a breach (or attempted breach) of security occurs, and quantify and qualify what sort of damage occurred or would have occurred
Recovery: Re-secure the system or data after the breach or damage has occurred and, where possible, undo whatever damage was done

Host Security/Physical Security

This type of security involves protecting the server from unauthorized access. Physical security/host security can be achieved by password protecting the physical server with steps such as setting up a bios password, placing the box in a locked room where only authorized users have access, applying OS security patches, checking and correcting the permissions on all OS related files and reviewing logs on a regular basis for any signs intrusion or attacks.

Tips for Improving Host Security:

Protect your systems with strong passwords
Check the file systems and set correct permissions and ownerships on all directories and files
Apply security patches to vulnerable software
Remove all unnecessary ttys and console logins by removing the entry from /etc/securetty
Check system logs (eg: /var/log/messages, /var/log/secure, etc.)
Set a password on the boot loader
Monitor the system

Network Security

Network security is one of the most important aspects of overall security. As mentioned above, no machine connected to the Internet is completely secure - so security administrators and server owners must always be alert and ensure that they stay informed of the latest security bugs and exploits. Failure to keep up with these vulnerabilities leaves users at risk of breaches, hacks, malware and other dangerous cyber attacks.

Advice for Improving Network Security:

Use custom security scripts which will send out notifications when sshing as root, creating a user with uid of 0, etc.
Set up an idle timeout, so that idle users will be logged out after a certain amount of time.
Restrict direct root login (comment out the PermitRootLogin login option in sshd_config).
Limit user’s resources (using pam, specify the limits for each user in /etc/security/limits.conf).
Hide the server details. Remove /etc/issues and /etc/issues.net, for example.
Install a firewall (eg: apf and iptables) and only allow ports that the box needs for its normal functions to operate; block all other ports to prevent mischief.
Deploy honeynets for intrusion detection.
Restrict ssh to specific IP addresses and specific users.
Use a quality VPN (like Wireguard) to encrypt data between you and your server.
Check router firmware for security vulnerabilities and eliminate potential bugs with a Linux firmware replacement.

Organizations that require greater control over hardware, network segmentation, and resource isolation often deploy bare metal servers to reduce shared-environment risks and strengthen the overall security posture of critical workloads.

Security is an Ongoing Balancing Act

Maintaining a secure system is an ongoing process. This process is all about balancing trade offs: administrators must continuously weigh investments in security with the costs and potential impacts to system performance and user productivity. Optimal security is a delicate balance of cost, user experience and risk. And all of these factors are constantly changing - digital threats are ever-evolving, users’ expectations shift and growing demands of infrastructures alter the environment being secured. The challenge is to fully understand these factors, how they impact each other and how they are changing, and to determine the amount and types of security needed to best meet the expectations of all parties involved.

Getting Started with System Hardening

System hardening - or the process of securing a system by reducing its surface of vulnerability - is essential to establishing and maintaining a good security posture. Some methodologies and best practices for hardening your Linux system include:

Keep up to date on security news by subscribing to our Linux Security Week newsletter, which summarizes the week’s most relevant open-source security news.
Create a good security policy. Conduct security audits on the basis of this policy.
Keep your OS updated by applying all patches.
Install a custom kernel with all unwanted services removed and patched.
Disable all unwanted services and harden the services you leave running.
Change file and directory permissions to tighten security.
Install a firewall and create good rule sets.
Test and audit the server on a regular basis (covered in more detail below).
Install an intrusion detection system and a log monitor.
Make your partitions secure.
Run a good backup system to recover data in case of an intrusion, crash or other type of destructive incident.
Install a log analyzer and check your logs frequently for any suspicious entries.
Install scripts to send out mail or enable notifications when a suspected security breach occurs.
After a security breach, try to find out how, when and through what the breach occurred. When you find a fix for it, document the details for future reference.

Security Audits: What Should be Checked?

A security audit aims to identify any vulnerabilities present in a system and suggest actions to improve these flaws. In a normal audit, the points below should be checked. A report with the results of this audit should be created.

Check intrusion detection. Use chkrootkit or rkhunter for this purpose.
Check for known bugs in the software installed on the server - the kernel, openssl, openssh, etc.
Scan all network ports and find out which ports are open. Report the open ports that should not be open and the program that is listening on them.
Check whether /tmp is secured.
Check for hidden processes.
Check for bad disk blocks in all partitions.
Check for unsafe file permissions.
Check whether the kernel has a ptrace vulnerability.
Check the memory.
Check if the server is an open email relay.
Check if the partitions have enough free space.
Check the size of the log files. It's better that the log size remains in megabytes.

How To Determine if You’re Being Hacked?

If you suspect that your system has been compromised, here are some very basic steps you can take to determine if you’re being hacked:

Check if your performance has degraded or if your machine is being overused.
Check if your server has any hidden processes running.
Install an intrusion detection system (IDS) like OSSEC or Snort to detect anomalies in network traffic.
Use intrusion detection tools like chkrootkit or rkhunter to check your system for rootkit infections.
Check your machine’s uptime.
Identify unknown processes and determine what they are doing.
Monitor users’ activity using commands “w” or “who”.
Check network traffic by running the command “iftop” with sudo.

The Bottom Line

Security is of utmost importance to a server - compromising on server security is compromising on the security of your system as a whole. Hence, a solid understanding of information security is a prerequisite to successful server ownership and administration.

Security is a continuous learning process - mistakes are inevitable, and can provide valuable insight for the future. Thus, it is critical that administrators take the time to fully understand both their successes and failures - enabling them to identify what is working and where there is room for improvement.

The host of recent attacks exploiting Linux systems has brought both the prevalence and the impact of poor system administration to light, and has shown the importance of diligently monitoring servers for anomalies or signs of compromise. It is time for system administrator to make server security a priority.

Have additional questions about securing your server? Leave them in the comments below - we’d love to help you out!