How Reverse Engineering Can Help Secure Your Linux Systems Against Malware

For many years malware was solely a threat to Windows users - but that era is over. Cyber criminals have come to view Linux as a viable target for their attacks due to the growing popularity of the open-source OS and the plethora of high-value devices it powers.

2019 and 2020 have been plagued by the emergence of dangerous Linux malware variants - CloudSnooper, EvilGnome and HiddenWasp, among many others - and this trend is expected to be magnified in 2021 and beyond, as Linux malware operators are recognizing great success with their malicious campaigns. Thus, taking proactive measures to secure your Linux systems against malware has never been more important. 

Reverse engineering, or the process of deconstructing malware in an artificial environment such as a Linux system to gain insight into its design, architecture and code, is a highly effective method of malware detection and analysis. This article will examine how reverse engineering can be used to secure Linux systems against malware and other exploits, and will introduce our favorite tools, toolkits and utilities for reverse engineering and malware scanning available to Linux users.

How Can Reverse Engineering Help Detect, Analyze and Protect against Malware?

Reverse engineering helps administrators identify, study and eliminate security risks on their systems, and then apply the knowledge they have gained to prevent future attacks. It involves disassembling - and sometimes decompiling - a software program that can often be classified as malware. By converting binary instructions to code mnemonics or higher level constructs, reverse engineers (often referred to as “reversers”) can analyze characteristics of a malicious program including its behavior, systems that it impacts and vulnerabilities that it exploits. These valuable details can then be used to create effective solutions capable of mitigating the program’s intended malicious effects.

Dynamic analysis, which relies on a closed system known as a sandbox to launch a malicious program in a secure environment and then watch what it does, is gaining importance in the realm of reverse engineering due to the speed and automation that it offers. As emerging malware strains continue to demonstrate increasingly complex techniques, reversers need more time to understand disassembled or decompiled code - and this is critical time during which malware could potentially be compromising a network. The use of dynamic analysis can make reverse engineering more efficient and effective; however, reverse engineers should not rely solely on dynamic techniques, as sophisticated malware variants often employ evasion techniques to detect that they are in a sandbox and delay or hide malicious activities.

The best approach to modern reverse engineering for malware detection and analysis is a two-pronged approach where dynamic analysis is used to automatically analyze the majority of threats, while reversers dedicate their time to analyzing and acquiring threat intelligence from the most sophisticated attacks.

Now that we’ve explored how reverse engineering can help you secure your Linux systems against malware, let’s take a look at some great open-source reverse engineering and malware scanning tools, toolkits and utilities available to assist in the process.

Our Top Tools, Toolkits & Utilities for Reverse Engineering & Malware Scanning on Linux

REMnux

REMnux is a free, versatile toolkit for reverse-engineering and malware analysis. It conveniently allows reversers and analysts to investigate malware without having to find, install and configure the tools needed to do so. REMnux offers a distro which can be either downloaded as a VM in the OVO format and then imported into your hypervisor, installed from scratch on a dedicated host, added to an existing system running a compatible version of Ubuntu, or run as a Docker container.

Chkrootkit

Chkrootkit is a widely-used free rootkit detector that locally scans for signs of a rootkit and hidden security holes on Unix/Linux systems. The program consists of a shell script that checks system binaries for rootkit modification, and uses ‘strings’ and ‘grep’ (Linux tool commands) to detect potential security threats. Chkrootkit can be used from an alternative directory or from a rescue disc to verify an already compromised system, and is capable of locating deleted entries in the “wtmp” and “lastlog” files, finding sniffer records or rootkit configuration files and checking for hidden entries in “/proc” or calls to the “readdir” program.

Chkrootkit can be downloaded here.

Rkhunter

Rkhunter is a powerful, user-friendly tool designed to inspect and analyze Linux systems for hidden security holes and scan for rootkits, backdoors and local exploits. Rkhunter thoroughly checks files, default directories, kernel modules, and misconfigured permissions and compares them to the records of databases to identify suspicious programs.

Rkhunter can be downloaded 1.

Lynis

Lynis is a popular and free malware and vulnerability scanning and auditing tool for Unix/Linux OSes. Lynis detects security holes and configuration flaws, performs firewall auditing and checks file/directory permissions, file integrity and installed software, but its capabilities extend far beyond these. In addition to exposing vulnerabilities, Lynis suggests mitigations for the flaws it identifies. 

Lynis can be downloaded Lynis downloads.

Learn how to install and use Lynis in this OpenSource.com tutorial.

LMD

Linux Malware Detect (LMD) is a full-featured malware scanner designed specifically for hosted environments; however, LMD can be used to detect threats on any Linux system. The renowned program uses a signature database to identify and rapidly terminate malicious code running on a system. To populate its database, LMD captures threat intelligence data from network edge intrusion detection (IDS) systems. This approach enables the program to generate new signatures for malware that is being actively used in attacks. LMD includes a full reporting system where administrators can view both current and past scan results and receive email alerts after each scan. The scanner can be integrated with the ClamAV engine for improved performance.

The Bottom Line

Linux malware is a growing concern for administrators, as both the prevalence and sophistication of variants targeting Linux systems continues to increase. That being said, the rise in Linux malware is not a reflection of the security of Linux, as the majority of attacks on Linux systems can be attributed to misconfigured servers and poor administration.

Testing and verifying server security on an ongoing basis is crucial in preventing attacks, and reverse engineering is an excellent method of detecting and analyzing malware on Linux systems, and gathering threat intelligence that can be used to prevent future attacks. There is an array of great open-source tools, toolkits and utilities for reverse engineering and malware scanning available to Linux users, the majority of which are powerful, user-friendly and free to download.

Have questions about reverse engineering? Currently using one or more of the tools that we’ve highlighted in this article? We'd love to hear about your experience and/or answer your questions! Please do not hesitate to reach out to us on social media: Twitter | Facebook