KWAS Academy
Learn. Build. Master.
QUICK START:HTMLCSSJavaScriptTypeScriptPythonSQLReactNext.jsNode.jsLinux & UbuntuKotlinSwiftC# / .NETJavaGoRustC++DSASystem DesignDevOpsCybersecurityAI / ML
Intermediate 20 min readModule: Module 9: System Logging, journalctl & Log Rotation

System Logging with journalctl & Logrotate

Query system and service logs with journalctl, filter by unit, priority, and timestamps, and manage disk space with logrotate configuration.

What You Will Learn in This Lesson

  • Querying binary systemd journal logs using journalctl
  • Filtering logs by unit (`-u`), boot session (`-b`), and priority level (`-p err`)
  • Viewing real-time live log streams (`journalctl -f`)
  • Configuring automated log rotation policies in /etc/logrotate.d/

Introduction & Core Concept

Linux servers generate continuous operational telemetry. On modern Ubuntu and Debian systems, systemd-journald captures kernel messages, system events, daemon output, and standard error streams in a high-speed indexed binary log format. The logrotate daemon automatically compresses, archives, and removes old text logs to protect disk space.
WHY DOES THIS MATTER IN THE REAL WORLD?

When servers crash or services encounter errors, system logs provide the definitive audit trail. Mastering journalctl filtering and logrotate policies ensures you can diagnose root causes rapidly without running out of server disk space.

Syntax & Structure

bash
journalctl -u nginx.service -f
journalctl -p err -b
journalctl --since "1 hour ago"
logrotate -d /etc/logrotate.conf

Diagnosing System Issues with journalctl and Logrotate

bash
bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
#!/usr/bin/env bash
# Production Log Analysis with journalctl
 
echo "=== 1. System Errors on Current Boot ==="
journalctl -b -p err..emerg --no-pager | head -n 20
 
echo -e "
=== 2. Inspecting Specific Service Logs (Last 30 Minutes) ==="
# journalctl -u myapp.service --since "30 min ago" --no-pager
 
echo -e "
=== 3. Custom Logrotate Configuration Example (/etc/logrotate.d/kwas-app) ==="
cat << 'EOF'
/var/log/kwas-app/*.log {
    daily
    missingok
    rotate 14
    compress
    delaycompress
    notifempty
    create 0640 appuser appuser
    sharedscripts
    postrotate
        systemctl reload kwas-app.service > /dev/null 2>&1 || true
    endscript
}
EOF

Line-by-Line Technical Breakdown

1Syslog Priority Levels: 0=Emergency, 1=Alert, 2=Critical, 3=Error, 4=Warning, 5=Notice, 6=Informational, 7=Debug. `journalctl -p 3` returns all entries at Error level or higher.

Try It Yourself (Interactive Editor)

Modify the code in real-time and click Run to test live browser output and console logs.

Intelligent Code Runner & Live Sandbox[BASH]
BASH SOURCE EDITOR
Interactive Live Code

Common Mistakes & How to Avoid Them

#1: Deleting an active open log file with rm without reloading the logging daemon.

If you rm a file that a running process still has open, Linux maintains the inode open in memory. The disk space is NOT freed until the process is restarted.

Incorrect / Antipattern
rm /var/log/app/access.log
Correct / Professional Solution
truncate -s 0 /var/log/app/access.log
# OR configure proper logrotate

Industry Best Practices & Professional Standards

  • Use `journalctl -u <service> -n 100 --no-pager` for rapid diagnostic inspection in scripts.
  • Always specify compression (`compress`, `delaycompress`) in logrotate configurations.
  • Truncate active log files (`truncate -s 0 file.log`) instead of deleting them directly when performing emergency disk cleanup.

Lesson Summary & Core Takeaways

  • journalctl provides indexed, binary log querying across all systemd units.
  • Filter logs by unit (`-u`), boot (`-b`), priority (`-p`), and timeframe (`--since`).
  • logrotate manages automated rotation, compression, and disk space preservation.