4. Log Structures & System Events

Logging and system event analysis form the backbone of modern cybersecurity operations. While advanced technologies such as AI-driven detection, intrusion prevention systems, and SIEM platforms often receive the most attention, their effectiveness depends fundamentally on the accuracy, completeness, and structure of system logs. Logs represent the chronological, immutable record of system behavior, enabling incident detection, forensic investigations, performance monitoring, compliance validation, and threat hunting. Without robust logging, attackers can operate silently, abuses become invisible, and forensic efforts fail to reconstruct what happened before or during a compromise.

This chapter provides a comprehensive overview of log structures and system events in both Linux and Windows environments. It covers how logs are created, what they contain, how they should be collected and interpreted, and how event data becomes actionable intelligence. Drawing on Chapple’s emphasis on security monitoring fundamentals and Stallings’ principles of system integrity mechanisms, this chapter aims to equip students with the expertise required to understand, analyze, and operationalize logs as a core element of cybersecurity practice.

Foundations of Log Management

Log management refers to the processes of generating, collecting, storing, analyzing, and protecting log data. It encompasses both the technical mechanisms of system event generation and the organizational policies that govern retention, analysis, and access.

The Purpose of Logging

Logs serve several essential security functions:

• Security Monitoring: Identifying suspicious processes, failed authentication attempts, unexpected privilege escalations, or abnormal network activity.
• Incident Response: Reconstructing attacker actions, determining the extent of compromise, and validating containment or remediation efforts.
• Compliance: Meeting regulatory requirements such as PCI-DSS, HIPAA, GDPR, and ISO 27001.
• Operational Auditing: Monitoring system performance, resource usage, and administrative activity.
• Forensics: Preserving immutable evidence for investigation or legal action.

Logs represent the “memory” of a system, once lost or overwritten, critical evidence can disappear forever.

Characteristics of High-Quality Logs

Effective logs must adhere to several key characteristics, many of which align with Stallings’ principles of system integrity:

1. Accuracy: Correctly reflect real system behaviors
2. Consistency: Uniform formatting to facilitate automated parsing
3. Completeness: Sufficient detail to support detection and investigation
4. Integrity: Protected from unauthorized modification
5. Timeliness: Events generated and transmitted promptly
6. Retention: Stored for an adequate period as defined by security policies

Logs that lack any of these qualities diminish the organization’s ability to detect and respond to threats.

Linux Log Structures

Linux systems rely heavily on textual logs stored under the /var/log directory. These logs provide detailed visibility across authentication attempts, kernel-level events, service activity, and general system behavior. They are formatted for ease of parsing and compatibility with centralized log management platforms.

Syslog Architecture

Syslog is the foundational logging protocol for many Linux distributions. It defines:

• Format of messages
• Facilities (auth, cron, kernel, mail, daemon, user, etc.)
• Severity levels (from emerg to debug)
• Destination rules for log storage or forwarding

Syslog messages follow a standardized format, typically:

<Priority>Timestamp Hostname Process[PID]: Message

This structured approach enables administrators to programmatically filter, sort, and route logs.

Syslog Implementations

Common implementations include:

• rsyslog (default on many distributions)
• syslog-ng
• journald (part of systemd ecosystem)

Modern systems often use rsyslog as a bridge between journald logs and external log servers or SIEM platforms.

Systemd Journald

Systemd introduced journald, a binary logging system that collects structured metadata including:

• Timestamps
• Process identifiers
• User and group IDs
• Service units
• Boot session IDs
• Kernel messages

Journald provides enhanced structure, indexing, and logging security features such as forward integrity.

Commands like journalctl allow powerful filtering, including:

• Logs from a specific service
• Logs since last boot
• Kernel logs
• Logs for a particular user

This deep visibility is essential for security investigation.

Key Linux Log Files

/var/log/auth.log or /var/log/secure

Tracks:

• Authentication attempts
• sudo usage
• SSH logins
• Privilege escalations
• PAM events

This log is fundamental for detecting brute-force attacks, lateral movement attempts, and unauthorized privilege use.

/var/log/messages & /var/log/syslog

Contain general system activity and daemon interactions.

/var/log/kern.log

Provides kernel-level alerts, detecting:

• Kernel panics
• Hardware failures
• Low-level driver issues

/var/log/dmesg

Logs kernel ring buffer messages, often used during boot diagnostics.

Service Logs

Specific daemons (e.g., Apache, Nginx, MySQL) maintain their own logs for performance analysis and intrusion detection.

Log Rotation

Linux uses logrotate to:

• Archive old logs
• Compress historical logs
• Define retention periods
• Prevent disks from filling up

Security practitioners must configure rotation policies carefully, overly aggressive deletion reduces forensic visibility.

Windows Event Logging System

Windows logging operates through a structured, XML-based event architecture that logs events in categorized channels. Because Windows systems are widely deployed in enterprises, understanding Event Viewer and Windows event channels is essential for defenders.

Windows Event Viewer Architecture

Windows uses the Event Logging Service to collect, store, and manage logs in .evtx format. These logs are structured and indexed for efficient search.

Windows logs are organized into:

Windows Logs

• Application: Events from installed applications
• Security: Authentication, logon events, access control actions
• System: Kernel and system events
• Setup: Installation-related events
• Forwarded Events: Collected from other hosts

Applications and Services Logs

• Fine-grained events for specific applications
• Provides deep troubleshooting and security telemetry

Critical Windows Event IDs

Certain event IDs are essential for security monitoring:

Authentication & Access:

• 4624: Successful logon
• 4625: Failed logon
• 4634: Logoff
• 4648: Logon using explicit credentials
• 4672: Special privileges assigned to new logon

Privilege Escalation & Lateral Movement:

• 4670: Permissions on objects changed
• 4688: New process created
• 4698: Scheduled task created
• 4769: Kerberos service ticket request

Persistence:

• 7045: New Windows service installed
• 4719: System audit policy changed

These event IDs are central to detecting malware, APT behavior, and insider threats.

Windows Registry & Sysmon Integration

Sysmon (System Monitor)

Part of the Sysinternals suite, enhances logging by capturing:

• Process creation and termination
• Network connections
• File creation events
• Registry modifications
• Hashes of executed binaries

When properly configured, Sysmon logs offer deep visibility into attacker behavior.

Sysmon logs are stored under:

Applications and Services Logs → Microsoft → Windows → Sysmon/Operational

Network and Security Device Logs

System logs are only part of the picture. Security professionals need to interpret logs from:

• Firewalls (Cisco ASA, Fortinet, Palo Alto)
• IDS/IPS systems (Snort, Suricata)
• Web proxies
• Load balancers
• VPN concentrators
• DNS servers
• DHCP servers

These logs help identify:

• Reconnaissance activity
• Port scanning
• Suspicious connections
• Data exfiltration
• Malware command-and-control patterns

Because logs are generated across distributed systems, centralized log management becomes necessary.

Log Integrity & Protection

Log integrity is critical in cybersecurity. Attackers often attempt to:

• Delete logs
• Modify entries
• Disable logging services
• Overwrite logs using log flood attacks

Security best practices include:

• Restricting log file permissions
• Using append-only filesystems
• Forwarding logs to remote servers
• Applying digital signatures or cryptographic hashing
• Leveraging journald forward-secrecy features

Logs must be protected as rigorously as other security assets.

Centralized Logging & SIEM Integration

Modern organizations cannot rely on local logs alone due to volume, complexity, and distributed architectures.

Centralized Logging Advantages

• Single pane of glass for analysis
• Correlation across systems
• Real-time alerting
• Long-term storage
• Compliance reporting
• Forensic continuity

SIEM platforms (Splunk, Elastic, QRadar, Sentinel) ingest and analyze logs, detecting anomalies using correlation rules, machine learning, and behavior models.

Detecting Suspicious Events & Attack Patterns

Logs reveal attacker behavior when properly interpreted.

Examples:

• Repeated failed logins → brute-force attack
• Sudden privilege escalations → compromise of a privileged account
• Unknown service installation → persistence mechanism
• Powershell execution with encoded commands → malicious scripting
• Unexpected outbound traffic → data exfiltration

Understanding normal system behavior is key to recognizing anomalies.

Retention Policies & Compliance

Regulatory standards require specific log retention periods:

• PCI-DSS → 1 year (3 months immediately available)
• HIPAA → 6 years
• SOX → 7 years

Longer retention improves forensic investigation but requires storage planning and encryption to ensure data confidentiality.

Log structures and system events represent one of the most critical areas of cybersecurity mastery. Whether analyzing SSH logs on Linux, parsing Kerberos ticket requests on Windows, or correlating firewall events with endpoint telemetry, defenders rely on logs to detect, investigate, and respond to cyber threats. Proper logging architecture combines technical configuration, operational discipline, centralized analysis, and strong security controls to guarantee that logs remain accurate, actionable, and tamper-resistant.

By understanding how events are generated, stored, analyzed, and protected, cybersecurity practitioners gain the insight necessary to build resilient monitoring capabilities and support advanced security operations such as threat hunting, behavioral analytics, and digital forensics.