Syslog

Mental model

Every switch, router, firewall, and server produces log messages — events, warnings, errors. Without centralization, those logs live on each device’s tiny local buffer and disappear when the buffer wraps. When something breaks at 3 AM, you’re SSHing into 12 devices reading scrollback.

Syslog is the standard fix: every device ships its logs to a central server, where they’re stored, indexed, and searched. Combined with synced clocks via NTP, this turns scattered log files into a single timeline you can correlate.

The format is dirt-simple — a line of text per message, prefixed with severity, facility, timestamp, and host. Modern syslog servers (Splunk, ELK, Graylog, Loki, Datadog) parse and index millions of these per second.

The 8 severity levels

Memory aid: “Every Awesome Cisco Engineer Will Need Ice-cream Daily” (Emergency, Alert, Critical, Error, Warning, Notice, Informational, Debug).

Production rule of thumb: log level 6 (informational) to the syslog server. Reserve 7 (debug) for active troubleshooting only — debug-level traffic floods the syslog server and saturates the management link.

Where Cisco devices can log

A Cisco device has multiple log “destinations,” each independently configurable:

You typically log:

• Console: warnings and above (level 4) — don’t spam the console
• Buffer: level 6 — keep recent local history
• Syslog server: level 6 — the real long-term record

Commands

Basic syslog server config

R1(config)# logging host 10.0.99.5
R1(config)# logging trap informational                ! level 6 and above to syslog server
R1(config)# logging source-interface Loopback0        ! consistent source IP for the server

Local buffer

R1(config)# logging buffered 16384                    ! 16 KB buffer (default is small)
R1(config)# logging buffered informational            ! level 6 and above
R1# show logging                                       ! view it
R1# clear logging                                      ! wipe it

Console + terminal

R1(config)# logging console warnings                  ! only level 4 and above to console
R1(config)# logging monitor informational             ! level 6 to vty
R1# terminal monitor                                   ! turn on log forwarding to your SSH session
R1# terminal no monitor                                ! turn it off when you're done

Timestamps (make every log entry useful)

R1(config)# service timestamps log datetime msec localtime show-timezone

This makes every log line look like:

*Aug 15 14:23:01.234 PDT: %LINK-3-UPDOWN: Interface GigabitEthernet0/0, changed state to up

The components: timestamp · facility (LINK) · severity (3) · mnemonic (UPDOWN) · message text.

Reading a Cisco log message

*Aug 15 14:23:01.234 PDT: %SYS-5-CONFIG_I: Configured from console by admin on vty0 (10.0.0.5)

Breakdown:

• *Aug 15 14:23:01.234 PDT — when (NTP-synced if you set up NTP)
• %SYS-5-CONFIG_I — facility-severity-mnemonic — SYS=facility, 5=severity (Notice), CONFIG_I=specific event
• The rest — human-readable description

The %FACILITY-N-MNEMONIC pattern is consistent across all Cisco IOS messages. Useful for filtering with grep.

Common mistakes

1. Sending debug-level (7) to the syslog server. Floods the server, fills disk, masks real signals. Always set logging trap informational (6) for the remote server.

2. No NTP. Every device timestamps logs in its own clock, which drifts. Correlation across devices becomes impossible. Always configure NTP before relying on syslog.

3. Forgetting service timestamps log datetime. Default timestamps are uptime-based (*00:01:23) instead of wall-clock. Useless for forensic work.

4. No logging source-interface. Without this, the device sources syslog from whichever interface routes to the server — potentially different each time. The server sees the same device with different IPs. Pin it to a loopback or management interface.

5. No buffer at all. If the syslog server is unreachable (network outage during the outage, naturally), all log info is lost. Always configure a local buffer too.

6. Logging passwords or secrets. Some auth failure messages can include the attempted username/password. Sanitize before storage. Avoid logging at debug level on auth subsystems.

7. Treating syslog like a database. It’s append-only text. For metric-style data (CPU %, interface counters), use SNMP/streaming telemetry, not syslog.

Lab to try tonight

1. Set up any syslog server (free options: Kiwi Syslog, Splunk Free, rsyslog on Linux).
2. On a Cisco router, configure:

ntp server pool.ntp.org
service timestamps log datetime msec localtime show-timezone
logging buffered 16384 informational
logging host <syslog-server-ip>
logging trap informational
logging source-interface Loopback0

3. Trigger some events: shut/no shut an interface. Make a config change. Login from SSH.
4. Watch entries appear on the syslog server in real time.
5. Try terminal monitor from an SSH session and trigger events — watch them appear in your terminal too.
6. Test the disconnect: turn off the syslog server, generate logs, turn server back on. Logs in the local buffer survived; the ones sent to the server during the outage didn’t.
7. Bonus: pipe logs into Grafana Loki + Promtail. Query / chart events over time.

Cheat strip