Real-Time Log Analysis and Transfer with Lnav and Rsync

Lnav and rsync are two command-line tools that thrive under pressure. Lnav gives you real-time, interactive log viewing. Rsync syncs files across systems with precision. Used together, they create a smooth workflow for monitoring and transferring logs without breaking stride.

Lnav reads compressed or plain text log files. It indexes them, highlights patterns, and lets you filter by regex or time range. Unlike less or tail -f, it parses timestamps and formats multiple logs into a single, ordered timeline. This makes it faster to isolate errors after a deploy or during an incident.

Rsync handles file transfer with delta encoding. It moves only the blocks that changed. When running over SSH, it secures data in motion. You can sync logs from production to a local machine without saturating bandwidth or interrupting ongoing writes.

A simple pattern emerges:

rsync -az --progress user@prod:/var/log/*.log /tmp/logs/
lnav /tmp/logs/

First, rsync pulls the latest logs down. Then lnav opens them in one indexed view. If you add the --append flag to rsync or run it on a schedule, you get near-live log aggregation on your local terminal.

This pair is also powerful in pre-production. You can sync staging logs to your laptop, load them in lnav, and verify request flows before sign-off. With filters and SQL-like queries in lnav, you can turn raw log lines into actionable metrics without importing them into a database.

To optimize performance, use rsync’s --bwlimit during large transfers to avoid network contention. For very active logs, consider rsync with --inplace to minimize rewrites. Lnav can read partial files without issue, making the two tools naturally compatible.

When incidents hit, having a repeatable lnav–rsync command chain saves minutes that can prevent downtime. Script it. Test it. Keep it ready.

See this workflow in action and move from zero to live in minutes at hoop.dev.