Monitoring tools can collect thousands of numbers. Most of them will never help you. The real skill is choosing a small set of signals that tell you whether users are affected, whether a resource is running out and whether something is about to fail. This guide covers what is worth watching on a typical Linux server, what each signal means and how to turn the numbers into alerts you will not learn to ignore.

Start From Symptoms, Not Dashboards

Good monitoring answers two questions: “Is it working for users right now?” and “Will it stop working soon?” The first is about service health and response, the second about resource trends. Begin with the user-facing checks, such as whether the web page responds, whether login works and whether a key query returns, and only then add machine-level metrics to explain why something is wrong.

A common failing is alerting on every resource metric. A server using most of its memory for cache may be perfectly healthy, while a server with idle CPU may still be broken because a process is hung. Metrics are clues; service checks are the verdict.

The Core Resource Signals

CPU. Look at utilization broken down by type: user, system, input/output wait and steal time on virtual machines. High user time suggests application work. High system time may indicate heavy kernel activity such as networking. High I/O wait suggests processes are blocked on storage. Steal time on a virtual server hints that the host is oversubscribed.

Load average. It counts processes that want to run or are waiting on uninterruptible work such as disk I/O. Interpret it relative to the number of CPU cores, and look at the trend instead of a single value.

Memory. Distinguish used memory from cache and available memory, since Linux deliberately uses spare memory for file caching. The numbers that matter are available memory and swap activity. Heavy ongoing swapping usually means real pressure. Also watch for out-of-memory killer events in the kernel log.

Disk space and inodes. Full filesystems cause dramatic failures: databases refuse writes, logs fail, services crash. Monitor both percentage used and inode usage, and watch the rate of growth so you can predict when space will run out.

Disk performance. Track read and write latency, queue depth and utilization where your tooling provides them. Rising latency often appears before users complain.

Network. Watch throughput, errors, dropped packets and connection counts. A sudden rise in established or half-open connections can signal either success or an attack.

Service-Level Signals

Machine metrics describe capacity. Service metrics describe behavior. Depending on your stack, consider:

• Request rate, error rate and response time for web services.
• Queue lengths and processing delays for background workers.
• Database connection counts, slow query counts and replication lag.
• Certificate expiry dates and DNS resolution.
• Process presence: is the expected daemon actually running?
• Backup job completion and freshness.

These often reveal problems that raw CPU or memory never would.

A Starter Set You Can Adopt

Area	Signal	Alert style
Availability	External check of the main service	Page when failing repeatedly
CPU	Utilization and I/O wait over time	Warn on sustained high values
Memory	Available memory, swap activity, OOM events	Warn on sustained pressure; page on kills
Disk space	Percent used, inodes, growth rate	Warn early, page when close to full
Disk performance	Latency and utilization	Warn on sustained degradation
Network	Errors, drops, connection counts	Warn on unusual change
Application	Error rate and response time	Page on user-visible impact
Housekeeping	Certificate expiry, backup freshness	Ticket with generous lead time

Choose thresholds from your own baseline rather than copying numbers from articles. What counts as high depends on your hardware and workload.

Alerts People Will Respect

Alert fatigue is the enemy of monitoring. Each alert should be actionable: if no one needs to do anything, it should not wake anyone. Separate the severities. Pages are for conditions that need a human now. Tickets or daily summaries are for things that need attention soon. Use sustained conditions (high for several minutes) rather than single spikes. Include in the alert text what is wrong, where, and a link or note to the runbook. Review noisy alerts regularly and either fix the cause or remove them.

A Hypothetical Example

A team runs a web application on one server and gets a flood of CPU alerts at the same time every night. The alerts are ignored. Later, a genuine disk-space problem appears in the same channel and is missed because everyone has learned to skim it. After a review, they find the nightly CPU spike is a scheduled backup and compression job. They remove the CPU alert for that window, add a disk-growth alert with a generous lead time, and add an external check for the website. The channel becomes quiet, so the next real alert gets noticed.

Keep History for Capacity Planning

Store metrics long enough to compare this month with last month. Trends turn into capacity plans: disk growing steadily gives you a date to act, memory use creeping upward after each release suggests a leak, and weekly peaks help justify upgrades. Retention and resolution settings trade detail for storage, so keep fine-grained data for recent weeks and coarser summaries for longer.

Frequently Asked Questions

Which monitoring tool should I use? Many open-source and hosted options work well. Choose one your team can operate and that supports your alerting workflow. The signals discussed here apply regardless of tool.

Is high load average always a problem? No. Compare it to core count and check what the processes are doing. A high load from many short CPU-bound tasks differs from one caused by blocked disk I/O.

Should I alert on high memory usage? Usually not on raw used memory, because the kernel uses spare RAM for caching. Alert on low available memory, heavy swapping or out-of-memory kills.

How many alerts are too many? If people stop reading them, you have too many. Aim for alerts that are rare enough to be taken seriously and each lead to a clear action.

Do I need monitoring for a single small server? Yes, even a simple external uptime check plus disk and backup alerts provides large value for little effort.

Next Step

Write down the three things that would hurt most if they failed silently on your server, then make sure each one has a check and an alert. Add further metrics only when they help answer a question you have actually had.