Tracking Progress of Long-Running Linux Tasks Without Killing Them

How can you tell if a long-running process is actually making progress or stuck? On Linux, there are several ways to check.

• Detailed process information via /proc/<pid>/status
  The /proc filesystem provides extensive information about every running process. By running cat /proc/<pid>/status, you can see key details such as the number of threads, memory usage, and the process state. The State: field shows whether the process is running (R), sleeping (S), or in uninterruptible sleep (D), giving clues about its current activity. This file also lists voluntary_ctxt_switches, which counts how many times the process voluntarily gave up the CPU (typically while waiting for I/O), and nonvoluntary_ctxt_switches, which counts how many times it was preempted by the scheduler to let another process run. High voluntary context switches combined with low CPU usage usually indicate the process is mostly waiting rather than stuck.

• Output and resource inspection
  Linux allows you to follow what a process is printing to stdout using tail -f /proc/<pid>/fd/1. This works whether stdout is redirected to a file or attached to a terminal. Additionally, running lsof -p <pid> lists all files, sockets, and network connections the process has open, helping you determine whether it is blocked on I/O or waiting for network responses.

• System calls inspection
  For deeper observation, strace -p <pid> shows the system calls a process is executing. Watching repeated calls or blocked operations can reveal whether a process is actively working, waiting for I/O, or potentially stuck in a loop.

• Monitor logs with journalctl
  If the process runs as a systemd service, you can use journalctl -u <service_name> -f to follow its logs in real time. This is particularly useful for background tasks or daemons that don’t produce stdout output. You can see whether the service is actively reporting progress, encountering errors, or retrying operations.