排查磁盘空间不足

背景

在开发机上调试，开debug级别日志过多，磁盘空间消耗极大
报错：

Less than 1 MiB is left on the system directory partition ('/dev/mapper/ubuntu--vg-ubuntu--lv')

解决办法(BY GPT-5)

Here’s a safe, step‑by‑step plan to recover space on your Ubuntu root filesystem (/ on LVM: /dev/mapper/ubuntu--vg-ubuntu--lv) and prevent it from happening again. Start with quick frees, then move to deeper cleanup or resizing.

Important: With <1 MiB free, commands can fail. If any command errors due to no space left, reboot into recovery mode (root shell) or a live USB to operate with minimal services.

1. See what’s using space

• Show filesystem usage:
  • df -h /
• Show large directories (top level):
  • sudo du -xhd1 / | sort -h
• Drill down into the biggest directory from above:
  • sudo du -xhd1 /var | sort -h
  • sudo du -xhd1 /home | sort -h
• Find giant files:
  • sudo find / -xdev -type f -size +500M -printf "%p %k KB\n" 2>/dev/null | sort -nr -k2

2. Quick, low‑risk cleanup

• Apt cache:
  • sudo apt-get clean
  • sudo apt-get autoremove --purge -y
• Old kernels (on Ubuntu with LVM, kernels live under /boot and root; removing unused kernels helps):
  • Check current kernel: uname -r
  • List installed kernels:
    • dpkg -l | grep -E "linux-image|linux-headers" | awk '/^ii/{print $2}' | sort
  • Remove older kernels (replace X.Y.Z-W):
    • sudo apt-get purge linux-image-X.Y.Z-W-generic linux-headers-X.Y.Z-W-generic
  • Or let Ubuntu handle it:
    • sudo ubuntu-drivers list 2>/dev/null >/dev/null || true
    • sudo apt-get autoremove --purge -y
• Journald logs (can be huge):
  • Check usage:
    • sudo journalctl --disk-usage
  • Vacuum to 200M or 2 weeks (pick one):
    • sudo journalctl --vacuum-size=200M
    • sudo journalctl --vacuum-time=2weeks
• Systemd journals persistent directory may be large:
  • sudo du -sh /var/log/journal 2>/dev/null
• Log rotation (force rotate then clean old .gz):
  • sudo logrotate -f /etc/logrotate.conf
  • sudo find /var/log -type f -name "*.gz" -delete
  • sudo find /var/log -type f -name "*.1" -delete
• Snap cleanup (often large):
  • List snaps and revisions:
    • snap list --all
  • Remove disabled (old) revisions:
    • sudo snap set system refresh.retain=2
    • for s in $(snap list --all | awk '/disabled/{print $1, $3}'); do sudo snap remove "$s"; done
• Docker/Podman images, containers, logs (if using):
  • Docker:
    • docker system df
    • docker system prune -af --volumes
  • Podman:
    • podman system prune -af
• Flatpak cache (if using):
  • flatpak uninstall --unused -y
  • flatpak repair
• Trash for all users:
  • sudo rm -rf /root/.local/share/Trash/*
  • sudo find /home -maxdepth 2 -type d -name ".local" -exec sh -c 'rm -rf "$0"/share/Trash/* 2>/dev/null' {} ;

3. Clear orphaned temp files

• sudo rm -rf /tmp/* /var/tmp/* 2>/dev/null
• Reboot services may recreate what’s needed.

4. Check for runaway logs or app data

• Very large single logs often indicate an app misbehaving.
  • sudo du -ahx /var/log | sort -h | tail -n 30
  • If a log file is huge but you need the process running, truncate instead of delete:
    • sudo truncate -s 0 /var/log/NAME.log

5. LVM: grow the root logical volume if there is free VG space

• See LVM layout and free space:
  • sudo vgs
  • sudo lvs
  • sudo pvs
• If VG (ubuntu-vg) shows VFree > 0, extend LV and filesystem online:
  • sudo lvextend -l +100%FREE /dev/ubuntu-vg/ubuntu-lv
  • For ext4:
    • sudo resize2fs /dev/ubuntu-vg/ubuntu-lv
  • For XFS:
    • sudo xfs_growfs /
• If no free space in VG, but your physical disk has unpartitioned free space:
  • Identify disk: lsblk -o name,fstype,size,mountpoint
  • If the PV is a partition (e.g., /dev/sda3), grow the partition to fill free disk using:
    • For GPT: sudo growpart /dev/sda 3
    • For MBR: use parted or fdisk carefully
  • Then grow the PV and LV:
    • sudo pvresize /dev/sda3
    • sudo lvextend -l +100%FREE /dev/ubuntu-vg/ubuntu-lv
    • Then resize filesystem (ext4: resize2fs; XFS: xfs_growfs /)
• If disk has no free space, you can add a new disk and extend LVM:
  • Attach new disk (e.g., /dev/sdb)
  • sudo pvcreate /dev/sdb
  • sudo vgextend ubuntu-vg /dev/sdb
  • sudo lvextend -l +100%FREE /dev/ubuntu-vg/ubuntu-lv
  • Resize filesystem as above

6. If /boot is a separate full partition (sometimes the true culprit)

• Check: lsblk -o NAME,SIZE,FSTYPE,MOUNTPOINT | grep boot
• If /boot is separate and full, remove old kernels specifically from /boot:
  • dpkg -l | grep linux-image
  • sudo apt-get purge linux-image-X.Y.Z-W-generic
• Ensure update-initramfs and grub updates run:
  • sudo update-initramfs -u -k all
  • sudo update-grub

7. Prevent future space exhaustion

• Limit journal size persistently:
  • Create or edit /etc/systemd/journald.conf:
    • SystemMaxUse=200M
    • SystemMaxFileSize=50M
  • sudo systemctl restart systemd-journald
• Reduce snap revision retention:
  • sudo snap set system refresh.retain=2
• Regularly clear apt caches:
  • sudo systemctl enable --now apt-daily.timer apt-daily-upgrade.timer
• Monitor with a simple cron check that warns under 15% free:
  • Example:
    • echo '*/30 * * * * root df -h / | awk "NR==2 && $5+0>85{print "Root FS over 85%: "$5}" | logger -t diskwatch' | sudo tee /etc/cron.d/diskwatch >/dev/null

If you want, paste the outputs of:

• df -h /
• sudo vgs; sudo lvs; sudo pvs
• sudo du -xhd1 / | sort -h | tail -n 20
• sudo journalctl --disk-usage
  and I’ll give you exact commands tailored to your system.