df and du shows different Size and recovering the space without killing the process
1. From lsof we can find the big 10 files which is deleted. And the first file is around ((2407845888 / 1024 ) /1024) = 2296 MB
[root@server1 fd]# lsof | grep deleted | sort -nr -k 7 | head
frmweb 21199 mislve 23u REG 253,0 2407845888 819375 /tmp/filesvuEFz.TMP (deleted)
frmweb 28933 mislve 27u REG 253,0 4032864 819378 /tmp/fileiTj11l.TMP (deleted)
frmweb 16365 mislve 24u REG 253,0 2248832 819483 /tmp/file5z0oen.TMP (deleted)
frmweb 28697 mislve 27u REG 253,0 959488 819238 /tmp/fileemrhVz.TMP (deleted)
frmweb 24794 mislve 27u REG 253,0 906304 819492 /tmp/file4d68i0.TMP (deleted)
frmweb 25383 mislve 24u REG 253,0 393216 819572 /tmp/file7powEH.TMP (deleted)
frmweb 25329 mislve 28u REG 253,0 386784 819355 /tmp/filembs5IG.TMP (deleted)
frmweb 24400 mislve 28u REG 253,0 307328 819228 /tmp/fileL8p1m7.TMP (deleted)
frmweb 2717 mislve 27u REG 253,0 277472 819290 /tmp/fileVzblkn.TMP (deleted)
frmweb 2002 mislve 24u REG 253,0 262784 819516 /tmp/fileUeZpzA.TMP (deleted)
2. One option is to kill the process or nullfy the File Descripter (FD).We can find the FD as shown below.
[root@server1 fd]# ls -l /proc/21199/fd/ | grep deleted
lrwx------ 1 mislve dba 64 Feb 19 13:00 23 -> /tmp/filesvuEFz.TMP (deleted)
3.Nullify the file descripter
[root@server1 fd]# echo /dev/null > /proc/21199/fd/23
4. Size of the Fileystem before and after
Before
[root@server1 ~]# df -h /
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
11G 5.8G 4.3G 58% /
After
[root@server1 ~]# df -h /
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
11G 3.6G 6.5G 36% /
