Apr 29, 2018 | unix.stackexchange.com
Grzegorz Wierzowiecki, Jul 29, 2012 at 10:02How to clear unused space with zeros ? (ext3, ext4)
I'm looking for something smarter thancat /dev/zero > /mnt/X/big_zero ; sync; rm /mnt/X/big_zero
Like FSArchiver is looking for "used space" and ignores unused, but opposite site.
Purpose: I'd like to compress partition images, so filling unused space with zeros is highly recommended.
Btw. For btrfs : Clear unused space with zeros (btrfs)
Mat, Jul 29, 2012 at 10:18Check this out: superuser.com/questions/19326/ – Mat Jul 29 '12 at 10:18
Totor, Jan 5, 2014 at 2:57Two different kind of answer are possible. What are you trying to achieve? Either 1) security, by forbidding someone to read those data, or 2) optimizing compression of the whole partition or [SSD performance]( en.wikipedia.org/wiki/Trim_(computing) ? – Totor Jan 5 '14 at 2:57
enzotib, Jul 29, 2012 at 11:45Such an utility is
From its description:UPDATE #1
Zerofree finds the unallocated, non-zeroed blocks in an ext2 or ext3 file-system and fills them with zeroes. This is useful if the device on which this file-system resides is a disk image. In this case, depending on the type of disk image, a secondary utility may be able to reduce the size of the disk image after zerofree has been run. Zerofree requires the file-system to be unmounted or mounted read-only.
The usual way to achieve the same result (zeroing the unused blocks) is to run "dd" do create a file full of zeroes that takes up the entire free space on the drive, and then delete this file. This has many disadvantages, which zerofree alleviates:
- it is slow
- it makes the disk image (temporarily) grow to its maximal extent
- it (temporarily) uses all free space on the disk, so other concurrent write actions may fail.
Zerofree has been written to be run from GNU/Linux systems installed as guest OSes inside a virtual machine. If this is not your case, you almost certainly don't need this package.
The description of the .deb package contains the following paragraph now which would imply this will work fine with ext4 too.
Description: zero free blocks from ext2, ext3 and ext4 file-systems Zerofree finds the unallocated blocks with non-zero value content in an ext2, ext3 or ext4 file-system and fills them with zeroes...
Grzegorz Wierzowiecki, Jul 29, 2012 at 14:08Is it official page of the tool intgat.tigress.co.uk/rmy/uml/index.html ? Do you think it's safe to use with ext4 ? – Grzegorz Wierzowiecki Jul 29 '12 at 14:08
enzotib, Jul 29, 2012 at 14:12@GrzegorzWierzowiecki: yes, that is the page, but for debian and friends it is already in the repos. I used on a ext4 partition on a virtual disk to successively shrink the disk file image, and had no problem. – enzotib Jul 29 '12 at 14:12
jlh, Mar 4, 2016 at 10:10This isn't equivalent to the crude
ddmethod in the original question, since it doesn't work on mounted file systems. – jlh Mar 4 '16 at 10:10
endolith, Oct 14, 2016 at 16:33zerofree page talks about a patch that lets you do "filesystem is mounted with the zerofree option" so that it always zeros out deleted files continuously. does this require recompiling the kernel then? is there an easier way to accomplish the same thing? – endolith Oct 14 '16 at 16:33
Hubbitus, Nov 23, 2016 at 22:20Be careful - I lost ext4 filesystem using
zerofreeon Astralinux (Debian based) – Hubbitus Nov 23 '16 at 22:20
Anon, Dec 27, 2015 at 17:53Summary of the methods (as mentioned in this question and elsewhere) to clear unused space on ext2/ext3/ext4: Zeroing unused space File system is not mounted
File system is mounted
- If the "disk" your filesystem is on is thin provisioned (e.g. a modern SSD supporting TRIM, a VM file whose format supports sparseness etc.) and your kernel says the block device understands it, you can use
e2fsck -E discard src_fsto discard unused space (requires e2fsprogs 1.42.2 or higher).
- Using zerofree to explicitly write zeros over unused blocks.
e2image -rap src_fs dest_fsto only copy blocks in use (new filesystem should be on an otherwise zero'd "disk", requires e2fsprogs 1.42.9 or higher).
- If the "disk" your filesystem is on is thin provisioned (e.g. a modern SSD supporting TRIM, a VM file whose format supports sparseness etc.), your kernel says the block device understands it and finally the ext filesystem driver supports it you can use
fstrim /mnt/fs/to ask the filesystem to discard unused space.
cat /dev/zero > /mnt/fs/zeros; sync; rm /mnt/fs/zeros(
sfillfrom secure-delete uses this technique). This method is inefficient, not recommended by Ted Ts'o (author of ext4), may not zero certain things and can slow down future fscks.
Having the filesystem unmounted will give better results than having it mounted. Discarding tends to be the fastest method when a lot of previously used space needs to be zeroed but using zerofree after the discard process can sometimes zero a little bit extra (depending on how discard is implemented on the "disk").Making the image file smaller Image is in a dedicated VM format
You will need to use an appropriate disk image tool (such asImage is a raw file
qemu-img convert src_image dst_image) to enable the zeroed space to be reclaimed and to allow the file representing the image to become smaller.
One of the following techniques can be used to make the file sparse (so runs of zero stop taking up space):
cp --sparse=always src_image dst_image.
fallocate -d src_image(requires util-linux v2.25 or higher).
These days it might easier to use a tool like virt-sparsify to do these steps and more in one go.Sources
cas, Jul 29, 2012 at 11:45
sfillfrom secure-delete can do this and several other related jobs.
e.g.sfill -l -l -z /mnt/XUPDATE #1
There is a source tree that appears to be used by the ArchLinux project on github that contains the source for
sfillwhich is a tool included in the package Secure-Delete.
Also a copy of
sfill's man page is here:
cas, Jul 29, 2012 at 12:04that URL is obsolete. no idea where its home page is now (or even if it still has one), but it's packaged for debian and ubuntu. probably other distros too. if you need source code, that can be found in the debian archives if you can't find it anywhere else. – cas Jul 29 '12 at 12:04
mwfearnley, Jul 31, 2017 at 13:04The obsolete manpage URL is fixed now. Looks like "Digipedia" is no longer a thing. – mwfearnley Jul 31 '17 at 13:04
psusi, Apr 2, 2014 at 15:27If you have e2fsprogs 1.42.9, then you can use
e2imageto create the partition image without the free space in the first place, so you can skip the zeroing step.
mwfearnley, Mar 3, 2017 at 13:36I couldn't (easily) find any info online about these parameters, but they are indeed given in the 1.42.9 release notes: e2fsprogs.sf.net/e2fsprogs-release.html#1.42.9 – mwfearnley Mar 3 '17 at 13:36
user64219, Apr 2, 2014 at 14:39You can use
sfill. It's a better solution for thin volumes.
Anthon, Apr 2, 2014 at 15:01If you want to comment on cas answer, wait until you have enough reputation to do so. – Anthon Apr 2 '14 at 15:01
derobert, Apr 2, 2014 at 17:01I think the answer is referring to manpages.ubuntu.com/manpages/lucid/man1/sfill.1.html ... which is at least an attempt at answering. ("online" in this case meaning "with the filesystem mounted", not "on the web"). – derobert Apr 2 '14 at 17:01