Faceți căutări pe acest blog

marți, 24 martie 2015

My experience with Btrfs on Debian - no space on device

I wish to share with you some of my experiances with BTRFS. Maybe it will be usefull.

I'm using btrfs on my /home partition, on my Debian laptop for a few months now. All seemed to be pretty good. The only issues that I had was the fact that filesystem access seemed to be a bit slow on some occasions, but generally things where ok.

This went on until today, when I noticed a complete slowdown of the system, to a point that it became unusable. Some symptoms: Google Chrome crashed and became unusable, removing files was very slow (rm -rf node_modules, nautilus deletion, etc. ) Intellij WebStorm was barelly starting, node.js was complaining about 'not enough space on device' .

I also checked my root partition to validate if the prbolem was btrfs related or not. My root partition, which is ext4 and I created/removed some files. Operations performed normally.

My system was thus unusable and I didn't had any idea why.

So I started investigating...

I checked btrfs file system size, but things where ok:

btrfs fi df /home showed about 10gb of free space

sudo btrfs fi df /home/
Data, single: total=229.80GiB, used=218.80GiB
System, single: total=32.00MiB, used=20.00KiB
Metadata, single: total=3.00GiB, used=2.49GiB
GlobalReserve, single: total=512.00MiB, used=0.00B

My fstab looked like this (I had cow active):

/dev/sda3 /home           btrfs    autodefrag,compress=lzo,space_cache        0       0

Not knowing what to do, I started changing /etc/fstab and rebooting my laptop. First I activated noatime, then I deactivated cow (with nodataow) but I did not notice any improvement.

The solution:

I was very close to re-install Debian on my machine but decided to give it one more chance. I didn't like that thought, since it would have ment a lot of work. I seated myself at the keyboard and started a btrfs defrag

"btrfs filesystem defrag /home"

While I was at it, I thought it will be nice to have some more free-space so I started to delete a bunch of big files. That took a few minutes and I got some extra space and Evrika !!!  my system was usable once more !

Data, single: total=229.80GiB, used=199.81GiB
System, single: total=32.00MiB, used=20.00KiB
Metadata, single: total=3.00GiB, used=2.49GiB
GlobalReserve, single: total=512.00MiB, used=0.00B

My current /etc/fstab :

/dev/sda3 /home           btrfs    nodatacow,autodefrag,noatime,compress=lzo,space_cache        0       0

What caused this?

I'm not very sure. It's kind of obvious that I've hit the btrfs low disk space problem. The strange thing is that I've been having ~ 10 Gb of free space for a long time (more than a 1-2 months) and I did not notice that behaviour. After more thought of what I did different the laste few days is that I started using some node.js build tools that generate a lot of writes on change ( broccoli js build tool, ember-cli ). I will continue to investigate this, but right now I'm happy to have a working system and will also migrate my  root partition to btrfs once Debian Jessie is released.

On the down side, it's a pitty that with btrfs you can do so much, but you can't use all your disk space, and you can make your system very unstable very easy.

$ sudo btrfs fi show
Label: none  uuid: 49d9c741-b00b-487b-be5c-49da02325a16
Total devices 1 FS bytes used 202.30GiB
devid    1 size 232.83GiB used 232.83GiB path /dev/sda3

Btrfs v3.17

$ uname -a
Linux daos 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt7-1 (2015-03-01) x86_64 GNU/Linux

Update:
More 'scientific' artiles related to the subject:
http://unix.stackexchange.com/questions/174446/btrfs-error-error-during-balancing-no-space-left-on-device
http://marc.merlins.org/perso/btrfs/post_2014-05-04_Fixing-Btrfs-Filesystem-Full-Problems.html

2 comentarii:

  1. Quick heads up: nodatacow disables compress=lzo

    RăspundețiȘtergere
  2. I had a similar issue with ext4 a while ago: with approximately a couple free of GBs out of 30 in my / partition, I was no longer able to boot. I cleaned some caches with a live disk and everything went back on tracks.

    RăspundețiȘtergere