Différences entre les versions de « Btrfs »

De Wiki Seb35
Aller à la navigation Aller à la recherche
m (ref)
(12 versions intermédiaires par le même utilisateur non affichées)
Ligne 1 : Ligne 1 :
I tried to have a root filesystem into btrfs for a [https://www.gandi.net Gandi] server, unsuccessfully for now. Here are my steps if it can be useful to others.
I tried to have a root filesystem into btrfs for a [https://www.gandi.net Gandi] server, and succeed after a number of trial-error. Here are my steps if it can be useful to others.


WARNING: this is EXPERIMENTAL and could cause DATA LOSS.
WARNING: this is EXPERIMENTAL and could cause DATA LOSS.
This was updated on 2020-09-24 for Debian 10 buster + GandiV5 and works probably also for Debian 11 bullseye. An [{{fullurl:Btrfs|oldid=658077}} older version] was for Debian 9 jessie or Ubuntu 18.04 LTS + GandiV4.


== Create the server ==
== Create the server ==


On https://v4.gandi.net, create a new server [https://v4.gandi.net/admin/iaas/vm/create] with e.g. 1 proc, 256 Mio RAM, 1 system disk, Ubuntu 18.04 LTS.
On [https://admin.gandi.net GandiV5], create a new server with e.g. 1 proc, 256 Mio RAM, 1 system disk, Debian 10 buster.


Update the packages:
Update the packages:
  apt update && apt upgrade -y
  apt update && apt upgrade -y


On https://v4.gandi.net, stop the server, open the page about the system disk, clone the system disk with another name, and attach this cloned disk to the server, start the server.
Define backports for Debian 10 (btrfs-progs must contain btrfs-convert, see [https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=870286 this Debian bug]) (this is not needed for Debian 11):
echo 'deb https://deb.debian.org/debian buster-backports main' > /etc/apt/sources.list.d/debian-backports.list
 
Install btrfs-progs:
apt update && apt install -t buster-backports btrfs-progs
 
On GandiV5, stop the server, open the page about the system disk, clone the system disk with another (definitive) name, and attach this cloned disk to the server, start the server.


== Convert to btrfs ==
== Convert to btrfs ==


:''Mostly from [https://btrfs.wiki.kernel.org/index.php/Conversion_from_Ext3]''
:''Mostly from [https://btrfs.wiki.kernel.org/index.php/Conversion_from_Ext3]''
The package btrfs-progs currently does not contain btrfs-convert ([https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=870286 this Debian bug]) because the risk of data loss is too great. Then we need to compile btrfs-progs.
Install dependencies:
apt install gcc g++ libblkid-dev liblzo2-dev zlib1g-dev libzstd-dev asciidoc xmlto libattr1-dev libext2fs-dev pkg-config python3-dev python3-setuptools
Download and compile btrfs-progs ([https://btrfs.wiki.kernel.org/index.php/Changelog changelog], [https://www.kernel.org/pub/linux/kernel/people/kdave/btrfs-progs/ releases]):
wget https://mirrors.edge.kernel.org/pub/linux/kernel/people/kdave/btrfs-progs/btrfs-progs-v4.20.1.tar.gz
tar xfz btrfs-progs-v4.20.1.tar.gz
cd btrfs-progs-v4.20.1
./configure --with-convert=ext2
make
make install
cd ..
rm -rf btrfs-progs-v4.20.1 btrfs-progs-v4.20.1.tar.gz


Display the active root filesystem, and you will be able to deduce what is the cloned filesystem:
Display the active root filesystem, and you will be able to deduce what is the cloned filesystem:
  ls /dev/xvd*
  ls /dev/xvd*
  mount|grep xvd
  mount|grep xvd
  blkid -o value -s TYPE /dev/xvda1 # and the same for others
  blkid


Let’s say /dev/xvdb1 is the clone filesystem, we convert it to btrfs:
Let’s say /dev/xvdb1 is the clone filesystem, we convert it to btrfs: ([https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs-convert man 8 btrfs-convert])
  fsck.ext4 -f /dev/xvdb1
  fsck.ext4 -f /dev/xvdb1 # optional
  btrfs-convert -p /dev/xvdb1
  btrfs-convert -p /dev/xvdb1


Here, I obtained the following error:
Here, I obtained the following error:
  root@test:~# btrfs-convert -p /dev/xvda1
  root@test:~# btrfs-convert -p /dev/xvdb1
  create btrfs filesystem:
  create btrfs filesystem:
  blocksize: 4096
  blocksize: 4096
Ligne 65 : Ligne 58 :
  Aborted
  Aborted


According to [https://github.com/kdave/btrfs-progs/issues/123] and [https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg82326.html], it can be worked around:
According to [https://github.com/kdave/btrfs-progs/issues/123] and [https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg82326.html], it can be worked around: (this command could take some hours, depending on disk size and number of files)
  btrfs-convert -d -p /dev/xvdb1
  btrfs-convert -d -p /dev/xvdb1
or even:
btrfs-convert -n -d -p /dev/xvdb1
Or, if still unsuccessful, try to add free space, or if still unsuccessfull, [https://github.com/kdave/btrfs-progs/issues/123#issuecomment-455877070 re-compile btrfs-progs in version 4.17.1] (this last try worked for me for 50 Gio disk with 24 Gio free space on Debian 9 jessie, expanded after its original size was 30 Gio (=4 Gio free space)).


Then, mount the filesystem and delete the old ext4 snapshot:
Then, mount the filesystem and delete the old ext4 snapshot:
  mount /dev/xvdb1 /mnt
  mount /dev/xvdb1 /mnt
  btrfs subvolume delete /mnt/ext2_saved
  btrfs subvolume delete /mnt/ext2_saved
btrfs filesystem defrag -r /mnt # could take dozen of minuts
btrfs balance start /mnt # could take hours


== Promote as root filesystem ==
== Promote as root filesystem ==


Not successful after a first try with [https://help.ubuntu.com/community/btrfs#Install_as_Root_on_earlier_Ubuntu_versions] and I abandonned here, see also [https://wiki.gentoo.org/wiki/Btrfs/System_Root_Guide].
:''Mostly from [https://help.ubuntu.com/community/btrfs#Install_as_Root_on_earlier_Ubuntu_versions]''
 
Enter the chrooted system:
for i in dev dev/pts proc sys; do mount --bind /$i /mnt/$i; done
chroot /mnt
blkid|grep xvdb1
vi /etc/fstab
 
Edit the root filesystem with: (the UUID is from the command blkid)
UUID=a74f5787-aee1-4981-b7e6-fbd3cb6ac919 /              btrfs    defaults 0      1
 
There are 3 things to change on this line: UUID, ext4 → btrfs, and the options now to "defaults".
 
(in vi, type "dd" to remove a line, "i" to enter in edit mode, Esc to quit edit mode, ":x" to save and quit.)
 
Update grub:
update-grub
 
Change the reference of the root device (will be xvda1 when it will be the root device):
sed -i s/xvdb1/xvda1/g /boot/grub/grub.cfg
 
Quit:
exit
for i in dev/pts dev proc sys; do umount /mnt/$i; done
umount /mnt
exit
 
== Define the cooked disk as boot disk ==
 
In Gandi V5 interface, stop the server, detach the old root disk and define the cooked disk as "Use to start".
 
Launch the server (it should correctly start, even if the /boot directory is on the main partition / (some old documents said it didn’t work because grub didn’t know the btrfs filesystem, but it is fixed now)).
 
df -hT
 
It should show something like:
/dev/xvda1    btrfs      50G  32G  19G  63% /
 
Delete the old root disk.
 
You can delete the packages grub-efi-amd64 and grub-efi-amd64-bin since they are not used, but keep /etc/default/grub with Gandi customisations:
cp -a /etc/default/grub /root/grub
apt-get purge -y grub-efi-amd64 grub-efi-amd64-bin
mv /root/grub /etc/default/grub
update-grub # to check it still work and you should reboot now to be sure it reboots correctly
 
== Troubleshootings ==
 
* During a conversion, the root filesystem was mounted read-only in btrfs; it was probably because I didn’t change the options "rw,noatime,errors=remount-ro" to "defaults"
 
== External links ==
 
* [https://wiki.xen.org/wiki/PvGrub2 PvGrub2 on Xen wiki]

Version du 9 juin 2021 à 14:34

I tried to have a root filesystem into btrfs for a Gandi server, and succeed after a number of trial-error. Here are my steps if it can be useful to others.

WARNING: this is EXPERIMENTAL and could cause DATA LOSS.

This was updated on 2020-09-24 for Debian 10 buster + GandiV5 and works probably also for Debian 11 bullseye. An older version was for Debian 9 jessie or Ubuntu 18.04 LTS + GandiV4.

Create the server

On GandiV5, create a new server with e.g. 1 proc, 256 Mio RAM, 1 system disk, Debian 10 buster.

Update the packages:

apt update && apt upgrade -y

Define backports for Debian 10 (btrfs-progs must contain btrfs-convert, see this Debian bug) (this is not needed for Debian 11):

echo 'deb https://deb.debian.org/debian buster-backports main' > /etc/apt/sources.list.d/debian-backports.list

Install btrfs-progs:

apt update && apt install -t buster-backports btrfs-progs

On GandiV5, stop the server, open the page about the system disk, clone the system disk with another (definitive) name, and attach this cloned disk to the server, start the server.

Convert to btrfs

Mostly from [1]

Display the active root filesystem, and you will be able to deduce what is the cloned filesystem:

ls /dev/xvd*
mount|grep xvd
blkid

Let’s say /dev/xvdb1 is the clone filesystem, we convert it to btrfs: (man 8 btrfs-convert)

fsck.ext4 -f /dev/xvdb1 # optional
btrfs-convert -p /dev/xvdb1

Here, I obtained the following error:

root@test:~# btrfs-convert -p /dev/xvdb1
create btrfs filesystem:
	blocksize: 4096
	nodesize:  16384
	features:  extref, skinny-metadata (default)
creating ext2 image file
creating btrfs metadata
Unable to find block group for 0 27081]
Unable to find block group for 0
Unable to find block group for 0
ctree.c:2245: split_leaf: BUG_ON `1` triggered, value 1
btrfs-convert(+0x11b5a)[0x559c159c1b5a]
btrfs-convert(+0x1589b)[0x559c159c589b]
btrfs-convert(btrfs_search_slot+0x269)[0x559c159c6401]
btrfs-convert(btrfs_insert_empty_items+0x92)[0x559c159c7b3c]
btrfs-convert(btrfs_record_file_extent+0x1bc)[0x559c159d46b4]
btrfs-convert(record_file_blocks+0x14a)[0x559c159bfe92]
btrfs-convert(+0x10349)[0x559c159c0349]
btrfs-convert(+0x1135f)[0x559c159c135f]
btrfs-convert(main+0x1f59)[0x559c159bdefb]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0x7f4c11c3bb97]
btrfs-convert(_start+0x2a)[0x559c159bb5ca]
Aborted

According to [2] and [3], it can be worked around: (this command could take some hours, depending on disk size and number of files)

btrfs-convert -d -p /dev/xvdb1

or even:

btrfs-convert -n -d -p /dev/xvdb1

Or, if still unsuccessful, try to add free space, or if still unsuccessfull, re-compile btrfs-progs in version 4.17.1 (this last try worked for me for 50 Gio disk with 24 Gio free space on Debian 9 jessie, expanded after its original size was 30 Gio (=4 Gio free space)).

Then, mount the filesystem and delete the old ext4 snapshot:

mount /dev/xvdb1 /mnt
btrfs subvolume delete /mnt/ext2_saved
btrfs filesystem defrag -r /mnt # could take dozen of minuts
btrfs balance start /mnt # could take hours

Promote as root filesystem

Mostly from [4]

Enter the chrooted system:

for i in dev dev/pts proc sys; do mount --bind /$i /mnt/$i; done
chroot /mnt
blkid|grep xvdb1
vi /etc/fstab

Edit the root filesystem with: (the UUID is from the command blkid)

UUID=a74f5787-aee1-4981-b7e6-fbd3cb6ac919 /               btrfs    defaults 0       1

There are 3 things to change on this line: UUID, ext4 → btrfs, and the options now to "defaults".

(in vi, type "dd" to remove a line, "i" to enter in edit mode, Esc to quit edit mode, ":x" to save and quit.)

Update grub:

update-grub

Change the reference of the root device (will be xvda1 when it will be the root device):

sed -i s/xvdb1/xvda1/g /boot/grub/grub.cfg

Quit:

exit
for i in dev/pts dev proc sys; do umount /mnt/$i; done
umount /mnt
exit

Define the cooked disk as boot disk

In Gandi V5 interface, stop the server, detach the old root disk and define the cooked disk as "Use to start".

Launch the server (it should correctly start, even if the /boot directory is on the main partition / (some old documents said it didn’t work because grub didn’t know the btrfs filesystem, but it is fixed now)).

df -hT

It should show something like:

/dev/xvda1     btrfs      50G   32G   19G  63% /

Delete the old root disk.

You can delete the packages grub-efi-amd64 and grub-efi-amd64-bin since they are not used, but keep /etc/default/grub with Gandi customisations:

cp -a /etc/default/grub /root/grub
apt-get purge -y grub-efi-amd64 grub-efi-amd64-bin
mv /root/grub /etc/default/grub
update-grub # to check it still work and you should reboot now to be sure it reboots correctly

Troubleshootings

  • During a conversion, the root filesystem was mounted read-only in btrfs; it was probably because I didn’t change the options "rw,noatime,errors=remount-ro" to "defaults"

External links