Différences entre les versions de « Btrfs »

De Wiki Seb35
Aller à la navigation Aller à la recherche
(update for Debian 11 Bullseye)
 
(15 versions intermédiaires par le même utilisateur non affichées)
Ligne 1 : Ligne 1 :
I tried to have a root filesystem into btrfs for a [https://www.gandi.net Gandi] server, and succeed after a number of trial-error. Here are my steps if it can be useful to others.
I tried to have a root filesystem into btrfs for a [https://www.gandi.net Gandi] server, and succeed after a number of trial-error. Here are my steps if it can be useful to others. For an introduction, see [https://en.wikipedia.org/wiki/Btrfs Wikipedia article] and/or [https://btrfs.wiki.kernel.org the dedicated wiki].


WARNING: this is EXPERIMENTAL and could cause DATA LOSS.
WARNING: this could cause DATA LOSS, be sure you have backups (possibly Gandi snapshots).


This was updated on 2020-09-24 for Debian 10 buster + GandiV5 and on 2022-02-02 for Debian 11 bullseye. An [{{fullurl:Btrfs|oldid=658077}} older version] was for Debian 9 jessie or Ubuntu 18.04 LTS + GandiV4.
This was updated on 2020-09-24 for Debian 10 buster + GandiV5, and on 2022-02-02 for Debian 11 bullseye + GandiV5. An [{{fullurl:Btrfs|oldid=658077}} older version] was for Debian 9 jessie or Ubuntu 18.04 LTS + GandiV4.


== Create the server ==
== Manual procedure ==


On [https://admin.gandi.net GandiV5], create a new server with e.g. 1 proc, 256 Mio RAM, 1 system disk, Debian 10 buster.
=== Create the server ===


Update the packages:
On [https://admin.gandi.net GandiV5], create a new server with e.g. 1 proc, 256 MiB RAM, 1 system disk of 5 GiB, Debian 10 buster or Debian 11 bullseye.
apt update && apt upgrade -y


For Debian 10 Buster (only), define backports (btrfs-progs must contain btrfs-convert, see [https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=870286 this Debian bug]) (this is not needed for Debian 11):
For Debian 10 buster (only), define backports (btrfs-progs must contain btrfs-convert, see [https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=870286 this Debian bug]) (this is not needed for Debian 11):
  echo 'deb https://deb.debian.org/debian buster-backports main' > /etc/apt/sources.list.d/debian-backports.list
  echo 'deb https://deb.debian.org/debian buster-backports main' > /etc/apt/sources.list.d/debian-backports.list


Install btrfs-progs:
Install btrfs-progs:
  apt update && apt install -t buster-backports btrfs-progs # Debian 10 Buster
  apt update && apt install -t buster-backports btrfs-progs # Debian 10 buster
  apt update && apt install btrfs-progs # Debian 11 Bullseye
  apt update && apt install btrfs-progs # Debian 11 bullseye


On GandiV5, stop the server, open the page about the system disk, clone the system disk with another (definitive) name, and attach this cloned disk to the server, start the server.
On GandiV5 :
# stop the server,
# open the page about the system disk in Volumes,
# clone the system disk with another (definitive) name (e.g. the current disk is "sys-myserver", you can define the copied disk as "myserver"),
# start the server,
# wait until the server is started,
# attach this cloned disk to the server.


== Convert to btrfs ==
ATTENTION: really start the server before attaching the cloned disk, else the root filesystem might be the cloned disk given they have the same UUID and /etc/fstab is using the UUID to select the root filesystem (it could be workarounded by setting /dev/xvda1 in /etc/fstab before stopin the server).
 
=== Convert to btrfs ===


:''Mostly from [https://btrfs.wiki.kernel.org/index.php/Conversion_from_Ext3]''
:''Mostly from [https://btrfs.wiki.kernel.org/index.php/Conversion_from_Ext3]''


Display the active root filesystem, and you will be able to deduce what is the cloned filesystem: (beware: given the two filesystems have the same UUID, the mounted root filesystem could be /dev/xvdb1 – it happened once)
Display the active root filesystem, and you will be able to deduce what is the cloned filesystem:
  ls /dev/xvd*
  ls /dev/xvd*
  mount|grep xvd
  mount|grep xvd
  blkid
  blkid
Copy the result of <code>blkid</code> in some text file on your computer (will be used later).


Let’s say /dev/xvdb1 is the clone filesystem, we convert it to btrfs: ([https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs-convert man 8 btrfs-convert])
Let’s say /dev/xvdb1 is the clone filesystem, we convert it to btrfs: ([https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs-convert man 8 btrfs-convert])
Ligne 72 : Ligne 81 :
  btrfs balance start /mnt # could take hours
  btrfs balance start /mnt # could take hours


== Promote as root filesystem ==
=== Promote as root filesystem ===


:''Mostly from [https://help.ubuntu.com/community/btrfs#Install_as_Root_on_earlier_Ubuntu_versions]''
:''Mostly from [https://help.ubuntu.com/community/btrfs#Install_as_Root_on_earlier_Ubuntu_versions]''


Enter the chrooted system:
Edit the /etc/fstab of the btrfs system:
uuid=`blkid|grep -P '^/dev/xvdb1: '|grep -o -P ' UUID="[0-9a-f-]+" '|cut -d'"' -f2`
sed -i -E 's/^UUID=[0-9a-f-]+ *\/ .*$/UUID=$uid \/              btrfs    defaults 0      1/' /mnt/etc/fstab
 
Update Grub in a chrooted system:
  for i in dev dev/pts proc sys; do mount --bind /$i /mnt/$i; done
  for i in dev dev/pts proc sys; do mount --bind /$i /mnt/$i; done
  chroot /mnt
  chroot /mnt update-grub
blkid|grep xvdb1
  for i in dev/pts dev proc sys; do umount /mnt/$i; done
vi /etc/fstab
umount /mnt
 
Edit the root filesystem with: (the UUID is from the command blkid)
UUID=a74f5787-aee1-4981-b7e6-fbd3cb6ac919 /              btrfs    defaults 0      1
 
There are 3 things to change on this line: UUID, ext4 → btrfs, and the options now to "defaults".
 
(in vi, type "dd" to remove a line, "i" to enter in edit mode, Esc to quit edit mode, ":x" to save and quit.)
 
Update grub:
update-grub
 
Change the reference of the root device (will be xvda1 when it will be the root device):
  sed -i s/xvdb1/xvda1/g /boot/grub/grub.cfg


Quit:
Quit:
exit
for i in dev/pts dev proc sys; do umount /mnt/$i; done
umount /mnt
  exit
  exit


== Define the cooked disk as boot disk ==
=== Define the cooked disk as boot disk ===


In Gandi V5 interface, stop the server, detach the old root disk and define the cooked disk as "Use to start".
In Gandi V5 interface, stop the server, detach the old root disk and define the cooked disk as "Use to start".
Ligne 120 : Ligne 117 :
  apt-mark manual grub-common grub2-common
  apt-mark manual grub-common grub2-common
  update-grub # to check it still work and you should reboot now to be sure it reboots correctly
  update-grub # to check it still work and you should reboot now to be sure it reboots correctly
== Scripted procedure ==
This script automates the conversion, with a slightly-modified procedure.
On [https://admin.gandi.net GandiV5], create a new server with e.g. 1 proc, 256 MiB RAM, 1 system disk of 5 GiB, Debian 11 bullseye.
On GandiV5 :
# stop the server,
# open the page about the system disk in Volumes,
# clone the system disk with another (definitive) name (e.g. the current disk is "sys-myserver", you can define the copied disk as "myserver"),
# start the server,
# wait until the server is started,
# attach this cloned disk to the server.
ATTENTION: really start the server before attaching the cloned disk, else the root filesystem might be the cloned disk given they have the same UUID and /etc/fstab is using the UUID to select the root filesystem (it could be workarounded by setting /dev/xvda1 in /etc/fstab before stopin the server).
# Launch this script as root: (<code>vi convert_gandiroot_to_btrfs.sh</code> + <code>chmod +x convert_gandiroot_to_btrfs.sh</code> + (<code>./convert_gandiroot_to_btrfs.sh</code>)
#: The script takes about 3 minutes to convert a 1.4 GiB disk.
<syntaxhighlight lang="shell">
#!/bin/sh
set -e
fs_type=`mount|grep -F ' on / '|grep -o -E ' type [a-z0-9]+ '|cut -d' ' -f3`
blkid=`blkid|grep -F ' LABEL="gandiroot" '`
nb_fs_gandiroot=`echo "$blkid"|cut -d: -f2-|uniq -c|tr -s ' '|cut -d' ' -f2`
if [ "$fs_type" != 'ext4' -o ! "$nb_fs_gandiroot" = 2 ]; then
    echo 'It is not possible to convert the disk. There should be two identical devices labeled "gandiroot" formatted in ext4'
    exit 2
fi
fs_gandiroot_ext4=`echo "$blkid"|head -n1|cut -d: -f1`
fs_gandiroot_btrfs=`echo "$blkid"|tail -n1|cut -d: -f1`
echo
echo "Current root filesystem: $fs_gandiroot_ext4 (ext4)"
echo "Future root filesystem: $fs_gandiroot_btrfs (currently ext4, will be btrfs)"
set -v
apt-get update
apt-get install -y btrfs-progs
fsck.ext4 -f "$fs_gandiroot_btrfs"
btrfs-convert -p "$fs_gandiroot_btrfs"
mount "$fs_gandiroot_btrfs" /mnt
btrfs subvolume delete -C /mnt/ext2_saved
btrfs filesystem defragment -r /mnt
btrfs balance start /mnt
new_uuid=`blkid|grep -P "^$fs_gandiroot_btrfs: "|grep -o -P ' UUID="[0-9a-f-]+" '|cut -d'"' -f2`
sed -i -E "s/^UUID=[0-9a-f-]+ *\/ .*$/UUID=$new_uuid \/              btrfs    defaults 0      1/" /mnt/etc/fstab
for i in dev dev/pts proc sys; do mount --bind /$i /mnt/$i; done
chroot /mnt update-grub
for i in dev/pts dev proc sys; do umount /mnt/$i; done
umount /mnt
</syntaxhighlight>
In Gandi V5 interface, stop the server, detach the old root disk and define the cooked disk as "Use to start".
Launch the server and execute:
df -hT
It should show something like:
/dev/xvda1    btrfs      5G  1.4G  3.5G  28% /
Delete the old root disk.


== Troubleshootings ==
== Troubleshootings ==

Version actuelle datée du 15 juin 2023 à 16:48

I tried to have a root filesystem into btrfs for a Gandi server, and succeed after a number of trial-error. Here are my steps if it can be useful to others. For an introduction, see Wikipedia article and/or the dedicated wiki.

WARNING: this could cause DATA LOSS, be sure you have backups (possibly Gandi snapshots).

This was updated on 2020-09-24 for Debian 10 buster + GandiV5, and on 2022-02-02 for Debian 11 bullseye + GandiV5. An older version was for Debian 9 jessie or Ubuntu 18.04 LTS + GandiV4.

Manual procedure

Create the server

On GandiV5, create a new server with e.g. 1 proc, 256 MiB RAM, 1 system disk of 5 GiB, Debian 10 buster or Debian 11 bullseye.

For Debian 10 buster (only), define backports (btrfs-progs must contain btrfs-convert, see this Debian bug) (this is not needed for Debian 11):

echo 'deb https://deb.debian.org/debian buster-backports main' > /etc/apt/sources.list.d/debian-backports.list

Install btrfs-progs:

apt update && apt install -t buster-backports btrfs-progs # Debian 10 buster
apt update && apt install btrfs-progs # Debian 11 bullseye

On GandiV5 :

  1. stop the server,
  2. open the page about the system disk in Volumes,
  3. clone the system disk with another (definitive) name (e.g. the current disk is "sys-myserver", you can define the copied disk as "myserver"),
  4. start the server,
  5. wait until the server is started,
  6. attach this cloned disk to the server.

ATTENTION: really start the server before attaching the cloned disk, else the root filesystem might be the cloned disk given they have the same UUID and /etc/fstab is using the UUID to select the root filesystem (it could be workarounded by setting /dev/xvda1 in /etc/fstab before stopin the server).

Convert to btrfs

Mostly from [1]

Display the active root filesystem, and you will be able to deduce what is the cloned filesystem:

ls /dev/xvd*
mount|grep xvd
blkid

Copy the result of blkid in some text file on your computer (will be used later).

Let’s say /dev/xvdb1 is the clone filesystem, we convert it to btrfs: (man 8 btrfs-convert)

fsck.ext4 -f /dev/xvdb1 # optional
btrfs-convert -p /dev/xvdb1

Here, I obtained the following error:

root@test:~# btrfs-convert -p /dev/xvdb1
create btrfs filesystem:
	blocksize: 4096
	nodesize:  16384
	features:  extref, skinny-metadata (default)
creating ext2 image file
creating btrfs metadata
Unable to find block group for 0 27081]
Unable to find block group for 0
Unable to find block group for 0
ctree.c:2245: split_leaf: BUG_ON `1` triggered, value 1
btrfs-convert(+0x11b5a)[0x559c159c1b5a]
btrfs-convert(+0x1589b)[0x559c159c589b]
btrfs-convert(btrfs_search_slot+0x269)[0x559c159c6401]
btrfs-convert(btrfs_insert_empty_items+0x92)[0x559c159c7b3c]
btrfs-convert(btrfs_record_file_extent+0x1bc)[0x559c159d46b4]
btrfs-convert(record_file_blocks+0x14a)[0x559c159bfe92]
btrfs-convert(+0x10349)[0x559c159c0349]
btrfs-convert(+0x1135f)[0x559c159c135f]
btrfs-convert(main+0x1f59)[0x559c159bdefb]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0x7f4c11c3bb97]
btrfs-convert(_start+0x2a)[0x559c159bb5ca]
Aborted

According to [2] and [3], it can be worked around: (this command could take some hours, depending on disk size and number of files)

btrfs-convert -d -p /dev/xvdb1

or even:

btrfs-convert -n -d -p /dev/xvdb1

Or, if still unsuccessful, try to add free space, or if still unsuccessfull, re-compile btrfs-progs in version 4.17.1 (this last try worked for me for 50 Gio disk with 24 Gio free space on Debian 9 jessie, expanded after its original size was 30 Gio (=4 Gio free space)).

Then, mount the filesystem and delete the old ext4 snapshot:

mount /dev/xvdb1 /mnt
btrfs subvolume delete /mnt/ext2_saved
btrfs filesystem defrag -r /mnt # could take dozen of minuts
btrfs balance start /mnt # could take hours

Promote as root filesystem

Mostly from [4]

Edit the /etc/fstab of the btrfs system:

uuid=`blkid|grep -P '^/dev/xvdb1: '|grep -o -P ' UUID="[0-9a-f-]+" '|cut -d'"' -f2`
sed -i -E 's/^UUID=[0-9a-f-]+ *\/ .*$/UUID=$uid \/               btrfs    defaults 0       1/' /mnt/etc/fstab

Update Grub in a chrooted system:

for i in dev dev/pts proc sys; do mount --bind /$i /mnt/$i; done
chroot /mnt update-grub
for i in dev/pts dev proc sys; do umount /mnt/$i; done
umount /mnt

Quit:

exit

Define the cooked disk as boot disk

In Gandi V5 interface, stop the server, detach the old root disk and define the cooked disk as "Use to start".

Launch the server (it should correctly start, even if the /boot directory is on the main partition / (some old documents said it didn’t work because grub didn’t know the btrfs filesystem, but it is fixed now)).

df -hT

It should show something like:

/dev/xvda1     btrfs      50G   32G   19G  63% /

Delete the old root disk.

You can delete the packages grub-efi-amd64, grub-efi-amd64-bin, grub-pc, grub-pc-bin since they are not used, but keep /etc/default/grub with Gandi customisations:

cp -a /etc/default/grub /root/grub
apt-get purge -y grub-efi-amd64 grub-efi-amd64-bin grub-pc grub-pc-bin
mv /root/grub /etc/default/grub
apt-mark manual grub-common grub2-common
update-grub # to check it still work and you should reboot now to be sure it reboots correctly

Scripted procedure

This script automates the conversion, with a slightly-modified procedure.

On GandiV5, create a new server with e.g. 1 proc, 256 MiB RAM, 1 system disk of 5 GiB, Debian 11 bullseye.

On GandiV5 :

  1. stop the server,
  2. open the page about the system disk in Volumes,
  3. clone the system disk with another (definitive) name (e.g. the current disk is "sys-myserver", you can define the copied disk as "myserver"),
  4. start the server,
  5. wait until the server is started,
  6. attach this cloned disk to the server.

ATTENTION: really start the server before attaching the cloned disk, else the root filesystem might be the cloned disk given they have the same UUID and /etc/fstab is using the UUID to select the root filesystem (it could be workarounded by setting /dev/xvda1 in /etc/fstab before stopin the server).

  1. Launch this script as root: (vi convert_gandiroot_to_btrfs.sh + chmod +x convert_gandiroot_to_btrfs.sh + (./convert_gandiroot_to_btrfs.sh)
    The script takes about 3 minutes to convert a 1.4 GiB disk.
#!/bin/sh

set -e

fs_type=`mount|grep -F ' on / '|grep -o -E ' type [a-z0-9]+ '|cut -d' ' -f3`
blkid=`blkid|grep -F ' LABEL="gandiroot" '`
nb_fs_gandiroot=`echo "$blkid"|cut -d: -f2-|uniq -c|tr -s ' '|cut -d' ' -f2`
if [ "$fs_type" != 'ext4' -o ! "$nb_fs_gandiroot" = 2 ]; then
    echo 'It is not possible to convert the disk. There should be two identical devices labeled "gandiroot" formatted in ext4'
    exit 2
fi
fs_gandiroot_ext4=`echo "$blkid"|head -n1|cut -d: -f1`
fs_gandiroot_btrfs=`echo "$blkid"|tail -n1|cut -d: -f1`

echo
echo "Current root filesystem: $fs_gandiroot_ext4 (ext4)"
echo "Future root filesystem: $fs_gandiroot_btrfs (currently ext4, will be btrfs)"

set -v

apt-get update

apt-get install -y btrfs-progs

fsck.ext4 -f "$fs_gandiroot_btrfs"

btrfs-convert -p "$fs_gandiroot_btrfs"

mount "$fs_gandiroot_btrfs" /mnt

btrfs subvolume delete -C /mnt/ext2_saved

btrfs filesystem defragment -r /mnt

btrfs balance start /mnt

new_uuid=`blkid|grep -P "^$fs_gandiroot_btrfs: "|grep -o -P ' UUID="[0-9a-f-]+" '|cut -d'"' -f2`

sed -i -E "s/^UUID=[0-9a-f-]+ *\/ .*$/UUID=$new_uuid \/               btrfs    defaults 0       1/" /mnt/etc/fstab

for i in dev dev/pts proc sys; do mount --bind /$i /mnt/$i; done

chroot /mnt update-grub

for i in dev/pts dev proc sys; do umount /mnt/$i; done

umount /mnt

In Gandi V5 interface, stop the server, detach the old root disk and define the cooked disk as "Use to start".

Launch the server and execute:

df -hT

It should show something like:

/dev/xvda1     btrfs      5G   1.4G   3.5G  28% /

Delete the old root disk.

Troubleshootings

  • During a conversion, the root filesystem was mounted read-only in btrfs; it was probably because I didn’t change the options "rw,noatime,errors=remount-ro" to "defaults"

External links