Btrfs
I tried to have a root filesystem into btrfs for a Gandi server, and succeed after a number of trial-error. Here are my steps if it can be useful to others.
WARNING: this is EXPERIMENTAL and could cause DATA LOSS.
Sommaire
Create the server
On https://v4.gandi.net, create a new server [1] with e.g. 1 proc, 256 Mio RAM, 1 system disk, Ubuntu 18.04 LTS or Debian 9.
Update the packages:
apt update && apt upgrade -y
On https://v4.gandi.net, stop the server, open the page about the system disk, clone the system disk with another name, and attach this cloned disk to the server, start the server.
Convert to btrfs
- Mostly from [2]
The package btrfs-progs currently does not contain btrfs-convert (this Debian bug) because the risk of data loss is too great. Then we need to compile btrfs-progs.
Install dependencies (Ubuntu 18.04 LTS):
apt install gcc g++ libblkid-dev liblzo2-dev zlib1g-dev libzstd-dev asciidoc xmlto libattr1-dev pkg-config python3-dev python3-setuptools libext2fs-dev
OR Install dependencies (Debian 9):
apt install gcc g++ libblkid-dev liblzo2-dev zlib1g-dev libzstd-dev asciidoc xmlto libattr1-dev pkg-config python3-dev python3-setuptools e2fslibs-dev
Download and compile btrfs-progs (changelog, releases):
wget https://mirrors.edge.kernel.org/pub/linux/kernel/people/kdave/btrfs-progs/btrfs-progs-v4.20.1.tar.gz tar xfz btrfs-progs-v4.20.1.tar.gz cd btrfs-progs-v4.20.1 ./configure --with-convert=ext2 make make install cd .. rm -rf btrfs-progs-v4.20.1 btrfs-progs-v4.20.1.tar.gz
Display the active root filesystem, and you will be able to deduce what is the cloned filesystem:
ls /dev/xvd* mount|grep xvd blkid -o value -s TYPE /dev/xvda1 # and the same for others
Let’s say /dev/xvdb1 is the clone filesystem, we convert it to btrfs: (man 8 btrfs-convert)
fsck.ext4 -f /dev/xvdb1 btrfs-convert -p /dev/xvdb1
Here, I obtained the following error:
root@test:~# btrfs-convert -p /dev/xvda1 create btrfs filesystem: blocksize: 4096 nodesize: 16384 features: extref, skinny-metadata (default) creating ext2 image file creating btrfs metadata Unable to find block group for 0 27081] Unable to find block group for 0 Unable to find block group for 0 ctree.c:2245: split_leaf: BUG_ON `1` triggered, value 1 btrfs-convert(+0x11b5a)[0x559c159c1b5a] btrfs-convert(+0x1589b)[0x559c159c589b] btrfs-convert(btrfs_search_slot+0x269)[0x559c159c6401] btrfs-convert(btrfs_insert_empty_items+0x92)[0x559c159c7b3c] btrfs-convert(btrfs_record_file_extent+0x1bc)[0x559c159d46b4] btrfs-convert(record_file_blocks+0x14a)[0x559c159bfe92] btrfs-convert(+0x10349)[0x559c159c0349] btrfs-convert(+0x1135f)[0x559c159c135f] btrfs-convert(main+0x1f59)[0x559c159bdefb] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0x7f4c11c3bb97] btrfs-convert(_start+0x2a)[0x559c159bb5ca] Aborted
According to [3] and [4], it can be worked around: (this command could take some hours, depending on disk size and number of files)
btrfs-convert -d -p /dev/xvdb1
or even:
btrfs-convert -n -d -p /dev/xvdb1
Or, if still unsuccessful, try to add free space, or if still unsuccessfull, re-compile btrfs-progs in version 4.17.1 (this last try worked for me for 50 Gio disk with 24 Gio free space, expanded after its original size was 30 Gio (=4 Gio free space)).
Then, mount the filesystem and delete the old ext4 snapshot:
mount /dev/xvdb1 /mnt btrfs subvolume delete /mnt/ext2_saved btrfs filesystem defrag -r /mnt # could take dozen of minuts btrfs balance start /mnt # could take hours
Promote as root filesystem
- Mostly from [5]
Enter the chrooted system:
for i in dev dev/pts proc sys; do mount --bind /$i /mnt/$i; done chroot /mnt blkid|grep xvdb1 vi /etc/fstab
Edit the root filesystem with: (the UUID is from the command blkid)
UUID=a74f5787-aee1-4981-b7e6-fbd3cb6ac919 / btrfs defaults 0 1
(in vi, type "dd" to remove a line, "i" to enter in edit mode, Esc to quit edit mode, ":x" to save and quit.)
vi /etc/grub.d/00_header
Comment the line:
#if [ -n "\${have_grubenv}" ]; then if [ -z "\${boot_once}" ]; then save_env recordfail; fi; fi
Update grub:
grub-install /dev/xvdb update-grub
Quit:
exit for i in dev dev/pts proc sys; do umount /mnt/$i; done umount /mnt exit
Define the cooked disk as boot disk
In Gandi V4 interface, stop the server, select the cooked disk as boot disk of this server (or detach the cooked disk and attach it as boot disk on another server).
Launch the server (it should correctly start, even if the /boot directory is on the main partition / (some old documents said it didn’t work because grub didn’t know the btrfs filesystem, but it is fixed now)).
df -hT
It should show something like:
/dev/xvda1 btrfs 50G 32G 19G 63% /
Debug data:
-
Start request GET /Btrfs?oldid=172181
HTTP HEADERS:
CONNECTION: Keep-Alive
HOST: wiki.seb35.fr
ACCEPT-ENCODING: br,gzip
IF-MODIFIED-SINCE: Wed, 03 Jun 2020 22:24:56 GMT
ACCEPT-LANGUAGE: en-US,en;q=0.5
ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
USER-AGENT: CCBot/2.0 (https://commoncrawl.org/faq/)
CONTENT-LENGTH:
CONTENT-TYPE: -
[caches] main: EmptyBagOStuff, message: SqlBagOStuff, parser: SqlBagOStuff
-
[caches] LocalisationCache: using store LCStoreCDB
-
Fully initialised
-
Connected to database 0 at localhost:/run/mysqld/mysqld.sock
-
Title::getRestrictionTypes: applicable restrictions to [[Btrfs]] are {edit,move}
-
[ContentHandler] Created handler for wikitext: WikitextContentHandler
-
IP: 3.227.247.17
-
Connected to database 0 at localhost:/run/mysqld/mysqld.sock
-
[DBPerformance] [GET] Expectation (masterConns <= 0) by MediaWiki::main not met:
[connect to localhost:/run/mysqld/mysqld.sock (wiki_seb35)]
TransactionProfiler.php line 271 calls wfBacktrace()
TransactionProfiler.php line 128 calls TransactionProfiler->reportExpectationViolated()
LoadBalancer.php line 517 calls TransactionProfiler->recordConnection()
SqlBagOStuff.php line 160 calls LoadBalancer->getConnection()
SqlBagOStuff.php line 245 calls SqlBagOStuff->getDB()
SqlBagOStuff.php line 219 calls SqlBagOStuff->getMulti()
MessageCache.php line 300 calls SqlBagOStuff->get()
MessageCache.php line 881 calls MessageCache->load()
MessageCache.php line 800 calls MessageCache->getMsgFromNamespace()
MessageCache.php line 737 calls MessageCache->getMessageFromFallbackChain()
Message.php line 1075 calls MessageCache->get()
Message.php line 698 calls Message->fetchMessage()
Message.php line 789 calls Message->toString()
OutputPage.php line 930 calls Message->text()
OutputPage.php line 977 calls OutputPage->setHTMLTitle()
Article.php line 512 calls OutputPage->setPageTitle()
ViewAction.php line 44 calls Article->view()
MediaWiki.php line 463 calls ViewAction->show()
MediaWiki.php line 269 calls MediaWiki->performAction()
MediaWiki.php line 634 calls MediaWiki->performRequest()
MediaWiki.php line 482 calls MediaWiki->main()
index.php line 41 calls MediaWiki->run()
index.php line 31 calls require() -
[MessageCache] MessageCache::load: Loading fr... got from global cache
-
Unstubbing $wgParser on call of $wgParser::firstCallInit from MessageCache::transform
-
Parser: using preprocessor: Preprocessor_DOM
-
Unstubbing $wgLang on call of $wgLang::_unstub from ParserOptions::__construct
-
Article::view using parser cache: no
-
Article::view: doing uncached parse
-
[Preprocessor] Saved preprocessor XML to memcached (key wiki_seb35:preprocess-xml:c5c1203a70ebb30499df22e3d811ea9d:0)
-
[Preprocessor] Saved preprocessor XML to memcached (key wiki_seb35:preprocess-xml:c5c1203a70ebb30499df22e3d811ea9d:0)
-
[username] User::isValidUserName: '3.227.247.17' invalid due to empty, IP, slash, length, or lowercase
-
[MessageCache] MessageCache::load: Loading en... got from global cache
-
Title::getRestrictionTypes: applicable restrictions to [[Btrfs]] are {edit,move}