+[[!img notmuch-logo.png alt="Notmuch logo" class="left"]]
# Optimize notmuch's performance
-## Xapian 1.2.x database format: chert vs. flint
+[[!toc levels=2]]
-Xapian 1.2 (in contrast to 1.0.x) uses a new database format called
-`chert` which is a lot faster than the old format `flint`.
+## Use of ecryptfs slows notmuch down.
-Check whether you are on chert or flint:
+I noticed that my `notmuch new` was really slow (several minutes) for
+less then 20 new emails (around 20K in my database).
- $ ls $(notmuch config get database.path)/.notmuch/xapian/iam*
- iamchert
+The performance tests in the notmuch source didn't show any real
+issue, even on a non-encrypted ext4 fs.
-If it does not say `iamchert`, but `iamflint`, than you should migrate
-(read on).
+After investigation and running different tests, a switch to a
+LUKS-btrfs based filesystem to replace the ext4-ecryptfs resolved the
+issue. It now takes some seconds to index hundreds of messages.
-There might be a file named `flintlock`, it is no indicator for your
-database format, but also used in case of `chert`.
+## Xapian database format: chert vs. flint vs. glass
-### Migrate from flint to chert
+Depending on the Xapian version that was used during the initial notmuch
+database creation, you might use another database format than the most recent
+one, which is often faster.
-1. make sure you are using xapian 1.2.x (depends on your distribution)
+Check whether you are on chert, flint or glass:
-XXX: add examples
+ $ ls $(notmuch config get database.path)/.notmuch/xapian/iam*
+ iamglass
-2. Backup your notmuch tags
+If it does not say `iamglass`, but `iamchert` or `iamflint`, than you should
+migrate (read on).
- $ cd
- $ notmuch dump notmuch-dump-b4chert
+There might be a file named `flintlock`, it is no indicator for your database
+format, but also used in case of `chert`.
-3. Move away your old database
+### Migrate from an older format to the most recent one (glass)
- $ mv $(notmuch config get database.path)/.notmuch/ notmuch-db-b4chert
+Make sure you are using xapian 1.4.x (depends on your distribution)
-4. Find and import your messages, a new and fresh database will be
-created.
+XXX: add examples for different distributions
- $ XAPIAN_PREFER_CHERT=1 notmuch new
+Backup your notmuch tags:
-I did not need the `XAPIAN_PREFER_CHERT`, but am not sure whether
-there are situations making it necessary (certain compile-time options
-to xapian?). This note is just here to be extended or removed by
-somebody more knowledgeable.
+ $ cd
+ $ notmuch dump --output=notmuch-dump-oldformat
-After the initial import it is definitely not necessary to define
-`XAPIAN_PREFER_CHERT` for subsequent calls of `notmuch new`.
+Move away your old database:
-5. Restore your tags
+ $ mv $(notmuch config get database.path)/.notmuch/ notmuch-db.bak
- $ notmuch restore notmuch-dump-b4chert
+Find and import your messages, a new and fresh database will be
+created:
-6. Check whether you are on chert now.
+ $ notmuch new
- $ ls $(notmuch config get database.path)/.notmuch/xapian/iam*
- iamchert
+Restore your tags:
-If it says `iamflint`, most likely you are still using xapian 1.0.x.
+ $ notmuch restore --input=notmuch-dump-oldformat
-7. Clean-up
+Check whether you are on glass now:
-If you are sure everything is fine, you can delete the old things. The
-dump of your tags you might want to keep anyway as a backup - on the
-other hand you should have some mechanism in place to make regular
-backups of your notmuch database.
+ $ ls $(notmuch config get database.path)/.notmuch/xapian/iam*
+ iamglass
+
+ If it says `iamflint` or `iamchert`, most likely you are still using an old
+ xapian version.
- $ rm notmuch-dump-b4chert
- $ rm -R notmuch-db-b4chert
+Clean-up: if you are sure everything is fine, you can delete the old
+things. The dump of your tags you might want to keep anyway as a
+backup - on the other hand you should have some mechanism in place to
+make regular backups of your notmuch database:
+
+ $ rm notmuch-dump-oldformat
+ $ rm -R notmuch-db.bak
### Compact your database
Backup your database:
$ cd
- $ notmuch dump notmuch-dump-b4compact
+ $ notmuch dump --output=notmuch-dump-b4compact
Compact it:
### More information about this in the email archives:
-* [Carl Worth](http://notmuchmail.org/pipermail/notmuch/2010/001105.html)
-
-* [Sebastian Spaeth](http://notmuchmail.org/pipermail/notmuch/2011/004048.html)
+* [Carl Worth](https://notmuchmail.org/pipermail/notmuch/2010/001105.html)
-* [Pieter Praet](http://notmuchmail.org/pipermail/notmuch/2011/004175.html)
+* [Sebastian Spaeth](https://notmuchmail.org/pipermail/notmuch/2011/004048.html)
+* [Pieter Praet](https://notmuchmail.org/pipermail/notmuch/2011/004175.html)
projects / notmuch-wiki / blobdiff