notmuch performance: chert vs flint, compacting

diff --git a/index.mdwn b/index.mdwn
index 172015e6ecc24f2311335112a14fe2affef359d0..9924de86312d169fdb13c34e411773d1d905e051 100644 (file)
--- a/index.mdwn
+++ b/index.mdwn
@@ -52,6 +52,7 @@ contain a reference for users.  This is what we have so far:
   * [[Initial_Tagging]]
   * [[Tips for using notmuch within Emacs|emacstips]]
   * [[Remote Usage|remoteusage]]
+  * [[Performance|performance]]
 
 Apart from the wiki, help is available via email.  Join the mailing
 list.  Read the archives.  Ask questions.

diff --git a/performance.mdwn b/performance.mdwn
new file mode 100644 (file)
index 0000000..dcd70e0
--- /dev/null
+++ b/performance.mdwn
@@ -0,0 +1,103 @@
+# Optimize notmuch's performance
+
+## Xapian 1.2.x database format: chert vs. flint
+
+Xapian 1.2 (in contrast to 1.0.x) uses a new database format called
+`chert` which is a lot faster than the old format `flint`.
+
+Check whether you are on chert or flint:
+
+    $ ls $(notmuch config get database.path)/.notmuch/xapian/iam*
+    iamchert
+
+If it does not say `iamchert`, but `iamflint`, than you should migrate
+(read on).
+
+There might be a file named `flintlock`, it is no indicator for your
+database format, but also used in case of `chert`.
+
+### Migrate from flint to chert
+
+1. make sure you are using xapian 1.2.x (depends on your distribution)
+
+XXX: add examples
+
+2. Backup your notmuch tags
+
+    $ cd
+    $ notmuch dump notmuch-dump-b4chert
+
+3. Move away your old database
+
+    $ mv $(notmuch config get database.path)/.notmuch/ notmuch-db-b4chert
+
+4. Find and import your messages, a new and fresh database will be
+created.
+
+    $ XAPIAN_PREFER_CHERT=1 notmuch new
+
+I did not need the `XAPIAN_PREFER_CHERT`, but am not sure whether
+there are situations making it necessary (certain compile-time options
+to xapian?). This note is just here to be extended or removed by
+somebody more knowledgeable.
+
+After the initial import it is definitely not necessary to define
+`XAPIAN_PREFER_CHERT` for subsequent calls of `notmuch new`.
+
+5. Restore your tags
+
+    $ notmuch restore notmuch-dump-b4chert
+
+6. Check whether you are on chert now.
+
+    $ ls $(notmuch config get database.path)/.notmuch/xapian/iam*
+    iamchert
+
+If it says `iamflint`, most likely you are still using xapian 1.0.x.
+
+7. Clean-up
+
+If you are sure everything is fine, you can delete the old things. The
+dump of your tags you might want to keep anyway as a backup - on the
+other hand you should have some mechanism in place to make regular
+backups of your notmuch database.
+
+    $ rm notmuch-dump-b4chert
+    $ rm -R notmuch-db-b4chert
+
+### Compact your database
+
+In order to greatly reduce the size of your database, you should
+compact it. Carl says "This functionality is not yet available in the
+Xapian library interface or else I would probably make notmuch call it
+after building the database." and Pieter "[compacting] will cause
+writes to be a bit more expensive in the short term, but will
+significantly reduce the db's size, which your SSD will surely
+appreciate."
+
+This needs xapian 1.2
+
+Backup your database:
+
+    $ cd
+    $ notmuch dump notmuch-dump-b4compact
+
+Compact it:
+
+    $ cd $(notmuch config get database.path)/.notmuch
+    $ xapian-compact --no-renumber xapian xapian-compacted
+
+Remove old and renamed compacted, if compacting went well:
+
+    $ rm -R xapian
+    $ mv xapian-compacted xapian
+
+
+### More information about this in the email archives:
+
+* [id:xyz, Carl Worth](http://notmuchmail.org/pipermail/notmuch/2010/001105.html)
+
+* [id:xyz, Sebastian Spaeth](http://notmuchmail.org/pipermail/notmuch/2011/004048.html)
+
+* [id:xyz, Pieter Praet](http://notmuchmail.org/pipermail/notmuch/2011/004175.html)
+