emacstips: how to add a user-agent header

[notmuch-wiki] / emacstips.mdwn

[[!img notmuch-logo.png alt="Notmuch logo" class="left"]]
#Tips and Tricks for using notmuch with Emacs

The main client based on notmuch is notmuch.el, which is included in the notmuch package. It is might, and allows you to configure a lot of things, however, it might not be immediately obvious how, and how the general workflow generally looks like. This [first section](#typical_workflow) will describe a typical workflow and setup, while the section [Advanced tips and tweaks](#advanced_tips) below, focuses on more specific questions.

<h2 id="typical_workflow">Typical setup and workflow</h2>

### Set up & receiving mails

Notmuch requires either a MailDir or a "mh"-style maildirectory to operate on. (It basically requires any setup in which each mail is in a file of it's own.) Most people use "[offlineimap](http://software.complete.org/software/projects/show/offlineimap)" or "[mbsync](http://isync.sourceforge.net/)" in order to synchronize their IMAP or pop mail server with a local mail store. offlineimap is quite useful and widely tested, it also offers a handy hook that will come in useful a bit later in our setup. 

So go install and configure offlineimap so you can simply run offlineimap and have an updated maildir. In the [Account xxx] section of your .offineimaprc file your can specify a "presynchook" and "postsynchook" command that will get run whenever you sync. Point _postsynchook_ to a script that gets run on every sync and that will do the automatic tagging and updating of your notmuch database.

The script will look somewhat like this:

>     #/bin/sh
>     # incorporate all new mails in the database
>     notmuch new > /dev/null
>     #apply some automatic tags
>     notmuch tag +notmuch from:notmuch@notmuchmail.org and not tag:notmuch
>     ...more tag rules...

If you don't need automatic tagging, just use "notmuch new > /dev/null" as your synchook. One advanced setup with automatic tagging has been described by James Vasile here (id: _87hbp5j9dv.fsf@hackervisions.org_). Carl Worth has described his script in this mail (id: _87r5o8stbj.fsf@yoom.home.cworth.org_) in the same thread.

> __A note on message ids__: Confused by those message ids? Get used to them, because they are an importance part of the notmuch workflow. Every send message has a unique message id, just like a website has a unique url. On the notmuch mailing list people will throw around message ids and expect people to just find the right mails, as this is what notmuch makes easy. In order to find a specific message, hit 's' for search and type: "id:messageidhere". If you incorporated the notmuch mail archive, you can e.g. try to find id:87r5o8stbj.fsf@yoom.home.cworth.org. Don't have the notmuch archive back to Feb 25, 2010? Fortunately gmane (and probably others allow to search for message id in their archives as well. You'll find this message under [http://mid.gmane.org/87r5o8stbj.fsf@yoom.home.cworth.org](http://mid.gmane.org/87r5o8stbj.fsf@yoom.home.cworth.org) (incidentally, this server seems to be down right now).

### Synchronizing notmuch tags with maildir flags
IMAP users rely on maildir flags that convey the status "seen", "replied", "trashed", in order to synchronize the status of their mail across mail clients (a [maildir flag](http://cr.yp.to/proto/maildir.html) is simply a defined part of their filename). notmuch will by default happily ignore those flags and will never modify them either. If you want to use your mail with non-notmuch clients too, please check out the entry on "[how to sync notmuch tags and maildir flags](#sync_maildir_flags)".

### Navigating & reading mails
OK, messages are now in your maildir and they have the tags you want them to have (of course you will be assigning more tags manually as you parse through your mail.). Now, on to actually using notmuch in emacs. If you added the correct bits to your .emacs file you will be able to start notmuch by typing "M-x notmuch" (or M-x notmuch-folder). If you want to start notmuch immediately when starting emacs you can also call emacs as "emacs -f notmuch" (and create a handy shortcut on your desktop for that).

notmuch (the search view) and notmuch-folder are the 2 main views that can be used to navigate through your mail.
The default notmuch "home" is a notmuch-search view with all all mail threads in which a message has been tagged as "inbox". Just navigate that page and read threads by hitting <RETURN> on one of them. This will open a notmuch thread view with all messages in this thread. Threads are a powerful construct in notmuch as a thread really represents a discussion around a single topic. In difference to other mail applications, you will often find very long threads in notmuch mail conversations, or people reviving very old threads with a reply.

The second way of navigating notmuch mails, is by using the notmuch-folder view which you can invoke with `M-x notmuch-folder` (or using the F key if you applied the patch from mail id: 1265889583-25406-1-git-send-email-dme@dme.org (Feb 11st, 2010)). For information on how to customize your notmuch-folder views, see [here](#customize_notmuch_folder). This view provides you with "virtual folders" that allow you to browse all emails topic-by-topic. For example, if you want to browse all notmuch related mail every 10 minutes, but check out mails from your boss only once a day, this will be one convenient way to do it. It will also give you a nice overview of how many (unread) emails you have for each folder. Those folders are virtual in that a mail can easily be part of several folders at the same time and the folders will be updated on-the-fly every time you hit `=`.

Last but not least, you can always search for a specific mail. Notmuch provides very powerful search possibilities which allows you to find mail threads easily without having to file them away into umpteenth mail folders as you might have been doing in your Thunderbird, Mail.app, or Outlook.

### search strategies
__[TODO, describe s, f, t keys and some exemplary searches]__.
__[TODO: describe common search strategies that notmuch users use]__

###Sending mail
Notmuch itself does not provide any facilities for sending mail. Fortunately, emacs comes with "message mode" which offers that functionality. This also means that there are various methods to invoke the mail sending command. One possibility is to use "CTRL-x m" which will invoke a new "message mail" window. The second possibility is to type 'm' when you are in a notmuch window. And finally, when looking at a message or thread in notmuch you can type 'r' to start a reply.

If you want to use mail address autocompletion, check out [bbdb](http://bbdb.sourceforge.net) which works nicely together with message mode.

Type your message and send it off with ctrl-c ctrl-c. By default message mode will use your /usr/sbin/sendmail command to send a mail, so make sure that works.
One annoying standard configuration of message mode is that it will hide the sent mail in your emacs frame stack, but it will not close it. If you type several mails in an emacs session they will accumulate and make switching between buffers more annoying. You can avoid that behavior by adding `(setq message-kill-buffer-on-exit t)` in your .emacs file which will really close the mail window after sending it.

Currently, there is a slight problem with copying the sent mail into your "outbox". While conventional MUAs will automatically copy your sent mail to a "Sent" folder, notmuch will never touch your mail store and your sent mails will not be searchable by default. When replying with 'r' notmuch will insert a BCC header to yourself, but when sending a mail with 'm' or 'ctrl-x m', it will not (and can not as ctrl-x m does not involve notmuch at all). Please see the specific advice on "[how to do FCC/BCC](#fcc)" for more details on how to set up notmuch to obey your preferences.

###3rd party application

As its name implies, notmuch isn't really doing that much (which is part of its appeal to many of us). However, it enables a whole ecosystem of surrounding scripts to work in conjunction with notmuch

* [notmuchsync](http://spaetz.github.com/notmuchsync/)
  notmuchsync is a python tool that allows synchronization of maildir flags to/from notmuch tags.
* [cnotmuch](http://pypi.python.org/pypi/cnotmuch) python bindings for the notmuch shared library. Extensive API documentation available.
* [addrlookup](http://github.com/spaetz/vala-notmuch) vala spplication that can be used for interactive email address completion.
* gitmuch?
* ...

-----

<h2 id="advanced_tips">Advanced tips and tweaks</h2>
* <span id="fcc">**How to do FCC/BCC...**</span>

  Any notmuch reply will automatically include your primary email
  address in a BCC so that any messages you send will (eventually) end
  up in your mail store as well. But this doesn't do anything for
  messages that you compose that are not replies.

  Another method is described in [this
  mail](http://mid.gmane.org/878wbj4nfq.fsf@servo.finestructure.net)
  (id:878wbj4nfq.fsf@servo.finestructure.net). It involves a trivial
  python script (which should be rewritten in lisp and integrated into
  the notmuch frontend, really, but is not difficult to setup.

  This is the code I needed in my .emacs file to make it work with
  the python wrapper that I called mddeliver.py:

          ;; fcc handler
          (defun maildir-deliver-region(destdir)
            (shell-command-on-region
              (point-min) (point-max)
              (concat "/usr/local/bin/mddeliver.py -c -s -d " destdir)))
          (setq message-fcc-handler-function 'maildir-deliver-region)
          (defun my-message-header-setup ()
            (message-add-header "Fcc: ~/mail/INBOX.Sent"))
          (add-hook 'message-send-hook 'my-message-header-setup)

* <span id="customize_notmuch_folder">How to customize notmuch-folders</span>

  There's a "notmuch-folder" command available in the emacs client
  that displays a list of "folders" and the number of messages in
  each. Each folder is simply a named search specification. To
  configure this mode, edit your ${HOME}/.emacs file and include text
  something like the following:

                (setq notmuch-folders '(("inbox" . "tag:inbox")
                                        ("unread" . "tag:inbox AND tag:unread")
                                        ("notmuch" . "tag:inbox AND to:notmuchmail.org")))

  Of course, you can have any number of folders, each configured
  with any supported search terms (see "notmuch help search-terms").

  Personally, I find it fairly useful to add "not tag:delete" to those
  views as I use that tag to mark messages as deleted and it
  automatically removes them from my standard views. Use whatever
  seems most useful to you.

* Viewing HTML messages with an external viewer

  The emacs client can often display an HTML message inline, but it
  sometimes fails for one reason or another, (or is perhaps inadequate
  if you really need to see the graphical presentation of the HTML
  message).

  In this case, it can be useful to display the message in an external
  viewer, such as a web browser. Here's a little script that Keith
  Packard wrote, which he calls view-html:

                #!/bin/sh
                dir=3D`mktemp -d`
                trap "rm -r $dir" 0
                cat "$@" > "$dir"/msg
                if munpack -C "$dir" -t < "$dir"/msg 2>&1 | grep 'Did not find'; then
                    sed -n '/[Hh][Tt][Mm][Ll]/,$p' "$dir"/msg > $dir/part1.html
                    rm "$dir"/msg
                fi
                for i in "$dir"/part*; do
                    if grep -q -i -e '<html>' -e 'text/html' "$i"; then
                        iceweasel "$i" &
                        sleep 3
                        exit 0
                    fi
                done

  Save that script somewhere in your ${PATH}, make it executable, and
  change the invocation of iceweasel to any other HTML viewer if
  necessary. Then within the emacs client, press "|" to pipe the
  current message, then type "view-html".

  Keith mentions the following caveat, "Note that if iceweasel isn't
  already running, it seems to shut down when the script exits. I
  don't know why."

* msmtp, message mode and multiple accounts

  As an alternative to running a mail server such as sendmail or
  postfix just to send email, it is possible to use
  [msmtp](http://msmtp.sourceforge.net/).  This small application will
  look like /usr/bin/sendmail to a MUA such as emacs message mode, but
  will just forward the email to an external SMTP server.  It's fairly
  easy to set up and it support several account for using different
  SMTP servers.  The msmtp pages have several examples.

  A typical scenario is that you want to use the company SMTP server
  for email coming from your company email address, and your personal
  server for personal email.  If msmtp is passed the envelope address
  on the command line (the -f/--from option) it will automatically
  pick the matching account.  The only trick here seems to be getting
  emacs to actually pass the envelope from.  There are a number of
  overlapping configuration variables that control this, and it's a
  little confusion, but setting these three works for me:

   - mail-specify-envelope-from: t

   - message-sendmail-envelope-from: header

   - mail-envelope-from: header

  With that in place, you need a .msmtprc with the accounts configured
  for the domains you want to send out using specific SMTP servers and
  the rest will go to the default account.

* <span id="sync_maildir_flags">how to sync notmuch tags and maildir
  flags</span>

  There is no built-in support for sunchronizing your notmuch tags
  with your imap maildir flags (seen, replied, trashed, etc). However
  there are currently 2 ways in which you can achieve
  synchronization. In order to sync new notmuch tags to maildir flags,
  there is a python tool called
  [notmuchsync](http://spaetz.github.com/notmuchsync/) that allows
  synchronization in either direction or pruning of deleted files.  

  The second solution allows for fast maildir flag to notmuch tag
  synchronization (and only in that direction) requires patching the
  notmuch source code. The patch has been posted in this mail
  *id:1267450136-31749-1-git-send-email-Sebastian@SSpaeth.de* and can
  be viewed as a [source diff in this git
  repository](http://github.com/spaetz/notmuch-all-feature/commit/df3b087cefb85e9d16fd17540e348bcb854dd7ee).

* <span id="address_completion">**how to get email address completion**</span>
  There are 2 solutions. Use "bbdb" which allows you to maintain a mail database and gives you mail address completion with the tab key.

  Alternatively, you use the notmuch database as a mail address book
  itself. You need a command line tool that outputs likely address
  candidates based on a search string. There is a python tool
  [notmuch_address.py](which can be fetched with `git clone
  http://jkr.acm.jhu.edu/git/notmuch_addresses.git`) (slower, but no
  compilation required so good for testing the setup) or the
  vala-based addrlookup (faster, but needs compiling).  This is how
  you compile the (3rd party) tool "addrlookup" to give you address
  completion:

  - you need the addrlookup binary, first of all. Grab http://github.com/spaetz/vala-notmuch/raw/static-sources/src/addrlookup.c and build it with *cc -o addrlookup addrlookup.c `pkg-config --cflags --libs gobject-2.0` -lnotmuch*. That should give you the binary that you can test already.

  - EUDC is integrated into emacs and can be used for tab completion
    of email addresses. The code I use is here
    http://gist.github.com/359425. It was announce in [this
    mail](http://mid.gmane.org/87fx3uflkx.fsf@jhu.edu)
    (id:87fx3uflkx.fsf@jhu.edu) which contains links to the git
    repositories which contain the files.

* <span id="insert_user_agent">**how to insert a user agent header**</span>

  Sometimes it is good to let the world know which email client you use, so others know which quirks to expect. And it is no shame to let others know that you are way ahead of mail2.0. Notmuch is mail3.0 (at least)!

  [This mail](http://mid.gmane.org/87y6gtnkch.fsf@SSpaeth.de) (id:87y6gtnkch.fsf@SSpaeth.de) posted some code, that -when inserted in your .emacs file- will add a User-Agent header (which is hidden during composing and in notmuch show, but which can be seen when viewing all headers of a mail):

  If you insert this slab of code, you will get a User-Agent header for notmuch 
          ;; set the User-Agent string whenever we invoke message mode
          (add-hook 'message-mode-hook '(lambda()
              ;; check if User-Agent is a required header and set it if not
              (if (not (memq 'User-Agent message-required-mail-headers))
                  (setq message-required-mail-headers 
                        (append message-required-mail-headers '(User-Agent))))
              ;; hide the User-Agent header if not already hidden
              (if (not (memq '"^User-Agent:" message-hidden-headers))
                  (setq message-hidden-headers 
                        (append message-hidden-headers '("^User-Agent:"))))
              ;; create user agent string
              (let ((notmuch-user-agent (concat 
                                         (substring (shell-command-to-string (concat notmuch-command " --version")) 0 -1)
                                         " (Emacs " emacs-version "/"
                                          system-configuration ")")))
                (setq message-newsreader notmuch-user-agent))
          ))