David Bremner [Sun, 16 Sep 2018 17:08:07 +0000 (14:08 -0300)]
contrib/notmuch-mutt: add install target
The main goal here is to be able to install the notmuch-mutt script
with an absolute shebang. I have tried to make the notmuch-mutt
Makefile use configure information from notmuch if available, but make
suitable guesses if not.
David Edmondson [Sat, 8 Sep 2018 11:49:40 +0000 (12:49 +0100)]
notmuch: Database paths without a leading / are relative to $HOME
If the database path specified in the configuration file does *not*
start with a /, presume that it is relative to $HOME and modify the
path used to open the database accordingly.
David Bremner [Sat, 8 Sep 2018 19:15:10 +0000 (16:15 -0300)]
debian: add unversioned emacs, drop emacs23
In current Debian sid / testing, emacs25 is a transitional package,
and emacs-gtk, emacs-lucid, and emacs-nox are the real packages.
Emacs 23 is not available in emacs after oldoldstable.
David Bremner [Thu, 30 Aug 2018 11:29:15 +0000 (08:29 -0300)]
lib: change parent strategy to use In-Reply-To if it looks sane
As reported by Sean Whitton, there are mailers (in particular the
Debian Bug Tracking System) that have sensible In-Reply-To headers,
but un-useful-for-notmuch References (in particular with the BTS, the
oldest reference is last). I looked at a sample of about 200K
messages, and only about 0.5% these had something other than a single
message-id in In-Reply-To. On this basis, if we see a single
message-id in In-Reply-To, consider that as authoritative.
David Bremner [Thu, 30 Aug 2018 11:29:14 +0000 (08:29 -0300)]
lib: add _notmuch_message_id_parse_strict
The idea is that if a message-id parses with this function, the MUA
generating it was probably sane, and in particular it's probably safe
to use the result as a parent from In-Reply-to.
David Bremner [Thu, 30 Aug 2018 11:29:12 +0000 (08:29 -0300)]
test/thread-replies: mangle In-Reply-To's
In a future commit, we will start trusting In-Reply-To's when they
look sane (i.e. a single message-id). Modify these tests so they will
keep passing (i.e. keep choosing References) when that happens.
David Bremner [Thu, 30 Aug 2018 11:29:11 +0000 (08:29 -0300)]
test: add known broken test for good In-Reply-To / bad References
The current scheme of choosing the replyto (i.e. the default parent
for threading purposes) does not work well for mailers that put
the oldest Reference last.
David Bremner [Thu, 30 Aug 2018 11:29:10 +0000 (08:29 -0300)]
lib/thread: change _resolve_thread_relationships to use depths
We (finally) implement the XXX comment. It requires a bit of care not
to reparent all of the possible toplevel messages.
_notmuch_messages_has_next is not ready to be a public function yet,
since it punts on the mset case. We know in the one case it is called,
the notmuch_messages_t is just a regular list / iterator.
David Bremner [Thu, 30 Aug 2018 11:29:07 +0000 (08:29 -0300)]
lib/thread: initial use of references as for fallback parenting
This is mainly to lay out the structure of the final code. The problem
isn't really solved yet, although some very simple cases are
better (hence the fixed test). We need two passes through the messages
because we need to be careful not to re-parent too many messages and
end up without any toplevel messages.
David Bremner [Mon, 3 Sep 2018 21:02:25 +0000 (18:02 -0300)]
lib: add _notmuch_message_list_empty
There is no public notmuch_message_list_t public interface, so to this
is added to the private API. We use it immediately in thread.cc;
future commits will use it further.
David Bremner [Thu, 30 Aug 2018 11:29:04 +0000 (08:29 -0300)]
lib/thread: sort sibling messages by date
For non-root messages, this should not should anything currently, as
the messages are already added in date order. In the future we will
add some non-root messages in a second pass out of order and the
sorting will be useful. It does fix the order of multiple
root-messages (although it is overkill for that).
David Bremner [Thu, 30 Aug 2018 11:29:02 +0000 (08:29 -0300)]
test: start threading test corpus
There are 3 threads here, two synthetic, and one anonymized one using
data from Gregor. They test various aspects of thread
ordering/construction in the presence of replies to ghost messages.
test: add known broken test for mislabeled Windows-1252 encoding
Messages that contain Windows-1252 are frequently mislabeled as ISO
8859-1, which may result in non-printable characters when displaying
the message. The test asserts that such characters (in this case
curved quotes) are displayed correctly.
David Edmondson [Sun, 26 Aug 2018 21:16:12 +0000 (22:16 +0100)]
emacs: Kill the stderr buffer when an async process completes
On some platforms (e.g. macOS), it is necessary to add a real sentinel
process for the error buffer used by `notmuch-start-notmuch' rather
than a no-op sentinel.
Many of the manpages didn't treat literal text as literal text. I've
tried to normalize some of the restructured text to make it a bit more
regular.
several of the synopsis lines are still untouched by this cleanup, but
i'm not sure what the right way to represent those is in .rst,
actually.
In particular find that if i rebuild the manpages, sometimes i end up
with some of the synopsis lines showing – (U+2013 EN DASH) where they
should have -- (2 × U+002D HYPHEN-MINUS) in the generated nroff
output, though i have not tracked down the source of this error yet.
David Bremner [Wed, 30 May 2018 02:03:17 +0000 (19:03 -0700)]
configure: check for links to/from libdir in ldconfig output
If e.g. /lib is a link to /usr/lib, then the latter may not show up in
the way we expect in the output of ldconfig. 'test foo -ef bar' checks
if foo and bar have the same device and inode numbers. Since (at least
in bash, dash, ksh, and zsh) the shell dereferences symlinks before
applying the test, this includes both the case where file1 is equal to
file2 and the case where one is a symlink to the other.
David Bremner [Mon, 21 May 2018 20:08:56 +0000 (13:08 -0700)]
docs: add initial documentation for notmuch-tag-jump
This is mainly to improve discoverability. It seems that doing
variable cross-references is not easy without using some sphinx
extension/customization.
David Bremner [Mon, 21 May 2018 20:08:53 +0000 (13:08 -0700)]
debian: install info pages as part of elpa (emacs) package
This is a simple way to make the info pages available, since the emacs
package system deals with the annoying bits. If there is many people
who want the info pages outside of emacs, we could consider a separate
binary package, but currently it doesn't seem worthwhile.
Add fancy new feature, which makes "notmuch show" capable of actually
indexing messages that it just decrypted.
This enables a workflow where messages can come in in the background
and be indexed using "--decrypt=auto". But when showing an encrypted
message for the first time, it gets automatically indexed.
This is something of a departure for "notmuch show" -- in particular,
because it requires read/write access to the database. However, this
might be a common use case -- people get mail delivered and indexed in
the background, but only want access to their secret key to happen
when they're directly interacting with notmuch itself.
In such a scenario, they couldn't search newly-delivered, encrypted
messages, but they could search for them once they've read them.
Documentation of this new feature also uses a table form, similar to
that found in the description of index.decrypt in notmuch-config(1).
A notmuch UI that wants to facilitate this workflow while also
offering an interactive search interface might instead make use of
these additional commands while the user is at the console:
Count received encrypted messages (if > 0, there are some things we
haven't yet tried to index, and therefore can't yet search):
notmuch count tag:encrypted and \
not property:index.decryption=success and \
not property:index.decryption=failure
Reindex those messages:
notmuch reindex --try-decrypt=true tag:encrypted and \
not property:index.decryption=success and \
not property:index.decryption=failure
cli/show: reindex when we learned new session keys about a message
If the number of session keys for a given message increased after
running "notmuch show" then we just learned something new that might
let us do automatic decryption. We should reindex this message using
our newfound knowledge.
cli: write session keys to database, if asked to do so
If the decryption policy is NOTMUCH_DECRYPT_TRUE, that means we want
to stash session keys in the database. Note that there is currently
no way from the command line to set it this way, though, so it is not
yet included in the test suite.
The user can already do this manually, of course, but (a) it's nice to
have a convenience function, and (b) exposing this interface means
that someone more clever with a _notmuch_string_map_t than i am can
write a more efficient version if they like, and it will just
accelerate the users of the convenience function.
We've had _notmuch_message_database() internally for a while, and it's
useful. It turns out to be useful on the other side of the library
interface as well (i'll use it later in this series for "notmuch
show"), so we expose it publicly now.
David Bremner [Sun, 29 Apr 2018 23:19:42 +0000 (20:19 -0300)]
test: use --full-scan in T050-new.sh
Wherever the test relies on directories being scanned, this option
should be used to avoid skipping them due to mtimes on directories
matching the database.
David Bremner [Sun, 29 Apr 2018 23:19:41 +0000 (20:19 -0300)]
test: add tests for notmuch new --full-scan
Most of these just check that adding the flag does not break existing
functionality. The one test that does check the full-scan
functionality had to be rewritten to output debugging info.
David Bremner [Sun, 29 Apr 2018 23:19:40 +0000 (20:19 -0300)]
CLI/new: add full-scan option
By default notmuch-new uses directory mtimes to optimize the scanning of
directories for new mail. This option allows turning that optimization
off e.g. for testing or debugging.
David Bremner [Sun, 28 Jan 2018 02:50:42 +0000 (22:50 -0400)]
lib: define specialized get_thread_id for use in thread subquery
The observation is that we are only using the messages to get there
thread_id, which is kindof a pessimal access pattern for the current
notmuch_message_get_thread_id
David Bremner [Tue, 26 Dec 2017 01:03:05 +0000 (21:03 -0400)]
lib: add thread subqueries.
This change allows queries of the form
thread:{from:me} and thread:{from:jian} and not thread:{from:dave}
This is still somewhat brute-force, but it's a big improvement over
both the shell script solution and the previous proposal [1], because it
does not build the whole thread structure just generate a
query. A further potential optimization is to replace the calls to
notmuch with more specialized Xapian code; in particular it's not
likely that reading all of the message metadata is a win here.
Correct URLs that have crept into the notmuch codebase with http://
when https:// is possible.
As part of this conversion, this changeset also indicates the current
preferred upstream URLs for both gmime and sup. the new URLs are
https-enabled, the old ones are not.
This also fixes T310-emacs.sh, thanks to Bremner for catching it.
David Bremner [Thu, 14 Dec 2017 14:32:34 +0000 (10:32 -0400)]
test: add known broken test for regexp search of second subject
We expect this to give the same answer as the non-regexp subject
search. It does not because the regexp search relies on the value
slot, which currently contains only one subject.