David Bremner [Sun, 6 Feb 2022 13:38:16 +0000 (09:38 -0400)]
doc: add configuration section to notmuch-insert(1)
This is partially redudant given some existing cross references, but
it is useful to have all of the config keys listed in one place, to
help keep track of them if nothing else.
David Bremner [Sun, 6 Feb 2022 13:07:13 +0000 (09:07 -0400)]
doc: alphabetize options in notmuch-config(1)
Originally (I think) these were in the order generated by notmuch
setup. As the number of options grows, and several are not in the
initial setup generated file, the original order becomes less useful
for users. This commit alphabetizes the keys to help users
search. There is only one content change, an added cross-reference
from user.other_email to user.primary_email.
David Bremner [Sat, 12 Feb 2022 02:47:03 +0000 (22:47 -0400)]
CLI/insert: escape envelope from
The idea is to do as little parsing and modification of the delivered
message as possible. Luckily the position of the "envelope header"
lets us escape it by replacing the first 5 characters of the stream
with a regular header name (with ':').
David Bremner [Sat, 5 Feb 2022 19:52:10 +0000 (15:52 -0400)]
test: start new corpus of test messages for indexing code
This particular message is not recognized by notmuch as mail, but is
fine according to e.g. mutt. The trigger for this bad behaviour seems
to be a second "From " ocurring at the beginning of the line but
inside an attachment.
David Bremner [Wed, 9 Feb 2022 01:01:16 +0000 (21:01 -0400)]
test: known broken test for list(db.config) in python-cffi bindings
As of notmuch 0.34.2 [1], the python-cffi bindings make available the
configuration from both a config file and the database when accessing
Database.config like a dictionary. It is therefore confusing that the
iterator operations only work on the configuration information stored
in the database.
David Bremner [Fri, 21 Jan 2022 23:38:53 +0000 (19:38 -0400)]
lib: strip trailing '/' from pathnames (sexp queries).
This changes makes the sexp query parser consistent with the infix one
in ignoring trailing '/'. Here we do a bit better and ignore any
number of trailing '/'.
David Bremner [Fri, 21 Jan 2022 23:38:52 +0000 (19:38 -0400)]
test: add multiple path, folder sexp query tests
This is mainly to make sure we get trailing / removal correct. Also
add regression test for path: in the infix parser matching the
existing one for folder:.
David Bremner [Fri, 21 Jan 2022 23:38:51 +0000 (19:38 -0400)]
lib: drop trailing slash for path and folder searches (infix)
This resolves an old bug reported by David Edmondson in 2014. The fix
is only needed for the "boolean" case, as probabilistic / phrase
searching already ignores punctuation.
This fix is only for the infix (xapian provided) query parser.
David Bremner [Fri, 21 Jan 2022 23:38:50 +0000 (19:38 -0400)]
test/sexp: tests for path, folder, including trailing '/' (sexp)
This duplicates the bug reported in [1], as well as adding some simple
regression tests for 'path' and 'folder' searches which were
previously missing for sexp syntax.
David Bremner [Fri, 21 Jan 2022 23:38:49 +0000 (19:38 -0400)]
test: known broken tests for trailing / in path search (infix)
In [1], David Edmondson observed that the trailing / added by many
completion mechanisms causes path searches to silently fail. This
test reproduces that bug for both `path:' and `folder:' searches.
David Bremner [Fri, 21 Jan 2022 23:38:48 +0000 (19:38 -0400)]
test: sanitize generated message files names
It is fragile to encode the generated names into tests, as it makes
tests break when e.g. new tests are added. There is a possibility
that this will hide certain failures; in that case meaningful filenames
should be chosen for the generated messages.
David Bremner [Thu, 20 Jan 2022 13:36:04 +0000 (09:36 -0400)]
test/sexpr-search: add known broken tests for lastmod queries
These are loosely modelled on the tests just above for date ranges,
since the error conditions are similar. Some ideas also borrowed from
T570-revision-tracking.
David Bremner [Thu, 20 Jan 2022 13:36:03 +0000 (09:36 -0400)]
lib/parse-sexp: support actual date queries.
The default argument processing overlaps somewhat with what is already
done in _notmuch_date_strings_to_query, but we can give more specific
error messages for the s-expression context.
The extra generality of _sexp_parse_range will be useful when we
implement additional range prefixes (at least 'lastmod' is needed).
David Bremner [Thu, 20 Jan 2022 03:22:07 +0000 (23:22 -0400)]
emacs: use cached encoded copy for fcc
This fixes the bug reported by dkg in [1]. The movement of the call to
n-m-setup-message-for-saving is so the cleanup of Fcc headers happens
in the encoded version (otherwise Fcc headers may be saved to disk).
David Bremner [Thu, 20 Jan 2022 01:23:30 +0000 (21:23 -0400)]
test/emacs: known broken test for matching fcc and sent message
Based on the method outlined by Daniel Kahn Gillmor in
id:87k1zm225v.fsf@fifthhorseman.net. With a delay of 0.2 seconds the
test becomes flaky on my machine. With a 1 second delay it fails
consistently for more than 1600 iterations.
David Bremner [Mon, 17 Jan 2022 23:54:52 +0000 (19:54 -0400)]
test/setup: add known broken test for single items
In [1] Ian observed that notmuch setup was inconsistent with notmuch
config set when adding single items, namely adding an unneeded
semi-colon at the end. This test replicates that bug.
jao [Sun, 23 Jan 2022 19:54:17 +0000 (19:54 +0000)]
emacs: customizable names for search buffers
Customizable names for buffers presenting search results, via two
custom variables (notmuch-search-buffer-name-format and
notmuch-saved-search-buffer-name-format), defaulting to values
currently used for plain searches and including too tree and
unthreaded search buffers.
David Bremner [Sat, 22 Jan 2022 17:44:40 +0000 (13:44 -0400)]
emacs: define, use option :disable-excludes for n-h-query-counts
Initially only use in notmuch-hello-insert-alltags. This is a more
narrow resolution of [1], which (unlike [2]) does not disable exclude
processing for regular saved searches.
David Bremner [Sat, 22 Jan 2022 17:44:39 +0000 (13:44 -0400)]
Revert "emacs: use --exclude=false when checking for empty searches"
Revert commit 8370e3cfe2dd8a79323613c2bbf2f11db6134dac, and remark the
corresponding test as broken. Also update the expected output of the
broken test to show excludes active in the user defined saved searches.
David Bremner [Sun, 9 Jan 2022 14:38:04 +0000 (10:38 -0400)]
lib/config: move g_key_File_get_string before continue
In [1] Austin Ray reported some memory leaks in
notmuch_database_open. One of those leaks is caused by jumping to the
next key without freeing val. This change avoids that leak.
David Bremner [Wed, 5 Jan 2022 01:06:05 +0000 (21:06 -0400)]
emacs: use --exclude=false when checking for empty searches
In particular tags that only occur on otherwise excluded messages do
not show up in "All tags" without this change. The reported numbers
_do_ take excludes into account, so it is possible that
searches (e.g. the aformentioned tags) will show up with a count of
zero.
David Bremner [Wed, 5 Jan 2022 01:06:04 +0000 (21:06 -0400)]
test/emacs: add known broken test for hidden tags
Gregor Zattler observed that tags could be unintentionally hidden in
the "All tags" view, and Tomi Ollia worked out [2] that the issue was tags
that only occured on excluded messages. This test reproduces that bug.
David Bremner [Tue, 18 Jan 2022 13:39:05 +0000 (09:39 -0400)]
test/emacs: known broken test for reply with extra headers set.
Although it makes sense for the extra headers to be added to the copy
of the message headers included in the sexp/json, it is a bit
surprising for them to show in the new message constructed for the
reply, especially when, as here, they are always missing/empty.
David Bremner [Sat, 1 Jan 2022 12:01:34 +0000 (08:01 -0400)]
CLI: stash pointer to database in sprinter structs
We already use an allocated (and presumably open) database as a talloc
context. Keeping the pointer in the allocated struct will allow us to
e.g. interrogate the configuration in a sprinter function without
threading the database all the way through the various levels of function.
David Bremner [Sun, 26 Dec 2021 12:24:10 +0000 (08:24 -0400)]
cli/new: only ignore .notmuch at top level
Since the bug was first reported in [1], notmuch has gained the
ability to have the database located outside the mail root, hence this
this change differs slightly from Jani's proposed solution [2] in not
using notmuch_database_get_path, but rather the already retrieved
mail_root.
David Bremner [Sun, 26 Dec 2021 12:24:09 +0000 (08:24 -0400)]
test: add known broken test for ignoring non-toplevel .notmuch
In [1] Rob observed that notmuch new ignored directories called
.notmuch everywhere in the tree, where they should only (and now, with
split configs, at most) be ignored at the top level. Add a test to
demonstrate the problem.
David Bremner [Sat, 25 Dec 2021 13:33:05 +0000 (09:33 -0400)]
lib/open: no default mail root in split configurations
If we know the configuration is split, but there is no mail root
defined, this indicates a (lack of) configuration error. Currently
this can only arise in XDG configurations.
David Bremner [Sat, 25 Dec 2021 13:33:02 +0000 (09:33 -0400)]
lib/open: use db struct as talloc ctx for choose_database_path
The extra talloc struct "local" was left over from before the notmuch
struct was allocated earlier. Having the notmuch struct available in
this function will allow more flexibility to track the configuration
variations (e.g. split vs. non-split).
David Bremner [Sat, 11 Dec 2021 12:49:12 +0000 (08:49 -0400)]
test/libconfig: add two tests for the config = "" case
If notmuch_database_open_with_config finds a database, but that
database is not in a legacy, non-split configuration, then it
currently incorrectly deduces the mail root and returns SUCCESS. Add
to two tests to demonstrate this bug.
David Bremner [Tue, 11 Jan 2022 23:52:30 +0000 (19:52 -0400)]
test/gpgsm: use --with-colons when calculating fingerprint.
As stressed by the gpg documentation, the non-'with-colons' output
format is subject to change, and indeed it did in 2.3.x (x<=3). This
should make the the test suite more robust against such changes.
jao [Mon, 10 Jan 2022 03:05:23 +0000 (03:05 +0000)]
emacs: make header line in notmuch-show buffers optional
New notmuch-show-header-line customizable boolean to allow inhibiting
a header line in notmuch-show-mode buffers (for instance, because one
prefers to just include Subject in notmuch-message-headers).
David Bremner [Sat, 8 Jan 2022 21:21:43 +0000 (17:21 -0400)]
bindings/python-cffi: search for config by default
The previous (pre-0.34.2) constructor searched for a config file but
only if the database path was not specified, and only to retrieve
database.path. Neither of the available options (CONFIG.SEARCH or
CONFIG.NONE) matches this semantics exactly, but CONFIG.SEARCH causes
less breakage for people who relied on the old behaviour to set their
database.path [1]. Since it also seems like the friendlier option in
the long run, this commit switches to CONFIG.SEARCH as default.
This requires a certain amount of updating the pytest tests, but most
users will actually have a config file, unlike the test environment.
Cédric Hannotier [Thu, 16 Dec 2021 17:40:53 +0000 (18:40 +0100)]
completion: use mail_root for path completion in bash/zsh
Since mail store and database directory can be different,
path and folder completions must use the directory given by
database.mail_root, not by database.path.
David Bremner [Sat, 1 Jan 2022 14:36:43 +0000 (10:36 -0400)]
bindings/python-cffi: add matched property to message objects
Existing users of the legacy python bindings use
message.get_flags(Message.FLAG.MATCH) to determine which messages in a
thread matched. Since the bindings don't provide get_flags anymore,
they should provide a property analogous to the existing "excluded"
property.
David Bremner [Fri, 24 Dec 2021 18:29:05 +0000 (14:29 -0400)]
build: move {C,CXX}FLAGS to the end of FINAL_{C,CXX}FLAGS
In addition to avoiding problems with user specified include paths
picking up an installed version of notmuch.h, this should also enable
users to override more options (in particular they could override
warning options since the last one takes effect).
David Bremner [Fri, 24 Dec 2021 18:29:03 +0000 (14:29 -0400)]
build: move LDFLAGS after notmuch libraries.
In [1] Ryan Schmidt reported a problem on macports [2] with notmuch
finding an existing installed version of libnotmuch during the build
when the user specified LDFLAGS including the libnotmuch install
directory.
This change should prevent that.
LDFLAGS also occurs in FINAL_LIBNOTMUCH_LDFLAGS. The only built
library linked to that is util/libtnotmuch_util.a, and that passed as
explicit (relative) path, and is thus not affected by -L.
David Bremner [Sat, 25 Dec 2021 23:31:12 +0000 (19:31 -0400)]
emacs: update coolj-line-prefix-regexp to make space optional
Jani reported that lines prefixed with '>' (as opposed to '> ') are
not highlighted properly [1]. David E responded with a updated regex
[2]. This change implements David E's suggestion.
David Bremner [Thu, 23 Dec 2021 01:44:45 +0000 (21:44 -0400)]
configure: calculate NOTMUCH_BUILDDIR, write to Makefile.config
This will correct the current use of an undefined variable when
setting LD_LIBRARY_PATH in doc/Makefile.local
It is tempting to try to replace the use of test/export-dirs.sh, but
this is not as simple as it looks, as NOTMUCH_BUILDDIR is used to
locate sh.config, so probably cannot also sensibly be used to define
it.
David Bremner [Sat, 25 Dec 2021 21:22:56 +0000 (17:22 -0400)]
lib/open: do not consider .notmuch alone as an existing database.
It makes perfect sense for users to want to pre-create .notmuch,
e.g. to install hooks, so we should handle the case of a .notmuch
directory without an actual xapian database more gracefully.
David Bremner [Sat, 25 Dec 2021 21:22:55 +0000 (17:22 -0400)]
test/new: add known broken test for missing xapian directory.
`notmuch new' should go ahead and create the xapian database if it is
missing, even in the case where the parent .notmuch (or equivalent)
directory exists.
David Bremner [Sun, 26 Dec 2021 16:20:44 +0000 (12:20 -0400)]
test: test folder renames
In [1] Mark Walters reported a problem with messages being removed
from the database when the parent directory was renamed. Jani Nikula
proposed [2] these tests but observed
This test is not suitable for merging since it's not deterministic.
After applying Jani's patch [3], the tests now pass deterministically,
and could usefully act as regression tests.
David Bremner [Fri, 24 Dec 2021 16:20:31 +0000 (12:20 -0400)]
doc: add dep. on stamp file for rebuilding gzipped man pages.
In [1] Daniel observed that the gzipped man pages were only being
rebuild every second time when building with `make -j4'. This may be
caused by a race condition between sphinx-build rebuilding the roff
files and the recipe to gzip them. This commit sequentializes these
two steps by making the stamp file a prerequisite for (all of) the
gzip files.