David Bremner [Sun, 10 Oct 2021 11:47:58 +0000 (08:47 -0300)]
devel: script for checking a commit (series)
'check-notmuch-commit' is an updated version of a script I have been
using (although not always as consistently as I should) before sending
patches to the list.
Although it requires a bit more tooling, encouraging people to use
check-notmuch-commit might reduce the number of round trips to the
list for style nitpicks.
David Bremner [Thu, 7 Oct 2021 11:59:06 +0000 (08:59 -0300)]
configure: check explicitely for python dev (include) files
As discussed at [1] we have received reports that the implicit check
using cffi.FFI().verify() is not reliable in all environments. Since
we already use pkg-config, and the python dev package should include a
.pc file [2], add an extra check using pkg-config. On at least
Debian, we have to know which version of python dev files with are
looking for, so calculate that first.
[1]: id:87im1g35ey.fsf@tethera.netid:87im1g35ey.fsf@tethera.net,
[2]: checked on Debian and Fedora
David Bremner [Tue, 28 Sep 2021 01:30:08 +0000 (22:30 -0300)]
test/T355-smime: Use key as exported by gpgsm
As reported in id:87h7pxiek3.fsf@tethera.net, the previous version of
the test is flaky. There is some so-far undebugged interaction between
openssl and gpgsm that causes the keys to fail to import. As a
potential workaround, use the key as exported by gpgsm, and eliminate
openssl from this particular pipeline.
jao [Tue, 31 Aug 2021 01:21:06 +0000 (02:21 +0100)]
emacs: notmuch-tree: customizable strings for drawing trees
New customizable variable, notmuch-tree-thread-symbols, that allows
tweaking of how trees in a forest are represented. For instance, one
can now choose to use an hyphen rather than a white space as a prefix,
or replace the character(s) used to draw arrows.
David Bremner [Sun, 29 Aug 2021 19:23:30 +0000 (12:23 -0700)]
emacs: wrap call-process-region
As with notmuch--process-lines, initial purpose is to provide a safe
binding for default-directory. This is enough to make notmuch-hello
robust against non-existent or corrupt values default-directory, but
probably not other views.
David Bremner [Sun, 29 Aug 2021 19:23:27 +0000 (12:23 -0700)]
test/emacs: provide macro test-log-error
Because of the way emacs reports errors, a test form can crash and not
change the main buffer. To work around this, capture both signalled
errors and any other messages.
David Bremner [Thu, 9 Sep 2021 12:51:03 +0000 (09:51 -0300)]
lib: use 'localhost' rather than fqdn for default mail address.
As discussed in the thread starting at [1], the fully qualified domain
name is a bit tricky to get reproducibly, might reveal information
people prefer to keep private, and somewhat unlikely to provide
reliable mail routing.
The new approach of $current_username@localhost is better for the
first two considerations, and probably at least as good as a test mail
address.
David Edmondson [Sun, 21 Feb 2021 15:19:02 +0000 (15:19 +0000)]
emacs: Allow functions in notmuch-{tree,unthreaded}-result-format
If the car of an element in notmuch-tree-result-format or
notmuch-unthreaded-result-format is a function, insert the result of
calling the function into the buffer.
Jonas Bernoulli [Mon, 2 Aug 2021 22:10:08 +0000 (00:10 +0200)]
emacs: notmuch-fcc-header-setup: fix regression
With [1: 16b2db09] we lost the (undocumented) option to use no Fcc
header only for From addresses matching a regexp. This brings back
that feature and documents it.
David Bremner [Tue, 24 Aug 2021 15:17:44 +0000 (08:17 -0700)]
CLI/tag: enable sexp queries
We have to rewrite _optimize_tag_query here because it is generating
a query string in the infix Xapian syntax. Luckily this is easy to do
with the sexp query syntax.
The change in each case is to call notmuch_query_create_with_syntax,
relying on the already inherited shared options. As a bonus we get
improved error handling from the new query creation API.
The remaining subcommand is 'tag', which is a bit trickier.
David Bremner [Tue, 24 Aug 2021 15:17:41 +0000 (08:17 -0700)]
lib/parse-sexp: apply macros
Macros implement lazy evaluation and lexical scope. The former is
needed to make certain natural constructs work sensibly (e.g. (tag
,param)) but the latter is mainly future-proofing in case the DSL is
is extended to allow local bindings.
For technical background, see chapters 6 and 17 of [1] (or some other
intermediate programming languages textbook).
David Bremner [Tue, 24 Aug 2021 15:17:39 +0000 (08:17 -0700)]
lib/parse-sexp: support saved s-expression queries
It turns out there is not really much code in query-fp.cc useful for
supporting the new syntax. The code we could potentially factor out
amounts to calling notmuch_database_get_config; both the key
construction and the parsing of the results are specific to the query
syntax involved.
David Bremner [Tue, 24 Aug 2021 15:17:38 +0000 (08:17 -0700)]
CLI/config support saving s-expression queries
This commit does not enable using saved s-expression queries, only
saving and retrieving them from the config file or the database. Use
in queries will be enabled in a following commit.
David Bremner [Tue, 24 Aug 2021 15:17:37 +0000 (08:17 -0700)]
lib/parse-sexp: handle saved queries
This provides functionality analogous to query: in the Xapian
QueryParser based parser. Perhaps counterintuitively, the saved
queries currently have to be in the original query syntax (i.e. not
s-expressions).
David Bremner [Tue, 24 Aug 2021 15:17:35 +0000 (08:17 -0700)]
lib/parse-sexp: parse user headers
One subtle aspect is the replacement of _find_prefix with
_notmuch_database_prefix, which understands user headers. Otherwise
the code mainly consists of creating a fake prefix record (since the
user prefixes are not in the prefix table) and error handling.
David Bremner [Tue, 24 Aug 2021 15:17:34 +0000 (08:17 -0700)]
lib/parse-sexp: support infix subqueries
This is necessary so that programs can take infix syntax queries from
a user and use the sexp query syntax to construct e.g. a refinement of
that query.
David Bremner [Tue, 24 Aug 2021 15:17:32 +0000 (08:17 -0700)]
lib/thread-fp: factor out query expansion, rewrite in Xapian
It will be convenient not to have to construct a notmuch query object
when parsing subqueries, so the commit rewrites the query
expansion (currently only used for thread:{} queries) using only
Xapian. As a bonus it seems about 15% faster in initial experiments.
David Bremner [Tue, 24 Aug 2021 15:17:31 +0000 (08:17 -0700)]
lib/query: factor out _notmuch_query_string_to_xapian_query
When dealing with recursive queries (i.e. thread:{foo}) it turns out
to be useful just to deal with the underlying Xapian objects, and not
wrap them in notmuch objects.
David Bremner [Tue, 24 Aug 2021 15:17:30 +0000 (08:17 -0700)]
lib: generate actual Xapian query for "*" and ""
The previous code had the somewhat bizarre effect that the (notmuch
specific) query string was "*" (interpreted as MatchAll) and the
allegedly parsed xapian_query was "MatchNothing".
David Bremner [Tue, 24 Aug 2021 15:17:29 +0000 (08:17 -0700)]
lib/parse-sexp: support regular expressions
At least to the degree that the Xapian QueryParser based parser
also supports them. Support short alias 'rx' as it seems to make more
complex queries nicer to read.
David Bremner [Tue, 24 Aug 2021 15:17:26 +0000 (08:17 -0700)]
lib/parse-sexp: handle unprefixed terms.
This is equivalent to adding the same field name "" for multiple
prefixes in the Xapian query parser, but we have to explicitely
construct the resulting query.
David Bremner [Tue, 24 Aug 2021 15:17:24 +0000 (08:17 -0700)]
lib/parse-sexp: 'starts-with' wildcard searches
The many tests potentially overkill, but they could catch typos in the
prefixes table. As a simplifying assumption, for now we assume a
single argument to the wildcard operator, as this matches the Xapian
semantics. The name 'starts-with' is chosen to emphasize the supported
case of wildcards in currrent (1.4.x) Xapian.
David Bremner [Tue, 24 Aug 2021 15:17:23 +0000 (08:17 -0700)]
lib/parse-sexp: add term prefix backed fields
We use "boolean" to describe fields that should generate terms
literally without stemming or phrase splitting. This terminology
might not be ideal but it is already enshrined in
notmuch-search-terms(7).
David Bremner [Tue, 24 Aug 2021 15:17:22 +0000 (08:17 -0700)]
lib/parse-sexp: support phrase queries.
Anything that is quoted or not purely word characters is considered a
phrase. Phrases are not stemmed, because the stems do not have
positional information in the database. It is less efficient to scan
the term twice, but it avoids a second pass to add prefixes, so maybe
it balances out. In any case, it seems unlikely query parsing is very
often a bottleneck.
David Bremner [Tue, 24 Aug 2021 15:17:19 +0000 (08:17 -0700)]
lib/parse-sexp: support and, not, and or.
All operations and (Xapian) fields will eventually have an entry in
the prefixes table. The flags field is just a placeholder for now, but
will eventually distinguish between various kinds of prefixes.
David Bremner [Tue, 24 Aug 2021 15:17:16 +0000 (08:17 -0700)]
lib/parse-sexp: parse single terms and the empty list.
There is not much of a parser here yet, but it already does some
useful error reporting. Most functionality sketched in the
documentation is not implemented yet; detailed documentation will
follow with the implementation.
David Bremner [Tue, 24 Aug 2021 15:17:13 +0000 (08:17 -0700)]
lib: define notmuch_query_create_with_syntax
Set the parsing syntax when the (notmuch) query object is
created. Initially the library always returns a trivial query that
matches all messages when using s-expression syntax.
It seems better to select the syntax at query creation time because
the lazy parsing is an implementation detail.
David Bremner [Tue, 24 Aug 2021 15:17:11 +0000 (08:17 -0700)]
configure: optional library sfsexp
The configure part is essentially the same as the other checks using
pkg-config. Since the optional inclusion of this feature changes what
options are available to the user, include it in the "built_with"
pseudo-configuration keys.
David Bremner [Tue, 24 Aug 2021 15:17:10 +0000 (08:17 -0700)]
CLI: make variable n_requested_db_uuid file scope.
It turns out that now that we pass an open database into the
subcommands, it is easy to check any requested uuid against the
database at the same time as we process the other shared
arguments. This results in overall less boilerplate code, as well as
making a CLI scope function and variable file scope in notmuch.c.
David Bremner [Sat, 4 Sep 2021 18:26:48 +0000 (11:26 -0700)]
debian: temporarily disable two subtests
These are failing on (surprisingly) the Debian amd64
autobuilder. There were also previous reports of failures on Ubuntu
s390x. Fixing this may require changing the way the default is
calculated.
David Bremner [Sun, 4 Jul 2021 13:16:38 +0000 (10:16 -0300)]
emacs: don't inline message/rfc822 parts without content
This avoids some ugly error messages and exceptions, and hopes that
some gnus method will display message/rfc822 parts that have only a
file, no :content part.
David Bremner [Sat, 3 Jul 2021 20:11:31 +0000 (17:11 -0300)]
test: known broken test for emacs display of message/rfc822 parts
There are at least 3 bugs present.
1) notmuch-show-insert-part-message/rfc822 assumes that message/rfc822
parts will have a ":content" property, but that turns out not to be
the case.
2) something deep in gnus wants gnus-newsgroup-charset, but that is
defined in gnus-sum, which is not loaded by default.
3) If gnus-sum is loaded, then the display of the message/rfc822 part
succeeds, but the buffer gets put into gnus-article-mode, which means
that, inter alia, notmuch text properties and keybindings get wiped.
lib: consider all instances of Delivered-To header
When using notmuch-reply and guessing the From: address from
Delivered-To headers, I had the wrong address chosen today. This was
because the messages from the notmuch list contain these headers in this
order:
Delivered-To: hannu.hartikainen@gmail.com
... Delivered-To: hannu@hrtk.in
In my .notmuch-config I have the following configuration:
Before this change, notmuch-reply would guess From: @gmail.com because
that is the first Delivered-To header present. After the change, the
primary address is chosen as I would expect.
Add a known broken subtest for guessing From: correctly when there are
multiple Delivered-To: headers. The address configured as primary_email
should get picked.