David Bremner [Sat, 30 Oct 2021 16:22:35 +0000 (13:22 -0300)]
python-cffi: switch to notmuch_database_{open,create}_with_config
Since release 0.32, libnotmuch provides searching for database and
configuration paths. This commit changes the python module notmuch2 to
use those facilities.
This fixes the bug reported in [1], along with a couple of the
deprecation warnings in the python bindings.
Database.default_path is deprecated, since it no longer faithfully
reflects what libnotmuch is doing, and it is also no longer used in
the bindings themselves.
This commit choose the default of config=CONFIG.EMPTY (equivalent to
passing "" to notmuch_database_open_with_config). This makes the
change upward compatible API-wise (at least as far as the test suite
verifies), but changing the default to CONFIG.SEARCH would probably be
more convenient for bindings users.
David Bremner [Thu, 28 Oct 2021 01:34:19 +0000 (22:34 -0300)]
lib/config: don't overwrite database.path if the caller passed it
If the user passed a path, and we opened it, then we consider that
definitive definition of "database.path". This makes libnotmuch
respond more gracefully to certain erroneous combinations of
NOTMUCH_CONFIG settings and config file contents.
David Bremner [Thu, 28 Oct 2021 01:34:17 +0000 (22:34 -0300)]
test: add known broken test for conflict with database parameter
This is arguably user error: having configuration file with bad
settings in it (and/or having a bad NOTMUCH_CONFIG environment
variable). On the other hand returning a different path than was
actually opened is definitely a bug.
David Bremner [Sat, 30 Oct 2021 18:49:20 +0000 (15:49 -0300)]
rename built_with.sexpr_query to built_with.sexp_queries
It is confusing to use two different names (sexp vs sexpr) when
compared with the command line option --query=sexp and (furthermore)
singular vs plural when compared with the man page title.
David Bremner [Mon, 11 Oct 2021 11:19:21 +0000 (08:19 -0300)]
emacs: add minimal docstring for notmuch-unthreaded
The missing docstring causes a blank in the notmuch-help display
[1]. Since the function is a simple wrapper for notmuch-tree, it seems
fair to forward the reader there for more detailed information.
David Bremner [Sat, 23 Oct 2021 13:22:37 +0000 (10:22 -0300)]
lib/load_config: document expectations for db on error, add tests
This is a bit different than n_d_{open,create}_with_config, since
there are several non-zero status codes where we do want to return a
non-NULL database structure.
David Bremner [Sat, 23 Oct 2021 13:22:36 +0000 (10:22 -0300)]
lib/create: fix memory leak, ensure *database=NULL on error
This code previously relied on _finish_open to free the notmuch struct
on errors (except for the case of database == NULL, which was a
potential double free). When we removed those frees from _finish_open,
we introduced a (small) memory leak.
In this commit, fix the memory leak, and harmonize the on-error
behaviour with n_d_open_with_config.
David Bremner [Sat, 23 Oct 2021 13:22:34 +0000 (10:22 -0300)]
lib/open: fix potential double-free, ensure *database=NULL on error
During refactoring for 0.32, the code that set notmuch=NULL on various
errors was moved into _finish_open. This meant that the the code which
relied on that to set *database to NULL on error was no longer
correct. It also introduced a potential double free, since the notmuch
struct was deallocated inside _finish_open (via n_d_destroy).
In this commit we revert to "allocator frees", and leave any cleanup
to the caller of _finish_open. This allows us to get back the
behaviour of setting *database to NULL with a small change. Other
callers of _finish_open will need free notmuch on errors.
David Bremner [Sat, 23 Oct 2021 13:22:33 +0000 (10:22 -0300)]
test: add two known broken tests for missing config files
The documentation claims that the database will be set to NULL in this
case, but it is currently not happening. Based on a reproducer [1]
from Austin Ray.
David Bremner [Mon, 11 Oct 2021 12:56:15 +0000 (09:56 -0300)]
doc/sexp-queries: replace definition lists with block quotes.
This document contains meaningful markup in the terms, which makeinfo
complains about. Replace the use of definition lists with regular
paragraphs containing quote blocks. This is accomplished by splitting
the "term" from the definition with a blank line.
David Bremner [Mon, 11 Oct 2021 12:56:14 +0000 (09:56 -0300)]
doc: remove explicit formatting of terms in definition lists
Sphinx-doc already formats the terms appropriately for a given
backend (bold in html and man). `makeinfo` complains noisily about
formatting inside a @item if we add our own explicit formatting.
This change may change the formatting in the info output. On the other
hand, the existing use of quotes for bold is not that great anyway.
In some places blank lines were removed to preserve the logical
structure of a definition list.
David Bremner [Sat, 12 Jun 2021 13:26:17 +0000 (10:26 -0300)]
CLI: move indexopts variable out of shared options block
This reduces the amount of global state. Furthermore, index options
can be set (in principle) in several ways, not just in the one
function for processing indexing command line options.
Jani Nikula [Wed, 13 Oct 2021 14:02:17 +0000 (17:02 +0300)]
cli: remove enum names from typedefs
There are some enum typedefs with the enum name:
typedef enum _name_t { ... } name_t;
We don't need or use the enum names _name_t for anything, and not all
of the enum typedefs have them. We have the typedefs specifically to
use the typedef name.
Jani Nikula [Wed, 13 Oct 2021 14:02:16 +0000 (17:02 +0300)]
lib: remove enum names from typedefs
There are some enum typedefs with the enum name:
typedef enum _name_t { ... } name_t;
We don't need or use the enum names _name_t for anything, and not all
of the enum typedefs have them. We have the typedefs specifically to
use the typedef name.
Jani Nikula [Wed, 13 Oct 2021 14:03:10 +0000 (17:03 +0300)]
lib: fix commented out NOTMUCH_DEPRECATED()
Remove the comment markers from the placeholder NOTMUCH_DEPRECATED(),
added in commit e5f3c3ed5024 ("lib: add stub for
notmuch_database_open_with_config").
David Bremner [Sun, 10 Oct 2021 11:47:58 +0000 (08:47 -0300)]
devel: script for checking a commit (series)
'check-notmuch-commit' is an updated version of a script I have been
using (although not always as consistently as I should) before sending
patches to the list.
Although it requires a bit more tooling, encouraging people to use
check-notmuch-commit might reduce the number of round trips to the
list for style nitpicks.
David Bremner [Thu, 7 Oct 2021 11:59:06 +0000 (08:59 -0300)]
configure: check explicitely for python dev (include) files
As discussed at [1] we have received reports that the implicit check
using cffi.FFI().verify() is not reliable in all environments. Since
we already use pkg-config, and the python dev package should include a
.pc file [2], add an extra check using pkg-config. On at least
Debian, we have to know which version of python dev files with are
looking for, so calculate that first.
[1]: id:87im1g35ey.fsf@tethera.netid:87im1g35ey.fsf@tethera.net,
[2]: checked on Debian and Fedora
David Bremner [Tue, 28 Sep 2021 01:30:08 +0000 (22:30 -0300)]
test/T355-smime: Use key as exported by gpgsm
As reported in id:87h7pxiek3.fsf@tethera.net, the previous version of
the test is flaky. There is some so-far undebugged interaction between
openssl and gpgsm that causes the keys to fail to import. As a
potential workaround, use the key as exported by gpgsm, and eliminate
openssl from this particular pipeline.
jao [Tue, 31 Aug 2021 01:21:06 +0000 (02:21 +0100)]
emacs: notmuch-tree: customizable strings for drawing trees
New customizable variable, notmuch-tree-thread-symbols, that allows
tweaking of how trees in a forest are represented. For instance, one
can now choose to use an hyphen rather than a white space as a prefix,
or replace the character(s) used to draw arrows.
David Bremner [Sun, 29 Aug 2021 19:23:30 +0000 (12:23 -0700)]
emacs: wrap call-process-region
As with notmuch--process-lines, initial purpose is to provide a safe
binding for default-directory. This is enough to make notmuch-hello
robust against non-existent or corrupt values default-directory, but
probably not other views.
David Bremner [Sun, 29 Aug 2021 19:23:27 +0000 (12:23 -0700)]
test/emacs: provide macro test-log-error
Because of the way emacs reports errors, a test form can crash and not
change the main buffer. To work around this, capture both signalled
errors and any other messages.
David Bremner [Thu, 9 Sep 2021 12:51:03 +0000 (09:51 -0300)]
lib: use 'localhost' rather than fqdn for default mail address.
As discussed in the thread starting at [1], the fully qualified domain
name is a bit tricky to get reproducibly, might reveal information
people prefer to keep private, and somewhat unlikely to provide
reliable mail routing.
The new approach of $current_username@localhost is better for the
first two considerations, and probably at least as good as a test mail
address.
David Edmondson [Sun, 21 Feb 2021 15:19:02 +0000 (15:19 +0000)]
emacs: Allow functions in notmuch-{tree,unthreaded}-result-format
If the car of an element in notmuch-tree-result-format or
notmuch-unthreaded-result-format is a function, insert the result of
calling the function into the buffer.
Jonas Bernoulli [Mon, 2 Aug 2021 22:10:08 +0000 (00:10 +0200)]
emacs: notmuch-fcc-header-setup: fix regression
With [1: 16b2db09] we lost the (undocumented) option to use no Fcc
header only for From addresses matching a regexp. This brings back
that feature and documents it.
David Bremner [Tue, 24 Aug 2021 15:17:44 +0000 (08:17 -0700)]
CLI/tag: enable sexp queries
We have to rewrite _optimize_tag_query here because it is generating
a query string in the infix Xapian syntax. Luckily this is easy to do
with the sexp query syntax.
The change in each case is to call notmuch_query_create_with_syntax,
relying on the already inherited shared options. As a bonus we get
improved error handling from the new query creation API.
The remaining subcommand is 'tag', which is a bit trickier.
David Bremner [Tue, 24 Aug 2021 15:17:41 +0000 (08:17 -0700)]
lib/parse-sexp: apply macros
Macros implement lazy evaluation and lexical scope. The former is
needed to make certain natural constructs work sensibly (e.g. (tag
,param)) but the latter is mainly future-proofing in case the DSL is
is extended to allow local bindings.
For technical background, see chapters 6 and 17 of [1] (or some other
intermediate programming languages textbook).
David Bremner [Tue, 24 Aug 2021 15:17:39 +0000 (08:17 -0700)]
lib/parse-sexp: support saved s-expression queries
It turns out there is not really much code in query-fp.cc useful for
supporting the new syntax. The code we could potentially factor out
amounts to calling notmuch_database_get_config; both the key
construction and the parsing of the results are specific to the query
syntax involved.
David Bremner [Tue, 24 Aug 2021 15:17:38 +0000 (08:17 -0700)]
CLI/config support saving s-expression queries
This commit does not enable using saved s-expression queries, only
saving and retrieving them from the config file or the database. Use
in queries will be enabled in a following commit.
David Bremner [Tue, 24 Aug 2021 15:17:37 +0000 (08:17 -0700)]
lib/parse-sexp: handle saved queries
This provides functionality analogous to query: in the Xapian
QueryParser based parser. Perhaps counterintuitively, the saved
queries currently have to be in the original query syntax (i.e. not
s-expressions).
David Bremner [Tue, 24 Aug 2021 15:17:35 +0000 (08:17 -0700)]
lib/parse-sexp: parse user headers
One subtle aspect is the replacement of _find_prefix with
_notmuch_database_prefix, which understands user headers. Otherwise
the code mainly consists of creating a fake prefix record (since the
user prefixes are not in the prefix table) and error handling.
David Bremner [Tue, 24 Aug 2021 15:17:34 +0000 (08:17 -0700)]
lib/parse-sexp: support infix subqueries
This is necessary so that programs can take infix syntax queries from
a user and use the sexp query syntax to construct e.g. a refinement of
that query.
David Bremner [Tue, 24 Aug 2021 15:17:32 +0000 (08:17 -0700)]
lib/thread-fp: factor out query expansion, rewrite in Xapian
It will be convenient not to have to construct a notmuch query object
when parsing subqueries, so the commit rewrites the query
expansion (currently only used for thread:{} queries) using only
Xapian. As a bonus it seems about 15% faster in initial experiments.
David Bremner [Tue, 24 Aug 2021 15:17:31 +0000 (08:17 -0700)]
lib/query: factor out _notmuch_query_string_to_xapian_query
When dealing with recursive queries (i.e. thread:{foo}) it turns out
to be useful just to deal with the underlying Xapian objects, and not
wrap them in notmuch objects.
David Bremner [Tue, 24 Aug 2021 15:17:30 +0000 (08:17 -0700)]
lib: generate actual Xapian query for "*" and ""
The previous code had the somewhat bizarre effect that the (notmuch
specific) query string was "*" (interpreted as MatchAll) and the
allegedly parsed xapian_query was "MatchNothing".
David Bremner [Tue, 24 Aug 2021 15:17:29 +0000 (08:17 -0700)]
lib/parse-sexp: support regular expressions
At least to the degree that the Xapian QueryParser based parser
also supports them. Support short alias 'rx' as it seems to make more
complex queries nicer to read.