master
Extra tests.
Improve some of the error messages emitted by fts5 when it encounters corruption.
Avoid corrupting the fts5 index if a value that was inserted via sqlite3_bind_blob() in a non-utf8 db is deleted.
Fix an incorrect tcl comment that appeared in many fts5 test files.
Clamp the nToken parameter to the fts5 snippet() function between 0 and 64. It has always been documented this way, but not previously implemented. Report [bugs:/info/2026-06-08T08:29:00Z | 2026-06-08T08:29:00Z].
Further tests and fixes for this branch.
Fix an OOB read caused by passing an out-of-range phrase number to fts5 auxiliary API xPhraseFirstColumn() in detail=columns mode.
Add extra tests for fts5, large rowids and "ORDER BY rowid DESC". No changes to code.
Change the error message text for SQLITE_ERROR to omit the part about "missing database" as that meaning is now obsolete (since approx SQLite 2.0).
Avoid a possible integer overflow when fts5 tokenizes a very large document. Only possible with non-standard builds that use large values of SQLITE_MAX_LENGTH. Bug [bugs:/info/2026-06-09T05:27:16Z | 2026-06-09T05:27:16Z].
Alternative implementation of fts5 locale=1 feature that allows blobs to be stored in indexed columns of fts5 locale=1 tables.
Add tests to restore coverage of fts5_tokenizer.c.
Fix a crash in fts5 that could occur if shadow tables are modified or removed.
Have fts5.xBestIndex return SQLITE_CONSTRAINT, instead of a large cost, when no usable plan can be found.
Enable "OR IGNORE" support via SQLITE_VTAB_CONSTRAINT_SUPPORT for internal-content (not contentless or external-content) fts5 tables.
Add the SQLITE_DEFAULT_DEFENSIVE compile-time option. Fix up test cases so that they work with DEFENSIVE enabled.
Improve the error message returned when fts5 finds a row is missing from its content table.
Add tests to restore coverage of fts5_config.c.
In fts5, avoid starting a new merge of level L if there exists already an ongoing merge of a level less than L.
Remove harmless "breakpoint" command inserted by the previous check-in
Avoid a possible buffer overrun in 32-bit builds of fts5 that could occur when processing corrupt records. Bug [bugs:/info/2026-06-14T12:21:15Z | 2026-06-14T12:21:15Z].
Fix another buffer overread in fts5 that could occur when processing corrupt records. Bug [bugs:/info/2026-06-03T03:54:40Z | 2026-06-03T03:54:40Z].
Optimize allocation of large tombstone arrays in fts5.
Fix further problems with deleting from corrupt fts5 databases in secure-delete mode.
Fix a buffer overread that could occur in fts5 within a memcmp() when doing an integrity-check on corrupted records.
Fix a problem in fts5 secure-delete code that could lead to a buffer overrun when dealing with a strategically corrupted database. Problem reported by YEONBA.
Avoid integer overflows (a) when dealing with zeroblob(N) calls when 2*N is larger than 2^31, and (b) in fts5 when a corrupt record is close to 2^31 bytes in size. Both of these require special builds with SQLITE_MAX_LENGTH set to a non-default value.
Fix a typo in fts5delete.test.
Fix a memory leak in fts5 that could occur when processing a query containing multiple classes of error.
Tcl_ChannelType implementations for Tcl9 apparently require that wideSeekProc be implemented. Also adjust minor test script issues for fts5 tests so that they can be run sequentially and so that they do not depend on the specific floating point output formats generated by Tcl.
Tests to improve coverage of fts5_expr.c.
Avoid reading the structure record from within the fts5 xConnect method.
Remove some 'breakpoint' commands in test files.
Ensure the fts5 xIntegrity method correctly returns error codes unrelated to corruption or missing SQL elements.
Fix a test problem causing an error in fts5fault9.test.
Allow fts5 to filter on multiple MATCH clauses in a single scan.
Add tests for the trigram tokenizer. Fix minor issues.
Allow fts5 trigram tables created with detail=column or detail=none to optimize LIKE and GLOB queries. Allow case-insensitive tables to optimize GLOB as well as LIKE.
Add tests and fixes for the new code on this branch.
Fix a problem with UPDATEs on fts5 tables that contain blob values.
Omit explain_i calls from TCL test scripts.
Fix a technically undefined signed integer overflow in fts5.
In fts5, avoid holding an open blob handle after an INSERT/UPDATE/DELETE statement, as this can interfere with sqlite3_interrupt(). Forum post [forum:/forumpost/95413eb410 | 95413eb410].
Update fts5 scan costs to take into account that a prefix query combined with a rowid equality constraint is still quite expensive.
Avoid evaluating special vtab operators (e.g. MATCH) that are part of ON clauses attached to left joins from being evaluated too early. Possible fix for [forum:/forumpost/428ef7c468 | forum post 428ef7c468].
Limit the number of nested NOT nodes in an fts5 expression to 256.
Fix an fts5 problem causing attempts to create locale=1 tables with more than 115 columns to fail. Bug [bugs:/info/2026-06-19T05:19:08Z | 2026-06-19T05:19:08Z].
Fix a problem in the fts5matchinfo.test file that prevented it from working on Mac.
Avoid signed integer overflow when an fts5 'merge' command is passed -2147483648 as a parameter.
Updates to where.c to handle vtabs in joins more similarly to regular tables.
Avoid a potential use-after-free in fts5. Report [bugs:/info/2026-06-08T08:45:27Z | 2026-06-08T08:45:27Z].
Allow an fts5 table or query to be configured to collect xInstToken data for any prefix terms as part of the first parse of the main index, if any.
Add tests for xInstToken() and prefix queries with various fts5 configurations.
Add tests for an fts5 NEAR() expression with a single argument phrase.
Avoid a potentially undefined left-shift in the fts5 porter tokenizer.
In fts5, account for the possibility of negative docids when allocating a buffer to store the results of merging detail=none prefix lists. Fix for report [bugs:/info/2026-08-04T08:49:27Z | 2026-08-04T08:49:27Z].
Add extra tests for [4b60a1c3].
Add the 'secure-delete' option to fts5. Used to configure fts5 to aggressively remove old full-text-index entries belonging to deleted or updated rows.
Fix a usan complaint about signed integer overflow.
Fix some problems with using fts5 options 'secure-delete' and detail=none together.
Fix an fts5 problem caused by a 'rebuild' followed by a DELETE in secure-delete mode.
Add tests for, and source code comments to, fts5. No changes to code.
Remove a stray "breakpoint" from a test script.
Approximately 100 typo corrections spanning the whole tree, submitted via [forum:0db9827f0464bc33|forum post 0db9827f0464bc33] and individually audited and verified. Affects only code comments, innocuous test strings, error message text in tool (not library) code, and configure-level help text.
Fix a problem that could cause fts5 to return incorrect answers when performing prefix queries on tokens with synonyms. Bug [bugs:/info/2026-06-14T11:11:06Z | 2026-06-14T11:11:06Z].
Prevent fts5 tokenizer unicode61 from considering '\0' to be a token characters, even if other characters of class "Cc" are.
Add a test to verify that the fts3 and fts5 "unicode61" tokenizers are byte for byte compatible.
Fix the fts5 xInstToken() API for prefix queries that do not use prefix-indexes. This is experimental.
Silently ignore nested porter tokenizers in FTS5. Having nested porter tokenizers is pointless, but it does use stack space unnecessarily. [bugs:/forumpost/a7766198f1|Bug report a7766198f1].
Improved robustness of parsing of tokenize= arguments in FTS5. [forum:/forumpost/171bcc2bcd|Forum post 171bcc2bcd].
Ensure tokenizers registered with xCreateTokenizer_v2() can be accessed using xFindTokenizer(), and that those registered with xCreateTokenizer() work with xFindTokenizer_v2().
Fix the fts5 trigram tokenizer so that it handles non-nul-terminated strings.
Fix a broken assert in fts5_expr.c.
Add the "remove_diacritics=2" option to the unicode61 tokenizer in both FTS5 and FTS3/4.
Allow UPDATEs of unindexed columns in fts5 contentless_unindexed=1 tables. Testing to come.
Have fts5 reject attempts to insert a non-integer, non-null value into a rowid column with SQLITE_MISMATCH.
Add tests for DELETE on contentless, contentless-delete and contentless-unindexed fts5 tables.
When removing an fts5 in secure-delete mode, defer setting the table version to SECUREDELETE until flushing data to disk. This prevents problems that can occur if there is a rollback or statement rollback operation.
Fix a problem in the xFilter() method of the fts5_vocab virtual table causing it to omit some result rows in some cases where a virtual cursor is used more than once by SQLite.