63 Commits (ea2f97e9ed8317dc9a62e45d9a807dca7e3700a5)

Author	SHA1	Message	Date
Kim Alvefur	7e16a71be8	mod_storage_internal: Fix queries with only start returning extra items ... Queries with start > last item would return one item, because there's some boundary condition in binary_search(). This is here fixed by always applying filters that omit items outside the requested range. See also 2374c7665d0b	9 months ago
Matthew Wild	afb9bc34a0	mod_storage_internal: Use UUIDv7 for message ids ... This matches what we use for SQL already, so provides some consistency. Client developers prefer sortable ids.	10 months ago
Kim Alvefur	1768a2f369	mod_storage_internal: Fix off-by-one when searching archive for ... Fixes a test case provided by MattJ where the very first item matched by a 'start' timestamp was not returned.	2 years ago
Matthew Wild	4b0463968a	mod_storage_internal, tests: Fix before/after combined with the 'reverse' flag	2 years ago
Kim Alvefur	97332bf56f	mod_storage_internal: Clear archive item count cache after experimental trim ... The previous count would be invalid at this point. Should be possible to math out how many items are left, but this is left as future work.	2 years ago
Kim Alvefur	c47cfefec0	mod_storage_internal: Close lazy-loading list abstraction after trim ... Should be done here too.	2 years ago
Kim Alvefur	15f487b789	mod_storage_internal: Only close lazy-loading list store abstractions ... Since datamanager can fall back to the old method of loading the whole list, which wouldn't come with a :close method.	2 years ago
Kim Alvefur	58988fbce5	mod_storage_internal: Close archive list after completion of iteration ... This closes the two FDs that the random access list abstraction uses, otherwise they are left to the garbage collector.	2 years ago
Kim Alvefur	4d69962644	mod_storage_internal: Don't report error when attempting to trim empty archive ... Fixes "Could not delete messages for room 'x': (nil)"	2 years ago
Kim Alvefur	069bcf619b	mod_storage_internal: Fix fast trimming of archive with exactly one item ... This method would previously never delete the first (and only) item since it works out which item should become the first item after the trim operation, which doesn't make sense when all should be removed. This also works as an optimization for when all the last item should be trimmed, thus items should be removed.	2 years ago
Kim Alvefur	41ea0ee4a4	mod_storage_internal: Use integer option method for cache size ... Missed this one in previous sweep	2 years ago
Kim Alvefur	71ad48095d	plugins: Use integer config API with interval specification where sensible ... Many of these fall into a few categories: - util.cache size, must be >= 1 - byte or item counts that logically can't be negative - port numbers that should be in 1..0xffff	2 years ago
Kim Alvefur	d03a9b2f7e	mod_storage_internal: Implement efficient deletion of oldest archive items ... Using the new shift function in datamanager, either the oldest items are removed or all the later items are moved into a new file that replaces the old. Hidden behind a feature flag for now.	2 years ago
Kim Alvefur	7259b41e23	mod_storage_internal: Use a binary search for time based ranges ... Iterating over an entire archive to find a few items in the far end from where iteration started is expensive, and probably more expensive with the lazy-loading of items added in the previous commit. Since we can now efficiently read items in random order, we can now use a binary search to find a better starting point for iteration.	5 years ago
Kim Alvefur	f4be79f6d6	mod_storage_internal: Lazy-load archive items while iterating ... Very large list files previously ran into limits of the Lua parser, or just caused Prosody to freeze while parsing. Using the new index we can parse individual items one at a time. This probably won't reduce overall CPU usage, probably the opposite, but it will reduce the number of items in memory at once and allow collection of items after we iterated past them.	5 years ago
Kim Alvefur	98922d54b1	plugins: Prefix module imports with prosody namespace	3 years ago
Kim Alvefur	faa381e3ed	mod_storage_internal: Fix error in time limited queries on items without 'when' field, fixes #1557	6 years ago
Kim Alvefur	c6adacaaad	mod_storage_internal: Stop storing XEP-0091 timestamp ... Should no longer be used by anything since the conversion of mod_offline to the archive API in 0.10.0, which was 4 years ago. The line clearing the property is left for a bit longer in case someone has very old offline messages or archived data.	4 years ago
Kim Alvefur	344d8e7d51	mod_storage_internal: Support query for set of IDs ... Based on implementation in mod_storage_memory	5 years ago
Kim Alvefur	7a4f8adb70	mod_storage_internal: Add support for full ID range query ... Based on the mod_storage_memory implementation	5 years ago
Matthew Wild	0919bb30d3	mod_storage_internal, mod_storage_memory: Add support for query.before ... Previously returned all results.	6 years ago
Kim Alvefur	d916ce38f6	mod_storage_internal: Fix keeping old timestamp in archive map API ... This led to a missing 'when' field on changed items, which would cause a traceack.	6 years ago
Kim Alvefur	82714e54a8	mod_storage_internal: Implement key-value API	6 years ago
Kim Alvefur	d45b4c026f	mod_storage_internal: Include last text message ... A protocol built on this API now allows showing a list of unread conversations with a counter, ordered by either oldest or newest message, along with the text body itself.	6 years ago
Kim Alvefur	9d5e31fd48	mod_storage_*: Also include timestmap of first message in :summary API ... For completeness along with most recent timestamp. Might be nice to be able to order by oldest unread message.	6 years ago
Kim Alvefur	d3c559bcc8	mod_storage_*: Include timestamp of latest message in :summary API ... Clients may want to show a list of conversations ordered by how timestamp of most recent message. The counts allow a badge with unread message counter.	6 years ago
Kim Alvefur	39cb87a158	mod_storage_*: Tweak :summary API to allow future expansion with more fields ... Eg might want to include last message, timestamp, chat state or other info.	6 years ago
Kim Alvefur	99f6c69507	mod_storage_internal: Return appropriate error even with empty archive	7 years ago
Kim Alvefur	b772308c93	mod_storage_internal: Return error if 'before' or 'after' are not found (partial fix for #1325)	7 years ago
Kim Alvefur	ef95de8871	mod_storage_internal: Add support for iterating over users in archive stores ... May help with writing a better migrator	7 years ago
Kim Alvefur	637b52a13e	mod_storage_internal,memory: Only return total count if requested	7 years ago
Kim Alvefur	04d52fd099	mod_storage_internal: Return a noop iterator if archive is empty (fixes #920)	9 years ago
Kim Alvefur	d7789712da	mod_storage_internal: Handle case of empty item store when deleting (fixes #910)	9 years ago
Kim Alvefur	83741da83a	mod_storage_internal: Correctly calculate number of deleted items (fixes #912)	9 years ago
Kim Alvefur	2e1a6cd73e	mod_storage_internal: Support the 'key' archive query field	9 years ago
Kim Alvefur	52cff8b490	mod_storage_internal: Add the dates method	9 years ago
Kim Alvefur	8be8224bba	mod_storage_internal: Add support for removing archived items	9 years ago
Kim Alvefur	ffb3266efa	mod_storage_internal: Add basic archive store implementation	9 years ago
Kim Alvefur	fefbf23013	mod_storage_internal: Ignore unused 'self' argument [luacheck]	9 years ago
Kim Alvefur	b1d9296789	mod_storage_internal: Reorder methods	9 years ago
Kim Alvefur	0ca328a00c	mod_storage_internal: Separate driver from keyval implementation	9 years ago
Kim Alvefur	5a4e252a6c	mod_storage_internal: Only apply truncate if there are more items than requested	8 years ago
Kim Alvefur	b158553c43	mod_storage_internal: Skip write if no items matched a deletion query	8 years ago
Kim Alvefur	06addc9a73	mod_storage_internal: Optimize truncation	8 years ago
Kim Alvefur	cb209b0f1e	mod_storage_internal: Allow truncating deletion at the beginning or end of an archive store	8 years ago
Kim Alvefur	5091ef0c46	mod_storage_internal: Add more extensive query support to archive:delete method	8 years ago
Kim Alvefur	2477d1c269	mod_storage_internal: Add support for archive key deduplication (like mod_storage_sql)	8 years ago
Kim Alvefur	2ba9c6ce77	mod_storage_{none,internal,sql}: Return error for unsupported (everything but keyval) store types	12 years ago
Kim Alvefur	196022f3f5	mod_storage_internal, datamanager: Add support for iterating over users with data in a store	13 years ago
Kim Alvefur	07bb8a42ce	storagemanager, datamanager, mod_storage_{internal,sql}: Replace list_stores() with an iterator version	13 years ago