You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

Tom Lane 261f89a976 Track the maximum possible frequency of non-MCE array elements. ... The lossy-counting algorithm that ANALYZE uses to identify most-common array elements has a notion of cutoff frequency: elements with frequency greater than that are guaranteed to be collected, elements with smaller frequencies are not. In cases where we find fewer MCEs than the stats target would permit us to store, the cutoff frequency provides valuable additional information, to wit that there are no non-MCEs with frequency greater than that. What the selectivity estimation functions actually use the "minfreq" entry for is as a ceiling on the possible frequency of non-MCEs, so using the cutoff rather than the lowest stored MCE frequency provides a tighter bound and more accurate estimates. Therefore, instead of redundantly storing the minimum observed MCE frequency, store the cutoff frequency when there are fewer tracked values than we want. (When there are more, then of course we cannot assert that no non-stored elements are above the cutoff frequency, since we're throwing away some that are; so we still use the minimum stored frequency in that case.) Notably, this works even when none of the values are common enough to be called MCEs. In such cases we previously stored nothing in the STATISTIC_KIND_MCELEM pg_statistic slot, which resulted in the selectivity functions falling back to default estimates. So in that case we want to construct a STATISTIC_KIND_MCELEM entry that contains no "values" but does have "numbers", to wit the three extra numbers that the MCELEM entry type defines. A small obstacle is that update_attstats() has traditionally stored a null, not an empty array, when passed zero "values" for a slot. That gives rise to an MCELEM entry that get_attstatsslot() will spit up on. The least risky solution seems to be to adjust update_attstats() so that it will emit a non-null (but possibly empty) array when the passed stavalues array pointer isn't NULL, rather than conditioning that on numvalues > 0. In other existing cases I don't believe that that changes anything. For consistency, handle the stanumbers array the same way. In passing, improve the comments in routines that use STATISTIC_KIND_MCELEM data. Particularly, explain why we use minfreq / 2 not minfreq as the estimate for non-MCE values. Thanks to Matt Long for the suggestion that we could apply this idea even when there are more than zero MCEs. Reported-by: Mark Frost <FROSTMAR@uk.ibm.com> Reported-by: Matt Long <matt@mattlong.org> Author: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/PH3PPF1C905D6E6F24A5C1A1A1D8345B593E16FA@PH3PPF1C905D6E6.namprd15.prod.outlook.com		4 months ago
..
bench	Update copyright for 2025	1 year ago
data	Fix integer-overflow problem in intarray's g_int_decompress().	2 years ago
expected	Fix GIN's shimTriConsistentFn to not corrupt its input.	10 months ago
sql	Fix GIN's shimTriConsistentFn to not corrupt its input.	10 months ago
.gitignore	…
Makefile	Remove deprecated containment operators for contrib types.	5 years ago
_int.h	Specialize intarray sorting	11 months ago
_int_bool.c	Remove useless casts to (void *)	1 year ago
_int_gin.c	…
_int_gist.c	Fix integer-overflow problem in intarray's g_int_decompress().	2 years ago
_int_op.c	Add missing Datum conversions	6 months ago
_int_selfuncs.c	Track the maximum possible frequency of non-MCE array elements.	4 months ago
_int_tool.c	Specialize intarray sorting	11 months ago
_intbig_gist.c	Use new overflow-safe integer comparison functions.	2 years ago
intarray--1.0--1.1.sql	…
intarray--1.1--1.2.sql	…
intarray--1.2--1.3.sql	…
intarray--1.2.sql	…
intarray--1.3--1.4.sql	…
intarray--1.4--1.5.sql	Remove deprecated containment operators for contrib types.	5 years ago
intarray.control	Remove deprecated containment operators for contrib types.	5 years ago
meson.build	Update copyright for 2025	1 year ago