9 Commits (3e6dfcfb0529a9d003e07a56f58d0ff3c46556bf)

Author	SHA1	Message	Date
Alexander Korotkov	6ab1dbd26b	Add NOT NULL checking of pg_stat_statements_reset() in tests ... This is preliminary patch. It adds NOT NULL checking for the result of pg_stat_statements_reset() function. It is needed for upcoming patch "Track statement entry timestamp" that will change the result type of this function to the timestamp of a reset performed. Discussion: https://postgr.es/m/flat/72e80e7b160a6eb189df9ef6f068cce3765d37f8.camel%40moonset.ru Author: Andrei Zubkov Reviewed-by: Julien Rouhaud, Hayato Kuroda, Yuki Seino, Chengxi Sun Reviewed-by: Anton Melnikov, Darren Rush, Michael Paquier, Sergei Kornilov Reviewed-by: Alena Rybakina, Andrei Lepikhov	2 years ago
Michael Paquier	108161bcb9	pg_stat_statements: Remove duplicated tests for SET statements ... This looks like a copy-paste mistake introduced in de2aca2885, that has added checks for more patterns of SET statements while ignoring the original test block that existed. Backpatch down to where this has been introduced, as this shaves some cycles. Author: Sergei Kornilov Discussion: https://postgr.es/m/5689421699428803@mail-sendbernar-production-main-46.myt.yp-c.yandex.net Backpatch-through: 16	2 years ago
Michael Paquier	11c34b342b	Show parameters of CALL as constants in pg_stat_statements ... This commit changes the query jumbling of CallStmt so as its IN/OUT parameters are able to show up as constants with a parameter symbol in pg_stat_statements, like: CALL proc1($1, $2); CALL proc2($1, $2, $3); The transformed FuncExpr is used in the query ID computation instead of the FuncCall generated by the parser, so as it is sensitive to the OID of the procedure and its list of input arguments. The output arguments are handled in a separate list in CallStmt, which is also included in the computation. Tests are added to pg_stat_statements to show how this affects CALL with IN/OUT parameters as well as overloaded functions. Like 638d42a3c5 or 31de7e60da, this improves the monitoring of workloads with a lot of CALL statements, preventing unnecessary bloat when these use different input (or event output) values. Author: Sami Imseih Discussion: https://postgr.es/m/B44FA29D-EBD0-4DD9-ABC2-16F1CB087074@amazon.com	2 years ago
Michael Paquier	bb45156f34	Show names of DEALLOCATE as constants in pg_stat_statements ... This commit switches query jumbling so as prepared statement names are treated as constants in DeallocateStmt. A boolean field is added to DeallocateStmt to make a distinction between ALL and named prepared statements, as "name" was used to make this difference before, NULL meaning DEALLOCATE ALL. Prior to this commit, DEALLOCATE was not tracked in pg_stat_statements, for the reason that it was not possible to treat its name parameter as a constant. Now that query jumbling applies to all the utility nodes, this reason does not apply anymore. Like 638d42a3c5, this can be a huge advantage for monitoring where prepared statement names are randomly generated, preventing bloat in pg_stat_statements. A couple of tests are added to track the new behavior. Author: Dagfinn Ilmari Mannsåker, Michael Paquier Reviewed-by: Julien Rouhaud Discussion: https://postgr.es/m/ZMhT9kNtJJsHw6jK@paquier.xyz	2 years ago
Michael Paquier	638d42a3c5	Show GIDs of two-phase commit commands as constants in pg_stat_statements ... This relies on the "location" field added to TransactionStmt in 31de7e6, now applied to the "gid" field used by 2PC commands. These commands are now reported like: COMMIT PREPARED $1 PREPARE TRANSACTION $1 ROLLBACK PREPARED $1 Applying constants for these commands is a huge advantage for workloads that rely a lot on 2PC commands with different GIDs. Some tests are added to track the new behavior. Reviewed-by: Julien Rouhaud Discussion: https://postgr.es/m/ZMhT9kNtJJsHw6jK@paquier.xyz	2 years ago
Michael Paquier	31de7e60da	Show savepoint names as constants in pg_stat_statements ... In pg_stat_statements, savepoint names now show up as constants with a parameter symbol, using as base query string the one added as a new entry to the PGSS hash table, leading to: RELEASE $1 ROLLBACK TO $1 SAVEPOINT $1 Applying constants to these query parts is a huge advantage for workloads that generate randomly savepoint points, like ORMs (Django is at the origin of this patch). The ODBC driver is a second layer that likes a lot savepoints, though it does not use a random naming pattern. A "location" field is added to TransactionStmt, now set only for savepoints. The savepoint name is ignored by the query jumbling. The location can be extended to other query patterns, if required, like 2PC commands. Some tests are added to pg_stat_statements for all the query patterns supported by the parser. ROLLBACK, ROLLBACK TO SAVEPOINT and ROLLBACK TRANSACTION TO SAVEPOINT have the same Node representation, so all these are equivalents. The same happens for RELEASE and RELEASE SAVEPOINT. Author: Greg Sabino Mullane Discussion: https://postgr.es/m/CAKAnmm+2s9PA4OaumwMJReWHk8qvJ_-g1WqxDRDAN1BSUfxyTw@mail.gmail.com	2 years ago
Michael Paquier	9a714b9d6e	Improve cleanup phases in regression tests of pg_stat_statements ... As shaped, two DROP ROLE queries included in "user_activity" were showing in the reports for "wal". The intention is to keep each test isolated and independent, so this is incorrect. This commit adds some calls to pg_stat_statements_reset() to clean up the statistics once each test finishes, so as there are no risks of overlap in the reports for individial scenarios. The addition in "user_activity" fixes the output of "wal". The new resets done in "level_tracking" and "utility" are added for consistency with the rest, though they do not affect the stats generated in the other tests. Oversight in d0028e3. Reported-by: Andrei Zubkov Discussion: https://postgr.es/m/7beb722dd016bf54f1c78bfd6d44a684e28da624.camel@moonset.ru	3 years ago
Michael Paquier	de2aca2885	Expand regression tests of pg_stat_statements for utility queries ... This commit adds more coverage for utility statements so as it is possible to track down all the effects of query normalization done for all the queries that use either Const or A_Const nodes, which are the nodes where normalization makes the most sense as they apply to constants (well, most of the time, really). This set of queries is extracted from an analysis done while looking at full dumps of the regression database when applying different levels of normalization to either Const or A_Const nodes for utilities, as of a minimal set of these, for: - All relkinds (CREATE, ALTER, DROP) - Policies - Cursors - Triggers - Types - Rules - Statistics - CALL - Transaction statements (isolation level, options) - EXPLAIN - COPY Note that pg_stat_statements is not switched yet to show any normalization for utilities, still it improves the default coverage of the query jumbling code (not by as much as enabling query jumbling on the main regression test suite, though): - queryjumblefuncs.funcs.c: 36.8% => 48.5% - queryjumblefuncs.switch.c: 33.2% => 43.1% Reviewed-by: Bertrand Drouvot Discussion: https://postgr.es/m/Y+MRdEq9W9XVa2AB@paquier.xyz	3 years ago
Michael Paquier	e8dbdb15db	Refactor tests of pg_stat_statements for planning, utility and level tracking ... pg_stat_statements.sql acts as the main file for all the core tests of the module, but things have become complicated to follow over the years as some of the sub-scenarios tested in this file rely on assumptions that come from completely different areas of it, like a GUC setup or a relation created previously. For example, row tracking for CTAS/COPY was looking at the number of plans, which was not necessary, or level tracking was mixed with checks on planner counts. This commit refactors the tests of pg_stat_statements, by moving test cases out of pg_stat_statements.sql into their own file, as of: - Planning-related tests in planning.sql, for [re]plan counts and top-level handling. These depend on pg_stat_statements.track_planning. - Utilities in utility.sql (pg_stat_statements.track_utility), that includes now the tests for: -- Row tracking for CTAS, CREATE MATERIALIZED VIEW, COPY. -- Basic utility statements. -- SET statements. - Tracking level, depending on pg_stat_statements.track. This part has been looking at scenarios with DO blocks, PL functions and SQL functions. pg_stat_statements.sql (still named the same for now) still includes some checks for role-level tracking and WAL generation metrics, that ought to become independent in the long term for clarity. While on it, this switches the order of the attributes when querying pg_stat_statements, the query field becoming last. This makes much easier the tracking of changes related to normalization, as queries are the only variable-length attributes queried (unaligned mode would be one extra choice, but that reduces the checks on the other fields). Test scenarios and their results match exactly with what was happening before this commit in terms of calls, number of plans, number of rows, cached data or level tracking, so this has no effect on the coverage in terms of what is produced by the reports in the table pg_stat_statements. A follow-up patch will extend more the tests of pg_stat_statements around utilities, so this split creates a foundation for this purpose, without complicating more pg_stat_statements.sql. Reviewed-by: Bertrand Drouvot Discussion: https://postgr.es/m/Y+MRdEq9W9XVa2AB@paquier.xyz	3 years ago