Refactor tests of pg_stat_statements for planning, utility and level tracking
pg_stat_statements.sql acts as the main file for all the core tests of
the module, but things have become complicated to follow over the years
as some of the sub-scenarios tested in this file rely on assumptions
that come from completely different areas of it, like a GUC setup or a
relation created previously. For example, row tracking for CTAS/COPY
was looking at the number of plans, which was not necessary, or level
tracking was mixed with checks on planner counts.
This commit refactors the tests of pg_stat_statements, by moving test
cases out of pg_stat_statements.sql into their own file, as of:
- Planning-related tests in planning.sql, for [re]plan counts and
top-level handling. These depend on pg_stat_statements.track_planning.
- Utilities in utility.sql (pg_stat_statements.track_utility), that
includes now the tests for:
-- Row tracking for CTAS, CREATE MATERIALIZED VIEW, COPY.
-- Basic utility statements.
-- SET statements.
- Tracking level, depending on pg_stat_statements.track. This part has
been looking at scenarios with DO blocks, PL functions and SQL
functions.
pg_stat_statements.sql (still named the same for now) still includes
some checks for role-level tracking and WAL generation metrics, that
ought to become independent in the long term for clarity.
While on it, this switches the order of the attributes when querying
pg_stat_statements, the query field becoming last. This makes much
easier the tracking of changes related to normalization, as queries are
the only variable-length attributes queried (unaligned mode would be one
extra choice, but that reduces the checks on the other fields).
Test scenarios and their results match exactly with what was happening
before this commit in terms of calls, number of plans, number of rows,
cached data or level tracking, so this has no effect on the coverage in
terms of what is produced by the reports in the table
pg_stat_statements. A follow-up patch will extend more the tests of
pg_stat_statements around utilities, so this split creates a foundation
for this purpose, without complicating more pg_stat_statements.sql.
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/Y+MRdEq9W9XVa2AB@paquier.xyz
3 years ago
|
|
|
--
|
|
|
|
-- Statement level tracking
|
|
|
|
--
|
|
|
|
SET pg_stat_statements.track_utility = TRUE;
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
Refactor tests of pg_stat_statements for planning, utility and level tracking
pg_stat_statements.sql acts as the main file for all the core tests of
the module, but things have become complicated to follow over the years
as some of the sub-scenarios tested in this file rely on assumptions
that come from completely different areas of it, like a GUC setup or a
relation created previously. For example, row tracking for CTAS/COPY
was looking at the number of plans, which was not necessary, or level
tracking was mixed with checks on planner counts.
This commit refactors the tests of pg_stat_statements, by moving test
cases out of pg_stat_statements.sql into their own file, as of:
- Planning-related tests in planning.sql, for [re]plan counts and
top-level handling. These depend on pg_stat_statements.track_planning.
- Utilities in utility.sql (pg_stat_statements.track_utility), that
includes now the tests for:
-- Row tracking for CTAS, CREATE MATERIALIZED VIEW, COPY.
-- Basic utility statements.
-- SET statements.
- Tracking level, depending on pg_stat_statements.track. This part has
been looking at scenarios with DO blocks, PL functions and SQL
functions.
pg_stat_statements.sql (still named the same for now) still includes
some checks for role-level tracking and WAL generation metrics, that
ought to become independent in the long term for clarity.
While on it, this switches the order of the attributes when querying
pg_stat_statements, the query field becoming last. This makes much
easier the tracking of changes related to normalization, as queries are
the only variable-length attributes queried (unaligned mode would be one
extra choice, but that reduces the checks on the other fields).
Test scenarios and their results match exactly with what was happening
before this commit in terms of calls, number of plans, number of rows,
cached data or level tracking, so this has no effect on the coverage in
terms of what is produced by the reports in the table
pg_stat_statements. A follow-up patch will extend more the tests of
pg_stat_statements around utilities, so this split creates a foundation
for this purpose, without complicating more pg_stat_statements.sql.
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/Y+MRdEq9W9XVa2AB@paquier.xyz
3 years ago
|
|
|
(1 row)
|
|
|
|
|
|
|
|
-- DO block - top-level tracking.
|
|
|
|
CREATE TABLE stats_track_tab (x int);
|
|
|
|
SET pg_stat_statements.track = 'top';
|
|
|
|
DELETE FROM stats_track_tab;
|
|
|
|
DO $$
|
|
|
|
BEGIN
|
|
|
|
DELETE FROM stats_track_tab;
|
|
|
|
END;
|
|
|
|
$$ LANGUAGE plpgsql;
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
WHERE query LIKE '%DELETE%' ORDER BY query COLLATE "C", toplevel;
|
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+--------------------------------
|
|
|
|
t | 1 | DELETE FROM stats_track_tab
|
|
|
|
t | 1 | DO $$ +
|
|
|
|
| | BEGIN +
|
|
|
|
| | DELETE FROM stats_track_tab;+
|
|
|
|
| | END; +
|
|
|
|
| | $$ LANGUAGE plpgsql
|
|
|
|
(2 rows)
|
|
|
|
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
Refactor tests of pg_stat_statements for planning, utility and level tracking
pg_stat_statements.sql acts as the main file for all the core tests of
the module, but things have become complicated to follow over the years
as some of the sub-scenarios tested in this file rely on assumptions
that come from completely different areas of it, like a GUC setup or a
relation created previously. For example, row tracking for CTAS/COPY
was looking at the number of plans, which was not necessary, or level
tracking was mixed with checks on planner counts.
This commit refactors the tests of pg_stat_statements, by moving test
cases out of pg_stat_statements.sql into their own file, as of:
- Planning-related tests in planning.sql, for [re]plan counts and
top-level handling. These depend on pg_stat_statements.track_planning.
- Utilities in utility.sql (pg_stat_statements.track_utility), that
includes now the tests for:
-- Row tracking for CTAS, CREATE MATERIALIZED VIEW, COPY.
-- Basic utility statements.
-- SET statements.
- Tracking level, depending on pg_stat_statements.track. This part has
been looking at scenarios with DO blocks, PL functions and SQL
functions.
pg_stat_statements.sql (still named the same for now) still includes
some checks for role-level tracking and WAL generation metrics, that
ought to become independent in the long term for clarity.
While on it, this switches the order of the attributes when querying
pg_stat_statements, the query field becoming last. This makes much
easier the tracking of changes related to normalization, as queries are
the only variable-length attributes queried (unaligned mode would be one
extra choice, but that reduces the checks on the other fields).
Test scenarios and their results match exactly with what was happening
before this commit in terms of calls, number of plans, number of rows,
cached data or level tracking, so this has no effect on the coverage in
terms of what is produced by the reports in the table
pg_stat_statements. A follow-up patch will extend more the tests of
pg_stat_statements around utilities, so this split creates a foundation
for this purpose, without complicating more pg_stat_statements.sql.
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/Y+MRdEq9W9XVa2AB@paquier.xyz
3 years ago
|
|
|
(1 row)
|
|
|
|
|
|
|
|
-- DO block - all-level tracking.
|
|
|
|
SET pg_stat_statements.track = 'all';
|
|
|
|
DELETE FROM stats_track_tab;
|
|
|
|
DO $$
|
|
|
|
BEGIN
|
|
|
|
DELETE FROM stats_track_tab;
|
|
|
|
END; $$;
|
|
|
|
DO LANGUAGE plpgsql $$
|
|
|
|
BEGIN
|
|
|
|
-- this is a SELECT
|
|
|
|
PERFORM 'hello world'::TEXT;
|
|
|
|
END; $$;
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C", toplevel;
|
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+----------------------------------------------------
|
Refactor tests of pg_stat_statements for planning, utility and level tracking
pg_stat_statements.sql acts as the main file for all the core tests of
the module, but things have become complicated to follow over the years
as some of the sub-scenarios tested in this file rely on assumptions
that come from completely different areas of it, like a GUC setup or a
relation created previously. For example, row tracking for CTAS/COPY
was looking at the number of plans, which was not necessary, or level
tracking was mixed with checks on planner counts.
This commit refactors the tests of pg_stat_statements, by moving test
cases out of pg_stat_statements.sql into their own file, as of:
- Planning-related tests in planning.sql, for [re]plan counts and
top-level handling. These depend on pg_stat_statements.track_planning.
- Utilities in utility.sql (pg_stat_statements.track_utility), that
includes now the tests for:
-- Row tracking for CTAS, CREATE MATERIALIZED VIEW, COPY.
-- Basic utility statements.
-- SET statements.
- Tracking level, depending on pg_stat_statements.track. This part has
been looking at scenarios with DO blocks, PL functions and SQL
functions.
pg_stat_statements.sql (still named the same for now) still includes
some checks for role-level tracking and WAL generation metrics, that
ought to become independent in the long term for clarity.
While on it, this switches the order of the attributes when querying
pg_stat_statements, the query field becoming last. This makes much
easier the tracking of changes related to normalization, as queries are
the only variable-length attributes queried (unaligned mode would be one
extra choice, but that reduces the checks on the other fields).
Test scenarios and their results match exactly with what was happening
before this commit in terms of calls, number of plans, number of rows,
cached data or level tracking, so this has no effect on the coverage in
terms of what is produced by the reports in the table
pg_stat_statements. A follow-up patch will extend more the tests of
pg_stat_statements around utilities, so this split creates a foundation
for this purpose, without complicating more pg_stat_statements.sql.
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/Y+MRdEq9W9XVa2AB@paquier.xyz
3 years ago
|
|
|
f | 1 | DELETE FROM stats_track_tab
|
|
|
|
t | 1 | DELETE FROM stats_track_tab
|
|
|
|
t | 1 | DO $$ +
|
|
|
|
| | BEGIN +
|
|
|
|
| | DELETE FROM stats_track_tab; +
|
Refactor tests of pg_stat_statements for planning, utility and level tracking
pg_stat_statements.sql acts as the main file for all the core tests of
the module, but things have become complicated to follow over the years
as some of the sub-scenarios tested in this file rely on assumptions
that come from completely different areas of it, like a GUC setup or a
relation created previously. For example, row tracking for CTAS/COPY
was looking at the number of plans, which was not necessary, or level
tracking was mixed with checks on planner counts.
This commit refactors the tests of pg_stat_statements, by moving test
cases out of pg_stat_statements.sql into their own file, as of:
- Planning-related tests in planning.sql, for [re]plan counts and
top-level handling. These depend on pg_stat_statements.track_planning.
- Utilities in utility.sql (pg_stat_statements.track_utility), that
includes now the tests for:
-- Row tracking for CTAS, CREATE MATERIALIZED VIEW, COPY.
-- Basic utility statements.
-- SET statements.
- Tracking level, depending on pg_stat_statements.track. This part has
been looking at scenarios with DO blocks, PL functions and SQL
functions.
pg_stat_statements.sql (still named the same for now) still includes
some checks for role-level tracking and WAL generation metrics, that
ought to become independent in the long term for clarity.
While on it, this switches the order of the attributes when querying
pg_stat_statements, the query field becoming last. This makes much
easier the tracking of changes related to normalization, as queries are
the only variable-length attributes queried (unaligned mode would be one
extra choice, but that reduces the checks on the other fields).
Test scenarios and their results match exactly with what was happening
before this commit in terms of calls, number of plans, number of rows,
cached data or level tracking, so this has no effect on the coverage in
terms of what is produced by the reports in the table
pg_stat_statements. A follow-up patch will extend more the tests of
pg_stat_statements around utilities, so this split creates a foundation
for this purpose, without complicating more pg_stat_statements.sql.
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/Y+MRdEq9W9XVa2AB@paquier.xyz
3 years ago
|
|
|
| | END; $$
|
|
|
|
t | 1 | DO LANGUAGE plpgsql $$ +
|
|
|
|
| | BEGIN +
|
|
|
|
| | -- this is a SELECT +
|
|
|
|
| | PERFORM 'hello world'::TEXT; +
|
Refactor tests of pg_stat_statements for planning, utility and level tracking
pg_stat_statements.sql acts as the main file for all the core tests of
the module, but things have become complicated to follow over the years
as some of the sub-scenarios tested in this file rely on assumptions
that come from completely different areas of it, like a GUC setup or a
relation created previously. For example, row tracking for CTAS/COPY
was looking at the number of plans, which was not necessary, or level
tracking was mixed with checks on planner counts.
This commit refactors the tests of pg_stat_statements, by moving test
cases out of pg_stat_statements.sql into their own file, as of:
- Planning-related tests in planning.sql, for [re]plan counts and
top-level handling. These depend on pg_stat_statements.track_planning.
- Utilities in utility.sql (pg_stat_statements.track_utility), that
includes now the tests for:
-- Row tracking for CTAS, CREATE MATERIALIZED VIEW, COPY.
-- Basic utility statements.
-- SET statements.
- Tracking level, depending on pg_stat_statements.track. This part has
been looking at scenarios with DO blocks, PL functions and SQL
functions.
pg_stat_statements.sql (still named the same for now) still includes
some checks for role-level tracking and WAL generation metrics, that
ought to become independent in the long term for clarity.
While on it, this switches the order of the attributes when querying
pg_stat_statements, the query field becoming last. This makes much
easier the tracking of changes related to normalization, as queries are
the only variable-length attributes queried (unaligned mode would be one
extra choice, but that reduces the checks on the other fields).
Test scenarios and their results match exactly with what was happening
before this commit in terms of calls, number of plans, number of rows,
cached data or level tracking, so this has no effect on the coverage in
terms of what is produced by the reports in the table
pg_stat_statements. A follow-up patch will extend more the tests of
pg_stat_statements around utilities, so this split creates a foundation
for this purpose, without complicating more pg_stat_statements.sql.
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/Y+MRdEq9W9XVa2AB@paquier.xyz
3 years ago
|
|
|
| | END; $$
|
|
|
|
f | 1 | SELECT $1::TEXT
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
Show values of SET statements as constants in pg_stat_statements
This is a continuation of work like 11c34b342bd7, done to reduce the
bloat of pg_stat_statements by applying more normalization to query
entries. This commit is able to detect and normalize values in
VariableSetStmt, resulting in:
SET conf_param = $1
Compared to other parse nodes, VariableSetStmt is embedded in much more
places in the parser, impacting many query patterns in
pg_stat_statements. A custom jumble function is used, with an extra
field in the node to decide if arguments should be included in the
jumbling or not, a location field being not enough for this purpose.
This approach allows for a finer tuning.
Clauses relying on one or more keywords are not normalized, for example:
* DEFAULT
* FROM CURRENT
* List of keywords. SET SESSION CHARACTERISTICS AS TRANSACTION,
where it is critical to differentiate different sets of options, is a
good example of why normalization should not happen.
Some queries use VariableSetStmt for some subclauses with SET, that also
have their values normalized:
- ALTER DATABASE
- ALTER ROLE
- ALTER SYSTEM
- CREATE/ALTER FUNCTION
ba90eac7a995 has added test coverage for most of the existing SET
patterns. The expected output of these tests shows the difference this
commit creates. Normalization could be perhaps applied to more portions
of the grammar but what is done here is conservative, and good enough as
a starting point.
Author: Greg Sabino Mullane, Michael Paquier
Discussion: https://postgr.es/m/36e5bffe-e989-194f-85c8-06e7bc88e6f7@amazon.com
Discussion: https://postgr.es/m/B44FA29D-EBD0-4DD9-ABC2-16F1CB087074@amazon.com
Discussion: https://postgr.es/m/CAKAnmmJtJY2jzQN91=2QAD2eAJAA-Per61eyO48-TyxEg-q0Rg@mail.gmail.com
12 months ago
|
|
|
t | 1 | SET pg_stat_statements.track = $1
|
Refactor tests of pg_stat_statements for planning, utility and level tracking
pg_stat_statements.sql acts as the main file for all the core tests of
the module, but things have become complicated to follow over the years
as some of the sub-scenarios tested in this file rely on assumptions
that come from completely different areas of it, like a GUC setup or a
relation created previously. For example, row tracking for CTAS/COPY
was looking at the number of plans, which was not necessary, or level
tracking was mixed with checks on planner counts.
This commit refactors the tests of pg_stat_statements, by moving test
cases out of pg_stat_statements.sql into their own file, as of:
- Planning-related tests in planning.sql, for [re]plan counts and
top-level handling. These depend on pg_stat_statements.track_planning.
- Utilities in utility.sql (pg_stat_statements.track_utility), that
includes now the tests for:
-- Row tracking for CTAS, CREATE MATERIALIZED VIEW, COPY.
-- Basic utility statements.
-- SET statements.
- Tracking level, depending on pg_stat_statements.track. This part has
been looking at scenarios with DO blocks, PL functions and SQL
functions.
pg_stat_statements.sql (still named the same for now) still includes
some checks for role-level tracking and WAL generation metrics, that
ought to become independent in the long term for clarity.
While on it, this switches the order of the attributes when querying
pg_stat_statements, the query field becoming last. This makes much
easier the tracking of changes related to normalization, as queries are
the only variable-length attributes queried (unaligned mode would be one
extra choice, but that reduces the checks on the other fields).
Test scenarios and their results match exactly with what was happening
before this commit in terms of calls, number of plans, number of rows,
cached data or level tracking, so this has no effect on the coverage in
terms of what is produced by the reports in the table
pg_stat_statements. A follow-up patch will extend more the tests of
pg_stat_statements around utilities, so this split creates a foundation
for this purpose, without complicating more pg_stat_statements.sql.
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/Y+MRdEq9W9XVa2AB@paquier.xyz
3 years ago
|
|
|
(7 rows)
|
|
|
|
|
|
|
|
-- Procedure with multiple utility statements.
|
|
|
|
CREATE OR REPLACE PROCEDURE proc_with_utility_stmt()
|
|
|
|
LANGUAGE SQL
|
|
|
|
AS $$
|
|
|
|
SHOW pg_stat_statements.track;
|
|
|
|
show pg_stat_statements.track;
|
|
|
|
SHOW pg_stat_statements.track_utility;
|
|
|
|
$$;
|
|
|
|
SET pg_stat_statements.track_utility = TRUE;
|
|
|
|
-- all-level tracking.
|
|
|
|
SET pg_stat_statements.track = 'all';
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
CALL proc_with_utility_stmt();
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C", toplevel;
|
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+----------------------------------------------------
|
|
|
|
t | 1 | CALL proc_with_utility_stmt()
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
|
|
|
f | 2 | SHOW pg_stat_statements.track
|
|
|
|
f | 1 | SHOW pg_stat_statements.track_utility
|
|
|
|
(4 rows)
|
|
|
|
|
|
|
|
-- top-level tracking.
|
|
|
|
SET pg_stat_statements.track = 'top';
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
CALL proc_with_utility_stmt();
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C", toplevel;
|
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+----------------------------------------------------
|
|
|
|
t | 1 | CALL proc_with_utility_stmt()
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
|
|
|
(2 rows)
|
|
|
|
|
|
|
|
-- EXPLAIN - all-level tracking.
|
|
|
|
CREATE TABLE test_table (x int);
|
|
|
|
SET pg_stat_statements.track = 'all';
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) SELECT 1;
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) (SELECT 1, 2);
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) TABLE stats_track_tab;
|
|
|
|
QUERY PLAN
|
|
|
|
-----------------------------
|
|
|
|
Seq Scan on stats_track_tab
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) (TABLE test_table);
|
|
|
|
QUERY PLAN
|
|
|
|
------------------------
|
|
|
|
Seq Scan on test_table
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) VALUES (1);
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) (VALUES (1, 2));
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) UPDATE stats_track_tab SET x = 1 WHERE x = 1;
|
|
|
|
QUERY PLAN
|
|
|
|
-----------------------------------
|
|
|
|
Update on stats_track_tab
|
|
|
|
-> Seq Scan on stats_track_tab
|
|
|
|
Filter: (x = 1)
|
|
|
|
(3 rows)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) DELETE FROM stats_track_tab;
|
|
|
|
QUERY PLAN
|
|
|
|
-----------------------------------
|
|
|
|
Delete on stats_track_tab
|
|
|
|
-> Seq Scan on stats_track_tab
|
|
|
|
(2 rows)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) INSERT INTO stats_track_tab VALUES ((1));
|
|
|
|
QUERY PLAN
|
|
|
|
---------------------------
|
|
|
|
Insert on stats_track_tab
|
|
|
|
-> Result
|
|
|
|
(2 rows)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) MERGE INTO stats_track_tab
|
|
|
|
USING (SELECT id FROM generate_series(1, 10) id) ON x = id
|
|
|
|
WHEN MATCHED THEN UPDATE SET x = id
|
|
|
|
WHEN NOT MATCHED THEN INSERT (x) VALUES (id);
|
|
|
|
QUERY PLAN
|
|
|
|
-------------------------------------------------------
|
|
|
|
Merge on stats_track_tab
|
|
|
|
-> Hash Right Join
|
|
|
|
Hash Cond: (stats_track_tab.x = id.id)
|
|
|
|
-> Seq Scan on stats_track_tab
|
|
|
|
-> Hash
|
|
|
|
-> Function Scan on generate_series id
|
|
|
|
(6 rows)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) SELECT 1 UNION SELECT 2;
|
|
|
|
QUERY PLAN
|
|
|
|
--------------------------
|
|
|
|
Unique
|
|
|
|
-> Sort
|
|
|
|
Sort Key: (1)
|
|
|
|
-> Append
|
|
|
|
-> Result
|
|
|
|
-> Result
|
|
|
|
(6 rows)
|
|
|
|
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C";
|
Track more precisely query locations for nested statements
Previously, a Query generated through the transform phase would have
unset stmt_location, tracking the starting point of a query string.
Extensions relying on the statement location to extract its relevant
parts in the source text string would fallback to use the whole
statement instead, leading to confusing results like in
pg_stat_statements for queries relying on nested queries, like:
- EXPLAIN, with top-level and nested query using the same query string,
and a query ID coming from the nested query when the non-top-level
entry.
- Multi-statements, with only partial portions of queries being
normalized.
- COPY TO with a query, SELECT or DMLs.
This patch improves things by keeping track of the statement locations
and propagate it to Query during transform, allowing PGSS to only show
the relevant part of the query for nested query. This leads to less
bloat in entries for non-top-level entries, as queries can now be
grouped within the same (toplevel, queryid) duos in pg_stat_statements.
The result gives a stricter one-one mapping between query IDs and its
query strings.
The regression tests introduced in 45e0ba30fc40 produce differences
reflecting the new logic.
Author: Anthonin Bonnefoy
Reviewed-by: Michael Paquier, Jian He
Discussion: https://postgr.es/m/CAO6_XqqM6S9bQ2qd=75W+yKATwoazxSNhv5sjW06fjGAtHbTUA@mail.gmail.com
11 months ago
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+--------------------------------------------------------------------
|
|
|
|
f | 1 | DELETE FROM stats_track_tab
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) (SELECT $1, $2)
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) (TABLE test_table)
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) (VALUES ($1, $2))
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) DELETE FROM stats_track_tab
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) INSERT INTO stats_track_tab VALUES (($1))
|
Track more precisely query locations for nested statements
Previously, a Query generated through the transform phase would have
unset stmt_location, tracking the starting point of a query string.
Extensions relying on the statement location to extract its relevant
parts in the source text string would fallback to use the whole
statement instead, leading to confusing results like in
pg_stat_statements for queries relying on nested queries, like:
- EXPLAIN, with top-level and nested query using the same query string,
and a query ID coming from the nested query when the non-top-level
entry.
- Multi-statements, with only partial portions of queries being
normalized.
- COPY TO with a query, SELECT or DMLs.
This patch improves things by keeping track of the statement locations
and propagate it to Query during transform, allowing PGSS to only show
the relevant part of the query for nested query. This leads to less
bloat in entries for non-top-level entries, as queries can now be
grouped within the same (toplevel, queryid) duos in pg_stat_statements.
The result gives a stricter one-one mapping between query IDs and its
query strings.
The regression tests introduced in 45e0ba30fc40 produce differences
reflecting the new logic.
Author: Anthonin Bonnefoy
Reviewed-by: Michael Paquier, Jian He
Discussion: https://postgr.es/m/CAO6_XqqM6S9bQ2qd=75W+yKATwoazxSNhv5sjW06fjGAtHbTUA@mail.gmail.com
11 months ago
|
|
|
t | 1 | EXPLAIN (COSTS OFF) MERGE INTO stats_track_tab +
|
|
|
|
| | USING (SELECT id FROM generate_series($1, $2) id) ON x = id +
|
|
|
|
| | WHEN MATCHED THEN UPDATE SET x = id +
|
|
|
|
| | WHEN NOT MATCHED THEN INSERT (x) VALUES (id)
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) SELECT $1
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) SELECT $1 UNION SELECT $2
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) TABLE stats_track_tab
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) UPDATE stats_track_tab SET x = $1 WHERE x = $2
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) VALUES ($1)
|
Track more precisely query locations for nested statements
Previously, a Query generated through the transform phase would have
unset stmt_location, tracking the starting point of a query string.
Extensions relying on the statement location to extract its relevant
parts in the source text string would fallback to use the whole
statement instead, leading to confusing results like in
pg_stat_statements for queries relying on nested queries, like:
- EXPLAIN, with top-level and nested query using the same query string,
and a query ID coming from the nested query when the non-top-level
entry.
- Multi-statements, with only partial portions of queries being
normalized.
- COPY TO with a query, SELECT or DMLs.
This patch improves things by keeping track of the statement locations
and propagate it to Query during transform, allowing PGSS to only show
the relevant part of the query for nested query. This leads to less
bloat in entries for non-top-level entries, as queries can now be
grouped within the same (toplevel, queryid) duos in pg_stat_statements.
The result gives a stricter one-one mapping between query IDs and its
query strings.
The regression tests introduced in 45e0ba30fc40 produce differences
reflecting the new logic.
Author: Anthonin Bonnefoy
Reviewed-by: Michael Paquier, Jian He
Discussion: https://postgr.es/m/CAO6_XqqM6S9bQ2qd=75W+yKATwoazxSNhv5sjW06fjGAtHbTUA@mail.gmail.com
11 months ago
|
|
|
f | 1 | INSERT INTO stats_track_tab VALUES (($1))
|
|
|
|
f | 1 | MERGE INTO stats_track_tab +
|
|
|
|
| | USING (SELECT id FROM generate_series($1, $2) id) ON x = id +
|
|
|
|
| | WHEN MATCHED THEN UPDATE SET x = id +
|
|
|
|
| | WHEN NOT MATCHED THEN INSERT (x) VALUES (id)
|
|
|
|
f | 1 | SELECT $1
|
|
|
|
f | 1 | SELECT $1 UNION SELECT $2
|
|
|
|
f | 1 | SELECT $1, $2
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
Track more precisely query locations for nested statements
Previously, a Query generated through the transform phase would have
unset stmt_location, tracking the starting point of a query string.
Extensions relying on the statement location to extract its relevant
parts in the source text string would fallback to use the whole
statement instead, leading to confusing results like in
pg_stat_statements for queries relying on nested queries, like:
- EXPLAIN, with top-level and nested query using the same query string,
and a query ID coming from the nested query when the non-top-level
entry.
- Multi-statements, with only partial portions of queries being
normalized.
- COPY TO with a query, SELECT or DMLs.
This patch improves things by keeping track of the statement locations
and propagate it to Query during transform, allowing PGSS to only show
the relevant part of the query for nested query. This leads to less
bloat in entries for non-top-level entries, as queries can now be
grouped within the same (toplevel, queryid) duos in pg_stat_statements.
The result gives a stricter one-one mapping between query IDs and its
query strings.
The regression tests introduced in 45e0ba30fc40 produce differences
reflecting the new logic.
Author: Anthonin Bonnefoy
Reviewed-by: Michael Paquier, Jian He
Discussion: https://postgr.es/m/CAO6_XqqM6S9bQ2qd=75W+yKATwoazxSNhv5sjW06fjGAtHbTUA@mail.gmail.com
11 months ago
|
|
|
f | 1 | TABLE stats_track_tab
|
|
|
|
f | 1 | TABLE test_table
|
|
|
|
f | 1 | UPDATE stats_track_tab SET x = $1 WHERE x = $2
|
|
|
|
f | 1 | VALUES ($1)
|
|
|
|
f | 1 | VALUES ($1, $2)
|
|
|
|
(23 rows)
|
|
|
|
|
|
|
|
-- EXPLAIN - top-level tracking.
|
|
|
|
SET pg_stat_statements.track = 'top';
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) SELECT 1;
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) (SELECT 1, 2);
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) TABLE stats_track_tab;
|
|
|
|
QUERY PLAN
|
|
|
|
-----------------------------
|
|
|
|
Seq Scan on stats_track_tab
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) (TABLE test_table);
|
|
|
|
QUERY PLAN
|
|
|
|
------------------------
|
|
|
|
Seq Scan on test_table
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) VALUES (1);
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) (VALUES (1, 2));
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) UPDATE stats_track_tab SET x = 1 WHERE x = 1;
|
|
|
|
QUERY PLAN
|
|
|
|
-----------------------------------
|
|
|
|
Update on stats_track_tab
|
|
|
|
-> Seq Scan on stats_track_tab
|
|
|
|
Filter: (x = 1)
|
|
|
|
(3 rows)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) DELETE FROM stats_track_tab;
|
|
|
|
QUERY PLAN
|
|
|
|
-----------------------------------
|
|
|
|
Delete on stats_track_tab
|
|
|
|
-> Seq Scan on stats_track_tab
|
|
|
|
(2 rows)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) INSERT INTO stats_track_tab VALUES ((1));
|
|
|
|
QUERY PLAN
|
|
|
|
---------------------------
|
|
|
|
Insert on stats_track_tab
|
|
|
|
-> Result
|
|
|
|
(2 rows)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) MERGE INTO stats_track_tab
|
|
|
|
USING (SELECT id FROM generate_series(1, 10) id) ON x = id
|
|
|
|
WHEN MATCHED THEN UPDATE SET x = id
|
|
|
|
WHEN NOT MATCHED THEN INSERT (x) VALUES (id);
|
|
|
|
QUERY PLAN
|
|
|
|
-------------------------------------------------------
|
|
|
|
Merge on stats_track_tab
|
|
|
|
-> Hash Right Join
|
|
|
|
Hash Cond: (stats_track_tab.x = id.id)
|
|
|
|
-> Seq Scan on stats_track_tab
|
|
|
|
-> Hash
|
|
|
|
-> Function Scan on generate_series id
|
|
|
|
(6 rows)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) SELECT 1 UNION SELECT 2;
|
|
|
|
QUERY PLAN
|
|
|
|
--------------------------
|
|
|
|
Unique
|
|
|
|
-> Sort
|
|
|
|
Sort Key: (1)
|
|
|
|
-> Append
|
|
|
|
-> Result
|
|
|
|
-> Result
|
|
|
|
(6 rows)
|
|
|
|
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C";
|
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+--------------------------------------------------------------------
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) (SELECT $1, $2)
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) (TABLE test_table)
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) (VALUES ($1, $2))
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) DELETE FROM stats_track_tab
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) INSERT INTO stats_track_tab VALUES (($1))
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) MERGE INTO stats_track_tab +
|
|
|
|
| | USING (SELECT id FROM generate_series($1, $2) id) ON x = id +
|
|
|
|
| | WHEN MATCHED THEN UPDATE SET x = id +
|
|
|
|
| | WHEN NOT MATCHED THEN INSERT (x) VALUES (id)
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) SELECT $1
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) SELECT $1 UNION SELECT $2
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) TABLE stats_track_tab
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) UPDATE stats_track_tab SET x = $1 WHERE x = $2
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) VALUES ($1)
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
|
|
|
(12 rows)
|
|
|
|
|
|
|
|
-- EXPLAIN - all-level tracking with multi-statement strings.
|
|
|
|
SET pg_stat_statements.track = 'all';
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
-- SELECT queries
|
|
|
|
EXPLAIN (COSTS OFF) SELECT 1\; EXPLAIN (COSTS OFF) SELECT 1, 2;
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) (SELECT 1, 2, 3)\; EXPLAIN (COSTS OFF) (SELECT 1, 2, 3, 4);
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) SELECT 1, 2 UNION SELECT 3, 4\; EXPLAIN (COSTS OFF) (SELECT 1, 2, 3) UNION SELECT 3, 4, 5;
|
|
|
|
QUERY PLAN
|
|
|
|
----------------------------
|
|
|
|
Unique
|
|
|
|
-> Sort
|
|
|
|
Sort Key: (1), (2)
|
|
|
|
-> Append
|
|
|
|
-> Result
|
|
|
|
-> Result
|
|
|
|
(6 rows)
|
|
|
|
|
|
|
|
QUERY PLAN
|
|
|
|
---------------------------------
|
|
|
|
Unique
|
|
|
|
-> Sort
|
|
|
|
Sort Key: (1), (2), (3)
|
|
|
|
-> Append
|
|
|
|
-> Result
|
|
|
|
-> Result
|
|
|
|
(6 rows)
|
|
|
|
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C";
|
Track more precisely query locations for nested statements
Previously, a Query generated through the transform phase would have
unset stmt_location, tracking the starting point of a query string.
Extensions relying on the statement location to extract its relevant
parts in the source text string would fallback to use the whole
statement instead, leading to confusing results like in
pg_stat_statements for queries relying on nested queries, like:
- EXPLAIN, with top-level and nested query using the same query string,
and a query ID coming from the nested query when the non-top-level
entry.
- Multi-statements, with only partial portions of queries being
normalized.
- COPY TO with a query, SELECT or DMLs.
This patch improves things by keeping track of the statement locations
and propagate it to Query during transform, allowing PGSS to only show
the relevant part of the query for nested query. This leads to less
bloat in entries for non-top-level entries, as queries can now be
grouped within the same (toplevel, queryid) duos in pg_stat_statements.
The result gives a stricter one-one mapping between query IDs and its
query strings.
The regression tests introduced in 45e0ba30fc40 produce differences
reflecting the new logic.
Author: Anthonin Bonnefoy
Reviewed-by: Michael Paquier, Jian He
Discussion: https://postgr.es/m/CAO6_XqqM6S9bQ2qd=75W+yKATwoazxSNhv5sjW06fjGAtHbTUA@mail.gmail.com
11 months ago
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+-----------------------------------------------------------------
|
|
|
|
f | 1 | (SELECT $1, $2, $3) UNION SELECT $4, $5, $6
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) (SELECT $1, $2, $3)
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) (SELECT $1, $2, $3) UNION SELECT $4, $5, $6
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) (SELECT $1, $2, $3, $4)
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) SELECT $1
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) SELECT $1, $2
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) SELECT $1, $2 UNION SELECT $3, $4
|
Track more precisely query locations for nested statements
Previously, a Query generated through the transform phase would have
unset stmt_location, tracking the starting point of a query string.
Extensions relying on the statement location to extract its relevant
parts in the source text string would fallback to use the whole
statement instead, leading to confusing results like in
pg_stat_statements for queries relying on nested queries, like:
- EXPLAIN, with top-level and nested query using the same query string,
and a query ID coming from the nested query when the non-top-level
entry.
- Multi-statements, with only partial portions of queries being
normalized.
- COPY TO with a query, SELECT or DMLs.
This patch improves things by keeping track of the statement locations
and propagate it to Query during transform, allowing PGSS to only show
the relevant part of the query for nested query. This leads to less
bloat in entries for non-top-level entries, as queries can now be
grouped within the same (toplevel, queryid) duos in pg_stat_statements.
The result gives a stricter one-one mapping between query IDs and its
query strings.
The regression tests introduced in 45e0ba30fc40 produce differences
reflecting the new logic.
Author: Anthonin Bonnefoy
Reviewed-by: Michael Paquier, Jian He
Discussion: https://postgr.es/m/CAO6_XqqM6S9bQ2qd=75W+yKATwoazxSNhv5sjW06fjGAtHbTUA@mail.gmail.com
11 months ago
|
|
|
f | 1 | SELECT $1
|
|
|
|
f | 1 | SELECT $1, $2
|
|
|
|
f | 1 | SELECT $1, $2 UNION SELECT $3, $4
|
|
|
|
f | 1 | SELECT $1, $2, $3
|
|
|
|
f | 1 | SELECT $1, $2, $3, $4
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
|
|
|
(13 rows)
|
|
|
|
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
-- Most DMLs
|
|
|
|
EXPLAIN (COSTS OFF) TABLE stats_track_tab\; EXPLAIN (COSTS OFF) (TABLE test_table);
|
|
|
|
QUERY PLAN
|
|
|
|
-----------------------------
|
|
|
|
Seq Scan on stats_track_tab
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
QUERY PLAN
|
|
|
|
------------------------
|
|
|
|
Seq Scan on test_table
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) VALUES (1)\; EXPLAIN (COSTS OFF) (VALUES (1, 2));
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) UPDATE stats_track_tab SET x = 1 WHERE x = 1\; EXPLAIN (COSTS OFF) UPDATE stats_track_tab SET x = 1;
|
|
|
|
QUERY PLAN
|
|
|
|
-----------------------------------
|
|
|
|
Update on stats_track_tab
|
|
|
|
-> Seq Scan on stats_track_tab
|
|
|
|
Filter: (x = 1)
|
|
|
|
(3 rows)
|
|
|
|
|
|
|
|
QUERY PLAN
|
|
|
|
-----------------------------------
|
|
|
|
Update on stats_track_tab
|
|
|
|
-> Seq Scan on stats_track_tab
|
|
|
|
(2 rows)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) DELETE FROM stats_track_tab\; EXPLAIN (COSTS OFF) DELETE FROM stats_track_tab WHERE x = 1;
|
|
|
|
QUERY PLAN
|
|
|
|
-----------------------------------
|
|
|
|
Delete on stats_track_tab
|
|
|
|
-> Seq Scan on stats_track_tab
|
|
|
|
(2 rows)
|
|
|
|
|
|
|
|
QUERY PLAN
|
|
|
|
-----------------------------------
|
|
|
|
Delete on stats_track_tab
|
|
|
|
-> Seq Scan on stats_track_tab
|
|
|
|
Filter: (x = 1)
|
|
|
|
(3 rows)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) INSERT INTO stats_track_tab VALUES ((1))\; EXPLAIN (COSTS OFF) INSERT INTO stats_track_tab VALUES (1), (2);
|
|
|
|
QUERY PLAN
|
|
|
|
---------------------------
|
|
|
|
Insert on stats_track_tab
|
|
|
|
-> Result
|
|
|
|
(2 rows)
|
|
|
|
|
|
|
|
QUERY PLAN
|
|
|
|
---------------------------------
|
|
|
|
Insert on stats_track_tab
|
|
|
|
-> Values Scan on "*VALUES*"
|
|
|
|
(2 rows)
|
|
|
|
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C";
|
Track more precisely query locations for nested statements
Previously, a Query generated through the transform phase would have
unset stmt_location, tracking the starting point of a query string.
Extensions relying on the statement location to extract its relevant
parts in the source text string would fallback to use the whole
statement instead, leading to confusing results like in
pg_stat_statements for queries relying on nested queries, like:
- EXPLAIN, with top-level and nested query using the same query string,
and a query ID coming from the nested query when the non-top-level
entry.
- Multi-statements, with only partial portions of queries being
normalized.
- COPY TO with a query, SELECT or DMLs.
This patch improves things by keeping track of the statement locations
and propagate it to Query during transform, allowing PGSS to only show
the relevant part of the query for nested query. This leads to less
bloat in entries for non-top-level entries, as queries can now be
grouped within the same (toplevel, queryid) duos in pg_stat_statements.
The result gives a stricter one-one mapping between query IDs and its
query strings.
The regression tests introduced in 45e0ba30fc40 produce differences
reflecting the new logic.
Author: Anthonin Bonnefoy
Reviewed-by: Michael Paquier, Jian He
Discussion: https://postgr.es/m/CAO6_XqqM6S9bQ2qd=75W+yKATwoazxSNhv5sjW06fjGAtHbTUA@mail.gmail.com
11 months ago
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+--------------------------------------------------------------------
|
|
|
|
f | 1 | DELETE FROM stats_track_tab
|
|
|
|
f | 1 | DELETE FROM stats_track_tab WHERE x = $1
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) (TABLE test_table)
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) (VALUES ($1, $2))
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) DELETE FROM stats_track_tab
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) DELETE FROM stats_track_tab WHERE x = $1
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) INSERT INTO stats_track_tab VALUES ($1), ($2)
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) INSERT INTO stats_track_tab VALUES (($1))
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) TABLE stats_track_tab
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) UPDATE stats_track_tab SET x = $1
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) UPDATE stats_track_tab SET x = $1 WHERE x = $2
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) VALUES ($1)
|
Track more precisely query locations for nested statements
Previously, a Query generated through the transform phase would have
unset stmt_location, tracking the starting point of a query string.
Extensions relying on the statement location to extract its relevant
parts in the source text string would fallback to use the whole
statement instead, leading to confusing results like in
pg_stat_statements for queries relying on nested queries, like:
- EXPLAIN, with top-level and nested query using the same query string,
and a query ID coming from the nested query when the non-top-level
entry.
- Multi-statements, with only partial portions of queries being
normalized.
- COPY TO with a query, SELECT or DMLs.
This patch improves things by keeping track of the statement locations
and propagate it to Query during transform, allowing PGSS to only show
the relevant part of the query for nested query. This leads to less
bloat in entries for non-top-level entries, as queries can now be
grouped within the same (toplevel, queryid) duos in pg_stat_statements.
The result gives a stricter one-one mapping between query IDs and its
query strings.
The regression tests introduced in 45e0ba30fc40 produce differences
reflecting the new logic.
Author: Anthonin Bonnefoy
Reviewed-by: Michael Paquier, Jian He
Discussion: https://postgr.es/m/CAO6_XqqM6S9bQ2qd=75W+yKATwoazxSNhv5sjW06fjGAtHbTUA@mail.gmail.com
11 months ago
|
|
|
f | 1 | INSERT INTO stats_track_tab VALUES ($1), ($2)
|
|
|
|
f | 1 | INSERT INTO stats_track_tab VALUES (($1))
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
Track more precisely query locations for nested statements
Previously, a Query generated through the transform phase would have
unset stmt_location, tracking the starting point of a query string.
Extensions relying on the statement location to extract its relevant
parts in the source text string would fallback to use the whole
statement instead, leading to confusing results like in
pg_stat_statements for queries relying on nested queries, like:
- EXPLAIN, with top-level and nested query using the same query string,
and a query ID coming from the nested query when the non-top-level
entry.
- Multi-statements, with only partial portions of queries being
normalized.
- COPY TO with a query, SELECT or DMLs.
This patch improves things by keeping track of the statement locations
and propagate it to Query during transform, allowing PGSS to only show
the relevant part of the query for nested query. This leads to less
bloat in entries for non-top-level entries, as queries can now be
grouped within the same (toplevel, queryid) duos in pg_stat_statements.
The result gives a stricter one-one mapping between query IDs and its
query strings.
The regression tests introduced in 45e0ba30fc40 produce differences
reflecting the new logic.
Author: Anthonin Bonnefoy
Reviewed-by: Michael Paquier, Jian He
Discussion: https://postgr.es/m/CAO6_XqqM6S9bQ2qd=75W+yKATwoazxSNhv5sjW06fjGAtHbTUA@mail.gmail.com
11 months ago
|
|
|
f | 1 | TABLE stats_track_tab
|
|
|
|
f | 1 | TABLE test_table
|
|
|
|
f | 1 | UPDATE stats_track_tab SET x = $1
|
|
|
|
f | 1 | UPDATE stats_track_tab SET x = $1 WHERE x = $2
|
|
|
|
f | 1 | VALUES ($1)
|
|
|
|
f | 1 | VALUES ($1, $2)
|
|
|
|
(21 rows)
|
|
|
|
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
-- MERGE, worth its own.
|
|
|
|
EXPLAIN (COSTS OFF) MERGE INTO stats_track_tab
|
|
|
|
USING (SELECT id FROM generate_series(1, 10) id) ON x = id
|
|
|
|
WHEN MATCHED THEN UPDATE SET x = id
|
|
|
|
WHEN NOT MATCHED THEN INSERT (x) VALUES (id)\; EXPLAIN (COSTS OFF) SELECT 1, 2, 3, 4, 5;
|
|
|
|
QUERY PLAN
|
|
|
|
-------------------------------------------------------
|
|
|
|
Merge on stats_track_tab
|
|
|
|
-> Hash Right Join
|
|
|
|
Hash Cond: (stats_track_tab.x = id.id)
|
|
|
|
-> Seq Scan on stats_track_tab
|
|
|
|
-> Hash
|
|
|
|
-> Function Scan on generate_series id
|
|
|
|
(6 rows)
|
|
|
|
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C";
|
Track more precisely query locations for nested statements
Previously, a Query generated through the transform phase would have
unset stmt_location, tracking the starting point of a query string.
Extensions relying on the statement location to extract its relevant
parts in the source text string would fallback to use the whole
statement instead, leading to confusing results like in
pg_stat_statements for queries relying on nested queries, like:
- EXPLAIN, with top-level and nested query using the same query string,
and a query ID coming from the nested query when the non-top-level
entry.
- Multi-statements, with only partial portions of queries being
normalized.
- COPY TO with a query, SELECT or DMLs.
This patch improves things by keeping track of the statement locations
and propagate it to Query during transform, allowing PGSS to only show
the relevant part of the query for nested query. This leads to less
bloat in entries for non-top-level entries, as queries can now be
grouped within the same (toplevel, queryid) duos in pg_stat_statements.
The result gives a stricter one-one mapping between query IDs and its
query strings.
The regression tests introduced in 45e0ba30fc40 produce differences
reflecting the new logic.
Author: Anthonin Bonnefoy
Reviewed-by: Michael Paquier, Jian He
Discussion: https://postgr.es/m/CAO6_XqqM6S9bQ2qd=75W+yKATwoazxSNhv5sjW06fjGAtHbTUA@mail.gmail.com
11 months ago
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+---------------------------------------------------------------
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) MERGE INTO stats_track_tab +
|
|
|
|
| | USING (SELECT id FROM generate_series($1, $2) id) ON x = id+
|
|
|
|
| | WHEN MATCHED THEN UPDATE SET x = id +
|
|
|
|
| | WHEN NOT MATCHED THEN INSERT (x) VALUES (id)
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) SELECT $1, $2, $3, $4, $5
|
Track more precisely query locations for nested statements
Previously, a Query generated through the transform phase would have
unset stmt_location, tracking the starting point of a query string.
Extensions relying on the statement location to extract its relevant
parts in the source text string would fallback to use the whole
statement instead, leading to confusing results like in
pg_stat_statements for queries relying on nested queries, like:
- EXPLAIN, with top-level and nested query using the same query string,
and a query ID coming from the nested query when the non-top-level
entry.
- Multi-statements, with only partial portions of queries being
normalized.
- COPY TO with a query, SELECT or DMLs.
This patch improves things by keeping track of the statement locations
and propagate it to Query during transform, allowing PGSS to only show
the relevant part of the query for nested query. This leads to less
bloat in entries for non-top-level entries, as queries can now be
grouped within the same (toplevel, queryid) duos in pg_stat_statements.
The result gives a stricter one-one mapping between query IDs and its
query strings.
The regression tests introduced in 45e0ba30fc40 produce differences
reflecting the new logic.
Author: Anthonin Bonnefoy
Reviewed-by: Michael Paquier, Jian He
Discussion: https://postgr.es/m/CAO6_XqqM6S9bQ2qd=75W+yKATwoazxSNhv5sjW06fjGAtHbTUA@mail.gmail.com
11 months ago
|
|
|
f | 1 | MERGE INTO stats_track_tab +
|
|
|
|
| | USING (SELECT id FROM generate_series($1, $2) id) ON x = id+
|
|
|
|
| | WHEN MATCHED THEN UPDATE SET x = id +
|
|
|
|
| | WHEN NOT MATCHED THEN INSERT (x) VALUES (id)
|
|
|
|
f | 1 | SELECT $1, $2, $3, $4, $5
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
|
|
|
(5 rows)
|
|
|
|
|
|
|
|
-- EXPLAIN - top-level tracking with multi-statement strings.
|
|
|
|
SET pg_stat_statements.track = 'top';
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) SELECT 1\; EXPLAIN (COSTS OFF) SELECT 1, 2;
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) (SELECT 1, 2, 3)\; EXPLAIN (COSTS OFF) (SELECT 1, 2, 3, 4);
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) TABLE stats_track_tab\; EXPLAIN (COSTS OFF) (TABLE test_table);
|
|
|
|
QUERY PLAN
|
|
|
|
-----------------------------
|
|
|
|
Seq Scan on stats_track_tab
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
QUERY PLAN
|
|
|
|
------------------------
|
|
|
|
Seq Scan on test_table
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) VALUES (1)\; EXPLAIN (COSTS OFF) (VALUES (1, 2));
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) UPDATE stats_track_tab SET x = 1 WHERE x = 1\; EXPLAIN (COSTS OFF) UPDATE stats_track_tab SET x = 1;
|
|
|
|
QUERY PLAN
|
|
|
|
-----------------------------------
|
|
|
|
Update on stats_track_tab
|
|
|
|
-> Seq Scan on stats_track_tab
|
|
|
|
Filter: (x = 1)
|
|
|
|
(3 rows)
|
|
|
|
|
|
|
|
QUERY PLAN
|
|
|
|
-----------------------------------
|
|
|
|
Update on stats_track_tab
|
|
|
|
-> Seq Scan on stats_track_tab
|
|
|
|
(2 rows)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) DELETE FROM stats_track_tab\; EXPLAIN (COSTS OFF) DELETE FROM stats_track_tab WHERE x = 1;
|
|
|
|
QUERY PLAN
|
|
|
|
-----------------------------------
|
|
|
|
Delete on stats_track_tab
|
|
|
|
-> Seq Scan on stats_track_tab
|
|
|
|
(2 rows)
|
|
|
|
|
|
|
|
QUERY PLAN
|
|
|
|
-----------------------------------
|
|
|
|
Delete on stats_track_tab
|
|
|
|
-> Seq Scan on stats_track_tab
|
|
|
|
Filter: (x = 1)
|
|
|
|
(3 rows)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) INSERT INTO stats_track_tab VALUES ((1))\; EXPLAIN (COSTS OFF) INSERT INTO stats_track_tab VALUES ((1), (2));
|
|
|
|
QUERY PLAN
|
|
|
|
---------------------------
|
|
|
|
Insert on stats_track_tab
|
|
|
|
-> Result
|
|
|
|
(2 rows)
|
|
|
|
|
|
|
|
ERROR: INSERT has more expressions than target columns
|
|
|
|
LINE 1: ...N (COSTS OFF) INSERT INTO stats_track_tab VALUES ((1), (2));
|
|
|
|
^
|
|
|
|
EXPLAIN (COSTS OFF) MERGE INTO stats_track_tab USING (SELECT id FROM generate_series(1, 10) id) ON x = id
|
|
|
|
WHEN MATCHED THEN UPDATE SET x = id
|
|
|
|
WHEN NOT MATCHED THEN INSERT (x) VALUES (id)\; EXPLAIN (COSTS OFF) SELECT 1, 2, 3, 4, 5;
|
|
|
|
QUERY PLAN
|
|
|
|
-------------------------------------------------------
|
|
|
|
Merge on stats_track_tab
|
|
|
|
-> Hash Right Join
|
|
|
|
Hash Cond: (stats_track_tab.x = id.id)
|
|
|
|
-> Seq Scan on stats_track_tab
|
|
|
|
-> Hash
|
|
|
|
-> Function Scan on generate_series id
|
|
|
|
(6 rows)
|
|
|
|
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) SELECT 1, 2 UNION SELECT 3, 4\; EXPLAIN (COSTS OFF) (SELECT 1, 2, 3) UNION SELECT 3, 4, 5;
|
|
|
|
QUERY PLAN
|
|
|
|
----------------------------
|
|
|
|
Unique
|
|
|
|
-> Sort
|
|
|
|
Sort Key: (1), (2)
|
|
|
|
-> Append
|
|
|
|
-> Result
|
|
|
|
-> Result
|
|
|
|
(6 rows)
|
|
|
|
|
|
|
|
QUERY PLAN
|
|
|
|
---------------------------------
|
|
|
|
Unique
|
|
|
|
-> Sort
|
|
|
|
Sort Key: (1), (2), (3)
|
|
|
|
-> Append
|
|
|
|
-> Result
|
|
|
|
-> Result
|
|
|
|
(6 rows)
|
|
|
|
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C";
|
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+------------------------------------------------------------------------------------------------------------
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) (SELECT $1, $2, $3)
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) (SELECT $1, $2, $3) UNION SELECT $4, $5, $6
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) (SELECT $1, $2, $3, $4)
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) (TABLE test_table)
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) (VALUES ($1, $2))
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) DELETE FROM stats_track_tab
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) DELETE FROM stats_track_tab WHERE x = $1
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) INSERT INTO stats_track_tab VALUES (($1))
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) MERGE INTO stats_track_tab USING (SELECT id FROM generate_series($1, $2) id) ON x = id+
|
|
|
|
| | WHEN MATCHED THEN UPDATE SET x = id +
|
|
|
|
| | WHEN NOT MATCHED THEN INSERT (x) VALUES (id)
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) SELECT $1
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) SELECT $1, $2
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) SELECT $1, $2 UNION SELECT $3, $4
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) SELECT $1, $2, $3, $4, $5
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) TABLE stats_track_tab
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) UPDATE stats_track_tab SET x = $1
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) UPDATE stats_track_tab SET x = $1 WHERE x = $2
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) VALUES ($1)
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
|
|
|
(18 rows)
|
|
|
|
|
|
|
|
-- EXPLAIN with CTEs - all-level tracking
|
|
|
|
SET pg_stat_statements.track = 'all';
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) WITH a AS (SELECT 4) SELECT 1;
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) (WITH a AS (SELECT 4) (SELECT 1, 2));
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) WITH a AS (SELECT 4) UPDATE stats_track_tab SET x = 1 WHERE x = 1;
|
|
|
|
QUERY PLAN
|
|
|
|
-----------------------------------
|
|
|
|
Update on stats_track_tab
|
|
|
|
-> Seq Scan on stats_track_tab
|
|
|
|
Filter: (x = 1)
|
|
|
|
(3 rows)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) WITH a AS (SELECT 4) DELETE FROM stats_track_tab;
|
|
|
|
QUERY PLAN
|
|
|
|
-----------------------------------
|
|
|
|
Delete on stats_track_tab
|
|
|
|
-> Seq Scan on stats_track_tab
|
|
|
|
(2 rows)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) WITH a AS (SELECT 4) INSERT INTO stats_track_tab VALUES ((1));
|
|
|
|
QUERY PLAN
|
|
|
|
---------------------------
|
|
|
|
Insert on stats_track_tab
|
|
|
|
-> Result
|
|
|
|
(2 rows)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) WITH a AS (SELECT 4) MERGE INTO stats_track_tab
|
|
|
|
USING (SELECT id FROM generate_series(1, 10) id) ON x = id
|
|
|
|
WHEN MATCHED THEN UPDATE SET x = id
|
|
|
|
WHEN NOT MATCHED THEN INSERT (x) VALUES (id);
|
|
|
|
QUERY PLAN
|
|
|
|
-------------------------------------------------------
|
|
|
|
Merge on stats_track_tab
|
|
|
|
-> Hash Right Join
|
|
|
|
Hash Cond: (stats_track_tab.x = id.id)
|
|
|
|
-> Seq Scan on stats_track_tab
|
|
|
|
-> Hash
|
|
|
|
-> Function Scan on generate_series id
|
|
|
|
(6 rows)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) WITH a AS (select 4) SELECT 1 UNION SELECT 2;
|
|
|
|
QUERY PLAN
|
|
|
|
--------------------------
|
|
|
|
Unique
|
|
|
|
-> Sort
|
|
|
|
Sort Key: (1)
|
|
|
|
-> Append
|
|
|
|
-> Result
|
|
|
|
-> Result
|
|
|
|
(6 rows)
|
|
|
|
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C";
|
Track more precisely query locations for nested statements
Previously, a Query generated through the transform phase would have
unset stmt_location, tracking the starting point of a query string.
Extensions relying on the statement location to extract its relevant
parts in the source text string would fallback to use the whole
statement instead, leading to confusing results like in
pg_stat_statements for queries relying on nested queries, like:
- EXPLAIN, with top-level and nested query using the same query string,
and a query ID coming from the nested query when the non-top-level
entry.
- Multi-statements, with only partial portions of queries being
normalized.
- COPY TO with a query, SELECT or DMLs.
This patch improves things by keeping track of the statement locations
and propagate it to Query during transform, allowing PGSS to only show
the relevant part of the query for nested query. This leads to less
bloat in entries for non-top-level entries, as queries can now be
grouped within the same (toplevel, queryid) duos in pg_stat_statements.
The result gives a stricter one-one mapping between query IDs and its
query strings.
The regression tests introduced in 45e0ba30fc40 produce differences
reflecting the new logic.
Author: Anthonin Bonnefoy
Reviewed-by: Michael Paquier, Jian He
Discussion: https://postgr.es/m/CAO6_XqqM6S9bQ2qd=75W+yKATwoazxSNhv5sjW06fjGAtHbTUA@mail.gmail.com
11 months ago
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+------------------------------------------------------------------------------------------
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) (WITH a AS (SELECT $1) (SELECT $2, $3))
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) WITH a AS (SELECT $1) DELETE FROM stats_track_tab
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) WITH a AS (SELECT $1) INSERT INTO stats_track_tab VALUES (($2))
|
Track more precisely query locations for nested statements
Previously, a Query generated through the transform phase would have
unset stmt_location, tracking the starting point of a query string.
Extensions relying on the statement location to extract its relevant
parts in the source text string would fallback to use the whole
statement instead, leading to confusing results like in
pg_stat_statements for queries relying on nested queries, like:
- EXPLAIN, with top-level and nested query using the same query string,
and a query ID coming from the nested query when the non-top-level
entry.
- Multi-statements, with only partial portions of queries being
normalized.
- COPY TO with a query, SELECT or DMLs.
This patch improves things by keeping track of the statement locations
and propagate it to Query during transform, allowing PGSS to only show
the relevant part of the query for nested query. This leads to less
bloat in entries for non-top-level entries, as queries can now be
grouped within the same (toplevel, queryid) duos in pg_stat_statements.
The result gives a stricter one-one mapping between query IDs and its
query strings.
The regression tests introduced in 45e0ba30fc40 produce differences
reflecting the new logic.
Author: Anthonin Bonnefoy
Reviewed-by: Michael Paquier, Jian He
Discussion: https://postgr.es/m/CAO6_XqqM6S9bQ2qd=75W+yKATwoazxSNhv5sjW06fjGAtHbTUA@mail.gmail.com
11 months ago
|
|
|
t | 1 | EXPLAIN (COSTS OFF) WITH a AS (SELECT $1) MERGE INTO stats_track_tab +
|
|
|
|
| | USING (SELECT id FROM generate_series($2, $3) id) ON x = id +
|
|
|
|
| | WHEN MATCHED THEN UPDATE SET x = id +
|
|
|
|
| | WHEN NOT MATCHED THEN INSERT (x) VALUES (id)
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) WITH a AS (SELECT $1) SELECT $2
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) WITH a AS (SELECT $1) UPDATE stats_track_tab SET x = $2 WHERE x = $3
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) WITH a AS (select $1) SELECT $2 UNION SELECT $3
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
Track more precisely query locations for nested statements
Previously, a Query generated through the transform phase would have
unset stmt_location, tracking the starting point of a query string.
Extensions relying on the statement location to extract its relevant
parts in the source text string would fallback to use the whole
statement instead, leading to confusing results like in
pg_stat_statements for queries relying on nested queries, like:
- EXPLAIN, with top-level and nested query using the same query string,
and a query ID coming from the nested query when the non-top-level
entry.
- Multi-statements, with only partial portions of queries being
normalized.
- COPY TO with a query, SELECT or DMLs.
This patch improves things by keeping track of the statement locations
and propagate it to Query during transform, allowing PGSS to only show
the relevant part of the query for nested query. This leads to less
bloat in entries for non-top-level entries, as queries can now be
grouped within the same (toplevel, queryid) duos in pg_stat_statements.
The result gives a stricter one-one mapping between query IDs and its
query strings.
The regression tests introduced in 45e0ba30fc40 produce differences
reflecting the new logic.
Author: Anthonin Bonnefoy
Reviewed-by: Michael Paquier, Jian He
Discussion: https://postgr.es/m/CAO6_XqqM6S9bQ2qd=75W+yKATwoazxSNhv5sjW06fjGAtHbTUA@mail.gmail.com
11 months ago
|
|
|
f | 1 | WITH a AS (SELECT $1) (SELECT $2, $3)
|
|
|
|
f | 1 | WITH a AS (SELECT $1) DELETE FROM stats_track_tab
|
|
|
|
f | 1 | WITH a AS (SELECT $1) INSERT INTO stats_track_tab VALUES (($2))
|
|
|
|
f | 1 | WITH a AS (SELECT $1) MERGE INTO stats_track_tab +
|
|
|
|
| | USING (SELECT id FROM generate_series($2, $3) id) ON x = id +
|
|
|
|
| | WHEN MATCHED THEN UPDATE SET x = id +
|
|
|
|
| | WHEN NOT MATCHED THEN INSERT (x) VALUES (id)
|
|
|
|
f | 1 | WITH a AS (SELECT $1) SELECT $2
|
|
|
|
f | 1 | WITH a AS (SELECT $1) UPDATE stats_track_tab SET x = $2 WHERE x = $3
|
|
|
|
f | 1 | WITH a AS (select $1) SELECT $2 UNION SELECT $3
|
|
|
|
(15 rows)
|
|
|
|
|
|
|
|
-- EXPLAIN with CTEs - top-level tracking
|
|
|
|
SET pg_stat_statements.track = 'top';
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) WITH a AS (SELECT 4) SELECT 1;
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) (WITH a AS (SELECT 4) (SELECT 1, 2));
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) WITH a AS (SELECT 4) UPDATE stats_track_tab SET x = 1 WHERE x = 1;
|
|
|
|
QUERY PLAN
|
|
|
|
-----------------------------------
|
|
|
|
Update on stats_track_tab
|
|
|
|
-> Seq Scan on stats_track_tab
|
|
|
|
Filter: (x = 1)
|
|
|
|
(3 rows)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) WITH a AS (SELECT 4) DELETE FROM stats_track_tab;
|
|
|
|
QUERY PLAN
|
|
|
|
-----------------------------------
|
|
|
|
Delete on stats_track_tab
|
|
|
|
-> Seq Scan on stats_track_tab
|
|
|
|
(2 rows)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) WITH a AS (SELECT 4) INSERT INTO stats_track_tab VALUES ((1));
|
|
|
|
QUERY PLAN
|
|
|
|
---------------------------
|
|
|
|
Insert on stats_track_tab
|
|
|
|
-> Result
|
|
|
|
(2 rows)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) WITH a AS (SELECT 4) MERGE INTO stats_track_tab
|
|
|
|
USING (SELECT id FROM generate_series(1, 10) id) ON x = id
|
|
|
|
WHEN MATCHED THEN UPDATE SET x = id
|
|
|
|
WHEN NOT MATCHED THEN INSERT (x) VALUES (id);
|
|
|
|
QUERY PLAN
|
|
|
|
-------------------------------------------------------
|
|
|
|
Merge on stats_track_tab
|
|
|
|
-> Hash Right Join
|
|
|
|
Hash Cond: (stats_track_tab.x = id.id)
|
|
|
|
-> Seq Scan on stats_track_tab
|
|
|
|
-> Hash
|
|
|
|
-> Function Scan on generate_series id
|
|
|
|
(6 rows)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) WITH a AS (select 4) SELECT 1 UNION SELECT 2;
|
|
|
|
QUERY PLAN
|
|
|
|
--------------------------
|
|
|
|
Unique
|
|
|
|
-> Sort
|
|
|
|
Sort Key: (1)
|
|
|
|
-> Append
|
|
|
|
-> Result
|
|
|
|
-> Result
|
|
|
|
(6 rows)
|
|
|
|
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C";
|
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+------------------------------------------------------------------------------------------
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) (WITH a AS (SELECT $1) (SELECT $2, $3))
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) WITH a AS (SELECT $1) DELETE FROM stats_track_tab
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) WITH a AS (SELECT $1) INSERT INTO stats_track_tab VALUES (($2))
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) WITH a AS (SELECT $1) MERGE INTO stats_track_tab +
|
|
|
|
| | USING (SELECT id FROM generate_series($2, $3) id) ON x = id +
|
|
|
|
| | WHEN MATCHED THEN UPDATE SET x = id +
|
|
|
|
| | WHEN NOT MATCHED THEN INSERT (x) VALUES (id)
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) WITH a AS (SELECT $1) SELECT $2
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) WITH a AS (SELECT $1) UPDATE stats_track_tab SET x = $2 WHERE x = $3
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) WITH a AS (select $1) SELECT $2 UNION SELECT $3
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
|
|
|
(8 rows)
|
|
|
|
|
|
|
|
-- Explain analyze, all-level tracking.
|
|
|
|
SET pg_stat_statements.track = 'all';
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF) SELECT 100;
|
EXPLAIN: Always use two fractional digits for row counts.
Commit ddb17e387aa28d61521227377b00f997756b8a27 attempted to avoid
confusing users by displaying digits after the decimal point only when
nloops > 1, since it's impossible to have a fraction row count after a
single iteration. However, this made the regression tests unstable since
parallal queries will have nloops>1 for all nodes below the Gather or
Gather Merge in normal cases, but if the workers don't start in time and
the leader finishes all the work, they will suddenly have nloops==1,
making it unpredictable whether the digits after the decimal point would
be displayed or not. Although 44cbba9a7f51a3888d5087fc94b23614ba2b81f2
seemed to fix the immediate failures, it may still be the case that there
are lower-probability failures elsewhere in the regression tests.
Various fixes are possible here. For example, it has previously been
proposed that we should try to display the digits after the decimal
point only if rows/nloops is an integer, but currently rows is storead
as a float so it's not theoretically an exact quantity -- precision
could be lost in extreme cases. It has also been proposed that we
should try to display the digits after the decimal point only if we're
under some sort of construct that could potentially cause looping
regardless of whether it actually does. While such ideas are not
without merit, this patch adopts the much simpler solution of always
display two decimal digits. If that approach stands up to scrutiny
from the buildfarm and human users, it spares us the trouble of doing
anything more complex; if not, we can reassess.
This commit incidentally reverts 44cbba9a7f51a3888d5087fc94b23614ba2b81f2,
which should no longer be needed.
Author: Robert Haas <robertmhaas@gmail.com>
Author: Ilia Evdokimov <ilya.evdokimov@tantorlabs.com>
Discussion: http://postgr.es/m/CA+TgmoazzVHn8sFOMFAEwoqBTDxKT45D7mvkyeHgqtoD2cn58Q@mail.gmail.com
7 months ago
|
|
|
QUERY PLAN
|
|
|
|
-----------------------------------
|
|
|
|
Result (actual rows=1.00 loops=1)
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
|
|
|
|
DECLARE foocur CURSOR FOR SELECT * FROM stats_track_tab;
|
EXPLAIN: Always use two fractional digits for row counts.
Commit ddb17e387aa28d61521227377b00f997756b8a27 attempted to avoid
confusing users by displaying digits after the decimal point only when
nloops > 1, since it's impossible to have a fraction row count after a
single iteration. However, this made the regression tests unstable since
parallal queries will have nloops>1 for all nodes below the Gather or
Gather Merge in normal cases, but if the workers don't start in time and
the leader finishes all the work, they will suddenly have nloops==1,
making it unpredictable whether the digits after the decimal point would
be displayed or not. Although 44cbba9a7f51a3888d5087fc94b23614ba2b81f2
seemed to fix the immediate failures, it may still be the case that there
are lower-probability failures elsewhere in the regression tests.
Various fixes are possible here. For example, it has previously been
proposed that we should try to display the digits after the decimal
point only if rows/nloops is an integer, but currently rows is storead
as a float so it's not theoretically an exact quantity -- precision
could be lost in extreme cases. It has also been proposed that we
should try to display the digits after the decimal point only if we're
under some sort of construct that could potentially cause looping
regardless of whether it actually does. While such ideas are not
without merit, this patch adopts the much simpler solution of always
display two decimal digits. If that approach stands up to scrutiny
from the buildfarm and human users, it spares us the trouble of doing
anything more complex; if not, we can reassess.
This commit incidentally reverts 44cbba9a7f51a3888d5087fc94b23614ba2b81f2,
which should no longer be needed.
Author: Robert Haas <robertmhaas@gmail.com>
Author: Ilia Evdokimov <ilya.evdokimov@tantorlabs.com>
Discussion: http://postgr.es/m/CA+TgmoazzVHn8sFOMFAEwoqBTDxKT45D7mvkyeHgqtoD2cn58Q@mail.gmail.com
7 months ago
|
|
|
QUERY PLAN
|
|
|
|
--------------------------------------------------------
|
|
|
|
Seq Scan on stats_track_tab (actual rows=0.00 loops=1)
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C";
|
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+------------------------------------------------------------------------------
|
|
|
|
t | 1 | EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF) +
|
|
|
|
| | DECLARE foocur CURSOR FOR SELECT * FROM stats_track_tab
|
|
|
|
t | 1 | EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF) SELECT $1
|
Track more precisely query locations for nested statements
Previously, a Query generated through the transform phase would have
unset stmt_location, tracking the starting point of a query string.
Extensions relying on the statement location to extract its relevant
parts in the source text string would fallback to use the whole
statement instead, leading to confusing results like in
pg_stat_statements for queries relying on nested queries, like:
- EXPLAIN, with top-level and nested query using the same query string,
and a query ID coming from the nested query when the non-top-level
entry.
- Multi-statements, with only partial portions of queries being
normalized.
- COPY TO with a query, SELECT or DMLs.
This patch improves things by keeping track of the statement locations
and propagate it to Query during transform, allowing PGSS to only show
the relevant part of the query for nested query. This leads to less
bloat in entries for non-top-level entries, as queries can now be
grouped within the same (toplevel, queryid) duos in pg_stat_statements.
The result gives a stricter one-one mapping between query IDs and its
query strings.
The regression tests introduced in 45e0ba30fc40 produce differences
reflecting the new logic.
Author: Anthonin Bonnefoy
Reviewed-by: Michael Paquier, Jian He
Discussion: https://postgr.es/m/CAO6_XqqM6S9bQ2qd=75W+yKATwoazxSNhv5sjW06fjGAtHbTUA@mail.gmail.com
11 months ago
|
|
|
f | 1 | SELECT $1
|
|
|
|
f | 1 | SELECT * FROM stats_track_tab
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
|
|
|
(5 rows)
|
|
|
|
|
|
|
|
-- Explain analyze, top tracking.
|
|
|
|
SET pg_stat_statements.track = 'top';
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF) SELECT 100;
|
EXPLAIN: Always use two fractional digits for row counts.
Commit ddb17e387aa28d61521227377b00f997756b8a27 attempted to avoid
confusing users by displaying digits after the decimal point only when
nloops > 1, since it's impossible to have a fraction row count after a
single iteration. However, this made the regression tests unstable since
parallal queries will have nloops>1 for all nodes below the Gather or
Gather Merge in normal cases, but if the workers don't start in time and
the leader finishes all the work, they will suddenly have nloops==1,
making it unpredictable whether the digits after the decimal point would
be displayed or not. Although 44cbba9a7f51a3888d5087fc94b23614ba2b81f2
seemed to fix the immediate failures, it may still be the case that there
are lower-probability failures elsewhere in the regression tests.
Various fixes are possible here. For example, it has previously been
proposed that we should try to display the digits after the decimal
point only if rows/nloops is an integer, but currently rows is storead
as a float so it's not theoretically an exact quantity -- precision
could be lost in extreme cases. It has also been proposed that we
should try to display the digits after the decimal point only if we're
under some sort of construct that could potentially cause looping
regardless of whether it actually does. While such ideas are not
without merit, this patch adopts the much simpler solution of always
display two decimal digits. If that approach stands up to scrutiny
from the buildfarm and human users, it spares us the trouble of doing
anything more complex; if not, we can reassess.
This commit incidentally reverts 44cbba9a7f51a3888d5087fc94b23614ba2b81f2,
which should no longer be needed.
Author: Robert Haas <robertmhaas@gmail.com>
Author: Ilia Evdokimov <ilya.evdokimov@tantorlabs.com>
Discussion: http://postgr.es/m/CA+TgmoazzVHn8sFOMFAEwoqBTDxKT45D7mvkyeHgqtoD2cn58Q@mail.gmail.com
7 months ago
|
|
|
QUERY PLAN
|
|
|
|
-----------------------------------
|
|
|
|
Result (actual rows=1.00 loops=1)
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF)
|
|
|
|
DECLARE foocur CURSOR FOR SELECT * FROM stats_track_tab;
|
EXPLAIN: Always use two fractional digits for row counts.
Commit ddb17e387aa28d61521227377b00f997756b8a27 attempted to avoid
confusing users by displaying digits after the decimal point only when
nloops > 1, since it's impossible to have a fraction row count after a
single iteration. However, this made the regression tests unstable since
parallal queries will have nloops>1 for all nodes below the Gather or
Gather Merge in normal cases, but if the workers don't start in time and
the leader finishes all the work, they will suddenly have nloops==1,
making it unpredictable whether the digits after the decimal point would
be displayed or not. Although 44cbba9a7f51a3888d5087fc94b23614ba2b81f2
seemed to fix the immediate failures, it may still be the case that there
are lower-probability failures elsewhere in the regression tests.
Various fixes are possible here. For example, it has previously been
proposed that we should try to display the digits after the decimal
point only if rows/nloops is an integer, but currently rows is storead
as a float so it's not theoretically an exact quantity -- precision
could be lost in extreme cases. It has also been proposed that we
should try to display the digits after the decimal point only if we're
under some sort of construct that could potentially cause looping
regardless of whether it actually does. While such ideas are not
without merit, this patch adopts the much simpler solution of always
display two decimal digits. If that approach stands up to scrutiny
from the buildfarm and human users, it spares us the trouble of doing
anything more complex; if not, we can reassess.
This commit incidentally reverts 44cbba9a7f51a3888d5087fc94b23614ba2b81f2,
which should no longer be needed.
Author: Robert Haas <robertmhaas@gmail.com>
Author: Ilia Evdokimov <ilya.evdokimov@tantorlabs.com>
Discussion: http://postgr.es/m/CA+TgmoazzVHn8sFOMFAEwoqBTDxKT45D7mvkyeHgqtoD2cn58Q@mail.gmail.com
7 months ago
|
|
|
QUERY PLAN
|
|
|
|
--------------------------------------------------------
|
|
|
|
Seq Scan on stats_track_tab (actual rows=0.00 loops=1)
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C";
|
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+------------------------------------------------------------------------------
|
|
|
|
t | 1 | EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF) +
|
|
|
|
| | DECLARE foocur CURSOR FOR SELECT * FROM stats_track_tab
|
|
|
|
t | 1 | EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF, TIMING OFF, BUFFERS OFF) SELECT $1
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
|
|
|
(3 rows)
|
|
|
|
|
|
|
|
-- Create Materialized View, all-level tracking.
|
|
|
|
SET pg_stat_statements.track = 'all';
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
CREATE MATERIALIZED VIEW pgss_materialized_view AS
|
|
|
|
SELECT * FROM generate_series(1, 5) as id;
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C";
|
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+----------------------------------------------------
|
|
|
|
t | 1 | CREATE MATERIALIZED VIEW pgss_materialized_view AS+
|
|
|
|
| | SELECT * FROM generate_series($1, $2) as id
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
|
|
|
(2 rows)
|
|
|
|
|
|
|
|
-- CREATE MATERIALIZED VIEW, top-level tracking.
|
|
|
|
SET pg_stat_statements.track = 'top';
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
CREATE MATERIALIZED VIEW pgss_materialized_view_2 AS
|
|
|
|
SELECT * FROM generate_series(1, 5) as id;
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C";
|
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+------------------------------------------------------
|
|
|
|
t | 1 | CREATE MATERIALIZED VIEW pgss_materialized_view_2 AS+
|
|
|
|
| | SELECT * FROM generate_series($1, $2) as id
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
|
|
|
(2 rows)
|
|
|
|
|
|
|
|
-- REFRESH MATERIALIZED VIEW, all-level tracking.
|
|
|
|
SET pg_stat_statements.track = 'all';
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
REFRESH MATERIALIZED VIEW pgss_materialized_view;
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C";
|
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+----------------------------------------------------
|
|
|
|
t | 1 | REFRESH MATERIALIZED VIEW pgss_materialized_view
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
|
|
|
(2 rows)
|
|
|
|
|
|
|
|
-- REFRESH MATERIALIZED VIEW, top-level tracking.
|
|
|
|
SET pg_stat_statements.track = 'top';
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
REFRESH MATERIALIZED VIEW pgss_materialized_view;
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C";
|
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+----------------------------------------------------
|
|
|
|
t | 1 | REFRESH MATERIALIZED VIEW pgss_materialized_view
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
|
|
|
(2 rows)
|
|
|
|
|
|
|
|
-- CREATE TABLE AS, all-level tracking.
|
|
|
|
SET pg_stat_statements.track = 'all';
|
|
|
|
PREPARE test_prepare_pgss AS select generate_series(1, 10);
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
CREATE TEMPORARY TABLE pgss_ctas_1 AS SELECT 1;
|
|
|
|
CREATE TEMPORARY TABLE pgss_ctas_2 AS EXECUTE test_prepare_pgss;
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C";
|
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+-----------------------------------------------------------------
|
|
|
|
t | 1 | CREATE TEMPORARY TABLE pgss_ctas_1 AS SELECT $1
|
|
|
|
t | 1 | CREATE TEMPORARY TABLE pgss_ctas_2 AS EXECUTE test_prepare_pgss
|
|
|
|
f | 1 | SELECT $1
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
Track more precisely query locations for nested statements
Previously, a Query generated through the transform phase would have
unset stmt_location, tracking the starting point of a query string.
Extensions relying on the statement location to extract its relevant
parts in the source text string would fallback to use the whole
statement instead, leading to confusing results like in
pg_stat_statements for queries relying on nested queries, like:
- EXPLAIN, with top-level and nested query using the same query string,
and a query ID coming from the nested query when the non-top-level
entry.
- Multi-statements, with only partial portions of queries being
normalized.
- COPY TO with a query, SELECT or DMLs.
This patch improves things by keeping track of the statement locations
and propagate it to Query during transform, allowing PGSS to only show
the relevant part of the query for nested query. This leads to less
bloat in entries for non-top-level entries, as queries can now be
grouped within the same (toplevel, queryid) duos in pg_stat_statements.
The result gives a stricter one-one mapping between query IDs and its
query strings.
The regression tests introduced in 45e0ba30fc40 produce differences
reflecting the new logic.
Author: Anthonin Bonnefoy
Reviewed-by: Michael Paquier, Jian He
Discussion: https://postgr.es/m/CAO6_XqqM6S9bQ2qd=75W+yKATwoazxSNhv5sjW06fjGAtHbTUA@mail.gmail.com
11 months ago
|
|
|
f | 1 | select generate_series($1, $2)
|
|
|
|
(5 rows)
|
|
|
|
|
|
|
|
-- CREATE TABLE AS, top-level tracking.
|
|
|
|
SET pg_stat_statements.track = 'top';
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
CREATE TEMPORARY TABLE pgss_ctas_3 AS SELECT 1;
|
|
|
|
CREATE TEMPORARY TABLE pgss_ctas_4 AS EXECUTE test_prepare_pgss;
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C";
|
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+-----------------------------------------------------------------
|
|
|
|
t | 1 | CREATE TEMPORARY TABLE pgss_ctas_3 AS SELECT $1
|
|
|
|
t | 1 | CREATE TEMPORARY TABLE pgss_ctas_4 AS EXECUTE test_prepare_pgss
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
|
|
|
(3 rows)
|
|
|
|
|
|
|
|
-- EXPLAIN with CREATE TABLE AS - all-level tracking.
|
|
|
|
SET pg_stat_statements.track = 'all';
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) CREATE TEMPORARY TABLE pgss_explain_ctas AS SELECT 1;
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C";
|
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+---------------------------------------------------------------------------
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) CREATE TEMPORARY TABLE pgss_explain_ctas AS SELECT $1
|
|
|
|
f | 1 | SELECT $1
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
|
|
|
(3 rows)
|
|
|
|
|
|
|
|
-- EXPLAIN with CREATE TABLE AS - top-level tracking.
|
|
|
|
SET pg_stat_statements.track = 'top';
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
EXPLAIN (COSTS OFF) CREATE TEMPORARY TABLE pgss_explain_ctas AS SELECT 1;
|
|
|
|
QUERY PLAN
|
|
|
|
------------
|
|
|
|
Result
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C";
|
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+---------------------------------------------------------------------------
|
|
|
|
t | 1 | EXPLAIN (COSTS OFF) CREATE TEMPORARY TABLE pgss_explain_ctas AS SELECT $1
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
|
|
|
(2 rows)
|
|
|
|
|
|
|
|
-- DECLARE CURSOR, all-level tracking.
|
|
|
|
SET pg_stat_statements.track = 'all';
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
BEGIN;
|
|
|
|
DECLARE FOOCUR CURSOR FOR SELECT * from stats_track_tab;
|
|
|
|
FETCH FORWARD 1 FROM foocur;
|
|
|
|
x
|
|
|
|
---
|
|
|
|
(0 rows)
|
|
|
|
|
|
|
|
CLOSE foocur;
|
|
|
|
COMMIT;
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C";
|
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+---------------------------------------------------------
|
|
|
|
t | 1 | BEGIN
|
|
|
|
t | 1 | CLOSE foocur
|
|
|
|
t | 1 | COMMIT
|
|
|
|
t | 1 | DECLARE FOOCUR CURSOR FOR SELECT * from stats_track_tab
|
|
|
|
t | 1 | FETCH FORWARD 1 FROM foocur
|
|
|
|
f | 1 | SELECT * from stats_track_tab
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
|
|
|
(7 rows)
|
|
|
|
|
|
|
|
-- DECLARE CURSOR, top-level tracking.
|
|
|
|
SET pg_stat_statements.track = 'top';
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
BEGIN;
|
|
|
|
DECLARE FOOCUR CURSOR FOR SELECT * FROM stats_track_tab;
|
|
|
|
FETCH FORWARD 1 FROM foocur;
|
|
|
|
x
|
|
|
|
---
|
|
|
|
(0 rows)
|
|
|
|
|
|
|
|
CLOSE foocur;
|
|
|
|
COMMIT;
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C";
|
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+---------------------------------------------------------
|
|
|
|
t | 1 | BEGIN
|
|
|
|
t | 1 | CLOSE foocur
|
|
|
|
t | 1 | COMMIT
|
|
|
|
t | 1 | DECLARE FOOCUR CURSOR FOR SELECT * FROM stats_track_tab
|
|
|
|
t | 1 | FETCH FORWARD 1 FROM foocur
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
|
|
|
(6 rows)
|
|
|
|
|
|
|
|
-- COPY - all-level tracking.
|
|
|
|
SET pg_stat_statements.track = 'all';
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
COPY (SELECT 1) TO stdout;
|
|
|
|
1
|
|
|
|
COPY (SELECT 1 UNION SELECT 2) TO stdout;
|
|
|
|
1
|
|
|
|
2
|
|
|
|
COPY (MERGE INTO stats_track_tab USING (SELECT 1 id) ON x = id
|
|
|
|
WHEN MATCHED THEN UPDATE SET x = id
|
|
|
|
WHEN NOT MATCHED THEN INSERT (x) VALUES (id) RETURNING x) TO stdout;
|
|
|
|
1
|
|
|
|
COPY (INSERT INTO stats_track_tab (x) VALUES (1) RETURNING x) TO stdout;
|
|
|
|
1
|
|
|
|
COPY (UPDATE stats_track_tab SET x = 2 WHERE x = 1 RETURNING x) TO stdout;
|
|
|
|
2
|
|
|
|
2
|
|
|
|
COPY (DELETE FROM stats_track_tab WHERE x = 2 RETURNING x) TO stdout;
|
|
|
|
2
|
|
|
|
2
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C";
|
Track more precisely query locations for nested statements
Previously, a Query generated through the transform phase would have
unset stmt_location, tracking the starting point of a query string.
Extensions relying on the statement location to extract its relevant
parts in the source text string would fallback to use the whole
statement instead, leading to confusing results like in
pg_stat_statements for queries relying on nested queries, like:
- EXPLAIN, with top-level and nested query using the same query string,
and a query ID coming from the nested query when the non-top-level
entry.
- Multi-statements, with only partial portions of queries being
normalized.
- COPY TO with a query, SELECT or DMLs.
This patch improves things by keeping track of the statement locations
and propagate it to Query during transform, allowing PGSS to only show
the relevant part of the query for nested query. This leads to less
bloat in entries for non-top-level entries, as queries can now be
grouped within the same (toplevel, queryid) duos in pg_stat_statements.
The result gives a stricter one-one mapping between query IDs and its
query strings.
The regression tests introduced in 45e0ba30fc40 produce differences
reflecting the new logic.
Author: Anthonin Bonnefoy
Reviewed-by: Michael Paquier, Jian He
Discussion: https://postgr.es/m/CAO6_XqqM6S9bQ2qd=75W+yKATwoazxSNhv5sjW06fjGAtHbTUA@mail.gmail.com
11 months ago
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+---------------------------------------------------------------------------
|
|
|
|
t | 1 | COPY (DELETE FROM stats_track_tab WHERE x = 2 RETURNING x) TO stdout
|
|
|
|
t | 1 | COPY (INSERT INTO stats_track_tab (x) VALUES (1) RETURNING x) TO stdout
|
Track more precisely query locations for nested statements
Previously, a Query generated through the transform phase would have
unset stmt_location, tracking the starting point of a query string.
Extensions relying on the statement location to extract its relevant
parts in the source text string would fallback to use the whole
statement instead, leading to confusing results like in
pg_stat_statements for queries relying on nested queries, like:
- EXPLAIN, with top-level and nested query using the same query string,
and a query ID coming from the nested query when the non-top-level
entry.
- Multi-statements, with only partial portions of queries being
normalized.
- COPY TO with a query, SELECT or DMLs.
This patch improves things by keeping track of the statement locations
and propagate it to Query during transform, allowing PGSS to only show
the relevant part of the query for nested query. This leads to less
bloat in entries for non-top-level entries, as queries can now be
grouped within the same (toplevel, queryid) duos in pg_stat_statements.
The result gives a stricter one-one mapping between query IDs and its
query strings.
The regression tests introduced in 45e0ba30fc40 produce differences
reflecting the new logic.
Author: Anthonin Bonnefoy
Reviewed-by: Michael Paquier, Jian He
Discussion: https://postgr.es/m/CAO6_XqqM6S9bQ2qd=75W+yKATwoazxSNhv5sjW06fjGAtHbTUA@mail.gmail.com
11 months ago
|
|
|
t | 1 | COPY (MERGE INTO stats_track_tab USING (SELECT 1 id) ON x = id +
|
|
|
|
| | WHEN MATCHED THEN UPDATE SET x = id +
|
|
|
|
| | WHEN NOT MATCHED THEN INSERT (x) VALUES (id) RETURNING x) TO stdout
|
|
|
|
t | 1 | COPY (SELECT 1 UNION SELECT 2) TO stdout
|
|
|
|
t | 1 | COPY (SELECT 1) TO stdout
|
|
|
|
t | 1 | COPY (UPDATE stats_track_tab SET x = 2 WHERE x = 1 RETURNING x) TO stdout
|
Track more precisely query locations for nested statements
Previously, a Query generated through the transform phase would have
unset stmt_location, tracking the starting point of a query string.
Extensions relying on the statement location to extract its relevant
parts in the source text string would fallback to use the whole
statement instead, leading to confusing results like in
pg_stat_statements for queries relying on nested queries, like:
- EXPLAIN, with top-level and nested query using the same query string,
and a query ID coming from the nested query when the non-top-level
entry.
- Multi-statements, with only partial portions of queries being
normalized.
- COPY TO with a query, SELECT or DMLs.
This patch improves things by keeping track of the statement locations
and propagate it to Query during transform, allowing PGSS to only show
the relevant part of the query for nested query. This leads to less
bloat in entries for non-top-level entries, as queries can now be
grouped within the same (toplevel, queryid) duos in pg_stat_statements.
The result gives a stricter one-one mapping between query IDs and its
query strings.
The regression tests introduced in 45e0ba30fc40 produce differences
reflecting the new logic.
Author: Anthonin Bonnefoy
Reviewed-by: Michael Paquier, Jian He
Discussion: https://postgr.es/m/CAO6_XqqM6S9bQ2qd=75W+yKATwoazxSNhv5sjW06fjGAtHbTUA@mail.gmail.com
11 months ago
|
|
|
f | 1 | DELETE FROM stats_track_tab WHERE x = $1 RETURNING x
|
|
|
|
f | 1 | INSERT INTO stats_track_tab (x) VALUES ($1) RETURNING x
|
|
|
|
f | 1 | MERGE INTO stats_track_tab USING (SELECT $1 id) ON x = id +
|
|
|
|
| | WHEN MATCHED THEN UPDATE SET x = id +
|
|
|
|
| | WHEN NOT MATCHED THEN INSERT (x) VALUES (id) RETURNING x
|
|
|
|
f | 1 | SELECT $1
|
|
|
|
f | 1 | SELECT $1 UNION SELECT $2
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
Track more precisely query locations for nested statements
Previously, a Query generated through the transform phase would have
unset stmt_location, tracking the starting point of a query string.
Extensions relying on the statement location to extract its relevant
parts in the source text string would fallback to use the whole
statement instead, leading to confusing results like in
pg_stat_statements for queries relying on nested queries, like:
- EXPLAIN, with top-level and nested query using the same query string,
and a query ID coming from the nested query when the non-top-level
entry.
- Multi-statements, with only partial portions of queries being
normalized.
- COPY TO with a query, SELECT or DMLs.
This patch improves things by keeping track of the statement locations
and propagate it to Query during transform, allowing PGSS to only show
the relevant part of the query for nested query. This leads to less
bloat in entries for non-top-level entries, as queries can now be
grouped within the same (toplevel, queryid) duos in pg_stat_statements.
The result gives a stricter one-one mapping between query IDs and its
query strings.
The regression tests introduced in 45e0ba30fc40 produce differences
reflecting the new logic.
Author: Anthonin Bonnefoy
Reviewed-by: Michael Paquier, Jian He
Discussion: https://postgr.es/m/CAO6_XqqM6S9bQ2qd=75W+yKATwoazxSNhv5sjW06fjGAtHbTUA@mail.gmail.com
11 months ago
|
|
|
f | 1 | UPDATE stats_track_tab SET x = $1 WHERE x = $2 RETURNING x
|
|
|
|
(13 rows)
|
|
|
|
|
|
|
|
-- COPY - top-level tracking.
|
|
|
|
SET pg_stat_statements.track = 'top';
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
COPY (SELECT 1) TO stdout;
|
|
|
|
1
|
|
|
|
COPY (SELECT 1 UNION SELECT 2) TO stdout;
|
|
|
|
1
|
|
|
|
2
|
|
|
|
COPY (MERGE INTO stats_track_tab USING (SELECT 1 id) ON x = id
|
|
|
|
WHEN MATCHED THEN UPDATE SET x = id
|
|
|
|
WHEN NOT MATCHED THEN INSERT (x) VALUES (id) RETURNING x) TO stdout;
|
|
|
|
1
|
|
|
|
COPY (INSERT INTO stats_track_tab (x) VALUES (1) RETURNING x) TO stdout;
|
|
|
|
1
|
|
|
|
COPY (UPDATE stats_track_tab SET x = 2 WHERE x = 1 RETURNING x) TO stdout;
|
|
|
|
2
|
|
|
|
2
|
|
|
|
COPY (DELETE FROM stats_track_tab WHERE x = 2 RETURNING x) TO stdout;
|
|
|
|
2
|
|
|
|
2
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C";
|
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+---------------------------------------------------------------------------
|
|
|
|
t | 1 | COPY (DELETE FROM stats_track_tab WHERE x = 2 RETURNING x) TO stdout
|
|
|
|
t | 1 | COPY (INSERT INTO stats_track_tab (x) VALUES (1) RETURNING x) TO stdout
|
|
|
|
t | 1 | COPY (MERGE INTO stats_track_tab USING (SELECT 1 id) ON x = id +
|
|
|
|
| | WHEN MATCHED THEN UPDATE SET x = id +
|
|
|
|
| | WHEN NOT MATCHED THEN INSERT (x) VALUES (id) RETURNING x) TO stdout
|
|
|
|
t | 1 | COPY (SELECT 1 UNION SELECT 2) TO stdout
|
|
|
|
t | 1 | COPY (SELECT 1) TO stdout
|
|
|
|
t | 1 | COPY (UPDATE stats_track_tab SET x = 2 WHERE x = 1 RETURNING x) TO stdout
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
|
|
|
(7 rows)
|
|
|
|
|
Fix some issues with tracking nesting level in pg_stat_statements.
When we decide that we don't want to track execution time of a
specific planner or ProcessUtility call, we still have to increment
the nesting depth, or we'll make the wrong determination of whether
we are at top level when considering nested statements. (PREPARE
and EXECUTE are exceptions, for reasons explained in the code.)
Counting planner nesting depth separately from executor nesting depth
was a mistake: it causes us to make the wrong determination of whether
we are at top level when considering nested statements that get
executed during planning (as a result of constant-folding of
functions, for example). Merge those counters into one.
In passing, get rid of the PGSS_HANDLED_UTILITY macro in favor of
explicitly listing statement types. It seems somewhat coincidental
that PREPARE and EXECUTE are handled alike in each of the places where
that was used: the reasoning tends to be different for each one.
Thus, the macro seems as likely to encourage future bugs as prevent
them, since it's quite unclear whether any future statement type that
might need special-casing here would also need the same choices at
each spot.
Sergei Kornilov, Julien Rouhaud, and Tom Lane, per bug #17552 from
Maxim Boguk. This is pretty clearly a bug fix, but it's also a
behavioral change that might surprise somebody, so no back-patch.
Discussion: https://postgr.es/m/17552-213b534c56ab5d02@postgresql.org
2 years ago
|
|
|
-- DO block - top-level tracking without utility.
|
|
|
|
SET pg_stat_statements.track = 'top';
|
|
|
|
SET pg_stat_statements.track_utility = FALSE;
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
Fix some issues with tracking nesting level in pg_stat_statements.
When we decide that we don't want to track execution time of a
specific planner or ProcessUtility call, we still have to increment
the nesting depth, or we'll make the wrong determination of whether
we are at top level when considering nested statements. (PREPARE
and EXECUTE are exceptions, for reasons explained in the code.)
Counting planner nesting depth separately from executor nesting depth
was a mistake: it causes us to make the wrong determination of whether
we are at top level when considering nested statements that get
executed during planning (as a result of constant-folding of
functions, for example). Merge those counters into one.
In passing, get rid of the PGSS_HANDLED_UTILITY macro in favor of
explicitly listing statement types. It seems somewhat coincidental
that PREPARE and EXECUTE are handled alike in each of the places where
that was used: the reasoning tends to be different for each one.
Thus, the macro seems as likely to encourage future bugs as prevent
them, since it's quite unclear whether any future statement type that
might need special-casing here would also need the same choices at
each spot.
Sergei Kornilov, Julien Rouhaud, and Tom Lane, per bug #17552 from
Maxim Boguk. This is pretty clearly a bug fix, but it's also a
behavioral change that might surprise somebody, so no back-patch.
Discussion: https://postgr.es/m/17552-213b534c56ab5d02@postgresql.org
2 years ago
|
|
|
(1 row)
|
|
|
|
|
|
|
|
DELETE FROM stats_track_tab;
|
|
|
|
DO $$
|
|
|
|
BEGIN
|
|
|
|
DELETE FROM stats_track_tab;
|
|
|
|
END; $$;
|
|
|
|
DO LANGUAGE plpgsql $$
|
|
|
|
BEGIN
|
|
|
|
-- this is a SELECT
|
|
|
|
PERFORM 'hello world'::TEXT;
|
|
|
|
END; $$;
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C", toplevel;
|
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+----------------------------------------------------
|
Fix some issues with tracking nesting level in pg_stat_statements.
When we decide that we don't want to track execution time of a
specific planner or ProcessUtility call, we still have to increment
the nesting depth, or we'll make the wrong determination of whether
we are at top level when considering nested statements. (PREPARE
and EXECUTE are exceptions, for reasons explained in the code.)
Counting planner nesting depth separately from executor nesting depth
was a mistake: it causes us to make the wrong determination of whether
we are at top level when considering nested statements that get
executed during planning (as a result of constant-folding of
functions, for example). Merge those counters into one.
In passing, get rid of the PGSS_HANDLED_UTILITY macro in favor of
explicitly listing statement types. It seems somewhat coincidental
that PREPARE and EXECUTE are handled alike in each of the places where
that was used: the reasoning tends to be different for each one.
Thus, the macro seems as likely to encourage future bugs as prevent
them, since it's quite unclear whether any future statement type that
might need special-casing here would also need the same choices at
each spot.
Sergei Kornilov, Julien Rouhaud, and Tom Lane, per bug #17552 from
Maxim Boguk. This is pretty clearly a bug fix, but it's also a
behavioral change that might surprise somebody, so no back-patch.
Discussion: https://postgr.es/m/17552-213b534c56ab5d02@postgresql.org
2 years ago
|
|
|
t | 1 | DELETE FROM stats_track_tab
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
Fix some issues with tracking nesting level in pg_stat_statements.
When we decide that we don't want to track execution time of a
specific planner or ProcessUtility call, we still have to increment
the nesting depth, or we'll make the wrong determination of whether
we are at top level when considering nested statements. (PREPARE
and EXECUTE are exceptions, for reasons explained in the code.)
Counting planner nesting depth separately from executor nesting depth
was a mistake: it causes us to make the wrong determination of whether
we are at top level when considering nested statements that get
executed during planning (as a result of constant-folding of
functions, for example). Merge those counters into one.
In passing, get rid of the PGSS_HANDLED_UTILITY macro in favor of
explicitly listing statement types. It seems somewhat coincidental
that PREPARE and EXECUTE are handled alike in each of the places where
that was used: the reasoning tends to be different for each one.
Thus, the macro seems as likely to encourage future bugs as prevent
them, since it's quite unclear whether any future statement type that
might need special-casing here would also need the same choices at
each spot.
Sergei Kornilov, Julien Rouhaud, and Tom Lane, per bug #17552 from
Maxim Boguk. This is pretty clearly a bug fix, but it's also a
behavioral change that might surprise somebody, so no back-patch.
Discussion: https://postgr.es/m/17552-213b534c56ab5d02@postgresql.org
2 years ago
|
|
|
(2 rows)
|
|
|
|
|
|
|
|
-- DO block - all-level tracking without utility.
|
|
|
|
SET pg_stat_statements.track = 'all';
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
Fix some issues with tracking nesting level in pg_stat_statements.
When we decide that we don't want to track execution time of a
specific planner or ProcessUtility call, we still have to increment
the nesting depth, or we'll make the wrong determination of whether
we are at top level when considering nested statements. (PREPARE
and EXECUTE are exceptions, for reasons explained in the code.)
Counting planner nesting depth separately from executor nesting depth
was a mistake: it causes us to make the wrong determination of whether
we are at top level when considering nested statements that get
executed during planning (as a result of constant-folding of
functions, for example). Merge those counters into one.
In passing, get rid of the PGSS_HANDLED_UTILITY macro in favor of
explicitly listing statement types. It seems somewhat coincidental
that PREPARE and EXECUTE are handled alike in each of the places where
that was used: the reasoning tends to be different for each one.
Thus, the macro seems as likely to encourage future bugs as prevent
them, since it's quite unclear whether any future statement type that
might need special-casing here would also need the same choices at
each spot.
Sergei Kornilov, Julien Rouhaud, and Tom Lane, per bug #17552 from
Maxim Boguk. This is pretty clearly a bug fix, but it's also a
behavioral change that might surprise somebody, so no back-patch.
Discussion: https://postgr.es/m/17552-213b534c56ab5d02@postgresql.org
2 years ago
|
|
|
(1 row)
|
|
|
|
|
|
|
|
DELETE FROM stats_track_tab;
|
|
|
|
DO $$
|
|
|
|
BEGIN
|
|
|
|
DELETE FROM stats_track_tab;
|
|
|
|
END; $$;
|
|
|
|
DO LANGUAGE plpgsql $$
|
|
|
|
BEGIN
|
|
|
|
-- this is a SELECT
|
|
|
|
PERFORM 'hello world'::TEXT;
|
|
|
|
END; $$;
|
|
|
|
SELECT toplevel, calls, query FROM pg_stat_statements
|
|
|
|
ORDER BY query COLLATE "C", toplevel;
|
|
|
|
toplevel | calls | query
|
|
|
|
----------+-------+----------------------------------------------------
|
Fix some issues with tracking nesting level in pg_stat_statements.
When we decide that we don't want to track execution time of a
specific planner or ProcessUtility call, we still have to increment
the nesting depth, or we'll make the wrong determination of whether
we are at top level when considering nested statements. (PREPARE
and EXECUTE are exceptions, for reasons explained in the code.)
Counting planner nesting depth separately from executor nesting depth
was a mistake: it causes us to make the wrong determination of whether
we are at top level when considering nested statements that get
executed during planning (as a result of constant-folding of
functions, for example). Merge those counters into one.
In passing, get rid of the PGSS_HANDLED_UTILITY macro in favor of
explicitly listing statement types. It seems somewhat coincidental
that PREPARE and EXECUTE are handled alike in each of the places where
that was used: the reasoning tends to be different for each one.
Thus, the macro seems as likely to encourage future bugs as prevent
them, since it's quite unclear whether any future statement type that
might need special-casing here would also need the same choices at
each spot.
Sergei Kornilov, Julien Rouhaud, and Tom Lane, per bug #17552 from
Maxim Boguk. This is pretty clearly a bug fix, but it's also a
behavioral change that might surprise somebody, so no back-patch.
Discussion: https://postgr.es/m/17552-213b534c56ab5d02@postgresql.org
2 years ago
|
|
|
f | 1 | DELETE FROM stats_track_tab
|
|
|
|
t | 1 | DELETE FROM stats_track_tab
|
|
|
|
f | 1 | SELECT $1::TEXT
|
|
|
|
t | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
Fix some issues with tracking nesting level in pg_stat_statements.
When we decide that we don't want to track execution time of a
specific planner or ProcessUtility call, we still have to increment
the nesting depth, or we'll make the wrong determination of whether
we are at top level when considering nested statements. (PREPARE
and EXECUTE are exceptions, for reasons explained in the code.)
Counting planner nesting depth separately from executor nesting depth
was a mistake: it causes us to make the wrong determination of whether
we are at top level when considering nested statements that get
executed during planning (as a result of constant-folding of
functions, for example). Merge those counters into one.
In passing, get rid of the PGSS_HANDLED_UTILITY macro in favor of
explicitly listing statement types. It seems somewhat coincidental
that PREPARE and EXECUTE are handled alike in each of the places where
that was used: the reasoning tends to be different for each one.
Thus, the macro seems as likely to encourage future bugs as prevent
them, since it's quite unclear whether any future statement type that
might need special-casing here would also need the same choices at
each spot.
Sergei Kornilov, Julien Rouhaud, and Tom Lane, per bug #17552 from
Maxim Boguk. This is pretty clearly a bug fix, but it's also a
behavioral change that might surprise somebody, so no back-patch.
Discussion: https://postgr.es/m/17552-213b534c56ab5d02@postgresql.org
2 years ago
|
|
|
(4 rows)
|
|
|
|
|
Refactor tests of pg_stat_statements for planning, utility and level tracking
pg_stat_statements.sql acts as the main file for all the core tests of
the module, but things have become complicated to follow over the years
as some of the sub-scenarios tested in this file rely on assumptions
that come from completely different areas of it, like a GUC setup or a
relation created previously. For example, row tracking for CTAS/COPY
was looking at the number of plans, which was not necessary, or level
tracking was mixed with checks on planner counts.
This commit refactors the tests of pg_stat_statements, by moving test
cases out of pg_stat_statements.sql into their own file, as of:
- Planning-related tests in planning.sql, for [re]plan counts and
top-level handling. These depend on pg_stat_statements.track_planning.
- Utilities in utility.sql (pg_stat_statements.track_utility), that
includes now the tests for:
-- Row tracking for CTAS, CREATE MATERIALIZED VIEW, COPY.
-- Basic utility statements.
-- SET statements.
- Tracking level, depending on pg_stat_statements.track. This part has
been looking at scenarios with DO blocks, PL functions and SQL
functions.
pg_stat_statements.sql (still named the same for now) still includes
some checks for role-level tracking and WAL generation metrics, that
ought to become independent in the long term for clarity.
While on it, this switches the order of the attributes when querying
pg_stat_statements, the query field becoming last. This makes much
easier the tracking of changes related to normalization, as queries are
the only variable-length attributes queried (unaligned mode would be one
extra choice, but that reduces the checks on the other fields).
Test scenarios and their results match exactly with what was happening
before this commit in terms of calls, number of plans, number of rows,
cached data or level tracking, so this has no effect on the coverage in
terms of what is produced by the reports in the table
pg_stat_statements. A follow-up patch will extend more the tests of
pg_stat_statements around utilities, so this split creates a foundation
for this purpose, without complicating more pg_stat_statements.sql.
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/Y+MRdEq9W9XVa2AB@paquier.xyz
3 years ago
|
|
|
-- PL/pgSQL function - top-level tracking.
|
|
|
|
SET pg_stat_statements.track = 'top';
|
|
|
|
SET pg_stat_statements.track_utility = FALSE;
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
Refactor tests of pg_stat_statements for planning, utility and level tracking
pg_stat_statements.sql acts as the main file for all the core tests of
the module, but things have become complicated to follow over the years
as some of the sub-scenarios tested in this file rely on assumptions
that come from completely different areas of it, like a GUC setup or a
relation created previously. For example, row tracking for CTAS/COPY
was looking at the number of plans, which was not necessary, or level
tracking was mixed with checks on planner counts.
This commit refactors the tests of pg_stat_statements, by moving test
cases out of pg_stat_statements.sql into their own file, as of:
- Planning-related tests in planning.sql, for [re]plan counts and
top-level handling. These depend on pg_stat_statements.track_planning.
- Utilities in utility.sql (pg_stat_statements.track_utility), that
includes now the tests for:
-- Row tracking for CTAS, CREATE MATERIALIZED VIEW, COPY.
-- Basic utility statements.
-- SET statements.
- Tracking level, depending on pg_stat_statements.track. This part has
been looking at scenarios with DO blocks, PL functions and SQL
functions.
pg_stat_statements.sql (still named the same for now) still includes
some checks for role-level tracking and WAL generation metrics, that
ought to become independent in the long term for clarity.
While on it, this switches the order of the attributes when querying
pg_stat_statements, the query field becoming last. This makes much
easier the tracking of changes related to normalization, as queries are
the only variable-length attributes queried (unaligned mode would be one
extra choice, but that reduces the checks on the other fields).
Test scenarios and their results match exactly with what was happening
before this commit in terms of calls, number of plans, number of rows,
cached data or level tracking, so this has no effect on the coverage in
terms of what is produced by the reports in the table
pg_stat_statements. A follow-up patch will extend more the tests of
pg_stat_statements around utilities, so this split creates a foundation
for this purpose, without complicating more pg_stat_statements.sql.
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/Y+MRdEq9W9XVa2AB@paquier.xyz
3 years ago
|
|
|
(1 row)
|
|
|
|
|
|
|
|
CREATE FUNCTION PLUS_TWO(i INTEGER) RETURNS INTEGER AS $$
|
|
|
|
DECLARE
|
|
|
|
r INTEGER;
|
|
|
|
BEGIN
|
|
|
|
SELECT (i + 1 + 1.0)::INTEGER INTO r;
|
|
|
|
RETURN r;
|
|
|
|
END; $$ LANGUAGE plpgsql;
|
|
|
|
SELECT PLUS_TWO(3);
|
|
|
|
plus_two
|
|
|
|
----------
|
|
|
|
5
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
SELECT PLUS_TWO(7);
|
|
|
|
plus_two
|
|
|
|
----------
|
|
|
|
9
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
-- SQL function --- use LIMIT to keep it from being inlined
|
|
|
|
CREATE FUNCTION PLUS_ONE(i INTEGER) RETURNS INTEGER AS
|
|
|
|
$$ SELECT (i + 1.0)::INTEGER LIMIT 1 $$ LANGUAGE SQL;
|
|
|
|
SELECT PLUS_ONE(8);
|
|
|
|
plus_one
|
|
|
|
----------
|
|
|
|
9
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
SELECT PLUS_ONE(10);
|
|
|
|
plus_one
|
|
|
|
----------
|
|
|
|
11
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
|
|
|
|
calls | rows | query
|
|
|
|
-------+------+----------------------------------------------------
|
Refactor tests of pg_stat_statements for planning, utility and level tracking
pg_stat_statements.sql acts as the main file for all the core tests of
the module, but things have become complicated to follow over the years
as some of the sub-scenarios tested in this file rely on assumptions
that come from completely different areas of it, like a GUC setup or a
relation created previously. For example, row tracking for CTAS/COPY
was looking at the number of plans, which was not necessary, or level
tracking was mixed with checks on planner counts.
This commit refactors the tests of pg_stat_statements, by moving test
cases out of pg_stat_statements.sql into their own file, as of:
- Planning-related tests in planning.sql, for [re]plan counts and
top-level handling. These depend on pg_stat_statements.track_planning.
- Utilities in utility.sql (pg_stat_statements.track_utility), that
includes now the tests for:
-- Row tracking for CTAS, CREATE MATERIALIZED VIEW, COPY.
-- Basic utility statements.
-- SET statements.
- Tracking level, depending on pg_stat_statements.track. This part has
been looking at scenarios with DO blocks, PL functions and SQL
functions.
pg_stat_statements.sql (still named the same for now) still includes
some checks for role-level tracking and WAL generation metrics, that
ought to become independent in the long term for clarity.
While on it, this switches the order of the attributes when querying
pg_stat_statements, the query field becoming last. This makes much
easier the tracking of changes related to normalization, as queries are
the only variable-length attributes queried (unaligned mode would be one
extra choice, but that reduces the checks on the other fields).
Test scenarios and their results match exactly with what was happening
before this commit in terms of calls, number of plans, number of rows,
cached data or level tracking, so this has no effect on the coverage in
terms of what is produced by the reports in the table
pg_stat_statements. A follow-up patch will extend more the tests of
pg_stat_statements around utilities, so this split creates a foundation
for this purpose, without complicating more pg_stat_statements.sql.
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/Y+MRdEq9W9XVa2AB@paquier.xyz
3 years ago
|
|
|
2 | 2 | SELECT PLUS_ONE($1)
|
|
|
|
2 | 2 | SELECT PLUS_TWO($1)
|
|
|
|
1 | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
Refactor tests of pg_stat_statements for planning, utility and level tracking
pg_stat_statements.sql acts as the main file for all the core tests of
the module, but things have become complicated to follow over the years
as some of the sub-scenarios tested in this file rely on assumptions
that come from completely different areas of it, like a GUC setup or a
relation created previously. For example, row tracking for CTAS/COPY
was looking at the number of plans, which was not necessary, or level
tracking was mixed with checks on planner counts.
This commit refactors the tests of pg_stat_statements, by moving test
cases out of pg_stat_statements.sql into their own file, as of:
- Planning-related tests in planning.sql, for [re]plan counts and
top-level handling. These depend on pg_stat_statements.track_planning.
- Utilities in utility.sql (pg_stat_statements.track_utility), that
includes now the tests for:
-- Row tracking for CTAS, CREATE MATERIALIZED VIEW, COPY.
-- Basic utility statements.
-- SET statements.
- Tracking level, depending on pg_stat_statements.track. This part has
been looking at scenarios with DO blocks, PL functions and SQL
functions.
pg_stat_statements.sql (still named the same for now) still includes
some checks for role-level tracking and WAL generation metrics, that
ought to become independent in the long term for clarity.
While on it, this switches the order of the attributes when querying
pg_stat_statements, the query field becoming last. This makes much
easier the tracking of changes related to normalization, as queries are
the only variable-length attributes queried (unaligned mode would be one
extra choice, but that reduces the checks on the other fields).
Test scenarios and their results match exactly with what was happening
before this commit in terms of calls, number of plans, number of rows,
cached data or level tracking, so this has no effect on the coverage in
terms of what is produced by the reports in the table
pg_stat_statements. A follow-up patch will extend more the tests of
pg_stat_statements around utilities, so this split creates a foundation
for this purpose, without complicating more pg_stat_statements.sql.
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/Y+MRdEq9W9XVa2AB@paquier.xyz
3 years ago
|
|
|
(3 rows)
|
|
|
|
|
Fix some issues with tracking nesting level in pg_stat_statements.
When we decide that we don't want to track execution time of a
specific planner or ProcessUtility call, we still have to increment
the nesting depth, or we'll make the wrong determination of whether
we are at top level when considering nested statements. (PREPARE
and EXECUTE are exceptions, for reasons explained in the code.)
Counting planner nesting depth separately from executor nesting depth
was a mistake: it causes us to make the wrong determination of whether
we are at top level when considering nested statements that get
executed during planning (as a result of constant-folding of
functions, for example). Merge those counters into one.
In passing, get rid of the PGSS_HANDLED_UTILITY macro in favor of
explicitly listing statement types. It seems somewhat coincidental
that PREPARE and EXECUTE are handled alike in each of the places where
that was used: the reasoning tends to be different for each one.
Thus, the macro seems as likely to encourage future bugs as prevent
them, since it's quite unclear whether any future statement type that
might need special-casing here would also need the same choices at
each spot.
Sergei Kornilov, Julien Rouhaud, and Tom Lane, per bug #17552 from
Maxim Boguk. This is pretty clearly a bug fix, but it's also a
behavioral change that might surprise somebody, so no back-patch.
Discussion: https://postgr.es/m/17552-213b534c56ab5d02@postgresql.org
2 years ago
|
|
|
-- immutable SQL function --- can be executed at plan time
|
|
|
|
CREATE FUNCTION PLUS_THREE(i INTEGER) RETURNS INTEGER AS
|
|
|
|
$$ SELECT i + 3 LIMIT 1 $$ IMMUTABLE LANGUAGE SQL;
|
|
|
|
SELECT PLUS_THREE(8);
|
|
|
|
plus_three
|
|
|
|
------------
|
|
|
|
11
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
SELECT PLUS_THREE(10);
|
|
|
|
plus_three
|
|
|
|
------------
|
|
|
|
13
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
SELECT toplevel, calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
|
|
|
|
toplevel | calls | rows | query
|
|
|
|
----------+-------+------+------------------------------------------------------------------------------
|
|
|
|
t | 2 | 2 | SELECT PLUS_ONE($1)
|
|
|
|
t | 2 | 2 | SELECT PLUS_THREE($1)
|
|
|
|
t | 2 | 2 | SELECT PLUS_TWO($1)
|
|
|
|
t | 1 | 3 | SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C"
|
|
|
|
t | 1 | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
Fix some issues with tracking nesting level in pg_stat_statements.
When we decide that we don't want to track execution time of a
specific planner or ProcessUtility call, we still have to increment
the nesting depth, or we'll make the wrong determination of whether
we are at top level when considering nested statements. (PREPARE
and EXECUTE are exceptions, for reasons explained in the code.)
Counting planner nesting depth separately from executor nesting depth
was a mistake: it causes us to make the wrong determination of whether
we are at top level when considering nested statements that get
executed during planning (as a result of constant-folding of
functions, for example). Merge those counters into one.
In passing, get rid of the PGSS_HANDLED_UTILITY macro in favor of
explicitly listing statement types. It seems somewhat coincidental
that PREPARE and EXECUTE are handled alike in each of the places where
that was used: the reasoning tends to be different for each one.
Thus, the macro seems as likely to encourage future bugs as prevent
them, since it's quite unclear whether any future statement type that
might need special-casing here would also need the same choices at
each spot.
Sergei Kornilov, Julien Rouhaud, and Tom Lane, per bug #17552 from
Maxim Boguk. This is pretty clearly a bug fix, but it's also a
behavioral change that might surprise somebody, so no back-patch.
Discussion: https://postgr.es/m/17552-213b534c56ab5d02@postgresql.org
2 years ago
|
|
|
(5 rows)
|
|
|
|
|
Refactor tests of pg_stat_statements for planning, utility and level tracking
pg_stat_statements.sql acts as the main file for all the core tests of
the module, but things have become complicated to follow over the years
as some of the sub-scenarios tested in this file rely on assumptions
that come from completely different areas of it, like a GUC setup or a
relation created previously. For example, row tracking for CTAS/COPY
was looking at the number of plans, which was not necessary, or level
tracking was mixed with checks on planner counts.
This commit refactors the tests of pg_stat_statements, by moving test
cases out of pg_stat_statements.sql into their own file, as of:
- Planning-related tests in planning.sql, for [re]plan counts and
top-level handling. These depend on pg_stat_statements.track_planning.
- Utilities in utility.sql (pg_stat_statements.track_utility), that
includes now the tests for:
-- Row tracking for CTAS, CREATE MATERIALIZED VIEW, COPY.
-- Basic utility statements.
-- SET statements.
- Tracking level, depending on pg_stat_statements.track. This part has
been looking at scenarios with DO blocks, PL functions and SQL
functions.
pg_stat_statements.sql (still named the same for now) still includes
some checks for role-level tracking and WAL generation metrics, that
ought to become independent in the long term for clarity.
While on it, this switches the order of the attributes when querying
pg_stat_statements, the query field becoming last. This makes much
easier the tracking of changes related to normalization, as queries are
the only variable-length attributes queried (unaligned mode would be one
extra choice, but that reduces the checks on the other fields).
Test scenarios and their results match exactly with what was happening
before this commit in terms of calls, number of plans, number of rows,
cached data or level tracking, so this has no effect on the coverage in
terms of what is produced by the reports in the table
pg_stat_statements. A follow-up patch will extend more the tests of
pg_stat_statements around utilities, so this split creates a foundation
for this purpose, without complicating more pg_stat_statements.sql.
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/Y+MRdEq9W9XVa2AB@paquier.xyz
3 years ago
|
|
|
-- PL/pgSQL function - all-level tracking.
|
|
|
|
SET pg_stat_statements.track = 'all';
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
Refactor tests of pg_stat_statements for planning, utility and level tracking
pg_stat_statements.sql acts as the main file for all the core tests of
the module, but things have become complicated to follow over the years
as some of the sub-scenarios tested in this file rely on assumptions
that come from completely different areas of it, like a GUC setup or a
relation created previously. For example, row tracking for CTAS/COPY
was looking at the number of plans, which was not necessary, or level
tracking was mixed with checks on planner counts.
This commit refactors the tests of pg_stat_statements, by moving test
cases out of pg_stat_statements.sql into their own file, as of:
- Planning-related tests in planning.sql, for [re]plan counts and
top-level handling. These depend on pg_stat_statements.track_planning.
- Utilities in utility.sql (pg_stat_statements.track_utility), that
includes now the tests for:
-- Row tracking for CTAS, CREATE MATERIALIZED VIEW, COPY.
-- Basic utility statements.
-- SET statements.
- Tracking level, depending on pg_stat_statements.track. This part has
been looking at scenarios with DO blocks, PL functions and SQL
functions.
pg_stat_statements.sql (still named the same for now) still includes
some checks for role-level tracking and WAL generation metrics, that
ought to become independent in the long term for clarity.
While on it, this switches the order of the attributes when querying
pg_stat_statements, the query field becoming last. This makes much
easier the tracking of changes related to normalization, as queries are
the only variable-length attributes queried (unaligned mode would be one
extra choice, but that reduces the checks on the other fields).
Test scenarios and their results match exactly with what was happening
before this commit in terms of calls, number of plans, number of rows,
cached data or level tracking, so this has no effect on the coverage in
terms of what is produced by the reports in the table
pg_stat_statements. A follow-up patch will extend more the tests of
pg_stat_statements around utilities, so this split creates a foundation
for this purpose, without complicating more pg_stat_statements.sql.
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/Y+MRdEq9W9XVa2AB@paquier.xyz
3 years ago
|
|
|
(1 row)
|
|
|
|
|
|
|
|
-- we drop and recreate the functions to avoid any caching funnies
|
|
|
|
DROP FUNCTION PLUS_ONE(INTEGER);
|
|
|
|
DROP FUNCTION PLUS_TWO(INTEGER);
|
Fix some issues with tracking nesting level in pg_stat_statements.
When we decide that we don't want to track execution time of a
specific planner or ProcessUtility call, we still have to increment
the nesting depth, or we'll make the wrong determination of whether
we are at top level when considering nested statements. (PREPARE
and EXECUTE are exceptions, for reasons explained in the code.)
Counting planner nesting depth separately from executor nesting depth
was a mistake: it causes us to make the wrong determination of whether
we are at top level when considering nested statements that get
executed during planning (as a result of constant-folding of
functions, for example). Merge those counters into one.
In passing, get rid of the PGSS_HANDLED_UTILITY macro in favor of
explicitly listing statement types. It seems somewhat coincidental
that PREPARE and EXECUTE are handled alike in each of the places where
that was used: the reasoning tends to be different for each one.
Thus, the macro seems as likely to encourage future bugs as prevent
them, since it's quite unclear whether any future statement type that
might need special-casing here would also need the same choices at
each spot.
Sergei Kornilov, Julien Rouhaud, and Tom Lane, per bug #17552 from
Maxim Boguk. This is pretty clearly a bug fix, but it's also a
behavioral change that might surprise somebody, so no back-patch.
Discussion: https://postgr.es/m/17552-213b534c56ab5d02@postgresql.org
2 years ago
|
|
|
DROP FUNCTION PLUS_THREE(INTEGER);
|
Refactor tests of pg_stat_statements for planning, utility and level tracking
pg_stat_statements.sql acts as the main file for all the core tests of
the module, but things have become complicated to follow over the years
as some of the sub-scenarios tested in this file rely on assumptions
that come from completely different areas of it, like a GUC setup or a
relation created previously. For example, row tracking for CTAS/COPY
was looking at the number of plans, which was not necessary, or level
tracking was mixed with checks on planner counts.
This commit refactors the tests of pg_stat_statements, by moving test
cases out of pg_stat_statements.sql into their own file, as of:
- Planning-related tests in planning.sql, for [re]plan counts and
top-level handling. These depend on pg_stat_statements.track_planning.
- Utilities in utility.sql (pg_stat_statements.track_utility), that
includes now the tests for:
-- Row tracking for CTAS, CREATE MATERIALIZED VIEW, COPY.
-- Basic utility statements.
-- SET statements.
- Tracking level, depending on pg_stat_statements.track. This part has
been looking at scenarios with DO blocks, PL functions and SQL
functions.
pg_stat_statements.sql (still named the same for now) still includes
some checks for role-level tracking and WAL generation metrics, that
ought to become independent in the long term for clarity.
While on it, this switches the order of the attributes when querying
pg_stat_statements, the query field becoming last. This makes much
easier the tracking of changes related to normalization, as queries are
the only variable-length attributes queried (unaligned mode would be one
extra choice, but that reduces the checks on the other fields).
Test scenarios and their results match exactly with what was happening
before this commit in terms of calls, number of plans, number of rows,
cached data or level tracking, so this has no effect on the coverage in
terms of what is produced by the reports in the table
pg_stat_statements. A follow-up patch will extend more the tests of
pg_stat_statements around utilities, so this split creates a foundation
for this purpose, without complicating more pg_stat_statements.sql.
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/Y+MRdEq9W9XVa2AB@paquier.xyz
3 years ago
|
|
|
-- PL/pgSQL function
|
|
|
|
CREATE FUNCTION PLUS_TWO(i INTEGER) RETURNS INTEGER AS $$
|
|
|
|
DECLARE
|
|
|
|
r INTEGER;
|
|
|
|
BEGIN
|
|
|
|
SELECT (i + 1 + 1.0)::INTEGER INTO r;
|
|
|
|
RETURN r;
|
|
|
|
END; $$ LANGUAGE plpgsql;
|
|
|
|
SELECT PLUS_TWO(-1);
|
|
|
|
plus_two
|
|
|
|
----------
|
|
|
|
1
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
SELECT PLUS_TWO(2);
|
|
|
|
plus_two
|
|
|
|
----------
|
|
|
|
4
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
-- SQL function --- use LIMIT to keep it from being inlined
|
|
|
|
CREATE FUNCTION PLUS_ONE(i INTEGER) RETURNS INTEGER AS
|
|
|
|
$$ SELECT (i + 1.0)::INTEGER LIMIT 1 $$ LANGUAGE SQL;
|
|
|
|
SELECT PLUS_ONE(3);
|
|
|
|
plus_one
|
|
|
|
----------
|
|
|
|
4
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
SELECT PLUS_ONE(1);
|
|
|
|
plus_one
|
|
|
|
----------
|
|
|
|
2
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
|
|
|
|
calls | rows | query
|
|
|
|
-------+------+----------------------------------------------------
|
Refactor tests of pg_stat_statements for planning, utility and level tracking
pg_stat_statements.sql acts as the main file for all the core tests of
the module, but things have become complicated to follow over the years
as some of the sub-scenarios tested in this file rely on assumptions
that come from completely different areas of it, like a GUC setup or a
relation created previously. For example, row tracking for CTAS/COPY
was looking at the number of plans, which was not necessary, or level
tracking was mixed with checks on planner counts.
This commit refactors the tests of pg_stat_statements, by moving test
cases out of pg_stat_statements.sql into their own file, as of:
- Planning-related tests in planning.sql, for [re]plan counts and
top-level handling. These depend on pg_stat_statements.track_planning.
- Utilities in utility.sql (pg_stat_statements.track_utility), that
includes now the tests for:
-- Row tracking for CTAS, CREATE MATERIALIZED VIEW, COPY.
-- Basic utility statements.
-- SET statements.
- Tracking level, depending on pg_stat_statements.track. This part has
been looking at scenarios with DO blocks, PL functions and SQL
functions.
pg_stat_statements.sql (still named the same for now) still includes
some checks for role-level tracking and WAL generation metrics, that
ought to become independent in the long term for clarity.
While on it, this switches the order of the attributes when querying
pg_stat_statements, the query field becoming last. This makes much
easier the tracking of changes related to normalization, as queries are
the only variable-length attributes queried (unaligned mode would be one
extra choice, but that reduces the checks on the other fields).
Test scenarios and their results match exactly with what was happening
before this commit in terms of calls, number of plans, number of rows,
cached data or level tracking, so this has no effect on the coverage in
terms of what is produced by the reports in the table
pg_stat_statements. A follow-up patch will extend more the tests of
pg_stat_statements around utilities, so this split creates a foundation
for this purpose, without complicating more pg_stat_statements.sql.
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/Y+MRdEq9W9XVa2AB@paquier.xyz
3 years ago
|
|
|
2 | 2 | SELECT (i + $2 + $3)::INTEGER
|
|
|
|
2 | 2 | SELECT (i + $2)::INTEGER LIMIT $3
|
|
|
|
2 | 2 | SELECT PLUS_ONE($1)
|
|
|
|
2 | 2 | SELECT PLUS_TWO($1)
|
|
|
|
1 | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
Refactor tests of pg_stat_statements for planning, utility and level tracking
pg_stat_statements.sql acts as the main file for all the core tests of
the module, but things have become complicated to follow over the years
as some of the sub-scenarios tested in this file rely on assumptions
that come from completely different areas of it, like a GUC setup or a
relation created previously. For example, row tracking for CTAS/COPY
was looking at the number of plans, which was not necessary, or level
tracking was mixed with checks on planner counts.
This commit refactors the tests of pg_stat_statements, by moving test
cases out of pg_stat_statements.sql into their own file, as of:
- Planning-related tests in planning.sql, for [re]plan counts and
top-level handling. These depend on pg_stat_statements.track_planning.
- Utilities in utility.sql (pg_stat_statements.track_utility), that
includes now the tests for:
-- Row tracking for CTAS, CREATE MATERIALIZED VIEW, COPY.
-- Basic utility statements.
-- SET statements.
- Tracking level, depending on pg_stat_statements.track. This part has
been looking at scenarios with DO blocks, PL functions and SQL
functions.
pg_stat_statements.sql (still named the same for now) still includes
some checks for role-level tracking and WAL generation metrics, that
ought to become independent in the long term for clarity.
While on it, this switches the order of the attributes when querying
pg_stat_statements, the query field becoming last. This makes much
easier the tracking of changes related to normalization, as queries are
the only variable-length attributes queried (unaligned mode would be one
extra choice, but that reduces the checks on the other fields).
Test scenarios and their results match exactly with what was happening
before this commit in terms of calls, number of plans, number of rows,
cached data or level tracking, so this has no effect on the coverage in
terms of what is produced by the reports in the table
pg_stat_statements. A follow-up patch will extend more the tests of
pg_stat_statements around utilities, so this split creates a foundation
for this purpose, without complicating more pg_stat_statements.sql.
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/Y+MRdEq9W9XVa2AB@paquier.xyz
3 years ago
|
|
|
(5 rows)
|
|
|
|
|
Fix some issues with tracking nesting level in pg_stat_statements.
When we decide that we don't want to track execution time of a
specific planner or ProcessUtility call, we still have to increment
the nesting depth, or we'll make the wrong determination of whether
we are at top level when considering nested statements. (PREPARE
and EXECUTE are exceptions, for reasons explained in the code.)
Counting planner nesting depth separately from executor nesting depth
was a mistake: it causes us to make the wrong determination of whether
we are at top level when considering nested statements that get
executed during planning (as a result of constant-folding of
functions, for example). Merge those counters into one.
In passing, get rid of the PGSS_HANDLED_UTILITY macro in favor of
explicitly listing statement types. It seems somewhat coincidental
that PREPARE and EXECUTE are handled alike in each of the places where
that was used: the reasoning tends to be different for each one.
Thus, the macro seems as likely to encourage future bugs as prevent
them, since it's quite unclear whether any future statement type that
might need special-casing here would also need the same choices at
each spot.
Sergei Kornilov, Julien Rouhaud, and Tom Lane, per bug #17552 from
Maxim Boguk. This is pretty clearly a bug fix, but it's also a
behavioral change that might surprise somebody, so no back-patch.
Discussion: https://postgr.es/m/17552-213b534c56ab5d02@postgresql.org
2 years ago
|
|
|
-- immutable SQL function --- can be executed at plan time
|
|
|
|
CREATE FUNCTION PLUS_THREE(i INTEGER) RETURNS INTEGER AS
|
|
|
|
$$ SELECT i + 3 LIMIT 1 $$ IMMUTABLE LANGUAGE SQL;
|
|
|
|
SELECT PLUS_THREE(8);
|
|
|
|
plus_three
|
|
|
|
------------
|
|
|
|
11
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
SELECT PLUS_THREE(10);
|
|
|
|
plus_three
|
|
|
|
------------
|
|
|
|
13
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
SELECT toplevel, calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
|
|
|
|
toplevel | calls | rows | query
|
|
|
|
----------+-------+------+------------------------------------------------------------------------------
|
|
|
|
f | 2 | 2 | SELECT (i + $2 + $3)::INTEGER
|
|
|
|
f | 2 | 2 | SELECT (i + $2)::INTEGER LIMIT $3
|
|
|
|
t | 2 | 2 | SELECT PLUS_ONE($1)
|
|
|
|
t | 2 | 2 | SELECT PLUS_THREE($1)
|
|
|
|
t | 2 | 2 | SELECT PLUS_TWO($1)
|
|
|
|
t | 1 | 5 | SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C"
|
|
|
|
f | 2 | 2 | SELECT i + $2 LIMIT $3
|
|
|
|
t | 1 | 1 | SELECT pg_stat_statements_reset() IS NOT NULL AS t
|
Fix some issues with tracking nesting level in pg_stat_statements.
When we decide that we don't want to track execution time of a
specific planner or ProcessUtility call, we still have to increment
the nesting depth, or we'll make the wrong determination of whether
we are at top level when considering nested statements. (PREPARE
and EXECUTE are exceptions, for reasons explained in the code.)
Counting planner nesting depth separately from executor nesting depth
was a mistake: it causes us to make the wrong determination of whether
we are at top level when considering nested statements that get
executed during planning (as a result of constant-folding of
functions, for example). Merge those counters into one.
In passing, get rid of the PGSS_HANDLED_UTILITY macro in favor of
explicitly listing statement types. It seems somewhat coincidental
that PREPARE and EXECUTE are handled alike in each of the places where
that was used: the reasoning tends to be different for each one.
Thus, the macro seems as likely to encourage future bugs as prevent
them, since it's quite unclear whether any future statement type that
might need special-casing here would also need the same choices at
each spot.
Sergei Kornilov, Julien Rouhaud, and Tom Lane, per bug #17552 from
Maxim Boguk. This is pretty clearly a bug fix, but it's also a
behavioral change that might surprise somebody, so no back-patch.
Discussion: https://postgr.es/m/17552-213b534c56ab5d02@postgresql.org
2 years ago
|
|
|
(8 rows)
|
|
|
|
|
Refactor tests of pg_stat_statements for planning, utility and level tracking
pg_stat_statements.sql acts as the main file for all the core tests of
the module, but things have become complicated to follow over the years
as some of the sub-scenarios tested in this file rely on assumptions
that come from completely different areas of it, like a GUC setup or a
relation created previously. For example, row tracking for CTAS/COPY
was looking at the number of plans, which was not necessary, or level
tracking was mixed with checks on planner counts.
This commit refactors the tests of pg_stat_statements, by moving test
cases out of pg_stat_statements.sql into their own file, as of:
- Planning-related tests in planning.sql, for [re]plan counts and
top-level handling. These depend on pg_stat_statements.track_planning.
- Utilities in utility.sql (pg_stat_statements.track_utility), that
includes now the tests for:
-- Row tracking for CTAS, CREATE MATERIALIZED VIEW, COPY.
-- Basic utility statements.
-- SET statements.
- Tracking level, depending on pg_stat_statements.track. This part has
been looking at scenarios with DO blocks, PL functions and SQL
functions.
pg_stat_statements.sql (still named the same for now) still includes
some checks for role-level tracking and WAL generation metrics, that
ought to become independent in the long term for clarity.
While on it, this switches the order of the attributes when querying
pg_stat_statements, the query field becoming last. This makes much
easier the tracking of changes related to normalization, as queries are
the only variable-length attributes queried (unaligned mode would be one
extra choice, but that reduces the checks on the other fields).
Test scenarios and their results match exactly with what was happening
before this commit in terms of calls, number of plans, number of rows,
cached data or level tracking, so this has no effect on the coverage in
terms of what is produced by the reports in the table
pg_stat_statements. A follow-up patch will extend more the tests of
pg_stat_statements around utilities, so this split creates a foundation
for this purpose, without complicating more pg_stat_statements.sql.
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/Y+MRdEq9W9XVa2AB@paquier.xyz
3 years ago
|
|
|
--
|
|
|
|
-- pg_stat_statements.track = none
|
|
|
|
--
|
|
|
|
SET pg_stat_statements.track = 'none';
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
Refactor tests of pg_stat_statements for planning, utility and level tracking
pg_stat_statements.sql acts as the main file for all the core tests of
the module, but things have become complicated to follow over the years
as some of the sub-scenarios tested in this file rely on assumptions
that come from completely different areas of it, like a GUC setup or a
relation created previously. For example, row tracking for CTAS/COPY
was looking at the number of plans, which was not necessary, or level
tracking was mixed with checks on planner counts.
This commit refactors the tests of pg_stat_statements, by moving test
cases out of pg_stat_statements.sql into their own file, as of:
- Planning-related tests in planning.sql, for [re]plan counts and
top-level handling. These depend on pg_stat_statements.track_planning.
- Utilities in utility.sql (pg_stat_statements.track_utility), that
includes now the tests for:
-- Row tracking for CTAS, CREATE MATERIALIZED VIEW, COPY.
-- Basic utility statements.
-- SET statements.
- Tracking level, depending on pg_stat_statements.track. This part has
been looking at scenarios with DO blocks, PL functions and SQL
functions.
pg_stat_statements.sql (still named the same for now) still includes
some checks for role-level tracking and WAL generation metrics, that
ought to become independent in the long term for clarity.
While on it, this switches the order of the attributes when querying
pg_stat_statements, the query field becoming last. This makes much
easier the tracking of changes related to normalization, as queries are
the only variable-length attributes queried (unaligned mode would be one
extra choice, but that reduces the checks on the other fields).
Test scenarios and their results match exactly with what was happening
before this commit in terms of calls, number of plans, number of rows,
cached data or level tracking, so this has no effect on the coverage in
terms of what is produced by the reports in the table
pg_stat_statements. A follow-up patch will extend more the tests of
pg_stat_statements around utilities, so this split creates a foundation
for this purpose, without complicating more pg_stat_statements.sql.
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/Y+MRdEq9W9XVa2AB@paquier.xyz
3 years ago
|
|
|
(1 row)
|
|
|
|
|
|
|
|
SELECT 1 AS "one";
|
|
|
|
one
|
|
|
|
-----
|
|
|
|
1
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
SELECT 1 + 1 AS "two";
|
|
|
|
two
|
|
|
|
-----
|
|
|
|
2
|
|
|
|
(1 row)
|
|
|
|
|
|
|
|
SELECT calls, rows, query FROM pg_stat_statements ORDER BY query COLLATE "C";
|
|
|
|
calls | rows | query
|
|
|
|
-------+------+-------
|
|
|
|
(0 rows)
|
|
|
|
|
|
|
|
SELECT pg_stat_statements_reset() IS NOT NULL AS t;
|
|
|
|
t
|
|
|
|
---
|
|
|
|
t
|
|
|
|
(1 row)
|
|
|
|
|