postgres

Commit Graph

Author	SHA1	Message	Date
Amit Langote	f9a468c664	Fix bogus ctid requirement for dummy-root partitioned targets ExecInitModifyTable() unconditionally required a ctid junk column even when the target was a partitioned table. This led to spurious "could not find junk ctid column" errors when all children were excluded and only the dummy root result relation remained. A partitioned table only appears in the result relations list when all leaf partitions have been pruned, leaving the dummy root as the sole entry. Assert this invariant (nrels == 1) and skip the ctid requirement. Also adjust ExecModifyTable() to tolerate invalid ri_RowIdAttNo for partitioned tables, which is safe since no rows will be processed in this case. Bug: #19099 Reported-by: Alexander Lakhin <exclusion@gmail.com> Author: Amit Langote <amitlangote09@gmail.com> Reviewed-by: Tender Wang <tndrwang@gmail.com> Reviewed-by: Kirill Reshke <reshkekirill@gmail.com> Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/19099-e05dcfa022fe553d%40postgresql.org Backpatch-through: 14	1 month ago
Fujii Masao	26cb14aea1	file_fdw: Support multi-line HEADER option. Commit `bc2f348` introduced multi-line HEADER support for COPY. This commit extends this capability to file_fdw, allowing multiple header lines to be skipped. Because CREATE/ALTER FOREIGN TABLE requires option values to be single-quoted, this commit also updates defGetCopyHeaderOption() to accept integer values specified as strings for HEADER option. Author: Shinya Kato <shinya11.kato@gmail.com> Reviewed-by: Fujii Masao <masao.fujii@gmail.com> Reviewed-by: songjinzhou <tsinghualucky912@foxmail.com> Reviewed-by: Japin Li <japinli@hotmail.com> Reviewed-by: Chao Li <li.evan.chao@gmail.com> Discussion: https://postgr.es/m/CAOzEurT+iwC47VHPMS+uJ4WSzvOLPsZ2F2_wopm8M7O+CZa3Xw@mail.gmail.com	1 month ago
Tom Lane	aa87f69c00	Disallow "=" in names of reloptions and foreign-data options. We store values for these options as array elements with the syntax "name=value", hence a name containing "=" confuses matters when it's time to read the array back in. Since validation of the options is often done (long) after this conversion to array format, that leads to confusing and off-point error messages. We can improve matters by rejecting names containing "=" up-front. (Probably a better design would have involved pairs of array elements, but it's too late now --- and anyway, there's no evident use-case for option names like this. We already reject such names in some other contexts such as GUCs.) Reported-by: Chapman Flack <jcflack@acm.org> Author: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Chapman Flack <jcflack@acm.org> Discussion: https://postgr.es/m/6830EB30.8090904@acm.org Backpatch-through: 13	9 months ago
Fujii Masao	631db1d4b2	file_fdw: Add regression tests for ON_ERROR and other options. This commit introduces regression tests to validate incorrect settings for the ON_ERROR, LOG_VERBOSITY, and REJECT_LIMIT options in file_fdw. Author: Atsushi Torikoshi Reviewed-by: Fujii Masao Suggested-by: Yugo Nagata Discussion: https://postgr.es/m/20241113231706.09e5b5ea9640289312835be8@sraoss.co.jp	1 year ago
Fujii Masao	6c8f670323	file_fdw: Add REJECT_LIMIT option to file_fdw. Commit `4ac2a9bece` introduced the REJECT_LIMIT option for the COPY command. This commit extends the support for this option to file_fdw. As well as REJECT_LIMIT option for COPY, this option limits the maximum number of erroneous rows that can be skipped. If the number of data type conversion errors exceeds this limit, accessing the file_fdw foreign table will fail with an error, even when on_error = 'ignore' is specified. Since the CREATE/ALTER FOREIGN TABLE commands require foreign table options to be single-quoted, this commit updates defGetCopyRejectLimitOption() to handle also string value for them, in addition to int64 value for COPY command option. Author: Atsushi Torikoshi Reviewed-by: Fujii Masao, Yugo Nagata, Kirill Reshke Discussion: https://postgr.es/m/bab68a9fc502b12693f0755b6f35f327@oss.nttdata.com	1 year ago
Fujii Masao	a1c4c8a9e1	file_fdw: Add on_error and log_verbosity options to file_fdw. In v17, the on_error and log_verbosity options were introduced for the COPY command. This commit extends support for these options to file_fdw. Setting on_error = 'ignore' for a file_fdw foreign table allows users to query it without errors, even when the input file contains malformed rows, by skipping the problematic rows. Both on_error and log_verbosity options apply to SELECT and ANALYZE operations on file_fdw foreign tables. Author: Atsushi Torikoshi Reviewed-by: Masahiko Sawada, Fujii Masao Discussion: https://postgr.es/m/ab59dad10490ea3734cf022b16c24cfd@oss.nttdata.com	1 year ago
Andrew Dunstan	9f8377f7a2	Add a DEFAULT option to COPY FROM This allows for a string which if an input field matches causes the column's default value to be inserted. The advantage of this is that the default can be inserted in some rows and not others, for which non-default data is available. The file_fdw extension is also modified to take allow use of this option. Israel Barth Rubio Discussion: https://postgr.es/m/CAO_rXXAcqesk6DsvioOZ5zmeEmpUN5ktZf-9=9yu+DTr0Xr8Uw@mail.gmail.com	3 years ago
Fujii Masao	ecc84b916f	Add regression test for TRUNCATE on foreign table not supporting TRUNCATE. file_fdw doesn't support INSERT, UPDATE, DELETE and TRUNCATE. It has the regression test that confirms that INSERT, UPDATE and DELETE fail on its foreign table, but not TRUNCATE yet. It's better to also test TRUNCATE fails on a foreign table not allowing TRUNCATE, for test coverage. This commit adds that regression test using file_fdw. Author: Yugo Nagata Reviewed-by: Fujii Masao Discussion: https://postgr.es/m/20220630104812.ec1556481452c019874f4ac9@sraoss.co.jp	4 years ago
Peter Eisentraut	072132f04e	Add header matching mode to COPY FROM COPY FROM supports the HEADER option to silently discard the header line from a CSV or text file. It is possible to load by mistake a file that matches the expected format, for example, if two text columns have been swapped, resulting in garbage in the database. This adds a new option value HEADER MATCH that checks the column names in the header line against the actual column names and errors out if they do not match. Author: Rémi Lapeyre <remi.lapeyre@lenstra.fr> Reviewed-by: Daniel Verite <daniel@manitou-mail.org> Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com> Discussion: https://www.postgresql.org/message-id/flat/CAF1-J-0PtCWMeLtswwGV2M70U26n4g33gpe1rcKQqe6wVQDrFA@mail.gmail.com	4 years ago
Peter Eisentraut	43f33dc018	Add HEADER support to COPY text format The COPY CSV format supports the HEADER option to output a header line. This patch adds the same option to the default text format. On input, the HEADER option causes the first line to be skipped, same as with CSV. Author: Rémi Lapeyre <remi.lapeyre@lenstra.fr> Discussion: https://www.postgresql.org/message-id/flat/CAF1-J-0PtCWMeLtswwGV2M70U26n4g33gpe1rcKQqe6wVQDrFA@mail.gmail.com	4 years ago
Tom Lane	dc9c3b0ff2	Remove dynamic translation of regression test scripts, step 2. "git mv" all the input/.source and output/.source files into the corresponding sql/ and expected/ directories. Then remove the pg_regress and Makefile infrastructure associated with dynamic translation. Discussion: https://postgr.es/m/1655733.1639871614@sss.pgh.pa.us	4 years ago
Tom Lane	7c5d0ae707	Add contrib/file_fdw foreign-data wrapper for reading files via COPY. This is both very useful in its own right, and an important test case for the core FDW support. This commit includes a small refactoring of copy.c to expose its option checking code as a separately callable function. The original patch submission duplicated hundreds of lines of that code, which seemed pretty unmaintainable. Shigeru Hanada, reviewed by Itagaki Takahiro and Tom Lane	15 years ago

12 Commits (e35add48ccc2e5aa94de360f1a43c6c150bda54a)