162 Commits (b1ecd19c5d19815b69e425d80f442bf2877cab76)

Author	SHA1	Message	Date
Andrew Morgan	ac99774dac	Rewrite complement.sh (#9685) ... This PR rewrites the original complement.sh script with a number of improvements: * We can now use a local checkout of Complement (configurable with `COMPLEMENT_DIR`), though the default behaviour still downloads the master branch. * You can now specify a regex of test names to run, or just run all tests. * We now use the Synapse test blacklist tag (so all tests will pass).	4 years ago
Andrew Morgan	f380bb77d1	Use 'dmypy run' in lint.sh instead of 'mypy' (#9701) ... For it's obvious performance benefits. `dmypy` support landed in #9692.	4 years ago
Quentin Gliech	d4c4798a25	Use interpreter from $PATH instead of absolute paths in various scripts using /usr/bin/env (#9689) ... On NixOS, `bash` isn't under `/bin/bash` but rather in some directory in `$PATH`. Locally, I've been patching those scripts to make them work. `/usr/bin/env` seems to be the only [portable way](https://unix.stackexchange.com/questions/29608/why-is-it-better-to-use-usr-bin-env-name-instead-of-path-to-name-as-my) to use binaries from the PATH as interpreters. Signed-off-by: Quentin Gliech <quentingliech@gmail.com>	4 years ago
Richard van der Hoff	0e35584734	federation_client: handle inline signing_keys in hs.yaml (#9647)	4 years ago
Richard van der Hoff	201178db1a	federation_client: stop adding URL prefix (#9645)	4 years ago
Andrew Morgan	22db45bd4d	Prevent the config-lint script erroring out on any sample_config changes (#9562) ... I noticed that I'd occasionally have `scripts-dev/lint.sh` fail when messing about with config options in my PR. The script calls `scripts-dev/config-lint.sh`, which attempts some validation on the sample config. It does this by using `sed` to edit the sample_config, and then seeing if the file changed using `git diff`. The problem is: if you changed the sample_config as part of your commit, this script will error regardless. This PR attempts to change the check so that existing, unstaged changes to the sample_config will not cause the script to report an invalid file.	4 years ago
Eric Eastwood	0a00b7ff14	Update black, and run auto formatting over the codebase (#9381) ... - Update black version to the latest - Run black auto formatting over the codebase - Run autoformatting according to [`docs/code_style.md `](80d6dc9783/docs/code_style.md) - Update `code_style.md` docs around installing black to use the correct version	4 years ago
Eric Eastwood	80d6dc9783	Remove conflicting sqlite tables that are "reserved" (shadow fts4 tables) (#9003) ... Remove conflicting sqlite tables that throw sqlite3.OperationalError: object name reserved for internal use: event_search_content when running the twisted unit tests. Fix #8996	4 years ago
Andrew Morgan	31c5382d7a	Align the directories linted in CI with the defaults in scripts-dev/lint.sh (#9191) ... The lists of source directories to lint between `tox.ini` and `lint.sh` became out of sync. This PR tightens them up and adds some comments reminding any future readers to keep the list in sync.	4 years ago
Patrick Cloke	5d34f40d49	Add type hints to the push module. (#8901)	4 years ago
Richard van der Hoff	97b35ee259	Add a script to sign arbitrary json objects. (#8772)	4 years ago
Erik Johnston	22eeb6bc54	Fix cache call signature to accept `on_invalidate`. (#8684) ... Cached functions accept an `on_invalidate` function, which we failed to add to the type signature. It's rarely used in the files that we have typed, which is why we haven't noticed it before.	4 years ago
Patrick Cloke	00b24aa545	Support generating structured logs in addition to standard logs. (#8607) ... This modifies the configuration of structured logging to be usable from the standard Python logging configuration. This also separates the formatting of logs from the transport allowing JSON logs to files or standard logs to sockets.	4 years ago
kleph	29ce6d43b5	Run mypy as part of the lint.sh script. (#8633)	4 years ago
Andrew Morgan	654e239b25	Add option to scripts-dev/lint.sh to only lint files changed since the last git commit (#8472) ... This PR makes several changes to the `./scripts-dev/lint.sh` script, which lints the codebase with a number of tools: * Adds usage information, with `-h` flag to show it. Otherwise it will show when providing an unknown flag. * Adds option `-d` which will check both staged and unstaged files that have changed since the last commit and add them to the list of files to lint. - Note that only files without an extension, or with a `.py` extension will be allowed. This prevents editing bash scripts causing the linters to break on non-python files. * Improves the print-out of which files/directories are being linted.	4 years ago
Patrick Cloke	d9b55bd830	Add Ubuntu 20.10 (Groovy Gorilla) to build scripts. (#8475)	4 years ago
Andrew Morgan	01f82bfe32	Remove docs/sphinx and related references (#8480) ... https://github.com/matrix-org/synapse/tree/develop/docs/sphinx doesn't seem to really be utilised or changed recently since the initial commit. I like the idea of exportable documentation of the codebase, but at the moment after running through the build instructions the generated website wasn't very useful...	4 years ago
Andrew Morgan	12f0d18611	Add support for running Complement against the local checkout (#8317) ... This PR adds a script that: * Builds the local Synapse checkout using our existing `docker/Dockerfile` image. * Downloads [Complement](https://github.com/matrix-org/complement/)'s source code. * Builds the [Synapse.Dockerfile](https://github.com/matrix-org/complement/blob/master/dockerfiles/Synapse.Dockerfile) using the above dockerfile as a base. * Builds and runs Complement against it. This set up differs slightly from [that of the dendrite repo](https://github.com/matrix-org/dendrite/blob/master/build/scripts/complement.sh) (`complement.sh`, `Complement.Dockerfile`), which instead stores a separate, but slightly modified, dockerfile in Dendrite's repo rather than running the one stored in Complement's repo. That synapse equivalent to that dockerfile (`Synapse.Dockerfile`) in Complement's repo is just based on top of `matrixdotorg/synapse:latest`, which we opt to build here locally. Thus copying over the files from Complement's repo wouldn't change any functionality, and would result in two instances of the same files. So just using the dockerfile in Complement's repo was decided upon instead.	4 years ago
Patrick Cloke	8a4a4186de	Simplify super() calls to Python 3 syntax. (#8344) ... This converts calls like super(Foo, self) -> super(). Generated with: sed -i "" -Ee 's/super\([^\(]+\)/super()/g' **/*.py	4 years ago
Jonathan de Jong	837293c314	Remove obsolete __future__ imports (#8337)	4 years ago
Patrick Cloke	c619253db8	Stop sub-classing object (#8249)	4 years ago
Erik Johnston	208e1d3eb3	Fix typing for `@cached` wrapped functions (#8240) ... This requires adding a mypy plugin to fiddle with the type signatures a bit.	4 years ago
Richard van der Hoff	88b9807ba4	Update the test federation client to handle streaming responses (#8130) ... Now that the server supports streaming back JSON responses, it would be nice to show the response as it is streamed, in the test tool.	4 years ago
Erik Johnston	a7bdf98d01	Rename database classes to make some sense (#8033)	4 years ago
Andrew Morgan	7cac9006d6	Spruce up the check-newsfragment CI output (#8024) ... This PR: * Reduces the amount of noise in the `check-newsfragment` CI output by hiding the dependency installation output by default. * Prints a link to the changelog/debian changelog section of the contributing guide if an error is found.	4 years ago
Richard van der Hoff	8078dec3be	Fix exit code for `check_line_terminators.sh` (#7970) ... If there are *no* files with CRLF line endings, then the xargs exits with a non-zero exit code (as expected), but then, since that is the last thing to happen in the script, the script as a whole exits non-zero, making the whole thing fail. using `if/then/fi` instead of `&& (...)` means that the script exits with a zero exit code.	4 years ago
Andrew Morgan	c4ce0da6fe	Add script for finding files with unix line terminators (#7965) ... This PRs adds a script to check for unix-line terminators in the repo. It will be used to address https://github.com/matrix-org/synapse/issues/7943 by adding the check to CI. I've changed the original script slightly as proposed in https://github.com/matrix-org/pipelines/pull/81#discussion_r460580664	4 years ago
Andrew Morgan	b7ddece2a6	Lint the contrib/ directory in CI and linting scripts, add synctl to linting script (#7914) ... Run `isort`, `flake8` and `black` over the `contrib/` directory and `synctl` script. The latter was already being done in CI, but now the linting script does it too. Fixes https://github.com/matrix-org/synapse/issues/7910	4 years ago
Gary Kim	f2af3e4fc5	Remove Ubuntu Eoan that is now EOL (#7888)	4 years ago
Will Hunt	62b1ce8539	isort 5 compatibility (#7786) ... The CI appears to use the latest version of isort, which is a problem when isort gets a major version bump. Rather than try to pin the version, I've done the necessary to make isort5 happy with synapse.	4 years ago
Andrew Morgan	177b2d0c19	Move flake8 to end. Don't exit script on failure (#7738)	4 years ago
Dagfinn Ilmari Mannsåker	a3f11567d9	Replace all remaining six usage with native Python 3 equivalents (#7704)	5 years ago
Richard van der Hoff	86d814cdde	Check the changelog number in check-newsfragment (#7623)	5 years ago
Patrick Cloke	b2b8699070	Remove Ubuntu Cosmic and Disco which are both EOL. (#7539)	5 years ago
Richard van der Hoff	1fc8914f76	update dh-virtualenv (#7526)	5 years ago
Richard van der Hoff	e6027562e2	remove `builtins.buffer` code from storage code ... this is no longer needed on python 3	5 years ago
Patrick Cloke	509e381afa	Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957) ... Ensure good comprehension hygiene using flake8-comprehensions.	5 years ago
Patrick Cloke	ba547ec3a9	Use BSD-compatible in-place editing for sed. (#6887)	5 years ago
Brendan Abolivier	07124d028d	Port synapse_port_db to async/await (#6718) ... * Raise an exception if there are pending background updates So we return with a non-0 code * Changelog * Port synapse_port_db to async/await * Port update_database to async/await * Add version string to mocked homeservers * Remove unused imports * Convert overseen bits to async/await * Fixup logging contexts * Fix imports * Add a way to print an error without raising an exception * Incorporate review	5 years ago
Richard van der Hoff	08815566bc	Automate generation of the sample and debian log configs (#6627)	5 years ago
Erik Johnston	2284eb3a53	Add database config class (#6513) ... This encapsulates config for a given database and is the way to get new connections.	5 years ago
Andrew Morgan	6676ee9c4a	Add dev script to generate full SQL schema files (#6394)	5 years ago
Erik Johnston	67c991b78f	Fix upgrade db script	5 years ago
Erik Johnston	8b77fc6506	Fix DB scripts	5 years ago
Erik Johnston	ee86abb2d6	Remove underscore from SQLBaseStore functions	5 years ago
Richard van der Hoff	4257feb20f	build debs for eoan and bullseye	5 years ago
Neil Pilgrim	befd58f47b	Document lint.sh & allow application to specified files only (#6312)	5 years ago
Andrew Morgan	2cab02f9d1	Update CI to run isort on scripts and scripts-dev (#6270)	5 years ago
Brendan Abolivier	14504ad573	Add CI for synapse_port_db (#6140) ... This adds: * a test sqlite database * a configuration file for the sqlite database * a configuration file for a postgresql database (using the credentials in `.buildkite/docker-compose.pyXX.pgXX.yaml`) as well as a new script named `.buildkite/scripts/test_synapse_port_db.sh` that: 1. installs Synapse 2. updates the test sqlite database to the latest schema and runs background updates on it 3. creates an empty postgresql database 4. run the `synapse_port_db` script to migrate the test sqlite database to the empty postgresql database (with coverage) Step `2` is done via a new script located at `scripts-dev/update_database`. The test sqlite database is extracted from a SyTest run, so that it can be considered as an actual homeserver's database with actual data in it.	5 years ago
Andrew Morgan	409c62b27b	Add config linting script that checks for bool casing (#6203) ... Add a linting script that enforces all boolean values in the default config be lowercase. This has annoyed me for a while so I decided to fix it.	5 years ago