watcha
/
watcha-synapse
mirror of https://github.com/watcha-fr/synapse
1
1
Fork 0
Code Issues Projects Releases Wiki Activity

Merge branch 'release-v1.86' into develop

Browse Source
1.103.0-whithout-watcha
Mathieu Velten 2 years ago
parent 21fea6b749 825c5909de
commit 87e5df9a6e
38 changed files with 103 additions and 99 deletions
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Show Stats Download Patch File Download Diff File
  1. 79
      CHANGES.md
  2. 1
      changelog.d/12504.misc
  3. 1
      changelog.d/14213.misc
  4. 1
      changelog.d/15388.feature
  5. 1
      changelog.d/15450.feature
  6. 1
      changelog.d/15520.feature
  7. 1
      changelog.d/15582.feature
  8. 1
      changelog.d/15609.bugfix
  9. 1
      changelog.d/15649.misc
  10. 1
      changelog.d/15650.misc
  11. 1
      changelog.d/15674.feature
  12. 1
      changelog.d/15675.misc
  13. 1
      changelog.d/15689.misc
  14. 1
      changelog.d/15690.misc
  15. 1
      changelog.d/15694.misc
  16. 1
      changelog.d/15695.bugfix
  17. 1
      changelog.d/15697.misc
  18. 1
      changelog.d/15705.feature
  19. 1
      changelog.d/15709.misc
  20. 1
      changelog.d/15710.feature
  21. 1
      changelog.d/15721.misc
  22. 1
      changelog.d/15723.misc
  23. 1
      changelog.d/15725.misc
  24. 1
      changelog.d/15726.misc
  25. 1
      changelog.d/15729.misc
  26. 1
      changelog.d/15731.misc
  27. 1
      changelog.d/15732.doc
  28. 1
      changelog.d/15733.misc
  29. 1
      changelog.d/15740.feature
  30. 1
      changelog.d/15752.misc
  31. 1
      changelog.d/15768.misc
  32. 12
      debian/changelog
  33. 26
      docs/usage/configuration/config_documentation.md
  34. 2
      pyproject.toml
  35. 10
      synapse/config/federation.py
  36. 21
      synapse/http/matrixfederationclient.py
  37. 2
      synapse/types/__init__.py
  38. 20
      tests/http/test_matrixfederationclient.py

79
CHANGES.md
Unescape View File

@ -1,3 +1,82 @@
Synapse 1.86.0rc2 (2023-06-14)
==============================
Bugfixes
--------
- Fix an error when having workers of different versions running. ([\#15774](https://github.com/matrix-org/synapse/issues/15774))
Synapse 1.86.0rc1 (2023-06-13)
==============================
This version was tagged but never released.
Features
--------
- Stable support for [MSC3882](https://github.com/matrix-org/matrix-spec-proposals/pull/3882) to allow an existing device/session to generate a login token for use on a new device/session. ([\#15388](https://github.com/matrix-org/synapse/issues/15388))
- Support resolving a room's [canonical alias](https://spec.matrix.org/v1.7/client-server-api/#mroomcanonical_alias) via the module API. ([\#15450](https://github.com/matrix-org/synapse/issues/15450))
- Enable support for [MSC3952](https://github.com/matrix-org/matrix-spec-proposals/pull/3952): intentional mentions. ([\#15520](https://github.com/matrix-org/synapse/issues/15520))
- Experimental [MSC3861](https://github.com/matrix-org/matrix-spec-proposals/pull/3861) support: delegate auth to an OIDC provider. ([\#15582](https://github.com/matrix-org/synapse/issues/15582))
- Add Synapse version deploy annotations to Grafana dashboard which enables easy correlation between behavior changes witnessed in a graph to a certain Synapse version and nail down regressions. ([\#15674](https://github.com/matrix-org/synapse/issues/15674))
- Add a catch-all * to the supported relation types when redacting an event and its related events. This is an update to [MSC3912](https://github.com/matrix-org/matrix-spec-proposals/pull/3861) implementation. ([\#15705](https://github.com/matrix-org/synapse/issues/15705))
- Speed up `/messages` by backfilling in the background when there are no backward extremities where we are directly paginating. ([\#15710](https://github.com/matrix-org/synapse/issues/15710))
- Expose a metric reporting the database background update status. ([\#15740](https://github.com/matrix-org/synapse/issues/15740))
Bugfixes
--------
- Correctly clear caches when we delete a room. ([\#15609](https://github.com/matrix-org/synapse/issues/15609))
- Check permissions for enabling encryption earlier during room creation to avoid creating broken rooms. ([\#15695](https://github.com/matrix-org/synapse/issues/15695))
Improved Documentation
----------------------
- Simplify query to find participating servers in a room. ([\#15732](https://github.com/matrix-org/synapse/issues/15732))
Internal Changes
----------------
- Log when events are (maybe unexpectedly) filtered out of responses in tests. ([\#14213](https://github.com/matrix-org/synapse/issues/14213))
- Read from column `full_user_id` rather than `user_id` of tables `profiles` and `user_filters`. ([\#15649](https://github.com/matrix-org/synapse/issues/15649))
- Add support for tracing functions which return `Awaitable`s. ([\#15650](https://github.com/matrix-org/synapse/issues/15650))
- Cache requests for user's devices over federation. ([\#15675](https://github.com/matrix-org/synapse/issues/15675))
- Add fully qualified docker image names to Dockerfiles. ([\#15689](https://github.com/matrix-org/synapse/issues/15689))
- Remove some unused code. ([\#15690](https://github.com/matrix-org/synapse/issues/15690))
- Improve type hints. ([\#15694](https://github.com/matrix-org/synapse/issues/15694), [\#15697](https://github.com/matrix-org/synapse/issues/15697))
- Update docstring and traces on `maybe_backfill()` functions. ([\#15709](https://github.com/matrix-org/synapse/issues/15709))
- Add context for when/why to use the `long_retries` option when sending Federation requests. ([\#15721](https://github.com/matrix-org/synapse/issues/15721))
- Removed some unused fields. ([\#15723](https://github.com/matrix-org/synapse/issues/15723))
- Update federation error to more plainly explain we can only authorize our own membership events. ([\#15725](https://github.com/matrix-org/synapse/issues/15725))
- Prevent the `latest_deps` and `twisted_trunk` daily GitHub Actions workflows from running on forks of the codebase. ([\#15726](https://github.com/matrix-org/synapse/issues/15726))
- Improve performance of user directory search. ([\#15729](https://github.com/matrix-org/synapse/issues/15729))
- Remove redundant table join with `room_memberships` when doing a `is_host_joined()`/`is_host_invited()` call (`membership` is already part of the `current_state_events`). ([\#15731](https://github.com/matrix-org/synapse/issues/15731))
- Remove superfluous `room_memberships` join from background update. ([\#15733](https://github.com/matrix-org/synapse/issues/15733))
- Speed up typechecking CI. ([\#15752](https://github.com/matrix-org/synapse/issues/15752))
- Bump minimum supported Rust version to 1.60.0. ([\#15768](https://github.com/matrix-org/synapse/issues/15768))
### Updates to locked dependencies
* Bump importlib-metadata from 6.1.0 to 6.6.0. ([\#15711](https://github.com/matrix-org/synapse/issues/15711))
* Bump library/redis from 6-bullseye to 7-bullseye in /docker. ([\#15712](https://github.com/matrix-org/synapse/issues/15712))
* Bump log from 0.4.18 to 0.4.19. ([\#15761](https://github.com/matrix-org/synapse/issues/15761))
* Bump phonenumbers from 8.13.11 to 8.13.13. ([\#15763](https://github.com/matrix-org/synapse/issues/15763))
* Bump pyasn1 from 0.4.8 to 0.5.0. ([\#15713](https://github.com/matrix-org/synapse/issues/15713))
* Bump pydantic from 1.10.8 to 1.10.9. ([\#15762](https://github.com/matrix-org/synapse/issues/15762))
* Bump pyo3-log from 0.8.1 to 0.8.2. ([\#15759](https://github.com/matrix-org/synapse/issues/15759))
* Bump pyopenssl from 23.1.1 to 23.2.0. ([\#15765](https://github.com/matrix-org/synapse/issues/15765))
* Bump regex from 1.7.3 to 1.8.4. ([\#15769](https://github.com/matrix-org/synapse/issues/15769))
* Bump sentry-sdk from 1.22.1 to 1.25.0. ([\#15714](https://github.com/matrix-org/synapse/issues/15714))
* Bump sentry-sdk from 1.25.0 to 1.25.1. ([\#15764](https://github.com/matrix-org/synapse/issues/15764))
* Bump serde from 1.0.163 to 1.0.164. ([\#15760](https://github.com/matrix-org/synapse/issues/15760))
* Bump types-jsonschema from 4.17.0.7 to 4.17.0.8. ([\#15716](https://github.com/matrix-org/synapse/issues/15716))
* Bump types-pyopenssl from 23.1.0.2 to 23.2.0.0. ([\#15766](https://github.com/matrix-org/synapse/issues/15766))
* Bump types-requests from 2.31.0.0 to 2.31.0.1. ([\#15715](https://github.com/matrix-org/synapse/issues/15715))
Synapse 1.85.2 (2023-06-08)
===========================

1
changelog.d/12504.misc
Unescape View File

@ -1 +0,0 @@
Allow for the configuration of max request retries and min/max retry delays in the matrix federation client.

1
changelog.d/14213.misc
Unescape View File

@ -1 +0,0 @@
Log when events are (maybe unexpectedly) filtered out of responses in tests.

1
changelog.d/15388.feature
Unescape View File

@ -1 +0,0 @@
Stable support for [MSC3882](https://github.com/matrix-org/matrix-spec-proposals/pull/3882) to allow an existing device/session to generate a login token for use on a new device/session.

1
changelog.d/15450.feature
Unescape View File

@ -1 +0,0 @@
Support resolving a room's [canonical alias](https://spec.matrix.org/v1.7/client-server-api/#mroomcanonical_alias) via the module API.

1
changelog.d/15520.feature
Unescape View File

@ -1 +0,0 @@
Enable support for [MSC3952](https://github.com/matrix-org/matrix-spec-proposals/pull/3952): intentional mentions.

1
changelog.d/15582.feature
Unescape View File

@ -1 +0,0 @@
Experimental [MSC3861](https://github.com/matrix-org/matrix-spec-proposals/pull/3861) support: delegate auth to an OIDC provider.

1
changelog.d/15609.bugfix
Unescape View File

@ -1 +0,0 @@
Correctly clear caches when we delete a room.

1
changelog.d/15649.misc
Unescape View File

@ -1 +0,0 @@
Read from column `full_user_id` rather than `user_id` of tables `profiles` and `user_filters`.

1
changelog.d/15650.misc
Unescape View File

@ -1 +0,0 @@
Add support for tracing functions which return `Awaitable`s.

1
changelog.d/15674.feature
Unescape View File

@ -1 +0,0 @@
Add Syanpse version deploy annotations to Grafana dashboard which enables easy correlation between behavior changes witnessed in a graph to a certain Synapse version and nail down regressions.

1
changelog.d/15675.misc
Unescape View File

@ -1 +0,0 @@
Cache requests for user's devices over federation.

1
changelog.d/15689.misc
Unescape View File

@ -1 +0,0 @@
Add fully qualified docker image names to Dockerfiles.

1
changelog.d/15690.misc
Unescape View File

@ -1 +0,0 @@
Remove some unused code.

1
changelog.d/15694.misc
Unescape View File

@ -1 +0,0 @@
Improve type hints.

1
changelog.d/15695.bugfix
Unescape View File

@ -1 +0,0 @@
Check permissions for enabling encryption earlier during room creation to avoid creating broken rooms.

1
changelog.d/15697.misc
Unescape View File

@ -1 +0,0 @@
Improve type hints.

1
changelog.d/15705.feature
Unescape View File

@ -1 +0,0 @@
Add a catch-all * to the supported relation types when redacting an event and its related events. This is an update to [MSC3912](https://github.com/matrix-org/matrix-spec-proposals/pull/3861) implementation.

1
changelog.d/15709.misc
Unescape View File

@ -1 +0,0 @@
Update docstring and traces on `maybe_backfill()` functions.

1
changelog.d/15710.feature
Unescape View File

@ -1 +0,0 @@
Speed up `/messages` by backfilling in the background when there are no backward extremities where we are directly paginating.

1
changelog.d/15721.misc
Unescape View File

@ -1 +0,0 @@
Add context for when/why to use the `long_retries` option when sending Federation requests.

1
changelog.d/15723.misc
Unescape View File

@ -1 +0,0 @@
Removed some unused fields.

1
changelog.d/15725.misc
Unescape View File

@ -1 +0,0 @@
Update federation error to more plainly explain we can only authorize our own membership events.

1
changelog.d/15726.misc
Unescape View File

@ -1 +0,0 @@
Prevent the `latest_deps` and `twisted_trunk` daily GitHub Actions workflows from running on forks of the codebase.

1
changelog.d/15729.misc
Unescape View File

@ -1 +0,0 @@
Improve performance of user directory search.

1
changelog.d/15731.misc
Unescape View File

@ -1 +0,0 @@
Remove redundant table join with `room_memberships` when doing a `is_host_joined()`/`is_host_invited()` call (`membership` is already part of the `current_state_events`).

1
changelog.d/15732.doc
Unescape View File

@ -1 +0,0 @@
Simplify query to find participating servers in a room.

1
changelog.d/15733.misc
Unescape View File

@ -1 +0,0 @@
Remove superfluous `room_memberships` join from background update.

1
changelog.d/15740.feature
Unescape View File

@ -1 +0,0 @@
Expose a metric reporting the database background update status.

1
changelog.d/15752.misc
Unescape View File

@ -1 +0,0 @@
Speed up typechecking CI.

1
changelog.d/15768.misc
Unescape View File

@ -1 +0,0 @@
Bump minimum supported Rust version to 1.60.0.

12
debian/changelog vendored
Unescape View File

@ -1,3 +1,15 @@
matrix-synapse-py3 (1.86.0~rc2) stable; urgency=medium
* New Synapse release 1.86.0rc2.
-- Synapse Packaging team <packages@matrix.org> Wed, 14 Jun 2023 12:16:27 +0200
matrix-synapse-py3 (1.86.0~rc1) stable; urgency=medium
* New Synapse release 1.86.0rc1.
-- Synapse Packaging team <packages@matrix.org> Tue, 13 Jun 2023 14:30:45 +0200
matrix-synapse-py3 (1.85.2) stable; urgency=medium
* New Synapse release 1.85.2.

26
docs/usage/configuration/config_documentation.md
Unescape View File

@ -1196,32 +1196,6 @@ Example configuration:
allow_device_name_lookup_over_federation: true
```
---
### `federation`
The federation section defines some sub-options related to federation.
The following options are related to configuring timeout and retry logic for one request,
independently of the others.
Short retry algorithm is used when something or someone will wait for the request to have an
answer, while long retry is used for requests that happen in the background,
like sending a federation transaction.
* `client_timeout`: timeout for the federation requests in seconds. Default to 60s.
* `max_short_retry_delay`: maximum delay to be used for the short retry algo in seconds. Default to 2s.
* `max_long_retry_delay`: maximum delay to be used for the short retry algo in seconds. Default to 60s.
* `max_short_retries`: maximum number of retries for the short retry algo. Default to 3 attempts.
* `max_long_retries`: maximum number of retries for the long retry algo. Default to 10 attempts.
Example configuration:
```yaml
federation:
client_timeout: 180
max_short_retry_delay: 7
max_long_retry_delay: 100
max_short_retries: 5
max_long_retries: 20
```
---
## Caching
Options related to caching.

2
pyproject.toml
Unescape View File

@ -89,7 +89,7 @@ manifest-path = "rust/Cargo.toml"
[tool.poetry]
name = "matrix-synapse"
version = "1.85.2"
version = "1.86.0rc2"
description = "Homeserver for the Matrix decentralised comms protocol"
authors = ["Matrix.org Team and Contributors <packages@matrix.org>"]
license = "Apache-2.0"

10
synapse/config/federation.py
Unescape View File

@ -22,8 +22,6 @@ class FederationConfig(Config):
section = "federation"
def read_config(self, config: JsonDict, **kwargs: Any) -> None:
federation_config = config.setdefault("federation", {})
# FIXME: federation_domain_whitelist needs sytests
self.federation_domain_whitelist: Optional[dict] = None
federation_domain_whitelist = config.get("federation_domain_whitelist", None)
@ -51,13 +49,5 @@ class FederationConfig(Config):
"allow_device_name_lookup_over_federation", False
)
# Allow for the configuration of timeout, max request retries
# and min/max retry delays in the matrix federation client.
self.client_timeout = federation_config.get("client_timeout", 60)
self.max_long_retry_delay = federation_config.get("max_long_retry_delay", 60)
self.max_short_retry_delay = federation_config.get("max_short_retry_delay", 2)
self.max_long_retries = federation_config.get("max_long_retries", 10)
self.max_short_retries = federation_config.get("max_short_retries", 3)
_METRICS_FOR_DOMAINS_SCHEMA = {"type": "array", "items": {"type": "string"}}

21
synapse/http/matrixfederationclient.py
Unescape View File

@ -95,6 +95,8 @@ incoming_responses_counter = Counter(
)
MAX_LONG_RETRIES = 10
MAX_SHORT_RETRIES = 3
MAXINT = sys.maxsize
@ -404,12 +406,7 @@ class MatrixFederationHttpClient:
self.clock = hs.get_clock()
self._store = hs.get_datastores().main
self.version_string_bytes = hs.version_string.encode("ascii")
self.default_timeout = hs.config.federation.client_timeout
self.max_long_retry_delay = hs.config.federation.max_long_retry_delay
self.max_short_retry_delay = hs.config.federation.max_short_retry_delay
self.max_long_retries = hs.config.federation.max_long_retries
self.max_short_retries = hs.config.federation.max_short_retries
self.default_timeout = 60
self._cooperator = Cooperator(scheduler=_make_scheduler(self.reactor))
@ -586,9 +583,9 @@ class MatrixFederationHttpClient:
# XXX: Would be much nicer to retry only at the transaction-layer
# (once we have reliable transactions in place)
if long_retries:
retries_left = self.max_long_retries
retries_left = MAX_LONG_RETRIES
else:
retries_left = self.max_short_retries
retries_left = MAX_SHORT_RETRIES
url_bytes = request.uri
url_str = url_bytes.decode("ascii")
@ -733,12 +730,12 @@ class MatrixFederationHttpClient:
if retries_left and not timeout:
if long_retries:
delay = 4 ** (self.max_long_retries + 1 - retries_left)
delay = min(delay, self.max_long_retry_delay)
delay = 4 ** (MAX_LONG_RETRIES + 1 - retries_left)
delay = min(delay, 60)
delay *= random.uniform(0.8, 1.4)
else:
delay = 0.5 * 2 ** (self.max_short_retries - retries_left)
delay = min(delay, self.max_short_retry_delay)
delay = 0.5 * 2 ** (MAX_SHORT_RETRIES - retries_left)
delay = min(delay, 2)
delay *= random.uniform(0.8, 1.4)
logger.debug(

2
synapse/types/__init__.py
Unescape View File

@ -177,7 +177,7 @@ class Requester:
user=UserID.from_string(input["user_id"]),
access_token_id=input["access_token_id"],
is_guest=input["is_guest"],
scope=set(input["scope"]),
scope=set(input.get("scope", [])),
shadow_banned=input["shadow_banned"],
device_id=input["device_id"],
app_service=appservice,

20
tests/http/test_matrixfederationclient.py
Unescape View File

@ -40,7 +40,7 @@ from synapse.server import HomeServer
from synapse.util import Clock
from tests.server import FakeTransport
from tests.unittest import HomeserverTestCase, override_config
from tests.unittest import HomeserverTestCase
def check_logcontext(context: LoggingContextOrSentinel) -> None:
@ -640,21 +640,3 @@ class FederationClientTests(HomeserverTestCase):
self.cl.build_auth_headers(
b"", b"GET", b"https://example.com", destination_is=b""
)
@override_config(
{
"federation": {
"client_timeout": 180,
"max_long_retry_delay": 100,
"max_short_retry_delay": 7,
"max_long_retries": 20,
"max_short_retries": 5,
}
}
)
def test_configurable_retry_and_delay_values(self) -> None:
self.assertEqual(self.cl.default_timeout, 180)
self.assertEqual(self.cl.max_long_retry_delay, 100)
self.assertEqual(self.cl.max_short_retry_delay, 7)
self.assertEqual(self.cl.max_long_retries, 20)
self.assertEqual(self.cl.max_short_retries, 5)

Write Preview
Loading…
Cancel
Save