Commit Graph

7610 Commits

Author SHA1 Message Date
Erik Johnston
6774f265b4 Fix building rust with nightly (#15906)
Also fix up a warning.
2023-07-10 16:24:04 +01:00
Eric Eastwood
6e731e86bf Placeholder changelog 2023-07-10 10:23:30 -05:00
Shay
8a529e4fb6 Stop running sytest on buster/python3.7 (#15892) 2023-07-07 12:04:55 -07:00
Shay
f25b0f8808 Stop writing to column user_id of tables profiles and user_filters (#15787) 2023-07-07 09:23:27 -07:00
Dirk Klimpel
677272caed Remove worker_replication_* settings from worker doc (#15872)
Co-authored-by: Mathieu Velten <mathieuv@matrix.org>
2023-07-07 08:09:41 +00:00
Jason Little
2481b7dfa4 Remove worker_replication_* deprecated settings, with helpful errors on startup (#15860)
Co-authored-by: reivilibre <oliverw@matrix.org>
2023-07-07 07:45:25 +00:00
sarthak shah
f19dd39dfc Update link to the clients webpage, fix #15825 (#15874) 2023-07-06 17:28:09 +02:00
Eric Eastwood
b07b14b494 Federation outbound proxy (#15773)
Allow configuring the set of workers to proxy outbound federation traffic through (`outbound_federation_restricted_to`).

This is useful when you have a worker setup with `federation_sender` instances responsible for sending outbound federation requests and want to make sure *all* outbound federation traffic goes through those instances. Before this change, the generic workers would still contact federation themselves for things like profile lookups, backfill, etc. This PR allows you to set more strict access controls/firewall for all workers and only allow the `federation_sender`'s to contact the outside world.

The original code is from @erikjohnston's branches which I've gotten in-shape to merge.
2023-07-05 18:53:55 -05:00
Eric Eastwood
561d06b481 Remove support for Python 3.7 (#15851)
Fix https://github.com/matrix-org/synapse/issues/15836
2023-07-05 18:45:42 -05:00
Erik Johnston
39d131b016 Add basic read/write lock (#15782) 2023-07-05 17:25:00 +01:00
Eric Eastwood
ce857c05d5 Add tracing to media /upload endpoint (#15850)
Add tracing instrumentation to media `/upload` code paths to investigate https://github.com/matrix-org/synapse/issues/15841
2023-07-05 10:22:21 -05:00
Sumner Evans
cc780b3f77 docs/admin_api: fix header level on 'Users' page (#15852)
Signed-off-by: Sumner Evans <sumner@beeper.com>
2023-07-05 16:15:56 +02:00
Jason Little
4cf9f92f39 Fix could not serialize access due to concurrent DELETE from presence_stream (#15826)
* Change update_presence to have a isolation level of READ_COMMITTED

* changelog
2023-07-05 11:44:02 +01:00
Erik Johnston
95a96b21eb Add foreign key constraint to event_forward_extremities. (#15751) 2023-07-05 09:43:19 +00:00
an0nfunc
c303eca8cc use Image.LANCZOS instead of Image.ANTIALIAS for thumbnail resize (#15876)
Image.ANTIALIAS is not defined in current pillow releases. Since ANTIALIAS was just using LANCZOS anyways, this is just a cosmetic change, but makes synapse work with most recent pillow releases.

Signed-off-by: Giovanni Harting <539@idlegandalf.com>
2023-07-05 10:52:12 +02:00
Michael Weimann
c8e81898b6 Add not_user_type param to the list accounts admin API (#15844)
Signed-off-by: Michael Weimann <michaelw@element.io>
2023-07-04 15:03:20 -07:00
Olivier Wilkinson (reivilibre)
861752b3aa Merge branch 'master' into develop 2023-07-04 17:40:37 +01:00
Olivier Wilkinson (reivilibre)
664ba14080 1.87.0 2023-07-04 16:25:33 +01:00
Paarth Shah
649848627c Pin pydantic to <2.0.0 (#15862)
Signed-off-by: Paarth Shah <mail@shahpaarth.com>
2023-07-04 16:22:33 +01:00
Paarth Shah
670d590f8a Pin pydantic to <2.0.0 (#15862)
Signed-off-by: Paarth Shah <mail@shahpaarth.com>
2023-07-04 09:33:24 +02:00
pacien
07d7cbfe69 devices: use combined ANY clause for faster cleanup (#15861)
Old device entries for the same user were being removed in individual
SQL commands, making the batch take way longer than necessary.

This combines the commands into a single one with a IN/ANY clause.

Example of log entry before the change, regularly observed with
"log_min_duration_statement = 10000" in PostgreSQL's config:

    LOG:  duration: 42538.282 ms  statement:
    DELETE FROM device_lists_stream
    WHERE user_id = '@someone' AND device_id = 'someid1'
    AND stream_id < 123456789
    ;
    DELETE FROM device_lists_stream
    WHERE user_id = '@someone' AND device_id = 'someid2'
    AND stream_id < 123456789
    ;
    [repeated for each device ID of that user, potentially a lot...]

With the patch applied on my instance for the past couple of days, I
no longer notice overly long statements of that particular kind.

Signed-off-by: pacien <pacien.trangirard@pacien.net>
2023-07-03 16:39:38 +02:00
reivilibre
cd8b73aa97 Fix the devenv up configuration which was ignoring the config overrides. (#15854)
* Fix use of config override directory in `devenv up`

`--config-directory` is for the generate config script; `-c` is for usage

* Add homeserver config override directory to gitignore

* Newsfile

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>

---------

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
2023-07-03 11:39:52 +01:00
reivilibre
53aa26eddc Add a timeout that aborts any Postgres statement taking more than 1 hour. (#15853)
* Add a timeout to Postgres statements

* Newsfile

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>

---------

Signed-off-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
2023-07-03 11:38:57 +01:00
Eric Eastwood
13fc89148c Split out 2022 changes from the changelog (#15846)
Split out 2022 changes from the changelog so the rendered version in GitHub doesn't timeout as much.
2023-06-28 15:10:33 -05:00
Eric Eastwood
472c2c72f6 Prepare changelog for v1.87.0rc1 2023-06-27 10:29:20 -05:00
Shay
78cfa55dad Fix sqlite user_filters upgrade (#15817) 2023-06-27 09:41:42 +01:00
Erik Johnston
25c55a9d22 Add login spam checker API (#15838) 2023-06-26 14:12:20 +00:00
Nicolas Werner
e0c39d6bb5 Fix forgotten rooms missing in initial sync (#15815)
If you leave a room and forget it, then rejoin it, the room would be
missing from the next initial sync.

fixes #13262

Signed-off-by: Nicolas Werner <n.werner@famedly.com>
2023-06-21 14:56:31 +01:00
Erik Johnston
289ce3b8d9 Fix harmless exception in port DB script (#15814)
The port DB script would try and run database background tasks, which
could fail if the data they acted on was in the process of being ported.
These exceptions were non fatal.

Fixes #15789
2023-06-21 13:20:46 +00:00
Erik Johnston
6c749c5124 Fix typo in faster join docs (#15812)
Fixes #15756
2023-06-21 11:34:32 +01:00
Mathieu Velten
496f73103d Allow for the configuration of max request retries and min/max retry delays in the matrix federation client (#15783) 2023-06-21 10:41:11 +02:00
Shay
6a5cf1a759 Fix Sytest environmental variable evaluation in CI (#15804) 2023-06-20 07:55:46 -07:00
ew-at-vier
2301a09d7a Fix admin api documentation typo (#15805)
* Fix admin api documentation typo

Signed-off-by: Eric Wolf <eric.wolf@vier.ai>
2023-06-20 10:45:26 +00:00
Eric Eastwood
887fa4b66b Switch from matrix:// to matrix-federation:// scheme for internal Synapse routing of outbound federation traffic (#15806)
`matrix://` is a registered specced scheme nowadays and doesn't make sense for
our internal to Synapse use case anymore. ([discussion]
(https://github.com/matrix-org/synapse/pull/15773#discussion_r1227598679))
2023-06-20 10:05:31 +01:00
Eric Eastwood
0f02f0b4da Remove experimental MSC2716 implementation to incrementally import history into existing rooms (#15748)
Context for why we're removing the implementation:

 - https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1487441010
 - https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1504262734

Anyone wanting to continue MSC2716, should also address these leftover tasks: https://github.com/matrix-org/synapse/issues/10737

Closes https://github.com/matrix-org/synapse/issues/10737 in the fact that it is not longer necessary to track those things.
2023-06-16 14:12:24 -05:00
Andrew Morgan
2ac6c3bbb5 Don't always lock "user_ips" table when performing non-native upsert (#15788) 2023-06-16 15:25:44 +01:00
Mathieu Velten
0618bf94cd push rules: fix internal conversion from _type to value (#15781)
Also fix wrong rule names for `is_user_mention` and `is_room_mention`.
2023-06-16 14:17:02 +02:00
Mathieu Velten
f63d4a3a65 Regularly try to wake up dests instead of waiting for next PDU/EDU (#15743) 2023-06-16 10:15:12 +00:00
Josh Qou
d939120421 Fix unsafe hotserving behaviour for non-multimedia uploads. (#15680)
* Fix unsafe hotserving behaviour for non-multimedia uploads.

* invert disposition assert

* test_media_storage.py: run lint

* test_base.py: /inline/attachment/s

* Only return attachment for disposition type, update tests

* Update synapse/media/_base.py

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

* Update changelog.d/15680.bugfix

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

* add attribution

* Update changelog.

---------

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2023-06-15 14:23:27 +01:00
Tulir Asokan
1404f68a03 Fix joining rooms through aliases where the alias server isn't a real homeserver (#15776) 2023-06-14 15:42:33 +01:00
Mathieu Velten
87e5df9a6e Merge branch 'release-v1.86' into develop 2023-06-14 14:54:19 +02:00
Mathieu Velten
825c5909de 1.86.0rc2 2023-06-14 12:17:29 +02:00
Mathieu Velten
14f9d9b452 Fix empty scope when having version mismatch between workers (#15774) 2023-06-14 11:53:55 +02:00
Jason Little
21fea6b749 Prefill events after invalidate not before when persisting events (#15758)
Fixes #15757
2023-06-14 09:42:18 +01:00
Eric Eastwood
8ddb2de553 Document looping_call() functionality that will wait for the given function to finish before scheduling another (#15772)
Thanks to @erikjohnston for clarifying, https://github.com/matrix-org/synapse/pull/15743#discussion_r1226544457

We don't have to worry about calls stacking up if the given function takes longer than the scheduled time.
2023-06-13 16:34:54 -05:00
Shay
553f2f53e7 Replace EventContext fields prev_group and delta_ids with field state_group_deltas (#15233) 2023-06-13 13:22:06 -07:00
Mathieu Velten
59ec4a0dc1 Fix MSC3983 support: only one OTK per device was returned through federation (#15770) 2023-06-13 19:51:47 +02:00
Eric Eastwood
0757d59ec4 Avoid backfill when we already have messages to return (#15737)
We now only block the client to backfill when we see a large gap in the events (more than 2 events missing in a row according to `depth`), more than 3 single-event holes, or not enough messages to fill the response. Otherwise, we return the messages directly to the client and backfill in the background for eventual consistency sake. 

Fix https://github.com/matrix-org/synapse/issues/15696
2023-06-13 12:31:08 -05:00
Patrick Cloke
df945e0d7c Fix MSC3983 support: Use the unstable /keys/claim federation endpoint if multiple keys are requested (#15755) 2023-06-13 18:07:55 +02:00
Mathieu Velten
9966eb10a3 1.86.0rc1 2023-06-13 14:30:51 +02:00