MSC4274: Inline media galleries via msgtypes #4274
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Johannes Marbach <[email protected]>
Johennes
force-pushed
the
johannes/msgtype-galleries
branch
from
d957ab5 to
4edac2e
Compare
Johennes
changed the title
turt2live
added
proposal
turt2live
reviewed
Mar 18, 2025
Member
There was a problem hiding this comment.
Implementation requirements:
- Client
Contributor
Author
There was a problem hiding this comment.
- ruma: events: add gallery msgtype for MSC4274Â ruma/ruma#2044
- Very draft matrix-rust-sdk: Add experimental support for MSC4274 (inline media galleries)Â matrix-rust-sdk#4838
| `org.matrix.custom.html` is supported. | ||
| - `formatted_body` (string): The formatted version of the body. Required if | ||
| `format` is specified. | ||
| - `itemtypes` (array): Ordered array of metadata for each item in the gallery. |
There was a problem hiding this comment.
Why is this called itemtypes and not simply items?
Contributor
Author
There was a problem hiding this comment.
Hm, I think I landed on this because itemtypes is basically the generalization of msgtype plus the associated data into an array. I agree that items would be clearer though.
Closed
1 task
This was referenced Apr 7, 2025
Hywan
pushed a commit
to matrix-org/matrix-rust-sdk
that referenced
this pull request
Apr 8, 2025
This was broken out of #4838 and is a preliminary step towards implementing [MSC4274](matrix-org/matrix-spec-proposals#4274). The `media_handles` field on `SendHandle` is turned into a vector so that it can hold handles for several media when upload a gallery later. Signed-off-by: Johannes Marbach <[email protected]>
bnjbvr
added a commit
to matrix-org/matrix-rust-sdk
that referenced
this pull request
Apr 24, 2025
…ueuedRequestKind::UploadFileWithThumbnail to prepare for MSC4274 gallery uploads (#4897) This was broken out of #4838 and is a preliminary step towards implementing [MSC4274](matrix-org/matrix-spec-proposals#4274). `SentRequestKey::Media` and `DependentQueuedRequestKind::UploadFileWithThumbnail` are generalized to allow chaining dependent media uploads and accumulating sent media sources. - [x] Public API changes documented in changelogs (optional) --------- Signed-off-by: Johannes Marbach <[email protected]> Co-authored-by: Benjamin Bouvier <[email protected]>
1 task
bnjbvr
pushed a commit
to matrix-org/matrix-rust-sdk
that referenced
this pull request
May 27, 2025
This was broken out of #4838 and is a step towards implementing [MSC4274](matrix-org/matrix-spec-proposals#4274). * The entry point for sending galleries via the send queue is a new method [`RoomSendQueue::send_gallery`](https://github.com/matrix-org/matrix-rust-sdk/pull/4977/files#diff-8752e86459c22cff470d7ca617240dcbdf222c3c6c98be2af2e43ddec071154cR362) which is a generalization of `RoomSendQueue::send_attachment`. * `send_gallery` takes as input parameters a [`GalleryConfig`](https://github.com/matrix-org/matrix-rust-sdk/pull/4977/files#diff-d38d74cec6159cf769c32bed496199146175f05428fc23ab13bc2c629900da3eR283) (containing info about the gallery itself, such as its caption) and a vector of [`GalleryItemInfo`](https://github.com/matrix-org/matrix-rust-sdk/pull/4977/files#diff-d38d74cec6159cf769c32bed496199146175f05428fc23ab13bc2c629900da3eR355)s (containing info about each image / file / etc. in the gallery). * `send_gallery` creates the gallery event content via [`Room:make_message_event`](https://github.com/matrix-org/matrix-rust-sdk/pull/4977/files#diff-474f20e47fdcf60feac3f839a81f82fbb2c1fd0bb406388b0262adb60216ce3bR2281) which was renamed from `make_attachment_event` to reflect the fact that it creates general `msgtype` events now. * `send_gallery` maps the passed item infos into [`GalleryItemQueueInfo`](https://github.com/matrix-org/matrix-rust-sdk/pull/4977/files#diff-d569f54c7901b5cd660aae77045c80dabbd3c251f0fa5891530469f35941327aR2008)s. This additional struct allows grouping all the metadata for a single gallery item together. * Finally `send_gallery` invokes [`QueueStorage::push_gallery`](https://github.com/matrix-org/matrix-rust-sdk/pull/4977/files#diff-d569f54c7901b5cd660aae77045c80dabbd3c251f0fa5891530469f35941327aR1294) which is a generalization of `QueueStorage::push_media`. * `send_gallery` pushes upload requests for the media and thumbnails to the queue in a "daisy chain" manner. The first thumbnail (or media if no thumbnail exists) is pushed as a `QueuedRequestKind::MediaUpload`. The remaining thumbnails and media are pushed as `DependentQueuedRequestKind::UploadFileOrThumbnail`s while chaining each request to the previous one. * Finally a [`DependentQueuedRequestKind::FinishGallery`](https://github.com/matrix-org/matrix-rust-sdk/pull/4977/files#diff-42a1a7ebe2446d916c9e013293bafc5eb16fd89bfe89248e3877ef031cc83ef2R265) is pushed to finalize the gallery upload (analogous to the existing `FinishUpload` for single media uploads). * The `FinishGallery` request is handled in [`QueueStorage::handle_dependent_finish_gallery_upload`](https://github.com/matrix-org/matrix-rust-sdk/pull/4977/files#diff-8752e86459c22cff470d7ca617240dcbdf222c3c6c98be2af2e43ddec071154cR628) which was modeled after `handle_dependent_finish_upload`. * To be able to map the temporary event source for each gallery item to the final `AccumulatedSentMediaInfo`, a hash map is used. * Using the hash map, the gallery event is then updated to use the actual media sources via [`update_gallery_event_after_upload`](https://github.com/matrix-org/matrix-rust-sdk/pull/4977/files#diff-8752e86459c22cff470d7ca617240dcbdf222c3c6c98be2af2e43ddec071154cR104) and a new method [`Room::make_gallery_item_type`](https://github.com/matrix-org/matrix-rust-sdk/pull/4977/files#diff-474f20e47fdcf60feac3f839a81f82fbb2c1fd0bb406388b0262adb60216ce3bR2303) * An [integration test](https://github.com/matrix-org/matrix-rust-sdk/pull/4977/files#diff-21532ad5467a69489d7913b8da7afd4d618b7e357ce94f769e6e60e395b58055R2048) has been added to demonstrate the functionality. This is relatively large, unfortunately, but including everything needed to actually send the event made it possible to also add a test for it. It would be nice if the amount of new code could be reduced but I'm struggling a bit to find ways to integrate galleries with the existing media uploads further. - [x] Public API changes documented in changelogs (optional) --------- Signed-off-by: Johannes Marbach <[email protected]>
Merged
1 task
bnjbvr
pushed a commit
to matrix-org/matrix-rust-sdk
that referenced
this pull request
Jun 3, 2025
This was broken out of #4838 and is a step towards implementing matrix-org/matrix-spec-proposals#4274. Building upon #4977, a new method `Timeline::send_gallery` in matrix-sdk-ui for sending galleries. - [x] Public API changes documented in changelogs (optional) --------- Signed-off-by: Johannes Marbach <[email protected]>
1 task
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Rendered
In line with matrix-org/matrix-spec#1700, the following disclosure applies:
I am a Systems Architect at gematik, Software Engineer at Filament and Unomed, Matrix community member and former Element employee. This proposal was written and published with my Filament hat on.