I'm CH Chethan Reddy. I'm currently pursuing Bachelors in Electronics and Communication Engineering at the National Institute of Technology, Trichy in India. In my free time I like to swim, work on projects I find interesting, watch movies and do a lot of other "interesting" stuff. This summer I have interned in Matrix.org for around 3-4 months under GSoC and in this blog I'll be sharing my experience.
Why Matrix.org?
Being passionate about privacy, I wanted to select an organization that actively works on that, and clearly, Matrix.org topped my list. For anyone who doesn't know what Matrix is, in simple words, It is a decentralized communication protocol that supports many awesome features like messaging, end-to-end encryption, voip/video calls support, etc. It uses simple RESTful http APIs which keeps things very simple. The best part of the protocol is, it is decentralized, so if you have a custom domain name, you can setup a synapse server in a jiffy. Knowing all these who wouldn't choose Matrix?
My Work
I worked on Desktop Client called Nheko. It is a light-weight Desktop Client that uses Qt Framework in the frontend and also uses a mtxclient library that implements Matrix-client server API written in C++. The scope of my project involved implementing Device-Verification and Cross-Signing in Nheko.
For people who have not used Matrix before, it is important to note that every device of a particular user is considered as a separate unit rather than user itself. For end-to-end encrypted chat to be viewable from a device, that particular device should be trusted by the sender's device. Device-verification and Cross-Signing are methods used for verifying.
Coming to the work I have implemented, Device-verification is completely implemented in Nheko. As of now, SAS verification with to-device and room message verification is supported in Nheko. As far as Cross-Signing is concerned, verifying signatures of Cross-Signing keys and showing the verified status of devices have been implemented. The only remaining unfinished parts are SSSS and signing the keys after verification. After that Cross-signing should be feature-complete.
I wished to implement SSSS during my GSoC period, but I couldn't because I did not anticipate the additional things which came with verification while making my proposal. The additional works included : implementing the UI, changes to the userprofile dialog, working on the caching of verified users, and user keys. Moreover, due to the current pandemic, there were a few sudden changes in my academic schedule, which interrupted my work to some extent. While contributing at times I was stuck on some bugs like verifying the signatures, some random crashes in UserProfile and setting up relations in Room-Verification. However, with the help of my mentors, I was able to fix these bugs.
I'm really happy about the work I have done. I am hoping to further work on Nheko in the future, complete Cross-Signing and work on additional features for Nheko.
Community and Mentors
The best part of my experience this summer was the the Matrix community and the learning I had. I am really lucky to be part of the Matrix community which has many passionate people collectively working on really cool projects with clients, bridges, servers, bots and the spec. A special thanks to Uhoreg and Sorunome who have helped me in navigating through the spec.
Last but not the least, a big shout-out to both my mentors, Nico and red_sky, who were always there to help me with any issue in the project in spite of their personal commitments and dealt with my stupid questions with utmost patience and kindness. Without their active help and guidance this experience definitely would not have been so fun and great, they have clearly surpassed the expectations I had from mentors.
Hello! I am Nikos, an MSc student in Computer Security. My original goal for this GSoC was to implement the basics of end-to-end encryption for Go-NEB, which is a popular and easily extendable bot for the Matrix protocol. I chose this project because it ticked off a lot of the boxes that I look for in a project: The Matrix protocol is very interesting and the e2e specification is fascinating, Go-NEB is written in Go which I wanted to learn better, plus I believe personally that crypto should be easy for everyone to use and adopted more.
My initial plan was to first create bindings in gomatrix for libolm, the C library that implements most of the necessary algorithms for Olm/Megolm, the protocols used in Matrix' end-to-end encryption. Then, I was going to write some nice APIs around those bindings so that most of the work done is hidden from the client that uses the library, and finally I would call these APIs from Go-NEB when needed to set up the encrypted sessions between devices in a room and use them to encrypt/decrypt messages as necessary. What we found out soon after GSoC started was that another library called mautrix-go that was based on gomatrix already did most of these things. My task then became to change the uses of gomatrix in Go-NEB for mautrix-go and make some slight changes to the latter if there were some incompatibilities. A month later, that task was done.
With two months to go in GSoC, my mentors came up with a new task: to implement a service for Go-NEB that allows other client developers to test their e2ee implementations easily. That resulted in a service called "cryptotest" which allows other clients in a specified room to execute some Neb commands which would trigger functions like the forwarding of keys between clients, another feature that had to be implemented in mautrix. Something else that I wanted to add and grabbed my interest was SAS device verification, a multi-step process that involves users comparing a string of numbers or emojis out-of-band to configure they are indeed talking to each other and not to a MITM. These were my main tasks during the second month and I was happy to have achieved them and made a decent verification API for clients to use. They were soon merged into mautrix and Go-NEB.
For my third month I proposed to implement a new Matrix feature called cross-signing, which was similar to SAS verification but dealt with verifying other users instead of their devices who would in turn verify their own devices, creating a graph of trust between users and devices. This functionality was strongly coupled with SSSS, another new feature that allows clients to store their encrypted secrets (in this case the multiple necessary signing keys) on the server, as well as in-room verification which allows verification between users. This task was more challenging as it uses multiple algorithms (some of which not in libolm) for deriving the keys, using them to encrypt other keys which in turn sign other keys. It was also more satisfying when I finally managed to generate the keys, store and retrieve them and then use them to sign another user and when I was done I felt like my understanding of cryptography levelled up.
All in all, I'm very satisfied with the overall experience. The spec was very clear in most cases and when it wasn't the community was always helpful and responsive. They were also happy to discuss the spec with me and explain the more intricate details. For that reason I wasn't asking questions directly to my mentors (besides for defining my tasks) but to the overall community. I'm also very grateful to my mentors, @uhoreg and @Kegan for picking me for this project and helping me with the planning aspect and reviewing my PRs. I'd lastly like to thank the maintainer of mautrix, @tulir, for reviewing and merging a lot of my work into his library.
The Matrix service at Linköping University has now passed its soft launch state and headed straight into full and proper deployment territory. (Though we're still not being overly vocal about it, don't want to crash our deployment on day one after all.)
They're simply announcing it on TWIM, perhaps the most high-profile place in the world!
We're doing a little bit of advertising of it to computer clubs and the like though, to slowly build up the users.
But then:
And just to add another slight update: already we have at least one conference planning on trying out the new Matrix server as well, so the user count might just end up exploding soon anyway.
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
In terms of Spec Core Team MSC focus for this week, we're sticking with the widget theme, but also adding something new. Travis is writing up a mahoosive widgets PR in addition to some widget MSCs. We're also adding on MSC2753 (peeking /sync), MSC2444 (peeking federation), which will both unlock some very exciting upcoming features (such as x-as-rooms)!
Erik has made an initial, naive implementation of a sharded Event Persister worker - the non-sharded version currently being one of the biggest bottlenecks on matrix.org right now. It still needs a bit of polish to be production ready, and Sytest will need some updating to support testing it, but otherwise progress is looking good!
Brendan is polishing his unread counts implementations, while Rich is writing up the push rules replacement tech in an MSC. Rich has also been optimizing the device code in Synapse to both simplify the code, as well as cut down on unnecessary device data being sent over federation.
Patrick has finished converting Synapse's database code to be fully async! This has the benefit of making profiling the matrix.org Synapse process much more sane, and we're already starting to find a few little places where Synapse is burning CPU unnecessarily! Now, Synapse won't be fully async'd until Twisted itself is, and we drop support for older Twisted versions that aren't, which is not soon. However the majority of the code that developers touch and that is executed being async has massive benefits either way 🙂
Andrew has been working on having a confirmation pop up after users click the link in their email, to help prevent phishing scams. He's also trying to finally make Synapse spec-compliant in accepting identifier dictionaries during user-interactive authentication.
Olivier has been continuing to help Synapse recover missed messages better after it has been offline for an extended period of time, as well as various fixes to push.
Expect a fresh release with all these changes soon!
As someone who had my own server outage this week I'm looking forward to seeing this!
Dendrite / gomatrixserverlib
Dendrite is a next-generation homeserver written in Go
This week we have been continuing on our bug-hunt mission and have been working on refactoring some significant parts of the codebase to simplify the API surfaces and avoid race conditions.
Changes this week include:
The current state server is in the process of being deprecated, with this functionality being folded into the roomserver
Support for rate limiting on certain CS API endpoints has been added
Support for password changes has been added, allowing users to change their own password
/sync no longer incorrectly lists some rooms in the leave section, which caused problems with Element Web and other clients
Roomserver input events are now handled in FIFO order with per-room workers
Roomserver input, query and perform packages are now split up nicely
The federation sender and keyserver now delay some startup tasks to allow the HTTP listeners to start more quickly
The federation sender is now used for backfilling and getting missing events
Some more SQLite writer bugs have been fixed
Room information in the roomserver is now handled in a common structure
The public rooms API now correctly searches remote servers (thanks rohitmohan96)
If contributing to Dendrite sounds like something you would be interested in, please take a look at these issues and join us in #dendrite-dev:matrix.org! There's also #dendrite:matrix.org for general Dendrite chat and updates.
Introducing a new bridge to the family, tomsg-matrix!
A friend of mine preferred using his own chat protocol, but having a chat protocol is useless without having people to talk to. Since I prefer using Matrix I thought it would be a fun exercise to try and create a bridge using Rust.
We are currently working on an Android-App to use an old smartphone as SMS-Gateway for matrix-sms-bridge, because using gammu in production (>50 SMS/day) seems not to be very reliable with our USB-modem.
I create service for sending corporate information: some service (for example zabbix) can send message to this service (over http-api) and service will create room (if needed), invite user and send message to this room. Service will change status of message in his database to readed when user will read this message. Also service can send any changes statuses of this message by callback url. Editing old messages also supported: for example zabbix can send UID problem to this service and it is will edit old "problem message" to "problem resolved".
I'm currently in the process of rebuilding Ditto - partly to incorporate rn-matrix, and partly to support a WEB VERSION
The new repo can be found here (https://gitlab.com/ditto-chat/ditto) (there's a new repo mostly because getting react-native-web to play nicely with the rest of the app was a challenge, to say the least)
Good progress this week, I just got end-to-end encryption working. The code still needs some cleanup and polishing to be able to release it in the wild, but expect a release somewhere next week with E2EE enabled.
gomuks now supports importing and exporting message decryption keys. It even supports exporting the keys for a specific room, e.g. for sharing them with someone else. To help with sharing exported key files with someone else, gomuks also got support for uploading media.
Next I'll try to figure out how nikofil's cross-signing/SSSS PR for mautrix-go works and use that in gomuks.
NovaChat is a new Matrix-based desktop client that aggregates all your chat networks into one app. We now have integrated bridges for Whatsapp, Twitter, Telegram, Hangouts, Slack, Messenger, Android Messages (SMS), Skype, Discord, Instagram and IRC.
Sept 4 Updates:
Continuous UI improvements. Everything is much denser now. See screenshot below.
New bridge alert: tulir just completed an Android Messages SMS bridge (which is a total hack but it works!) for NovaChat.
Added Discord Bridge. Thanks for building this Sorunome!
In the works...
Still working on our iMessage bridge, using jailbroken iPhone 4s
Marking messages as 'unread'
Sign up for the NovaChat beta. Or send me a DM @eric:nova.chat!
Nheko
Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at
Lurkki added an emoji completer to the message area! This was a super requested feature and it is pretty awesome, that someone picked this up and implemented it! This means you can finally write :fire: and have it expand to 🔥! Seems like this will also lead to a few more improvements to our other completers down the line (user, room, etc completions), so thank you very much Lurkki!
I've been mostly reviewing PRs this week (and trying to get some other work in line, like MSCs, synapse stuff, etc). With this hopefully Chethans work on cross-signing will be merged soon, which will give us most of the building blocks to have cross-signing in Nheko (his work is basically only missing Secure Secret Storage and Sharing). We hope to fix the few remaining build issues in the next week or so.
On the side I have been cleaning some UI stuff to simplify our room settings dialog and allow advanced users to hide/show certain event types in the timeline. This will probably take a bit more time to be finished though.
IMPORTANT: Due to legal reasons (USA cryptography export), end-to-end encryption has been disabled in the google play and ios builds for now. They will be re-enabled once we got all that figured out.
The F-Droid builds still have e2ee. As the play store and fdroid builds use the same build signature, you can simply install the fdroid version over the other one. More information on that here: https://fluffychat.im/en/fdroid.html
Features
Pin and unpin chats
Implement event aggregations
Implement message edits
Render reactions
Add / Remove reactions by tapping on existing reactions
Fixes:
Don't re-render the room list nearly as often, increasing performance
Various fixes for sending messages on bad networks
I removed SchildiChat from the Google Play for now, due to the complications of providing encryption-capable software from US servers. Huge thanks to the FluffyChat team for providing insight into their investigations concerning the US export regulations!
On the positive side, SchildiChat is now finally available on F-Droid from the official repos.
Furthermore, I have set up my own F-Droid repository, where you can get updates that are compatible with the Google Play Store variant and which support push notifications.
On the implementation side, I can report the following:
Element 1.0.5 has been merged
The "low priority" category now remembers its expand/collapse-state across restarts
New setting to hide member state changes and reactions from the chat overview
The category unread counter badge now also displays the number of chats with new messages in case none of these has notifications enabled
This week, we should have fixed the background crash that was happening in TestFlight 1.0.7. TestFlight 1.0.9 has been submitted yesterday.
In parallel, we started to modernise the AppDelegate class, the entry point of the app. This is the very first step of several improvements we want to do (this is not only to rewrite it in Swift).
We have just merged a few big PRs to develop: add emails and phone numbers to Matrix account, rework of upload attachment management, and other stuff related to 1-1 calls. It will live on develop for the week-end and we will prepare the release v1.0.6 at the beginning of next week.
Built on the matrix-js-sdk, this NPM package will help any React Native developer drop matrix into their app.
Been working on this since the beginning of the summer - this package is split into two, one for the data and one with default UI components for those who want to drop chat into their app quickly (not required).
Would love feedback on docs, installation, and usage, so try it out and drop me a line!
Over the past couple of weeks, jplatte has been hard at work making sure we can introduce new fields to request and response types as non-breaking updates. Thanks to @nicholaswyoung, consumers can opt-in to breaking changes to ensure compliance with the spec on each update.
Besides that, we implemented another endpoint for the Federation API, bringing us up to 20/31 endpoints implemented.
matrix-docker-ansible-deploy can now help you set up matrix-registration - an application that lets you keep your Matrix server's registration private, but still allow certain users (those having a unique registration link) to register by themselves.
The interest was high. Some didn't know, that bridges exist in Matrix.
!!! Well thank you Oleg for letting them know.
Dept of Ping 🏓
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
Rank
Hostname
Median MS
1
fairydust.space
376
2
neko.dev
444.5
3
tchncs.de
565.5
4
autodie.be
958
5
matrix.vgorcum.com
1124
6
maescool.be
1435
7
lermer.nl
1989
8
utzutzutz.net
2207
9
bau-ha.us
2965
10
rollyourown.xyz
3013
That's all I know 🏁
See you next week, and be sure to stop by #twim:matrix.org with your updates!
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
In terms of Spec Core Team MSC focus for this week, we're taking off MSC2674 and MSC1544 while the author works through feedback, and instead looking at getting widgets in line.
The spec for widgets has so far lived on a collaborative document that hasn't been updated in some time. As a result multiple, separate implementations have sprouted. We'd like to nail this down. So the focus for this week will be: MSC1960 (widget auth mechanics) and MSC1236 (general widgets).
Dept of GSoC 🎓️
The coding phase of Google Summer of Code is over! That said there is still a bunch of stuff to do, code is still appearing and we're now entering the time of many forms to fill in in order for people to get paid.
A little pre-emptive, but wow it's been a good summer! SIX students, all did well!
GSoC coding is over and there's now an async/await ported matrix-ircd! Before merging that into master, it would be nice to have it tested a bit more. To install it, you first need Cargo, Rust's package manager (Installation). With that set up, just run
For the GSoC project HTML embeddable matrix chat this week -
Added an 'auto' option for the theme (read and set the theme from the device)-
Added a 'custom' highlight color (meant to be modified by users).
Restyled various components.
More news to come from this project - going to be used in a really exciting live event in the near future. Arnav has accidentally found a niche. :D
One more thing: This project needs a new name (riot-embedded I don't like.) If you have ideas you can fling them at arnav-t or post in #twim:matrix.org.
Dept of Servers 🏢
Dendrite / gomatrixserverlib
Dendrite is a next-generation homeserver written in Go
If contributing to Dendrite sounds like something you would be interested in, please take a look at these issues and join us in #dendrite-dev:matrix.org! There's also #dendrite:matrix.org for general Dendrite chat.
This week Erik has been working on sharding the Event Persister worker as we aim further for horizontal scalability for Synapse. Some of the complication comes from ensuring that sysadmin's don't misconfigure things where multiple event persisters persist events in the same room.
Patrick has been continuing on the async/await train and has just submitted the final PR which async's our runInteraction method - the one that most database calls go through. Synapse is nearly 100% async! He's also been submitting a flurry of shadow ban PRs, which will be another powerful tool for moderation.
Brendan and Rich are continuing to work on the Notifications rewrite, with Brendan finishing up unread counts, and Rich polishing up designs for the new replacement for push rules.
Andrew (who dat? - ed.) is continuing to work on the knocking MSC and has fleshed out the CS and SS details. There are still some open questions though, so please give feedback if you notice anything off!
Finally, Olivier has been working on making sure you get your missed messages when your homeserver has been offline for a while, as well as various Sygnal fixes and matrix-synapse-ldap3.
Just pushed an update to my matrix-synapse chart, fixing the volume permission configuration - for those who want to run Synapse as non-root without using a storage class that supports setting rights based on fsGroup
and then
Just pushed the 1.19.1 tags for my K8s-optimized container image, and a new helm chart version to go with it.
Did you know that Ananace is running a Matrix server for Linköpings universitet?
Another True Fact: "universitet" appears to be Swedish for "university".
I recently published a new matrix <-> mattermost bridge - matrix-appservice-mattermost. The new bridge already supports many features and is rapidly improving, with the current focus being setting up proper integration tests.
New bridge! This is actually really fully featured considering that we've only just heard about it! Looking forward to seeing more from dalcde and team in future.
Honestly I didn't realise Florian was a setting in the power levels. Is this in the spec? Very good seeing Rust SDK getting such usage, we will see more from this project I hope!
A multi-account Matrix client that allows you to authorize webapps to access parts of your account, designed with collaboration in mind.
This is the webapp I've been talking about for a while that allows webpages limited access to your account. Currently, apps can view the name, avatar, alias, and type of a room that's been opened with the app, and that's just about it.
The big interesting thing about mxapps right now is the architecture. Just about all of the logic is running in a shared worker. This means that you can duplicate as many tabs as you want and it will all act as the same client. It also has multi-account support.
Features coming ~soon~ at some point will be full state event support and easy-to-follow docs on how to make an app, plus Vue plugins to make development really easy.
You can checkout the live version on mxapps.kb1rd.net, but it is a bit useless...
Released 0.0.34 this week with some fixes to the room list sorting and fixing several crashes. If you clear your session, it will also sort low priority rooms at the bottom of the room list, even if unread, making it a bit more usable until we add more UX refinements to the room list.
Most of the effort this week however has gone into working on end-to-end encryption, having investigated using the rust-sdk for a couple of days, and now looking into a custom implementation based on libolm. This is well on its way with key upload working and device tracking nearing completion.
Getting e2ee will make this a very compelling client! Remember to watch Matrix Live with Bruno above. If you've watched it, remember we have 100s of episodes of Matrix Live to catch up on!
Nheko
Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at
I just merged the new event store a few days ago. This means Nheko now doesn't keep messages in RAM! Everything is stored on disk and mapped into memory from disk. This may reduce memory usage a tiny bit, but may increase latency while scrolling a bit. Try it out and give your feedback!
For the developers reading this, I collected some ramblings on my blog here: https://blog.neko.dev/posts/nhekos-event-store-v2.html . Yes, I designed that site when I was like 15, I know the dark theme is a bit too much... Just use Firefox and the alternate stylesheet for light mode! Anyway, I'm very interested to hear how other clients implement their event stores!
Element-Android v1.0.5 has been released to the PlayStore. We are now working to improve user experience with 1-1 calls. We will add support to add and remove emails to your Matrix account, and we are still fixing bugs...
This week, we made 2 release candidates available through TestFlight. New features are :
app access protection with PIN code, TouchID or FaceID
the come back of the incoming native call screen
A huge thanks to the community for the feedbacks. It helped us to discover issues like a crash in background due to PushKit. The coming TF (1.0.7) should fix all of them
A PR has been created for cross-signing and SSSS support! It also supports accepting in-room verification requests and will soon also support creating them, after which it should be ready for reviewing. Afterwards, with some minor changes to clients using the library, cross-signing should be available.
A Matrix bot using the library matrix-nio and Docker (specifically aiodocker) for running sandboxed code, allowing users to write their own factoid commands which can perform any task required. The bot launches a new instance of itself every time it is invited to a room, where it can then be configured for that channel by a management website. There is also a web IDE for the factoids, so you can edit and test the bot's output.
Bash-like factoid and command input/output redirection
Run untrusted code for PHP, Python, JavaScript and Java in Docker containers
Manage the instance of the bot from a management server (realtime log output over websockets, schedule commands to run, manage trusted devices and more)
Web interface for factoid editing (create, save, edit and test with live output). IDE from ace from cloud9
Cool! There are so many bot creation systems for Matrix now!
There is one month left until Hacktoberfest🎃! Prepare your repos now!
It's a great time to get friends and newcomers involved with open source and your projects.
During October, merged PRs to any(!) public GitHub repo will count towards a person's score. With four PRs, they get a very neat looking t-shirt from the organizer DigitalOcean. They can also ask for a tree to be planted instead.
As a maintainer,
Add a "Hacktoberfest" label (color suggestion: #FF7518) on issues to point people to tasks with a clear scope.
Let people know that you're interested in contributions and will review them.
Add the label "invalid" on a PR that's spam or a minor contribution that shouldn't count (fixing typos, blindly upgrading dependencies, etc.).
Matrix benefits from Hacktoberfest, but do keep in mind that this is a commercial project from DigitalOcean.
Dept of Ping 🏓
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
Rank
Hostname
Median MS
1
fairydust.space
374
2
neko.dev
526
3
loves.shitposting.chat
566
4
yyyyyyy.ml
619
5
kif.rocks
669
6
maescool.be
737.5
7
matrix.vgorcum.com
754
8
autodie.be
839
9
hmlw.me
966.5
10
fick.es
1053.5
That's all I know 🏁
See you next week, and be sure to stop by #twim:matrix.org with your updates!
Nico, one of the Nheko client maintainers, discusses the state of the project and shares various opinions about Matrix, chat clients, physics, the local weather and so on.
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
In terms of Spec Core Team MSC focus for this week, MSC1960 will be the only one on the list. The two MSCs from last time will be put aside for this week while we focus on implementation.
Dept of GSoC 🎓️
Nheko(GSoC)
Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at
This week we added some more federation readers to matrix.org which should noticeably improve federation lag.
Patrick has continued on with the async/await-a-thon and we are starting to enter the home stretch. Of the things we care about only runInteraction remains. Once done we can get on with profiling the main process.
On the sharding side the last big milestone is to get the event persister to shard. I know this comes up every week, but we now have a rough working implementation for non-backfilled events. Next steps are to get backfill working, get the tests passing, go through review and then ship. I asked Erik if it would be live on matrix.org within 2 weeks. He didn’t say, at least I didn’t hear him say no…>
Aside from that Rich is working on the push rules redesign, the next steps are to design the APIs in more detail and then make a formal spec proposal. Brendan is also in notifications land and his work on room unread counts is going through final review.
Finally, Andrew has been working with Sorunome to dust off the knock feature, both on the spec and the Synapse side.
Romeo and Juliet benchmark to compare Matrix server implementations
This week I created the first(?) Matrix server benchmark and ran it on Synapse, Dendrite and Conduit. The test will go through the entire Romeo and Juliet play and create users for each character and sends messages for each line they say.
The results are very interesting:
Synapse:
timo neilalexander Half-Shot
# default (sqlite)
time 5m0.870s
# postgres:
time 1m46.319s 1m8s
Dendrite:
# default (sqlite)
time 6m8.802s 0m38.502s
# postgres:
time 2m45.387s 0m53.691s
Conduit:
# default (sled)
time 0m4.184s 0m2.776s 0m2.935s
There is some question here about how federation and state resolution will affect performance on these tests.
Synapse Deployment 📥️
There is rather a lot in this section this week (see also Ops 🛠 section below.) I'll consider making this a more structured (table?) feature in future.
Hello! Just recently I made public an SELinux policy module I developed for Synapse. This policy is one I developed early on during my trip down the SELinux rabbit hole, and I have been using it in production for some time. At the moment, this policy module is built for the Gentoo SELinux policy (not RHEL/CentOS/Fedora) and does not currently have systemd support, nor has it been tested in all environments, i.e. with workers. I am publishing this policy in the hopes that it will be useful and I am open to any contributions, even those to help tighten the policy if possible. Finally, I have also made public a policy module for coturn, which I am also using in my own personal environment under the same conditions. Both the Synapse policy and the coturn policy can be found on Github. Thank you!
The ansible collection famedly.matrix has seen another release, 0.1.2, which updates the element role to 1.7.4 and the synapse role to 1.19.0. As always, get it from Ansible Galaxy, source from Gitlab, and for any questions join #ansible:famedly.de.
Just pushed the 1.19 tags for the K8s-optimized Synapse image, as well as an updated chart version for it that also takes advantage of the added /health endpoint.
As channels are really popular in Telegram, I wrote a quick and dirty program/bot which can mirror (text and image) public Telegram channels to Matrix rooms. This should make it easier for Telegram users to consider a switch.
Making the rooms read-only via permissions prevents messages from users.
It provides a good basis for someone trying to make a real 1to1 mirror. Please fork this project and host some channels publicly ;)
This project is functionally quite different from the more established mautrix-telegram: it is purely designed to mirror content from Telegram public rooms.
Hello all, I'm going to be quick off the mark and announce matrix-appservice-irc release 0.20.1! This release contains a ton of bug fixes and quality of life changes that you never knew you were missing. Please update and report back in the usual channels as always. Thanks!
Dept of Clients 📱
Nheko
Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at
I've been working on trying to finish the event store (when I wasn't distracted by video calls with some people). This should allow limited offline use and reduce memory use as well as allow some other features in the future. If you want to try it out, you can build this PR or try out the nightlies with new-event-store in their name.
If you decide to try it out, backup your cache or at least BACKUP YOUR ENCRYPTION KEYS! I won't be responding to feedback immediately, since I am currently hiking, but I should get to it this weekend or next week. Come discuss it in #nheko:ocean.joedonofry.com, if you have some feedback or issues.
We are still fixing issues on Element Android. Hopefully we will release version 1.0.5 of the application and of the Matrix SDK in the coming days!
later, fidèle à sa parole:
Element Android 1.0.5 has been published on the beta channel of the PlayStore. If everything is fine it will be pushed to production next week. I've also published SDK v1.0.5, and push translations of the PlayStore home page of Element to German, Chinese, English US, Hungarian, Portuguese (Brazil), Russian, Slovak, Swedish and Ukrainian.
This week, we improved customisation for forks. We add possibility to customize default HTTP headers (like "User-Agent") and improve login and room settings screens customisation. We started to improve project documentation. We also made some fixes like fixing registration validation link in email.
Updated various links and strings for the new brand
Renewed Windows code signing cert
Removed rebranding toast
All cross-signing keys now shared with other devices during verification
This week
Mode to require secure backup during registration
Continue work on deferred cross-signing setup
Light weight Hydrogen client continues, adding key features and design tweaks
Notifications continues in background while push rules evolve
Matrix.to link previews in progress
Hydrogen
Hydrogen is a web-based client designed to be very lightweight, and with few dependencies.
Anyone who wants to discuss Hydrogen can join #hydrogen:matrix.org.
avatars and display names in the timeline and for rooms
implemented the logic for the member list, but no UI for this yet
room list sorting by last message timestamp, unread state and badge count
clear unread state when reading a room
some visual tweaks in the timeline and room list
bug fixes
If you have checked out hydrogen before, you'll need to clear your session (the little clear button in the session picker) to get correct avatars and display names everywhere.
Next week I'll get started on E2EE, which should make the project a lot more useful. I'll look into leveraging the nice rust matrix-crypto-sdk for this.
Mirage 0.6.1
miruka hit us with a late breaking update about Mirage!
Mirage 0.6.1 was released today:
Added
SSO authentication support
Homeserver browser:
To add a new account, you will be asked first to pick one of the
listed public server (list data from anchel.nl) or to manually enter a server address
Typing in the server address field will also filter the public server list,
Up/Down or (Shift+)Tab and Enter can be used to navigate it by keyboard
If the address doesn't have a scheme://, auto-detect whether the server
supports HTTPS or only HTTP
Use the .well-known API if possible to resolve domains to the actual
homeserver's URL, e.g. matrix.org resolves to
https://matrix-client.matrix.org
The server address field will remember the last homeserver that was
connected to
Room members autocompletion:
Type @ followed by one or more characters in the composer,
or one or more characters and hit (Shift+)Tab to trigger username/user ID
autocompletion
Only autocompleted names will be turned into mentions, unlike before
where any word in a sent message that happened to be someone's name would
mention them
Full image viewer for matrix image messages and URL previews:
Click on a thumbnail in the timeline to open the image viewer
Middle click on a thumbnail (or use the option in the context menu)
to open the image externally
Left click on the image (mouse only): expand to window size if the
image's origin size is smaller than the window,
else expand to original size
Tap on the image (touch screen/pen only): reveal the info and button bars
when auto-hidden (bars will auto-hide only when they overlap with a big
enough displayed image)
Any mouse movement: reveal auto-hidden bars
Double click on the image: toggle full screen
Middle click anywhere: open externally
Right click anywhere: close the viewer, back to chat
Drag when displayed image is bigger than window to pan
Wheel to pan up/down, hold shift or alt to pan left/right
Ctrl+wheel to control zoom
Buttons to control rotation, scale mode, full screen, GIF play/pause
and GIF speed
New keyboard shortcuts are available for all these actions,
see keys.imageViewer in the config file (will be automatically updated
when you start Mirage 0.6.1)
Add media.openExternallyOnClick setting to swap the new
click and middle click on thumbnails behavior
Room and member filter fields now support (Shift+)Tab navigation, in addition
to Up/Down
Add a colored left border to the currently highlighted item in list views
(e.g. room list, members list, etc) to improve visibility
Themes:
Add controls.listView.highlightBorder and
controls.listView.highlightBorderThickness properties (can be set to 0)
Add the chat.userAutoCompletion section
Changed
Messages context menu:
Use a cleaner icon for the "Copy text" entry
Replace the confusing broken "Copy media address" entry with:
Copy media address: visible for non-encrypted media, always
copies the HTTP URL
Copy local path: always visible for already downloaded media, even if
they were downloaded before mirage was started
The openMessagesLinks keybind (default Ctrl+O) is renamed to
openMessagesLinksOrFiles and can now also open media message files
Using the openMessagesLinksOrFiles keybind on a reply will now ignore the
matrix.to links contained in the "In reply to XYZ" header
Pressing Ctrl+C to copy selected/highlighted non-encrypted media messages
will copy their HTTP URL instead of the filename
Retry downloading image thumbnails if they fail with a 404 or 500+ server
error (uploads sometimes take a few seconds to become available on the
server)
Non-encrypted media messages are now always downloaded on click and opened
with a desktop application (or the image viewer), instead of
being opened in a browser
Compress thumbnails and clipboard images in a separate process, to avoid
blocking every other backend operation while the compression is running
Reduce the level of optimization applied to clipboard images,
the previous setting was too slow for large PNG (10MB+)
Increase applied scrolling velocity when using the
scrollPageUp/scrollPageDown keybinds, now similar to how it was before
Mirage 0.6.0
Don't catch SIGQUIT (Ctrl+\ in terminal) and SIGTERM signals, exit immediately
Slightly increase the top/bottom padding to the multi-account bar in the
left pane
Dependencies: minimum nio version bumped to 0.15.0
We wanted to take a little bit of time to show off a state resolution library for Ruma, courtesy of @DevinR528 and @Timo. Ruma has a new crate to make resolving state easier, state-res! The goal of this crate is to be the base for federation, it provides the necessary utilities to resolve state, sort power events (reverse topological ordering), sort events (based on mainline depth), and a lot of event authentication functions. It is still in the early stages of development, but will be hopefully proved out in Conduit.
Other things that happened this week:
Added the invite endpoints from the Federation API
Started requiring strong types for event-sending endpoints
Augmented error types with error-specific fields (see the livestream here)
Speaking of livestreams, @jplatte streams coding sessions from time to time on Twitch. If you're interested in watching some coding streams to get more familiar with the Ruma codebase or learn Rust in general, follow us at [#ruma-livestreams:matrix.org](https://matrix.to/#/#ruma-livestreams:matrix.org)!
Very impressed that these features are landing in Ruma first, with the intention of getting them into Conduit.
SSSS and cross-signing are slowly making their way to mautrix-go! A WIP PR exists for both features and is being developed upon. This will allow any clients that use the library to eventually be able to use SSSS and verify the trustworthiness of other devices based on cross-signing.
Device verification was merged to neb! It needs human intervention to approve that the SAS match, of course.
A new service called "cryptotest" was added that allows other clients to exchange e2e-related events with neb to test their capabilities (such as encryption, room key forwarding, SAS verification)
Residents of Germany can apply for the Prototype Fund until the end of September. They are looking for new open source ideas or features and fund about 20 projects twice a year with 47,500 EUR for six months (starting in April for this round).
Previously funded chat-related projects include Briar and OpenPush.
It would be great to see a project building on Matrix.
Dept of Ping 🏓
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
Rank
Hostname
Median MS
1
fairydust.space
345
2
maescool.be
865
3
chatserver.ca
985
4
nuclearlimes.co.uk
1206
5
utzutzutz.net
1826
6
nct08.de
1885
7
kapsi.fi
1994.5
8
mailstation.de
2324
9
envs.net
4053
10
kittenface.studio
4076
That's all I know 🏁
See you next week, and be sure to stop by #twim:matrix.org with your updates!
1.19.0 contains lots of async/await changes, 27 separate PRs to be precise(thanks Patrick). These changes will provide some level of performance improvement, but the real value to us is that it will improve our ability to profile Synapse more effectively.
It also features improved logging performance, this is in part to reduce IO but also to reduce the total amount logged. We’d be really interested to get some feedback on if the changes help. You may also want to take a look at our new logging defaults to see if they would help reduce total disk space usage.
Some admins will rejoice that we have added a /health end point to every configured HTTP listener which should make it easier to monitor multi-worker installs.
Finally 1.19.0 contains improvements to the Admin API allowing the admin to optionally purge rooms on deletion.
Get the new releases from any of the usual sources mentioned at https://github.com/matrix-org/synapse/blob/master/INSTALL.md. 1.19.0 is on github here.
Add a /health endpoint to every configured HTTP listener that can be used as a health check endpoint by load balancers. (#8048)
Allow login to be blocked based on the values of SAML attributes. (#8052)
Allow guest access to the GET /_matrix/client/r0/rooms/{room_id}/members endpoint, according to MSC2689. Contributed by Awesome Technologies Innovationslabor GmbH. (#7314)
Bugfixes
Fix a bug introduced in Synapse v1.7.2 which caused inaccurate membership counts in the room directory. (#7977)
Fix a long standing bug: 'Duplicate key value violates unique constraint "event_relations_id"' when message retention is configured. (#7978)
Fix "no create event in auth events" when trying to reject invitation after inviter leaves. Bug introduced in Synapse v1.10.0. (#7980)
Fix various comments and minor discrepancies in server notices code. (#7996)
Fix a long standing bug where HTTP HEAD requests resulted in a 400 error. (#7999)
Fix a long-standing bug which caused two copies of some log lines to be written when synctl was used along with a MemoryHandler logger. (#8011, #8012)
In completely unrelated news to my earlier K8s updates, Linköping University has now soft-launched Matrix for use both internally - and hopefully with other universities and institutions in the future.
Se the k8s updates from Ananace below. Is this the first .se university offering Matrix? Also, great hostname!
We've finally opened up https://translate.riot.im/projects/matrix-doc/sas-emoji-v1 to translate the emoji used by clients to ensure that users are securely verifying each other. The idea is that clients should always have a consistent name for these emoji, even if developed by completely different teams. If your client supports encryption verification and has translations, please transition to using matrix-doc for your translations: https://github.com/matrix-org/matrix-doc/pull/2728 has some details, and #matrix-spec:matrix.org would love to help out.
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
Some of the team was on holiday last week, hence the noticeable lack of progress 😇 In terms of Spec Core Team MSC focus for this week, we're keeping on with the same two as last week: MSC2674 (aggregations pt1: relationships) and MSC1544 (QR code verification).
lots of async/await changes, 27 separate PRs (thanks Patrick)
improved logging performance, this is in part to reduce IO but also to reduce the total amount logged. We’d be really interested to get some feedback on if the changes help. You may also want to take a look at our new logging defaults to see if they would help reduce total disk space usage.
Further improvements to the Admin API allowing admin to optionally purge rooms on deletion.
Expect 1.19.0 to land early next week.
Aside from that Erik has been continuing with sharding the event persister, this is a big job, so we’re still a little way off having something to ship, but I’ll keep you updated.
Big news this week, after spending a good chunk of time trying to rework notifications to make the UX more intuitive, we have decided that the best (and only) way forward is to completely rethink how the push rules work. Rich is currently redesigning the whole thing, and once done we’ll resume on the client side.
Having spent much of the past 3 months trying to improve the performance of large scale instances we will soon start looking at improving the out of the box experience for smaller installs and are in the early stages of thinking about what that might look like. I’ll update more over time, but the two obvious use cases to target are
Federating instance, with 1-5 users, resource constrained.
Federating instance with a few hundred users.
Our (relatively small) changes to logging in 1.19.0 are the first signs of this shift. We still need to ship event persistence sharding and also move more functionality from the main process (dependent on async/await), but once done it clears the way for new projects.
anoa brought it to our attention that this graph looks like a "ocean-beach waterline dropoff".
This week I worked a bit on federation! Last week we were able to query room directories over federation; This week I looked into what's needed to join one of those rooms:
Resolve room aliases over federation
Send make_join request when a user tries to join an unknown room
Use make_join response to create send_join request (insert origin, origin_server_ts and add hashes and signatures)
After implementing all those things, I could actually see Conduit's join event on Synapse!
Of course a lot more work is needed to make this really work,
like creating the room on the Conduit side and sending and receiving federation
transactions, but this is a start.
Other news:
Docker support (thanks to @weasy, @valkum and @paul)
Room upgrade support (almost done, thanks to @Faelar)
Thanks to everyone who supports me on Liberapay or Bitcoin!
Dendrite / gomatrixserverlib
Dendrite is a next-generation homeserver written in Go
Sytest has now been updated to allow skipping tests for endpoints that have been deprecated in the spec. This may be useful if you are writing a homeserver and have no plans to implement deprecated endpoints, e.g. /initialSync or /events.
Use the command line parameter --exclude-deprecated when starting Sytest to skip these tests!
Just pushed some WIP (v0.1) charts for both Synapse and the matrix-media-repo project, written with Redis, workers/sharding, PVC constraints, and other large deployment thoughts in mind.
Note that these charts - the Synapse one in particular - basically require you to have a fully featured cluster, with a regex-capable ingress, automated cert manager, and RBAC. They do not require you to have a ReadWriteMany capable storage class though, even when using workers.
Nheko is a desktop client using Qt, Boost.Asio and C++17. It supports E2EE (with the notable exception being device verification for now) and intends to be full featured and nice to look at
So, we've been silent for a bit, but that doesn't mean we haven't been busy, so I have a little bit of a bigger announcement to make today:
Trilene implemented voice call support in Nheko! We just merged this and you can try it out now!
Some caveats apply at the moment, but will be resolved in the near future:
Currently you will need to have a patched version of GStreamer. The current releases of GStreamer have a bug when your turnserver uses a password, that contains a ':'. Since this is the default for coturn, I'm guessing this affects everyone.
You will need to build Nheko yourself. Currently we do not have GStreamer available in our CI pipeline, so we disabled support for voice calls in those builds (apart from the flatpak build, but that also needs some further validation and a new GStreamer release). We will release proper builds with VOIP support once GStreamer is actually released and available with the fixes we need.
There are still some other issues, when relying on a turnserver to connect a call. We will fix them one at a time.
Big shoutout to trilene for implementing all of this! I can finally call people from Nheko, which is super awesome!
This was the first of the bigger features, that have been cooking for a while, in the near future we should also finish the new event store, which reduced memory usage and provides some kind of offline support, and Chethan will probably finish the first iteration of cross-signing soon too. After that we'll focus on the smaller features again. ;-)
BIG update! Looking forward to trying out voice calls. ☎️ Maybe we'll be calling Nico for Matrix Live soon..?
We are still fixing issues following the Element release. We are also working to add Jitsi conference call support.
We have exported a first beta version of the SDK from Element Android. It is available here: https://github.com/matrix-org/matrix-android-sdk2 and can be integrated in any Android app as a regular gradle dependency. We will update the SDK after each release of Element Android.
It includes following changes since the last TWIM:
The single-list overview screen is now optional. If you prefer the tabbed layout for separating direct messages and rooms, as known from Element for Android, but still want SchildiChat's message bubbles, you can get that combination now too!
Push notifications have been enabled for the Play Store variant
Some regular readers might remember, in the past I've put a few entries here about a side-project called Brawl. Two weeks ago, we took Brawl and started building a new client on top of it code-named Hydrogen.
As the name implies, Hydrogen tries to be the lightest Element. It is written entirely in vanilla javascript (no React, no Webpack) for complete control, structured as an MVVM app, leveraging the raw performance of indexeddb. On huge accounts where Element-web uses up to 1GB of heap memory, Hydrogen sips away with a mere 15mb. It is very early days though, and Hydrogen has very little features: you can receive and send unencrypted messages and that's it.
Hydrogen also aims to support a wider range of browsers, from IE11 (particularly for orgs with locked-down machines) all the way to a fully featured PWA that works well on mobile. We should also be able to easily add progressive loading, which proved hard in the past.
Another goal is to make it easy to embed parts of the app (like the room view) into another app or website. It could also be really nice to have a place to play and experiment with new tech for element proper - much as Dendrite has influenced Synapse.
As mentioned, it's early days, and lots of things are missing or broken, so please don't file any issues yet. Feel free to play around with it at https://hydrogen.element.io/ though!
Lots of exciting things happened in Ruma this week:
We have officially implemented over half (18/31) of the Federation API endpoints!
Special thanks to @agraven and @q-b!
We've added the only new endpoint from r0.6.1 of the C2S spec.
We have begun work on adding support for borrowed types for requests. This
should reduce the memory allocations required for clients to send requests to
a server.
Exported some macros to make creating custom endpoints and events that
much easier.
New Contributors
This week, we have gotten help from a few new contributors:
There's news from ansible land! Ansible recently introduced a new packaging format, the "collection", and we've now started moving our ansible content into collections. This means the matrix modules I twim'ed a few months back are now easier to install than ever before, and the synapse/element roles are included in there as well (thanks to madonius for the contributions there). The roles have been regularly updated for a while now, and the element role isn't even called riot anymore! You can get the collection on Ansible Galaxy, with the source being available over on GitLab. In case of any questions, feel free to hop over into #ansible:famedly.de
If Ansible were a film franchise "Ansible: The Collection" would be the long awaited boxset.
opsdroid is getting close to a release with the matrix connector rewritten to use the matrix-nio library. If you are feeling brave and want to help us iron out any bugs before the release, please install the master branch and test it out for your matrix bots. Please report any issues on the GitHub issue tracker or come talk in #opsdroid-developers:matrix.org. Hopefully, a release with this and lots of other goodies will be coming in a couple of weeks.
I’ve created a compact and portable home server based on Matrix protocol, it’s a piece of hardware which is easy to use and set up. I’m working on a new client APP similar to other typical messenger GUI, but this server we named it ‘Privee’ supports any messenger developed with Matrix protocol. I’ve launched a campaign on Kickstarter this week, please come and support us! Thank you!
https://www.kickstarter.com/projects/privee/privee?ref=beibnw
Privee is an interesting project alright! The project creators are based in Hong Kong, and are working from a desire to have more control over their communications. The plan is to run homeservers on a small computer physically located with the user, similar to plans outlined by NovaChat.
This is an early-stage project (hence starting on KickStarter.) While exciting and ambitious, I encourage everyone to do their own research! Learn more and chat with terry_hello (who has done a good job handling questions already!) in #Privee:matrix.org.
Dept of Ping 🏓
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
Rank
Hostname
Median MS
1
helderferreira.io
470
2
maescool.be
486.5
3
matrix.vgorcum.com
570.5
4
mchus.pro
730
5
envs.net
770
6
maunium.net
983
7
matrix.org
1261
8
acmelabs.space
1282.5
9
chatserver.ca
1387
10
productionservers.net
1433.5
That's all I know 🏁
See you next week, and be sure to stop by #twim:matrix.org with your updates!
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
Handle SAS verification by adding an endpoint where the decimal SAS code generated by another device can be sent. If this code matches with go-neb's generated code, the verification succeeds and the device is marked as trusted.
Add a cryptotest service which can be used for other clients to test their functionalities related to e2ee. This currently includes a challenge / response, key forwarding functions (key request / key forward) and SAS verification. SSSS and cross-signing are planned for the future. (PR is under review)
Dept of P2P 👥
iOS P2P Demo
Dendrite is a next-generation homeserver written in Go
v1.2.0 is out now, with a bunch of bug fixes and some added features:
Thumbnail support for apng and webp images (thanks Sorunome!)
Experimental (but pretty stable) support for Redis as a cache - instructions here
Optional oEmbed support for things like previewing tweets - turn it on here
Dynamic thumbnail support
Per-user quotas
v1.2.0 is also much faster at file uploads, however some areas of matrix-media-repo can still be improved - general performance work is planned for the next release.
We are happy to announce the beta milestone for our first versioned release of Dendrite, which we anticipate to land in the next few weeks. Once these issues are cleared, we'll have made the big breaking changes that we want to make and will have a migration mechanism in place for database schema upgrades.
This will make life significantly easier for anyone who wants to start experimenting with and running their own Dendrite homeservers without unexpected breakages. More updates to follow soon - join us in #dendrite:matrix.org for the latest!
This week, Dendrite's end-to-end encryption support has also been mostly rounded off with the exception of a few minor bugs and edge-cases.
Changes this week include:
TLS certificate verification is now enabled by default
A new federation_disable_tls_validation config option has been added to override the above
Added support for processing inbound device lists from federation
Local device list changes are now sent outbound over federation
One-time key counts are now sent in /sync
Direct server signing key fetches are now limited to 15 seconds
State events that fail signature checks on federated room joins are now properly ignored
A number of SQLite database lock bugs have been fixed using TransactionWriter
A bug when finding joined hosts in the federation sender in SQLite mode has been fixed
Logging levels have been adjusted in a number of areas
Spec compliance:
Client-Server APIs: 51%, same as last week
Server-Server APIs: 54%, up from 53% last week
Asked whether this progress means that matrix.org would be running on Dendrite next year, Neil said he was certain that it would definitely be ready:
This week in Synapse land we continued with our performance push making further progress on the async/await-athon. We will move into the storage layer this coming week which will leave the caching layer and a few other bits outstanding. Separate work continued to get the event persister into a shardable state. The first step being to replace the event stream token with its own type and then threading that through the code base.
We have also continued to improve on our push capabilities and released a new version of Sygnal (0.8.2) with support for pushing to iOS devices via Firebase.
We are also very close to getting a workable version of the new push rules. The idea is to test it out in the wild before getting it enshrined in the spec.
Next week will see a continued focus on sharding and async/awaiting, more notifications work, logging improvements, and a host of bug fixes!
I released v0.2.0 of the maubot RSS plugin. The most recent addition is support for JSON feeds, stolen frominspired by FantasyCookie17's jsonfeed-bot.
Other changes since v0.1.0:
Per-feed toggle to send posts as m.text instead of m.notice
Automatically moving subscriptions across room upgrades
Some fixes for handling weird feeds
maubot/rss is a maubot plugin that posts feed updates to Matrix rooms. It supports RSS/Atom/JSON feeds, customizing message templates per-feed/room, and more.
All my bridges now have options for automatic e2ee key sharing in portal rooms. Currently they don't have verification support (other than manually marking as verified in the database), so enabling key sharing isn't entirely secure yet. I'll probably invent some sort of web verification interface after nikofil adds cross-signing to mautrix-go.
This also means mautrix-python now supports sending and responding to key requests. I'll port the interactive verification and future cross-signing code from mautrix-go to mautrix-python at some point too.
4.4.0 is out as promised last week! Get it off Flathub now.
After we announced our intention to release, we discovered a regression that prevented joining new rooms because there was a mixup between room aliases and room id during a switch from internal structures to ruma-identifiers. Luckily we managed to fix it in time and make the release as intended.
Full changelog since 4.2.2
New features
Videos now have a preview directly in history
New videos player: click on a preview to open it
Improved audio file support, with a nicer presentation and seeking
Redacted messages are removed from history
Edited messages are replaced with their new content and show a small icon
Translations have been updated
Spellcheck language is saved per room
New swipe-back gesture to go back from room to room list in narrow view
New swipe left and right gesture to navigate through media in media view
SOCKS proxy are now also supported
Bugfixes
Thumbnail for pasted images have been fixed
File type detection for uploads is more reliable
Typing notifications are now only sent when text is actually typed in the message input
Under the hood
Code refactor is still going
Some improvements to logging, including access token removal for privacy
Element Android 1.0.4 🚀 has been released to the store, it contains several bug fixes, and is adding back some moderators functionality (see list of banned users for a room + actions)
The team has been mainly working on stabilisation + refactoring of the current code base.
We are also trying to give more visibility on what is coming next:
SchildiChat is a new fork of Element for Android (previously RiotX), which features some UI changes, to make it more comfortable to use for people who are used to other common instant messengers.
The most important changes are the following:
The overview screen lists all chats in one list instead of the tabbed layout in Element
Message bubbles (optional)
Theme choice based on the system-wide dark/light theme on recent Android versions
It is now available on Google Play, it might appear on F-Droid too in the future.
As per this issue comment, I was asked to inform this channel of the new AppImage for the Element Desktop Matrix Client. 🎉. Interested users may get them on appimage.github.io or on my releases page, offering Continuous builds (refreshed every day) for stable and beta builds 😃
I have changed the archive feature. Added a splash screen. Further simplified settings. Login times work faster compared to FC. I am connecting it to my Jitsi instance. which operates p2p and is completely analytics free. Message notifications are analytics free. It will eventually be an extension to a videocall app developed by myself. It adds variety to the ecosystem. It strictly follows license rules, credits the developers and links to their wonderful work. And it is a great thing to do in my spare time ☺️
Hey all! I built yet another bot library. matrixbz was built with the intention of making it easy to draft matrix bots. Check out the github - I've added some examples where you can build a bot in ~10 lines of python code. matrixbz features:
Auth - you can specify user(s) who are authorized to call commands. bot only accepts invites from those users.
Cache - you can cache results for particular command invocations.
Hemppa the bot is a general use bot that has lot of modules for various uses and super easy API to create more. Hemppa received support for Mastodon. Now you can login to your Mastodon account and send toots via Matrix. Next major feature would be to subscribe to hashtags and users and get notifications to Matrix rooms. https://github.com/vranki/hemppa#mastodon
The Hackers on Planet Earth conference went almost entirely self-hosted this year due to not being able to hold an in-person event, and used Matrix to go virtual! hopeless was the robot-heart of the conference, providing community+room ACLs for attendees. The bot also has a bunch of other features including:
configured admins sending notices to rooms
ability to manually and programmatically schedule announcements for scheduled events happening, like workshops or talks
the ability for users to re-join rooms and generate invites for themselves as necessary
admin commands for inviting attendees and presenters without tickets
an attempt to be friendly and solve people's common problems without a real person having to assist (surprisingly effective!)
Many thanks to the nio-template for existing, otherwise this wouldn't have been possible!
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
Rank
Hostname
Median MS
1
fairydust.space
426
2
privacytools.io
438
3
saferutar.com
487
4
maunium.net
534
5
heitkoetter.net
570
6
chatcloud.net
640
7
encom.eu.org
658.5
8
envs.net
697
9
matrix.vgorcum.com
847
10
neko.dev
879
That's all I know 🏁
See you next week, and be sure to stop by #twim:matrix.org with your updates!
[Matrix will] be rolled out by the German education system to provide collaboration tools for half a million seats in the states of Schleswig-Holstein and Hamburg
Here's your weekly spec update! The heart of Matrix is the specification - and this is modified by Matrix Spec Change (MSC) proposals. Learn more about how the process works at https://matrix.org/docs/spec/proposals.
In terms of Spec Core Team MSC focus for this week, we'll be continuing on with MSC2674 (aggregations pt1: relationships) and adding MSC1544 (QR code verification) on top.
E2E Dendrite work is progressing, and we're making a lot of breaking changes ahead of our first stable release which you can expect in the coming weeks.
As for the changes this week:
Add a bare bones user directory. This is mainly to benefit the iOS Yggdrasil demo.
The dendrite-config.yaml configuration format has been overhauled. It's still in review but expect big changes here!
Fixed a bug with SQLite when handling large rooms (thanks @HenrikSolver!)
Device lists now work locally, with federation support to come.
This week we shipped 1.18.0 which contained support for our worker sharding performance projects that you’ve been hearing about recently.
We also shipped a security and feature release of Sygnal (actually two each…). The main new feature being the ability to run behind an HTTP proxy which is useful in environments that mandate that all traffic go through a proxy.
Aside from that week we continued our async/await-athon and Patrick provided a visual update
We’ve been documenting worker support and diving into sharding the event persisters. This is not a small job since it means every other part of the code base that assumes an integer stream id will need to be updated to be aware of multiple stream ids from multiple persisters.
Over in feature land we’ve been working on the notifications project and most of the backend support is done. The project will mean more intuitive defaults as well as a much better client UX in configuring notification settings.
Just pushed the 1.18.0 K8s-optimized Synapse images, now only doing the debian-based version as like upstream - though still including jemalloc. So latest/v1.18.0 and *-debian will be the same in this version and going forwards. Additionally, the *-debian tags will only remain for a release or two more before disappearing, as they're now superfluous.
I finally added automatic backfilling options to mautrix-telegram. Like my other bridges, it can backfill old messages when creating portals and missed messages when restarting, and it can disable notifications in the room while backfilling.
I also added bridging for your own read receipts, so if you read a message with another Telegram client, the bridge will mark it as read on your Matrix account with double puppeting.
gomuks can now do interactive verification for e2ee. Most of it is thanks to nikofil's SAS verification PR to mautrix-go, I just made a UI for using it. Currently gomuks can only send verification requests, but I'll add some interface for accepting incoming requests soon.
Related to verification, there are some new commands for verifying/unverifying/blacklisting devices and viewing device lists. There's also a toggle to disable sending messages to unverified devices.
This is super exciting, raises gomuks to the level of a daily driver IMO.
SAS verification support has been merged! Support for it should also be landing in go-neb soon, both for verifying with the bot's device and for testing with go-neb's client testing service. Next up, looking at SSSS and eventually cross-verification support.
We, some German Synapse admins, have created a survey about the usage of Matrix-Synapse for private statistic purpose (results will be published alike). As suggested in #synapse:matrix.org, this could also be a topic for "TWIM".
Here we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
Rank
Hostname
Median MS
1
kleinhirn.net
366
2
fairydust.space
383
3
tchncs.de
394.5
4
mjdsystems.ca
506
5
matrix.org
587
6
nobelium.no
753.5
7
pixie.town
761
8
matrix.linux.pizza
1111
9
uraziel.de
1230.5
10
yyyyyyy.ml
1314
That's all I know 🏁
See you next week, and be sure to stop by #twim:matrix.org with your updates!
