Sidebar
Issue Tracker
Have you ever wished for a social media platform that combines the best features of Lemmy and image boards with customizable feed algorithms? I know I have. Here's what my perfect social media platform would look like: 1. **A mix of Lemmy and image board**: My ideal platform would allow users to post both text-based content and images, using tags instead of communities. With a view like Lemmy[^1], and another grid-view like an image board[^2]. 1. **User curation**: Unlike Lemmy, where images are difficult to find again due to the lack of tags , my ideal platform would have well-curated images with tags for easy searching[^2]. 1. Advanced search could also be implemented, as suggested in the Lemmy issue [#3788](https://github.com/LemmyNet/lemmy/issues/3788). 1. **User trust levels and community moderation**: A hierarchical trust level system, similar to Discourse’s trust levels[^3], could distribute the responsibility among users and reduce the burden on admins. Trust levels would be assigned for each community based on user activity and voting affinity with the admin, allowing admins to shape their instance according to their preferences without micromanaging every aspect of the community. This idea is also discussed in the Lemmy issue [#3548](https://github.com/LemmyNet/lemmy/issues/3548). 1. **Customizable feed algorithms**: One of the best things about Lemmy is that users can choose their own algorithm for their home feed[^1]. My ideal platform would take this a step further by allowing users to customize their feed algorithms like in Bluesky[^4]. 1. **Machine learning algorithms**: To make the feed even more personalized, my ideal platform would use machine learning algorithms to suggest posts to users based on their activity on the platform[^5]. For example, if a user frequently upvotes posts about cats, the platform would suggest more cat-related posts to that user. 1. **One-size-fits-all image format**: Image boards are known for their simple, one-size-fits-all image format[^2]. My ideal platform would adopt this format to make it easy for users to share images without worrying about formatting issues. [^1]: [Lemmy](https://alternativeto.net/software/lemmy/about/) [^2]: [Imageboard](https://en.wikipedia.org/wiki/Imageboard) [^3]: [Understanding Discourse Trust Levels](https://blog.discourse.org/2018/06/understanding-discourse-trust-levels/) [^4]: [Bluesky custom feeds and algorithms](https://www.theverge.com/2023/5/26/23739174/bluesky-custom-feeds-algorithms-twitter-alternative) [^5]: [How to implement personalized feed ranking](https://stackoverflow.com/questions/50663740/how-to-implement-personalized-feed-ranking)
cross-posted from: https://lemmy.world/post/1723295 > I've noticed that there are a few communities that tend to dominate when viewing all. Some days it gets to where looking at all isn't very different than just looking at Memes@lemmy.ml or 196@lemmy.blahaj.zone. > > > Before someone says "you can just block communities you don't want to see," it's not that I never want to see them, it's that I want to be able to have a view that shows me what is new and popular in a wide variety of communities. I appreciate seeing a few good memes in my feed. The problem is when that's all I see. Changing the sort from active to hot or top x days doesn't have much effect on which communities dominate, so that isn't the solution either. > > > "You can just subscribe to communities you like". True, but that has the effect of narrowing what I see. I'd like a view that showed me new things I never thought to subscribe to. > > > Lemmy devs - if you are reading this - it would be nice to have a feed that limited the number of posts showing up from any particular community. It could be a simple cutoff of 2 or 3 posts, or maybe some sort of weighting function to cause additional posts from the same community to appear lower in the sort order for that feed. > > > I'd love to hear what devs and other users think about this. > > Edit: To everyone saying "just sort be new" - yes, that has its uses, but it only solves part of the problem. I'd like a feed that shows me what is new *and popular*, but from more than just one or two communities.
github.com
### Requirements - [X] Is this a bug report? For questions or discussions use https://lemmy.ml/c/lemmy_support - [X] Did you check to see if this issue already exists? - [X] Is this only a single bug? Do not put multiple bugs in one issue. - [X] Is this a backend issue? Use the [lemmy-ui](https://github.com/LemmyNet/lemmy-ui) repo for UI / frontend issues. ### Summary We have a user with a few hundred comments who has crashed the site twice today while trying to delete his account. This query ends up running for a long time and locks subsequent updates to `comment`: ``` UPDATE "comment" SET "content" = $1, "deleted" = $2, "updated" = $3 WHERE ("comment"."creator_id" = $4) RETURNING "comment"."id", "comment"."creator_id", "comment"."post_id", "comment"."content", "comment"."removed", "comment"."published", "comment"."updated", "comment"."deleted", "comment"."ap_id", "comment"."local", "comment"."path", "comment"."distinguished", "comment"."language_id" ``` This was running for 8 minutes before I killed it. The user in question has 352 comments and 3073 entries in comment_like. This doesn't seem like such a large amount that there should be significant impact from a user deletion. ### Steps to Reproduce I haven't been able to reproduce this with a test user, so far only this one external user keeps causing it on our site. I've had to disable the /api/v3/user/delete_account URL for now. ### Technical Details Logs are too noisy but this is triggered by a post to /api/v3/user/delete_account from Jerboa ### Version 0.18.2 ### Lemmy Instance URL lemmy.ca
github.com
### Requirements - [X] Is this a bug report? For questions or discussions use https://lemmy.ml/c/lemmy_support - [X] Did you check to see if this issue already exists? - [X] Is this only a single bug? Do not put multiple bugs in one issue. - [X] Is this a backend issue? Use the [lemmy-ui](https://github.com/LemmyNet/lemmy-ui) repo for UI / frontend issues. ### Summary When on the page of a user on a remote instance, the linked RSS feed returns "Record not found" ### Steps to Reproduce 1. Go to the page of a remote user, on a local instance (https://lemmy.world/u/dessalines@lemmy.ml) 2. Click the RSS button 3. The linked page returns "Record not found" (https://lemmy.world/feeds/u/dessalines@lemmy.ml.xml) ### Technical Details For example, https://lemmy.world/u/dessalines@lemmy.ml links to https://lemmy.world/feeds/u/dessalines@lemmy.ml.xml and returns "Record not found" The page of a local user correctly returns an RSS feed (https://lemmy.ml/u/dessalines links to https://lemmy.ml/feeds/u/dessalines.xml which returns an RSS record) The RSS feed for a remote _community_ on a local instance returns the same "Record not found". (https://lemmy.world/feeds/c/fediverse@lemmy.ml.xml) See lemmy-ui issue (https://github.com/LemmyNet/lemmy-ui/issues/1954) for a related user interface bug. ### Version BE 0.18.2 ### Lemmy Instance URL lemmy.ml, lemmy.world, lemmy.ca, etc
github.com
…bout the LEMMY_DATABASE_URL format
github.com
### Requirements - [X] Is this a bug report? For questions or discussions use https://lemmy.ml/c/lemmy_support - [X] Did you check to see if this issue already exists? - [X] Is this only a single bug? Do not put multiple bugs in one issue. - [X] Is this a backend issue? Use the [lemmy-ui](https://github.com/LemmyNet/lemmy-ui) repo for UI / frontend issues. ### Summary When calling api/v3/comment/list you get a list of all comments on that post. Including any removed ones. The only change removed ones have is that the "removed" field is set to true. This is a massive problem because it delegates the obscuring of removed content to the front end. You can view these despite not being being logged in. When I used a mobile app that didn't take into account this removed flag, I was met with some disgusting NSFL imagery I'd rather not have seen. I strongly recommend obscuring the content of removed comments from the API if the user is not logged in or if the user is not a mod of the community/ not an instance owner. I understand the need to keep this information in the case of reversing moderation decisions and the modlog, but there is zero reason for non-mods and non-admins to have access to it in the normal endpoints. Furthermore, the baton should not be passed to front end developers either. The source of truth should be the backend, and the backend should enforce it. Finally this should happen with other places where comments are listed and posts are viewed. Comments deleted by the user should not be visible to anyone in the API besides the user and maybe mods/admins. I'm not sure what other endpoints it'll apply to, but in my opinion this is paramount. ### Steps to Reproduce 1. Create a post in a community you moderate 2. Create a comment on that post 3. Remove that comment 4. Open up dev tools 5. Go to that post again 6. Look at the http response. The JSON response has all identifying info removed, and the "removed" flag circled.  ### Technical Details n/a ### Version 0.18.2 ### Lemmy Instance URL _No response_
github.com
Credit to @phiresky for this idea, originally posted in comments of #2994 This PR adds `community_id` to `post_aggregates` (& a new index on `post_aggregates`) to enable joining `community` directly to `post_aggregates` when querying posts. **On lemm.ee, this optimization speeds up the query for front page of subscribed posts ~1000x, from several seconds to to just milliseconds.** You can check a before/after of query plans here: https://gist.github.com/sunaurus/856e03165bb0c0010505afeebde45230
github.com
### Requirements - [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support - [X] Did you check to see if this issue already exists? - [X] Is this only a feature request? Do not put multiple feature requests in one issue. - [X] Is this a backend issue? Use the [lemmy-ui](https://github.com/LemmyNet/lemmy-ui) repo for UI / frontend issues. ### Is your proposal related to a problem? at first glance, with lemmy URLs I can't tell the context of a URL off the bat. (for example, which community it is in, which post a comment is a part of). with the URLs for any community post just being **{lemmy domain}/post/{post id}** and the URLs for any comment being **{lemmy domain}/comment/{comment id}** ### Describe the solution you'd like. Changing these URLs to include: - the community that they were posted in - the post that they are a comment on The result would be: - Posts looking like **\{lemmy domain\}/c/\{community name\}/post/\{post id\}** - Comments looking like **\{lemmy domain\}/c/\{community name\}/post/\{post id\}/comment/\{comment id\}** ### Describe alternatives you've considered. I don't have other ideas for URL formats. This seemed the neatest to me. ### Additional context _No response_
github.com
### Requirements - [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support - [X] Did you check to see if this issue already exists? - [X] Is this only a feature request? Do not put multiple feature requests in one issue. - [X] Is this a backend issue? Use the [lemmy-ui](https://github.com/LemmyNet/lemmy-ui) repo for UI / frontend issues. ### Is your proposal related to a problem? Some admins want to manage the amount of storage needed for image uploads and so have instituted limits on image sizes. Examples include lemmy.ee at 100kB (https://lemm.ee/post/25065) and beehaw.org at 4000x4000 (https://github.com/LemmyNet/lemmy/issues/3473#issuecomment-1620520547). There does not seem to be a way to set, enforce or expose these rules via the API, so these appear to be implemented in NGINX, giving a 413 Payload Too Large error in the case of lemm.ee. As a result, when a client app is attempting to upload an offending image, all it can currently tell the user is that the image is somehow too large and that they must find out for themself what the rules are. ### Describe the solution you'd like. The ideal would be that the instance handles the required resizing for the user. If this is not seen as an attractive approach, it would then be extremely helpful for client apps if this information could be exposed via the API, so that they could query this information before attempting an upload and automatically handle the resizing to the instance's requirements on the user's behalf. My initial inclination is that the bounding-box limit would generally be easier for developers to work with. ### Describe alternatives you've considered. The lemm.ee link above suggests that users should use other image hosts if their files are too large, but this is not an attractive option for app developers or end users, as if either requires the developers to make a choice of third-party host on behalf of their users (which some will doubtless have reasons for disliking) or it requires the users to make choices or take actions which they may not understand. Most users in most cases would rather just see their images resized, perhaps with a note in the app to inform them that this had happened in case they wanted to deal with this differently. ### Additional context _No response_
github.com
replaced with new branch
github.com
This is an urgent test addition to highlight the problem with comment deletes not replicating when a remote-server creates the comment, the home server has no code to replicate delete of comment to all the downstream subscribe servers. Gamma serves as an example of the downstream servers subscribed who are not getting the delete in 0.18.2 version. The intention here is to put more developer eyes on https://github.com/LemmyNet/lemmy/issues/3625
github.com
### Requirements - [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support - [X] Did you check to see if this issue already exists? - [X] Is this only a feature request? Do not put multiple feature requests in one issue. - [X] Is this a backend issue? Use the [lemmy-ui](https://github.com/LemmyNet/lemmy-ui) repo for UI / frontend issues. ### Is your proposal related to a problem? Just a lack of the ability to show silent appreciation ### Describe the solution you'd like. Lemmy is not a commercial project, nor should it be. Gifts as designed by other social media are a toxic money dump that prey on users' good will. Gifts to other users should not be "purchased" from some central authority (though a case could be made that server maintainers could be the distributors of gifts, and at that point it's up to them?). I think theres a much more wholesome way to actually have a "gift economy" - if a post or comment of mine receives a "gift" from someone, that gift goes into my inventory. The only use for that gift in my inventory is to gift it to another post. ### Describe alternatives you've considered. more or less mused upon below ### Additional context I think this can create a positive feedback loop where users can show their appreciation to each other and feel motivated to do so But there are a couple of issues: - Where do gifts come from? Does every user just have one of each by default? Do servers distribute them based on their own rules? Are they earned via engagement? - If servers determine this, this could help differentiate communities from one another, but also increase "choice paralysis" when choosing a server. - This could also allow servers to opt out of gifting entirely if they choose to keep things simpler - What of people who don't give back? They become gift leeches and all the gifts from the community disappear into black holes. - Auto-regen periods defined by the server? I.e. after one week, if a user has no "thank you" gift, they will be granted one in their inventory - What would the gifts be, and who decides? - Do servers decide? Does lemmy only support certain gifts? If servers decide, how do we limit or support gifting between users of X server on posts of Y servers?
github.com
### Requirements - [X] Is this a bug report? For questions or discussions use https://lemmy.ml/c/lemmy_support - [X] Did you check to see if this issue already exists? - [X] Is this only a single bug? Do not put multiple bugs in one issue. - [X] Is this a backend issue? Use the [lemmy-ui](https://github.com/LemmyNet/lemmy-ui) repo for UI / frontend issues. ### Summary If you browse with show read posts enabled, and read every post on page 1 of content, when you navigate to page 2, it will show you page 2 of "unread" content, and you need to go back to page one. ### Steps to Reproduce 1. Disable "Show Read Posts" 2. Browse a community - I suggest a community with consistent content but a relatively static sort - e.g. !spaceporn@lemmy.fmhy.ml w/TopMonth 3. "Read" each post on the 1st page (e.g. upvote all of them) - Pay attention to top couple of posts on page - Notice vote counts at bottom of page 5. Browse to Page 2 - Notice large drop in post vote count between end of page 1 and page 2 - Pay attention to top couple of posts on page 6. Browse back to page 1 - Notice vote counts at top of page and bottom of page aligns between the end of the original page 1 and the viewed page "2" - Notice Page 1 content is different from original page 1 content ### Technical Details I believe this is a pagination issue when constructing the offsets used for pagination, there may need to be a mechanism to deduct or track the state of read posts. ### Version BE: 18. ### Lemmy Instance URL lemmy.fmhy.ml
cross-posted from: https://merv.news/post/26663 > most people i know use google by searching whatever question they have and including the word “reddit” at the end to find reddit threads since it currently has the most useful information. > > As Lemmy gets more and more filled with useful threads and reviews it would be great if we can collectively improve Lemmy’s SEO so just including the word lemmy in a search will show lemmy threads related to the search. > > The obscure tlds used in lemmy servers don’t help and lemmy.com currently redirects to lemm.ee. Is there a way we can improve the SEO of all instances or have lemmy.com be a aggregator of threads from many Lemmy servers?
I miss this function from reddit. I used it often to find if a post has already been submitted. Also, it was useful to see what else was posted from this domain. I hope some day this will come to Lemmy. Examples: 1. https://www.reddit.com/domain/hillelwayne.com/ 2. https://www.reddit.com/domain/hillelwayne.com/top/?sort=top&t=all
github.com
### Requirements - [X] Is this a bug report? For questions or discussions use https://lemmy.ml/c/lemmy_support - [X] Did you check to see if this issue already exists? - [X] Is this only a single bug? Do not put multiple bugs in one issue. - [X] Is this a backend issue? Use the [lemmy-ui](https://github.com/LemmyNet/lemmy-ui) repo for UI / frontend issues. ### Summary If you receive an abusive DM there is no way to remove it from your inbox without admin help. ### Steps to Reproduce Ideally a person would be able to delete all private messages in their inbox, regardless of whether they created them or not. But if this is not possible, blocking the abuse account should hide all DMs from them. Reproduction: 1. Person B send messages to person A 2. Person A block person B 3. See person B messages still show up in inbox of person A 4. Observe person A also has no way to delete person B's messages. Their stuck it person A's inbox forever unless an admin intervenes. ### Technical Details N/A ### Version 0.18.2 ### Lemmy Instance URL _No response_
github.com
### Requirements - [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support - [X] Did you check to see if this issue already exists? - [X] Is this only a feature request? Do not put multiple feature requests in one issue. - [X] Is this a backend issue? Use the [lemmy-ui](https://github.com/LemmyNet/lemmy-ui) repo for UI / frontend issues. ### Is your proposal related to a problem? allow for .mp3 .ogg etc. uploads ### Describe the solution you'd like. uploading audio files ### Describe alternatives you've considered. n/a ### Additional context n/a
github.com
### Requirements - [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support - [X] Did you check to see if this issue already exists? - [X] Is this only a feature request? Do not put multiple feature requests in one issue. - [X] Is this a backend issue? Use the [lemmy-ui](https://github.com/LemmyNet/lemmy-ui) repo for UI / frontend issues. ### Is your proposal related to a problem? As a moderator I need to know when there are new posts in the communities I moderate so I can fulfill my role. https://lemmy.world/post/1320681 ### Describe the solution you'd like. A new option in settings to receive a notification for every new post only in the communities I moderate. Current notifications for new posts ### Describe alternatives you've considered. Creating a second account subscribed only to the communities I moderate and enable new post notification on that account. https://lemmy.world/comment/1323243 ### Additional context _No response_
github.com
### Requirements - [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support - [X] Did you check to see if this issue already exists? - [X] Is this only a feature request? Do not put multiple feature requests in one issue. - [X] Is this a backend issue? Use the [lemmy-ui](https://github.com/LemmyNet/lemmy-ui) repo for UI / frontend issues. ### Is your proposal related to a problem? Many videos from external sites fail to properly embed on Lemmy clients. Video detection from a URL is based on opengraph tags (specifically `og:video`), which a lot of sites intentionally don't send, and should be made more intelligent to ensure better content sharing. This is a major sore point for Lemmy at the moment. ### Describe the solution you'd like. To facilitate better video embedding, the [`fetch_site_metadata`](https://github.com/LemmyNet/lemmy/blob/main/crates/api_common/src/request.rs#L19) function needs to be improved - this way, no UI changes will be required. One easy and cheap way to do this might be to use something like `yt-dlp`, which can extract direct video URLs from indirect URLs. There are [Rust wrappers](https://crates.io/crates/youtube_dl) available for the library. For example, consider the imgur URL `https://imgur.com/gallery/hOPRxdR`. From this, yt-dlp is able to extract the direct MP4: ``` ~ ❯ yt-dlp https://imgur.com/gallery/hOPRxdR -g https://i.imgur.com/txMlHj7.mp4 ``` This process should fail fast. If any error is encountered, simply ditch the approach and fall back to opengraph. #### Considerations 1. This can be too aggressive: for example, YouTube and `v.reddit` links can be resolved, but they are `m3u8` streams that can't be played by the UI in a `<video>` tag. [Only three formats are supported](https://www.w3schools.com/tags/tag_video.asp#:~:text=There%20are%20three%20supported%20video,MP4%2C%20WebM%2C%20and%20OGG.): `mp4`, `webm`, and `ogg`. This can be tuned in the command: ``` ~ ❯ yt-dlp https://imgur.com/gallery/hOPRxdR -g -f 'best[ext=webm]/best[ext=mp4]/best[ext=ogg]' https://i.imgur.com/txMlHj7.mp4 ``` 2. Error handling is required in case a URL can't be selected: ``` ~ ❯ yt-dlp https://v.redd.it/s2426qje27cb1 -g -f 'best[ext=webm]/best[ext=mp4]/best[ext=ogg]' ERROR: [Reddit] s2426qje27cb1: Requested format is not available. Use --list-formats for a list of available formats ``` This is a non-zero exit code, so the normal fail-fast behavior might be enough. ### Describe alternatives you've considered. I considered using `yt-dlp` to also do the downloading of a video to `pictrs`, but that seems like unnecessary storage use. I'm sure there are alternative solutions out there. ### Additional context Sample Rust code (I am not a Rust programmer so don't judge) ```rust fn run_ytdl(input: &str) -> Result<YoutubeDlOutput, youtube_dl::Error> { YoutubeDl::new(input) .download(false) .format("best[ext=webm]/best[ext=mp4]/best[ext=ogg]") .socket_timeout("10") .run() } fn extract_direct_url(input: &str) -> Option<String> { let output = run_ytdl(input).ok()?; match output { YoutubeDlOutput::Playlist(playlist) => playlist.entries?.get(0)?.clone().url, YoutubeDlOutput::SingleVideo(video) => video.url } } ```
github.com
If a database is temporarily unavailable at the start of a scheduled task, the resulting panic will permanently crash the scheduled tasks thread. This PR replaces the panic with an error log.
github.com
### Requirements - [x] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support - [X] Did you check to see if this issue already exists? - [X] Is this only a feature request? Do not put multiple feature requests in one issue. - [X] Is this a backend issue? Use the [lemmy-ui](https://github.com/LemmyNet/lemmy-ui) repo for UI / frontend issues. ### Is your proposal related to a problem? When I open Lemmy as an average user, I want to see posts that: - have active discussions - are from within the last day - are highly upvoted However, the `Hot` sort type shows me extremely recent posts with few votes or comments, and `Active` mostly shows me posts that are between one and two days old. ### Describe the solution you'd like. Create additional sort types: - `Active Six Hours` - `Active Twelve Hours` - `Active Day` - `Active Two Days` With `Active Six Hours`, the latest comment time no longer updates after six hours past the post published time. With `Active Twelve Hours`, the latest comment time no longer updates past twelve hours after the post published time, and so on an so forth. [`Active Two Days` would be the current default behavior of the `Active` sorting algorithm.](https://join-lemmy.org/docs/contributors/07-ranking-algo.html#lemmy) ### Describe alternatives you've considered. - Set the default sorting algorithm to `Hot` - Adjust the `Gravity` value until the average age of posts on the front page of lemmy.ml is twelve hours. ### Additional context Posts on the topic: - [YSK: Feel like you only see the same 2-day old content? At least on lemmy.world, you can change your homepage’s default sorting type & scope to “Hot” and “All”](https://lemmy.world/post/994692) - [FYI sort by “Top Day” if you’re tired of seeing the same top posts from 2-3 days ago](https://lemmy.world/post/192630)
github.com
**This PR is not complete (missing items detailed below), but I am submitting it already as a draft to get some early feedback.** Please check the description below before checking code - I would really appreciate feedback on the overall design which is included in the description. But comments on the partially complete code are of course welcome as well. ---- # Introduction This PR contains an overhaul of Lemmy authentication. It introduces three new authentication tokens: access tokens, refresh tokens, and api tokens (more details below). The changes are intended to be backwards compatible - the existing /login endpoint will become deprecated but will remain operational until we are ready to remove it in a future version. ## What is wrong with our current authentication? 1. Auth tokens never expire: https://github.com/LemmyNet/lemmy/issues/3364 2. Auth sessions can't be revoked by users 3. There is no support for httpOnly cookie based auth: https://github.com/LemmyNet/lemmy-ui/issues/1252 4. There is no support for api token based auth - all 3rd party apps require user passwords 5. All auth tokens have full access to everything, their scope can't be limited This PR contains intends to solve all these issues. ## Proposed solution This PR proposes to replace the existing auth token with 3 new types of tokens: ### Access token This token can be acquired with either a refresh token or an API token. The new access token is intended to be a backwards compatible drop-in replacement for the existing auth token, with a few key differences: * It expires within 5 minutes (so even if it leaks, it can only be abused within 5 minutes of the leak) * It contains a `method` claim, which can be used later to limit certain activities to specific methods (for example, disallow password changes if the access token was obtained via an API token) ### Refresh token This token can be acquired using username + password (+ 2fa). It lives in a secure httpOnly cookie (can't be read from browser js), which is limited only to the /api/v3/get_access_token path. This is intended only for trusted web interfaces (such as lemmy-ui) and can be used to create access tokens with full access to the user. Each refresh token can be considered a separate "session". Each token records its last use time, as well as last use ip address - these values can be displayed to users in some new security UI so they get an overview of their active sessions. Each refresh token expires 2 weeks after it was last used, or when revoked manually by a user. ### API token This token must be manually created by users with a specific label and expiry date. This is intended for 3rd party apps to avoid users from entering their passwords directly into untrusted code. The api token can be used similarly to refresh tokens to request access tokens, but the created access tokens would have limited access. Each API token will also record their last use time as well as last use ip address. API tokens expire after their user defined expiry date, or when revoked manually. ---- To summarize the general flow: 1. Acquire either a refresh token (if trusted web ui) or an API token (if 3rd party app) 2. Request access token using the token from step 1 3. Make all API requests with access token from step 2 4. If access token is close to expiry (or last request failed due to token), get a new access token (and retry last request) 5. If getting access token fails due to a token error, assume the (refresh or api) token has expired and go back to step 1 ## Rollout plan 1. Release the new logic in a minor Lemmy version 2. Add a migration guide to release notes to allow app developers to migrate to the new APIs 3. Update Lemmy-ui to use the new endpoints 4. After some time has passed, remove the old /login endpoint in a backwards-incompatible Lemmy update ## TODO in this PR * Add refresh token list & revoke endpoints * Add api token create & list & revoke endpoints * Disallow some actions (new api token creation + password change + reading user e-mail?) when access token method is `Api` * Add some tests ## TODO in future PRs * Switch lemmy-ui to use new authentication * Add security page to lemmy-ui, where users can see and revoke their sessions (refresh tokens), as well as see/revoke/create API tokens * Add method for 3rd party apps to redirect users to an API token creation page (with a potential return_url to automatically get back to the app with the created token)
github.com
When re-running the first cargo clippy command in fix-clippy.sh, the build time of db_views is now 311.1s instead of 1281.9s Helps with #3610
github.com
When re-running the first cargo clippy command in fix-clippy.sh, the build time of db_views is now 311.1s instead of 1281.9s Helps with #3610
github.com
### Requirements - [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support - [X] Did you check to see if this issue already exists? - [X] Is this only a feature request? Do not put multiple feature requests in one issue. - [X] Is this a backend issue? Use the [lemmy-ui](https://github.com/LemmyNet/lemmy-ui) repo for UI / frontend issues. ### Is your proposal related to a problem? hacking? ### Describe the solution you'd like. Pitch Jerboa should have an opt-in feature for servers that lets them be temporarily taken down remotely for their own safety by the Jerboa non-profit in case of an extreme security vulnerability requiring an update Motivation This would protect the servers of those who opt-in along with the users on that server. As it is an opt-in, no one will be left annoyed. All users who join the server alongside the owner would be aware of a potential takedown at any moment and would know of its importance and how it is solely for their safety. Servers can freely opt-in or out. ### Describe alternatives you've considered. sending out a please update notice? ### Additional context mastodon got lucky that mozilla paid pentesters who reported it and it got patched before it coukd be exploited, lemmy migjt not be so fortjnate
github.com
### Requirements - [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support - [X] Did you check to see if this issue already exists? - [X] Is this only a feature request? Do not put multiple feature requests in one issue. - [X] Is this a backend issue? Use the [lemmy-ui](https://github.com/LemmyNet/lemmy-ui) repo for UI / frontend issues. ### Is your proposal related to a problem? hacking? ### Describe the solution you'd like. Pitch Jerboa should have an opt-in feature for servers that lets them be temporarily taken down remotely for their own safety by the Jerboa non-profit in case of an extreme security vulnerability requiring an update Motivation This would protect the servers of those who opt-in along with the users on that server. As it is an opt-in, no one will be left annoyed. All users who join the server alongside the owner would be aware of a potential takedown at any moment and would know of its importance and how it is solely for their safety. Servers can freely opt-in or out. ### Describe alternatives you've considered. sending out a please update notice? ### Additional context mastodon got lucky that mozilla paid pentesters who reported it and it got patched before it coukd be exploited, lemmy migjt not be so fortjnate
github.com
### Requirements - [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support - [X] Did you check to see if this issue already exists? - [X] Is this only a feature request? Do not put multiple feature requests in one issue. - [X] Is this a backend issue? Use the [lemmy-ui](https://github.com/LemmyNet/lemmy-ui) repo for UI / frontend issues. ### Is your proposal related to a problem? hacking? ### Describe the solution you'd like. Pitch Jerboa should have an opt-in feature for servers that lets them be temporarily taken down remotely for their own safety by the Jerboa non-profit in case of an extreme security vulnerability requiring an update Motivation This would protect the servers of those who opt-in along with the users on that server. As it is an opt-in, no one will be left annoyed. All users who join the server alongside the owner would be aware of a potential takedown at any moment and would know of its importance and how it is solely for their safety. Servers can freely opt-in or out. ### Describe alternatives you've considered. sending out a please update notice? ### Additional context mastodon got lucky that mozilla paid pentesters who reported it and it got patched before it coukd be exploited, lemmy migjt not be so fortjnate
github.com
### Requirements - [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support - [X] Did you check to see if this issue already exists? - [X] Is this only a feature request? Do not put multiple feature requests in one issue. - [X] Is this a backend issue? Use the [lemmy-ui](https://github.com/LemmyNet/lemmy-ui) repo for UI / frontend issues. ### Is your proposal related to a problem? hacking? ### Describe the solution you'd like. Pitch Jerboa should have an opt-in feature for servers that lets them be temporarily taken down remotely for their own safety by the Jerboa non-profit in case of an extreme security vulnerability requiring an update Motivation This would protect the servers of those who opt-in along with the users on that server. As it is an opt-in, no one will be left annoyed. All users who join the server alongside the owner would be aware of a potential takedown at any moment and would know of its importance and how it is solely for their safety. Servers can freely opt-in or out. ### Describe alternatives you've considered. sending out a please update notice? ### Additional context mastodon got lucky that mozilla paid pentesters who reported it and it got patched before it coukd be exploited, lemmy migjt not be so fortjnate
github.com
### Requirements - [X] Is this a bug report? For questions or discussions use https://lemmy.ml/c/lemmy_support - [X] Did you check to see if this issue already exists? - [X] Is this only a single bug? Do not put multiple bugs in one issue. - [X] Is this a backend issue? Use the [lemmy-ui](https://github.com/LemmyNet/lemmy-ui) repo for UI / frontend issues. ### Summary lemmy_server code fails to convert RawAnnouncableActivities into a AnnouncableActivities:Page before reaching logic to reject Page. https://github.com/LemmyNet/lemmy/blob/e9e76549a88cfbdab36f00d302cceabcaaa24f4c/crates/apub/src/activities/community/announce.rs#L47 ``` let activity: AnnouncableActivities = self.clone().try_into()?; // This is only for sending, not receiving so we reject it. if let AnnouncableActivities::Page(_) = activity { return Err(LemmyErrorType::CannotReceivePage)?; } ``` The code errors on the first line with `data did not match any variant of untagged enum AnnouncableActivities`, not reaching the second line of code that would return `Err(LemmyErrorType::CannotReceivePage)`. ### Steps to Reproduce 1. Subscribe a lemmy instance to various bots mentioned below 2. Look in server error logs for `data did not match any variant of untagged enum AnnouncableActivities` 3. capture the raw incoming data to analyze Example of incoming raw data causing this match problem: `RawAnnouncableActivities { id: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("lemmit.online")), port: None, path: "/activities/announce/f513b770-c440-48f3-b0f0-21317b9e85b7", query: None, fragment: None }, actor: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("lemmit.online")), port: None, path: "/c/comics", query: None, fragment: None }, other: {"to": Array [String("https://www.w3.org/ns/activitystreams#Public")], "object": Object {"id": String("https://lemmit.online/post/201322"), "actor": String("https://lemmit.online/u/bot"), "type": String("Page"), "attributedTo": String("https://lemmit.online/u/bot"), "to": Array [String("https://lemmit.online/c/comics"), String("https://www.w3.org/ns/activitystreams#Public")], "name": String("I guess my son qualifies as my pet as well."), "cc": Array [], "content": String("<h5>This is an automated archive made by the <a href=\"https://lemmit.online/post/14692\">Lemmit Bot</a>.</h5>\n<p>The original was posted on <a href=\"https://old.reddit.com/r/comics/comments/151el8i/i_guess_my_son_qualifies_as_my_pet_as_well/\">/r/comics</a> by <a href=\"https://old.reddit.com/u/AlloyComics\">/u/AlloyComics</a> on 2023-07-16 21:08:16.</p>\n"), "mediaType": String("text/html"), "source": Object {"content": String("##### This is an automated archive made by the [Lemmit Bot](https://lemmit.online/post/14692).\nThe original was posted on [/r/comics](https://old.reddit.com/r/comics/comments/151el8i/i_guess_my_son_qualifies_as_my_pet_as_well/) by [/u/AlloyComics](https://old.reddit.com/u/AlloyComics) on 2023-07-16 21:08:16.\n"), "mediaType": String("text/markdown")}, "attachment": Array [Object {"href": String("https://i.redd.it/q53smvggldcb1.png"), "type": String("Link")}], "commentsEnabled": Bool(true), "sensitive": Bool(false), "published": String("2023-07-17T01:58:34.158916+00:00"), "language": Object {"identifier": String("en"), "name": String("English")}, "audience": String("https://lemmit.online/c/comics")}, "cc": Array [String("https://lemmit.online/c/comics/followers")], "type": String("Announce")} }` ### Technical Details It seems bods are generating these type: Page https://lemmy.world/u/MatchThreadBot https://lemmit.online/u/bot Example of a post that comes as type: Page and causes this problem: https://lemmit.online/post/201322 ### Version BE: 0.18.2 ### Lemmy Instance URL _No response_
github.com
### Requirements - [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support - [X] Did you check to see if this issue already exists? - [X] Is this only a feature request? Do not put multiple feature requests in one issue. - [X] Is this a backend issue? Use the [lemmy-ui](https://github.com/LemmyNet/lemmy-ui) repo for UI / frontend issues. ### Is your proposal related to a problem? Users can receive unsolicited private or direct messages (PM or DM) in their inbox. Although a feature like OM or DM can be valuable to some, it can also lead to unwanted, abusive, commercial or otherwise unsolicited messages without the user being able to block them before they are received. Related issue (deleting unwanted PM or DM): https://github.com/LemmyNet/lemmy/issues/3629 ### Describe the solution you'd like. Add a checkbox in settings where a user can disable a PM or DM being received ### Describe alternatives you've considered. Add the line "No DM" in my profile ### Additional context _No response_
github.com
### Requirements - [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support - [X] Did you check to see if this issue already exists? - [X] Is this only a feature request? Do not put multiple feature requests in one issue. - [X] Is this a backend issue? Use the [lemmy-ui](https://github.com/LemmyNet/lemmy-ui) repo for UI / frontend issues. ### Is your proposal related to a problem? I have just heared about the XSS volunerability and wanted to update my instance ASAP, but was left realizing that since 0.17.3, there has not been any new arm64 builds submitted to Docker Hub at all. So just in case, I checked [the docs](https://join-lemmy.org/docs/administration/install_docker.html) to see if I might just be using the wrong containers, but no - all the ones I use. ### Describe the solution you'd like. I would like to see arm64 builds return. Even if they are only found on `:latest`, that'd still be enough for me - I use Watchtower to autp-update my containers anyway. ### Describe alternatives you've considered. Provide arm-tagged images like older versions (i.e. `0.17.3-arm64-linux`). ### Additional context _No response_
github.com
### Requirements - [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support - [X] Did you check to see if this issue already exists? - [X] Is this only a feature request? Do not put multiple feature requests in one issue. - [X] Is this a backend issue? Use the [lemmy-ui](https://github.com/LemmyNet/lemmy-ui) repo for UI / frontend issues. ### Is your proposal related to a problem? Yes, the proposal is related to the problem of underrepresentation of smaller instances on the main feed in Lemmy, as mentioned in issue #1026. ### Describe the solution you'd like. The solution proposed is to balance instance scores based on monthly active users instead of the size of the community. By considering the monthly active users, smaller instances will have a fair chance of being represented on the main feed, ensuring a more diverse representation of instances. ### Describe alternatives you've considered. The approach considered on issue #1026 can be particularly important in the subscribed feed, where communities should be equally represented. Or even better allow for weighted community subscription as proposed in issue #3518. This would ensure that you see more from communities that you like more. ### Additional context Balancing communities would dilute the uniqueness and individual themes of each instance. Instead of balancing communities, balancing instance scores based on monthly active users would accomplish the goal of achieving a more inclusive and diverse experience for users while still maintaining the individuality of each instance. Considering the possibility of Meta making a Reddit alternative with millions of active users, the feed of every instance would be dominated by the communities of that instance. Implementing a balanced scoring system based on monthly active users would help prevent this domination and promote a more diverse and inclusive experience for users on Lemmy.
github.com
If a database is temporarily unavailable at the start of a scheduled task, the resulting panic will permanently crash the scheduled tasks thread. This PR replaces the panic with an error log.
github.com
### Requirements - [x] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support - [X] Did you check to see if this issue already exists? - [X] Is this only a feature request? Do not put multiple feature requests in one issue. - [X] Is this a backend issue? Use the [lemmy-ui](https://github.com/LemmyNet/lemmy-ui) repo for UI / frontend issues. ### Is your proposal related to a problem? When I open Lemmy as an average user, I want to see posts that: - have active discussions - are from within the last day - are highly upvoted However, the `Hot` sort type shows me extremely recent posts with few votes or comments, and `Active` mostly shows me posts that are between one and two days old. ### Describe the solution you'd like. Create additional sort types: - `Active Six Hours` - `Active Twelve Hours` - `Active Day` - `Active Two Days` With `Active Six Hours`, the latest comment time no longer updates after six hours past the post published time. With `Active Twelve Hours`, the latest comment time no longer updates past twelve hours after the post published time, and so on an so forth. [`Active Two Days` would be the current default behavior of the `Active` sorting algorithm.](https://join-lemmy.org/docs/contributors/07-ranking-algo.html#lemmy) ### Describe alternatives you've considered. - Set the default sorting algorithm to `Hot` - Adjust the `Gravity` value until the average age of posts on the front page of lemmy.ml is twelve hours. ### Additional context Posts on the topic: - [YSK: Feel like you only see the same 2-day old content? At least on lemmy.world, you can change your homepage’s default sorting type & scope to “Hot” and “All”](https://lemmy.world/post/994692) - [FYI sort by “Top Day” if you’re tired of seeing the same top posts from 2-3 days ago](https://lemmy.world/post/192630)
github.com
**This PR is not complete (missing items detailed below), but I am submitting it already as a draft to get some early feedback.** Please check the description below before checking code - I would really appreciate feedback on the overall design which is included in the description. But comments on the partially complete code are of course welcome as well. ---- # Introduction This PR contains an overhaul of Lemmy authentication. It introduces three new authentication tokens: access tokens, refresh tokens, and api tokens (more details below). The changes are intended to be backwards compatible - the existing /login endpoint will become deprecated but will remain operational until we are ready to remove it in a future version. ## What is wrong with our current authentication? 1. Auth tokens never expire: https://github.com/LemmyNet/lemmy/issues/3364 2. Auth sessions can't be revoked by users 3. There is no support for httpOnly cookie based auth: https://github.com/LemmyNet/lemmy-ui/issues/1252 4. There is no support for api token based auth - all 3rd party apps require user passwords 5. All auth tokens have full access to everything, their scope can't be limited This PR contains intends to solve all these issues. ## Proposed solution This PR proposes to replace the existing auth token with 3 new types of tokens: ### Access token This token can be acquired with either a refresh token or an API token. The new access token is intended to be a backwards compatible drop-in replacement for the existing auth token, with a few key differences: * It expires within 5 minutes (so even if it leaks, it can only be abused within 5 minutes of the leak) * It contains a `method` claim, which can be used later to limit certain activities to specific methods (for example, disallow password changes if the access token was obtained via an API token) ### Refresh token This token can be acquired using username + password (+ 2fa). It lives in a secure httpOnly cookie (can't be read from browser js), which is limited only to the /api/v3/get_access_token path. This is intended only for trusted web interfaces (such as lemmy-ui) and can be used to create access tokens with full access to the user. Each refresh token can be considered a separate "session". Each token records its last use time, as well as last use ip address - these values can be displayed to users in some new security UI so they get an overview of their active sessions. Each refresh token expires 2 weeks after it was last used, or when revoked manually by a user. ### API token This token must be manually created by users with a specific label and expiry date. This is intended for 3rd party apps to avoid users from entering their passwords directly into untrusted code. The api token can be used similarly to refresh tokens to request access tokens, but the created access tokens could have limited access in the future. Each API token will also record their last use time as well as last use ip address. API tokens expire after their user defined expiry date, or when revoked manually. ---- To summarize the general flow: 1. Acquire either a refresh token (if trusted web ui) or an API token (if 3rd party app) 2. Request access token using the token from step 1 3. Make all API requests with access token from step 2 4. If access token is close to expiry (or last request failed due to token), get a new access token (and retry last request) 5. If getting access token fails due to back refresh/api token, tell the user they need to reauthenticate ## Rollout plan 1. Release the new logic in a minor Lemmy version 2. Add a migration guide to release notes to allow app developers to migrate to the new APIs 3. Update Lemmy-ui to use the new endpoints 4. After some time has passed, remove the old /login endpoint in a backwards-incompatible Lemmy update ## TODO in this PR * Add refresh token list & revoke endpoints * Add api token create & list & revoke endpoints * Disallow some actions (new api token creation + password change + reading user e-mail?) when access token method is `Api` * Add some tests ## TODO in future PRs * Switch lemmy-ui to use new authentication * Add security page to lemmy-ui, where users can see and revoke their sessions (refresh tokens), as well as see/revoke/create API tokens * Add method for 3rd party apps to redirect users to an API token creation page (with a potential return_url to automatically get back to the app with the created token)
github.com
If a database is temporarily unavailable at the start of a scheduled task, the resulting panic will permanently crash the scheduled tasks thread. This PR replaces the panic with an error log.
github.com
> When a community is deleted if someone searches for it, it doesn't appear on the communities and if he then tries to create it the button keeps spinning without any warning that the community already exists. The only people who can restore the community are the admin and the person who deleted it in the first place. I think there should be a warning indicating that the community already exists and the button shouldn't spin. Or the moderators could be kicked out and the person trying to create the community made the new moderator. I don't see any reason for moderators to sit on a removed community. > > *Originally posted by [ghost](https://github.com/ghost) in [#1105](https://github.com/LemmyNet/lemmy-ui/issues/1105)*
github.com
### Requirements - [X] This is a bug report, and if not, please post to https://lemmy.ml/c/lemmy_support instead. - [X] Please [check](https://github.com/LemmyNet/lemmy-ui/issues) to see if this issue already exists. - [X] It's a single bug. Do not report multiple bugs in one issue. - [X] It's a frontend issue, not a backend issue; Otherwise please create an issue on the [backend repo](https://github.com/LemmyNet/lemmy) instead. ### Summary Docker setup. After updating to the 0.18.0 images for both lemmy-ui and lemmy backend, the lemmy-ui logs show a pictrs error, and the main site returns a "Server error". ``` FetchError: request to https://SITE_URL_REDACTED/pictrs/image/a29da3fc-b6ce-4e59-82b0-1a9c94f8faed.webp failed, reason: connect ECONNREFUSED 127.0.1.1:443 at ClientRequest.<anonymous> (/app/node_modules/node-fetch/lib/index.js:1505:11) at ClientRequest.emit (node:events:511:28) at TLSSocket.socketErrorListener (node:_http_client:495:9) at TLSSocket.emit (node:events:511:28) at emitErrorNT (node:internal/streams/destroy:151:8) at emitErrorCloseNT (node:internal/streams/destroy:116:3) at process.processTicksAndRejections (node:internal/process/task_queues:82:21) { type: 'system', errno: 'ECONNREFUSED', code: 'ECONNREFUSED' } ``` ### Steps to Reproduce 1. Update docker-compose.yml to 0.18.0 2. docker-compose down 3. docker-compose up 4. access site ### Technical Details Docker setup. After updating to the 0.18.0 images for both lemmy-ui and lemmy backend, the lemmy-ui logs show a pictrs error, and the main site returns a "Server error". ``` FetchError: request to https://SITE_URL_REDACTED/pictrs/image/a29da3fc-b6ce-4e59-82b0-1a9c94f8faed.webp failed, reason: connect ECONNREFUSED 127.0.1.1:443 at ClientRequest.<anonymous> (/app/node_modules/node-fetch/lib/index.js:1505:11) at ClientRequest.emit (node:events:511:28) at TLSSocket.socketErrorListener (node:_http_client:495:9) at TLSSocket.emit (node:events:511:28) at emitErrorNT (node:internal/streams/destroy:151:8) at emitErrorCloseNT (node:internal/streams/destroy:116:3) at process.processTicksAndRejections (node:internal/process/task_queues:82:21) { type: 'system', errno: 'ECONNREFUSED', code: 'ECONNREFUSED' } ``` ### Lemmy Instance Version 0.18.0 ### Lemmy Instance URL _No response_
github.com
related to https://github.com/LemmyNet/lemmy/pull/3318
github.com
### Requirements - [X] Is this a feature request? For questions or discussions use https://lemmy.ml/c/lemmy_support - [X] Did you check to see if this issue already exists? - [X] Is this only a feature request? Do not put multiple feature requests in one issue. - [X] Is this a backend issue? Use the [lemmy-ui](https://github.com/LemmyNet/lemmy-ui) repo for UI / frontend issues. ### Is your proposal related to a problem? In the web UI, when visiting the `/communities` page, I would like the ability to sort by clicking the column headers. I believe this may be a backend issue, though, as I can't seem to supply any sort values in the API call to `/community/list` that have the effect I want. I have never written any Rust before, so feel free to tell me I have no idea what I'm talking about, but it would appear that the `sort` URL parameter just gets passed straight to the query builder. This request type is [defined here](https://github.com/LemmyNet/lemmy/blob/main/crates/api_common/src/community.rs#L78C24-L78C24) as a `SortType`, which is defined as [an enum here](https://github.com/LemmyNet/lemmy/blob/main/crates/db_schema/src/lib.rs#L51-L69). The values in that enum do not correspond to much of the data in the `counts` section of the `/community/list` response, making me think the sorting is ultimately ignored. Furthermore, if I query the server with `&sort=Hot` one might think that the results would be sorted by their `hot_rank`, but this does not appear to be the case. Again, if I'm off base here, please point me in the right direction. ### Describe the solution you'd like. Seems the solution is to implement listing communities sorted by various count values in the API, then to expose the new sorting features through the lemmy-ui project. ### Describe alternatives you've considered. I have used the tools at https://lemmyverse.net/ to get to the data I want. There I can filter down to my instance, view its communities, and sort by things like subscriber count and activity and so on. ### Additional context _No response_