lemmy_admin Lemmy Administration Docker container logging
Jump
  • OdiousStooge OdiousStooge 1 year ago 100%

    Huge thank you! I had a feeling something like this was going on but had no idea how to troubleshoot/fix.

    My pictrs and lemmy containers were the biggest between 3-8 GB (significant for a smaller instance) after a couple weeks.

    For anyone who finds this, in addition to what OP provided here, another command I found helpful (since I am a docker noob 🙂) to find the name of the currently running containers:

    docker ps --format '{{.Name}}'

    1
  • lemmy_support Lemmy Support Tips for "Care and Feeding" of my Instance - Disk Usage Rising
    Jump
  • OdiousStooge OdiousStooge 1 year ago 100%

    If anyone stumbles across this, the command to connect to the DB is (run from the root of your lemmy install - assuming a docker install):

    docker-compose exec postgres psql -U lemmy

    1
  • lemmy_support Lemmy Support Tips for "Care and Feeding" of my Instance - Disk Usage Rising
    Jump
  • OdiousStooge OdiousStooge 1 year ago 100%

    After some tinkering, **yes ** this indeed was my issue. The logs for pictrs and lemmy in particular were between 3 and 8 gb only after a couple weeks of info level logging.

    Steps to fix (the post above has more detail but adding my full workflow in case that helps folks, some of this wasn't super apparent to me) - these steps assume a docker/ansible install:

    1. SSH to your instance.

    2. Change to your instance install dir

    most likely: cd /srv/lemmy/{domain.name}

    1. List currently running containers

    docker ps --format '{{.Name}}'

    Now for each docker container name:

    1. Find the path/name of the associated log file:

    docker inspect --format='{{.LogPath}}' {one of the container names from above}

    1. Optionally check the file size of the log

    ls -lh {path to log file from the inspect command}

    1. Clear the log

    truncate -s 0 {path to log file from the inspect command}

    After you have cleared any logs you want to clear:

    1. Modify docker-compose.yml adding the following to each container:
    logging:
          driver: "json-file"
          options:
            max-size: "100m"
    
    1. Restart the containers

    docker-compose restart

    2
  • lemmy_support Lemmy Support Tips for "Care and Feeding" of my Instance - Disk Usage Rising
    Jump
  • OdiousStooge OdiousStooge 1 year ago 100%

    UPDATE:

    If anyone else is running into consistently rising disk I am pretty sure this is my issue (RE logs running with no cap):

    https://lemmy.eus/post/172518

    Trying out ^ and will update with my findings if it helps.

    1
  • lemmy_support Lemmy Support Tips for "Care and Feeding" of my Instance - Disk Usage Rising
    Jump
  • OdiousStooge OdiousStooge 1 year ago 100%

    Gotcha thanks! Thats good to know. Based on the originating ticket: https://github.com/LemmyNet/lemmy/issues/1133

    Sounds like it might be safe for me to purge that table a bit more often as well.

    Dumb question, how are you profiling (RE your mention of getting a better idea of which tables might be bloated) your DB? Just SSHing into your box and direct connecting to DB? Or are there other recommended workflows?

    1
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearLE
    Lemmy Support OdiousStooge 1 year ago 100%
    Tips for "Care and Feeding" of my Instance - Disk Usage Rising

    Any helpful tips for general care and feeding I should be doing on a regular basis? I know I need to keep an eye on updates and re-run my ansible setup form time to time to stay up to date. But I have also been keeping an eye on my VPS metrics to see when/if I need to beef up the server. One thing I am noticing is a steadily increasing disk utilization (which mostly makes sense except its seeming a bit faster than I expected as most all media is links to external sites rather than uploading media directly to my instance). Anything I can do to manage that short of just adding more space? Like are there logs/cached content that need to be purged from time to time? Thank you!

    9
    6
    wow World of Warcraft Classic HC Plans?
    Jump
  • OdiousStooge OdiousStooge 1 year ago 100%
  • weirdaiart
    Weird AI Art OdiousStooge 1 year ago 100%
    Scary Busey
    2
    0
    technology Technology Reddit’s API rug pull
    Jump
  • OdiousStooge OdiousStooge 1 year ago 100%

    Fire ship is great. Feel like it's the perfect mix of memery and we'll researched hot takes.

    Hadn't heard the note he makes about VCs before. Interesting... Defo assumed some of this was sparked by ChatGPTs success though and how much they leveraged the data API for training sets.

    1
  • food Food and Cooking Browned Butter and Sage Gnocchi
    Jump
  • OdiousStooge OdiousStooge 1 year ago 100%

    Damn that looks amazing. Love me those fluffy little potato pillows.

    1
  • javascript JavaScript community A Better Way to Type React Components
    Jump
  • OdiousStooge OdiousStooge 1 year ago 100%

    I flip flop on TS. I really liked it when I was using Angular a bunch as it really felt like a first class citizen there.

    In react projects it's been a bit of a pain having to sift through starter kits/setup my own config for TS and such.

    I defo like the dynamism of JS, but end of the day, if I'm working on a project with other peeps, it's nice to have some type safety and data contracts/type ahead.

    1
  • wow
    World of Warcraft OdiousStooge 1 year ago 100%
    Classic HC Plans?

    Folks been planning their HC toons/profs for when official drops? Been trialing a few on unofficial Bloodsail and having a blast. Feeling good about Priest with ench (until GMW) and then tailoring/skinning.

    10
    2
    diablo
    Diablo OdiousStooge 1 year ago 100%
    Anyone else a D4 alt-o-holic?

    I get to like 20/30 and then want to try another class. Loving the mechanics on all of them! Hard to settle.

    12
    10
    asklemmy Asklemmy How are you going to avoid using reddit?
    Jump
  • OdiousStooge OdiousStooge 1 year ago 100%

    I removed reddit sync from my phone screen and put jerboa in it's place. Muscle memory 😎

    4
  • lemmy_support Lemmy Support Any way to "allow all" on a new instance?
    Jump
  • OdiousStooge OdiousStooge 1 year ago 100%

    💖 man ima dumb dumb. Most excellent, thank you!

    8
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearLE
    Lemmy Support OdiousStooge 1 year ago 75%
    Any way to "allow all" on a new instance?

    I have just been hand-jamming lemmy domains I know about in the `allowed instances` section of my new instance. Is there a way to just allow all? Or maybe a running list somewhere of domains I can copy/paste? Seeing a bunch of: `WARN Error encountered while processing incoming HTTP request: lemmy_server::root_span_builder: Domain is not in allowlist` Flowing through my logs and from what I can tell, it doesn't seem to tell me WHICH domain I should be trying to add 🥲 Any thoughts? Thank you!

    2
    4
    "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearGE
    General OdiousStooge 1 year ago 100%
    How to make NSFW Communities Discoverable?

    It is my (perhaps incorrect) understanding that NSFW communities are hidden from the global search unless you are registered/logged in at that instance. This makes it tough for federated users to find your stuff. Any tips on making that easier? For example, I am registered at `lemmy.ml` but wanna see y'alls `goodies` 👁️👄👁️ I also run an instance that allows NSFW content so curious for helping discoverability of that instance as well. Was thinking of maybe adding a 'site map-esque' concept to a general/welcome community's sidebar with the appropriate search strings. ie, something like: `If you'd like to subscribe to a community put this in your search bar...` `Community 1 is blah` `!{community_name_1}@{instance_url}` `Community 2 is blem` `!{community_name_2}@{instance_url}` `Community 3 is shmlep` `!{community_name_3}@{instance_url}` Any better way?

    12
    3
    fediverse Fediverse *Permanently Deleted*
    Jump
  • OdiousStooge OdiousStooge 1 year ago 100%

    Thanks!. Yeah I gotta figure that out. Something funky going on.

    There are several communities in there atm but I'm getting some "odd" federation behavior.

    If you are interested in federating there are two ways I have seen it work (again, warning, note that it is NSFW 😀 ):

    • use your search feature on your main instance and enter !butt_frenzy@booty.world then you can subscribe

    • or, from your main instance, alter your url to: {your main instance}/c/butt_frenzy@booty.world so for example if your main instance is lemmy.ml that would be: https://lemmy.ml/c/butt_frenzy@booty.world

    ^ butt_frenzy is one of the communities on booty.world for example.

    Gotta be a better way, and I am probs doing something wrong. Will work on it. Open to feedback if anyone has any advice though haha.

    Thanks.

    1
  • lemmy_support Lemmy Support Help Troubleshooting Email Verification on Self Hosted
    Jump
  • OdiousStooge OdiousStooge 1 year ago 100%

    Noice many thanks! I'll give it a shot.

    2
  • lemmy_support Lemmy Support Help Troubleshooting Email Verification on Self Hosted
    Jump
  • OdiousStooge OdiousStooge 1 year ago 100%

    Oh snap, yes I am using Digital Ocean.

    Looks like you might have forgotten to add the link, mind chucking that in when you get a chance?

    Thanks, this seems like a good lead as well.

    3
  • lemmy_support Lemmy Support Help Troubleshooting Email Verification on Self Hosted
    Jump
  • OdiousStooge OdiousStooge 1 year ago 100%

    Cool thanks! Last thing for now, and then I'll go do some debugging.

    Dumb Q: The lemmy install includes an SMTP server correct? Like I don't need to use an external service for that?

    2
  • lemmy_support Lemmy Support Help Troubleshooting Email Verification on Self Hosted
    Jump
  • OdiousStooge OdiousStooge 1 year ago 100%

    Nice! Thanks your mention of the config.hjson makes me wonder lol. I probably goofed that too.

    I did the ansible install which I believe just adds orchestration on top of the Docker install. I'll ssh in and try the docker-compose command.

    Do you know, if I did goof the email config, can I just tweak the config locally and then re-run ansible? Or do I need to do some manual tweaks to the deployed solution? Or I suppose at this point it might be easier to just blow the instance away and start fresh.

    2
  • "Initials" by "Florian Körner", licensed under "CC0 1.0". / Remix of the original. - Created with dicebear.comInitialsFlorian Körnerhttps://github.com/dicebear/dicebearLE
    Lemmy Support OdiousStooge 1 year ago 100%
    Help Troubleshooting Email Verification on Self Hosted

    Any good resources/Discord servers (to discuss with other devs) for troubleshooting user login/email verification issues? I have a self-hosted instance and was able to make the admin user, but when trying to create a new test user, the user does not get the verification email. I assume its related to this issue because I see the same issue but on login rather than sign up: https://github.com/LemmyNet/lemmy-ui/issues/1080 I am able to create the new user but get no email verification and then when I try to login the login button gets stuck with the spinner. No frontend console logs so I assume something is dying server-side and not bubbling the error. Really just looking for some guidance/links to docs on how I can access logs to troubleshoot server-side errors (and my gut tells me maybe an email issue but I am a novice in this space so not sure). Any help is appreciated. Thank you.

    8
    30