• umbrella@lemmy.ml
    link
    fedilink
    arrow-up
    1
    ·
    2 months ago

    this week i sudo shutdown now our main service right at the end of the workday because i tought it was a local terminal.

    not a bright move.

    • Trainguyrom@reddthat.com
      link
      fedilink
      English
      arrow-up
      1
      ·
      2 months ago

      I was making after hours config changes on a pair of mostly-but-not-entirely redundant Cisco L3 switches which basically controlled the entire network at that location. While updating the running configs I mixed up which ssh session was which switch and accidentally gave both switches the same IP address, and before I noticed the error I copied the running config to the startup config.

      Due to other limitations and the fact that these changes were to fix DNS issues (and therefore I couldn’t rely on DNS to save me) I ended up keeping sshing in by IP until I got the right switch and trying to make the change before my session died due to dropped packets from the mucked up network situation I had created. That easily added a couple of hours of cleanup to the maintainence I was doing

    • LiveLM@lemmy.zip
      link
      fedilink
      English
      arrow-up
      1
      ·
      2 months ago

      Best thing I did was change my shell prompt so I can easily tell when it isn’t my machine

    • SavvyWolf@pawb.social
      link
      fedilink
      English
      arrow-up
      1
      ·
      2 months ago

      There’s a package called molly-guard which will check to see if you are connected via ssh when you try to shut it down. If you are, it will ask you for the hostname of the system to make sure you’re shutting down the right one.

      Very usefull program to just throw onto servers.