To do data backups
And test your backups.
You don’t have backups until youve restored a backup.
What’s your process for testing it?
Depends on the kind of backup really
Just a folder full of duplicate files? Try to open them.
Having a drive image you can restore from? Take an extra drive and try to “restore” the contents of your backup onto it. You use the extra drive because if you just use your primary drive you may brick yourself.
There’s definitely types of backups I’m not covering here but you should do research into the type of backups you want to use and the restoration process, and basically try the restoration process intermittently.
I kinda got lucky. I wanted to install a new, bigger SSD. At the same time, I had been wanting to start doing backups, but was too lazy to set it up. Two birds, one stone. Set up backups, tested it by copying everything to the new SSD. Everything worked first try!
I used rsnapshot for backups. I made a little container that spins up, pulls my data, then shuts down. And then I made a script that does that and made cron jobs for it.
I once set an S3 lifecycle setting that accidentally affected 3 years worth of logs to Glacier. The next morning I woke up to a billing alert and an AWS bill with an extra $250k in charges (our normal run rate was $30k/month at the time). Basically I spent my entire add annual cloud budget for the year overnight.
Thankfully after an email to our account rep and a bunch of back and forth I was able to get the charges reduced to $4,300.
Is cloud even cheaper than managing your own infrastructure anymore?
The problem is having a competent team to manage your infrastructure. You can do a lot with a handful of people - but you need competences spanning a lot of areas, and finding that is pretty hard.
If you can get a competent team the only advantage cloud still has is the ability to quickly scale up and down - but if there might be a need for that it’d still be better to go hybrid, most on your own hardware, and just the prepared ability to quickly bring up cloud workers if needed. The cost savings of properly doing it yourself are so huge that it still might be cheaper to just have some pre-provisioned standby hardware for that, though.
If I never have to buzz into another colo and stand in the exhaust of hundreds of servers again, it’s worth every single penny. If I never have to plan for capacity weeks to years in advance again, its worth every penny.
Depends on your needs. If you expect to grow fast and unpredictably, or have extreme burst workloads (at my company it fluctuates between requiring ~10 cpus to ~50,000, and between 0 GPUs and dozens) or if you need several complex types of services and no people at hand who can manage them, it can be way cheaper. If you just need a few servers, a tape backup and a database, actual hardware has always been cheaper.
It depends on the workload. Some workloads do well on other people’s computers, some are better on your own computers. One size does not fit all.
deleted by creator
Yeah luckily Amazon is good about mess-ups that are one-time like this. Was the cost because you were pushing to, or retrieving from Glacier?
Deleting from. We move logstores and I added an ageout policy for anything over 1 day, to “easily” empty a bucket overnight. I forgot that I had been cycling stuff to glacier after 6 months, and there were 3 years of logs in there.
was hanging out with friends getting some drinks, we decided to walk through our old campus.
there was a roof I always used to climb up while in college to chill on, so I did that.
after finished, while hopped up on liquid courage, I decided to jump down.
did so and shattered my heel.
spent the entire summer immobile and required a surgery that ended up costing me about $5k out of pocket.
have mostly recovered now, but it’s still not as good as the other foot, and I know it’s going to hurt like hell when i’m old.
don’t be like me, don’t do stupid shit while drunk.
I was about to joke about the fact that you have to pay for health, but then I saw that you’ve never recovered. I’m very sorry, buddy.
It’s not horrible, I can run and hike right now, just not at 100%.
what are you gonna do though
Alcohol. The cause of, and solution to, all of life’s problems.
-Homer J Simpson
Did something somewhat similar. Got drunk after final exams. We decided we were special agents and were rolling over car hoods/bonnets (like they do in chases). There was a van (think A-Team), and since it doesn’t have much of a hood, I rolled off the roof instead.
Pretty sure I cracked or broke a rib (there was literally a loud pop). Couldn’t sit up, cough, sneeze, or breathe deeply for about 2 weeks without intense pain. I never went to doctor though because I thought, “well, it’s a rib, what are they gonna do? Put a cast around my whole body?”
Now I have discomfort every day, and pain between 2-3/10 after any sports. Tough when your youth idiocy catches up with you! Felt so invincible back then.
I’ll drink to that
If you have a four-year scholarship, for God’s sake, make sure you graduate in four years!
Or, as I did, don’t drop a class mid-term because it’s not going well and end up sliding into part-time status. Poof, scholarship gone. I woulda been better off taking the F.
Never let the car run out of gas. I was on the highway and the destination gas station was in sight. Well, even after putting more gas in from a Jerry can it wouldn’t start because debris clogged the fuel filter. Getting it towed + repaired was like $1000 when I could have just stopped at a gas station earlier.
Before buying your fitst home:
- bring someone with more experience than you to have a look at it, maybe even a professional
- scout out the area (on foot) during the day, evening and night
- visit local businesses like cafés, restaurants, bakeries etc.
- look at statistics like crime and air quality
- have a talk with the neighbors, get a sense of the community if you can, otherwise just observe while taking walks
- if applicable, call the home owner’s representative (or whatever the equivalent is where you live), ask them about the home, neighborhood, community, expenses, plans for the future etc.
- have a set budget of how much you want to spend on it before you move in, don’t overstep that amount
Nowadays if you do all those steps someone else will have bought the house before you’re done
That’s basically why you need an outstanding real estate agent.
Look at mister fancy pants who can buy a home.
That starting the work is half the work. I wasted a lot of time procrastinating, it took me shamefully long to realize that if I could just start an activity for 5 minutes, taking it to completion is then relatively easy
Not all landscapers can “landscape”. Hired a guy to build a pad for a shed which included a small retaining wall. The guy doesn’t own a level, and the end result is visibly not level. I showed him with my laser level what was going on, and he didn’t believe me. He started adding MORE material to the high spot.
He was aggressive about needing to be paid. Very aggressive. I paid him since he knows where we live. Unless we sue him and win, we’re out $4800, and to have it done correctly (with a fancier wall) will be $6500.
TLDR: Don’t hire a lawn service company to build anything.
That sucks, I’m sorry that happened. But landscapers are not concrete people. I will say that any of either profession I’ve dealt with were aggressive about payment. I wouldn’t be surprised if the guy tried to give you a change order for additional money?
It didn’t need any concrete, I hired him to do a stone wall. I didn’t want anything fancy, and had the whole thing been level I would have been happy enough with it.
thats when you take a fraction of the money you would have paid him, and buy security cameras for your house. High quality security cameras, with night vision.
Then contact a lawyer for damages to undo what he did.
I have a good 4M camera covering the entire front of our house already. Driveway, front yard, front door, etc. We’ll see if I take him to court. It’s really not worth the effort and time at this point, but it would be fun to waste his time even if I lose.
boy I wish I was rich enough to throw away thousands of dollars with a shrug and “To much effort to deal with it”
Damn bunch cynical people saying don’t get married. Maybe don’t get married to someone unless you’re sure, and get a prenuptial. There are advantages, legal and financially, of being married.
Everyone tends to extrapolate from their own experiences. My wife and I got married about a month after we met, for complicated reasons. We’ve been married for just over 20 years now. Mostly very happily! I don’t recommend our path to anyone, but the fact is that you just never know.
I wanted a newer car, so I rolled my existing auto loan into the newer vehicles loan. So easy right?
I was upside down on it for years and years. It’s so disheartening to drive a vehicle that’s falling apart and stranding you everywhere but still owe $10k on it. It was an awful decision that took years of pain but that was my lesson on buying things I can afford.
Art school isn’t worth it, period. I got a far better art education through my local community college by far, from instructors who weren’t incrediblely stuck up and full of themselves.
That was an 80k expense that I’m still paying off almost 20 years later, and I didn’t even finish my degree.
I went back to get my AS at a CC and took some art classes there. 10/10, far better instruction for a fraction of the price.
deleted by creator
Woman I liked borrowed money from me to buy a motorcycle, and in a drunk haze I told her to keep the bike as a valentine’s present.
Ooo 😮
Don’t invest in crypto.
Not to buy a game’s merchandise from the other side of the world (shipping price was around the same price if not more expensive than the product itself)
All my friends know: The moment I get a zombie apocalypse or similar confirmed, I’m ransacking and then burning down the customs building. They’re criminals and I want them to die first with the fall of society
I was in vacation in Japan. We ended our trip in Tokyo because my partner and I are into gaming and I knew we’d buy stuff. One thing we bought was bigger and a bit of a hassle to pack. I wondered how much it would have been if I had bought it online and shipped it. Turns out I lot of the newer stuff we bought we could have bought from Amazon Japan and shipped directly home for a reasonable rate (probably less than the cost of the overpriced duffle bag we bought).
Not me personally, but one of my career mentor’s friend’s took down the entirety of Google Ads as an intern for like 10 minutes. Apparently it was a multi-million dollar mistake, but they fixed the issue so it couldn’t happen again and all was well afterward.
In my first couple months, I broke Amazon so that no-one in Europe could buy video for a few hours. On a Friday, right before going on a week’s vacation.
The way that the ensuing investigation and response was carried out - 100% blame-free, and focused on “how did these tools let him down? How can we make sure no-one ever makes that same mistake again?” - gave me a career-long interest in Software Resiliency and Incident Management.
Junior dev: “I fucked up bad, I’m so fired”
Senior dev: " I have 3 production outages named after me lol"
Source: https://twitter.com/CarlaNotarobot/status/1481458190722207747
Yep. And every time there’s a thread about an Internet service having an outage, there’s some kid saying “oh, someone’s getting so fired for this one!”
Yeah, the competent business folks know that if you fire people for outages, you lose everyone who even stands a chance of preventing outages. And you tell the rest of your staff to hide problems. Businesses that do that kind of thing tend to end up with a valuation in the single digits.
Not always.
Sometimes the internet service outage is due to a car taking out a green box or a pole.
If they fire the backhoe driver, then backhoe drivers will never learn.
Oh yeah, That too. Plus trenchers, and hole boring for like fences or billboards.
If an intern (or damn near any employee) can be in a position to single handedly take down that scale of system it’s not the intern that should be fired - it’s the architect that baked that kind of weakness in the first place.
You’re not a real SRE until you’ve caused at least a $100K outage. You’re not a good SRE until you’ve fixed it so nobody can ever make that particular one again.
deleted by creator
