Munu Emergency Blog

Aug. 20, 2011 -- DNS Problem

2011-08-20T15:03:00.000-07:00

Looks like it's not just The Jawa Report and Ace of Spades HQ, but all of mu.nu. It's some kind of DNS error.

We'll be back when we can.

Fidel Castro is dead again

2007-08-25T17:19:00.000-07:00

Well, since no one is using this blog for anything else nowadays, might as well pimp something for posterity:

Fidel Castro is dead. What, Castro dead you say? No, but when Fidel Castro dies now you'll know where to look.

Moo-noo go booom booom

2006-11-29T11:07:00.000-08:00

What up with mu.nu?

MuNu down...again

2006-07-21T15:09:00.000-07:00

This is the third time in four days that MuNu has been down. We should try to use the emergency blog more effectively to get the word out on what is going on.

Of course, none of it matters if Pixy is soundly asleep right now. :)

Cheers!

P.S. I'm still waiting on those instructions from Eric Grumbles old log about how to generate Technorati tags (or del.icio.us tags or Flickr tags...it's all the same) from MT keywords. There are plenty of ways described on the web, but they all require this plugin or that to be installed, and I'm not quite sure what MuNu has in its library of plugins. Apparently Eric hit upon the formula that worked for this installation of MT.

P.P.S Does this matter with the Minx migration? Will anything I do today still work in a few weeks?

Grrrrrrrrrr!!!!!!

2006-06-02T20:58:00.000-07:00

Kei has crashed again.

Yuri is still getting DoSed.

Waiting on a reboot of Kei now.

AAARGH!!!

2006-06-02T18:50:00.000-07:00

Yuri is getting hit by a denial-of-service attack.

Kei seems to have suffered a disk failure. We will be rebooting, and we'll see how that goes.

What fun.

Yuri Down - DOS Attack?

2006-06-02T16:41:00.000-07:00

One of our servers has been taken out by what looks like a denial-of-service attack. We are investigating the cause and will do whatever we can to bring it back on line.

MuNu down today?

2006-02-20T18:13:00.000-08:00

Can't get to the maintenance screen at blog.mu.nu. Something up (or down)?

Network Maintenance

2006-01-29T00:16:00.000-08:00

Sorry everyone. Our hosting provider is doing some urgent maintenance on their network, and it's taking a little bit longer than expected. Munu should be back very soon.

What's the status?

2006-01-19T14:14:00.000-08:00

mu.nu seems to be having problems today. Any idea what the prognosis is?

Network Problems

2005-12-12T23:25:00.000-08:00

There seem to be some major routing issues with parts of the internet at the moment, and they may be preventing you from reaching mu.nu. I can't currently reach either mu.nu or the hosting company (DedicatedNOW), but the indications are that the problems are occuring long before the traffic reaches DedicatedNOW's network. In fact, the packets are travelling from Australia to America, coming back again, and then disappearing entirely.

I'm hoping this will be fixed soon.

Oh, and WebHostingTalk, the site I usually turn to for information in such circumstances, is down right now for a software upgrade. Yay.

Update: Now traffic headed for mu.nu isn't even leaving Australia.

Update: Coming back online now. Telstra have re-routed their link through Chicago. Exetel still seem to be working on it. Chilli are going via LA and New York. Can't check iiNet right now. If you still can't reach the server, it means that your ISP hasn't found a way around the problem yet.

2005-11-11T23:38:00.000-08:00

Mu.nu is currently shut down to move to the new servers. While we are shut down, you will not be able to post or comment on any of the mu.nu sites.

Do not panic!

Make sure you have the following supplies at hand:

12 sheets of 1/4 inch plywood, at least 3' by 4'
6 gallons of water
At least one dozen batteries in each of the common sizes
String
Sticky tape
Wire
Tinned food such as beans; at least three tins per person per day

Use the sticky tape and string to construct a cubby house (or "fort") out of the plywood. The tins of beans can be used to defend your fort. Either throw them at people who approach, or, in close combat, ding them on the head.

For entertainment you can try touching the batteries to your tongue. Wire them in sequence and parallel and see what difference it makes!

For extra value, spill the water on the carpet and blame it on the dog.

Not Drowning, Moving!

2005-11-11T21:52:00.000-08:00

Hi everyone. mu.nu isn't down, it's moving to a new location. We will be back in a few hours, bigger and more annoying than ever before.

So take a break and break open a frosty can of Duff Beer. Beer. It's good for you.

Duff Beer is a product of MuNu Global Manufacturing and World Domination, Inc. Beer is not good for you. At least, our beer isn't.

MuNu is down?

2005-10-19T12:36:00.000-07:00

Or, at the very least, I can't get to anything.

Another flaky day?

2005-09-21T09:25:00.000-07:00

Wednesday, 21 Sept: Noticing a lot of "Server Error" pages when I try to access my blog...

Crap, Again

2005-09-19T22:54:00.000-07:00

The server is down. Again. Logged a ticket with the hosting company.

:(

Fixed! Don't know what the problem was just yet, but at least it's working.

Actually, I think they might have the Critical Need Detector wired up backwards. I did a full backup of everything on the server yesterday. Just as I had done the day before the Big Crash.

Hmm.

Pinin' For The Fjords

2005-09-03T03:13:00.000-07:00

This is a planned outage. Do not adjust your bookmarks! We will be back before you know it!

Fsck!

2005-08-27T05:12:00.000-07:00

The server has decided that it has to do a full filesystem check right now. This normally does this once every six months after a reboot. And today is the lucky day.

This does mean that the server isn't broken, but there are something like a million files on there (thanks to the busy Munuvians), so it will take a little while to run. Probably on the order of an hour, and we're half an hour into that already.

Oh yeah: A filesystem check on Unix (or Linux) is done by a program called fsck.

Update: Coming up now. Sorry about the delay.

Bugger

2005-08-27T04:54:00.000-07:00

That didn't work. The system will be back shortly, but we will have to schedule yet another outage to finish the work. :(

Dum De Dum

2005-08-27T04:18:00.000-07:00

The second disk drive is being replaced right now. We'll be back any moment... Any moment now...

The Blog Ate My Homework - I Swear...

2005-08-26T15:49:00.000-07:00

Funny thing happened on the way to publishing a post this morning. I hit publish, it published, but it didn't post. I dutifully refreshed the page & nothing. I rebuilt the entire site - nada. The entries are still on my editing page & saved but for some reason they just won't post to the site. I ended up posting the two entries at my back up site (publicola.blogspot.com)but I'm still puzzled.

Has this happened to any of ya'll? Ladies tell me do you understand?
And all my fellas do you feel my pain?

Planned Outrage: Saturday 27th August

2005-08-26T03:19:00.000-07:00

The server will be down for about an hour beginning 5AM CST while the other ~~hell demon~~ disk drive is ~~exorcised~~ replaced.

Hopefully that will fix all of our present problems, so we can get on with creating new and better problems.

Damn And Double Damn

2005-08-25T06:44:00.000-07:00

We are now getting error messages from the other disk drive. If you've ended up here, it probably means the server is down while we replace that disk as well. It shouldn't take so long this time, since everything is already backed up.

I'm going to try rebooting and see if that helps at all. If we can make it to the weekend I can schedule the second hard disk to be replaced so that there's a minimum of downtime, but if it gets really bad I'll have them replace it right away.

And As The Blog Sinks Slowly In The West

2005-08-24T18:52:00.000-07:00

This blog will now drift peacefully off to sleep, hopefully for a very long time.

Back In Business!

2005-08-24T11:26:00.000-07:00

We're up and running! Yay!

It might be a bit slow for the first few hours because the RAID array is rebuilding, but it is definitely working and nothing has been lost.

Thanks for your patience everyone! Now my bed is calling me. :)

We'll Be Back

2005-08-24T11:25:00.000-07:00

MuNu isn't dead, just pining for the fjords. When I'm done putting 10,000 volts through it, it will voom good as new.

There doesn't seem to be any way to sticky posts, so I kicked this into the middle of next week.

(If you were looking for something other than MuNu, such as a forum or a portal site, then it was hosted on the MuNu server and will be back along with everything else as soon as we can possibly manage it.)

Good News!

2005-08-24T11:17:00.000-07:00

The cheesy disk drive has been kicked to the kerb! I repeat, the cheesy disk drive has been kicked to the kerb!

The techies are testing things now, but it's looking good.

Eric Keeps Grumbling

2005-08-24T10:41:00.000-07:00

First off, Pixy, you have done an awesome job with MuNu. I know this is only temporary and you'll get us back up and running soon. In the meantime, I'll be grumbling on, as usual, at The Life, Liberty, Property Group blog until such time as Eric's Grumbles Before The Grave is back up and running on MuNu!

Okay. Nearly There...

2005-08-24T10:36:00.000-07:00

Everything is across onto the backup server, and all the main databases have been restored. A few databases for individual users wouldn't load (including one important one) but I can get most things running now. The databases are from yesterday, but I managed to copy all the individual files across, so we hardly lost anything.

But!

Tech support at the hosting company is replacing the failed disk right now. (Yay!) If that works, then we can just go back to our happy MuNu-ing. If it fails, I will bring MuNu up on the backup server and off we go.

Hang tight for another hour or so, kids, we're nearly there!

Titles? We Don't Need No ....

2005-08-24T06:04:00.000-07:00

Eh. Blogger. Can't really bitch too much about it, but I'll try anyway. :P Actually it's not that bad. It'd be nice to have an extended entry so I don't take up too much space bitching/not bitching about blogspot, but beggars can't be choosers.

I'll be honest: my knowledge of things PC & internet is mainly pre-jurassic. So when like most of you I tried to click on my site I was a bit perplexed. Hell, I figured the ATFU finally shut me down. Then I checked some other Munuvians & discovered that it as system wide. I still don't know exactly what went wrong, but then again I can't claim I knew exactly what goes right 99.995% of the time.

Confederate Yankee offers to post the URl of any Munuvians on his back up site. E-mail him at confederateyankee AT gmail DOT com. (Replace AT with @ & DOT with . respectively).

Basil is accepting trackbacks of Munuvian back up sites at http://www.basilsblog.net/2005/08/mu_nu_where_are.html

The lovely & talented Beth is offering to let Munuvians post on her site & she as well is accepting trackbacks. Click here for the relevant info.

Many thanks to all three of those fine young bloggers for lending a hand. Or more approriately Yay! (what the hell does that mean anyway??) Aftet the revolution their names will be on the list of Friends of the People.

& thanks to Pixy who is busting his ass trying to get things up & running again so we can not post for days at a time but still pull up our pages hourly in a vain attempt to get "inspired".

Finally a message to all our Munuvian brothers & sisters trapped without a convenient way to post:

The chair is against the wall. The chair is against the wall.
Babygirl think she cute. Babygirl think she cute.
Cadilacs can only swim through epsom salted water. Cadilacs can only swim through epsom salted water.

That is all.

Update: 08-24-05 07:09 MST

That is not all. I have received word from our...well let's just say I have recieved word. Words rather; there was more than one involved. Now the message:

The jaded eagle flies at midnight. The jaded eagle flies at midnight.

You know what to do. Carry on.

Damn And Blast

2005-08-24T05:26:00.000-07:00

The database restore failed somewhere near line 2.8 million of the 9.8 million line backup file.

Which is bad.

I've just cooked up a little Python program to split the backup file into individual databases so I can restore them one at a time. We'll see if that works a bit better.

Update: Okay, off and running again. That's the good side of dumping to an ASCII format - you can hack it up if you have to. (The bad side is that you sometimes have to.)

Sleep, That Wits The Knavelled Reave Of Care

2005-08-24T05:09:00.000-07:00

I'm going to see if I can grab a couple of hours of this elusive commodity while the database restore runs. Everything seems to be going well... Relatively speaking.

...

Ah crap, the restore fell over. :(

Meh

2005-08-24T04:49:00.000-07:00

Gave up on trying to dump the SQL databases again; it was going to take something like 20 hours. I've kicked off the restore from yesterday's backup, and that is going to take long enough by itself.

Fun fun fun...

Trying To Back Up MySQL

2005-08-24T03:41:00.000-07:00

It's sort of working. Not fast, though.

Timing Is Everything

2005-08-24T03:17:00.000-07:00

Our hosting company's tech support website seems to have gone offline. Nice.

Homeward Bound

2005-08-24T00:22:00.000-07:00

I'm heading off home now, so I'll be offline for an hour or so. Just in case something goes wrong...

Nevermind.

You Know...

2005-08-23T23:49:00.000-07:00

Blogger doesn't suck as much as it used to.

Compared to Movable Type on a server with a stuffed hard disk, for example.

Rsyncing Feeling

2005-08-23T23:47:00.000-07:00

Well, it's doing better than expected in that it hasn't died completely. Every so often it just freezes for a minute or three. But I do want to get all the stuff that was posted since the last backup, so no choice really.

Suckiness

2005-08-23T23:45:00.000-07:00

It takes forever just to update that little place-keeper post on munu, so further updates will be found here.

If that rotten disk would just drop dead, everything would be fine again. Sigh.

MuNu Done Blowed Up Real Good

2005-08-23T23:39:00.000-07:00

Hi munu fans.

The munu server has blowed up. The various problems we've been having with spam floods and denial-of-service attacks and getting Farked have been real, but the reason they were killing the server was an intermittent drive failure.

That failure is no longer intermittent.

Our last full offsite backup was yesterday, so this could have happened at a worse time. I'm working on recovering any updates from the last few hours, and after that I'll be bringing things up on our backup server. This may take a few hours, so please bear with us.

(Yes, we have mirrored disks. If one of them had simply failed outright, this wouldn't have been a problem. Instead, it's gone completely mad, generating spurious errors and generally screwing things up.)

Update 16:13 AEST: All user files and critical system files are on the backup server, and I'm attempting to bring it up to date with rsync. I've logged a request for the faulty drive to be replaced; at this point we don't have much to lose even if they get it wrong and pull out the good drive...

Minor update: The rsync is doing much better than I'd expected, so I'll let it run. After that, I'll try for a backup of MySQL so we'll be completely up to date.