Whatsapp facebook and linkedin down?

It's the insulation loonies that have taken down social media!

I don't believe they are capable of doing anything as useful as that!

So not all bad then :-)

Been looking at this. Watch out for this story. What has been done is very very comprehensive and deliberate I think. Looks like DNS is down as well as BGP routes to the servers.

The BGP takes some serious doing, and the coincidence rules out a fat finger

I do wonder if they have been hacked.

Michael Chare brought next idea :

They suggested it was something to do with a new configuration which went wrong. Sounds a bit lame to me, they should have been able to instantly revert back to the old configuration.

Sounds like they have in fact been hacked.

They use Twitter instead.

Actually any company who can break the links between their numeric host addresses and their friendly names seems to me to be inept in the extreme!

Maybe there is more to this than have been admitted?

Brian

Last Thursday, one of my pcs alerted me that the hosts file had been altered in an illegal way, would I like the security system to restore the last known good file? I allowed it it then downloaded a load of anti virus updates and did a scan, removing one so called potentially unwanted program. This is the sort of thing that occurs when somebody attempts to change the numeric addresses allied to the friendly host names so you go to bogus sites or click through a data mining site. I've only seen it on one machine connected via 02gifgaff to the internet, my Virgin connected machine has had no such weird goings on as it has security software in its router. It kind of makes you wonder though if there are people out there altering stuff on major systems of servers which could cause loads of chaos. I don't use Facebook or the other services Facebook own, though may have to use whatsapp at some time in the future merely due to the groups that people are now creating on it. Brian

Or they are just incompetent. They probably sacked the people who built it and those people installed a bomb so if they ever lost their jobs it would trigger a monumental mess costing them a lot of money. I've heard of that sort of thing being done before. Brian

For some reason the Messenger app disappeared from my mobile device, probably overnight. I don't remember uninstalling it. Perhaps it just slid down behind the sofa...

Reversing the changes is great if you've got an alternate method to get in to do so. If your mistake has cut off all your datacentres from the internet and locked all the doors, it's a bit more challenging ...

Harry Bloomfield Esq. snipped-for-privacy@harrym1byt.plus.com> wrote

Its not that simple with DNS changes.

Nope.

You know what they say about never attributing to malice what can be explained by incompetence?

Its seems one of the problems was that their control infrastructure also runs on the platform and infrastructure as their main customer facing services. So when the change they pushed out killed access to the platform, it not only knocked all the users off, it also locked out their own control infrastructure, to the point that some people could not even get into buildings!

It's somewhat akin to taking apart your boat engine, and then throwing the tool kit overboard.

Both, sort of. Someone broadcast BGP updates that effectively killed routing to theor IP block, even internally (FSVO internally). Whether that was finger trouble or malicious is yet to be determined.

Apparently it took a while to fix becuase they about about 100k routers and they didn't have OOB access to at least some of them.

That wasn't the problem. DNS worked for a while (because caching) but the actual IP block was inaccessible. Then the caches expired and the authoritative DNS servers were inaccessible.

Not if they couldn't get in to their internal systems to do that. From what I read all the secure areas stayed locked up, and they couldnt get in via any networks

We have all managed to misnconfigure a router or server so its no longer reachable over the network and needs a 'factory reset'.

If its also controlling physical access to itself, you are, frankly, f***ed.

We used to have dial up modems on the servers to allow us to remotely reboot them if we couldn't get to them over the internet

I think probably not. But we will see.

AIUI It was in fact BGP changes that simply removed the DNS servers from the known internet....

I tried a traceroute. No one had routed their block elsewhere. I got as far as my ISP and then that was it - there were no routes for their servers at all on the internet.

No. Its doubtful there are 100k backbone routers running BGP in the world.

These are ISP edge routers and they probably had no more than 4-5 at any given major server center. My impression is that they could access them but that wasn't the issue - the issue was how they were being updated by some database feeding them bad data.

All you need is a typo in the IP address and that's it. Gone

We always had a modem and, separately, ISDN to all routers.