VOIPo down?

[Russell]

My conversation was cut-off. Both phone lines on the RT31P2 are dark. Going to the control panel I see no devices registered for me. Anyone else having issues? Tried rebooting the RT31P2 to no avail.

[sal]

I am fine here. No problems at all.

[Russell]

I plugged in my PAP2T and it registered fine. The RT31P2 is still functioning fine as a router but phone lines dead - and it dropped my call.

[Russell]

And now the RT31P2 lines have registered.

[Russell]

So, I've switched back to the RT31P2 and unplugged the PAP2T. The reference number for the dropped call was a53d192b-c74b515b@192.168.2.41 - I'd love to know what happened.

[burris]

No problems here.

[Xponder1]

Just looked and both lights are out on the phone ports of my PAP2T

Back up again as of 2:31AM

[usa2k]

All green from the early post until now ...

[VOIPoTim]

We were just told that some emergency network mainteane at Softlayer required a few routers to be rebooted in all their DCs so this is likely why a few of you guys saw an issue for a few minutes.

Here is the update they gave us. We have nothing in Seattle which is the only one still pending.

Date: 11-30-2009 Start time (CDT): 12:30 Services Affected: Public Network Device: All FCR Routers Location: All Duration: Times Listed Below Description: ==================================================
PROGRESS UPDATE

This maintenance has started. The following router(s) are either in progress or completed.

FCR01.WDC01 - COMPLETED
FCR01.DAL01 - COMPLETED
FCR02.DAL01 - COMPLETED
FCR03.DAL01 - COMPLETED
FCR04.DAL01 - COMPLETED
FCR01.SEA01 - IN PROGRESS

==================================================

Emergency Data Center Maintenance [11/30/2009]
Date: Monday, November 30, 2009 (11/30/2009)
Services affected: Public Network
Devices affected: FCR01.DAL01, FCR02.DAL01, FCR03.DAL01, FCR04.DAL01, FCR01.WDC01, FCR01.SEA01

==================================================

The following notification is to inform you of an EMERGENCY MAINTENANCE event on the SoftLayer Technologies IP Network. Due to the severity, the time frame between this notice and the maintenance window has been shortened.

Event Times:

12:00 CST / 01:00 EST - FCR01.WDC01
12:30 CST - FCR01.DAL01
01:00 CST - FCR02.DAL01
01:30 CST - FCR03.DAL01
02:00 CST - FCR04.DAL01
02:30 CST / 12:30 PST - FCR01.SEA01


Summary:
SoftLayer Technologies in conjunction with Cisco Systems have identified a service impacting bug in the current IOS version running on this router. A vendor recommended upgrade has been provided as a resolution and needs to be implemented immediately.


Original Issue Reference:
With the introduction of 4-byte ASNs (Autonomous System Numbers), a newly devised method of injecting a specifically crafted prefix into the global route table allows for the BGP process to be reset on older IOS code.

http://www.cisco.com/warp/public/707/cisco-sa-20090729-bgp.shtml
https://forums.softlayer.com/showthread.php?t=4585


Customer Impact:
During this maintenance, customers will notice a complete loss of connectivity to their servers on the frontend network (public network). Backend network (private network) connectivity will NOT be impacted during this maintenance. While the upgrade duration is scheduled for 1 hour, we only expect between 15-20 minutes of downtime as the routers reload and fully converge. Again, this will NOT impact the backend network (private network) for customer servers.

A notice will also be posted in the portal prior to the maintenance, during the maintenance informing you of the progress, and after with completion details.

==================================================

If you have any problems with your connection to your services after this maintenance window, or if you have any questions regarding the maintenance at any point, please call us at 214-442-0603 or open a ticket in the customer portal.

We appreciate your patience during this maintenance and welcome any feedback you may have.

Sincerely,
SoftLayer Network Team
https://status.softlayer.com (https://status.softlayer.com/)

[bwyatt]

Just a note - for whatever value it may be... three voipo devices here... two were on and working fine this morning... one had lost connection with both lines. I rebooted and it reconnected. Time: 6:25am EST

[Russell]

My conversation being cutoff does not seem to have occurred during the listed times - the time it occurred was a few minutes before my original post around 8:30 pm EST. As previously mentioned, the reference number for the dropped call was a53d192b-c74b515b@192.168.2.41 - I'd still love to know what happened.

[VOIPoTim]

My conversation being cutoff does not seem to have occurred during the listed times - the time it occurred was a few minutes before my original post around 8:30 pm EST. As previously mentioned, the reference number for the dropped call was a53d192b-c74b515b@192.168.2.41 - I'd still love to know what happened.

Send in a ticket if you haven't already and an engineer will be able to look it up for you.

[Russell]

Send in a ticket if you haven't already and an engineer will be able to look it up for you.
Done. BTX-428509 created. Mentioned that all is well so it's NOT urgent.

[Russell]

[deleted text]