Resellers: Testing Michigan Datacenter

[VOIPoTim]

Hey Guys,

We're doing some casual testing of a few new datacenters that we are considering adding into the mix so that there is an alternative option outside of our primary datacenter available for you guys to use if you prefer it for some reason.

Right now, we're testing a facility in Michigan. If you'd like to try it out, you can connect to sip7.voipwelcome.com instead of sip.voipwelcome.com. All resold accounts can register to it (and all direct VOIPo accounts with BYOD enabled).

Please let me know what your connectivity is like to that facility and if you feel it's a better or worse connection for you. Traceroutes would be very helpful with feedback.

Our Dallas datacenter is owned by IBM now and is rock solid with excellent connectivity for most people, but one big difference with it and some of the others we're evaluating is that it has a lot of direct ISP peering in its routing so if a user is using a major ISP like Cox or Comcast the traffic is likely being routed directly to it. Other facilities we're looking at adding as an alternative don't have as much peering so they may have better routing for customers not on major ISPs or for certain geographical areas.

Current Facility: Softlayer - http://www.softlayer.com/about/network/carriers
sip.voipwelcome.com

Possible Addition/Alternative: Liquidweb - http://www.liquidweb.com/datacenter/network.html
sip7.voipwelcome.com

Again, this is just something we're testing. We are extremely happy with our current facility but have a few datacenters we're considering adding to the mix and this one in Michigan is currently our top choice so we'd like to get some testing. We have one reseller that seems to have a poor connection to our Dallas facility for some reason so we're re-exploring adding some diversity to the SIP server mix.

If anyone wants to test, please let me know how your connection (and post traceroutes if possible to compare). We'd like to get as much "real-world" data as possible.

[uf_shane]

ISP = Suddenlink


Tracing route to sip7.voipwelcome.com [72.52.231.45]
over a maximum of 30 hops:


1 1 ms <1 ms <1 ms CISCO18933 [192.168.19.4]
2 * * * Request timed out.
3 9 ms 15 ms 16 ms 173-219-249-152-link.sta.suddenlink.net [173.219.*.*]
4 9 ms 10 ms 9 ms 173-219-249-148-link.sta.suddenlink.net [173.219.*.*]
5 37 ms 37 ms 37 ms 173-219-249-47-link.sta.suddenlink.net [173.219.249.47]
6 40 ms 38 ms 41 ms 173-219-249-250-link.sta.suddenlink.net [173.219.249.250]
7 53 ms 41 ms 38 ms cdm-66-76-30-133.tylrtx.suddenlink.net [66.76.30.133]
8 67 ms 62 ms 64 ms xe-7-2-1.edge5.Dallas3.Level3.net [4.71.122.17]
9 59 ms 61 ms 66 ms vlan70.csw2.Dallas1.Level3.net [4.69.145.126]
10 59 ms 59 ms 75 ms ae-71-71.ebr1.Dallas1.Level3.net [4.69.151.138]
11 61 ms 65 ms 64 ms ae-14-14.ebr2.Chicago2.Level3.net [4.69.151.117]
12 63 ms 61 ms 65 ms ae-204-3604.edge2.Chicago2.Level3.net [4.69.158.61]
13 61 ms 61 ms 68 ms CWIE-LLC.edge2.Chicago2.Level3.net [4.59.29.82]
14 68 ms 70 ms 81 ms lw-core4-te91.rtr.liquidweb.com [209.59.157.206]
15 66 ms 69 ms 70 ms lw-dc3-dist15.rtr.liquidweb.com [69.167.128.201]
16 71 ms 73 ms 83 ms 72.52.231.45


Trace complete.


Tracing route to sip.voipwelcome.com [67.228.182.2]
over a maximum of 30 hops:


1 <1 ms <1 ms <1 ms CISCO18933 [192.168.19.4]
2 * * * Request timed out.
3 10 ms 10 ms 10 ms 173-219-249-152-link.sta.suddenlink.net [173.219.*.*]
4 11 ms 11 ms 12 ms 173-219-249-148-link.sta.suddenlink.net [173.219.*.*]
5 44 ms 39 ms 38 ms 173-219-249-47-link.sta.suddenlink.net [173.219.249.47]
6 38 ms 40 ms 39 ms 173-219-249-250-link.sta.suddenlink.net [173.219.249.250]
7 42 ms 42 ms 41 ms cdm-66-76-30-133.tylrtx.suddenlink.net [66.76.30.133]
8 41 ms 41 ms 40 ms 173-219-254-169-link.tex.sta.suddenlink.net [173.219.254.169]
9 44 ms 44 ms 44 ms te1-5.bbr01.eq01.dal01.networklayer.com [206.223.118.24]
10 46 ms 43 ms 53 ms ae0.dar02.sr01.dal01.networklayer.com [173.192.18.253]
11 42 ms 44 ms 43 ms po2.fcr04.sr05.dal01.networklayer.com [66.228.118.218]
12 49 ms 58 ms 57 ms 67.228.182.2-static.reverse.softlayer.com [67.228.182.2]


Trace complete.

[chevyman]

here is from seattle

Tracing route to sip.voipwelcome.com [67.228.182.2]
over a maximum of 30 hops:

1 <1 ms <1 ms 1 ms 192.168.0.10
2 17 ms 18 ms 17 ms tukw-dsl-gw65.tukw.qwest.net [63.231.10.65]
3 17 ms 17 ms 18 ms tukw-agw1.inet.qwest.net [71.217.186.1]
4 18 ms 18 ms 17 ms sea-brdr-02.inet.qwest.net [67.14.41.18]
5 18 ms 17 ms 20 ms sea-b1-link.telia.net [80.239.194.29]
6 36 ms 37 ms 36 ms sjo-bb1-link.telia.net [213.155.134.80]
7 36 ms 36 ms 61 ms xe-1-1-1.bbr02.eq01.sjc02.networklayer.com [213.248.102.222]
8 45 ms 44 ms 44 ms ae0.bbr02.cs01.lax01.networklayer.com [173.192.18.151]
9 44 ms 45 ms 44 ms ae7.bbr01.cs01.lax01.networklayer.com [173.192.18.166]
10 72 ms 72 ms 72 ms ae19.bbr01.eq01.dal03.networklayer.com [173.192.18.140]
11 70 ms 71 ms 71 ms ae0.dar02.sr01.dal01.networklayer.com [173.192.18.253]
12 71 ms 71 ms 71 ms po2.fcr04.sr05.dal01.networklayer.com [66.228.118.218]
13 71 ms 71 ms 71 ms 67.228.182.2-static.reverse.softlayer.com [67.228.182.2]

Trace complete.


Tracing route to sip7.voipwelcome.com [72.52.231.45]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms 192.168.0.10
2 18 ms 18 ms 19 ms tukw-dsl-gw65.tukw.qwest.net [63.231.10.65]
3 45 ms 17 ms 17 ms tukw-agw1.inet.qwest.net [71.217.186.1]
4 18 ms 18 ms 18 ms sea-brdr-02.inet.qwest.net [67.14.41.18]
5 64 ms 18 ms 19 ms 208.173.49.93
6 20 ms 19 ms 20 ms msr1-te-0-5-1-0.sea.savvis.net [206.28.98.174]
7 69 ms 69 ms 69 ms cr1-te-0-5-0-2.den.savvis.net [206.28.96.185]
8 75 ms 76 ms 75 ms cr1-ten-0-4-0-1.chd.savvis.net[204.70.192.134]
9 76 ms 74 ms 75 ms ber1-te-1-0-0.chicagoequinix.savvis.net[204.70.196.22]
10 74 ms 75 ms 74 ms 208.173.176.222
11 81 ms 81 ms 81 ms lw-dc2-core6-te9-2.rtr.liquidweb.com [209.59.157.226]
12 80 ms 81 ms 81 ms lw-dc3-dist15.rtr.liquidweb.com [69.167.128.205]
13 81 ms 81 ms 81 ms 72.52.231.45

[redoneusa]

Tracing route to sip7.voipwelcome.com [72.52.231.45]
over a maximum of 30 hops:

1 3 ms 1 ms 1 ms dlinkrouter.gha.chartermi.net [192.168.0.1]
2 10 ms 8 ms 14 ms 10.152.80.1
3 8 ms 9 ms 9 ms dtr04trcymi-tge-0-1-0-14.trcy.mi.charter.com [96
.34.40.42]
4 42 ms 25 ms 17 ms crr02trcymi-tge-0-7-0-4.trcy.mi.charter.com [96.
34.33.12]
5 21 ms 21 ms 28 ms crr02aldlmi-tge-0-3-0-0.aldl.mi.charter.com [96.
34.38.88]
6 19 ms 18 ms 18 ms bbr01aldlmi-bue-2.aldl.mi.charter.com [96.34.2.1
0]
7 82 ms 22 ms 22 ms bbr01chcgil-bue-4.chcg.il.charter.com [96.34.0.9
9]
8 22 ms 29 ms 37 ms prr02chcgil-bue-3.chcg.il.charter.com [96.34.3.1
14]
9 24 ms 39 ms 21 ms 23-30-206-169-static.hfc.comcastbusiness.net [23
.30.206.169]
10 23 ms 26 ms 22 ms he-3-3-0-0-cr01.350ecermak.il.ibone.comcast.net
[68.86.83.161]
11 18 ms 20 ms 27 ms be-14-pe03.350ecermak.il.ibone.comcast.net [68.8
6.82.210]
12 23 ms 24 ms 22 ms 66.208.216.86
13 34 ms 26 ms 39 ms lw-dc2-core6-te9-2.rtr.liquidweb.com [209.59.157
.226]
14 25 ms 30 ms 25 ms lw-dc3-dist15.rtr.liquidweb.com [69.167.128.205]

15 32 ms 32 ms 24 ms 72.52.231.45

Trace complete.

[GreenLantern]

I don't know why sip7 is so much better, but I've had to move a lot of accounts over to it again this week.

Original server may have fewer hops and faster pings, but it is just flaky and moving customers to sip7 has been a lifesaver.

[jsiepka]

How is the testing on SIP7 going? After a rocky startup we have had some good luck so I don't want to move my accounts over as things have been stable. Well at least for the moment. :)

So if I understand correctly the advantage to using SIP7 would be for clients that use Comcast. 90% of my clients are using Comcast business class internet services within the Chicago area, and Northern Indiana.

[jsiepka]

================================================== ========================
VisualRoute trace to sip7.voipwelcome.com
================================================== ========================


This trace was started on Jul 11, 2014 10:48:36 PM.
The host 'sip7.voipwelcome.com' has been found, and is reachable in 13 hops. Also, it responded to HTTP requests on port 80 (it is running server Apache/2.2.16 (Debian), which responded in 71ms). The TTL value of packets received from it is 53.In general this route offers a good throughput, with hops responding on average within 16ms. However, hop 9 in network 'Comcast Cable Communications Holdings, Inc' is noticeably slower than others. This slow speed could perhaps be accounted for by the fact that the remote network is located far away. The DNS lookup was completed almost instantaneously (less than 2ms - this may be the result of caching).

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| Hop | %Loss | IP Address | Node Name | Location | Tzone | ms | Graph | Network |
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| 0 | | 10.2.1.13 | MRCCTVMOBILE2.tech151.com | | | 0 | x | [Local Network] |
| 1 | | 10.2.0.200 | tech151.com | | | 2 | x | [Local Network] |
| 2 | | 50.194.101.46 | 50-194-101-46-static.hfc.comcastbusiness.net | Joliet, IL, USA | -06:00 | 1 | x | TECH JOE SIEPKA |
| 3 | | 50.148.100.1 | c-50-148-100-1.hsd1.il.comcast.net | Springfield, IL, USA | -06:00 | 9 | x | Comcast Cable Communications Holdings, Inc |
| 4 | | 162.151.34.17 | te-0-3-0-10-sur04.romeoville.il.chicago.comcast.net | Chicago, IL, USA | -06:00 | 10 | x | Early registration addresses |
| 5 | | 69.139.203.189 | te-2-10-0-9-ar01.area4.il.chicago.comcast.net | Chicago, IL, USA | -06:00 | 13 | x | Comcast Cable Communications, Inc. |
| 6 | | 68.86.94.125 | he-3-14-0-0-cr01.350ecermak.il.ibone.comcast.net | Mt Laurel, usa | | 14 | x | Comcast Cable Communications, Inc. |
| 7 | | 68.86.85.138 | he-0-13-0-0-pe03.350ecermak.il.ibone.comcast.net | Mt Laurel, usa | | 13 | x | Comcast Cable Communications, Inc. |
| 8 | | 50.242.150.130 | 50-242-150-130-static.hfc.comcastbusiness.net | (Australia) | | 72 | -x-------- | - |
| 9 | | 209.59.157.244 | lw-core5.rtr.liquidweb.com | | | 18 | x | - |
| 10 | | 69.167.128.201 | lw-dc3-dist15.rtr.liquidweb.com | | | 20 | x | - |
| 11 | | 72.52.231.45 | sip7.voipwelcome.com | Lansing, MI, USA | | 19 | x | Liquid Web, Inc. |
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

[VOIPoTim]

How is the testing on SIP7 going? After a rocky startup we have had some good luck so I don't want to move my accounts over as things have been stable. Well at least for the moment. :)

So if I understand correctly the advantage to using SIP7 would be for clients that use Comcast. 90% of my clients are using Comcast business class internet services within the Chicago area, and Northern Indiana.

SIP7 is still available for resellers/BYOD users.

It's not necessarily better for Comcast...it's just in a different datacenter so for some people it may have a better connection.

SIP is in Dallas. SIP7 is in Michigan.

You can check ping times and do a traceroute to sip7 vs sip to see which one looks better for your connection.

[searchformission]

Joe, my building is using Comcast business class internet, in a residential area in the Hyde Park area of Chicago.
I recently signed up with a reseller that uses the SIP7 datacenter. My Obi110 box has no problem connecting. Our soft-phone setup has been working well also.
My parent uses VOIPo's service out in California for almost 4 years. I tested their BYOD setup here at our building, and the SIP-datacenter in Dallas came through clear as well.

Tim, I am glad you guys are spreading out to other data centers. I read something recently that Softlayer/IBM is expanding their data centers like crazy. Softlayer and IBM knows how to run their business.
It's good that VOIPo add some diversity for customers, as not everyone enjoy the same Internet setup/experience as others. If it's not too expensive to add, consider adding one in Toronto or somewhere near for your Canadian clients.

A quick question:
Can a client (whether with VOIPo or through one of the VOIPo's reseller) choose which SIP server works for them, simply by changing sip.voipwelcome.com --> sip7.voipwelcome.com (and vice versa)?

[jsiepka]

Here is a trace and we are seeing better results using SIP7


Target Name: sip.voipwelcome.com
IP: 67.228.182.2
Date/Time: 8/2/2014 1:13:06 AM

1 1 ms 4 ms tech151.com [10.2.0.200]
2 1 ms 1 ms 50-194-101-46-static.hfc.comcastbusiness.net [50.194.101.46]
3 9 ms 10 ms c-50-148-100-1.hsd1.il.comcast.net [50.148.100.1]
4 7 ms 10 ms te-0-3-0-10-sur04.romeoville.il.chicago.comcast.net [162.151.34.17]
5 N/A 12 ms 162-151-44-181-static.hfc.comcastbusiness.net [162.151.44.181]
6 14 ms 15 ms pos-1-10-0-0-cr01.sanjose.ca.ibone.comcast.net [68.86.88.1]
7 15 ms 11 ms pos-1-2-0-0-pe01.350ecermak.il.ibone.comcast.net [68.86.86.78]
8 12 ms 14 ms xe-0/0/0.bbr01.eq01.chi01.networklayer.com [75.149.228.98]
9 45 ms 40 ms ae20.bbr01.eq01.dal03.networklayer.com [173.192.18.136]
10 43 ms 42 ms ae0.dar01.sr01.dal01.networklayer.com [173.192.18.211]
11 59 ms 66 ms po1.fcr04.sr05.dal01.networklayer.com [66.228.118.214]
12 41 ms 50 ms 67.228.182.2-static.reverse.softlayer.com [67.228.182.2]

Ping statistics for sip.voipwelcome.com
Packets: Sent = 2, Received = 2, Lost = 0 (0.0%)
Round Trip Times: Minimum = 41ms, Maximum = 50ms, Average = 45ms


Target Name: sip7.voipwelcome.com
IP: 72.52.231.45
Date/Time: 8/2/2014 1:13:59 AM

1 3 ms tech151.com [10.2.0.200]
2 2 ms 50-194-101-46-static.hfc.comcastbusiness.net [50.194.101.46]
3 7 ms c-50-148-100-1.hsd1.il.comcast.net [50.148.100.1]
4 10 ms te-0-3-0-10-sur04.romeoville.il.chicago.comcast.net [162.151.34.17]
5 15 ms te-2-10-0-12-ar01.area4.il.chicago.comcast.net [68.86.187.169]
6 13 ms he-3-14-0-0-cr01.350ecermak.il.ibone.comcast.net [68.86.94.125]
7 9 ms he-0-13-0-0-pe03.350ecermak.il.ibone.comcast.net [68.86.85.138]
8 166 ms [50.242.150.126]
9 15 ms lw-core5.rtr.liquidweb.com [209.59.157.244]
10 18 ms lw-dc3-dist15.rtr.liquidweb.com [69.167.128.201]
11 15 ms [72.52.231.45]

Ping statistics for sip7.voipwelcome.com
Packets: Sent = 1, Received = 1, Lost = 0 (0.0%)
Round Trip Times: Minimum = 15ms, Maximum = 15ms, Average = 15ms

[GreenLantern]

Michigan data center (SIP7) was toast today. Just illustrates how important it is to have more than one data center.

I would really like to see a 3rd or even a 4th regional server... such as North, South, East and West.

Also, if resellers could map their own domains, we could quickly change DNS in the event of an outage... so our customers could be moved to another server all at once... then moved back to their regional server after the outage. Or voipo could do that on their end.

[chevyman]

So sip7 doesn't go to Michigan any more?

[GreenLantern]

No, it is being retired.

[VOIPoTim]

So sip7 doesn't go to Michigan any more?

Correct. We had too many outages and issues with it so we redirected it back to Dallas. It did not prove to be as stable as our Dallas facility.

[racerdude]

When can we expect to see the new West Coast server you mentioned previously?

[VOIPoTim]

When can we expect to see the new West Coast server you mentioned previously?

No ETA at this time. In the testing we've done, we've not found anything as stable as our Dallas setup.

[racerdude]

If you've eliminated the Michigan server that means you have all your eggs in one basket in Dallas. Don't you want to have redundancy at multiple sites in case Dallas goes down?

[VOIPoTim]

If you've eliminated the Michigan server that means you have all your eggs in one basket in Dallas. Don't you want to have redundancy at multiple sites in case Dallas goes down?

Not quite. We're already in 3 different physical facilities in Dallas...our datacenter partner Softlayer (owned by IBM) has multiple physical datacenters there an we use a mix of them for redundancy.

Here's some more info:

http://www.softlayer.com/data-centers

http://www.softlayer.com/network

One of the nice things that makes it perform so well for us is that they have direct peering with almost all the major consumer ISPs that our customers use for more direct connections vs routing through multiple backbone providers in addition to pretty robust connectivity. They also have very solid private networks (isolated from the public) that connect the DCs for us to do backups and behind the scenes stuff on either the public network or the completely different private network for security, redundancy, etc. Dallas is also central and has pretty solid connectivity to the entire country.

We're still open to adding more options in the future (Softlayer is definitely not cheap...we pay them 5 figures monthly) so it's definitely a possibility. We just haven't had good experiences with trials with other providers.

[GreenLantern]

well, I hate to point this out, but service at the TX server went down this morning.

my customers who were still on MI server were not affected and worked great.

[chpalmer]

Both sip.voipwelcome.com and sip7.voipwelcome.com resolve to a Dallas location now. Were your people using an IP address directly? Otherwise...

Tracing route to sip.voipwelcome.com [67.228.182.2]

Tracing route to sip7.voipwelcome.com [67.228.182.2]

9 56 ms 52 ms 55 ms po2.fcr04.sr05.dal01.networklayer.com [66.228.118.218]
10 52 ms 53 ms 54 ms 67.228.182.2-static.reverse.softlayer.com [67.228.182.2]

[GreenLantern]

Yes, I have a handful of clients hard coded to the Michigan location. Been busy moving them off all month, but a few were still there today. They were unaffected by the outage.