Calling issues in 919 area code

[Russell]

I'm having trouble calling several numbers in the 919 area code. This is getting to be extremely frustrating for members of my family as opening a new ticket to create a custom route for each such number is not really a viable option. My wife is standing by my side and furious. Using a cell-phone confirms the numbers are valid (that is the calls go through). The latest number to cause an issue is from a local mall 919-572-8800. Tim, what would you advise me to do? i tried both 7 digit as well as 11 digit dialing and get a fast busy.

[Russell]

Same issue with another number yesterday. One of my wife's coworkers. I'm obviously not going to post that number since it's a residence (as opposed to the number above which is for a shopping mall).

[burris]

My wife is telling me we have the same problems in the 305 area

[fisamo]

Russell--

I'm in the 919 area code, and I was able to connect with no issue. (Had to use the 'wrong number' line.) Are you still seeing fast busy when calling this number? Also, are you calling from within 919, or do you live elsewhere?

[usa2k]

I The latest number to cause an issue is from a local mall 919-572-8800. 10 digit dialing connected OK for me,

[burris]

I think the issue might be that there are various PSTN carriers and routes that connect from both sides of a call.
This is what makes it difficult at times to pin down the exact problem. I know that during the beta days, they had to do some re-routing for me as some of the calls, all in the same AC would or would not connect.

[Russell]

Russell--

I'm in the 919 area code, and I was able to connect with no issue. (Had to use the 'wrong number' line.) Are you still seeing fast busy when calling this number? Also, are you calling from within 919, or do you live elsewhere?
I'm within 919 not far from Apex :-). I'll try again tonight when I get home.

[scott2020]

I have had lots of problems calling numbers in the 417 area code. It happens about every 3-4 weeks, where I have to open a ticket and they do some custom call route. It only lasts a few weeks, then when we call it either has dead air, some strange ring and a subsequent disconnect, or some low quality connection. I have family members experiencing the same problem. Unfortunately last time it happened on a Friday and wasn't fixed until Monday. Needless to say my wife not being able to call her mom for 3 days did not go over well.

[abward]

Russell,

I live on the edge of Cary/Apex, and ours is running normally.

[Brody]

I seem to be having the same problem in the 561 area code. Some numbers just give dead air, while they actually ring on the other end; no audio is ever established. There seems to be no rhyme or reason to it! Cody re-routed one problem for me, but now my parents are having an issue with another number...

[scott2020]

I can't call anything in 573 area code right now. I can call other numbers in other area codes, but 573 is dead for me now. Too bad it's my local area. WAF dropping fast at this point...

[bwyatt]

This problem is reoccurring in various forms on various threads. Same problem happens here in the 276 area code. It has been better this week, but still occurs without rhyme or reason on both incoming and outgoing.

[scott2020]

I see the "forward ports" response a lot because that is supposed to fix audio problems, but it doesn't help me. I have tried both with and without forwarding and about every combination of settings, different routers, DD-WRT, Tomato firmware, and end up with the same results. Tech support has been responsive, and I had the most excellent service for many months. Just in the past 2 months maybe I have had more problems than normal.

[abward]

I seem to be having the same problem in the 561 area code. Some numbers just give dead air, while they actually ring on the other end; no audio is ever established. There seems to be no rhyme or reason to it! Cody re-routed one problem for me, but now my parents are having an issue with another number...

A family member tells me we had this today too, in the 919 area code.

[burris]

I seem to be having the same problem in the 561 area code. Some numbers just give dead air, while they actually ring on the other end; no audio is ever established. There seems to be no rhyme or reason to it! Cody re-routed one problem for me, but now my parents are having an issue with another number...

Looks like you're in the Palm Beach area. From my previous post you can tell I'm down in the Miami area.
I had these problems a few times this morning, but got a bit better.

Almost seems like something common is affecting a bunch of us.

[Russell]

I'm within 919 not far from Apex :-). I'll try again tonight when I get home.

Calling the mall works fine now. Hopefully, issues in the other area codes have also been resolved

[Brody]

Unfortunately, not. Certain calls in the 561 area code (like my sister-in-law) are unreachable via VOIPo. I know it's not me, as another relative who I've recruited to VOIPo cannot call them either. Frustrating, as everything has worked so well for me for quite some time!

[Russell]

Unfortunately, not. Certain calls in the 561 area code (like my sister-in-law) are unreachable via VOIPo. I know it's not me, as another relative who I've recruited to VOIPo cannot call them either. Frustrating, as everything has worked so well for me for quite some time!

Yes, I have similar feelings and hope this is not a sign of the future. VOIPo worked great at the start. Support is still excellent, but in the past few weeks there have been more than one time when I've not been able to make a call via VOIPo (to numbers that worked before like my wife's work phone) with the symptoms described in this thread - so I know it's got nothing to do with my setup. If I open a ticket, as I said support is excellent and the issue will be fixed (the typical response is that the call to that number has been rerouted), but what scares me is the state of the VOIPo foundation that needs such "patches" to be functional. Have we moved to lower cost "upstream" providers that result in such issues? And, with my wife not being able to call the local mall yesterday and one of her co-workers the day before ... the WAF has taken a dive. It'll be reassuring to hear an explanation, but what will really do the trick is flawless connections over a period of time.

To end on a positive note, I was looking for a product today and made several calls to local businesses and all of them completed as expected.

[VOIPoTim]

If you have problem calls, it's important that you report the specific numbers to support@voipo.com so the actual carriers that service those numbers can be identified and the issues corrected.

Old Procedure

We have tightened up our policies surrounding re-routing to make things more consistent. In the past reps were sometimes re-routing an entire area code or NPANXX combo and any that were re-routed incorrectly like this were cleared recently so the routing can be corrected properly.

There were two key issues with updating routing that way which is why it is not standard procedure for us. This ad hoc approach was not effective and often caused erratic issues where issues would crop up again and it was drastically increasing costs.

1) When an entire area code or NPA-NXX combo is re-routed, it affects a huge range of numbers. In most cases, there are many different carriers within a range of number. As an example, 949-829-XXXX is a block where new numbers were issued to AT&T, Cost Plus, and InfoTelecom. There are also many numbers which have been ported in from other carriers in this same range.

So what has happened in the past is say someone has an issue calling issue calling to an AT&T number in that 949-829-XXXX area. In the past with our "use your discretion" policy regarding routing, some reps would re-route the entire NPA-NXX out through an alternate trunk or carrier on our network. This may solve the issue for the customer who had an issue with calling the AT&T number, but in turn would cause an entirely new issue with customers calling Infotelecom numbers within the same block. A rep then may re-route back out the original way which would solve the issue calling an Infotelecom number, but break the AT&T one again.

This is why some users reported outgoing calling issues being fixed and then coming back and fixed and coming back (all the free form re-routes reps were making).

2) The second factor is cost. When re-routing an entire NPA-NXX, it was often isolated to one specific carrier within that network (like in the example above AT&T or Infotelecom) and by re-routing the entire block, every single number was re-routed including ones that did not need to be re-routed. This greatly increased costs. Just for some perspective, just 100 of the "custom routes" that were added in our system were generating $15K more in monthly costs than routing the the original assigned carriers.

When looking at all the custom routes in place and analyzing all traffic, we had less than 3% of traffic re-routed and our total overall costs were literally 3x as much as if we had used the original carriers for that 3%. I'm sure you can appreciate that it's not a good thing to pay 3X as much on the total cost of providing service for 3% of traffic.

On a side note, this is not a matter of "cheaper" carriers vs more expensive ones. We are very confident in all of the carriers we work with and have good relationships with them all. In some areas though, different carriers have different situations and are significantly cheaper in those particular areas. As an example, if a carrier is a CLEC somewhere, it's almost always going to be cheaper for them to handle the traffic than it is for a carrier who is reselling and handing off to another CLEC. Pacwest is an example here that is VERY competitive in their territory (mostly west coast), but is very expensive for routing traffic elsewhere (even though they will do it) because they just pass it on to someone else. There are also some situations where we have partnerships with other companies for various reasons. One is a provider which has a very large PAYG base (but has only been mentioned a handful of times on BBR) with a lot of telemarketing type traffic. Our residential traffic has longer calls which help balance out their very short calls to give them a better overall traffic profile.

By re-routing only affected traffic and working with the carriers involved to solve the issues rather than simply routing around them, this is a non-issue.


New Procedure

First, we've moved all broad re-routing to Tier II techs only. Front line support can re-route any individual numbers that you report as having an issue, but they do not have the ability to re-route entire areas.

Re-routing the specific destinations will give some level of immediate resolution, but won't resolve the broad issue....re-routing ALL calls in that area wouldn't either and may cause more issues with other providers in the area.

Our system tracks what it perceives to be failed calls. We use this information in combination with your reported failed calls to find trends. When we see a trend in an area, our Tier II department is contacting the NOCs or the appropriate carriers to work with them to identify and fix the issue so that the core issue can be resolved rather than simply routing around them. Most respond very quickly when we have sample numbers and concrete stats (like x% of calls to you failed here), but some other companies may be slower. If they are slow to respond to us, Tier II has the discretion to do a temporary mass re-route for an area as long as it meets certain criteria and we've had a minimum number of failed calls (either reported or detected by system) to there. Overall, there are some cheap/low quality carriers out there, but that's not really the issue here. It's just a matter of some carriers being more competitive in their native areas or due to special situations.

This does a few things.

1) It solves the core issue rather than "patching it" by simply routing around it. This makes it less likely to occur again and also it's not going to "break" other working routes by re-routing them away in the mass re-route.

2) It keeps our costs in reality since we are not re-routing numbers that we should not be to more expensive carriers.

Big Picture

Overall, there may be some apparent new issues that came from reversing some "broad" changes that reps made in the past rather than dealing with the issue directly. The good news is that as we work through any of those, this will be less likely to occur again, will make the experience much more consistent, and will cut out outgoing call costs by 2/3.

If you have problem calls, please report them so we have valid data to use to get them resolved. Many customers will say "all calls in the area" and then when we test numbers in that area they all are fine. 99% of the time it's carrier specific in terms of it may be all AT&T Wireless numbers in a destination or something along those lines rather than all carriers there.

[VOIPoTim]

Russell & abward - The majority of the reported issues this week were in 919. We were actually seeing a 35% failure rate to some providers within 919 and this was pretty evenly split between two of our carriers. We have been working with one directly since that % is not even within the realm of being acceptable and it seems to be resolved now. If you still have issues, please report them.

Brody and burris - The second problem area (but at a much much lesser extent than 919) were in South FL 305 & 561. We have been working on these areas as well and are very close to resolution, but it has been more isolated in terms of the specific areas involved, so I'm not as confident in it as 919 at this point. We're continuing to work with carriers involved. Please report any new issues.



This problem is reoccurring in various forms on various threads. Same problem happens here in the 276 area code. It has been better this week, but still occurs without rhyme or reason on both incoming and outgoing.

Bwatt - If you had an issue on incoming, it wouldn't be related to these outgoing issues. Support will need to work with you to get it resolved. From the notes on your account, it looks like you have several VOIPo devices in one location. Often times routers have issues sometimes with multiple devices if it mixes up any of the randomized ports for the audio streams. If no one has mentioned anything like this in troubleshooting before, please open a ticket and ask to go to Tier II in a ticket and mention it or link to this. PM me the ticket number and I'll have someone call you Monday to work through it and resolve.

[abward]

Thanks Tim. I will keep an eye on it, and will open a ticket if it happens again.

[Russell]

Russell & abward - The majority of the reported issues this week were in 919. We were actually seeing a 35% failure rate to some providers within 919 and this was pretty evenly split between two of our carriers. We have been working with one directly since that % is not even within the realm of being acceptable and it seems to be resolved now. If you still have issues, please report them.


Thanks for your long and detailed responses, Tim. Hopefully, 919 is back to the way it was some weeks ago. And, yes, I'll report future issues.

And, at the risk of not being PC, Merry Christmas!

[burris]

Hi, Tim..

As you will remember, I come from the world of POTS, and as a result, am totally empathetic with the routing problems you and everyone else in telephony experience.
We also had our NOC only re-route on a one on one basis, unless of course, the carrier went down, in which case an alternate route would be immediately implemented. What people need to understand is that in order to survive, one must seek out the least cost routing that works, and stay with them so long as they perform. If you are simply going to send everything through ATT, Level3, Verizon, etc., you would maybe have better uptime, but you would have to raise your prices considerably.
In this respect, VOIP is not much different than POTS.

From inception, it appears that you are on top of running a first class operation and from the results, what you are doing is working.
For me, if I have to re-dial a number because it dropped, it's not the end of the world. From my wife, I hear about it. After all, I can't imagine anyone who owns a computer or who is on the internet, not having the same kinds of problems. I only post or submit a ticket when I think it might detail the larger scope of an impending problem.

Anyway, have the rest of a great Holiday and keep up the good work.

burris

[scott2020]

Thanks Tim. I know my problems stem from calling rural areas, where unfortunately it costs considerably more to terminate the call regardless of carrier. I experiment with several providers, and most of them either sound awful or have various different problems. VOIPo has been good about fixing me up when I get terrible audio, but it must cost more. Not saying VOIPo won't do the right thing, but I wonder if once a month or so some calling tables aren't updated somewhere at upstream carriers, resulting in trouble again at least for me. My thing is I know the audio can be fine to the rural areas I call, so that is what I have come to expect. I get annoyed when an upstream carrier somewhere reroutes my call through somewhere cheaper and audio sounds like it is under water or worse. I understand VOIP and can tolerate things and it is nice to work with a company who will make it right.