New Database Servers

[VOIPoTim]

We're adding more database servers today.

This should not cause any issues, but I wanted to post ahead of time and let you know that if you see any quirkiness in vPanel, it could be related.

Again, no issues are expected from this and it'll increase stability quite a bit.

[abward]

Tim,

Is there any particular reason why you don't do this sort of thing during an off-hours maintenance window, like 03:00 Sunday morning, or something like that? Seems to me that you really don't want to me generating more possible complaints these days.

IMHO, no change (unless a sev 1, of course) should be made during production hours.

[burris]

Tim,

Is there any particular reason why you don't do this sort of thing during an off-hours maintenance window, like 03:00 Sunday morning, or something like that? Seems to me that you really don't want to me generating more possible complaints these days.

IMHO, no change (unless a sev 1, of course) should be made during production hours.

I wonder if that's the result of the outages many are experiencing now..

[VOIPoTim]

Tim,

Is there any particular reason why you don't do this sort of thing during an off-hours maintenance window, like 03:00 Sunday morning, or something like that? Seems to me that you really don't want to me generating more possible complaints these days.

IMHO, no change (unless a sev 1, of course) should be made during production hours.

This was a pretty critical change (will eliminate the latency issues that have come up a lot lately).

Some people don't agree with the timing, but I don't want to do something at night and then risk something happening when support is not fully staffed to handle the flood of calls.

The main reason we do this stuff during the day is so that our support center is fully staffed and available if issues do come up.

[fisamo]

Some people don't agree with the timing, but I don't want to do something at night and then risk something happening when support is not fully staffed to handle the flood of calls.

The main reason we do this stuff during the day is so that our support center is fully staffed and available if issues do come up.

OK, I understand your reason for making changes like this during the day. However, abward brings up an excellent point--more people are going to be using their phone at 4pm on a weekday than at 3am on a Sunday. The "flood" of calls, therefore, would more likely (than at 4pm on a weekday) be a "trickle".

Scott2020 also brings up a good point that the phone should work 24/7. After all, you never know when an emergency call will need to be made. Well, you do have to perform your maintenance sometime, and when your call volume is the lowest is the best time for it to be done, even if your support staff is all sleeping (OFF the clock, of course! ).

Perhaps the best way to handle these potential issues is to have a maintenance recording for your support line. In the case of this database server addition/reconfiguration, it could have said, "On Sunday March 15 at 3AM Central time, Voipo began scheduled maintenance. These activities were anticipated to be complete by 4AM and should not affect anyone's service. If your service is not working, we apologize for any inconvenience, and we invite you to submit a ticket by typing in your voipo number now. We will restore your service as soon as we can."

You could then log all tickets submitted at that time so you can be sure any affected customers (who reported problems) are back online.

[usa2k]

Good thoughts. My ISP often has recorded messages on their support line.

"If your service is in Livonia, WideOpenWest is aware of a cable service issue ..."

[HelpNYC]

Agree................

This was a pretty critical change (will eliminate the latency issues that have come up a lot lately).

Some people don't agree with the timing, but I don't want to do something at night and then risk something happening when support is not fully staffed to handle the flood of calls.

The main reason we do this stuff during the day is so that our support center is fully staffed and available if issues do come up.

[VOIPoTim]

When we added new database servers to the mix a few minutes ago, there was a table that was not fully synced for some reason and left about 3000 records out for a few minutes.

It is being restored/pushed to all database servers right now and those clients should be back up now.

[abward]

I think the results today speak for themselves. I rest my case, but I won't rub it in anymore.

[scott2020]

Cut them some slack. Tim doesn't have to post anything in the forums about it. At least they make announcements so people know what is going on, instead of just doing something and denying anything has changed if there are problems.