high jitter on new server

[Wil]

I'm seeing really high jitter (lag) on the new server around 7:30 am UTC. The following is from "-ping" inside the client run repeatedly over the course of a minute or so. Note how it jumps from a typical 30-70 ms up to as high as 1300 ms and then immediately back down.

Min: 29.5ms Max: 31.5ms Avg: 37.0ms
Min: 31.0ms Max: 47.0ms Avg: 42.6ms
Min: 28.5ms Max: 36.5ms Avg: 38.3ms
Min: 27.5ms Max: 63.0ms Avg: 49.4ms
Min: 30.0ms Max: 65.0ms Avg: 43.9ms
Min: 31.0ms Max: 180.0ms Avg: 79.2ms
Min: 31.0ms Max: 72.5ms Avg: 56.0ms
Min: 31.0ms Max: 203.0ms Avg: 77.2ms
Min: 29.0ms Max: 48.0ms Avg: 43.5ms
Min: 29.5ms Max: 112.0ms Avg: 56.7ms
Min: 30.5ms Max: 49.5ms Avg: 40.9ms
Min: 30.0ms Max: 97.0ms Avg: 53.9ms
Min: 30.5ms Max: 47.0ms Avg: 42.3ms
Min: 28.0ms Max: 173.5ms Avg: 78.5ms
Min: 30.5ms Max: 46.5ms Avg: 41.1ms
Min: 30.5ms Max: 43.0ms Avg: 39.7ms
Min: 27.5ms Max: 38.5ms Avg: 40.1ms
Min: 31.0ms Max: 148.0ms Avg: 71.6ms
Min: 26.5ms Max: 82.5ms Avg: 51.1ms
Min: 30.5ms Max: 1339.7ms Avg: 361.9ms
Min: 29.5ms Max: 189.0ms Avg: 99.5ms
Min: 30.5ms Max: 142.0ms Avg: 75.0ms
Min: 30.5ms Max: 80.0ms Avg: 50.5ms
Min: 30.0ms Max: 170.5ms Avg: 74.8ms
Min: 30.5ms Max: 34.0ms Avg: 37.9ms
Min: 30.5ms Max: 230.0ms Avg: 87.9ms
Min: 29.0ms Max: 47.5ms Avg: 42.0ms
Min: 26.5ms Max: 408.6ms Avg: 121.8ms
Min: 28.0ms Max: 132.5ms Avg: 69.9ms
Min: 28.5ms Max: 46.5ms Avg: 40.2ms
Min: 27.5ms Max: 66.0ms Avg: 49.2ms
Min: 30.5ms Max: 161.5ms Avg: 65.7ms
Min: 30.0ms Max: 315.5ms Avg: 94.0ms
Min: 31.5ms Max: 46.0ms Avg: 43.4ms
Min: 30.5ms Max: 39.5ms Avg: 39.1ms
Min: 30.5ms Max: 269.5ms Avg: 88.7ms
Min: 30.5ms Max: 104.0ms Avg: 61.3ms
Min: 30.5ms Max: 32.0ms Avg: 37.6ms
Min: 31.5ms Max: 47.0ms Avg: 49.2ms
Min: 31.0ms Max: 94.0ms Avg: 58.7ms
Min: 28.0ms Max: 47.0ms Avg: 41.6ms
Min: 29.0ms Max: 48.0ms Avg: 42.1ms
Min: 28.5ms Max: 77.5ms Avg: 52.8ms
Min: 31.0ms Max: 61.0ms Avg: 44.8ms
Min: 29.0ms Max: 48.5ms Avg: 40.7ms
Min: 30.5ms Max: 188.5ms Avg: 105.0ms
Min: 30.5ms Max: 32.5ms Avg: 37.7ms
Min: 30.5ms Max: 57.0ms Avg: 44.4ms
Min: 31.0ms Max: 33.5ms Avg: 38.1ms
Min: 31.0ms Max: 46.0ms Avg: 45.2ms
Min: 32.5ms Max: 134.5ms Avg: 73.1ms
Min: 26.5ms Max: 46.5ms Avg: 42.0ms
Min: 28.0ms Max: 47.0ms Avg: 44.5ms
Min: 29.5ms Max: 48.0ms Avg: 41.8ms
Min: 30.5ms Max: 41.0ms Avg: 39.4ms
Min: 30.5ms Max: 49.5ms Avg: 46.8ms
Min: 30.0ms Max: 264.5ms Avg: 101.5ms
Min: 30.0ms Max: 34.0ms Avg: 38.1ms
Min: 31.5ms Max: 811.6ms Avg: 203.6ms
Min: 35.5ms Max: 239.0ms Avg: 94.8ms
Min: 27.0ms Max: 52.0ms Avg: 41.1ms
Min: 28.0ms Max: 75.5ms Avg: 46.3ms
Min: 31.0ms Max: 110.0ms Avg: 61.0ms
Min: 29.0ms Max: 109.0ms Avg: 59.3ms
Min: 29.5ms Max: 40.0ms Avg: 39.2ms

[Wil]

OS level pings at the same time are 20 ms and very steady.

[Pariah]

I'm getting the same. Out of curiosity, where are you located?

[grunger]

around 150ms all the time at my place.

[Wil]

I'm getting the same. Out of curiosity, where are you located?

Virginia, US. But I've eliminated most of the variables for it being network lag: OS level pings are steady with essentially no packet loss and I've tried two different ISPs which don't share a common path until iweb's (the hosting company's) ISP Cogent. This acts like server lag, not network lag.

traceroute -T -p 60 shard.uoex.net
traceroute to shard.uoex.net (72.55.146.174), 30 hops max, 60 byte packets
1 lily.p.dirtside.com (192.168.99.12) 0.413 ms 0.456 ms 0.513 ms
2 L300.WASHDC-VFTTP-91.verizon-gni.net (173.73.47.1) 1.766 ms 1.986 ms 2.480 ms
3 G0-3-4-3.WASHDC-LCR-22.verizon-gni.net (130.81.48.122) 16.717 ms 16.784 ms 16.777 ms
4 ae24-0.RES-BB-RTR2.verizon-gni.net (130.81.174.228) 58.909 ms * so-14-0-0-0.RES-BB-RTR2.verizon-gni.net (130.81.22.56) 4.654 ms
5 so-4-0-3-0.RES-BB-RTR1.verizon-gni.net (130.81.23.238) 5.389 ms 0.ae2.BR1.IAD8.ALTER.NET (140.222.229.165) 7.705 ms 7.817 ms
6 te0-1-0-14.ccr41.iad02.atlas.cogentco.com (154.54.12.45) 7.654 ms 0.xe-7-1-0.BR2.IAD8.ALTER.NET (152.63.38.125) 6.512 ms te0-6-0-14.ccr41.iad02.atlas.cogentco.com (154.54.10.225) 7.966 ms
7 be2176.ccr21.dca01.atlas.cogentco.com (154.54.41.54) 9.188 ms be2113.mpd21.dca01.atlas.cogentco.com (154.54.6.170) 8.235 ms be2176.ccr21.dca01.atlas.cogentco.com (154.54.41.54) 8.275 ms
8 be2176.ccr21.dca01.atlas.cogentco.com (154.54.41.54) 6.936 ms be2151.mpd22.jfk02.atlas.cogentco.com (154.54.40.74) 12.435 ms be2112.mpd22.dca01.atlas.cogentco.com (154.54.5.234) 4.744 ms
9 be2149.ccr22.jfk02.atlas.cogentco.com (154.54.31.126) 10.978 ms be2107.ccr21.alb02.atlas.cogentco.com (154.54.3.94) 16.128 ms be2149.ccr22.jfk02.atlas.cogentco.com (154.54.31.126) 11.042 ms
10 be2088.ccr21.ymq02.atlas.cogentco.com (154.54.43.17) 22.042 ms be2108.ccr21.alb02.atlas.cogentco.com (154.54.3.134) 14.237 ms 14.812 ms
11 te8-4.cl-core06.cogent.mtl.iweb.com (38.104.154.202) 20.008 ms 38.122.42.122 (38.122.42.122) 20.217 ms 19.884 ms
12 po22.cr3.mtl.iweb.com (184.107.1.2) 19.542 ms 18.875 ms 19.294 ms
13 * po22.cr3.mtl.iweb.com (184.107.1.2) 18.895 ms *
14 72.55.146.174 (72.55.146.174) 18.471 ms 19.236 ms 17.920 ms

traceroute -T -p 60 shard.uoex.net
traceroute to shard.uoex.net (72.55.146.174), 30 hops max, 60 byte packets
1 10.1.192.1 (10.1.192.1) 8.867 ms 8.934 ms 8.834 ms
2 ip68-100-3-113.dc.dc.cox.net (68.100.3.113) 8.946 ms 9.869 ms 8.792 ms
3 ip68-100-3-113.dc.dc.cox.net (68.100.3.113) 12.234 ms 12.492 ms 12.012 ms
4 mrfddsrj02gex070002.rd.dc.cox.net (68.100.0.145) 12.629 ms 11.301 ms 11.198 ms
5 68.1.4.246 (68.1.4.246) 12.391 ms 12.320 ms 11.964 ms
6 te0-16-0-15.ccr41.iad02.atlas.cogentco.com (38.122.62.165) 12.224 ms te0-18-0-15.ccr41.iad02.atlas.cogentco.com (154.54.14.21) 9.359 ms te0-16-0-15.ccr41.iad02.atlas.cogentco.com (38.122.62.165) 8.566 ms
7 be2177.ccr22.dca01.atlas.cogentco.com (154.54.41.206) 9.563 ms be2176.ccr21.dca01.atlas.cogentco.com (154.54.41.54) 14.038 ms 12.554 ms
8 be2150.mpd21.jfk02.atlas.cogentco.com (154.54.31.130) 18.980 ms be2148.ccr21.jfk02.atlas.cogentco.com (154.54.31.118) 18.812 ms be2149.ccr22.jfk02.atlas.cogentco.com (154.54.31.126) 18.459 ms
9 be2108.ccr21.alb02.atlas.cogentco.com (154.54.3.134) 20.363 ms be2109.ccr21.alb02.atlas.cogentco.com (154.54.3.138) 20.187 ms be2107.ccr21.alb02.atlas.cogentco.com (154.54.3.94) 24.758 ms
10 be2088.ccr21.ymq02.atlas.cogentco.com (154.54.43.17) 29.594 ms 29.716 ms 29.425 ms
11 38.122.42.122 (38.122.42.122) 26.984 ms te8-4.cl-core06.cogent.mtl.iweb.com (38.104.154.202) 25.776 ms 29.463 ms
12 po22.cr3.mtl.iweb.com (184.107.1.2) 230.427 ms 229.497 ms 227.125 ms
13 te9-3.dr5.mtl.iweb.com (67.205.127.93) 27.754 ms * *
14 72.55.146.174 (72.55.146.174) 27.347 ms 26.258 ms 29.984 ms


Also, look what happens at the tcp level for a "-ping 1" lag I recorded as 640 ms:

send ping packet:
11:25:21.577082 IP 71.246.241.146.60556 > 72.55.146.174.60: Flags [P.], seq 2137:2143, ack 21515, win 16425, length 6
Server TCP ack of ping packet
11:25:21.698954 IP 72.55.146.174.60 > 71.246.241.146.60556: Flags [.], ack 2143, win 253, length 0
Server responds with ping response packet:
11:25:22.305805 IP 72.55.146.174.60 > 71.246.241.146.60556: Flags [P.], seq 21515:21518, ack 2143, win 253, length 3
client TCP ack and additional data sent:
11:25:22.306084 IP 71.246.241.146.60556 > 72.55.146.174.60: Flags [P.], seq 2143:2145, ack 21518, win 16424, length 2

Notice the timing. I get a TCP ack from the server OS in 13 ms but the server application doesn't respond for an additional 600 ms.

[heyyo]

I'm extremely jittery as well. Pretty much unplayable.

[Acacia]

i was only able to log in for a moment and then got booted off. now unable to connect all together

[Wil]

i was only able to log in for a moment and then got booted off. now unable to connect all together

+C reverted the server to the old IP address yesterday. If your DNS hasn't caught up you may still be trying to access the one that had the lag.

[Acacia]

i was only able to log in for a moment and then got booted off. now unable to connect all together

+C reverted the server to the old IP address yesterday. If your DNS hasn't caught up you may still be trying to access the one that had the lag.

i flushed my dns again. and it still won't connect