From the server side

[jdstroy]

Hi, all,

The last time I was here, I had some trouble getting incoming calls through a symmetric NAT. I finally figured out that the problem mostly occurs on inbound calls. Outbound calls work out fairly well, but almost all inbound calls get the "Awaiting Acknowledgment" message, followed by "ACK timeout". Keep in mind, this only happens on a symmetric NAT (which, for me, happens to be 5 out of 7 days every week).

I grew weary of tweaking my UA's settings, and switching between X-lite, Phoner, and SJPhone, so I switched to IPtel.org's SIP service. It works well, bidirectionally relaying RTP streams every time. No more "Awaiting Acknowledgment" message! Every time, that is, when I can connect! Again, outbound calls are just fine, but about 9/10ths of inbound calls end up with SIP "480 Temporarily Unavailable" messages. Uhoh. And this is behind both a Symmetric NAT and a Full cone NAT. (For the record, IPtel's SIP service works better in connected calls; but Voxalot's works better in terms of calls connected.)

So I tried an old (trusty? untrusty?) SIP provider that I didn't really use before: AOL. Oh... no. AOL. I was expecting the "It's so easy to use, no wonder it's #1 (in dissatisfied customers)" experience, but surprisingly, that isn't the case! AOL, in fact, sends the SIP and RTP data through to me correctly, through a symmetric NAT!

Now, this is, to me, very cool. I can finally get phone calls working again. But I'm back here at Voxalot again... because AOL's SIP proxy only accepts PSTN calls (e.g. SIP-to-SIP doesn't seem to work right).

That puts me back here at Voxalot again. Why? Ordinary phone service sucks--SIP makes things more interesting! Okay, okay, not really. I'm interested in getting the best of both worlds -- SIP and PSTN. What I'd like to know is if I can convince someone to check out what's going on on the server side. Or even send me dumps relevant to my problem. The thing that I'd like to know is: why do my inbound calls (usually from a non-Voxalot user) mostly end up with the "Awaiting Acknowledgment" message if:

1) SIP data streams are (correctly) delivered to me; and
2) RTP data streams are (supposedly) proxied through Voxalot?

I think I can safely assume #1 if I can "sense" all the incoming call notification (Ring) messages.

#2 assumes that proxied RTP helps with completing my calls. I think it's safe to assume that an RTP proxy helps, given AOL's SIP service works reliably for me.

... but I suspect that the RTP data isn't going through Voxalot as it should. According to what I've seen on Wireshark, anyway.

Can anyone shed some insight? Is there anyone from Voxalot that would be able to look in to network dumps, or that would be able to provide me with relevant network dumps? Even logging would be nice...

Thanks

[emoci]

1. Under default conditions VoXalot will proxy the RTP stream
2. VoXalot will try to make a point-to-point connection via re-invites if you adjust your settings as noted here for specific providers: HowTo: 6 Steps To Optimize Your Audio - Voxalot FAQ

3. However Boatman also identifies that in cases where NAT setup appears problematic, VoXalot will take to proxying RTP streams regardlessly... see http://forum.voxalot.com/voxalot-sup...html#post18519

Should you continue to have trouble with getting VoXalot to play nice with your setup, I would next try MySipSwitch...because they actually do not concern themselves with RTP media at all, it may actually be more adapted to the situation...and of course you can still use VoXalot from within MSS...

[jdstroy]

Hmm, nice to hear from you again, emoci!

I think I want Voxalot to proxy the RTP payload. I'm sitting behind a symmetric NAT that makes life troublesome. Without NAT payload traversal, I end up with the typical one-sided calls. So #1 sounds like a good idea, and #2 a not-so-good idea on my end (yes, I've tried services without RTP proxy support, and no, it doesn't work here). And #3 appears to be what Voxalot is doing... but only if I'm calling out.

I actually have used MSS; but MSS doesn't work at this site, again due to symmetric NAT handling. SIP sessions go through, RTP sessions get blocked.

I'm just hoping that I could possibly get debug information from the server side.

[emoci]

Originally Posted by jdstroy

Hmm, nice to hear from you again, emoci!

I think I want Voxalot to proxy the RTP payload. I'm sitting behind a symmetric NAT that makes life troublesome. Without NAT payload traversal, I end up with the typical one-sided calls. So #1 sounds like a good idea, and #2 a not-so-good idea on my end (yes, I've tried services without RTP proxy support, and no, it doesn't work here). And #3 appears to be what Voxalot is doing... but only if I'm calling out.

I actually have used MSS; but MSS doesn't work at this site, again due to symmetric NAT handling. SIP sessions go through, RTP sessions get blocked.

I'm just hoping that I could possibly get debug information from the server side.

Do you have any sort of control over NAT settings ... at your current location (eg. would you have access to port forwarding...)?

Logs from the server end are hard to come by (Mods have no access to the backend either)...

Are you already grabbing logs on your end...?

One other thing to try, sub MSS for VoXalot.... they have an option that will let you monitor in real time what their server sees for your acct. as you make and receive calls (I am assuming your issue with MSS and VoXalot is similar, and one can help resolve one by resolving the other...)

Finally, I've been using Zoiper, for the only reason that it can be used in a no-install USB Stick scenario.... but I've found it's STUN sever to work quite well through a variety of different networks... also it allows for comparing connection issues by ruling out setup dependent problems since you can setup and test on one end, send to someone else with the exact same settings and have them try to run it, hence ruling out everything else but the NAT settings... (in worst case scenario, it sounds like you may at least be able to have both IPTel and VoXalot running simultaneously...which Zoiper supports, one for incoming calls and the other for outgoing...)

Last but not least, you've tried VoXalot, MSS, what about PBXes....?

[boatman]

Originally Posted by jdstroy

I think I want Voxalot to proxy the RTP payload.

Any idea why that isn't that already happening.
I assume you have set Symmetric NAT Handling to 'Yes', and there may be a setting called "Optimize Audio Path" which you would have set as no. Is it not working as advertised? Have you actually used a packet sniffer to see source/destination address of RTP packets?

You said that the trouble was getting incoming calls. Do you want me to do a test call to you with my packet sniffer running?

[jdstroy]

Originally Posted by emoci

Do you have any sort of control over NAT settings ... at your current location (eg. would you have access to port forwarding...)?

No, I'm connected via a university network. The university IT services group is not willing to make any exceptions in their firewall. :/

Originally Posted by emoci

Logs from the server end are hard to come by (Mods have no access to the backend either)...

Originally Posted by emoci

Are you already grabbing logs on your end...?

Logs and packet captures, yep.

Originally Posted by emoci

One other thing to try, sub MSS for VoXalot.... they have an option that will let you monitor in real time what their server sees for your acct. as you make and receive calls (I am assuming your issue with MSS and VoXalot is similar, and one can help resolve one by resolving the other...)

Registration looks fine on MSS. Outbound calls don't work (outgoing only audio).

Inbound calls on MSS are similar to the situation on Voxalot (Awaiting Acknowledgment).

The logs for SJPhone on MSS are on pastebin:

Register OK
Outbound call failure
Outbound call hangup

Inbound call failure
Inbound call hangup
Inbound call (complete, ACK timeout)

MSS Dialplan:

exten = 100,1,Switch(anon,,303@sip.blueface.ie)
exten = 101,1,Switch(anon,,612@fwd.pulver.com)

MSS events (couldn't capture everything, but there isn't anything interesting)

And packet captures... I can do that, too, but not at the same time as debug logging. I can show them, though, as needed.

The only problem with "resolving" the issue between me and MSS is that it would be near impossible, if I'm not mistaken, to resolve it. MSS doesn't switch RTP payloads, which means that the audio streams would never make it for me. They'd reach my NAT and stop dead in its tracks. Sorta like what's happening with incoming Voxalot calls for me. =}

Originally Posted by emoci

Finally, I've been using Zoiper, for the only reason that it can be used in a no-install USB Stick scenario.... but I've found it's STUN sever to work quite well through a variety of different networks... also it allows for comparing connection issues by ruling out setup dependent problems since you can setup and test on one end, send to someone else with the exact same settings and have them try to run it, hence ruling out everything else but the NAT settings... (in worst case scenario, it sounds like you may at least be able to have both IPTel and VoXalot running simultaneously...which Zoiper supports, one for incoming calls and the other for outgoing...)

Originally Posted by emoci

Last but not least, you've tried VoXalot, MSS, what about PBXes....?

I recall your recommendation of Zoiper, but I haven't tried it. I've already a pile of SIP soft phones, but if it comes down to using it or nothing else, I'll happily give it a spin. I've got SJPhone, X-lite, and Phoner installed; I have had PhonerLite, sipxphone, phonegaim, and Mozilla Zap by 8x8; I haven't tried 3CXPhone, ConnectFusion, Zoiper, Avaya's SIP Softphone (was that what I downloaded?)...

Right now, neither IPTel nor VoXalot work well enough for me to combine IPTel/Voxalot. IPTel's outgoing calls will either complete successfully, or produce an error immediately after dialing. (Errors rarely happen on outbound calls to other parties.) On the other side of the spectrum, VoXalot's outgoing calls often complete successfully, but when they don't complete successfully, I'm left with a dead line (e.g. one-sided conversations).

Incoming calls are also broken; IPTel's servers return error code 408 to about 20 bad calls for every good call. Incoming calls through Voxalot almost never connect with both streams (they all end up with Awaiting Acknowledgment).

I've not tried PBXes, as I'm not really sure how to configure it. I've played with the settings several times, but I've never gotten a "perfect" setup for what I want. (In fact, I never really have gotten it to work the way I wanted, but I'll guess it's a User Error. )

Right now, I've set up GrandCentral<->AOL Voice/SIP to just do ordinary PSTN. (And sometimes I'm throwing Jaduka's dukaDial in to the mix.) It does, in fact, work!

I used to have this (which worked):

GrandCentral<->IPKall<->FWD

I'm trying to reproduce the old setup with either Voxalot or anything else:

GrandCentral<->IPKall<->Other SIP provider

Originally Posted by boatman

Any idea why that isn't that already happening.

Originally Posted by boatman

I assume you have set Symmetric NAT Handling to 'Yes', and there may be a setting called "Optimize Audio Path" which you would have set as no. Is it not working as advertised? Have you actually used a packet sniffer to see source/destination address of RTP packets?

Right, and right. Symmetric NAT Handling is "Yes", and "Optimize Audio Path" is "No." It definitely doesn't seem to be working as advertised for inbound calls; at least, not for me. Outbound calls are significantly better, with a hit-to-miss ratio of about 20 to 1. Outbound calls to a number on Voxalot are more often completed successfully than outbound calls to another network.

Yes, I have used a packet sniffer on the line, and I see that outbound calls work just fine for the most part. The RTP payloads usually travel to/from VoXalot without a hitch when I'm calling outbound. When receiving inbound calls, I believe (last time I ran Wireshark, anyway) that SIP data comes in correctly. I definitely know that the RTP data goes out, but doesn't come in. This is different between Voxalot and IPtel.org:

- Whereas Voxalot shows proper signalling, RTP payload just doesn't seem to be proxied.
- On the other hand, IPtel shows no signalling when then 408 response is received on remote. However, when calls DO get through, the RTP payload is ALWAYS sent via IPtel.

Originally Posted by boatman

You said that the trouble was getting incoming calls. Do you want me to do a test call to you with my packet sniffer running?

Thanks for the offer. Sure, that might be insightful. When do you have time to do this?

[boatman]

During the test call RTP packets passed to and from an IP address in Los Angeles; 64.34.173.199. I'm guessing that's not your IP address, so that must be a Voxalot server.

Your SIP client is SJphone/1.65.377a (SJ Labs). After examining the SIP/SDP packet sent during call setup I have listed some possible sources of the problem below.

1. In the message header of the SIP/SDP packet, SJphone sent this contact address: <sip:798554@129.252.103.250:5060;natp2us1=yes>

Some SIP clients may be trying to send SIP packets to 129.252.103.250 when they should be sending all SIP packets to voxalot.com in order to pass through the symmetric NAT at your location. Ideally the contact address should be 798554@voxalot.com. You may be able to achieve this by setting SJphone to "use outbound proxy" and "use outbound proxy in dialog", if SJphone has such settings. Make sure outbound proxy is set to the same as proxy.

2. In the SIP/SDP packet there are two connection information lines. the first one is incorrect, probably it's your LAN IP address. The second one is correct if RTP packets are to be handled by Voxalot's server. Maybe the second line was inserted by Voxalot.

Connection Information (c): IN IP4 192.168.200.1
Connection Information (c): IN IP4 64.34.173.199

The fact that SJphone is including your LAN IP address in SIP packets may indicate that STUN is not working.

[jdstroy]

Hi boatman; thanks for the call and analysis!

64.34.173.199 == premium.voxalot.com, so yes, it's from voxalot.
129.252.103.250 is (one of the) the externally connected IP addresses at my location.

1. I will attempt to set the SIP Contact URI to 798554@us.voxalot.com (via the Full Address of Record option in SJPhone). Outbound proxy and Outbound proxy for NAT are both set to us.voxalot.com:443 (was us.voxalot.com:5060 during the test call), strict outbound proxy.

2. Your analysis is correct. 192.168.200.1 is the internal IP address assigned to the local machine to connect to the NAT service here. STUN is definitely not working here, as I've disabled it in SJPhone. Please correct me if that is not the proper action for a host behind symmetric NAT.

I'll be back to edit this when I find out if it works!

For the while, though, Jeff Pulver's FWD still appears to work -- but I'm definitely not going to count on that!

[boatman]

Rather than us.voxalot.com you should use voxalot.com in your contact address. This is especially important for inbound calls. I'm not sure how all the servers fit together, but the regional servers (us.voxalot.com, eu.voxalot.com, au.voxalot.com) are best used only for outbound calling.

I am not sure about stun, try with and without stun. Make sure inbound calls are coming to you through voxalot.com and not us.voxalot.com.

The LAN internal IP address should never appear in a SIP packet as a contact address. The contact address is normally your public IP address, but if that doesn't work then you might have success using 64.34.173.199 as contact address.

You are going to need a packet monitoring program such as Wireshark in order to view the contents of your SIP packets. Don't trust that your softphone will construct the packets the way it should.

[jdstroy]

Hi boatman,

Thanks for the tip. I've changed the option for Full Address of Record in SJPhone to 798554@voxalot.com. Setting 798554@us.voxalot.com appears to have increased the incoming call success rate. I'm not seeing any "Awaiting Acknowledgment" statuses anymore. However, I still see "Contact: <sip:798554@192.168.200.1>" occasionally. I'm not sure how to change that.

I also encountered one call session (with Full Address of Record set to 798554@voxalot.com) where only one leg of the call (the outbound segment) was established.

I did encounter a peculiar issue before when 798554@us.voxalot.com was set as Full Address of Record in SJPhone. Some of the incoming calls were not received (they were sent to voicemail) after a long duration of time passed. I'm waiting to see if this will happen again.

I'm also periodically getting bad data from Voxalot, it seems, after calls are established. According to Wireshark, I'm getting UDP data (from premium.voxalot.com:443/64.34.173.199:443) containing 4 bytes, 0x00000000, directed at port 1024, where SJPhone is transmitting and listening (for SIP signalling data). SJPhone freaks out and prints "ERROR Discarded bad packet: Packet too short" into the error log, but otherwise works fine.

As for STUN on SNATs, a quick search on Google suggests that SNATs don't play well with STUN, as expected.

I will try to update when I can test this more thoroughly. I've got my fingers crossed!