NAT trouble with some Websites

Murus
chrs
Posts: 18
Joined: Tue Apr 12, 2016 7:37 pm

NAT trouble with some Websites

Post by chrs » Mon Apr 18, 2016 2:34 pm

I set up a Mac Mini 2011 OSX 10.11.4 for internet sharing with Murus from a DSL connection via ppp0 to a LAN on en0.

At first I thought everything was working ok but then I detected that some websites do not load, they just stall forever.
It appears to be the big ones witch open lots of connections like http://www.tumblr.com or google maps. Other websites like apple.com work without any apparent problems.

I restarted with clean config and one static client IP, DHCP off, Apple Firewall and Internet Sharing is off, fixed external DNS, and all services NAT allowed, logging all blocked
enabled; and the problem persists.
If the browser is runnning on the router Mac itself all websites work, no problem.

There is nothing in the logs to see; I have no clue where to look and for what kind of error to search: it seems NAT related since any blocking should be logged, but why only some Websites?

Any ideas?

chrs
Posts: 18
Joined: Tue Apr 12, 2016 7:37 pm

Re: NAT trouble with some Websites

Post by chrs » Tue Apr 19, 2016 11:17 am

In addition to my first post I just found something strange in the logs when trying to conect to google.de (replaced my ip with ME):

Code: Select all

Apr 19 12:38:09 Mini11 pf[470]: 00:02:57.157595 rule 14/0(match): block in on ppp0: 92.226.2.25.443 > ME.39009: Flags [.], seq 3034937291:3034938003, ack 1685310788, win 235, options [nop,nop,TS val 3389692439 ecr 2756273201], length 712
Apr 19 12:38:09 Mini11 pf[470]: 00:00:00.000371 rule 14/0(match): block in on ppp0: 92.226.2.25 > ME: ip-proto-6
Apr 19 12:38:09 Mini11 pf[470]: 00:00:00.001099 rule 14/0(match): block in on ppp0: 92.226.2.25.443 > ME.39009: Flags [.], seq 2048:2760, ack 1, win 235, options [nop,nop,TS val 3389692439 ecr 2756273201], length 712
Apr 19 12:38:09 Mini11 pf[470]: 00:00:00.000677 rule 14/0(match): block in on ppp0: 92.226.2.25 > ME: ip-proto-6
Apr 19 12:38:09 Mini11 pf[470]: 00:00:00.019717 rule 14/0(match): block in on ppp0: 92.226.2.25.443 > ME.39009: Flags [.], seq 0:712, ack 1, win 235, options [nop,nop,TS val 3389692461 ecr 2756273238], length 712
Apr 19 12:38:09 Mini11 pf[470]: 00:00:00.000394 rule 14/0(match): block in on ppp0: 92.226.2.25 > ME: ip-proto-6
Apr 19 12:38:09 Mini11 pf[470]: 00:00:00.220358 rule 14/0(match): block in on ppp0: 92.226.2.25.443 > ME.39009: Flags [.], seq 0:712, ack 1, win 235, options [nop,nop,TS val 3389692681 ecr 2756273238], length 712
Apr 19 12:38:09 Mini11 pf[470]: 00:00:00.000381 rule 14/0(match): block in on ppp0: 92.226.2.25 > ME: ip-proto-6
Apr 19 12:38:10 Mini11 pf[470]: 00:00:00.439700 rule 14/0(match): block in on ppp0: 92.226.2.25.443 > ME.39009: Flags [.], seq 0:712, ack 1, win 235, options [nop,nop,TS val 3389693121 ecr 2756273238], length 712
Apr 19 12:38:10 Mini11 pf[470]: 00:00:00.000348 rule 14/0(match): block in on ppp0: 92.226.2.25 > ME: ip-proto-6
Apr 19 12:38:11 Mini11 pf[470]: 00:00:00.879361 rule 14/0(match): block in on ppp0: 92.226.2.25.443 > ME.39009: Flags [.], seq 0:712, ack 1, win 235, options [nop,nop,TS val 3389694001 ecr 2756273238], length 712
Apr 19 12:38:11 Mini11 pf[470]: 00:00:00.000469 rule 14/0(match): block in on ppp0: 92.226.2.25 > ME: ip-proto-6
Apr 19 12:38:12 Mini11 pf[470]: 00:00:01.759082 rule 14/0(match): block in on ppp0: 92.226.2.25.443 > ME.39009: Flags [.], seq 0:712, ack 1, win 235, options [nop,nop,TS val 3389695761 ecr 2756273238], length 712
Apr 19 12:38:12 Mini11 pf[470]: 00:00:00.000023 rule 14/0(match): block in on ppp0: 92.226.2.25 > ME: ip-proto-6
Apr 19 12:38:16 Mini11 pf[470]: 00:00:03.520656 rule 14/0(match): block in on ppp0: 92.226.2.25.443 > ME.39009: Flags [.], seq 0:712, ack 1, win 235, options [nop,nop,TS val 3389699281 ecr 2756273238], length 712
Apr 19 12:38:16 Mini11 pf[470]: 00:00:00.000130 rule 14/0(match): block in on ppp0: 92.226.2.25 > ME: ip-proto-6
Apr 19 12:38:50 Mini11 pf[470]: 00:00:34.181233 rule 14/0(match): block in on ppp0: 95.213.255.88.80 > ME.46368: Flags [.], ack 1774605602, win 16384, length 0
What's going on here?

hany
Posts: 480
Joined: Wed Dec 10, 2014 5:20 pm

Re: NAT trouble with some Websites

Post by hany » Wed Apr 20, 2016 2:22 pm

There is nothing in the logs to see; I have no clue where to look and for what kind of error to search: it seems NAT related since any blocking should be logged, but why only some Websites?
that's strange, I've never faced such issue.
However I need some more information about your network configuration.
Is your WAN interface using a public IP address? How did you configure your clients?

chrs
Posts: 18
Joined: Tue Apr 12, 2016 7:37 pm

Re: NAT trouble with some Websites

Post by chrs » Thu Apr 21, 2016 11:23 am

My situation:

-ADSL Modem in Bridge mode

-Mac Mini Server 5 / 10.11.4 doing PPPoE on Apple USB Ethernet to Modem ppp0 and internal ethernet en0 running Murus as NAT router 192.168.1.1

Code: Select all

ifconfig ppp0
ppp0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1492
	inet 92.231.14.54 --> 62.52.200.233 netmask 0xff000000 

Mini11: chris$ ifconfig en0
en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
	options=10b<RXCSUM,TXCSUM,VLAN_HWTAGGING,AV>
	ether c8:2a:14:59:15:8b 
	inet6 fe80::ca2a:14ff:fe59:158b%en0 prefixlen 64 scopeid 0x4 
	inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
	nd6 options=1<PERFORMNUD>
	media: autoselect (1000baseT <full-duplex,flow-control>)
	status: active
Mini11: chris$ 
- Client Mac 192.168.1.25 browsing the web

Code: Select all

M12:~ chris$ ifconfig en0
en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
	options=b<RXCSUM,TXCSUM,VLAN_HWTAGGING>
	ether 0c:4d:e9:9a:54:60 
	inet6 fe80::e4d:e9ff:fe9a:5460%en0 prefixlen 64 scopeid 0x4 
	inet 192.168.1.25 netmask 0xffffff00 broadcast 192.168.1.255
	media: autoselect (1000baseT <full-duplex,flow-control>)
	status: active
All IPs static manual configuration, the router gets public IP from ISP changing on every connect.

Hope this helps.

chrs
Posts: 18
Joined: Tue Apr 12, 2016 7:37 pm

Re: NAT trouble with some Websites

Post by chrs » Fri Apr 22, 2016 10:08 am

I believe I got it.

It's an MTU issue with the PPPoE connection and some webservers. I don' understand what's going on on their side but reducing my client Macs MTU to 1482 solved the problem.

But that is not a really good solution. I think I need to enable mss clamping on ppp0 interface like the old natd in ipfw days did. How do I do this with Murus?

hany
Posts: 480
Joined: Wed Dec 10, 2014 5:20 pm

Re: NAT trouble with some Websites

Post by hany » Sat Apr 23, 2016 2:29 pm

I believe I got it.

It's an MTU issue with the PPPoE connection and some webservers. I don' understand what's going on on their side but reducing my client Macs MTU to 1482 solved the problem.
wow, chrs, this is really interesting. That's something really new for me.
But that is not a really good solution. I think I need to enable mss clamping on ppp0 interface like the old natd in ipfw days did. How do I do this with Murus?
Honestly... I have absolutely no idea of what you are talking about :) But this sounds interesting, I'll immediately have a look at the issue.
If this "mss clamping" can be done with pf, it will be possible to add it to a future Murus release. Please allow us some time to do our research and tests :)

Thanks so much for your hint!

chrs
Posts: 18
Joined: Tue Apr 12, 2016 7:37 pm

Re: NAT trouble with some Websites

Post by chrs » Sat Apr 23, 2016 2:58 pm

If this "mss clamping" can be done with pf, it will be possible to add it to a future Murus release
Please do so. It must be done with pf. While ipfw was used, clamping was done by natd (see man natd for the option). If NAT is done with pf, clamping must be done by pf. I now manually hacked this into murus.conf because adding it as a custom handmade rule has no effect because it must be before Apple scrub anchor in the config.

Just add (optional) this line to murus.conf near the top when ppp0 is the WAN interface:

Code: Select all

scrub on ppp0 all max-mss 1440
1440 is a bit low but is a safe bet and most recommended value; perhaps a few more bytes will work. Add this as an option to your NAT dialog and make your customers using DSL via PPPoE happy.

For more info for example consult http://linuxtechres.blogspot.de/2010/03 ... ernet.html

hany
Posts: 480
Joined: Wed Dec 10, 2014 5:20 pm

Re: NAT trouble with some Websites

Post by hany » Sat Apr 23, 2016 5:06 pm

chrs,

absolutely great :) thank you.
I'm going to add this feature right now.
I will post a Murus beta here in this thread so you can try it and confirm me it works.

hany
Posts: 480
Joined: Wed Dec 10, 2014 5:20 pm

Re: NAT trouble with some Websites

Post by hany » Sat Apr 23, 2016 6:51 pm

chrs,

you can get a Murus test build here:

http://www.murusfirewall.com/downloads/murus-177.zip

I've added a checkbox in NAT window: "Clamp mss". Check it to add the scrub rule for WAN interface. MSS value is 1440 by default and can't be changed.
Please let me know if it works :) Thanks so much!!!

hany
Posts: 480
Joined: Wed Dec 10, 2014 5:20 pm

Re: NAT trouble with some Websites

Post by hany » Sat Apr 23, 2016 8:43 pm

chrs,

I've added the option to change the MSS value. However 1440 is still the default value when the option is checked.
http://www.murusfirewall.com/downloads/murus-178.zip

I'll add some info in the NAT section of Murus manual for PPPoE users.
Let me know if something is missing.

Thanks a lot for your help.

Post Reply