site not blocking when it should be

Comments

34 comments

  • Avatar
    methom90wh

    Further to this if I just go to xhamster.com (without the www) then I also proceed unblocked.

  • Avatar
    rotblitz

    How did you block this site, via category or by individually blocking?  If the latter, and you have www.xhamster.com in your "always block" list, it will not block m.xhamster.com, of course.  You had to have xhamster.com in your blacklist to make it work for all subdomains, not just www.

    Also, did you flush your caches after settings changes?  Or do you use IPv6 connectivity over the internet?

    If it isn't any of those, then post the complete plain text output of the following diagnostic commands here:

       nslookup -type=txt debug.opendns.com. 
       nslookup www.exampleadultsite.com
       nslookup www.xhamster.com.
       nslookup m.xhamster.com.

    "I'm on a static IP, using PFSense firewall in front of everyone and blocking port 53 requests"

    This is all irrelevant for your issue.  Where do you have the OpenDNS resolver addresses configured?  Did you ensure to use OpenDNS resolver addresses only, not any others, or leaving DNS server fields empty?

  • Avatar
    methom90wh

    Thanks for yu response rotblitz.

    I tried adding xhamster.com and m.xhamster.com to my always block list but they are still coming through.  I did fluch the dns entries on the PC I was using and restarted the dnsmasq service in pfsense. The OpenDNS resolver addresses are stored in PFsense and are the only servers that are listed (the other 2 fields are empty).

    Here are the results from the nslookups:

    C:\Users\Matt>nslookup -type=txt debug.opendns.com
    Server:  fw01.localdomain
    Address:  192.168.61.1

    Non-authoritative answer:
    debug.opendns.com       text =

            "server 7.syd"
    debug.opendns.com       text =

            "flags 20 0 2F6 D00FF00300814C3"
    debug.opendns.com       text =

            "originid 18762691"
    debug.opendns.com       text =

            "actype 2"
    debug.opendns.com       text =

            "bundle 5491861"
    debug.opendns.com       text =

            "source 222.154.235.3:59259"

    C:\Users\Matt>nslookup www.playboy.com
    Server:  fw01.localdomain
    Address:  192.168.61.1

    Non-authoritative answer:
    Name:    www.playboy.com
    Addresses:  67.215.65.130
              67.215.65.130


    C:\Users\Matt>nslookup www.xhamster.com
    Server:  fw01.localdomain
    Address:  192.168.61.1

    Non-authoritative answer:
    Name:    www.xhamster.com
    Addresses:  67.215.65.131
              67.215.65.131


    C:\Users\Matt>nslookup m.xhamster.com
    Server:  fw01.localdomain
    Address:  192.168.61.1

    Non-authoritative answer:
    Name:    m.xhamster.com
    Addresses:  67.215.65.131
              67.215.65.131

     

  • Avatar
    rotblitz

    You're using OpenDNS, data centre Sydney, and your IP address 222.154.235.3 is registered with OpenDNS network ID 18762691.  You have configured the OpenDNS resolver addresses on a device fw01.localdomain [192.168.61.1].

    "The OpenDNS resolver addresses are stored in PFsense and are the only servers that are listed (the other 2 fields are empty)."

    So fill these two other fields with 208.67.222.220 and 208.67.220.222.  Else you will be using OpenDNS randomly only.

    One of the commands was "nslookup www.exampleadultsite.com.", but not "nslookup www.playboy.com".  The site www.exampleadultsite.com really exists and is owned by OpenDNS for testing purposes...

    Well, www.playboy.com is being blocked by category (returned IP 67.215.65.130 for hit-adult.opendns.com), whereas www.xhamster.com and m.xhamster.com are being blocked individually (returned IP 67.215.65.131 for hit-block.opendns.com).  You can remove all xhamster entries from your "always block" list, because they would be blocked nevertheless by category.

    Are you still able to visit xhamster.com and m.xhamster.com?

    Does ping return the real IP addresses of these domains?

       ping xhamster.com 
       ping m.xhamster.com

    Then you didn't correctly flush both, your local resolver cache, also on PFSense, and your browser cache, or the browser being used does not use your system settings, but somehow circumvents OpenDNS.  What browser are you using?

  • Avatar
    methom90wh

    I've added the two new DNS servers. no change.

    C:\Users\Matt>nslookup www.exampleadultsite.com
    Server:  fw01.localdomain
    Address:  192.168.61.1

    Non-authoritative answer:
    Name:    www.exampleadultsite.com
    Addresses:  67.215.65.130
              67.215.65.130

     

    The pings are below.  It's not a local caching issue because I can navigate to new pages and they load fine.

     

    C:\Users\Matt>ping www.xhamster.com

    Pinging www.xhamster.com [67.215.65.131] with 32 bytes of data:
    Reply from 67.215.65.131: bytes=32 time=76ms TTL=54
    Reply from 67.215.65.131: bytes=32 time=76ms TTL=54
    Reply from 67.215.65.131: bytes=32 time=76ms TTL=54
    Reply from 67.215.65.131: bytes=32 time=74ms TTL=54

    Ping statistics for 67.215.65.131:
        Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
    Approximate round trip times in milli-seconds:
        Minimum = 74ms, Maximum = 76ms, Average = 75ms

    C:\Users\Matt>ping m.xhamster.com

    Pinging m.xhamster.com [67.215.65.131] with 32 bytes of data:
    Reply from 67.215.65.131: bytes=32 time=77ms TTL=54
    Reply from 67.215.65.131: bytes=32 time=79ms TTL=54
    Reply from 67.215.65.131: bytes=32 time=76ms TTL=54
    Reply from 67.215.65.131: bytes=32 time=85ms TTL=54

    Ping statistics for 67.215.65.131:
        Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
    Approximate round trip times in milli-seconds:
        Minimum = 76ms, Maximum = 85ms, Average = 79ms

    C:\Users\Matt>ping www.xhamster.com

    Pinging www.xhamster.com [67.215.65.131] with 32 bytes of data:
    Reply from 67.215.65.131: bytes=32 time=84ms TTL=54
    Reply from 67.215.65.131: bytes=32 time=106ms TTL=54
    Reply from 67.215.65.131: bytes=32 time=132ms TTL=54
    Reply from 67.215.65.131: bytes=32 time=135ms TTL=54

    Ping statistics for 67.215.65.131:
        Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
    Approximate round trip times in milli-seconds:
        Minimum = 84ms, Maximum = 135ms, Average = 114ms


    I'm using firefox but I just tried chrome and get the same issue.  The users can't circumvent OpenDNS because there is no way to bypass PFSense and still connect to the internet.

  • Avatar
    Alexander Harrison

    From the sounds of the above, since xhamster.com appears to have been visited before the blocks were in place, the most likely culprit of the lack of blocking is cached non-blocked DNS entries and browser cache data. 

    We'd also recommend, as mentioned above, that if there are multiple DNS server addresses to fill each one with OpenDNS addresses: 208.67.220.220 and 208.67.222.222 are the main two, and 208.67.220.222 and 208.67.222.220 are two additional for a 3rd and 4th slot. 

  • Avatar
    methom90wh

    Hi Alexander.  The category blocks for xhamster would of been in place when I first setup OpenDNS on my network months ago.

    I only noticed m.xhamster.com because I was looking at my sons phone browser history. 

    I added the domains to always block to see if that would help.

    I'm not sure why the xhamster domain is being so differcult.  I also added reddit.com and imgur.com to my always block list and after 3 mins they are blocking correctly.

  • Avatar
    rotblitz

    From your outputs everything is perfect and blocking should take effect.  OpenDNS doesn't return the real IP addresses for these sites, but their own ones which would redirect to the block page.  There's nothing more OpenDNS could do for you.

    That said, if you can still visit these sites, then your browsers disregard the system (computer and PFSense) settings.  Do you have a proxy configured in some way?  Or use browser-addons which use proxy technology?  Or do you use an internal proxy server or VPN technology?  These would be good reasons why your browsers circumvent your OpenDNS settings.  What message does http://welcome.opendns.com/ show up with?

    "blocking port 53 requests that aren't directed at the PFSense interface...  The users can't circumvent OpenDNS because there is no way to bypass PFSense and still connect to the internet."

    This is what you think.  If there's some form of proxy or VPN in use, it is still possible to circumvent OpenDNS, despite your port 53 blocking.

  • Avatar
    methom90wh

    Hi.  There is no proxy or VPN in play.  A linux box which has been off for months shows the same issue.  Interestingly part of the m.xhamster.com page is being blocked by OpenDNS (syndication.exoclick.com).

    I agree that the nslookup and ping commands show that openDNS is being used and is returning the right information.

    I just can't figure out why this one site is not being blocked.  As I said imgur and reddit were added about an hour ago and they are being blocked with no issue on the same PC's.  I could understand it better if OpenDNS wasn't being used at all but it seems that for this one domain something is causing an issue for me.

  • Avatar
    Alexander Harrison

    Is there any chance that the phone visiting m.xhamster.com was visited over the cellular network which would have been an unfiltered request?

  • Avatar
    methom90wh

    No data allowed over the cell network but it'spossible they got to m.xhamster.com from another wireless site (maybe a friends house) that hasn't blocked it.

    I still can't figure out why PC's that have never been there before are showing the same issue.

  • Avatar
    Alexander Harrison

    Based on your account, any requests that are making it to OpenDNS from your registered IP have been filtered. Somehow, some requests aren't making it through to OpenDNS, or aren't leaving your network from the IP address registered to your Dashboard. Based on the test in your earlier reply, that lookup did report that it was associated with your account. 

    A way to try and diagnose the issue is to run the following nslookup command across the computers that aren't working and see if any report a originid that is correct (18762691): nslookup -type=txt debug.opendns.com. An in-browser test is to visit http://welcome.opendns.com

  • Avatar
    methom90wh

    The output from that nslookup shows the correct originID and the welcome page looks fine.

     

  • Avatar
    Alexander Harrison

    All these indications lead to the setup working correctly. Next time it's not working right, follow up with the results of a diagnostic test with the instructions from https://support.opendns.com/entries/21841580 if nslookup -type=txt debug.opendns.com shows some incorrect information. The key to tracking down the issue would be to catch it when it isn't working. 

    I'd also confirm that the filtering is working in Incognito/Private browsing if the filtering isn't working in the browser. There is a chance a browser extension is being used to bypass OpenDNS like the near-VPN ZenMate. Incognito mode disables all addons so it can be used as a test. 

  • Avatar
    methom90wh

    See results at https://opendnsupdate.appspot.com/d/6157300683243520

    All the tests I've done so far have mostly been in private mode.

    I'm pretty confident of the browser setup.  This issue is happening on my own PC as well as a base install linux box.

    I also allowed all port 53 traffic on the LAN and logged it.  Everything on my PC is going to the PFSense interface and not a 3rd party server.

  • Avatar
    Alexander Harrison

    Everything does appear to be configured correctly and m.xhamster.com is being blocked when using the default DNS servers. Direct access to OpenDNS (208.67.222.222) on port 53 is being blocked; however, a nslookup to a third party DNS provider Level3 (4.2.2.1) was able to complete successfully and return the IP for m.xhamster.com. 

    Results for: nslookup m.xhamster.com. 4.2.2.1
    stdout:
    Server:  a.resolvers.level3.net
    Address:  4.2.2.1
    Name:    m.xhamster.com
    Addresses:  2a02:b48:4000:1::4248
    	  2a02:b48:4000:1::4247
    	  2a02:b48:4000:1::4246
    	  2a02:b48:4000:1::4249
    	  88.208.24.59
    	  88.208.24.58
    	  88.208.24.56
    	  88.208.24.57
    

     This indicates that it may be possible to use a different DNS server manually configured on a device and that the firewall isn't blocking other DNS providers like its expected to. You did say you opened up port 53 - and you should see this allowed DNS request for m.xhamster.com resolving unblocked to 4.2.2.1. 

  • Avatar
    methom90wh

    See results at https://opendnsupdate.appspot.com/d/5981648734650368

    I did a second test.  Not sure why direct access to 208.67.222.222 would be blocked.  Maybe I opened 53 up after I started the test.

    Looking now the fw hasn't blocked any traffic.

    Once I have this sorted I'll lock down 53 again to prevent people from using a 3rd party DNS.

  • Avatar
    methom90wh

    I haven't got to the bottom of this yet but it is related to my ISP.  Via my normal ISP I have the problem but when I used my phone to provide internet to my laptop OpenDNS blocked all 3 sites as expected.

    I probably won't get a chance to fix this until November as I'm just about to go on holiday.

    Thanks for all the help!

  • Avatar
    rotblitz

    "I haven't got to the bottom of this yet but it is related to my ISP."

    There may be a mismatch between your IP address used to send DNS traffic and your IP address used to send HTTP traffic.

    Your DNS IP address:    nslookup myip.opendns.com.
    Your web IP address:     http://myip.dnsomatic.com/

    Are those different?

    Or is your ISP using a proxy or cache or NAT? 
    http://www.lagado.com/proxy-test 
    http://www.lagado.com/tools/cache-test

  • Avatar
    Alexander Harrison

    "Via my normal ISP I have the problem but when I used my phone to provide internet to my laptop OpenDNS blocked all 3 sites as expected."

    Chances are, your IP address registered to your account is the Phone IP address rather than the ISP Internet IP address. Your current registered IP is 222.154.X.X. Is this what you see when visiting http://myip.dnsomatic.com from your ISP connection? If not, then your Phone IP is registered to your account. Since you don't have an active Updater client, this would result in only your phone connection being filtered and your ISP connection isn't being touched. 

    The other possibility is the one with the ISP issue that has been discussed above, which would take some more work to resolve. 

  • Avatar
    methom90wh

    "Chances are, your IP address registered to your account is the Phone IP address rather than the ISP Internet IP address."

    Nah, It is my usual internet provider.  When I tried the domains with my phone instead of the OpenDNS block page it went to the mobile companys main page but in the URL I could see the category tags that the www site would of been taged with.

     

     

     

  • Avatar
    methom90wh

    Thanks for your help everyone.

    My phone is with Vodafone in NZ and Telecom provide the main house line.  When I go to the domain on my phone it's being redirected to the main Vodafone.co.nz www page and in the URL I can see the category tags that would be associated with the domain.  I did change the phone connection to the OpenDNS IPs so maybe Vodafone subscribe.

    Telecom provide Internet and phone to the house over ADSL.  I rang them yesterday to ask about any issues with caching and they tried to say it's probably an issue with my ADSL-modem and said they doubt it's an issue with any upstream cache.

     

    So perhaps 2 issues.

    1) dnsomatic.com returned a different IP to my Telecom static IP.  222.153.122.196 from dnsomatic and 222.154.235.3 from the nslookup.  The 2nd IP address here is the static one assigned to my house.  If I reload this page a few times then I get the 2nd IP address.

    2) It looks like I failed the cache-test.  I got the same ID from the page after clearing my cache, I also got the same ID again when I tried it from a different PC that is attached directly to the ADSL modem whereas my laptop is behind PFSense.

     

  • Avatar
    rotblitz

    Yes, the trend with possibly different web and DNS IP addresses and the cache test confirm that your ISP is using caching technology.  This is well known for NZ and has been discussed a lot in the old forums https://forums.opendns.com/ a lot already.  Because NZ is pretty isolated from the rest of the world from a network perspective, it would be hard and slow to surf the internet without this caching technology.

    That said, it is the IP address returned from "nslookup myip.opendns.com." which must be registered at https://dashboard.opendns.com/settings/ for your content filtering and the stats taking effect.  And it is the IP address from http://myip.dnsomatic.com/ which must be registered for your customization of block pages taking effect.

    Because these are different, you have the choice to e.g. not to use customization of block pages or to manage a second OpenDNS network, one with your DNS IP address and one with your web IP address.

     

    This is the best bet you have, and you are still not be able to consistently see the results you expect from your use of OpenDNS.  This is because of your ISP's caching.  For example, if you have example.com blocked and your browser requests it, then your ISP analyses this, and if they just have example.com in their cache by chance, they'll present it to you without loading the content from the original site example.com and regardless of the DNS query result, and your OpenDNS settings are simply disregarded in this case.

    To go back to your original problem, this is certainly the case with m.xhamster.com too where many of your ISP's customers may visit it with their smartphones, so it's almost in your ISP's cache.

    You may ask your ISP if there is a possibility to opt out from caching.  Please note that this may significantly slow down your internet speed, but you would be able to use OpenDNS consistently then.

  • Avatar
    methom90wh

    Is there anything I can do inside my LAN to work around this ?

    I agree that removing myself from the cache might be worse.

  • Avatar
    rotblitz

    No, you can't do anything inside your LAN, because this is not where the problem is.

  • Avatar
    rotblitz

    Ah wait, it depends on what you want to achieve within your LAN.  You could use the local hosts file on each device to block domains as you want, in addition to OpenDNS.  The hosts file has priority over DNS.

    https://startpage.com/do/search?q=hosts+file+block

    You can also add domains to the blacklist in most browsers.  Or you could run an internal DNS server or proxy server to filter content yourself.

  • Avatar
    methom90wh

    I've found a fix for the domain in question.

    PFSense has a DNS forwarder which allows you to resolve "local" hostnames before going out the OpenDNS servers.  There are options to override a domain and send it to a different DNS server.  I've used this option and put in a fake 192.168.x.x IP address.  This causes any traffic to that domain to timeout.

    While not perfect at least I have a way to plug any holes I find.

    Thanks again for all your help.

  • Avatar
    methom90wh

    Would DNSCcrypt help my situation at all ?

    Would that be enough to side step my ISP intercepting requests.

  • Avatar
    rotblitz

    No, as I said, your ISP doesn't care about your DNS queries, no matter if encrypted or not.  They look after the URL to see if they have that in the cache and disregard any DNS in this case.  Only if its not in the cache, they load it from the original site, using your DNS query result.  That said, the interception is not at the DNS level, but at the HTTP/HTTPS level.  And exactly this is the problem.

  • Avatar
    zamanoof

    i hop solve the problem of xhamster.com bcs many of users or maybe all users , u can check the link they comment the website was work https://domain.opendns.com/xhamster.com 

    regards :(

Please sign in to leave a comment.