Lengthy HOSTS file or black-hole list

Comments

15 comments

  • Avatar
    Anthony Honciano

    With the Umbrella subscription, we can assist you with importing a list of domains to your dashboard. In general, OpenDNS is designed for providing security and content filtering to your networks. We do offer a category for blocking sites that are marked as adware, as well as a list of other categories at http://domain.opendns.com. If the site is accessed via a domain name, then our system will block it for you.

    We do offer a full 14 day free trial at http://www.opendns.com/enterprise-security/packages-and-pricing, you will be able to use everything that Umbrella has to offer with no obligation. You will be able to test this on your network along with other features such as iOS security and the Umbrella Roaming Client.

    Feel free to send us a message at support@opendns.com if you have any questions with setting up the service. We also have our account specialists team at sales@opendns.com, they will also be able to help you with any questions you have about the service.

    Best regards, 

     

    0
    Comment actions Permalink
  • Avatar
    rotblitz

    "I am willing to upgrade or pay for a way to easily submit a list of domains that I need to blacklist."

    It's free: http://community.opendns.com/domaintagging/submit/
    Submit 1000 at a time.  Clearly, it takes some time to get all voted on and approved by the community.

    "This ad blocking is the elephant in the room here at OpenDNS."

    If you want to significantly slow down your surfing experience, then you go for ad blocking by an external service like OpenDNS, else you do ad blocking locally which speeds up your surfing experience.

    "For all I know, another proxy server might work better..."

    Yep, you say it !

    0
    Comment actions Permalink
  • Avatar
    uruiamme

    Thanks for the link to the submission form. Worth a shot.

    Well, the choices for proxy servers are a bit sparse nowadays. Not that there was ever a ton of them back 10 years ago or so. Speed versus complexity is the decision. I could implement ANYTHING here myself, but I just have a lot of other things I could be doing lately. I was hoping that one of those categories offered by OpenDNS was "ads" and not "adware." In a quick spurt of holiday computer work, I cobbled together a hosts file-like block list using my local DNS server. But that is pretty limited in scope and just serves as a test for now.

    I am not entirely sure that the OpenDNS system will be noticeably slower than doing something locally. The proxy would have to run on something, and while the same thing that runs DNS could run the proxy, I am a bit leery of proxies. There is a way to use the "wpad" server which I think I implemented about 10 years ago... but I remember more about manually entering in proxy servers on a LOT of boxes. I think newer browsers are better adapted to this, so it might work by just adding a CNAME to my DNS server and running the wpad proxy script... but I don't relish the idea. https://en.wikipedia.org/wiki/Web_Proxy_Autodiscovery_Protocol

    Specifically, rotblitz, I think your comment about ads being blocked by OpenDNS would  "significantly slow down your surfing experience" is a very inaccurate statement. I know you answer a lot of questions here and you have quite a bit of experience, but really, who would think that DNS lookups by OpenDNS are anything but fast? (and still be using this service?) My network sends dozens of requests up the wire per minute to OpenDNS right now, so where was my slowdown on any of that? The speed of a blocked lookup to a DNS resolver is essentially faster than a recursive lookup in which there is no glue, for example, and the client has to ask for multiple RRs in succession until the name is resolved. Sorry if I got over your head, but your statement is technically wrong. A blocked or filtered domain name via OpenDNS would be fast enough. Will it be a little faster or slower than my Intel Core i7 server? Or my Pentium Pro server (now retired?) It will probably vary, but something tells me that it won't affect me as much as the headaches of one more app to configure and maintain. So what will affect my surfing experience more is messing with local filters.

    0
    Comment actions Permalink
  • Avatar
    uruiamme

    Hey Anthony,

    You gave me this link, and of all things, the first domain up for voting was in the "Advertising" category.

    http://community.opendns.com/domaintagging/

    1. So I will ask again, this time from one IT guy to another. Which product will allow me to select the "Advertising" category as a group of domains that I can block?

    2. And the other question is this: for that product, can I submit a large list of domains that I can use as part of my custom black list?


    Here is the list of 59 categories that I can select with the free account.

    1. Academic Fraud
    2. Adult Themes
    3. Adware
    4. Alcohol
    5. Anime/Manga/Webcomic
    6. Auctions
    7. Automotive
    8. Blogs
    9. Business Services
    10. Chat
    11. Classifieds
    12. Dating
    13. Drugs
    14. Ecommerce/Shopping
    15. Educational Institutions
    16. File Storage
    17. Financial Institutions
    18. Forums/Message boards
    19. Gambling
    20. Games
    21. German Youth Protection
    22. Government
    23. Hate/Discrimination
    24. Health and Fitness
    25. Humor
    26. Instant Messaging
    27. Jobs/Employment
    28. Lingerie/Bikini
    29. Movies
    30. Music
    31. News/Media
    32. Non-Profits
    33. Nudity
    34. P2P/File sharing
    35. Parked Domains
    36. Photo Sharing
    37. Podcasts
    38. Politics
    39. Pornography
    40. Portals
    41. Proxy/Anonymizer
    42. Radio
    43. Religious
    44. Research/Reference
    45. Search Engines
    46. Sexuality
    47. Social Networking
    48. Software/Technology
    49. Sports
    50. Tasteless
    51. Television
    52. Tobacco
    53. Travel
    54. Typo Squatting
    55. Video Sharing
    56. Visual Search Engines
    57. Weapons
    58. Web Spam
    59. Webmail


    Above, I highlighted the ones that do not appear on the list over at http://community.opendns.com/domaintagging/ (there are 57). Now below, here is the list of categories that appear at domain tagging, but are not available to select on the free account.

    1. Advertising

    Bingo. Exactly. This is the elephant in the room, Anthony. "Advertising" is the only category not listed at the free control panel and being submitted/voted upon by the community. Is all of the community work is going to waste on this category? Or do you offer this as a category to block using Umbrella? Did anyone ever realize that the community can tag a domain as "Advertising" but this category cannot be selected in the control panel? And that you cannot select "Adware" in the dropdown menu at http://community.opendns.com/domaintagging/submit/ ? But "Adware" is available at the control panel for free accounts?

    My guess is that there is some internal review that tries to distinguish between these two categories... surely you guys know the difference. And maybe you figure that people in the community don't know, so you just allow them to choose "Advertising" for domains that will eventually be split up internally?

    0
    Comment actions Permalink
  • Avatar
    rotblitz

    "I think your comment about ads being blocked by OpenDNS would  "significantly slow down your surfing experience" is a very inaccurate statement."

    Not convinced yet?  This will convince you: http://forums.opendns.com/comments.php?DiscussionID=7784

    "My network sends dozens of requests up the wire per minute to OpenDNS right now, so where was my slowdown on any of that?"

    You missed the point.  It's not about normal DNS traffic, it's about the blocking mechanism.

    0
    Comment actions Permalink
  • Avatar
    uruiamme

    rotblitz, thanks for playing, but you are wrong again. The page you linked to is simply wrong about how a potential ad block would work. Sit back and learn something.

    First off, the servers used by OpenDNS, and presumably their engineers, have some rudimentary clue as to how blocking should work. Because when I do a simple test of the erroneous and lengthy hypothetical ad blocking ... and I will quote: "Think about what happens if they have 5 or 12 ads embedded. :shocked:" I do not get the predicted results.

    Let's do this the tried-and-tested way: a real example, and not a hypothetical.

    1. Configure OpenDNS to block porn websites.
    2. Type in http://www.porntube.com/junk.html into a browser window.
    3. Observe the error message delivered to your browser, using redirects and such. Oh noes!
    4. Now type in http://www.porntube.com/junk.jpg into a browser window.
    5. Observe that in this case, a simple and effective "404 - Not Found" error message shows up.
    6. Congrats. You have just proved wrong the theory that using OpenDNS for ad blocking will "significantly slow down your surfing experience."

    Now, I am not sure why the OpenDNS folks stopped at a few file extensions for this behavior. I noticed that an FLV and MP3 extension brought up the redirect HTML page but GIF and JPG delivered the 404. But I don't see why I can't implement a simple black hole of blocked-website.com in my local DNS server. It would take a few seconds to implement, and those requests get routed to the trash heap. It doesn't take much to keep running my DNS server, which is just doing recursive lookups to openDNS anyway for most things.

    So to answer your question: no. I am not convinced yet. The blocking mechanism looks mediocre, but it doesn't appear that it will be a dramatic change in performance if used for ad blocking. I could run it up the flagpole and see who salutes I guess... meaning I could collate a bunch of ad servers on a bloated media/tv/portal website and configure them into my personal OpenDNS account as always blocks... visit the site, and see what happens when I surf it in a pristine browser.

    0
    Comment actions Permalink
  • Avatar
    rotblitz

    "I will quote: "Think about what happens if they have 5 or 12 ads embedded. :shocked:" I do not get the predicted results."

    You're right,  Some of this may have been cached locally, so DNS lookups may not be needed for every repeating instance, but the more important load of ad content will still take place.

    "Let's do this the tried-and-tested way: a real example"

    I took my results from measuring it with DNS query sniffer and Fiddler2.  But you're right, my results are nearly 4 years old.  They significantly changed the blocking mechanism in between to introduce even more lookups and HTTP 301/302 webhop redirects, so there is even more traffic nowadays.  And yes, they differentiate the blocking also by object type in between, in your example assuming that a GIF or JPEG image isn't called alone, but possibly within a page loaded by a domain which may or may not be blocked.

    I may post newer and better examples as I find time to collect and compile the results.

    "I could collate a bunch of ad servers on a bloated media/tv/portal website and configure them into my personal OpenDNS account as always blocks... visit the site, and see what happens when I surf it in a pristine browser."

    A good idea!  But hard to measure this way without certain tools.  A stopwatch may not be sufficient.

    0
    Comment actions Permalink
  • Avatar
    uruiamme

    So Anthony, do the Umbrella offerings permit me to utilize the "Advertising" category to be blocked? And can I pick and choose other sites, submitting my custom list for the groups I specify?

    0
    Comment actions Permalink
  • Avatar
    Anthony Honciano

    Hi there,

    The advertisement category is not available in the filter options, but you do have the ability to blacklist any domain you want. If you have a large list of domains, we can port those for you into a separate blacklist and whitelists, which can be used on multiple networks via our policy system.

    Feel free to visit http://www.opendns.com/enterprise-security/packages-and-pricing to test drive the Umbrella Dashboard, and let us know if you have any questions.

    0
    Comment actions Permalink
  • Avatar
    uruiamme

    According to the boss man at http://blog.opendns.com/2014/05/29/no-more-ads/ and other news outlets, advertising and security make strange bedfellows. Can I recommend you re-evaluate this and see if Umbrella could be permit users to choose the advertising category? If security is my aim (and it is certainly one of them for my own network), blocking ads that are typical vectors for unwanted security problems would almost certainly be good business sense for OpenDNS and Umbrella. Should I go out on a limb and tell you that the ones served up by OpenDNS on the ill-fated Guide are already blocked quite easily and have been by those (like me) who care for years? And that while I started this question with the stated goal of bandwidth reduction, security (as stated by David Ulevitch, and probably quoting me) is a good enough reason:

    "There’s also the elephant in the room: ads and security don’t mix. " and "It’s clear to us that they are fundamentally incompatible."

    I note that David said this on May 29, while my comment above is dated May 26. So this was a timely discussion we are having, and I appreciate Mr. Ulevitch's understanding of my bandwidth goals. I would also list another compelling reason that I need to do it: for the same reason I block pornography and nudity (in my case for censorship reasons), I would like to have the ability to block ads at the DNS level and to do so with (what should be) the valuable list of advertising domains. After trying web-based ad blocking with proxy servers for a long time (so easy to bypass and difficult to maintain) and with browser plugins (nice but not all ads are delivered to browsers) and with rudimentary HOSTS files (this dates back to the 20th century, with pitfalls associated with a management and infrastructure nightmare), I am willing to try a DNS solution. There are some hybrid systems out there based on the proxy server/DNS server and firewall appliances out there, but I think a DNS solution can beat them at that game for less money on my part.

    I want to thank the OpenDNS community for classifying the thousands of websites as "Advertising." And while I know that maintaining a proper and realistic list of those servers could get burdensome, I also know that there are community-based systems that have done it for years... and DNS is just one more layer that can utilize that hard work. I have seen it happen with mail server operators blocking spam at essentially the domain name level, at least as far back as 10+ years ago with SMTP plugins. A DNS solution for them would actually skip a step in that combat. So while you are looking at "Advertising" as a catergory, I would recommend also a "Spam" category, and feed that block list to a company's SMTP server. It's the same old RBL (Real-time Block list) idea ratcheted up to the DNS call rather than doing a call to a RBL database hosted on a fake DNS server. I may be talking a bit over your head if you aren't familiar with SMTP and spam blocking, but back in my heyday as a systems administrator I was tasked with blocking both web-based ads and email-based ads. The solutions back then were worlds apart, but a DNS solution can do double duty... kill two birds with one DNS server.

     

    The more I think back to my email-blocking of the early 2000s, I seem to remember some nasty malware that could pop up in email. This was before a lot of people used the Big Email providers that put the ISPs out of business for "free email account." Since Gmail, MSN, Yahoo, AOL, and most large free accounts have had virus-free mail for years, and spam-free mail for (well...) months... :) We may forget that there is still a lot of junk to block out there. So I would say security was a big deal back then. Maybe it only faded after a lot of the automatic virus scanners became entrenched in email? So while I blocked those viruses, I guess it was really just an ad-on to block the Spam. I think I hated the Spam more than the malware, though, but only until someone got a virus. Then it was a mess.

    So ad-blocking, security, and bandwidth reduction. Three important issues for (surely) lots of businesses. I think OpenDNS should advertise these three ideas as goals to achieve a happy corporate customer.

    0
    Comment actions Permalink
  • Avatar
    Anthony Honciano

    The "no more ads" announcement is based on the free home and premium users guide page, that was driven by advertisements. When a user requested a domain that is non existent (NXDOMAIN), our servers would intercept the error and redirect the user to the guide page in attempt to search for the correct domain or an alternate site. Similar to a search results page. This is also true for our block pages, when a user is filtered or blocked from a site, they were redirected to our block pages that were driven by advertisements. Both the Guide page and Block pages will be free of advertisements. Please visit http://www.opendns.com/no-more-ads/ for additional details.

    With this said, OpenDNS does not have a feature to filter advertisement domains, for one of the reasons that some of the sources are hosted on popular CDN's for other sites. You are more than welcome to bring your ideas regarding ad blocking to our Idea Bank at https://support.opendns.com/forums/21211727-Idea-Bank.

    0
    Comment actions Permalink
  • Avatar
    popowsda

    Thanks, @uruiamme, for bringing some sense to the OpenDNS Community Forums.  If you wish to pursue Anthony's suggestion and bring your thoughts to the Idea Bank, please start at https://support.opendns.com/entries/41751214-Enable-Web-Content-Filtering-to-block-Advertising-category.

    0
    Comment actions Permalink
  • Avatar
    popowsda

    Anthony, I want to address your most recent comments above.  It's quite disingenuous for you to say OpenDNS does not permit customers to block the Advertising category via Web Content Filtering because (even if only in part) "some of the [ad] sources are hosted on popular CDN's for other sites."  In fact, some online games also are hosted on popular CDN's, but OpenDNS still allows customers to block the Games category.  And besides, nobody realistically expects to block all ads via the Advertising category.  We're simply requesting to have Advertising presented (alongside Games) as a category for blocking.

    Do you have another (perhaps more credible) explanation for why it's not permitted?  Is the political pressure from the advertising industry too great to let customers block them categorically?

    0
    Comment actions Permalink
  • Avatar
    rotblitz

    "OpenDNS still allows customers to block the Games category."

    But CDN domains should not and do not belong to the Games category.  Or would you have an example at hand?

    "Do you have another (perhaps more credible) explanation for why it's not permitted?"

    I did say it already.  They don't want to have increased support efforts due to the many complaints from people saying their web browsing is so slow.  What else?  Ad blocking is a pure local task, of course.  You don't want any network traffic for something you don't want to see aynway.

    I bet they would allow blocking with category advertisement if you founded a support company catching all their complaints to deal with for free.  Would you?

    0
    Comment actions Permalink
  • Avatar
    Anthony Honciano

    Thank you for your detailed response and idea, Popowsda. 

    At this time, we do not offer a feature to block advertisements. If you haven't already posted this into our Idea Bank, please do so at https://support.opendns.com/forums/21211727-Idea-Bank. Our project managers regularly review and comment on user submissions.

    Thank you.

     

    0
    Comment actions Permalink

Post is closed for comments.