Home > IPDeny > CIDR Zone file merging for higher performance

CIDR Zone file merging for higher performance

UPDATE: AUGUST 15th, 2014. IPv4 country aggregated IP address blocks are now offered for free downloads. Read our official blog post about aggregated IP address blocks and enjoy!

We got an email from Cusimano.com person suggesting us to use CIDR zone file merging script from zwitterion.org. We have ran a few in-house tests and this Perl based script effectively decreases CIDR IP block count up to 60% or even more.

The less firewall rules you have set-up in your packet filter (server or firewall) the less matching needs to be done – this effectively decreases load and usage. In the next upcoming week we will fully implement this feature into freely available country zone files.

Thank you for your continued support and feel free to send us more suggestions or recommendations!

Categories: IPDeny Tags:
  1. T Hackque
    August 2nd, 2010 at 09:57 | #1

    Why not use NetAddr::IP’s built-in compaction? It’s supported, debugged, and effective.

    By chance, I created a wrapper script for it recently; it will read any number of input files (or stdin) and write to stdout. 11 lines of code – and it works on your current zone files.

    Note that if you use more than one country file, it may still be worth running this script over all the files that you use even after ipdeny does merging – there may be additional optimizations possible when countries have adjacent blocks.

    enjoy,

    –tlhackque
    #!/usr/bin/perl

    use strict;
    use warnings;

    use NetAddr::IP;

    my @addresses = ();

    while( ) {
    chomp;
    s/\s*#.*$//;

    next if( !length );

    push @addresses, NetAddr::IP->new( $_ );
    }

    exit unless( @addresses );

    #print STDERR ( (scalar @addresses), ” addresses input, ” );

    @addresses = NetAddr::IP::Compact(@addresses);
    #print STDERR ( (scalar @addresses), ” addresses output\n” );

    print join( “\n”, @addresses ), “\n”;

    exit;

  2. T Hackque
    August 2nd, 2010 at 10:01 | #2

    Umm, your blog software seems to have removed a few crucial characters, let’s ty this:

    #!/usr/bin/perl

    use strict;
    use warnings;

    use NetAddr::IP;

    my @addresses = ();

    while( ) {
    chomp;
    s/\s*#.*$//;

    next if( !length );

    push @addresses, NetAddr::IP->new( $_ );
    }

    exit unless( @addresses );

    #print STDERR ( (scalar @addresses), ” addresses input, ” );

    @addresses = NetAddr::IP::Compact(@addresses);
    #print STDERR ( (scalar @addresses), ” addresses output\n” );

    print join( “\n”, @addresses ), “\n”;

    exit;

  3. T Hackque
    August 2nd, 2010 at 10:05 | #3

    well, that didn’t work. the while has an input operator (less-than greater-than) in it’s condition. I’m not going to fight your software to get it there.

    If the site admin (only) wants a clean, executable and correctly formatted copy, contact me via e-mail.

  4. August 3rd, 2010 at 08:32 | #4

    I have made significant modifications to a script initially posted by Vivek Gite at http://www.cyberciti.biz/faq/block-entier-country-using-iptables to address the performance issues and cache the zone updates from ipdeny for an iptables based firewall. The rationale for the changes and the altered script is available at http://psind.com/blog/2010/07/31/targeted-ip-blocking-align-web-services-to-your-target-markets/

  5. IT_Architect
    December 16th, 2010 at 19:05 | #5

    @T Hackque
    I don’t know Perl. Can I use this script to compress IP addresses by calling it with a file name from the command line or….

    Thanks!

  6. rwilcher
    September 16th, 2014 at 14:40 | #6

    This site supplies insane goodness to the net.
    Please keep it up. My attack perimeter has shrunk big time.