|
|
|
|
Snort Forums Archive
Archive Home » Rules » Rules file gzip changed?
Please note that the categories listed below represent an archived version of our forums pages. To view the current version and be able to post and reply to threads, please register and login here to go to the full forums pages.
[ Notice: Full Version of This Topic ]
Rules file gzip changed?
Posted by NetWhiz on March 10, 2005 10:09:15
I used to use Perl IO::Socket::INET sockets to retrieve the rulesets. Now, using:
http://www.snort.org/pub-bin/oinkmaster.cgi//snortrules-snapshot-2.3.tar.gz
The file is downloaded, but it is invalid. Nothing has changed with the code except for the new tarball
file. When using the above URL in a browser or wget, the file is downloaded fine. Using Perl, opening
the socket and then reading in the file:
while (<$sock>) {
print TARFILE $_ unless $header;
$header = 0 if /^\r?\n$/;
}
makes a file that is a little larger and with a lot more "lines":
Correct file: 180348 10 Mar 13:54 snortrules-snapshot-2.3.tar.gz
Perl file: 180537 Mar 10 14:50 snortrules-snapshot-2.3.tar.gz
This is very strange as it should work w/o issue. Has something changed with the file to cause this?
Does anyone have a way using only Perl to get the file? wget is not an option for this situation.
Thanks,
NetWhiz |
|
Posted by novowels on March 12, 2005 09:30:15
I'm guessing it is a problem with the $header stuff. You should be downloading a proper .tar.gz without any headers from that url so no skips or conversions are required.
Look at the oinkmaster sources for how they do it with LWP.
From http://cvs.sourceforge.net/viewcvs.py/oinkmaster/oinkmaster/oinkmaster.pl?view=markup
my $ua = LWP::UserAgent->new();
$ua->env_proxy;
my $request = HTTP::Request->new(GET => $url);
my $response = $ua->request($request, $localfile);
|
|
Posted by mwright on March 12, 2005 18:14:35
I just downloaded snortrules-snapshot-2.2.tar.gz and snortrules-snapshot-CURRENT.tar.gz and got gzip errors on both. MD5 checksums for both files are not the same as what is reported on the website.
Both files reported the same MD5 checksum:
MD5 (snortrules-snapshot-CURRENT.tar.gz) = 283e34815b7b425efd1192717898f3ac
MD5 (snortrules-snapshot-2.2.tar.gz) = 283e34815b7b425efd1192717898f3ac |
|
Posted by icekicker on March 13, 2005 19:47:56
Yeah im not unable to grab the rules either. It says they are not in gzip format. And oinkmaster is saying they failed the integrity check. |
|
Posted by NetWhiz on March 18, 2005 12:48:28
LWP is not an option right now since it is not installed by default on this distro. The raw socket get should work w/o issue. It worked before they revamped their site, just not after. And yes, headers were always an issue and easy enough to get around. Something is just wrong here. wget can get it but a raw socket retrieve does nto work.
NetWhiz |
|
Posted by Gesp on April 03, 2005 13:48:09
Having the same problem, I have solved with
my $x = index($buf, "\r\n\037\213"); # \037\213 is .gz beginning
my $split_length = 2;
if ($x == -1) {
$errormessage = "$Lang::tr{'invalid loaded file'} 1";
return 0;
} else {
$buf = substr($buf,$x + $split_length);
}
$x = index($buf, "\000\r\n\060\r\n"); # end
if ($x == -1) {
$errormessage = "$Lang::tr{'invalid loaded file'} 2";
return 0;
} else {
$split_length = 1;
$buf = substr($buf,0,$x + $split_length);
}
$buf =~ s/\r\n\w{2,}\r\n//g; # zap some more stuff
Tested to work with 2.2 and 2.3 actual rules but is experimental as I don't know what are these supplementary datas I had to remove to retrieve the loaded file. |
|
Posted by Gesp on April 16, 2005 04:53:28
The answer is that supplemental data should be caused by Transfer-Encoding: chunked
To retrieve md5
$md5buf =~ /(.?\r\n\w{1,}\r\n)(\w+)(\W.)/; # zap headers
and it is in $2
to remove chunk lenght mark for the rules file
$buf =~ s/\r\n\w{1,}\r\n//g; # zap chunk lenght
|
|
Posted by NetWhiz on May 15, 2005 07:27:05
Can you either post or send me what you got working? I have tried your code additions to get the file out
of the stream, but it keeps failing whenn it gets to the "my $x = index($buffer, "\r\n\037\213");" line b/c
$x then has a value of -1;
Thanks,
NetWhiz |
|
Posted by NetWhiz on May 15, 2005 07:55:47
Nevermind .. worked it out. Thhank you for your help with this.
NetWhiz |
|
|
|
|
|
