Contents

Extending Snort

Plugins

Plugins have an associated API defined for each type, all of which share a common header, called the BaseApi. A dynamic library makes its plugins available by exporting the snort_plugins symbol, which is a null terminated array of BaseApi pointers.

The BaseApi includes type, name, API version, plugin version, and function pointers for constructing and destructing a Module. The specific API add various other data and functions for their given roles.

Modules

If we are defining a new Inspector called, say, gadget, it might be configured in snort.lua like this:

gadget =
{
    brain = true,
    claw = 3
}

When the gadget table is processed, Snort will look for a module called gadget. If that Module has an associated API, it will be used to configure a new instance of the plugin. In this case, a GadgetModule would be instantiated, brain and claw would be set, and the Module instance would be passed to the GadgetInspector constructor.

Module has three key virtual methods:

  • begin() - called when Snort starts processing the associated Lua table. This is a good place to allocate any required data and set defaults.

  • set() - called to set each parameter after validation.

  • end() - called when Snort finishes processing the associated Lua table. This is where additional integrity checks of related parameters should be done.

The configured Module is passed to the plugin constructor which pulls the configuration data from the Module. For non-trivial configurations, the working paradigm is that Module hands a pointer to the configured data to the plugin instance which takes ownership.

Note that there is at most one instance of a given Module, even if multiple plugin instances are created which use that Module. (Multiple instances require Snort binding configuration.)

A module’s usage determines how it may be configured:

  • Global: configured at most once, outside any policy.

  • Context: configured at most once in a network policy, eg event_queue.

  • Inspect: configured in an inspection policy aka network analysis policy (NAP); stream and service inspectors like stream_tcp and smtp are multitons, ie they may be configured more than once in a policy, while others like binder are singletons and can be configured at most once per policy.

  • Detect: configured at most once in an IPS policy, eg ips.

Inspectors

There are several types of inspector, which determines which inspectors are executed when:

  • IT_CONTROL - process all packets before detection

  • IT_FIRST - analyze 1st pkt of new flow and 1st pkt after reload of ongoing flow (eg reputation)

  • IT_NETWORK - processes packets w/o service (e.g. arp_spoof, back_orifice)

  • IT_PACKET - used to process raw packets only (e.g. normalizer)

  • IT_PASSIVE - for configuration only or for handling inspection events (eg ftp_client and binder)

  • IT_PROBE - process all packets after all the detection (e.g. perf_monitor, port_scan)

  • IT_SERVICE - for analyzing PDUs eg http_inspect, ftp_server, telnet, etc.

  • IT_STREAM - for flow tracking, ip defrag, and TCP reassembly; also for processing files directly or TCP payload-only streams

  • IT_WIZARD - determines which service inspector to use if none explicitly bound by matching start-of-flow patterns

Codecs

The Snort Codecs decipher raw packets. These Codecs are now completely pluggable; almost every Snort Codec can be built dynamically and replaced with an alternative, customized Codec. The pluggable nature has also made it easier to build new Codecs for protocols without having to touch the Snort code base.

The first step in creating a Codec is defining its class and protocol. Every Codec must inherit from the Snort Codec class defined in "framework/codec.h". The following is an example Codec named "example" and has an associated struct that is 14 bytes long.

#include <cstdint>
#include <arpa/inet.h>
#include “framework/codec.h”
#include "main/snort_types.h"
#define EX_NAME “example”
#define EX_HELP “example codec help string”
struct Example
{
    uint8_t dst[6];
    uint8_t src[6];
    uint16_t ethertype;
    static inline uint8_t size()
    { return 14; }
}
class ExCodec : public Codec
{
public:
    ExCodec() : Codec(EX_NAME) { }
    ~ExCodec() { }
    bool decode(const RawData&, CodecData&, DecodeData&) override;
    void get_protocol_ids(std::vector<uint16_t>&) override;
};

After defining ExCodec, the next step is adding the Codec’s decode functionality. The function below does this by implementing a valid decode function. The first parameter, which is the RawData struct, provides both a pointer to the raw data that has come from a wire and the length of that raw data. The function takes this information and validates that there are enough bytes for this protocol. If the raw data’s length is less than 14 bytes, the function returns false and Snort discards the packet; the packet is neither inspected nor processed. If the length is greater than 14 bytes, the function populates two fields in the CodecData struct, next_prot_id and lyr_len. The lyr_len field tells Snort the number of bytes that this layer contains. The next_prot_id field provides Snort the value of the next EtherType or IP protocol number.

bool ExCodec::decode(const RawData& raw, CodecData& codec, DecodeData&)
{
   if ( raw.len < Example::size() )
       return false;
    const Example* const ex = reinterpret_cast<const Example*>(raw.data);
    codec.next_prot_id = ntohs(ex->ethertype);
    codec.lyr_len = ex->size();
    return true;
}

For instance, assume this decode function receives the following raw data with a validated length of 32 bytes:

00 11 22 33 44 55 66 77    88 99 aa bb 08 00 45 00
00 38 00 01 00 00 40 06    5c ac 0a 01 02 03 0a 09

The Example struct’s EtherType field is the 13 and 14 bytes. Therefore, this function tells Snort that the next protocol has an EtherType of 0x0800. Additionally, since the lyr_len is set to 14, Snort knows that the next protocol begins 14 bytes after the beginning of this protocol. The Codec with EtherType 0x0800, which happens to be the IPv4 Codec, will receive the following data with a validated length of 18 ( == 32 – 14):

45 00 00 38 00 01 00 00    40 06 5c ac 0a 01 02 03
0a 09

How does Snort know that the IPv4 Codec has an EtherType of 0x0800? The Codec class has a second virtual function named get_protocol_ids(). When implementing the function, a Codec can register for any number of values between 0x0000 - 0xFFFF. Then, if the next_proto_id is set to a value for which this Codec has registered, this Codec’s decode function will be called. As a general note, the protocol ids between [0, 0x00FF] are IP protocol numbers, [0x0100, 0x05FF] are custom types, and [0x0600, 0xFFFF] are EtherTypes.

For example, in the get_protocol_ids function below, the ExCodec registers for the protocols numbers 17, 787, and 2054. 17 happens to be the protocol number for UDP while 2054 is ARP’s EtherType. Therefore, this Codec will now attempt to decode UDP and ARP data. Additionally, if any Codec sets the next_protocol_id to 787, ExCodec’s decode function will be called. Some custom protocols are already defined in the file "protocols/protocol_ids.h"

void ExCodec::get_protocol_ids(std::vector<uint16_t>&v)
{
    v.push_back(0x0011); // == 17  == UDP
    v.push_back(0x1313); // == 787  == custom
    v.push_back(0x0806); // == 2054  == ARP
}

To register a Codec for Data Link Type’s rather than protocols, the function get_data_link_type() can be similarly implemented.

The final step to creating a pluggable Codec is the snort_plugins array. This array is important because when Snort loads a dynamic library, the program only find plugins that are inside the snort_plugins array. In other words, if a plugin has not been added to the snort_plugins array, that plugin will not be loaded into Snort.

Although the details will not be covered in this post, the following code snippet is a basic CodecApi that Snort can load. This snippet can be copied and used with only three minor changes. First, in the function ctor, ExCodec should be replaced with the name of the Codec that is being built. Second, EX_NAME must match the Codec’s name or Snort will be unable to load this Codec. Third, EX_HELP should be replaced with the general description of this Codec. Once this code snippet has been added, ExCodec is ready to be compiled and plugged into Snort.

static Codec* ctor(Module*)
{ return new ExCodec; }
static void dtor(Codec *cd)
{ delete cd; }
static const CodecApi ex_api =
{
    {
        PT_CODEC,
        EX_NAME,
        EX_HELP,
        CDAPI_PLUGIN_V0,
        0,
        nullptr,
        nullptr,
    },
    nullptr, // pointer to a function called during Snort's startup.
    nullptr, // pointer to a function called during Snort's exit.
    nullptr, // pointer to a function called during thread's startup.
    nullptr, // pointer to a function called during thread's destruction.
    ctor, // pointer to the codec constructor.
    dtor, // pointer to the codec destructor.
};
SO_PUBLIC const BaseApi* snort_plugins[] =
{
    &ex_api.base,
    nullptr
};

Two example Codecs are available in the extra directory on git and the extra tarball on the Snort page. One of those examples is the Token Ring Codec while the other example is the PIM Codec.

As a final note, there are four more virtual functions that a Codec should implement: encode, format, update, and log. If the functions are not implemented Snort will not throw any errors. However, Snort may also be unable to accomplish some of its basic functionality.

  • encode is called whenever Snort actively responds and needs to builds a packet, i.e. whenever a rule using an IPS ACTION like react, reject, or rewrite is triggered. This function is used to build the response packet protocol by protocol.

  • format is called when Snort is rebuilding a packet. For instance, every time Snort reassembles a TCP stream or IP fragment, format is called. Generally, this function either swaps any source and destination fields in the protocol or does nothing.

  • update is similar to format in that it is called when Snort is reassembling a packet. Unlike format, this function only sets length fields.

  • log is called when either the log_codecs logger or a custom logger that calls PacketManager::log_protocols is used when running Snort.

IPS Actions

Action plugins specify a builtin action in the API which is used to determine verdict. (Conversely, builtin actions don’t have an associated plugin function.)

Trace Loggers

The Trace Loggers print trace messages. They can be implemented as inspector plugins.

The first step is creating a custom logger by inheriting from the Snort TraceLogger class. The following is an example TraceLogger.

class FooLogger : public TraceLogger
{
public:
    void log(const char*, const char*, uint8_t, const char*, const Packet*) override
    { printf("%s%s\n", "Foo", "Bar"); }
};

To instantiate logger objects it’s needed to create a logger factory derived from the Snort TraceLoggerFactory class.

class FooFactory : public TraceLoggerFactory
{
public:
    TraceLogger* instantiate() override
    { return new FooLogger(); }
};

Once the Factory is created, Inspector and appropriate Module are needed. Inspector::configure() must initialize the logger factory.

bool FooInspector::configure(SnortConfig* sc) override
{
    return TraceApi::override_logger_factory(sc, new FooFactory());
}

Coding Style

All new code should try to follow these style guidelines. These are not yet firm so feedback is welcome to get something we can live with.

General

  • Generally try to follow https://google.github.io/styleguide/cppguide.html, but there are some differences documented here.

  • Each source directory should have a dev_notes.txt file summarizing the key points and design decisions for the code in that directory. These are built into the developers guide.

  • Makefile.am and CMakeLists.txt should have the same files listed in alpha order. This makes it easier to maintain both build systems.

  • All new code must come with unit tests providing 95% coverage or better.

  • Generally, Catch is preferred for tests in the source file and CppUTest is preferred for test executables in a test subdirectory.

C++ Specific

  • Do not use exceptions. Exception-safe code is non-trivial and we have ported legacy code that makes use of exceptions unwise. There are a few exceptions to this rule for the memory manager, shell, etc. Other code should handle errors as errors.

  • Do not use dynamic_cast or RTTI. Although compilers are getting better all the time, there is a time and space cost to this that is easily avoided.

  • Use smart pointers judiciously as they aren’t free. If you would have to roll your own, then use a smart pointer. If you just need a dtor to delete something, write the dtor.

  • Prefer and over && and or over || for new source files.

  • Use nullptr instead of NULL.

  • Use new, delete, and their [] counterparts instead of malloc and free except where realloc must be used. But try not to use realloc. New and delete can’t return nullptr so no need to check. And Snort’s memory manager will ensure that we live within our memory budget.

  • Use references in lieu of pointers wherever possible.

  • Use the order public, protected, private top to bottom in a class declaration.

  • Keep inline functions in a class declaration very brief, preferably just one line. If you need a more complex inline function, move the definition below the class declaration.

  • The goal is to have highly readable class declarations. The user shouldn’t have to sift through implementation details to see what is available to the client.

  • Any using statements in source files should be added only after all includes have been declared.

Naming

  • Use camel case for namespaces, classes, and types like WhizBangPdfChecker.

  • Use lower case identifiers with underscore separators, e.g. some_function() and my_var.

  • Do not start or end variable names with an underscore. This has a good chance of conflicting with macro and/or system definitions.

  • Use lower case filenames with underscores.

Comments

  • Write comments sparingly with a mind towards future proofing. Often the comments can be obviated with better code. Clear code is better than a comment.

  • Heed Tim Ottinger’s Rules on Comments (https://disqus.com/by/tim_ottinger/):

    1. Comments should only say what the code is incapable of saying.

    2. Comments that repeat (or pre-state) what the code is doing must be removed.

    3. If the code CAN say what the comment is saying, it must be changed at least until rule #2 is in force.

  • Function comment blocks are generally just noise that quickly becomes obsolete. If you absolutely must comment on parameters, put each on a separate line along with the comment. That way changing the signature may prompt a change to the comments too.

  • Use FIXIT (not FIXTHIS or TODO or whatever) to mark things left for a day or even just a minute. That way we can find them easily and won’t lose track of them.

  • Presently using FIXIT-X where X is one of the characters below. Place A and W comments on the exact warning line so we can match up comments and build output. Supporting comments can be added above.

  • A = known static analysis issue

  • D = deprecated - code to be removed after users update

  • E = enhancement - next steps for incomplete features (not a bug)

  • H = high priority - urgent deficiency

  • L = low priority - cleanup or similar technical debt (not a bug)

  • M = medium priority - suspected non-urgent deficiency

  • P = performance issue (not a bug)

  • W = warning - known compiler warning

  • Put the copyright(s) and license in a comment block at the top of each source file (.h and .cc). Don’t bother with trivial scripts and make foo. Some interesting Lua code should get a comment block too. Copy and paste exactly from src/main.h (don’t reformat).

  • Put author, description, etc. in separate comment(s) following the license. Do not put such comments in the middle of the license foo. Be sure to put the author line ahead of the header guard to exclude them from the developers guide. Use the following format, and include a mention to the original author if this is derived work:

    // ips_dnp3_obj.cc author Maya Dagon <mdagon@cisco.com>
    // based on work by Ryan Jordan
  • Each header should have a comment immediately after the header guard to give an overview of the file so the reader knows what’s going on.

  • Use the following comment on switch cases that intentionally fall through to the next case to suppress compiler warning on known valid cases:

    // fallthrough

Logging

  • Messages intended for the user should not look like debug messages. Eg, the function name should not be included. It is generally unhelpful to include pointers.

  • Most debug messages should just be deleted.

  • Don’t bang your error messages (no !). The user feels bad enough about the problem already w/o you shouting at him.

Types

  • Use logical types to make the code clearer and to help the compiler catch problems. typedef uint16_t Port; bool foo(Port) is way better than int foo(int port).

  • Use forward declarations (e.g. struct SnortConfig;) instead of void*.

  • Try not to use extern data unless absolutely necessary and then put the extern in an appropriate header. Exceptions for things used in exactly one place like BaseApi pointers.

  • Use const liberally. In most cases, const char* s = "foo" should be const char* const s = "foo". The former goes in the initialized data section and the latter in read only data section.

  • But use const char s[] = "foo" instead of const char* s = "foo" when possible. The latter form allocates a pointer variable and the data while the former allocates only the data.

  • Use static wherever possible to minimize public symbols and eliminate unneeded relocations.

  • Declare functions virtual only in the parent class introducing the function (not in a derived class that is overriding the function). This makes it clear which class introduces the function.

  • Declare functions as override if they are intended to override a function. This makes it possible to find derived implementations that didn’t get updated and therefore won’t get called due a change in the parent signature.

  • Use bool functions instead of int unless there is truly a need for multiple error returns. The C-style use of zero for success and -1 for error is less readable and often leads to messy code that either ignores the various errors anyway or needlessly and ineffectively tries to do something about them. Generally that code is not updated if new errors are added.

Macros (aka defines)

  • In many cases, even in C++, use #define name "value" instead of a const char* const name = "value" because it will eliminate a symbol from the binary.

  • Use inline functions instead of macros where possible (pretty much all cases except where stringification is necessary). Functions offer better typing, avoid re-expansions, and a debugger can break there.

  • All macros except simple const values should be wrapped in () and all args should be wrapped in () too to avoid surprises upon expansion. Example:

    #define SEQ_LT(a,b)  ((int)((a) - (b)) <  0)
  • Multiline macros should be blocked (i.e. inside { }) to avoid if-else type surprises.

Formatting

  • Try to keep all source files under 2500 lines. 3000 is the max allowed. If you need more lines, chances are that the code needs to be refactored.

  • Indent 4 space chars … no tabs!

  • If you need to indent many times, something could be rewritten or restructured to make it clearer. Fewer indents is generally easier to write, easier to read, and overall better code.

  • Braces go on the line immediately following a new scope (function signature, if, else, loop, switch, etc.

  • Use consistent spacing and line breaks. Always indent 4 spaces from the breaking line. Keep lines less than 120 chars; it greatly helps readability.

    No:
        calling_a_func_with_a_long_name(arg1,
                                        arg2,
                                        arg3);
    Yes:
        calling_a_func_with_a_long_name(
            arg1, arg2, arg3);
  • Put function signature on one line, except when breaking for the arg list:

    No:
        inline
        bool foo()
        { // ...
    Yes:
        inline bool foo()
        { // ...
  • Put conditional code on the line following the if so it is easy to break on the conditional block:

    No:
        if ( test ) foo();
    Yes:
        if ( test )
            foo();

Headers

  • Don’t hesitate to create a new header if it is needed. Don’t lump unrelated stuff into an header because it is convenient.

  • Write header guards like this (leading underscores are reserved for system stuff). In my_header.h:

    #ifndef MY_HEADER_H
    #define MY_HEADER_H
    // ...
    #endif
  • Includes from a different directory should specify parent directory. This makes it clear exactly what is included and avoids the primordial soup that results from using -I this -I that -I the_other_thing … .

    // given:
    src/foo/foo.cc
    src/bar/bar.cc
    src/bar/baz.cc
    // in baz.cc
    #include "bar.h"
    // in foo.cc
    #include "bar/bar.h"
  • Includes within installed headers should specify parent directory.

  • Just because it is a #define doesn’t mean it goes in a header. Everything should be scoped as tightly as possible. Shared implementation declarations should go in a separate header from the interface. And so on.

  • All .cc files should include config.h with the standard block shown below immediately following the initial comment blocks and before anything else. This presents a consistent view of all included header files as well as access to any other configure-time definitions. To avoid issues, no .h files should include config.h, even if not installed.

    #ifdef HAVE_CONFIG_H
    #include "config.h"
    #endif
  • A .cc should include its own .h before any others aside from the aforementioned config.h (including system headers). This ensures that the header stands on its own and can be used by clients without include prerequisites and the developer will be the first to find a dependency issue.

  • Split headers included from the local directory into a final block of headers. For a .cc file, the final order of sets of header includes should look like this:

    1. config.h

    2. its own .h file

    3. system headers (.h/.hpp/.hxx)

    4. C++ standard library headers (no file extension)

    5. Snort headers external to the local directory (path-prefixed)

    6. Snort headers in the local directory

  • Include required headers, all required headers, and nothing but required headers. Don’t just clone a bunch of headers because it is convenient.

  • Keep includes in alphabetical order. This makes it easier to maintain, avoid duplicates, etc.

  • Do not put using statements in headers unless they are tightly scoped.

Warnings

  • With g++, use at least these compiler flags:

    -Wall -Wextra -pedantic -Wformat -Wformat-security
    -Wunused-but-set-variable -Wno-deprecated-declarations
    -fsanitize=address -fno-omit-frame-pointer
  • With clang, use at least these compiler flags:

    -Wall -Wextra -pedantic -Wformat -Wformat-security
    -Wno-deprecated-declarations
    -fsanitize=address -fno-omit-frame-pointer
  • Two macros (PADDING_GUARD_BEGIN and PADDING_GUARD_END) are provided by utils/cpp_macros.h. These should be used to surround any structure used as a hash key with a raw comparator or that would otherwise suffer from unintentional padding. A compiler warning will be generated if any structure definition is automatically padded between the macro invocations.

  • Then Fix All Warnings and Aborts. None Allowed.

Uncrustify

Currently using uncrustify from at https://github.com/bengardner/uncrustify to reformat legacy code and anything that happens to need a makeover at some point.

The working config is crusty.cfg in the top level directory. It does well but will munge some things. Specially formatted INDENT-OFF comments were added in 2 places to avoid a real mess.

You can use uncrustify something like this:

uncrustify -c crusty.cfg --replace file.cc

Commit Messages

Commit messages should follow some basic guidelines:

  • Messages should have a subject line followed by an optional body separated by an empty line.

  • The subject line should be formatted like "<topic>: <summary>".

  • All lines should be 72 characters or less. Subject lines at 50 characters or less are encouraged.

Source Code

src/

This directory contains the program entry point, thread management, and control functions.

  • The main / foreground thread services control inputs from signals, the command line shell (if enabled), etc.

  • The packet / background threads service one input source apiece.

The main_loop() starts a new Pig when a new source (interface or pcap, etc.) is available if the number of running Pigs is less than configured.

It also does housekeeping functions like servicing signal flags, shell commands, etc.

The shell has to be explicitly enabled at build time to be available and then must be configured at run time to be activated. Multiple simultaneous remote shells are supported.

Unit test and benchmark testbuild options also impact actual execution.

Reload is implemented by swapping a thread local config pointer by each running Pig. The inspector manager is called to empty trash if the main loop is not otherwise busy.

Reload policy is implemented by cloning the thread local config and overwriting the policy map and the inspection policy in the main thread. The inspector list from the old config’s inspection policy is copied into the inspection policy of the new config. After the config pointer is cloned, the new inspection policy elements (reloadable) such as inspectors, binder, wizard etc are read and instantiated. The inspector list of the new config is updated by swapping out the old inspectors, binder etc. with the newly instantiated elements. The reloaded inspectors, binders and other inspection policy elements are marked for deletion. After the new inspection policy is loaded, the thread local config pointer is swapped with the new cloned config by running Pig. This happens in the packet thread. The inspector manager is then called to delete any reloaded policy elements and empty trash.

main.h

Path = src/main.h

#ifndef MAIN_H
#define MAIN_H

struct lua_State;

const char* get_prompt();

// commands provided by the snort module
int main_delete_inspector(lua_State* = nullptr);
int main_dump_stats(lua_State* = nullptr);
int main_log_command(lua_State* = nullptr);
int main_dump_heap_stats(lua_State* = nullptr);
int main_reset_stats(lua_State* = nullptr);
int main_set_watchdog_params(lua_State* = nullptr);
int main_rotate_stats(lua_State* = nullptr);
int main_reload_config(lua_State* = nullptr);
int main_reload_policy(lua_State* = nullptr);
int main_reload_daq(lua_State* = nullptr);
int main_reload_hosts(lua_State* = nullptr);
int main_show_config_generation(lua_State* = nullptr);
int main_process(lua_State* = nullptr);
int main_pause(lua_State* = nullptr);
int main_resume(lua_State* = nullptr);
int main_quit(lua_State* = nullptr);
int main_help(lua_State* = nullptr);
int convert_counter_type(const char* type);

#ifdef SHELL
int main_dump_plugins(lua_State* = nullptr);
int main_detach(lua_State* = nullptr);
#endif

void main_poke(unsigned);

#endif

actions/

IPS actions allow you to execute custom responses to events. Unlike loggers, these are invoked before thresholding and can be used to control external agents (including loggers).

IPS rules have an associated type that determines what kind of action they trigger. The rule types defined in this module are:

  • log pass alert drop block

It also defines 3 active responses: * react reject rewrite

Reject performs active response to shutdown hostile network session by injecting TCP resets (TCP connections) or ICMP unreachable packets.

React sends an HTML page to the client, a RST to the server and blocks the flow. It is using payload_injector utility. payload_injector should be configured when react is used.

Rewrite enables overwrite packet contents based on "replace" option in the rules. Note that using "rewrite" action without "replace" option will raise the appropriate rule alert but will not overwrite the packet payload. Rewrite/replace works for raw packets only.

Ips actions are all pluggable and implemented as subclasses of IpsAction action. Each ips action instance has an instance of the active action that is used to perform the active response.

IpsAction::exec represents the action taken by the ips actions after rule evaluation during packet processing. This may include logging, alerting marking the packet as passed, etc. IpsActions::exec when called without the otn will do the active action such as drop/block/reset without alerting/logging or marking the packet as passed.

ActiveAction::delayed_exec is called during the post processing of a packet to send active responses.

actions.h

Path = src/actions/actions.h

#ifndef ACTIONS_H
#define ACTIONS_H

// Define action types and provide hooks to apply a given action to a packet

#include <cstdint>
#include <string>

#include "main/snort_types.h"

struct OptTreeNode;

namespace snort
{
struct Packet;
}

class SO_PUBLIC Actions
{
public:
    using Type = uint8_t;
public:
    static std::string get_string(Type);
    static Type get_type(const char*);
    static Type get_max_types();
    static bool is_valid_action(Type);
    static std::string get_default_priorities(bool alert_before_pass = false);

    static void pass();
    static void log(snort::Packet*, const OptTreeNode*);
    static void alert(snort::Packet*, const OptTreeNode*);
};
#endif

ips_actions.h

Path = src/actions/ips_actions.h

#ifndef IPS_ACTIONS_H
#define IPS_ACTIONS_H

void load_actions();

#endif

catch/

Unit and Benchmark Tests

This directory contains unit-test and benchmark-test interfaces.

In order to enable unit tests, configure snort with --enable-unit-tests.

To enable benchmark tests, configure snort with --enable-benchmark-tests. For benchmarking is also preferred to configure a non-debug build with optimizations.

The relevant licensing information for catch.hpp, as required by the Boost Software License, is provided in this directory, in the file LICENSE_1_0.txt

snort_catch.h

Path = src/catch/snort_catch.h

#ifndef SNORT_CATCH_H
#define SNORT_CATCH_H

// This header adds a wrapper around Catch to facilitate testing in dynamic
// modules. Because of quirks in global storage (as used by Catch), an
// installer in the main binary is necessary to access Catch's global state and
// provide test cases from dynamic plugins to the global list of tests to be
// run. This header should be used instead of including catch.hpp directly.

// pragma for running unit tests on dynamic modules
#pragma GCC visibility push(default)
#include "catch.hpp"
#pragma GCC visibility pop

#include "main/snort_types.h"

namespace snort
{
class TestCaseInstaller
{
public:
    SO_PUBLIC TestCaseInstaller(void(*fun)(), const char* name);
    SO_PUBLIC TestCaseInstaller(void(*fun)(), const char* name, const char* group);
};

/*  translate this:
        SNORT_TEST_CASE("my_test", "[snort_tests"])
        {
            REQUIRE(true);
        }

    to this:
        static void snort_test1();
        static TestCaseInstaller test_test_installer1(snort_test1, "my_test", "[snort_tests]");
        static void snort_test1()
        {
            REQUIRE(true);
        }
*/
#define SNORT_CATCH_UNIQUE_NAME_LINE2( name, line ) name##line
#define SNORT_CATCH_UNIQUE_NAME_LINE( name, line ) SNORT_CATCH_UNIQUE_NAME_LINE2(name, line)
#define SNORT_CATCH_UNIQUE_NAME( name ) SNORT_CATCH_UNIQUE_NAME_LINE(name, __LINE__)
#define MAKE_SNORT_TEST_CASE(fun, ...) \
    static void fun(); \
    static snort::TestCaseInstaller SNORT_CATCH_UNIQUE_NAME(test_case_installer)(fun, __VA_ARGS__); \
    static void fun()

#undef TEST_CASE
#define TEST_CASE(...) MAKE_SNORT_TEST_CASE(SNORT_CATCH_UNIQUE_NAME(snort_test), __VA_ARGS__)
}

#endif

unit_test.h

Path = src/catch/unit_test.h

#ifndef UNIT_TEST_H
#define UNIT_TEST_H

#include "main/snort_types.h"

// Unit test interface

void catch_set_filter(const char* s);

bool catch_enabled();

int catch_test();

#endif

codecs/

This directory contains all of the codecs used for parsing raw fields contained within captured frames. The codecs contained do not perform processing beyond basic per-frame validation. They handle protocols from link layer through transport.

Codecs here conform to the API defined by src/framework/codec.h.

codec_api.h

Path = src/codecs/codec_api.h

#ifndef CODEC_API_H
#define CODEC_API_H

void load_codecs();

#endif

codec_module.h

Path = src/codecs/codec_module.h

#ifndef CODECS_CODEC_MODULE_H
#define CODECS_CODEC_MODULE_H

#include <cstdint>

#include "framework/module.h"
#include "main/snort_types.h"

namespace snort
{
class Trace;
}

extern THREAD_LOCAL const snort::Trace* decode_trace;

namespace snort
{
constexpr int GID_DECODE = 116;

//-----------------------------------------------------
// remember to add rules to preproc_rules/decoder.rules
// add the new decoder rules to the following enum.

enum CodecSid : uint32_t
{
    DECODE_NOT_IPV4_DGRAM = 1,
    DECODE_IPV4_INVALID_HEADER_LEN = 2,
    DECODE_IPV4_DGRAM_LT_IPHDR = 3,
    DECODE_IPV4OPT_BADLEN = 4,
    DECODE_IPV4OPT_TRUNCATED = 5,
    DECODE_IPV4_DGRAM_GT_CAPLEN = 6,

    DECODE_TCP_DGRAM_LT_TCPHDR = 45,
    DECODE_TCP_INVALID_OFFSET = 46,
    DECODE_TCP_LARGE_OFFSET = 47,

    DECODE_TCPOPT_BADLEN = 54,
    DECODE_TCPOPT_TRUNCATED = 55,
    DECODE_TCPOPT_TTCP = 56,
    DECODE_TCPOPT_OBSOLETE = 57,
    DECODE_TCPOPT_EXPERIMENTAL = 58,
    DECODE_TCPOPT_WSCALE_INVALID = 59,

    DECODE_UDP_DGRAM_LT_UDPHDR = 95,
    DECODE_UDP_DGRAM_INVALID_LENGTH = 96,
    DECODE_UDP_DGRAM_SHORT_PACKET = 97,
    DECODE_UDP_DGRAM_LONG_PACKET = 98,

    DECODE_ICMP_DGRAM_LT_ICMPHDR = 105,
    DECODE_ICMP_DGRAM_LT_TIMESTAMPHDR = 106,
    DECODE_ICMP_DGRAM_LT_ADDRHDR = 107,

    DECODE_ARP_TRUNCATED = 109,
    DECODE_EAPOL_TRUNCATED = 110,
    DECODE_EAPKEY_TRUNCATED = 111,
    DECODE_EAP_TRUNCATED = 112,

    DECODE_BAD_PPPOE = 120,
    DECODE_BAD_VLAN = 130,
    DECODE_BAD_LLC_HEADER = 131,
    DECODE_BAD_LLC_OTHER = 132,
    DECODE_BAD_80211_ETHLLC = 133,
    DECODE_BAD_80211_OTHER = 134,

    DECODE_BAD_TRH = 140,
    DECODE_BAD_TR_ETHLLC = 141,
    DECODE_BAD_TR_MR_LEN = 142,
    DECODE_BAD_TRHMR = 143,

    DECODE_BAD_TRAFFIC_LOOPBACK = 150,
    DECODE_BAD_TRAFFIC_SAME_SRCDST = 151,

    DECODE_GRE_DGRAM_LT_GREHDR = 160,
    DECODE_GRE_MULTIPLE_ENCAPSULATION = 161,
    DECODE_GRE_INVALID_VERSION = 162,
    DECODE_GRE_INVALID_HEADER = 163,
    DECODE_GRE_V1_INVALID_HEADER = 164,
    DECODE_GRE_TRANS_DGRAM_LT_TRANSHDR = 165,

    DECODE_BAD_MPLS = 170,
    DECODE_BAD_MPLS_LABEL0 = 171,
    DECODE_BAD_MPLS_LABEL1 = 172,
    DECODE_BAD_MPLS_LABEL2 = 173,
    DECODE_BAD_MPLS_LABEL3 = 174,
    DECODE_MPLS_RESERVED_LABEL = 175,
    DECODE_MPLS_LABEL_STACK = 176,

    DECODE_GENEVE_DGRAM_LT_GENEVE_HDR = 180,
    DECODE_GENEVE_INVALID_VERSION,
    DECODE_GENEVE_INVALID_HEADER,
    DECODE_GENEVE_INVALID_FLAGS,
    DECODE_GENEVE_INVALID_OPTIONS,

    DECODE_ICMP_ORIG_IP_TRUNCATED = 250,
    DECODE_ICMP_ORIG_IP_VER_MISMATCH = 251,
    DECODE_ICMP_ORIG_DGRAM_LT_ORIG_IP = 252,
    DECODE_ICMP_ORIG_PAYLOAD_LT_64 = 253,
    DECODE_ICMP_ORIG_PAYLOAD_GT_576 = 254,
    DECODE_ICMP_ORIG_IP_WITH_FRAGOFFSET = 255,

    DECODE_IPV6_MIN_TTL = 270,
    DECODE_IPV6_IS_NOT = 271,
    DECODE_IPV6_TRUNCATED_EXT = 272,
    DECODE_IPV6_TRUNCATED = 273,
    DECODE_IPV6_DGRAM_LT_IPHDR = 274,
    DECODE_IPV6_DGRAM_GT_CAPLEN = 275,
    DECODE_IPV6_DST_ZERO = 276,
    DECODE_IPV6_SRC_MULTICAST = 277,
    DECODE_IPV6_DST_RESERVED_MULTICAST = 278,
    DECODE_IPV6_BAD_OPT_TYPE = 279,
    DECODE_IPV6_BAD_MULTICAST_SCOPE = 280,
    DECODE_IPV6_BAD_NEXT_HEADER = 281,
    DECODE_IPV6_ROUTE_AND_HOPBYHOP = 282,
    DECODE_IPV6_TWO_ROUTE_HEADERS = 283,

    DECODE_ICMPV6_TOO_BIG_BAD_MTU = 285,
    DECODE_ICMPV6_UNREACHABLE_NON_RFC_2463_CODE = 286,
    DECODE_ICMPV6_SOLICITATION_BAD_CODE = 287,
    DECODE_ICMPV6_ADVERT_BAD_CODE = 288,
    DECODE_ICMPV6_SOLICITATION_BAD_RESERVED = 289,
    DECODE_ICMPV6_ADVERT_BAD_REACHABLE = 290,

    DECODE_IPV6_TUNNELED_IPV4_TRUNCATED = 291,
    DECODE_IPV6_DSTOPTS_WITH_ROUTING = 292,
    DECODE_IP_MULTIPLE_ENCAPSULATION = 293,

    DECODE_ESP_HEADER_TRUNC = 294,
    DECODE_IPV6_BAD_OPT_LEN = 295,
    DECODE_IPV6_UNORDERED_EXTENSIONS = 296,

    DECODE_GTP_MULTIPLE_ENCAPSULATION = 297,
    DECODE_GTP_BAD_LEN = 298,

    DECODE_TCP_XMAS = 400,
    DECODE_TCP_NMAP_XMAS,
    DECODE_DOS_NAPTHA,
    DECODE_SYN_TO_MULTICAST,
    DECODE_ZERO_TTL,
    DECODE_BAD_FRAGBITS,
    DECODE_UDP_IPV6_ZERO_CHECKSUM,
    DECODE_IP4_LEN_OFFSET,
    DECODE_IP4_SRC_THIS_NET,
    DECODE_IP4_DST_THIS_NET,
    DECODE_IP4_SRC_MULTICAST,
    DECODE_IP4_SRC_RESERVED,
    DECODE_IP4_DST_RESERVED,
    DECODE_IP4_SRC_BROADCAST,
    DECODE_IP4_DST_BROADCAST,
    DECODE_ICMP4_DST_MULTICAST,
    DECODE_ICMP4_DST_BROADCAST,
    DECODE_ICMP4_TYPE_OTHER = 418,
    DECODE_TCP_BAD_URP,
    DECODE_TCP_SYN_FIN,
    DECODE_TCP_SYN_RST,
    DECODE_TCP_MUST_ACK,
    DECODE_TCP_NO_SYN_ACK_RST,
    DECODE_ETH_HDR_TRUNC,
    DECODE_IP4_HDR_TRUNC,
    DECODE_ICMP4_HDR_TRUNC,
    DECODE_ICMP6_HDR_TRUNC,
    DECODE_IP4_MIN_TTL,
    DECODE_IP6_ZERO_HOP_LIMIT,
    DECODE_IP4_DF_OFFSET, // = 430
    DECODE_ICMP6_TYPE_OTHER,
    DECODE_ICMP6_DST_MULTICAST,
    DECODE_TCP_SHAFT_SYNFLOOD,
    DECODE_ICMP_PING_NMAP,
    DECODE_ICMP_ICMPENUM,
    DECODE_ICMP_REDIRECT_HOST,
    DECODE_ICMP_REDIRECT_NET,
    DECODE_ICMP_TRACEROUTE_IPOPTS,
    DECODE_ICMP_SOURCE_QUENCH,
    DECODE_ICMP_BROADSCAN_SMURF_SCANNER, // = 440
    DECODE_ICMP_DST_UNREACH_ADMIN_PROHIBITED,
    DECODE_ICMP_DST_UNREACH_DST_HOST_PROHIBITED,
    DECODE_ICMP_DST_UNREACH_DST_NET_PROHIBITED,
    DECODE_IP_OPTION_SET,
    DECODE_UDP_LARGE_PACKET,
    DECODE_TCP_PORT_ZERO,
    DECODE_UDP_PORT_ZERO,
    DECODE_IP_RESERVED_FRAG_BIT,
    DECODE_IP_UNASSIGNED_PROTO,
    DECODE_IP_BAD_PROTO, // = 450
    DECODE_ICMP_PATH_MTU_DOS,
    DECODE_ICMP_DOS_ATTEMPT,
    DECODE_IPV6_ISATAP_SPOOF,
    DECODE_PGM_NAK_OVERFLOW,
    DECODE_IGMP_OPTIONS_DOS,
    DECODE_IP6_EXCESS_EXT_HDR,
    DECODE_ICMPV6_UNREACHABLE_NON_RFC_4443_CODE,
    DECODE_IPV6_BAD_FRAG_PKT,
    DECODE_ZERO_LENGTH_FRAG,
    DECODE_ICMPV6_NODE_INFO_BAD_CODE, // = 460
    DECODE_IPV6_ROUTE_ZERO,
    DECODE_ERSPAN_HDR_VERSION_MISMATCH,
    DECODE_ERSPAN2_DGRAM_LT_HDR,
    DECODE_ERSPAN3_DGRAM_LT_HDR,
    DECODE_AUTH_HDR_TRUNC,
    DECODE_AUTH_HDR_BAD_LEN,
    DECODE_FPATH_HDR_TRUNC,
    DECODE_CISCO_META_HDR_TRUNC,
    DECODE_CISCO_META_HDR_OPT_LEN,
    DECODE_CISCO_META_HDR_OPT_TYPE, // = 470
    DECODE_CISCO_META_HDR_SGT,
    DECODE_TOO_MANY_LAYERS,
    DECODE_BAD_ETHER_TYPE,
    DECODE_ICMP6_NOT_IP6,
    DECODE_MIPV6_BAD_PAYLOAD_PROTO,
    DECODE_IPV6_SRC_RESERVED,
    DECODE_IPV6_DST_RESERVED,
    DECODE_INDEX_MAX
};

//-------------------------------------------------------------------------
// module
//-------------------------------------------------------------------------

class BaseCodecModule : public Module
{
public:
    BaseCodecModule(const char* s, const char* h) : Module(s, h)
    { }

    BaseCodecModule(const char* s, const char* h, const Parameter* p, bool is_list = false)
        : Module(s, h, p, is_list) { }

    unsigned get_gid() const override
    { return GID_DECODE; }

    Usage get_usage() const override
    { return CONTEXT; }
};

class SO_PUBLIC CodecModule : public BaseCodecModule
{
public:
    CodecModule();

    const RuleMap* get_rules() const override;

    void set_trace(const Trace*) const override;
    const TraceOption* get_trace_options() const override;
};
}

#endif

codecs/ip/

All codecs under this directory handle data that would be seen directly following or under IP headers.

checksum.h

Path = src/codecs/ip/checksum.h

#ifndef CODECS_CHECKSUM_H
#define CODECS_CHECKSUM_H

#include <cstddef>

#include <protocols/protocol_ids.h>

namespace checksum
{
union Pseudoheader
{
    struct
    {
        uint32_t sip;
        uint32_t dip;
        uint8_t zero;
        IpProtocol protocol;
        uint16_t len;
    } hdr;
    uint16_t arr[6];
    static_assert(sizeof(hdr) == sizeof(arr), "IPv4 pseudoheader must be 12 bytes");
};

union Pseudoheader6
{
    struct
    {
        uint32_t sip[4];
        uint32_t dip[4];
        uint8_t zero;
        IpProtocol protocol;
        uint16_t len;
    } hdr;
    uint16_t arr[18];
    static_assert(sizeof(hdr) == sizeof(arr), "IPv6 pseudoheader must be 36 bytes");
};

//  calculate the checksum for this general case.
static uint16_t cksum_add(const uint16_t* buf, std::size_t buf_len);
inline uint16_t tcp_cksum(const uint16_t* buf, std::size_t len, const Pseudoheader&);
inline uint16_t tcp_cksum(const uint16_t* buf, std::size_t len, const Pseudoheader6&);
inline uint16_t udp_cksum(const uint16_t* buf, std::size_t len, const Pseudoheader&);
inline uint16_t udp_cksum(const uint16_t* buf, std::size_t len, const Pseudoheader6&);
inline uint16_t icmp_cksum(const uint16_t* buf, std::size_t len, const Pseudoheader6&);
inline uint16_t icmp_cksum(const uint16_t* buf, std::size_t len);
inline uint16_t ip_cksum(const uint16_t* buf, std::size_t len);

/*
 *  NOTE: Since multiple dynamic libraries use checksums, the choice
 *          is to either include all of the checksum details in a header,
 *          or ensure I include these symbols for every linker which
 *          can be used. Obviously, setting correct linker flags is
 *          significantly more difficult, so these functions will all
 *          stay in a header file
 */

/*
 *  IT IS HIGHLY RECOMMENDED to use the above API. Rathern than calling
 *  any of of the following recommendations directly
 */
namespace detail
{
inline uint16_t cksum_add(const uint16_t* buf, std::size_t len, uint32_t cksum)
{
    const uint16_t* sp = buf;

    // if pointer is 16 bit aligned calculate checksum in tight loop...
    // gcc 5.4 -O3 generates unaligned quadword instructions that crash; fixed in gcc 8.0.1
    if ( !( reinterpret_cast<std::uintptr_t>(sp) & 0x01 ) )
    {
        while ( len > 1 )
        {
            cksum += *sp++;
            len -= 2;
        }
    }
    else if ( len > 1 )
    {
        std::size_t sn = ((len / 2) & 0xF);  // == len/2 % 16
        std::size_t n = (((len / 2) + 15) / 16);   // ceiling of (len / 2) / 16

        switch (sn)
        {
        case 0:
            sn = 16;
            cksum += sp[15];    // fallthrough
        case 15:
            cksum += sp[14];    // fallthrough
        case 14:
            cksum += sp[13];    // fallthrough
        case 13:
            cksum += sp[12];    // fallthrough
        case 12:
            cksum += sp[11];    // fallthrough
        case 11:
            cksum += sp[10];    // fallthrough
        case 10:
            cksum += sp[9];     // fallthrough
        case 9:
            cksum += sp[8];     // fallthrough
        case 8:
            cksum  += sp[7];    // fallthrough
        case 7:
            cksum += sp[6];     // fallthrough
        case 6:
            cksum += sp[5];     // fallthrough
        case 5:
            cksum += sp[4];     // fallthrough
        case 4:
            cksum += sp[3];     // fallthrough
        case 3:
            cksum += sp[2];     // fallthrough
        case 2:
            cksum += sp[1];     // fallthrough
        case 1:
            cksum += sp[0];
        }
        sp += sn;

        /* unroll loop using Duff's device. */
        while (--n > 0)
        {
            cksum += sp[0];
            cksum += sp[1];
            cksum += sp[2];
            cksum += sp[3];
            cksum += sp[4];
            cksum += sp[5];
            cksum += sp[6];
            cksum += sp[7];
            cksum += sp[8];
            cksum += sp[9];
            cksum += sp[10];
            cksum += sp[11];
            cksum += sp[12];
            cksum += sp[13];
            cksum += sp[14];
            cksum += sp[15];
            sp += 16;
        }
    }

    // if len is odd, sum in the last byte...
    if ( len & 0x01 )
        cksum += *((const uint8_t*) sp);

    cksum  = (cksum >> 16) + (cksum & 0x0000ffff);
    cksum += (cksum >> 16);

    return (uint16_t)(~cksum);
}

inline void add_ipv4_pseudoheader(const Pseudoheader& ph4, uint32_t& cksum)
{
    const uint16_t* h = ph4.arr;

    /* ipv4 pseudo header must have 12 bytes */
    cksum += h[0];
    cksum += h[1];
    cksum += h[2];
    cksum += h[3];
    cksum += h[4];
    cksum += h[5];
}

inline void add_ipv6_pseudoheader(const Pseudoheader6& ph6, uint32_t& cksum)
{
    const uint16_t* h = ph6.arr;

    /* PseudoHeader must have 36 bytes */
    cksum += h[0];
    cksum += h[1];
    cksum += h[2];
    cksum += h[3];
    cksum += h[4];
    cksum += h[5];
    cksum += h[6];
    cksum += h[7];
    cksum += h[8];
    cksum += h[9];
    cksum += h[10];
    cksum += h[11];
    cksum += h[12];
    cksum += h[13];
    cksum += h[14];
    cksum += h[15];
    cksum += h[16];
    cksum += h[17];
}

inline void add_tcp_header(const uint16_t*& d,
    std::size_t& len,
    uint32_t& cksum)
{
    /* TCP hdr must have 20 hdr bytes */
    cksum += d[0];
    cksum += d[1];
    cksum += d[2];
    cksum += d[3];
    cksum += d[4];
    cksum += d[5];
    cksum += d[6];
    cksum += d[7];
    cksum += d[8];
    cksum += d[9];
    d += 10;
    len -= 20;
}

inline void add_udp_header(const uint16_t*& d,
    size_t& len,
    uint32_t& cksum)
{
    /* UDP must have 8 hdr bytes */
    cksum += d[0];
    cksum += d[1];
    cksum += d[2];
    cksum += d[3];
    len -= 8;
    d += 4;
}

inline void add_ip_header(const uint16_t*& d,
    std::size_t& len,
    uint32_t& cksum)
{
    /* IP must be >= 20 bytes */
    cksum += d[0];
    cksum += d[1];
    cksum += d[2];
    cksum += d[3];
    cksum += d[4];
    cksum += d[5];
    cksum += d[6];
    cksum += d[7];
    cksum += d[8];
    cksum += d[9];
    d += 10;
    len -= 20;
}
} // namespace detail

inline uint16_t icmp_cksum(const uint16_t* buf,
    std::size_t len,
    const Pseudoheader6& ph)
{
    uint32_t cksum = 0;

    detail::add_ipv6_pseudoheader(ph, cksum);
    return detail::cksum_add(buf, len, cksum);
}

inline uint16_t icmp_cksum(const uint16_t* buf, size_t len)
{
    return detail::cksum_add(buf, len, 0);
}

inline uint16_t tcp_cksum(const uint16_t* h,
    std::size_t len,
    const Pseudoheader& ph)
{
    uint32_t cksum = 0;

    detail::add_ipv4_pseudoheader(ph, cksum);
    detail::add_tcp_header(h, len, cksum);
    return detail::cksum_add(h, len, cksum);
}

inline uint16_t tcp_cksum(const uint16_t* buf,
    std::size_t len,
    const Pseudoheader6& ph)
{
    uint32_t cksum = 0;

    detail::add_ipv6_pseudoheader(ph, cksum);
    detail::add_tcp_header(buf, len, cksum);
    return detail::cksum_add(buf, len, cksum);
}

inline uint16_t udp_cksum(const uint16_t* buf,
    std::size_t len,
    const Pseudoheader& ph)
{
    uint32_t cksum = 0;

    detail::add_ipv4_pseudoheader(ph, cksum);
    detail::add_udp_header(buf, len, cksum);
    return detail::cksum_add(buf, len, cksum);
}

inline uint16_t udp_cksum(const uint16_t* buf,
    std::size_t len,
    const Pseudoheader6& ph)
{
    uint32_t cksum = 0;

    detail::add_ipv6_pseudoheader(ph, cksum);
    detail::add_udp_header(buf, len, cksum);
    return detail::cksum_add(buf, len, cksum);
}

inline uint16_t ip_cksum(const uint16_t* buf, std::size_t len)
{
    uint32_t cksum = 0;

    detail::add_ip_header(buf, len, cksum);
    return detail::cksum_add(buf, len, cksum);
}

inline uint16_t cksum_add(const uint16_t* buf, std::size_t len)
{ return detail::cksum_add(buf, len, 0); }
} // namespace checksum

#endif  /* CODECS_CHECKSUM_H */

These codecs handle link-layer protocols that would be presented beyond the root encapsulation defined by the capture data-link type.

codecs/misc/

This directory contains codecs that do not fall under the classifications of the other codec directories. These codecs primarily handle IP tunneling protocols.

codecs/misc/test/

codecs/root/

This directory contains codecs for all of the top-level codecs used. These codecs represent the first layer of encapsulation encountered within a captured frame, as defined by interface data-link types.

connectors/

Connectors house the collection of the plugin-type Connector. As defined …/framework/connector.h, the Connector object implements a simplex communications channel that is in-turn used by SideChannel objects.

Connectors have the standard plugin api and instantiation/destruction protocols.

The file_connector writes messages to a file and reads messages from a file.

Configuration entries map side channels to connector instances.

connectors.h

Path = src/connectors/connectors.h

#ifndef CONNECTORS_H
#define CONNECTORS_H

void load_connectors();

#endif

connectors/file_connector/

Implement a connector plugin that reads/writes side channel messages from/to files.

Each connector implements a simplex channel, either transmit or receive. In- turn, each SideChannel owns a transmit and/or a receive connector object.

The "format = text" option in the configuration sets text file input/output else binary file format is used.

In binary files, an additional FileConnector message header is pre-pended to each side channel message written to the output file. This header specifies the file format version and the length of the side channel message. This length does not include the file connector message header, but does include the side channel message header.

The utility get_instance_file() is used to uniquely name the files. The complete file name convention is:

The input file is named file_connector_<name>_receive and the output file is named file_connector_<name>_transmit. Where the <name> is specified "name" field in the configuration. These names are in turn pre-pended with the get_instance_file() naming convention.

The file_connector Connector configuration results in ONE ConnectorCommon object which is used to contain a list of all Connectors being configured. A vector<> in the ConnectorCommon object holds individual Connector config objects. The ConnectorManager then uses this vector<> to instantiate the set of desired Connectors.

file_connector_config.h

Path = src/connectors/file_connector/file_connector_config.h

#ifndef FILE_CONNECTOR_CONFIG_H
#define FILE_CONNECTOR_CONFIG_H

#include <string>
#include <vector>

#include "framework/connector.h"

class FileConnectorConfig : public snort::ConnectorConfig
{
public:
    FileConnectorConfig()
    { direction = snort::Connector::CONN_UNDEFINED; text_format = false; }

    bool text_format;
    std::string name;

    typedef std::vector<FileConnectorConfig*> FileConnectorConfigSet;
};

#endif

file_connector.h

Path = src/connectors/file_connector/file_connector.h

#ifndef FILE_CONNECTOR_H
#define FILE_CONNECTOR_H

#include <fstream>

#include "framework/connector.h"

#include "file_connector_config.h"

#define FILE_FORMAT_VERSION (1)

//-------------------------------------------------------------------------
// class stuff
//-------------------------------------------------------------------------

class __attribute__((__packed__)) FileConnectorMsgHdr
{
public:
    FileConnectorMsgHdr(uint32_t length)
    { version = FILE_FORMAT_VERSION; connector_msg_length = length; }

    uint16_t version;
    uint32_t connector_msg_length;
};

class FileConnectorMsgHandle : public snort::ConnectorMsgHandle
{
public:
    FileConnectorMsgHandle(const uint32_t length);
    ~FileConnectorMsgHandle();
    snort::ConnectorMsg connector_msg;
};

class FileConnectorCommon : public snort::ConnectorCommon
{
public:
    FileConnectorCommon(FileConnectorConfig::FileConnectorConfigSet*);
    ~FileConnectorCommon();
};

class FileConnector : public snort::Connector
{
public:
    FileConnector(FileConnectorConfig*);
    snort::ConnectorMsgHandle* alloc_message(const uint32_t, const uint8_t**) override;
    void discard_message(snort::ConnectorMsgHandle*) override;
    bool transmit_message(snort::ConnectorMsgHandle*) override;
    snort::ConnectorMsgHandle* receive_message(bool) override;

    snort::ConnectorMsg* get_connector_msg(snort::ConnectorMsgHandle* handle) override
    { return( &((FileConnectorMsgHandle*)handle)->connector_msg ); }
    Direction get_connector_direction() override
    { return( ((const FileConnectorConfig*)config)->direction ); }

    std::fstream file;

private:
    snort::ConnectorMsgHandle* receive_message_binary();
    snort::ConnectorMsgHandle* receive_message_text();
};

#endif

file_connector_module.h

Path = src/connectors/file_connector/file_connector_module.h

#ifndef FILE_CONNECTOR_MODULE_H
#define FILE_CONNECTOR_MODULE_H

#include "framework/module.h"

#include "file_connector_config.h"

#define FILE_CONNECTOR_NAME "file_connector"
#define FILE_CONNECTOR_HELP "implement the file based connector"

class FileConnectorModule : public snort::Module
{
public:
    FileConnectorModule();
    ~FileConnectorModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    FileConnectorConfig::FileConnectorConfigSet* get_and_clear_config();

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;

    snort::ProfileStats* get_profile() const override;

    Usage get_usage() const override
    { return GLOBAL; }

private:
    FileConnectorConfig::FileConnectorConfigSet* config_set;
    FileConnectorConfig* config = nullptr;
};

#endif

connectors/file_connector/test/

connectors/tcp_connector/

Implement a connector plugin that reads and writes side channel messages across a TCP stream channel.

Each connector implements a duplex channel, both transmit and receive. When used by a side_channel object, a single TcpConnector object is used for both the transmit and receive connectors.

An additional TcpConnector message header is pre-pended to each side channel message transmitted. This header specifies the protocol format version and the length of the side channel message. This length does not include the tcp connector message header, but does include the side channel message header.

The tcp_connector Connector configuration results in ONE ConnectorCommon object which is used to contain a list of all Connectors being configured. A vector<> in the ConnectorCommon object holds individual Connector config objects. The ConnectorManager then uses this vector<> to instantiate the set of desired Connectors.

TCP connector configuration includes a partner address, base port numbers, and connection setup direction. The actual port number used is the base port number added to the thread instance value.

A TCP connector can be either the active partner and initiate the TCP connection or can be the passive partner and expect to be called by the active side. This is controlled by the setup configuration element.

Receive messages are managed via separate thread and ring buffer queue structure. The thread’s purpose is to read whole side channel messages from the stream and insert them into the queue. Then the packet processing thread is able to read whole side messages from the queue.

tcp_connector_config.h

Path = src/connectors/tcp_connector/tcp_connector_config.h

#ifndef TCP_CONNECTOR_CONFIG_H
#define TCP_CONNECTOR_CONFIG_H

#include <vector>

#include "framework/connector.h"

class TcpConnectorConfig : public snort::ConnectorConfig
{
public:
    enum Setup { CALL, ANSWER };
    TcpConnectorConfig()
    { direction = snort::Connector::CONN_DUPLEX; async_receive = true; }

    uint16_t base_port = 0;
    std::string address;
    Setup setup = {};
    bool async_receive;

    typedef std::vector<TcpConnectorConfig*> TcpConnectorConfigSet;
};

#endif

tcp_connector.h

Path = src/connectors/tcp_connector/tcp_connector.h

#ifndef TCP_CONNECTOR_H
#define TCP_CONNECTOR_H

#include <thread>

#include "framework/connector.h"
#include "helpers/ring.h"

#include "tcp_connector_config.h"

#define TCP_FORMAT_VERSION (1)

//-------------------------------------------------------------------------
// class stuff
//-------------------------------------------------------------------------

class __attribute__((__packed__)) TcpConnectorMsgHdr
{
public:
    TcpConnectorMsgHdr() : version(0), connector_msg_length(0)
    { }
    TcpConnectorMsgHdr(uint32_t length)
    { version = TCP_FORMAT_VERSION; connector_msg_length = length; }

    uint8_t version;
    uint16_t connector_msg_length;
};

class TcpConnectorMsgHandle : public snort::ConnectorMsgHandle
{
public:
    TcpConnectorMsgHandle(const uint32_t length);
    ~TcpConnectorMsgHandle();
    snort::ConnectorMsg connector_msg;
};

class TcpConnectorCommon : public snort::ConnectorCommon
{
public:
    TcpConnectorCommon(TcpConnectorConfig::TcpConnectorConfigSet*);
    ~TcpConnectorCommon();
};

class TcpConnector : public snort::Connector
{
public:
    typedef Ring<TcpConnectorMsgHandle*> ReceiveRing;

    TcpConnector(TcpConnectorConfig*, int sock_fd);
    ~TcpConnector() override;
    snort::ConnectorMsgHandle* alloc_message(const uint32_t, const uint8_t**) override;
    void discard_message(snort::ConnectorMsgHandle*) override;
    bool transmit_message(snort::ConnectorMsgHandle*) override;
    snort::ConnectorMsgHandle* receive_message(bool) override;

    snort::ConnectorMsg* get_connector_msg(snort::ConnectorMsgHandle* handle) override
    { return( &((TcpConnectorMsgHandle*)handle)->connector_msg ); }
    Direction get_connector_direction() override
    { return Connector::CONN_DUPLEX; }
    void process_receive();

    int sock_fd;

private:
    bool run_thread = false;
    std::thread* receive_thread;
    void start_receive_thread();
    void stop_receive_thread();
    void receive_processing_thread();
    ReceiveRing* receive_ring;
};

#endif

tcp_connector_module.h

Path = src/connectors/tcp_connector/tcp_connector_module.h

#ifndef TCP_CONNECTOR_MODULE_H
#define TCP_CONNECTOR_MODULE_H

#include "framework/module.h"

#include "tcp_connector_config.h"

#define TCP_CONNECTOR_NAME "tcp_connector"
#define TCP_CONNECTOR_HELP "implement the tcp stream connector"

class TcpConnectorModule : public snort::Module
{
public:
    TcpConnectorModule();
    ~TcpConnectorModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    TcpConnectorConfig::TcpConnectorConfigSet* get_and_clear_config();

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;

    snort::ProfileStats* get_profile() const override;

    Usage get_usage() const override
    { return GLOBAL; }

private:
    TcpConnectorConfig::TcpConnectorConfigSet* config_set;
    TcpConnectorConfig* config = nullptr;
};

#endif

connectors/tcp_connector/test/

control/

This module provides functions for registering and running handlers that are called when Snort is not doing anything more important.

control.h

Path = src/control/control.h

#ifndef CONTROL_H
#define CONTROL_H

#include <cstdarg>
#include <ctime>
#include <queue>
#include <string>
#include <vector>

#include "main/snort_types.h"

struct lua_State;

class ControlConn
{
public:
    ControlConn(int fd, bool local);
    ~ControlConn();

    ControlConn(const ControlConn&) = delete;
    ControlConn& operator=(const ControlConn&) = delete;

    int get_fd() const { return fd; }
    class Shell* get_shell() const { return shell; }

    void block();
    void unblock();
    void remove();
    bool show_prompt();
    bool is_blocked() const { return blocked; }
    bool is_closed() const { return (fd == -1); }
    bool is_removed() const { return removed; }
    bool has_pending_command() const { return !pending_commands.empty(); }
    time_t get_touched() const;
    std::string get_current_command() const { return pending_commands.front(); }

    void configure() const;
    int read_commands();
    int execute_commands();
    void shutdown();

    void set_user_network_policy();

    SO_PUBLIC bool is_local() const { return local; }
    SO_PUBLIC bool respond(const char* format, va_list& ap);
    SO_PUBLIC bool respond(const char* format, ...) __attribute__((format (printf, 2, 3)));
    SO_PUBLIC static ControlConn* query_from_lua(const lua_State*);

    static void log_command(const std::string& module, bool log);
    static unsigned increment_pending_cmds_count() { return ++pending_cmds_count; }
    static unsigned decrement_pending_cmds_count() { return --pending_cmds_count; }

private:
    void touch();
    bool loggable(const std::string& command);

private:
    std::queue<std::string> pending_commands;
    std::string next_command;
    class Shell *shell;
    int fd;
    int blocked = 0; // a number of commands blocking the channel
    bool local = false;
    bool removed = false;
    time_t touched;

    static std::vector<std::string> log_exclusion_list;
    static unsigned pending_cmds_count; //counter to serialize commands across control connections
};

#define LogRespond(cn, ...)       do { if (cn) cn->respond(__VA_ARGS__); else LogMessage(__VA_ARGS__); } while(0)
#define LogfRespond(cn, fh, ...)  do { if (cn) cn->respond(__VA_ARGS__); else LogMessage(fh, __VA_ARGS__); } while(0)

#endif

control_mgmt.h

Path = src/control/control_mgmt.h

#ifndef CONTROL_MGMT_H
#define CONTROL_MGMT_H

class ControlConn;
struct lua_State;

namespace snort
{
struct SnortConfig;
}

class ControlMgmt
{
public:
    static bool add_control(int fd, bool local_control);
    static void reconfigure_controls();

    static int socket_init(const snort::SnortConfig*);
    static void socket_term();

    static ControlConn* find_control(const lua_State*);

    static bool service_users();
};

#endif

decompress/

The components in this area implement several file decompression mechanisms. They provide real-time decompression to permit inspection (rules) of the decompressed content.

In particular the components support these decompression options:

  1. Decompress SWF (Adobe Flash) files compressed with the ZLIB algorithm

  2. Optionally decompress SWF files compressed with the LZMA algorithm. This is only available if Snort ++ is built with the optional LZMA support.

  3. Decompress the Deflate compressed portions of PDF files.

The three modes are individually enabled/disabled at initialization time.

All parsing and decompression is incremental and allows inspection to proceed as the file is received and processed.

SWF File Processing:

SWF files exist in three forms: 1) uncompressed, 2) ZLIB compressed, and 3) LZMA compressed. SWF files begin with a file signature block (always uncompressed) to indicate the format of the balance of the file. The balance of the file is formatted and processed as specified.

PDF files are significantly more complex as the compressed content is embedded within the PDF syntax and one file may contain one or many compressed segments.

Thus the PDF decompression engine implements a lightweight PDF file parser to locate the PDF Stream segments and then attempt to decompress Streams that are filtered with the FlateDecode compression. Streams are binary objects that are used for much of the PDF actual content. A Stream object can be labeled with a Filter option to indicate that the Steam is encoded in some fashion, perhaps having multiple cascaded Filters.

The current implementation supports the most common FlateDecode Filter option and does not support cascaded Filters (including cascaded FlateDecode’s).

The decompressor processors can indicate several error situations. There are two mechanisms used to relay these error codes to the calling context. Some errors terminate processing and are passed to the caller in the decompression context structure. Other errors are inline and may be passed back to the caller via a provided call-back function.

Possible error conditions are:

  • FILE_DECOMP_ERR_SWF_ZLIB_FAILURE - The ZLIB decompression engine returned an error.

  • FILE_DECOMP_ERR_SWF_LZMA_FAILURE - The LZMA decompression engine returned an error.

  • FILE_DECOMP_ERR_PDF_DEFL_FAILURE - The Deflate (form of ZLIB) decompression engine returned at error.

  • FILE_DECOMP_ERR_PDF_UNSUP_COMP_TYPE - An unsupported PDF Stream Filter type was encountered,

  • FILE_DECOMP_ERR_PDF_CASC_COMP - Cascasded FlateDecode Stream Filters were encountered.

  • FILE_DECOMP_ERR_PDF_PARSE_FAILURE - Error while parsing the PDF file.

file_decomp.h

Path = src/decompress/file_decomp.h

#ifndef FILE_DECOMP_H
#define FILE_DECOMP_H

// File_Decomp global typedefs (used in child objects)

#include <cstring>

#include "main/snort_types.h"

#define NEW_OFFICE_FRMT 120
#define OLD_OFFICE_FRMT  27

/* Function return codes used internally and with caller */
// FIXIT-L these need to be split into internal-only codes and things that may be returned to the
// application. The codes used by PDF and SWF should be standardized. PDF is returning BlockIn and
// BlockOut while SWF is using OK. There also needs to be clarity about what Error means and what
// should be done about it. Is it just an indicator of programming error or are there operational
// cases where it occurs? No idea whether Complete and Eof are real things and what should be done
// about them.

enum fd_status_t
{
    File_Decomp_DecompError = -2,  /* Error from decompression */
    File_Decomp_Error = -1,        /* Error from decompression */
    File_Decomp_OK = 0,
    File_Decomp_NoSig = 1,         /* No file signature located */
    File_Decomp_Complete = 2,      /* Completed */
    File_Decomp_BlockOut = 3,      /* Blocked due to lack of output space */
    File_Decomp_BlockIn = 4,       /* Blocked due to lack in input data */
    File_Decomp_Eof = 5            /* End of file located */
};

enum file_compression_type_t
{
    FILE_COMPRESSION_TYPE_NONE,
    FILE_COMPRESSION_TYPE_DEFLATE,
    FILE_COMPRESSION_TYPE_ZLIB,
    FILE_COMPRESSION_TYPE_LZMA,
    FILE_COMPRESSION_TYPE_MAX
};

/* Potential decompression modes, passed in at initialization time. */
#define FILE_SWF_LZMA_BIT    (0x00000001)
#define FILE_SWF_ZLIB_BIT    (0x00000002)
#define FILE_PDF_DEFL_BIT    (0x00000004)
#define FILE_ZIP_DEFL_BIT    (0x00000008)
#define FILE_VBA_EXTR_BIT    (0x00000010)

#define FILE_PDF_ANY         (FILE_PDF_DEFL_BIT)
#define FILE_SWF_ANY         (FILE_SWF_LZMA_BIT | FILE_SWF_ZLIB_BIT)
#define FILE_ZIP_ANY         (FILE_ZIP_DEFL_BIT | FILE_VBA_EXTR_BIT)

/* Error codes either passed to caller via the session->Error_Alert of
   the File_Decomp_Alert() call-back function. */
enum FileDecompError
{
    FILE_DECOMP_ERR_SWF_ZLIB_FAILURE,
    FILE_DECOMP_ERR_SWF_LZMA_FAILURE,
    FILE_DECOMP_ERR_PDF_DEFL_FAILURE,
    FILE_DECOMP_ERR_PDF_UNSUP_COMP_TYPE,
    FILE_DECOMP_ERR_PDF_CASC_COMP,
    FILE_DECOMP_ERR_PDF_PARSE_FAILURE,
    FILE_DECOMP_ERR_ZIP_PARSE_FAILURE,
    FILE_DECOMP_ERR_ZIP_DEFL_FAILURE
};

/* Private Types */
enum file_type_t
{
    FILE_TYPE_NONE,
    FILE_TYPE_SWF,
    FILE_TYPE_PDF,
    FILE_TYPE_ZIP,
    FILE_TYPE_MAX
};

enum fd_states_t
{
    STATE_NEW,        /* Session created */
    STATE_READY,      /* Session created and ready for content, no file/decomp selected */
    STATE_ACTIVE,     /* Decompressor inited and ready for content */
    STATE_COMPLETE    /* Decompression completed */
};

/* Primary file decompression session state context */
struct fd_session_t
{
    // FIXIT-L replace with abstract base class pointer used for
    // PDF or SWF subclass and eliminate switches on File_Type
    union
    {
        struct fd_PDF_t* PDF;
        struct fd_SWF_t* SWF;
        struct fd_ZIP_t* ZIP;
    };

    const uint8_t* Next_In;     // next input byte
    uint8_t* Next_Out;          // next output byte should be put there

    // Alerting callback
    void (* Alert_Callback)(void* Context, int Event);
    void* Alert_Context;

    uint32_t Avail_In;   // number of bytes available at next_in
    uint32_t Total_In;   // total number of input bytes read so far

    uint32_t Avail_Out;  // remaining free space at next_out
    uint32_t Total_Out;  // total number of bytes output so far

    // Configuration settings
    // FIXIT-RC Compr_Depth and Decompr_Depth only support OHI and eventually should be removed
    uint32_t Compr_Depth;
    uint32_t Decompr_Depth;
    uint32_t Modes;      // Bit mapped set of potential file/algo modes

    int Error_Event;     // Specific event indicated by DecomprError return

    // Internal State
    uint8_t File_Type;   // Active file type
    uint8_t Decomp_Type; // Active decompression type
    uint8_t Sig_State;   // Sig search state machine
    uint8_t State;       // main state machine
    uint8_t* ole_data_ptr; // compressed ole file.
    uint32_t ole_data_len;
    bool vba_analysis;

    void get_ole_data(uint8_t*& ole_data_ptr, uint32_t& ole_data_len)
    {
        ole_data_ptr = this->ole_data_ptr;
        ole_data_len = this->ole_data_len;
    }

    void ole_data_reset()
    {
        ole_data_ptr = nullptr;
        ole_data_len = 0;
    }
};

/* Macros */

/* Macros used to sync my decompression context with that
   of the underlying decompression engine context. */
#ifndef SYNC_IN
#define SYNC_IN(dest) \
    dest->next_in = const_cast<Bytef*>(SessionPtr->Next_In); \
    (dest)->avail_in = SessionPtr->Avail_In; \
    (dest)->total_in = SessionPtr->Total_In; \
    (dest)->next_out = SessionPtr->Next_Out; \
    (dest)->avail_out = SessionPtr->Avail_Out; \
    (dest)->total_out = SessionPtr->Total_Out;
#endif

#ifndef SYNC_OUT
#define SYNC_OUT(src) \
    SessionPtr->Next_In = (const uint8_t*)(src)->next_in; \
    SessionPtr->Avail_In = (src)->avail_in; \
    SessionPtr->Total_In = (src)->total_in; \
    SessionPtr->Next_Out = (uint8_t*)(src)->next_out; \
    SessionPtr->Avail_Out = (src)->avail_out; \
    SessionPtr->Total_Out = (src)->total_out;
#endif

/* Inline Functions */

/* If available, look at the next available byte in the input queue */
inline bool Peek_1(fd_session_t* SessionPtr, uint8_t* c)
{
    if ( (SessionPtr->Next_In != nullptr) && (SessionPtr->Avail_In > 0) )
    {
        *c = *(SessionPtr->Next_In);
        return( true );
    }
    else
        return( false );
}

/* If available, get a byte from the input queue */
inline bool Get_1(fd_session_t* SessionPtr, uint8_t* c)
{
    if ( (SessionPtr->Next_In != nullptr) && (SessionPtr->Avail_In > 0) )
    {
        *c = *(SessionPtr->Next_In)++;
        SessionPtr->Avail_In -= 1;
        SessionPtr->Total_In += 1;
        return( true );
    }
    else
        return( false );
}

/* If available, get N bytes from the input queue.  All N must be
   available for this call to succeed. */
inline bool Get_N(fd_session_t* SessionPtr, const uint8_t** c, uint32_t N)
{
    if ( (SessionPtr->Next_In != nullptr) && (SessionPtr->Avail_In >= N) )
    {
        *c = SessionPtr->Next_In;
        SessionPtr->Next_In += N;
        SessionPtr->Avail_In -= N;
        SessionPtr->Total_In += N;
        return( true );
    }
    else
        return( false );
}

/* If there's room in the output queue, put one byte. */
inline bool Put_1(fd_session_t* SessionPtr, uint8_t c)
{
    if ( (SessionPtr->Next_Out != nullptr) && (SessionPtr->Avail_Out > 0) )
    {
        *(SessionPtr->Next_Out)++ = c;
        SessionPtr->Avail_Out -= 1;
        SessionPtr->Total_Out += 1;
        return( true );
    }
    else
        return( false );
}

/* If the output queue has room available, place N bytes onto the queue.
   The output queue must have space for N bytes for this call to succeed. */
inline bool Put_N(fd_session_t* SessionPtr, const uint8_t* c, uint32_t N)
{
    if ( (SessionPtr->Next_Out != nullptr) && (SessionPtr->Avail_Out >= N) )
    {
        strncpy( (char*)SessionPtr->Next_Out, (const char*)c, N);
        SessionPtr->Next_Out += N;
        SessionPtr->Avail_Out -= N;
        SessionPtr->Total_Out += N;
        return( true );
    }
    else
        return( false );
}

/* If the input queue has at least one byte available AND there's at
   space for at least one byte in the output queue, then move one byte. */
inline bool Move_1(fd_session_t* SessionPtr)
{
    if ( (SessionPtr->Next_Out != nullptr) && (SessionPtr->Avail_Out > 0) &&
        (SessionPtr->Next_In != nullptr) && (SessionPtr->Avail_In > 0) )
    {
        *(SessionPtr->Next_Out) = *(SessionPtr->Next_In);
        SessionPtr->Next_In += 1;
        SessionPtr->Next_Out += 1;
        SessionPtr->Avail_In -= 1;
        SessionPtr->Avail_Out -= 1;
        SessionPtr->Total_In += 1;
        SessionPtr->Total_Out += 1;
        return( true );
    }
    else
        return( false );
}

/* If the input queue has at least N bytes available AND there's at
   space for at least N bytes in the output queue, then move all N bytes. */
inline bool Move_N(fd_session_t* SessionPtr, uint32_t N)
{
    if ( (SessionPtr->Next_Out != nullptr) && (SessionPtr->Avail_Out >= N) &&
        (SessionPtr->Next_In != nullptr) && (SessionPtr->Avail_In >= N) )
    {
        memcpy( (char*)SessionPtr->Next_Out, (const char*)SessionPtr->Next_In, N);
        SessionPtr->Next_In += N;
        SessionPtr->Next_Out += N;
        SessionPtr->Avail_In -= N;
        SessionPtr->Avail_Out -= N;
        SessionPtr->Total_In += N;
        SessionPtr->Total_Out += N;
        return( true );
    }
    else
        return( false );
}

/* API Functions */
namespace snort
{
/* Create a new decompression session object */
SO_PUBLIC fd_session_t* File_Decomp_New();

/* Initialize the session */
SO_PUBLIC fd_status_t File_Decomp_Init(fd_session_t*);

/* Run the incremental decompression engine */
SO_PUBLIC fd_status_t File_Decomp(fd_session_t*);

/* Close the decomp session processing */
SO_PUBLIC fd_status_t File_Decomp_End(fd_session_t*);

/* Close the current decomp session, but setup for another */
SO_PUBLIC fd_status_t File_Decomp_Reset(fd_session_t*);

/* Abort and delete the session */
SO_PUBLIC fd_status_t File_Decomp_StopFree(fd_session_t*);

/* Delete the session object */
SO_PUBLIC void File_Decomp_Free(fd_session_t*);

/* Call the error alerting call-back function */
SO_PUBLIC void File_Decomp_Alert(fd_session_t*, int Event);
}
#endif

file_decomp_pdf.h

Path = src/decompress/file_decomp_pdf.h

#ifndef FILE_DECOMP_PDF_H
#define FILE_DECOMP_PDF_H

#include <zlib.h>

#include "file_decomp.h"

#define ELEM_BUF_LEN        (12)
#define FILTER_SPEC_BUF_LEN (40)
#define PARSE_STACK_LEN     (12)

/* FIXIT-RC Other than the API prototypes, the other parts of this header should
   be private to file_decomp_pdf. */

enum fd_PDF_States
{
    PDF_STATE_NEW,
    PDF_STATE_LOCATE_STREAM,     /* Found sig bytes, looking for dictionary & stream */
    PDF_STATE_INIT_STREAM,       /* Init stream */
    PDF_STATE_PROCESS_STREAM     /* Processing stream */
};

struct fd_PDF_Parse_Stack_t
{
    uint8_t State;
    uint8_t Sub_State;
};

struct fd_PDF_Parse_t
{
    const uint8_t* xref_tok;
    uint32_t Obj_Number;
    uint32_t Gen_Number;
    uint8_t Parse_Stack_Index;
    uint8_t Sub_State;
    uint8_t State;
    uint8_t Dict_Nesting_Cnt;
    uint8_t Elem_Index;
    uint8_t Filter_Spec_Index;
    uint8_t Elem_Buf[ELEM_BUF_LEN];
    uint8_t Filter_Spec_Buf[FILTER_SPEC_BUF_LEN+1];
    fd_PDF_Parse_Stack_t Parse_Stack[PARSE_STACK_LEN];
};

struct fd_PDF_Deflate_t
{
    z_stream StreamDeflate;
};

struct fd_PDF_t
{
    union
    {
        fd_PDF_Deflate_t Deflate;
    } PDF_Decomp_State;
    fd_PDF_Parse_t Parse;
    uint8_t Decomp_Type;
    uint8_t State;
};

/* API Functions */

/* Init the PDF decompressor */
fd_status_t File_Decomp_Init_PDF(fd_session_t*);

/* Run the incremental PDF file parser/decompressor */
fd_status_t File_Decomp_PDF(fd_session_t*);

/* End the decompressor */
fd_status_t File_Decomp_End_PDF(fd_session_t*);

#endif

file_decomp_swf.h

Path = src/decompress/file_decomp_swf.h

#ifndef FILE_DECOMP_SWF_H
#define FILE_DECOMP_SWF_H

#ifdef HAVE_LZMA
#include <lzma.h>
#endif
#include <zlib.h>

#include "file_decomp.h"

/* FIXIT-RC Other than the API prototypes, the other parts of this header should
   be private to file_decomp_swf. */

/* Both ZLIB & LZMA files have an uncompressed eight byte header.  The signature is
   three bytes.  The header consists of a three byte sig, a one byte version,
   and a four byte uncompressed length (little-endian).  */

#define SWF_SIG_LEN       (3)
#define SWF_VER_LEN       (1)
#define SWF_UCL_LEN       (4)

/* LZMA Files have an additional nine bytes of header prior to the compressed data.
   This includes a four byte compressed length (little-endian) and five bytes
   of LZMA properties. */
#define SWF_LZMA_CML_LEN  (4)
#define SWF_LZMA_PRP_LEN  (5)

/* AFTER the sig, the max number of header bytes to fetch.
   VER+UCL+CML+LZMA_PRP -> 14 bytes. */
#define SWF_MAX_HEADER    (14)

/* Types */

enum fd_SWF_States
{
    SWF_STATE_NEW,
    SWF_STATE_GET_HEADER,     /* Found sig bytes, looking for end of uncomp header */
    SWF_STATE_PROC_HEADER,    /* Found header bytes, now process the header */
    SWF_STATE_DATA            /* Done with header, looking for start of data */
};

struct fd_SWF_t
{
    z_stream StreamZLIB;
#ifdef HAVE_LZMA
    lzma_stream StreamLZMA;
#endif
    uint8_t Header_Bytes[SWF_MAX_HEADER];
    uint8_t State;
    uint8_t Header_Len;
    uint8_t Header_Cnt;
};

/* API Functions */

/* Initialize the SWF file decompressor */
fd_status_t File_Decomp_Init_SWF(fd_session_t*);

/* Process the file incrementally */
fd_status_t File_Decomp_SWF(fd_session_t*);

/* End the SWF file decompression */
fd_status_t File_Decomp_End_SWF(fd_session_t*);

#endif

file_decomp_zip.h

Path = src/decompress/file_decomp_zip.h

#ifndef FILE_DECOMP_ZIP_H
#define FILE_DECOMP_ZIP_H

#include <zlib.h>

#include "file_decomp.h"

namespace snort
{
class BoyerMooreSearchCase;
}

#define MACRO_BINNAME_LEN 14

static const char* const macro_binname = "vbaProject.bin";

static const uint32_t ZIP_LOCAL_HEADER = 0x04034B50;
static const uint8_t header_pattern[4] = { 0x50, 0x4B, 0x03, 0x04 };
static const uint8_t DATA_DESC_BIT = 0x08;

enum fd_ZIP_states
{
    ZIP_STATE_LH,             // local header (4 bytes)

    // skipped:
    // ZIP_STATE_VER,         // version (2 bytes)

    ZIP_STATE_BITFLAG,        // bitflag (2 bytes)
    ZIP_STATE_METHOD,         // compression method (2 bytes)

    // skipped:
    // ZIP_STATE_MODTIME,     // modification time (2 bytes)
    // ZIP_STATE_MODDATE,     // modification date (2 bytes)
    // ZIP_STATE_CRC,         // CRC-32 (4 bytes)

    ZIP_STATE_COMPSIZE,       // compressed size (4 bytes)

    // skipped:
    // ZIP_STATE_UNCOMPSIZE,  // uncompressed size (4 bytes)

    ZIP_STATE_FILENAMELEN,    // filename length (2 bytes)
    ZIP_STATE_EXTRALEN,       // extra field length (2 bytes)

    ZIP_STATE_FILENAME,    // filename field (filenamelen bytes)

    //skipped:
    // ZIP_STATE_EXTRA,       // extra field (extralen bytes)
    // ZIP_STATE_STREAM,      // compressed stream (compsize bytes)

    ZIP_STATE_OLE_FILE,
    ZIP_STATE_INFLATE_INIT,   // initialize zlib inflate
    ZIP_STATE_INFLATE,        // perform zlib inflate
    ZIP_STATE_SEARCH,         // search for local header
    ZIP_STATE_SKIP            // skip state
};

struct fd_ZIP_t
{
    // zlib stream
    z_stream Stream;

    // decompression progress
    uint32_t progress;

    // ZIP fields
    uint32_t local_header;
    uint16_t bitflag;
    bool data_descriptor;
    uint16_t method;
    uint32_t compressed_size;
    uint16_t filename_length;
    uint16_t extra_length;
    char* file_name;
    // field index
    uint32_t Index;

    // current parser state
    fd_ZIP_states State;
    uint32_t Length;

    // next parser state
    fd_ZIP_states Next;
    uint32_t Next_Length;

    // local file header searcher
    snort::BoyerMooreSearchCase* header_searcher;
};

// allocate and set initial ZIP state
fd_status_t File_Decomp_Init_ZIP(fd_session_t*);

// end ZIP processing
fd_status_t File_Decomp_End_ZIP(fd_session_t*);

// run the ZIP state machine
fd_status_t File_Decomp_ZIP(fd_session_t*);

#endif

file_olefile.h

Path = src/decompress/file_olefile.h

#ifndef FILE_OLE_H
#define FILE_OLE_H

#include "file_oleheader.h"

#include <memory>
#include <unordered_map>

#include "detection/detection_engine.h"
#include "helpers/literal_search.h"
#include "ips_options/ips_vba_data.h"
#include "trace/trace_api.h"
#include "utils/util.h"
#include "utils/util_utf.h"

#define OLE_MAX_FILENAME_LEN_UTF16  64
#define OLE_MAX_FILENAME_ASCII      32

#define OLE_HEADER_LEN             512
#define DIR_ENTRY_SIZE             128
#define ROOT_ENTRY             "Root Entry"

#define SIG_COMP_CONTAINER        0x01
#define VBA_COMPRESSION_WINDOW    4096
#define MAX_VBA_BUFFER_LEN       16384

#define INVALID_SECTOR              -1

#define DIR_FILE_TYPE_OFFSET        66
#define DIR_COLOR_OFFSET            67
#define DIR_LEFT_SIB_OFFSET         68
#define DIR_RIGHT_SIB_OFFSET        72
#define DIR_ROOT_NODE_OFFSET        76
#define DIR_CLS_ID_OFFSET           80
#define DIR_STARTING_SEC_OFFSET    116
#define DIR_STREAM_SIZE_OFFSET     120
#define DIR_NEXT_ENTR_OFFSET       128

#define CURRENT_PACKET snort::DetectionEngine::get_current_packet()

#define VBA_DEBUG(module_name, module_id, log_level, p, ...) \
    trace_logf(log_level, module_name , module_id, p, __VA_ARGS__)

#define memcpy_id(destn, dsize, src, ssize) \
    ((dsize>=ssize) ? memcpy(destn, src, ssize) : memcpy( \
    destn, src, dsize))

enum object_type
{
    EMPTY = 0x00,
    STORAGE = 0x01,
    STREAM = 0x02,
    ROOT_STORAGE = 0x05
};

enum color_flag
{
    RED = 0x00,
    BLACK = 0x01
};

enum sector_type
{
    FAT_SECTOR = 0,
    MINIFAT_SECTOR = 1
};

int32_t cli_readn(const uint8_t*& fd, uint32_t& data_len, void* buff, int32_t count);

struct FileProperty
{
public:
    void set_name(uint8_t* f_name)
    {
        name = (char*)f_name;
    }

    char* get_name()
    {
        return name;
    }

    void set_file_type(const uint8_t* buf)
    {
        file_type = (object_type)*buf;
    }

    object_type get_file_type()
    {
        return file_type;
    }

    void set_color(const uint8_t* buf)
    {
        color = (color_flag)*buf;
    }

    color_flag get_color()
    {
        return color;
    }

    void set_lef_sib_id(const uint8_t* buf, byte_order_endianess endian)
    {
        lef_sib_id = (!endian) ? LETOHL_UNALIGNED(buf) : BETOHL_UNALIGNED(buf);
    }

    int32_t get_lef_sib_id()
    {
        return lef_sib_id;
    }

    void set_rig_sib_id(const uint8_t* buf, byte_order_endianess endian)
    {
        rig_sib_id = (!endian) ? LETOHL_UNALIGNED(buf) : BETOHL_UNALIGNED(buf);
    }

    int32_t get_rig_sib_id()
    {
        return rig_sib_id;
    }

    void set_root_node_id(const uint8_t* buf, byte_order_endianess endian)
    {
        root_node_id = (!endian) ? LETOHL_UNALIGNED(buf) : BETOHL_UNALIGNED(buf);
    }

    int32_t get_root_node_id()
    {
        return root_node_id;
    }

    void set_cls_id(const uint8_t* buf)
    {
        memcpy_id(cls_id, sizeof(cls_id), buf, 16);
    }

    char* get_cls_id()
    {
        return cls_id;
    }

    void set_starting_sector(const uint8_t* buf, byte_order_endianess endian)
    {
        starting_sector = (!endian) ? LETOHL_UNALIGNED(buf) : BETOHL_UNALIGNED(buf);
    }

    int32_t get_starting_sector()
    {
        return starting_sector;
    }

    void set_stream_size(const uint8_t* buf, byte_order_endianess endian)
    {
        stream_size = (!endian) ? LETOHLL_UNALIGNED(buf) : BETOHLL_UNALIGNED(buf);
    }

    int64_t get_stream_size()
    {
        return stream_size;
    }

    FileProperty() = default;

private:
    char* name = nullptr;
    object_type file_type = EMPTY;
    color_flag color = RED;
    int32_t lef_sib_id = 0;
    int32_t rig_sib_id = 0;
    int32_t root_node_id = 0;
    char cls_id[16] = {};
    int32_t starting_sector = 0;
    int64_t stream_size = 0;
};

class DirectoryList
{
public:
    std::list<FileProperty*> oleentry;
    snort::UtfDecodeSession* utf_state = nullptr;

    bool is_file_exists(char* name);
    FileProperty* get_file_node(char* name);
    int32_t get_file_sector(char* name);
    bool is_mini_sector(char* name);
    void set_mini_stream_sector(int32_t mini_stream_sector)
    {
        this->mini_stream_sector = mini_stream_sector;
    }

    int32_t get_mini_stream_sector()
    {
        return mini_stream_sector;
    }

    DirectoryList() = default;

    ~DirectoryList();

private:
    int32_t mini_stream_sector = -1;
};

class OleFile
{
public:
    bool parse_ole_header();
    void populate_fat_list();
    void populate_mini_fat_list();
    void walk_directory_list();
    void find_and_extract_vba(uint8_t*&, uint32_t&);
    int32_t get_next_fat_sector(int32_t sec_id);
    int32_t get_next_mini_fat_sector(int32_t sec_id);
    int32_t get_fat_offset(int32_t sec_id);
    int32_t get_mini_fat_offset(int32_t sec_id);
    int32_t get_file_offset(const uint8_t*, uint32_t data_len);

    void decompression(const uint8_t* data, uint32_t& data_len, uint8_t*& buffer,
        uint32_t& buffer_ofset);
    uint32_t find_bytes_to_copy(uint32_t byte_offset, uint32_t data_len,
                                   uint32_t stream_size, uint16_t sector_size);

    int search_nocase(const uint8_t* buffer, unsigned buffer_len) const
    {
        return searcher->search(search_handle, buffer, buffer_len);
    }

    OleFile(const uint8_t* file_buf, const uint32_t buf_len)
    {
        this->file_buf = file_buf;
        this->buf_len = buf_len;
    }

    ~OleFile()
    {
        delete header;
        delete dir_list;
        delete[] fat_list;
        delete[] mini_fat_list;
    }


private:
    void get_file_data(FileProperty*, uint8_t*&, uint32_t&);
    const uint8_t* file_buf;
    uint32_t buf_len;

    OleHeader* header = nullptr;
    DirectoryList* dir_list = nullptr;

    int32_t* fat_list = nullptr;
    int32_t fat_list_len = 0;
    int32_t* mini_fat_list = nullptr;
    int32_t mini_fat_list_len = 0;
};

void oleprocess(const uint8_t* const, const uint32_t, uint8_t*&, uint32_t&);
#endif

file_oleheader.h

Path = src/decompress/file_oleheader.h

#ifndef FILE_OLE_HEADER_H
#define FILE_OLE_HEADER_H

#include <cmath>
#include <cstring>

#include "main/snort_types.h"
#include "utils/endian.h"

#define MAX_DIFAT_SECTORS                 109

#define HEADER_MINOR_VER_OFFSET            24
#define HEADER_MAJOR_VER_OFFSET            26
#define HEADER_BYTE_ORDER_OFFSET           28
#define HEADER_SECTR_SIZE_OFFSET           30
#define HEADER_MIN_SECTR_SIZE_OFFSET       32
#define HEADER_DIR_SECTR_CNT_OFFSET        40
#define HEADER_FAT_SECTR_CNT_OFFSET        44
#define HEADER_FIRST_DIR_SECTR_OFFSET      48
#define HEADER_MINFAT_CUTOFF_OFFSET        56
#define HEADER_FIRST_MINFAT_OFFSET         60
#define HEADER_MINFAT_COUNT_OFFSET         64
#define HEADER_FIRST_DIFAT_OFFSET          68
#define HEADER_DIFAT_CNT_OFFSET            72
#define HEADER_DIFAT_ARRY_OFFSET           76

enum byte_order_endianess
{
    LITL_END = 0,
    BIG_END = 1
};

class OleHeader
{
public:
    bool set_byte_order(const uint8_t* buf);
    byte_order_endianess get_byte_order();
    bool match_ole_sig(const uint8_t* buf);
    void set_minor_version(const uint8_t* buf);
    uint16_t get_minor_version();
    void set_major_version(const uint8_t* buf);
    uint16_t get_major_version();
    void set_sector_size(const uint8_t* buf);
    uint16_t get_sector_size();
    void set_mini_sector_size(const uint8_t* buf);
    uint16_t get_mini_sector_size();
    void set_dir_sector_count(const uint8_t* buf);
    int32_t get_dir_sector_count();
    void set_first_dir(const uint8_t* buf);
    int32_t get_first_dir();
    void set_difat_count(const uint8_t* buf);
    int32_t get_difat_count();
    void set_fat_sector_count(const uint8_t* buf);
    int32_t get_fat_sector_count();
    void set_minifat_cutoff(const uint8_t* buf);
    uint32_t get_minifat_cutoff();
    void set_first_minifat(const uint8_t* buf);
    int32_t get_first_minifat();
    void set_minifat_count(const uint8_t* buf);
    int32_t get_minifat_count();
    void set_first_difat(const uint8_t* buf);
    int32_t get_first_difat();
    void set_difat_array(const uint8_t* buf);
    int32_t get_difat_array(int num);
    OleHeader() = default;

private:
    unsigned char sig[8] = {}; //0xD0, 0xCF, 0x11, 0xE0, 0xA1, 0xB1, 0x1A, 0xE1
    uint16_t minor_version = 0;
    uint16_t major_version = 0;
    uint16_t byte_order = 0;
    uint16_t sector_size = 0;
    uint16_t mini_sector_size = 0;
    int32_t dir_sector_count = 0;
    int32_t first_dir = -1;
    int32_t difat_count = 0;
    int32_t fat_sector_count = 0;
    uint32_t minifat_cutoff = 0;
    int32_t first_minifat = 0;
    int32_t minifat_count = 0;
    int32_t first_difat = 0;
    int32_t difat_array[MAX_DIFAT_SECTORS] = {};

    byte_order_endianess byte_order_endian = LITL_END;
};
#endif

decompress/test/

detection/

Rules are grouped by ports and services. An MPSE instance is created for:

  • protocol and source port(s)

  • protocol and dest ports(s)

  • protocol any ports

  • service to server

  • service to client

For each fast pattern match state, a detection option tree is created which allows Snort to efficiently evaluate a set of rules. The non-leaf nodes in this tree reference an IpsOption instance. The leaf nodes are OTNs, which represents the rule body. Attached to the OTN is one or more RTNs, which represent the rule head. There is one RTN for each policy in which the rule appears. (There is just one instance of each unique RTN in each policy to save space.) The RTN criteria are evaluated last to determine if an event should be generated.

Note that the fast pattern detection code refers to qualified events and non-qualified events. The latter are just fast pattern hits for which no rule fired. The former are fast pattern hits for which a rule actually fired.

Rules w/o fast patterns are grouped per the above and evaluated for each packet for which the group is selected. These are definitely bad for performance.

The following was written by Norton and Roelker on 2002/05/15 and predates the use of services but is still applicable.

Fast Packet Classification for Rule and Pattern Matching in SNORT

A simple method for grouping rules into lists and looking them up quickly in realtime.

There is a natural problem when aggregating rules into pattern groups for performing multi-pattern matching not seen with single pattern Boyer-Moore strategies. The problem is how to group the rules efficiently when considering that there are multiple parameters which govern what rules to apply to each packet or connection. The parameters sip, dip, sport, dport, and flags form an enormous address space of possible packets that must be tested in realtime against a subset of rule patterns. Methods to group patterns precisely based on all of these parameters can quickly become complicated by both algorithmic implications and implementation details. The procedure described herein is quick and simple.

Some implementations of MPSE have the capability to perform searches for multiple pattern groups over the same piece of data efficiently, rather than requiring a separate search for each group. A batch of searches can be built up from a single piece of data and passed to the search_batch() MPSE method, allowing MPSE specific optimization of how to carry out the searches to be performed.

The methodology presented here to solve this problem is based on the premise that we can use the source and destination ports to isolate pattern groups for pattern matching, and rely on an event validation procedure to authenticate other parameters such as sip, dip and flags after a pattern match is made. An intrinsic assumption here is that most sip and dip values will be acceptable and that the big gain in performance is due to the fact that by isolating traffic based on services (ports) we gain the most benefit. Additionally, and just as important, is the requirement that we can perform a multi-pattern recognition-inspection phase on a large set of patterns many times quicker than we can apply a single pattern test against many single patterns.

The current implementation assumes that for each rule the src and dst ports each have one of 2 possible values. Either a specific port number or the ANYPORT designation. This does allow us to handle port ranges and NOT port rules as well.

We make the following assumptions about classifying packets based on ports:

  1. There are Unique ports which represent special services. For example, ports 21,25,80,110,etc.

  2. Patterns can be grouped into Unique Pattern groups, and a Generic Pattern Group

    1. Unique pattern groups exist for source ports 21,25,80,110,etc.

    2. Unique pattern groups exist for destination ports 21,25,80,etc.

    3. A Generic pattern group exists for rules applied to every combination of source and destination ports.

We make the following assumptions about packet traffic:

  1. Well behaved traffic has one Unique port and one ephemeral port for most packets and sometimes legitimately, as in the case of DNS, has two unique ports that are the same. But we always determine that packets with two different but Unique ports is bogus, and should generate an alert. For example, if you have traffic going from port 80 to port 20.

  2. In fact, state could tell us which side of this connection is a service and which side is a client. Than we could handle this packet more precisely, but this is a rare situation and is still bogus. We can choose not to do pattern inspections on these packets or to do complete inspections.

Rules are placed into each group as follows:

  1. Src Port == Unique Service, Dst Port == ANY → Unique Src Port Table Src Port == Unique Service, Dst Port == Unique → Unique Src & Dst Port Tables

  2. Dst Port == Unique Service, Src Port == ANY → Unique Dst Port Table Dst Port == Unique Service, Src Port == Unique → Unique Dst & Src Port Tables

  3. Dst Port == ANY, Src Port == ANY → Generic Rule Set, And add to all Unique Src/Dst Rule Sets that have entries

  4. !Dst or !Src Port is the same as ANY Dst or ANY Src port respectively

  5. DstA:DstB is treated as an ANY port group, same for SrcA:SrcB

Initialization

For each rule check the dst-port, if it’s specific, then add it to the dst table. If the dst-port is Any port, then do not add it to the dst port table. Repeat this for the src-port.

If the rule has Any for both ports then it’s added generic rule list.

Also, fill in the Unique-Conflicts array, this indicates if it’s OK to have the same Unique service port for both destination and source. This will force an alert if it’s not ok. We optionally pattern match against this anyway.

Processing Rules

When packets arrive:

  1. Categorize the Port Uniqueness:

    1. Check the DstPort[DstPort] for possible rules, if no entry,then no rules exist for this packet with this destination.

    2. Check the SrcPort[SrcPort] for possible rules, if no entry,then no rules exist for this packet with this source.

  2. Process the Uniqueness:

    If a AND !b has rules or !a AND b has rules then
       match against those rules
    If a AND b have rules then
        if( sourcePort != DstPort )
            Alert on this traffic and optionally match both rule sets
       else if( SourcePort == DstPort )
           Check the Unique-Conflicts array for allowable conflicts
           if( NOT allowed )
               Alert on this traffic, optionally match the rules
    else
        match both sets of rules against this traffic
    If( !a AND ! b )  then
        Pattern Match against the Generic Rules ( these apply to all packets)

Pseudocode

PORT_RULE_MAP * prm;
PortGroup  *src, *dst, *generic;
RULE * prule; //user defined rule structure for user rules
prm = prmNewMap();
for( each rule )
{
    prule = ....get a rule pointer
    prmAddRule( prm, prule->dport, prule->sport, prule );
}
prmCompileGroups( prm );
while( sniff-packets )
{
    ....
    stat = prmFindRuleGroup( prm, dport, sport, &src, &dst, &generic );
    switch( stat )
    {
    case 0:  // No rules at all
        break;
    case 1:  // Dst Rules
        // pass 'dst->pgPatData', 'dst->pgPatDataUri' to the pattern engine
        break;
    case 2:  // Src Rules
        // pass 'src->pgPatData', 'src->pgPatDataUri' to the pattern engine
        break;
    case 3:  // Src/Dst Rules - Both ports represent Unique service ports
        // pass 'src->pgPatData' ,'src->pgPatDataUri' to the pattern engine
        // pass 'dst->pgPatData'  'src->pgPatDataUri' to the pattern engine
        break;
    case 4:  // Generic Rules Only
        // pass 'generic->pgPatData' to the pattern engine
        // pass 'generic->pgPatDataUri' to the pattern engine
        break;
    }
}

Stateful Signature Evaluation

Detection module processes data in block mode. Some features are available to extend limits of the block mode: large PDU with accumulated data (rebuilt packets), flowbits, and stateful signature evaluation (SSE). For data which forms a contiguous flow, a rule evaluation started in one packet may end up in subsequent packets.

From the user perspective, rule options are evaluated in left-to-right order, doing their calculations, checks, setting cursor position, switching buffers, etc. If at some point the cursor position is set beyond the current buffer boundary, when the rule’s evaluation context is stored on the flow and the rule fails for the current packet. Later, when more data arrives, Detection module checks if the flow has suspended contexts and resumes their evaluation after rule group selection and fast-pattern search are done but before other rules evaluated.

The Continuation concept represents a suspended evaluation:

  1. IPS options are evaluated in a known order, where they form pairs when the 1st option passes the evaluation context to the next option.

  2. The Continuation is merely the context (or a state) with a few things attached. So, it is somewhere between options, rather than in an option.

  3. The statement above lets us deal with a single evaluation pass, which is independent of what goes before and after it. (Avoiding multiple children nodes).

  4. Evaluation flow goes forward and backwards on this pass. The pass could be attempted several times, depending on presence of retry IPS options.

  5. On each attempt, the following could happen:

Forward:

1st option 2nd option Continuation is …

NO_MATCH

not evaluated

created and attached to the parent option (if cursor is awaiting data)

MATCH

NO_MATCH

created and attached to the child option (if original cursor was awaiting data)

MATCH

MATCH

not created (for this pass, but a subsequent pass can have its own continuation)

At this point, it doesn’t matter what happens after in the sub-tree. If the 1st and the 2nd options matched, that means a continuation is not needed here.

Backward:

option children Continuations spawned by the option …

(matched)

no match or partially matched

remain attached to the flow

(matched)

all matched

are recalled and erased from the flow

So, if some branches in the sub-tree failed to match, then continuations created at this node will give another try to these branches (later when more data become available). If sub-tree fully matched, then continuations are not needed (since the rule has fired) and will be recalled.

Pending continuations from the flow are picked up and updated/evaluated with respect to the buffer’s source (e.g. flow direction, file context, etc.)

In case of multiple calls for detection (the same packet and IPS context), continuations are created as usual. But just-created continuations are not evaluated immediately on the same packet, they will wait their turn in the next PDU. Additionally, if an inspector calls for detection on a single data block (like, a full attachment in HTTP), continuations can be disabled by providing no_flow flag to file_data buffer or any other buffer to indicate that the block is complete and no continuations needed.

Matched Buffers Dumps

Buffers that have been matched during detection are collected and stored on IPS context until logging takes place. Amount of collected buffers is represented with "detection.buf_dumps" peg count;

Caveats:

  1. Only buffers included in the rules explicitly will be collected.

  2. For multiple detection cases, all found instances of a matched buffer will be logged, even ones that may not be related to the rule.

context_switcher.h

Path = src/detection/context_switcher.h

#ifndef CONTEXT_SWITCHER_H
#define CONTEXT_SWITCHER_H

// ContextSwitcher maintains a set of contexts, only one of which can be
// active at any time. the normal workflow is:
//
// 1.  start and stop are called at the beginning and end of each wire
// packet which activates and releases one context from among those
// available.
//
// 2.  during processing interrupt and complete should be called to start
// and finish processing of a generated pseudo packet. it is possible to
// interrupt pseudo packets. complete may return without doing anything if
// dependent contexts were suspended.
//
// 3.  suspend may be called to pause the current context and activate the
// prior. multiple contexts may be suspended.
//
// 4.  there is no ordering of idle contexts. busy contexts are in strict LIFO
// order. context dependency chains are maintained in depth-first order by Flow.

#include <vector>

#include "detection/ips_context_chain.h"
#include "utils/primed_allocator.h"

namespace snort
{
class Flow;
class IpsContext;
class IpsContextData;
}

// FIXIT-E add the hold to catch offloads that don't return
class ContextSwitcher
{
public:
    ~ContextSwitcher();

    void push(snort::IpsContext*);

    void start();
    void stop();
    void abort();

    snort::IpsContext* interrupt();
    snort::IpsContext* complete();

    void suspend();
    void resume(snort::IpsContext*);

    snort::IpsContext* get_context() const;
    snort::IpsContext* get_next() const;

    snort::IpsContextData* get_context_data(unsigned id) const;
    void set_context_data(unsigned id, snort::IpsContextData*) const;

    unsigned idle_count() const;
    unsigned busy_count() const;

public:
    snort::IpsContextChain non_flow_chain;

private:
    std::vector<snort::IpsContext*> idle;
    std::vector<snort::IpsContext*> busy;
    std::vector<snort::IpsContext*> contexts;
};

#endif

detect.h

Path = src/detection/detect.h

#ifndef DETECT_H
#define DETECT_H

#include "detection/rules.h"
#include "main/snort_types.h"
#include "main/thread.h"

namespace snort
{
struct Packet;
struct ProfileStats;
}

extern THREAD_LOCAL snort::ProfileStats eventqPerfStats;

// main loop hooks
bool snort_ignore(snort::Packet*);
bool snort_log(snort::Packet*);

// alerts
void CallLogFuncs(snort::Packet*, ListHead*, struct Event*, const char*);
void CallLogFuncs(snort::Packet*, const OptTreeNode*, ListHead*);
void CallAlertFuncs(snort::Packet*, const OptTreeNode*, ListHead*);

void enable_tags();
void check_tags(snort::Packet*);

#endif

detection_continuation.h

Path = src/detection/detection_continuation.h

#ifndef DETECTION_CONTINUATION_H
#define DETECTION_CONTINUATION_H

#include "framework/cursor.h"
#include "framework/ips_option.h"
#include "ips_options/extract.h"
#include "latency/rule_latency.h"
#include "latency/rule_latency_state.h"
#include "main/snort_config.h"
#include "main/thread_config.h"
#include "protocols/packet.h"
#include "trace/trace_api.h"
#include "utils/grouped_list.h"
#include "utils/stats.h"

#include "detection_options.h"
#include "detect_trace.h"
#include "ips_context.h"
#include "rule_option_types.h"
#include "treenodes.h"

class Continuation
{
public:
    template <bool opt_parent>
    static inline void postpone(const Cursor&,
        const detection_option_tree_node_t&, const detection_option_eval_data_t&);

    static inline void recall(dot_node_state_t&, const snort::Packet*);

    inline bool is_reloaded() const;

    inline void eval(snort::Packet&);

private:
    Continuation(unsigned max_cnt) : states_cnt(0), states_cnt_max(max_cnt),
        reload_id(snort::SnortConfig::get_thread_reload_id())
    { }

    template <bool opt_parent>
    inline void add(const Cursor&,
        const detection_option_tree_node_t&, const detection_option_eval_data_t&);

    struct State
    {
        State() : data(), root(), selector(nullptr), node(nullptr), waypoint(0),
            original_waypoint(0), sid(0), packet_number(0), opt_parent(false)
        {
            for (uint8_t i = 0; i < NUM_IPS_OPTIONS_VARS; ++i)
                byte_extract_vars[i] = 0;
        }

        State(const detection_option_tree_node_t& n, const detection_option_eval_data_t& d,
            snort::IpsOption* s, unsigned wp, uint64_t id, bool p) : data(d),
            root(1, d.otn),
            selector(s), node(const_cast<detection_option_tree_node_t*>(&n)), waypoint(wp),
            original_waypoint(wp), sid(id), packet_number(d.p->context->packet_number),
            opt_parent(p)
        {
            for (uint8_t i = 0; i < NUM_IPS_OPTIONS_VARS; ++i)
                snort::GetVarValueByIndex(&byte_extract_vars[i], i);

            root.children = &node;
        }

        inline bool eval(snort::Packet&);

        detection_option_eval_data_t data;
        detection_option_tree_root_t root;
        snort::IpsOption* selector;
        detection_option_tree_node_t* node;
        unsigned waypoint;
        const unsigned original_waypoint;
        uint64_t sid;
        uint64_t packet_number;
        uint32_t byte_extract_vars[NUM_IPS_OPTIONS_VARS];
        bool opt_parent;
    };

    using LState = snort::GroupedList<State>;

    LState states;
    unsigned states_cnt;
    const unsigned states_cnt_max;
    const unsigned reload_id;
};

template <bool opt_parent>
void Continuation::postpone(const Cursor& cursor,
    const detection_option_tree_node_t& node, const detection_option_eval_data_t& data)
{
    if (!cursor.awaiting_data())
        return;

    if (!data.p->has_paf_payload())
        return;

    assert(data.p->flow);

    auto cont = data.p->flow->ips_cont;

    if (!cont)
    {
        auto max_cnt = snort::SnortConfig::get_conf()->max_continuations;

        if (!max_cnt)
            return;

        cont = data.p->flow->ips_cont = new Continuation(max_cnt);
        snort::pc.cont_flows++;
    }

    cont->add<opt_parent>(cursor, node, data);
}

void Continuation::recall(dot_node_state_t& nst,
     const snort::Packet* p)
{
    if (nst.last_check.context_num != nst.context_num or
        nst.last_check.run_num != nst.run_num)
        return;

    auto cnt = LState::erase_group((LState*&)nst.conts);
    assert(nst.conts == nullptr);
    assert(cnt);

    debug_logf(detection_trace, TRACE_CONT, nullptr,
        "The path matched, dropping %u continuation(s)\n", cnt);

    assert(p);
    assert(p->flow);
    assert(p->flow->ips_cont);

    auto cont = p->flow->ips_cont;

    assert(cnt <= cont->states_cnt);
    cont->states_cnt -= cnt;
    snort::pc.cont_recalls += cnt;
}

bool Continuation::is_reloaded() const
{
    return snort::SnortConfig::get_thread_reload_id() != reload_id;
}

void Continuation::eval(snort::Packet& p)
{
    if (!p.has_paf_payload())
        return;

    debug_logf(detection_trace, TRACE_CONT, nullptr,
        "Processing %u continuation(s)\n", states_cnt);

    if (states_cnt > snort::pc.cont_max_num)
        snort::pc.cont_max_num = states_cnt;

    auto i = states.get_next();

    while (i != &states)
    {
        auto st = i;
        i = i->get_next();

        if ((**st).eval(p))
        {
            assert(0 < states_cnt);
            assert(st != &states);
            --states_cnt;
            st->leave_group();
            delete st;
        }
    }
}

bool Continuation::State::eval(snort::Packet& p)
{
    Cursor cursor(&p);

    // Create local object to follow latency workflow
    RuleLatency::Context rule_latency_ctx(root, &p);

    if (RuleLatency::suspended())
        return true;

    if (packet_number == p.context->packet_number)
        return false;

    if (selector)
        selector->eval(cursor, &p);

    if (sid != cursor.id())
        return false;

    if (cursor.is_buffer_accumulated())
    {
        debug_logf(detection_trace, TRACE_CONT, nullptr,
            "Continuation dropped due to accumulated data in '%s'\n", cursor.get_name());

        return true;
    }

    cursor.set_pos(waypoint);

    if (cursor.awaiting_data(true) or cursor.size() == 0)
    {
        waypoint = cursor.get_next_pos();
        debug_logf(detection_trace, TRACE_CONT, data.p,
            "Continuation postponed, %u bytes to go\n", waypoint);
        return false;
    }

    assert(cursor.get_name());
    debug_logf(detection_trace, TRACE_CONT, data.p,
        "Cursor reached the position, evaluating sub-tree with "
        "current buffer '%s'\n", cursor.get_name());

    data.pmd = nullptr;
    data.p = &p;

    int result = 0;
    snort::pc.cont_evals++;

    for (uint8_t i = 0; i < NUM_IPS_OPTIONS_VARS; ++i)
        snort::SetVarValueByIndex(byte_extract_vars[i], i);

    const detection_option_tree_node_t* root_node = root.children[0];

    if (opt_parent)
    {
        for (int i = 0; i < root_node->num_children; ++i)
            result += detection_option_node_evaluate(root_node->children[i], data, cursor);
    }
    else
    {
        result = detection_option_node_evaluate(root_node, data, cursor);
    }

    if (data.leaf_reached and !data.otn->sigInfo.file_id)
    {
        data.p->context->matched_buffers.emplace_back(cursor.get_name(), cursor.buffer(), cursor.size());
        debug_logf(detection_trace, TRACE_BUFFER, data.p, "Collecting \"%s\" buffer of size %u on continuation root\n",
            cursor.get_name(), cursor.size());
        snort::pc.buf_dumps++;
    }

    clear_trace_cursor_info();

    if (result)
    {
        snort::pc.cont_matches++;
        snort::pc.cont_match_distance += original_waypoint;
    }
    else
    {
        snort::pc.cont_mismatches++;
        snort::pc.cont_mismatch_distance += original_waypoint;
    }

    return true;
}

template <bool opt_parent>
void Continuation::add(const Cursor& cursor,
    const detection_option_tree_node_t& node, const detection_option_eval_data_t& data)
{
    auto selector = data.buf_selector;
    auto pos = cursor.get_next_pos();
    auto sid = cursor.id();
    auto nst = node.state + snort::get_instance_id();
    assert(nst);

    if (nst->last_check.context_num != nst->context_num or
        nst->last_check.run_num != nst->run_num)
    {
        nst->context_num = nst->last_check.context_num;
        nst->run_num = nst->last_check.run_num;
        nst->conts = nullptr;
    }

    if (states_cnt < states_cnt_max)
    {
        ++states_cnt;
        new LState(states, (LState*&)nst->conts, node, data, selector, pos, sid, opt_parent);
    }
    else
    {
        debug_logf(detection_trace, TRACE_CONT, data.p,
            "Max size reached (%u), rejecting the oldest continuation\n",
            states_cnt_max);

        auto st = states.get_next();
        assert(st != &states);

        if ((**st).packet_number == data.p->context->packet_number)
            st->leave_group();
        delete st;

        new LState(states, (LState*&)nst->conts, node, data, selector, pos, sid, opt_parent);
    }

    snort::pc.cont_creations++;

    assert(cursor.get_name());
    assert(!selector || selector->get_name());
    debug_logf(detection_trace, TRACE_CONT, data.p,
        "Adding a continuation: "
        "current buffer '%s', buffer selector '%s'; %u more bytes needed\n",
        cursor.get_name(), selector ? selector->get_name() : "N/A", pos);
}

#endif

detection_engine.h

Path = src/detection/detection_engine.h

#ifndef DETECTION_ENGINE_H
#define DETECTION_ENGINE_H

// DetectionEngine manages a detection context.  To detect a rebuilt
// packet (PDU), first call set_next_packet().  If rebuild is successful,
// then instantiate a new DetectionEngine to detect that packet.

#include "detection/detection_util.h"
#include "detection/ips_context.h"
#include "main/snort_types.h"

struct DataPointer;
struct Replacement;

namespace snort
{
struct Packet;
class Flow;
class IpsContext;
class IpsContextChain;
class IpsContextData;

class SO_PUBLIC DetectionEngine
{
public:
    DetectionEngine();
    ~DetectionEngine();

public:
    static void thread_init();
    static void thread_term();

    static void reset();

    static IpsContext* get_context();

    static Packet* get_current_packet();
    static Packet* get_current_wire_packet();
    static Packet* set_next_packet(const Packet* parent = nullptr, Flow* flow = nullptr);
    static uint8_t* get_next_buffer(unsigned& max);

    static bool offload(Packet*);

    static void onload(Flow*);
    static void onload();
    static void idle();

    static void set_encode_packet(Packet*);
    static Packet* get_encode_packet();

    static void set_file_data(const DataPointer& dp);
    static void set_file_data(const DataPointer& dp, uint64_t id, bool is_accum, bool no_flow);
    static const DataPointer& get_file_data(const IpsContext*);
    static const DataPointer& get_file_data(const IpsContext*, uint64_t& id, bool& drop_sse, bool& no_sse);

    static uint8_t* get_buffer(unsigned& max);
    static inline DataPointer get_alt_buffer(const Packet*);
    static inline DataBuffer& acquire_alt_buffer(const Packet*);
    static inline void reset_alt_buffer(Packet*);

    static void set_data(unsigned id, IpsContextData*);
    static IpsContextData* get_data(unsigned id);
    static IpsContextData* get_data(unsigned id, IpsContext*);

    static void add_replacement(const std::string&, unsigned);
    static bool get_replacement(std::string&, unsigned&);
    static void clear_replacement();

    static bool detect(Packet*, bool offload_ok = false);
    static bool inspect(Packet*);

    static int queue_event(const struct OptTreeNode*);
    static int queue_event(unsigned gid, unsigned sid);

    static void disable_all(Packet*);
    static bool all_disabled(Packet*);

    static void disable_content(Packet*);
    static void enable_content(Packet*);
    static bool content_enabled(Packet*);

    static IpsContext::ActiveRules get_detects(Packet*);
    static void set_detects(Packet*, IpsContext::ActiveRules);

    static void set_check_tags(bool enable = true);
    static bool get_check_tags();

    static void wait_for_context();

private:
    static struct SF_EVENTQ* get_event_queue();
    static bool do_offload(snort::Packet*);
    static void offload_thread(IpsContext*);
    static void complete(snort::Packet*);
    static void resume(snort::Packet*);
    static void resume_ready_suspends(const IpsContextChain&);

    static int log_events(Packet*);
    static void clear_events(Packet*);
    static void finish_inspect_with_latency(Packet*);
    static void finish_inspect(Packet*, bool inspected);
    static void finish_packet(Packet*, bool flow_deletion = false);

private:
    IpsContext* context;
};

DataPointer DetectionEngine::get_alt_buffer(const Packet* p)
{
    assert(p);
    auto& alt_buf = p->context->alt_data;

    return { alt_buf.data, alt_buf.len };
}

DataBuffer& DetectionEngine::acquire_alt_buffer(const Packet* p)
{
    assert(p);

    auto& alt_buf = p->context->alt_data;

    if (!alt_buf.data)
        alt_buf.allocate_data();

    return alt_buf;
}

void snort::DetectionEngine::reset_alt_buffer(Packet *p)
{ p->context->alt_data.len = 0; }

static inline void set_file_data(const uint8_t* p, unsigned n)
{
    DataPointer dp { p, n };
    DetectionEngine::set_file_data(dp);
}

static inline void set_file_data(const uint8_t* p, unsigned n, uint64_t id, bool is_accum = false, bool no_flow = false)
{
    DataPointer dp { p, n };
    DetectionEngine::set_file_data(dp, id, is_accum, no_flow);
}

static inline void clear_file_data()
{ set_file_data(nullptr, 0); }

} // namespace snort
#endif

detection_module.h

Path = src/detection/detection_module.h

#ifndef DETECTION_MODULE_H
#define DETECTION_MODULE_H

#include "framework/module.h"

namespace snort
{
class DetectionModule : public Module
{
public:
    DetectionModule();

    bool begin(const char*, int, snort::SnortConfig*) override;
    bool set(const char*, Value&, SnortConfig*) override;
    bool end(const char*, int, SnortConfig*) override;

    const PegInfo* get_pegs() const override
    { return pc_names; }

    PegCount* get_counts() const override
    { return (PegCount*) &pc; }

    Usage get_usage() const override
    { return GLOBAL; }

    void set_trace(const Trace*) const override;
    const TraceOption* get_trace_options() const override;

private:
    bool add_service_extension(snort::SnortConfig*);

    std::string service;
    std::vector<std::string> extend_to;
};
}

#endif // DETECTION_MODULE_H

detection_options.h

Path = src/detection/detection_options.h

#ifndef DETECTION_OPTIONS_H
#define DETECTION_OPTIONS_H

// Support functions for rule option tree
//
// This implements tree processing for rule options, evaluating common
// detection options only once per pattern match.
//
// These trees are instantiated at parse time, one per MPSE match state.
// Eval, profiling, and latency data are attached in an array sized per max
// packet threads.

#include <sys/time.h>

#include "detection/rule_option_types.h"
#include "latency/rule_latency_state.h"
#include "main/thread_config.h"
#include "time/clock_defs.h"
#include "trace/trace_api.h"
#include "utils/util.h"

namespace snort
{
class HashNode;
class IpsOption;
class XHash;
struct Packet;
struct SnortConfig;
}
struct RuleLatencyState;
struct SigInfo;
struct OtnState;

typedef int (* eval_func_t)(void* option_data, class Cursor&, snort::Packet*);

// this is per packet thread
struct dot_node_state_t
{
    int result;
    struct
    {
        struct timeval ts;
        uint64_t context_num;
        uint32_t rebuild_flag;
        uint16_t run_num;
        char result;
        char flowbit_failed;
    } last_check;
    void* conts;
    uint64_t context_num;
    uint16_t run_num;

    // FIXIT-L perf profiler stuff should be factored of the node state struct
    hr_duration elapsed;
    hr_duration elapsed_match;
    hr_duration elapsed_no_match;
    uint64_t checks;
    uint64_t disables;

    unsigned latency_timeouts;
    unsigned latency_suspends;

    dot_node_state_t()
    {
        result = 0;
        conts = nullptr;
        memset(&last_check, 0, sizeof(last_check));
        context_num = run_num = 0;
        reset_profiling();
    }

    // FIXIT-L perf profiler stuff should be factored of the node state struct
    void update(hr_duration delta, bool match)
    {
        elapsed += delta;

        if ( match )
            elapsed_match += delta;
        else
            elapsed_no_match += delta;

        ++checks;
    }

    void reset_profiling()
    {
        elapsed = elapsed_match = elapsed_no_match = 0_ticks;
        checks = disables = 0;
        latency_suspends = latency_timeouts = 0;
    }
};

struct detection_option_tree_node_t;

struct detection_option_tree_bud_t
{
    int relative_children;
    int num_children;
    detection_option_tree_node_t** children;
    const struct OptTreeNode* otn;

    detection_option_tree_bud_t()
        : relative_children(0), num_children(0), children(nullptr), otn(nullptr) {}

    detection_option_tree_bud_t(int num, const OptTreeNode* _otn)
        : relative_children(0), num_children(num), children(nullptr), otn(_otn) {}

protected:
    ~detection_option_tree_bud_t() {}
};

struct detection_option_tree_node_t : public detection_option_tree_bud_t
{
    eval_func_t evaluate;
    void* option_data;
    dot_node_state_t* state;
    int is_relative;
    option_type_t option_type;

    detection_option_tree_node_t(option_type_t type, void* data) :
        evaluate(nullptr), option_data(data), is_relative(0), option_type(type)
    {
        state = new dot_node_state_t[snort::ThreadConfig::get_instance_max()];
    }

    ~detection_option_tree_node_t()
    {
        for (int i = 0; i < num_children; i++)
            delete children[i];

        snort_free(children);
        delete[] state;
    }
};

struct detection_option_tree_root_t : public detection_option_tree_bud_t
{
    RuleLatencyState* latency_state;

    detection_option_tree_root_t() : latency_state(nullptr) {}

    detection_option_tree_root_t(OptTreeNode* _otn)
        : detection_option_tree_bud_t(0, _otn)
    {
        latency_state = new RuleLatencyState[snort::ThreadConfig::get_instance_max()]();
    }

    detection_option_tree_root_t(int num, const OptTreeNode* _otn)
        : detection_option_tree_bud_t(num, _otn)
    {
        latency_state = new RuleLatencyState[snort::ThreadConfig::get_instance_max()]();
    }

    ~detection_option_tree_root_t()
    {
        delete[] latency_state;
    }
};

struct detection_option_eval_data_t
{
    const void* pmd;
    snort::Packet* p;
    snort::IpsOption* buf_selector;
    const struct OptTreeNode* otn;  // first rule in current processed tree
    char leaf_reached;
    char flowbit_failed;
    char flowbit_noalert;

    detection_option_eval_data_t()
        : pmd(nullptr), p(nullptr), buf_selector(nullptr), otn(nullptr)
        , leaf_reached(0), flowbit_failed(0), flowbit_noalert(0) {}

    detection_option_eval_data_t(snort::Packet* packet, const OptTreeNode* otn,
        const void* match_data = nullptr) : pmd(match_data), p(packet), buf_selector(nullptr)
        , otn(otn), leaf_reached(0), flowbit_failed(0), flowbit_noalert(0) {}

    detection_option_eval_data_t(const detection_option_eval_data_t& m)
        : pmd(m.pmd), p(m.p), buf_selector(m.buf_selector), otn(m.otn)
        , leaf_reached(m.leaf_reached), flowbit_failed(m.flowbit_failed), flowbit_noalert(m.flowbit_noalert) {}
};

// return existing data or add given and return nullptr
void* add_detection_option(struct snort::SnortConfig*, option_type_t, void*);
void* add_detection_option_tree(struct snort::SnortConfig*, detection_option_tree_node_t*);

int detection_option_node_evaluate(
    const detection_option_tree_node_t*, detection_option_eval_data_t&, const class Cursor&);

void print_option_tree(detection_option_tree_node_t*, int level);
void detection_option_tree_update_otn_stats(std::vector<snort::HashNode*>&,
    std::unordered_map<SigInfo*, OtnState>&, unsigned);
void detection_option_tree_reset_otn_stats(std::vector<snort::HashNode*>&, unsigned);

void free_detection_option_root(void** existing_tree);

#endif

detection_util.h

Path = src/detection/detection_util.h

#ifndef DETECTION_UTIL_H
#define DETECTION_UTIL_H

// this is a legacy junk-drawer file that needs to be refactored
// it provides file and alt data and event trace foo.

#include <cassert>

#include "actions/actions.h"
#include "main/snort_config.h"

#define DECODE_BLEN 65535

struct DataPointer
{
    DataPointer(const uint8_t* d, unsigned l) :
        data(d), len(l) {}
    const uint8_t* data;
    unsigned len;
};

struct DataBuffer
{
    static constexpr unsigned decode_blen = DECODE_BLEN;

    DataBuffer() = default;
    DataBuffer(const DataBuffer&) = delete;
    DataBuffer& operator=(const DataBuffer&) = delete;
    ~DataBuffer()
    { delete [] data; }

    void allocate_data()
    {
        assert(nullptr == data);
        const_cast<uint8_t*&>(data) = new uint8_t[DECODE_BLEN];
    }

    uint8_t* const data = nullptr;
    unsigned len = 0;
};

struct MatchedBuffer
{
    MatchedBuffer(const char* const n, const uint8_t* const d, unsigned s) :
        name(n), data(d), size(s)
    {}
    MatchedBuffer() = delete;

    const char* const name = nullptr;
    const uint8_t* const data = nullptr;
    unsigned size = 0;
};

// FIXIT-RC event trace should be placed in its own files
void EventTrace_Init();
void EventTrace_Term();

void EventTrace_Log(const snort::Packet*, const OptTreeNode*, Actions::Type action);

inline int EventTrace_IsEnabled(const snort::SnortConfig* sc)
{
    return ( sc->event_trace_max > 0 );
}

#endif

detect_trace.h

Path = src/detection/detect_trace.h

#ifndef DETECT_TRACE_H
#define DETECT_TRACE_H

// Detection trace utility

#include "framework/cursor.h"
#include "main/snort_types.h"
#include "main/thread.h"

namespace snort
{
struct Packet;
class Trace;
}

extern THREAD_LOCAL const snort::Trace* detection_trace;

struct detection_option_tree_node_t;
struct PatternMatchData;

enum
{
    TRACE_DETECTION_ENGINE = 0,
    TRACE_RULE_EVAL,
    TRACE_BUFFER,
    TRACE_RULE_VARS,
    TRACE_FP_SEARCH,
    TRACE_PKT_DETECTION,
    TRACE_OPTION_TREE,
    TRACE_TAG,
    TRACE_CONT,
};

void clear_trace_cursor_info();
void print_pkt_info(snort::Packet* p, const char*);
void print_pattern(const PatternMatchData* pmd, snort::Packet*);
void dump_buffer(const uint8_t* buff, unsigned len, snort::Packet*);
void node_eval_trace(const detection_option_tree_node_t* node, const Cursor& cursor, snort::Packet*);

#endif

fp_config.h

Path = src/detection/fp_config.h

#ifndef FP_CONFIG_H
#define FP_CONFIG_H

#include <string>

namespace snort
{
    struct MpseApi;
}

// this is a basically a factory for creating MPSE

#define PL_BLEEDOVER_WARNINGS_ENABLED        0x01
#define PL_DEBUG_PRINT_NC_DETECT_RULES       0x02
#define PL_DEBUG_PRINT_RULEGROUP_BUILD       0x04
#define PL_DEBUG_PRINT_RULEGROUPS_UNCOMPILED 0x08
#define PL_DEBUG_PRINT_RULEGROUPS_COMPILED   0x10
#define PL_SINGLE_RULE_GROUP                 0x20

class FastPatternConfig
{
public:
    FastPatternConfig();

    void set_debug_mode()
    { debug = true; }

    bool get_debug_mode() const
    { return debug; }

    void set_stream_insert(bool enable)
    { inspect_stream_insert = enable; }

    bool get_stream_insert() const
    { return inspect_stream_insert; }

    void set_max_queue_events(unsigned num_events)
    { max_queue_events = num_events; }

    unsigned get_max_queue_events() const
    { return max_queue_events; }

    void set_bleed_over_port_limit(unsigned n)
    { bleedover_port_limit = n; }

    int get_bleed_over_port_limit() const
    { return bleedover_port_limit; }

    int get_single_rule_group() const
    { return portlists_flags & PL_SINGLE_RULE_GROUP; }

    int get_bleed_over_warnings() const
    { return portlists_flags & PL_BLEEDOVER_WARNINGS_ENABLED; }

    int get_debug_print_nc_rules() const
    { return portlists_flags & PL_DEBUG_PRINT_NC_DETECT_RULES; }

    int get_debug_print_rule_group_build_details() const
    { return portlists_flags & PL_DEBUG_PRINT_RULEGROUP_BUILD; }

    int get_debug_print_rule_groups_compiled() const
    { return portlists_flags & PL_DEBUG_PRINT_RULEGROUPS_COMPILED; }

    int get_debug_print_rule_groups_uncompiled() const
    { return portlists_flags & PL_DEBUG_PRINT_RULEGROUPS_UNCOMPILED; }

    void set_debug_print_fast_patterns(bool b)
    { debug_print_fast_pattern = b; }

    bool get_debug_print_fast_patterns() const
    { return debug_print_fast_pattern; }

    void set_split_any_any(bool enable)
    { split_any_any = enable; }

    bool get_split_any_any() const
    { return split_any_any; }

    void set_single_rule_group()
    { portlists_flags |= PL_SINGLE_RULE_GROUP; }

    void set_bleed_over_warnings()
    { portlists_flags |= PL_BLEEDOVER_WARNINGS_ENABLED; }

    void set_debug_print_nc_rules()
    { portlists_flags |= PL_DEBUG_PRINT_NC_DETECT_RULES; }

    void set_debug_print_rule_group_build_details()
    { portlists_flags |= PL_DEBUG_PRINT_RULEGROUP_BUILD; }

    void set_debug_print_rule_groups_compiled()
    { portlists_flags |= PL_DEBUG_PRINT_RULEGROUPS_COMPILED; }

    void set_debug_print_rule_groups_uncompiled()
    { portlists_flags |= PL_DEBUG_PRINT_RULEGROUPS_UNCOMPILED; }

    void set_rule_db_dir(const char* s)
    { rule_db_dir = s; }

    const std::string& get_rule_db_dir() const
    { return rule_db_dir; }

    bool set_search_method(const char*);
    const char* get_search_method() const;

    bool set_offload_search_method(const char*);
    void set_max_pattern_len(unsigned);
    void set_queue_limit(unsigned);

    unsigned get_queue_limit() const
    { return queue_limit; }

    const snort::MpseApi* get_search_api() const
    { return search_api; }

    const snort::MpseApi* get_offload_search_api() const
    { return offload_search_api; }

    int get_num_patterns_truncated() const
    { return num_patterns_truncated; }

    unsigned set_max(unsigned bytes);

    bool deduplicate() const
    { return dedup; }

private:
    const snort::MpseApi* search_api = nullptr;
    const snort::MpseApi* offload_search_api = nullptr;

    bool inspect_stream_insert = true;
    bool split_any_any = false;
    bool debug_print_fast_pattern = false;
    bool debug = false;
    bool dedup = true;

    unsigned max_queue_events = 5;
    unsigned bleedover_port_limit = 1024;
    unsigned max_pattern_len = 0;

    unsigned queue_limit = 0;

    int portlists_flags = 0;
    unsigned num_patterns_truncated = 0;  // due to max_pattern_len

    std::string rule_db_dir;
};

#endif

fp_create.h

Path = src/detection/fp_create.h

#ifndef FPCREATE_H
#define FPCREATE_H

// this is where rule groups are compiled and MPSE are instantiated

#include <string>
#include "ports/port_group.h"

namespace snort
{
struct SnortConfig;
}

struct PMX
{
    struct PatternMatchData* pmd;
    RULE_NODE rule_node;
};

/* Used for negative content list */
struct NCListNode
{
    PMX* pmx;
    NCListNode* next;
};

/*
**  This is the main routine to create a FastPacket inspection
**  engine.  It reads in the snort list of RTNs and OTNs and
**  assigns them to PORT_MAPS.
*/
int fpCreateFastPacketDetection(snort::SnortConfig*);
void fpDeleteFastPacketDetection(snort::SnortConfig*);
void get_pattern_info(const PatternMatchData* pmd, std::string& hex, std::string& txt, std::string& opts);

#endif

fp_detect.h

Path = src/detection/fp_detect.h

#ifndef FPDETECT_H
#define FPDETECT_H

// this is where the high-level fast pattern matching action is
// rule groups are selected based on traffic and any fast pattern
// matches trigger rule tree evaluation.

#include "main/thread.h"
#include "profiler/profiler_defs.h"
#include "target_based/snort_protocols.h"

#define REBUILD_FLAGS (PKT_REBUILT_FRAG | PKT_REBUILT_STREAM)

namespace snort
{
class IpsContext;
struct Packet;
struct ProfileStats;
}

class Cursor;
struct RuleGroup;
struct OptTreeNode;

extern THREAD_LOCAL snort::ProfileStats mpsePerfStats;
extern THREAD_LOCAL snort::ProfileStats rulePerfStats;

struct RuleTreeNode;
int fpLogEvent(const RuleTreeNode*, const OptTreeNode*, snort::Packet*);
bool fp_eval_rtn(RuleTreeNode*, snort::Packet*, int check_ports);
int fp_eval_option(void*, Cursor&, snort::Packet*);
void populate_trace_data();

#define MAX_NUM_RULE_TYPES 16   // max number of allowed rule types

/*
**  This define is for the number of unique events
**  to match before choosing which event to log.
**  (Since we can only log one.) This define is the limit.
*/
#define MAX_EVENT_MATCH 100

/*
**  The events that are matched get held in this structure,
**  and iMatchIndex gets set to the event that holds the
**  highest priority.
*/
struct MatchInfo
{
    const OptTreeNode* MatchArray[MAX_EVENT_MATCH];
    unsigned iMatchCount;
    unsigned iMatchIndex;
    unsigned iMatchMaxLen;
};

/*
**  This structure holds information that is
**  referenced during setwise pattern matches.
**  It also contains information regarding the
**  number of matches that have occurred and
**  the event to log based on the event comparison
**  function.
*/
struct OtnxMatchData
{
    MatchInfo* matchInfo;
    bool have_match;
};

int fpAddMatch(OtnxMatchData*, const OptTreeNode*);

void fp_set_context(snort::IpsContext&);
void fp_clear_context(const snort::IpsContext&);

void fp_full(snort::Packet*);
void fp_partial(snort::Packet*);
void fp_complete(snort::Packet*, bool search = false);
void fp_eval_service_group(snort::Packet*, SnortProtocolId);

#endif

fp_utils.h

Path = src/detection/fp_utils.h

#ifndef FP_UTILS_H
#define FP_UTILS_H

// fast pattern utilities

#include <string>
#include <vector>

#include "framework/ips_option.h"
#include "framework/mpse.h"
#include "ports/port_group.h"

struct OptFpList;
struct OptTreeNode;

struct PatternMatchData* get_pmd(OptFpList*, SnortProtocolId, snort::RuleDirection);

bool make_fast_pattern_only(const OptFpList*, const PatternMatchData*);
bool is_fast_pattern_only(const OptTreeNode*, const OptFpList*, snort::Mpse::MpseType);
bool is_flowbit_setter(const OptFpList*);

PatternMatcher::Type get_pm_type(const std::string& buf);

bool set_fp_content(OptTreeNode*);

std::vector <PatternMatchData*> get_fp_content(
    OptTreeNode*, OptFpList*& pat, snort::IpsOption*& buf, bool srvc, bool only_literals, bool& exclude);

void queue_mpse(snort::Mpse*);
unsigned compile_mpses(struct snort::SnortConfig*, bool parallel = false);

bool has_service_rule_opt(OptTreeNode*);
void validate_services(struct snort::SnortConfig*, OptTreeNode*);

unsigned fp_serialize(const struct snort::SnortConfig*, const std::string& dir);
unsigned fp_deserialize(const struct snort::SnortConfig*, const std::string& dir);

void update_buffer_map(const char** bufs, const char* svc);
void add_default_services(struct snort::SnortConfig*, const std::string&, OptTreeNode*);

extern const char* section_to_str[];

#endif

ips_context_chain.h

Path = src/detection/ips_context_chain.h

#ifndef IPS_CONTEXT_CHAIN_H
#define IPS_CONTEXT_CHAIN_H

// IpsContextChain provides an interface for maintaining dependencies between
// IpsContexts. This class is provided to handle all linking and ensure only
// the tips of dependency chains are able to be processed, enforcing strict
// processing order.

namespace snort
{
class IpsContext;
class IpsContextChain
{
public:
    void abort()
    { _front = _back = nullptr; }

    IpsContext* front() const
    { return _front; }

    IpsContext* back() const
    { return _back; }

    void pop();
    void push_back(IpsContext*);

private:
    IpsContext* _front = nullptr;
    IpsContext* _back = nullptr;
};
}

#endif

ips_context_data.h

Path = src/detection/ips_context_data.h

#ifndef IPS_CONTEXT_DATA_H
#define IPS_CONTEXT_DATA_H

#include "main/snort_types.h"

#include "detection/detection_engine.h"

namespace snort
{
class SO_PUBLIC IpsContextData
{
public:
    virtual ~IpsContextData() = default;

    static unsigned get_ips_id();
    // Only unit tests can call this function to clear the id
    static void clear_ips_id();

    template<typename T>
    static T* get(unsigned ips_id)
    {
        T* data = (T*)DetectionEngine::get_data(ips_id);
        if ( ! data )
        {
            data = new T;
            DetectionEngine::set_data(ips_id, data);
        }
        return data;
    }
    virtual void clear() {}

protected:
    IpsContextData() = default;

private:
    static unsigned ips_id;
};
}
#endif

ips_context.h

Path = src/detection/ips_context.h

#ifndef IPS_CONTEXT_H
#define IPS_CONTEXT_H

// IpsContext provides access to all the state required for detection of a
// single packet.  the state is stored in IpsContextData instances, which
// are accessed by id.

#include <list>

#include "detection/detection_util.h"
#include "framework/codec.h"
#include "framework/mpse.h"
#include "framework/mpse_batch.h"
#include "main/snort_types.h"
#include "protocols/packet.h" // required to get a decent decl of pkth

class MpseStash;
struct OtnxMatchData;
struct SF_EVENTQ;
struct RegexRequest;

namespace snort
{
class IpsContextData;
struct SnortConfig;
struct Replacement
{
    std::string data;
    unsigned offset;
};

struct FlowSnapshot
{
    uint32_t session_flags;
    SnortProtocolId proto_id;
};

class SO_PUBLIC IpsContext
{
public:
    using Callback = void(*)(IpsContext*);
    enum State { IDLE, BUSY, SUSPENDED };

    IpsContext(unsigned size = 0);  // defaults to max id
    ~IpsContext();

    IpsContext(const IpsContext&) = delete;
    IpsContext& operator=(const IpsContext&) = delete;

    void setup();
    void clear();

    void set_context_data(unsigned id, IpsContextData*);
    IpsContextData* get_context_data(unsigned id) const;

    void snapshot_flow(Flow*);

    uint32_t get_session_flags()
    { return flow.session_flags; }

    SnortProtocolId get_snort_protocol_id()
    { return flow.proto_id; }

    void set_snort_protocol_id(SnortProtocolId id)
    { flow.proto_id = id; }

    void disable_detection();
    void disable_inspection();

    enum ActiveRules
    { NONE, NON_CONTENT, CONTENT };

    void register_post_callback(Callback callback)
    { post_callbacks.emplace_back(callback); }

    void clear_callbacks()
    { post_callbacks.clear(); }

    bool has_callbacks() const
    { return !post_callbacks.empty(); }

    void post_detection();

    void link(IpsContext* next)
    {
        assert(!next->depends_on);
        assert(!next->next_to_process);
        assert(!next_to_process);

        next->depends_on = this;
        next_to_process = next;
    }

    void unlink()
    {
        assert(!depends_on);
        if ( next_to_process )
        {
            assert(next_to_process->depends_on == this);
            next_to_process->depends_on = nullptr;
        }
        next_to_process = nullptr;
    }

    IpsContext* dependencies() const
    { return depends_on; }

    IpsContext* next() const
    { return next_to_process; }

    void abort()
    {
        if ( next_to_process )
            next_to_process->depends_on = depends_on;

        if ( depends_on )
            depends_on->next_to_process = next_to_process;

        depends_on = next_to_process = nullptr;
    }

public:
    std::vector<Replacement> rpl;

    Packet* packet;
    Packet* wire_packet = nullptr;
    Packet* encode_packet;
    DAQ_PktHdr_t* pkth;
    uint8_t* buf;

    const SnortConfig* conf = nullptr;
    MpseBatch searches;
    MpseStash* stash;
    OtnxMatchData* otnx;
    std::list<RegexRequest*>::iterator regex_req_it;
    SF_EVENTQ* equeue;

    DataPointer file_data = DataPointer(nullptr, 0);
    uint64_t file_data_id = 0;
    bool file_data_drop_sse = false;
    bool file_data_no_sse = false;
    DataBuffer alt_data = {};
    unsigned file_pos = 0;
    bool file_type_process = false;

    uint64_t context_num;
    uint64_t packet_number = 0;
    ActiveRules active_rules;
    State state;
    bool check_tags;
    bool clear_inspectors;

    static const unsigned buf_size = Codec::PKT_MAX;

    // FIXIT-L eliminate max_ips_id and just resize data vector.
    // Only 5 inspectors currently use the ips context data.
    static constexpr unsigned max_ips_id = 8;

    std::vector<MatchedBuffer> matched_buffers;

private:
    FlowSnapshot flow = {};
    std::vector<IpsContextData*> data;
    std::vector<unsigned> ids_in_use;  // for indirection; FIXIT-P evaluate alternatives
    std::vector<Callback> post_callbacks;
    IpsContext* depends_on;
    IpsContext* next_to_process;
    bool remove_gadget = false;
};
}
#endif

pattern_match_data.h

Path = src/detection/pattern_match_data.h

#ifndef PATTERN_MATCH_DATA_H
#define PATTERN_MATCH_DATA_H

#include <sys/time.h>

#include <cctype>
#include <string>
#include <vector>

#include "framework/ips_option.h"  // FIXIT-L not a good dependency

struct PmdLastCheck
{
    struct timeval ts;
    uint64_t context_num;
    uint32_t rebuild_flag;
    uint16_t run_num;
};

struct PatternMatchData
{
    // FIXIT-L wasting some memory here:
    // - this is not used by content option logic directly
    // - and only used on current eval (not across packets)
    // (partly mitigated by only allocating if exception_flag is set)
    //
    /* Set if fast pattern matcher found a content in the packet,
       but the rule option specifies a negated content. Only
       applies to negative contents that are not relative */
    PmdLastCheck* last_check;
    const char* sticky_buf = nullptr;  // provides context to contents
    //----------------------------------------------------------------
    // data above this point is for framework use only!
    //----------------------------------------------------------------

    const char* pattern_buf; // app layer pattern to match on
    unsigned pattern_size;   // size of app layer pattern

    int offset;              // pattern search start offset
    unsigned depth;          // pattern search depth

    enum
    {
        NEGATED  = 0x01,
        NO_CASE  = 0x02,
        RELATIVE = 0x04,
        LITERAL  = 0x08,
        FAST_PAT = 0x10,
        NO_FP    = 0x20,
    };

    uint16_t flags = 0;          // from above enum
    uint16_t mpse_flags;     // passed through to mpse

    uint16_t fp_offset;
    uint16_t fp_length;

    bool is_unbounded() const
    { return !depth; }

    void set_fast_pattern()
    { flags |= FAST_PAT; }

    void set_negated()
    { flags |= NEGATED; }

    void set_no_case()
    { flags |= NO_CASE; }

    void set_relative()
    { flags |= RELATIVE; }

    void set_literal()
    { flags |= LITERAL; }

    bool is_fast_pattern() const
    { return (flags & FAST_PAT) != 0; }

    bool is_negated() const
    { return (flags & NEGATED) != 0; }

    bool is_no_case() const
    { return (flags & NO_CASE) != 0; }

    bool is_relative() const
    { return (flags & RELATIVE) != 0; }

    bool is_literal() const
    { return (flags & LITERAL) != 0; }

    bool can_be_fp() const;

    bool has_alpha() const;
};

typedef std::vector<PatternMatchData*> PatternMatchVector;

inline bool PatternMatchData::can_be_fp() const
{
    if ( !pattern_buf or !pattern_size )
        return false;

    if ( flags & NO_FP )
        return false;

    if ( !is_negated() )
        return true;

    // Negative contents can only be considered if they are not
    // relative and don't have any offset or depth.  This is because
    // the pattern matcher does not take these into consideration and
    // may find the content in a non-relevant section of the payload
    // and thus disable the rule when it shouldn't be.

    // Also case sensitive patterns cannot be considered since patterns
    // are inserted into the pattern matcher without case which may
    // lead to false negatives.

    if ( is_relative() or !is_no_case() or offset or depth )
        return false;

    return true;
}

inline bool PatternMatchData::has_alpha() const
{
    unsigned tmp_offset = static_cast<unsigned>(fp_offset);
    unsigned tmp_length = fp_length ? fp_length : pattern_size;

    for ( unsigned idx = 0; idx < tmp_length; ++idx )
    {
        if ( isalpha(pattern_buf[tmp_offset + idx]) )
            return true;
    }
    return false;
}

#endif

pcrm.h

Path = src/detection/pcrm.h

#ifndef PCRM_H
#define PCRM_H

// Packet Classification-Rule Manager
// rule groups by source and dest ports as well as any
// (generic refers to any)

#include "ports/port_group.h"
#include "protocols/packet.h"

#define ANYPORT (-1)

struct PORT_RULE_MAP
{
    int prmNumDstRules;
    int prmNumSrcRules;
    int prmNumGenericRules;

    int prmNumDstGroups;
    int prmNumSrcGroups;

    RuleGroup* prmSrcPort[snort::MAX_PORTS];
    RuleGroup* prmDstPort[snort::MAX_PORTS];
    RuleGroup* prmGeneric;
};

PORT_RULE_MAP* prmNewMap();

int prmFindRuleGroupTcp(PORT_RULE_MAP*, int, int, RuleGroup**, RuleGroup**, RuleGroup**);
int prmFindRuleGroupUdp(PORT_RULE_MAP*, int, int, RuleGroup**, RuleGroup**, RuleGroup**);
int prmFindRuleGroupIp(PORT_RULE_MAP*, int, RuleGroup**, RuleGroup**);
int prmFindRuleGroupIcmp(PORT_RULE_MAP*, int, RuleGroup**, RuleGroup**);

#endif

regex_offload.h

Path = src/detection/regex_offload.h

#ifndef REGEX_OFFLOAD_H
#define REGEX_OFFLOAD_H

// RegexOffload provides an interface to fast pattern search accelerators.
// There are two flavors: MPSE and thread.  The MpseRegexOffload interfaces to
// an MPSE that is capable of regex offload such as the RXP whereas
// ThreadRegexOffload implements the regex search in auxiliary threads w/o
// requiring extra MPSE instances.  presently all offload is per packet thread;
// packet threads do not share offload resources.

#include <condition_variable>
#include <list>
#include <thread>

namespace snort
{
class Flow;
struct Packet;
struct SnortConfig;
}
struct RegexRequest;

class RegexOffload
{
public:
    static RegexOffload* get_offloader(unsigned max, bool async);
    virtual ~RegexOffload();

    virtual void stop();

    virtual void put(snort::Packet*) = 0;
    virtual bool get(snort::Packet*&) = 0;

    unsigned available() const
    { return idle.size(); }

    unsigned count() const
    { return busy.size(); }

    bool on_hold(const snort::Flow*) const;

protected:
    RegexOffload(unsigned max);

protected:
    std::list<RegexRequest*> busy;
    std::list<RegexRequest*> idle;
};

class MpseRegexOffload : public RegexOffload
{
public:
    MpseRegexOffload(unsigned max);

    void put(snort::Packet*) override;
    bool get(snort::Packet*&) override;
};

class ThreadRegexOffload : public RegexOffload
{
public:
    ThreadRegexOffload(unsigned max);
    ~ThreadRegexOffload() override;

    void stop() override;

    void put(snort::Packet*) override;
    bool get(snort::Packet*&) override;

private:
    static void worker(RegexRequest*, const snort::SnortConfig*, unsigned id);
};

#endif

rtn_checks.h

Path = src/detection/rtn_checks.h

#ifndef RTN_CHECKS_H
#define RTN_CHECKS_H

namespace snort
{
    struct Packet;
}
struct RuleFpList;
struct RuleTreeNode;

// parsing
int RuleListEnd(snort::Packet*, RuleTreeNode*, RuleFpList*, int);

// detection
int CheckBidirectional(snort::Packet*, RuleTreeNode*, RuleFpList*, int);

int CheckProto(snort::Packet*, RuleTreeNode*, RuleFpList*, int);
int CheckSrcIP(snort::Packet*, RuleTreeNode*, RuleFpList*, int);
int CheckDstIP(snort::Packet*, RuleTreeNode*, RuleFpList*, int);

int CheckSrcPortEqual(snort::Packet*, RuleTreeNode*, RuleFpList*, int);
int CheckDstPortEqual(snort::Packet*, RuleTreeNode*, RuleFpList*, int);
int CheckSrcPortNotEq(snort::Packet*, RuleTreeNode*, RuleFpList*, int);
int CheckDstPortNotEq(snort::Packet*, RuleTreeNode*, RuleFpList*, int);

#endif

rule_option_types.h

Path = src/detection/rule_option_types.h

#ifndef RULE_OPTION_TYPES_H
#define RULE_OPTION_TYPES_H

enum option_type_t
{
    RULE_OPTION_TYPE_LEAF_NODE,    // internal use by rule compiler
    RULE_OPTION_TYPE_CONTENT,      // ideally would be eliminated
    RULE_OPTION_TYPE_FLOWBIT,      // ideally would be eliminated
    RULE_OPTION_TYPE_OTHER         // for all new buffer independent rule options
};

#endif

rules.h

Path = src/detection/rules.h

#ifndef RULES_H
#define RULES_H

// misc rule and rule list support
// FIXIT-L refactor this header

#include <map>
#include <string>

#include "actions/actions.h"
#include "main/policy.h"

#define GID_DEFAULT          1
#define GID_SESSION        135

#define GID_BUILTIN_MIN     40
#define GID_BUILTIN_MAX    999

#define SESSION_EVENT_SYN_RX 1
#define SESSION_EVENT_SETUP  2
#define SESSION_EVENT_CLEAR  3

#define EventIsInternal(gid) ((gid) == GID_SESSION)

namespace snort
{
    class IpsAction;
    struct SnortConfig;
}
struct OutputSet;
struct RuleTreeNode;

struct ListHead
{
    OutputSet* LogList;
    OutputSet* AlertList;
    struct RuleListNode* ruleListNode;
};

// for top-level rule lists by type (alert, drop, etc.)
struct RuleListNode
{
    ListHead* RuleList;   /* The rule list associated with this node */
    Actions::Type mode;        /* the rule mode */
    unsigned evalIndex;        /* eval index for this rule set */
    char* name;           /* name of this rule list */
    RuleListNode* next;   /* the next RuleListNode */
};

struct RuleKey
{
    unsigned policy_id;
    unsigned gid;
    unsigned sid;

    friend bool operator< (const RuleKey&, const RuleKey&);
};

struct RuleState
{
    std::string rule_action;
    uint8_t action;
    IpsPolicy::Enable enable;
};

class RuleStateMap
{
public:
    void add(const RuleKey& key, const RuleState& state)
    { map[key] = state; }

    void apply(snort::SnortConfig*);

private:
    RuleTreeNode* dup_rtn(RuleTreeNode*, IpsPolicy*);
    void update_rtn(snort::SnortConfig*, RuleTreeNode*, const RuleState&);
    void apply(snort::SnortConfig*, OptTreeNode*, unsigned ips_num, const RuleState&);

private:
    std::map<RuleKey, RuleState> map;
};

#endif

service_map.h

Path = src/detection/service_map.h

#ifndef SERVICE_MAP_H
#define SERVICE_MAP_H

//  for managing rule groups by service
//  direction to client and to server are separate

#include <vector>

#include "target_based/snort_protocols.h"

namespace snort
{
struct SnortConfig;
class GHash;
}
struct RuleGroup;

//  Service Rule Map Master Table
struct srmm_table_t
{
    snort::GHash* to_srv;
    snort::GHash* to_cli;
};

srmm_table_t* ServiceMapNew();
void ServiceMapFree(srmm_table_t*);

srmm_table_t* ServiceRuleGroupMapNew();
void ServiceRuleGroupMapFree(srmm_table_t*);

void fpPrintServiceRuleGroupSummary(snort::SnortConfig*);
void fpCreateServiceMaps(snort::SnortConfig*);

//  Service/Protocol Ordinal To RuleGroup table
typedef std::vector<RuleGroup*> RuleGroupVector;

struct sopg_table_t
{
    sopg_table_t(unsigned size);
    RuleGroup* get_port_group(bool c2s, SnortProtocolId svc);

    RuleGroupVector to_srv;
    RuleGroupVector to_cli;
};


#endif

sfrim.h

Path = src/detection/sfrim.h

#ifndef SFRIM_H
#define SFRIM_H

// provides an ordinal for each rule so they can be looked up by a number
// used during parse time when rules are compiled

struct rule_index_map_t;

rule_index_map_t* RuleIndexMapCreate();
void RuleIndexMapFree(rule_index_map_t*);

int RuleIndexMapAdd(rule_index_map_t*, unsigned gid, unsigned sid);
bool RuleIndexMapGet(rule_index_map_t* map, int index, unsigned& gid, unsigned& sid);

#endif

signature.h

Path = src/detection/signature.h

#ifndef SIGNATURE_H
#define SIGNATURE_H

// basic non-detection signature info:  gid, sid, rev, class, priority, etc.

#include <cstdint>
#include <cstdio>
#include <string>

#include "rules.h"
#include "target_based/snort_protocols.h"

namespace snort
{
class GHash;
struct SnortConfig;
}

struct OptTreeNode;

struct ReferenceSystem
{
    ReferenceSystem(const std::string& n, const char* u) : name(n), url(u) { }
    std::string name;
    std::string url;
};

const ReferenceSystem* reference_system_add(snort::SnortConfig*, const std::string&, const char* = "");

struct ReferenceNode
{
    ReferenceNode(const ReferenceSystem* sys, const std::string& id) : system(sys), id(id) { }
    const ReferenceSystem* system;
    std::string id;
};

void add_reference(snort::SnortConfig*, OptTreeNode*, const std::string& sys, const std::string& id);

struct ClassType
{
    ClassType(const char* s, const char* txt, unsigned pri, int id) :
        name(s), text(txt), priority(pri), id(id) { }

    std::string name;
    std::string text;
    unsigned priority;
    int id;
};

void add_classification(snort::SnortConfig*, const char* name, const char* text, unsigned priority);

const ClassType* get_classification(snort::SnortConfig*, const char*);

struct SignatureServiceInfo
{
    SignatureServiceInfo(const char* s, SnortProtocolId proto) :
        service(s), snort_protocol_id(proto) { }
    std::string service;
    SnortProtocolId snort_protocol_id;
};

struct OtnKey
{
    uint32_t gid;
    uint32_t sid;
};

enum Target
{ TARGET_NONE, TARGET_SRC, TARGET_DST, TARGET_MAX = TARGET_DST };

struct SigInfo
{
    std::string message;
    std::string* body = nullptr;

    std::vector<const ReferenceNode*> refs;
    std::vector<SignatureServiceInfo> services;

    const ClassType* class_type = nullptr;

    uint32_t gid = GID_DEFAULT;
    uint32_t sid = 0;
    uint32_t rev = 0;

    uint32_t class_id = 0;
    uint32_t priority = 0;

    bool builtin = false;
    Target target = TARGET_NONE;
    uint64_t file_id = 0;
};

snort::GHash* OtnLookupNew();
void OtnLookupAdd(snort::GHash*, OptTreeNode*);
OptTreeNode* OtnLookup(snort::GHash*, uint32_t gid, uint32_t sid);
void OtnLookupFree(snort::GHash*);
void OtnRemove(snort::GHash*, OptTreeNode*);

void dump_msg_map(const snort::SnortConfig*);
void dump_rule_deps(const snort::SnortConfig*);
void dump_rule_meta(const snort::SnortConfig*);
void dump_rule_state(const snort::SnortConfig*);

#endif

tag.h

Path = src/detection/tag.h

#ifndef TAG_H
#define TAG_H

// rule option tag causes logging of some number of subsequent packets
// following an alert.  this module is use by the tag option to implement
// that functionality.  uses its own hash table.
//
// FIXIT-L convert tags to use flow instead of hash table.

#include <cstdint>

namespace snort
{
struct Packet;
}

struct OptTreeNode;
struct Event;

#define GID_TAG       2
#define TAG_LOG_PKT   1

#define TAG_SESSION   1
#define TAG_HOST      2
#define TAG_HOST_SRC  3
#define TAG_HOST_DST  4

#define TAG_METRIC_SECONDS    0x01
#define TAG_METRIC_PACKETS    0x02
#define TAG_METRIC_BYTES      0x04
#define TAG_METRIC_UNLIMITED  0x08
#define TAG_METRIC_SESSION    0x10

struct TagData
{
    int tag_type;       /* tag type (session/host) */
    int tag_metric;     /* (packets | seconds | bytes) units */
    int tag_direction;  /* source or dest, used for host tagging */

    uint32_t tag_seconds;    /* number of "seconds" units to tag for */
    uint32_t tag_packets;    /* number of "packets" units to tag for */
    uint32_t tag_bytes;      /* number of "type" units to tag for */
};

void InitTag();
void CleanupTag();
int CheckTagList(snort::Packet*, Event&, void**);
void SetTags(const snort::Packet*, const OptTreeNode*, uint16_t);

#endif

treenodes.h

Path = src/detection/treenodes.h

#ifndef TREENODES_H
#define TREENODES_H

// rule header (RTN) and body (OTN) nodes

#include <string>

#include "actions/actions.h"
#include "detection/signature.h"
#include "detection/rule_option_types.h"
#include "framework/pdu_section.h"
#include "main/policy.h"
#include "main/snort_types.h"
#include "ports/port_group.h"
#include "time/clock_defs.h"

namespace snort
{
class IpsOption;
struct Packet;
}
struct RuleTreeNode;
struct PortObject;
struct OutputSet;
struct TagData;
struct sfip_var_t;

/* same as the rule header FP list */
struct OptFpList
{
    snort::IpsOption* ips_opt;

    int (* OptTestFunc)(void* option_data, class Cursor&, snort::Packet*);

    OptFpList* next;

    unsigned char isRelative;
    option_type_t type;
};

struct OtnState
{
    // profiling
    // FIXIT-L factor the profiling stuff out
    hr_duration elapsed = 0_ticks;
    hr_duration elapsed_match = 0_ticks;
    hr_duration elapsed_no_match = 0_ticks;

    uint64_t checks = 0;
    uint64_t matches = 0;
    uint8_t noalerts = 0;
    uint64_t alerts = 0;

    uint64_t latency_timeouts = 0;
    uint64_t latency_suspends = 0;

    bool is_active() const
    { return elapsed > CLOCK_ZERO || checks > 0; }
};

/* function pointer list for rule head nodes */
// FIXIT-L use bit mask to determine what header checks to do
// cheaper than traversing a list and uses much less memory
struct RuleFpList
{
    /* context data for this test */
    void* context = nullptr;

    /* rule check function pointer */
    int (* RuleHeadFunc)(snort::Packet*, RuleTreeNode*, RuleFpList*, int) = nullptr;

    /* pointer to the next rule function node */
    RuleFpList* next = nullptr;
};

struct RuleHeader
{
    RuleHeader(const char* s) : action(s) { }

    std::string action;
    std::string proto;
    std::string src_nets;
    std::string src_ports;
    std::string dir;
    std::string dst_nets;
    std::string dst_ports;
};

// one of these per rule per policy
// represents head part of rule
struct RuleTreeNode
{
    using Flag = uint8_t;
    static constexpr Flag ENABLED       = 0x01;
    static constexpr Flag ANY_SRC_PORT  = 0x02;
    static constexpr Flag ANY_DST_PORT  = 0x04;
    static constexpr Flag ANY_FLAGS     = 0x08;
    static constexpr Flag BIDIRECTIONAL = 0x10;
    static constexpr Flag ANY_SRC_IP    = 0x20;
    static constexpr Flag ANY_DST_IP    = 0x40;
    static constexpr Flag USER_MODE     = 0x80;

    RuleFpList* rule_func = nullptr; /* match functions.. (Bidirectional etc.. ) */
    RuleHeader* header = nullptr;

    sfip_var_t* sip = nullptr;
    sfip_var_t* dip = nullptr;

    PortObject* src_portobject = nullptr;
    PortObject* dst_portobject = nullptr;

    struct ListHead* listhead = nullptr;

    SnortProtocolId snort_protocol_id = 0;

    // reference count from otn.
    // Multiple OTNs can reference this RTN with the same policy.
    unsigned int otnRefCount = 0; // FIXIT-L shared_ptr?

    Actions::Type action = 0;

    uint8_t flags = 0;

    void set_enabled()
    { flags |= ENABLED; }

    void clear_enabled()
    { flags &= (~ENABLED); }

    bool enabled() const
    { return (flags & ENABLED) != 0; }

    bool user_mode() const
    { return (flags & USER_MODE) != 0; }

    bool any_src_port() const
    { return (flags & ANY_SRC_PORT) != 0; }

    bool any_dst_port() const
    { return (flags & ANY_DST_PORT) != 0; }

    bool any_any_port() const
    { return any_src_port() and any_dst_port(); }
};

// one of these for each rule
// represents body part of rule
struct OptTreeNode
{
    ~OptTreeNode();

    using Flag = uint8_t;
    static constexpr Flag WARNED_FP  = 0x01;
    static constexpr Flag STATELESS  = 0x02;
    static constexpr Flag RULE_STATE = 0x04;
    static constexpr Flag META_MATCH = 0x08;
    static constexpr Flag TO_CLIENT  = 0x10;
    static constexpr Flag TO_SERVER  = 0x20;
    static constexpr Flag BIT_CHECK  = 0x40;
    static constexpr Flag SVC_ONLY   = 0x80;

    /* metadata about signature */
    SigInfo sigInfo;
    char* soid = nullptr;

    /* plugin/detection functions go here */
    OptFpList* opt_func = nullptr;
    OutputSet* outputFuncs = nullptr; /* per sid enabled output functions */
    snort::IpsOption* agent = nullptr;
    const char** buffer_setters = nullptr;

    OptFpList* normal_fp_only = nullptr;
    OptFpList* offload_fp_only = nullptr;

    struct THD_NODE* detection_filter = nullptr; /* if present, evaluated last, after header checks */
    TagData* tag = nullptr;

    // ptr to list of RTNs (head part); indexed by policyId
    RuleTreeNode** proto_nodes = nullptr;
    OtnState* state = nullptr;

    unsigned evalIndex = 0;       /* where this rule sits in the evaluation sets */
    unsigned ruleIndex = 0; // unique index
    uint32_t num_detection_opts = 0;
    SnortProtocolId snort_protocol_id = 0;    // Added for integrity checks during rule parsing.
    unsigned short proto_node_num = 0;
    uint16_t longestPatternLen = 0;
    IpsPolicy::Enable enable = IpsPolicy::Enable::DISABLED;
    Flag flags = 0;

    enum SectionDir { SECT_TO_SRV = 0, SECT_TO_CLIENT, SECT_DIR__MAX };
    snort::section_flags sections[SECT_DIR__MAX] = { section_to_flag(snort::PS_NONE),
        section_to_flag(snort::PS_NONE) };

    void set_warned_fp()
    { flags |= WARNED_FP; }

    bool warned_fp() const
    { return (flags & WARNED_FP) != 0; }

    void set_stateless()
    { flags |= STATELESS; }

    bool stateless() const
    { return (flags & STATELESS) != 0; }

    void set_enabled(IpsPolicy::Enable e)
    { enable = e; flags |= RULE_STATE; }

    bool is_rule_state_stub() const
    { return (flags & RULE_STATE) != 0; }

    bool enabled_somewhere() const
    {
        for ( unsigned i = 0; i < proto_node_num; i++ )
            if ( proto_nodes[i] and proto_nodes[i]->enabled() )
                return true;

        return false;
    }

    void set_metadata_match()
    { flags |= META_MATCH; }

    bool metadata_matched() const
    { return (flags & META_MATCH) != 0; }

    void set_to_client()
    { flags |= TO_CLIENT; }

    bool to_client() const
    { return (flags & TO_CLIENT) != 0; }

    void set_to_server()
    { flags |= TO_SERVER; }

    bool to_server() const
    { return (flags & TO_SERVER) != 0; }

    void set_flowbits_check()
    { flags |= BIT_CHECK; }

    bool checks_flowbits() const
    { return (flags & BIT_CHECK) != 0; }

    void set_service_only()
    { flags |= SVC_ONLY; }

    bool service_only() const
    { return (flags & SVC_ONLY) != 0; }

    void update_fp(snort::IpsOption*);

    bool to_client_err() const
    { return sections[SECT_TO_CLIENT] == section_to_flag(snort::PS_ERROR); }

    bool to_server_err() const
    { return sections[SECT_TO_SRV] == section_to_flag(snort::PS_ERROR); }
};

typedef int (* RuleOptEvalFunc)(void*, Cursor&, snort::Packet*);
OptFpList* AddOptFuncToList(RuleOptEvalFunc, OptTreeNode*);

namespace snort
{
SO_PUBLIC bool otn_has_plugin(OptTreeNode* otn, const char* name);
}

bool otn_set_agent(OptTreeNode*, snort::IpsOption*);

void otn_trigger_actions(const OptTreeNode*, snort::Packet*);

#endif

dump_config/

This directory contains classes related to Snort configuration dump.

Snort supports dumping config into a file for JSON and text formats.

  • ConfigData

    The ConfigData structure represents the internal config data of a particular Lua file.
    A unique shell is associated with every base and targeted policy file.
    Every module in a Lua file is represented by the General Tree.
    The Tree node can be one of the following types: TreeConfigNode, ValueConfigNode.
    The TreeConfigNode represents a table or list.
    The ValueConfigNode represents a config value itself.
  • ConfigOutput

    The ConfigOutput class is a base class that encapsulates dumping config into a file.
    Pure virtual function dump() should be overridden by derived classes for particular
    output format.
  • JsonAllConfigOutput

    The JsonAllConfigOutput class inherits from ConfigOutput and dumps base and targeted
    policy file in JSON format.
  • TextConfigOutput

    The TextConfigOutput class inherits from ConfigOutput and dumps base and targeted
    policy file in text format.

config_data.h

Path = src/dump_config/config_data.h

#ifndef CONFIG_DATA_H
#define CONFIG_DATA_H

#include <list>

#include "framework/value.h"

class BaseConfigNode;

using ConfigTrees = std::list<BaseConfigNode*>;

class BaseConfigNode
{
public:
    BaseConfigNode(BaseConfigNode* parent);
    virtual ~BaseConfigNode() = default;

    virtual std::string get_name() const = 0;
    virtual snort::Parameter::Type get_type() const = 0;
    virtual BaseConfigNode* get_node(const std::string& name) = 0;
    virtual void set_value(const snort::Value&) {}
    virtual const snort::Value* get_value() const { return nullptr; }

    const ConfigTrees& get_children() const
    { return children; }

    BaseConfigNode* get_parent_node() const
    { return parent; }

    void add_child_node(BaseConfigNode* node);

    static void clear_nodes(BaseConfigNode* root);
    static void sort_nodes(BaseConfigNode* node);

protected:
    ConfigTrees children;
    BaseConfigNode* parent = nullptr;
};

class TreeConfigNode : public BaseConfigNode
{
public:
    TreeConfigNode(BaseConfigNode* parent, const std::string& node_name,
        const snort::Parameter::Type node_type);

private:
    std::string get_name() const override
    { return name; }

    snort::Parameter::Type get_type() const override
    { return type; }

    BaseConfigNode* get_node(const std::string& name) override;

private:
    std::string name;
    snort::Parameter::Type type = snort::Parameter::PT_MAX;
};

class ValueConfigNode : public BaseConfigNode
{
public:
    ValueConfigNode(BaseConfigNode* parent, const snort::Value& value,
        const std::string& name = "");

private:
    std::string get_name() const override
    { return !custom_name.empty() ? custom_name : value.get_name(); }

    snort::Parameter::Type get_type() const override
    { return value.get_param_type(); }

    const snort::Value* get_value() const override
    { return &value; }

    void set_value(const snort::Value& v) override;
    BaseConfigNode* get_node(const std::string& name) override;

private:
    snort::Value value;
    bool multi_value = false;
    std::string custom_name;
};

class ConfigData
{
public:
    ConfigData(const char* file_name);

    void add_config_tree(BaseConfigNode* root)
    { config_trees.push_back(root); }

    void sort();
    void clear();

public:
    std::string file_name;
    ConfigTrees config_trees;
};

#endif // CONFIG_DATA_H

config_output.h

Path = src/dump_config/config_output.h

#ifndef CONFIG_OUTPUT_H
#define CONFIG_OUTPUT_H

class ConfigData;

class ConfigOutput
{
public:
    ConfigOutput() = default;
    virtual ~ConfigOutput() = default;

    void dump_config(ConfigData&);

private:
    virtual void dump(const ConfigData&) = 0;
};

#endif // CONFIG_OUTPUT_H

json_config_output.h

Path = src/dump_config/json_config_output.h

#ifndef JSON_CONFIG_OUTPUT_H
#define JSON_CONFIG_OUTPUT_H

#include "config_output.h"
#include "helpers/json_stream.h"

class BaseConfigNode;

class JsonAllConfigOutput : public ConfigOutput
{
public:
    JsonAllConfigOutput();
    ~JsonAllConfigOutput() override;

private:
    void dump(const ConfigData&) override;

private:
    snort::JsonStream json;
};

class JsonTopConfigOutput : public ConfigOutput
{
public:
    JsonTopConfigOutput() : ConfigOutput(), json(std::cout) {}

private:
    void dump(const ConfigData&) override;

private:
    snort::JsonStream json;
};

#endif // JSON_CONFIG_OUTPUT_H

text_config_output.h

Path = src/dump_config/text_config_output.h

#ifndef TEXT_CONFIG_OUTPUT_H
#define TEXT_CONFIG_OUTPUT_H

#include <string>

#include "config_output.h"

class BaseConfigNode;

class TextConfigOutput : public ConfigOutput
{
public:
    TextConfigOutput() = default;

private:
    void dump(const ConfigData&) override;
};

#endif // TEXT_CONFIG_OUTPUT_H

events/

This unit manages the event queue. Widely used utility GenerateSnortEvent() is in event_wrapper.h.

The event queue has a configurable maximum number of events, which are preallocated and stored in a linked list.

There are multiple instances of the event queue accessed via a simple stack. A push is done before processing a rebuilt packet or rebuilt payload after which a pop is done. During that time any wire packet events are still pending in the event queue higher up the stack. This ensures that the events for each packet (wire or rebuilt) are processed separately.

event.h

Path = src/events/event.h

#ifndef EVENT_H
#define EVENT_H

#include "main/thread.h"

struct SigInfo;

/* we must use fixed size of 32 bits, because on-disk
 * format of savefiles uses 32-bit tv_sec (and tv_usec)
 */
struct sf_timeval32
{
    uint32_t tv_sec;      /* seconds */
    uint32_t tv_usec;     /* microseconds */
};

struct Event
{
    SigInfo* sig_info = nullptr;
    struct sf_timeval32 ref_time = { 0, 0 };   /* reference time for the event reference */
    const char* alt_msg = nullptr;
    std::string action_string;
    const char** buffs_to_dump = nullptr;

    Event() = default;
    Event(SigInfo& si)
    { sig_info = &si; }

    uint32_t get_event_id() const { return event_id; }
    void set_event_id(uint32_t id) { event_id = id; }

    uint32_t get_event_reference() const { return event_reference; }
    void set_event_reference(uint32_t ref) { event_reference = ref; }

    void update_event_id(uint16_t log_id);
    void update_event_id_and_ref(uint16_t log_id);
    SO_PUBLIC static uint32_t update_and_get_event_id();

    void set_event(uint32_t gid, uint32_t sid, uint32_t rev,
        uint32_t classification, uint32_t priority, uint16_t event_ref,
        uint16_t log_id, const struct timeval& tv, const std::string& act = "");


private:
    uint32_t event_id = 0;
    uint32_t event_reference = 0; // reference to other events that have gone off,
                                  // such as in the case of tagged packets...
};

uint16_t get_event_id();
void incr_event_id();

#endif

event_queue.h

Path = src/events/event_queue.h

#ifndef EVENT_QUEUE_H
#define EVENT_QUEUE_H

#include "main/snort_types.h"

#define SNORT_EVENTQ_PRIORITY    1
#define SNORT_EVENTQ_CONTENT_LEN 2

struct EventQueueConfig
{
    unsigned max_events;
    unsigned log_events;
    int order;
    int process_all_events;
};

struct EventNode
{
    const struct OptTreeNode* otn;
    const struct RuleTreeNode* rtn;
};

EventQueueConfig* EventQueueConfigNew();
void EventQueueConfigFree(EventQueueConfig*);

#endif

sfeventq.h

Path = src/events/sfeventq.h

#ifndef SFEVENTQ_H
#define SFEVENTQ_H

struct SF_EVENTQ_NODE
{
    void* event;

    SF_EVENTQ_NODE* prev;
    SF_EVENTQ_NODE* next;
};

struct SF_EVENTQ
{
    /*
    **  Handles the actual ordering and memory
    **  of the event queue and it's nodes.
    */
    SF_EVENTQ_NODE* head;
    SF_EVENTQ_NODE* last;

    SF_EVENTQ_NODE* node_mem;
    char* event_mem;

    /*
    **  The reserve event allows us to allocate one extra node
    **  and compare against the last event in the queue to determine
    **  if the incoming event is a higher priority than the last
    **  event in the queue.
    */
    char* reserve_event;

    /*
    **  Queue configuration
    */
    int max_nodes;
    int log_nodes;
    int event_size;

    /*
    **  This element tracks the current number of
    **  nodes in the event queue.
    */
    int cur_nodes;
    int cur_events;
    unsigned fails;
};

SF_EVENTQ* sfeventq_new(int max_nodes, int log_nodes, int event_size);
void* sfeventq_event_alloc(SF_EVENTQ*);
unsigned sfeventq_reset(SF_EVENTQ*);  // returns fail count since last reset
int sfeventq_add(SF_EVENTQ*, void* event);
int sfeventq_action(SF_EVENTQ*, int (* action_func)(void* event, void* user), void* user);
void sfeventq_free(SF_EVENTQ*);

#endif

file_api/

This directory contains all the file processing related classes and APIs

  • file_api: provides the interfaces for file processing, used by service inspectors such as HTTP, SMTP, POP, IMAP, SMB, and FTP etc.

  • File capture: provides the ability to capture file data and save them in the mempool, then they can be stored to disk. Currently, files can be saved to the logging folder. Writing to disk is done by a separate thread that will not block packet thread. When a file is available to store, it will be put into a queue. The writer thread will read from this queue to write to disk. In the multiple packet thread case, many threads will write into this queue and one writer thread serves all of them. Thread synchronization is done by mutex and conditional variables for the queue. In the future, we will add support for multiple writer threads to improve performance when multiple disks are used.

  • File libraries: provides file type identification and file signature calculation

  • file_id: file rules must contain file_meta and at least one fast-pattern option.

circular_buffer.h

Path = src/file_api/circular_buffer.h

#ifndef CIRCULAR_BUFFER_H
#define CIRCULAR_BUFFER_H

//  Circular buffer is thread safe for one writer and one reader thread
//  This implementation is inspired by one slot open approach.
//  See http://en.wikipedia.org/wiki/Circular_buffer

#include "main/snort_types.h"

#define CB_SUCCESS    0  // FIXIT-RC use bool
#define CB_FAIL      (-1)

// Opaque buffer element type.  This would be defined by the application.
typedef void* ElemType;

struct _CircularBuffer;
typedef struct _CircularBuffer CircularBuffer;

// Initialize buffer based on number of elements
CircularBuffer* cbuffer_init(uint64_t size);

void cbuffer_free(CircularBuffer* cb);

int cbuffer_is_full(CircularBuffer* cb); // FIXIT-RC use bool
int cbuffer_is_empty(CircularBuffer* cb); // FIXIT-RC use bool

// Returns number of elements in use
uint64_t cbuffer_used(CircularBuffer* cb);

// Returns total number of elements
uint64_t cbuffer_size(CircularBuffer* cb);

// Returns CB_SUCCESS or CB_FAIL
int cbuffer_write(CircularBuffer* cb, const ElemType elem);

// Read one element from the buffer and remove it from buffer
// Returns CB_SUCCESS or CB_FAIL
int cbuffer_read(CircularBuffer* cb, ElemType* elem);

#endif

file_api.h

Path = src/file_api/file_api.h

#ifndef FILE_API_H
#define FILE_API_H

// File API provides all the convenient functions that are used by inspectors.
// Currently, it provides three sets of APIs: file processing, MIME processing,
// and configurations.

#include <atomic>
#include <bitset>
#include <cstring>
#include <string>

#include "main/snort_config.h"
#include "main/snort_types.h"

#define     ENABLE_FILE_TYPE_IDENTIFICATION      0x1
#define     ENABLE_FILE_SIGNATURE_SHA256         0x2
#define     ENABLE_FILE_CAPTURE                  0x4
#define     FILE_ALL_ON                          0xFFFFFFFF
#define     FILE_ALL_OFF                         0x00000000

enum FileAction
{
    FILE_ACTION_DEFAULT = 0,
    FILE_RESUME_BLOCK,
    FILE_RESUME_LOG
};

#define UTF_16_LE_BOM "\xFF\xFE"
#define UTF_16_LE_BOM_LEN 2

enum FileVerdict
{
    FILE_VERDICT_UNKNOWN = 0,
    FILE_VERDICT_LOG,
    FILE_VERDICT_STOP,
    FILE_VERDICT_BLOCK,
    FILE_VERDICT_REJECT,
    FILE_VERDICT_PENDING,
    FILE_VERDICT_STOP_CAPTURE,
    FILE_VERDICT_MAX
};

enum FilePosition
{
    SNORT_FILE_POSITION_UNKNOWN,
    SNORT_FILE_START,
    SNORT_FILE_MIDDLE,
    SNORT_FILE_END,
    SNORT_FILE_FULL
};

enum FileCaptureState
{
    FILE_CAPTURE_SUCCESS = 0,
    FILE_CAPTURE_MIN,                 /*smaller than file capture min*/
    FILE_CAPTURE_MAX,                 /*larger than file capture max*/
    FILE_CAPTURE_MEMCAP,              /*memcap reached, no more file buffer*/
    FILE_CAPTURE_FAIL                 /*Other file capture failures*/
};

enum FileSigState
{
    FILE_SIG_PROCESSING = 0,
    FILE_SIG_DEPTH_FAIL,              /*larger than file signature depth*/
    FILE_SIG_FLUSH,
    FILE_SIG_DONE
};

enum FileProcessType
{
    SNORT_FILE_TYPE_ID,
    SNORT_FILE_SHA256,
    SNORT_FILE_CAPTURE
};

enum FileDirection
{
    FILE_DOWNLOAD,
    FILE_UPLOAD
};

enum FileCharEncoding
{
    SNORT_CHAR_ENCODING_ASCII,
    SNORT_CHAR_ENCODING_UTF_16LE
};

struct FileState
{
    FileCaptureState capture_state;
    FileSigState sig_state;
};

namespace snort
{
#define FILE_ID_MAX          1024
typedef std::bitset<FILE_ID_MAX> FileTypeBitSet;

class FileContext;
class FileInfo;
class Flow;
struct Packet;

class UserFileDataBase
{
public:
    UserFileDataBase() = default;
    virtual ~UserFileDataBase() = default;
    UserFileDataBase(const UserFileDataBase& other);
    UserFileDataBase& operator=(const UserFileDataBase& other);

private:
    void copy(const UserFileDataBase& other);
};

class SO_PUBLIC FilePolicyBase
{
public:

    FilePolicyBase();
    virtual ~FilePolicyBase();

    // This is called when a new flow is queried for the first time
    // Check & update what file policy is enabled on this flow/file
    virtual void policy_check(Flow*, FileInfo*) { }

    // This is called after file type is known
    virtual FileVerdict type_lookup(Packet*, FileInfo*)
    { return FILE_VERDICT_UNKNOWN; }

    // This is called after file signature is complete
    virtual FileVerdict signature_lookup(Packet*, FileInfo*)
    { return FILE_VERDICT_UNKNOWN; }

    virtual void log_file_action(Flow*, FileInfo*, FileAction) { }

    void add_ref()
    { ++ref_count; }

    SO_PUBLIC static void delete_file_policy(FilePolicyBase*);

private:
    std::atomic_uint ref_count;
};

inline void initFilePosition(FilePosition* position, uint64_t processed_size)
{
    *position = SNORT_FILE_START;
    if (processed_size)
        *position = SNORT_FILE_MIDDLE;
}

inline void updateFilePosition(FilePosition* position, uint64_t processed_size)
{
    if ((*position == SNORT_FILE_END) || (*position == SNORT_FILE_FULL))
        *position = SNORT_FILE_START;
    else if (processed_size)
        *position = SNORT_FILE_MIDDLE;
}

inline void finalFilePosition(FilePosition* position)
{
    if (*position == SNORT_FILE_START)
        *position = SNORT_FILE_FULL;
    else if (*position != SNORT_FILE_FULL)
        *position = SNORT_FILE_END;
}

inline bool isFileStart(FilePosition position)
{
    return ((position == SNORT_FILE_START) || (position == SNORT_FILE_FULL));
}

inline bool isFileEnd(FilePosition position)
{
    return ((position == SNORT_FILE_END) || (position == SNORT_FILE_FULL));
}

inline FileCharEncoding get_character_encoding(const char* file_name, size_t length)
{
    FileCharEncoding encoding = SNORT_CHAR_ENCODING_ASCII;
    if (length >= UTF_16_LE_BOM_LEN)
    {
        if (memcmp(file_name, UTF_16_LE_BOM, UTF_16_LE_BOM_LEN) == 0)
            encoding = SNORT_CHAR_ENCODING_UTF_16LE;
    }

    return encoding;
}

SO_PUBLIC uint64_t get_file_processed_size(Flow* flow);
SO_PUBLIC FilePosition get_file_position(Packet* pkt);
SO_PUBLIC void set_rule_id_from_type(SnortConfig* sc, uint64_t id, std::string type,
    std::string file_category, std::string file_version, std::vector<std::string> file_groups);
SO_PUBLIC void get_magic_rule_ids_from_type(const std::string& type,
    const std::string& version, FileTypeBitSet& ids_set, SnortConfig*);
}
#endif

file_cache.h

Path = src/file_api/file_cache.h

#ifndef FILE_CACHE_H
#define FILE_CACHE_H

#include <mutex>

#include "sfip/sf_ip.h"
#include "utils/cpp_macros.h"

#include "file_config.h"

class ExpectedFileCache;

class FileCache
{
public:

PADDING_GUARD_BEGIN
    struct FileHashKey
    {
        snort::SfIp sip;
        int16_t sgroup;
        snort::SfIp dip;
        int16_t dgroup;
        uint64_t file_id;
        uint16_t asid;
        uint16_t padding[3];
    };
PADDING_GUARD_END

    struct FileNode
    {
        struct timeval cache_expire_time = {0, 0};
        snort::FileContext* file;
    };

    FileCache(int64_t max_files_cached);
    ~FileCache();

    void set_block_timeout(int64_t);
    void set_lookup_timeout(int64_t);
    void set_max_files(int64_t);

    snort::FileContext* get_file(snort::Flow*, uint64_t file_id, bool to_create);
    FileVerdict cached_verdict_lookup(snort::Packet*, snort::FileInfo*,
        snort::FilePolicyBase*);
    bool apply_verdict(snort::Packet*, snort::FileContext*, FileVerdict, bool resume,
        snort::FilePolicyBase*);

private:
    snort::FileContext* add(const FileHashKey&, int64_t timeout);
    snort::FileContext* find(const FileHashKey&, int64_t);
    snort::FileContext* find_add(const FileHashKey&, int64_t);
    snort::FileContext* get_file(snort::Flow*, uint64_t file_id, bool to_create,
        int64_t timeout);
    FileVerdict check_verdict(snort::Packet*, snort::FileInfo*, snort::FilePolicyBase*);
    int store_verdict(snort::Flow*, snort::FileInfo*, int64_t timeout);

    /* The hash table of expected files */
    ExpectedFileCache* fileHash = nullptr;
    int64_t block_timeout = DEFAULT_FILE_BLOCK_TIMEOUT;
    int64_t lookup_timeout = DEFAULT_FILE_LOOKUP_TIMEOUT;
    int64_t max_files = DEFAULT_MAX_FILES_CACHED;
    std::mutex cache_mutex;
};

#endif

file_capture.h

Path = src/file_api/file_capture.h

#ifndef FILE_CAPTURE_H
#define FILE_CAPTURE_H

// There are several steps for file capture:
// 1) To improve performance, file data are stored in file mempool first by
//    calling file_capture_process() during file data processing.
// 2) If file capture is needed, file_capture_reserve() should be called to
//    allow file data remains in mempool. Even if a session is closed, the file
//     data will stay in the mempool.
// 3) Then file data can be read through file_capture_read()
// 4) Finally, file data must be released from mempool file_capture_release()

#include <condition_variable>
#include <mutex>
#include <queue>
#include <thread>

#include "file_api.h"

class FileMemPool;

namespace snort
{
class FileInfo;

struct FileCaptureBlock
{
    uint32_t length;
    FileCaptureBlock* next;  /* next block of file data */
};

class SO_PUBLIC FileCapture
{
public:
    FileCapture(int64_t capture_min_size, int64_t capture_max_size);
    ~FileCapture();

    // this must be called during snort init
    static void init(int64_t memcap, int64_t block_size);

    // Capture file data to local buffer
    // This is the main function call to enable file capture
    FileCaptureState process_buffer(const uint8_t* file_data, int data_size,
        FilePosition pos);

    // Preserve the file in memory until it is released
    FileCaptureState reserve_file(const snort::FileInfo*);

    // Get the file that is reserved in memory, this should be called repeatedly
    // until nullptr is returned to get the full file
    // Returns:
    //   the next memory block
    //   nullptr: end of file or fail to get file
    FileCaptureBlock* get_file_data(uint8_t** buff, int* size);

    // Store files on local disk
    void store_file();

    // Store file to disk asynchronously
    void store_file_async();

    // Log file capture mempool usage
    static void print_mem_usage();

    // Exit file capture, release all file capture memory etc,
    // this must be called when snort exits
    static void exit();

    static FileCaptureState error_capture(FileCaptureState);

    static int64_t get_block_size() { return capture_block_size; }

    snort::FileInfo* get_file_info() { return file_info; }

    int64_t get_max_file_capture_size() { return capture_max_size; }
    int64_t get_file_capture_size() { return capture_size; }
    void get_file_reset() { current_block = head; }

private:

    static void init_mempool(int64_t max_file_mem, int64_t block_size);
    static void writer_thread();
    inline FileCaptureBlock* create_file_buffer();
    inline FileCaptureState save_to_file_buffer(const uint8_t* file_data, int data_size,
        int64_t max_size);
    void write_file_data(uint8_t* buf, size_t buf_len, FILE* fh);

    static FileMemPool* file_mempool;
    static int64_t capture_block_size;
    static std::mutex capture_mutex;
    static std::condition_variable capture_cv;
    static std::thread* file_storer;
    static std::queue<FileCapture*> files_waiting;
    static bool running;

    uint64_t capture_size;
    FileCaptureBlock* last;  /* last block of file data */
    FileCaptureBlock* head;  /* first block of file data */
    FileCaptureBlock* current_block = nullptr;  /* current block of file data */
    const uint8_t* current_data;  /*current file data*/
    uint32_t current_data_len;
    FileCaptureState capture_state;
    snort::FileInfo* file_info = nullptr;
    int64_t capture_min_size;
    int64_t capture_max_size;
};
}

#endif

file_config.h

Path = src/file_api/file_config.h

#ifndef FILE_CONFIG_H
#define FILE_CONFIG_H

// This provides the basic configuration for file processing
#include "main/snort_config.h"
#include "detection/detection_engine.h"
#include "file_api/file_identifier.h"

#define DEFAULT_FILE_TYPE_DEPTH 1460
#define DEFAULT_FILE_SIGNATURE_DEPTH 10485760 /*10 Mbytes*/
#define DEFAULT_FILE_SHOW_DATA_DEPTH 100
#define DEFAULT_FILE_BLOCK_TIMEOUT 86400 /*1 day*/
#define DEFAULT_FILE_LOOKUP_TIMEOUT 2    /*2 seconds*/
#define DEFAULT_FILE_CAPTURE_MEM            100         // 100 MiB
#define DEFAULT_FILE_CAPTURE_MAX_SIZE       1048576     // 1 MiB
#define DEFAULT_FILE_CAPTURE_MIN_SIZE       0           // 0
#define DEFAULT_FILE_CAPTURE_BLOCK_SIZE     32768       // 32 KiB
#define DEFAULT_MAX_FILES_CACHED            65536
#define DEFAULT_MAX_FILES_PER_FLOW          128

#define FILE_ID_NAME "file_id"
#define FILE_ID_HELP "configure file identification"

class FileConfig
{
public:
    const FileMeta* get_rule_from_id(uint32_t) const;
    void get_magic_rule_ids_from_type(const std::string&, const std::string&,
        snort::FileTypeBitSet&) const;
    void process_file_rule(FileMeta&);
    uint32_t find_file_type_id(const uint8_t* buf, int len, uint64_t file_offset, void** context);
    std::string file_type_name(uint32_t id) const;

    int64_t file_type_depth = DEFAULT_FILE_TYPE_DEPTH;
    int64_t file_signature_depth = DEFAULT_FILE_SIGNATURE_DEPTH;
    int64_t file_block_timeout = DEFAULT_FILE_BLOCK_TIMEOUT;
    int64_t file_lookup_timeout = DEFAULT_FILE_LOOKUP_TIMEOUT;
    bool block_timeout_lookup = false;
    int64_t capture_memcap = DEFAULT_FILE_CAPTURE_MEM;
    int64_t capture_max_size = DEFAULT_FILE_CAPTURE_MAX_SIZE;
    int64_t capture_min_size = DEFAULT_FILE_CAPTURE_MIN_SIZE;
    int64_t capture_block_size = DEFAULT_FILE_CAPTURE_BLOCK_SIZE;
    int64_t file_depth =  0;
    int64_t max_files_cached = DEFAULT_MAX_FILES_CACHED;
    uint64_t max_files_per_flow = DEFAULT_MAX_FILES_PER_FLOW;

    int64_t show_data_depth = DEFAULT_FILE_SHOW_DATA_DEPTH;
    bool trace_type = false;
    bool trace_signature = false;
    bool trace_stream = false;
    SnortProtocolId snort_protocol_id = UNKNOWN_PROTOCOL_ID;

private:
    FileIdentifier fileIdentifier;
};

std::string file_type_name(uint32_t id);
FileConfig* get_file_config(const snort::SnortConfig* sc = nullptr);
#endif

file_flows.h

Path = src/file_api/file_flows.h

#ifndef FILE_FLOWS_H
#define FILE_FLOWS_H

// This provides a wrapper to manage several file contexts

#include "flow/flow.h"
#include "main/snort_types.h"
#include "utils/event_gen.h"

#include "file_api.h"
#include "file_module.h"

#include <map>

using FileEventGen = EventGen<EVENT__MAX_VALUE, EVENT__NONE, FILE_ID_GID>;

namespace snort
{
class FileContext;
class Flow;

class FileInspect : public Inspector
{
public:
    FileInspect(FileIdModule*);
    ~FileInspect() override;
    void eval(Packet*) override { }
    bool configure(SnortConfig*) override;
    void show(const SnortConfig*) const override;
    FileConfig* config;
};

class SO_PUBLIC FileFlows : public FlowData
{
public:

    FileFlows(Flow* f, FileInspect* inspect) : FlowData(file_flow_data_id, inspect), flow(f) { }
    ~FileFlows() override;
    std::mutex file_flow_context_mutex;
    static void init()
    { file_flow_data_id = FlowData::create_flow_data_id(); }

    void handle_retransmit(Packet*) override;

    // Factory method to get file flows
    static FileFlows* get_file_flows(Flow*, bool to_create=true);
    static FilePolicyBase* get_file_policy(Flow*);

    FileContext* get_current_file_context();

    void set_current_file_context(FileContext*);

    // Get file context based on file id, create it if does not exist
    FileContext* get_file_context(uint64_t file_id, bool to_create,
        bool& is_new_context, uint64_t multi_file_processing_id=0);
    // Get a partially processed file context from the flow object
    FileContext* get_partially_processed_context(uint64_t file_id);
    // Remove a file from the flow object when processing is complete
    void remove_processed_file_context(uint64_t file_id);

    void remove_processed_file_context(uint64_t file_id, uint64_t multi_file_processing_id);

    uint64_t get_new_file_instance();

    bool set_file_name(const uint8_t* fname, uint32_t name_size, uint64_t file_id=0,
        uint64_t multi_file_processing_id=0, const uint8_t* url=nullptr, uint32_t url_size=0);

    void set_sig_gen_state( bool enable )
    {
        gen_signature = enable;
    }

    void add_pending_file(uint64_t file_id);

    // This is used when there is only one file per session
    bool file_process(Packet* p, const uint8_t* file_data, int data_size, FilePosition,
        bool upload, size_t file_index = 0);

    // This is used for each file context. Support multiple files per session
    bool file_process(Packet* p, uint64_t file_id, const uint8_t* file_data,
        int data_size, uint64_t offset, FileDirection, uint64_t multi_file_processing_id=0,
        FilePosition=SNORT_FILE_POSITION_UNKNOWN);

    static unsigned file_flow_data_id;

    void set_file_policy(FilePolicyBase* fp)
    {
        assert(fp);
        FilePolicyBase::delete_file_policy(file_policy);
        fp->add_ref();
        file_policy = fp;
    }
    FilePolicyBase* get_file_policy() { return file_policy; }

private:
    void init_file_context(FileDirection, FileContext*);
    FileContext* find_main_file_context(FilePosition, FileDirection, size_t id = 0);
    FileContext* main_context = nullptr;
    FileContext* current_context = nullptr;
    uint64_t current_file_id = 0;
    uint64_t pending_file_id = 0;
    bool gen_signature = false;
    Flow* flow = nullptr;
    FilePolicyBase* file_policy = nullptr;

    std::unordered_map<uint64_t, FileContext*> partially_processed_contexts;
    bool current_context_delete_pending = false;
    FileEventGen events;
};
}
#endif

file_identifier.h

Path = src/file_api/file_identifier.h

#ifndef FILE_IDENTIFIER_H
#define FILE_IDENTIFIER_H

// File type identification is based on file magic. To improve the detection
// performance, a trie is created to scan file data once. Currently, only the
// most specific file type is returned.

#include <list>
#include <vector>

#include "file_lib.h"

namespace snort
{
class GHash;
}

#define MAX_BRANCH (UINT8_MAX + 1)

enum IdNodeState
{
    ID_NODE_NEW,
    ID_NODE_USED,
    ID_NODE_SHARED
};

class FileMeta
{
public:
    FileMeta() { }
    FileMeta(uint type_id, const std::string& type_name, const std::string& file_category, const std::string&
        file_version, const std::vector<std::string>& file_groups)
        : id(type_id), type(type_name),category(file_category),version(file_version),groups(
            file_groups) { }
    void clear();
    uint32_t rev = 0;
    uint32_t id = 0;
    std::string type;
    std::string category;
    std::string version;
    std::vector<std::string> groups;
};

struct IdentifierNode
{
    uint32_t type_id;       /* magic content to match*/
    IdNodeState state;
    uint32_t offset;            /* offset from file start */
    struct IdentifierNode* next[MAX_BRANCH]; /* pointer to an array of 256 identifiers pointers*/
};

typedef std::list<void* >  IDMemoryBlocks;

class FileIdentifier
{
public:
    ~FileIdentifier();
    uint32_t memory_usage() const { return memory_used; }
    void add_file_id(FileMeta& rule);
    uint32_t find_file_type_id(const uint8_t* buf, int len, uint64_t offset, void** context);
    const FileMeta* get_rule_from_id(uint32_t) const;
    void get_magic_rule_ids_from_type(const std::string&, const std::string&,
        snort::FileTypeBitSet&) const;

private:
    void init_merge_hash();
    void* calloc_mem(size_t size);
    void set_node_state_shared(IdentifierNode* start);
    IdentifierNode* clone_node(IdentifierNode* start);
    bool update_next(IdentifierNode* start, IdentifierNode** next_ptr, IdentifierNode* append);
    void update_trie(IdentifierNode* start, IdentifierNode* append);

    /*properties*/
    IdentifierNode* identifier_root = nullptr; /*Root of magic tries*/
    uint32_t memory_used = 0; /*Track memory usage*/
    snort::GHash* identifier_merge_hash = nullptr;
    FileMeta file_magic_rules[FILE_ID_MAX + 1];
    IDMemoryBlocks id_memory_blocks;
};

#endif

file_lib.h

Path = src/file_api/file_lib.h

#ifndef FILE_LIB_H
#define FILE_LIB_H

// This will be basis of file class

#include <mutex>
#include <ostream>
#include <string>

#include "file_api/file_api.h"
#include "utils/util.h"

#define SNORT_FILE_TYPE_UNKNOWN          UINT16_MAX
#define SNORT_FILE_TYPE_CONTINUE         0

const std::string VerdictName[] =
{"Unknown", "Log", "Stop", "Block", "Reset", "Pending", "Stop Capture", "INVALID"};

class FileConfig;
class FileSegments;

namespace snort
{
class FileCapture;
class FileInspect;
class Flow;

class SO_PUBLIC FileInfo
{
public:
    virtual ~FileInfo();
    FileInfo() = default;
    FileInfo(const FileInfo& other);
    FileInfo& operator=(const FileInfo& other);
    uint32_t get_file_type() const;
    void set_file_type(uint64_t index);
    void set_file_name(const char* file_name, uint32_t name_size);
    void set_url(const char* url, uint32_t url_size);
    std::string& get_file_name();
    std::string& get_url();
    // Whether file name has been set (could be empty file name)
    bool is_file_name_set() const { return file_name_set; }
    bool is_url_set() const { return url_set; }

    void set_file_size(uint64_t size);
    uint64_t get_file_size() const;
    void set_file_direction(FileDirection dir);
    FileDirection get_file_direction() const;
    uint8_t* get_file_sig_sha256() const;
    std::string sha_to_string(const uint8_t* sha256);
    void set_file_id(uint64_t index);
    uint64_t get_file_id() const;

    // Configuration functions
    void config_file_type(bool enabled);
    bool is_file_type_enabled();
    void config_file_signature(bool enabled);
    bool is_file_signature_enabled();
    void config_file_capture(bool enabled);
    bool is_file_capture_enabled();
    void set_policy_id(uint32_t id);
    uint32_t get_policy_id();
    void set_file_data(UserFileDataBase* fd);
    UserFileDataBase* get_file_data() const;
    void copy(const FileInfo& other, bool clear_data = true);
    // Preserve the file in memory until it is released
    // The file reserved will be returned and it will be detached from file context/session
    FileCaptureState reserve_file(FileCapture*& dest);
    int64_t get_max_file_capture_size();

    FileState get_file_state() { return file_state; }

    FileVerdict verdict = FILE_VERDICT_UNKNOWN;
    bool processing_complete = false;
    std::mutex user_file_data_mutex;
    struct timeval pending_expire_time = {0, 0};

protected:
    std::string file_name;
    bool file_name_set = false;
    std::string url;
    bool url_set = false;
    uint64_t file_size = 0;
    FileDirection direction = FILE_DOWNLOAD;
    uint32_t file_type_id = SNORT_FILE_TYPE_CONTINUE;
    uint8_t* sha256 = nullptr;
    uint64_t file_id = 0;
    FileCapture* file_capture = nullptr;
    bool file_type_enabled = false;
    bool file_signature_enabled = false;
    bool file_capture_enabled = false;
    FileState file_state = { FILE_CAPTURE_SUCCESS, FILE_SIG_PROCESSING };
    uint32_t policy_id = 0;
    UserFileDataBase* user_file_data = nullptr;
};

class SO_PUBLIC FileContext : public FileInfo
{
public:
    FileContext();
    ~FileContext() override;

    void check_policy(Flow*, FileDirection, FilePolicyBase*);

    // main processing functions

    // Return:
    //    true: continue processing/log/block this file
    //    false: ignore this file
    bool process(Packet*, const uint8_t* file_data, int data_size, FilePosition, FilePolicyBase*);
    bool process(Packet*, const uint8_t* file_data, int data_size, uint64_t offset, FilePolicyBase*,
        FilePosition position=SNORT_FILE_POSITION_UNKNOWN);
    void process_file_signature_sha256(const uint8_t* file_data, int data_size, FilePosition);
    void update_file_size(int data_size, FilePosition position);
    void stop_file_capture();
    FileCaptureState process_file_capture(const uint8_t* file_data, int data_size, FilePosition);
    void log_file_event(Flow*, FilePolicyBase*);
    FileVerdict file_signature_lookup(Packet*);

    void set_signature_state(bool gen_sig);

    //File properties
    uint64_t get_processed_bytes();

    void print_file_sha256(std::ostream&);
    void print_file_name(std::ostream&);
    static void print_file_data(FILE* fp, const uint8_t* data, int len, int max_depth);
    void print(std::ostream&);
    char* get_UTF8_fname(size_t* converted_len);
    void set_not_cacheable() { cacheable = false; }
    bool is_cacheable() { return cacheable; }
    bool segments_queued() { return (file_segments != nullptr); }

private:
    uint64_t processed_bytes = 0;
    void* file_type_context;
    void* file_signature_context;
    FileSegments* file_segments;
    FileInspect* inspector;
    FileConfig*  config;
    bool cacheable = true;

    void finalize_file_type();
    void finish_signature_lookup(Packet*, bool, FilePolicyBase*);
    void find_file_type_from_ips(Packet*, const uint8_t *file_data, int data_size, FilePosition);
    void process_file_type(Packet*, const uint8_t* file_data, int data_size, FilePosition);
};
}
#endif

file_mempool.h

Path = src/file_api/file_mempool.h

#ifndef FILE_MEMPOOL_H
#define FILE_MEMPOOL_H

//  This mempool implementation has very efficient alloc/free operations.
//  In addition, it provides thread-safe alloc/free for one allocation/free
//  thread and one release thread.
//  One more bonus: Double free detection is also added into this library
//  This is a thread safe version of memory pool for one writer and one reader thread

#include <mutex>

#include "circular_buffer.h"

#define FILE_MEM_SUCCESS    0  // FIXIT-RC use bool
#define FILE_MEM_FAIL      (-1)

class FileMemPool
{
public:

    FileMemPool(uint64_t num_objects, size_t obj_size);
    ~FileMemPool();

    // Allocate a new object from the FileMemPool
    // Note: Memory block will not be zeroed for performance
    // Returns: a pointer to the FileMemPool object on success, nullptr on failure
    void* m_alloc();

    // This must be called by the same thread calling file_mempool_alloc()
    // Return: FILE_MEM_SUCCESS or FILE_MEM_FAIL
    int m_free(void* obj);

    // This can be called by a different thread calling file_mempool_alloc()
    // Return: FILE_MEM_SUCCESS or FILE_MEM_FAIL
    int m_release(void* obj);

    //Returns number of elements allocated
    uint64_t allocated();

    // Returns number of elements freed in current buffer
    uint64_t freed();

    // Returns number of elements released in current buffer
    uint64_t released();

    // Returns total number of elements in current buffer
    uint64_t total_objects() { return total; }

private:

    void free_pools();
    int remove(CircularBuffer* cb, void* obj);

    void** datapool = nullptr; /* memory buffer */
    uint64_t total = 0;
    CircularBuffer* free_list = nullptr;
    CircularBuffer* released_list = nullptr;
    size_t obj_size = 0;
    std::mutex pool_mutex;
};

#endif

file_module.h

Path = src/file_api/file_module.h

#ifndef FILE_MODULE_H
#define FILE_MODULE_H

#include <string>

#include "framework/module.h"

#include "file_config.h"
#include "file_identifier.h"
#include "trace/trace_api.h"
#include "utils/util.h"
//-------------------------------------------------------------------------
// file_id module
//-------------------------------------------------------------------------

static const uint32_t FILE_ID_GID = 150;

#define FILE_DEBUG(module_name, module_id, log_level, p, ...) \
    trace_logf(log_level, module_name , module_id, p, __VA_ARGS__)

#define GET_CURRENT_PACKET snort::DetectionEngine::get_current_packet()

extern THREAD_LOCAL const snort::Trace* file_trace;

class FileIdModule : public snort::Module
{
public:
    FileIdModule();
    ~FileIdModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    snort::ProfileStats* get_profile() const override;
    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;

    void sum_stats(bool) override;

    void load_config(FileConfig*& dst);

    void reset_stats() override;

    Usage get_usage() const override
    { return GLOBAL; }

    void show_dynamic_stats() override;

    unsigned get_gid() const override
    { return FILE_ID_GID; }

    void set_trace(const snort::Trace*) const override;
    const snort::TraceOption* get_trace_options() const override;
    const snort::RuleMap* get_rules() const override;

private:
    FileMeta rule;
    FileConfig *fc = nullptr;
    std::string magic_file;
};

enum FileSid
{
    EVENT__NONE = -1,
    EVENT_FILE_DROPPED_OVER_LIMIT = 1,
    EVENT__MAX_VALUE
};

#endif

file_segment.h

Path = src/file_api/file_segment.h

#ifndef FILE_SEGMENT_H
#define FILE_SEGMENT_H

// Segmented file data reassemble and processing

#include <string>

#include "file_api.h"

namespace snort
{
class Flow;
}
class FileConfig;

class FileSegment
{
public:
    FileSegment() = default;
    ~FileSegment();

    // Use single list for simplicity
    FileSegment* next = nullptr;
    uint32_t offset = 0;
    std::string* data = nullptr;
};

class FileSegments
{
public:
    FileSegments(snort::FileContext*);
    ~FileSegments();

    void clear();

    // Process file segments with current_offset specified. If file segment is out of order,
    // it will be put into the file segments queue.
    int process(snort::Packet*, const uint8_t* file_data, uint64_t data_size, uint64_t offset,
        snort::FilePolicyBase*, FilePosition position=SNORT_FILE_POSITION_UNKNOWN);

private:
    FileSegment* head = nullptr;
    uint64_t current_offset;
    snort::FileContext* context = nullptr;

    void add(const uint8_t* file_data, int64_t data_size, uint64_t offset);
    void insert_segment(const uint8_t* file_data, int64_t data_size, uint64_t offset, bool find_left,  FileSegment** left);
    FilePosition get_file_position(uint64_t data_size, uint64_t file_size);
    int process_one(snort::Packet*, const uint8_t* file_data, int data_size, snort::FilePolicyBase*,
        FilePosition position=SNORT_FILE_POSITION_UNKNOWN);
    int process_all(snort::Packet*, snort::FilePolicyBase*);
};

#endif

file_service.h

Path = src/file_api/file_service.h

#ifndef FILE_SERVICE_H
#define FILE_SERVICE_H

// This provides a wrapper to start/stop file service

#include "main/snort_config.h"
#include "main/snort_types.h"
#include "mime/file_mime_config.h"

class FileEnforcer;
class FileCache;

namespace snort
{
class SO_PUBLIC FileService
{
public:
    // This must be called when snort restarts
    static void init();

    // Called after permission is dropped
    static void post_init();

    // Called during reload
    static void verify_reload(const SnortConfig*);

    // This must be called when snort exits
    static void close();

    static void thread_init();
    static void thread_term();

    static void enable_file_type();
    static void enable_file_signature();
    static void enable_file_capture();
    static bool is_file_type_id_enabled() { return file_type_id_enabled; }
    static bool is_file_signature_enabled() { return file_signature_enabled; }
    static bool is_file_capture_enabled() { return file_capture_enabled; }
    static bool is_file_service_enabled();
    static int64_t get_max_file_depth();
    static void reset_depths();

    static FileCache* get_file_cache() { return file_cache; }
    static DecodeConfig decode_conf;

private:
    static bool file_type_id_enabled;
    static bool file_signature_enabled;
    static bool file_capture_enabled;
    static bool file_processing_initiated;
    static FileCache* file_cache;
};
} // namespace snort
#endif

file_stats.h

Path = src/file_api/file_stats.h

#ifndef FILE_STATS_H
#define FILE_STATS_H

#include "framework/counts.h"
#include "main/thread.h"

#include "file_api.h"
#include "file_config.h"

#define MAX_PROTOCOL_ORDINAL 8192  // FIXIT-L use std::vector and get_protocol_count()

struct FileCounts
{
    PegCount files_total;
    PegCount file_data_total;
    PegCount cache_add_fails;
    PegCount files_over_flow_limit_not_processed;
    PegCount max_concurrent_files_per_flow;
    PegCount files_buffered_total;
    PegCount files_released_total;
    PegCount files_freed_total;
    PegCount files_captured_total;
    PegCount file_memcap_failures_total;
    PegCount file_memcap_failures_reserve;  // This happens during reserve
    PegCount file_reserve_failures;         // This happens during reserve
    PegCount file_size_min;                 // This happens during reserve
    PegCount file_size_max;                 // This happens during reserve
    PegCount file_within_packet;
    PegCount file_buffers_used_max;         // maximum buffers used simultaneously
    PegCount file_buffers_allocated_total;
    PegCount file_buffers_freed_total;
    PegCount file_buffers_released_total;
    PegCount file_buffers_free_errors;
    PegCount file_buffers_release_errors;
};

struct FileStats
{
    PegCount files_processed[FILE_ID_MAX + 1][2];
    PegCount signatures_processed[FILE_ID_MAX + 1][2];
    PegCount verdicts_type[FILE_VERDICT_MAX];
    PegCount verdicts_signature[FILE_VERDICT_MAX];
    PegCount files_by_proto[MAX_PROTOCOL_ORDINAL + 1];
    PegCount signatures_by_proto[MAX_PROTOCOL_ORDINAL + 1];
    PegCount data_processed[FILE_ID_MAX + 1][2];
};

extern THREAD_LOCAL FileCounts file_counts;
extern THREAD_LOCAL FileStats* file_stats;

void file_stats_init();
void file_stats_term();

void file_stats_sum();
void file_stats_print();
void file_stats_clear();

#endif

filters/

A collection of several different event and detection filtering function. The types of filters implemented here include:

Detection Filter - One of the last steps of the rule evaluation process. A detection filter can prevent a rule from firing based on a simple threshold. For example, only generate an alert if the filter has been evaluated N times in M time period. Events are accumulated into a multithreaded hash structure, to allow for real-time eventing as soon as the threshold value is crossed, regardless of which thread(s) processed the prior, non-eventing packets.

Rate Filter - Based on configuration options, generically track multiple occurrences of the same event/address tuples. The configuration can specify a limit where-by if the tracked limit is exceeded, the action of the event is changed. For instance, the first N occurrences of Event X in time period Y can Alert, but if this rate is exceeded subsequent occurrence will Drop. This function can be used to protect against DOS type of attacks.

Event Filter - After the rules engine generates whatever actions it needs to, the Event Filter is then invoked to filter the logging of these events. Once again, tracking by event/address tuples, block the logging of events if the configured counts per time is exceeded. This will tend to reduce the logging system load for rules that fire too often.

All of the filters in this area are a collection of similar services brought together to share the same event tracking logic. sfthreshold.cc implements a generic threshold tracking mechanism using a hash table. This hash structure permits the various filter/threshold components to build event tracking facilities.

Detection filter support the detection_filter rule option. Rate and event filters have builtin modules defined in main/modules.cc. Those module definitions should be refactored into the appropriate filter directory.

detection_filter.h

Path = src/filters/detection_filter.h

#ifndef DETECTION_FILTER_H
#define DETECTION_FILTER_H

// the use of threshold standalone, in config, and in rules is deprecated.
// - standalone and config are replaced with event_filter.
// - within a rule is replaced with detection_filter.
//
// both detection_filter and event_filter use the same basic mechanism.
// however, detection_filter is evaluated as the final step in rule matching.
// and thereby controls event generation.  event_filter is evaluated after
// the event is queued, and thereby controls which events get logged.

#include "profiler/profiler.h"

namespace snort
{

extern THREAD_LOCAL snort::ProfileStats detectionFilterPerfStats;

struct SfIp;
}

struct DetectionFilterConfig
{
    unsigned memcap;
    int count;
    int enabled;
};

DetectionFilterConfig* DetectionFilterConfigNew();
void DetectionFilterConfigFree(DetectionFilterConfig*);

void detection_filter_init(DetectionFilterConfig*);
void detection_filter_term();

int detection_filter_test(void*, const snort::SfIp* sip, const snort::SfIp* dip, long curtime);
struct THD_NODE* detection_filter_create(DetectionFilterConfig*, struct THDX_STRUCT*);

#endif

rate_filter.h

Path = src/filters/rate_filter.h

#ifndef RATE_FILTER_H
#define RATE_FILTER_H

// rate filter interface for Snort
namespace snort
{
struct Packet;
struct SnortConfig;
}
struct RateFilterConfig;
struct tSFRFConfigNode;
struct OptTreeNode;

RateFilterConfig* RateFilter_ConfigNew();
void RateFilter_ConfigFree(RateFilterConfig*);
void RateFilter_Cleanup();

int RateFilter_Create(snort::SnortConfig* sc, RateFilterConfig*, tSFRFConfigNode*);
int RateFilter_Test(const OptTreeNode*, snort::Packet*);

#endif

sfrf.h

Path = src/filters/sfrf.h

#ifndef SFRF_H
#define SFRF_H

#include <ctime>
#include <mutex>

#include "actions/actions.h"
#include "framework/counts.h"
#include "main/policy.h"
#include "sfip/sf_ip.h"
#include "sfip/sf_ipvar.h"

namespace snort
{
class GHash;
struct SfIp;
struct SnortConfig;
}

#define SFRF_MAX_GENID 8129

typedef enum
{
    SFRF_TRACK_BY_SRC = 1,
    SFRF_TRACK_BY_DST,
    SFRF_TRACK_BY_RULE,
    SFRF_TRACK_BY_MAX
} SFRF_TRACK;


typedef enum
{
    SFRF_COUNT_NOP,
    SFRF_COUNT_RESET,
    SFRF_COUNT_INCREMENT,
    SFRF_COUNT_DECREMENT,
    SFRF_COUNT_MAX
} SFRF_COUNT_OPERATION;

typedef enum
{
    FS_NEW = 0, FS_OFF, FS_ON, FS_MAX
} FilterState;

struct tSFRFConfigNode
{
    int tid;
    unsigned gid;
    unsigned sid;
    PolicyId policyId;
    SFRF_TRACK tracking;
    unsigned count;
    unsigned seconds;
    Actions::Type newAction;
    unsigned timeout;
    sfip_var_t* applyTo;
};

struct tSFRFSidNode
{
    PolicyId policyId;
    unsigned gid;
    unsigned sid;
    struct sf_list* configNodeList;
};

struct tSFRFGenHashKey
{
    PolicyId policyId;
    unsigned sid;
};

struct RateFilterConfig
{
    /* Array of hash, indexed by gid. Each array element is a hash, which
     * is keyed on sid/policyId and data is a tSFRFSidNode node.
     */
    snort::GHash* genHash [SFRF_MAX_GENID];

    unsigned memcap;
    unsigned noRevertCount;
    int count;
    int internal_event_mask;
};

struct RateFilterStats
{
    PegCount xhash_nomem_peg = 0;
};

void SFRF_Delete();
void SFRF_Flush();
int SFRF_ConfigAdd(snort::SnortConfig*, RateFilterConfig*, tSFRFConfigNode*);

int SFRF_TestThreshold(RateFilterConfig *config, unsigned gid, unsigned sid,
    PolicyId policyid, const snort::SfIp *sip, const snort::SfIp *dip,
    time_t curTime, SFRF_COUNT_OPERATION);

void SFRF_ShowObjects(RateFilterConfig*);

int SFRF_Alloc(unsigned int memcap);

inline void enable_internal_event(RateFilterConfig* config, uint32_t sid)
{
    if (config == nullptr)
        return;

    config->internal_event_mask |= (1 << sid);
}

inline bool is_internal_event_enabled(RateFilterConfig* config, uint32_t sid)
{
    if (config == nullptr)
        return false;

    return (config->internal_event_mask & (1 << sid));
}

static std::mutex sfrf_hash_mutex;

#endif

sfthd.h

Path = src/filters/sfthd.h

#ifndef SFTHD_H
#define SFTHD_H

#include "framework/counts.h"
#include "main/policy.h"
#include "sfip/sf_ip.h"
#include "utils/cpp_macros.h"

#include <mutex>
#include <vector>

namespace snort
{
class GHash;
class XHash;
struct SnortConfig;
}

typedef struct sf_list SF_LIST;

static std::mutex sfthd_hash_mutex;

/*!
    Max GEN_ID value - Set this to the Max Used by Snort, this is used for the
    dimensions of the gen_id lookup array.

    Rows in each hash table, by gen_id.
*/
#define THD_MAX_GENID     8129
#define THD_GEN_ID_1_ROWS 4096
#define THD_GEN_ID_ROWS   512

#define THD_NO_THRESHOLD (-1)

#define THD_TOO_MANY_THDOBJ (-15)

/*!
   Type of Thresholding
*/
enum
{
    THD_TYPE_LIMIT,
    THD_TYPE_THRESHOLD,
    THD_TYPE_BOTH,
    THD_TYPE_SUPPRESS,
    THD_TYPE_DETECT
};

/*
   Very high priority for suppression objects
   users priorities are limited to this minus one
*/
#define THD_PRIORITY_SUPPRESS 1000000

/*!
   Tracking by src, or by dst
*/
enum
{
    THD_TRK_NONE,  // suppress only
    THD_TRK_SRC,
    THD_TRK_DST
};

/*!
    THD_IP_NODE

    Dynamic hashed node data - added and deleted during runtime
    These are added during run-time, and recycled if we max out memory usage.
*/
struct THD_IP_NODE
{
    unsigned count;
    unsigned prev;
    time_t tstart;
    time_t tlast;
};

/*!
    THD_IP_NODE_KEY

    HASH Key to lookup and store Ip nodes. The structure now tracks thresholds for different
    policies. This destroys locality of reference and may cause poor performance.
*/
PADDING_GUARD_BEGIN
struct THD_IP_NODE_KEY
{
    int thd_id;
    PolicyId policyId;
    snort::SfIp ip;
    uint16_t padding;
};

struct THD_IP_GNODE_KEY
{
    unsigned gen_id;
    unsigned sig_id;
    PolicyId policyId;
    snort::SfIp ip;
    uint16_t padding;
};
PADDING_GUARD_END

/*!
    A Thresholding Object
    These are created at program startup, and remain static.
    The THD_IP_NODE elements are dynamic.
*/
struct THD_NODE
{
    int thd_id = 0;        /* Id of this node */
    unsigned gen_id = 0;   /* Keep these around if needed */
    unsigned sig_id = 0;
    int tracking = 0;      /* by_src, by_dst */
    int type = 0;
    int priority = 0;
    int count = 0;
    unsigned seconds = 0;
    sfip_var_t* ip_address = nullptr;
};

/*!
    The THD_ITEM acts as a container of gen_id+sig_id based threshold objects,
    this allows multiple threshold objects to be applied to a single
    gen_id+sig_id pair. The sflist is created using the priority field,
    so highest priority objects are first in the list. When processing the
    highest priority object will trigger first.

    These are static data elements, built at program startup.
*/
struct THD_ITEM
{
    PolicyId policyId;
    unsigned gen_id; /* just so we know what gen_id we are */
    unsigned sig_id;
    /*
     * List of THD_NODE's - walk this list and hash the
     * 'THD_NODE->sfthd_id + src_ip or dst_ip' to get the correct THD_IP_NODE.
     */
    SF_LIST* sfthd_node_list;
};

// Temporary structure useful when parsing the Snort rules
struct THDX_STRUCT
{
    unsigned gen_id;
    unsigned sig_id;
    unsigned seconds;

    int type;
    int count;
    int tracking;
    int priority;

    sfip_var_t* ip_address;
};

struct tThdItemKey
{
    PolicyId policyId;
    unsigned sig_id;
};

/*!
    THD_STRUCT

    The main thresholding data structure.

    Local and global threshold thd_id's are all unique, so we use just one
    ip_nodes lookup table
 */
struct THD_STRUCT
{
    snort::XHash* ip_nodes;   /* Global hash of active IP's key=THD_IP_NODE_KEY, data=THD_IP_NODE */
    snort::XHash* ip_gnodes;  /* Global hash of active IP's key=THD_IP_GNODE_KEY, data=THD_IP_GNODE */
};

struct ThresholdObjects
{
    int count = 0;  /* Total number of thresholding/suppression objects */
    std::vector<snort::GHash*> sfthd_vector;    /* Local Hash of THD_ITEM nodes,  lookup by key=sig_id */
    /*
    * Vector of unordered maps, each map indexed by generator ID (genId) and containing
    * unique pointers to THD_NODE structures. The outer vector is indexed by policyId
    * and can dynamically vary in size. Each map represents THD_NODE structures for a specific
    * policyId, with the key being the genId. This allows direct access to THD_NODE structures
    * by their generator ID within a given policy context, optimizing lookup and management
    * of thresholding nodes.
    */
    std::vector<std::unordered_map<uint32_t, std::unique_ptr<THD_NODE>>> sfthd_gvector;

    PolicyId numPoliciesAllocated = 0;
};

struct EventFilterStats
{
    PegCount xhash_nomem_peg_local = 0;
    PegCount xhash_nomem_peg_global = 0;
};

/*
 * Prototypes
 */
// lbytes = local threshold memcap
// gbytes = global threshold memcap (0 to disable global)
THD_STRUCT* sfthd_new(unsigned lbytes, unsigned gbytes);
snort::XHash* sfthd_local_new(unsigned bytes);
snort::XHash* sfthd_global_new(unsigned bytes);
void sfthd_free(THD_STRUCT*);
ThresholdObjects* sfthd_objs_new();
void sfthd_objs_free(ThresholdObjects*);

int sfthd_test_rule(snort::XHash* rule_hash, THD_NODE* sfthd_node,
    const snort::SfIp* sip, const snort::SfIp* dip, long curtime, PolicyId policy_id);

THD_NODE* sfthd_create_rule_threshold(
    int id,
    int tracking,
    int type,
    int count,
    unsigned int seconds
    );
void sfthd_node_free(THD_NODE*);

int sfthd_create_threshold(snort::SnortConfig*, ThresholdObjects*, unsigned gen_id,
    unsigned sig_id, int tracking, int type, int priority, int count,
    unsigned seconds, sfip_var_t* ip_address, PolicyId policy_id);

//  1: don't log due to event_filter
//  0: log
// -1: don't log due to suppress
int sfthd_test_threshold(ThresholdObjects*, THD_STRUCT*, unsigned gen_id, unsigned sig_id,
    const snort::SfIp* sip, const snort::SfIp* dip, long curtime, PolicyId policy_id);

snort::XHash* sfthd_new_hash(unsigned, size_t, size_t);

int sfthd_test_local(snort::XHash* local_hash, THD_NODE* sfthd_node, const snort::SfIp* sip,
    const snort::SfIp* dip, time_t curtime, PolicyId policy_id);

#ifdef THD_DEBUG
int sfthd_show_objects(THD_STRUCT* thd);
#endif

#endif

sfthreshold.h

Path = src/filters/sfthreshold.h

#ifndef SFTHRESHOLD_H
#define SFTHRESHOLD_H

#include "main/policy.h"

namespace snort
{
struct SfIp;
struct SnortConfig;
}
struct THDX_STRUCT;
struct ThresholdObjects;

struct ThresholdConfig
{
    ThresholdObjects* thd_objs;
    unsigned memcap;
    int enabled;
};

ThresholdConfig* ThresholdConfigNew();
void ThresholdConfigFree(ThresholdConfig*);
void sfthreshold_reset();
int sfthreshold_create(snort::SnortConfig*, ThresholdConfig*, THDX_STRUCT*, PolicyId);
int sfthreshold_test(
    unsigned int, unsigned int, const snort::SfIp*, const snort::SfIp*, long curtime,
    PolicyId);
void sfthreshold_free();

int sfthreshold_alloc(unsigned int l_memcap, unsigned int g_memcap);

#endif

flow/

Flows are preallocated at startup and stored in protocol specific caches. FlowKey is used for quick look up in the cache hash table.

Each flow may have associated inspectors:

  • clouseau is the Wizard bound to the flow to help determine the appropriate service inspector

  • gadget is the service inspector

  • data is a passive service inspector such as a client config.

FlowData is used by various inspectors to store specific data on the flow for later use. Any inspector may store data on the flow, not just clouseau gadget.

FlowData reference counts the associated inspector so that the inspector can be freed (via garbage collection) after a reload.

There are many flags that may be set on a flow to indicate session tracking state, disposition, etc.

High Availability

HighAvailability (ha.cc, ha.h) serves to synchronize session state between high availability partners. The primary purpose is to exchange session state with the goal of keeping the session caches in sync. Other clients may also register with the HA service to exchange additional session data by implementing one or more FlowHAClient classes.

HA uses Side Channel Connectors and/or DAQ module IOCTLs to transmit and receive HA state messages. A full duplex Side Channel is required for Side Channel communications. A DAQ module that supports the IOCTLs for setting and getting HA state is required for a functional DAQ-backed setup. Modules that do not support these IOCTLs will silently fail and present no HA state on future packets in the flow. HA state will be queried from the DAQ module prior to new flow creation when the appropriate DAQ packet message flag has been set on the initiating packet.

The HA subsystem exchanges two high level message types: - DELETE: Indicate to the partner that a session has been removed. No additional HA client status will be exchanged. (Not used for DAQ-backed storage.) - UPDATE: Indicate all other state changes. The message always includes the session state and optionally may include state from other HA clients. By default, the update messages are incremental. In the case of DAQ-backed storage, the update messages are always fully formed.

The HA subsystem implements these classes: - HighAvailabilityManager - A collection of static elements providing the top-most interface to HA capabilities. - HAMessage - A wrapper around the actual message and includes a cursor and convenience functions for producer/consumer activity. Passed around among all message handing classes/methods. - HighAvailability - If HA is enabled, instantiated in each packet thread and provides all primary HA functionality for the thread. Referenced via a THREAD_LOCAL object pointer. - FlowHAState - One per flow and referenced via a pointer in the Flow object. Contains all dynamic state information. A set of get’ers and set’ers are implemented to help manage the flags, client pending flags, and timing information. - FlowHAClient - All HA coordination is managed via a set of FlowHAClient’s. Every function that desires to use HA capabilities defines one FlowHAClient object within the packet processing thread. Each client is provided a handle bit-mask as part of the pending logic. The client may set the pending bit to indicate the desire to add content to the HA message. A maximum of 17 clients may existing (numbers 1-16, corresponding to the 16 bits of the pending flags. Client 0 is the session, is always present, and is handled as a special case. Client 0 is the fundamental session HA state sync functionality. Other clients are optional.

09/25/2023 In response to the need for more nuanced management of different protocol types within the flow_cache, we’ve leveraged the newly added feature in xhash. This enhancement includes the introduction of Protocol-Based Least Recently Used (LRU) caches, which substantially improves the handling of multiple protocol types in the cache. With this implementation, during various pruning sessions (idle, excess, and timeout etc), we’ve adopted a round-robin method to cycle through the LRU lists designated for each protocol type. This methodology ensures that every protocol, irrespective of its inherent timeout characteristics, is given an equal opportunity for pruning and retirement from the cache. This adjustment to flow_cache not only addresses the previously observed bias during pruning sessions, especially concerning UDP flows, but lays down a framework for more advanced and controlled data management within the cache moving forward.

deferred_trust.h

Path = src/flow/deferred_trust.h

#ifndef DEFERRED_TRUST_H
#define DEFERRED_TRUST_H

#include <cstdint>
#include <forward_list>

#include "main/snort_types.h"

namespace snort
{

class Active;
class Flow;
struct Packet;

class DeferredTrust
{
    // This class is used to delay session trust. This is used in cases where
    // a module needs to continue inspecting a session to enforce policy.
    // A module calls set_deferred_trust with the on parameter true to begin
    // deferring. The sets the state to TRUST_DEFER_ON. If trust session is called
    // while deferring, the state is changed to TRUST_DEFER_DEFERRING to the trust action.
    // A module calls set_deferred_trust with the on parameter false to stop deferring.
    // When all modules have stopped deferring, the state is checked. If the state is
    // TRUST_DEFER_DEFERRING, the state is changed to TRUST_DEFER_DO_TRUST. Otherwise, the state
    // is set to TRUST_DEFER_OFF.
    // The TRUST_DEFER_DO_TRUST state is checked at the end of packet processing. If the state
    // is TRUST_DEFER_DO_TRUST and the action is ACT_ALLOW, the session is trusted.
    // If a drop, block or reset action occurs while deferring, deferring is stopped and the
    // block or blocklist version is enforced.
    // The module_id, a unique module identifier created by calling
    // FlowData::create_flow_data_id(), is used to track the modules that are currently deferring.
    // This allows the module to use trusted deferring without needing to track the deferring
    // state of the module.
    enum DeferredTrustState : uint8_t
    {
        TRUST_DEFER_OFF = 0,
        TRUST_DEFER_ON,
        TRUST_DEFER_DEFERRING,
        TRUST_DEFER_DO_TRUST,
    };

public:
    DeferredTrust() = default;
    ~DeferredTrust() = default;
    SO_PUBLIC void set_deferred_trust(unsigned module_id, bool on);
    bool is_active() const
    { return TRUST_DEFER_ON == deferred_trust || TRUST_DEFER_DEFERRING == deferred_trust; }
    bool try_trust()
    {
        if (TRUST_DEFER_ON == deferred_trust)
            deferred_trust = TRUST_DEFER_DEFERRING;
        return TRUST_DEFER_DEFERRING != deferred_trust;
    }
    bool is_deferred() const
    { return TRUST_DEFER_DEFERRING == deferred_trust; }
    void clear()
    {
        deferred_trust = TRUST_DEFER_OFF;
        deferred_trust_modules.clear();
    }
    void finalize(Active&);

protected:
    std::forward_list<unsigned> deferred_trust_modules;
    DeferredTrustState deferred_trust = TRUST_DEFER_OFF;
};

}

#endif

expect_cache.h

Path = src/flow/expect_cache.h

#ifndef EXPECT_CACHE_H
#define EXPECT_CACHE_H

// ExpectCache is used to track anticipated flows (like ftp data channels).
// when the flow is found, it updated with the given info.

//-------------------------------------------------------------------------
// data structs
// -- key has IP address and port pairs; one port must be zero (wild card)
//    forming a 3-tuple
// -- node struct is stored in hash table by key
// -- each node struct has one or more list structs linked together
// -- each list struct has a list of flow data
// -- when a new expect is added, a new list struct is created if a new
//    node is created or the last list struct of an existing node already
//    has the same preproc id in the flow data list
// -- when a new expect is added, the last list struct is used if the
//    given preproc id is not already in the flow data list
// -- nodes are preallocated and stored in hash table; if there is no node
//    available when an expect is added, LRU nodes are pruned
// -- list structs are also preallocated and stored in free list; if there
//    is no list struct available when an expect is added, LRU nodes are
//    pruned freeing up both nodes and list structs
// -- the number of list structs per node is capped at MAX_LIST; once
//    reached, requests to add new expects requiring new list structs fail
// -- the number of data structs per list struct is not capped
// -- example:  ftp preproc adds a new 3-tuple twice for 2 expected data
//    channels -> new node with 2 list structs linked to it
// -- example:  ftp preproc adds a new 3-tuple once and then another
//    preproc expects the same 3-tuple -> new node with one list struct
//    is created for ftp and the next request goes in that same list
//    struct
// -- new list structs are appended to node's list struct chain
// -- matching expected sessions are pulled off from the head of the node's
//    list struct chain
//
// FIXIT-M expiration is by node struct but should be by list struct, ie
//    individual sessions, not all sessions to a given 3-tuple
//    (this would make pruning a little harder unless we add linkage
//    a la FlowCache)
//-------------------------------------------------------------------------
#include <vector>
#include "flow/flow_key.h"
#include "target_based/snort_protocols.h"

struct ExpectNode;

namespace snort
{
class Flow;
class FlowData;
struct Packet;

struct SO_PUBLIC ExpectFlow
{
    struct ExpectFlow* next;
    snort::FlowData* data;

    ~ExpectFlow();
    void clear();
    int add_flow_data(snort::FlowData*);
    snort::FlowData* get_flow_data(unsigned);
    static std::vector<ExpectFlow*>* get_expect_flows();
    static void reset_expect_flows();
    static void handle_expected_flows(const snort::Packet*);
};
}

class ExpectCache
{
public:
    ExpectCache(uint32_t max);
    ~ExpectCache();

    ExpectCache(const ExpectCache&) = delete;
    ExpectCache& operator=(const ExpectCache&) = delete;

    int add_flow(const snort::Packet *ctrlPkt, PktType, IpProtocol, const snort::SfIp* cliIP,
        uint16_t cliPort, const snort::SfIp* srvIP, uint16_t srvPort, char direction,
        snort::FlowData*, SnortProtocolId snort_protocol_id = UNKNOWN_PROTOCOL_ID,
        bool swap_app_direction = false, bool expect_multi = false, bool bidirectional = false,
        bool expect_persist = false);

    bool check(snort::Packet*, snort::Flow*);

    unsigned long get_expects() { return expects; }
    unsigned long get_realized() { return realized; }
    unsigned long get_prunes() { return prunes; }
    unsigned long get_overflows() { return overflows; }
    void reset_stats()
    {
        expects = 0;
        realized = 0;
        prunes = 0;
        overflows = 0;
    }

private:
    void prune_lru();

    ExpectNode* get_node(snort::FlowKey&, bool&);
    snort::ExpectFlow* get_flow(ExpectNode*, uint32_t, int16_t);
    bool set_data(ExpectNode*, snort::ExpectFlow*&, snort::FlowData*);
    ExpectNode* find_node_by_packet(snort::Packet*, snort::FlowKey&);
    bool process_expected(ExpectNode*, snort::FlowKey&, snort::Packet*, snort::Flow*);

private:
    class ZHash* hash_table;
    ExpectNode* nodes;
    snort::ExpectFlow* pool;
    snort::ExpectFlow* free_list;

    unsigned long expects = 0;
    unsigned long realized = 0;
    unsigned long prunes = 0;
    unsigned long overflows = 0;
};

#endif

flow_cache.h

Path = src/flow/flow_cache.h

#ifndef FLOW_CACHE_H
#define FLOW_CACHE_H

// there is a FlowCache instance for each protocol.
// Flows are stored in a ZHash instance by FlowKey.

#include <ctime>
#include <type_traits>

#include "framework/counts.h"
#include "main/thread.h"

#include "flow_config.h"
#include "prune_stats.h"

namespace snort
{
class Flow;
struct FlowKey;
}

class FlowUniList;

class FlowCache
{
public:
    FlowCache(const FlowCacheConfig&);
    ~FlowCache();

    FlowCache(const FlowCache&) = delete;
    FlowCache& operator=(const FlowCache&) = delete;

    snort::Flow* find(const snort::FlowKey*);
    snort::Flow* allocate(const snort::FlowKey*);

    bool release(snort::Flow*, PruneReason = PruneReason::NONE, bool do_cleanup = true);

    unsigned prune_idle(uint32_t thetime, const snort::Flow* save_me);
    unsigned prune_excess(const snort::Flow* save_me);
    bool prune_one(PruneReason, bool do_cleanup, uint8_t type = 0);
    unsigned timeout(unsigned num_flows, time_t cur_time);
    unsigned delete_flows(unsigned num_to_delete);
    unsigned prune_multiple(PruneReason, bool do_cleanup);

    unsigned purge();
    unsigned get_count();

    unsigned get_max_flows() const
    { return config.max_flows; }

    PegCount get_total_prunes() const
    { return prune_stats.get_total(); }

    PegCount get_prunes(PruneReason reason) const
    { return prune_stats.get(reason); }

    PegCount get_proto_prune_count(PruneReason reason, PktType type) const
    { return prune_stats.get_proto_prune_count(reason,type); }

    PegCount get_total_deletes() const
    { return delete_stats.get_total(); }

    PegCount get_deletes(FlowDeleteState state) const
    { return delete_stats.get(state); }

    void reset_stats()
    {
        prune_stats = PruneStats();
        delete_stats = FlowDeleteStats();
    }

    void unlink_uni(snort::Flow*);

    void set_flow_cache_config(const FlowCacheConfig& cfg)
    { config = cfg; }

    const FlowCacheConfig& get_flow_cache_config() const
    { return config; }

    unsigned get_flows_allocated() const;

    size_t uni_flows_size() const;
    size_t uni_ip_flows_size() const;
    size_t flows_size() const;

private:
    void delete_uni();
    void push(snort::Flow*);
    void link_uni(snort::Flow*);
    void remove(snort::Flow*);
    void retire(snort::Flow*);
    unsigned prune_unis(PktType);
    unsigned delete_active_flows
        (unsigned mode, unsigned num_to_delete, unsigned &deleted);

private:
    static const unsigned cleanup_flows = 1;
    FlowCacheConfig config;
    uint32_t flags;

    class ZHash* hash_table;
    FlowUniList* uni_flows;
    FlowUniList* uni_ip_flows;

    PruneStats prune_stats;
    FlowDeleteStats delete_stats;
};
#endif

flow_config.h

Path = src/flow/flow_config.h

#ifndef FLOW_CONFIG_H
#define FLOW_CONFIG_H

#include "framework/decode_data.h"

// configured by the stream module
struct FlowTypeConfig
{
    unsigned nominal_timeout = 0;
};

struct FlowCacheConfig
{
    unsigned max_flows = 0;
    unsigned pruning_timeout = 0;
    FlowTypeConfig proto[to_utype(PktType::MAX)];
    unsigned prune_flows = 0;
};

#endif

flow_control.h

Path = src/flow/flow_control.h

#ifndef FLOW_CONTROL_H
#define FLOW_CONTROL_H

// this is where all the flow caches are managed and where all flows are
// processed.  flows are pruned as needed to process new flows.

#include <cstdint>
#include <vector>

#include "flow/flow_config.h"
#include "framework/counts.h"
#include "framework/decode_data.h"
#include "framework/inspector.h"

namespace snort
{
class Flow;
class FlowData;
struct FlowKey;
struct Packet;
struct SfIp;
}
class FlowCache;

enum class PruneReason : uint8_t;
enum class FlowDeleteState : uint8_t;

class FlowControl
{
public:
    FlowControl(const FlowCacheConfig& fc);
    ~FlowControl();

    void set_flow_cache_config(const FlowCacheConfig& cfg);
    const FlowCacheConfig& get_flow_cache_config() const;
    void init_proto(PktType, snort::InspectSsnFunc);
    void init_exp(uint32_t max);
    unsigned get_flows_allocated() const;

    bool process(PktType, snort::Packet*, bool* new_flow = nullptr);
    snort::Flow* find_flow(const snort::FlowKey*);
    snort::Flow* new_flow(const snort::FlowKey*);
    void release_flow(const snort::FlowKey*);
    void release_flow(snort::Flow*, PruneReason);
    void purge_flows();
    unsigned delete_flows(unsigned num_to_delete);
    bool prune_one(PruneReason, bool do_cleanup);
    snort::Flow* stale_flow_cleanup(FlowCache*, snort::Flow*, snort::Packet*);
    void timeout_flows(unsigned int, time_t cur_time);
    void check_expected_flow(snort::Flow*, snort::Packet*);
    unsigned prune_multiple(PruneReason, bool do_cleanup);

    int add_expected_ignore(
        const snort::Packet* ctrlPkt, PktType, IpProtocol,
        const snort::SfIp *srcIP, uint16_t srcPort,
        const snort::SfIp *dstIP, uint16_t dstPort,
        char direction, snort::FlowData*);

    int add_expected(const snort::Packet* ctrlPkt, PktType, IpProtocol, const snort::SfIp *srcIP,
        uint16_t srcPort, const snort::SfIp *dstIP, uint16_t dstPort, SnortProtocolId snort_protocol_id,
        snort::FlowData*, bool swap_app_direction = false, bool expect_multi = false,
        bool bidirectional = false, bool expect_persist = false);

    class ExpectCache* get_exp_cache()
    { return exp_cache; }

    PegCount get_flows()
    { return num_flows; }

    PegCount get_total_prunes() const;
    PegCount get_prunes(PruneReason) const;
    PegCount get_proto_prune_count(PruneReason, PktType) const;
    PegCount get_total_deletes() const;
    PegCount get_deletes(FlowDeleteState state) const;
    void clear_counts();

    PegCount get_uni_flows() const;
    PegCount get_uni_ip_flows() const;
    PegCount get_num_flows() const;

private:
    void set_key(snort::FlowKey*, snort::Packet*);
    unsigned process(snort::Flow*, snort::Packet*, bool new_ha_flow);
    void update_stats(snort::Flow*, snort::Packet*);

private:
    snort::InspectSsnFunc get_proto_session[to_utype(PktType::MAX)] = {};
    PegCount num_flows = 0;
    FlowCache* cache = nullptr;
    snort::Flow* mem = nullptr;
    class ExpectCache* exp_cache = nullptr;
    PktType last_pkt_type = PktType::NONE;
};

#endif

flow_data.h

Path = src/flow/flow_data.h

#ifndef FLOW_DATA_H
#define FLOW_DATA_H

#include "main/snort_types.h"

namespace snort
{
class Inspector;
struct Packet;

class SO_PUBLIC FlowData
{
public:
    FlowData(unsigned u, Inspector* = nullptr);
    virtual ~FlowData();

    unsigned get_id()
    { return id; }

    static unsigned create_flow_data_id()
    { return ++flow_data_id; }

    Inspector* get_handler() { return handler; }

    // deprecated - do not implement
    virtual size_t size_of() { return 0; }

    virtual void handle_expected(Packet*) { }
    virtual void handle_retransmit(Packet*) { }
    virtual void handle_eof(Packet*) { }

public:  // FIXIT-L privatize
    FlowData* next;
    FlowData* prev;

private:
    static unsigned flow_data_id;
    Inspector* handler;
    unsigned id;
};

// The flow data created from SO rules must use RuleFlowData
// to support reload
class SO_PUBLIC RuleFlowData : public FlowData
{
protected:
    RuleFlowData(unsigned u);
public:
    ~RuleFlowData() override = default;
};

}
#endif

flow.h

Path = src/flow/flow.h

#ifndef FLOW_H
#define FLOW_H

// Flow is the object that captures all the data we know about a session,
// including IP for defragmentation and TCP for desegmentation.  For all
// protocols, it used to track connection status bindings, and inspector
// state.  Inspector state is stored in FlowData, and Flow manages a list
// of FlowData items.

#include <string>
#include <sys/time.h>

#include <daq_common.h>

#include "detection/ips_context_chain.h"
#include "flow/deferred_trust.h"
#include "flow/flow_data.h"
#include "flow/flow_stash.h"
#include "framework/data_bus.h"
#include "framework/decode_data.h"
#include "framework/inspector.h"
#include "network_inspectors/appid/application_ids.h"
#include "protocols/layer.h"
#include "sfip/sf_ip.h"
#include "target_based/snort_protocols.h"

#define SSNFLAG_SEEN_CLIENT         0x00000001
#define SSNFLAG_SEEN_SENDER         0x00000001
#define SSNFLAG_SEEN_SERVER         0x00000002
#define SSNFLAG_SEEN_RESPONDER      0x00000002

#define SSNFLAG_ESTABLISHED         0x00000004
#define SSNFLAG_MIDSTREAM           0x00000008 /* picked up midstream */

#define SSNFLAG_ECN_CLIENT_QUERY    0x00000010
#define SSNFLAG_ECN_SERVER_REPLY    0x00000020
#define SSNFLAG_CLIENT_FIN          0x00000040 /* server sent fin */
#define SSNFLAG_SERVER_FIN          0x00000080 /* client sent fin */

#define SSNFLAG_COUNTED_INITIALIZE  0x00000100
#define SSNFLAG_COUNTED_ESTABLISH   0x00000200
#define SSNFLAG_COUNTED_CLOSING     0x00000400
#define SSNFLAG_COUNTED_CLOSED      0x00000800

#define SSNFLAG_TIMEDOUT            0x00001000
#define SSNFLAG_PRUNED              0x00002000
#define SSNFLAG_RESET               0x00004000

#define SSNFLAG_TCP_PSEUDO_EST      0x00008000

#define SSNFLAG_DROP_CLIENT         0x00010000
#define SSNFLAG_DROP_SERVER         0x00020000

#define SSNFLAG_STREAM_ORDER_BAD    0x00100000
#define SSNFLAG_CLIENT_SWAP         0x00200000
#define SSNFLAG_CLIENT_SWAPPED      0x00400000

#define SSNFLAG_PROXIED             0x01000000
#define SSNFLAG_NO_DETECT_TO_CLIENT 0x02000000
#define SSNFLAG_NO_DETECT_TO_SERVER 0x04000000

#define SSNFLAG_ABORT_CLIENT        0x10000000
#define SSNFLAG_ABORT_SERVER        0x20000000

#define SSNFLAG_HARD_EXPIRATION     0x40000000
#define SSNFLAG_KEEP_FLOW           0x80000000

#define SSNFLAG_NONE                0x00000000 /* nothing, an MT bag of chips */

#define SSNFLAG_SEEN_BOTH (SSNFLAG_SEEN_SERVER | SSNFLAG_SEEN_CLIENT)
#define SSNFLAG_BLOCK (SSNFLAG_DROP_CLIENT|SSNFLAG_DROP_SERVER)

#define STREAM_STATE_NONE              0x0000
#define STREAM_STATE_ESTABLISHED       0x0001
#define STREAM_STATE_DROP_CLIENT       0x0002
#define STREAM_STATE_DROP_SERVER       0x0004
#define STREAM_STATE_MIDSTREAM         0x0008
#define STREAM_STATE_TIMEDOUT          0x0010
#define STREAM_STATE_UNREACH           0x0020
#define STREAM_STATE_CLOSED            0x0040
#define STREAM_STATE_BLOCK_PENDING     0x0080
#define STREAM_STATE_RELEASING         0x0100

class Continuation;
class BitOp;
class Session;

namespace snort
{
class FlowHAState;
struct FlowKey;
class IpsContext;
struct Packet;

typedef void (* StreamAppDataFree)(void*);

struct FilteringState
{
    uint8_t generation_id = 0;
    bool matched = false;

    void clear()
    {
        generation_id = 0;
        matched = false;
    }

    bool was_checked(uint8_t id) const
    {
        return generation_id and (generation_id == id);
    }

    void set_matched(uint8_t id, bool match)
    {
        generation_id = id;
        matched = match;
    }
};

struct FlowStats
{
    uint64_t client_pkts;
    uint64_t server_pkts;
    uint64_t client_bytes;
    uint64_t server_bytes;
    struct timeval start_time;
    uint64_t total_flow_latency;
};

struct LwState
{
    uint32_t session_flags;

    int16_t ipprotocol;
    SnortProtocolId snort_protocol_id;

    char direction;
    char ignore_direction;
};

class SO_PUBLIC StreamFlowIntf
{
public:
    virtual FlowData* get_stream_flow_data(const Flow* flow) = 0;
    virtual void set_stream_flow_data(Flow* flow, FlowData* flow_data) = 0;
    virtual void get_stream_id(const Flow* flow, int64_t& stream_id) = 0;
    virtual void* get_hi_msg_section(const Flow* flow) = 0;
    virtual void set_hi_msg_section(Flow* flow, void* section) = 0;
    virtual AppId get_appid_from_stream(const Flow*) { return APP_ID_NONE; }
    // Stream based flows should override this interface to return parent flow
    // when child flow is passed as input
    virtual Flow* get_stream_parent_flow(Flow* cflow) { return cflow; }
};

// this struct is organized by member size for compactness
class SO_PUBLIC Flow
{
public:
    enum class FlowState : uint8_t
    {
        SETUP = 0,
        INSPECT,
        BLOCK,
        RESET,
        ALLOW
    };
    Flow() = default;
    virtual ~Flow();

    Flow(const Flow&) = delete;
    Flow& operator=(const Flow&) = delete;

    void init(PktType);

    void flush(bool do_cleanup = true);
    void reset(bool do_cleanup = true);
    void restart(bool dump_flow_data = true);
    void clear(bool dump_flow_data = true);

    int set_flow_data(FlowData*);
    FlowData* get_flow_data(uint32_t proto) const;
    void free_flow_data(uint32_t proto);
    void free_flow_data(FlowData*);
    void free_flow_data();

    void call_handlers(Packet* p, bool eof = false);
    void markup_packet_flags(Packet*);
    void set_client_initiate(Packet*);
    void set_direction(Packet*);
    void set_expire(const Packet*, uint64_t timeout);
    bool expired(const Packet*) const;
    void set_ttl(Packet*, bool client);
    void set_mpls_layer_per_dir(Packet*);
    Layer get_mpls_layer_per_dir(bool);
    void swap_roles();
    void set_service(Packet*, const char* new_service);
    bool get_attr(const std::string& key, int32_t& val) const;
    bool get_attr(const std::string& key, std::string& val) const;
    void set_attr(const std::string& key, const int32_t& val);
    void set_attr(const std::string& key, const std::string& val);
    // Use this API when the publisher of the attribute allocated memory for it and can give up its
    // ownership after the call.
    void set_attr(const std::string& key, std::string* val)
    {
        assert(stash);
        stash->store(key, val);
    }

    template<typename T>
    bool get_attr(const std::string& key, T& val) const
    {
        assert(stash);
        return stash->get(key, val);
    }

    template<typename T>
    void set_attr(const std::string& key, const T& val)
    {
        assert(stash);
        stash->store(key, val);
    }

    uint32_t update_session_flags(uint32_t ssn_flags)
    { return ssn_state.session_flags = ssn_flags; }

    uint32_t set_session_flags(uint32_t ssn_flags)
    { return ssn_state.session_flags |= ssn_flags; }

    uint32_t get_session_flags() const
    { return ssn_state.session_flags; }

    uint32_t clear_session_flags(uint32_t ssn_flags)
    { return ssn_state.session_flags &= ~ssn_flags; }

    uint32_t clear_session_state(uint32_t ssn_state)
    { return session_state &= ~ssn_state; }

    void set_to_client_detection(bool enable);
    void set_to_server_detection(bool enable);

    int get_ignore_direction() const
    { return ssn_state.ignore_direction; }

    int set_ignore_direction(char ignore_direction)
    {
        ssn_state.ignore_direction = ignore_direction;
        return ssn_state.ignore_direction;
    }

    bool two_way_traffic() const
    { return (ssn_state.session_flags & SSNFLAG_SEEN_BOTH) == SSNFLAG_SEEN_BOTH; }

    bool is_pdu_inorder(uint8_t dir) const;

    bool is_direction_aborted(bool from_client) const;

    void set_proxied()
    { ssn_state.session_flags |= SSNFLAG_PROXIED; }

    bool is_proxied() const
    { return (ssn_state.session_flags & SSNFLAG_PROXIED) != 0; }

    bool is_stream() const
    { return pkt_type == PktType::TCP or pkt_type == PktType::USER; }

    void block()
    { ssn_state.session_flags |= SSNFLAG_BLOCK; }

    bool was_blocked() const
    { return (ssn_state.session_flags & SSNFLAG_BLOCK) != 0; }

    bool full_inspection() const
    { return (flow_state <= FlowState::INSPECT) and !is_inspection_disabled(); }

    void set_state(FlowState fs)
    { flow_state = fs; }

    void set_client(Inspector* ins)
    {
        if (ssn_client)
            ssn_client->rem_ref();
        ssn_client = ins;
        if (ssn_client)
            ssn_client->add_ref();
    }

    void set_server(Inspector* ins)
    {
        if (ssn_server)
            ssn_server->rem_ref();
        ssn_server = ins;
        if (ssn_server)
            ssn_server->add_ref();
    }

    void set_clouseau(Inspector* ins)
    {
        clouseau = ins;
        clouseau->add_ref();
    }

    void clear_clouseau()
    {
        clouseau->rem_ref();
        clouseau = nullptr;
    }

    bool searching_for_service()
    {
        return clouseau != nullptr;
    }

    void set_gadget(Inspector* ins)
    {
        gadget = ins;
        gadget->add_ref();
    }

    void clear_gadget()
    {
        gadget->rem_ref();
        gadget = nullptr;
        if (assistant_gadget != nullptr)
            clear_assistant_gadget();
    }

    void set_assistant_gadget(Inspector* ins)
    {
        assistant_gadget = ins;
        assistant_gadget->add_ref();
    }

    void clear_assistant_gadget()
    {
        assistant_gadget->rem_ref();
        assistant_gadget = nullptr;
    }

    void set_data(Inspector* pd)
    {
        data = pd;
        data->add_ref();
    }

    void clear_data()
    {
        data->rem_ref();
        data = nullptr;
    }

    virtual void disable_inspection()
    { flags.disable_inspect = true; }

    bool is_inspection_disabled() const
    { return flags.disable_inspect; }

    bool is_suspended() const
    { return context_chain.front(); }

    void set_default_session_timeout(uint32_t dst, bool force)
    {
        if (force || (default_session_timeout == 0))
            default_session_timeout = dst;
    }

    void set_hard_expiration()
    { ssn_state.session_flags |= SSNFLAG_HARD_EXPIRATION; }

    bool is_hard_expiration() const
    { return (ssn_state.session_flags & SSNFLAG_HARD_EXPIRATION) != 0; }

    void set_deferred_trust(unsigned module_id, bool on)
    { deferred_trust.set_deferred_trust(module_id, on); }

    bool cannot_trust() const
    { return deferred_trust.is_active(); }

    bool try_trust()
    { return deferred_trust.try_trust(); }

    void stop_deferring_trust()
    { deferred_trust.clear(); }

    void finalize_trust(Active& active)
    {
        deferred_trust.finalize(active);
    }

    void trust();

    bool trust_is_deferred() const
    { return deferred_trust.is_deferred(); }

    void set_idle_timeout(unsigned timeout)
    { idle_timeout = timeout; }

public:  // FIXIT-M privatize if possible
    // fields are organized by initialization and size to minimize
    // void space

    std::unordered_map<uint32_t, std::unique_ptr<FlowData>> flow_data;

    DeferredTrust deferred_trust;

    const FlowKey* key = nullptr;
    BitOp* bitop = nullptr;
    FlowHAState* ha_state = nullptr;
    FlowStash* stash = nullptr;

    uint8_t ip_proto = 0;
    PktType pkt_type = PktType::NONE; // ^^

    // these fields are always set; not zeroed
    Flow* prev = nullptr;
    Flow* next = nullptr;
    Session* session = nullptr;
    Inspector* ssn_client = nullptr;
    Inspector* ssn_server = nullptr;
    Continuation* ips_cont = nullptr;

    long last_data_seen = 0;
    Layer mpls_client = {};
    Layer mpls_server = {};

    IpsContextChain context_chain;
    FlowData* current_flow_data = nullptr;
    FlowStats flowstats = {};
    StreamFlowIntf* stream_intf = nullptr;

    SfIp client_ip = {};
    SfIp server_ip = {};

    LwState ssn_state = {};
    LwState previous_ssn_state = {};

    Inspector* clouseau = nullptr;  // service identifier
    Inspector* gadget = nullptr;    // service handler
    Inspector* assistant_gadget = nullptr;
    Inspector* data = nullptr;
    const char* service = nullptr;

    uint64_t expire_time = 0;

    unsigned network_policy_id = 0;
    unsigned inspection_policy_id = 0;
    unsigned ips_policy_id = 0;
    unsigned reload_id = 0;
    uint32_t tenant = 0;
    uint32_t default_session_timeout = 0;
    uint32_t idle_timeout = 0;
    int32_t client_intf = 0;
    int32_t server_intf = 0;

    int16_t client_group = 0;
    int16_t server_group = 0;

    uint16_t client_port = 0;
    uint16_t server_port = 0;

    uint16_t ssn_policy = 0;
    uint16_t session_state = 0;

    uint8_t inner_client_ttl = 0;
    uint8_t inner_server_ttl = 0;
    uint8_t outer_client_ttl = 0;
    uint8_t outer_server_ttl = 0;

    uint8_t response_count = 0;

    struct
    {
        bool client_initiated : 1;  // Set if the first packet on the flow was from the side that is
                                    // currently considered to be the client
        bool app_direction_swapped : 1; // Packet direction swapped from application perspective
        bool disable_inspect : 1;
        bool trigger_detained_packet_event : 1;
        bool trigger_finalize_event : 1;
        bool use_direct_inject : 1;
        bool data_decrypted : 1;    // indicate data in current flow is decrypted TLS application data
        bool snort_proto_id_set_by_ha : 1;
        bool efd_flow : 1;  // Indicate that current flow is an elephant flow
        bool svc_event_generated : 1; // Set if FLOW_NO_SERVICE_EVENT was generated for this flow
        bool retry_queued : 1; // Set if a packet was queued for retry for this flow
        bool ha_flow : 1; // Set if this flow was created by an HA message
        bool disable_reassembly_by_ips : 1; // Set if IPS has disabled reassembly for this flow
        bool ips_block_event_suppressed : 1; // Set if event filters have suppressed a block ips event
        bool ips_wblock_event_suppressed : 1; // set if event filters have suppressed a would block/drop ips event
        bool ips_pblock_event_suppressed : 1; // set if event filters have suppressed a partial block ips event
    } flags = {};

    FlowState flow_state = FlowState::SETUP;

    FilteringState filtering_state;

    DAQ_Verdict last_verdict = MAX_DAQ_VERDICT;

private:
    void clean();
};

inline void Flow::set_to_client_detection(bool enable)
{
    if ( enable )
        ssn_state.session_flags &= ~SSNFLAG_NO_DETECT_TO_CLIENT;
    else
        ssn_state.session_flags |= SSNFLAG_NO_DETECT_TO_CLIENT;
}

inline void Flow::set_to_server_detection(bool enable)
{
    if ( enable )
        ssn_state.session_flags &= ~SSNFLAG_NO_DETECT_TO_SERVER;
    else
        ssn_state.session_flags |= SSNFLAG_NO_DETECT_TO_SERVER;
}
}

#endif

flow_key.h

Path = src/flow/flow_key.h

#ifndef FLOW_KEY_H
#define FLOW_KEY_H

// FlowKey is used to store Flows in the caches.  the data members are
// sequenced to avoid void space.

#include <cstdint>

#include <daq_common.h>

#include "framework/decode_data.h"
#include "hash/hash_key_operations.h"
#include "utils/cpp_macros.h"

class HashKeyOperations;

namespace snort
{
struct SfIp;
struct SnortConfig;

class FlowHashKeyOps : public HashKeyOperations
{
public:
    FlowHashKeyOps(int rows)
        : HashKeyOperations(rows)
    { }

    unsigned do_hash(const unsigned char* k, int len) override;
    bool key_compare(const void* k1, const void* k2, size_t) override;
};

PADDING_GUARD_BEGIN
struct SO_PUBLIC FlowKey
{
    uint32_t   ip_l[4]; /* Low IP */
    uint32_t   ip_h[4]; /* High IP */
    uint32_t   mplsLabel;
    uint32_t   addressSpaceId;
    uint32_t   tenant_id;
    uint16_t   port_l;  /* Low Port - 0 if ICMP */
    uint16_t   port_h;  /* High Port - 0 if ICMP */
    int16_t    group_l;
    int16_t    group_h;
    uint16_t   vlan_tag;
    uint16_t   padding;
    uint8_t    ip_protocol;
    PktType    pkt_type;
    uint8_t    version;
    struct
    {
        bool group_used : 1;
        uint8_t padding_bits : 7;
    } flags;

    /* The init() functions return true if the key IP/port fields were actively
        normalized, reversing the source and destination addresses internally.
        The IP-only init() will always return false as we will not reorder its
        addresses at this time. */
    bool init(
        const SnortConfig*, PktType, IpProtocol,
        const snort::SfIp *srcIP, uint16_t srcPort,
        const snort::SfIp *dstIP, uint16_t dstPort,
        uint16_t vlanId, uint32_t mplsId, uint32_t addrSpaceId, uint32_t tid,
        int16_t group_h = DAQ_PKTHDR_UNKNOWN, int16_t group_l = DAQ_PKTHDR_UNKNOWN);

    bool init(
        const SnortConfig*, PktType, IpProtocol,
        const snort::SfIp *srcIP, const snort::SfIp *dstIP,
        uint32_t id, uint16_t vlanId,
        uint32_t mplsId, uint32_t addrSpaceId, uint32_t tid,
        int16_t group_h = DAQ_PKTHDR_UNKNOWN, int16_t group_l = DAQ_PKTHDR_UNKNOWN);

    bool init(
        const SnortConfig*, PktType, IpProtocol,
        const snort::SfIp *srcIP, uint16_t srcPort,
        const snort::SfIp *dstIP, uint16_t dstPort,
        uint16_t vlanId, uint32_t mplsId, const DAQ_PktHdr_t&);

    bool init(
        const SnortConfig*, PktType, IpProtocol,
        const snort::SfIp *srcIP, const snort::SfIp *dstIP,
        uint32_t id, uint16_t vlanId, uint32_t mplsId, const DAQ_PktHdr_t&);

    void init_mpls(const SnortConfig*, uint32_t);
    void init_vlan(const SnortConfig*, uint16_t);
    void init_address_space(const SnortConfig*, uint32_t);
    void init_groups(int16_t, int16_t, bool);

    static bool is_equal(const FlowKey* k1, const FlowKey* k2)
    {
        return 0 == memcmp(k1, k2, sizeof(FlowKey));
    }


private:
    bool init4(IpProtocol, const snort::SfIp *srcIP, uint16_t srcPort,
        const snort::SfIp *dstIP, uint16_t dstPort, bool order = true);

    bool init6(IpProtocol, const snort::SfIp *srcIP, uint16_t srcPort,
        const snort::SfIp *dstIP, uint16_t dstPort, bool order = true);
};
PADDING_GUARD_END

}

#endif

flow_stash.h

Path = src/flow/flow_stash.h

#ifndef FLOW_STASH_H
#define FLOW_STASH_H

#include <list>
#include <map>
#include <string>
#include <unordered_map>

#include "main/snort_config.h"
#include "main/snort_types.h"
#include "sfip/sf_ip.h"

#include "stash_item.h"

namespace snort
{

class SO_PUBLIC FlowStash
{
public:
    ~FlowStash();
    void reset();

    bool get(const std::string& key, int32_t& val);
    bool get(const std::string& key, uint32_t& val);
    bool get(const std::string& key, std::string& val);
    bool get(const std::string& key, StashGenericObject* &val);

    void store(const std::string& key, int32_t val, unsigned pubid = 0, unsigned evid = 0);
    void store(const std::string& key, uint32_t val, unsigned pubid = 0, unsigned evid = 0);
    void store(const std::string& key, const std::string& val, unsigned pubid = 0, unsigned evid = 0);
    void store(const std::string& key, std::string* val, unsigned pubid = 0, unsigned evid = 0);
    void store(const std::string& key, StashGenericObject* val, unsigned pubid = 0, unsigned evid = 0);

    bool store(const snort::SfIp&, const SnortConfig* sc = nullptr);

    std::list<snort::SfIp>& get_aux_ip_list()
    { return aux_ip_fifo; }

private:
    std::list<snort::SfIp> aux_ip_fifo;
    std::unordered_map<std::string, StashItem*> container;

    template<typename T>
    bool get(const std::string& key, T& val, StashItemType type);
    template<typename T>
    void store(const std::string& key, T& val, StashItemType type, unsigned = 0, unsigned = 0);
    void store(const std::string& key, StashGenericObject* &val, StashItemType type, unsigned, unsigned);
};

}

#endif

flow_uni_list.h

Path = src/flow/flow_uni_list.h

#ifndef FLOW_UNI_LIST_H
#define FLOW_UNI_LIST_H

#include "flow.h"

class FlowUniList
{
public:

    FlowUniList()
    {
        head = new snort::Flow;
        tail = new snort::Flow;
        head->next = tail;
        tail->prev = head;
        head->prev = nullptr;
        tail->next = nullptr;
    }

    ~FlowUniList()
    {
        delete head;
        delete tail;
    }

    void link_uni(snort::Flow* flow)
    {
        flow->next = head->next;
        flow->prev = head;
        head->next->prev = flow;
        head->next = flow;
        ++count;
    }

    bool unlink_uni(snort::Flow* flow)
    {
        // FIXIT-H: Checking flow->prev is a defensive fix but doesn't resolve the
        // root cause of how next and prev could be different.
        if ( !flow->next or !flow->prev )
            return false;

        flow->next->prev = flow->prev;
        flow->prev->next = flow->next;
        flow->next = flow->prev = nullptr;
        --count;
        return true;
    }

    snort::Flow* get_oldest_uni()
    {
        return ( tail->prev != head ) ? tail->prev : nullptr;
    }

    snort::Flow* get_prev(snort::Flow* flow)
    {
        return ( flow->prev != head ) ? flow->prev : nullptr;
    }

    unsigned get_count() const
    { return count; }

private:
    snort::Flow* head = nullptr;
    snort::Flow* tail = nullptr;
    unsigned count = 0;

};

#endif

ha.h

Path = src/flow/ha.h

#ifndef HA_H
#define HA_H

#include <daq_common.h>

#include <cassert>

#include "framework/bits.h"
#include "main/thread.h"

//-------------------------------------------------------------------------

struct HighAvailabilityConfig;

namespace snort
{
class Flow;
struct FlowKey;
struct Packet;
struct ProfileStats;

// The FlowHAHandle is the dynamically allocated index used uniquely identify
//   the client.  Used both in the API and HA messages.
// Handle 0 is defined to be the primary session client.
// NOTE: The type, masks, and count values must be in sync,
typedef uint16_t FlowHAClientHandle;
constexpr FlowHAClientHandle ALL_CLIENTS = 0xffff;

// Each active flow will have an associated FlowHAState instance.
class SO_PUBLIC FlowHAState
{
public:
    static constexpr uint8_t CRITICAL = 0x80;
    static constexpr uint8_t MAJOR = 0x40;

    enum : uint8_t
    {
        NEW = 0x01,
        MODIFIED = 0x02,
        DELETED = 0x04,
        STANDBY = 0x08,
        NEW_SESSION = 0x10,
    };

    FlowHAState();

    void set_pending(FlowHAClientHandle);
    void clear_pending(FlowHAClientHandle);
    bool check_pending(FlowHAClientHandle);
    void set(uint8_t state);
    void add(uint8_t state);
    void clear(uint8_t state);
    bool check_any(uint8_t state);
    static void config_timers(struct timeval, struct timeval);
    bool sync_interval_elapsed();
    void init_next_update();
    void set_next_update();
    void reset();

private:
    static constexpr uint8_t INITIAL_STATE = 0x00;
    static constexpr uint16_t NONE_PENDING = 0x0000;
    static struct timeval min_session_lifetime;
    static struct timeval min_sync_interval;

    struct timeval next_update;
    uint16_t pending = NONE_PENDING;
    uint8_t state = NEW | NEW_SESSION;
};

// Describe the message being produced or consumed.
class HAMessage
{
public:
    HAMessage(uint8_t* buffer, uint32_t buffer_length) :
        buffer(buffer), buffer_length(buffer_length), cursor(buffer) { }

    bool fits(uint32_t size) const { return size <= (buffer_length - (cursor - buffer)); }

    void advance_cursor(uint32_t size)
    {
        assert(fits(size));
        cursor += size;
    }

    void reset_cursor(uint8_t* pos = nullptr)
    {
        if (pos)
        {
            assert(pos >= buffer && pos <= (buffer + buffer_length));
            cursor = pos;
        }
        else
            cursor = buffer;
    }

    uint32_t cursor_position() const { return (uint32_t) (cursor - buffer); }

    uint8_t* buffer;
    const uint32_t buffer_length;
    uint8_t* cursor;
};

// A FlowHAClient subclass for each producer/consumer of flow HA data
class SO_PUBLIC FlowHAClient
{
public:
    virtual ~FlowHAClient() = default;
    virtual bool consume(snort::Flow*&, const snort::FlowKey*, snort::HAMessage&, uint8_t size) = 0;
    virtual bool produce(snort::Flow&, snort::HAMessage&) = 0;
    virtual bool is_update_required(snort::Flow*) { return false; }
    virtual uint8_t get_message_size(Flow&) { return max_length; }

    FlowHAClientHandle handle = 0;  // Actual handle for the instance
    uint8_t index = 0;
    uint8_t max_length;

protected:
    FlowHAClient(uint8_t length, bool session_client);
};

// Top level management of HighAvailability components.
class SO_PUBLIC HighAvailabilityManager
{
public:
    static void configure(HighAvailabilityConfig*);
    static void thread_init();
    static void thread_term_beginning(); // thread is about to be terminated
    static void thread_term();
    static void term();

    // true if we are configured and able to process
    static bool active();

    // Within packet processing, analyze the packet and flow for potential update messages
    static void process_update(snort::Flow*, snort::Packet*);

    // Anytime a flow is deleted, potentially generate a deletion message
    static void process_deletion(snort::Flow&);

    // Look for and dispatch receive messages.
    static void process_receive();
    static void set_modified(snort::Flow*);
    static bool in_standby(snort::Flow*);

    // Attempt to import HA data from the Packet
    static Flow* import(snort::Packet& p, snort::FlowKey& key);

private:
    static void reset_config();

    HighAvailabilityManager() = delete;
    static bool use_daq_channel;
    static PortBitSet* ports;
};
}

#endif

ha_module.h

Path = src/flow/ha_module.h

#ifndef HA_MODULE_H
#define HA_MODULE_H

#include <sys/time.h>

#include "framework/module.h"
#include "main/thread.h"

#define HA_NAME "high_availability"
#define HA_HELP "implement flow tracking high availability"

struct HighAvailabilityConfig
{
    ~HighAvailabilityConfig() { delete ports; }

    bool enabled;
    bool daq_channel;
    PortBitSet* ports = nullptr;
    struct timeval min_session_lifetime;
    struct timeval min_sync_interval;
};

class HighAvailabilityModule : public snort::Module
{
public:
    HighAvailabilityModule();
    ~HighAvailabilityModule() override;

    bool begin(const char*, int, snort::SnortConfig*) override;
    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;

    snort::ProfileStats* get_profile() const override;

    Usage get_usage() const override
    { return GLOBAL; }

private:
    HighAvailabilityConfig* config;
};

struct HAStats
{
    PegCount msgs_recv;
    PegCount update_msgs_recv;
    PegCount update_msgs_recv_no_flow;
    PegCount update_msgs_consumed;
    PegCount delete_msgs_consumed;
    PegCount daq_stores;
    PegCount daq_imports;
    PegCount key_mismatch;
    PegCount msg_version_mismatch;
    PegCount msg_length_mismatch;
    PegCount truncated_msgs;
    PegCount unknown_key_type;
    PegCount unknown_client_idx;
    PegCount client_consume_errors;
};

extern THREAD_LOCAL HAStats ha_stats;
extern THREAD_LOCAL snort::ProfileStats ha_perf_stats;

#endif

prune_stats.h

Path = src/flow/prune_stats.h

#ifndef PRUNE_STATS_H
#define PRUNE_STATS_H

#include <cstdint>
#include <type_traits>

#include "framework/counts.h"

enum class PruneReason : uint8_t
{
    EXCESS,
    UNI,
    MEMCAP,
    HA,
    STALE,
        IDLE_MAX_FLOWS,
        IDLE_PROTOCOL_TIMEOUT,
    NONE,
    MAX
};

struct ProtoPruneStats
{
    using proto_t = std::underlying_type_t<PktType>;
    PegCount proto_counts[static_cast<proto_t>(PktType::MAX)] { };

    PegCount get_total() const
    {
        PegCount total = 0;
        for ( proto_t i = 0; i < static_cast<proto_t>(PktType::MAX); ++i )
            total += proto_counts[i];

        return total;
    }

    PegCount& get(PktType type)
    { return proto_counts[static_cast<proto_t>(type)]; }

    const PegCount& get(PktType type) const
    { return proto_counts[static_cast<proto_t>(type)]; }

    void update(PktType type)
    { ++get(type); }
};

struct PruneStats
{
    using reason_t = std::underlying_type<PruneReason>::type;

    PegCount prunes[static_cast<reason_t>(PruneReason::MAX)] { };
    ProtoPruneStats protoPruneStats[static_cast<reason_t>(PruneReason::MAX)] { };

    PegCount get_total() const
    {
        PegCount total = 0;
        for ( reason_t i = 0; i < static_cast<reason_t>(PruneReason::NONE); ++i )
            total += prunes[i];

        return total;
    }

    PegCount& get(PruneReason reason)
    { return prunes[static_cast<reason_t>(reason)]; }

    const PegCount& get(PruneReason reason) const
    { return prunes[static_cast<reason_t>(reason)]; }

    void update(PruneReason reason, PktType type = PktType::NONE)
    {
        ++get(reason);
        protoPruneStats[static_cast<reason_t>(reason)].update(type);
    }

    PegCount& get_proto_prune_count(PruneReason reason, PktType type)
    { return protoPruneStats[static_cast<reason_t>(reason)].get(type); }

    const PegCount& get_proto_prune_count(PruneReason reason, PktType type) const
    { return protoPruneStats[static_cast<reason_t>(reason)].get(type); }

    PegCount get_proto_prune_count(PktType type) const
    {
        PegCount total = 0;
        for ( reason_t i = 0; i < static_cast<reason_t>(PruneReason::NONE); ++i )
            total += protoPruneStats[i].get(type);

        return total;
    }
};

enum class FlowDeleteState : uint8_t
{
    FREELIST,
    ALLOWED,
    OFFLOADED,
    BLOCKED,
    MAX
};

struct FlowDeleteStats
{
    using state_t = std::underlying_type<FlowDeleteState>::type;

    PegCount deletes[static_cast<state_t>(FlowDeleteState::MAX)] { };

    PegCount get_total() const
    {
        PegCount total = 0;
        for ( state_t i = 0; i < static_cast<state_t>(FlowDeleteState::MAX); ++i )
            total += deletes[i];

        return total;
    }
    PegCount& get(FlowDeleteState state)
    { return deletes[static_cast<state_t>(state)]; }

    const PegCount& get(FlowDeleteState state) const
    { return deletes[static_cast<state_t>(state)]; }

    void update(FlowDeleteState state)
    { ++get(state); }
};

#endif

session.h

Path = src/flow/session.h

#ifndef SESSION_H
#define SESSION_H

// Session is an abstract base class for the various protocol subclasses.
// the subclasses do the actual work of tracking, reassembly, etc.

#include <cassert>
#include "stream/stream.h"

namespace snort
{
class Flow;
struct Packet;
struct SfIp;
class StreamSplitter;
}

class Session
{
public:
    virtual ~Session() = default;

    virtual bool setup(snort::Packet*) { return true; }
    virtual void update_direction(char /*dir*/, const snort::SfIp*, uint16_t /*port*/)
    {
        // UDP is currently the only stream that implements this. Others should not be calling.
        assert(false);
    }
    virtual int process(snort::Packet*) { return 0; }

    virtual void restart(snort::Packet*) { }
    virtual void precheck(snort::Packet*) { }
    virtual void clear() = 0;
    virtual void cleanup(snort::Packet* = nullptr) { clear(); }

    virtual bool add_alert(snort::Packet*, uint32_t /*gid*/, uint32_t /*sid*/) { return false; }
    virtual bool check_alerted(snort::Packet*, uint32_t /*gid*/, uint32_t /*sid*/) { return false; }

    virtual int update_alert(
        snort::Packet*, uint32_t /*gid*/, uint32_t /*sid*/,
        uint32_t /*event_id*/, uint32_t /*event_second*/) { return 0; }

    virtual void flush() { }
    virtual void flush_client(snort::Packet*) { }
    virtual void flush_server(snort::Packet*) { }
    virtual void flush_talker(snort::Packet*, bool /*final_flush */ = false) { }
    virtual void flush_listener(snort::Packet*, bool /*final_flush */ = false) { }

    virtual void set_splitter(bool /*c2s*/, snort::StreamSplitter*) { assert(false); }
    virtual snort::StreamSplitter* get_splitter(bool /*c2s*/) { return nullptr; }

    virtual void set_extra_data(snort::Packet*, uint32_t /*flag*/) { }

    virtual bool is_sequenced(uint8_t /*dir*/) const { return true; }
    virtual bool are_packets_missing(uint8_t /*dir*/) const { return false; }
    virtual bool are_client_segments_queued() const { return false; }

    virtual void disable_reassembly(snort::Flow*) { }
    virtual uint8_t get_reassembly_direction() const { return SSN_DIR_NONE; }
    virtual uint8_t missing_in_reassembled(uint8_t /*dir*/) const { return SSN_MISSING_NONE; }

    virtual bool set_packet_action_to_hold(snort::Packet*) { return false; }

protected:
    Session(snort::Flow* f) { flow = f; }

public:
    snort::Flow* flow;  // FIXIT-L use reference?
};

/* These should be tracked by all Session subclasses. Add to top of peg list.
 * Having these predefined stats improves consistency and provides convenience.
 */
#define SESSION_PEGS(module) \
    { CountType::SUM, "sessions", "total " module " sessions" }, \
    { CountType::MAX, "max", "max " module " sessions" }, \
    { CountType::SUM, "created", module " session trackers created" }, \
    { CountType::SUM, "released", module " session trackers released" }, \
    { CountType::SUM, "timeouts", module " session timeouts" }, \
    { CountType::SUM, "prunes", module " session prunes" }

// See above. Add to end of stats array.
#define SESSION_STATS \
    PegCount sessions; \
    PegCount max; \
    PegCount created; \
    PegCount released; \
    PegCount timeouts; \
    PegCount prunes

// Do not change the semantics of max. Max = the highest seen during the perf interval.
// To obtain max over the entire run, determine the maximum of reported max pegs.
#define SESSION_STATS_ADD(stats) \
    { \
        (stats).sessions++; \
        (stats).created++; \
        if ( (stats).max < (stats).sessions ) \
            (stats).max = (stats).sessions; \
    }

#endif

stash_item.h

Path = src/flow/stash_item.h

#ifndef STASH_ITEM_H
#define STASH_ITEM_H

#include <cstdint>
#include <string>

#define STASH_APPID_DATA "appid_data"

#define STASH_GENERIC_OBJECT_APPID 1
#define STASH_GENERIC_OBJECT_MIME 2

namespace snort
{

class StashGenericObject
{
public:
    StashGenericObject(int type) : object_type(type)
    { }

    virtual ~StashGenericObject() = default;

    int get_object_type() const
    { return object_type; }

private:
    int object_type;
};

enum StashItemType
{
    STASH_ITEM_TYPE_INT32,
    STASH_ITEM_TYPE_UINT32,
    STASH_ITEM_TYPE_STRING,
    STASH_ITEM_TYPE_GENERIC_OBJECT
};

union StashItemVal
{
    int32_t int32_val;
    uint32_t uint32_val;
    std::string* str_val;
    StashGenericObject* generic_obj_val;
};

class StashItem
{
public:
    StashItem(int32_t int32_val)
    {
        type = STASH_ITEM_TYPE_INT32;
        val.int32_val = int32_val;
    }

    StashItem(uint32_t uint32_val)
    {
        type = STASH_ITEM_TYPE_UINT32;
        val.uint32_val = uint32_val;
    }

    StashItem(const std::string& str_val)
    {
        type = STASH_ITEM_TYPE_STRING;
        val.str_val = new std::string(str_val);
    }

    StashItem(std::string* str_val)
    {
        type = STASH_ITEM_TYPE_STRING;
        val.str_val = str_val;
    }

    StashItem(StashGenericObject* obj)
    {
        type = STASH_ITEM_TYPE_GENERIC_OBJECT;
        val.generic_obj_val = obj;
    }

    ~StashItem()
    {
        switch (type)
        {
        case STASH_ITEM_TYPE_STRING:
            delete val.str_val;
            break;
        case STASH_ITEM_TYPE_GENERIC_OBJECT:
            delete val.generic_obj_val;
        default:
            break;
        }
    }

    StashItemType get_type() const
    { return type; }

    void get_val(int32_t& int32_val) const
    { int32_val = val.int32_val; }

    void get_val(uint32_t& uint32_val) const
    { uint32_val = val.uint32_val; }

    void get_val(std::string& str_val) const
    { str_val = *(val.str_val); }

    void get_val(StashGenericObject* &obj_val) const
    { obj_val = val.generic_obj_val; }

private:
    StashItemType type;
    StashItemVal val;
};

}

#endif

flow/test/

flow_stubs.h

Path = src/flow/test/flow_stubs.h

#ifndef FLOW_STUBS_H
#define FLOW_STUBS_H

#include "framework/data_bus.h"
#include "log/messages.h"
#include "main/policy.h"
#include "main/snort_config.h"
#include "main/thread_config.h"
#include "packet_tracer/packet_tracer.h"
#include "protocols/layer.h"
#include "protocols/packet.h"
#include "stream/stream.h"

namespace snort
{
void ErrorMessage(const char*,...) { }
void LogMessage(const char*,...) { }

void DataBus::publish(unsigned, unsigned, DataEvent&, Flow*) { }
void DataBus::publish(unsigned, unsigned, const uint8_t*, unsigned, Flow*) { }
void DataBus::publish(unsigned, unsigned, Packet*, Flow*) { }

Packet::Packet(bool)
{
    memset((char*) this , 0, sizeof(*this));
    ip_proto_next = IpProtocol::PROTO_NOT_SET;
    packet_flags = PKT_FROM_CLIENT;
}
Packet::~Packet()  = default;
uint32_t Packet::get_flow_geneve_vni() const { return 0; }

THREAD_LOCAL PacketTracer* s_pkt_trace = nullptr;

PacketTracer::~PacketTracer() = default;
void PacketTracer::log(const char*, ...) { }
void PacketTracer::open_file() { }
void PacketTracer::dump_to_daq(Packet*) { }
void PacketTracer::reset(bool) { }
void PacketTracer::pause() { }
void PacketTracer::unpause() { }

namespace layer
{
const vlan::VlanTagHdr* get_vlan_layer(const Packet* const) { return nullptr; }
}

void Stream::drop_traffic(const Packet*, char) { }
bool Stream::blocked_flow(Packet*) { return true; }
void Stream::stop_inspection(Flow*, Packet*, char, int32_t, int) { }

NetworkPolicy* get_network_policy() { return nullptr; }
InspectionPolicy* get_inspection_policy() { return nullptr; }
IpsPolicy* get_ips_policy() { return nullptr; }
void set_network_policy(NetworkPolicy*) { }
void set_inspection_policy(InspectionPolicy*) { }
void set_ips_policy(IpsPolicy*) { }
unsigned SnortConfig::get_thread_reload_id() { return 0; }
}

void set_network_policy(unsigned) { }
void set_inspection_policy(unsigned) { }
void set_ips_policy(const snort::SnortConfig*, unsigned) { }
void select_default_policy(const _daq_pkt_hdr&, const snort::SnortConfig*) { }

#endif

framework/

This directory provides base classes and support for the various Snort++ plugins. Module provides an abstracted interface to the common plugin features such as configuration. Module corresponds to a top-level Lua table of the same name.

Note that plugins must keep configuration (parse time) separate from state (run time). A plugin must store its state on the flow in FlowData or in thread local data. In some cases thread local data is handled by an array attached to configuration with one element per packet thread, however those cases are rare and should only be needed by the framework code, not the plugins.

PacketConstraints allow you to match packets and flows against a 6-tuple. ( ip_proto; src_ip; dst_ip; src_port; dst_port; tenant)

base_api.h

Path = src/framework/base_api.h

#ifndef BASE_API_H
#define BASE_API_H

// BaseApi is the struct at the front of every plugin api and provides the
// data necessary for common management of plugins.  in addition to basic
// usage fields, it provides module instantiation and release functions, as
// well as additional data to help detect mismatched builds etc.

#include <cstdint>

// this is the current version of the base api
// must be prefixed to subtype version
#define BASE_API_VERSION 16

// set options to API_OPTIONS to ensure compatibility
#ifndef API_OPTIONS
#include "framework/api_options.h"
#endif

// set the reserved field to this to be future proof
#define API_RESERVED 0

enum PlugType
{
    PT_CODEC,
    PT_INSPECTOR,
    PT_IPS_ACTION,
    PT_IPS_OPTION,
    PT_SEARCH_ENGINE,
    PT_SO_RULE,
    PT_LOGGER,
    PT_CONNECTOR,
    PT_POLICY_SELECTOR,
    PT_MAX
};

namespace snort
{
class Module;
typedef Module* (* ModNewFunc)();
typedef void (* ModDelFunc)(Module*);

// if we inherit this we can't use a static initializer list :(
// so BaseApi must be the prefix (ie 1st member) of all plugin api
struct BaseApi
{
    PlugType type;
    uint32_t size;
    uint32_t api_version;
    uint32_t version;
    uint64_t reserved;
    const char* options;
    const char* name;
    const char* help;
    snort::ModNewFunc mod_ctor;
    ModDelFunc mod_dtor;
};
}
#endif

bits.h

Path = src/framework/bits.h

#ifndef BITS_H
#define BITS_H

// common types used throughout the code

#include <bitset>

typedef std::bitset<65536> PortBitSet;
typedef std::bitset<4096> VlanBitSet;
typedef std::bitset<256> ByteBitSet;

#endif

codec.h

Path = src/framework/codec.h

#ifndef FRAMEWORK_CODEC_H
#define FRAMEWORK_CODEC_H

// Codec is a type of plugin that provides protocol-specific encoding and
// decoding.

#include <cstdint>
#include <vector>

#include "framework/base_api.h"
#include "framework/decode_data.h"
#include "protocols/layer.h"
#include "utils/cpp_macros.h"

struct TextLog;
struct _daq_msg;

namespace snort
{
enum CodecSid : uint32_t;

namespace ip
{
class IpApi;
}
namespace tcp
{
struct TCPHdr;
}
namespace udp
{
struct UDPHdr;
}
namespace icmp
{
struct ICMPHdr;
}

class Flow;

// Used by root codecs to add their DLT to their HELP string
#define ADD_DLT(help, x) help " (DLT " STRINGIFY_MX(x) ")"

constexpr uint8_t MIN_TTL = 64;
constexpr uint8_t MAX_TTL = 255;

struct RawData
{
    const struct _daq_msg* daq_msg;
    const uint8_t* data;
    uint32_t len;

    RawData(const struct _daq_msg* daq_msg, const uint8_t* data, uint32_t len) :
        daq_msg(daq_msg), data(data), len(len) { }
};

/*  Decode Flags */
constexpr uint16_t CODEC_DF = 0x0001;    /* don't fragment flag */

// packet may have incorrect encapsulation layer.  don't alert if "next
// layer" is invalid.  If decode fails with this bit set, PacketManager
// will back out to the previous layer.  IMPORTANT:  This bit can ONLY be
// set if the DECODE_ENCAP_LAYER flag was was previously set.
constexpr uint16_t CODEC_UNSURE_ENCAP = 0x0002;

// DO NOT USE THIS LAYER!!  --  use DECODE_ENCAP_LAYER
constexpr uint16_t CODEC_SAVE_LAYER = 0x0004;

// If encapsulation decode fails, back out to this layer This will be
// cleared by PacketManager between decodes This flag automatically sets
// DECODE_ENCAP_LAYER for the next layer (and only the next layer).
constexpr uint16_t CODEC_ENCAP_LAYER = (CODEC_SAVE_LAYER | CODEC_UNSURE_ENCAP );

// used to check ip6 extension order
constexpr uint16_t CODEC_ROUTING_SEEN = 0x0008;

// used by icmp4 for alerting
constexpr uint16_t CODEC_IPOPT_RR_SEEN = 0x0010;

// used by IGMP for alerting
constexpr uint16_t CODEC_IPOPT_RTRALT_SEEN = 0x0020;

// used by IGMP for alerting
constexpr uint16_t CODEC_IPOPT_LEN_THREE = 0x0040;

// used in IPv6 Codec
constexpr uint16_t CODEC_TEREDO_SEEN = 0x0080;

constexpr uint16_t CODEC_STREAM_REBUILT = 0x0100;
constexpr uint16_t CODEC_NON_IP_TUNNEL = 0x0200;

constexpr uint16_t CODEC_IP6_EXT_OOO = 0x0400;
constexpr uint16_t CODEC_IP6_BAD_OPT = 0x0800;

constexpr uint16_t CODEC_ETHER_NEXT = 0x1000;
constexpr uint16_t CODEC_COMPOUND = 0x2000;
constexpr uint16_t CODEC_LAYERS_EXCEEDED = 0x400;

constexpr uint16_t CODEC_IPOPT_FLAGS = (CODEC_IPOPT_RR_SEEN |
    CODEC_IPOPT_RTRALT_SEEN | CODEC_IPOPT_LEN_THREE);

struct SnortConfig;

constexpr uint8_t COMPOUND_LAYERS_MAX = 8;

struct CompoundLayer
{
    Layer layer;
    uint32_t proto_bits;
};

struct CodecData
{
    const SnortConfig* conf;

    /* This section will get reset before every decode() function call */
    ProtocolId next_prot_id;        /* protocol type of the next layer */
    uint16_t lyr_len = 0;           /* length of the valid portion of this layer */
    uint16_t invalid_bytes = 0;     /* length of the INVALID part of this layer */
    uint32_t proto_bits = 0;        /* protocols contained within this packet
                                        -- will be propogated to Snort++ Packet struct*/

    /* Reset before each decode of packet begins */

    /*  Codec specific fields.  These fields are only relevant to codecs. */
    uint16_t codec_flags = 0;   /* flags used while decoding */
    uint8_t ip_layer_cnt = 0;

    uint8_t ip6_extension_count = 0;
    uint8_t curr_ip6_extension = 0;
    IpProtocol ip6_csum_proto = IpProtocol::IP;   /* Used for IPv6 checksums */
    bool tunnel_bypass = false;

    CompoundLayer compound_layers[COMPOUND_LAYERS_MAX]{};
    uint8_t compound_layer_cnt = 0;

    CodecData(const SnortConfig* sc, ProtocolId init_prot) : conf(sc), next_prot_id(init_prot)
    { }

    bool inline is_cooked() const
    { return codec_flags & CODEC_STREAM_REBUILT; }
};

typedef uint64_t EncodeFlags;
constexpr EncodeFlags ENC_FLAG_FWD = 0x8000000000000000;  // send in forward direction
constexpr EncodeFlags ENC_FLAG_SEQ = 0x4000000000000000;  // VAL bits contain seq adj
constexpr EncodeFlags ENC_FLAG_ID  = 0x2000000000000000;  // use randomized IP ID
constexpr EncodeFlags ENC_FLAG_NET = 0x1000000000000000;  // stop after innermost network (ip4/6)
                                                          // layer
constexpr EncodeFlags ENC_FLAG_DEF = 0x0800000000000000;  // stop before innermost ip4 opts or ip6
                                                          // frag header
constexpr EncodeFlags ENC_FLAG_RAW = 0x0400000000000000;  // don't encode outer eth header (this is
                                                          // raw ip)
constexpr EncodeFlags ENC_FLAG_PAY = 0x0200000000000000;  // set to when a TCP payload is attached
constexpr EncodeFlags ENC_FLAG_PSH = 0x0100000000000000;  // set by PacketManager when TCP should
                                                          // set PUSH flag
constexpr EncodeFlags ENC_FLAG_FIN = 0x0080000000000000;  // set by PacketManager when TCP should
                                                          // set FIN flag
constexpr EncodeFlags ENC_FLAG_TTL = 0x0040000000000000;  // set by PacketManager when TCP should
                                                          // set FIN flag
constexpr EncodeFlags ENC_FLAG_INLINE = 0x0020000000000000;  // set by PacketManager when TCP
                                                             // should set FIN flag
constexpr EncodeFlags ENC_FLAG_RST_CLNT = 0x0010000000000000;  // finish with a client RST packet
constexpr EncodeFlags ENC_FLAG_RST_SRVR = 0x0008000000000000;  // finish with a server RST packet
constexpr EncodeFlags ENC_FLAG_VAL = 0x00000000FFFFFFFF;  // bits for adjusting seq and/or ack

constexpr uint8_t ENC_PROTO_UNSET = 0xFF;

struct SO_PUBLIC EncState
{
    const ip::IpApi& ip_api; /* IP related information. Good for checksums */
    EncodeFlags flags;
    const uint16_t dsize; /* for non-inline, TCP sequence numbers */
    ProtocolId next_ethertype; /*  set the next encoder 'proto' field to this value. */
    IpProtocol next_proto; /*  set the next encoder 'proto' field to this value. */
    const uint8_t ttl;

    EncState(const ip::IpApi& api, EncodeFlags f, IpProtocol pr,
        uint8_t t, uint16_t data_size);

    inline bool next_proto_set() const
    { return (next_proto != IpProtocol::PROTO_NOT_SET); }

    inline bool ethertype_set() const
    { return next_ethertype != ProtocolId::ETHERTYPE_NOT_SET; }

    inline bool forward() const
    { return flags & ENC_FLAG_FWD; }

    uint8_t get_ttl(uint8_t lyr_ttl) const;
};

struct SO_PUBLIC Buffer
{
public:
    Buffer(uint8_t* buf, uint32_t size);

    inline uint8_t* data() const
    { return base; }

    uint32_t size() const
    { return end; }

    inline bool allocate(uint32_t len)
    {
        if ( (end + len) > max_len )
            return false;

        end += len;
        base -= len;
        return true;
    }

    inline void clear()
    { base = base + end; end = 0; off = 0; }

private:
    uint8_t* base; /* start of data */
    uint32_t end;       /* end of data */
    const uint32_t max_len;   /* size of allocation */

public:
    uint32_t off;       /* offset into data */
};

typedef uint8_t UpdateFlags;
constexpr UpdateFlags UPD_COOKED = 0x01;
constexpr UpdateFlags UPD_MODIFIED = 0x02;
constexpr UpdateFlags UPD_RESIZED = 0x04;
constexpr UpdateFlags UPD_REBUILT_FRAG = 0x08;

/*  Codec Class */

class SO_PUBLIC Codec
{
public:
    virtual ~Codec() = default;

    // PKT_MAX = ETHERNET_HEADER_LEN + VLAN_HEADER + ETHERNET_MTU + IP_MAXPACKET

    /* PKT_MAX is sized to ensure that any reassembled packet
     * can accommodate a full datagram at innermost layer
     *
     * ETHERNET_HEADER_LEN == 14
     * VLAN_HEADER == 4
     * ETHERNET_MTU == 1500
     * IP_MAXPACKET ==  65535  FIXIT-L use Packet::max_dsize
     */
    static const uint32_t PKT_MAX = 14 + 4 + 1500 + 65535;

    /*  Codec Initialization */

    // Get the codec's name
    inline const char* get_name() const
    { return name; }
    // Registers this Codec's data link type (as defined by libpcap)
    virtual void get_data_link_type(std::vector<int>&) // FIXIT-M return a vector == efficient in
                                                       // c++11
    { }
    // Register the code's protocol ID's and Ethertypes
    virtual void get_protocol_ids(std::vector<ProtocolId>&)  // FIXIT-M return a vector ==
                                                           // efficient in c++11
    { }

    /*
     * Main decoding function!  Will get called when decoding a packet.
     *
     * PARAMS:
     *      const RawData& = struct containing information about the
     *                      current packet's raw data
     *
     *      CodecData& = Pass information the PacketManager and other
     *              codecs. IMPORTANT FIELDS TO SET IN YOUR CODEC -
     *         next_prot_id   = protocol type of the next layer
     *         lyr_len        = The length of the valid part layer
     *         invalid bytes  = number of invalid bytes between the end of
     *                          the this layer's valid length and the next
     *                           layer. For instance,
     *                          when decoding IP, if there are 20 bytes of
     *                          options but only 12 bytes are valid.
     *
     *                          data.lyr_len = MIN_IP_HEADER_LEN + 12;
     *                          data.invalid_bytes = 8    === 20 - 12
     *
     *      DecodeData& = Data which will be sent to the rest of Snort++.
     *                      contains convenience pointers and information
     *                      about this packet.
     **/
    virtual bool decode(const RawData&, CodecData&, DecodeData&)=0;

    /*
     *  Log this layer's information
     *  PARAMS:
     *          TextLog* = the logger. Defined in "text_log.h"
     *          const uint8_t *raw_pkt = the same data seen during decode
     *          const uint16_t lyr_len = This layer's validated length ==
     *                                   lyr_len set during decode.
     */
    virtual void log(TextLog* const, const uint8_t* /*raw_pkt*/, const uint16_t /*lyr_len*/)
    { }

    /*
     * Encoding -- active response!!
     *
     * Encode the current packet. Encoding starts with the innermost
     * layer and working outwards.  All encoders MUST call the
     * Buffer.allocate() function before writing to output buffer.
     * PARAMS:
     *        uint8_t* raw_in =  A pointer to the raw input which was decoded.
     *              This is the same pointer which was given to decode().
     *        uint16_t len == the value to which '(CodecData&).lyr_len' was set during decode.
     *              I.e., the validated length of this layer. Some protocols,
     *              like IPv4 (original ipv4 header may contain invalid options
     *              which we don't want to copy) and GTP have dynamic lengths.
     *              So, this parameter ensure the encode() function doesn't
     *              need to revalidate and recalculate the length.
     *        EncState& = The current EncState struct
     *        Buffer& = the packet which will be sent. All inward layers will already
     *              be set.
     *
     * NOTE:  all functions MUST call the Buffer.allocate() function before
     *          manipulating memory.
     */
    virtual bool encode(const uint8_t* const /*raw_in */,
        const uint16_t /*raw_len*/,
        EncState&,
        Buffer&, Flow*)
    { return true; }

    /*
     * Update this layers checksums and length fields.  Used
     * when rebuilding packets and Snort is in inline mode.
     *
     *  updated_len MUST be set to this layers updated length.
     */
    virtual void update(
        const ip::IpApi&,
        const EncodeFlags /*flags*/,
        uint8_t* /*raw_pkt*/,  /* The data associated with the current layer */
        uint16_t lyr_len,  /* This layers previously calculated length */
        uint32_t& updated_len) /* Update!  The length to end of packet */
    { updated_len += lyr_len; }

    // formatter
    virtual void format(bool /*reverse*/, uint8_t* /*raw_pkt*/, DecodeData&)
    { }

protected:
    Codec(const char* s)
    { name = s; }

    // Create an event with the Codec GID
    void codec_event(const CodecData &, CodecSid);
    // Check the Hop and DST IPv6 extension
    bool CheckIPV6HopOptions(const RawData&, CodecData&);
    // NOTE:: data.next_prot_id MUST be set before calling this!!
    void CheckIPv6ExtensionOrder(CodecData&, const IpProtocol);

private:
    const char* name;
};

//-------------------------------------------------------------------------
// api
//-------------------------------------------------------------------------

// this is the current version of the api
#define CDAPI_VERSION ((BASE_API_VERSION << 16) | 1)

typedef Codec* (* CdNewFunc)(Module*);
typedef void (* CdDelFunc)(Codec*);
typedef void (* CdAuxFunc)();

struct CodecApi
{
    BaseApi base;

    // these may be nullptr
    CdAuxFunc pinit;  // initialize global plugin data
    CdAuxFunc pterm;  // clean-up pinit()

    CdAuxFunc tinit;  // initialize thread-local plugin data
    CdAuxFunc tterm;  // clean-up tinit()

    // these must be set
    CdNewFunc ctor;   // get eval optional instance data
    CdDelFunc dtor;   // clean up instance data
};
}
#endif /* FRAMEWORK_CODEC_H */

connector.h

Path = src/framework/connector.h

#ifndef CONNECTOR_H
#define CONNECTOR_H

// Connector provides out-of-band communications among packet processing
// threads, high-availability partners, and other threads.

#include <string>
#include <vector>

#include "framework/base_api.h"
#include "main/snort_types.h"

namespace snort
{
// this is the current version of the api
#define CONNECTOR_API_VERSION ((BASE_API_VERSION << 16) | 0)

//-------------------------------------------------------------------------
// api for class
// ctor, dtor are in main thread
// other methods are packet thread specific
//-------------------------------------------------------------------------

struct ConnectorApi;
class ConnectorConfig;

struct ConnectorMsg
{
    uint32_t length;
    uint8_t* data;
};

class ConnectorMsgHandle
{
};

class SO_PUBLIC Connector
{
public:
    enum Direction
    {
        CONN_UNDEFINED,
        CONN_RECEIVE,
        CONN_TRANSMIT,
        CONN_DUPLEX
    };

    virtual ~Connector() = default;

    virtual ConnectorMsgHandle* alloc_message(const uint32_t, const uint8_t**) = 0;
    virtual void discard_message(ConnectorMsgHandle*) = 0;
    virtual bool transmit_message(ConnectorMsgHandle*) = 0;
    virtual ConnectorMsgHandle* receive_message(bool block) = 0;
    virtual ConnectorMsg* get_connector_msg(ConnectorMsgHandle*) = 0;
    virtual Direction get_connector_direction() = 0;

    const std::string connector_name;
    const ConnectorConfig* config = nullptr;

protected:
    Connector() = default;
};

class ConnectorConfig
{
public:
    typedef std::vector<ConnectorConfig*> ConfigSet;
    Connector::Direction direction;
    std::string connector_name;

    virtual ~ConnectorConfig() = default;
};

class SO_PUBLIC ConnectorCommon
{
public:
    ConnectorConfig::ConfigSet* config_set;
};

typedef ConnectorCommon* (* ConnectorNewFunc)(Module*);
typedef void (* ConnectorDelFunc)(ConnectorCommon*);
typedef Connector* (* ConnectorThreadInitFunc)(ConnectorConfig*);
typedef void (* ConnectorThreadTermFunc)(Connector*);
typedef void (* ConnectorFunc)();

struct ConnectorApi
{
    BaseApi base;
    unsigned flags;

    ConnectorFunc pinit;     // plugin init
    ConnectorFunc pterm;     // cleanup pinit()
    ConnectorThreadInitFunc tinit;     // thread local init
    ConnectorThreadTermFunc tterm;     // cleanup tinit()

    ConnectorNewFunc ctor;
    ConnectorDelFunc dtor;
};
}
#endif

counts.h

Path = src/framework/counts.h

#ifndef COUNTS_H
#define COUNTS_H

// basic stats support - note that where these are used, the number of
// elements in stats must be the same as the number of elements in the peg
// info (and in the same sequence).

#include "main/snort_types.h"

typedef uint64_t PegCount;

enum CountType
{
    END,   // sentinel value
    SUM,   // tracks cumulative total number of items seen (eg #events)
    NOW,   // gives snapshot of current number of items (eg current #sessions)
    MAX,   // tracks maximum value seen (eg max #sessions)
};

struct SimpleStats
{
    PegCount total_packets;
};

struct PegInfo
{
    CountType type;
    const char* name;
    const char* help;
};


namespace snort
{
SO_PUBLIC extern const struct PegInfo simple_pegs[];
} // namespace snort

#define array_size(a) (sizeof(a)/sizeof((a)[0]))

#endif

cursor.h

Path = src/framework/cursor.h

#ifndef CURSOR_H
#define CURSOR_H

// Cursor provides a formal way of using buffers when doing detection with
// IpsOptions.

#include <assert.h>
#include <cstdint>
#include <cstring>
#include <vector>

#include "main/snort_types.h"

namespace snort
{
struct Packet;
}

class CursorData
{
public:
    CursorData(unsigned u) : id(u) {}
    virtual ~CursorData() = default;
    virtual CursorData* clone() = 0;

    unsigned get_id() const
    { return id; }

    static unsigned create_cursor_data_id()
    { return ++cursor_data_id; }

private:
    static unsigned cursor_data_id;
    unsigned id;
};

class SO_PUBLIC Cursor
{
public:
    Cursor() = default;
    Cursor(snort::Packet*);
    Cursor(const Cursor&);

    Cursor& operator=(const Cursor&) = delete;

    ~Cursor()
    {
        if (!data)
            return;

        for (CursorData*& cd : *data)
            delete cd;

        delete data;
    }

    const char* get_name() const
    { return name; }

    bool is(const char* s) const
    { return !strcmp(name, s); }

    void reset(snort::Packet*);

    void set(const char* s, const uint8_t* b, unsigned n, bool ext = false)
    {
        name = s; buf = b; buf_size = n; current_pos = delta = 0;
        extensible = ext and n > 0;
        buf_id = 0;
    }

    void set(const char* s, const uint8_t* b, unsigned n, unsigned pos_file, bool ext = false)
    {
        file_pos = pos_file;
        name = s; buf = b; buf_size = n; current_pos = delta = 0;
        extensible = ext and n > 0;
        buf_id = 0;
    }

    void set(const char* s, uint64_t id, const uint8_t* b, unsigned n, bool ext = false)
    {
        set(s, b, n, ext);
        buf_id = id;
    }

    void set(const char* s, uint64_t id, const uint8_t* b, unsigned n, unsigned pos_file, bool ext = false)
    {
        set(s, b, n, pos_file, ext);
        buf_id = id;
    }

    uint64_t id() const
    { return buf_id; }

    const uint8_t* buffer() const
    { return buf; }

    unsigned size() const
    { return buf_size; }

    // the NEXT octect after last in buffer
    // (this pointer is out of bounds)
    const uint8_t* endo() const
    { return buf + buf_size; }

    const uint8_t* start() const
    { return buf + current_pos; }

    unsigned length() const
    { return buf_size - current_pos; }

    unsigned get_pos() const
    { return current_pos; }

    unsigned get_delta() const
    { return delta; }

    CursorData* get_data(unsigned id) const;

    bool add_pos(unsigned n)
    {
        current_pos += n;
        return !(current_pos > buf_size);
    }

    // current_pos and delta may go 1 byte after end
    bool set_pos(unsigned n)
    {
        current_pos = n;
        return !(current_pos > buf_size);
    }

    bool set_pos_file(unsigned n)
    {
        file_pos = n;
        return true;
    }

    bool set_accumulation(bool is_accum)
    {
        is_accumulated = is_accum;
        return true;
    }

    unsigned get_file_pos() const
    {
        return file_pos;
    }

    bool is_buffer_accumulated() const
    {
        return is_accumulated;
    }

    bool set_delta(unsigned n)
    {
        if (n > buf_size)
            return false;
        delta = n;
        return true;
    }

    void set_data(CursorData* cd);

    bool awaiting_data() const
    { return extensible and current_pos >= buf_size; }

    bool awaiting_data(bool force_ext) const
    { return force_ext and current_pos >= buf_size; }

    unsigned get_next_pos() const
    {
        assert(current_pos >= buf_size);
        return current_pos - buf_size;
    }

    typedef std::vector<CursorData*> CursorDataVec;

private:
    unsigned buf_size = 0;
    unsigned current_pos = 0;
    unsigned delta = 0;            // loop offset
    unsigned file_pos = 0;         // file pos
    const uint8_t* buf = nullptr;
    const char* name = nullptr;    // rule option name ("pkt_data", "http_uri", etc.)
    CursorDataVec* data = nullptr; // data stored on the cursor
    bool extensible = false;       // if the buffer could have more data in a continuation
    uint64_t buf_id = 0;           // source buffer ID
    bool is_accumulated = false;
};

#endif

data_bus.h

Path = src/framework/data_bus.h

#ifndef DATA_BUS_H
#define DATA_BUS_H

// DataEvents are the product of inspection, not detection.  They can be
// used to implement flexible processing w/o hardcoding the logic to call
// specific functions under specific conditions.  By using DataEvents with
// a publish-subscribe mechanism, it is possible to add custom processing
// at arbitrary points, eg when service is identified, or when a URI is
// available, or when a flow clears.

#include <string>
#include <unordered_map>
#include <unordered_set>
#include <vector>

#include "main/snort_types.h"

namespace snort
{
class Flow;
struct Packet;
struct SnortConfig;

struct PubKey
{
    const char* publisher;
    unsigned num_events;
};

class DataEvent
{
public:
    virtual ~DataEvent() = default;

    virtual const Packet* get_packet() const
    { return nullptr; }

    virtual const uint8_t* get_data()
    { return nullptr; }

    virtual const uint8_t* get_data(unsigned& len)
    { len = 0; return nullptr; }

    virtual const uint8_t* get_normalized_data(unsigned& len)
    { return get_data(len); }

protected:
    DataEvent() = default;
};

class BareDataEvent final : public DataEvent
{
public:
    BareDataEvent() = default;
    ~BareDataEvent() override = default;
};

class DataHandler
{
public:
    virtual ~DataHandler() = default;

    virtual void handle(DataEvent&, Flow*) { }
    const char* module_name;
    bool cloned;

    // order is desired position in the call sequence of handlers: 1 = first, 0 = last
    // the sequence among subscribers with the same order is not specified
    unsigned order = 0;

protected:
    DataHandler(std::nullptr_t) = delete;
    DataHandler(const char* mod_name) : module_name(mod_name), cloned(false) { }
};

class SO_PUBLIC DataBus
{
public:
    DataBus();
    ~DataBus();

    // configure time methods - main thread only
    static unsigned init();
    void clone(DataBus& from, const char* exclude_name = nullptr);

    // publishers must register their key and use given id to publish
    static unsigned get_id(const PubKey&);

    static bool valid(unsigned pub_id)
    { return pub_id != 0; }

    // FIXIT-L ideally these would not be static or would take an inspection policy*
    static void subscribe(const PubKey&, unsigned id, DataHandler*);
    static void subscribe_network(const PubKey&, unsigned id, DataHandler*);
    static void subscribe_global(const PubKey&, unsigned id, DataHandler*, SnortConfig&);

    // FIXIT-L these should be called during cleanup
    static void unsubscribe(const PubKey&, unsigned id, DataHandler*);
    static void unsubscribe_network(const PubKey&, unsigned id, DataHandler*);
    static void unsubscribe_global(const PubKey&, unsigned id, DataHandler*, SnortConfig&);

    // runtime methods
    static void publish(unsigned pub_id, unsigned evt_id, DataEvent&, Flow* = nullptr);

    // convenience methods
    static void publish(unsigned pub_id, unsigned evt_id, const uint8_t*, unsigned, Flow* = nullptr);
    static void publish(unsigned pub_id, unsigned evt_id, Packet*, Flow* = nullptr);

private:
    void _subscribe(unsigned pub_id, unsigned evt_id, DataHandler*);
    void _subscribe(const PubKey&, unsigned evt_id, DataHandler*);
    void _unsubscribe(const PubKey&, unsigned evt_id, const DataHandler*);
    void _publish(unsigned pub_id, unsigned evt_id, DataEvent&, Flow*) const;

private:
    typedef std::vector<DataHandler*> SubList;
    std::vector<SubList> pub_sub;
};
}

#endif

decode_data.h

Path = src/framework/decode_data.h

#ifndef FRAMEWORK_DECODE_DATA_H
#define FRAMEWORK_DECODE_DATA_H

// Captures decode information from Codecs.

#include "protocols/ip.h"
#include "protocols/mpls.h"

namespace snort
{
namespace tcp
{
struct TCPHdr;
}
namespace udp
{
struct UDPHdr;
}
namespace icmp
{
struct ICMPHdr;
}
}

/* NOTE: if A protocol is added, update DecodeFlags! */
enum class PktType : std::uint8_t
{
    NONE, IP, TCP, UDP, ICMP, USER, FILE, PDU, MAX
};

// the first several of these bits must map to PktType
// eg PROTO_BIT__IP == BIT(PktType::IP), etc.
#define PROTO_BIT__NONE             0x000000
#define PROTO_BIT__IP               0x000001
#define PROTO_BIT__TCP              0x000002
#define PROTO_BIT__UDP              0x000004
#define PROTO_BIT__ICMP             0x000008
#define PROTO_BIT__USER             0x000010
#define PROTO_BIT__FILE             0x000020
#define PROTO_BIT__PDU              0x000040
#define PROTO_BIT__TEREDO           0x000080
#define PROTO_BIT__GTP              0x000100
#define PROTO_BIT__MPLS             0x000200
#define PROTO_BIT__VLAN             0x000400
#define PROTO_BIT__ETH              0x000800
#define PROTO_BIT__TCP_EMBED_ICMP   0x001000
#define PROTO_BIT__UDP_EMBED_ICMP   0x002000
#define PROTO_BIT__ICMP_EMBED_ICMP  0x004000
#define PROTO_BIT__ICMP_EMBED_OTHER 0x008000
#define PROTO_BIT__IP6_EXT          0x010000
#define PROTO_BIT__CISCO_META_DATA  0x020000
#define PROTO_BIT__VXLAN            0x040000
#define PROTO_BIT__UDP_TUNNELED     0x080000
#define PROTO_BIT__OTHER            0x100000
#define PROTO_BIT__GENEVE           0x200000
#define PROTO_BIT__ARP              0x400000
#define PROTO_BIT__ALL              0x7FFFFF

#define PROTO_BIT__ICMP_EMBED \
    (PROTO_BIT__TCP_EMBED_ICMP | PROTO_BIT__UDP_EMBED_ICMP | \
    PROTO_BIT__ICMP_EMBED_ICMP | PROTO_BIT__ICMP_EMBED_OTHER)

#define PROTO_BIT__ANY_IP   (PROTO_BIT__IP | PROTO_BIT__TCP | PROTO_BIT__UDP | PROTO_BIT__ICMP)
#define PROTO_BIT__ANY_PDU  (PROTO_BIT__TCP | PROTO_BIT__UDP | PROTO_BIT__PDU)
#define PROTO_BIT__ANY_SSN  (PROTO_BIT__ANY_IP | PROTO_BIT__PDU | PROTO_BIT__FILE | PROTO_BIT__USER)
#define PROTO_BIT__ANY_TYPE (PROTO_BIT__ANY_SSN | PROTO_BIT__ARP)

enum DecodeFlags : std::uint16_t
{
    DECODE_ERR_CKSUM_IP =   0x0001,  // error flags
    DECODE_ERR_CKSUM_TCP =  0x0002,
    DECODE_ERR_CKSUM_UDP =  0x0004,
    DECODE_ERR_CKSUM_ICMP = 0x0008,
    DECODE_ERR_BAD_TTL =    0x0010,

    DECODE_ERR_CKSUM_ALL = ( DECODE_ERR_CKSUM_IP | DECODE_ERR_CKSUM_TCP |
        DECODE_ERR_CKSUM_UDP | DECODE_ERR_CKSUM_ICMP ),
    DECODE_ERR_FLAGS = ( DECODE_ERR_CKSUM_ALL | DECODE_ERR_BAD_TTL ),

    DECODE_PKT_TRUST =      0x0020,  // trust this packet
    DECODE_FRAG =           0x0040,  // ip - fragmented packet
    DECODE_MF =             0x0080,  // ip - more fragments
    DECODE_DF =             0x0100,  // ip - don't fragment

    // using decode flags in lieu of creating user layer for now
    DECODE_C2S =            0x0200,  // user - client to server
    DECODE_SOF =            0x0400,  // user - start of flow
    DECODE_EOF =            0x0800,  // user - end of flow
    DECODE_GTP =            0x1000,

    DECODE_TCP_MSS =        0x2000,
    DECODE_TCP_TS =         0x4000,
    DECODE_TCP_WS =         0x8000,
};

struct DecodeData
{
    /*
     * these three pointers are each referenced literally
     * dozens if not hundreds of times.  NOTHING else should be added!!
     */
    const snort::tcp::TCPHdr* tcph = nullptr;
    const snort::udp::UDPHdr* udph = nullptr;
    const snort::icmp::ICMPHdr* icmph = nullptr;

    uint16_t sp = 0;    /* source port (TCP/UDP) */
    uint16_t dp = 0;    /* dest port (TCP/UDP) */

    uint16_t decode_flags = 0;
    PktType type = PktType::NONE;

    snort::ip::IpApi ip_api;
    snort::mpls::MplsHdr mplsHdr = {};

    inline void reset()
    {
        tcph = nullptr;
        udph = nullptr;
        icmph = nullptr;
        sp = 0;
        dp = 0;
        decode_flags = 0;
        type = PktType::NONE;
        ip_api.reset();
        mplsHdr = {};
    }

    inline void set_pkt_type(PktType pkt_type)
    { type = pkt_type; }

    inline PktType get_pkt_type() const
    { return type; }
};

#endif

endianness.h

Path = src/framework/endianness.h

#ifndef ENDIANNESS_H
#define ENDIANNESS_H

#include "main/snort_types.h"

namespace snort
{
class SO_PUBLIC Endianness
{
public:
    Endianness() = default;
    virtual ~Endianness() = default;
    virtual bool get_offset_endianness(int32_t offset, uint8_t& endian) = 0;
};
}
#endif

file_policy.h

Path = src/framework/file_policy.h

#ifndef FILE_POLICY_H
#define FILE_POLICY_H

#include <map>
#include <vector>

#include "file_api/file_api.h"

namespace snort
{
class FileInfo;
}

struct FileVerdictWhen
{
    uint32_t type_id;
    std::string sha256;
};

struct FileVerdictUse
{
    FileVerdict verdict = FILE_VERDICT_UNKNOWN;
    bool type_enabled = false;
    bool signature_enabled = false;
    bool capture_enabled = false;
};

class FileRule
{
public:
    FileVerdictWhen when;
    FileVerdictUse use;

    FileRule();
    void clear();
};

class FilePolicy: public snort::FilePolicyBase
{
public:

    FilePolicy() = default;
    ~FilePolicy() override = default;

    void policy_check(snort::Flow*, snort::FileInfo*) override;

    // This is called after file type is known
    FileVerdict type_lookup(snort::Packet*, snort::FileInfo*) override;

    // This is called after file signature is complete
    FileVerdict signature_lookup(snort::Packet*, snort::FileInfo*) override;

    void add_file_id(FileRule&);
    void set_file_type(bool enabled);
    void set_file_signature(bool enabled);
    void set_file_capture(bool enabled);
    bool get_file_type() const;
    bool get_file_signature() const;
    bool get_file_capture() const;
    void load();
    void set_verdict_delay(int64_t delay) { verdict_delay = delay; }

private:
    FileRule& match_file_rule(snort::Flow*, snort::FileInfo*);
    FileVerdict match_file_signature(snort::Flow*, snort::FileInfo*);
    std::vector<FileRule> file_rules;
    std::map<std::string, FileVerdict> file_shas;
    bool type_enabled = false;
    bool signature_enabled = false;
    bool capture_enabled = false;
    int64_t verdict_delay = 0;
};

#endif

inspector.h

Path = src/framework/inspector.h

#ifndef INSPECTOR_H
#define INSPECTOR_H

// Inspectors are the workhorse that do all the heavy lifting between
// decoding a packet and detection.  There are several types that operate
// in different ways.  These correspond to Snort 2X preprocessors.

#include <atomic>
#include <cstring>
#include <memory>
#include <vector>

#include "framework/base_api.h"
#include "main/thread.h"
#include "target_based/snort_protocols.h"

class Session;

namespace snort
{
struct SnortConfig;
struct Packet;

// this is the current version of the api
#define INSAPI_VERSION ((BASE_API_VERSION << 16) | 0)

struct InspectionBuffer
{
    enum Type
    {
        // these are the only generic rule options
        IBT_VBA, IBT_JS_DATA,

        // FIXIT-M all of these should be eliminated
        IBT_KEY, IBT_HEADER, IBT_BODY,

        IBT_MAX
    };
    const uint8_t* data;
    unsigned len;
    bool is_accumulated = false;
};

struct InspectApi;

//-------------------------------------------------------------------------
// api for class
//-------------------------------------------------------------------------

class ThreadSpecificData;

class SO_PUBLIC Inspector
{
public:
    // main thread functions
    virtual ~Inspector();

    Inspector(const Inspector&) = delete;
    Inspector& operator=(const Inspector&) = delete;

    // access external dependencies here
    // return verification status
    virtual bool configure(SnortConfig*) { return true; }

    // cleanup for inspector instance removal from the running configuration
    // this is only called for inspectors in the default inspection policy that
    // were present in the prior snort configuration and were removed in the snort
    // configuration that is being loaded during a reload_config command
    virtual void tear_down(SnortConfig*) { }

    // called on controls after everything is configured
    // return true if there is nothing to do ever based on config
    virtual bool disable(SnortConfig*) { return false; }

    virtual void show(const SnortConfig*) const { }

    // Specific to Binders to notify them of an inspector being removed from the policy
    // FIXIT-L Probably shouldn't be part of the base Inspector class
    virtual void remove_inspector_binding(SnortConfig*, const char*) { }

    // packet thread functions
    // tinit, tterm called on default policy instance only
    virtual void tinit() { }   // allocate configurable thread local
    virtual void tterm() { }   // purge only, deallocate via api

    // screen incoming packets; only liked packets go to eval
    // default filter is per api proto / paf
    virtual bool likes(Packet*);

    // clear is a bookend to eval() for the active service inspector
    // clear is called when Snort is done with the previously eval'd
    // packet to release any thread-local or flow-based data
    virtual void eval(Packet*) = 0;
    virtual void clear(Packet*) { }

    // framework support
    unsigned get_ref(unsigned i) { return ref_count[i]; }
    void set_ref(unsigned i, unsigned r) { ref_count[i] = r; }

    void add_ref();
    void rem_ref();

    // Reference counts for the inspector that are not thread specific
    void add_global_ref();
    void rem_global_ref();

    bool is_inactive();

    void set_service(SnortProtocolId snort_protocol_id_param)
    { snort_protocol_id = snort_protocol_id_param; }

    SnortProtocolId get_service() const { return snort_protocol_id; }

    // for well known buffers
    // well known buffers may be included among generic below,
    // but they must be accessible from here
    virtual bool get_buf(InspectionBuffer::Type, Packet*, InspectionBuffer&)
    { return false; }

    // for generic buffers
    // key is listed in api buffers
    // id-1 is zero based index into buffers array
    unsigned get_buf_id(const char* key);
    virtual bool get_buf(const char* key, Packet*, InspectionBuffer&);
    virtual bool get_buf(unsigned /*id*/, Packet*, InspectionBuffer&)
    { return false; }

    virtual bool get_fp_buf(InspectionBuffer::Type ibt, Packet* p, InspectionBuffer& bf)
    { return get_buf(ibt, p, bf); }

    // IT_SERVICE only
    virtual class StreamSplitter* get_splitter(bool to_server);

    void set_api(const InspectApi* p)
    { api = p; }

    const InspectApi* get_api()
    { return api; }

    const char* get_name() const;

    void set_alias_name(const char* name)
    { alias_name = name; }

    const char* get_alias_name() const
    { return alias_name; }

    void set_network_policy_user_id(uint64_t user_id)
    {
        network_policy_user_id = user_id;
        network_policy_user_id_set = true;
    }

    bool get_network_policy_user_id(uint64_t& user_id) const
    {
        user_id = network_policy_user_id;
        return network_policy_user_id_set;
    }

    virtual bool is_control_channel() const
    { return false; }

    virtual bool can_carve_files() const
    { return false; }

    virtual bool can_start_tls() const
    { return false; }

    virtual bool supports_no_ips() const
    { return false; }

    void allocate_thread_storage();
    void set_thread_specific_data(void*);
    void* get_thread_specific_data() const;
    void copy_thread_storage(Inspector*);

    virtual void install_reload_handler(SnortConfig*)
    { }

    virtual const uint8_t* adjust_log_packet(Packet*, uint16_t&)
    { return nullptr; }

public:
    static THREAD_LOCAL unsigned slot;

protected:
    // main thread functions
    Inspector();  // internal init only at this point

private:
    const InspectApi* api = nullptr;
    std::shared_ptr<ThreadSpecificData> thread_specific_data;
    std::atomic_uint* ref_count;
    SnortProtocolId snort_protocol_id = 0;
    // FIXIT-E Use std::string to avoid storing a pointer to external std::string buffers
    const char* alias_name = nullptr;
    uint64_t network_policy_user_id = 0;
    bool network_policy_user_id_set = false;
};

// at present there is no sequencing among like types except that appid
// is always first among controls.

enum InspectorType
{
    IT_PASSIVE,  // config only, or data consumer (eg file_log, binder, ftp_client)
    IT_WIZARD,   // guesses service inspector
    IT_PACKET,   // processes raw packets only (eg normalize, capture)
    IT_STREAM,   // flow tracking and reassembly (eg ip, tcp, udp)
    IT_FIRST,    // analyze 1st pkt of new flow and 1st pkt after reload of ongoing flow (eg rep)
    IT_NETWORK,  // process packets w/o service (eg arp, bo)
    IT_SERVICE,  // extract and analyze service PDUs (eg dce, http, ssl)
    IT_CONTROL,  // process all packets before detection (eg appid)
    IT_PROBE,    // process all packets after detection (eg perf_monitor, port_scan)
    IT_FILE,     // file identification inspector
    IT_MAX
};

typedef Inspector* (* InspectNew)(Module*);
typedef void (* InspectDelFunc)(Inspector*);
typedef void (* InspectFunc)();
typedef Session* (* InspectSsnFunc)(class Flow*);

struct InspectApi
{
    BaseApi base;
    InspectorType type;
    uint32_t proto_bits;

    const char** buffers;  // null terminated list of exported buffers
    const char* service;   // nullptr when type != IT_SERVICE

    InspectFunc pinit;     // plugin init
    InspectFunc pterm;     // cleanup pinit()
    InspectFunc tinit;     // thread local init
    InspectFunc tterm;     // cleanup tinit()
    InspectNew ctor;       // instantiate inspector from Module data
    InspectDelFunc dtor;   // release inspector instance
    InspectSsnFunc ssn;    // get new session tracker
    InspectFunc reset;     // clear stats

    static const char* get_type(InspectorType type);
};

inline const char* Inspector::get_name() const
{ return api->base.name; }
}

#endif

ips_action.h

Path = src/framework/ips_action.h

#ifndef IPS_ACTION_H
#define IPS_ACTION_H

// IpsAction provides custom rule actions that are executed when a
// detection event is generated regardless of whether the event is logged.
// These can be used to execute external controls like updating an external
// firewall.

#include "framework/base_api.h"
#include "main/snort_types.h"
#include "packet_io/active_action.h"

// this is the current version of the api
#define ACTAPI_VERSION ((BASE_API_VERSION << 16) | 1)

//-------------------------------------------------------------------------
// api for class
//-------------------------------------------------------------------------

struct OptTreeNode;
namespace snort
{
struct Packet;

class SO_PUBLIC IpsAction
{
public:
    enum IpsActionPriority : uint16_t
    {
        IAP_OTHER = 1,
        IAP_LOG = 10,
        IAP_ALERT = 20,
        IAP_REWRITE = 30,
        IAP_DROP = 40,
        IAP_BLOCK = 50,
        IAP_REJECT = 60,
        IAP_PASS = 70,
        IAP_MAX = IAP_PASS
    };

public:
    virtual ~IpsAction() = default;
    const char* get_name() const { return name; }
    ActiveAction* get_active_action() const { return active_action; }

    virtual void exec(Packet*, const OptTreeNode* otn = nullptr) = 0;
    virtual bool drops_traffic() { return false; }

protected:
    IpsAction(const char* s, ActiveAction* a)
    {
        active_action = a;
        name = s;
    }

private:
    const char* name;
    ActiveAction* active_action;
};

typedef void (* IpsActFunc)();
typedef IpsAction* (* ActNewFunc)(class Module*);
typedef void (* ActDelFunc)(IpsAction*);

struct ActionApi
{
    BaseApi base;

    IpsAction::IpsActionPriority priority;

    IpsActFunc pinit;
    IpsActFunc pterm;
    IpsActFunc tinit;
    IpsActFunc tterm;

    ActNewFunc ctor;
    ActDelFunc dtor;
};
}
#endif

ips_option.h

Path = src/framework/ips_option.h

#ifndef IPS_OPTION_H
#define IPS_OPTION_H

// All IPS rule keywords are realized as IpsOptions instantiated when rules
// are parsed.

#include "detection/rule_option_types.h"
#include "framework/base_api.h"
#include "main/snort_types.h"
#include "target_based/snort_protocols.h"

#include "pdu_section.h"

//-------------------------------------------------------------------------
// api for class
// eval and action are packet thread specific
//-------------------------------------------------------------------------

class Cursor;
struct OptTreeNode;
struct PatternMatchData;

namespace snort
{
struct Packet;
struct SnortConfig;
class Module;

// this is the current version of the api
#define IPSAPI_VERSION ((BASE_API_VERSION << 16) | 0)

enum CursorActionType
{
    CAT_NONE,
    CAT_READ,
    CAT_ADJUST,
    CAT_SET_OTHER,
    CAT_SET_RAW,
    CAT_SET_FAST_PATTERN,
};

enum RuleDirection
{
    RULE_FROM_CLIENT,
    RULE_FROM_SERVER,
    RULE_WO_DIR
};

class SO_PUBLIC IpsOption
{
public:
    virtual ~IpsOption() = default;

    // main thread
    virtual uint32_t hash() const;
    virtual bool operator==(const IpsOption& ips) const;

    bool operator!=(const IpsOption& ips) const
    { return !(*this == ips); }

    virtual bool is_agent() { return false; }

    // packet threads
    virtual bool is_relative() { return false; }

    // 2nd cursor is deprecated, do not use
    virtual bool retry(Cursor&, const Cursor&) { return false; }
    virtual void action(Packet*) { }

    enum EvalStatus { NO_MATCH, MATCH, NO_ALERT, FAILED_BIT };
    virtual EvalStatus eval(Cursor&, Packet*) { return MATCH; }

    virtual CursorActionType get_cursor_type() const
    { return CAT_NONE; }

    // for fast-pattern options like content
    virtual PatternMatchData* get_pattern(SnortProtocolId, RuleDirection = RULE_WO_DIR)
    { return nullptr; }

    virtual PatternMatchData* get_alternate_pattern()
    { return nullptr; }

    option_type_t get_type() const { return type; }
    const char* get_name() const { return name; }

    bool is_buffer_setter() const
    { return get_cursor_type() > CAT_ADJUST; }

    const char* get_buffer()
    { return buffer; }

    virtual section_flags get_pdu_section(bool to_server) const;

protected:
    IpsOption(const char* s, option_type_t t = RULE_OPTION_TYPE_OTHER);

private:
    const char* name;
    const char* buffer = "error"; // FIXIT-API to be deleted; here to avoid an api update
    option_type_t type;
};

enum RuleOptType
{
    OPT_TYPE_LOGGING,
    OPT_TYPE_DETECTION,
    OPT_TYPE_META,
    OPT_TYPE_MAX
};

typedef void (* IpsOptFunc)(const SnortConfig*);

typedef IpsOption* (* IpsNewFunc)(Module*, OptTreeNode*);
typedef void (* IpsDelFunc)(IpsOption*);

struct IpsApi
{
    BaseApi base;
    RuleOptType type;

    unsigned max_per_rule;
    unsigned protos;

    IpsOptFunc pinit;
    IpsOptFunc pterm;
    IpsOptFunc tinit;
    IpsOptFunc tterm;
    IpsNewFunc ctor;
    IpsDelFunc dtor;
    IpsOptFunc verify;
};
}
#endif

logger.h

Path = src/framework/logger.h

#ifndef LOGGER_H
#define LOGGER_H

// Logger is used to log packets and events.  Events are thresholded before
// they reach the Logger.  Packets may be logged along with events or as a
// result of tagging.

#include "framework/base_api.h"
#include "main/snort_types.h"

struct Event;
namespace snort
{
struct Packet;

// this is the current version of the api
#define LOGAPI_VERSION ((BASE_API_VERSION << 16) | 0)

#define OUTPUT_TYPE_FLAG__NONE  0x0
#define OUTPUT_TYPE_FLAG__ALERT 0x1
#define OUTPUT_TYPE_FLAG__LOG   0x2

//-------------------------------------------------------------------------
// api for class
// ctor, dtor are in main thread
// other methods are packet thread specific
//-------------------------------------------------------------------------

struct LogApi;

class SO_PUBLIC Logger
{
public:
    virtual ~Logger() = default;

    virtual void open() { }
    virtual void close() { }
    virtual void reset() { }
    virtual void reload() { }

    virtual void alert(Packet*, const char*, const Event&) { }
    virtual void log(Packet*, const char*, Event*) { }

    void set_api(const LogApi* p)
    { api = p; }

    const LogApi* get_api()
    { return api; }

protected:
    Logger() = default;

private:
    const LogApi* api = nullptr;
};

typedef Logger* (* LogNewFunc)(class Module*);
typedef void (* LogDelFunc)(Logger*);

struct LogApi
{
    BaseApi base;
    unsigned flags;
    LogNewFunc ctor;
    LogDelFunc dtor;
};
}
#endif

lua_api.h

Path = src/framework/lua_api.h

#ifndef LUA_API_H
#define LUA_API_H

// LuaApi makes Lua scripts standard plugins

#include <string>

namespace snort
{
struct BaseApi;
}

class LuaApi
{
public:
    virtual ~LuaApi() = default;
    virtual const snort::BaseApi* get_base() const = 0;

    std::string name;
    std::string chunk;

protected:
    LuaApi(const std::string& s, const std::string& c) : name(s), chunk(c)
    { }
};

#endif

module.h

Path = src/framework/module.h

#ifndef MODULE_H
#define MODULE_H

// Module provides a data-driven way to manage much of Snort++.  For
// example, it provides an interface to configured components.  There is at
// most one instance of a Module at any time.  A given module instance is
// used to configure all related components.  As such it stores data only
// for the sake of constructing the next component instance.
//
// Module will set all parameter defaults immediately after calling
// begin() so defaults should not be explicitly set in begin() or a ctor
// called by begin, except as needed for infrastructure and sanity.
//
// Note that there are no internal default lists.  Put appropriate default
// lists in snort_defaults.lua or some such.  Each list item, however, will
// have any defaults applied.

#include <cassert>
#include <string>
#include <vector>

#include "framework/counts.h"
#include "framework/parameter.h"
#include "framework/value.h"
#include "main/snort_types.h"
#include "utils/stats.h"

struct lua_State;

namespace snort
{
class ModuleManager;
class Trace;
struct ProfileStats;
struct SnortConfig;
struct TraceOption;

using LuaCFunction = int(*)(lua_State*);

struct Command
{
    const char* name;
    LuaCFunction func;
    const Parameter* params;
    const char* help;
    // the flag determines if the command is allowed to run in parallel with other control commands
    bool can_run_in_parallel = false;

    std::string get_arg_list() const;
};

struct RuleMap
{
    unsigned sid;
    const char* msg;
};

class SO_PUBLIC Module
{
public:
    virtual ~Module() = default;

    // configuration:
    // for lists (tables with numeric indices):
    // int == 0 is list container
    // int > 0 is list item
    virtual bool begin(const char*, int, SnortConfig*)
    { return true; }

    virtual bool end(const char*, int, SnortConfig*)
    { return true; }

    virtual bool set(const char*, Value&, SnortConfig*)
    { return true; }

    virtual void set_trace(const Trace*) const { }

    virtual const TraceOption* get_trace_options() const
    { return nullptr; }

    // used to match parameters with $var names like <gid:sid> for rule_state
    virtual bool matches(const char* /*param_name*/, std::string& /*lua_name*/)
    { return false; }

    // ips events:
    virtual unsigned get_gid() const
    { return 0; }

    const char* get_name() const
    { return name ? name : params->name; }

    bool is_table() const
    { return (name != nullptr); }

    bool is_list() const
    { return list; }

    Parameter::Type get_type() const
    {
        if ( is_list() )
            return Parameter::PT_LIST;
        else if ( is_table() )
            return Parameter::PT_TABLE;
        else
            return params->type;
    }

    const char* get_help() const
    { return help; }

    const Parameter* get_parameters() const
    { return params; }

    virtual const Command* get_commands() const
    { return nullptr; }

    virtual const RuleMap* get_rules() const
    { return nullptr; }

    virtual const PegInfo* get_pegs() const
    { return nullptr; }

    virtual bool counts_need_prep() const
    { return false; }

    virtual void prep_counts(bool) { }

    // counts and profile are thread local
    virtual PegCount* get_counts() const
    { return nullptr; }

    virtual PegCount get_global_count(const char* name) const;

    virtual int get_num_counts() const
    { return num_counts; }

    virtual ProfileStats* get_profile() const
    { return nullptr; }

    // implement above -or- below
    virtual ProfileStats* get_profile(
        unsigned /*index*/, const char*& /*name*/, const char*& /*parent*/) const
    { return nullptr; }

    virtual const char* get_defaults() const
    { return nullptr; }

    virtual bool global_stats() const
    { return false; }

    virtual void sum_stats(bool dump_stats);
    virtual void show_interval_stats(IndexVec&, FILE*);
    virtual void show_stats();
    virtual void reset_stats();
    virtual void init_stats(bool new_thread=false);
    virtual void main_accumulate_stats();
    virtual void show_dynamic_stats() {}
    void clear_global_active_counters();

    // Wrappers to check that lists are not tables
    bool verified_begin(const char*, int, SnortConfig*);
    bool verified_set(const char*, Value&, SnortConfig*);
    bool verified_end(const char*, int, SnortConfig*);

    enum Usage
    {
        GLOBAL,
        CONTEXT,
        INSPECT,
        DETECT
    };

    virtual Usage get_usage() const
    { return CONTEXT; }

    virtual bool is_bindable() const
    { return false; }

protected:
    Module(const char* name, const char* help);
    Module(const char* name, const char* help, const Parameter*, bool is_list = false);

    void set_params(const Parameter* p)
    { params = p; }
    std::vector<unsigned> dump_stats_initialized;
    std::vector<std::vector<PegCount>> counts;
    std::vector<std::vector<PegCount>> dump_stats_counts;
    std::vector<PegCount> dump_stats_results;
    int num_counts = -1;

    void set_peg_count(int index, PegCount value, bool dump_stats)
    {
        assert(index < num_counts);
        if (dump_stats)
            dump_stats_counts[get_instance_id()][index] = value;
        else
            counts[get_instance_id()][index] = value;
    }

    void set_max_peg_count(int index, PegCount value, bool dump_stats)
    {
        assert(index < num_counts);
        if (dump_stats)
        {
            if(value > dump_stats_counts[get_instance_id()][index])
                dump_stats_counts[get_instance_id()][index] = value;
        }
        else
        {
            if(value > counts[get_instance_id()][index])
                counts[get_instance_id()][index] = value;
        }
    }

    void add_peg_count(int index, PegCount value, bool dump_stats)
    {
        assert(index < num_counts);
        if (dump_stats)
            dump_stats_counts[get_instance_id()][index] += value;
        else
            counts[get_instance_id()][index] += value;
    }

private:
    friend ModuleManager;
    void init(const char*, const char* = nullptr);


    const char* name;
    const char* help;

    const Parameter* params;
    bool list;
    int table_level = 0;

};
}
#endif

mpse_batch.h

Path = src/framework/mpse_batch.h

#ifndef MPSE_BATCH_H
#define MPSE_BATCH_H

#include <unordered_map>
#include <vector>

#include "framework/mpse.h"
#include "main/snort_types.h"

namespace snort
{
class SO_PUBLIC MpseGroup
{
public:
    MpseGroup()
    { normal_mpse = nullptr; offload_mpse = nullptr; }

    MpseGroup(Mpse* normal)
    { normal_mpse = normal; offload_mpse = nullptr; }

    ~MpseGroup();

    Mpse* get_normal_mpse() const
    { return normal_mpse; }

    // Offload will only be actioned if the offload_search_api is configured.
    // If the offload_search_api is the same as the normal_search_api then
    // only the normal mpse will get created and thus if the offload mpse
    // is requested the normal mpse will be returned otherwise the offload_mpse
    // will get returned
    Mpse* get_offload_mpse() const
    { return offload_mpse ? offload_mpse : normal_mpse; }

    bool create_normal_mpse(const SnortConfig*, const MpseAgent* agent);
    bool create_normal_mpse(const SnortConfig*, const char*);

    bool create_offload_mpse(const SnortConfig*, const MpseAgent* agent);
    bool create_offload_mpse(const SnortConfig*);

    inline bool can_fallback() const
    { return get_offload_mpse() != normal_mpse; }

public:  // FIXIT-L privatize
    Mpse* normal_mpse;
    Mpse* offload_mpse;
    bool normal_is_dup = false;
    bool offload_is_dup = false;
};

template<typename BUF = const uint8_t*, typename LEN = unsigned>
struct MpseBatchKey
{
    BUF buf;
    LEN len;
    MpseBatchKey(BUF b, LEN n)
    {
        this->buf = b;
        this->len = n;
    }

    bool operator==(const MpseBatchKey &k) const
    {
        return buf == k.buf && len == k.len;
    }
};

struct MpseBatchKeyHash
{
    template <class BUF, class LEN>
    std::size_t operator()(const MpseBatchKey<BUF, LEN> &k) const
    {
        std::size_t h1 = std::hash<BUF>()(k.buf);
        std::size_t h2 = std::hash<LEN>()(k.len);

        return h1 ^ h2;
    }
};

class MpseBatchItem
{
public:
    std::vector<MpseGroup*> so;
    bool done;
    bool error;
    int matches;

    MpseBatchItem(MpseGroup* s = nullptr)
    { if (s) so.push_back(s); done = false; error = false; matches = 0; }
};

struct MpseBatch
{
    MpseMatch mf;
    void* context;
    std::unordered_map<MpseBatchKey<>, MpseBatchItem, MpseBatchKeyHash> items;

    void search();
    Mpse::MpseRespType receive_responses();

    void offload_search();
    Mpse::MpseRespType receive_offload_responses();

    bool search_sync();
    bool can_fallback() const;

    static Mpse::MpseRespType poll_responses(MpseBatch*& batch)
    { return Mpse::poll_responses(batch, snort::Mpse::MPSE_TYPE_NORMAL); }

    static Mpse::MpseRespType poll_offload_responses(MpseBatch*& batch)
    { return Mpse::poll_responses(batch, snort::Mpse::MPSE_TYPE_OFFLOAD); }

};

inline void MpseBatch::search()
{
    items.begin()->second.so[0]->get_normal_mpse()->search(*this, Mpse::MPSE_TYPE_NORMAL);
}

inline Mpse::MpseRespType MpseBatch::receive_responses()
{
    return items.begin()->second.so[0]->get_normal_mpse()->
        receive_responses(*this, Mpse::MPSE_TYPE_NORMAL);
}

inline void MpseBatch::offload_search()
{
    assert(items.begin()->second.so[0]->get_offload_mpse());

    items.begin()->second.so[0]->get_offload_mpse()->
        search(*this, Mpse::MPSE_TYPE_OFFLOAD);
}

inline Mpse::MpseRespType MpseBatch::receive_offload_responses()
{
    assert(items.begin()->second.so[0]->get_offload_mpse());

    return items.begin()->second.so[0]->get_offload_mpse()->
        receive_responses(*this, Mpse::MPSE_TYPE_OFFLOAD);
}

inline bool MpseBatch::can_fallback() const
{
    return items.begin()->second.so[0]->can_fallback();
}
}

#endif

mpse.h

Path = src/framework/mpse.h

#ifndef MPSE_H
#define MPSE_H

// MPSE = Multi-Pattern Search Engine - ie fast pattern matching. The key
// methods of an MPSE are the ability to add patterns, compile a state
// machine from the patterns, and search either a single buffer or a set
// of (related) buffers for patterns.

#include <cassert>
#include <string>

#include "framework/base_api.h"
#include "main/snort_types.h"
#include "main/thread.h"
#include "search_engines/search_common.h"

namespace snort
{
// this is the current version of the api
#define SEAPI_VERSION ((BASE_API_VERSION << 16) | 0)

struct SnortConfig;
class Mpse;
struct MpseApi;
struct MpseBatch;
struct ProfileStats;

class SO_PUBLIC Mpse
{
public:
    enum MpseType
    {
        MPSE_TYPE_NORMAL = 0,
        MPSE_TYPE_OFFLOAD = 1
    };

    enum MpseRespType
    {
        MPSE_RESP_COMPLETE_FAIL    = -1,
        MPSE_RESP_NOT_COMPLETE     = 0,
        MPSE_RESP_COMPLETE_SUCCESS = 1
    };

    virtual ~Mpse() = default;

    struct PatternDescriptor
    {
        bool no_case;
        bool negated;
        bool literal;
        bool multi_match;

        unsigned flags;

        PatternDescriptor(
            bool noc = false, bool neg = false, bool lit = false, bool multi = false, unsigned f = 0)
        { no_case = noc; negated = neg; literal = lit; multi_match = multi; flags = f; }
    };

    virtual int add_pattern(
        const uint8_t* pat, unsigned len, const PatternDescriptor&, void* user) = 0;

    virtual int prep_patterns(SnortConfig*) = 0;

    virtual void reuse_search() { }

    int search(
        const uint8_t* T, int n, MpseMatch, void* context, int* current_state);

    virtual int search_all(
        const uint8_t* T, int n, MpseMatch, void* context, int* current_state);

    void search(MpseBatch&, MpseType);

    virtual MpseRespType receive_responses(MpseBatch&, MpseType)
    { return MPSE_RESP_COMPLETE_SUCCESS; }

    static MpseRespType poll_responses(MpseBatch*&, MpseType);

    virtual void set_opt(int) { }
    virtual int print_info() { return 0; }
    virtual int get_pattern_count() const { return 0; }

    virtual bool serialize(uint8_t*&, size_t&) const { return false; }
    virtual bool deserialize(const uint8_t*, size_t) { return false; }
    virtual void get_hash(std::string&) { }

    const char* get_method() { return method.c_str(); }
    void set_verbose(bool b = true) { verbose = b; }

    void set_api(const MpseApi* p) { api = p; }
    const MpseApi* get_api() { return api; }

protected:
    Mpse(const char* method);

    virtual int _search(
        const uint8_t* T, int n, MpseMatch, void* context, int* current_state) = 0;

    virtual void _search(MpseBatch&, MpseType);

private:
    std::string method;
    int verbose = 0;
    const MpseApi* api = nullptr;
};

typedef void (* MpseOptFunc)(SnortConfig*);
typedef void (* MpseExeFunc)();

typedef Mpse* (* MpseNewFunc)(const SnortConfig*, class Module*, const MpseAgent*);
typedef void (* MpseDelFunc)(Mpse*);

typedef Mpse::MpseRespType (* MpsePollFunc)(MpseBatch*&, Mpse::MpseType);

#define MPSE_BASE   0x00  // no optional features
#define MPSE_REGEX  0x02  // supports regex patterns
#define MPSE_ASYNC  0x04  // does asynchronous (lookaside) searches
#define MPSE_MTBLD  0x08  // support multithreaded / parallel compilation

struct MpseApi
{
    BaseApi base;
    uint32_t flags;

    MpseOptFunc activate;
    MpseOptFunc setup;
    MpseExeFunc start;
    MpseExeFunc stop;
    MpseNewFunc ctor;
    MpseDelFunc dtor;
    MpseExeFunc init;
    MpseExeFunc print;
    MpsePollFunc poll;
};
}
#endif

packet_constraints.h

Path = src/framework/packet_constraints.h

#ifndef PACKET_CONSTRAINTS_H
#define PACKET_CONSTRAINTS_H

#include <vector>

#include "protocols/protocol_ids.h"
#include "sfip/sf_ip.h"

namespace snort
{

class Flow;
struct Packet;

struct PacketConstraints
{
    enum SetBits : uint8_t {
        IP_PROTO = 1,
        SRC_IP   = 1 << 1,
        DST_IP   = 1 << 2,
        SRC_PORT = 1 << 3,
        DST_PORT = 1 << 4,
        TENANT   = 1 << 5,
    };

    PacketConstraints() = default;
    PacketConstraints(IpProtocol ip_proto, uint16_t src_port, uint16_t dst_port,
        const snort::SfIp& src_ip, const snort::SfIp& dst_ip, uint8_t set_bits, bool match,
        const std::vector<uint32_t>& tenants = std::vector<uint32_t>()) :
        ip_proto(ip_proto), src_port(src_port), dst_port(dst_port),
        src_ip(src_ip), dst_ip(dst_ip), set_bits(set_bits), match(match), tenants(tenants)
    {}
    PacketConstraints(const PacketConstraints&);
    PacketConstraints& operator=(const PacketConstraints& other);

    bool operator==(const PacketConstraints& other) const;

    bool packet_match(const Packet& p) const;
    bool flow_match(const Flow& f) const;

    IpProtocol ip_proto = IpProtocol::PROTO_NOT_SET;
    uint16_t src_port = 0;
    uint16_t dst_port = 0;
    snort::SfIp src_ip;
    snort::SfIp dst_ip;

    uint8_t set_bits = 0;

    bool match = true;

    std::vector<uint32_t> tenants;
};

} // namespace snort

#endif

parameter.h

Path = src/framework/parameter.h

#ifndef PARAMETER_H
#define PARAMETER_H

// Parameter provides basic parsing from Lua types into meaningful C++
// types.  Modules support a list of parameters.
//
// number ranges are given by:
// nullptr -> any
// # | #: | :# | #:#
// where # is any valid pos|neg dec|hex|octal number

#include <functional>
#include "main/snort_types.h"

namespace snort
{
class Value;

struct SO_PUBLIC Parameter
{
    using RangeQuery = std::function<const char*()>;

    enum Type
    {
        PT_TABLE,      // range is Parameter*, no default
        PT_LIST,       // range is Parameter*, no default
        PT_DYNAMIC,    // range is RangeQuery*
        PT_BOOL,       // if you are reading this, get more coffee
        PT_INT,        // signed 53 bits or less determined by range
        PT_INTERVAL,   // string that defines an interval, bounds within range
        PT_REAL,       // double
        PT_PORT,       // 0 to 64K-1 unless specified otherwise
        PT_STRING,     // any string less than len chars
                       // range = "(optional)" if not required (eg on cmd line)
        PT_SELECT,     // any string appearing in range
        PT_MULTI,      // one or more strings appearing in range
        PT_ENUM,       // string converted to unsigned by range sequence
        PT_MAC,        // 6-byte mac address
        PT_IP4,        // inet_addr() compatible
        PT_ADDR,       // ip4 or ip6 CIDR
        PT_BIT_LIST,   // string that converts to bitset
        PT_INT_LIST,   // string that contains ints
        PT_ADDR_LIST,  // Snort 2 ip list in [ ]
        PT_IMPLIED,    // rule option args w/o values eg relative
        PT_MAX
    };
    const char* name;
    Type type;
    const void* range;  // nullptr|const char*|RangeQuery*|const Parameter*
    const char* deflt;
    const char* help;

    Parameter(const char* n, Type t, const void* r, const char* d, const char* h) :
        name(n), type(t), range(r), deflt(d), help(h) { }

    const char* get_type() const;
    const char* get_range() const;

    bool validate(Value&) const;

    bool is_positional() const
    { return ( name && *name == '~' ); }

    bool is_wild_card() const
    { return ( name && *name == '*' ); }

    bool is_table() const
    { return ( type == PT_TABLE || type == PT_LIST ); }

    bool is_quoted() const
    { return ( type > PT_PORT ); }

    bool get_bool() const;
    double get_number() const;
    const char* get_string() const;

    static const Parameter* find(const Parameter*, const char*);

    // 0-based; -1 if not found; list is | delimited
    static int index(const char* list, const char* key);

    // convert string to long long (including 'maxN' literals)
    static int64_t get_int(const char*);
    static int64_t get_int(const char*, bool&);

    // convert string to unsigned long long (including 'maxN' literals)
    static uint64_t get_uint(const char*);
    static uint64_t get_uint(const char*, bool&);
};
}
#endif

pdu_section.h

Path = src/framework/pdu_section.h

#ifndef PDU_SECTION_H
#define PDU_SECTION_H

// PDU section in which an IPS option provides the buffer.
// The sections are ordered from earliest to latest.
// The latest section is used to determine the rule group.
// Currently only used by ips options that apply to HTTP. The rest default to PS_NONE.

#include "main/snort_types.h"

namespace snort
{
// PS_HEADER_BODY is used for rule options that can work on both header and body.
// It was added to make the rule timing selection easier:
// - if combined with header the rule should still be evaluated in both header and body.
// - if combined with body or trailer should be evaluated at body/trailer.
// PS_ERROR is used for invalid combination of sections:
// trailer and body sections can be combined only if it's a request trailer in a to_client direction
// When updating this enum, also update section_to_str
enum PduSection { PS_NONE = 0, PS_HEADER, PS_HEADER_BODY, PS_BODY, PS_TRAILER, PS_MAX = PS_TRAILER,
    PS_ERROR };

// Bitmask with all of supported sections
using section_flags = uint16_t;

inline section_flags section_to_flag(PduSection sect)
{ return 1<<sect; }
}

#endif

policy_selector.h

Path = src/framework/policy_selector.h

#ifndef POLICY_SELECTOR_H
#define POLICY_SELECTOR_H

// Policy selectors provide a method to select the network policy and default inspection
// and IPS policies for a given packet

#include <string>

#include "framework/base_api.h"
#include "framework/counts.h"
#include "main/snort_types.h"

struct _daq_flow_stats;
struct _daq_pkt_hdr;

namespace snort
{
#define POLICY_SELECTOR_API_VERSION ((BASE_API_VERSION << 16) | 0)

struct Packet;
class PolicySelector;
struct PolicySelectorApi;
struct SnortConfig;

struct PolicySelectStats
{
    PegCount packets;
    PegCount no_match;
};

struct PolicySelectUse
{
    std::string stringify() const
    { return "file = " + name; }

    std::string name;
    unsigned network_index;
    unsigned inspection_index;
    unsigned ips_index;
};

typedef PolicySelector* (*SelectorNewFunc)(Module*);
typedef void (*SelectorDeleteFunc)(PolicySelector*);

struct PolicySelectorApi
{
    BaseApi base;
    SelectorNewFunc ctor;
    SelectorDeleteFunc dtor;
};

class SO_PUBLIC PolicySelector
{
public:
    PolicySelector() = delete;
    PolicySelector(const PolicySelector&) = delete;
    virtual ~PolicySelector() = default;

    static void free_policy_selector(PolicySelector* ps)
    {
        if (ps)
            ps->get_api()->dtor(ps);
    }
    const PolicySelectorApi* get_api() const
    { return api; }
    virtual bool select_default_policies(const _daq_pkt_hdr&, const SnortConfig*) = 0;
    virtual bool select_default_policies(const _daq_flow_stats&, const SnortConfig*) = 0;
    virtual void show() const = 0;

protected:
    explicit PolicySelector(const PolicySelectorApi* api) : api(api)
    { }
    const PolicySelectorApi* api;
};
}
#endif

range.h

Path = src/framework/range.h

#ifndef FRAMEWORK_RANGE_H
#define FRAMEWORK_RANGE_H

// RangeCheck supports common IpsOption evaluation syntax and semantics.

#include "main/snort_types.h"

// unfortunately, <> was implemented inconsistently.  eg:
// dsize implements <> as ( a <= c && c <= b ) and
// icode implements <> as ( a < c && c < b )

// <> is implemented icode style but we add explicit options
// <=> for dsize style and >< for icode style so rule options
// can coerce <> if needed for backwards compatibility

namespace snort
{
class SO_PUBLIC RangeCheck
{
public:
    enum Op
    {
        // =  !  <   <=  >   >=  <>  <=>
        EQ, NOT, LT, LE, GT, GE, LG, LEG, MAX
    };

    Op op = MAX;
    int64_t min = 0;
    int64_t max = 0;

    bool operator==(const RangeCheck&) const;

    void init();
    bool is_set() const;
    // FIXIT-L add ttl style syntax
    bool parse(const char* s);
    bool eval(int64_t) const;
    bool validate(const char* s, const char* r);
    uint32_t hash() const;
};
}
#endif

so_rule.h

Path = src/framework/so_rule.h

#ifndef SO_RULE_H
#define SO_RULE_H

// SO rule = shared object rule; allows implementing arbitrary C++ for
// detection below and beyond the text rule options.  An SO rule is just
// like a text rule except that it can call function hooks. It can also
// define its own rule options and any other plugins it may need.

#include "framework/base_api.h"
#include "framework/ips_option.h"
#include "main/snort_types.h"

namespace snort
{
struct Packet;
}

// this is the current version of the api
#define SOAPI_VERSION ((BASE_API_VERSION << 16) | 0)

//-------------------------------------------------------------------------
// rule format is:  header ( [<stub opts>;] soid:<tag>; [<remaining opts>;] )
// <remaining opts> may include so opts like so:<key>;
// ctor(<key>) returns eval func and optional data
// data is freed with dtor(data)

typedef snort::IpsOption::EvalStatus (* SoEvalFunc)(void*, class Cursor&, snort::Packet*);
typedef SoEvalFunc (* SoNewFunc)(const char* key, void**);
typedef void (* SoDelFunc)(void*);
typedef void (* SoAuxFunc)();

struct SoApi
{
    snort::BaseApi base;

    const uint8_t* rule;
    unsigned length;

    // these may be nullptr
    SoAuxFunc pinit;  // initialize global plugin data
    SoAuxFunc pterm;  // clean-up pinit()

    SoAuxFunc tinit;  // initialize thread-local plugin data
    SoAuxFunc tterm;  // clean-up tinit()

    // these must be set
    SoNewFunc ctor;   // get eval with optional instance data
    SoDelFunc dtor;   // clean up instance data
};

#endif

value.h

Path = src/framework/value.h

#ifndef VALUE_H
#define VALUE_H

// Value is used to represent Lua bool, number, and string.

#include <algorithm>
#include <cstring>
#include <sstream>

#include "framework/bits.h"
#include "framework/parameter.h"
#include "main/snort_types.h"

namespace snort
{
struct SfCidr;
struct SfIp;

class SO_PUBLIC Value
{
public:
    static const unsigned mask_bits = 52; // ieee 754 significand
    enum ValueType { VT_BOOL, VT_NUM, VT_UNUM, VT_STR, VT_REAL };

    Value(bool b)
    { set(b); }

    Value(double d)
    { set(d); }

    Value(int64_t i)
    { set(i); }

    Value(uint64_t u)
    { set(u); }

    Value(const char* s)
    { set(s); set_origin(s); }

    Value(const Value& v) :
        type(v.type),
        unum(v.unum),
        num(v.num),
        real(v.real),
        str(v.str),
        origin_str(v.origin_str),
        ss(nullptr),
        param(v.param)
    {}

    Value& operator=(const Value& v)
    {
        if ( this == &v )
            return *this;

        delete ss;
        ss = nullptr;

        type = v.type;
        num = v.num;
        unum = v.unum;
        real = v.real;
        str = v.str;
        origin_str = v.origin_str;
        param = v.param;

        return *this;
    }

    ValueType get_type() const
    { return type; }

    ~Value()
    { delete ss; }

    void set(bool b)
    { type = VT_BOOL; unum = b ? 1 : 0; }

    void set(double d)
    { type = VT_REAL; real = d; }

    void set(uint64_t n)
    { type = VT_UNUM; unum = n; }

    void set(int64_t n)
    { type = VT_NUM; num = n; }

    void set(const char* s)
    { type = VT_STR; str = s; }

    void set_origin(const char* val)
    { origin_str = val; }

    void set(const uint8_t* s, unsigned len)
    { type = VT_STR; str.assign((const char*)s, len); }

    void set(const Parameter* p)
    { param = p; }

    void set_enum(unsigned u)
    { type = VT_UNUM; unum = u; }

    void set_aux(uint64_t u)
    { type = VT_UNUM; unum = u; }

    const char* get_name() const
    { return param ? param->name : nullptr; }

    bool is(const char* s) const
    { return param ? !strcmp(param->name, s) : false; }

    bool has_default() const
    { return param ? param->deflt != nullptr : false; }

    bool get_bool() const
    { return 0 != ((VT_REAL == type) ? real : unum); }

    size_t get_size() const
    { return (VT_REAL == type) ? (size_t)real : ((VT_UNUM == type) ? (size_t)unum : (size_t)num); }

    uint8_t get_uint8() const
    { return (VT_REAL == type) ? (uint8_t)real : ((VT_UNUM == type) ? (uint8_t)unum : (uint8_t)num); }

    int16_t get_int16() const
    { return (VT_REAL == type) ? (int16_t)real : ((VT_UNUM == type) ? (int16_t)unum : (int16_t)num); }

    uint16_t get_uint16() const
    { return (VT_REAL == type) ? (uint16_t)real : ((VT_UNUM == type) ? (uint16_t)unum : (uint16_t)num); }

    int32_t get_int32() const
    { return (VT_REAL == type) ? (int32_t)real : ((VT_UNUM == type) ? (int32_t)unum : (int32_t)num); }

    uint32_t get_uint32() const
    { return (VT_REAL == type) ? (uint32_t)real : ((VT_UNUM == type) ? (uint32_t)unum : (uint32_t)num); }

    int64_t get_int64() const
    { return (VT_REAL == type) ? (int64_t)real : ((VT_UNUM == type) ? (int64_t)unum : (int64_t)num); }

    uint64_t get_uint64() const
    { return (VT_REAL == type) ? (uint64_t)real : ((VT_UNUM == type) ? (uint64_t)unum : (uint64_t)num); }

    double get_real() const
    { return real; }

    const uint8_t* get_buffer(unsigned& n) const
    { n = (unsigned)str.size(); return (const uint8_t*)str.data(); }

    const char* get_string() const
    { return str.c_str(); }

    std::string get_as_string() const;
    Parameter::Type get_param_type() const;
    std::string get_origin_string() const;

    std::string get_unquoted_string() const
    {
        if ( str.length() < 2 )
            return str;

        std::string tmp = str;

        if ( tmp.front() == '"' and tmp.back() == '"' )
        {
            tmp.erase(0, 1);
            tmp.erase(tmp.size() - 1, 1);
        }

        return tmp;
    }

    bool strtol(long&) const;
    bool strtol(long&, const std::string&) const;
    bool strtoul(unsigned long&) const;
    bool strtoul(unsigned long&, const std::string&) const;

    bool operator==(const char* s) const
    { return str == s; }

    bool operator==(uint64_t n) const
    { return n == ((VT_UNUM == type) ? unum : (uint64_t)num); }

    bool operator==(double d) const
    { return real == d; }

    void get_bits(PortBitSet&) const;
    void get_bits(VlanBitSet&) const;
    void get_bits(ByteBitSet&) const;

    void lower()
    { std::transform(str.begin(), str.end(), str.begin(), ::tolower); }

    void upper()
    { std::transform(str.begin(), str.end(), str.begin(), ::toupper); }

    uint32_t get_ip4() const
    { return (VT_REAL == type) ? (uint32_t)real : (uint32_t)unum; }
    void get_mac(uint8_t (&mac)[6]) const;
    void get_addr(uint8_t (&addr)[16]) const;
    void get_addr_ip4(uint8_t (&addr)[4]) const;
    void get_addr_ip6(uint8_t (&addr)[16]) const;
    void get_addr(SfIp&) const;
    void get_addr(SfCidr&) const;

    void set_first_token();
    bool get_next_token(std::string&);
    bool get_next_csv_token(std::string&);

    // set/clear flag based on get_bool()
    void update_mask(uint8_t& mask, uint8_t flag, bool invert = false);
    void update_mask(uint16_t& mask, uint16_t flag, bool invert = false);
    void update_mask(uint32_t& mask, uint32_t flag, bool invert = false);
    void update_mask(uint64_t& mask, uint64_t flag, bool invert = false);

private:
    ValueType type;
    uint64_t unum = 0;
    int64_t num = 0;
    double real = 0;
    std::string str;
    std::string origin_str;
    std::stringstream* ss = nullptr;
    const Parameter* param = nullptr;
};
}
#endif

framework/test/

hash/

Message digests and hash maps/table implementations:

  • md5: open source implementation based on Colin Plumb’s code.

  • sha2: open source implementation by Aaron Gifford.

  • ghash: Generic hash table

  • xhash: Hash table with supports memcap and automatic memory recovery when out of memory.

  • zhash: zero runtime allocations/preallocated hash table.

Use of the above hashing utilities is primarily for use by pre-existing code. For new code, use standard template library and C++11 features.

For thread-safe shared caches:

  • lru_cache_shared: A thread-safe LRU map.

09/25/2023

A vector of pointers to HashLruCache objects, vector<HashLruCache*>, has been introduced to manage multiple types of LRUs within xhash. This enhancement facilitates more sophisticated handling of varied data types within the same xhash. With this advancement, greater control over data pruning in the LRU is achieved, depending on the type of data. This feature is valuable when there’s a necessity to retain certain data in the LRU for durations longer or shorter than other data. The utilization of this feature is optional. During initialization, the number of LRUs to be created can be specified. If not specified, a single LRU will be created by default.

Segmented Shared LRU Cache The SegmentedLruCache class is a layer built atop the existing LruCacheShared class, designed to mitigate bottlenecks in multi-threaded environments, thereby bolstering scalability. Without altering the core caching logic, it divides the cache into multiple segments, defaulting to four. This structure drastically reduces contention among threads, allowing for improved performance. The segmented approach is generic and configurable, enabling easy adaptation for different modules while preserving the fundamental LRU cache behavior. Through this strategic modification, the pathway for enhanced scalability and future advancements is significantly broadened, making the caching mechanism more robust and adaptable to evolving computational demands. check host_attributes.cc for example usage.

ghash.h

Path = src/hash/ghash.h

#ifndef GHASH_H
#define GHASH_H

// generic hash table - stores and maps key + data pairs

#include <cstring>
#include "hash_key_operations.h"
#include "main/snort_types.h"

namespace snort
{
struct GHashNode
{
    struct GHashNode* next;
    struct GHashNode* prev;
    const void* key;
    void* data;
};

typedef void (* gHashFree)(void*);

class SO_PUBLIC GHash
{
public:
    GHash(int nrows, unsigned keysize, bool userkey, gHashFree);
    ~GHash();

    int insert(const void* const key, void* const data);
    int remove(const void* const key);
    void* find(const void* const key);
    GHashNode* find_first();
    GHashNode* find_next();
    void set_hashkey_ops(HashKeyOperations*);

    unsigned get_count() const
    { return count; }

private:
    GHashNode* find_node(const void* const key, unsigned index);
    int free_node(unsigned index, GHashNode*);
    void next();

    unsigned get_key_length(const void* const key)
    { return ( keysize > 0  ) ? keysize : strlen((const char*)key) + 1; }

    unsigned get_index(const void* const key)
    {
        unsigned hashkey = hashfcn->do_hash((const unsigned char*)key, get_key_length(key));
        return hashkey % nrows;
    }

    unsigned keysize;     // bytes in key, if < 0 -> keys are strings
    bool userkey;          // user owns the key */
    gHashFree userfree;
    int nrows;            // # rows int the hash table use a prime number 211, 9871
    HashKeyOperations* hashfcn;
    GHashNode** table;    // array of node ptr's
    unsigned count;       // total # nodes in table
    int crow;             // findfirst/next row in table
    GHashNode* cnode;     // findfirst/next node ptr

};


}
#endif

hash_defs.h

Path = src/hash/hash_defs.h

#ifndef HASH_DEFS_H
#define HASH_DEFS_H

#include "hash_key_operations.h"
#include "main/snort_types.h"

namespace snort
{
#define HASH_NOMEM     (-2)
#define HASH_NOT_FOUND (-1)
#define HASH_OK        0
#define HASH_INTABLE   1
#define HASH_PENDING   2

class HashNode
{
public:
    HashNode* gnext; // lru or free node list
    HashNode* gprev;
    HashNode* next;  // hash row node list
    HashNode* prev;
    void* key;
    void* data;
    int rindex;
};
}
#endif

hashes.h

Path = src/hash/hashes.h

#ifndef HASHES_H
#define HASHES_H

#include "main/snort_types.h"

namespace snort
{
#define MD5_HASH_SIZE    16
#define SHA256_HASH_SIZE 32
#define SHA512_HASH_SIZE 64
#define MAX_HASH_SIZE    64

// digest must be buffer of size given above
SO_PUBLIC void md5(const unsigned char* data, size_t size, unsigned char* digest);
SO_PUBLIC void sha256(const unsigned char* data, size_t size, unsigned char* digest);
SO_PUBLIC void sha512(const unsigned char* data, size_t size, unsigned char* digest);
}
#endif

hash_key_operations.h

Path = src/hash/hash_key_operations.h

#ifndef HASH_KEY_OPERATIONS_H
#define HASH_KEY_OPERATIONS_H

#include "main/snort_types.h"

namespace
{
    inline uint32_t rot(uint32_t x, unsigned k) { return (x << k) | (x >> (32 - k)); }
    inline void mix(uint32_t& a, uint32_t& b, uint32_t& c)
    {
        a -= c; a ^= rot(c, 4); c += b;
        b -= a; b ^= rot(a, 6); a += c;
        c -= b; c ^= rot(b, 8); b += a;
        a -= c; a ^= rot(c,16); c += b;
        b -= a; b ^= rot(a,19); a += c;
        c -= b; c ^= rot(b, 4); b += a;
    }
    inline void finalize(uint32_t& a, uint32_t& b, uint32_t& c)
    {
        c ^= b; c -= rot(b,14);
        a ^= c; a -= rot(c,11);
        b ^= a; b -= rot(a,25);
        c ^= b; c -= rot(b,16);
        a ^= c; a -= rot(c,4);
        b ^= a; b -= rot(a,14);
        c ^= b; c -= rot(b,24);
    }
}

namespace snort
{
SO_PUBLIC void mix_str(
    uint32_t& a, uint32_t& b, uint32_t& c,
    // n == 0 => strlen(s)
    const char* s, unsigned n = 0);

SO_PUBLIC uint32_t str_to_hash(const uint8_t *str, size_t length);

static inline int hash_nearest_power_of_2(int nrows)
{
    nrows -= 1;
    for (unsigned i = 1; i < sizeof(nrows) * 8; i <<= 1)
        nrows = nrows | (nrows >> i);
    nrows += 1;

    return nrows;
}

static inline uint64_t hash_mac(const uint8_t* a)
{
    uint64_t hash = a[0];
    hash = (hash << 8) | a[1];
    hash = (hash << 8) | a[2];
    hash = (hash << 8) | a[3];
    hash = (hash << 8) | a[4];
    hash = (hash << 8) | a[5];
    return hash;
}

class HashKeyOperations
{
public:
    HashKeyOperations(int rows);
    virtual ~HashKeyOperations() = default;

    virtual unsigned do_hash(const unsigned char* key, int len);
    virtual bool key_compare(const void* key1, const void* key2, size_t len);

protected:
    unsigned seed;
    unsigned scale;
    unsigned hardener;
};
}

#endif

hash_lru_cache.h

Path = src/hash/hash_lru_cache.h

#ifndef HASH_LRU_CACHE_H
#define HASH_LRU_CACHE_H

#include "hash_defs.h"

class HashLruCache
{
public:
    HashLruCache();

    void insert(snort::HashNode*);
    void touch(snort::HashNode*);
    void remove_node(snort::HashNode*);

    snort::HashNode* get_lru_node()
    {
        cursor = tail;
        return cursor;
    }

    snort::HashNode* get_next_lru_node()
    {
        if ( cursor )
            cursor = cursor->gprev;
        return cursor;
    }

    snort::HashNode* get_current_node()
    { return cursor; }

    void* get_mru_user_data()
    { return ( head ) ? head->data : nullptr; }

    void* get_lru_user_data()
    { return ( tail ) ? tail->data : nullptr; }

    snort::HashNode* remove_lru_node()
    {
        snort::HashNode* hnode = tail;
        if ( hnode )
            remove_node(hnode);

        return hnode;
    }

private:
    snort::HashNode* head = nullptr;
    snort::HashNode* tail = nullptr;
    snort::HashNode* cursor = nullptr;
};

#endif

lru_cache_local.h

Path = src/hash/lru_cache_local.h

#ifndef LRU_CACHE_LOCAL_H
#define LRU_CACHE_LOCAL_H

// LruCacheLocal - A simple thread-unsafe memcap-enforced least-recently-used cache.

#include <list>
#include <unordered_map>
#include <vector>

#include "framework/counts.h"

#define LRU_CACHE_LOCAL_PEGS(module) \
    { CountType::SUM, "cache_adds", module " cache added new entry" }, \
    { CountType::SUM, "cache_hits", module " cache found existing entry" }, \
    { CountType::SUM, "cache_misses", module " cache did not find entry" }, \
    { CountType::SUM, "cache_replaces", module " cache found entry and replaced its value" }, \
    { CountType::SUM, "cache_max", module " cache's maximum byte usage"}, \
    { CountType::SUM, "cache_prunes", module " cache pruned entry to make space for new entry" }, \
    { CountType::SUM, "cache_removes", module " cache removed existing entry"}

struct LruCacheLocalStats
{
    PegCount cache_adds;
    PegCount cache_hits;
    PegCount cache_misses;
    PegCount cache_replaces;
    PegCount cache_max;
    PegCount cache_prunes;
    PegCount cache_removes;
};

template<typename Key, typename Value, typename Hash>
class LruCacheLocal
{
public:
    LruCacheLocal(const size_t sz, struct LruCacheLocalStats& st)
        : max_size(sz), current_size(0), stats(st) { }

    virtual ~LruCacheLocal() = default;

    // Return 1 if an entry associated with the key exists, else return 0
    int count(const Key&);

    // Return the entry associated with the key; insert new entry if absent
    Value& find_else_create(const Key&, bool* is_new = nullptr);

    // If key does not exist, insert the key-value pair and return true;
    // else return false replacing the existing value if asked
    bool add(const Key&, const Value&, bool replace = false);

    // If key does not exist, return false;
    // else remove the entry associated with key
    bool remove(const Key&);

    // Copy all key-value pairs from the cache
    void get_all_values(std::vector<std::pair<Key, Value>>&);

protected:
    using LruList = std::list<std::pair<Key, Value>>;
    using LruListIter = typename LruList::iterator;
    using LruMap = std::unordered_map<Key, LruListIter, Hash>;
    using LruMapIter = typename LruMap::iterator;

    void prune();
    void add_entry(const Key&, const Value&);

    static constexpr size_t entry_size = 2 * sizeof(Key) + sizeof(Value) + sizeof(LruListIter);
    const size_t max_size;
    size_t current_size;
    LruList list;
    LruMap map;
    struct LruCacheLocalStats& stats;
};

template<typename Key, typename Value, typename Hash>
void LruCacheLocal<Key, Value, Hash>::prune()
{
    if ( !max_size )
        return;

    while ( current_size > max_size and !list.empty() )
    {
        auto it = --list.end();
        map.erase(it->first);
        list.erase(it);
        current_size -= entry_size;
        ++stats.cache_prunes;
    }
}

template<typename Key, typename Value, typename Hash>
int LruCacheLocal<Key, Value, Hash>::count(const Key& key)
{
    return map.count(key);
}

template<typename Key, typename Value, typename Hash>
void LruCacheLocal<Key, Value, Hash>::add_entry(const Key& key, const Value& value)
{
    stats.cache_adds++;
    list.emplace_front(std::make_pair(key, value));
    map[key] = list.begin();
    current_size += entry_size;
    prune();
    if ( stats.cache_max < current_size )
        stats.cache_max = current_size;
}

template<typename Key, typename Value, typename Hash>
Value& LruCacheLocal<Key, Value, Hash>::find_else_create(const Key& key, bool* is_new)
{
    auto it = map.find(key);
    if (it == map.end())
    {
        stats.cache_misses++;
        add_entry(key, Value());
        if ( is_new )
            *is_new = true;
        return list.begin()->second;
    }

    stats.cache_hits++;
    list.splice(list.begin(), list, it->second);
    return list.begin()->second;
}

template<typename Key, typename Value, typename Hash>
bool LruCacheLocal<Key, Value, Hash>::add(const Key& key, const Value& value, bool replace)
{
    auto it = map.find(key);
    if (it == map.end())
    {
        stats.cache_misses++;
        add_entry(key, value);
        return true;
    }

    stats.cache_hits++;
    list.splice(list.begin(), list, it->second);
    if ( replace )
    {
        it->second->second = value;
        stats.cache_replaces++;
    }
    return false;
}

template<typename Key, typename Value, typename Hash>
bool LruCacheLocal<Key, Value, Hash>::remove(const Key& key)
{
    auto it = map.find(key);
    if (it == map.end())
    {
        return false;
    }
    list.erase(it->second);
    map.erase(it);
    current_size -= entry_size;
    stats.cache_removes++;
    return true;
}

template<typename Key, typename Value, typename Hash>
void LruCacheLocal<Key, Value, Hash>::get_all_values(std::vector<std::pair<Key, Value>>& kv)
{
    for (auto& entry : list )
        kv.emplace_back(entry);
}

#endif

lru_cache_shared.h

Path = src/hash/lru_cache_shared.h

#ifndef LRU_CACHE_SHARED_H
#define LRU_CACHE_SHARED_H

// LruCacheShared -- Implements a thread-safe unordered map where the
// least-recently-used (LRU) entries are removed once a fixed size is hit.

#include <atomic>
#include <cassert>
#include <list>
#include <memory>
#include <mutex>
#include <typeinfo>
#include <unordered_map>
#include <vector>

#include "framework/counts.h"

extern const PegInfo lru_cache_shared_peg_names[];

struct LruCacheSharedStats
{
    PegCount adds = 0;          // an insert that added new entry
    PegCount alloc_prunes = 0;  // when an old entry is removed to make room for a new entry
    PegCount bytes_in_use = 0;  // current bytes in use
    PegCount items_in_use = 0;  // current items in cache
    PegCount find_hits = 0;     // found entry in cache
    PegCount find_misses = 0;   // did not find entry in cache
    PegCount reload_prunes = 0; // when an old entry is removed due to lower memcap during reload
    PegCount removes = 0;       // found entry and removed it
    PegCount replaced = 0;      // found entry and replaced it
};

enum class LcsInsertStatus {
    LCS_ITEM_PRESENT,
    LCS_ITEM_INSERTED,
    LCS_ITEM_REPLACED
};

template<typename Key, typename Value, typename Hash, typename Eq = std::equal_to<Key>,
    typename Purgatory = std::vector<std::shared_ptr<Value>>>
class LruCacheShared
{
public:

    //  Do not allow default constructor, copy constructor or assignment
    //  operator.  Cannot safely copy the LruCacheShared due to the mutex
    //  lock.
    LruCacheShared() = delete;
    LruCacheShared(const LruCacheShared& arg) = delete;
    LruCacheShared& operator=(const LruCacheShared& arg) = delete;

    LruCacheShared(const size_t initial_size) :
        max_size(initial_size), current_size(0) { }

    virtual ~LruCacheShared() = default;

    using Data = std::shared_ptr<Value>;
    using ValueType = Value;
    using KeyType = Key;

    // Return data entry associated with key. If doesn't exist, return nullptr.
    Data find(const Key&);

    // Return data entry associated with key. If doesn't exist, create a new entry.
    Data operator[](const Key&);

    // Same as operator[]; additionally, sets the boolean if a new entry is created.
    Data find_else_create(const Key&, bool* new_data);

    // Returns true if found or replaced, takes a ref to a user managed entry
    bool find_else_insert(const Key&, Data&, bool replace = false);

    // Returns the found or inserted data, takes a ref to user managed entry.
    Data find_else_insert(const Key&, Data&, LcsInsertStatus*,
        bool replace = false);

    // Return all data from the LruCache in order (most recently used to least)
    std::vector<std::pair<Key, Data>> get_all_data();

    //  Get current number of elements in the LruCache.
    size_t size()
    {
        std::lock_guard<std::mutex> cache_lock(cache_mutex);
        return list.size();
    }

    virtual size_t mem_size()
    {
        std::lock_guard<std::mutex> cache_lock(cache_mutex);
        return list.size() * mem_chunk;
    }

    size_t get_max_size() const
    {
        return max_size;
    }

    //  Modify the maximum number of entries allowed in the cache. If the size is reduced,
    //  the oldest entries are removed. This pruning doesn't utilize reload resource tuner.
    bool set_max_size(size_t newsize);

    //  Remove entry associated with Key.
    //  Returns true if entry existed, false otherwise.
    virtual bool remove(const Key&);

    //  Remove entry associated with key and return removed data.
    //  Returns true and copy of data if entry existed.  Returns false if
    //  entry did not exist.
    virtual bool remove(const Key&, Data&);

    const PegInfo* get_pegs() const
    { return lru_cache_shared_peg_names; }

    const PegCount* get_counts() const
    { return (const PegCount*)&stats; }

    void lock()
    { cache_mutex.lock(); }

    void unlock()
    { cache_mutex.unlock(); }

protected:
    using LruList = std::list<std::pair<Key, Data>>;
    using LruListIter = typename LruList::iterator;
    using LruMap = std::unordered_map<Key, LruListIter, Hash, Eq>;
    using LruMapIter = typename LruMap::iterator;

    static constexpr size_t mem_chunk = sizeof(Data) + sizeof(Value);

    std::atomic<size_t> max_size; // Once max_size elements are in the cache, start to
                                  // remove the least-recently-used elements.

    std::atomic<size_t> current_size;// Number of entries currently in the cache.

    std::mutex cache_mutex;
    LruList list;  //  Contains key/data pairs. Maintains LRU order with
                   //  least recently used at the end.
    LruMap map;    //  Maps key to list iterator for fast lookup.

    struct LruCacheSharedStats stats;

    // The reason for these functions is to allow derived classes to do their
    // size book keeping differently (e.g. host_cache). This effectively
    // decouples the current_size variable from the actual size in memory,
    // so these functions should only be called when something is actually
    // added or removed from memory (e.g. in find_else_insert, remove, etc).
    virtual void increase_size(ValueType* value_ptr=nullptr)
    {
        UNUSED(value_ptr);
        current_size++;
    }

    virtual void decrease_size(ValueType* value_ptr=nullptr)
    {
        UNUSED(value_ptr);
        current_size--;
    }

    // Caller must lock and unlock. Don't use this during snort reload for which
    // we need gradual pruning and size reduction via reload resource tuner.
    void prune(Purgatory& data)
    {
        LruListIter list_iter;
        assert(data.empty());
        while (current_size > max_size && !list.empty())
        {
            list_iter = --list.end();
            data.emplace_back(list_iter->second); // increase reference count
            decrease_size(list_iter->second.get());
            map.erase(list_iter->first);
            list.erase(list_iter);
            ++stats.alloc_prunes;
        }
    }
};

template<typename Key, typename Value, typename Hash, typename Eq, typename Purgatory>
bool LruCacheShared<Key, Value, Hash, Eq, Purgatory>::set_max_size(size_t newsize)
{
    if (newsize == 0)
        return false;   //  Not allowed to set size to zero.

    // Like with remove(), we need local temporary references to data being
    // deleted, to avoid race condition. This data needs to self-destruct
    // after the cache_lock does.
    Purgatory data;

    std::lock_guard<std::mutex> cache_lock(cache_mutex);

    //  Remove the oldest entries if we have to reduce cache size.
    max_size = newsize;

    prune(data);

    return true;
}

template<typename Key, typename Value, typename Hash, typename Eq, typename Purgatory>
std::shared_ptr<Value> LruCacheShared<Key, Value, Hash, Eq, Purgatory>::find(const Key& key)
{
    std::lock_guard<std::mutex> cache_lock(cache_mutex);

    auto map_iter = map.find(key);
    if (map_iter == map.end())
    {
        // coverity[missing_lock]
        stats.find_misses++;
        return nullptr;
    }

    //  Move entry to front of LruList
    list.splice(list.begin(), list, map_iter->second);
    // coverity[missing_lock]
    stats.find_hits++;
    return map_iter->second->second;
}

template<typename Key, typename Value, typename Hash, typename Eq, typename Purgatory>
std::shared_ptr<Value> LruCacheShared<Key, Value, Hash, Eq, Purgatory>::operator[](const Key& key)
{
    return find_else_create(key, nullptr);
}

template<typename Key, typename Value, typename Hash, typename Eq, typename Purgatory>
std::shared_ptr<Value> LruCacheShared<Key, Value, Hash, Eq, Purgatory>::find_else_create(const Key& key,
    bool* new_data)
{
    // As with remove and operator[], we need a temporary list of references
    // to delay the destruction of the items being removed by prune().
    // This is one instance where we cannot get by with directly locking and
    // unlocking the cache_mutex, because the cache must be locked when we
    // return the data pointer (below), or else, some other thread might
    // delete it before we got a chance to return it.
    Purgatory tmp_data;

    std::lock_guard<std::mutex> cache_lock(cache_mutex);

    auto map_iter = map.find(key);
    if (map_iter != map.end())
    {
        // coverity[missing_lock]
        stats.find_hits++;
        list.splice(list.begin(), list, map_iter->second); // update LRU
        return map_iter->second->second;
    }

    // coverity[missing_lock]
    stats.find_misses++;
    // coverity[missing_lock]
    stats.adds++;
    if ( new_data )
        *new_data = true;
    Data data = Data(new Value);

    //  Add key/data pair to front of list.
    list.emplace_front(std::make_pair(key, data));
    increase_size(data.get());

    //  Add list iterator for the new entry to map.
    map[key] = list.begin();

    prune(tmp_data);

    return data;
}

template<typename Key, typename Value, typename Hash, typename Eq, typename Purgatory>
bool LruCacheShared<Key, Value, Hash, Eq, Purgatory>::find_else_insert(const Key& key, Data& data,
    bool replace)
{
    Purgatory tmp_data;

    std::lock_guard<std::mutex> cache_lock(cache_mutex);

    auto map_iter = map.find(key);
    if (map_iter != map.end())
    {
        // coverity[missing_lock]
        stats.find_hits++;
        if (replace)
        {
            // Explicitly calling the reset so its more clear that destructor could be called for the object
            decrease_size(map_iter->second->second.get());
            map_iter->second->second.reset();
            map_iter->second->second = data;
            increase_size(map_iter->second->second.get());
            // coverity[missing_lock]
            stats.replaced++;
        }
        list.splice(list.begin(), list, map_iter->second); // update LRU
        return true;
    }
    // coverity[missing_lock]
    stats.find_misses++;
    // coverity[missing_lock]
    stats.adds++;

    //  Add key/data pair to front of list.
    list.emplace_front(std::make_pair(key, data));
    increase_size(data.get());

    //  Add list iterator for the new entry to map.
    map[key] = list.begin();

    prune(tmp_data);

    return false;
}

template<typename Key, typename Value, typename Hash, typename Eq, typename Purgatory>
std::shared_ptr<Value> LruCacheShared<Key, Value, Hash, Eq, Purgatory>::find_else_insert(const Key& key, Data& data,
    LcsInsertStatus* status, bool replace)
{
    Purgatory tmp_data;

    std::lock_guard<std::mutex> cache_lock(cache_mutex);

    auto map_iter = map.find(key);
    if (map_iter != map.end())
    {
        stats.find_hits++;
        if (status) *status = LcsInsertStatus::LCS_ITEM_PRESENT;
        if (replace)
        {
            // Explicitly calling the reset so its more clear that destructor could be called for the object
            decrease_size(map_iter->second->second.get());
            map_iter->second->second.reset();
            map_iter->second->second = data;
            increase_size(map_iter->second->second.get());
            stats.replaced++;
            if (status) *status = LcsInsertStatus::LCS_ITEM_REPLACED;
        }
        list.splice(list.begin(), list, map_iter->second); // update LRU
        return map_iter->second->second;
    }

    stats.find_misses++;
    stats.adds++;
    if (status) *status = LcsInsertStatus::LCS_ITEM_INSERTED;

    //  Add key/data pair to front of list.
    list.emplace_front(std::make_pair(key, data));
    increase_size(data.get());

    //  Add list iterator for the new entry to map.
    map[key] = list.begin();

    prune(tmp_data);

    return data;
}

template<typename Key, typename Value, typename Hash, typename Eq, typename Purgatory>
std::vector<std::pair<Key, std::shared_ptr<Value>>> LruCacheShared<Key, Value, Hash, Eq, Purgatory>::get_all_data()
{
    std::vector<std::pair<Key, Data> > vec;
    std::lock_guard<std::mutex> cache_lock(cache_mutex);

    vec.reserve(list.size());
    std::copy(list.cbegin(), list.cend(), std::back_inserter(vec));
    return vec;
}

template<typename Key, typename Value, typename Hash, typename Eq, typename Purgatory>
bool LruCacheShared<Key, Value, Hash, Eq, Purgatory>::remove(const Key& key)
{
    // There is a potential race condition here, when the destructor of
    // the object being removed needs to call back into the cache and lock
    // the cache (e.g. via an allocator) to update the size of the cache.
    //
    // The shared pointer below fixes this condition by increasing the
    // reference count to the object being deleted, thus delaying the
    // call to the destructor until after the cache is unlocked.
    //
    // In particular, since the cache must be unlocked when data self-destructs
    // data must be defined before cache_lock. Do not change the order of
    // data and cache_lock!
    Data data;

    std::lock_guard<std::mutex> cache_lock(cache_mutex);

    auto map_iter = map.find(key);
    if (map_iter == map.end())
        return false;   //  Key is not in LruCache.

    data = map_iter->second->second;

    decrease_size(data.get());
    list.erase(map_iter->second);
    map.erase(map_iter);
    stats.removes++;

    assert( data.use_count() > 0 );

    // Now, data can go out of scope and if it needs to lock again while
    // deleting the Value object, it can do so.

    return true;
}

template<typename Key, typename Value, typename Hash, typename Eq, typename Purgatory>
bool LruCacheShared<Key, Value, Hash, Eq, Purgatory>::remove(const Key& key, Data& data)
{
    std::lock_guard<std::mutex> cache_lock(cache_mutex);

    auto map_iter = map.find(key);
    if (map_iter == map.end())
        return false;   //  Key is not in LruCache.

    data = map_iter->second->second;

    decrease_size(data.get());
    list.erase(map_iter->second);
    map.erase(map_iter);
    stats.removes++;

    assert( data.use_count() > 0 );

    return true;
}

#endif

lru_segmented_cache_shared.h

Path = src/hash/lru_segmented_cache_shared.h

#ifndef LRU_SEGMENTED_CACHE_SHARED_H
#define LRU_SEGMENTED_CACHE_SHARED_H

#include <algorithm>
#include <cassert>
#include <numeric>
#include <vector>

#include "lru_cache_shared.h"

#define DEFAULT_SEGMENT_COUNT 4

template<typename Key, typename Value, typename Hash = std::hash<Key>, typename Eq = std::equal_to<Key>>
class SegmentedLruCache
{
public:

    using LruCacheType = LruCacheShared<Key, Value, Hash, Eq>;
    using Data = typename LruCacheType::Data;

    SegmentedLruCache(const size_t initial_size, std::size_t segment_count = DEFAULT_SEGMENT_COUNT)
        :segment_count(segment_count)
    {
        assert( segment_count > 0 && ( segment_count & (segment_count - 1)) == 0 );

        segments.resize(segment_count);
        for( auto& segment : segments )
            segment = std::make_unique<LruCacheType>(initial_size/segment_count);

        assert( segment_count == segments.size() );
    }

    virtual ~SegmentedLruCache() = default;

    Data find(const Key& key)
    {
        std::size_t segment_idx = get_segment_idx(key);
        return segments[segment_idx]->find(key);
    }

    Data operator[](const Key& key)
    {
        std::size_t segment_idx = get_segment_idx(key);
        return (*segments[segment_idx])[key];
    }

    bool remove(const Key& key)
    {
        std::size_t segment_idx = get_segment_idx(key);
        return segments[segment_idx]->remove(key);
    }

    bool remove(const Key& key, Data& data)
    {
        std::size_t idx = get_segment_idx(key);
        return segments[idx]->remove(key, data);
    }

    Data find_else_create(const Key& key, bool* new_data)
    {
        std::size_t segment_idx = get_segment_idx(key);
        return segments[segment_idx]->find_else_create(key, new_data);
    }

    bool find_else_insert(const Key& key, std::shared_ptr<Value>& data, bool replace = false)
    {
        std::size_t segment_idx = get_segment_idx(key);
        return segments[segment_idx]->find_else_insert(key, data, replace);
    }

    std::shared_ptr<Value> find_else_insert(const Key& key, std::shared_ptr<Value>& data, LcsInsertStatus* status, bool replace = false)
    {
        std::size_t segment_idx = get_segment_idx(key);
        return segments[segment_idx]->find_else_insert(key, data, status, replace);
    }

    bool set_max_size(size_t max_size)
    {
        bool success = true;
        size_t memcap_per_segment = max_size / segment_count;
        for ( const auto& segment : segments )
        {
            if ( !segment->set_max_size(memcap_per_segment) )
                success = false;
        }
        return success;
    }

    std::vector<std::pair<Key, std::shared_ptr<Value>>> get_all_data()
    {
        std::vector<std::pair<Key, std::shared_ptr<Value>>> all_data;

        for ( const auto& cache : segments )
        {
            auto cache_data = cache->get_all_data();
            all_data.insert(all_data.end(), cache_data.begin(), cache_data.end());
        }
        return all_data;
    }

    size_t mem_size() const
    {
        size_t mem_size = 0;
        for ( const auto& cache : segments )
            mem_size += cache->mem_size();
        return mem_size;
    }

    const PegInfo* get_pegs()
    {
        return lru_cache_shared_peg_names;
    }

    const PegCount* get_counts()
    {
        PegCount* pcs = (PegCount*)&counts;
        const PegInfo* pegs = get_pegs();

        for ( int i = 0; pegs[i].type != CountType::END; i++ )
        {
            PegCount c = 0;
            for ( const auto& cache : segments )
            {
                c += cache->get_counts()[i];
            }
            pcs[i] = c;
        }
        return (const PegCount*)&counts;
    }

    size_t size() const
    {
        size_t total_size = 0;
        for ( const auto& cache : segments )
            total_size += cache->size();
        return total_size;
    }

    size_t get_max_size() const
    {
        size_t max_size = 0;
        for ( const auto& cache : segments )
            max_size += cache->get_max_size();
        return max_size;
    }

    size_t get_segment_count() const
    {
        return segment_count;
    }

protected:
    std::size_t segment_count = DEFAULT_SEGMENT_COUNT;

private:
    std::vector<std::unique_ptr<LruCacheType>> segments;
    struct LruCacheSharedStats counts;

    //derived class can implement their own get_segment_idx if needed
    virtual std::size_t get_segment_idx(Key val)
    {
        if ( segment_count == 1 )
            return 0;
        const uint8_t* bytes = reinterpret_cast<const uint8_t*>(&val);
        uint8_t result = 0;
        for ( size_t i = 0; i < sizeof(Key); ++i )
            result ^= bytes[i];
        // Assumes segment_count is a power of 2 always
        // This is a fast way to do a modulo operation
        return result & (segment_count - 1);
    }
};

#endif // LRU_SEGMENTED_CACHE_SHARED_H

primetable.h

Path = src/hash/primetable.h

#ifndef PRIMETABLE_H
#define PRIMETABLE_H

int nearest_prime(int n);

#endif

xhash.h

Path = src/hash/xhash.h

#ifndef XHASH_H
#define XHASH_H

// generic hash table - stores and maps key + data pairs
// (supports memcap and automatic memory recovery when out of memory)
#include <vector>

#include "framework/counts.h"
#include "main/snort_types.h"
#include "utils/memcap_allocator.h"

class HashLruCache;

namespace snort
{
class HashKeyOperations;
class HashNode;

struct XHashStats
{
    PegCount nodes_created = 0;
    PegCount memcap_prunes = 0;
    PegCount memcap_deletes = 0;
    PegCount release_recycles = 0;
    PegCount release_deletes = 0;
};

class SO_PUBLIC XHash
{
public:
    XHash(int rows, int keysize, uint8_t num_lru_caches = 1);
    XHash(int rows, int keysize, int datasize, unsigned long memcap, uint8_t num_lru_caches = 1);
    virtual ~XHash();

    int insert(const void* key, void* data);
    HashNode* find_node(const void* key);
    HashNode* find_first_node();
    HashNode* find_next_node();
    void* get_user_data();
    void* get_user_data(const void* key, uint8_t type = 0);
    void release(uint8_t type = 0);
    int release_node(const void* key, uint8_t type = 0);
    int release_node(HashNode* node, uint8_t type = 0);
    void* get_mru_user_data(uint8_t type = 0);
    void* get_lru_user_data(uint8_t type = 0);
    bool delete_lru_node(uint8_t type = 0);
    void clear_hash();
    bool full() const { return !fhead; }

    // set max hash nodes, 0 == no limit
    void set_max_nodes(int max)
    { max_nodes = max; }

    unsigned get_num_nodes()
    { return num_nodes; }

    unsigned get_num_free_nodes()
    { return num_free_nodes; }

    void set_memcap(unsigned long memcap)
    { mem_allocator->set_mem_capacity(memcap); }

    unsigned long get_memcap()
    { return mem_allocator->get_mem_capacity(); }

    unsigned long get_mem_used()
    { return mem_allocator->get_mem_allocated(); }

    const XHashStats& get_stats() const
    { return stats; }

    virtual int tune_memory_resources(unsigned work_limit, unsigned& num_freed);

protected:
    void initialize(HashKeyOperations*);
    void initialize();

    void initialize_node(HashNode*, const void* key, void* data, int index, uint8_t type = 0);
    HashNode* allocate_node(const void* key, void* data, int index);
    HashNode* find_node_row(const void* key, int& rindex, uint8_t type = 0);
    void link_node(HashNode*);
    void unlink_node(HashNode*);
    bool delete_a_node();
    void save_free_node(HashNode*);
    HashNode* get_free_node();
    void delete_hash_table();

    virtual bool is_node_recovery_ok(HashNode*)
    { return true; }

    virtual void free_user_data(HashNode*)
    { }

    MemCapAllocator* mem_allocator = nullptr;
    std::vector<HashLruCache*> lru_caches;  // Multiple LRU caches

    unsigned nrows = 0;
    unsigned keysize = 0;
    unsigned num_nodes = 0;
    unsigned num_free_nodes = 0;
    bool recycle_nodes = true;
    bool anr_enabled = true;
    uint8_t num_lru_caches = 1;

private:
    HashNode** table = nullptr;
    HashKeyOperations* hashkey_ops = nullptr;
    HashNode* cursor = nullptr;
    HashNode* fhead = nullptr;
    unsigned datasize = 0;
    unsigned long mem_cap = 0;
    unsigned max_nodes = 0;
    unsigned crow = 0;
    XHashStats stats;

    void set_number_of_rows(int nrows);
    void move_to_front(HashNode*, uint8_t type = 0);
    bool delete_free_node();
    HashNode* release_lru_node(uint8_t type = 0);
    void update_cursor();
    void purge_free_list();
};

} // namespace snort
#endif

zhash.h

Path = src/hash/zhash.h

#ifndef ZHASH_H
#define ZHASH_H

#include <cstddef>

#include "hash/xhash.h"

class ZHash : public snort::XHash
{
public:
    ZHash(int nrows, int keysize, uint8_t lru_count = 1, bool recycle = true);

    ZHash(const ZHash&) = delete;
    ZHash& operator=(const ZHash&) = delete;

    void* push(void* p);
    void* pop();

    void* get(const void* key, uint8_t type = 0);
    void* remove(uint8_t type = 0);

    void* lru_first(uint8_t type = 0);
    void* lru_next(uint8_t type = 0);
    void* lru_current(uint8_t type = 0);
    void lru_touch(uint8_t type = 0);
};

#endif

hash/test/

helpers/

This directory contains new utility classes and methods for use by the framework.

base64_encoder.h

Path = src/helpers/base64_encoder.h

#ifndef BASE64_ENCODER_H
#define BASE64_ENCODER_H

// this is based on the excellent work by devolve found at
// https://sourceforge.net/projects/libb64/.

// usage: instantiate, encode+, finish
// buf must hold 2*length_in

#include <cstdint>
#include "main/snort_types.h"

namespace snort
{
class SO_PUBLIC Base64Encoder
{
public:
    Base64Encoder()
    { reset(); }

    unsigned encode(const uint8_t* plain_text, unsigned length, char* buf);
    unsigned finish(char* buf);

    void reset()
    { step = step_A; state = 0; }

private:
    enum Steps { step_A, step_B, step_C };
    Steps step;
    uint8_t state;
};
}
#endif

bitop.h

Path = src/helpers/bitop.h

#ifndef BITOP_H
#define BITOP_H

// A simple, dynamically sized bit vector implementation

#include <cstdint>
#include <vector>

class BitOp
{
public:
    BitOp(size_t bit)
    { bit_buf.resize(index(bit) + 1); }

    ~BitOp() = default;

    BitOp(const BitOp&) = delete;
    BitOp& operator=(const BitOp&) = delete;

    void set(unsigned int bit);
    bool is_set(unsigned int bit) const;
    void clear(unsigned int bit);

private:
    size_t size() const
    { return bit_buf.size(); }

    size_t index(size_t bit) const
    { return (bit + 7) >> 3; }

    uint8_t& byte(size_t bit)
    { return bit_buf[index(bit)]; }

    uint8_t mask(size_t bit) const
    { return (uint8_t)(0x80 >> (bit & 7)); }

    std::vector<uint8_t> bit_buf;
};

// -----------------------------------------------------------------------------
// implementation
// -----------------------------------------------------------------------------

inline void BitOp::set(unsigned int bit)
{
    if ( index(bit) >= size() )
        bit_buf.resize(index(bit) + 1);
    byte(bit) |= mask(bit);
}

inline bool BitOp::is_set(unsigned int bit) const
{
    if ( index(bit) >= size() )
        return false;
    return bit_buf[index(bit)] & mask(bit);
}

inline void BitOp::clear(unsigned int bit)
{
    if ( index(bit) >= size() )
        return;
    byte(bit) &= ~mask(bit);
}

#endif

boyer_moore_search.h

Path = src/helpers/boyer_moore_search.h

#ifndef BOYER_MOORE_SEARCH_H
#define BOYER_MOORE_SEARCH_H

// Boyer-Moore literal content matching routines (single pattern)
// use LiteralSearch::instantiate to get hyperscan if available

#include "helpers/literal_search.h"
#include "main/snort_types.h"

namespace snort
{

class SO_PUBLIC BoyerMooreSearch : public LiteralSearch
{
protected:
    BoyerMooreSearch(const uint8_t* pattern, unsigned pattern_len);

protected:
    void make_skip();

    const uint8_t* pattern;
    unsigned pattern_len;
    unsigned last;

    unsigned skip[256];
};

class SO_PUBLIC BoyerMooreSearchCase : public BoyerMooreSearch
{
public:
    BoyerMooreSearchCase(const uint8_t* pat, unsigned pat_len) :
        BoyerMooreSearch(pat, pat_len) { }

    int search(const uint8_t* buffer, unsigned buffer_len) const;

    int search(void*, const uint8_t* buffer, unsigned buffer_len) const override
    { return search(buffer, buffer_len); }
};

class SO_PUBLIC BoyerMooreSearchNoCase : public BoyerMooreSearch
{
public:
    BoyerMooreSearchNoCase(const uint8_t* pat, unsigned pat_len) :
        BoyerMooreSearch(pat, pat_len) { }

    int search(const uint8_t* buffer, unsigned buffer_len) const;

    int search(void*, const uint8_t* buffer, unsigned buffer_len) const override
    { return search(buffer, buffer_len); }
};

}
#endif

buffer_data.h

Path = src/helpers/buffer_data.h

#ifndef BUFFER_DATA_H
#define BUFFER_DATA_H

#include <cstdint>

#include "main/snort_types.h"

namespace snort
{
class SO_PUBLIC BufferData
{
public:
    BufferData(int32_t length, const uint8_t* data_, bool own_the_buffer_);
    BufferData() = default;

    ~BufferData();

    int32_t length() const;
    const uint8_t* data_ptr() const;

    void set(int32_t length, const uint8_t* data_, bool own_the_buffer_);

    void reset();

    static const BufferData buffer_null;

private:
    int32_t len = 0;
    const uint8_t* data = nullptr;
    bool own_the_buffer = false;
};
}
#endif

chunk.h

Path = src/helpers/chunk.h

#ifndef CHUNK_H
#define CHUNK_H

// Lua chunk support

#include <string>

// FIXIT-L merge with helpers/lua
bool init_chunk(struct lua_State*, std::string& chunk, const char* name, std::string& args);

#endif

directory.h

Path = src/helpers/directory.h

#ifndef DIRECTORY_H
#define DIRECTORY_H

// simple directory traversal

#include <dirent.h>
#include <string>

class Directory
{
public:
    Directory(const char*, const char* filter = nullptr);
    ~Directory();

    int error_on_open();
    void rewind();
    const char* next();

private:
    DIR* dir;
    std::string root;
    std::string filter;
    std::string path;
    unsigned len;
    Directory* sub;
    int error;
};

#endif

discovery_filter.h

Path = src/helpers/discovery_filter.h

#ifndef DISCOVERY_FILTER_H
#define DISCOVERY_FILTER_H

#include <unordered_map>

#include "protocols/packet.h"
#include "sfip/sf_ipvar.h"
#include "sfip/sf_vartable.h"

enum FilterType { DF_APP, DF_HOST, DF_USER, DF_MAX };
enum FlowCheckDirection { DF_NONE, DF_CLIENT, DF_SERVER };

typedef int32_t IntfType; // matching daq header
#define DF_ANY_INTF INT32_MAX

// Holds configurations to filter traffic discovery based network address, port, and interface
class DiscoveryFilter
{
public:
    DiscoveryFilter(const std::string& conf_path);
    ~DiscoveryFilter();

    // If flag is provided (preferable), results are stored in flag to avoid future lookups
    bool is_app_monitored(const snort::Packet* p, uint8_t* flag = nullptr);
    bool is_host_monitored(const snort::Packet* p, uint8_t* flag = nullptr,
        const snort::SfIp* ip = nullptr, FlowCheckDirection flowdir = FlowCheckDirection::DF_NONE);
    bool is_user_monitored(const snort::Packet* p, uint8_t* flag = nullptr);

private:

    enum Direction { CLIENT, SERVER, NUM_DIRECTIONS };

    bool is_monitored(const snort::Packet* p, FilterType type, uint8_t& flag,
        uint8_t checked, uint8_t monitored, const snort::SfIp* ip = nullptr,
        FlowCheckDirection flowdir = FlowCheckDirection::DF_NONE);
    bool is_monitored(const snort::Packet* p, FilterType type, const snort::SfIp* ip = nullptr,
        FlowCheckDirection flowdir = FlowCheckDirection::DF_NONE);
    void add_ip(FilterType type, IntfType intf, std::string& ip);
    sfip_var_t* get_list(FilterType type, IntfType intf, bool exclude_empty = false);

    // add ip for port exclusion
    void add_ip(Direction dir, uint16_t proto, uint16_t port, const std::string& ip);
    sfip_var_t* get_port_list(Direction dir, uint32_t key);

    inline uint32_t proto_port_key(uint16_t proto, uint16_t port) const
    {
        return (proto << 16) | port;
    }

    bool is_port_excluded(const snort::Packet* p);

    std::unordered_map<IntfType, sfip_var_t*> intf_ip_list[DF_MAX];
    vartable_t* vartable = nullptr;

    // Internal cache for sfip_var_t indexed by protocol x port, for port
    // exclusion.
    std::unordered_map<uint32_t, sfip_var_t*> port_ip_list[NUM_DIRECTIONS];

#ifdef UNIT_TEST
    friend bool is_port_excluded_test(DiscoveryFilter& df, snort::Packet* p);
#endif
};

#endif

flag_context.h

Path = src/helpers/flag_context.h

#ifndef FLAG_CONTEXT_H
#define FLAG_CONTEXT_H

template<typename T>
struct FlagContext
{
    FlagContext(T& dst, T flags) : dst(dst), flags(flags)
    { dst |= flags; }

    ~FlagContext()
    { dst &= ~flags; }

    T& dst;
    T flags;
};

#endif

hyper_scratch_allocator.h

Path = src/helpers/hyper_scratch_allocator.h

#ifndef HYPER_SCRATCH_ALLOCATOR_H
#define HYPER_SCRATCH_ALLOCATOR_H

#include "helpers/scratch_allocator.h"

#include <hs_compile.h>
#include <hs_runtime.h>

#include "main/snort_config.h"
#include "main/snort_types.h"
#include "main/thread.h"

//--------------------------------------------------------------------------
// scratch management
//--------------------------------------------------------------------------

namespace snort
{
struct SnortConfig;

class SO_PUBLIC HyperScratchAllocator : public ScratchAllocator
{
public:
    ~HyperScratchAllocator() override;

    bool setup(SnortConfig*) override;
    void cleanup(SnortConfig*) override;
    void update(SnortConfig*) override
    { }
    bool allocate(hs_database_t*);

    hs_scratch_t* get()
    { return get(SnortConfig::get_conf(), snort::get_instance_id()); }

private:
    hs_scratch_t** get_addr(SnortConfig* sc, unsigned idx)
    { return (hs_scratch_t**)&sc->state[idx][id]; }

    hs_scratch_t* get(const SnortConfig* sc, unsigned idx)
    { return (hs_scratch_t*)sc->state[idx][id]; }

    void set(SnortConfig* sc, unsigned idx, void* pv)
    { sc->state[idx][id] = pv; }

private:
    hs_scratch_t* scratch = nullptr;
};

}
#endif

hyper_search.h

Path = src/helpers/hyper_search.h

#ifndef HYPER_SEARCH_H
#define HYPER_SEARCH_H

// Hyperscan-based literal content matching (single pattern)
// use LiteralSearch::instantiate to fallback to boyer-moore
// if hyperscan is not available.

#include "helpers/literal_search.h"
#include "main/snort_types.h"

extern "C" struct hs_database;

namespace snort
{

class SO_PUBLIC HyperSearch : public snort::LiteralSearch
{
public:
    using Handle = snort::LiteralSearch::Handle;

    static Handle* setup();        // call from module ctor
    static void cleanup(Handle*);  // call from module dtor

    HyperSearch(Handle*, const uint8_t* pattern, unsigned pattern_len, bool no_case = false);
    ~HyperSearch() override;

    int search(Handle*, const uint8_t* buffer, unsigned buffer_len) const override;

private:
    struct hs_database* db;
    unsigned pattern_len;
};

}
#endif

json_stream.h

Path = src/helpers/json_stream.h

#ifndef JSON_STREAM_H
#define JSON_STREAM_H

// Simple output stream for outputting JSON data.

#include <iostream>
#include "main/snort_types.h"

namespace snort
{
class SO_PUBLIC JsonStream
{
public:
    JsonStream(std::ostream& o) : out(o) { }
    ~JsonStream() = default;

    void open(const char* key = nullptr);
    void close();

    void open_array(const char* key = nullptr);
    void close_array();

    void put(const char* key);    // null
    void put(const char* key, int64_t val);
    void uput(const char* key, uint64_t val);
    void put(const char* key, const char* val);
    void put(const char* key, const std::string& val);
    void put(const char* key, double val, int precision);

    void put_true(const char* key);
    void put_false(const char* key);

    void put_eol();

private:
    void split();

private:
    std::ostream& out;
    bool sep = false;
    unsigned level = 0;
    unsigned level_array = 0;
};
}
#endif

literal_search.h

Path = src/helpers/literal_search.h

#ifndef LITERAL_SEARCH_H
#define LITERAL_SEARCH_H

// literal content matching (single pattern)
// used eg with content during signature evaluation

#include "main/snort_types.h"

namespace snort
{

class SO_PUBLIC LiteralSearch
{
public:
    using Handle = void;

    static Handle* setup();        // call from module ctor
    static void cleanup(Handle*);  // call from module dtor

    static LiteralSearch* instantiate(
            Handle*, const uint8_t* pattern, unsigned pattern_len, bool no_case = false, bool hs = false);
    virtual ~LiteralSearch() = default;

    virtual int search(Handle*, const uint8_t* buffer, unsigned buffer_len) const = 0;

protected:
    LiteralSearch() = default;
};

}
#endif

markup.h

Path = src/helpers/markup.h

#ifndef MARKUP_H
#define MARKUP_H

// used to format help and list output for inclusion into user manual

#include <string>

class Markup
{
public:
    static void enable(bool = true);

    static const char* head(unsigned level = 1);
    static const char* item();

    static const char* emphasis_on();
    static const char* emphasis_off();
    static const std::string& emphasis(const std::string&);
    static const std::string& escape(const char* const);
    static const std::string& escape(const std::string&);

private:
    static bool enabled;
};

#endif

process.h

Path = src/helpers/process.h

#ifndef PROCESS_H
#define PROCESS_H

#include "main/snort_types.h"

// process oriented services like signal handling, heap info, etc.

enum PigSignal
{
    PIG_SIG_NONE,
    PIG_SIG_QUIT,
    PIG_SIG_TERM,
    PIG_SIG_INT,
    PIG_SIG_RELOAD_CONFIG,
    PIG_SIG_RELOAD_HOSTS,
    PIG_SIG_DUMP_STATS,
    PIG_SIG_ROTATE_STATS,
    PIG_SIG_MAX
};

PigSignal get_pending_signal();
const char* get_signal_name(PigSignal);

void init_signals();
void term_signals();
SO_PUBLIC void install_oops_handler();
SO_PUBLIC void remove_oops_handler();
void help_signals();

void daemonize();
void set_quick_exit(bool);
void set_main_thread();

void trim_heap();

#endif

ring.h

Path = src/helpers/ring.h

#ifndef RING_H
#define RING_H

// Simple ring implementation

#include "ring_logic.h"

template <typename T>
class Ring
{
public:
    Ring<T>(int size);
    ~Ring<T>();

    Ring<T>(const Ring<T>&) = delete;
    Ring<T>& operator=(const Ring<T>&) = delete;

    T* read();
    bool pop();

    T* write();
    bool push();

    T get(T);
    bool put(T);

    int count();
    bool full();
    bool empty();

private:
    RingLogic logic;
    T* store;
};

template <typename T>
Ring<T>::Ring (int size) : logic(size)
{
    store = new T[size];
}

template <typename T>
Ring<T>::~Ring ()
{
    delete[] store;
}

template <typename T>
T* Ring<T>::read()
{
    int ix = logic.read();
    return (ix < 0) ? nullptr : store + ix;
}

template <typename T>
T* Ring<T>::write()
{
    int ix = logic.write();
    return (ix < 0) ? nullptr : store + ix;
}

template <typename T>
bool Ring<T>::push()
{
    return logic.push();
}

template <typename T>
bool Ring<T>::pop()
{
    return logic.pop();
}

template <typename T>
T Ring<T>::get(T v)
{
    T* p = read();
    if ( !p )
        return v;
    v = *p;
    pop();
    return v;
}

template <typename T>
bool Ring<T>::put(T v)
{
    T* p = write();
    if ( !p )
        return false;
    *p = v;
    push();
    return true;
}

template <typename T>
int Ring<T>::count()
{
    return logic.count();
}

template <typename T>
bool Ring<T>::full()
{
    return logic.full();
}

template <typename T>
bool Ring<T>::empty()
{
    return logic.empty();
}

#endif

ring_logic.h

Path = src/helpers/ring_logic.h

#ifndef RING_LOGIC_H
#define RING_LOGIC_H

// Logic for simple ring implementation

class RingLogic
{
public:
    RingLogic(int size);

    // return next available position or -1
    int read();
    int write();

    // return true if index advanced
    bool push();
    bool pop();

    int count();
    bool full();
    bool empty();

private:
    int next(int ix)
    { return ( ++ix < sz ) ? ix : 0; }

private:
    int sz;
    volatile int rx;
    volatile int wx;
};

inline RingLogic::RingLogic(int size)
{
    sz = size;
    rx = 0;
    wx = 1;
}

inline int RingLogic::read()
{
    int nx = next(rx);
    return ( nx == wx ) ? -1 : nx;
}

inline int RingLogic::write()
{
    int nx = next(wx);
    return ( nx == rx ) ? -1 : wx;
}

inline bool RingLogic::push()
{
    int nx = next(wx);
    if ( nx == rx )
        return false;
    wx = nx;
    return true;
}

inline bool RingLogic::pop()
{
    int nx = next(rx);
    if ( nx == wx )
        return false;
    rx = nx;
    return true;
}

inline int RingLogic::count()
{
    int c = wx - rx - 1;
    if ( c < 0 )
        c += sz;
    return c;
}

inline bool RingLogic::full()
{
    return ( count() == sz );
}

inline bool RingLogic::empty()
{
    return ( count() == 0 );
}

#endif

scratch_allocator.h

Path = src/helpers/scratch_allocator.h

#ifndef SCRATCH_ALLOCATOR_H
#define SCRATCH_ALLOCATOR_H

// manages scratch memory - allocates required memory for each packet thread
// in SnortConfig.state[slot][id] where 0 <= slot < SnortConfig.num_slots and
// id = SnortConfig::request_scratch().  The use of scratch memory is strictly
// per packet, it can not be referenced on a flow, as it will change with each
// config reload.
//
// setup() should return false if no memory was allocated otherwise cleanup()
// will be called when the config is deleted.  this can happen eg if the
// associated module is not used in the current configuration.
//
// scratch allocators may use a prototype to allocate the packet thread
// memory.  the prototype should be freed in setup to avoid leaks and to
// ensure the prototypes for different configs are not interdependent (eg
// preventing a decrease in required scratch).

#include "main/snort_types.h"

namespace snort
{
struct SnortConfig;

class SO_PUBLIC ScratchAllocator
{
public:
    virtual ~ScratchAllocator();

    virtual bool setup(SnortConfig*) = 0;
    virtual void cleanup(SnortConfig*) = 0;
    virtual void update(SnortConfig*) = 0;

    int get_id() { return id; }

protected:
    ScratchAllocator();

    int id;
};

typedef bool (* ScratchSetup)(SnortConfig*);
typedef void (* ScratchCleanup)(SnortConfig*);
typedef void (* ScratchUpdate)(SnortConfig*);

class SO_PUBLIC SimpleScratchAllocator : public ScratchAllocator
{
public:
    SimpleScratchAllocator(ScratchSetup fs, ScratchCleanup fc, ScratchUpdate fu = nullptr)
        : fsetup(fs), fcleanup(fc), fupdate(fu) { }

    bool setup(SnortConfig* sc) override
    { return fsetup(sc); }

    void cleanup(SnortConfig* sc) override
    { fcleanup(sc); }

    void update(SnortConfig* sc) override
    {
        if (fupdate)
            fupdate(sc);
    }

private:
    ScratchSetup fsetup;
    ScratchCleanup fcleanup;
    ScratchUpdate fupdate;
};

}
#endif

sigsafe.h

Path = src/helpers/sigsafe.h

#ifndef SIGSAFE_H
#define SIGSAFE_H

#include <cstddef>
#include <cstdint>

#include "main/snort_types.h"

class SO_PUBLIC SigSafePrinter
{
public:
    SigSafePrinter(char *buf, size_t size);
    SigSafePrinter(int fd) : fd(fd) { }

    void hex_dump(const uint8_t* data, unsigned len);
    void printf(const char* format, ...);

private:
    void write_string(const char* str);

private:
    char* buf = nullptr;
    size_t buf_size = 0;
    size_t buf_idx = 0;
    int fd = -1;
};

#endif

helpers/test/

host_tracker/

This directory contains the implementation of the HostTracker components.

  • The HostTracker object contains information that is known or discovered about a host. It provides an API to get/set host data in a thread-safe manner.

  • The global host_cache is used to cache HostTracker objects so that they can be shared between threads.

    • The host_cache holds a shared_ptr to each HostTracker object. This allows the HostTracker to be removed from the host cache without invalidating the HostTracker held by other threads.

  • The HostTrackerModule is used to read in initial known information about hosts, populate HostTracker objects, and place them in the host_cache.

  • The HostCache object is a thread-safe global LRU cache. The cache is shared between all packet threads. It contains HostTracker objects and provides a way for packet threads to store and retrieve data about hosts as it is discovered. In the long run this cache will replace the current Hosts table and will be the central, shared repository for data about hosts.

  • The HostCacheModule is used to configure the HostCache’s size.

Memory Usage Issues

  • The host cache can grow fairly big, so we want to cap it at a certain size. This size can be set in the lua configuration file as host_cache.memcap and will be read in and honored by host_cache_module.cc.

  • The LruCacheShared class defined in hash/lru_cache_shared.h tracks its memory usage in terms of number of items it contains. That is fine as long as the items in the cache have constant size at run-time, since in that case the memory in bytes is a constant multiple of the number of items.

However, in the case of HostTracker items, the memory usage does not remain constant at run-time. The HostTracker class contains a vector<HostApplication>, which can grow indefinitely as snort discovers more services on a given host.

The LruCacheShared container and the items within know nothing about each other. This independence is desirable and we want to maintain it. However, when an item owned by the cache grows, it must - in good faith - update the cache size, or the cache won’t know that it now owns more memory. This breaks the independence between the cache and its items.

We address this problem by passing a custom allocator to the STL containers that the HostTracker contains - for example, vector<HostApplication>. The allocator gets called by STL whenever the vector requests or releases memory. In turn, the allocator calls back into the global host cache object, updating it with the size that it just allocated or de-allocated. This way, although the HostTracker communicates with the host cache (indirectly, through the allocator), the memory accounting is done automatically by the allocator, transparently to the HostTracker users. The alternative would be that each time new information is added to the HostTracker, the user updates the cache explicitly - which is prone to error.

Every container HostTracker might contain must be instantiated with our custom allocator as a parameter.

Memory Usage vs. Number of Items

In some cases it is preferable that the size of the cache is measured in number of items within the cache, whereas in other cases (HostTracker) we measure the size of the cache in bytes.

Upon careful analysis, the only difference between the two cases is that we must increase/decrease the size by 1 when size is measured in number of items, and by sizeof(Item) when the size is measured in bytes. The rest of the code (insert, prune, remove, etc.) remains the same.

Consequently, we can have the LruCacheShared class measure its size in number of items, and derive from it another cache class - LruCacheSharedMemcap - that measures size in bytes. All we need to do is provide virtual increase_size() / decrease_size() functions in the base class, which will update the size by the appropriate amount in each case. See host_cache.h and hash/lru_cache_shared.h.

The LruCacheShared need not know anything about the custom allocator described above, since it operates under the assumption that its items do not grow at run-time. All size accounting can be done at item insertion time.

The derived LruCacheSharedMemcap, however, must contain an update() function to be used solely by the allocator. The update() function must lock the cache.

The prune(), increase_size() and decrease_size() functions do not lock the cache. They must be called exclusively from functions like insert() or remove(), that do lock. On the other hand, the update() function in the derived cache class has to lock the cache, as it is called asynchronously from different threads (via the allocator).

Allocator Implementation Issues

There is a circular dependency between the HostCache, the HostTracker and the allocator that needs to be broken. This is true in the HostTracker case, but it will be true for any cache item that requires an allocator.

The allocator is ephemeral, it comes into existence inside STL for a brief period of time, allocates/deallocates memory and then it gets destroyed. STL assumes the allocator constructor has no parameters, so we can’t pass a (pointer to a) host cache object to the allocator upon construction. Therefore, the allocator must have an internal host cache pointer that gets set in the constructor to the global host cache instance. Hence, the allocator must know about the host cache.

On the other hand, the host cache must know about its item (the HostTracker) at instantiation time.

Finally, the HostTracker must know about the allocator, so it can inform the cache about memory changes.

This is a circular dependency:

     ------> Allocator ----
    |                      |
    |                      |
    |                      V
HostCache <----------- HostTracker

It is common to implement a class template (like Allocator, in our case) in a single .h file. However, to break this dependency, we have to split the Allocator into a .h and a .cc file. We include the .h file in HostTracker and declare HostCache extern only in the .cc file. Then, we have to include the .cc file also in the HostTracker implementation file because Allocator is templated. See host_cache.h, cache_allocator.h, cache_allocator.cc, host_tracker.h and host_tracker.cc.

Illustrative examples are test/cache_allocator_test.cc (standalone host cache / allocator example) and test/host_cache_allocator_ht_test.cc (host_cache / allocator with host tracker example).

13/08/2023

To address the issue of contention due to mutex locks when Snort is configured to run a large number (over 100) of threads with a single host_cache, we introduced a new layer: "host_cache_segmented". This layer operates on multiple cache segments, thus significantly reducing the locking contention that was previously observed.

The segmented host cache is not a replacement but rather an enhancement layer above the existing host_cache. With this architecture, there can be more than one host_cache, now referred to as a "segment". Each segment functions as an LRU cache, just like the previous singular host_cache. Importantly, there has been no change in the LRU cache design or its logic.

Whenever a new key-data pair is added to a segment, its allocator needs updating. This ensures that the peg counts and visibility metrics are accurate for that specific segment. The find_else_create method of the segmented cache takes care of this, ensuring that each key-data pair is correctly associated with its segment.

Each of these cache segments can operate independently, allowing for more efficient parallel processing. This not only reduces the time threads spend waiting for locks but also better utilizes multi-core systems by allowing simultaneous read and write operations in different cache segments.

The number of segments and the memcap are both configurable, providing flexibility for tuning based on the specific requirements of the deployment environment and the workload. Furthermore, this segmented approach scales well with the increase in the number of threads, making it a robust solution for high-performance, multi-threaded environments.

In summary, the introduction of the "host_cache_segmented" layer represents a significant step forward in the performance and scalability of Snort in multi-threaded environments. This enhancement not only provides immediate benefits in terms of improved throughput but also paves the way for further performance optimizations in the future. ----------------- | Snort Threads | ----------------- | v ------------------------------- | Host Cache Segmented Layer | ------------------------------- | v ------------------------------------------------- | Cache Segment 1 | Cache Segment 2 | … | -------------------------------------------------

cache_allocator.h

Path = src/host_tracker/cache_allocator.h

#ifndef CACHE_ALLOCATOR_H
#define CACHE_ALLOCATOR_H

#include <cassert>

#include "cache_interface.h"

template <class T>
class CacheAlloc : public std::allocator<T>
{
public:

    template <class U>
    struct rebind
    {
        typedef CacheAlloc<U> other;
    };

    T* allocate(std::size_t n);
    void deallocate(T* p, std::size_t n) noexcept;
    void set_lru(CacheInterface* c) { lru = c; }
    CacheInterface* get_lru() const { return lru; }

protected:

    CacheInterface* lru = nullptr;
};

template <class T>
T* CacheAlloc<T>::allocate(std::size_t n)
{
    size_t sz = n * sizeof(T);
    T* out = std::allocator<T>::allocate(n);
    lru->update(sz);
    return out;
}

template <class T>
void CacheAlloc<T>::deallocate(T* p, std::size_t n) noexcept
{
    size_t sz = n * sizeof(T);
    std::allocator<T>::deallocate(p, n);
    lru->update(-(int) sz);
}


// Trivial derived allocator, pointing to their own host cache.
// HostCacheAllocIp has a CacheInterface* pointing to an lru cache
// instantiated using snort::SfIp as the key. See host_cache.h.
// We can create different cache types by instantiating the lru cache using
// different keys and derive here allocators with CacheInterface*
// pointing to the appropriate lru cache object.
template <class T>
class HostCacheAllocIp : public CacheAlloc<T>
{
public:

    using Base = CacheAlloc<T>;
    // This needs to be in every derived class:
    template <class U>
    struct rebind
    {
        typedef HostCacheAllocIp<U> other;
    };

    using CacheAlloc<T>::lru;

    void set_cache(CacheInterface* hci) { Base::set_lru(hci); }
    CacheInterface* get_cache_ptr() { return Base::get_lru(); }

    template <class U>
    HostCacheAllocIp(const HostCacheAllocIp<U>& other)
    {
        this->lru = other.get_lru();
    }

    template <class U>
    HostCacheAllocIp(HostCacheAllocIp<U>&& other)  noexcept
    {
        this->lru = other.get_lru();
    }

    HostCacheAllocIp();

};

#endif

cache_interface.h

Path = src/host_tracker/cache_interface.h

#ifndef CACHE_INTERFACE_H
#define CACHE_INTERFACE_H

class CacheInterface
{
public:

    // This abstract class is the interface of the host cache for the allocator.
    //
    // Only the allocator calls update(), in the derived class. In turn,
    // the allocator is called, for instance, from HostTracker::add_service(),
    // which locks the host tracker but not the cache. Therefore, update()
    // must lock the cache.
    //
    // Note that any cache item object that is not yet owned by the cache
    // will increase / decrease the current_size of the cache any time it
    // adds / removes something to itself. Case in point: HostTracker.
    //
    // Therefore, if the cache items are containers that can grow dynamically,
    // then those items should be added to the cache first, and only accessed
    // via the cache. Then, any size change of the item, will legitimately
    // and correctly update the current_size of the cache.
    //
    // In concrete terms, never have a zombie HostTracker object outside
    // the host cache add or remove stuff to itself, as that will incorrectly
    // change the current_size of the cache.
    virtual void update(int size) = 0;
};

#endif

host_cache.h

Path = src/host_tracker/host_cache.h

#ifndef HOST_CACHE_H
#define HOST_CACHE_H

// The host cache is used to cache information about hosts so that it can
// be shared among threads.

#include <cassert>

#include "hash/lru_cache_shared.h"
#include "host_tracker.h"
#include "log/messages.h"
#include "main/snort_config.h"
#include "sfip/sf_ip.h"
#include "utils/stats.h"

#include "cache_allocator.h"
#include "cache_interface.h"

// Default host cache size in bytes.
#define LRU_CACHE_INITIAL_SIZE 8388608 // 8 MB

// Used to create hash of key for indexing into cache.
//
// Note that both HashIp and IpEqualTo below ignore the IP family.
// This means that 1.2.3.4 and ::ffff:0102:0304 will be treated
// as equal (same host).
struct HashIp
{
    size_t operator()(const snort::SfIp& ip) const
    {
        const uint64_t* ip64 = (const uint64_t*) ip.get_ip6_ptr();
        return std::hash<uint64_t>() (ip64[0]) ^
               std::hash<uint64_t>() (ip64[1]);
    }
};

struct IpEqualTo
{
    bool operator()(const snort::SfIp &lhs, const snort::SfIp &rhs) const
    {
        return lhs.fast_eq6(rhs);
    }
};

template<typename Key, typename Value, typename Hash, typename Eq = std::equal_to<Key>,
    typename Purgatory = std::vector<std::shared_ptr<Value>>>
class LruCacheSharedMemcap : public LruCacheShared<Key, Value, Hash, Eq, Purgatory>,
    public CacheInterface
{
public:
    using LruBase = LruCacheShared<Key, Value, Hash, Eq, Purgatory>;
    using LruBase::cache_mutex;
    using LruBase::current_size;
    using LruBase::list;
    using LruBase::map;
    using LruBase::max_size;
    using LruBase::mem_chunk;
    using LruBase::stats;
    using Data = typename LruBase::Data;
    using LruListIter = typename LruBase::LruListIter;
    using ValueType = typename LruBase::ValueType;

    LruCacheSharedMemcap() = delete;
    LruCacheSharedMemcap(const LruCacheSharedMemcap& arg) = delete;
    LruCacheSharedMemcap& operator=(const LruCacheSharedMemcap& arg) = delete;

    LruCacheSharedMemcap(const size_t sz) : LruCacheShared<Key, Value, Hash, Eq, Purgatory>(sz),
        valid_id(invalid_id+1) {}

    size_t mem_size() override
    {
        return current_size;
    }

    void print_config()
    {
        if ( snort::SnortConfig::log_verbose() )
        {
            snort::LogLabel("host_cache");
            snort::LogMessage("    memcap: %zu bytes\n", max_size.load());
        }

    }

    // If the new memcap causes pruning, don't modify max_size and return true signifying that
    // a gradual pruning/resizing is needed. Otherwise, modify max_size and return false.
    bool reload_resize(size_t new_size)
    {
        if ( current_size > new_size )
            return true;

        max_size = new_size;
        return false;
    }

    // Prune a few entries at each call and set the max_size to the current watermark.
    // Return true when the desired memcap is reached.
    bool reload_prune(size_t new_size, unsigned max_prune)
    {
        std::unique_lock<std::mutex> reload_lock(reload_mutex, std::try_to_lock);
        if ( !reload_lock.owns_lock() )
            return false; // some other thread wins this round

        // Since decrease_size() does not account associated objects in host_tracker,
        // we may over-prune if we remove max_prune entries in a single attempt. Instead,
        // we acquire lock, hold data, release lock, and delete data in each iteration.
        while ( max_prune-- > 0 )
        {
            // Get a local temporary reference of data being deleted (as if a trash can).
            // To avoid race condition, data needs to self-destruct after the cache_lock does.
            Data data;
            std::lock_guard<std::mutex> cache_lock(cache_mutex);

            if ( !list.empty() )
            {
                max_size.store(current_size);
                if ( max_size > new_size )
                {
                    LruListIter list_iter = --list.end();
                    data = list_iter->second; // increase reference count
                    decrease_size();
                    max_size -= mem_chunk; // in sync with current_size
                    map.erase(list_iter->first);
                    list.erase(list_iter);
                    ++stats.reload_prunes;
                }
            }

            if ( max_size <= new_size or list.empty() )
            {
                max_size = new_size;
                return true;
            }
        }

        return false;
    }

    bool is_valid(size_t id) const
    {
        return id == valid_id;
    }

    void invalidate()
    {
        valid_id++;
    }

    size_t get_valid_id() const { return valid_id; }

    static constexpr size_t invalid_id = 0;

    template <class T>
    friend class HostCacheAllocIp;

private:

    // Only the allocator calls this. The allocator, in turn, is called e.g.
    // from HostTracker::add_service(), which locks the host tracker
    // but not the cache. Therefore, update() must lock the cache.
    //
    // Note that any cache item object that is not yet owned by the cache
    // will increase / decrease the current_size of the cache any time it
    // adds / removes something to itself. Case in point: HostTracker.
    //
    // Therefore, if the cache items are containers that can grow dynamically,
    // then those items should be added to the cache first, and only accessed
    // via the cache. Then, any size change of the item, will legitimately
    // and correctly update the current_size of the cache.
    //
    // In concrete terms, never have a standalone HostTracker object outside
    // the host cache add or remove stuff to itself, as that will incorrectly
    // change the current_size of the cache.

    void update(int size) override
    {
        if ( size < 0 )
        {
            assert( current_size >= (size_t) -size);
        }
        if ( (current_size += size) > max_size )
        {
            // Same idea as in LruCacheShared::remove(), use shared pointers
            // to hold the pruned data until after the cache is unlocked.
            // Do not change the order of data and cache_lock, as the data must
            // self destruct after cache_lock.
            Purgatory data;
            std::lock_guard<std::mutex> cache_lock(cache_mutex);
            LruBase::prune(data);
        }
    }

    void increase_size(ValueType* value_ptr=nullptr) override
    {
        UNUSED(value_ptr);
        current_size += mem_chunk;
    }

    void decrease_size(ValueType* value_ptr=nullptr) override
    {
        UNUSED(value_ptr);
        assert( current_size >= mem_chunk );
        current_size -= mem_chunk;
    }

    std::atomic<size_t> valid_id;

    std::mutex reload_mutex;
    friend class TEST_host_cache_module_misc_Test; // for unit test
};


class HTPurgatory
{
public:

    ~HTPurgatory()
    {
        for (auto& ht : data)
        {
            ht->remove_flows();
        }
    }

    bool empty() const {
        return data.empty();
    }

    void emplace_back(std::shared_ptr<snort::HostTracker>& ht)
    {
        data.emplace_back(ht);
    }

    std::vector<std::shared_ptr<snort::HostTracker>> data;
};

typedef LruCacheSharedMemcap<snort::SfIp, snort::HostTracker, HashIp, IpEqualTo, HTPurgatory>
    HostCacheIpSpec;

// Since the LruCacheShared and LruCacheSharedMemcap templates make no
// assumptions about the item, we have to derive our host cache
// from the specialization, if we want to make use of things within the item.
class HostCacheIp : public HostCacheIpSpec
{
public:
    HostCacheIp(const size_t initial_size) : HostCacheIpSpec(initial_size) { }

    bool remove(const KeyType& key) override
    {
        LruBase::Data data;
        return remove(key, data);
    }

    bool remove(const KeyType& key, LruBase::Data& data) override
    {
        bool out = LruBase::remove(key, data);
        data->remove_flows();
        return out;
    }
};


#endif

host_cache_module.h

Path = src/host_tracker/host_cache_module.h

#ifndef HOST_CACHE_MODULE_H
#define HOST_CACHE_MODULE_H

//  Loads host cache configuration data.

#include <string>

#include "framework/module.h"
#include "main/snort.h"
#include "main/reload_tuner.h"
#include "trace/trace_api.h"

#include "host_cache.h"
#include "host_cache_segmented.h"

#define HOST_CACHE_NAME "host_cache"
#define HOST_CACHE_HELP "global LRU cache of host_tracker data about hosts"

class HostCacheReloadTuner : public snort::ReloadResourceTuner
{
public:
    explicit HostCacheReloadTuner(size_t memcap) : memcap(memcap) { }
    bool tinit() override
    { return host_cache.reload_resize(memcap); }

    bool tune_idle_context() override
    { return host_cache.reload_prune(memcap, max_work_idle); }

    bool tune_packet_context() override
    { return host_cache.reload_prune(memcap, max_work); }

private:
    size_t memcap;
};

class HostCacheModule : public snort::Module
{
public:
    HostCacheModule();
    ~HostCacheModule() override;

    bool end(const char*, int, snort::SnortConfig*) override;
    bool set(const char*, snort::Value&, snort::SnortConfig*) override;

    const snort::Command* get_commands() const override;
    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    void prep_counts(bool) override;
    void sum_stats(bool) override;
    void reset_stats() override;

    // in sum_stats, just populate the counts vector with whatever we have now
    bool global_stats() const override
    { return true; }

    Usage get_usage() const override
    { return GLOBAL; }

    void log_host_cache(const char* file_name, bool verbose = false);
    std::string get_host_cache_stats();
    std::string get_host_cache_segment_stats(int seg_idx);

    void set_trace(const snort::Trace*) const override;
    const snort::TraceOption* get_trace_options() const override;

private:
    std::string dump_file;
    size_t memcap = 0;
    uint8_t segments = 1;
};
extern THREAD_LOCAL const snort::Trace* host_cache_trace;

#endif

host_cache_segmented.h

Path = src/host_tracker/host_cache_segmented.h

#ifndef HOST_CACHE_SEGMENTED_H
#define HOST_CACHE_SEGMENTED_H

#ifdef HAVE_CONFIG_H
#include "config.h"
#endif

#include <atomic>
#include <cassert>
#include <numeric>

#include "host_cache.h"
#include "log/messages.h"

#define DEFAULT_HOST_CACHE_SEGMENTS 4

extern SO_PUBLIC HostCacheIp default_host_cache;
extern THREAD_LOCAL struct LruCacheSharedStats host_cache_counts;

template<typename Key, typename Value>
class HostCacheSegmented
{
public:
    HostCacheSegmented() :
        segment_count(DEFAULT_HOST_CACHE_SEGMENTS),
        memcap_per_segment(LRU_CACHE_INITIAL_SIZE) { }
    HostCacheSegmented(uint8_t segment_count, size_t memcap_per_segment);

    void init();
    void term();
    void setup(uint8_t , size_t );

    const PegInfo* get_pegs() { return lru_cache_shared_peg_names; }
    size_t get_memcap_per_segment() { return memcap_per_segment.load(); }
    size_t get_valid_id(uint8_t idx);
    uint8_t get_segments() { return segment_count; }
    size_t get_max_size();
    size_t get_mem_chunk();
    PegCount* get_counts();

    void set_segments(uint8_t segments) { segment_count = segments; }
    void print_config();
    bool set_max_size(size_t max_size);
    bool reload_resize(size_t memcap_per_segment);
    bool reload_prune(size_t new_size, unsigned max_prune);
    void invalidate();
    void update_counts();
    void reset_counts();

    std::shared_ptr<Value> operator[](const Key& key);

    uint8_t get_segment_idx(Key val);
    std::shared_ptr<Value> find(const Key& key);
    std::shared_ptr<Value> find_else_create(const Key& key, bool* new_data);
    std::vector<std::pair<Key, std::shared_ptr<Value>>> get_all_data();
    bool find_else_insert(const Key& key, std::shared_ptr<Value>& value);
    bool remove(const Key& key);
    bool remove(const Key& key, typename LruCacheSharedMemcap
        <snort::SfIp, snort::HostTracker, HashIp, IpEqualTo, HTPurgatory>::Data& data);
    size_t mem_size();

    std::vector<HostCacheIp*> seg_list;
    HostCacheIp* default_cache = &default_host_cache; // Default cache used for host tracker

private:
    uint8_t segment_count;
    std::atomic<size_t> memcap_per_segment;
    struct LruCacheSharedStats counts;
    bool init_done = false;
    std::mutex stats_lock;
};


template<typename Key, typename Value>
HostCacheSegmented<Key, Value>::HostCacheSegmented(uint8_t segment_count, size_t memcap_per_segment) :
    segment_count(segment_count),
    memcap_per_segment(memcap_per_segment)
{
    assert(segment_count > 0);

    for (size_t i = 0; i < this->segment_count; ++i)
    {
        auto cache = new HostCacheIp(this->memcap_per_segment);
        seg_list.emplace_back((HostCacheIp*)cache);
    }
    init_done = true;
}

template<typename Key, typename Value>
void HostCacheSegmented<Key, Value>::init()
{
    if(init_done or seg_list.size() >= segment_count)
        return;

    assert(segment_count > 0);

    for (size_t i = 0; i < segment_count; ++i)
    {
        auto cache = new HostCacheIp(memcap_per_segment.load());
        seg_list.emplace_back((HostCacheIp*)cache);
    }
    init_done = true;
}

template<typename Key, typename Value>
void HostCacheSegmented<Key, Value>::term()
{
    for (auto cache : seg_list)
    {
        if (cache)
            delete cache;
    }
}

template<typename Key, typename Value>
void HostCacheSegmented<Key, Value>::setup(uint8_t segs, size_t memcap )
{
    assert(segment_count > 0);

    segment_count = segs;
    memcap_per_segment = memcap/segs;
    set_max_size(memcap);
}

template<typename Key, typename Value>
size_t HostCacheSegmented<Key, Value>::get_valid_id(uint8_t idx)
{
    if(idx < seg_list.size())
        return seg_list[idx]->get_valid_id();
    return 0;
}

template<typename Key, typename Value>
void HostCacheSegmented<Key, Value>::print_config()
{
    if ( snort::SnortConfig::log_verbose() )
    {
        snort::LogLabel("host_cache");
        snort::LogMessage("    memcap: %zu bytes\n", get_max_size());
    }
}

template<typename Key, typename Value>
std::shared_ptr<Value> HostCacheSegmented<Key, Value>::operator[](const Key& key)
{
    return find_else_create(key, nullptr);
}

/**
 * Sets the maximum size for the entire cache, which is distributed equally
 * among all the segments.
 */
template<typename Key, typename Value>
bool HostCacheSegmented<Key, Value>::set_max_size(size_t max_size)
{
    bool success = true;
    memcap_per_segment = max_size/segment_count;
    for (auto cache : seg_list)
    {
        if (!cache->set_max_size(memcap_per_segment))
            success = false;
    }
    return success;
}

/**
 * Resize the cache based on the provided memory capacity, distributing the
 * memory equally among all the segments. If any segment fails to resize,
 * the operation is considered unsuccessful.
 */
template<typename Key, typename Value>
bool HostCacheSegmented<Key, Value>::reload_resize(size_t memcap)
{
    bool success = true;
    memcap_per_segment = memcap/segment_count;
    for (auto cache : seg_list)
    {
        if (!cache->reload_resize(memcap_per_segment.load()))
            success = false;
    }
    return success;
}

// Computes the index of the segment where a given key-value pair belongs.
template<typename Key, typename Value>
uint8_t HostCacheSegmented<Key, Value>::get_segment_idx(Key val)
{
    const uint8_t* bytes = reinterpret_cast<const uint8_t*>(&val);
    uint8_t result = 0;
    for (size_t i = 0; i < sizeof(Key); ++i)
        result ^= bytes[i];
    //Assumes segment_count is a power of 2 always
    //This is a fast way to do a modulo operation
    return result & (segment_count - 1);
}

//Retrieves all the data stored across all the segments of the cache.
template<typename Key, typename Value>
std::vector<std::pair<Key, std::shared_ptr<Value>>>  HostCacheSegmented<Key,Value>::get_all_data()
{
    std::vector<std::pair<Key, std::shared_ptr<Value>>> all_data;

    for (auto cache : seg_list)
    {
        auto cache_data = cache->get_all_data();
        all_data.insert(all_data.end(), cache_data.begin(), cache_data.end());
    }
    return all_data;
}

template<typename Key, typename Value>
std::shared_ptr<Value> HostCacheSegmented<Key, Value>::find(const Key& key)
{
    uint8_t idx = get_segment_idx(key);
    return seg_list[idx]->find(key);
}

/**
 * Updates the internal counts of the host cache. This method aggregates the
 * counts from all segments and updates the overall counts for the cache.
 */
template<typename Key, typename Value>
void HostCacheSegmented<Key, Value>::update_counts()
{
    std::lock_guard<std::mutex> guard(stats_lock);
    PegCount* pcs = (PegCount*)&counts;
    const PegInfo* pegs = get_pegs();

    for (int i = 0; pegs[i].type != CountType::END; i++)
        pcs[i] = 0;

    for (auto cache : seg_list)
    {
        const PegCount* cache_counts = cache->get_counts();
        cache->lock();
        cache->stats.bytes_in_use = cache->current_size;
        cache->stats.items_in_use = cache->list.size();
        for (int i = 0; pegs[i].type != CountType::END; i++)
            pcs[i] += cache_counts[i];
        cache->unlock();
    }
    host_cache_counts = counts;
}

template<typename Key, typename Value>
void HostCacheSegmented<Key, Value>::reset_counts()
{
    std::lock_guard<std::mutex> guard(stats_lock);
    const PegInfo* pegs = get_pegs();

    for (auto cache : seg_list)
    {
        PegCount* cache_counts = reinterpret_cast<PegCount*> (&cache->stats);
        cache->lock();
        for (int i = 0; pegs[i].type != CountType::END; i++)
            cache_counts[i] = 0;
        cache->unlock();
    }
}


template<typename Key, typename Value>
std::shared_ptr<Value> HostCacheSegmented<Key, Value>:: find_else_create(const Key& key, bool* new_data)
{
    // Determine the segment index where the key-value pair resides or should reside
    uint8_t idx = get_segment_idx(key);
    bool new_data_local = false;

    // Retrieve or create the entry for the key in the determined segment
    auto ht = seg_list[idx]->find_else_create(key, &new_data_local);
    if(new_data_local)
    {
        // If a new entry was created, update its cache interface and visibility
        ht->update_cache_interface(idx);
        ht->init_visibility(seg_list[idx]->get_valid_id());
    }
    if(new_data)
        *new_data = new_data_local;
    return ht;
}

template<typename Key, typename Value>
bool HostCacheSegmented<Key, Value>::find_else_insert(const Key& key, std::shared_ptr<Value>& value)
{
    uint8_t idx = get_segment_idx(key);
    return seg_list[idx]->find_else_insert(key, value, false);
}

template<typename Key, typename Value>
PegCount* HostCacheSegmented<Key, Value>::get_counts()
{
    if( init_done )
        update_counts();

    return (PegCount*)&host_cache_counts;
}

template<typename Key, typename Value>
void HostCacheSegmented<Key, Value>::invalidate()
{
    for( auto cache: seg_list)
    {
        cache->invalidate();
    }
}

template<typename Key, typename Value>
bool HostCacheSegmented<Key, Value>::reload_prune(size_t new_size, unsigned max_prune)
{
    bool success = true;
    memcap_per_segment = new_size/segment_count;
    for (auto cache : seg_list)
    {
        if (!cache->reload_prune(memcap_per_segment, max_prune))
            success = false;
    }
    return success;
}

template<typename Key, typename Value>
size_t HostCacheSegmented<Key, Value>::mem_size()
{
    size_t mem_size = 0;
    for (auto cache : seg_list)
    {
        if(cache)
            mem_size += cache->mem_size();
    }
    return mem_size;
}

template<typename Key, typename Value>
size_t HostCacheSegmented<Key, Value>::get_max_size()
{
    size_t max_size = 0;
    for (auto cache : seg_list)
        max_size += cache->get_max_size();
    return max_size;
}

template<typename Key, typename Value>
size_t HostCacheSegmented<Key, Value>::get_mem_chunk()
{
    //Assumes all segments have the same mem_chunk
    return seg_list[0]->mem_chunk;
}

template<typename Key, typename Value>
bool HostCacheSegmented<Key, Value>::remove(const Key& key)
{
    uint8_t idx = get_segment_idx(key);
    return seg_list[idx]->remove(key);
}

template<typename Key, typename Value>
bool HostCacheSegmented<Key, Value>::remove(const Key& key, typename LruCacheSharedMemcap<snort::SfIp, snort::HostTracker, HashIp, IpEqualTo, HTPurgatory>::Data& data)
{
    uint8_t idx = get_segment_idx(key);
    return seg_list[idx]->remove(key, data);
}

/*
Warning!!!: update_allocator and update_set_allocator don't copy data to old container
but erase it for speed. Use with care!!!
*/
template <template <typename, typename...> class Container, typename T, typename Alloc>
void update_allocator(Container<T, Alloc>& cont, CacheInterface* new_lru)
{
    Alloc new_allocator;
    new_allocator.set_cache(new_lru);
    cont = std::move(Container<T, Alloc>(new_allocator));
}

template <template <typename, typename, typename...> class Container, typename T, typename Comp, typename Alloc>
void update_set_allocator(Container<T, Comp, Alloc>& cont, CacheInterface* new_lru)
{
    Alloc new_allocator;
    new_allocator.set_cache(new_lru);
    cont = std::move(Container<T, Comp, Alloc> (new_allocator));
}


typedef HostCacheSegmented<snort::SfIp, snort::HostTracker> HostCacheSegmentedIp;
extern SO_PUBLIC HostCacheSegmentedIp host_cache;

#endif // HOST_CACHE_SEGMENTED_H

host_tracker.h

Path = src/host_tracker/host_tracker.h

#ifndef HOST_TRACKER_H
#define HOST_TRACKER_H

// The HostTracker class holds information known about a host (may be from
// configuration or dynamic discovery).  It provides a thread-safe API to
// set/get the host data.

#include <cstring>
#include <mutex>
#include <list>
#include <set>
#include <unordered_set>
#include <vector>

#include "framework/counts.h"
#include "main/snort_types.h"
#include "main/thread.h"
#include "network_inspectors/appid/application_ids.h"
#include "protocols/protocol_ids.h"
#include "protocols/vlan.h"
#include "time/packet_time.h"

#include "cache_allocator.h"

struct HostTrackerStats
{
    PegCount service_adds;
    PegCount service_finds;
};

extern THREAD_LOCAL struct HostTrackerStats host_tracker_stats;

class RNAFlow;

namespace snort
{
#define INFO_SIZE 32
#define MAC_SIZE 6
extern const uint8_t zero_mac[MAC_SIZE];

struct HostMac
{
    HostMac() : ttl(0), primary(0), last_seen(0)
    { memset(mac, 0, MAC_SIZE); }

    HostMac(uint8_t p_ttl, const uint8_t* p_mac, uint8_t p_primary, uint32_t p_last_seen)
        : ttl(p_ttl), primary(p_primary), last_seen (p_last_seen) { memcpy(mac, p_mac, MAC_SIZE); }

    // the type and order below should match logger's serialization
    uint8_t ttl;
    uint8_t mac[MAC_SIZE];
    uint8_t primary;
    uint32_t last_seen;
};

struct HostApplicationInfo
{
    HostApplicationInfo() = default;
    HostApplicationInfo(const char *ver, const char *ven);
    char vendor[INFO_SIZE] = { '\0' };
    char version[INFO_SIZE] = { '\0' };

    friend class HostTracker;
private:
    bool visibility = true;
};

typedef HostCacheAllocIp<HostApplicationInfo> HostAppInfoAllocator;
typedef std::pair<AppId, bool> Payload_t;
typedef std::vector<Payload_t, HostCacheAllocIp<Payload_t>> PayloadVector;

struct HostApplication
{
    HostApplication() = default;
    HostApplication(Port pt, IpProtocol pr, AppId ap, bool in, uint32_t ht = 0, uint32_t ls = 0,
        bool banner = false) : port(pt), proto(pr), appid(ap), inferred_appid(in), hits(ht),
        last_seen(ls), banner_updated(banner) { }
    HostApplication(const HostApplication& ha): port(ha.port), proto(ha.proto), appid(ha.appid),
        inferred_appid(ha.inferred_appid), hits(ha.hits), last_seen(ha.last_seen),
        num_visible_payloads(ha.num_visible_payloads), info(ha.info), payloads(ha.payloads),
        visibility(ha.visibility) { }

    HostApplication& operator=(const HostApplication& ha)
    {
        port = ha.port;
        proto = ha.proto;
        appid = ha.appid;
        inferred_appid = ha.inferred_appid;
        hits = ha.hits;
        last_seen = ha.last_seen;
        info = ha.info;
        payloads = ha.payloads;
        visibility = ha.visibility;
        banner_updated = ha.banner_updated;
        num_visible_payloads = ha.num_visible_payloads;
        memcpy(user, ha.user, sizeof(user));
        user_login = ha.user_login;
        return *this;
    }

    Port port = 0;
    IpProtocol proto = IpProtocol::PROTO_NOT_SET;
    AppId appid = APP_ID_NONE;
    bool inferred_appid = false;
    uint32_t hits = 0;
    uint32_t last_seen = 0;
    char user[INFO_SIZE] = { '\0' };
    uint8_t user_login = 0;
    bool banner_updated = false;
    size_t num_visible_payloads = 0;

    std::vector<HostApplicationInfo, HostAppInfoAllocator> info;
    PayloadVector payloads;

    friend class HostTracker;

// visibility is public in UT only, to avoid extra lock/unlock funcs used only by UT
#ifndef UNIT_TEST
private:
#endif
    bool visibility = true;

};

struct HostClient
{
    HostClient() = default;
    HostClient(AppId clientid, const char *ver, AppId ser);
    AppId id = APP_ID_NONE;
    char version[INFO_SIZE] = { '\0' };
    AppId service = APP_ID_NONE;
    PayloadVector payloads;
    size_t num_visible_payloads = 0;

    bool operator==(const HostClient& c) const
    {
        return id == c.id and service == c.service;
    }

    friend class HostTracker;

private:
    bool visibility = true;
};

struct DeviceFingerprint
{
    DeviceFingerprint(uint32_t id, uint32_t type, bool jb, const char* dev);
    uint32_t fpid;
    uint32_t fp_type;
    bool jail_broken;
    char device[INFO_SIZE] = { '\0' };
};

enum HostType : std::uint32_t
{
    HOST_TYPE_HOST = 0,
    HOST_TYPE_ROUTER,
    HOST_TYPE_BRIDGE,
    HOST_TYPE_NAT,
    HOST_TYPE_LB
};

class HostMac_t : public HostMac
{
public:
    HostMac_t(uint8_t p_ttl, const uint8_t* p_mac, uint8_t p_primary, uint32_t p_last_seen)
        : HostMac(p_ttl, p_mac, p_primary, p_last_seen) {}

    HostMac_t& operator=(const HostMac_t& hm)
    {
        ttl = hm.ttl;
        primary = hm.primary;
        last_seen = hm.last_seen;
        visibility = hm.visibility;
        memcpy(mac, hm.mac, MAC_SIZE);
        return *this;
    }

    bool visibility = true;
};

#define MIN_BOOT_TIME    10
#define MIN_TTL_DIFF     16

typedef HostCacheAllocIp<HostApplication> HostAppAllocator;
typedef HostCacheAllocIp<HostClient> HostClientAllocator;
typedef HostCacheAllocIp<DeviceFingerprint> HostDeviceFpAllocator;

class SO_PUBLIC HostTracker
{
public:

    typedef std::pair<uint16_t, bool> NetProto_t;
    typedef std::pair<uint8_t, bool> XProto_t;

    HostTracker();

    void update_last_seen();
    uint32_t get_last_seen() const
    {
        std::lock_guard<std::mutex> lck(host_tracker_lock);
        return last_seen;
    }

    void update_last_event(uint32_t time = 0);
    uint32_t get_last_event() const
    {
        std::lock_guard<std::mutex> lck(host_tracker_lock);
        return last_event;
    }

    std::vector<uint16_t> get_network_protos()
    {
        std::vector<uint16_t> out_protos;
        std::lock_guard<std::mutex> lck(host_tracker_lock);
        for (const auto& proto : network_protos)
            if ( proto.second )
                out_protos.emplace_back(proto.first);
        return out_protos;
    }

    std::vector<uint16_t> get_xport_protos()
    {
        std::vector<uint16_t> out_protos;
        std::lock_guard<std::mutex> lck(host_tracker_lock);
        for (const auto& proto : xport_protos)
            if ( proto.second )
                out_protos.emplace_back(proto.first);
        return out_protos;
    }

    void set_host_type(HostType rht)
    {
        std::lock_guard<std::mutex> lck(host_tracker_lock);
        host_type = rht;
    }

    HostType get_host_type() const
    {
        std::lock_guard<std::mutex> lck(host_tracker_lock);
        return host_type;
    }

    uint8_t get_hops()
    {
        std::lock_guard<std::mutex> lck(host_tracker_lock);
        return hops;
    }

    void update_hops(uint8_t h)
    {
        std::lock_guard<std::mutex> lck(host_tracker_lock);
        hops = h;
    }

    bool add_client_payload(HostClient&, AppId, size_t);

    // Returns true if a new mac entry is added, false otherwise
    bool add_mac(const uint8_t* mac, uint8_t ttl, uint8_t primary);

    // Returns true if a mac entry TTL is updated and decreased, false otherwise
    bool update_mac_ttl(const uint8_t* mac, uint8_t new_ttl);

    // Returns true if we changed primary (false->true), false otherwise
    bool make_primary(const uint8_t* mac);

    // Returns true if a new payload entry added, false otherwise
    bool add_payload(HostApplication&, Port, IpProtocol, const AppId payload,
        const AppId service, size_t max_payloads);

    // Returns true after resetting hops if there is a primary mac
    bool reset_hops_if_primary();

    // Returns true and copy of the matching HostMac, false if no match...
    bool get_hostmac(const uint8_t* mac, HostMac& hm);

    const uint8_t* get_last_seen_mac(uint8_t*);

    void update_vlan(uint16_t vth_pri_cfi_vlan, uint16_t vth_proto);
    bool has_same_vlan(uint16_t);
    void get_vlan_details(uint8_t& cfi, uint8_t& priority, uint16_t& vid);

    // The caller owns and deletes the copied list of mac addresses
    void copy_data(uint8_t& p_hops, uint32_t& p_last_seen, std::list<HostMac>*& p_macs);

    bool add_network_proto(const uint16_t type);
    bool add_xport_proto(const uint8_t type);

    // Appid may not be identified always. Inferred means dynamic/runtime
    // appid detected from one flow to another flow such as BitTorrent.
    bool add_service(Port, IpProtocol,
        AppId appid = APP_ID_NONE, bool inferred_appid = false, bool* added = nullptr);
    bool add_service(const HostApplication&, bool* added = nullptr);
    void clear_service(HostApplication&);
    void update_service_port(HostApplication&, Port);
    void update_service_proto(HostApplication&, IpProtocol);

    AppId get_appid(Port, IpProtocol, bool inferred_only = false,
        bool allow_port_wildcard = false);

    size_t get_service_count();

    HostApplication add_service(Port, IpProtocol, uint32_t, bool&, AppId appid = APP_ID_NONE);

    void update_service(const HostApplication&);
    bool update_service_info(HostApplication&, const char* vendor, const char* version,
        uint16_t max_info);
    bool update_service_banner(Port, IpProtocol);
    bool update_service_user(Port, IpProtocol, const char* username, uint32_t lseen,
        uint16_t max_services, bool success);
    void remove_inferred_services();

    size_t get_client_count();
    HostClient find_or_add_client(AppId id, const char* version, AppId service,
        bool& is_new);
    bool add_tcp_fingerprint(uint32_t fpid);
    bool add_ua_fingerprint(uint32_t fpid, uint32_t fp_type, bool jail_broken,
        const char* device_info, uint8_t max_devices);
    bool add_udp_fingerprint(uint32_t fpid);
    bool add_smb_fingerprint(uint32_t fpid);

    bool add_cpe_os_hash(uint32_t hash);

    //  This should be updated whenever HostTracker data members are changed
    void stringify(std::string& str);

    uint8_t get_ip_ttl() const
    {
        std::lock_guard<std::mutex> lck(host_tracker_lock);
        return ip_ttl;
    }

    void set_ip_ttl(uint8_t ttl)
    {
        std::lock_guard<std::mutex> lck(host_tracker_lock);
        ip_ttl = ttl;
    }

    uint32_t get_nat_count_start() const
    {
        std::lock_guard<std::mutex> lck(host_tracker_lock);
        return nat_count_start;
    }

    void set_nat_count_start(uint32_t natCountStart)
    {
        std::lock_guard<std::mutex> lck(host_tracker_lock);
        nat_count_start = natCountStart;
    }

    uint32_t get_nat_count() const
    {
        std::lock_guard<std::mutex> lck(host_tracker_lock);
        return nat_count;
    }

    void set_nat_count(uint32_t v = 0)
    {
        std::lock_guard<std::mutex> lck(host_tracker_lock);
        nat_count = v;
    }

    uint32_t inc_nat_count()
    {
        std::lock_guard<std::mutex> lck(host_tracker_lock);
        return ++nat_count;
    }

    void set_cache_idx(uint8_t idx)
    {
        std::lock_guard<std::mutex> lck(host_tracker_lock);
        cache_idx = idx;
    }

    void init_visibility(size_t v)
    {
        std::lock_guard<std::mutex> lck(host_tracker_lock);
        visibility = v;
    }

    uint8_t get_cache_idx() const
    {
        return cache_idx;
    }

    bool set_netbios_name(const char*);

    bool set_visibility(bool v = true);
    size_t get_visibility() const {return visibility;}


    bool is_visible() const;

    // the control delete commands do not actually remove objects from
    // the host tracker, but just mark them as invisible, until rediscovered.
    bool set_network_proto_visibility(uint16_t proto, bool v = true);
    bool set_xproto_visibility(uint8_t proto, bool v = true);
    bool set_service_visibility(Port, IpProtocol, bool v = true);
    bool set_client_visibility(const HostClient&, bool v = true);

#ifdef UNIT_TEST
    // Caller is responsible for checking visibility
    std::vector<HostApplication, HostAppAllocator> get_services()
    {
        std::lock_guard<std::mutex> lck(host_tracker_lock);
        return services;
    }

    // Caller is responsible for checking visibility
    std::vector<HostClient, HostClientAllocator> get_clients()
    {
        std::lock_guard<std::mutex> lck(host_tracker_lock);
        return clients;
    }
#endif

    void add_flow(RNAFlow*);
    void remove_flows();
    void remove_flow(RNAFlow*);

    void update_cache_interface( uint8_t idx );
    CacheInterface * get_cache_interface() { return cache_interface; }

private:

    mutable std::mutex host_tracker_lock; // ensure that updates to a shared object are safe
    mutable std::mutex flows_lock;        // protect the flows set separately
    uint8_t hops = ~0;                 // hops from the snort inspector, e.g., zero for ARP
    uint32_t last_seen;           // the last time this host was seen
    uint32_t last_event = ~0;          // the last time an event was generated

    // list guarantees iterator validity on insertion
    std::list<HostMac_t, HostCacheAllocIp<HostMac_t>> macs;
    std::vector<NetProto_t, HostCacheAllocIp<NetProto_t>> network_protos;
    std::vector<XProto_t, HostCacheAllocIp<XProto_t>> xport_protos;
    std::vector<HostApplication, HostAppAllocator> services;
    std::vector<HostClient, HostClientAllocator> clients;
    std::set<uint32_t, std::less<uint32_t>, HostCacheAllocIp<uint32_t>> tcp_fpids;
    std::set<uint32_t, std::less<uint32_t>, HostCacheAllocIp<uint32_t>> udp_fpids;
    std::set<uint32_t, std::less<uint32_t>, HostCacheAllocIp<uint32_t>> smb_fpids;
    std::set<uint32_t, std::less<uint32_t>, HostCacheAllocIp<uint32_t>> cpe_fpids;
    std::vector<DeviceFingerprint, HostDeviceFpAllocator> ua_fps;
    std::string netbios_name;

    // flows that we belong to
    std::unordered_set<RNAFlow*> flows;

    bool vlan_tag_present = false;
    vlan::VlanTagHdr vlan_tag = {};
    HostType host_type = HOST_TYPE_HOST;
    uint8_t ip_ttl = 0;
    uint32_t nat_count = 0;
    uint32_t nat_count_start;     // the time nat counting starts for this host

    size_t visibility;
    uint8_t cache_idx = 0;

    uint32_t num_visible_services = 0;
    uint32_t num_visible_clients = 0;
    uint32_t num_visible_macs = 0;

    CacheInterface * cache_interface = nullptr;

    // These three do not lock independently; they are used by payload discovery and called
    // from add_payload(HostApplication&, Port, IpProtocol, AppId, AppId, size_t); where the
    // lock is actually obtained
    bool add_payload_no_lock(const AppId, HostApplication*, size_t);
    HostApplication* find_service_no_lock(Port, IpProtocol, AppId);
    void update_ha_no_lock(HostApplication& dst, HostApplication& src);

    HostApplication* find_and_add_service_no_lock(Port, IpProtocol, uint32_t lseen,
        bool& is_new, AppId, uint16_t max_services = 0);

    // Sets all payloads visible or invisible
    void set_payload_visibility_no_lock(PayloadVector& pv, bool v, size_t& num_vis);

    // Hide / delete the constructor from the outside world. We don't want to
    // have zombie host trackers, i.e. host tracker objects that live outside
    // the host cache.
    HostTracker( const HostTracker& ) = delete;
    HostTracker( const HostTracker&& ) = delete;

    HostTracker& operator=( const HostTracker& ) = delete;
    HostTracker& operator=( const HostTracker&& ) = delete;

    // Only the host cache can create them ...
    template<class Key, class Value, class Hash>
    friend class LruCacheShared;

    // ... and some unit tests. See Utest.h and UtestMacros.h in cpputest.
    friend class TEST_host_tracker_add_find_service_test_Test;
    friend class TEST_host_tracker_stringify_Test;
};
} // namespace snort
#endif

host_tracker_module.h

Path = src/host_tracker/host_tracker_module.h

#ifndef HOST_TRACKER_MODULE_H
#define HOST_TRACKER_MODULE_H

//  Loads host configuration data.

// FIXIT-M temporarily create new HostTracker module to test new
// HostTracker object.  May eventually replace old Hosts module with this
// one.

#include <cassert>
#include <vector>

#include "framework/module.h"
#include "host_tracker/cache_allocator.cc"
#include "host_tracker/host_cache.h"

#define host_tracker_help \
    "configure hosts"
#define HOST_TRACKER_NAME "host_tracker"

class HostTrackerModule : public snort::Module
{
public:
    HostTrackerModule() :
        snort::Module(HOST_TRACKER_NAME, host_tracker_help, host_tracker_params, true) { }

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    void init_data();

    Usage get_usage() const override
    { return GLOBAL; }

private:
    static const snort::Parameter host_tracker_params[];
    static const snort::Parameter service_params[];

    std::vector<snort::HostApplication> apps;
    snort::HostApplication app;
    snort::SfIp addr = {};
};

#endif

host_tracker/test/

ips_options/

Standard IPS rule option such as "content", "pcre", "flowbits" etc.. (non-preprocessor specific) implemented as IpsOption subclasses.

Most of the IpsOptions can be built statically or dynamically. Several, however, such as content, are still tightly coupled with the code and can only be built statically. The code will hopefully evolve and eliminate these cases.

IPS options are not guaranteed to get all they need for evaluation. For example, a packet may not have a flow assigned to it, or the flow may not have a gadget present. Not all options need specific inputs, so, if one does, that option should validate input values. Service-only options may not be guaranteed to get a gadget on the flow.

Several options use RangeCheck to implement upper and/or lower bound semantics. The Snort 2X options had various implementations of ranges so 3X differs in some places.

The "regex" and "sd_pattern" options both use hyperscan for pattern matching. Hyperscan is an "optional" dependency for Snort3; These rule options will not exist without satisfying that dependency.

Hyperscan documentation can be found online https://intel.github.io/hyperscan/dev-reference

The "sd_pattern" will be used as a fast pattern in the future (like "regex") for performance.

"replace" option has the following restrictions: - Content and replacement are aligned to the right side of the matching content and are limited not by the size of the matching content, but by the boundaries of the packet. - Be aware that after the match there should be enough room left for the "replace" content in the matched packet. If there is not enough space for the "replace" content the rule will not match. - Only the first occurrence of the content will be replaced. - "replace" works for raw packets only. So, TCP data must either fit under the "pkt_data" buffer requirements or one should enable detection on TCP payload before reassembly: search_engine.detect_raw_tcp=true.

"js_data" option is used by Enhanced JavaScript Normalizer to access normalized data. It’s implemented as a generic IPS buffer and follows the JIT approach. "js_data" buffer must be specified in the list of buffers available for a particular inspector.

extract.h

Path = src/ips_options/extract.h

#ifndef EXTRACT_H
#define EXTRACT_H

#include "framework/cursor.h"
#include "framework/endianness.h"
#include "main/snort_types.h"
#include "main/thread.h"
#include "protocols/packet.h"

#define ENDIAN_BIG    0x1
#define ENDIAN_LITTLE 0x2
#define ENDIAN_FUNC   0x4

#define PARSELEN      10
#define MAX_BYTES_TO_GRAB 4

#define NUM_IPS_OPTIONS_VARS 2
#define IPS_OPTIONS_NO_VAR (-1)
#define INVALID_VAR_ERR_STR "%s uses an undefined rule option variable (%s)"

namespace snort
{

struct ByteData
{
    uint32_t base;
    uint32_t bitmask_val;
    uint32_t bytes_to_extract;
    int32_t offset;
    uint8_t endianness;
    bool relative_flag;
    bool string_convert_flag;
};

SO_PUBLIC int string_extract(
    int bytes_to_grab, int base, const uint8_t* ptr,
    const uint8_t* start, const uint8_t* end, uint32_t* value);

SO_PUBLIC int byte_extract(
    int endianness, int bytes_to_grab, const uint8_t* ptr,
    const uint8_t* start, const uint8_t* end, uint32_t* value);

void set_cursor_bounds(const ByteData& settings, const Cursor& c,
    const uint8_t*& start, const uint8_t*& ptr, const uint8_t*& end);

int32_t data_extraction(const ByteData& settings, Packet* p,
    uint32_t& result_var, const uint8_t* start,
    const uint8_t* ptr, const uint8_t* end);

int32_t extract_data(const ByteData& settings, const Cursor& c, Packet* p,
    uint32_t& result_var);

SO_PUBLIC void set_byte_order(uint8_t& order, uint8_t flag, const char* opt);

SO_PUBLIC uint32_t getNumberTailingZerosInBitmask(uint32_t);
SO_PUBLIC uint8_t numBytesInBitmask(uint32_t);

// Used during parsing
SO_PUBLIC int8_t GetVarByName(const char* name);
SO_PUBLIC int8_t AddVarNameToList(const char* name);
// Called at the end of rule parsing
SO_PUBLIC void ClearIpsOptionsVars();
// Used during eval
SO_PUBLIC int GetVarValueByIndex(uint32_t* dst, uint8_t var_number);
SO_PUBLIC int SetVarValueByIndex(uint32_t value, uint8_t var_number);
}
#endif

ips_flowbits.h

Path = src/ips_options/ips_flowbits.h

#ifndef IPS_FLOWBITS_H
#define IPS_FLOWBITS_H

#include <string>
#include <vector>

bool flowbits_setter(void*);
void get_flowbits_dependencies(void*, bool& set, std::vector<std::string>& bits);
void flowbits_counts(unsigned& total, unsigned& unchecked, unsigned& unset);

#endif

ips_options.h

Path = src/ips_options/ips_options.h

#ifndef IPS_OPTIONS_H
#define IPS_OPTIONS_H

void load_ips_options();

#endif

sd_credit_card.h

Path = src/ips_options/sd_credit_card.h

#ifndef SD_CREDIT_CARD_H
#define SD_CREDIT_CARD_H

#include <cstdint>

int SdLuhnAlgorithm(const uint8_t *buf, unsigned long long buflen);

#endif

ips_options/test/

js_norm/

Enhanced JavaScript Normalizer is a stateful JavaScript whitespace and identifiers normalizer. JSNorm is a basic implementation, so other modules can use it right away or provide some customization to it. Normalizer will remove all extraneous whitespace and newlines, keeping a single space where syntactically necessary. Comments will be removed, but contents of string literals will be kept intact. Any string literals, added by the plus operator, will be concatenated. This also works for functions that result in string literals. Semicolons will be inserted, if not already present, according to ECMAScript automatic semicolon insertion rules.

All JavaScript identifier names, except those from the ident_ignore or prop_ignore lists, will be substituted with unified names in the following format: var_0000 → var_ffff. So, the number of unique identifiers available is 65536 names per transaction. If Normalizer overruns the configured limit, built-in alert is generated.

A config option to set the limit manually: * js_norm.identifier_depth

Identifiers from the ident_ignore list will be placed as is, without substitution. Starting with the listed identifier, any chain of dot accessors, brackets and function calls will be kept intact. For example: * console.log("bar") * document.getElementById("id").text * eval("script") * foo["bar"]

Ignored identifiers are configured via the following config option, it accepts a list of object and function names: * js_norm.ident_ignore = { console, document, eval, foo }

When a variable assignment that aliases an identifier from the list is found, the assignment will be tracked, and subsequent occurrences of the variable will be replaced with the stored value. This substitution will follow JavaScript variable scope limits.

For example:

var a = console.log
a("hello")  // will be substituted to 'console.log("hello")'
a.foo.bar() // will be normalized as 'console.log.foo.bar()'. When variable is 'de-aliased',
            // following identifiers are not normalized, just like identifiers from ident_ignore

When an object is created using a new keyword, and the class/constructor is found in ident_ignore list, the object will be tracked, and although its own identifier will be converted to normal form its property and function calls will be kept intact, as with ignored identifiers.

For example: var obj = new Array() obj.insert(1,2,3) // will be normalized to var_0000.insert(1,2,3)

For properties and methods of objects that can be created implicitly, there is a prop_ignore list. All names in the call chain after the first property or method from the list has been occurred will not be normalized.

Note that identifiers are normalized by name, i.e. an identifier and a property with the same name will be normalized to the same value. However, the ignore lists act separately on identifiers and properties.

For example:

js_norm.prop_ignore = { 'split' }
in: "string".toUpperCase().split("").reverse().join("");
out: "string".var_0000().split("").reverse().join("");

In addition to the scope tracking, JS Normalizer specifically tracks unescape-like JavaScript functions (unescape, decodeURI, decodeURIComponent, String.fromCharCode, String.fromCodePoint). This allows detection of unescape functions nested within other unescape functions, which is a potential indicator of a multilevel obfuscation. The definition of a function call depends on identifier substitution, so such identifiers must be included in the ignore list in order to use this feature. After determining the unescape sequence, it is decoded into the corresponding string, and the name of unescape function will not be present in the output. Single-byte escape sequences within the string and template literals which are arguments of unescape, decodeURI and decodeURIComponent functions will be decoded according to ISO/IEC 8859-1 (Latin-1) charset. Except these cases, escape sequences and code points will be decoded to UTF-8 format.

For example:

unescape('\u0062\u0061\u0072')              -> 'bar'
decodeURI('%62%61%72')                      -> 'bar'
decodeURIComponent('\x62\x61\x72')          -> 'bar'
String.fromCharCode(98, 0x0061, 0x72)       -> 'bar'
String.fromCodePoint(65600, 65601, 0x10042) -> '𐁀𐁁𐁂'

Supported formats follow

\xXX
\uXXXX
\u{XXXX}
%XX
\uXX
%uXXXX
decimal code point
hexadecimal code point

JS Normalizer is able to decode mixed encoding sequences. However, a built-in alert rises in such case.

JS Normalizer’s syntax parser follows ECMA-262 standard. For various features, tracking of variable scope and individual brackets is done in accordance to the standard. Additionally, Normalizer enforces standard limits on HTML content in JavaScript: * no nesting tags allowed, i.e. two opening tags in a row * script closing tag is not allowed in string literals, block comments, regular expression literals, etc.

If source JavaScript is syntactically incorrect (containing a bad token, brackets mismatch, HTML-tags, etc) Normalizer fires corresponding built-in rule and abandons the current script, though the already-processed data remains in the output buffer.

Enhanced JavaScript Normalizer has some trace messages available. Trace options follow:

  • trace.module.js_norm.proc turns on messages from script processing flow.

    Verbosity levels:

    1. Script opening tag detected (available in release build)

    2. Attributes of detected script (available in release build)

    3. Normalizer return code (available in release build)

    4. Contexts management (debug build only)

    5. Parser states (debug build only)

    6. Input stream states (debug build only)

  • trace.module.js_norm.dump dumps JavaScript data from processing layers.

    Verbosity levels:

    1. js_data buffer as it is being passed to detection (available in release build)

    2. (no messages available currently)

    3. Payload passed to Normalizer (available in release build)

    4. Temporary buffer (debug build only)

    5. Matched token (debug build only)

    6. Identifier substitution (debug build only)

PDF parser follows "PDF 32000-1:2008 First Edition 2008-7-1 Document management Portable document format Part 1: PDF 1.7". Known limitations: * Nested dictionaries are not fully supported. Properties of the last object are tracked. Once the nested object ends, it clears all info about the object type. * Nested dictionaries are not allowed in JavaScript-type dictionary. * JavaScript in streams is tracked only when a reference to that stream is found earlier in that file. * Compressed JavaScript streams are handled correctly only if PDF decompression is enabled (http_inspect.decompress_pdf = true, and the same option for other inspectors)

js_config.h

Path = src/js_norm/js_config.h

#ifndef JS_CONFIG_H
#define JS_CONFIG_H

#include <string>
#include <unordered_set>

struct JSNormConfig
{
    int64_t bytes_depth = -1;
    int32_t identifier_depth = 0xffff;
    uint8_t max_template_nesting = 32;
    uint32_t max_bracket_depth = 256;
    uint32_t max_scope_depth = 256;
    std::unordered_set<std::string> ignored_ids;
    std::unordered_set<std::string> ignored_props;
};

#endif

js_enum.h

Path = src/js_norm/js_enum.h

#ifndef JS_ENUM_H
#define JS_ENUM_H

#include "utils/event_gen.h"

namespace jsn
{

static constexpr unsigned js_gid = 154;

enum
{
    TRACE_PROC = 0,
    TRACE_DUMP,
    TRACE_PDF_PROC,
    TRACE_PDF_DUMP
};

// This enum must be synchronized with JSNormModule::peg_names[] in js_norm_module.cc
enum PEG_COUNT
{
    PEG_BYTES = 0,
    PEG_IDENTIFIERS,
    PEG_IDENTIFIER_OVERFLOWS,
    PEG_COUNT_MAX
};

// This enum must be synchronized with JSNormModule::events[] in js_norm_module.cc
enum EventSid
{
    EVENT__NONE = -1,
    EVENT_NEST_UNESCAPE_FUNC = 1,
    EVENT_MIXED_UNESCAPE_SEQUENCE = 2,
    EVENT_BAD_TOKEN = 3,
    EVENT_OPENING_TAG = 4,
    EVENT_CLOSING_TAG = 5,
    EVENT_IDENTIFIER_OVERFLOW = 6,
    EVENT_BRACKET_NEST_OVERFLOW = 7,
    EVENT_DATA_LOST = 8,
    EVENT_SCOPE_NEST_OVERFLOW = 9,
    EVENT__MAX_VALUE
};

}

using JSEvents = EventGen<jsn::EVENT__MAX_VALUE, jsn::EVENT__NONE, jsn::js_gid>;

#endif

js_identifier_ctx.h

Path = src/js_norm/js_identifier_ctx.h

#ifndef JS_IDENTIFIER_CTX
#define JS_IDENTIFIER_CTX

#include <cstdint>
#include <list>
#include <string>
#include <unordered_map>
#include <unordered_set>
#include <vector>

namespace jsn
{

enum JSProgramScopeType : unsigned int
{
    GLOBAL = 0,     // the global scope (the initial one)
    FUNCTION,       // function declaration
    BLOCK,          // block of code and object declaration
    PROG_SCOPE_TYPE_MAX
};

class JSIdentifier
{
public:
    virtual ~JSIdentifier() = default;

    virtual const char* substitute(const char* identifier, bool is_property) = 0;
    virtual void add_alias(const char* alias, const std::string&& value) = 0;
    virtual const char* alias_lookup(const char* alias) const = 0;
    virtual bool is_ignored(const char* identifier) const = 0;

    virtual bool scope_push(JSProgramScopeType) = 0;
    virtual bool scope_pop(JSProgramScopeType) = 0;

    virtual void reset() = 0;
};

class JSIdentifierCtx : public JSIdentifier
{
public:
    JSIdentifierCtx(int32_t depth, uint32_t max_scope_depth,
        const std::unordered_set<std::string>& ignored_ids_list,
        const std::unordered_set<std::string>& ignored_props_list);

    virtual const char* substitute(const char* identifier, bool is_property) override;
    virtual void add_alias(const char* alias, const std::string&& value) override;
    virtual const char* alias_lookup(const char* alias) const override;
    virtual bool is_ignored(const char* identifier) const override;

    virtual bool scope_push(JSProgramScopeType) override;
    virtual bool scope_pop(JSProgramScopeType) override;

    virtual void reset() override;

private:

    struct NormId
    {
        const char* id_name = nullptr;
        const char* prop_name = nullptr;
        uint8_t type = 0;
    };

    using Alias = std::vector<std::string>;
    using AliasRef = std::list<Alias*>;
    using AliasMap = std::unordered_map<std::string, Alias>;
    using NameMap = std::unordered_map<std::string, NormId>;

    class ProgramScope
    {
    public:
        ProgramScope(JSProgramScopeType t) : t(t)
        {}

        ~ProgramScope()
        { for (auto a : to_remove) a->pop_back(); }

        void reference(Alias& a)
        { to_remove.push_back(&a); }

        JSProgramScopeType type() const
        { return t; }

    private:
        JSProgramScopeType t;
        AliasRef to_remove{};
    };

    inline const char* substitute(unsigned char c, bool is_property);
    inline bool is_substituted(const NormId& id, bool is_property);
    inline const char* acquire_norm_name(NormId& id);
    inline void init_ignored_names();

    // do not swap next two lines, the destructor frees them in the reverse order
    AliasMap aliases;
    std::list<ProgramScope> scopes;

    NormId id_fast[256];
    NameMap id_names;
    const std::unordered_set<std::string>& ignored_ids_list;
    const std::unordered_set<std::string>& ignored_props_list;

    const char* norm_name;
    const char* norm_name_end;
    uint32_t max_scope_depth;

// advanced program scope access for testing
#if defined(CATCH_TEST_BUILD) || defined(BENCHMARK_TEST)
public:
    // compare scope list with the passed pattern
    bool scope_check(const std::list<JSProgramScopeType>& compare) const;
    const std::list<JSProgramScopeType> get_types() const;
#endif // CATCH_TEST_BUILD
};

}

#endif // JS_IDENTIFIER_CTX

js_normalizer.h

Path = src/js_norm/js_normalizer.h

#ifndef JS_NORMALIZER_H
#define JS_NORMALIZER_H

#include "main/snort_types.h"

#include <FlexLexer.h>

#include "js_tokenizer.h"

#include "utils/streambuf.h"

namespace jsn
{

class JSNormalizer
{
public:
    JSNormalizer(JSIdentifier& js_ident_ctx, size_t depth,
        uint8_t max_template_nesting, uint32_t max_bracket_depth,
        int tmp_cap_size = JSTOKENIZER_BUF_MAX_SIZE);
    ~JSNormalizer();

    JSTokenizer::JSRet normalize(const char* src, size_t src_len,
        bool external_script = false);

    const char* get_src_next() const
    { return src_next; }

    void reset_depth()
    { rem_bytes = depth; }

    const char* take_script()
    { tokenizer.reset_output(); return out_buf.take_data(); }

    const char* get_script() const
    { return out_buf.data(); }

    size_t script_size()
    { return out.tellp(); }

    bool is_unescape_nesting_seen() const
    { return tokenizer.is_unescape_nesting_seen(); }

    bool is_mixed_encoding_seen() const
    { return tokenizer.is_mixed_encoding_seen(); }

    bool is_opening_tag_seen() const
    { return tokenizer.is_opening_tag_seen(); }

    bool is_closing_tag_seen() const
    { return tokenizer.is_closing_tag_seen(); }

    bool is_buffer_adjusted() const
    { return tokenizer.is_buffer_adjusted(); }

#if defined(CATCH_TEST_BUILD) || defined(BENCHMARK_TEST)
    const char* get_tmp_buf() const
    { return tmp_buf; }
    size_t get_tmp_buf_size() const
    { return tmp_buf_size; }
    const JSTokenizer& get_tokenizer() const
    { return tokenizer; }
#endif // CATCH_TEST_BUILD || BENCHMARK_TEST

#ifdef BENCHMARK_TEST
    void rewind_output()
    { out_buf.pubseekoff(0, std::ios_base::beg, std::ios_base::out); }
#endif

private:
    size_t depth;
    size_t rem_bytes;
    bool unlim;
    const char* src_next;

    char* tmp_buf;
    size_t tmp_buf_size;

    snort::istreambuf_glue in_buf;
    snort::ostreambuf_infl out_buf;
    std::istream in;
    std::ostream out;
    JSTokenizer tokenizer;
};

}

#endif //JS_NORMALIZER_H

js_norm.h

Path = src/js_norm/js_norm.h

#ifndef JS_NORM_H
#define JS_NORM_H

#include "utils/event_gen.h"

#include "js_config.h"
#include "js_enum.h"

namespace jsn
{
class JSIdentifier;
class JSNormalizer;

const char* ret2str(int);
}

namespace snort
{

class SO_PUBLIC JSNorm
{
public:
    JSNorm(JSNormConfig*, bool ext_script_type = false, uint32_t generation_id = 0);
    JSNorm(const JSNorm&) = delete;
    virtual ~JSNorm();

    void tick()
    { ++pdu_cnt; }

    void normalize(const void*, size_t, const void*&, size_t&);
    void get_data(const void*&, size_t&);
    void flush_data(const void*&, size_t&);
    void flush_data();

    uint32_t get_generation_id() const
    { return generation_id; }

protected:
    virtual bool pre_proc();
    virtual bool post_proc(int);

    bool alive;
    uint32_t pdu_cnt;

    const uint8_t* src_ptr;
    const uint8_t* src_end;

    jsn::JSIdentifier* idn_ctx;
    jsn::JSNormalizer* jsn_ctx;
    bool ext_script_type;

    JSEvents events;
    JSNormConfig* config;
    uint32_t generation_id;
};

}

#endif

js_norm_module.h

Path = src/js_norm/js_norm_module.h

#ifndef JS_NORM_MODULE_H
#define JS_NORM_MODULE_H

#include "framework/module.h"
#include "main/policy.h"
#include "profiler/profiler.h"

#include "js_config.h"
#include "js_enum.h"

namespace snort
{
class Trace;
}

extern THREAD_LOCAL const snort::Trace* js_trace;

class JSNormModule : public snort::Module
{
public:
    JSNormModule();
    ~JSNormModule() override;

    bool begin(const char*, int, snort::SnortConfig*) override;
    bool set(const char*, snort::Value&, snort::SnortConfig*) override;

    void set_trace(const snort::Trace*) const override;
    const snort::TraceOption* get_trace_options() const override;

    unsigned get_gid() const override;

    const snort::RuleMap* get_rules() const override
    { return events; }

    const PegInfo* get_pegs() const override
    { return peg_names; }

    PegCount* get_counts() const override
    { return peg_counts; }

    snort::ProfileStats* get_profile() const override
    { return &profile_stats; }

    Usage get_usage() const override
    { return INSPECT; }

    static void increment_peg_counts(jsn::PEG_COUNT counter)
    { peg_counts[counter]++; }

    static void increment_peg_counts(jsn::PEG_COUNT counter, uint64_t value)
    { peg_counts[counter] += value; }

    static PegCount get_peg_counts(jsn::PEG_COUNT counter)
    { return peg_counts[counter]; }

private:
    static const snort::Parameter params[];
    static const snort::RuleMap events[];
    static const PegInfo peg_names[];

    static THREAD_LOCAL PegCount peg_counts[];
    static THREAD_LOCAL snort::ProfileStats profile_stats;

    JSNormConfig* config;
};

#endif

js_pdf_norm.h

Path = src/js_norm/js_pdf_norm.h

#ifndef JS_PDF_NORM_H
#define JS_PDF_NORM_H

#include <FlexLexer.h>
#include <cstring>

#include "js_norm/js_norm.h"
#include "js_norm/pdf_tokenizer.h"
#include "utils/streambuf.h"

namespace snort
{

class SO_PUBLIC PDFJSNorm : public JSNorm
{
public:
    static bool is_pdf(const void* data, size_t len)
    {
        constexpr char magic[] = "%PDF-1.";
        constexpr int magic_len = sizeof(magic) - 1;
        return magic_len < len and !strncmp((const char*)data, magic, magic_len);
    }

    PDFJSNorm(JSNormConfig* cfg, uint32_t gen_id) :
        JSNorm(cfg, false, gen_id),
        pdf_in(&buf_pdf_in), pdf_out(&buf_pdf_out), extractor(pdf_in, pdf_out)
    { }

protected:
    bool pre_proc() override;
    bool post_proc(int) override;

private:
    snort::istreambuf_glue buf_pdf_in;
    snort::ostreambuf_infl buf_pdf_out;
    std::istream pdf_in;
    std::ostream pdf_out;
    jsn::PDFTokenizer extractor;
};

}

#endif

js_tokenizer.h

Path = src/js_norm/js_tokenizer.h

#ifndef JS_TOKENIZER_H
#define JS_TOKENIZER_H

#include <array>
#include <sstream>
#include <stack>
#include <vector>

// The longest pattern has 9 characters " < / s c r i p t > ",
// 8 of them can reside in 1st chunk
// Each character in the identifier forms its own group (pattern matching case),
// i.e. in the current implementation IDENTIFIER has " . " rule.
#define JSTOKENIZER_MAX_STATES 8

// To hold potentially long identifiers
#define JSTOKENIZER_BUF_MAX_SIZE 256

namespace jsn
{

enum JSProgramScopeType : unsigned int;

class JSIdentifier;
#if defined(CATCH_TEST_BUILD) || defined(BENCHMARK_TEST)
class JSTokenizerTester;
class JSTestConfig;
#endif // CATCH_TEST_BUILD || BENCHMARK_TEST

class SO_PUBLIC JSTokenizer : public yyFlexLexer
{
private:
    enum JSToken
    {
        UNDEFINED = 0,
        IDENTIFIER,
        KEYWORD,
        PUNCTUATOR,
        OPERATOR,
        LITERAL,
        DIRECTIVE,
        DOT,
        COLON,
        CLOSING_BRACKET,
        KEYWORD_VAR_DECL,             // var, let, const
        KEYWORD_FUNCTION,
        KEYWORD_BLOCK,                // for all block-definition keywords e.g. if, else, for, etc.
        KEYWORD_CLASS,
        OPERATOR_ASSIGNMENT,
        OPERATOR_COMPLEX_ASSIGNMENT,
        OPERATOR_COMPARISON,
        OPERATOR_LOGICAL,
        OPERATOR_SHIFT
    };

    enum ScopeType
    {
        GLOBAL = 0,  // not in the brackets (the initial one)
        BRACES,      // {}
        PARENTHESES, // ()
        BRACKETS     // []
    };
    enum ScopeMetaType
    {
        NOT_SET = 0,
        ARROW_FUNCTION, // arrow function
        FUNCTION,       // function
        BLOCK,          // if, else, for, while, do, with, switch, try, catch, finally, block of code
        OBJECT,         // object definition, class definition
        SCOPE_META_TYPE_MAX
    };
    enum FuncType
    {
        NOT_FUNC = 0,
        GENERAL,
        UNESCAPE,
        CHAR_CODE
    };
    struct Scope
    {
        Scope(ScopeType t) :
            type(t), meta_type(ScopeMetaType::NOT_SET), func_call_type(FuncType::NOT_FUNC),
            ident_norm(true), block_param(false), do_loop(false), encoding(0), char_code_str(false),
            in_object(false)
        {}

        ScopeType type;
        ScopeMetaType meta_type;
        FuncType func_call_type;
        bool ident_norm;
        bool block_param;
        bool do_loop;
        uint32_t encoding;
        bool char_code_str;
        bool in_object;
    };

    enum ASIGroup
    {
        ASI_OTHER = 0,
        ASI_GROUP_1,    // {
        ASI_GROUP_2,    // }
        ASI_GROUP_3,    // [ (
        ASI_GROUP_4,    // ]
        ASI_GROUP_5,    // )
        ASI_GROUP_6,    // + -
        ASI_GROUP_7,    // this true false null identifier literal
                        //IDENTIFIER + LITERAL + KEYWORD_LITERAL
        ASI_GROUP_8,    // ++ --
        ASI_GROUP_9,    // continue break return debugger // same as KEYWORD_BA
        ASI_GROUP_10,   // var function new delete void typeof if do while for with
                        // switch throw try ~ +
        ASI_GROUP_MAX
    };

    enum AliasState
    {
        ALIAS_NONE = 0,
        ALIAS_DEFINITION, // var a
        ALIAS_PREFIX,     // var a +%possible PDU split%
                          // to handle ambiguity between a++, a+=, and a + b
        ALIAS_EQUALS,     // var a =
        ALIAS_NEW,        // var a = new
        ALIAS_VALUE       // var a = eval
    };

    template <class T>
    using VStack = std::stack<T, std::vector<T>>;

public:
    enum JSRet
    {
        EOS = 0,
        SCRIPT_ENDED,
        SCRIPT_CONTINUE,
        CLOSING_TAG,
        BAD_TOKEN,
        IDENTIFIER_OVERFLOW,
        TEMPLATE_NESTING_OVERFLOW,
        BRACKET_NESTING_OVERFLOW,
        SCOPE_NESTING_OVERFLOW,
        WRONG_CLOSING_SYMBOL,
        ENDED_IN_INNER_SCOPE,
        MAX
    };

    JSTokenizer() = delete;
    explicit JSTokenizer(std::istream& in, std::ostream& out, JSIdentifier& ident_ctx,
        uint8_t max_template_nesting, uint32_t max_bracket_depth, char*& buf, size_t& buf_size,
        int cap_size = JSTOKENIZER_BUF_MAX_SIZE);
    ~JSTokenizer() override;

    JSRet process(size_t& bytes_in, bool external_script = false);

    void reset_output()
    { ignored_id_pos = -1; }

    bool is_unescape_nesting_seen() const;
    bool is_mixed_encoding_seen() const;
    bool is_opening_tag_seen() const;
    bool is_closing_tag_seen() const;
    bool is_buffer_adjusted() const;

private:
    int yylex() override;

    void switch_to_initial();
    void switch_to_temporal(const std::string& data);
    JSRet eval_eof();
    JSRet do_spacing(JSToken cur_token);
    JSRet do_operator_spacing();
    JSRet do_semicolon_insertion(ASIGroup current);
    JSRet do_identifier_substitution(const char* lexeme, bool id_part);
    JSRet push_identifier(const char* ident);
    bool unescape(const char* lexeme);
    bool concatenate();
    void process_punctuator(JSToken tok = PUNCTUATOR);
    void skip_punctuator();
    void process_closing_brace();
    JSRet process_subst_open();

    bool states_process();
    void states_correct(int);
    void states_reset();
    void states_over();
    void states_adjust();

    // scope stack servicing
    JSRet scope_push(ScopeType);
    JSRet scope_pop(ScopeType);
    Scope& scope_cur();

    // program scope stack servicing
    JSRet p_scope_push(ScopeMetaType);
    JSRet p_scope_pop(ScopeMetaType);

    // interactions with the current scope
    bool global_scope();
    void set_meta_type(ScopeMetaType);
    ScopeMetaType meta_type();
    void set_ident_norm(bool);
    bool ident_norm();
    void set_func_call_type(FuncType);
    FuncType func_call_type();
    FuncType detect_func_type();
    void check_function_nesting(FuncType);
    void check_mixed_encoding(uint32_t);
    void set_block_param(bool);
    bool block_param();
    void set_do_loop(bool);
    bool do_loop();

    void set_encoding(uint32_t f)
    { scope_cur().encoding |= f; }

    uint32_t encoding()
    { return scope_cur().encoding; }

    void set_char_code_str(bool f)
    { scope_cur().char_code_str = f; }

    bool char_code_str()
    { return scope_cur().char_code_str; }

    void set_in_object(bool f)
    { scope_cur().in_object = f; }

    bool in_object()
    { return scope_cur().in_object; }

    static JSProgramScopeType m2p(ScopeMetaType);
    static const char* m2str(ScopeMetaType);
    static bool is_operator(JSToken);

    void dealias_clear_mutated(bool id_part);
    void dealias_increment();
    void dealias_identifier(bool id_part, bool assignment_start);
    void dealias_reset();
    void dealias_prefix_reset();
    void dealias_equals(bool complex);
    void dealias_append();
    void dealias_finalize();

    //rule handlers
    JSRet html_closing_script_tag();
    JSRet literal_dq_string_start();
    JSRet literal_sq_string_start();
    JSRet literal_template_start();
    JSRet literal_regex_start();
    JSRet literal_regex_end();
    JSRet literal_regex_g_open();
    JSRet literal_regex_g_close();
    void div_assignment_operator();
    JSRet open_brace();
    JSRet close_brace();
    JSRet open_parenthesis();
    JSRet close_parenthesis();
    JSRet open_bracket();
    JSRet close_bracket();
    JSRet punctuator_prefix();
    void dot_accessor();
    JSRet punctuator_arrow();
    JSRet punctuator_semicolon();
    void punctuator_colon();
    void operator_comparison();
    void operator_complex_assignment();
    void operator_logical();
    void operator_shift();
    void punctuator_comma();
    JSRet use_strict_directive();
    JSRet use_strict_directive_sc();
    JSRet keyword_var_decl();
    JSRet keyword_function();
    JSRet keyword_catch();
    JSRet keyword_while();
    JSRet keyword_B();
    JSRet keyword_new();
    JSRet keyword_BA();
    JSRet keyword_finally();
    JSRet keyword_do();
    JSRet keyword_class();
    JSRet keyword_other();
    void operator_assignment();
    JSRet operator_prefix();
    JSRet operator_incr_decr();
    JSRet general_operator();
    JSRet general_literal();
    JSRet general_identifier();
    void general_unicode();
    void escaped_unicode_latin_1();
    void escaped_unicode_utf_8();
    void escaped_code_point();
    void escaped_url_sequence_latin_1();
    void lit_int_code_point(int base);
    void char_code_no_match();
    void explicit_otag();
    void ctag_in_regex();

    static const char* p_scope_codes[];

    void* cur_buffer;
    void* tmp_buffer = nullptr;
    std::stringstream tmp;

    std::stringstream aliased;
    std::string alias;
    std::string last_dealiased;
    AliasState alias_state = ALIAS_NONE;
    bool prefix_increment = false;
    bool dealias_stored = false;
    bool unescape_nest_seen = false;
    bool mixed_encoding_seen = false;
    bool opening_tag_seen = false;
    bool closing_tag_seen = false;

    uint8_t max_template_nesting;
    VStack<uint16_t> brace_depth;
    JSToken token = UNDEFINED;
    ASIGroup previous_group = ASI_OTHER;
    JSIdentifier& ident_ctx;
    size_t bytes_read;
    size_t tmp_bytes_read;
    uint32_t tokens_read;
    uint32_t tmp_tokens_read;
    bool ext_script;
    VStack<char> regex_stack;

    struct
    {
        JSToken token = UNDEFINED;          // the token before
        int orig_len = 0;                   // current token original length
        int norm_len = 0;                   // normalized length of previous tokens
        int sc = 0;                         // current Starting Condition (0 means NOT_SET)
        int correction = 0;                 // correction length
    } states[JSTOKENIZER_MAX_STATES];
    int sp = 0;                             // points to the top of states
    int eof_sp = 0;                         // points to the last state before the EOF
    JSToken eof_token = UNDEFINED;          // the last token before the EOF
    int eof_sc = 0;                         // the last Starting Condition before the EOF
    int bytes_skip = 0;                     // num of bytes to skip of processing in the next chunk

    char*& tmp_buf;
    size_t& tmp_buf_size;
    const int tmp_cap_size;

    bool newline_found = false;
    bool adjusted_data = false;              // flag for resetting the continuation in case of adjusting js_data
    constexpr static bool insert_semicolon[ASI_GROUP_MAX][ASI_GROUP_MAX]
    {
        {false, false, false, false, false, false, false, false, false, false, false,},
        {false, false, false, false, false, false, false, false, false, false, false,},
        {false, false, false, false, false, false, false, false, false, false, false,},
        {false, false, false, false, false, false, false, false, false, false, false,},
        {false, true,  false, false, false, false, false, true,  true,  true,  true, },
        {false, false, false, false, false, false, false, true,  true,  true,  true, },
        {false, false, false, false, false, false, false, false, false, false, false,},
        {false, true,  false, false, false, false, false, true,  true,  true,  true, },
        {false, true,  false, true,  false, false, false, true,  true,  true,  true, },
        {false, true,  false, true,  false, false, true,  true,  true,  true,  true, },
        {false, false, false, false, false, false, false, false, false, false, false,}
    };

    std::streampos ignored_id_pos;
    struct FunctionIdentifier
    {
        bool operator< (const FunctionIdentifier& other) const
        { return identifier.size() < other.identifier.size(); }

        std::string identifier;
        FuncType type;
    };

    const std::array<FunctionIdentifier, 5> function_identifiers
    {{
        {"unescape",             FuncType::UNESCAPE },
        {"decodeURI",            FuncType::UNESCAPE },
        {"decodeURIComponent",   FuncType::UNESCAPE },
        {"String.fromCharCode",  FuncType::CHAR_CODE},
        {"String.fromCodePoint", FuncType::CHAR_CODE}
    }};

    const uint32_t max_bracket_depth;
    std::stack<Scope> scope_stack;

#if defined(CATCH_TEST_BUILD) || defined(BENCHMARK_TEST)
    friend JSTokenizerTester;
    friend JSTestConfig;
#endif // CATCH_TEST_BUILD || BENCHMARK_TEST
};

}

#endif // JS_TOKENIZER_H

pdf_tokenizer.h

Path = src/js_norm/pdf_tokenizer.h

#ifndef PDF_TOKENIZER_H
#define PDF_TOKENIZER_H

#include <cstring>
#include <sstream>
#include <unordered_set>

#include "main/snort_types.h"

#define PDFTOKENIZER_NAME_MAX_SIZE 16

namespace jsn
{

class SO_PUBLIC PDFTokenizer : public yyFlexLexer
{
public:
    enum PDFRet
    {
        EOS = 0,
        NOT_NAME_IN_DICTIONARY_KEY,
        INCOMPLETE_ARRAY_IN_DICTIONARY,
        STREAM_NO_LENGTH,
        UNEXPECTED_SYMBOL,
        MAX
    };

    PDFTokenizer() = delete;
    explicit PDFTokenizer(std::istream& in, std::ostream& out);
    ~PDFTokenizer() override;

    PDFRet process();

private:
    int yylex() override;

    PDFRet h_dict_open();
    PDFRet h_dict_close();
    PDFRet h_dict_name();
    PDFRet h_dict_other();
    inline bool h_lit_str();
    inline bool h_hex_str();
    inline bool h_lit_open();
    inline bool h_lit_close();
    PDFRet h_lit_unescape();
    PDFRet h_lit_oct2chr();
    PDFRet h_hex_hex2chr();
    PDFRet h_hex_hex2chr_u16();
    PDFRet h_lit_u16();
    PDFRet h_lit_u16_unescape();
    PDFRet h_stream_open();
    PDFRet h_stream();
    bool h_stream_close();
    void h_stream_length();
    void h_ref();
    void h_ind_obj_open();
    inline void h_ind_obj_close();
    void h_u16_start();
    void h_u16_break();
    void h_u16_hex_start();
    void h_u16_hex_break();

    PDFRet u16_eval(uint8_t byte);
    void u16_to_u8(uint32_t code);

    struct ObjectString
    {
        void clear()
        { parenthesis_level = 0; }

        int parenthesis_level = 0;
    };

    struct ObjectArray
    {
        void clear()
        { nesting_level = 0; }

        int nesting_level = 0;
    };

    struct ObjectDictionary
    {
        void clear()
        { key_value = true; array_level = 0; }

        bool key_value = true;
        int array_level = 0;
    };

    struct DictionaryEntry
    {
        void clear()
        { key[0] = '\0'; }

        char key[PDFTOKENIZER_NAME_MAX_SIZE] = {0};
    };

    struct Stream
    {
        int rem_length = -1;
        bool is_js = false;
    };

    ObjectString obj_string;
    ObjectArray obj_array;
    ObjectDictionary obj_dictionary;
    DictionaryEntry obj_entry;
    Stream obj_stream;
    std::unordered_set<unsigned int> js_stream_refs;

    // represents UTF-16BE code point
    struct
    {
        uint16_t high = 0;
        uint16_t low = 0;
        int cur_byte = 0;
    } u16_state;
};

bool PDFTokenizer::h_lit_str()
{
    return obj_dictionary.array_level == obj_array.nesting_level and !strcmp(obj_entry.key, "/JS");
}

bool PDFTokenizer::h_hex_str()
{
    return obj_dictionary.array_level == obj_array.nesting_level and !strcmp(obj_entry.key, "/JS");
}

bool PDFTokenizer::h_lit_open()
{
    return ++obj_string.parenthesis_level == 1;
}

bool PDFTokenizer::h_lit_close()
{
    return --obj_string.parenthesis_level == 0;
}

void PDFTokenizer::h_ind_obj_close()
{
    obj_stream.is_js = false;
}

}

#endif

js_norm/test/

This folder contains unit tests for utils folder and testing fixtures/utilities for these tests.

js_test_utils: Provides constants and testing functions for JavaScript normalizer tests. Test functions to check normalization and scope, both simple and multi-PDU, are made configurable through derivable configs and overrides.

Use examples:

    test_normalization("abc", "var_0000"); // will use params from default_config

    test_normalization("abc", "var_0000", {return_code(JSTokenizer::SCRIPT_CONTINUE)});
    // add check for the tokenizer return code, use default_config for everything else

    auto config = default_config.derive({norm_depth(10), check_mixed_encoding(true)});
    config.test_normalization("abc", "var_0000", {return_code(JSTokenizer::SCRIPT_CONTINUE)});
    // create derived config to make a reusable setup

js_test_option: Provides a mechanism to make configurable testing setup with a number of optional, parametrized settings. Any option is passed in params as a variant and is stored as a ConfigSet field. Checklist to add a new option with an already present type: 1. Add the option to ConfigType enum; 2. Add the option as a Field to ConfigSet; 3. Add the option to a matching type case in Config copy constructor; 4. If the option type has a destructor (e.g. unordered_set), add the option to the matching type case in the Config destructor; 5. Add field assignment to the set_overrides test case; 6. Add a named constuctor that returns Config with your option as the type.

js_test_stubs: Provides stubs for external dependencies.

js_test_options.h

Path = src/js_norm/test/js_test_options.h

#ifndef JS_TEST_OPTIONS_H
#define JS_TEST_OPTIONS_H

#include <list>
#include <tuple>
#include <unordered_map>
#include <unordered_set>
#include <vector>

#include "js_norm/js_identifier_ctx.h"
#include "js_norm/js_normalizer.h"

using namespace jsn;

typedef std::unordered_set<std::string> StringSet;

enum ConfigType
{
    ReturnCode,
    NormalizeIdentifiers,
    NormalizeAsExternal,
    IgnoredIdsList,
    IgnoredPropertiesList,
    NormDepth,
    IdentifierDepth,
    MaxTemplateNesting,
    MaxBracketDepth,
    MaxScopeDepth,
    MaxTokenBufSize,
    CheckMixedEncoding,
    CheckUnescapeNesting,
    UseExpectedForLastPDU,
    ExpectedCursorPos,
    CheckOpenTag,
    CheckClosingTag,
    TemporaryBuffer
};

struct Config
{
    ConfigType type;
    union
    {
        JSTokenizer::JSRet val_jsret;
        bool val_bool;
        int val_int;
        StringSet val_string_set;
        std::string val_string;
    };

    Config(const Config& other);
    Config(ConfigType type, JSTokenizer::JSRet val) : type(type), val_jsret(val){}
    Config(ConfigType type, bool val) : type(type), val_bool(val){}
    Config(ConfigType type, int val) : type(type), val_int(val){}
    Config(ConfigType type, const StringSet& val) : type(type), val_string_set(val){}
    Config(ConfigType type, const std::string& val) : type(type), val_string(val){}
    ~Config();
};

typedef std::initializer_list<Config> Overrides;

class ConfigSet
{
protected:
    template<class T>
    class Field
    {
        T val;
        bool set = false;

    public:
        Field()
        { }
        Field(const T& val): val(val)
        { }
        operator const T&() const
        { return val; }
        T& operator =(const T& new_value)
        { val = new_value; set = true; return val; }
        bool is_set() const
        { return set; }
        void unset()
        { set = false; }
    };

public:
    Field<JSTokenizer::JSRet> return_code;
    Field<bool> normalize_identifiers;
    Field<bool> normalize_as_external;
    Field<StringSet> ignored_ids_list;
    Field<StringSet> ignored_properties_list;
    Field<int> norm_depth;
    Field<int> identifier_depth;
    Field<int> max_template_nesting;
    Field<int> max_bracket_depth;
    Field<int> max_scope_depth;
    Field<int> max_token_buf_size;
    Field<bool> check_mixed_encoding;
    Field<bool> check_unescape_nesting;

    // If true, check only new normalized part, otherwise check the whole normalized script
    Field<bool> use_expected_for_last_pdu;
    Field<int> expected_cursor_pos;
    Field<bool> check_open_tag;
    Field<bool> check_closing_tag;

    // Add check for contents of the temporary buffer
    Field<std::string> temporary_buffer;

protected:
    void set_overrides(const Overrides& overrides);
};

Config return_code(JSTokenizer::JSRet val);
Config normalize_identifiers(bool val);
Config normalize_as_external(bool val);
Config ignored_ids_list(StringSet val);
Config ignored_properties_list(StringSet val);
Config norm_depth(int val);
Config identifier_depth(int val);
Config max_template_nesting(int val);
Config max_bracket_depth(int val);
Config max_scope_depth(int val);
Config max_token_buf_size(int val);
Config check_mixed_encoding(bool val);
Config check_unescape_nesting(bool val);

// If true, check only new normalized part, otherwise check the whole normalized script
Config use_expected_for_last_pdu(bool val);
Config expected_cursor_pos(int val);
Config check_open_tag(bool val);
Config check_closing_tag(bool val);

// Add check for contents of the temporary buffer
Config temporary_buffer(std::string val);

#endif

js_test_utils.h

Path = src/js_norm/test/js_test_utils.h

#ifndef JS_TEST_UTILS_H
#define JS_TEST_UTILS_H

#include <list>
#include <tuple>
#include <unordered_set>
#include <utility>
#include <vector>

#include "js_norm/js_identifier_ctx.h"
#include "js_norm/js_normalizer.h"

#include "js_test_options.h"

constexpr int unlim_depth = -1;

namespace jsn
{

class JSIdentifierCtxStub : public JSIdentifier
{
public:
    JSIdentifierCtxStub() = default;

    const char* substitute(const char* identifier, bool) override
    { return identifier; }
    virtual void add_alias(const char*, const std::string&&) override {}
    virtual const char* alias_lookup(const char* alias) const override
    { return alias; }
    bool is_ignored(const char*) const override
    { return false; }
    bool scope_push(JSProgramScopeType) override { return true; }
    bool scope_pop(JSProgramScopeType) override { return true; }
    void reset() override {}
};

class JSTestConfig;

class JSTokenizerTester
{
public:
    typedef JSTokenizer::FuncType FuncType;

    JSTokenizerTester(const JSTestConfig& conf);

    void do_pdu(const std::string& source);
    void check_output(const std::string& expected);
    void run_checks(const JSTestConfig& checks);

    JSIdentifierCtx ident_ctx;
    JSIdentifierCtxStub ident_ctx_stub;
    JSNormalizer normalizer;

private:
    const JSTestConfig& config;
    JSTokenizer::JSRet last_return = JSTokenizer::EOS;
    std::string last_source;
};

typedef JSTokenizerTester::FuncType FuncType;

// source, expected for a single PDU
typedef std::pair<std::string, std::string> PduCase;

// source, expected, and current scope type stack for a single PDU
typedef std::tuple<std::string, std::string, std::list<JSProgramScopeType>> ScopedPduCase;
typedef std::tuple<const char*, const char*, std::list<FuncType>> FunctionScopeCase;

class JSTestConfig : public ConfigSet
{
public:
    JSTestConfig(const Overrides& values);
    JSTestConfig derive(const Overrides& values) const;

    JSNormalizer&& make_normalizer() const;

    void test_scope(const std::string& context, const std::list<JSProgramScopeType>& stack) const;
    void test_scope(const std::string& context, const std::list<JSProgramScopeType>& stack,
        const Overrides& overrides) const;

    void test_function_scopes(const std::list<FunctionScopeCase>& pdus);
    void test_function_scopes(const std::list<FunctionScopeCase>& pdus, const Overrides& overrides);

    void test_normalization(const std::string& source, const std::string& expected) const;
    void test_normalization(const std::string& source, const std::string& expected, const Overrides& overrides) const;

    void test_normalization(const std::vector<PduCase>& pdus) const;
    void test_normalization(const std::vector<PduCase>& pdus, const Overrides& overrides) const;

    void test_normalization(const std::list<ScopedPduCase>& pdus) const;
    void test_normalization(const std::list<ScopedPduCase>& pdus, const Overrides& overrides) const;

    void test_normalization_combined(const std::list<std::string>& pdu_sources,
        const std::string& total_expected) const;
    void test_normalization_combined(const std::list<std::string>& pdu_sources,
        const std::string& total_expected, const Overrides& overrides) const;
};

static const JSTestConfig default_config({
    norm_depth(65535),
    identifier_depth(65535),
    max_template_nesting(4),
    max_bracket_depth(256),
    max_scope_depth(256),
    max_token_buf_size(256),
    ignored_ids_list({
        "console", "eval", "document", "unescape", "decodeURI", "decodeURIComponent", "String",
        "name", "u"}),
    ignored_properties_list({
        "watch", "unwatch", "split", "reverse", "join", "name", "w", "catch", "finally"}),
    normalize_identifiers(true)
});

}

void test_scope(const std::string& context, const std::list<jsn::JSProgramScopeType>& stack);
void test_normalization(const std::string& source, const std::string& expected, const Overrides& overrides = {});
void test_normalization_noident(const std::string& source, const std::string& expected,
    const Overrides& overrides = {});
void test_normalization_bad(const std::string& source, const std::string& expected, JSTokenizer::JSRet eret);
void test_normalization_mixed_encoding(const std::string& source, const std::string& expected);
void test_normalization(const std::vector<PduCase>& pdus, const Overrides& overrides = {});
void test_normalization(const std::list<ScopedPduCase>& pdus, const Overrides& overrides = {});
void test_normalization_combined(const std::list<std::string>& pdu_sources, const std::string& total_expected,
    const Overrides& overrides = {});

#endif

latency/

The latency directory contains code for latency logging and management.

  • Packet latency: tracks and manages latency in packet processing. Packet latency maintains an internal stack of timers for each packet processing context. The API can be queried to check if a packet should be fastpathed, and upon context exit, either an event, a log message or both can be generated based on whether the packet was fastpathed and depending on how the manager was configured.

  • Rule latency: tracks and manages latency in rule tree evaluation. Rule latency works much like packet latency. Instead of fastpath the API contains an enabled() check that tests whether the current rule tree should be evaluated. Pushing a rule tree onto the stack has the side effect of re-enabling the rule tree if conditions are met. Popping a rule tree off of the stack has the side effect of suspending the rule tree if certain conditions are met. Re-enables, suspends, and rule tree timeouts all propagate log events. Re-enables and suspends propagate alerts.

    Pushing a rule tree side-effect: A rule tree is re-enabled if
    1) it is currently disabled and 2) the suspend timeout is nonzero
    and has been reached.
    Popping a rule tree side-effect: A rule tree is suspended if
    1) it is timed out and 2) the timeout threshold is met or
    exceeded.

latency_config.h

Path = src/latency/latency_config.h

#ifndef LATENCY_CONFIG_H
#define LATENCY_CONFIG_H

#include "packet_latency_config.h"
#include "rule_latency_config.h"

namespace snort
{
class Trace;
}

extern THREAD_LOCAL const snort::Trace* latency_trace;

struct LatencyConfig
{
    PacketLatencyConfig packet_latency;
    RuleLatencyConfig rule_latency;
};

#endif

latency_module.h

Path = src/latency/latency_module.h

#ifndef LATENCY_MODULE_H
#define LATENCY_MODULE_H

#include "framework/module.h"

class LatencyModule : public snort::Module
{
public:
    LatencyModule();

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    const snort::RuleMap* get_rules() const override;
    unsigned get_gid() const override;

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;

    Usage get_usage() const override
    { return CONTEXT; }

    void set_trace(const snort::Trace*) const override;
    const snort::TraceOption* get_trace_options() const override;
};

#endif

latency_rules.h

Path = src/latency/latency_rules.h

#ifndef LATENCY_RULES_H
#define LATENCY_RULES_H

#define GID_LATENCY 134

// SIDs
#define LATENCY_EVENT_RULE_TREE_SUSPENDED   1
#define LATENCY_EVENT_RULE_TREE_ENABLED     2
#define LATENCY_EVENT_PACKET_FASTPATHED     3

#endif

latency_stats.h

Path = src/latency/latency_stats.h

#ifndef LATENCY_STATS_H
#define LATENCY_STATS_H

#include "main/thread.h"
#include "framework/counts.h"

struct LatencyStats
{
    PegCount total_packets;
    PegCount total_usecs;
    PegCount max_usecs;
    PegCount packet_timeouts;
    PegCount total_rule_evals;
    PegCount rule_eval_timeouts;
    PegCount rule_tree_enables;
};

extern THREAD_LOCAL LatencyStats latency_stats;

#endif

latency_timer.h

Path = src/latency/latency_timer.h

#ifndef LATENCY_TIMER_H
#define LATENCY_TIMER_H

#include "time/stopwatch.h"

template<typename Clock>
class LatencyTimer
{
public:
    using duration = typename Clock::duration;

    LatencyTimer(duration d) :
        max_time(d)
    { sw.start(); }

    duration elapsed() const
    { return sw.get(); }

    bool timed_out() const
    { return (max_time > CLOCK_ZERO) && (elapsed() > max_time); }

private:
    duration max_time;
    Stopwatch<Clock> sw;
};

#endif

latency_util.h

Path = src/latency/latency_util.h

#ifndef LATENCY_UTIL_H
#define LATENCY_UTIL_H

template<typename T>
struct ReferenceWrapper
{
    virtual const T* operator->() const = 0;
};

template<typename Event>
struct EventingWrapper
{
    virtual void handle(const Event&) = 0;
};

#endif

packet_latency_config.h

Path = src/latency/packet_latency_config.h

#ifndef PACKET_LATENCY_CONFIG_H
#define PACKET_LATENCY_CONFIG_H

#include "time/clock_defs.h"

struct PacketLatencyConfig
{
    hr_duration max_time = CLOCK_ZERO;
    bool fastpath = false;
    bool force_enable = false;
#ifdef REG_TEST
    bool test_timeout = false;
#endif

    bool enabled() const
    {
#ifdef REG_TEST
        if ( test_timeout )
            return true;
#endif
        return max_time > CLOCK_ZERO;
    }

    bool force_enabled() const
    {
        return force_enable;
    }
};

#endif

packet_latency.h

Path = src/latency/packet_latency.h

#ifndef PACKET_LATENCY_H
#define PACKET_LATENCY_H

namespace snort
{
struct Packet;
}

class PacketLatency
{
public:
    static void push();
    static void pop(const snort::Packet*);
    static bool fastpath();

    static void tterm();

    class Context
    {
    public:
        Context(const snort::Packet* p) : p(p) { PacketLatency::push(); }
        ~Context() { PacketLatency::pop(p); }

    private:
        const snort::Packet* p;
    };
};

#endif

rule_latency_config.h

Path = src/latency/rule_latency_config.h

#ifndef RULE_LATENCY_CONFIG_H
#define RULE_LATENCY_CONFIG_H

#include "time/clock_defs.h"

struct RuleLatencyConfig
{
    hr_duration max_time = 0_ticks;
    bool suspend = false;
    unsigned suspend_threshold = 0;
    hr_duration max_suspend_time = 0_ticks;
#ifdef REG_TEST
    bool test_timeout = false;
#endif

    bool enabled() const
    {
#ifdef REG_TEST
        if ( test_timeout )
            return true;
#endif
        return max_time > 0_ticks;
    }
    bool allow_reenable() const { return max_suspend_time > 0_ticks; }
};

#endif

rule_latency.h

Path = src/latency/rule_latency.h

#ifndef RULE_LATENCY_H
#define RULE_LATENCY_H

struct detection_option_tree_root_t;
namespace snort
{
struct Packet;
}

class RuleLatency
{
public:
    static void push(const detection_option_tree_root_t&, snort::Packet*);
    static void pop();
    static bool suspended();

    static void tterm();

    class Context
    {
    public:
        Context(const detection_option_tree_root_t& root, snort::Packet* p)
        { RuleLatency::push(root, p); }

        ~Context()
        { RuleLatency::pop(); }
    };
};

#endif

rule_latency_state.h

Path = src/latency/rule_latency_state.h

#ifndef RULE_LATENCY_STATE_H
#define RULE_LATENCY_STATE_H

#include "time/clock_defs.h"

struct RuleLatencyState
{
    hr_time suspend_time { };
    unsigned timeouts = 0;
    bool suspended = false;

    void enable()
    {
        timeouts = 0;
        suspended = false;
    }

    void suspend(hr_time cur_time)
    {
        suspend_time = cur_time;
        suspended = true;
    }
};

#endif

log/

Text output logging facilities are located here:

  • log - provides convenience functions for global packet logging.

  • log_text - provides convenience functions for logging with a TextLog.

  • messages - provides Dumper class and message logging facilities.

    Class ConfigLogger is implemented to provide functions to format
    config verbose output from Inspector::show().
  • obfuscator - provides an API for logging packets w/o revealing sensitive data.

    class Obfuscator is implemented with a std::set to provide in order
    access for potentially out-of-order insertions.
    Currently does not support streaming mode.  It should be possible to
    iterate over contiguous chunks of data, alternating between obfuscated
    and plain.
  • text_log - provides a class like implementation (TextLog) for multiple instances of text-based log files.

log.h

Path = src/log/log.h

#ifndef LOG_H
#define LOG_H

#include <cstdio>

#include "main/snort_types.h"

namespace snort
{
namespace tcp { struct TCPHdr; }
struct Packet;

SO_PUBLIC void CreateTCPFlagString(const tcp::TCPHdr* const, char*);
}

FILE* OpenAlertFile(const char*, bool is_critical=true);
int RollAlertFile(const char*);

void OpenLogger();
void CloseLogger();
void LogIPPkt(snort::Packet*);
void LogFlow(snort::Packet*);
void LogNetData(const uint8_t* data, const int len, snort::Packet*);

#endif

log_text.h

Path = src/log/log_text.h

#ifndef LOG_TEXT_H
#define LOG_TEXT_H

// Use these methods to write to a TextLog

#include "log/text_log.h"

struct Event;

namespace snort
{
struct Packet;
namespace eth { struct EtherHdr; }
namespace ip { struct IP4Hdr; }
namespace tcp { struct TCPHdr; }

SO_PUBLIC void LogTimeStamp(TextLog*, Packet*);
SO_PUBLIC void LogPriorityData(TextLog*, const Event&);
SO_PUBLIC void LogXrefs(TextLog*, const Event&);

SO_PUBLIC void LogIPPkt(TextLog*, Packet*);
SO_PUBLIC void LogPayload(TextLog*, Packet*);
SO_PUBLIC bool LogAppID(TextLog*, Packet*);

SO_PUBLIC void LogNetData(
        TextLog*, const uint8_t* data, const int len, Packet*, const char* buf_name = nullptr,
        const char* ins_name = nullptr);

SO_PUBLIC void Log2ndHeader(TextLog*, Packet*);
SO_PUBLIC void LogTCPHeader(TextLog*, Packet*);
SO_PUBLIC void LogUDPHeader(TextLog*, Packet*);
SO_PUBLIC void LogICMPHeader(TextLog*, Packet*);

SO_PUBLIC void LogEthAddrs(TextLog*, const eth::EtherHdr*);
SO_PUBLIC void LogIpAddrs(TextLog*, Packet*);
SO_PUBLIC void LogIPHeader(TextLog*, Packet*);

SO_PUBLIC void LogIpOptions(TextLog*, const ip::IP4Hdr*, uint16_t valid_ip4_len);
SO_PUBLIC void LogTcpOptions(TextLog*, const tcp::TCPHdr*, uint16_t valid_tcp_len);
}

#endif

messages.h

Path = src/log/messages.h

#ifndef MESSAGES_H
#define MESSAGES_H

#include <arpa/inet.h>
#include <cstdio>
#include <cstdarg>
#include <string>
#include <ctime>

#include "main/snort_types.h"

#define LOG_DIV "--------------------------------------------------"

#ifndef __GNUC__
#define __attribute__(x)  /*NOTHING*/
#endif

#define STD_BUF 1024

enum WarningGroup
{
    WARN_DAQ, WARN_CONF, WARN_CONF_STRICT, WARN_VARS,
    WARN_SYMBOLS, WARN_SCRIPTS, WARN_HOSTS, WARN_RULES,
    WARN_FLOWBITS, WARN_PLUGINS,
    WARN_MAX
};

void reset_parse_errors();
unsigned get_parse_errors();
unsigned get_parse_warnings();
void reset_reload_errors();
unsigned get_reload_errors();
std::string& get_reload_errors_description();

namespace snort
{
SO_PUBLIC void ParseWarning(WarningGroup, const char*, ...) __attribute__((format (printf, 2, 3)));
SO_PUBLIC void ParseError(const char*, ...) __attribute__((format (printf, 1, 2)));
SO_PUBLIC void ReloadError(const char*, ...) __attribute__((format (printf, 1, 2)));
[[noreturn]] SO_PUBLIC void ParseAbort(const char*, ...) __attribute__((format (printf, 1, 2)));

SO_PUBLIC void LogMessage(const char*, va_list& ap);
SO_PUBLIC void LogMessage(const char*, ...) __attribute__((format (printf, 1, 2)));
SO_PUBLIC void LogMessage(FILE*, const char*, ...) __attribute__((format (printf, 2, 3)));
SO_PUBLIC void WarningMessage(const char*, ...) __attribute__((format (printf, 1, 2)));
SO_PUBLIC void WarningMessage(const char*, va_list& ap);
SO_PUBLIC void ErrorMessage(const char*, ...) __attribute__((format (printf, 1, 2)));
SO_PUBLIC void ErrorMessage(const char*, va_list& ap);

class SO_PUBLIC ConfigLogger final
{
public:
    ConfigLogger() = delete;

    static void log_option(const char* caption);

    static bool log_flag(const char* caption, bool flag, bool subopt = false);

    static void log_limit(const char* caption, int val, int unlim, bool subopt = false);
    static void log_limit(const char* caption, unsigned int val, unsigned int unlim, bool subopt = false);
    static void log_limit(const char* caption, long val, int unlim, bool subopt = false);
    static void log_limit(const char* caption, unsigned long val, unsigned int unlim, bool subopt = false);
    static void log_limit(const char* caption, long long val, int unlim, bool subopt = false);
    static void log_limit(const char* caption, unsigned long long val, unsigned int unlim, bool subopt = false);

    static void log_limit(const char* caption, int val, int unlim, int disable, bool subopt = false);
    static void log_limit(const char* caption, unsigned int val, unsigned int unlim, unsigned int disable, bool subopt = false);
    static void log_limit(const char* caption, long val, int unlim, int disable, bool subopt = false);
    static void log_limit(const char* caption, unsigned long val, unsigned int unlim, unsigned int disable, bool subopt = false);
    static void log_limit(const char* caption, long long val, int unlim, int disable, bool subopt = false);
    static void log_limit(const char* caption, unsigned long long val, unsigned int unlim, unsigned int disable, bool subopt = false);

    static void log_value(const char* caption, int n, const char* descr, bool subopt = false);

    static void log_value(const char* caption, int n, bool subopt = false);
    static void log_value(const char* caption, unsigned int n, bool subopt = false);
    static void log_value(const char* caption, long n, bool subopt = false);
    static void log_value(const char* caption, unsigned long n, bool subopt = false);
    static void log_value(const char* caption, long long n, bool subopt = false);
    static void log_value(const char* caption, unsigned long long n, bool subopt = false);
    static void log_value(const char* caption, double n, bool subopt = false);
    static void log_value(const char* caption, const char* str, bool subopt = false);

    static void log_list(const char* caption, const char* list, const char* prefix = " ", bool subopt = false);
    static void log_list(const char* list);
private:
    static constexpr int indention = 25;
    static constexpr int max_line_len = 75;
};

// FIXIT-RC do not call FatalError() during runtime
[[noreturn]] SO_PUBLIC void FatalError(const char*, ...) __attribute__((format (printf, 1, 2)));

NORETURN_ASSERT void log_safec_error(const char*, void*, int);

class Dumper
{
public:
    Dumper(const char* s = nullptr, unsigned n = 3)
    {
        max = n; idx = 0;
        if ( s )
            LogMessage("%s\n", s);
    }

    ~Dumper()
    {
        if ( idx % max )
            LogMessage("\n");
    }

    void dump(const char* s, unsigned v = 0)
    {
        const char* eol = !(++idx % max) ? "\n" : "";
        LogMessage("    %18.18s(v%u)%s", s, v, eol);
    }

    void dump(const char* s, const char* t)
    {
        LogMessage("%s::%s\n", s, t);
    }

private:
    unsigned max;
    unsigned idx;
};
}

#endif

obfuscator.h

Path = src/log/obfuscator.h

#ifndef OBFUSCATOR_H
#define OBFUSCATOR_H

#include <cstddef>
#include <cstdint>
#include <set>
#include <string>
#include <unordered_map>

#include "main/snort_types.h"

namespace snort
{
struct ObfuscatorBlock
{
    // Only used by `class Obfuscator`
    ObfuscatorBlock(uint32_t off, uint32_t len)
        : offset(off), length(len)
    { }

    // Used for iterations
    ObfuscatorBlock() = default;

    uint32_t offset = 0;
    size_t length = 0;
};

class SO_PUBLIC Obfuscator
{
public:
    struct BlockCompare
    {
        bool operator() (const ObfuscatorBlock& l, const ObfuscatorBlock& r) const
        { return l.offset < r.offset; }
    };

    using ObSet = std::set<ObfuscatorBlock, BlockCompare>;
    using BufBlocks = std::unordered_map<std::string/*buf_name*/, ObSet>;
    using const_iterator = ObSet::const_iterator;
    using iterator = ObSet::iterator;

    Obfuscator()
    {
        cur_buf = buffer_blocks.begin();
    }

    void push(uint32_t offset, uint32_t length)
    {
        if (cur_buf == buffer_blocks.end())
            set_buffer("");
        const auto push_res = cur_buf->second.emplace(offset, length);

        if (!push_res.second and length > push_res.first->length)
        {
            cur_buf->second.erase(push_res.first);
            cur_buf->second.emplace(offset, length);
        }
    }

    bool select_buffer(const char* buf_key)
    {
        if (!buf_key)
            return false;

        auto buf = buffer_blocks.find(buf_key);
        if (buf == buffer_blocks.end())
            return false;
        cur_buf = buf;
        return true;
    }

    void set_buffer(const char* buf_key)
    {
        if (!buf_key)
            return;

        cur_buf = buffer_blocks.emplace(buf_key, ObSet()).first;
    }

    const_iterator begin() const
    { return cur_buf->second.cbegin(); }

    const_iterator end() const
    { return cur_buf->second.cend(); }

    bool first(ObfuscatorBlock &b);
    bool next(ObfuscatorBlock &b);

    char get_mask_char()
    { return mask_char; }

private:
    BufBlocks buffer_blocks;
    BufBlocks::iterator cur_buf;
    iterator it;
    static constexpr char mask_char = 'X';
};
}

#endif

text_log.h

Path = src/log/text_log.h

#ifndef TEXT_LOG_H
#define TEXT_LOG_H

/*
 * Declares a TextLog_*() api for buffered logging.  This allows
 * relatively painless transition from fprintf(), fwrite(), etc.
 * to a buffer that is formatted in memory and written with one
 * fwrite().
 *
 * Additionally, the file is capped at a maximum size.  Beyond
 * that, the file is closed, renamed, and reopened.  The current
 * file always has the same name.  Old files are renamed to that
 * name plus a timestamp.
 */

#include <cstring>

#include "main/snort_types.h"

#define K_BYTES (1024)
#define M_BYTES (K_BYTES*K_BYTES)
#define G_BYTES (K_BYTES*M_BYTES)

// FIXIT-L need a LogMessage based subclass of TextLog
// or some such to get stdout or syslog
struct TextLog;

namespace snort
{
SO_PUBLIC TextLog* TextLog_Init(
    const char* name, unsigned int maxBuf = 0, size_t maxFile = 0, bool is_critical=true);
SO_PUBLIC void TextLog_Term(TextLog*);

SO_PUBLIC bool TextLog_Putc(TextLog* const, char);
SO_PUBLIC bool TextLog_Quote(TextLog* const, const char*);
SO_PUBLIC bool TextLog_Write(TextLog* const, const char*, int len);
SO_PUBLIC bool TextLog_Print(TextLog* const, const char* format, ...) __attribute__((format (printf, 2, 3)));

SO_PUBLIC bool TextLog_Flush(TextLog* const);
SO_PUBLIC int TextLog_Avail(TextLog* const);
SO_PUBLIC void TextLog_Reset(TextLog* const);
} // namespace snort

/*-------------------------------------------------------------------
  * helper functions
  *-------------------------------------------------------------------
  */
inline bool TextLog_NewLine(TextLog* const txt)
{
    return snort::TextLog_Putc(txt, '\n');
}

inline bool TextLog_Puts(TextLog* const txt, const char* str)
{
    return snort::TextLog_Write(txt, str, strlen(str));
}

#endif

u2_packet.h

Path = src/log/u2_packet.h

#ifndef U2_PACKET_H
#define U2_PACKET_H

// generate eth:ip:tcp headers for compatibility  with legacy tool chains
// that must have packets

#include "protocols/eth.h"
#include "protocols/ipv4.h"
#include "protocols/ipv6.h"
#include "protocols/tcp.h"

#include "main/snort_types.h"

namespace snort
{
struct Packet;

class SO_PUBLIC U2PseudoHeader
{
public:
    U2PseudoHeader(const Packet*);

    uint8_t* get_data();
    uint16_t get_size();
    uint16_t get_dsize();

private:
    void cook_eth(const Packet*, eth::EtherHdr&);
    void cook_ip4(const Packet*, ip::IP4Hdr&);
    void cook_ip6(const Packet*, ip::IP6Hdr&);
    void cook_tcp(const Packet*, tcp::TCPHdr&);

    static const unsigned offset = 2;
    static const unsigned max_size =
        offset + sizeof(eth::EtherHdr) + sizeof(ip::IP6Hdr) + sizeof(tcp::TCPHdr);

    uint16_t size = 0;
    uint16_t dsize = 0;

    union
    {
        struct
        {
            uint8_t align[offset];
            eth::EtherHdr eth;
            ip::IP4Hdr ip4;
            tcp::TCPHdr tcp;
        } v4;

        struct
        {
            uint8_t align[offset];
            eth::EtherHdr eth;
            ip::IP6Hdr ip6;
            tcp::TCPHdr tcp;
        } v6;

        uint8_t buf[max_size];
    } u;
};
}
#endif

unified2.h

Path = src/log/unified2.h

#ifndef UNIFIED2_H
#define UNIFIED2_H

// Unified logging (events and packets) shared header.

#include <netinet/in.h>

#include "protocols/protocol_ids.h"

// OBSOLETE (no longer generated):
// #define UNIFIED2_EVENT                 1
// #define UNIFIED2_IDS_EVENT             7
// #define UNIFIED2_IDS_EVENT_IPV6       72
// #define UNIFIED2_IDS_EVENT_MPLS       99
// #define UNIFIED2_IDS_EVENT_IPV6_MPLS 100
// #define UNIFIED2_IDS_EVENT_APPSTAT   113

// CURRENT
#define UNIFIED2_PACKET                2
#define UNIFIED2_BUFFER                3  // !legacy_events
#define UNIFIED2_IDS_EVENT_VLAN      104  // legacy_events
#define UNIFIED2_IDS_EVENT_IPV6_VLAN 105  // legacy_events
#define UNIFIED2_EXTRA_DATA          110
#define UNIFIED2_EVENT3              114

#define MAX_EVENT_APPNAME_LEN         64

/* Data structure used for serialization of Unified2 Records */
struct Serial_Unified2_Header
{
    uint32_t type;
    uint32_t length;
};

// UNIFIED2_EVENT3 = type 114
struct Unified2Event
{
    uint32_t snort_id;

    uint32_t event_id;
    uint32_t event_second;
    uint32_t event_microsecond;

    uint32_t rule_gid;
    uint32_t rule_sid;
    uint32_t rule_rev;
    uint32_t rule_class;
    uint32_t rule_priority;

    // everything above this point is common to all prior event records
    // try to keep the same for things like barnyard2

    uint32_t policy_id_context;
    uint32_t policy_id_inspect;
    uint32_t policy_id_detect;

    uint32_t pkt_src_ip[4];
    uint32_t pkt_dst_ip[4];
    uint32_t pkt_mpls_label;

    uint16_t pkt_src_port_itype;
    uint16_t pkt_dst_port_icode;
    uint16_t pkt_vlan_id;
    uint16_t unused;

    uint8_t pkt_ip_ver;  // 0x4 or 0x6, high nybble is src, low is dst
    uint8_t pkt_ip_proto;

    uint8_t snort_status;  // allow=0, can't, would, force
    uint8_t snort_action;  // pass=0, drop, block, reset

    char app_name[MAX_EVENT_APPNAME_LEN];
};

// UNIFIED2_IDS_EVENT_VLAN = type 104
struct Unified2IDSEvent
{
    uint32_t sensor_id;
    uint32_t event_id;
    uint32_t event_second;
    uint32_t event_microsecond;
    uint32_t signature_id;
    uint32_t generator_id;
    uint32_t signature_revision;
    uint32_t classification_id;
    uint32_t priority_id;
    uint32_t ip_source;
    uint32_t ip_destination;
    uint16_t sport_itype;
    uint16_t dport_icode;
    IpProtocol ip_proto;
    uint8_t impact_flag; // overloads packet_action
    uint8_t impact;
    uint8_t blocked;
    uint32_t mpls_label;
    uint16_t vlanId;
    uint16_t pad2; // Policy ID
    char app_name[MAX_EVENT_APPNAME_LEN];
};

// UNIFIED2_IDS_EVENT_IPV6_VLAN = type 105
struct Unified2IDSEventIPv6
{
    uint32_t sensor_id;
    uint32_t event_id;
    uint32_t event_second;
    uint32_t event_microsecond;
    uint32_t signature_id;
    uint32_t generator_id;
    uint32_t signature_revision;
    uint32_t classification_id;
    uint32_t priority_id;
    struct in6_addr ip_source;
    struct in6_addr ip_destination;
    uint16_t sport_itype;
    uint16_t dport_icode;
    IpProtocol ip_proto;
    uint8_t impact_flag;
    uint8_t impact;
    uint8_t blocked;
    uint32_t mpls_label;
    uint16_t vlanId;
    uint16_t pad2; /*could be IPS Policy local id to support local sensor alerts*/
    char app_name[MAX_EVENT_APPNAME_LEN];
};

// UNIFIED2_PACKET = type 2
struct Serial_Unified2Packet
{
    uint32_t sensor_id;
    uint32_t event_id;
    uint32_t event_second;
    uint32_t packet_second;
    uint32_t packet_microsecond;
    uint32_t linktype;
    uint32_t packet_length;
    uint8_t packet_data[4];
};

struct Unified2ExtraDataHdr
{
    uint32_t event_type;
    uint32_t event_length;
};

// UNIFIED2_EXTRA_DATA - type 110
struct SerialUnified2ExtraData
{
    uint32_t sensor_id;
    uint32_t event_id;
    uint32_t event_second;
    uint32_t type;         // EventInfo
    uint32_t data_type;    // EventDataType
    uint32_t blob_length;  // Length of the data + sizeof(blob_length) + sizeof(data_type)
};

struct Data_Blob
{
    uint32_t length;
    const uint8_t* data;
};

enum EventInfoEnum
{
    EVENT_INFO_XFF_IPV4 = 1,
    EVENT_INFO_XFF_IPV6,
    EVENT_INFO_REVIEWED_BY,
    EVENT_INFO_GZIP_DATA,
    EVENT_INFO_SMTP_FILENAME,
    EVENT_INFO_SMTP_MAILFROM,
    EVENT_INFO_SMTP_RCPTTO,
    EVENT_INFO_SMTP_EMAIL_HDRS,
    EVENT_INFO_HTTP_URI,
    EVENT_INFO_HTTP_HOSTNAME,
    EVENT_INFO_IPV6_SRC,  // deprecated
    EVENT_INFO_IPV6_DST,  // deprecated
    EVENT_INFO_JSNORM_DATA
};

enum EventDataType
{
    EVENT_DATA_TYPE_BLOB = 1,
    EVENT_DATA_TYPE_MAX
};

#define EVENT_TYPE_EXTRA_DATA   4

#define MAX_XFF_WRITE_BUF_LENGTH \
    (sizeof(Serial_Unified2_Header) + \
    sizeof(Unified2ExtraDataHdr) + sizeof(SerialUnified2ExtraData) \
    + sizeof(struct in6_addr))

#endif

loggers/

Logger subclasses that provide logging and event alerting facilities.

unified2 is currently the best logger for serializing various data like events and packets and is the only Logger supporting extra data fields. Currently only the SMTP and HTTP inspectors produce extra data.

There is separate utility called u2spewfoo provided under tools/ that can dump the binary u2 log in text format.

This will likely be replaced with a FlatBuffer implementation.

loggers.h

Path = src/loggers/loggers.h

#ifndef LOGGERS_H
#define LOGGERS_H

void load_loggers();

#endif

log/test/

lua/

Templates and helpers for interfacing with luajit virtual machines are here:

  • State is a RAII wrapper to the lua_State* pointer

  • ManageStack prevents the stack from growing too large and ensures that the stack has enough room.

  • Args provides type checking access to the lua stack from within an embedded C/C\++ function

  • lua_iface.h contains wrappers for exposing C\++ objects to lua

  • lua_ref.h provides a very basic reference tracking system to prevent Lua from garbage collecting objects that may be in use but are otherwise untracked in the lua vm.

  • lua_stack.h provides templated functions to interface with the stack

lua.h

Path = src/lua/lua.h

#ifndef LUA_H
#define LUA_H

// methods and templates for the C++ / LuaJIT interface

#include <lua.hpp>

#include "main/snort_types.h"

namespace Lua
{

// Resource manager for lua_State
class State
{
public:
    State(bool openlibs = true);
    ~State();

    State(State&) = delete;
    State& operator=(State&) = delete;

    // Enable move constructor
    State(State&&) noexcept;
    State& operator=(State&&);

    lua_State* get_ptr()
    { return state; }

    operator lua_State*()
    { return get_ptr(); }

private:
    lua_State* state;
};

// Stack maintainer for lua_State
class SO_PUBLIC ManageStack
{
public:
    ManageStack(lua_State* L, int extra = 0);

    ManageStack(ManageStack&) = delete;
    ManageStack(ManageStack&&) = delete;

    ~ManageStack();

private:
    lua_State* state;
    int top;
};

}
#endif

lua_ref.h

Path = src/lua/lua_ref.h

#ifndef LUA_REF_H
#define LUA_REF_H

// Keep references to C objects in Lua to prevent premature garbage collection

struct lua_State;

namespace Lua
{
void add_ref(lua_State*, void*, const char*, int);
void remove_ref(lua_State*, void*, const char*);
void remove_refs(lua_State*, void*);
}
#endif

lua_script.h

Path = src/lua/lua_script.h

#ifndef LUA_SCRIPT_H
#define LUA_SCRIPT_H

#include <string>

#include <lua.hpp>

#define LUA_DIR_SEP '/'
#define SCRIPT_DIR_VARNAME "SCRIPT_DIR"

namespace Lua
{
inline void set_script_dir(
    lua_State* L, const std::string& varname, const std::string& path)
{
    std::string dir = path.substr(0, path.rfind(LUA_DIR_SEP));
    lua_pushlstring(L, dir.c_str(), dir.size());
    lua_setglobal(L, varname.c_str());
}
}
#endif

lua_stack.h

Path = src/lua/lua_stack.h

#ifndef LUA_STACK_H
#define LUA_STACK_H

#include <lua.hpp>

#include <string>

namespace Lua
{
template<typename T>
inline constexpr bool IsInteger()
{ return std::is_integral<T>::value && !std::is_same<T, bool>::value; }

template<typename T, typename Integral = void, typename Unsigned = void>
struct Stack {};

// unsigned integer
template<typename T>
struct Stack<T, typename std::enable_if<IsInteger<T>()>::type,
    typename std::enable_if<std::is_unsigned<T>::value>::type>
{
    static void push(lua_State* L, const T& v)
    { lua_pushinteger(L, v); }

    static T get(lua_State* L, int n)
    { return lua_tointeger(L, n); }

    static constexpr int type()
    { return LUA_TNUMBER; }

    static bool validate(lua_State* L, int n, T& v)
    {
        if ( lua_type(L, n) != type() )
            return false;

        lua_Integer tmp = lua_tointeger(L, n);
        if ( tmp < 0 )
            return false;

        v = tmp;
        return true;
    }

    static bool validate(lua_State* L, int n)
    {
        T v;
        return validate(L, n, v);
    }
};

// integer
template<typename T>
struct Stack<T, typename std::enable_if<IsInteger<T>()>::type,
    typename std::enable_if<!std::is_unsigned<T>::value>::type>
{
    static void push(lua_State* L, const T& v)
    { lua_pushinteger(L, v); }

    static T get(lua_State* L, int n)
    { return lua_tointeger(L, n); }

    static constexpr int type()
    { return LUA_TNUMBER; }

    static bool validate(lua_State* L, int n, T& v)
    {
        if ( lua_type(L, n) != type() )
            return false;

        v = lua_tointeger(L, n);
        return true;
    }

    static bool validate(lua_State* L, int n)
    {
        T v;
        return validate(L, n, v);
    }
};

// default
template<typename T>
struct Stack<T, typename std::enable_if<!IsInteger<T>()>::type>
{
    static void push(lua_State*, T);
    static void push(lua_State*, T, size_t);

    static T get(lua_State*, int);
    static T get(lua_State*, int, size_t&);

    static constexpr int type();

    static bool validate(lua_State* L, int n, T& v)
    {
        if ( lua_type(L, n) != type() )
            return false;

        v = get(L, n);
        return true;
    }

    static bool validate(lua_State* L, int n)
    {
        T v;
        return validate(L, n, v);
    }

    static bool validate(lua_State*, int, T&, size_t&);
};

// const char*
template<>
inline void Stack<const char*>::push(lua_State* L, const char* s)
{ lua_pushstring(L, s); }

template<>
inline void Stack<const char*>::push(lua_State* L, const char* s, size_t len)
{ lua_pushlstring(L, s, len); }

template<>
inline const char* Stack<const char*>::get(lua_State* L, int n)
{ return lua_tostring(L, n); }

template<>
inline const char* Stack<const char*>::get(lua_State* L, int n, size_t& len)
{ return lua_tolstring(L, n, &len); }

template<>
inline constexpr int Stack<const char*>::type()
{ return LUA_TSTRING; }

template<>
inline bool Stack<const char*>::validate(
    lua_State* L, int n, const char*& v, size_t& len)
{
    if ( lua_type(L, n) != type() )
        return false;

    v = get(L, n, len);
    return true;
}

// string
template<>
inline void Stack<std::string>::push(lua_State* L, std::string s)
{ lua_pushlstring(L, s.c_str(), s.length()); }

template<>
inline std::string Stack<std::string>::get(lua_State* L, int n)
{
    size_t len = 0;
    const char* s = lua_tolstring(L, n, &len);
    return std::string(s, len);
}

template<>
inline constexpr int Stack<std::string>::type()
{ return LUA_TSTRING; }

// bool
template<>
inline void Stack<bool>::push(lua_State* L, bool v)
{ lua_pushboolean(L, v); }

template<>
inline bool Stack<bool>::get(lua_State* L, int n)
{ return lua_toboolean(L, n); }

template<>
inline constexpr int Stack<bool>::type()
{ return LUA_TBOOLEAN; }
}
#endif

lua_table.h

Path = src/lua/lua_table.h

#ifndef LUA_TABLE_H
#define LUA_TABLE_H

#include <vector>

#include "lua_stack.h"

namespace Lua
{
struct Table
{
    lua_State* L;
    int index;

    template<typename T>
    inline bool get(int n, T& v)
    {
        if ( lua_type(L, n) == Stack<T>::type() )
        {
            v = Stack<T>::get(L, n);
            return true;
        }

        return false;
    }

    inline bool set_field_from_stack(const char* k, int v)
    {
        lua_pushstring(L, k);
        lua_pushvalue(L, v);
        lua_settable(L, index);
        return true;
    }

    inline bool raw_set_field_from_stack(const char* k, int v)
    {
        lua_pushstring(L, k);
        lua_pushvalue(L, v);
        lua_rawset(L, index);
        return true;
    }

    inline bool get_field_to_stack(const char* k)
    {
        lua_getfield(L, index, k);
        return true;
    }

    template<typename T>
    inline bool get_field(const char* k, T& v)
    {
        lua_getfield(L, index, k);
        bool rv = get(-1, v);
        lua_pop(L, 1);
        return rv;
    }

    template<typename T>
    inline bool get_default(const char* k, T& v, T d = 0)
    {
        if ( !get_field<T>(k, v) )
        {
            v = d;
            return false;
        }

        return true;
    }

    template<typename T>
    inline bool raw_get_field(const char* k, T& v)
    {
        lua_pushstring(L, k);
        lua_rawget(L, index);
        bool rv = get(-1, v);
        lua_pop(L, 1);
        return rv;
    }

    template<typename T>
    bool raw_get_field(int i, T& v)
    {
        lua_rawgeti(L, index, i);
        bool rv = get(-1, v);
        lua_pop(L, 1);
        return rv;
    }

    template<typename T>
    bool set_field(const char* k, const T& v)
    {
        Stack<T>::push(L, v);
        lua_setfield(L, index, k);
        return false;
    }

    template<typename T>
    bool raw_set_field(const char* k, const T& v)
    {
        lua_pushstring(L, k);
        Stack<T>::push(L, v);
        lua_rawset(L, index);
        return false;
    }

    template<typename T>
    bool raw_set_field(int i, const T& v)
    {
        Stack<T>::push(L, v);
        lua_rawseti(L, index, i);
        return true;
    }

    Table(lua_State* state, int t) : L(state), index(t) { }
};

template<typename T>
inline void fill_table_from_vector(lua_State* L, int tindex, std::vector<T>& vec)
{
    Table table(L, tindex);
    int i = 0;
    for ( auto v : vec )
        table.raw_set_field(++i, v);
}
}
#endif

lua/test/

lua_test_common.h

Path = src/lua/test/lua_test_common.h

#ifndef LUA_TEST_COMMON_H
#define LUA_TEST_COMMON_H

//#include <utility>
#include <lua.hpp>

inline void l_end_lua_state(lua_State*& L_ptr)
{
    if ( L_ptr )
    {
        lua_close(L_ptr);
        L_ptr = nullptr;
    }
}

inline void l_reset_lua_state(lua_State*& L_ptr)
{
    l_end_lua_state(L_ptr);
    L_ptr = luaL_newstate();
    luaL_openlibs(L_ptr);
}

template<typename T, size_t N>
inline constexpr size_t sizeofArray(T (&)[N])
{ return N; }

#endif

main/

This directory provides the top-level application objects and services. SnortConfig is used heavily throughout the code and should be updated so that builtin modules can attach state in a generic but readily accessible fashion.

On Analyzer states and commands:

The Analyzer life cycle is managed as a finite state machine. It will start in the NEW state and will transition to the INITIALIZED state once the object is called as part of spinning off a packet thread. Further transitions will be prompted by commands from the main thread. From INITIALIZED, it will go to STARTED via the START command. Similarly, it will go from STARTED to RUNNING via the RUN command. Finally, it will end up in the STOPPED state when the Analyzer object has finished executing. This can be prompted by the STOP command, but may also happen if the Analyzer finishes its operation for other reasons (such as encountering an error condition). The one other state an Analyzer may be in is PAUSED, which will occur when it receives the PAUSE command while in the RUNNING state. A subsequent RESUME command will switch it back from PAUSED to RUNNING. One of the primary drivers of this state machine pattern is to allow the main thread to have synchronization points with the packet threads such that it can drop privileges at the correct time (based on the limitations of the selected DAQ module) prior to starting packet processing.

Two other commands are currently available: SWAP and ROTATE. The SWAP command will swap in a new configuration at the earliest convenience, and the ROTATE command will cause open per-thread output files to be closed, rotated, and reopened anew.

On Control connections and management:

Remote control connections can be created using tcp sockets or unix sockets. Each control connection (local and/or remote) has a request and shell associated with it. The asynchronous control commands within the control connection are blocking and control connections are not returned the shell prompt until the control commands are completed.

Re THREAD_LOCAL defined in thread.h:

In clang, this code compiles (std::array has, for all intents and purposes, a constexpr constructor). gcc, on the other hand has stricter requirements for constexpr expressions (tested on gcc 4.8, 4.9, and 5) and the code will not compile. This article implies that at the very least, the clang treatment is more up-to-date:

In any case, it does not matter right now, because we are only defining THREAD_LOCAL → thread_local on compilers that do not support extern declarations of variables with the GNU thread attribute (AKA clang). In the future, we may want to consider making the code compatible with either TLS attribute, independent of the compiler (and of course declspec(thread) for windows, whenever we cross that bridge).

Re Use of libhwloc in thread_config.cc:

The Portable Hardware Locality (hwloc) library provides a nice, platform-independent abstraction layer for CPU and memory architecture information and management. Currently it is being used as a cross-platform mechanism for managing CPU affinity of threads, but it will be used in the future for NUMA (non-uniform memory access) awareness among other things.

Use of libnuma in thread_config.cc:

The libnuma library offers a straightforward programming interface to the NUMA (Non Uniform Memory Access) policy that is supported by the Linux kernel. Preferred memory mode support for NUMA systems has been added. set_mempolicy() is used to establish the memory policy for packet threads operating on different nodes within a NUMA system. The libnuma-dev library must be installed to enable this support. In systems without NUMA architecture, this feature will not affect system performance or behavior. This, alongside with libhwloc, presents an efficient cross-platform mechanism for thread configuration and managing CPU affinity of threads, not only considering CPU architecture but also memory access policies, providing a more balanced and optimized execution environment.

ac_shell_cmd.h

Path = src/main/ac_shell_cmd.h

#ifndef AC_SHELL_CMD_H
#define AC_SHELL_CMD_H

#include "main/analyzer.h"
#include "main/analyzer_command.h"

class ControlConn;

class ACShellCmd : public snort::AnalyzerCommand
{
public:
    ACShellCmd() = delete;
    ACShellCmd(ControlConn*, snort::AnalyzerCommand*);
    bool execute(Analyzer&, void**) override;
    bool need_update_reload_id() const override
    { return ac->need_update_reload_id(); }
    const char* stringify() override { return ac->stringify(); }
    ~ACShellCmd() override;

private:
    snort::AnalyzerCommand* ac;
};

#endif

analyzer_command.h

Path = src/main/analyzer_command.h

#ifndef ANALYZER_COMMANDS_H
#define ANALYZER_COMMANDS_H

#include <cstdarg>
#include <vector>

#include "main/snort_types.h"

class Analyzer;
class ControlConn;
class Swapper;

namespace snort
{
class ScratchAllocator;
struct SnortConfig;

class SFDAQInstance;

class AnalyzerCommand
{
public:
    AnalyzerCommand() : AnalyzerCommand(nullptr)
    { }
    explicit AnalyzerCommand(ControlConn* conn) : ctrlcon(conn)
    { }
    virtual ~AnalyzerCommand() = default;
    virtual bool execute(Analyzer&, void**) = 0;
    virtual bool need_update_reload_id() const
    { return false; }
    virtual const char* stringify() = 0;
    unsigned get() { return ++ref_count; }
    unsigned put() { return --ref_count; }
    SO_PUBLIC void log_message(const char* format, ...) __attribute__((format (printf, 2, 3)));
    SO_PUBLIC static void log_message(ControlConn*, const char* format, ...) __attribute__((format (printf, 2, 3)));
    SO_PUBLIC static snort::SFDAQInstance* get_daq_instance(Analyzer& analyzer);

    ControlConn* ctrlcon;

private:
    static void log_message(ControlConn*, const char* format, va_list& ap);
    unsigned ref_count = 0;
};
}

class ACGetStats : public snort::AnalyzerCommand
{
public:
    ACGetStats(ControlConn* conn) : AnalyzerCommand(conn)
    { }
    bool execute(Analyzer&, void**) override;
    const char* stringify() override { return "GET_STATS"; }
    ~ACGetStats() override;
};

typedef enum clear_counter_type
{
    TYPE_INVALID=-1,
    TYPE_DAQ=0,
    TYPE_MODULE,
    TYPE_APPID,
    TYPE_FILE_ID,
    TYPE_SNORT,
    TYPE_HA,
        TYPE_ALL
} clear_counter_type_t;

// FIXIT-M Will replace this vector with an unordered map of
// <clear_counter_type, clear_counter_type_string_map> when
// will come up with more granular form of clearing module stats.
static std::vector<const char*> clear_counter_type_string_map
{
    "daq",
    "module",
    "appid",
    "file_id",
    "snort",
    "high_availability"
};

class ACResetStats : public snort::AnalyzerCommand
{
public:
    explicit ACResetStats(clear_counter_type_t requested_type);
    bool execute(Analyzer&, void**) override;
    const char* stringify() override { return "RESET_STATS"; }
    ~ACResetStats() override;
private:
    clear_counter_type_t requested_type;
};

class ACPause : public snort::AnalyzerCommand
{
public:
    bool execute(Analyzer&, void**) override;
    const char* stringify() override { return "PAUSE"; }
};

class ACResume : public snort::AnalyzerCommand
{
public:
    ACResume(uint64_t msg_cnt): msg_cnt(msg_cnt) { }
    bool execute(Analyzer&, void**) override;
    const char* stringify() override { return "RESUME"; }
private:
    uint64_t msg_cnt;
};

class ACRotate : public snort::AnalyzerCommand
{
public:
    bool execute(Analyzer&, void**) override;
    const char* stringify() override { return "ROTATE"; }
};

class ACRun : public snort::AnalyzerCommand
{
public:
    ACRun() = delete;
    ACRun(bool is_paused = false ) { paused = is_paused; }
    bool execute(Analyzer&, void**) override;
    const char* stringify() override { return "RUN"; }
private:
    bool paused = false;
};

class ACStart : public snort::AnalyzerCommand
{
public:
    bool execute(Analyzer&, void**) override;
    const char* stringify() override { return "START"; }
};

class ACStop : public snort::AnalyzerCommand
{
public:
    bool execute(Analyzer&, void**) override;
    const char* stringify() override { return "STOP"; }
};

class ACSwap : public snort::AnalyzerCommand
{
public:
    ACSwap() = delete;
    ACSwap(Swapper* ps, ControlConn* conn) : AnalyzerCommand(conn), ps(ps)
    { }
    bool execute(Analyzer&, void**) override;
    bool need_update_reload_id() const override
    { return true; }
    const char* stringify() override { return "SWAP"; }
    ~ACSwap() override;
private:
    Swapper *ps;
};

class ACHostAttributesSwap : public snort::AnalyzerCommand
{
public:
    ACHostAttributesSwap(ControlConn* conn) : AnalyzerCommand(conn)
    { }
    bool execute(Analyzer&, void**) override;
    const char* stringify() override { return "HOST_ATTRIBUTES_SWAP"; }
    ~ACHostAttributesSwap() override;
};

class ACDAQSwap : public snort::AnalyzerCommand
{
public:
    bool execute(Analyzer&, void**) override;
    const char* stringify() override { return "DAQ_SWAP"; }
    ~ACDAQSwap() override;
};

class ACScratchUpdate : public snort::AnalyzerCommand
{
public:
    ACScratchUpdate(snort::SnortConfig* sc, std::vector<snort::ScratchAllocator*>& handlers,
        ControlConn* conn) : AnalyzerCommand(conn), sc(sc), handlers(handlers)
    { }
    bool execute(Analyzer&, void**) override;
    const char* stringify() override { return "SCRATCH_UPDATE"; }
    ~ACScratchUpdate() override;
private:
    snort::SnortConfig* sc;
    std::vector<snort::ScratchAllocator*>& handlers;
};

namespace snort
{
// from main.cc
#ifdef REG_TEST
void main_unicast_command(AnalyzerCommand*, unsigned target, ControlConn* = nullptr);
#endif
SO_PUBLIC void main_broadcast_command(snort::AnalyzerCommand*, ControlConn* = nullptr);
}

#endif

analyzer.h

Path = src/main/analyzer.h

#ifndef ANALYZER_H
#define ANALYZER_H

// Analyzer provides the packet acquisition and processing loop.  Since it
// runs in a different thread, it also provides a command facility so that
// to control the thread and swap configuration.

#include <daq_common.h>

#include <atomic>
#include <list>
#include <mutex>
#include <queue>
#include <string>

#include "main/snort_types.h"
#include "thread.h"

class ContextSwitcher;
class OopsHandler;
class RetryQueue;
class Swapper;

namespace snort
{
class AnalyzerCommand;
class Flow;
class SFDAQInstance;
struct Packet;
struct SnortConfig;
struct ProfileStats;
}

typedef bool (* MainHook_f)(snort::Packet*);

class UncompletedAnalyzerCommand
{
public:
    UncompletedAnalyzerCommand(snort::AnalyzerCommand* ac, void* acs) : command(ac), state(acs)
    { }

    snort::AnalyzerCommand* command;
    void* state;
};

class Analyzer
{
public:
    enum class State {
        NEW,
        INITIALIZED,
        STARTED,
        RUNNING,
        PAUSED,
        STOPPED,
        FAILED,
        NUM_STATES
    };

    SO_PUBLIC static Analyzer* get_local_analyzer();
    static ContextSwitcher* get_switcher();
    static void set_main_hook(MainHook_f);

    Analyzer() = delete;
    Analyzer(snort::SFDAQInstance*, unsigned id, const char* source, uint64_t msg_cnt = 0);
    ~Analyzer();

    void operator()(Swapper*, uint16_t run_num);

    State get_state() { return state; }
    const char* get_state_string();
    const char* get_source() { return source.c_str(); }

    void set_pause_after_cnt(uint64_t msg_cnt) { pause_after_cnt = msg_cnt; }
    void set_skip_cnt(uint64_t msg_cnt) { skip_cnt = msg_cnt; }

    void execute(snort::AnalyzerCommand*);

    void post_process_packet(snort::Packet*);
    bool process_rebuilt_packet(snort::Packet*, const DAQ_PktHdr_t*, const uint8_t* pkt, uint32_t pktlen);
    SO_PUBLIC bool inspect_rebuilt(snort::Packet*);
    void finalize_daq_message(DAQ_Msg_h, DAQ_Verdict);
    void add_to_retry_queue(DAQ_Msg_h, snort::Flow*);

    // Functions called by analyzer commands
    void start();
    void run(bool paused = false);
    void stop();
    void pause();
    void resume(uint64_t msg_cnt);
    void reload_daq();
    void reinit(const snort::SnortConfig*);
    void stop_removed(const snort::SnortConfig*);
    void rotate();
    snort::SFDAQInstance* get_daq_instance() { return daq_instance; }

    bool is_idling() const
    { return idling; }

private:
    void analyze();
    bool handle_command();
    void handle_commands();
    void handle_uncompleted_commands();
    DAQ_RecvStatus process_messages();
    void process_daq_msg(DAQ_Msg_h, bool retry);
    void process_daq_pkt_msg(DAQ_Msg_h, bool retry);
    void post_process_daq_pkt_msg(snort::Packet*);
    void process_retry_queue();
    void set_state(State);
    void idle();
    bool init_privileged();
    void init_unprivileged();
    void term();
    void show_source();
    void add_command_to_uncompleted_queue(snort::AnalyzerCommand*, void*);
    void add_command_to_completed_queue(snort::AnalyzerCommand*);

public:
    std::queue<snort::AnalyzerCommand*> completed_work_queue;
    std::mutex completed_work_queue_mutex;
    std::queue<snort::AnalyzerCommand*> pending_work_queue;

private:
    std::atomic<State> state;
    unsigned id;
    bool exit_requested = false;
    bool idling = false;
    uint64_t exit_after_cnt;
    uint64_t pause_after_cnt = 0;
    uint64_t skip_cnt = 0;
    std::string source;
    snort::SFDAQInstance* daq_instance;
    RetryQueue* retry_queue;
    OopsHandler* oops_handler;
    ContextSwitcher* switcher = nullptr;
    std::mutex pending_work_queue_mutex;
    std::list<UncompletedAnalyzerCommand*> uncompleted_work_queue;
};

extern THREAD_LOCAL snort::ProfileStats daqPerfStats;

#endif

help.h

Path = src/main/help.h

#ifndef HELP_H
#define HELP_H

// utility methods that provide output modes other than the normal packet
// processing.  these are called based on command line arguments.

namespace snort
{
struct SnortConfig;
}

void config_markup(snort::SnortConfig*, const char*);

void help_args(const char* pfx);

[[noreturn]] void help_basic(snort::SnortConfig*, const char*);
[[noreturn]] void help_buffers(snort::SnortConfig* sc, const char*);
[[noreturn]] void help_builtin(snort::SnortConfig* sc, const char*);
[[noreturn]] void help_commands(snort::SnortConfig* sc, const char*);
[[noreturn]] void help_config(snort::SnortConfig* sc, const char*);
[[noreturn]] void help_counts(snort::SnortConfig* sc, const char*);
[[noreturn]] void help_gids(snort::SnortConfig* sc, const char*);
[[noreturn]] void help_limits(snort::SnortConfig* sc, const char*);
[[noreturn]] void help_module(snort::SnortConfig* sc, const char*);
[[noreturn]] void help_modules(snort::SnortConfig* sc, const char*);
[[noreturn]] void help_modules_json(snort::SnortConfig* sc, const char*);
[[noreturn]] void help_options(snort::SnortConfig*, const char*);
[[noreturn]] void help_plugins(snort::SnortConfig* sc, const char*);
[[noreturn]] void help_signals(snort::SnortConfig*, const char*);
[[noreturn]] void help_usage(snort::SnortConfig*, const char*);
[[noreturn]] void help_version(snort::SnortConfig*);

[[noreturn]] void list_daqs(snort::SnortConfig* sc);
[[noreturn]] void list_modules(snort::SnortConfig* sc, const char*);
[[noreturn]] void list_plugins(snort::SnortConfig* sc, const char*);

[[noreturn]] void dump_builtin_rules(snort::SnortConfig* sc, const char*, const char*);
[[noreturn]] void dump_defaults(snort::SnortConfig* sc, const char*);
[[noreturn]] void dump_dynamic_rules(snort::SnortConfig* sc, const char*);
[[noreturn]] void dump_rule_hex(snort::SnortConfig* sc, const char*);
[[noreturn]] void dump_rule_text(snort::SnortConfig* sc, const char*);
[[noreturn]] void dump_version(snort::SnortConfig* sc);

#endif

modules.h

Path = src/main/modules.h

#ifndef MODULES_H
#define MODULES_H

// this is for builtin module initialization.
// ideally, modules.cc would be refactored into several files.

#include "framework/counts.h"
#include "main/thread.h"


void module_init();
const char* get_lua_defaults();

#endif

network_module.h

Path = src/main/network_module.h

#ifndef NETWORK_MODULE_H
#define NETWORK_MODULE_H

#include "framework/module.h"


class NetworkModule : public snort::Module
{
public:
    NetworkModule();
    ~NetworkModule() override = default;
    bool set(const char*, snort::Value&, snort::SnortConfig*) override;

    const snort::Command* get_commands() const override;

    Usage get_usage() const override
    { return CONTEXT; }
};

#endif

oops_handler.h

Path = src/main/oops_handler.h

#ifndef OOPS_HANDLER_H
#define OOPS_HANDLER_H

#include <daq_common.h>

#include "packet_io/sfdaq_instance.h"

class OopsHandler
{
public:
    static void handle_crash(int fd);

    OopsHandler() = default;
    ~OopsHandler() = default;

    void tinit();
    void set_current_message(DAQ_Msg_h, snort::SFDAQInstance*);
    void tterm();

private:
    void eternalize(int fd);

private:
    DAQ_Msg_h msg = nullptr;
    // Eternalized data
    DAQ_MsgType type = static_cast<DAQ_MsgType>(0);
    uint8_t header[UINT16_MAX] = { };
    size_t header_len = 0;
    uint8_t data[UINT16_MAX] = { };
    uint32_t data_len = 0;
    uint8_t priv_data[UINT16_MAX] = { };
    uint16_t priv_data_len = 0;
};

#endif

policy.h

Path = src/main/policy.h

#ifndef SNORT_POLICY_H
#define SNORT_POLICY_H

// the following policy types are defined:
//
// -- network - for packet handling
// -- inspection - for flow handling
// -- ips - for rule handling

#ifdef HAVE_UUID
#include <uuid.h>
#else
typedef unsigned char uuid_t[16];
#endif

#include <algorithm>
#include <climits>
#include <map>
#include <memory>
#include <unordered_map>
#include <vector>

#include "framework/data_bus.h"

namespace snort
{
class FilePolicyBase;
class GHash;
class IpsAction;
class PolicySelector;
struct SnortConfig;
}

struct _daq_flow_stats;
struct _daq_pkt_hdr;
struct JSNormConfig;
struct PortTable;
struct vartable_t;
struct sfip_var_t;

#define UNDEFINED_NETWORK_USER_POLICY_ID UINT64_MAX

typedef unsigned int PolicyId;
typedef snort::GHash PortVarTable;

enum PolicyMode
{
    POLICY_MODE__PASSIVE,
    POLICY_MODE__INLINE,
    POLICY_MODE__INLINE_TEST,
    POLICY_MODE__MAX
};

// FIXIT-L split into separate headers

//-------------------------------------------------------------------------
// navigator stuff
//-------------------------------------------------------------------------

struct InspectionPolicy;
struct IpsPolicy;
struct NetworkPolicy;
class Shell;

namespace snort
{
SO_PUBLIC NetworkPolicy* get_network_policy();
NetworkPolicy* get_network_parse_policy();
SO_PUBLIC InspectionPolicy* get_inspection_policy();
SO_PUBLIC IpsPolicy* get_ips_policy();

SO_PUBLIC void set_network_policy(NetworkPolicy*);
void set_network_parse_policy(NetworkPolicy*);
SO_PUBLIC void set_inspection_policy(InspectionPolicy*);
SO_PUBLIC void set_ips_policy(IpsPolicy*);

SO_PUBLIC NetworkPolicy* get_default_network_policy(const snort::SnortConfig*);
// Based on currently set network policy
SO_PUBLIC InspectionPolicy* get_user_inspection_policy(uint64_t policy_id);

SO_PUBLIC IpsPolicy* get_ips_policy(const snort::SnortConfig*, unsigned i = 0);
// Based on currently set network policy
SO_PUBLIC IpsPolicy* get_default_ips_policy(const snort::SnortConfig*);
SO_PUBLIC IpsPolicy* get_user_ips_policy(const snort::SnortConfig*, unsigned policy_id);
SO_PUBLIC IpsPolicy* get_empty_ips_policy(const snort::SnortConfig*);
}

void set_network_policy(unsigned = 0);
void set_inspection_policy(unsigned = 0);
void set_ips_policy(const snort::SnortConfig*, unsigned = 0);

void set_policies(const snort::SnortConfig*, Shell*);
void set_default_policy(const snort::SnortConfig*);
void select_default_policy(const _daq_pkt_hdr&, const snort::SnortConfig*);
void select_default_policy(const _daq_flow_stats&, const snort::SnortConfig*);

bool only_inspection_policy();
bool only_ips_policy();

//-------------------------------------------------------------------------
// traffic stuff
//-------------------------------------------------------------------------

enum ChecksumFlag
{
    CHECKSUM_FLAG__IP   = 0x00000001,
    CHECKSUM_FLAG__TCP  = 0x00000002,
    CHECKSUM_FLAG__UDP  = 0x00000004,
    CHECKSUM_FLAG__ICMP = 0x00000008,
    CHECKSUM_FLAG__ALL  = 0x0000000f,
    CHECKSUM_FLAG__DEF  = 0x80000000
};

enum DecodeEventFlag
{
    DECODE_EVENT_FLAG__DEFAULT = 0x00000001
};

//-------------------------------------------------------------------------
// inspection stuff
//-------------------------------------------------------------------------

struct InspectionPolicy
{
public:
    InspectionPolicy(PolicyId = 0);
    InspectionPolicy(InspectionPolicy* old_inspection_policy);
    ~InspectionPolicy();

    void configure();

public:
    PolicyId policy_id = 0;
    PolicyMode policy_mode = POLICY_MODE__MAX;
    uint64_t user_policy_id = 0;
    uuid_t uuid{};

    struct FrameworkPolicy* framework_policy;
    snort::DataBus dbus;
    bool cloned;

    JSNormConfig* jsn_config = nullptr;

private:
    void init(InspectionPolicy* old_inspection_policy);
};

//-------------------------------------------------------------------------
// Network stuff
//-------------------------------------------------------------------------

class FilePolicy;
class FileRule;
struct IpsPolicy;

struct NetworkPolicy
{
public:
    NetworkPolicy(PolicyId = 0, PolicyId default_ips_id = 0);
    NetworkPolicy(NetworkPolicy*, const char*);
    ~NetworkPolicy();

    InspectionPolicy* get_inspection_policy(unsigned i = 0)
    { return i < inspection_policy.size() ? inspection_policy[i] : nullptr; }
    unsigned inspection_policy_count()
    { return inspection_policy.size(); }
    void setup_inspection_policies();
    InspectionPolicy* get_user_inspection_policy(uint64_t user_id) const;
    void set_user_inspection(InspectionPolicy* p)
    { user_inspection[p->user_policy_id] = p; }

    IpsPolicy* get_default_ips_policy(const snort::SnortConfig* sc)
    { return snort::get_ips_policy(sc, default_ips_policy_id); }

    void add_file_policy_rule(FileRule& file_rule);
    snort::FilePolicyBase* get_base_file_policy() const;
    FilePolicy* get_file_policy() const;

    bool checksum_drops(uint16_t codec_cksum_err_flag)
    { return (checksum_drop & codec_cksum_err_flag) != 0; }

    bool ip_checksums()
    { return (checksum_eval & CHECKSUM_FLAG__IP) != 0; }

    bool udp_checksums()
    { return (checksum_eval & CHECKSUM_FLAG__UDP) != 0; }

    bool tcp_checksums()
    { return (checksum_eval & CHECKSUM_FLAG__TCP) != 0; }

    bool icmp_checksums()
    { return (checksum_eval & CHECKSUM_FLAG__ICMP) != 0; }

protected:
    FilePolicy* file_policy;

public:
    struct TrafficPolicy* traffic_policy;
    snort::DataBus dbus;

    std::vector<InspectionPolicy*> inspection_policy;
    std::unordered_map<uint64_t, InspectionPolicy*> user_inspection;

    PolicyId policy_id = 0;
    uint64_t user_policy_id = 0;
    PolicyId default_ips_policy_id = 0;

    // minimum possible (allows all but errors to pass by default)
    uint8_t min_ttl = 1;
    uint8_t new_ttl = 5;

    uint32_t checksum_eval = CHECKSUM_FLAG__ALL | CHECKSUM_FLAG__DEF;
    uint32_t checksum_drop = CHECKSUM_FLAG__DEF;
    uint32_t normal_mask = 0;
    bool cloned = false;

private:
    void init(NetworkPolicy*, const char*);
};

//-------------------------------------------------------------------------
// detection stuff
//-------------------------------------------------------------------------

struct IpsPolicy
{
public:
    enum Enable : uint8_t { DISABLED, ENABLED, INHERIT_ENABLE };

    IpsPolicy(PolicyId = 0);
    ~IpsPolicy();

public:
    PolicyId policy_id;
    uint64_t user_policy_id = 0;
    uuid_t uuid{};

    PolicyMode policy_mode = POLICY_MODE__MAX;
    bool enable_builtin_rules;
    int rules_loaded = 0;
    int rules_shared = 0;

    std::string includer;
    std::string include;

    std::string rules;
    std::string states;

    uint32_t var_id;

    struct VarEntry* var_table;
    vartable_t* ip_vartable;

    /* The portobjects in these are attached to rtns and used during runtime */
    PortVarTable* portVarTable;     /* named entries, uses a hash table */
    PortTable* nonamePortVarTable;  /* un-named entries */

    Enable default_rule_state = INHERIT_ENABLE;

    bool obfuscate_pii;

    std::string action_override;
    std::map<std::string, std::string> action_map;

    // Holds all plugin actions associated with this policy
    std::vector<snort::IpsAction*> action;
};

//-------------------------------------------------------------------------
// binding stuff
//-------------------------------------------------------------------------

struct PolicyTuple
{
    InspectionPolicy* inspection = nullptr;
    IpsPolicy* ips = nullptr;
    NetworkPolicy* network = nullptr;
    NetworkPolicy* network_parse = nullptr;

    PolicyTuple(InspectionPolicy* ins_pol, IpsPolicy* ips_pol, NetworkPolicy* net_pol,
        NetworkPolicy* net_parse) :
        inspection(ins_pol), ips(ips_pol), network(net_pol), network_parse(net_parse)
    { }
};

struct GlobalInspectorPolicy;
class SingleInstanceInspectorPolicy;

class PolicyMap
{
public:
    PolicyMap(PolicyMap* old_map = nullptr, const char* exclude_name = nullptr);
    ~PolicyMap();

    InspectionPolicy* add_inspection_shell(Shell*);
    IpsPolicy* add_ips_shell(Shell*);
    std::shared_ptr<PolicyTuple> add_shell(Shell*, NetworkPolicy*);
    std::shared_ptr<PolicyTuple> get_policies(Shell* sh);

    Shell* get_shell(unsigned i = 0)
    { return i < shells.size() ? shells[i] : nullptr; }

    bool setup_network_policies();

    void set_user_ips(IpsPolicy* p)
    { user_ips[p->user_policy_id] = p; }

    NetworkPolicy* get_user_network(uint64_t user_id) const;

    IpsPolicy* get_user_ips(uint64_t user_id)
    {
        auto it = user_ips.find(user_id);
        return it == user_ips.end() ? nullptr : it->second;
    }

    NetworkPolicy* get_network_policy(unsigned i = 0)
    { return i < network_policy.size() ? network_policy[i] : nullptr; }
    unsigned network_policy_count()
    { return network_policy.size(); }

    IpsPolicy* get_ips_policy(unsigned i = 0)
    { return i < ips_policy.size() ? ips_policy[i] : nullptr; }
    unsigned ips_policy_count()
    { return ips_policy.size(); }
    IpsPolicy* get_empty_ips()
    { return empty_ips_policy; }

    unsigned shells_count()
    { return shells.size(); }

    void set_cloned(bool state)
    { cloned = state; }

    snort::PolicySelector* get_policy_selector() const
    { return selector; }

    void set_policy_selector(snort::PolicySelector* new_selector)
    { selector = new_selector; }

    SingleInstanceInspectorPolicy* get_file_id()
    { return file_id; }

    SingleInstanceInspectorPolicy* get_flow_tracking()
    { return flow_tracking; }

    GlobalInspectorPolicy* get_global_inspector_policy()
    { return global_inspector_policy; }

    const Shell* get_shell_by_policy(unsigned id) const
    {
        auto it = std::find_if(std::begin(shell_map), std::end(shell_map),
            [=](auto&& p) { return p.second->ips and p.second->ips->policy_id == id; });

        return (it == std::end(shell_map)) ? nullptr : it->first;
    }

    bool get_inspector_tinit_complete(unsigned instance_id) const
    { return inspector_tinit_complete[instance_id]; }

    void set_inspector_tinit_complete(unsigned instance_id, bool val)
    { inspector_tinit_complete[instance_id] = val; }

private:
    void clone(PolicyMap *old_map, const char* exclude_name);
    bool set_user_network(NetworkPolicy* p);

    std::vector<Shell*> shells;
    std::vector<NetworkPolicy*> network_policy;
    std::vector<IpsPolicy*> ips_policy;
    IpsPolicy* empty_ips_policy;

    std::unordered_map<Shell*, std::shared_ptr<PolicyTuple>> shell_map;
    std::unordered_map<uint64_t, NetworkPolicy*> user_network;
    std::unordered_map<uint64_t, IpsPolicy*> user_ips;

    snort::PolicySelector* selector = nullptr;
    SingleInstanceInspectorPolicy* file_id;
    SingleInstanceInspectorPolicy* flow_tracking;
    GlobalInspectorPolicy* global_inspector_policy;

    bool* inspector_tinit_complete;
    bool cloned = false;
};

#endif

reload_tracker.h

Path = src/main/reload_tracker.h

#ifndef RELOAD_TRACKER_H
#define RELOAD_TRACKER_H

#include <string>

#include "main/snort_types.h"

class ControlConn;

namespace snort
{

class SO_PUBLIC ReloadTracker
{
public:
    ReloadTracker() = delete;
    static bool start(ControlConn* ctrlcon);
    static void end(const ControlConn* ctrlcon, bool prompt=false);
    static void failed(const ControlConn* ctrlcon, const char* reason);
    static void update(const ControlConn* ctrlcon, const char* status);

private:
    static bool reload_in_progress;
    static std::string current_command;
    static ControlConn* ctrl;
};

}

#endif

reload_tuner.h

Path = src/main/reload_tuner.h

#ifndef RELOAD_TUNER_H
#define RELOAD_TUNER_H

namespace snort
{

class ReloadResourceTuner
{
public:
    static const unsigned RELOAD_MAX_WORK_PER_PACKET = 3;
    // be aggressive when idle as analyzer gets chance once in every second only due to daq timeout
    static const unsigned RELOAD_MAX_WORK_WHEN_IDLE = 32767;

    virtual ~ReloadResourceTuner() = default;

    // returns true if resource tuning required, false otherwise
    virtual bool tinit() = 0;

    // each of these returns true if resource tuning is complete, false otherwise
    virtual bool tune_packet_context() = 0;
    virtual bool tune_idle_context() = 0;

protected:
    ReloadResourceTuner() = default;

    unsigned max_work = RELOAD_MAX_WORK_PER_PACKET;
    unsigned max_work_idle = RELOAD_MAX_WORK_WHEN_IDLE;
};

class ReloadSwapper : public ReloadResourceTuner
{
public:
    virtual ~ReloadSwapper() override = default;

    // each of these returns true if resource tuning is complete, false otherwise
    bool tune_packet_context() override
    { return true; }
    bool tune_idle_context() override
    { return true; }

    bool tinit() override
    {
        tswap();
        return false;
    }

    virtual void tswap() = 0;

protected:
    ReloadSwapper() = default;
};

}

#endif

shell.h

Path = src/main/shell.h

#ifndef SHELL_H
#define SHELL_H

// Shell encapsulates a Lua state.  There is one for each policy file.

#include <list>
#include <set>
#include <stack>
#include <string>

#include "dump_config/config_data.h"
#include "framework/parameter.h"
#include "policy.h"

struct lua_State;

class BaseConfigNode;
class ConfigOutput;

namespace snort
{
struct SnortConfig;
class Value;
}

class Shell
{
public:
    typedef std::set<std::string> Allowlist;

    Shell(const char* file = nullptr, bool load_defaults = false);
    ~Shell();

    void set_file(const char*);
    void set_overrides(const char*);
    void set_overrides(Shell*);

    bool configure(snort::SnortConfig*, bool is_root = false);
    void install(const char*, const struct luaL_Reg*);
    void execute(const char*, std::string&);

    const char* get_file() const
    { return file.c_str(); }

    const char* get_from() const
    { return parse_from.c_str(); }

    bool get_loaded() const
    { return loaded; }

    lua_State* get_lua() const
    { return lua; }

    void set_user_network_policy();

public:
    static bool is_trusted(const std::string& key);
    static void allowlist_append(const char* keyword, bool is_prefix);

    static void config_open_table(bool is_root_node, bool is_list, int idx,
        const std::string& table_name, const snort::Parameter* p);
    static void set_config_value(const std::string& fqn, const snort::Value& value);
    static void add_config_child_node(const std::string& node_name, snort::Parameter::Type type,
        bool is_root_list_item);
    static void update_current_config_node(const std::string& node_name = "");
    static void config_close_table();
    static void set_config_output(ConfigOutput* config_output);
    static void clear_config_output();

    static void set_lua_sandbox(const char* s)
    { lua_sandbox = s; }

    static void set_network_policy_user_id(lua_State*, uint64_t user_id);

private:
    static void add_config_root_node(const std::string& root_name, snort::Parameter::Type type);

private:
    [[noreturn]] static int panic(lua_State*);
    static Shell* get_current_shell();

private:
    static std::string fatal;
    static std::stack<Shell*> current_shells;
    static ConfigOutput* s_config_output;
    static BaseConfigNode* s_current_node;
    static bool s_close_table;
    static std::string lua_sandbox;
    static const char* const lua_shell_id;

private:
    void clear_allowlist()
    {
        allowlist.clear();
        internal_allowlist.clear();
        allowlist_prefixes.clear();
    }

    const Allowlist& get_allowlist() const
    { return allowlist; }

    const Allowlist& get_internal_allowlist() const
    { return internal_allowlist; }

    const Allowlist& get_allowlist_prefixes() const
    { return allowlist_prefixes; }

    void print_allowlist() const;
    void allowlist_update(const char* keyword, bool is_prefix);

    bool load_lua_sandbox();
    bool set_sandbox_env();
    bool load_string(const char* s, bool load_in_sandbox, const char* message);
    bool load_config(const char* file, bool load_in_sandbox);

private:
    bool loaded;
    bool bootstrapped = false;
    lua_State* lua;
    std::string file;
    std::string parse_from;
    std::string overrides;
    Allowlist allowlist;
    Allowlist internal_allowlist;
    Allowlist allowlist_prefixes;
    ConfigData config_data;
    uint64_t network_user_policy_id = UNDEFINED_NETWORK_USER_POLICY_ID;
    bool load_defaults;
};

#endif

snort_config.h

Path = src/main/snort_config.h

#ifndef SNORT_CONFIG_H
#define SNORT_CONFIG_H

// SnortConfig encapsulates all data loaded from the config files.
// FIXIT-L privatize most of this stuff.

#include <sys/types.h>

#include <list>
#include <mutex>
#include <unordered_map>
#include <vector>

#include "actions/actions.h"
#include "events/event_queue.h"
#include "framework/bits.h"
#include "helpers/scratch_allocator.h"
#include "main/policy.h"
#include "main/thread.h"
#include "sfip/sf_cidr.h"

#define DEFAULT_LOG_DIR "."

enum RunFlag
{
    RUN_FLAG__READ                = 0x00000001,
    RUN_FLAG__DAEMON              = 0x00000002,
    RUN_FLAG__DUMP_MSG_MAP        = 0x00000004,
    RUN_FLAG__DUMP_RULE_META      = 0x00000008,

    RUN_FLAG__INLINE              = 0x00000010,
    RUN_FLAG__STATIC_HASH         = 0x00000020,
    RUN_FLAG__CREATE_PID_FILE     = 0x00000040,
    RUN_FLAG__NO_LOCK_PID_FILE    = 0x00000080,

    RUN_FLAG__ALERT_BEFORE_PASS   = 0x00000100,
    RUN_FLAG__CONF_ERROR_OUT      = 0x00000200,
    RUN_FLAG__PROCESS_ALL_EVENTS  = 0x00000400,
    RUN_FLAG__INLINE_TEST         = 0x00000800,

    RUN_FLAG__PCAP_SHOW           = 0x00001000,
    RUN_FLAG__SHOW_FILE_CODES     = 0x00002000,
    RUN_FLAG__PAUSE               = 0x00004000,
    RUN_FLAG__NO_PCRE             = 0x00008000,

    RUN_FLAG__DUMP_RULE_STATE     = 0x00010000,
    RUN_FLAG__DUMP_RULE_DEPS      = 0x00020000,
    RUN_FLAG__TEST                = 0x00040000,
    RUN_FLAG__MEM_CHECK           = 0x00080000,

    RUN_FLAG__TRACK_ON_SYN        = 0x00100000,
    RUN_FLAG__IP_FRAGS_ONLY       = 0x00200000,
    RUN_FLAG__TEST_FEATURES       = 0x00400000,

#ifdef SHELL
    RUN_FLAG__SHELL               = 0x01000000,
#endif
};

enum OutputFlag
{
    OUTPUT_FLAG__LINE_BUFFER       = 0x00000001,
    OUTPUT_FLAG__VERBOSE_DUMP      = 0x00000002,
    OUTPUT_FLAG__CHAR_DATA         = 0x00000004,
    OUTPUT_FLAG__APP_DATA          = 0x00000008,

    OUTPUT_FLAG__SHOW_DATA_LINK    = 0x00000010,
    OUTPUT_FLAG__USE_UTC           = 0x00000020,
    OUTPUT_FLAG__INCLUDE_YEAR      = 0x00000040,
    /* Note using this alters the packet - can't be used inline */
    OUTPUT_FLAG__OBFUSCATE         = 0x00000080,

    OUTPUT_FLAG__ALERT_IFACE       = 0x00000100,
    OUTPUT_FLAG__NO_TIMESTAMP      = 0x00000200,
    OUTPUT_FLAG__ALERTS            = 0x00000400,
    OUTPUT_FLAG__WIDE_HEX          = 0x00000800,

    OUTPUT_FLAG__ALERT_REFS        = 0x00001000,
};

enum LoggingFlag
{
    LOGGING_FLAG__VERBOSE         = 0x00000001,
    LOGGING_FLAG__QUIET           = 0x00000002,
    LOGGING_FLAG__SYSLOG          = 0x00000004,
    LOGGING_FLAG__SHOW_PLUGINS    = 0x00000008,
};

enum TunnelFlags
{
    TUNNEL_GTP    = 0x01,
    TUNNEL_TEREDO = 0x02,
    TUNNEL_6IN4   = 0x04,
    TUNNEL_4IN6   = 0x08,
    TUNNEL_4IN4   = 0x10,
    TUNNEL_6IN6   = 0x20,
    TUNNEL_GRE    = 0x40,
    TUNNEL_MPLS   = 0x80,
    TUNNEL_VXLAN  = 0x100,
    TUNNEL_GENEVE = 0x200
};

enum DumpConfigType
{
    DUMP_CONFIG_NONE = 0,
    DUMP_CONFIG_JSON_ALL,
    DUMP_CONFIG_JSON_TOP,
    DUMP_CONFIG_TEXT
};

class ConfigOutput;
class ControlConn;
class FastPatternConfig;
class RuleStateMap;
class TraceConfig;

struct srmm_table_t;
struct sopg_table_t;
struct ClassType;
struct DetectionFilterConfig;
struct EventQueueConfig;
struct FrameworkConfig;
struct HighAvailabilityConfig;
struct IpsActionsConfig;
struct LatencyConfig;
struct MemoryConfig;
struct PayloadInjectorConfig;
struct Plugins;
struct PORT_RULE_MAP;
struct RateFilterConfig;
struct ReferenceSystem;
struct RuleListNode;
struct RulePortTables;
struct SFDAQConfig;
struct SoRules;
struct ThresholdConfig;

namespace snort
{
class GHash;
class ProtocolReference;
class ReloadResourceTuner;
class ThreadConfig;
class XHash;
struct ProfilerConfig;

struct SnortConfig
{
private:
    void init(const SnortConfig* const, ProtocolReference*, const char* exclude_name);

public:
    SnortConfig(const SnortConfig* const other_conf = nullptr, const char* exclude_name = nullptr);
    SnortConfig(ProtocolReference* protocol_reference);
    ~SnortConfig();

    SnortConfig(const SnortConfig&) = delete;

    void setup();
    void post_setup();
    void update_scratch(ControlConn*);
    bool verify() const;

    void merge(const SnortConfig*);
    void clone(const SnortConfig* const);

private:
    static uint32_t logging_flags;

public:
    static uint32_t warning_flags;

    //------------------------------------------------------
    // alert module stuff
    std::string rule_order;

    SfCidr homenet;

    //------------------------------------------------------
    // output module stuff
#ifdef REG_TEST
    // FIXIT-M builtin modules should set SnortConfig defaults instead
    uint32_t output_flags = OUTPUT_FLAG__WIDE_HEX;
#else
    uint32_t output_flags = 0;
#endif
    uint32_t tagged_packet_limit = 256;
    uint16_t event_trace_max = 0;

    std::string log_dir;

    //------------------------------------------------------
    // daq stuff
    SFDAQConfig* daq_config;

    //------------------------------------------------------
    // detection module stuff
    // FIXIT-L pcre_match_limit* are interdependent
    // somehow a packet thread needs a much lower setting
    long int pcre_match_limit = 1500;
    long int pcre_match_limit_recursion = 1500;

    int pcre_ovector_size = 0;
    bool pcre_override = true;

    uint32_t run_flags = 0;

    unsigned offload_limit = 99999;  // disabled
    unsigned offload_threads = 0;    // disabled

    bool hyperscan_literals = false;
    bool pcre_to_regex = false;

    bool global_rule_state = false;
    bool global_default_rule_state = true;
    bool allow_missing_so_rules = false;
    bool enable_strict_reduction = false;
    uint16_t max_continuations = 1024;

    std::unordered_map<std::string, std::vector<std::string>> service_extension =
        {
             { "http", {"http2", "http3"} },
             { "netbios-ssn", {"dcerpc"} },
        };

    //------------------------------------------------------
    // process stuff

    // user_id and group_id should be initialized to -1 by default, because
    // chown() use this later, -1 means no change to user_id/group_id
    int user_id = -1;
    int group_id = -1;
    uint16_t watchdog_timer = 0;
    uint16_t watchdog_min_thread_count = 1;
    bool dirty_pig = false;

    std::string chroot_dir;        /* -t or config chroot */
    std::string include_path;
    std::string plugin_path;
    std::string rule_db_dir;
    std::vector<std::string> script_paths;

    mode_t file_mask = 0;

    //------------------------------------------------------
    // decode module stuff
    uint8_t num_layers = 0;
    uint8_t max_ip6_extensions = 0;
    uint8_t max_ip_layers = 0;

    bool enable_esp = false;
    bool address_anomaly_check_enabled = false;

    //------------------------------------------------------
    // active stuff
    uint8_t respond_attempts = 0;
    uint8_t max_responses = 0;
    uint8_t min_interval = 0;
    uint8_t* eth_dst = nullptr;

    std::string respond_device;
    std::string output;

    //------------------------------------------------------
    // attribute tables stuff
    std::string attribute_hosts_file;
    uint32_t max_attribute_hosts = 0;
    uint32_t max_attribute_services_per_host = 0;
    uint32_t max_metadata_services = 0;
    uint32_t segment_count_host = 4;

    //------------------------------------------------------
    // packet module stuff
    bool asid_agnostic = false;
    bool mpls_agnostic = true;
    bool vlan_agnostic = false;

    uint64_t pkt_cnt = 0;           /* -n */
    uint64_t pkt_skip = 0;
    uint64_t pkt_pause_cnt = 0;

    std::string bpf_file;          /* -F or config bpf_file */

    //------------------------------------------------------
    // various modules
    FastPatternConfig* fast_pattern_config = nullptr;
    EventQueueConfig* event_queue_config = nullptr;
    PayloadInjectorConfig* payload_injector_config = nullptr;

    /* policy specific? */
    ThresholdConfig* threshold_config = nullptr;
    RateFilterConfig* rate_filter_config = nullptr;
    DetectionFilterConfig* detection_filter_config = nullptr;

    //------------------------------------------------------
    // FIXIT-L command line only stuff, add to conf / module

    uint16_t event_log_id = 0;
    SfCidr obfuscation_net;
    std::string bpf_filter;
    std::string metadata_filter;

    //------------------------------------------------------
    // FIXIT-L non-module stuff - separate config from derived state?
    std::string run_prefix;
    uint16_t id_offset = 0;
    bool id_subdir = false;
    bool id_zero = false;

    bool stdin_rules = false;

    std::string pid_filename;
    std::string orig_log_dir;      /* set in case of chroot */

    int thiszone = 0;

    std::unordered_map<std::string, ClassType*> classifications;
    std::unordered_map<std::string, ReferenceSystem*> references;

    RuleStateMap* rule_states = nullptr;
    GHash* otn_map = nullptr;

    ProtocolReference* proto_ref = nullptr;

    unsigned num_rule_types = 0;
    RuleListNode* rule_lists = nullptr;
    int* evalOrder = nullptr;

    IpsActionsConfig* ips_actions_config = nullptr;
    FrameworkConfig* framework_config = nullptr;

    /* master port list table */
    RulePortTables* port_tables = nullptr;

    /* The port-rule-maps map the src-dst ports to rules for
     * udp and tcp, for Ip we map the dst port as the protocol,
     * and for Icmp we map the dst port to the Icmp type. This
     * allows us to use the decode packet information to in O(1)
     * select a group of rules to apply to the packet.  These
     * rules may or may not have content.  We process the content
     * 1st and then the no content rules for udp/tcp and icmp, and
     * then we process the ip rules. */
    PORT_RULE_MAP* prmIpRTNX = nullptr;
    PORT_RULE_MAP* prmIcmpRTNX = nullptr;
    PORT_RULE_MAP* prmTcpRTNX = nullptr;
    PORT_RULE_MAP* prmUdpRTNX = nullptr;

    srmm_table_t* srmmTable = nullptr;   /* srvc rule map master table */
    srmm_table_t* spgmmTable = nullptr;  /* srvc port_group map master table */
    sopg_table_t* sopgTable = nullptr;   /* service-ordinal to port_group table */

    XHash* detection_option_hash_table = nullptr;
    XHash* detection_option_tree_hash_table = nullptr;
    XHash* rtn_hash_table = nullptr;

    PolicyMap* policy_map = nullptr;
    std::string tweaks;

    DataBus* global_dbus = nullptr;

    uint16_t tunnel_mask = 0;

    int16_t max_aux_ip = 16;

    // FIXIT-L this is temporary for legacy paf_max required only for HI;
    // it is not appropriate for multiple stream_tcp with different
    // paf_max; the HI splitter should pull from there
    unsigned max_pdu = 16384;

    //------------------------------------------------------
    ProfilerConfig* profiler = nullptr;
    LatencyConfig* latency = nullptr;

    unsigned remote_control_port = 0;
    std::string remote_control_socket;

    MemoryConfig* memory = nullptr;
    //------------------------------------------------------

    std::vector<ScratchAllocator*> scratchers;
    std::vector<void *>* state = nullptr;
    unsigned num_slots = 0;

    ThreadConfig* thread_config;
    HighAvailabilityConfig* ha_config = nullptr;
    TraceConfig* trace_config = nullptr;

    // TraceConfig instance which used by TraceSwap control channel command
    TraceConfig* overlay_trace_config = nullptr;

    //------------------------------------------------------
    //Reload inspector related

    bool cloned = false;
    Plugins* plugins = nullptr;
    SoRules* so_rules = nullptr;

    DumpConfigType dump_config_type = DUMP_CONFIG_NONE;
private:
    std::list<ReloadResourceTuner*> reload_tuners;
    static std::mutex reload_id_mutex;
    unsigned reload_id = 0;
    static std::mutex static_names_mutex;
    static std::unordered_map<std::string, std::string> static_names;

public:
    //------------------------------------------------------
    // decoding related
    uint8_t get_num_layers() const
    { return num_layers; }

    // curr_layer is the zero based ip6 options
    bool hit_ip6_maxopts(uint8_t curr_opt) const
    { return max_ip6_extensions && (curr_opt >= max_ip6_extensions); }

    // curr_ip is the zero based ip layer
    bool hit_ip_maxlayers(uint8_t curr_ip) const
    { return max_ip_layers && (curr_ip >= max_ip_layers); }

    //------------------------------------------------------
    // Non-static mutator methods

    void add_plugin_path(const char*);
    void add_script_path(const char*);
    void enable_syslog();
    void set_alert_before_pass(bool);
    void set_alert_mode(const char*);
    void set_chroot_dir(const char*);
    void set_create_pid_file(bool);
    void set_daemon(bool);
    void set_decode_data_link(bool);
    void set_dirty_pig(bool);
    void set_dst_mac(const char*);
    void set_dump_chars_only(bool);
    void set_dump_payload(bool);
    void set_dump_payload_verbose(bool);
    void set_gid(const char*);
    void set_log_dir(const char*);
    void set_log_mode(const char*);
    void set_no_logging_timestamps(bool);
    void set_obfuscate(bool);
    void set_obfuscation_mask(const char*);
    void set_overlay_trace_config(TraceConfig*);
    void set_include_path(const char*);
    void set_process_all_events(bool);
    void set_rule_db_dir(const char*);
    void set_show_year(bool);
    void set_tunnel_verdicts(const char*);
    void set_tweaks(const char*);
    void set_uid(const char*);
    void set_umask(uint32_t);
    void set_utc(bool);
    void set_watchdog(uint16_t);
    void set_watchdog_min_thread_count(uint16_t);
    SO_PUBLIC bool set_latency_enable();

    //------------------------------------------------------
    // accessor methods

    bool esp_decoding() const
    { return enable_esp; }

    bool is_address_anomaly_check_enabled() const
    { return address_anomaly_check_enabled; }

    bool aux_ip_is_enabled() const
    { return max_aux_ip >= 0; }

    // mode related
    bool dump_config_mode() const
    { return dump_config_type > DUMP_CONFIG_NONE; }

    bool dump_msg_map() const
    { return run_flags & RUN_FLAG__DUMP_MSG_MAP; }

    bool dump_rule_meta() const
    { return run_flags & RUN_FLAG__DUMP_RULE_META; }

    bool dump_rule_state() const
    { return run_flags & RUN_FLAG__DUMP_RULE_STATE; }

    bool dump_rule_deps() const
    { return run_flags & RUN_FLAG__DUMP_RULE_DEPS; }

    bool dump_rule_info() const
    { return dump_msg_map() or dump_rule_meta() or dump_rule_deps() or dump_rule_state(); }

    bool test_mode() const
    { return run_flags & RUN_FLAG__TEST; }

    bool mem_check() const
    { return run_flags & RUN_FLAG__MEM_CHECK; }

    bool daemon_mode() const
    { return run_flags & RUN_FLAG__DAEMON; }

    bool read_mode() const
    { return run_flags & RUN_FLAG__READ; }

    bool ips_inline_mode() const
    {
        // cppcheck-suppress nullPointer
        return get_ips_policy()->policy_mode == POLICY_MODE__INLINE;
    }

    bool ips_inline_test_mode() const
    { return get_ips_policy()->policy_mode == POLICY_MODE__INLINE_TEST; }

    bool nap_inline_mode() const
    { return get_inspection_policy()->policy_mode == POLICY_MODE__INLINE; }

    bool ips_passive_mode() const
    { return get_ips_policy()->policy_mode == POLICY_MODE__PASSIVE; }

    bool show_file_codes() const
    { return run_flags & RUN_FLAG__SHOW_FILE_CODES; }

    bool adaptor_inline_mode() const
    { return run_flags & RUN_FLAG__INLINE; }

    bool adaptor_inline_test_mode() const
    { return run_flags & RUN_FLAG__INLINE_TEST; }

    // event stuff
    uint16_t get_event_log_id() const
    { return event_log_id; }

    bool process_all_events() const
    { return event_queue_config->process_all_events; }

    int get_eval_index(Actions::Type type) const
    { return evalOrder[type]; }

    // output stuff
    bool output_include_year() const
    { return output_flags & OUTPUT_FLAG__INCLUDE_YEAR; }

    bool output_use_utc() const
    { return output_flags & OUTPUT_FLAG__USE_UTC; }

    bool output_datalink() const
    { return output_flags & OUTPUT_FLAG__SHOW_DATA_LINK; }

    bool verbose_byte_dump() const
    { return output_flags & OUTPUT_FLAG__VERBOSE_DUMP; }

    bool obfuscate() const
    { return output_flags & OUTPUT_FLAG__OBFUSCATE; }

    bool output_app_data() const
    { return output_flags & OUTPUT_FLAG__APP_DATA; }

    bool output_char_data() const
    { return output_flags & OUTPUT_FLAG__CHAR_DATA; }

    bool alert_interface() const
    { return output_flags & OUTPUT_FLAG__ALERT_IFACE; }

    bool output_no_timestamp() const
    { return output_flags & OUTPUT_FLAG__NO_TIMESTAMP; }

    bool line_buffered_logging() const
    { return output_flags & OUTPUT_FLAG__LINE_BUFFER; }

    bool output_wide_hex() const
    { return output_flags & OUTPUT_FLAG__WIDE_HEX; }

    bool alert_refs() const
    { return output_flags & OUTPUT_FLAG__ALERT_REFS; }

    // run flags
    bool no_lock_pid_file() const
    { return run_flags & RUN_FLAG__NO_LOCK_PID_FILE; }

    bool create_pid_file() const
    { return run_flags & RUN_FLAG__CREATE_PID_FILE; }

    bool pcap_show() const
    { return run_flags & RUN_FLAG__PCAP_SHOW; }

    bool alert_before_pass() const
    { return run_flags & RUN_FLAG__ALERT_BEFORE_PASS; }

    bool no_pcre() const
    { return run_flags & RUN_FLAG__NO_PCRE; }

    bool conf_error_out() const
    { return run_flags & RUN_FLAG__CONF_ERROR_OUT; }

    bool test_features() const
    { return run_flags & RUN_FLAG__TEST_FEATURES; }

    // other stuff
    uint8_t min_ttl() const
    { return get_network_policy()->min_ttl; }

    uint8_t new_ttl() const
    { return get_network_policy()->new_ttl; }

    long int get_pcre_match_limit() const
    { return pcre_match_limit; }

    long int get_pcre_match_limit_recursion() const
    { return pcre_match_limit_recursion; }

    const ProfilerConfig* get_profiler() const
    { return profiler; }

    long int get_tagged_packet_limit() const
    { return tagged_packet_limit; }

    uint32_t get_max_attribute_hosts() const
    { return max_attribute_hosts; }

    uint32_t get_segment_count_host() const
    { return segment_count_host; }

    uint32_t get_max_services_per_host() const
    { return max_attribute_services_per_host; }

    int get_uid() const
    { return user_id; }

    int get_gid() const
    { return group_id; }

    bool get_mpls_agnostic() const
    { return mpls_agnostic; }

    bool get_vlan_agnostic() const
    { return vlan_agnostic; }

    bool address_space_agnostic() const
    { return asid_agnostic; }

    bool change_privileges() const
    { return user_id != -1 || group_id != -1 || !chroot_dir.empty(); }

    bool track_on_syn() const
    { return (run_flags & RUN_FLAG__TRACK_ON_SYN) != 0; }

    bool ip_frags_only() const
    { return (run_flags & RUN_FLAG__IP_FRAGS_ONLY) != 0; }

    void clear_run_flags(RunFlag flag)
    { run_flags &= ~flag; }

    void set_run_flags(RunFlag flag)
    { run_flags |= flag; }

    const std::list<ReloadResourceTuner*>& get_reload_resource_tuners() const
    { return reload_tuners; }

    void clear_reload_resource_tuner_list();

    void update_reload_id();

    unsigned get_reload_id() const
    { return reload_id; }

    bool get_default_rule_state() const;

    ConfigOutput* create_config_output() const;

    SO_PUBLIC bool tunnel_bypass_enabled(uint16_t proto) const;

    // FIXIT-L snort_conf needed for static hash before initialized
    static bool static_hash()
    { return get_conf() && get_conf()->run_flags & RUN_FLAG__STATIC_HASH; }

    // This requests an entry in the scratch space vector and calls setup /
    // cleanup as appropriate
    SO_PUBLIC static int request_scratch(ScratchAllocator*);
    SO_PUBLIC static void release_scratch(int);

    // runtime access to const config - especially for packet threads
    // prefer access via packet->context->conf
    SO_PUBLIC static const SnortConfig* get_conf();
    // Thread local copy of the reload_id needed for commands that cause reevaluation
    SO_PUBLIC static unsigned get_thread_reload_id();
    SO_PUBLIC static void update_thread_reload_id();

    // runtime access to mutable config - main thread only, and only special cases
    SO_PUBLIC static SnortConfig* get_main_conf();

    static void set_conf(const SnortConfig*);

    SO_PUBLIC void register_reload_handler(ReloadResourceTuner*);

    static void cleanup_fatal_error();

    // logging stuff
    static void enable_log_syslog()
    { logging_flags |= LOGGING_FLAG__SYSLOG; }

    static bool log_syslog()
    { return logging_flags & LOGGING_FLAG__SYSLOG; }

    static void set_log_quiet(bool enabled)
    {
        if (enabled)
            logging_flags |= LOGGING_FLAG__QUIET;
        else
            logging_flags &= ~LOGGING_FLAG__QUIET;
    }

    static bool log_quiet()
    { return logging_flags & LOGGING_FLAG__QUIET; }

    static void enable_log_verbose()
    { logging_flags |= LOGGING_FLAG__VERBOSE; }

    static bool log_verbose()
    { return logging_flags & LOGGING_FLAG__VERBOSE; }

    static void enable_log_show_plugins()
    { logging_flags |= LOGGING_FLAG__SHOW_PLUGINS; }

    static bool log_show_plugins()
    { return logging_flags & LOGGING_FLAG__SHOW_PLUGINS; }

    SO_PUBLIC static const char* get_static_name(const char* name);
};
}

#endif

snort.h

Path = src/main/snort.h

#ifndef SNORT_H
#define SNORT_H

// Snort is the top-level application class.
#include <daq_common.h>

#include "main/snort_types.h"

class ContextSwitcher;

namespace snort
{
class Flow;
class SFDAQInstance;
struct Packet;
struct SnortConfig;

class Snort
{
public:
    static SnortConfig* get_reload_config(const char* fname, const char* plugin_path,
        const SnortConfig* old);
    static SnortConfig* get_updated_policy(SnortConfig*, const char* fname, const char* iname);
    static void setup(int argc, char* argv[]);
    static bool drop_privileges();
    static void do_pidfile();
    static void cleanup();

    static bool has_dropped_privileges();
    SO_PUBLIC static bool is_reloading();
    inline SO_PUBLIC static bool is_exiting() { return already_exiting; }

private:
    static void init(int, char**);
    static void term();
    static void clean_exit(int);
    static void reload_failure_cleanup(SnortConfig*);

private:
    static bool initializing;
    static bool reloading;
    static bool privileges_dropped;
    static bool already_exiting;
};

// RAII-style mechanism for removal and reinstallation of Snort's crash handler
class SO_PUBLIC OopsHandlerSuspend
{
public:
    OopsHandlerSuspend();
    ~OopsHandlerSuspend();
};
}

#endif

snort_module.h

Path = src/main/snort_module.h

#ifndef SNORT_MODULE_H
#define SNORT_MODULE_H

// the snort module is for handling command line args,
// shell commands, and basic application stats

#include "main/thread.h"

namespace snort
{
class Module;
class Trace;
}

extern THREAD_LOCAL const snort::Trace* snort_trace;

snort::Module* get_snort_module();

enum
{
    TRACE_INSPECTOR_MANAGER = 0,
    TRACE_MAIN,
};

#endif

snort_types.h

Path = src/main/snort_types.h

#ifndef SNORT_TYPES_H
#define SNORT_TYPES_H

// defines common types if not already defined

#include <cinttypes>
#include <cstddef>
#include <cstdint>

typedef uint16_t Port;

/* use these macros for 64 bit format portability */
#define STDu64 "%" PRIu64
#define FMTu64(fmt) "%" fmt PRIu64

#define STDi64 "%" PRIi64
#define FMTi64(fmt) "%" fmt PRIi64

#define STDx64 "%" PRIx64
#define FMTx64(fmt) "%" fmt PRIx64

#define UNUSED(x) (void)(x)

#ifdef NDEBUG
#define NORETURN_ASSERT
#else
#define NORETURN_ASSERT [[noreturn]]
#endif

#ifndef SO_PUBLIC
#if defined _WIN32 || defined __CYGWIN__
#  ifdef __GNUC__
#    define SO_PUBLIC __attribute__((dllimport))
#  else
#    define SO_PUBLIC __declspec(dllimport)
#  endif
#  define DLL_LOCAL
#else
#  ifdef HAVE_VISIBILITY
#    define SO_PUBLIC  __attribute__ ((visibility("default")))
#    define SO_PRIVATE __attribute__ ((visibility("hidden")))
#  else
#    define SO_PUBLIC
#    define SO_PRIVATE
#  endif
#endif
#endif

#if !defined(__GNUC__) || __GNUC__ < 2 || \
    (__GNUC__ == 2 && __GNUC_MINOR__ < 5)
#define __attribute__(x)    /* delete __attribute__ if non-gcc or gcc1 */
#endif

#endif

swapper.h

Path = src/main/swapper.h

#ifndef SWAPPER_H
#define SWAPPER_H

// used to make thread local, pointer-based config swaps by packet threads

#include "main/snort_types.h"

namespace snort
{
struct SnortConfig;
}

class Analyzer;

class SO_PUBLIC Swapper
{
public:
    Swapper(snort::SnortConfig*);
    Swapper(const snort::SnortConfig* sold, snort::SnortConfig* snew);
    Swapper();
    ~Swapper();

    void apply(Analyzer&);
    void finish(Analyzer&);
    snort::SnortConfig* get_new_conf() { return new_conf; }

private:
    const snort::SnortConfig* old_conf;
    snort::SnortConfig* new_conf;
};

#endif

thread_config.h

Path = src/main/thread_config.h

#ifndef THREAD_CONFIG_H
#define THREAD_CONFIG_H

#include <hwloc.h>
#include <map>
#include <string>

#include "main/thread.h"

struct CpuSet;

namespace snort
{

class SO_PUBLIC ThreadConfig
{
public:
    static bool init();
    static CpuSet* validate_cpuset_string(const char*);
    static void destroy_cpuset(CpuSet*);
    static void set_instance_max(unsigned);
    static unsigned get_instance_max();
    static void term();
    static void start_watchdog();
    static void preemptive_kick();
    static void set_instance_tid(int);
    static int get_instance_tid(int);

    ~ThreadConfig();
    void apply_thread_policy(SThreadType type, unsigned id);
    void set_thread_affinity(SThreadType, unsigned id, CpuSet*);
    void set_named_thread_affinity(const std::string&, CpuSet*);
    void implement_thread_affinity(SThreadType, unsigned id);
    void implement_named_thread_affinity(const std::string& name);
    bool implement_thread_mempolicy(SThreadType type, unsigned id);

    static constexpr unsigned int DEFAULT_THREAD_ID = 0;

private:
    struct TypeIdPair
    {
        SThreadType type;
        unsigned id;
    };
    struct TypeIdPairComparer
    {
        bool operator()(const TypeIdPair& left, const TypeIdPair& right) const
        {
            if (left.type < right.type)
                return true;
            if (left.type == right.type && left.id < right.id)
                return true;
            return false;
        }
    };
    std::map<TypeIdPair, CpuSet*, TypeIdPairComparer> thread_affinity;
    std::map<std::string, CpuSet*> named_thread_affinity;

    bool set_preferred_mempolicy(int node);
    int get_numa_node(hwloc_topology_t, hwloc_cpuset_t);
};
}
#endif

thread.h

Path = src/main/thread.h

#ifndef THREAD_H
#define THREAD_H

// basic thread management utilities

#include <string>

#include "main/snort_types.h"

#define THREAD_LOCAL_TBD
//#define THREAD_LOCAL // for single-threaded debugging

// `__thread` is a gnu extension that at present is slightly faster than
// `thread_local` (possibly due to the lack of dynamic initialization)
#ifdef USE_THREAD_LOCAL
#    define THREAD_LOCAL thread_local
#else
#    define THREAD_LOCAL __thread
#endif

enum SThreadType
{
    STHREAD_TYPE_OTHER,
    STHREAD_TYPE_PACKET,
    STHREAD_TYPE_MAIN
};

void set_instance_id(unsigned);
void set_thread_type(SThreadType);

void set_run_num(uint16_t);
uint16_t get_run_num();

namespace snort
{
SO_PUBLIC unsigned get_instance_id();
SO_PUBLIC SThreadType get_thread_type();

SO_PUBLIC inline bool in_main_thread()
{ return get_thread_type() == STHREAD_TYPE_MAIN; }

SO_PUBLIC inline bool is_packet_thread()
{ return get_thread_type() == STHREAD_TYPE_PACKET; }

// all modules that use packet thread files should call this function to
// get a packet thread specific path.  name should be the module name or
// derived therefrom.
SO_PUBLIC const char* get_instance_file(std::string&, const char* name);
}

#endif

main/test/

managers/

Managers provide the ability to instantiate and utilize the various plugins in Snort++. There is a separate manager class for each plugin type, plus one for modules, plugins, scripts, and packets.

The only plugin that is reloadable is Inspector. It has reference counts so that it won’t be freed while an active flow is using it.

Only the action, codec, and inspector managers have thread local state:

  • action manager has an action function

  • codec manager has the grinder and related stats

  • inspector manager has a flag to control calling the clear method

Some Lua files are here as they are coupled closely with C++ code in this directory (module_manager.cc):

  • bootstrap.lua provides FFI for adding table names and prefixes to snort’s lua whitelist. This whitelist is used to ignore warnings when snort modules are not found for table names. This file also has the FFI used to resolve file includes in lua file.

  • finalize.lua provides the ability to parse a Lua configuration. It is much easier to traverse the Lua tables via Lua itself. This file leverages the LuaJIT FFI to open and close tables and set values.

  • snort_plugin.lua provides FFI for script plugins (ips options and loggers). This is pretty thin at the moment and should be expanded to provide more data.

These Lua files get installed in LUA_PATH.

Module manager recursively sets default values for all parameters within a module. While list items have default values, default lists are not provided by modules; that is strictly done in Lua with snort_defaults.lua. This not only simplifies the code somewhat, it also makes the most sense from a user perspective.

The ConnectorManager (and associated) classes manage the set of Connector objects. One ConnectorCommon is created to encapsulate a vector of configuration objects. At thread startup, these config objects are used to instantiate thread specific instances of each configured Connector.

action_manager.h

Path = src/managers/action_manager.h

#ifndef ACTION_MANAGER_H
#define ACTION_MANAGER_H

// Factory for IpsActions.  Also manages their associated action queue,
// which is just a single response deferred until end of current packet
// processing.

#include "actions/actions.h"
#include "framework/ips_action.h"
#include "framework/module.h"

namespace snort
{
struct ActionApi;
class IpsAction;
struct SnortConfig;
struct Packet;
}

struct IpsPolicy;

//-------------------------------------------------------------------------

class ActionManager
{
public:
    static void add_plugin(const snort::ActionApi*);
    static void release_plugins();
    static void dump_plugins();

    static void new_config(snort::SnortConfig*);
    static void delete_config(snort::SnortConfig*);

    static void instantiate(const snort::ActionApi*, snort::Module*,
            snort::SnortConfig*, IpsPolicy* ips = nullptr );
    static void initialize_policies(snort::SnortConfig*);

    static std::string get_action_string(Actions::Type);
    static Actions::Type get_action_type(const char*);
    static Actions::Type get_max_action_types();
    static std::string get_action_priorities(bool);

    static void thread_init(const snort::SnortConfig*);
    static void thread_reinit(const snort::SnortConfig*);
    static void thread_term();
};

#endif

codec_manager.h

Path = src/managers/codec_manager.h

#ifndef MANAGERS_CODEC_MANAGER_H
#define MANAGERS_CODEC_MANAGER_H

// Factory for Codecs.  Runtime support is provided by PacketManager.

#include <array>
#include <vector>

#include "main/thread.h"
#include "protocols/protocol_ids.h"

namespace snort
{
class Codec;
struct CodecApi;
class Module;
class PacketManager;
struct ProfileStats;
struct SnortConfig;
}

//-------------------------------------------------------------------------

extern THREAD_LOCAL snort::ProfileStats decodePerfStats;

/*
 *  CodecManager class
 */
class CodecManager
{
public:
    friend class snort::PacketManager;

    // global plugin initializer
    static void add_plugin(const struct snort::CodecApi*);
    // instantiate a specific codec with a codec specific Module
    static void instantiate(const snort::CodecApi*, snort::Module*, snort::SnortConfig*);
    // instantiate any codec for which a module has not been provided.
    static void instantiate();
    // destroy all global codec related information
    static void release_plugins();
    // initialize the current threads DLT and Packet struct
    static void thread_init(const snort::SnortConfig*);
    // destroy thread_local data
    static void thread_term();
    // print all of the codec plugins
    static void dump_plugins();

    static uint8_t get_max_layers()
    { return max_layers; }

private:
    struct CodecApiWrapper;

    static std::vector<CodecApiWrapper> s_codecs;
    static std::array<ProtocolIndex, num_protocol_ids> s_proto_map;
    static std::array<snort::Codec*, UINT8_MAX> s_protocols;

    static THREAD_LOCAL ProtocolId grinder_id;
    static THREAD_LOCAL ProtocolIndex grinder;
    static THREAD_LOCAL uint8_t max_layers;

    /*
     * Private helper functions.  These are all declared here
     * because they need access to private variables.
     */

    // Private struct defined in an anonymous namespace.
    static void instantiate(CodecApiWrapper&, snort::Module*, snort::SnortConfig*);
    static CodecApiWrapper& get_api_wrapper(const snort::CodecApi* cd_api);
    static uint8_t get_codec(const char* const keyword);
};

#endif

connector_manager.h

Path = src/managers/connector_manager.h

#ifndef CONNECTOR_MANAGER_H
#define CONNECTOR_MANAGER_H

// Factory for Connectors.

#include <string>

namespace snort
{
class Connector;
struct ConnectorApi;
class Module;
struct SnortConfig;
}

//-------------------------------------------------------------------------

class ConnectorManager
{
public:
    static void add_plugin(const snort::ConnectorApi* api);
    static void dump_plugins();
    static void release_plugins();

    static void instantiate(const snort::ConnectorApi*, snort::Module*, snort::SnortConfig*);

    static void thread_init();
    static void thread_term();

    /* get_connector() returns the thread-specific object. */
    static snort::Connector* get_connector(const std::string& name);
};

#endif

event_manager.h

Path = src/managers/event_manager.h

#ifndef EVENT_MANAGER_H
#define EVENT_MANAGER_H

// Factory for Loggers.
// OutputSet is a group of Loggers that can be attached to external data.
// Also provides runtime logging.

#include "framework/logger.h"
#include "framework/module.h"

#define OUTPUT_TYPE_FLAG__NONE  0x0
#define OUTPUT_TYPE_FLAG__ALERT 0x1
#define OUTPUT_TYPE_FLAG__LOG   0x2

namespace snort
{
struct LogApi;
struct Packet;
struct SnortConfig;
}
struct Event;
struct OutputSet;

//-------------------------------------------------------------------------

class EventManager
{
public:
    static void add_plugin(const snort::LogApi*);
    static void dump_plugins();
    static void release_plugins();

    static void instantiate(const char*, snort::SnortConfig*);
    static void instantiate(const snort::LogApi*, snort::Module*, snort::SnortConfig*);

    static unsigned get_output_type_flags(char*);

    static void add_output(OutputSet**, snort::Logger*);
    static void copy_outputs(OutputSet* dst, OutputSet* src);
    static void release_outputs(OutputSet*);

    static void open_outputs();
    static void close_outputs();
    static void reload_outputs();

    static void call_alerters(OutputSet*, snort::Packet*, const char* message, const Event&);
    static void call_loggers(OutputSet*, snort::Packet*, const char* message, Event*);

    static void enable_alerts(bool b) { alert_enabled = b; }
    static void enable_logs(bool b) { log_enabled = b; }

private:
    static void instantiate(struct Output*, snort::Module*, snort::SnortConfig*);

    static bool alert_enabled;
    static bool log_enabled;
};

#endif

inspector_manager.h

Path = src/managers/inspector_manager.h

#ifndef INSPECTOR_MANAGER_H
#define INSPECTOR_MANAGER_H

// Factory for Inspectors.
// Also provides packet evaluation.

#include <map>

#include "framework/inspector.h"
#include "framework/module.h"

class Binder;
class SingleInstanceInspectorPolicy;
struct InspectorList;
struct InspectionPolicy;
struct NetworkPolicy;
struct PHInstance;
struct GlobalInspectorPolicy;

namespace snort
{
struct Packet;
struct SnortConfig;

//-------------------------------------------------------------------------

class InspectorManager
{
public:
    static void add_plugin(const InspectApi* api);
    static void dump_plugins();
    static void dump_buffers();
    static void release_plugins();

    static void global_init();

    static std::vector<const InspectApi*> get_apis();
    static const char* get_inspector_type(const char* name);

    static void new_policy(NetworkPolicy*, NetworkPolicy*);
    static void delete_policy(NetworkPolicy*, bool cloned);

    static void new_policy(InspectionPolicy*, InspectionPolicy*);
    static void delete_policy(InspectionPolicy*, bool cloned);

    static void update_policy(SnortConfig* sc);

    static void new_config(SnortConfig*);
    static void delete_config(SnortConfig*);

    static void instantiate(
        const InspectApi*, Module*, SnortConfig*, const char* name = nullptr);

    static bool delete_inspector(SnortConfig*, const char* iname);
    static void free_inspector(Inspector*);
    static SingleInstanceInspectorPolicy* create_single_instance_inspector_policy();
    static void destroy_single_instance_inspector(SingleInstanceInspectorPolicy*);
    static GlobalInspectorPolicy* create_global_inspector_policy(GlobalInspectorPolicy* = nullptr);
    static void destroy_global_inspector_policy(GlobalInspectorPolicy*, bool cloned);
    static InspectSsnFunc get_session(uint16_t proto);

    SO_PUBLIC static Inspector* get_file_inspector(const SnortConfig* = nullptr);

    // This assumes that, in a multi-tenant scenario, this is called with the correct network and inspection
    // policies are set correctly
    SO_PUBLIC static Inspector* get_inspector(const char* key, bool dflt_only = false, const SnortConfig* = nullptr);

    // This cannot be called in or before the inspector configure phase for a new snort config during reload
    SO_PUBLIC static Inspector* get_inspector(const char* key, Module::Usage, InspectorType);

    SO_PUBLIC static Inspector* get_service_inspector_by_service(const char*);
    static Inspector* get_service_inspector_by_id(const SnortProtocolId);

    SO_PUBLIC static Binder* get_binder();

    SO_PUBLIC static Inspector* acquire_file_inspector();
    SO_PUBLIC static void release(Inspector*);

    static bool configure(SnortConfig*, bool cloned = false);
    static void prepare_inspectors(SnortConfig*);
    static void prepare_controls(SnortConfig*);
    static std::string generate_inspector_label(const PHInstance*);
    static void print_config(SnortConfig*);

    static void thread_init(const SnortConfig*);
    static void thread_reinit(const SnortConfig*);
    static void thread_stop_removed(const SnortConfig*);

    static void thread_stop(const SnortConfig*);
    static void thread_term();

    static void execute(Packet*);
    static void probe(Packet*);

    static void clear(Packet*);
    static void empty_trash();
    static void reconcile_inspectors(const SnortConfig*, SnortConfig*, bool cloned = false);
    static void clear_removed_inspectors(SnortConfig*);

private:
    static void bumble(Packet*);
    template<bool T> static void full_inspection(Packet*);
    template<bool T> static void internal_execute(Packet*);
    static void sort_inspector_list(const InspectorList* il,
        std::map<const std::string, const PHInstance*>& sorted_ilist);
};
}
#endif

ips_manager.h

Path = src/managers/ips_manager.h

#ifndef IPS_MANAGER_H
#define IPS_MANAGER_H

// Factory for IpsOptions.
// Runtime use of IpsOptions is via detection option tree.

#include "detection/detection_options.h"
#include "framework/ips_option.h"
#include "framework/module.h"

namespace snort
{
    struct IpsApi;
    class IpsOption;
    struct SnortConfig;
}

//-------------------------------------------------------------------------

class IpsManager
{
public:
    static void add_plugin(const snort::IpsApi*);
    static void dump_plugins();
    static void release_plugins();
    static void instantiate(const snort::IpsApi*, snort::Module*, snort::SnortConfig*);

    static bool option_begin(snort::SnortConfig*, const char* key, SnortProtocolId);
    static bool option_set(
        snort::SnortConfig*, const char* key, const char* opt, const char* val);
    static snort::IpsOption* option_end(
        snort::SnortConfig*, OptTreeNode*, SnortProtocolId, const char* key, snort::RuleOptType&);

    static void delete_option(snort::IpsOption*);
    static const char* get_option_keyword();

    SO_PUBLIC static const snort::IpsApi* get_option_api(const char* keyword);

    static void global_init(const snort::SnortConfig*);
    static void global_term(const snort::SnortConfig*);

    static void reset_options();
    static void setup_options(const snort::SnortConfig*);
    static void clear_options(const snort::SnortConfig*);

    static bool verify(snort::SnortConfig*);
};

#endif

lua_plugin_defs.h

Path = src/managers/lua_plugin_defs.h

#ifndef LUA_PLUGIN_DEFS_H
#define LUA_PLUGIN_DEFS_H

#include <cstdint>

struct SnortBuffer
{
    const char* type;
    const uint8_t* data;
    unsigned len;
};

extern "C"
const struct SnortBuffer* get_buffer();

struct SnortEvent
{
    unsigned gid;
    unsigned sid;
    unsigned rev;

    uint32_t event_id;
    uint32_t event_ref;

    const char* msg;
    const char* svc;
};

extern "C"
const struct SnortEvent* get_event();

struct SnortPacket
{
    // FIXIT-L add ip addrs and other useful foo to lua packet
    const char* type;
    uint64_t num;
    unsigned sp;
    unsigned dp;
};

extern "C"
const struct SnortPacket* get_packet();

#endif

module_manager.h

Path = src/managers/module_manager.h

#ifndef MODULE_MANAGER_H
#define MODULE_MANAGER_H

// Factory for Modules, including all builtin and plugin modules.
// Modules are strictly used during parse time.

#include <cstdint>
#include <list>
#include <mutex>
#include <set>
#include <string>

#include "framework/counts.h"
#include "main/analyzer_command.h"
#include "main/snort_types.h"

//-------------------------------------------------------------------------

class Shell;

namespace snort
{
struct BaseApi;
class Module;
struct SnortConfig;

class ModuleManager
{
public:
    static void init();
    static void term();

    static void add_module(Module*, const BaseApi* = nullptr);
    SO_PUBLIC static Module* get_module(const char*);
    static Module* get_default_module(const char*, SnortConfig*);
    SO_PUBLIC static std::list<Module*> get_all_modules();

    static const char* get_lua_coreinit();
    static const char* get_includer(const char* module);

    static void list_modules(const char* = nullptr);
    static void dump_modules();
    static void show_modules();
    static void show_modules_json();
    static void show_module(const char*);

    static bool gid_in_use(uint32_t);

    // output for matching module name; prefix is sufficient if not exact
    static void show_configs(const char* = nullptr, bool exact = false);
    static void show_commands(const char* = nullptr, bool exact = false);
    static void show_gids(const char* = nullptr, bool exact = false);
    static void show_pegs(const char* = nullptr, bool exact = false);
    static void show_rules(const char* = nullptr, bool exact = false);

    static void dump_rules(const char* = nullptr, const char* opts = nullptr);
    static void dump_defaults(const char* = nullptr);

    static void load_params();
    static const struct Parameter* get_parameter(const char* table, const char* option);

    static void load_commands(Shell*);
    static void load_rules(SnortConfig*);
    static void set_config(SnortConfig*);

    static void reset_errors();
    static unsigned get_errors();

    static PegCount* get_stats(const char* name);
    static void dump_stats(const char* skip = nullptr, bool dynamic = false);
    static void accumulate_dump_stats();
    static void init_stats();
    static void add_thread_stats_entry(const char* name);

    static void accumulate(const char* except = "snort");
    static void accumulate_module(const char* name);

    static void reset_stats(SnortConfig*);
    static void reset_stats(clear_counter_type_t);
    static void reset_module_stats(const char* name);

    static void clear_global_active_counters();
    static bool is_parallel_cmd(std::string control_cmd);
    static std::string remove_whitespace(std::string& control_cmd);


    static std::set<uint32_t> gids;
    SO_PUBLIC static std::mutex stats_mutex;
    SO_PUBLIC static const char* dynamic_stats_modules;
};
}

#endif

mpse_manager.h

Path = src/managers/mpse_manager.h

#ifndef MPSE_MANAGER_H
#define MPSE_MANAGER_H

// Factory for Mpse.  The same Mpse type is used for rule matching as well
// as searching by inspectors with a SearchTool.  Runtime use of the Mpse
// is by the fast pattern detection module.

#include "framework/module.h"

namespace snort
{
struct MpseApi;
class Mpse;
struct SnortConfig;
}

//-------------------------------------------------------------------------

class MpseManager
{
public:
    static void add_plugin(const snort::MpseApi*);
    static void release_plugins();
    static void dump_plugins();

    static void instantiate(const snort::MpseApi*, snort::Module*, snort::SnortConfig*);
    static const snort::MpseApi* get_search_api(const char* type);
    static void delete_search_engine(snort::Mpse*);

    static snort::Mpse* get_search_engine(
        const snort::SnortConfig* sc,const snort::MpseApi* api, const struct MpseAgent*);

    static void activate_search_engine(const snort::MpseApi*, snort::SnortConfig*);
    static void setup_search_engine(const snort::MpseApi*, snort::SnortConfig*);
    static void start_search_engine(const snort::MpseApi*);
    static void stop_search_engine(const snort::MpseApi*);
    static bool is_async_capable(const snort::MpseApi*);
    static bool is_regex_capable(const snort::MpseApi*);
    static bool parallel_compiles(const snort::MpseApi*);
    static bool is_poll_capable(const snort::MpseApi* api);
    static void print_mpse_summary(const snort::MpseApi*);
    static void print_search_engine_stats();
};

#endif

plugin_manager.h

Path = src/managers/plugin_manager.h

#ifndef PLUGIN_MANAGER_H
#define PLUGIN_MANAGER_H

//-------------------------------------------------------------------------
// Manages all plugins.  Loading plugins is a 3 step process:
//
// 1. ScriptManager loads available scripts and generates an API
//    for each.
// 2. PluginManager loads all API including static, dynamic, generated.
//    PluginManager retains only the latest version of each.
// 3. API managers, such as IpsManager, use the API to instantiate plugins
//    based on configuration.
//-------------------------------------------------------------------------

#include <map>
#include <memory>
#include <string>

#include "framework/base_api.h"

namespace snort
{
class Module;
struct SnortConfig;
}

class PluginManager
{
public:
    // plugin methods
    static void load_plugins(const snort::BaseApi**);
    static void load_plugins(const std::string& lib_paths);

    static void list_plugins();
    static void show_plugins();
    static void dump_plugins();
    static void release_plugins();

    static PlugType get_type(const char*);
    static const char* get_type_name(PlugType);

    static const snort::BaseApi* get_api(PlugType, const char* name);
    static const char* get_current_plugin();

    static void instantiate(const snort::BaseApi*, snort::Module*, snort::SnortConfig*);
    static void instantiate(const snort::BaseApi*, snort::Module*, snort::SnortConfig*,
        const char* name);

    static const char* get_available_plugins(PlugType);
    static void load_so_plugins(snort::SnortConfig*, bool is_reload = false);
    static void reload_so_plugins(const char*, snort::SnortConfig*);
    static void reload_so_plugins_cleanup(snort::SnortConfig*);
};

struct Plugin;
struct Plugins
{
    std::map<std::string, Plugin> plug_map;
    ~Plugins();
};

struct SoHandle
{
    void* handle;

    SoHandle(void* h) : handle(h) { }
    ~SoHandle();
};

using SoHandlePtr = std::shared_ptr<SoHandle>;

#endif

policy_selector_manager.h

Path = src/managers/policy_selector_manager.h

#ifndef POLICY_SELECTOR_MANAGER_H
#define POLICY_SELECTOR_MANAGER_H

// Policy selectors are used in a multi-tenant configuration to select all policies
// based on layer 1 criteria. This is the manager for a given policy selector.
// Only one selector is permitted in the configuration and it must be at the top level.

namespace snort
{
struct PolicySelectorApi;
class Module;
struct SnortConfig;
}

//-------------------------------------------------------------------------

class PolicySelectorManager
{
public:
    static void add_plugin(const snort::PolicySelectorApi* api);
    static void dump_plugins();
    static void release_plugins();

    static void instantiate(const snort::PolicySelectorApi*, snort::Module*, snort::SnortConfig*);
    static void print_config(const snort::SnortConfig*);
};

#endif

script_manager.h

Path = src/managers/script_manager.h

#ifndef SCRIPT_MANAGER_H
#define SCRIPT_MANAGER_H

// Factory for Lua script plugins used for IpsOptions and Loggers implemented
// in Lua.  Runtime use is via the actual plugin type manager.

#include <string>
#include <vector>

#include "framework/base_api.h"

//-------------------------------------------------------------------------

class ScriptManager
{
public:
    static void load_scripts(const std::vector<std::string>& paths);
    static void release_scripts();
    static const snort::BaseApi** get_plugins();
    static std::string* get_chunk(const char* key);
};

#endif

so_manager.h

Path = src/managers/so_manager.h

#ifndef SO_MANAGER_H
#define SO_MANAGER_H

// Factory for shared object rules.
// Runtime is same as for text rules.
#include <list>
#include "framework/so_rule.h"
#include "managers/plugin_manager.h"

namespace snort
{
struct SnortConfig;
class Inspector;
}

//-------------------------------------------------------------------------
struct SoRules
{
    std::list<const SoApi*> api;
    std::list<SoHandlePtr> handles;
    snort::Inspector* proxy = nullptr;
    ~SoRules();
};

class SoManager
{
public:
    static void add_plugin(const SoApi*, snort::SnortConfig*, SoHandlePtr);
    static void dump_plugins();

    static void instantiate(const SoApi*);

    // soid is arg to soid option, so is arg to so option
    static const char* get_so_rule(const char* soid, snort::SnortConfig* sc = nullptr);
    static SoEvalFunc get_so_eval(const char* soid, const char* so,
        void** data, snort::SnortConfig* sc = nullptr);
    static void delete_so_data(const char* soid, void*, SoRules*);

    static void rule_to_hex(const char* file);
    static void rule_to_text(const char* file);
    static void dump_rule_stubs(const char*, snort::SnortConfig*);
    static void load_so_proxy();
};

#endif

managers/test/

memory/

Snort provides two memory related features: management and profiling. Memory management ensures that Snort uses no more than a configured cap and is built with --enable-jemalloc (jemalloc required). Profiling displays memory usage by component and is enabled with --enable-memory-profiling. These features are independent and may be used together. jemalloc is required for management because it supports the necessary introspection in an efficient manner. Profiling is not enabled by default because of the runtime overhead incurred by overloading new and delete.

Memory management is implemented by periodically comparing the total process usage reported by jemalloc ("stats.allocated") against the configured cap. If over limit, packet threads commence a reap cycle and prune LRU flows up to the configured prune target. This is achieved by capturing the deallocated total (via "thread.deallocatedp") at the start of the reap cycle and checking the current difference and pruning a single flow if the target has not been reached. At most one flow is pruned per packet. You can learn more about the jemalloc api at http://jemalloc.net/jemalloc.3.html.

Snort will go over the configured cap by a (relatively) small amount so the configured cap should be below the hard limit, eg the limit enforced by cgroups. The default values for memory.interval and memory.prune_target generally work well so you should only have to set memory.cap and memory.threshold. If cap is set to the hard limit, set threshold to 99% or so (or just set cap lower and threshold to 100%). memory.threshold is provided for convenience; the actual effective limit is the product of cap * threshold..

Some things to keep in mind:

  • jemalloc updates summary stats like process total once per epoch. Snort will bump the epoch once per memory.interval ms. Performance suffers if interval is too low.

  • Snort generally frees memory in the same thread that allocated it, however this is not always true. Shared caches may delete LRU entries when updated leading to freeing by different packet threads. File capture will free memory allocated by a packet thread in a capture thread. This means that the memory.allocated and memory.deallocated pegs can be confusing and should be considered independently.

  • Since only packet threads take action to enforce the cap, packet threads will be forced to compensate for non-packet threads. This could lead to an apparent out of memory condition if non-packet threads allocate and hold significant memory. Therefore, non-packet threads should be configured with some cap and that amount should be factored into the fixed startup cost.

tcmalloc builds (--enable-tcmalloc) do not support memory management. A process total is available from the tcmalloc extensions but it lacks a thread deallocated number. A scheme could be implemented that released prune_target flows but that is not planned. tcmalloc does provide a performance advantage over glibc so that may be preferred for deployments that don’t need memory management, however internal tests show jemalloc performs better than tcmalloc for more than about 8 threads / cores.

Files pertaining to management (all under src/memory/):

  • heap_interface: implements a jemalloc interface if enabled or a nerfed interface if disabled. A custom HeapInterface (and pruner) can be provided by a plugin for testing. MemoryCap provides methods to install them.

  • memory_cap: implementation of reap cycles using heap interface.

  • memory_config: defines config struct.

  • memory_module: glue between Snort and memory features.

Files pertaining to profiling (all under src/memory/):

  • memory_allocator: overloads call the allocator to actually malloc and free memory.

  • memory_overloads: implements new and delete overloads using allocator.

The current iteration of the memory manager is exclusively preemptive. MemoryCap::free_space is called by the analyzer before each DAQ message is processed. If thread_usage > thread_limit, a single flow will be pruned. Demand-based pruning, ie enforcing that each allocation stays below limit, is both not necessary and bug prone (due to reentrancy when pruning causes flushing causes more allocations).

This implementation has the following limitations:

  • If the profiler is built, it only tracks memory allocated with C++ new. Specifically, direct calls to malloc or calloc which may be made by libraries are not tracked.

  • Non-packet threads are assumed to have bounded memory usage, eg via a cache.

  • Heap managers tend to acquire memory from the system quickly and release back much more slowly, if ever. It is also relatively expensive to force a release.

  • Therefore, pruning a single flow almost certainly won’t release memory back to the system.

For these reasons, the goal of the memory manager is to prevent allocation past the limit rather than try to reclaim memory allocated past the limit. This means that the configured limit must be well enough below the actual hard limit, for example the limit enforced by cgroups, such that the processing of a single packet will not push us over. It must also allow for additional heap overhead.

heap_interface.h

Path = src/memory/heap_interface.h

#ifndef HEAP_INTERFACE_H
#define HEAP_INTERFACE_H

#include <cstdint>

class ControlConn;
namespace memory
{

class HeapInterface
{
public:
    virtual ~HeapInterface() { }

    virtual void main_init() = 0;
    virtual void thread_init() = 0;

    virtual void get_process_total(uint64_t& epoch, uint64_t& total) = 0;
    virtual void get_thread_allocs(uint64_t& alloc, uint64_t& dealloc) = 0;

    virtual void print_stats(ControlConn*) { }
    virtual void get_aux_counts(uint64_t& app_all, uint64_t& active, uint64_t& resident, uint64_t& retained)
    { app_all = active = resident = retained = 0; }

    static HeapInterface* get_instance();

protected:
    HeapInterface() { }
};

}

#endif

memory_allocator.h

Path = src/memory/memory_allocator.h

#ifndef MEMORY_ALLOCATOR_H
#define MEMORY_ALLOCATOR_H

#include <cstddef>

namespace memory
{

struct MemoryAllocator
{
    static void* allocate(size_t);
    static void deallocate(void*);
};

inline void* MemoryAllocator::allocate(size_t n)
{ return malloc(n); }

inline void MemoryAllocator::deallocate(void* p)
{ free(p); }

} // namespace memory

#endif

memory_cap.h

Path = src/memory/memory_cap.h

#ifndef MEMORY_CAP_H
#define MEMORY_CAP_H

#include "memory/heap_interface.h"

#include <cstddef>

#include "framework/counts.h"
#include "main/snort_types.h"

struct MemoryConfig;
class ControlConn;

namespace memory
{

struct MemoryCounts
{
    PegCount start_up_use;
    PegCount cur_in_use;
    PegCount max_in_use;
    PegCount epochs;
    PegCount allocated;
    PegCount deallocated;
    PegCount reap_cycles;
    PegCount reap_attempts;
    PegCount reap_failures;
    PegCount reap_aborts;
    PegCount reap_decrease;
    PegCount reap_increase;
    // reporting only
    PegCount app_all;
    PegCount active;
    PegCount resident;
    PegCount retained;
};

typedef bool (*PruneHandler)();

class SO_PUBLIC MemoryCap
{
public:
    // main thread
    static void init(unsigned num_threads);
    static void term();

    // main thread - in configure
    static void set_heap_interface(HeapInterface*);
    static void set_pruner(PruneHandler);

    // main thread - after configure
    static void start(const MemoryConfig&, PruneHandler);
    static void stop();
    static void print(bool verbose, bool init = false);

    // packet threads
    static void thread_init();
    static void thread_term();
    static void free_space();

    // main and packet threads
    static MemoryCounts& get_mem_stats();

    // main thread - shutdown
    static void update_pegs();

    static void dump_mem_stats(ControlConn*);
#if defined(REG_TEST) || defined(UNIT_TEST)
    static void test_main_check();
#endif
};

}

#endif

memory_config.h

Path = src/memory/memory_config.h

#ifndef MEMORY_CONFIG_H
#define MEMORY_CONFIG_H

#include <cstddef>

struct MemoryConfig
{
    size_t cap = 0;
    unsigned threshold = 0;
    unsigned interval = 1000;
    unsigned prune_target = 1048576;
    bool enabled = false;

    constexpr MemoryConfig() = default;
};

#endif

memory_module.h

Path = src/memory/memory_module.h

#ifndef MEMORY_MODULE_H
#define MEMORY_MODULE_H

#include "framework/module.h"

class MemoryModule : public snort::Module
{
public:
    MemoryModule();

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    Usage get_usage() const override
    { return GLOBAL; }

    void set_trace(const snort::Trace*) const override;
    const snort::TraceOption* get_trace_options() const override;
};

extern THREAD_LOCAL const snort::Trace* memory_trace;

#endif

memory_overloads.h

Path = src/memory/memory_overloads.h

#ifndef MEMORY_OVERLOADS_H
#define MEMORY_OVERLOADS_H

#include <new>

// -----------------------------------------------------------------------------
// new /delete replacements
// -----------------------------------------------------------------------------

// these don't have to be visible to operate as replacements


void* operator new(std::size_t);
void* operator new[](std::size_t);
void* operator new(std::size_t, const std::nothrow_t&) noexcept;
void* operator new[](std::size_t, const std::nothrow_t&) noexcept;
void operator delete(void*) noexcept;
void operator delete[](void*) noexcept;
void operator delete(void*, const std::nothrow_t&) noexcept;
void operator delete[](void*, const std::nothrow_t&) noexcept;
void operator delete(void*, std::size_t) noexcept;
void operator delete[](void*, std::size_t) noexcept;
void operator delete[](void*, std::size_t, const std::nothrow_t&) noexcept;
void operator delete(void*, std::size_t, const std::nothrow_t&) noexcept;

#endif

memory/test/

mime/

This directory contains all the mime processing related classes and APIs

  • MIME processing: provides the common MIME header and MIME body processing for service inspectors such as HTTP, SMTP, POP, and IMAP.

  • Decode: supports Base64, UU-encoding, QP-encoding, and Bit-encoding

  • Log: logs file names and email headers

  • Configuration: configure decode and log

  • PAF: provides common processing for PAF (Protocol Aware Flushing)

decode_b64.h

Path = src/mime/decode_b64.h

#ifndef DECODE_B64_H
#define DECODE_B64_H

// Email attachment decoder

#include "main/snort_types.h"
#include "mime/decode_base.h"

class B64Decode : public DataDecode
{
public:
    B64Decode(int max_depth, int detect_depth);
    ~B64Decode() override;

    // Main function to decode file data
    DecodeResult decode_data(const uint8_t* start, const uint8_t* end, uint8_t* decode_buf) override;

    void reset_decode_state() override;

private:
    class DecodeBuffer* buffer = nullptr;
};

namespace snort
{
// FIXIT-L inbuf should probably be const uint8_t*
SO_PUBLIC int sf_base64decode(
    uint8_t* inbuf, uint32_t inbuf_size,
    uint8_t* outbuf, uint32_t outbuf_size,
    uint32_t* bytes_written
    );
}
#endif

decode_base.h

Path = src/mime/decode_base.h

#ifndef DECODE_BASE_H
#define DECODE_BASE_H

// Email attachment decoder

#include <cstdint>

enum DecodeResult
{
    DECODE_SUCCESS,
    DECODE_EXCEEDED, // Decode Complete when we reach the max depths
    DECODE_FAIL
};

class DataDecode
{
public:
    DataDecode(int max_depth, int detect_depth);
    virtual ~DataDecode() = default;

    // Main function to decode file data
    virtual DecodeResult decode_data(const uint8_t* start, const uint8_t* end, uint8_t* decode_buf) = 0;

    // Retrieve the decoded data the previous decode_data() call
    int get_decoded_data(const uint8_t** buf,  uint32_t* size);

    virtual void reset_decoded_bytes();

    virtual void reset_decode_state();

    // Used to limit number of bytes examined for rule evaluation
    int get_detection_depth();

protected:
    uint32_t decoded_bytes = 0;
    uint32_t decode_bytes_read;
    const uint8_t* decodePtr = nullptr;
    int detection_depth;
};

#endif

decode_bit.h

Path = src/mime/decode_bit.h

#ifndef DECODE_BIT_H
#define DECODE_BIT_H

// Email attachment decoder

#include "mime/decode_base.h"

class BitDecode : public DataDecode
{
public:
    BitDecode(int max_depth, int detect_depth);

    // Main function to decode file data
    DecodeResult decode_data(const uint8_t* start, const uint8_t* end, uint8_t* decode_buf) override;

    void reset_decode_state() override;

private:
    int decode_depth;
};

#endif

decode_buffer.h

Path = src/mime/decode_buffer.h

#ifndef DECODE_BUFFER_H
#define DECODE_BUFFER_H

// Manage decode/encode buffers

#include "main/snort_types.h"

class DecodeBuffer
{
public:
    DecodeBuffer(int max_depth);
    ~DecodeBuffer();

    // Make sure buffer is available and restore the buffer saved
    bool check_restore_buffer();

    // Save buffer for future use (restore)
    void save_buffer(uint8_t* buff, uint32_t buff_size);

    // Move forward buffer pointer
    void update_buffer(uint32_t act_encode_size, uint32_t act_decode_size);

    void reset_saved();
    uint8_t* get_encode_buff() { return encodeBuf; }
    uint32_t get_decode_bytes_read() { return decode_bytes_read; }
    uint32_t get_decode_avail();
    uint32_t get_encode_avail();
    uint32_t get_prev_encoded_bytes() { return prev_encoded_bytes; }

private:
    uint32_t buf_size;
    uint32_t prev_encoded_bytes;
    uint8_t* prev_encoded_buf;
    uint8_t* encodeBuf = nullptr;
    uint32_t encode_bytes_read;
    uint32_t decode_bytes_read;
    int code_depth;
};

#endif

decode_qp.h

Path = src/mime/decode_qp.h

#ifndef DECODE_QP_H
#define DECODE_QP_H

// Email attachment decoder

#include "mime/decode_base.h"

class QPDecode : public DataDecode
{
public:
    QPDecode(int max_depth, int detect_depth);
    ~QPDecode() override;

    // Main function to decode file data
    DecodeResult decode_data(const uint8_t* start, const uint8_t* end, uint8_t* decode_buf) override;

    void reset_decode_state() override;

private:
    class DecodeBuffer* buffer = nullptr;
};

int sf_qpdecode(const char* src, uint32_t slen, char* dst, uint32_t dlen, uint32_t* bytes_read,
    uint32_t* bytes_copied);

#endif

decode_uu.h

Path = src/mime/decode_uu.h

#ifndef DECODE_UU_H
#define DECODE_UU_H

// UU decoder

#include "mime/decode_base.h"

class UUDecode : public DataDecode
{
public:
    UUDecode(int max_depth, int detect_depth);
    ~UUDecode() override;

    // Main function to decode file data
    DecodeResult decode_data(const uint8_t* start, const uint8_t* end, uint8_t* decode_buf) override;

    void reset_decode_state() override;

private:
    bool begin_found = false;
    bool end_found = false;
    class DecodeBuffer* buffer = nullptr;
};

int sf_uudecode(uint8_t* src, uint32_t slen, uint8_t* dst, uint32_t dlen, uint32_t* bytes_read,
    uint32_t* bytes_copied, bool* begin_found, bool* end_found);

#endif

file_mime_config.h

Path = src/mime/file_mime_config.h

#ifndef FILE_MIME_CONFIG_H
#define FILE_MIME_CONFIG_H

// List of MIME decode and log configuration functions
#include "main/snort_types.h"

/*These are temporary values*/
#define DEFAULT_MIME_MEMCAP           838860
#define DEFAULT_DEPTH                 0
#define DEFAULT_DECOMP                100000
#define MAX_LOG_MEMCAP                104857600
#define MIN_LOG_MEMCAP                3276
#define MIN_MIME_MEM                  3276
#define MAX_DEPTH                     65536
#define MIN_DEPTH                     (-1)

namespace snort
{
class SO_PUBLIC DecodeConfig
{
public:
    void set_ignore_data(bool);
    bool is_ignore_data() const;

    void set_b64_depth(int);
    int get_b64_depth() const;

    void set_qp_depth(int);
    int get_qp_depth() const;

    void set_bitenc_depth(int);
    int get_bitenc_depth() const;

    void set_uu_depth(int);
    int get_uu_depth() const;

    void set_decompress_pdf(bool);
    bool is_decompress_pdf() const;

    void set_decompress_swf(bool);
    bool is_decompress_swf() const;

    void set_decompress_zip(bool);
    bool is_decompress_zip() const;

    void set_decompress_vba(bool);
    bool is_decompress_vba() const;

    void set_decompress_buffer_size(uint32_t);
    uint32_t get_decompress_buffer_size() const;

    int64_t get_file_depth() const;
    bool is_decoding_enabled() const;
    void sync_all_depths();
    void show(bool = false) const;
    int get_max_depth(int) const;

private:
    bool ignore_data = false;
    int b64_depth  = DEFAULT_DEPTH;
    int qp_depth = DEFAULT_DEPTH;
    int bitenc_depth = DEFAULT_DEPTH;
    int uu_depth = DEFAULT_DEPTH;
    bool decompress_pdf = false;
    bool decompress_swf = false;
    bool decompress_zip = false;
    bool decompress_vba = false;
    uint32_t decompress_buffer_size = DEFAULT_DECOMP;
    int64_t file_depth = MIN_DEPTH;
    bool decode_enabled = true;
};
}
#endif

file_mime_context_data.h

Path = src/mime/file_mime_context_data.h

#ifndef FILE_MIME_CONTEXT_DATA_H
#define FILE_MIME_CONTEXT_DATA_H

#include "detection/ips_context_data.h"

class MimeDecodeContextData : public snort::IpsContextData
{
public:
    MimeDecodeContextData();
    ~MimeDecodeContextData() override;

    static unsigned mime_ips_id;

    uint8_t* decode_buf = nullptr;
    uint8_t* decompress_buf = nullptr;
    uint32_t decompress_buf_size = 0;

    static void init();

    static uint8_t* get_decode_buf();
    static uint8_t* get_decompress_buf();
    static uint32_t get_decompress_buf_size();
};

#endif

file_mime_decode.h

Path = src/mime/file_mime_decode.h

#ifndef FILE_MIME_DECODE_H
#define FILE_MIME_DECODE_H

// Email attachment decoder, supports Base64, QP, UU, and Bit7/8

#include "decompress/file_decomp.h"
#include "framework/counts.h"
#include "helpers/buffer_data.h"
#include "main/snort_types.h"
#include "mime/decode_base.h"
#include "mime/file_mime_config.h"

namespace snort
{

enum DecodeType
{
    DECODE_NONE = 0,
    DECODE_BITENC,
    DECODE_B64,
    DECODE_QP,
    DECODE_UU,
};

struct MimeStats
{
    PegCount b64_attachments;
    PegCount b64_bytes;
    PegCount qp_attachments;
    PegCount qp_bytes;
    PegCount uu_attachments;
    PegCount uu_bytes;
    PegCount bitenc_attachments;
    PegCount bitenc_bytes;
};

class SO_PUBLIC MimeDecode
{
public:
    MimeDecode(const snort::DecodeConfig* conf);
    ~MimeDecode();

    // get the decode type from buffer
    void process_decode_type(const char* start, int length);
    void finalize_decoder(MimeStats* mime_stats);

    // Main function to decode file data
    DecodeResult decode_data(const uint8_t* start, const uint8_t* end);

    // Retrieve the decoded data the previous decode_data() call
    int get_decoded_data(const uint8_t** buf,  uint32_t* size);

    int get_detection_depth();

    void clear_decode_state();
    void reset_decoded_bytes();

    DecodeType get_decode_type();

    void file_decomp_reset();
    void file_decomp_init();

    DecodeResult decompress_data(const uint8_t* buf_in, uint32_t size_in,
                                 const uint8_t*& buf_out, uint32_t& size_out);
    const BufferData& _get_ole_buf();
    const BufferData& get_decomp_vba_data();
    void clear_decomp_vba_data();

    static void init();

private:
    void get_ole_data();
    DecodeType decode_type = DECODE_NONE;
    const snort::DecodeConfig* config;
    DataDecode* decoder = nullptr;
    fd_session_t* fd_state = nullptr;
    BufferData ole_data;
    BufferData decompressed_vba_data;
};

} // namespace snort

#endif

file_mime_log.h

Path = src/mime/file_mime_log.h

#ifndef FILE_MIME_LOG_H
#define FILE_MIME_LOG_H

// File name will be extracted from MIME header
// Email headers and emails are also stored in the log buffer

#include <cstdint>
#include "main/snort_types.h"
#include "flow/flow_stash.h"
#define STASH_EXTRADATA_MIME "mime_data"

namespace snort
{
enum EmailUserType
{
    EMAIL_SENDER,
    EMAIL_RECIPIENT
};

struct MailLogConfig
{
    bool log_mailfrom = false;
    bool log_rcptto = false;
    bool log_filename = false;
    bool log_email_hdrs = false;
    uint32_t email_hdrs_log_depth = 0;
};

class Flow;

class SO_PUBLIC MailLogState : public snort::StashGenericObject
{
public:
    MailLogState(MailLogConfig* conf);
    ~MailLogState() override;

    /* accumulate MIME attachment filenames. The filenames are appended by commas */
    int log_file_name(const uint8_t* start, int length);

    int log_email_hdrs(const uint8_t* start, int length);
    int log_email_id(const uint8_t* start, int length, EmailUserType);

    void get_file_name(uint8_t** buf, uint32_t* len);
    void get_email_hdrs(uint8_t** buf, uint32_t* len);
    void get_email_id(uint8_t** buf, uint32_t* len, EmailUserType);

    bool is_file_name_present() const;
    bool is_email_hdrs_present() const;
    bool is_email_from_present() const;
    bool is_email_to_present() const;

private:
    int log_flags = 0;
    uint8_t* buf = nullptr;
    unsigned char* emailHdrs = nullptr;
    uint32_t log_depth = 0;
    uint32_t hdrs_logged;
    uint8_t* recipients = nullptr;
    uint16_t rcpts_logged;
    uint8_t* senders = nullptr;
    uint16_t snds_logged;
    uint8_t* filenames = nullptr;
    uint16_t file_logged;
    uint16_t file_current;
};
}

#endif

file_mime_paf.h

Path = src/mime/file_mime_paf.h

#ifndef FILE_MIME_PAF_H
#define FILE_MIME_PAF_H

// Provides list of MIME processing functions. Encoded file data will be decoded
// and file name will be extracted from MIME header

#include "main/snort_types.h"

/* State tracker for data */
enum MimeDataState
{
    MIME_PAF_FINDING_BOUNDARY_STATE,
    MIME_PAF_FOUND_FIRST_BOUNDARY_STATE,
    MIME_PAF_FOUND_BOUNDARY_STATE
};

/* State tracker for Boundary Signature */
enum MimeBoundaryState
{
    MIME_PAF_BOUNDARY_UNKNOWN = 0,      /* UNKNOWN */
    MIME_PAF_BOUNDARY_CR,               /* '\r' */
    MIME_PAF_BOUNDARY_LF,               /* '\n' */
    MIME_PAF_BOUNDARY_HYPEN_FIRST,      /* First '-' */
    MIME_PAF_BOUNDARY_HYPEN_SECOND      /* Second '-' */
};

/* State tracker for end of pop/smtp command */
enum DataEndState
{
    PAF_DATA_END_UNKNOWN,         /* Start or UNKNOWN */
    PAF_DATA_END_FIRST_CR,        /* First '\r' */
    PAF_DATA_END_FIRST_LF,        /* First '\n' */
    PAF_DATA_END_DOT,             /* '.' */
    PAF_DATA_END_SECOND_CR,       /* Second '\r' */
    PAF_DATA_END_SECOND_LF        /* Second '\n' */
};

#define MAX_MIME_BOUNDARY_LEN  70  /* Max length of boundary string, defined in RFC 2046 */

struct MimeDataPafInfo
{
    MimeDataState data_state;
    char boundary[ MAX_MIME_BOUNDARY_LEN + 1];  /* MIME boundary string + '\0' */
    int boundary_len;
    int boundary_search_len;
    const char* boundary_search;
    MimeBoundaryState boundary_state;
};

inline bool scanning_boundary(MimeDataPafInfo* mime_info, uint32_t boundary_start, uint32_t* fp)
{
    if (boundary_start
        && mime_info->data_state != MIME_PAF_FINDING_BOUNDARY_STATE
        && mime_info->boundary_state >= MIME_PAF_BOUNDARY_HYPEN_FIRST)
    {
        *fp = boundary_start;
        return true;
    }

    return false;
}

namespace snort
{
SO_PUBLIC void reset_mime_paf_state(MimeDataPafInfo*);

/*  Process data boundary and flush each file based on boundary*/
SO_PUBLIC bool process_mime_paf_data(MimeDataPafInfo*,  uint8_t val);
SO_PUBLIC bool check_data_end(void* end_state,  uint8_t val);
}

#endif

file_mime_process.h

Path = src/mime/file_mime_process.h

#ifndef FILE_MIME_PROCESS_H
#define FILE_MIME_PROCESS_H

// Provides list of MIME processing functions. Encoded file data will be decoded
// and file name will be extracted from MIME header
#include <string>
#include "file_api/file_api.h"
#include "mime/file_mime_config.h"
#include "mime/file_mime_decode.h"
#include "mime/file_mime_log.h"
#include "mime/file_mime_paf.h"

namespace snort
{
/* state flags */
#define MIME_FLAG_IN_CONTENT_TYPE            0x00000002
#define MIME_FLAG_GOT_BOUNDARY               0x00000004
#define MIME_FLAG_SEEN_HEADERS               0x00000008
#define MIME_FLAG_IN_CONT_TRANS_ENC          0x00000010
#define MIME_FLAG_FILE_ATTACH                0x00000020
#define MIME_FLAG_MULTIPLE_EMAIL_ATTACH      0x00000040
#define MIME_FLAG_MIME_END                   0x00000080
#define MIME_FLAG_IN_CONT_DISP               0x00000200
#define MIME_FLAG_IN_CONT_DISP_CONT          0x00000400

#define STATE_DATA_INIT    0
#define STATE_DATA_HEADER  1    /* Data header section of data state */
#define STATE_DATA_BODY    2    /* Data body section of data state */
#define STATE_MIME_HEADER  3    /* MIME header section within data section */

enum FilenameState
{
    CONT_DISP_FILENAME_PARAM_NAME,
    CONT_DISP_FILENAME_PARAM_EQUALS,
    CONT_DISP_FILENAME_PARAM_VALUE_QUOTE,
    CONT_DISP_FILENAME_PARAM_VALUE
};

/* Maximum length of header chars before colon, based on Exim 4.32 exploit */
#define MAX_HEADER_NAME_LEN 64

class SO_PUBLIC MimeSession
{
public:
    MimeSession(Packet*, const DecodeConfig*, MailLogConfig*, uint64_t base_file_id=0,
        const uint8_t* uri=nullptr, const int32_t uri_length=0);
    virtual ~MimeSession();

    MimeSession(const MimeSession&) = delete;
    MimeSession& operator=(const MimeSession&) = delete;

    static void init();
    static void exit();

    const uint8_t* process_mime_data(Packet*, const uint8_t *data, int data_size,
        bool upload, FilePosition);

    int get_data_state();
    void set_data_state(int);
    MailLogState* get_log_state();
    void set_mime_stats(MimeStats*);

    const BufferData& get_ole_buf();
    const BufferData& get_vba_inspect_buf();

    struct AttachmentBuffer
    {
        const uint8_t* data = nullptr;
        uint32_t length = 0;
        bool finished = true;
    };

    const AttachmentBuffer get_attachment() { return attachment; }

protected:
    MimeDecode* decode_state = nullptr;

private:
    int data_state = STATE_DATA_INIT;
    int state_flags = 0;
    MimeDataPafInfo mime_boundary;
    const DecodeConfig* decode_conf = nullptr;
    MailLogConfig* log_config = nullptr;
    MailLogState* log_state = nullptr;
    MimeStats* mime_stats = nullptr;
    FilenameState filename_state = CONT_DISP_FILENAME_PARAM_NAME;
    std::string filename;
    bool continue_inspecting_file = true;
    // This counter is not an accurate count of files; used only for creating a unique mime_file_id
    uint32_t file_counter = 0;
    uint32_t file_offset = 0;
    uint64_t session_base_file_id = 0;
    uint64_t current_file_cache_file_id = 0;
    uint64_t current_multiprocessing_file_id = 0;
    const uint8_t* uri;
    const int32_t uri_length;
    uint64_t get_file_cache_file_id();
    uint64_t get_multiprocessing_file_id();
    void mime_file_process(Packet* p, const uint8_t* data, int data_size,
        FilePosition position, bool upload);
    void reset_part_state();

    // Individual service inspectors may have different implementations for these
    virtual int handle_header_line(const uint8_t*, const uint8_t*, int, Packet*) { return 0; }
    virtual int normalize_data(const uint8_t*, const uint8_t*, Packet*) { return 0; }
    virtual void decode_alert() { }
    virtual void decompress_alert() { }
    virtual void reset_state(Flow*) { }
    virtual bool is_end_of_data(Flow*) { return false; }

    void reset_mime_state();
    void setup_attachment_processing();
    const uint8_t* process_mime_header(Packet*, const uint8_t* ptr, const uint8_t* data_end_marker);
    bool process_header_line(const uint8_t*& ptr, const uint8_t* eol, const uint8_t* eolm, const
        uint8_t* start_hdr, Packet* p);
    const uint8_t* process_mime_body(const uint8_t* ptr, const uint8_t* data_end, FilePosition);
    const uint8_t* process_mime_data_paf(Packet*, const uint8_t* start, const uint8_t* end,
        bool upload, FilePosition);
    int extract_file_name(const char*& start, int length);

    uint8_t* partial_header = nullptr;      // single header line split into multiple sections
    uint32_t partial_header_len = 0;
    uint8_t* partial_data = nullptr;        // attachment's trailing bytes (suspected boundary)
    uint32_t partial_data_len = 0;
    uint8_t* rebuilt_data = nullptr;        // prepended attachment data for detection module

    AttachmentBuffer attachment;            // decoded and uncompressed file body
};
}
#endif

network_inspectors/

A set of miscellaneous network inspectors for packet analysis, performance monitoring, policy binding, etc. They are grouped here as they operate and the network and/or IP protocol layer.

The collection includes:

binder - The flow to config mapping (policy selection)

arp_spoof - Monitor ARP requests/replies for consistency. Locate spoofing attempts and ARP cache inconsistancies

port_scan - A tool to attempt to locate IP port scanning activity.

perf_monitor - Although not strictly a network inspector, this module monitors Snort++ performance criteria. Implemented as a network_inspector as it processes each valid packet.

normalize - A collection of IP/ICMP/TCP and potentially UDP frame level normalizations.

packet_capture - A tool for dumping the wire packets that Snort receives.

kaizen - Machine learning based exploit detector capable of detecting novel attacks fitting known vulnerability types. Kaizen uses a neural network provided by a model file to detect exploit patterns. The Kaizen Snort module subscribes to HTTP events published by the HTTP inspector, performs inference on HTTP queries/posts, and generates events if the neural network detects an exploit.

This entire set of inspectors is instantiated as a group via network_inspectors.cc

network_inspectors.h

Path = src/network_inspectors/network_inspectors.h

#ifndef NETWORK_INSPECTORS_H
#define NETWORK_INSPECTORS_H

void load_network_inspectors();

#endif

network_inspectors/appid/

The Application Identification inspector (AppId) is a network inspector that inspects the packets of a flow to discover the application (client, service, payload) that is running over the network connection. Example applications include:

  • service: HTTP

  • client: Chrome

  • payload: Facebook

AppId inspects packets from the client and server endpoints to identify the applications on each end of the connection.

The AppIdModule and AppIdInspector classes subclass the framework Module and Inspector classes to provide the logic to plug this inspector into the snort framework. AppIdModule represents the configuration for AppId and AppIdInspector provides the packet processing context. An AppId inspector is instantiated for each packet thread created by the framework.

AppId registers to receive any IP packet, it does not process rebuilt packets.

AppIdModule contains all the logic to process the AppId inspector Lua configuration which is identified by the appid keyword. This configuration includes settings for logging, statistics, etc. and also the directory path or filename for three additional configuration items:

  • directory for the application id mapping table and Lua detectors

  • path to legacy RNA configuration (appid uses a small part of this to configure monitored networks)

  • path to third party application identification configuration

The configuration syntax for these items has not been converted to use Lua so currently custom parsing code is used to process these files. Converting these to use Lua is on the AppId roadmap.

The application id mapping table is a key component of AppId and must be configured in order for detection to work for rules with the appids option. The value of the appids rule option is a list of the names of the applications required for a match to that rule (the match logic is any of the named applications). AppId uses numbers to identify applications and the mapping table defines the association of each number to the application name.

The set of Lua detectors that AppId loads are located in the odp/lua subdirectory of the directory that contains the mapping configuration file.

The legacy RNA configuration is processed by the AppIdContext class. This is currently not supported so no additional details provided here at this time. This section should be updated once this feature is supported.

<TBD: Provide description for third party configuration>

AppId maintains three categories of statistics counters:

  1. Statistics on general events such as packets processed that are known at build time. These statistics are defined as part of AppIdModule and use the PegCounts framework and reporting mechanism of the ModuleManager. See appid_module.[h|cc] for implementation of these counts.

  2. Statistics on detector discovery events. These are counts for each application that is defined in the application id mapping table and thus discoverable by AppId. This is not known until configuration is processed at runtime so the data structures for the counts are built dynamically during AppId initialization. These counts also use the PegCounts type and a custom print method is provided to dump the counts when required. See appid_peg_counts.[h|cc] for implementation of these counts.

  3. AppId periodic bucket statistics. These statistics count the amount of data processed by AppId and are periodically dumped to file for post processing. These statistics are legacy and are a candidate for review to determine if they should be refactored to use the enhanced snort3 statistics capabilities. See appid_stats.[h|cc] for implementation of these counts.

Supporting the appids option on detection rules is one of the primary features of AppId and this support is implemented in the AppIdIpsOption class. This is a subclass of the IpsOption with an IpsApi defined to plug into the detection framework. The AppIdIpsOption eval method is called for all rules that have the appids option to determine if any of the applications discovered on a flow match the rule. The application id mapping table is required to map the numerical ids maintained by AppId to the corresponding name that is used in IPS rules. Since this table is loaded after the framework has initialized all the rule options AppIdIpsOption must generate its lookup table on the first call to check for a match. This is not ideal but until the rule option framework supports a post-config callback to handle situations like this AppId will need this workaround.

The application discovery state for a flow is maintained in the AppIdSession object that AppId instantiates for each flow. This is typically done of the first packet of the flow, however for expected flows this object may have already been allocated. AppIdSession is a subclass of FlowData and currently is a kitchen sink of data and methods to support discovery and report results to interested parties. Many of the member variables are public to support access from legacy code. Refactoring this class to improve organization and encapsulation would be a worthy undertaking.

The application discovery process for a flow is managed in the AppIdDiscovery class or the client or service discovery classes derived from this class. An instance of the client and service discovery classes is created during initialization and these classes in turn instantiate each of builtin detectors for its category (client or service). Detectors register with discovery the ports and patterns they are interested in and when a flow matches the port or the packet payload matches a pattern registered then the detector is added to the list of candidates to do more detailed inspection of the payload for the current packet. Once the list of candidates is created each detector is dispatched in turn to examine the packet.

External detectors coded in Lua are also loading during the initialization process and these detectors use AppId’s Lua API to register themselves and the ports and patterns to match for selecting them as candidates to inspect a flow.

<NOTE: add details for how third-party discovery fits into this process>

Application detectors are the workhorses of the AppId inspector. Detectors inspect packets for either the server side or the client side (there are a few exceptions where a client detector may look at packets in both directions in some scenarios). Common behavior for both detector types is implemented in the AppIdDetector base class. ClientDetector and ServiceDetector subclass AppIdDetector to implement behavior that is common to all detectors with each category. Finally each application specific detector is a subclass of either the client or service detector class.

When a detector is instantiated it registers the ports for which it should be dispatched when packets arrive on that port and it also registers the set of patterns to match against the content of the packet payloads. The application ids the detector supports and flags indicating additional meta-data the detector may discover are also registered. When a detector is selected based on a port or pattern match the discovery framework will call the validate function of the detector to initiate detailed inspection of the payload. The detector will determine one of the following:

  1. The flow is not a match for its applications. In this case the detector is removed from the list of candidates and discovery moves on to the next detector.

  2. The flow may match but more packets must be inspected to make a final decision. In this case the detector remains a candidate and discovery moves on to the next detector.

  3. The flow is a match. In this case the AppIdSession is updated with the detected application id and any meta-data that has been extracted. If more meta-data is expected the detector will continue to be dispatched on subsequent packets until that process completes. Otherwise the detection process is finished. In either case the list of any other candidate detectors is purged.

As mentioned before, Lua detectors are client and service detectors written in Lua. LuaClientDetector and LuaServiceDetector subclass ClientDetector and ServiceDetector respectively to represent client and service Lua detectors.

AppId’s Lua API relies on Lua States to interact with Lua detectors. Every thread has an independent state and the detectors need to be initialized once per thread/state, whereas the C detectors, are only initialized once in the control thread. To mimic this behavior, session information is stored in a LuaStateDescriptor object and is associated with every detector in a thread/state. The Lua detectors are initialized once in the control thread and are shared by all the threads/states.

During initialization and discovery there are multiple callbacks between C and Lua functions. To keep track of the session and detector another layer of abstraction is added. The LuaObject class encapsulates a Lua detector and a LuaStateDescriptor object, it is stored in the Lua State and is accessed by C functions during Lua callbacks. LuaServiceObject and LuaClientObject subclass LuaObject to represent stateful (session) LuaServiceDetectors and LuaClientDetectors.

When a Lua detector is initialized, a unique environment/table is created in the global registry of the Lua State and a LuaObject object is created in C. And when a detector is activated, a call is made to it’s initialization function in Lua code and the corresponding LuaObject object is stored in it’s local stack. Callbacks to C functions to register ports and patterns are processed only in the control thread and ignored in the packet processing threads.

During discovery, if a Lua detector is selected based on a port or pattern and "validate" is called, the table corresponding to that detector is pulled from the Lua State and a call is made to the corresponding "validate" function in Lua code. The "validate" function in Lua can in turn make callbacks to C functions and shares its local stack with the C function. These functions make sure that the call is made only during discovery before executing.

A custom first packet lua detector API which would map IP address, port and protocol on the very first packet to application protocol (service appid), client application (client appid) and web application (payload appid). This API is only used if a user creates a custom lua detector containing the IP, port, protocol values to be mapped to AppIDs. The first packet API shall offer performance improvements, reinspection and early detection of the traffic.

The values are stored in a cache during load time which are parsed in the lua detector API from the values passed through the custom detector lua file. During runtime, for every session, the first packet API is invoked, to find any cache entries for the first packet. If there’s an entry in the cache, the found appids are assigned accordingly, if no entries are found in the cache, further discovery is carried out on the incoming traffic. Here, there could be two scenarios, if the reinspection flag is enabled, discovery process is further continued and appids found on first packet may or may not change, else if it is disabled, the discovery is stopped at the first packet itself and appids remains the same for this entire session.

LUA detectors can log messages at any point within their workflow ( e.g initialisation, deinitialisation and packet processing ) through snort via the detector_log_snort_message ( LUA cLog ) API. This function requires 2 arguments - log_level and message. The log_level argument tells snort what information needs to be logged. LUA_LOG_TRACE corresponds to packet-centric logging and thus instructs snort to attempt to add packet 5-tuple information to the log messages. Control logic should use the LUA_LOG_INFO log_level to log messages at a higher urgency and without packet information. Packet processing log should use LUA_LOG_TRACE instead, as described above.

appid_api.h

Path = src/network_inspectors/appid/appid_api.h

#ifndef APPID_API_H
#define APPID_API_H

#include "flow/flow.h"
#include "sfip/sf_ip.h"
#include "appid_session_api.h"
#include "application_ids.h"

enum class IpProtocol : uint8_t;

class AppIdContext;
class AppIdSession;

namespace snort
{
// -----------------------------------------------------------------------------
// AppId API
// -----------------------------------------------------------------------------

class SO_PUBLIC AppIdApi
{
public:
    SO_PRIVATE AppIdApi() = default;

    AppIdSession* get_appid_session(const Flow& flow);
    const char* get_application_name(AppId app_id, OdpContext& odp_ctxt);
    const char* get_application_name(AppId app_id, const Flow& flow);
    const char* get_application_name(const Flow& flow, bool from_client);
    AppId get_application_id(const char* appName, const AppIdContext& ctxt);
    bool ssl_app_group_id_lookup(Flow* flow, const char*, const char*, const char*,
        const char*, bool, AppId& service_id, AppId& client_id, AppId& payload_id);
    const AppIdSessionApi* get_appid_session_api(const Flow& flow) const;
    bool is_inspection_needed(const Inspector& g) const;
    const char* get_appid_detector_directory() const;

    bool is_service_http_type(AppId service_id) const
    {
        switch (service_id)
        {
            case APP_ID_HTTP:
            case APP_ID_HTTPS:
            case APP_ID_HTTP2:
            case APP_ID_FTPS:
            case APP_ID_IMAPS:
            case APP_ID_IRCS:
            case APP_ID_LDAPS:
            case APP_ID_NNTPS:
            case APP_ID_POP3S:
            case APP_ID_SMTPS:
            case APP_ID_SSHELL:
            case APP_ID_SSL:
            case APP_ID_QUIC:
            case APP_ID_HTTP3:
                return true;
        }

        return false;
    }
};

SO_PUBLIC extern AppIdApi appid_api;
}
#endif

appid_app_descriptor.h

Path = src/network_inspectors/appid/appid_app_descriptor.h

#ifndef APPID_APP_DESCRIPTOR_H
#define APPID_APP_DESCRIPTOR_H

// The ApplicationDescriptor class and its subclasses contain the state info for
// detected applications.  It provides and API for detectors to call when an
// application is detected to set, get, update, or reset this information.
// When the application is first detected or when it is updated to a different
// application than the current setting the PegCount statistic for that application
// is incremented.

#include <string>

#include "protocols/packet.h"
#include "pub_sub/appid_events.h"
#include "utils/util.h"

#include "appid_types.h"
#include "application_ids.h"

class AppIdDetector;
class AppIdSession;
class OdpContext;

class ApplicationDescriptor
{
public:
    ApplicationDescriptor() = default;
    virtual ~ApplicationDescriptor() = default;

    virtual void reset()
    {
        my_id = APP_ID_NONE;
        my_version.clear();
    }

    virtual void update(AppId id, char* version)
    {
        set_id(id);
        set_version(version);
    }

    AppId get_id() const
    {
        return my_id;
    }

    virtual void set_id(AppId app_id);

    virtual void set_id(const snort::Packet& p, AppIdSession& asd, AppidSessionDirection dir, AppId app_id, AppidChangeBits& change_bits);

    const char* get_version() const
    {
        return my_version.empty() ? nullptr : my_version.c_str();
    }

    void set_version(const char* version)
    {
        if ( version )
            my_version = version;
    }

    void reset_version()
    {
        my_version.clear();
    }

private:
    AppId my_id = APP_ID_NONE;
    std::string my_version;
};

struct AppIdServiceSubtype
{
    AppIdServiceSubtype* next = nullptr;
    std::string service;
    std::string vendor;
    std::string version;
};

class ServiceAppDescriptor : public ApplicationDescriptor
{
public:
    ServiceAppDescriptor()
    {
        service_ip.clear();
    }

    ~ServiceAppDescriptor() override
    {
        AppIdServiceSubtype* tmp_subtype = subtype;
        while (tmp_subtype)
        {
            subtype = tmp_subtype->next;
            delete tmp_subtype;
            tmp_subtype = subtype;
        }
    }

    void set_id(AppId app_id, OdpContext& odp_ctxt);

    void reset() override
    {
        ApplicationDescriptor::reset();
        my_vendor.clear();
        port_service_id = APP_ID_NONE;

        AppIdServiceSubtype* tmp_subtype = subtype;
        while (tmp_subtype)
        {
            subtype = tmp_subtype->next;
            delete tmp_subtype;
            tmp_subtype = subtype;
        }
        service_ip.clear();
        service_port = 0;
        service_group = DAQ_PKTHDR_UNKNOWN;
    }

    AppId get_port_service_id() const
    {
        return port_service_id;
    }

    void set_port_service_id(AppId id);

    bool get_deferred() const
    {
        return deferred;
    }

    const char* get_vendor() const
    {
        return my_vendor.empty() ? nullptr : my_vendor.c_str();
    }

    void set_vendor(const char* vendor, AppidChangeBits& change_bits)
    {
        if ( vendor )
        {
            my_vendor = vendor;
            change_bits.set(APPID_SERVICE_INFO_BIT);
        }
    }

    void add_subtype(AppIdServiceSubtype& more_subtype, AppidChangeBits& change_bits)
    {
        AppIdServiceSubtype** tmp_subtype;

        for (tmp_subtype = &subtype; *tmp_subtype; tmp_subtype = &(*tmp_subtype)->next)
            ;
        *tmp_subtype = &more_subtype;
        change_bits.set(APPID_SERVICE_INFO_BIT);
    }

    const AppIdServiceSubtype* get_subtype() const
    {
        return subtype;
    }

    void set_service_ip(const snort::SfIp& ip)
    {
        service_ip = ip;
    }

    const snort::SfIp& get_service_ip() const
    {
        return service_ip;
    }

    bool is_service_ip_set() const
    {
        return service_ip.is_set();
    }

    void set_service_port(uint16_t port)
    {
        service_port = port;
    }

    uint16_t get_service_port() const
    {
        return service_port;
    }

    void set_service_group(int16_t group)
    {
        service_group = group;
    }

    int16_t get_service_group() const
    {
        return service_group;
    }

    void set_alpn_service_app_id(AppId id)
    {
        alpn_service_app_id = id;
    }

    AppId get_alpn_service_app_id() const
    {
        return alpn_service_app_id;
    }

private:
    AppId port_service_id = APP_ID_NONE;
    AppId alpn_service_app_id = APP_ID_NONE;
    bool deferred = false;
    using ApplicationDescriptor::set_id;
    std::string my_vendor;
    AppIdServiceSubtype* subtype = nullptr;
    snort::SfIp service_ip;
    uint16_t service_port = 0;
    int16_t service_group = DAQ_PKTHDR_UNKNOWN;
};

class ClientAppDescriptor : public ApplicationDescriptor
{
public:
    ClientAppDescriptor() = default;

    void reset() override
    {
        ApplicationDescriptor::reset();
        my_username.clear();
        my_user_id = APP_ID_NONE;
        my_client_detect_type = CLIENT_APP_DETECT_APPID;
    }

    void update_user(AppId app_id, const char* username, AppidChangeBits& change_bits);

    AppId get_user_id() const
    {
        return my_user_id;
    }

    const char* get_username() const
    {
        return my_username.empty() ? nullptr : my_username.c_str();
    }

    void set_eve_client_app_id(AppId id)
    {
        eve_client_app_id = id;
    }

    AppId get_eve_client_app_id() const
    {
        return eve_client_app_id;
    }

    void set_eve_client_app_detect_type(ClientAppDetectType client_app_detect_type)
    {
        my_client_detect_type = client_app_detect_type;
    }

    ClientAppDetectType get_client_app_detect_type() const
    {
        return my_client_detect_type;
    }

private:
    std::string my_username;
    AppId my_user_id = APP_ID_NONE;
    AppId eve_client_app_id = APP_ID_NONE;
    ClientAppDetectType my_client_detect_type = CLIENT_APP_DETECT_APPID;
};

class PayloadAppDescriptor : public ApplicationDescriptor
{
public:
    PayloadAppDescriptor() = default;

    void reset() override
    {
        ApplicationDescriptor::reset();
    }
};

#endif

appid_cip_event_handler.h

Path = src/network_inspectors/appid/appid_cip_event_handler.h

#ifndef APPID_CIP_EVENT_HANDLER_H
#define APPID_CIP_EVENT_HANDLER_H

#include "pub_sub/cip_events.h"
#include "appid_detector.h"
#include "appid_inspector.h"

class CipEventHandler : public snort::DataHandler
{
public:
    CipEventHandler(AppIdInspector& inspector) :
        DataHandler(MOD_NAME), inspector(inspector)
    { }

    void handle(snort::DataEvent&, snort::Flow*) override;

private:
    void client_handler(AppIdSession&);
    void service_handler(const snort::Packet&, AppIdSession&);

    AppIdInspector& inspector;
};

#endif //APPID_CIP_EVENT_HANDLER_H

appid_config.h

Path = src/network_inspectors/appid/appid_config.h

#ifndef APP_ID_CONFIG_H
#define APP_ID_CONFIG_H

#include <array>
#include <string>

#include "helpers/discovery_filter.h"
#include "target_based/snort_protocols.h"

#include "app_info_table.h"
#include "client_plugins/client_discovery.h"
#include "client_plugins/eve_ca_patterns.h"
#include "detector_plugins/cip_patterns.h"
#include "detector_plugins/dns_patterns.h"
#include "detector_plugins/http_url_patterns.h"
#include "detector_plugins/sip_patterns.h"
#include "detector_plugins/ssl_patterns.h"
#include "host_port_app_cache.h"
#include "length_app_cache.h"
#include "lua_detector_flow_api.h"
#include "lua_detector_module.h"
#include "service_plugins/alpn_patterns.h"
#include "service_plugins/service_discovery.h"
#include "detector_plugins/ssh_patterns.h"
#include "tp_appid_module_api.h"
#include "utils/sflsq.h"

#define APP_ID_PORT_ARRAY_SIZE  65536

#define MIN_MAX_BYTES_BEFORE_SERVICE_FAIL 1024
#define MIN_MAX_PKTS_BEFORE_SERVICE_FAIL 2
#define MIN_MAX_PKT_BEFORE_SERVICE_FAIL_IGNORE_BYTES 2

#define DEFAULT_MAX_BYTES_BEFORE_SERVICE_FAIL 4096
#define DEFAULT_MAX_PKTS_BEFORE_SERVICE_FAIL  5
#define DEFAULT_MAX_PKT_BEFORE_SERVICE_FAIL_IGNORE_BYTES 10


enum SnortProtoIdIndex
{
    PROTO_INDEX_UNSYNCHRONIZED = 0,
    PROTO_INDEX_FTP_DATA,
    PROTO_INDEX_HTTP2,
    PROTO_INDEX_REXEC,
    PROTO_INDEX_RSH_ERROR,
    PROTO_INDEX_SNMP,
    PROTO_INDEX_SUNRPC,
    PROTO_INDEX_TFTP,
    PROTO_INDEX_SIP,
    PROTO_INDEX_SSH,
    PROTO_INDEX_CIP,

    PROTO_INDEX_MAX
};

class AppIdInspector;
class PatternClientDetector;
class PatternServiceDetector;
class SipUdpClientDetector;
class SipServiceDetector;

class AppIdConfig
{
public:
    AppIdConfig() = default;
    ~AppIdConfig();

    // FIXIT-L: DECRYPT_DEBUG - Move this to ssl-module
#ifdef REG_TEST
    // To manually restart appid detection for an SSL-decrypted flow (single session only),
    // indicate the first packet from where the flow is decrypted (usually immediately
    // after certificate-exchange). Such manual detection is disabled by default (0).
    uint32_t first_decrypted_packet_debug = 0;
    bool log_eve_process_client_mappings = false;
    bool log_alpn_service_mappings = false;
    bool log_memory_and_pattern_count = false;
#endif
    bool log_stats = false;
    uint32_t app_stats_period = 300;
    uint32_t app_stats_rollover_size = 0;
    const char* app_detector_dir = nullptr;
    std::string tp_appid_path = "";
    std::string tp_appid_config = "";
    bool tp_appid_stats_enable = false;
    bool tp_appid_config_dump = false;
    size_t memcap = 0;
    bool list_odp_detectors = false;
    bool log_all_sessions = false;
    bool enable_rna_filter = false;
    std::string rna_conf_path = "";
    SnortProtocolId snort_proto_ids[PROTO_INDEX_MAX] = {};
    void show() const;
};

class OdpContext
{
public:
    bool dns_host_reporting = true;
    bool referred_appId_disabled = false;
    bool mdns_user_reporting = true;
    bool chp_userid_disabled = false;
    bool is_host_port_app_cache_runtime = false;
    bool check_host_port_app_cache = false;
    bool check_host_cache_unknown_ssl = false;
    bool ftp_userid_disabled = false;
    bool chp_body_collection_disabled = false;
    bool need_reinspection = false;
    AppId first_pkt_service_id = 0;
    AppId first_pkt_payload_id = 0;
    AppId first_pkt_client_id = 0;
    uint32_t chp_body_collection_max = 0;
    uint32_t rtmp_max_packets = 15;
    uint32_t max_tp_flow_depth = 5;
    bool tp_allow_probes = false;
    uint32_t host_port_app_cache_lookup_interval = 10;
    uint32_t host_port_app_cache_lookup_range = 100000;
    bool allow_port_wildcard_host_cache = false;
    bool recheck_for_portservice_appid = false;
    uint64_t max_bytes_before_service_fail = DEFAULT_MAX_BYTES_BEFORE_SERVICE_FAIL;
    uint16_t max_packet_before_service_fail = DEFAULT_MAX_PKTS_BEFORE_SERVICE_FAIL;
    uint16_t max_packet_service_fail_ignore_bytes = DEFAULT_MAX_PKT_BEFORE_SERVICE_FAIL_IGNORE_BYTES;
    FirstPktAppIdDiscovered first_pkt_appid_prefix = NO_APPID_FOUND;
    bool eve_http_client = true;

    OdpContext(const AppIdConfig&, snort::SnortConfig*);
    void initialize(AppIdInspector& inspector);
    void reload();

    uint32_t get_version() const
    {
        return version;
    }

    AppInfoManager& get_app_info_mgr()
    {
        return app_info_mgr;
    }

    ClientDiscovery& get_client_disco_mgr()
    {
        return client_disco_mgr;
    }

    ServiceDiscovery& get_service_disco_mgr()
    {
        return service_disco_mgr;
    }

    HostPortVal* host_port_cache_find(const snort::SfIp* ip, uint16_t port, IpProtocol proto)
    {
        return host_port_cache.find(ip, port, proto, *this);
    }

    bool host_port_cache_add(const snort::SnortConfig* sc, const snort::SfIp* ip, uint16_t port,
        IpProtocol proto, unsigned type, AppId appid)
    {
        return host_port_cache.add(sc, ip, port, proto, type, appid);
    }

    bool host_first_pkt_add(const snort::SnortConfig* sc, const snort::SfIp* ip, uint32_t* netmask, uint16_t port,
        IpProtocol proto, AppId protocol_appid, AppId client_appid, AppId web_appid, unsigned reinspect)
    {
        return first_pkt_cache.add_host(sc, ip, netmask, port, proto, protocol_appid, client_appid, web_appid, reinspect);
    }

    HostAppIdsVal* host_first_pkt_find(const snort::SfIp* ip, uint16_t port, IpProtocol proto)
    {
        return first_pkt_cache.find_on_first_pkt(ip, port, proto, *this);
    }

    AppId length_cache_find(const LengthKey& key)
    {
        return length_cache.find(key);
    }

    bool length_cache_add(const LengthKey& key, AppId val)
    {
        return length_cache.add(key, val);
    }

    CipPatternMatchers& get_cip_matchers()
    {
        return cip_matchers;
    }

    DnsPatternMatchers& get_dns_matchers()
    {
        return dns_matchers;
    }

    HttpPatternMatchers& get_http_matchers()
    {
        return http_matchers;
    }

    EveCaPatternMatchers& get_eve_ca_matchers()
    {
        return eve_ca_matchers;
    }

    SipPatternMatchers& get_sip_matchers()
    {
        return sip_matchers;
    }

    SslPatternMatchers& get_ssl_matchers()
    {
        return ssl_matchers;
    }

    SshPatternMatchers& get_ssh_matchers()
    {
        return ssh_matchers;
    }

    PatternClientDetector& get_client_pattern_detector()
    {
        return *client_pattern_detector;
    }

    PatternServiceDetector& get_service_pattern_detector()
    {
        return *service_pattern_detector;
    }

    AlpnPatternMatchers& get_alpn_matchers()
    {
        return alpn_matchers;
    }

    unsigned get_pattern_count();
    void add_port_service_id(IpProtocol, uint16_t, AppId);
    void add_protocol_service_id(IpProtocol, AppId);
    AppId get_port_service_id(IpProtocol, uint16_t);
    AppId get_protocol_service_id(IpProtocol);
    void set_client_and_service_detectors();
    SipUdpClientDetector* get_sip_client_detector();
    SipServiceDetector* get_sip_service_detector();

private:
    AppInfoManager app_info_mgr;
    ClientDiscovery client_disco_mgr;
    HostPortCache host_port_cache;
    HostPortCache first_pkt_cache;
    LengthCache length_cache;
    CipPatternMatchers cip_matchers;
    DnsPatternMatchers dns_matchers;
    HttpPatternMatchers http_matchers;
    EveCaPatternMatchers eve_ca_matchers;
    ServiceDiscovery service_disco_mgr;
    SipPatternMatchers sip_matchers;
    SslPatternMatchers ssl_matchers;
    SshPatternMatchers ssh_matchers;
    PatternClientDetector* client_pattern_detector;
    PatternServiceDetector* service_pattern_detector;
    AlpnPatternMatchers alpn_matchers;

    std::array<AppId, APP_ID_PORT_ARRAY_SIZE> tcp_port_only = {}; // port-only TCP services
    std::array<AppId, APP_ID_PORT_ARRAY_SIZE> udp_port_only = {}; // port-only UDP services
    std::array<AppId, 256> ip_protocol = {}; // non-TCP / UDP protocol services

    uint32_t version;
    static uint32_t next_version;
};

class OdpThreadContext
{
public:
    ~OdpThreadContext();
    void initialize(const snort::SnortConfig*, AppIdContext& ctxt, bool is_control=false,
        bool reload_odp=false);

    void set_lua_detector_mgr(LuaDetectorManager& mgr)
    {
        lua_detector_mgr = &mgr;
    }

    LuaDetectorManager& get_lua_detector_mgr() const
    {
        assert(lua_detector_mgr);
        return *lua_detector_mgr;
    }

private:
    LuaDetectorManager* lua_detector_mgr = nullptr;
};

class AppIdContext
{
public:
    AppIdContext(AppIdConfig& config) : config(config)
    { }

    ~AppIdContext()
    {
        if (discovery_filter)
            delete discovery_filter;
    }

    OdpContext& get_odp_ctxt() const
    {
        assert(odp_ctxt);
        return *odp_ctxt;
    }
    DiscoveryFilter* get_discovery_filter() const
    {
        return discovery_filter;
    }

    ThirdPartyAppIdContext* get_tp_appid_ctxt() const
    { return tp_appid_ctxt; }

    static void delete_tp_appid_ctxt()
    { delete tp_appid_ctxt; }

    void create_odp_ctxt();
    void create_tp_appid_ctxt();
    bool init_appid(snort::SnortConfig*, AppIdInspector&);
    static void pterm();
    void show() const;

    AppIdConfig& config;

private:
    DiscoveryFilter* discovery_filter = nullptr;
    static OdpContext* odp_ctxt;
    static ThirdPartyAppIdContext* tp_appid_ctxt;
};

#endif

appid_data_decrypt_event_handler.h

Path = src/network_inspectors/appid/appid_data_decrypt_event_handler.h

#ifndef APPID_DATA_DECRYPT_EVENT_HANDLER_H
#define APPID_DATA_DECRYPT_EVENT_HANDLER_H

#include "framework/data_bus.h"

#include "appid_module.h"

class DataDecryptEventHandler : public snort::DataHandler
{
public:
    DataDecryptEventHandler(AppIdInspector& inspector) : DataHandler(MOD_NAME), inspector(inspector)
    { }

    void handle(snort::DataEvent& event, snort::Flow* flow) override;

private:
    AppIdInspector& inspector;
};

#endif

appid_dcerpc_event_handler.h

Path = src/network_inspectors/appid/appid_dcerpc_event_handler.h

#ifndef APPID_DCERPC_EVENT_HANDLER_H
#define APPID_DCERPC_EVENT_HANDLER_H

#include "pub_sub/dcerpc_events.h"

#include "appid_session.h"
#include "service_plugins/service_detector.h"

class DceExpSsnEventHandler : public snort::DataHandler
{
public:
    DceExpSsnEventHandler() : DataHandler(MOD_NAME) { }

    void handle(snort::DataEvent& event, snort::Flow* flow) override
    {
        assert(flow);

        AppIdSession* asd = snort::appid_api.get_appid_session(*flow);
        if (!asd or
            !asd->get_session_flags(APPID_SESSION_DISCOVER_APP | APPID_SESSION_SPECIAL_MONITORED))
                return;
        else
        {
            // Skip sessions using old odp context after reload detectors
            if (!pkt_thread_odp_ctxt or
                (pkt_thread_odp_ctxt->get_version() != asd->get_odp_ctxt_version()))
                return;
        }

        DceExpectedSessionEvent& map_resp_event = static_cast<DceExpectedSessionEvent&>(event);

        const snort::Packet* pkt = map_resp_event.get_packet();
        const snort::SfIp* src_ip = map_resp_event.get_src_ip();
        const snort::SfIp* dst_ip = map_resp_event.get_dst_ip();
        uint16_t src_port = map_resp_event.get_src_port();
        uint16_t dst_port = map_resp_event.get_dst_port();
        IpProtocol proto = map_resp_event.get_ip_proto();
        SnortProtocolId protocol_id = map_resp_event.get_proto_id();

        OdpContext& odp_ctxt = asd->get_odp_ctxt();
        AppIdSession* fp = AppIdSession::create_future_session(pkt, src_ip, src_port,
            dst_ip, dst_port, proto, protocol_id, odp_ctxt);

        if (fp) // initialize data session
        {
            fp->set_service_id(APP_ID_DCE_RPC, odp_ctxt);
            asd->initialize_future_session(*fp, APPID_SESSION_IGNORE_ID_FLAGS);
        }
    }
};

#endif // APPID_DCERPC_EVENT_HANDLER_H

appid_debug.h

Path = src/network_inspectors/appid/appid_debug.h

#ifndef APPID_DEBUG_H
#define APPID_DEBUG_H

#include <cstring>

#include <daq_common.h>

#include "detection/detection_engine.h"
#include "protocols/ipv6.h"
#include "protocols/protocol_ids.h"
#include "main/thread.h"
#include "sfip/sf_ip.h"

class AppIdSession;
namespace snort
{
    class Flow;
    struct Packet;
}

#define CURRENT_PACKET snort::DetectionEngine::get_current_packet()

void appid_log(const snort::Packet*, const uint8_t log_level, const char*, ...);

struct AppIdDebugSessionConstraints
{
    snort::SfIp sip;
    bool sip_flag = false;
    snort::SfIp dip;
    bool dip_flag = false;
    uint16_t sport;
    uint16_t dport;
    IpProtocol protocol = IpProtocol::PROTO_NOT_SET;
    std::vector<uint32_t> tenants;
    bool proto_match(IpProtocol proto) const
    {
        return (protocol == IpProtocol::PROTO_NOT_SET or protocol == proto);
    }
    bool port_match(uint16_t p1, uint16_t p2) const
    {
        return (!sport or sport == p1) and (!dport or dport == p2);
    }
    bool ip_match(const uint32_t* ip1, const uint32_t* ip2) const
    {
        return
            ((!sip_flag or !memcmp(sip.get_ip6_ptr(), ip1, sizeof(snort::ip::snort_in6_addr))) and
             (!dip_flag or !memcmp(dip.get_ip6_ptr(), ip2, sizeof(snort::ip::snort_in6_addr))));
    }
    bool tenant_match(uint32_t tenant_id) const
    {
        if (tenant_id && !tenants.empty())
        {
            auto it = std::find_if(tenants.cbegin(), tenants.cend(),
                [tenant_id](uint32_t t){ return t == tenant_id; });

            if (it == tenants.cend())
                return false;
        }
        return true;
    }
};

class AppIdDebug
{
public:
    AppIdDebug() = default;

    void activate(const uint32_t* ip1, const uint32_t* ip2, uint16_t port1, uint16_t port2,
        IpProtocol protocol, const int version, uint32_t address_space_id,
        const AppIdSession* session, bool log_all_sessions, uint32_t tenant_id, int16_t group1 = DAQ_PKTHDR_UNKNOWN,
        int16_t group2 = DAQ_PKTHDR_UNKNOWN, bool inter_group_flow = false);
    void activate(const snort::Flow *flow, const AppIdSession* session, bool log_all_sessions);
    void set_constraints(const char *desc, const AppIdDebugSessionConstraints* constraints);

    bool is_enabled() { return enabled; }
    void set_enabled(bool enable) { enabled = enable; }

    bool is_active() { return active; }
    void deactivate() { active = false; }

    const char* get_debug_session() const
    {
        return debugstr.c_str();
    }

private:
    bool enabled = false;
    bool active = false;
    AppIdDebugSessionConstraints info = {};
    std::string debugstr;
};

extern THREAD_LOCAL AppIdDebug* appidDebug;

#endif

appid_detector.h

Path = src/network_inspectors/appid/appid_detector.h

#ifndef APPID_DETECTOR_H
#define APPID_DETECTOR_H

#include <vector>

#include "flow/flow.h"

#include "appid_discovery.h"
#include "appid_session.h"
#include "application_ids.h"
#include "service_state.h"

class AppIdContext;
class AppIdInspector;
class LuaStateDescriptor;

namespace snort
{
struct Packet;
}

#define STATE_ID_MAX_VALID_COUNT 5

enum DetectorType
{
    DETECTOR_TYPE_DECODER =  0,
    DETECTOR_TYPE_NETFLOW,
    DETECTOR_TYPE_PORT,
    DETECTOR_TYPE_DERIVED,
    DETECTOR_TYPE_CONFLICT,
    DETECTOR_TYPE_PATTERN,
    DETECTOR_TYPE_NOT_SET
};

typedef std::vector<AppRegistryEntry> FlowApplicationInfo;

struct AppIdFlowContentPattern
{
    const uint8_t* pattern;
    unsigned length;
    int index;
    int nocase;
    unsigned appId;
};
typedef std::vector<AppIdFlowContentPattern> AppIdFlowContentPatterns;

struct ServiceDetectorPort
{
    uint16_t port;
    IpProtocol proto;
    bool reversed_validation;
};
typedef std::vector<ServiceDetectorPort> ServiceDetectorPorts;

class AppIdDiscoveryArgs
{
public:
    AppIdDiscoveryArgs(const uint8_t* data, uint16_t size, AppidSessionDirection dir,
        AppIdSession& asd, snort::Packet* p, AppidChangeBits& cb) : data(data),
        size(size), dir(dir), asd(asd), pkt(p), change_bits(cb)
    {}

    const uint8_t* data;
    uint16_t size;
    AppidSessionDirection dir;
    AppIdSession& asd;
    snort::Packet* pkt;
    AppidChangeBits& change_bits;
};

// These numbers are what Lua (VDB/ODP) gives us. If these numbers are ever changed,
// we need to change get_code_string() code to avoid misinterpretations.
enum APPID_STATUS_CODE
{
    APPID_SUCCESS = 0,
    APPID_INPROCESS = 10,
    APPID_NEED_REASSEMBLY = 11,
    APPID_NOT_COMPATIBLE = 12,
    APPID_INVALID_CLIENT = 13,
    APPID_REVERSED = 14,
    APPID_NOMATCH = 100,
    APPID_ENULL = -10,
    APPID_EINVALID = -11,
    APPID_ENOMEM = -12
};

class AppIdDetector
{
public:
    AppIdDetector() = default;
    virtual ~AppIdDetector() = default;

    virtual int initialize(AppIdInspector&);
    virtual void reload();
    virtual void do_custom_init() { }
    virtual void do_custom_reload() { }
    virtual int validate(AppIdDiscoveryArgs&) = 0;
    virtual void register_appid(AppId, unsigned extractsInfo, OdpContext& odp_ctxt) = 0;

    virtual void* data_get(AppIdSession&);
    virtual int data_add(AppIdSession&, void*, AppIdFreeFCN);
    virtual void add_user(AppIdSession&, const char*, AppId, bool, AppidChangeBits&);
    virtual void add_payload(AppIdSession&, AppId);
    virtual void add_app(AppIdSession& asd, AppId service_id, AppId client_id, const char* version, AppidChangeBits& change_bits)
    {
        if ( version )
            asd.set_client_version(version, change_bits);

        asd.set_client_detected();
        asd.client_inferred_service_id = service_id;
        asd.set_client_id(client_id);
    }
    virtual void add_app(const snort::Packet&, AppIdSession&, AppidSessionDirection, AppId, AppId, const char*, AppidChangeBits&);
    const char* get_code_string(APPID_STATUS_CODE) const;

    const std::string& get_name() const
    { return name; }

    const std::string& get_log_name() const
    { return log_name.empty()? name : log_name; }

    unsigned get_minimum_matches() const
    { return minimum_matches; }

    unsigned int get_precedence() const
    { return precedence; }

    unsigned get_flow_data_index() const
    { return flow_data_index; }

    bool is_custom_detector() const
    { return custom_detector; }

    AppIdDiscovery& get_handler() const
    { return *handler; }

    bool is_client() const
    { return client; }

    virtual LuaStateDescriptor* validate_lua_state(bool /*packet_context*/)
    { return nullptr; }

protected:
    AppIdDiscovery* handler = nullptr;
    std::string name;     // unique name to map detector; can be UUID file name for lua-detector
    std::string log_name; // name from detector package info; can be same as 'name' for c-detector
    bool client = false;
    bool enabled = true;
    bool custom_detector = false;
    IpProtocol proto = IpProtocol::PROTO_NOT_SET;
    unsigned minimum_matches = 0;
    unsigned int precedence = 0;
    bool provides_user = false;
    unsigned flow_data_index = 0;
    unsigned detectorType = DETECTOR_TYPE_NOT_SET;

    AppIdFlowContentPatterns tcp_patterns;
    AppIdFlowContentPatterns udp_patterns;
    FlowApplicationInfo appid_registry;
    ServiceDetectorPorts service_ports;
};

#endif

appid_discovery.h

Path = src/network_inspectors/appid/appid_discovery.h

#ifndef APPID_DISCOVERY_H
#define APPID_DISCOVERY_H

#include <map>
#include <string>
#include <vector>

#include "flow/flow.h"
#include "protocols/protocol_ids.h"
#include "pub_sub/appid_events.h"
#include "search_engines/search_tool.h"
#include "utils/util.h"

#include "appid_types.h"
#include "application_ids.h"

class AppIdInspector;
class AppIdSession;
class AppIdDetector;
class ServiceDetector;
struct ServiceDetectorPort;
class ThirdPartyAppIdContext;
class OdpContext;

namespace snort
{
struct Packet;
}

#define SCAN_HTTP_VIA_FLAG          (1<<0)
#define SCAN_HTTP_USER_AGENT_FLAG   (1<<1)
#define SCAN_HTTP_HOST_URL_FLAG     (1<<2)
#define SCAN_SSL_CERTIFICATE_FLAG   (1<<3)
#define SCAN_SSL_HOST_FLAG          (1<<4)
#define SCAN_HOST_PORT_FLAG         (1<<5)
#define SCAN_HTTP_VENDOR_FLAG       (1<<6)
#define SCAN_HTTP_XWORKINGWITH_FLAG (1<<7)
#define SCAN_HTTP_CONTENT_TYPE_FLAG (1<<8)
#define SCAN_HTTP_URI_FLAG          (1<<9)
#define SCAN_CERTVIZ_ENABLED_FLAG   (1<<10)
#define SCAN_SPOOFED_SNI_FLAG       (1<<11)

enum FirstPktAppIdDiscovered
{
    NO_APPID_FOUND = 0,
    FIRST_CLIENT_APPID_FOUND,
    FIRST_SERVICE_APPID_FOUND,
    FIRST_PAYLOAD_APPID_FOUND,
    FIRST_CLIENT_PAYLOAD_APPID_FOUND,
    FIRST_SERVICE_PAYLOAD_APPID_FOUND,
    FIRST_SERVICE_CLIENT_APPID_FOUND,
    FIRST_ALL_APPID_FOUND
};

class AppIdPatternMatchNode
{
public:
    AppIdPatternMatchNode(AppIdDetector* detector, int start, unsigned len)
        : service(detector), pattern_start_pos(start), size(len) { }

    bool valid_match(int end_position)
    {
        if ( pattern_start_pos >= 0 && pattern_start_pos != (end_position - (int)size) )
            return false;
        else
            return true;
    }

    AppIdDetector* service;
    int pattern_start_pos;
    unsigned size;
};

struct ServiceMatch
{
    struct ServiceMatch* next;
    unsigned count;
    unsigned size;
    ServiceDetector* service = nullptr;
};

typedef std::map<std::string, AppIdDetector*> AppIdDetectors;
typedef AppIdDetectors::iterator AppIdDetectorsIterator;

class AppIdDiscovery
{
public:
    AppIdDiscovery() = default;
    virtual ~AppIdDiscovery();

    AppIdDiscovery(const AppIdDiscovery&) = delete;
    AppIdDiscovery& operator=(const AppIdDiscovery&) = delete;

    static void tterm();

    virtual void initialize(AppIdInspector&) = 0;
    virtual void reload() = 0;
    virtual void register_detector(const std::string&, AppIdDetector*,  IpProtocol);
    virtual void add_pattern_data(AppIdDetector*, snort::SearchTool&, int position,
        const uint8_t* const pattern, unsigned size, unsigned nocase);
    virtual void register_tcp_pattern(AppIdDetector*, const uint8_t* const pattern, unsigned size,
        int position, unsigned nocase);
    virtual void register_udp_pattern(AppIdDetector*, const uint8_t* const pattern, unsigned size,
        int position, unsigned nocase);
    virtual int add_service_port(AppIdDetector*, const ServiceDetectorPort&);

    static void do_application_discovery(snort::Packet* p, AppIdInspector&,
        OdpContext&, ThirdPartyAppIdContext*);

    AppIdDetectors* get_tcp_detectors()
    {
        return &tcp_detectors;
    }

    AppIdDetectors* get_udp_detectors()
    {
        return &udp_detectors;
    }

protected:
    AppIdDetectors tcp_detectors;
    AppIdDetectors udp_detectors;
    snort::SearchTool tcp_patterns;
    int tcp_pattern_count = 0;
    snort::SearchTool udp_patterns;
    int udp_pattern_count = 0;
    std::vector<AppIdPatternMatchNode*> pattern_data;

private:
    static bool do_pre_discovery(snort::Packet* p, AppIdSession*& asd, AppIdInspector& inspector,
        IpProtocol& protocol, IpProtocol& outer_protocol, AppidSessionDirection& direction,
        OdpContext& odp_ctxt);
    static bool do_discovery(snort::Packet* p, AppIdSession& asd, IpProtocol protocol,
        IpProtocol outer_protocol, AppidSessionDirection direction, AppId& service_id,
        AppId& client_id, AppId& payload_id, AppId& misc_id, AppidChangeBits& change_bits,
        ThirdPartyAppIdContext* tp_appid_ctxt);
    static void do_post_discovery(snort::Packet* p, AppIdSession& asd,
        bool is_discovery_done, AppId service_id, AppId client_id, AppId payload_id, AppId misc_id,
        AppidChangeBits& change_bits);
    static void do_port_based_discovery(snort::Packet* p, AppIdSession& asd, IpProtocol protocol,
        AppidSessionDirection direction);
    static bool do_host_port_based_discovery(snort::Packet* p, AppIdSession& asd,
        IpProtocol protocol, AppidSessionDirection direction, ThirdPartyAppIdContext* tp_appid_ctxt);
    static bool detect_on_first_pkt(snort::Packet* p, AppIdSession& asd, IpProtocol protocol,
        AppidSessionDirection direction, AppId& service_id, AppId& client_id, AppId& payload_id);
};
#endif

appid_dns_session.h

Path = src/network_inspectors/appid/appid_dns_session.h

#ifndef APPID_DNS_SESSION_H
#define APPID_DNS_SESSION_H

#include <string>
#include "pub_sub/appid_events.h"

#define DNS_GOT_QUERY    0x01
#define DNS_GOT_RESPONSE 0x02

class AppIdDnsSession
{
public:
    AppIdDnsSession() { }
    ~AppIdDnsSession() { }

    void reset()
    {
        host.clear();
        state = 0;
        response_type = 0;
        id = 0;
        host_offset = 0;
        record_type = 0;
        ttl = 0;
    }

    uint8_t get_state() const
    { return state; }

    void set_state(uint8_t state)
    { this->state = state; }

    uint16_t get_id() const
    { return id; }

    void set_id(uint16_t id)
    { this->id = id; }

    uint16_t get_record_type() const
    { return record_type; }

    void set_record_type(uint16_t recordType)
    { record_type = recordType; }

    uint32_t get_ttl() const
    { return ttl; }

    void set_ttl(uint32_t ttl)
    { this->ttl = ttl; }

    uint8_t get_response_type() const
    { return response_type; }

    void set_response_type(uint8_t responseType)
    { response_type = responseType; }

    const char* get_host() const
    { return host.c_str(); }

    void set_host(char* host, AppidChangeBits& change_bits)
    {
        this->host = host;
        change_bits.set(APPID_DNS_HOST_BIT);
    }

    uint32_t get_host_len() const
    { return host.size(); }

    uint16_t get_host_offset() const
    { return host_offset; }

    void set_host_offset(uint16_t hostOffset)
    { host_offset = hostOffset; }

protected:
    uint8_t state = 0;
    uint8_t response_type = 0;
    uint16_t id = 0;
    uint16_t record_type = 0;
    uint32_t ttl = 0;
    std::string host;
    uint16_t host_offset = 0;
};
#endif

appid_eve_process_event_handler.h

Path = src/network_inspectors/appid/appid_eve_process_event_handler.h

#ifndef APPID_EVE_PROCESS_EVENT_HANDLER_H
#define APPID_EVE_PROCESS_EVENT_HANDLER_H

#include "pub_sub/eve_process_event.h"
#include "appid_module.h"

class AppIdEveProcessEventHandler : public snort::DataHandler
{
public:
    AppIdEveProcessEventHandler(AppIdInspector& inspector) :
        DataHandler(MOD_NAME), inspector(inspector) { }

    void handle(snort::DataEvent& event, snort::Flow* flow) override;

private:
    AppIdInspector& inspector;
};

#endif

appid_ha.h

Path = src/network_inspectors/appid/appid_ha.h

#ifndef APPID_HA_H
#define APPID_HA_H

#include "flow/flow.h"
#include "flow/ha.h"
#include "appid_inspector.h"

#define APPID_HA_MAX_FIELD_LEN 32

enum AppIdHAAppType
{
    APPID_HA_APP_SERVICE = 0,
    APPID_HA_APP_CLIENT,
    APPID_HA_APP_PAYLOAD,
    APPID_HA_APP_MISC,
    APPID_HA_APP_REFERRED,
    APPID_HA_APP_CLIENT_INFERRED_SERVICE,
    APPID_HA_APP_PORT_SERVICE,
    APPID_HA_APP_TP,
    APPID_HA_APP_TP_PAYLOAD,
    APPID_HA_APP_MAX
};

struct AppIdSessionHAApps
{
    uint16_t flags;
    AppId appId[APPID_HA_APP_MAX];
};

struct AppIdSessionHAHttp
{
    char url[APPID_HA_MAX_FIELD_LEN];
    char host[APPID_HA_MAX_FIELD_LEN];
};

struct AppIdSessionHATlsHost
{
    char tls_host[APPID_HA_MAX_FIELD_LEN];
};

class AppIdHAAppsClient : public snort::FlowHAClient
{
public:
    AppIdHAAppsClient() : FlowHAClient(sizeof(AppIdSessionHAApps), false) { }
    bool consume(snort::Flow*&, const snort::FlowKey*, snort::HAMessage&, uint8_t size) override;
    bool produce(snort::Flow&, snort::HAMessage&) override;
};

class AppIdHAHttpClient : public snort::FlowHAClient
{
public:
    AppIdHAHttpClient() : FlowHAClient(sizeof(AppIdSessionHAHttp), false) { }
    bool consume(snort::Flow*&, const snort::FlowKey*, snort::HAMessage&, uint8_t size) override;
    bool produce(snort::Flow&, snort::HAMessage&) override;
};

class AppIdHATlsHostClient : public snort::FlowHAClient
{
public:
    AppIdHATlsHostClient() : FlowHAClient(sizeof(AppIdSessionHATlsHost), false) { }
    bool consume(snort::Flow*&, const snort::FlowKey*, snort::HAMessage&, uint8_t size) override;
    bool produce(snort::Flow&, snort::HAMessage&) override;
};

class AppIdHAManager
{
public:
    static void tinit()
    {
        if ( snort::HighAvailabilityManager::active() )
        {
            ha_apps_client = new AppIdHAAppsClient;
            ha_http_client = new AppIdHAHttpClient;
            ha_tls_host_client = new AppIdHATlsHostClient;
        }
    }
    static void tterm()
    {
        if ( snort::HighAvailabilityManager::active() )
        {
            delete ha_apps_client;
            delete ha_http_client;
            delete ha_tls_host_client;
        }
    }

    static THREAD_LOCAL AppIdHAAppsClient* ha_apps_client;
    static THREAD_LOCAL AppIdHAHttpClient* ha_http_client;
    static THREAD_LOCAL AppIdHATlsHostClient* ha_tls_host_client;
};

#endif

appid_http_event_handler.h

Path = src/network_inspectors/appid/appid_http_event_handler.h

#ifndef APPID_HTTP_EVENT_HANDLER_H
#define APPID_HTTP_EVENT_HANDLER_H

#include "pub_sub/http_events.h"

#include "appid_module.h"

namespace snort
{
class Flow;
}

class AppIdInspector;

class HttpEventHandler : public snort::DataHandler
{
public:
    enum HttpEventType
    {
        REQUEST_EVENT,
        RESPONSE_EVENT,
    };

    HttpEventHandler(HttpEventType type, AppIdInspector& inspector) :
        DataHandler(MOD_NAME), event_type(type), inspector(inspector)
    { }

    void handle(snort::DataEvent&, snort::Flow*) override;

private:
    HttpEventType event_type;
    AppIdInspector& inspector;
};

#endif

appid_http_session.h

Path = src/network_inspectors/appid/appid_http_session.h

#ifndef APPID_HTTP_SESSION_H
#define APPID_HTTP_SESSION_H

#include <string>
#include <utility>

#include "flow/flow.h"
#include "pub_sub/appid_events.h"
#include "sfip/sf_ip.h"

#include "appid_app_descriptor.h"
#include "appid_types.h"
#include "application_ids.h"

class AppIdSession;
class ChpMatchDescriptor;
class HttpPatternMatchers;

#define RESPONSE_CODE_PACKET_THRESHHOLD 0

// These values are used in Lua code as raw numbers. Do NOT reassign new values.
#define APP_TYPE_SERVICE    0x1
#define APP_TYPE_CLIENT     0x2
#define APP_TYPE_PAYLOAD    0x4

struct TunnelDest
{
    snort::SfIp ip;
    uint16_t port = 0;
    TunnelDest(const char* string_srcip, uint16_t tun_port)
    {
        ip.clear();
        if ( ip.set(string_srcip) == SFIP_SUCCESS )
            port = tun_port;
    }
};

class AppIdHttpSession
{
public:
    typedef std::pair<uint16_t,uint16_t> pair_t;

    AppIdHttpSession(AppIdSession&, int64_t);
    virtual ~AppIdHttpSession();
    ClientAppDescriptor client;
    PayloadAppDescriptor payload;
    AppId referred_payload_app_id = APP_ID_NONE;
    AppId misc_app_id = APP_ID_NONE;

    int process_http_packet(AppidSessionDirection direction, AppidChangeBits& change_bits,
        HttpPatternMatchers& http_matchers);

    void update_url(AppidChangeBits& change_bits);
    void set_field(HttpFieldIds id, const std::string* str, AppidChangeBits& change_bits);
    void set_field(HttpFieldIds id, const uint8_t* str, int32_t len, AppidChangeBits& change_bits);
    void set_req_body_field(HttpFieldIds id, const uint8_t* str, int32_t len,
        AppidChangeBits& change_bits);

    const std::string* get_field(HttpFieldIds id) const
    { return meta_data[id]; }

    const char* get_cfield(HttpFieldIds id) const
    { return meta_data[id] != nullptr ? meta_data[id]->c_str() : nullptr; }

    void set_is_webdav(bool webdav)
    { is_webdav = webdav; }

    AppId get_chp_candidate() const
    { return chp_candidate; }

    bool is_chp_finished() const
    { return chp_finished; }

    bool is_chp_hold_flow() const
    { return chp_hold_flow; }

    void set_chp_hold_flow(bool chpHoldFlow = false)
    { chp_hold_flow = chpHoldFlow; }

    AppId get_chp_alt_candidate() const
    { return chp_alt_candidate; }

    void set_chp_alt_candidate(AppId chpAltCandidate = APP_ID_NONE)
    { chp_alt_candidate = chpAltCandidate; }

    bool is_skip_simple_detect() const
    { return skip_simple_detect; }

    void set_skip_simple_detect(bool skipSimpleDetect = false)
    { skip_simple_detect = skipSimpleDetect; }

    void set_chp_finished(bool chpFinished = false)
    { chp_finished = chpFinished; }

    void set_tun_dest();

    const TunnelDest* get_tun_dest() const
    { return tun_dest; }

    void free_tun_dest()
    {
        delete tun_dest;
        tun_dest = nullptr;
    }

    void reset_ptype_scan_counts();

    int get_ptype_scan_count(enum HttpFieldIds type) const
    { return ptype_scan_counts[type]; }

    void clear_all_fields();
    void set_client(AppId, AppidChangeBits&, const char*, const char* version = nullptr);
    void set_payload(AppId, AppidChangeBits&, const char* type = nullptr,
        const char* version = nullptr);
    void set_referred_payload(AppId, AppidChangeBits&);

    int64_t get_httpx_stream_id() const
    {
        return httpx_stream_id;
    }
    void set_rcvd_full_req_body(bool req_full_body)
    {
        rcvd_full_req_body = req_full_body;
    }
    bool get_rcvd_full_req_body()
    {
        return rcvd_full_req_body;
    }

    void set_tunnel(bool tunnel)
    {
        is_tunnel = tunnel;
    }

    bool get_tunnel()
    {
        return is_tunnel;
    }
protected:

    void init_chp_match_descriptor(ChpMatchDescriptor& cmd);
    bool initial_chp_sweep(ChpMatchDescriptor&, HttpPatternMatchers&);
    void process_chp_buffers(AppidChangeBits&, HttpPatternMatchers&);
    void free_chp_matches(ChpMatchDescriptor& cmd, unsigned max_matches);
    void set_http_change_bits(AppidChangeBits& change_bits, HttpFieldIds id);
    void set_scan_flags(HttpFieldIds id);
    void print_field(HttpFieldIds id, const std::string* str);

    AppIdSession& asd;

    // FIXIT-M the meta data buffers in this array are only set from
    // third party (tp_appid_utils.cc) and from http inspect
    // (appid_http_event_handler.cc). The set_field functions should
    // only be accessible to those functions/classes, but the process
    // functions in tp_appid_utils.cc are static. Thus the public
    // set_field() functions in AppIdHttpSession. We do need set functions
    // for this array, as old pointers need to be deleted upon set().
    const std::string* meta_data[NUM_METADATA_FIELDS] = { };

    bool is_webdav = false;
    bool chp_finished = false;
    AppId chp_candidate = APP_ID_NONE;
    AppId chp_alt_candidate = APP_ID_NONE;
    bool chp_hold_flow = false;
    int total_found = 0;
    unsigned app_type_flags = 0;
    int num_matches = 0;
    int num_scans = 0;
    bool skip_simple_detect = false;
    int ptype_req_counts[NUM_HTTP_FIELDS] = { };
    int ptype_scan_counts[NUM_HTTP_FIELDS] = { };
    const TunnelDest* tun_dest = nullptr;
#if RESPONSE_CODE_PACKET_THRESHHOLD
    unsigned response_code_packets = 0;
#endif
    int64_t httpx_stream_id = -1;
    bool is_payload_processed = false;
    bool rcvd_full_req_body = false;
    bool is_tunnel = false;
};

#endif

appid_httpx_req_body_event_handler.h

Path = src/network_inspectors/appid/appid_httpx_req_body_event_handler.h

#ifndef APPID_HTTPX_REQ_BODY_EVENT_HANDLER_H
#define APPID_HTTPX_REQ_BODY_EVENT_HANDLER_H

#include "pub_sub/http_request_body_event.h"

class AppIdHttpXReqBodyEventHandler : public snort::DataHandler
{
public:
    AppIdHttpXReqBodyEventHandler() : DataHandler(MOD_NAME){ }
    void handle(snort::DataEvent& event, snort::Flow* flow) override
    {
        if (!pkt_thread_odp_ctxt)
            return;
        assert(flow);
        snort::Packet* p = snort::DetectionEngine::get_current_packet();
        assert(p);
        AppIdSession* asd = snort::appid_api.get_appid_session(*flow);

        if (!asd or
            !asd->get_session_flags(APPID_SESSION_DISCOVER_APP | APPID_SESSION_SPECIAL_MONITORED))
            return;
        // Skip sessions using old odp context after reload detectors
        if (pkt_thread_odp_ctxt->get_version() != asd->get_odp_ctxt_version())
            return;
        snort::HttpRequestBodyEvent* http_req_body = (snort::HttpRequestBodyEvent*)&event;
        AppIdHttpSession* hsession = asd->get_matching_http_session(
            http_req_body->get_httpx_stream_id());

        if (!hsession)
            return;

        const uint8_t* header_start;
        int32_t header_length;
        int32_t offset;
        AppidChangeBits change_bits;
        header_start = http_req_body->get_request_body_data(header_length, offset);
        if (hsession->get_field(REQ_BODY_FID) and
            !asd->get_session_flags(APPID_SESSION_APP_REINSPECT))
            hsession->set_chp_finished(false);

        hsession->set_req_body_field(REQ_BODY_FID, header_start, header_length, change_bits);
        hsession->process_http_packet(APP_ID_FROM_INITIATOR, change_bits,
            asd->get_odp_ctxt().get_http_matchers());
        asd->publish_appid_event(change_bits, *p, true, asd->get_api().get_hsessions_size() - 1);

        bool last_req_rcvd = http_req_body->is_last_request_body_piece();
        if (last_req_rcvd)
            hsession->set_rcvd_full_req_body(last_req_rcvd);
    }
};
#endif

appid_inspector.h

Path = src/network_inspectors/appid/appid_inspector.h

#ifndef APPID_INSPECTOR_H
#define APPID_INSPECTOR_H

#include "appid_config.h"
#include "appid_module.h"
#include "application_ids.h"
#include "flow/flow.h"

namespace snort
{
struct Packet;
struct SnortConfig;
}

class AppIdInspector : public snort::Inspector
{
public:

    AppIdInspector(AppIdModule&);
    ~AppIdInspector() override;

    bool configure(snort::SnortConfig*) override;
    void show(const snort::SnortConfig*) const override;
    void tinit() override;
    void tterm() override;
    void tear_down(snort::SnortConfig*) override;
    void eval(snort::Packet*) override;
    AppIdContext& get_ctxt() const;
    const AppIdConfig& get_config() const { return *config; }

    static unsigned get_pub_id() { return pub_id; }

private:
    const AppIdConfig* config = nullptr;
    AppIdContext* ctxt = nullptr;
    static unsigned pub_id;
};

extern const snort::InspectApi appid_inspector_api;

extern THREAD_LOCAL OdpThreadContext* odp_thread_local_ctxt;
extern THREAD_LOCAL OdpContext* pkt_thread_odp_ctxt;
extern THREAD_LOCAL ThirdPartyAppIdContext* pkt_thread_tp_appid_ctxt;

#endif

appid_module.h

Path = src/network_inspectors/appid/appid_module.h

#ifndef APPID_MODULE_H
#define APPID_MODULE_H

#include <unordered_map>
#include <vector>

#include "framework/module.h"
#include "main/analyzer.h"
#include "main/analyzer_command.h"
#include "main/reload_tuner.h"

#include "appid_config.h"
#include "appid_peg_counts.h"

namespace snort
{
struct SnortConfig;
class Trace;
}

extern THREAD_LOCAL snort::ProfileStats appid_perf_stats;
extern THREAD_LOCAL snort::ProfileStats tp_appid_perf_stats;
extern THREAD_LOCAL const snort::Trace* appid_trace;

#define MOD_NAME "appid"
#define MOD_HELP "application and service identification"
#define MOD_USAGE snort::Module::GLOBAL

class AppIdReloadTuner : public snort::ReloadResourceTuner
{
public:
    explicit AppIdReloadTuner(size_t memcap) : memcap(memcap) { }
    ~AppIdReloadTuner() override = default;

    bool tinit() override;
    bool tune_packet_context() override
    {
        return tune_resources( max_work );
    }
    bool tune_idle_context() override
    {
        return tune_resources( max_work_idle );
    }

    friend class AppIdModule;

private:
    size_t memcap;

    bool tune_resources(unsigned work_limit);
};

extern THREAD_LOCAL AppIdStats appid_stats;

class AppIdModule : public snort::Module
{
public:
    AppIdModule();
    ~AppIdModule() override = default;

    bool begin(const char*, int, snort::SnortConfig*) override;
    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    const snort::Command* get_commands() const override;
    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile(
        unsigned i, const char*& name, const char*& parent) const override;

    const AppIdConfig* get_data();

    void reset_stats() override;

    Usage get_usage() const override
    { return MOD_USAGE; }
    void sum_stats(bool) override;
    void show_dynamic_stats() override;

    void set_trace(const snort::Trace*) const override;
    const snort::TraceOption* get_trace_options() const override;

private:
    AppIdConfig* config;
};

class ACThirdPartyAppIdCleanup : public snort::AnalyzerCommand
{
public:
    bool execute(Analyzer&, void**) override;
    const char* stringify() override { return "THIRD_PARTY_APPID_CLEANUP"; }
};

#endif

appid_opportunistic_tls_event_handler.h

Path = src/network_inspectors/appid/appid_opportunistic_tls_event_handler.h

#ifndef APPID_OPPORTUNISTIC_TLS_EVENT_HANDLER_H
#define APPID_OPPORTUNISTIC_TLS_EVENT_HANDLER_H

#include "pub_sub/opportunistic_tls_event.h"
#include "appid_session.h"

class AppIdOpportunisticTlsEventHandler : public snort::DataHandler
{
public:
    AppIdOpportunisticTlsEventHandler() : DataHandler(MOD_NAME) { }

    void handle(snort::DataEvent&, snort::Flow* flow) override
    {
        assert(flow);
        AppIdSession* asd = snort::appid_api.get_appid_session(*flow);
        if (!asd or
            !asd->get_session_flags(APPID_SESSION_DISCOVER_APP | APPID_SESSION_SPECIAL_MONITORED))
                return;

        // Skip sessions using old odp context after reload detectors
        if (!pkt_thread_odp_ctxt or
            (pkt_thread_odp_ctxt->get_version() != asd->get_odp_ctxt_version()))
            return;

        asd->set_session_flags(APPID_SESSION_OPPORTUNISTIC_TLS);
    }
};

#endif

appid_peg_counts.h

Path = src/network_inspectors/appid/appid_peg_counts.h

#ifndef APPID_PEG_COUNTS_H
#define APPID_PEG_COUNTS_H

// The AppIdPegCounts class provides an API to manage the dynamic peg counts maintained by AppId.
// AppId defines peg counts that are known at compile time in appid_module.h. The counts here are
// for each application that it can detect.  This list of applications is not known until the
// appMapping.data configuration file is loaded so methods are provided to dynamically
// initialize the PegCount array when that file is loaded.
// Functions for incrementing the peg counts are also provided.
// The AppId can be a very large number so using it as the array index is not practical.
// Packet threads are using dynamic pegs, and std::map that is used to translate the AppId to its
// array index.
// Only the main thread is using a static array.

#include <string>

#include "application_ids.h"
#include "framework/counts.h"

struct AppIdStats
{
    PegCount packets;
    PegCount processed_packets;
    PegCount ignored_packets;
    PegCount total_sessions;
    PegCount service_cache_prunes;
    PegCount service_cache_adds;
    PegCount service_cache_removes;
    PegCount odp_reload_ignored_pkts;
    PegCount tp_reload_ignored_pkts;
    PegCount bytes_in_use;
    PegCount items_in_use;
};

class AppIdPegCounts
{
public:
    static void add_app_peg_info(std::string app_name, AppId);
    static void init_pegs();
    static void cleanup_pegs();
    static void init_peg_info();
    static void cleanup_peg_info();
    static void cleanup_dynamic_sum();

    static void inc_service_count(AppId id);
    static void inc_client_count(AppId id);
    static void inc_payload_count(AppId id);
    static void inc_user_count(AppId id);
    static void inc_misc_count(AppId id);
    static void inc_referred_count(AppId id);

    static void sum_stats();
    static void print();
};
#endif

appid_service_event_handler.h

Path = src/network_inspectors/appid/appid_service_event_handler.h

#ifndef APPID_SERVICE_EVENT_HANDLER_H
#define APPID_SERVICE_EVENT_HANDLER_H

#include "framework/data_bus.h"

#include "appid_module.h"

namespace snort
{
class Flow;
}

class AppIdServiceEventHandler : public snort::DataHandler
{
public:
    AppIdServiceEventHandler(AppIdInspector& inspector) :
        DataHandler(MOD_NAME), inspector(inspector)
    { }

    void handle(snort::DataEvent&, snort::Flow* flow) override;

private:
    AppIdInspector& inspector;
};

#endif

appid_session_api.h

Path = src/network_inspectors/appid/appid_session_api.h

#ifndef APPID_SESSION_API_H
#define APPID_SESSION_API_H

#include "flow/flow.h"
#include "flow/stash_item.h"
#include "main/snort_types.h"
#include "pub_sub/appid_events.h"
#include "sfip/sf_ip.h"
#include "utils/util.h"
#include "appid_dns_session.h"
#include "appid_http_session.h"
#include "application_ids.h"

class AppIdDnsSession;
class AppIdSession;

namespace snort
{
#define APPID_SESSION_RESPONDER_MONITORED   (1ULL << 0)
#define APPID_SESSION_INITIATOR_MONITORED   (1ULL << 1)
#define APPID_SESSION_SPECIAL_MONITORED     (1ULL << 2)
#define APPID_SESSION_FUTURE_FLOW           (1ULL << 3)
#define APPID_SESSION_EXPECTED_EVALUATE     (1ULL << 4)
#define APPID_SESSION_DISCOVER_USER         (1ULL << 5)
#define APPID_SESSION_HAS_DHCP_FP           (1ULL << 6)
#define APPID_SESSION_HAS_DHCP_INFO         (1ULL << 7)
#define APPID_SESSION_HAS_SMB_INFO          (1ULL << 8)
#define APPID_SESSION_MID                   (1ULL << 9)
#define APPID_SESSION_OOO                   (1ULL << 10)
#define APPID_SESSION_SYN_RST               (1ULL << 11)
// Service missed the first UDP packet in a flow. This causes detectors to see traffic in reverse direction.
#define APPID_SESSION_UDP_REVERSED          (1ULL << 12)
#define APPID_SESSION_HTTP_SESSION          (1ULL << 13)
/**Service protocol was detected */
#define APPID_SESSION_SERVICE_DETECTED      (1ULL << 14)
/**Finished with client app detection */
#define APPID_SESSION_CLIENT_DETECTED       (1ULL << 15)
/**Flow is a data connection not a service */
#define APPID_SESSION_NOT_A_SERVICE         (1ULL << 16)
#define APPID_SESSION_DECRYPTED             (1ULL << 17)
#define APPID_SESSION_SERVICE_DELETED       (1ULL << 18)
//The following attributes are references only with appId
/**Continue calling the routine after the service has been identified. */
#define APPID_SESSION_CONTINUE              (1ULL << 19)
/**Call service detection even if the host does not exist */
#define APPID_SESSION_IGNORE_HOST           (1ULL << 20)
/**Service protocol had incompatible client data */
#define APPID_SESSION_INCOMPATIBLE          (1ULL << 21)
/**we are ready to see out of network Server packets */
#define APPID_SESSION_CLIENT_GETS_SERVER_PACKETS    (1ULL << 22)
#define APPID_SESSION_DISCOVER_APP          (1ULL << 23)
#define APPID_SESSION_PORT_SERVICE_DONE     (1ULL << 24)
#define APPID_SESSION_ADDITIONAL_PACKET     (1ULL << 25)
#define APPID_SESSION_RESPONDER_CHECKED     (1ULL << 26)
#define APPID_SESSION_INITIATOR_CHECKED     (1ULL << 27)
#define APPID_SESSION_SSL_SESSION           (1ULL << 28)
#define APPID_SESSION_SPDY_SESSION          (1ULL << 29)
#define APPID_SESSION_ENCRYPTED             (1ULL << 30)
#define APPID_SESSION_APP_REINSPECT         (1ULL << 31)
#define APPID_SESSION_RESPONSE_CODE_CHECKED (1ULL << 32)
#define APPID_SESSION_REXEC_STDERR          (1ULL << 33)
#define APPID_SESSION_CHP_INSPECTING        (1ULL << 34)
#define APPID_SESSION_STICKY_SERVICE        (1ULL << 35)
#define APPID_SESSION_APP_REINSPECT_SSL     (1ULL << 36)
#define APPID_SESSION_NO_TPI                (1ULL << 37)
#define APPID_SESSION_FUTURE_FLOW_IDED      (1ULL << 38)
#define APPID_SESSION_OOO_CHECK_TP          (1ULL << 39)
#define APPID_SESSION_PAYLOAD_SEEN          (1ULL << 40)
#define APPID_SESSION_HOST_CACHE_MATCHED    (1ULL << 41)
#define APPID_SESSION_DECRYPT_MONITOR       (1ULL << 42)
#define APPID_SESSION_HTTP_TUNNEL           (1ULL << 43)
#define APPID_SESSION_OPPORTUNISTIC_TLS     (1ULL << 44)
#define APPID_SESSION_FIRST_PKT_CACHE_MATCHED    (1ULL << 45)
#define APPID_SESSION_DO_NOT_DECRYPT        (1ULL << 46)
#define APPID_SESSION_EARLY_SSH_DETECTED        (1ULL << 47)
#define APPID_SESSION_IGNORE_ID_FLAGS \
    (APPID_SESSION_FUTURE_FLOW | \
    APPID_SESSION_NOT_A_SERVICE | \
    APPID_SESSION_NO_TPI | \
    APPID_SESSION_SERVICE_DETECTED | \
    APPID_SESSION_PORT_SERVICE_DONE)
const uint64_t APPID_SESSION_ALL_FLAGS = 0xFFFFFFFFFFFFFFFFULL;

class SO_PUBLIC AppIdSessionApi : public StashGenericObject
{
public:
    AppId get_service_app_id() const;
    void get_service_info(const char*& vendor, const char*& version,
        const AppIdServiceSubtype*& subtype) const;
    const char* get_user_info(AppId& service, bool& login) const;
    AppId get_misc_app_id(uint32_t stream_index = 0) const;
    AppId get_client_app_id(uint32_t stream_index = 0) const;
    AppId get_payload_app_id(uint32_t stream_index = 0) const;
    AppId get_referred_app_id(uint32_t stream_index = 0) const;
    void get_app_id(AppId& service, AppId& client, AppId& payload, AppId& misc, AppId& referred,
        uint32_t stream_index = 0) const;
    void get_app_id(AppId* service, AppId* client, AppId* payload, AppId* misc, AppId* referred,
        uint32_t stream_index = 0) const;
    bool is_appid_inspecting_session() const;
    bool is_appid_available(uint32_t stream_index = 0) const;
    const char* get_client_info(uint32_t stream_index = 0) const;
    uint64_t get_appid_session_attribute(uint64_t flag) const;
    const SfIp* get_initiator_ip() const;
    const SfIp& get_service_ip() const;
    uint16_t get_service_port() const;
    const AppIdDnsSession* get_dns_session() const;
    const AppIdHttpSession* get_http_session(uint32_t stream_index = 0) const;
    const AppIdHttpSession* get_matching_http_session(int64_t stream_id) const;
    const char* get_tls_host() const;
    bool is_http_inspection_done() const;
    const char* get_netbios_name() const;
    const char* get_netbios_domain() const;
    ClientAppDetectType get_client_app_detect_type() const;

    // For protocols such as HTTP2/HTTP3 which can have multiple streams within a single flow,
    // get_first_stream_* methods return the appids in the first stream seen in a packet.
    void get_first_stream_app_ids(AppId& service, AppId& client, AppId& payload, AppId& misc) const;
    void get_first_stream_app_ids(AppId& service, AppId& client, AppId& payload) const;

    ~AppIdSessionApi() override
    {
        delete_session_data();
    }

    uint32_t get_hsessions_size() const
    {
        return hsessions.size();
    }

    const std::string& get_session_id() const
    {
        return session_id;
    }

    void set_user_logged_in() { flags.user_logged_in = true; }

    void clear_user_logged_in() { flags.user_logged_in = false; }

protected:
    AppIdSessionApi(const AppIdSession* asd, const SfIp& ip);

private:
    const AppIdSession* asd = nullptr;
    AppId application_ids[APP_PROTOID_MAX] =
        { APP_ID_NONE, APP_ID_NONE, APP_ID_NONE, APP_ID_NONE, APP_ID_NONE };
    struct
    {
        bool published : 1;
        bool stored_in_stash : 1;
        bool finished : 1;
        bool user_logged_in : 1;
    } flags = {};
    std::vector<AppIdHttpSession*> hsessions;
    AppIdDnsSession* dsession = nullptr;
    snort::SfIp initiator_ip;
    ServiceAppDescriptor service;
    char* tls_host = nullptr;
    char* netbios_name = nullptr;
    char* netbios_domain = nullptr;
    std::string session_id;

    // Following two fields are used only for non-http sessions. For HTTP traffic,
    // these fields are maintained inside AppIdHttpSession.
    // Note: RTMP traffic is treated like HTTP in AppId
    ClientAppDescriptor client;
    PayloadAppDescriptor payload;

    static THREAD_LOCAL uint32_t appid_flow_data_id;

    void set_ss_application_ids(AppId service, AppId client, AppId payload, AppId misc,
        AppId referred, AppidChangeBits& change_bits, Flow& flow);
    void set_ss_application_ids(AppId client, AppId payload, AppidChangeBits& change_bits, Flow& flow);
    void set_ss_application_ids_payload(AppId payload, AppidChangeBits& change_bits, Flow& flow);
    void set_application_ids_service(AppId service_id, AppidChangeBits& change_bits, Flow& flow);
    void set_netbios_name(AppidChangeBits& change_bits, const char* name);
    void set_netbios_domain(AppidChangeBits& change_bits, const char* domain);
    bool prefer_eve_client_over_appid_http_client() const;

    AppIdHttpSession* get_hsession(uint32_t stream_index = 0) const;

    void delete_session_data()
    {
        delete_all_http_sessions();
        snort_free(tls_host);
        snort_free(netbios_name);
        snort_free(netbios_domain);
        delete dsession;
    }

    void delete_all_http_sessions()
    {
        for (auto hsession : hsessions)
            delete hsession;
        hsessions.clear();
    }

    void set_tls_host(const char* host)
    {
        if (host)
        {
            if (tls_host)
                snort_free(tls_host);
            tls_host = snort_strdup(host);
        }
    }

    friend AppIdSession;
};

}
#endif

appid_session.h

Path = src/network_inspectors/appid/appid_session.h

#ifndef APPID_SESSION_H
#define APPID_SESSION_H

#include <map>
#include <mutex>
#include <string>
#include <tuple>
#include <unordered_map>

#include <daq_common.h>
#include "pub_sub/appid_events.h"

#include "app_info_table.h"
#include "appid_api.h"
#include "appid_app_descriptor.h"
#include "appid_config.h"
#include "appid_http_session.h"
#include "appid_types.h"
#include "application_ids.h"
#include "detector_plugins/http_url_patterns.h"
#include "length_app_cache.h"
#include "service_state.h"

namespace snort
{
    class AppIdSessionApi;
}

class ClientDetector;
class ServiceDetector;
class AppIdDnsSession;
class AppIdHttpSession;
class ThirdPartyAppIdSession;

using AppIdFreeFCN = void (*)(void*);

const uint8_t* service_strstr(const uint8_t* haystack, unsigned haystack_len,
    const uint8_t* needle, unsigned needle_len);

#define SF_DEBUG_FILE   stdout
#define MIN_SFTP_PACKET_COUNT   30
#define MAX_SFTP_PACKET_COUNT   55

#define APPID_SESSION_DATA_NONE                  0
#define APPID_SESSION_DATA_SERVICE_MODSTATE_BIT  0x20000000
#define APPID_SESSION_DATA_CLIENT_MODSTATE_BIT   0x40000000
#define APPID_SESSION_DATA_DETECTOR_MODSTATE_BIT 0x80000000

enum APPID_DISCOVERY_STATE
{
    APPID_DISCO_STATE_NONE = 0,
    APPID_DISCO_STATE_STATEFUL,
    APPID_DISCO_STATE_FINISHED
};

class AppIdFlowData
{
public:
    AppIdFlowData(void* data, unsigned id, AppIdFreeFCN fcn) :
        fd_data(data), fd_id(id), fd_free(fcn)
    { }

    ~AppIdFlowData()
    {
        if (fd_data && fd_free)
            fd_free(fd_data);
    }

    void* fd_data;
    unsigned fd_id;
    AppIdFreeFCN fd_free;
};
typedef std::unordered_map<unsigned, AppIdFlowData*>::const_iterator AppIdFlowDataIter;

enum MatchedTlsType
{
    MATCHED_TLS_NONE = 0,
    MATCHED_TLS_HOST,
    MATCHED_TLS_FIRST_SAN,
    MATCHED_TLS_CNAME,
    MATCHED_TLS_ORG_UNIT,
};

class TlsSession
{
public:
    TlsSession()
    { }

    ~TlsSession()
    {
        if (tls_host)
            snort_free(tls_host);
        if (tls_first_alt_name)
            snort_free(tls_first_alt_name);
        if (tls_cname)
            snort_free(tls_cname);
        if (tls_org_unit)
            snort_free(tls_org_unit);
    }

    const char* get_tls_host() const
    {
        switch (matched_tls_type)
        {
            case MATCHED_TLS_HOST:
                return tls_host;
            case MATCHED_TLS_FIRST_SAN:
                return tls_first_alt_name;
            case MATCHED_TLS_CNAME:
                return tls_cname;
            default:
                if (tls_host)
                    return tls_host;
                else if (tls_first_alt_name)
                    return tls_first_alt_name;
                else if (tls_cname)
                    return tls_cname;
        }
        return nullptr;
    }

    const char* get_tls_first_alt_name() const { return tls_first_alt_name; }

    const char* get_tls_cname() const { return tls_cname; }

    const char* get_tls_org_unit() const { return tls_org_unit; }

    bool get_tls_handshake_done() const { return tls_handshake_done; }

    // Duplicate only if len > 0, otherwise simply set (i.e., own the argument)
    void set_tls_host(const char* new_tls_host, uint32_t len, bool published=false)
    {
        if (tls_host)
            snort_free(tls_host);
        if (!new_tls_host or *new_tls_host == '\0')
        {
            tls_host = nullptr;
            return;
        }
        tls_host = len? snort::snort_strndup(new_tls_host,len) : const_cast<char*>(new_tls_host);

        if (!published)
            tls_host_unpublished = true;
    }

    void set_tls_host(const char* new_tls_host, uint32_t len, AppidChangeBits& change_bits)
    {
        set_tls_host(new_tls_host, len, true);
        if (new_tls_host and *new_tls_host != '\0')
            change_bits.set(APPID_TLSHOST_BIT);
    }

    void set_tls_first_alt_name(const char* new_tls_first_alt_name, uint32_t len, AppidChangeBits& change_bits)
    {
        if (tls_first_alt_name)
            snort_free(tls_first_alt_name);
        if (!new_tls_first_alt_name or *new_tls_first_alt_name == '\0')
        {
            tls_first_alt_name = nullptr;
            return;
        }
        tls_first_alt_name = len? snort::snort_strndup(new_tls_first_alt_name, len) :
            const_cast<char*>(new_tls_first_alt_name);
        if (!tls_host)
            change_bits.set(APPID_TLSHOST_BIT);
    }

    void set_tls_cname(const char* new_tls_cname, uint32_t len, AppidChangeBits& change_bits)
    {
        if (tls_cname)
            snort_free(tls_cname);
        if (!new_tls_cname or *new_tls_cname == '\0')
        {
            tls_cname = nullptr;
            return;
        }
        tls_cname = len? snort::snort_strndup(new_tls_cname,len) :
            const_cast<char*>(new_tls_cname);
        if (tls_host == nullptr)
            change_bits.set(APPID_TLSHOST_BIT);
    }

    void set_tls_org_unit(const char* new_tls_org_unit, uint32_t len)
    {
        if (tls_org_unit)
            snort_free(tls_org_unit);
        if (!new_tls_org_unit or *new_tls_org_unit == '\0')
        {
            tls_org_unit = nullptr;
            return;
        }
        tls_org_unit = len? snort::snort_strndup(new_tls_org_unit,len) :
            const_cast<char*>(new_tls_org_unit);
    }

    void set_tls_handshake_done() { tls_handshake_done = true; }

    void set_matched_tls_type(MatchedTlsType type)
    {
        matched_tls_type = type;
    }

    void set_tls_host_unpublished(bool val) { tls_host_unpublished = val; }

    bool is_tls_host_unpublished() const { return tls_host_unpublished; }

private:
    char* tls_host = nullptr;
    char* tls_first_alt_name = nullptr;
    char* tls_cname = nullptr;
    char* tls_org_unit = nullptr;
    bool tls_handshake_done = false;
    bool tls_host_unpublished = false;
    MatchedTlsType matched_tls_type = MATCHED_TLS_NONE;
};

class AppIdSession : public snort::FlowData
{
public:
    AppIdSession(IpProtocol, const snort::SfIp*, uint16_t port, AppIdInspector&,
        OdpContext&, uint32_t asid, uint32_t tenant_id);
    ~AppIdSession() override;

    static AppIdSession* allocate_session(const snort::Packet*, IpProtocol,
        AppidSessionDirection, AppIdInspector&, OdpContext&);
    static AppIdSession* create_future_session(const snort::Packet*, const snort::SfIp*, uint16_t,
        const snort::SfIp*, uint16_t, IpProtocol, SnortProtocolId, OdpContext&,
        bool swap_app_direction=false, bool bidirectional=false, bool expect_persist=false);
    void initialize_future_session(AppIdSession&, uint64_t);

    snort::Flow* flow = nullptr;
    AppIdConfig& config;
    std::unordered_map<unsigned, AppIdFlowData*> flow_data;
    uint64_t flags = 0;
    uint16_t initiator_port = 0;
    uint32_t tenant_id = 0;
    uint32_t asid = 0;

    uint16_t session_packet_count = 0;
    uint16_t init_pkts_without_reply = 0;
    uint64_t init_bytes_without_reply = 0;

    IpProtocol protocol = IpProtocol::PROTO_NOT_SET;
    uint8_t previous_tcp_flags = 0;

    // AppId matching service side
    APPID_DISCOVERY_STATE service_disco_state = APPID_DISCO_STATE_NONE;
    SESSION_SERVICE_SEARCH_STATE service_search_state = SESSION_SERVICE_SEARCH_STATE::START;
    ServiceDetector* service_detector = nullptr;
    std::vector<ServiceDetector*> service_candidates;

    // Following field is used only for non-http sessions. For HTTP traffic,
    // this field is maintained inside AppIdHttpSession.
    AppId misc_app_id = APP_ID_NONE;

    // AppId matching client side
    APPID_DISCOVERY_STATE client_disco_state = APPID_DISCO_STATE_NONE;
    AppId client_inferred_service_id = APP_ID_NONE;
    ClientDetector* client_detector = nullptr;
    std::map<std::string, ClientDetector*> client_candidates;
    bool tried_reverse_service = false;

    TlsSession* tsession = nullptr;
    unsigned scan_flags = 0;
    ThirdPartyAppIdSession* tpsession = nullptr;
    uint16_t init_tpPackets = 0;
    uint16_t resp_tpPackets = 0;
    bool tp_reinspect_by_initiator = false;
    SnortProtocolId snort_protocol_id = UNKNOWN_PROTOCOL_ID;

    /* Length-based detectors. */
    LengthKey length_sequence;

    struct
    {
        uint32_t first_packet_second;
        uint32_t last_packet_second;
        uint64_t initiator_bytes;
        uint64_t responder_bytes;
    } stats = { 0, 0, 0, 0 };

    //appIds picked from encrypted session.
    struct
    {
        AppId service_id;
        AppId client_id;
        AppId payload_id;
        AppId misc_id;
        AppId referred_id;
    } encrypted = { APP_ID_NONE, APP_ID_NONE, APP_ID_NONE, APP_ID_NONE, APP_ID_NONE };

    bool in_expected_cache = false;
    static unsigned inspector_id;
    static std::mutex inferred_svcs_lock;

    static void init() { inspector_id = FlowData::create_flow_data_id(); }

    void set_session_flags(uint64_t set_flags) { flags |= set_flags; }
    void clear_session_flags(uint64_t clear_flags) { flags &= ~clear_flags; }
    uint64_t get_session_flags(uint64_t get_flags) const { return (flags & get_flags); }
    void set_service_detected() { flags |= APPID_SESSION_SERVICE_DETECTED; }
    bool is_service_detected() const { return ((flags & APPID_SESSION_SERVICE_DETECTED) == 0) ?
        false : true; }
    void set_client_detected() { flags |= APPID_SESSION_CLIENT_DETECTED; }
    bool is_client_detected() const { return ((flags & APPID_SESSION_CLIENT_DETECTED) == 0) ?
        false : true; }
    bool is_decrypted() const { return ((flags & APPID_SESSION_DECRYPTED) == 0) ? false : true; }
    bool is_svc_taking_too_much_time() const;

    void* get_flow_data(unsigned id) const;
    int add_flow_data(void* data, unsigned id, AppIdFreeFCN);
    int add_flow_data_id(uint16_t port, ServiceDetector*);
    void free_flow_data_by_id(unsigned id);
    void free_flow_data_by_mask(unsigned mask);
    void free_flow_data();

    AppId pick_service_app_id() const;
    // pick_ss_* and set_ss_* methods below are for application protocols that support only a single
    // stream in a flow. They should not be used for HTTP2/HTTP3 sessions which can have multiple
    // streams within a single flow
    AppId pick_ss_misc_app_id() const;
    AppId pick_ss_client_app_id() const;
    AppId pick_ss_payload_app_id() const;
    AppId check_first_pkt_tp_payload_app_id() const;
    AppId pick_ss_payload_app_id(AppId service_id) const;
    AppId pick_ss_referred_payload_app_id() const;

    void set_ss_application_ids(AppId service, AppId client, AppId payload, AppId misc,
        AppId referred, AppidChangeBits& change_bits);
    void set_ss_application_ids(AppId client, AppId payload, AppidChangeBits& change_bits);
    void set_ss_application_ids_payload(AppId payload, AppidChangeBits& change_bits);
    void set_application_ids_service(AppId service_id, AppidChangeBits& change_bits);

    void examine_ssl_metadata(AppidChangeBits& change_bits);
    void set_client_appid_data(AppId, AppidChangeBits& change_bits, char* version = nullptr);
    void set_client_appid_data(AppId, char* version = nullptr, bool published=false);
    void set_service_appid_data(AppId, AppidChangeBits& change_bits, char* version = nullptr);
    void set_payload_appid_data(AppId, char* version = nullptr);
    void check_app_detection_restart(AppidChangeBits& change_bits,
        ThirdPartyAppIdContext* tp_appid_ctxt);
    void check_ssl_detection_restart(AppidChangeBits& change_bits,
        ThirdPartyAppIdContext* tp_appid_ctxt);
    void check_tunnel_detection_restart();
    void update_encrypted_app_id(AppId);
    void examine_rtmp_metadata(AppidChangeBits& change_bits);
    void sync_with_snort_protocol_id(AppId, snort::Packet*);
    void stop_service_inspection(snort::Packet*,  AppidSessionDirection);

    void clear_http_flags();
    void clear_http_data();
    void reset_session_data(AppidChangeBits& change_bits);

    AppIdHttpSession* get_http_session(uint32_t stream_index = 0) const;
    AppIdHttpSession* create_http_session(int64_t stream_id = -1);
    AppIdHttpSession* get_matching_http_session(int64_t stream_id) const;
    void delete_all_http_sessions();

    AppIdDnsSession* create_dns_session();
    AppIdDnsSession* get_dns_session() const;

    bool is_tp_appid_done() const;
    bool is_tp_processing_done() const;
    bool is_tp_appid_available() const;

    void set_tp_app_id(const snort::Packet& p, AppidSessionDirection dir, AppId app_id,
        AppidChangeBits& change_bits);
    void set_tp_payload_app_id(const snort::Packet& p, AppidSessionDirection dir, AppId app_id,
        AppidChangeBits& change_bits);
    void publish_appid_event(AppidChangeBits&, const snort::Packet&, bool is_httpx = false,
        uint32_t httpx_stream_index = 0);

    bool need_to_delete_tp_conn(ThirdPartyAppIdContext*) const;

    inline void set_tp_app_id(AppId app_id)
    {
        if (tp_app_id != app_id)
        {
            tp_app_id = app_id;
            tp_app_id_deferred = odp_ctxt.get_app_info_mgr().get_app_info_flags
                (tp_app_id, APPINFO_FLAG_DEFER);
        }
    }

    inline void set_tp_payload_app_id(AppId app_id)
    {
        if (tp_payload_app_id != app_id)
        {
            tp_payload_app_id = app_id;
            tp_payload_app_id_deferred = odp_ctxt.get_app_info_mgr().get_app_info_flags
                (tp_payload_app_id, APPINFO_FLAG_DEFER_PAYLOAD);
        }
    }

    inline AppId get_tp_app_id() const
    {
        return tp_app_id;
    }

    inline AppId get_tp_payload_app_id() const
    {
        return tp_payload_app_id;
    }

    inline uint16_t is_inferred_svcs_ver_updated()
    {
        if (my_inferred_svcs_ver == inferred_svcs_ver)
            return false;
        my_inferred_svcs_ver = inferred_svcs_ver;
        return true;
    }

    static inline void incr_inferred_svcs_ver()
    {
        inferred_svcs_ver++;
        if (inferred_svcs_ver == 0)
            inferred_svcs_ver++;
    }

    uint16_t get_prev_httpx_raw_packet() const
    {
        return prev_httpx_raw_packet;
    }

    void set_prev_httpx_raw_packet(uint16_t packet_num)
    {
        prev_httpx_raw_packet = packet_num;
    }

    const snort::AppIdSessionApi& get_api() const
    {
        return api;
    }

    AppId get_service_id() const
    {
        return api.service.get_id();
    }

    void set_service_id(AppId id, OdpContext &ctxt)
    {
        api.service.set_id(id, ctxt);
    }

    AppId get_port_service_id() const
    {
        return api.service.get_port_service_id();
    }

    void set_port_service_id(AppId id)
    {
        api.service.set_port_service_id(id);
    }

    void set_service_version(const char* version, AppidChangeBits& change_bits)
    {
        if (!version)
            return;
        api.service.set_version(version);
        change_bits.set(APPID_SERVICE_INFO_BIT);
    }

    void set_service_vendor(const char* vendor, AppidChangeBits& change_bits)
    {
        api.service.set_vendor(vendor, change_bits);
    }

    void add_service_subtype(AppIdServiceSubtype& subtype, AppidChangeBits& change_bits)
    {
        api.service.add_subtype(subtype, change_bits);
    }

    AppId get_client_id() const
    {
        return api.client.get_id();
    }

    void set_client_id(AppId id)
    {
        api.client.set_id(id);
    }

    void set_client_id(const snort::Packet& p, AppidSessionDirection dir, AppId id, AppidChangeBits& change_bits)
    {
        api.client.set_id(p, *this, dir, id, change_bits);
    }

    void set_client_version(const char* version, AppidChangeBits& change_bits)
    {
        if (!version)
            return;
        api.client.set_version(version);
        change_bits.set(APPID_CLIENT_INFO_BIT);
    }

    const char* get_client_user() const
    {
        return api.client.get_username();
    }

    AppId get_client_user_id() const
    {
        return api.client.get_user_id();
    }

    void set_client_user(AppId id, const char* username, AppidChangeBits& change_bits)
    {
        api.client.update_user(id, username, change_bits);
    }

    void set_eve_client_app_id(AppId id)
    {
        api.client.set_eve_client_app_id(id);
    }

    AppId get_eve_client_app_id() const
    {
        return api.client.get_eve_client_app_id();
    }

    bool use_eve_client_app_id() const
    {
        if (api.client.get_eve_client_app_id() <= APP_ID_NONE)
            return false;

        if (get_session_flags(APPID_SESSION_HTTP_SESSION))
        {
            if (odp_ctxt.eve_http_client)
                api.client.reset_version();
            return odp_ctxt.eve_http_client;
        }
        else
            return (api.client.get_id() == APP_ID_SSL_CLIENT or api.client.get_id() <= APP_ID_NONE);
    }

    void set_alpn_service_app_id(AppId id)
    {
        api.service.set_alpn_service_app_id(id);
    }

    AppId get_alpn_service_app_id() const
    {
        return api.service.get_alpn_service_app_id();
    }

    AppId get_payload_id() const
    {
        return api.payload.get_id();
    }

    void set_payload_id(AppId id)
    {
        api.payload.set_id(id);
    }

    const snort::SfIp& get_initiator_ip() const
    {
        return api.initiator_ip;
    }

    void set_initiator_ip(const snort::SfIp& ip)
    {
        api.initiator_ip = ip;
    }

    void set_service_ip(const snort::SfIp& ip)
    {
        api.service.set_service_ip(ip);
    }

    void set_service_port(uint16_t port)
    {
        api.service.set_service_port(port);
    }

    void set_tls_host(const AppidChangeBits& change_bits)
    {
        if (tsession and change_bits[APPID_TLSHOST_BIT])
            api.set_tls_host(tsession->get_tls_host());
    }

    void set_tls_host(const char* tls_host)
    {
        api.set_tls_host(tls_host);
    }

    void set_tls_host()
    {
        if (tsession and tsession->is_tls_host_unpublished())
            api.set_tls_host(tsession->get_tls_host());
    }

    void set_netbios_name(AppidChangeBits& change_bits, const char *name)
    {
        api.set_netbios_name(change_bits, name);
    }

    void set_netbios_domain(AppidChangeBits& change_bits, const char *domain)
    {
        api.set_netbios_domain(change_bits, domain);
    }

    OdpContext& get_odp_ctxt() const
    {
        return odp_ctxt;
    }

    uint32_t get_odp_ctxt_version() const
    {
        return odp_ctxt_version;
    }

    ThirdPartyAppIdContext* get_tp_appid_ctxt() const
    {
        return tp_appid_ctxt;
    }

    void set_server_info(const snort::SfIp& ip, uint16_t port, int16_t group = DAQ_PKTHDR_UNKNOWN)
    {
        api.service.set_service_ip(ip);
        api.service.set_service_port(port);
        api.service.set_service_group(group);
    }

    std::tuple<const snort::SfIp*, uint16_t, int16_t>  get_server_info() const
    {
        return std::make_tuple(&api.service.get_service_ip(), api.service.get_service_port(),
            api.service.get_service_group());
    }

    uint16_t get_service_port() const
    {
        return api.service.get_service_port();
    }

    bool is_service_ip_set() const
    {
        return api.service.is_service_ip_set();
    }

    void set_user_logged_in()
    {
        api.set_user_logged_in();
    }

    void clear_user_logged_in()
    {
        api.clear_user_logged_in();
    }

    void set_consumed_ha_data(bool val)
    {
        consumed_ha_data = val;
    }

    bool has_no_service_candidate() const
    {
        return no_service_candidate;
    }

    void set_no_service_candidate()
    {
        no_service_candidate = true;
    }

    bool has_no_service_inspector() const
    {
        return no_service_inspector;
    }

    void set_no_service_inspector()
    {
        no_service_inspector = true;
    }

    void set_client_info_unpublished(bool val)
    {
        client_info_unpublished = val;
    }

    bool is_client_info_unpublished()
    {
        return client_info_unpublished;
    }

    inline bool is_encrypted_oportunistic_tls_session()
    {
        return get_session_flags(APPID_SESSION_OPPORTUNISTIC_TLS) and !flow->flags.data_decrypted;
    }

private:
    uint16_t prev_httpx_raw_packet = 0;

    void reinit_session_data(AppidChangeBits& change_bits, ThirdPartyAppIdContext* tp_appid_ctxt);
    void delete_session_data();

    bool tp_app_id_deferred = false;
    bool tp_payload_app_id_deferred = false;

    // appId determined by 3rd party library
    AppId tp_app_id = APP_ID_NONE;
    AppId tp_payload_app_id = APP_ID_NONE;

    uint16_t my_inferred_svcs_ver = 0;
    snort::AppIdSessionApi& api;
    static uint16_t inferred_svcs_ver;
    OdpContext& odp_ctxt;
    uint32_t odp_ctxt_version;
    ThirdPartyAppIdContext* tp_appid_ctxt = nullptr;
    bool consumed_ha_data = false;
    bool no_service_candidate = false;
    bool no_service_inspector = false;
    bool client_info_unpublished = false;
};

#endif

appid_ssh_event_handler.h

Path = src/network_inspectors/appid/appid_ssh_event_handler.h

#ifndef APPID_SSH_EVENT_HANDLER_H
#define APPID_SSH_EVENT_HANDLER_H

#include "pub_sub/ssh_events.h"

#include "appid_module.h"

class SshEventHandler : public snort::DataHandler
{
public:
    SshEventHandler() : snort::DataHandler(MOD_NAME)
    { id = snort::FlowData::create_flow_data_id(); }

    void handle(snort::DataEvent &, snort::Flow *) override;

private:
    static unsigned int id;
};

struct SshAppIdInfo
{
    std::string vendor;
    std::string version;
    bool finished = false;
};

struct SshEventFlowData
{
    SshAppIdInfo service_info;
    SshAppIdInfo client_info;
    bool failed = false;
};

#endif

appid_stats.h

Path = src/network_inspectors/appid/appid_stats.h

#ifndef APPID_STATS_H
#define APPID_STATS_H

#include <cstdio>
#include <ctime>
#include <map>

#include "utils/sflsq.h"
#include "utils/util.h"

#include "application_ids.h"

class AppIdSession;
class AppIdConfig;

struct AppIdStatRecord
{
    std::string app_name;
    uint64_t initiator_bytes = 0;
    uint64_t responder_bytes = 0;

    AppIdStatRecord(const char* app_name, uint64_t initiator_bytes, uint64_t responder_bytes) :
        app_name(app_name), initiator_bytes(initiator_bytes), responder_bytes(responder_bytes)
    { }
};

struct StatsBucket
{
    time_t start_time = 0;
    std::map<AppId, AppIdStatRecord> apps_tree;
    struct
    {
        size_t tx_byte_cnt = 0;
        size_t rx_byte_cnt = 0;
    } totalStats;
    uint32_t app_record_cnt = 0;
};

class AppIdStatistics
{
public:
    AppIdStatistics() = delete;
    ~AppIdStatistics();

    static AppIdStatistics* initialize_manager(const AppIdConfig&);
    static AppIdStatistics* get_stats_manager();
    static void cleanup();
    void update(const AppIdSession&);
    void flush();

private:
    AppIdStatistics(const AppIdConfig&);

    time_t get_time()
    {
        auto now = time(nullptr);
        return now - (now % bucket_interval);
    }

    void start_stats_period(time_t start_time)
    {
        bucket_start = start_time;
        bucket_end = bucket_start + bucket_interval;
    }

    void end_stats_period();
    StatsBucket* get_stats_bucket(time_t);
    void open_stats_log_file();
    void dump_statistics();

    SF_LIST* curr_buckets = nullptr;
    SF_LIST* log_buckets = nullptr;
    struct TextLog* log = nullptr;
    time_t bucket_start = 0;
    time_t bucket_interval = 0;
    time_t bucket_end = 0;
    size_t roll_size = 0;
    bool log_err = false;
};

#endif

appid_types.h

Path = src/network_inspectors/appid/appid_types.h

#ifndef APPID_TYPES_H
#define APPID_TYPES_H
#include <cstdint>
// These values are used in Lua code as raw numbers. Do NOT reassign new values.
// 0 - 8 (inclusive)       : used heavily in CHP code. DO NOT CHANGE.
// 9 - NUM_METADATA_FIELDS : extra metadata buffers, beyond CHP.
// NUM_METADATA_FIELDS     : must always follow the last metadata FID.
// NUM_HTTP_FIELDS         : number of CHP fields, so always RSP_BODY_FID + 1
enum HttpFieldIds : uint8_t
{
    // 0-8: CHP fields. DO NOT CHANGE

    // Request-side headers
    REQ_AGENT_FID,          // 0
    REQ_HOST_FID,           // 1
    REQ_REFERER_FID,        // 2
    REQ_URI_FID,            // 3
    REQ_COOKIE_FID,         // 4
    REQ_BODY_FID,           // 5
    // Response-side headers
    RSP_CONTENT_TYPE_FID,   // 6
    RSP_LOCATION_FID,       // 7
    RSP_BODY_FID,           // 8

    // extra (non-CHP) metadata fields.
    MISC_VIA_FID,           // 9
    MISC_RESP_CODE_FID,     // 10
    MISC_SERVER_FID,        // 11
    MISC_XWW_FID,           // 12
    MISC_URL_FID,           // 13

    // Total number of metadata fields, always first after actual FIDs.
    NUM_METADATA_FIELDS,    // 14

    // Number of CHP fields, always 1 past RSP_BODY_FIELD
    NUM_HTTP_FIELDS = MISC_VIA_FID,
    MAX_KEY_PATTERN = REQ_URI_FID,     // DO NOT CHANGE, used in CHP
};

enum AppidSessionDirection
{
    APP_ID_FROM_INITIATOR,
    APP_ID_FROM_RESPONDER,
    APP_ID_APPID_SESSION_DIRECTION_MAX
};

enum ClientAppDetectType
{
    CLIENT_APP_DETECT_APPID = 0,
    CLIENT_APP_DETECT_TLS_FP
};

#endif

app_info_table.h

Path = src/network_inspectors/appid/app_info_table.h

#ifndef APP_INFO_TABLE_H
#define APP_INFO_TABLE_H

#include <unordered_map>
#include <vector>

#include "application_ids.h"

#include "flow/flow.h"
#include "framework/counts.h"
#include "main/thread.h"
#include "target_based/snort_protocols.h"
#include "utils/util.h"

#define APP_PRIORITY_DEFAULT 2
#define SF_APPID_MAX            40000
#define SF_APPID_BUILDIN_MAX    30000
#define SF_APPID_CSD_MIN        1000000
#define SF_APPID_DYNAMIC_MIN    2000000

class AppIdConfig;
class ClientDetector;
class OdpContext;
class ServiceDetector;

enum AppInfoFlags
{
    APPINFO_FLAG_SERVICE_ADDITIONAL   = (1<<0),
    APPINFO_FLAG_SERVICE_UDP_REVERSED = (1<<1),
    APPINFO_FLAG_CLIENT_ADDITIONAL    = (1<<2),
    APPINFO_FLAG_CLIENT_USER          = (1<<3),
    APPINFO_FLAG_ACTIVE               = (1<<4),
    APPINFO_FLAG_SSL_INSPECT          = (1<<5),
    APPINFO_FLAG_REFERRED             = (1<<6),
    APPINFO_FLAG_DEFER                = (1<<7),

    APPINFO_FLAG_IGNORE               = (1<<8),
    APPINFO_FLAG_PERSISTENT           = (1<<9),
    APPINFO_FLAG_TP_CLIENT            = (1<<10),
    APPINFO_FLAG_DEFER_PAYLOAD        = (1<<11),
    APPINFO_FLAG_CLIENT_DETECTOR_CALLBACK = (1<<12),
    APPINFO_FLAG_SERVICE_DETECTOR_CALLBACK = (1<<13)
};

class AppInfoTableEntry
{
public:
    AppInfoTableEntry(AppId id, char* name);
    AppInfoTableEntry(AppId id, char* name, AppId sid, AppId cid, AppId pid);
    ~AppInfoTableEntry();

    AppId appId;
    uint32_t serviceId;
    uint32_t clientId;
    uint32_t payloadId;
    SnortProtocolId snort_protocol_id = UNKNOWN_PROTOCOL_ID;
    uint32_t flags = 0;
    uint32_t priority = APP_PRIORITY_DEFAULT;
    ClientDetector* client_detector = nullptr;
    ServiceDetector* service_detector = nullptr;
    char* app_name = nullptr;
    char* app_name_key = nullptr;
};

typedef std::unordered_map<AppId, AppInfoTableEntry*> AppInfoTable;
typedef std::unordered_map<std::string, AppInfoTableEntry*> AppInfoNameTable;

class AppInfoManager
{
public:
    AppInfoTableEntry* get_app_info_entry(AppId);
    AppInfoTableEntry* add_dynamic_app_entry(const char* app_name);
    AppId get_appid_by_service_id(uint32_t);
    AppId get_appid_by_client_id(uint32_t);
    AppId get_appid_by_payload_id(uint32_t);
    void set_app_info_active(AppId);
    const char* get_app_name(AppId);
    const char* get_app_name_key(AppId);
    static char * strdup_to_lower(const char *app_name);
    int32_t get_appid_by_name(const char* app_name);
    bool configured();

    void set_app_info_flags(AppId appId, unsigned flags)
    {
        AppInfoTableEntry* entry = get_app_info_entry(appId);
        if ( entry )
            entry->flags |= flags;
    }

    void clear_app_info_flags(AppId appId, unsigned flags)
    {
        AppInfoTableEntry* entry = get_app_info_entry(appId);
        if ( entry )
            entry->flags &= (~flags);
    }

    unsigned get_app_info_flags(AppId app_id, unsigned flags)
    {
        AppInfoTableEntry* entry = get_app_info_entry(app_id);
        return entry ? entry->flags & flags : 0;
    }

    void set_app_info_priority(AppId appId, unsigned priority)
    {
        AppInfoTableEntry* entry = get_app_info_entry(appId);
        if ( entry )
            entry->priority |= priority;
    }

    unsigned get_priority(AppId app_id)
    {
        AppInfoTableEntry* entry = get_app_info_entry(app_id);
        return entry ? entry->priority : 0;
    }

    void init_appid_info_table(const AppIdConfig&, snort::SnortConfig*, OdpContext& odp_ctxt);
    void cleanup_appid_info_table();
    void dump_app_info_table();
    SnortProtocolId add_appid_protocol_reference(const char* protocol, snort::SnortConfig*);
    void dump_appid_configurations(const std::string&) const;

private:
    void load_odp_config(OdpContext&, const char* path);
    AppInfoTableEntry* get_app_info_entry(AppId appId, const AppInfoTable&);
    bool is_existing_entry(AppInfoTableEntry* entry);
    AppInfoTableEntry* find_app_info_by_name(const char* app_name);
    bool add_entry_to_app_info_name_table(const char* app_name, AppInfoTableEntry* entry);
    AppId get_static_app_info_entry(AppId appid);

    AppInfoTable app_info_table;
    AppInfoTable app_info_service_table;
    AppInfoTable app_info_client_table;
    AppInfoTable app_info_payload_table;
    AppInfoNameTable app_info_name_table;
    AppId next_custom_appid = SF_APPID_DYNAMIC_MIN;
    AppInfoTable custom_app_info_table;
};

#endif

application_ids.h

Path = src/network_inspectors/appid/application_ids.h

#ifndef APPLICATION_IDS_H
#define APPLICATION_IDS_H

#include <cstdint>

enum AppProtoIdIndex
{
    APP_PROTOID_SERVICE,
    APP_PROTOID_CLIENT,
    APP_PROTOID_PAYLOAD,
    APP_PROTOID_MISC,
    APP_PROTOID_REFERRED,
    APP_PROTOID_MAX
};

typedef int32_t AppId;

enum ApplicationIds : AppId
{
    APP_ID_UNKNOWN                        = -1,    // searched and not found any matching app id
    APP_ID_NONE                           = 0,     // AppId not searched

    APP_ID_3COM_TSMUX                     = 2,
    APP_ID_8021Q                          = 3,
    APP_ID_914CG                          = 4,
    APP_ID_ACA_SERVICES                   = 5,
    APP_ID_ACI                            = 6,
    APP_ID_ACR_NEMA                       = 7,
    APP_ID_ACTIVE_DIRECTORY               = 8,
    APP_ID_ACTIVESYNC                     = 9,
    APP_ID_AD_BACKUP                      = 10,
    APP_ID_AD_DRS                         = 11,
    APP_ID_AD_DSAOP                       = 12,
    APP_ID_AD_DSROL                       = 13,
    APP_ID_AD_NSP                         = 14,
    APP_ID_ADOBE                          = 15,
    APP_ID_AD_RESTORE                     = 16,
    APP_ID_ADRIVE                         = 17,
    APP_ID_AD_XDS                         = 18,
    APP_ID_AED512                         = 19,
    APP_ID_AFP                            = 20,
    APP_ID_AH                             = 21,
    APP_ID_AJP                            = 22,
    APP_ID_ALIAS                          = 23,
    APP_ID_AMAZON                         = 24,
    APP_ID_ANET                           = 25,
    APP_ID_ANSA_NOTIFY                    = 26,
    APP_ID_ANSA_REX_TRADER                = 27,
    APP_ID_APPLE_ARP                      = 28,
    APP_ID_APPLEJUICE                     = 29,
    APP_ID_APPLESHARE                     = 30,
    APP_ID_APPLETALK                      = 31,
    APP_ID_APPLE_UPDATE                   = 32,
    APP_ID_ARCISDMS                       = 33,
    APP_ID_ARIEL                          = 34,
    APP_ID_ARNS                           = 35,
    APP_ID_ARP                            = 36,
    APP_ID_ASA                            = 37,
    APP_ID_ASTRAWEB                       = 38,
    APP_ID_ATM_FATE                       = 39,
    APP_ID_ATM_MPOA                       = 40,
    APP_ID_AUDITD                         = 41,
    APP_ID_AUDIT                          = 42,
    APP_ID_AURORA                         = 43,
    APP_ID_AVG                            = 44,
    APP_ID_AVIRA                          = 45,
    APP_ID_AVOCENT                        = 46,
    APP_ID_BACKBLAZE                      = 47,
    APP_ID_BACKPACK                       = 48,
    APP_ID_BATTLEFIELD                    = 49,
    APP_ID_BATTLE_NET                     = 50,
    APP_ID_BEETPH                         = 51,
    APP_ID_BFTP                           = 52,
    APP_ID_BGMP                           = 53,
    APP_ID_BH611                          = 54,
    APP_ID_BHEVENT                        = 55,
    APP_ID_BHFHS                          = 56,
    APP_ID_BHMDS                          = 57,
    APP_ID_BING                           = 58,
    APP_ID_BITDEFENDER                    = 59,
    APP_ID_BITS                           = 60,
    APP_ID_BITTORRENT                     = 61,
    APP_ID_BLACKBOARD                     = 62,
    APP_ID_BLACKJACK                      = 63,
    APP_ID_BLAZEFS                        = 64,
    APP_ID_BLIDM                          = 65,
    APP_ID_BNET                           = 66,
    APP_ID_CABLEPORT_AX                   = 67,
    APP_ID_CAICCI                         = 68,
    APP_ID_CAILIC                         = 69,
    APP_ID_CAP                            = 70,
    APP_ID_CDC                            = 71,
    APP_ID_CFDPTKT                        = 72,
    APP_ID_CHARGEN                        = 73,
    APP_ID_CHECK_POINT                    = 74,
    APP_ID_CISCO_DRP                      = 76,
    APP_ID_CISCO_FNATIVE                  = 77,
    APP_ID_CISCO_GDP                      = 78,
    APP_ID_CISCO_SLA                      = 79,
    APP_ID_CISCO_SYSMAINT                 = 80,
    APP_ID_CISCO_TNATIVE                  = 81,
    APP_ID_CITRIX_CGP                     = 82,
    APP_ID_CITRIX_ICA                     = 83,
    APP_ID_CITRIX_IMA                     = 84,
    APP_ID_CITRIX_JEDI                    = 85,
    APP_ID_CITRIX_LICENSING               = 86,
    APP_ID_CITRIX_ONLINE                  = 87,
    APP_ID_CITRIX_RTMP                    = 88,
    APP_ID_CITRIX_SLG                     = 89,
    APP_ID_CITRIX_WANSCALER               = 90,
    APP_ID_CL1                            = 91,
    APP_ID_CLEARCASE                      = 92,
    APP_ID_CLOANTO                        = 93,
    APP_ID_CMIP                           = 94,
    APP_ID_CODA_AUTH                      = 95,
    APP_ID_COMMVAULT                      = 96,
    APP_ID_COMPRESSNET                    = 97,
    APP_ID_COMSCM                         = 98,
    APP_ID_CORBA                          = 99,
    APP_ID_CORERJD                        = 100,
    APP_ID_COVIA_CI                       = 101,
    APP_ID_CSISGWP                        = 102,
    APP_ID_CSNET_NS                       = 103,
    APP_ID_CTF                            = 104,
    APP_ID_CVCHOSTD                       = 105,
    APP_ID_DASP                           = 106,
    APP_ID_DATEX_ASN                      = 107,
    APP_ID_DBASE                          = 108,
    APP_ID_DCAP                           = 109,
    APP_ID_DCCP                           = 110,
    APP_ID_DCP                            = 111,
    APP_ID_DEC_AUTH                       = 112,
    APP_ID_DEC_DEBUG                      = 113,
    APP_ID_DECVMS                         = 114,
    APP_ID_DEOS                           = 115,
    APP_ID_DHCPV6                         = 116,
    APP_ID_DIGG                           = 117,
    APP_ID_DIRECT_CONNECT                 = 118,
    APP_ID_DIRECT                         = 119,
    APP_ID_DIXIE                          = 120,
    APP_ID_DLS                            = 121,
    APP_ID_DNA_CML                        = 122,
    APP_ID_DNSIX                          = 123,
    APP_ID_DPSI                           = 124,
    APP_ID_DROPBOX                        = 125,
    APP_ID_DSFGW                          = 126,
    APP_ID_DSP3270                        = 127,
    APP_ID_DSP                            = 128,
    APP_ID_DSSETUP                        = 129,
    APP_ID_DTAG                           = 130,
    APP_ID_DTK                            = 131,
    APP_ID_EBAY                           = 132,
    APP_ID_EBAY_BID                       = 133,
    APP_ID_EBAY_SEARCH                    = 134,
    APP_ID_EBAY_WATCH                     = 135,
    APP_ID_EBUDDY                         = 136,
    APP_ID_EGP                            = 137,
    APP_ID_EMBLNDT                        = 138,
    APP_ID_EMFIS                          = 139,
    APP_ID_ENTRUSTTIME                    = 140,
    APP_ID_EPMAP                          = 141,
    APP_ID_ERPC                           = 142,
    APP_ID_ESET                           = 143,
    APP_ID_ESP                            = 144,
    APP_ID_ESRO                           = 145,
    APP_ID_ETH                            = 146,
    APP_ID_ETOS                           = 147,
    APP_ID_SAFARI_MOBILE_DUMMY            = 148,
    APP_ID_EXCHANGE                       = 1780,
    APP_ID_FACEBOOK_APPS                  = 149,
    APP_ID_FARK                           = 150,
    APP_ID_FARMVILLE                      = 151,
    APP_ID_FASP                           = 152,
    APP_ID_FASTTRACK                      = 153,
    APP_ID_FATMEN                         = 154,
    APP_ID_FILEMAKER                      = 155,
    APP_ID_FILER_CX                       = 156,
    APP_ID_FILESTUBE                      = 157,
    APP_ID_FLASHGET                       = 158,
    APP_ID_FLICKR                         = 159,
    APP_ID_FLIXSTER                       = 160,
    APP_ID_FOGBUGZ                        = 161,
    APP_ID_F_PROT                         = 162,
    APP_ID_FREECAST                       = 163,
    APP_ID_FRIENDFEED                     = 164,
    APP_ID_FTP_CONTROL                    = 165,
    APP_ID_FTP_DATA                       = 166,
    APP_ID_FTPSDATA                       = 167,
    APP_ID_FTPS                           = 168,
    APP_ID_FXP                            = 169,
    APP_ID_GACP                           = 170,
    APP_ID_GANGLIA                        = 171,
    APP_ID_GENESIS_PPP                    = 172,
    APP_ID_GENIE                          = 173,
    APP_ID_GENRAD                         = 174,
    APP_ID_GIGANEWS                       = 175,
    APP_ID_GIOP                           = 176,
    APP_ID_GIST                           = 177,
    APP_ID_GOOGLE_APIS                    = 178,
    APP_ID_GOOGLE_APP_ENGINE              = 179,
    APP_ID_GOOGLE_DOCS                    = 180,
    APP_ID_GOOGLE_TALK_GADGET             = 182,
    APP_ID_GOOGLE_TALK                    = 183,
    APP_ID_GOOGLE                         = 184,
    APP_ID_GOOGLE_TRANSLATE               = 185,
    APP_ID_GOOGLE_VIDEO                   = 186,
    APP_ID_GOTOMEETING                    = 187,
    APP_ID_GPFS                           = 188,
    APP_ID_GRE                            = 189,
    APP_ID_GROUPWISE                      = 190,
    APP_ID_GSIFTP                         = 191,
    APP_ID_GSS_LICENSE                    = 192,
    APP_ID_H_225                          = 193,
    APP_ID_H_245                          = 194,
    APP_ID_H_248                          = 195,
    APP_ID_H_323                          = 196,
    APP_ID_HASSLE                         = 197,
    APP_ID_HDAP                           = 198,
    APP_ID_HEMS                           = 199,
    APP_ID_HIVESTOR                       = 200,
    APP_ID_HL7                            = 201,
    APP_ID_HOPSTER                        = 202,
    APP_ID_HOSTNAME                       = 203,
    APP_ID_HOTFILE                        = 204,
    APP_ID_HOTMAIL                        = 205,
    APP_ID_HP_PERF                        = 206,
    APP_ID_HP_VMM                         = 207,
    APP_ID_HTTP_AUDIO                     = 208,
    APP_ID_HTTPMGT                        = 209,
    APP_ID_HTTP_VIDEO                     = 210,
    APP_ID_HYPER_G                        = 211,
    APP_ID_IASD                           = 212,
    APP_ID_IBM_OPC                        = 213,
    APP_ID_ICA_BROWSER                    = 214,
    APP_ID_ICAD                           = 215,
    APP_ID_ICAP                           = 216,
    APP_ID_ICA                            = 217,
    APP_ID_ICESHARE                       = 218,
    APP_ID_ICP                            = 221,
    APP_ID_ICQ2GO                         = 222,
    APP_ID_IDP                            = 223,
    APP_ID_IGMP                           = 224,
    APP_ID_IKE                            = 225,
    APP_ID_IMGAMES                        = 226,
    APP_ID_IMSP                           = 227,
    APP_ID_INBUSINESS                     = 228,
    APP_ID_INFORMIX                       = 229,
    APP_ID_INFOSEEK                       = 230,
    APP_ID_INFOSTORE                      = 231,
    APP_ID_INGRES_NET                     = 232,
    APP_ID_IPCOMP                         = 233,
    APP_ID_IPIP                           = 234,
    APP_ID_IP                             = 235,
    APP_ID_IPSEC                          = 236,
    APP_ID_IPV6                           = 237,
    APP_ID_IPX                            = 238,
    APP_ID_IRC                            = 239,
    APP_ID_IRCU                           = 240,
    APP_ID_IS_99                          = 241,
    APP_ID_ISAKMP                         = 242,
    APP_ID_ISCHAT                         = 243,
    APP_ID_ISI_GRAPHICS                   = 244,
    APP_ID_ISOIP                          = 245,
    APP_ID_ISO_TSAP                       = 246,
    APP_ID_JARGON                         = 247,
    APP_ID_KASPERSKY                      = 248,
    APP_ID_KBLOCK                         = 249,
    APP_ID_KFTPDATA                       = 250,
    APP_ID_KFTP                           = 251,
    APP_ID_KIS                            = 252,
    APP_ID_KNETCMP                        = 253,
    APP_ID_KRYPTOLAN                      = 254,
    APP_ID_KTELNET                        = 255,
    APP_ID_KUGOO                          = 256,
    APP_ID_KVM                            = 257,
    APP_ID_KWDB                           = 258,
    APP_ID_L2TP                           = 259,
    APP_ID_LA_MAINT                       = 260,
    APP_ID_LAST_FM                        = 261,
    APP_ID_LEGENT                         = 262,
    APP_ID_LINK                           = 263,
    APP_ID_LIVE365                        = 264,
    APP_ID_LIVEMEETING                    = 265,
    APP_ID_LIVESTATION                    = 266,
    APP_ID_LLMNR                          = 267,
    APP_ID_LOCUS_CONN                     = 268,
    APP_ID_LOCUS_MAP                      = 269,
    APP_ID_LOGMEIN                        = 270,
    APP_ID_LSARPC                         = 271,
    APP_ID_MAFIAWARS                      = 272,
    APP_ID_MAGENTA_LOGIC                  = 273,
    APP_ID_MAGICJACK                      = 274,
    APP_ID_MAILQ                          = 275,
    APP_ID_MANET                          = 276,
    APP_ID_MAPI                           = 277,
    APP_ID_MASQDIALER                     = 278,
    APP_ID_MATIP                          = 279,
    APP_ID_MCAFEE                         = 280,
    APP_ID_MC_FTP                         = 281,
    APP_ID_MCIDAS                         = 282,
    APP_ID_MCK_IVPIP                      = 283,
    APP_ID_MEDIAFIRE                      = 285,
    APP_ID_MEEBO                          = 286,
    APP_ID_MEETING_MAKER                  = 287,
    APP_ID_META5                          = 288,
    APP_ID_METAGRAM                       = 289,
    APP_ID_MF_COBOL                       = 290,
    APP_ID_MFTP                           = 291,
    APP_ID_MINI_SQL                       = 292,
    APP_ID_MIT_ML_DEV                     = 293,
    APP_ID_MIT_SPOOLER                    = 294,
    APP_ID_MIXI                           = 295,
    APP_ID_MOBILEIP                       = 296,
    APP_ID_MORTGAGEWARE                   = 297,
    APP_ID_MPLS_MULTICAST                 = 298,
    APP_ID_MPLS_UNICAST                   = 299,
    APP_ID_MPM                            = 300,
    APP_ID_MPP                            = 301,
    APP_ID_MPTN                           = 302,
    APP_ID_MS_CRS                         = 303,
    APP_ID_MSDN                           = 304,
    APP_ID_MSG                            = 305,
    APP_ID_MSMQ                           = 306,
    APP_ID_MSNP                           = 307,
    APP_ID_MSN                            = 308,
    APP_ID_MS_OLAP                        = 309,
    APP_ID_MS_ONLINE                      = 310,
    APP_ID_MSP                            = 311,
    APP_ID_MS_SQL                         = 312,
    APP_ID_MTA                            = 313,
    APP_ID_MULTIPLEX                      = 314,
    APP_ID_MUMPS                          = 315,
    APP_ID_MYSPACE_CHAT                   = 316,
    APP_ID_MYSPACE                        = 317,
    APP_ID_NAMP                           = 318,
    APP_ID_NAPSTER                        = 319,
    APP_ID_NCED                           = 320,
    APP_ID_NCLD                           = 321,
    APP_ID_NDS_AUTH                       = 322,
    APP_ID_NETBIOS                        = 323,
    APP_ID_NETINFO                        = 324,
    APP_ID_NETLOGON                       = 325,
    APP_ID_NETMEETING                     = 326,
    APP_ID_NETSC                          = 327,
    APP_ID_NETSCOUT                       = 328,
    APP_ID_NETWARE                        = 329,
    APP_ID_NFA                            = 330,
    APP_ID_NFS                            = 331,
    APP_ID_NI_FTP                         = 332,
    APP_ID_NI_MAIL                        = 333,
    APP_ID_NIP                            = 334,
    APP_ID_NNSP                           = 335,
    APP_ID_NOVABACKUP                     = 336,
    APP_ID_NPP                            = 337,
    APP_ID_NSIIOPS                        = 338,
    APP_ID_NSRMP                          = 339,
    APP_ID_NSS                            = 340,
    APP_ID_NSSTP                          = 341,
    APP_ID_NXEDIT                         = 342,
    APP_ID_NXTSTEP                        = 343,
    APP_ID_OCBINDER                       = 344,
    APP_ID_OCSERVER                       = 345,
    APP_ID_OCS                            = 346,
    APP_ID_ODMR                           = 347,
    APP_ID_OFTP                           = 348,
    APP_ID_OFTPS                          = 349,
    APP_ID_ONMUX                          = 350,
    APP_ID_OPALIS_ROBOT                   = 351,
    APP_ID_OPENPORT                       = 352,
    APP_ID_OPENVPN                        = 353,
    APP_ID_ORACLE_SQLNET                  = 355,
    APP_ID_ORKUT                          = 356,
    APP_ID_OSCAR                          = 357,
    APP_ID_OSUNMS                         = 358,
    APP_ID_PANDA                          = 359,
    APP_ID_PARTYPOKER                     = 360,
    APP_ID_PAWSERV                        = 361,
    APP_ID_PCMAIL                         = 362,
    APP_ID_PDAP                           = 363,
    APP_ID_PERSONALLINK                   = 364,
    APP_ID_PFTP                           = 365,
    APP_ID_PIM                            = 366,
    APP_ID_PIP                            = 367,
    APP_ID_PKIX_TIMESTAMP                 = 368,
    APP_ID_PLAXO                          = 369,
    APP_ID_POP2                           = 370,
    APP_ID_PPLIVE                         = 371,
    APP_ID_PPP_DISCOVERY                  = 372,
    APP_ID_PPP_SESSION                    = 373,
    APP_ID_PPSTREAM                       = 374,
    APP_ID_PPTP                           = 375,
    APP_ID_PRINTSRV                       = 376,
    APP_ID_PROFILE                        = 377,
    APP_ID_PROSPERO                       = 378,
    APP_ID_PTP                            = 379,
    APP_ID_PUP                            = 380,
    APP_ID_PWDGEN                         = 381,
    APP_ID_QBIK                           = 382,
    APP_ID_QFT                            = 383,
    APP_ID_QMTP                           = 384,
    APP_ID_QOTD                           = 385,
    APP_ID_QQ                             = 386,
    APP_ID_QUICKTIME                      = 387,
    APP_ID_RAP                            = 388,
    APP_ID_RARP                           = 389,
    APP_ID_REMAIL                         = 390,
    APP_ID_REMOTE_JOB_SERVICE             = 391,
    APP_ID_REMOTE_TELNET                  = 392,
    APP_ID_RESCAP                         = 393,
    APP_ID_RFR                            = 394,
    APP_ID_RIP                            = 395,
    APP_ID_RIS                            = 396,
    APP_ID_RJE                            = 397,
    APP_ID_RLOGIN                         = 398,
    APP_ID_RLP                            = 399,
    APP_ID_RMT                            = 400,
    APP_ID_RPC2PMAP                       = 401,
    APP_ID_RRP                            = 402,
    APP_ID_RSH                            = 403,
    APP_ID_RSVD                           = 404,
    APP_ID_RSVP                           = 405,
    APP_ID_RSVP_TUNNEL                    = 406,
    APP_ID_RTCP                           = 407,
    APP_ID_RTSPS                          = 408,
    APP_ID_SAMR                           = 409,
    APP_ID_SAP_HOSTCONTROL                = 410,
    APP_ID_SBNTBCST                       = 411,
    APP_ID_SCOI2DLG                       = 412,
    APP_ID_SCSI_ST                        = 413,
    APP_ID_SCTP                           = 414,
    APP_ID_SECOND_LIFE                    = 415,
    APP_ID_SECURSIGHT                     = 416,
    APP_ID_SEMANTIX                       = 417,
    APP_ID_SEND                           = 418,
    APP_ID_SET                            = 419,
    APP_ID_SFTP                           = 420,
    APP_ID_SGCP                           = 421,
    APP_ID_SGMP                           = 422,
    APP_ID_SHAREPOINT                     = 423,
    APP_ID_SHRINKWRAP                     = 424,
    APP_ID_SILVERPLATTER                  = 425,
    APP_ID_SIP                            = 426,
    APP_ID_SKYPE_AUTH                     = 428,
    APP_ID_SKYPE_OUT                      = 429,
    APP_ID_SKYPE_P2P                      = 430,
    APP_ID_SKYPE_PROBE                    = 431,
    APP_ID_SLINGBOX                       = 432,
    APP_ID_SMAKYNET                       = 433,
    APP_ID_SMART_SDP                      = 434,
    APP_ID_SMPTE                          = 435,
    APP_ID_SMSP                           = 436,
    APP_ID_SMUX                           = 437,
    APP_ID_SNA_GATEWAY                    = 438,
    APP_ID_SNET                           = 439,
    APP_ID_SNPP                           = 440,
    APP_ID_SOFTPC                         = 441,
    APP_ID_SOULSEEK                       = 442,
    APP_ID_SQL_SERVICES                   = 443,
    APP_ID_SRC                            = 444,
    APP_ID_SRMP                           = 445,
    APP_ID_SRS_SEND                       = 446,
    APP_ID_SSDP                           = 447,
    APP_ID_STATIONLAUNCHER                = 448,
    APP_ID_STATSRV                        = 449,
    APP_ID_STORE_ADMIN                    = 450,
    APP_ID_SU_MIT_TELNET                  = 451,
    APP_ID_SUN_RPC                        = 452,
    APP_ID_SUPDUP                         = 453,
    APP_ID_SUPERNEWS                      = 454,
    APP_ID_SURMEAS                        = 455,
    APP_ID_SVRLOC                         = 456,
    APP_ID_SWIFT_RVFP                     = 457,
    APP_ID_SYBASE_SQL                     = 458,
    APP_ID_SYMANTEC_SYSTEM_CENTER         = 459,
    APP_ID_SYNOPTICS                      = 460,
    APP_ID_SYSATT                         = 461,
    APP_ID_SYSLOG                         = 462,
    APP_ID_SYSTAT                         = 463,
    APP_ID_TACACS                         = 464,
    APP_ID_TAC_NEWS                       = 465,
    APP_ID_TCPMUX                         = 466,
    APP_ID_TCP                            = 467,
    APP_ID_TEXAR                          = 468,
    APP_ID_TFTPS                          = 469,
    APP_ID_TIME                           = 470,
    APP_ID_TMOBILE                        = 471,
    APP_ID_TOBIT                          = 472,
    APP_ID_TOR                            = 473,
    APP_ID_TRIPWIRE                       = 474,
    APP_ID_TUMBLR                         = 475,
    APP_ID_UAAC                           = 476,
    APP_ID_UARPS                          = 477,
    APP_ID_UC4                            = 478,
    APP_ID_UDP                            = 479,
    APP_ID_UIS                            = 480,
    APP_ID_ULSTPROC                       = 481,
    APP_ID_UMA                            = 482,
    APP_ID_UNICENTER                      = 483,
    APP_ID_UNIDATA_LDM                    = 484,
    APP_ID_UNIFY                          = 485,
    APP_ID_UPS                            = 486,
    APP_ID_USENET                         = 487,
    APP_ID_UTMP                           = 489,
    APP_ID_UUCP                           = 490,
    APP_ID_VCHAT                          = 491,
    APP_ID_VETTCP                         = 492,
    APP_ID_VMNET                          = 493,
    APP_ID_VMPWSCS                        = 494,
    APP_ID_VONAGE                         = 495,
    APP_ID_VSLMP                          = 496,
    APP_ID_VUZE                           = 497,
    APP_ID_WCCP                           = 498,
    APP_ID_WEBFILTER                      = 499,
    APP_ID_WEBLOGIC                       = 500,
    APP_ID_WIKIPEDIA                      = 501,
    APP_ID_WINDOWS_LIVE                   = 502,
    APP_ID_WINDOWS_MEDIA                  = 503,
    APP_ID_WINNY                          = 504,
    APP_ID_WINS                           = 505,
    APP_ID_WORDPRESS                      = 506,
    APP_ID_WORLD_OF_WARCRAFT              = 507,
    APP_ID_X_224                          = 508,
    APP_ID_X_25                           = 509,
    APP_ID_XANGA                          = 510,
    APP_ID_XBONE                          = 511,
    APP_ID_XBOX_LIVE                      = 512,
    APP_ID_XDMCP                          = 513,
    APP_ID_XFER                           = 514,
    APP_ID_XMPP                           = 515,
    APP_ID_XNS_AUTHENTICATION             = 516,
    APP_ID_XNS_CLEARINGHOUSE              = 517,
    APP_ID_XNS_MAIL                       = 518,
    APP_ID_XNS_TIME                       = 519,
    APP_ID_XNS                            = 520,
    APP_ID_XYPLEX                         = 521,
    APP_ID_YAHOO_GAMES                    = 522,
    APP_ID_YAHOO                          = 524,
    APP_ID_Z3950                          = 525,
    APP_ID_ZANNET                         = 526,
    APP_ID_ZEBRA                          = 527,
    APP_ID_ZOHO                           = 528,
    APP_ID_ZOHO_CHAT                      = 529,
    APP_ID_ZOHO_MAIL                      = 530,
    APP_ID_ZOHO_SHARE                     = 531,
    APP_ID_ZOHO_WIKI                      = 532,
    APP_ID_ZYNGA                          = 533,
    APP_ID_ZYNGA_POKER                    = 534,
    APP_ID_1_800_FLOWERS                  = 535,
    APP_ID_100BAO                         = 536,
    APP_ID_2CHANNEL                       = 537,
    APP_ID_6_PM                           = 538,
    APP_ID_ACE_HARDWARE_CORPORATION       = 539,
    APP_ID_ADDICTING_GAMES                = 540,
    APP_ID_ADOBE_UPDATE                   = 541,
    APP_ID_ADORAMA                        = 542,
    APP_ID_AIM_EXPRESS                    = 543,
    APP_ID_AMERICAN_EXPRESS               = 544,
    APP_ID_ANDROID_BROWSER                = 545,
    APP_ID_AOL_EMAIL                      = 546,
    APP_ID_AOL_SOFTWARE                   = 549,
    APP_ID_APPLE_EMAIL                    = 550,
    APP_ID_APPLE_STORE                    = 551,
    APP_ID_ARCSERVE                       = 552,
    APP_ID_ARES                           = 553,
    APP_ID_ARGOS                          = 554,
    APP_ID_ATOM                           = 555,
    APP_ID_ATOM_COM                       = 556,
    APP_ID_AUTOBLOG                       = 557,
    APP_ID_AUTOTRADER_COM                 = 558,
    APP_ID_B_H_PHOTO_VIDEO                = 559,
    APP_ID_BANK_OF_AMERICA                = 560,
    APP_ID_BARNES_AND_NOBLE               = 561,
    APP_ID_BARNEYS_NEW_YORK               = 562,
    APP_ID_BASECAMP                       = 563,
    APP_ID_BATTLENET                      = 564,
    APP_ID_BEARSHARE                      = 565,
    APP_ID_BEBO                           = 566,
    APP_ID_BEST_BUY                       = 567,
    APP_ID_BEWEEVEE                       = 568,
    APP_ID_BGP                            = 569,
    APP_ID_BITTORRENT_CLIENT              = 570,
    APP_ID_BITTRACKER_CLIENT              = 571,
    APP_ID_BLACK_DECKER_CORPORATION       = 572,
    APP_ID_BLACKBERRY_BROWSER             = 573,
    APP_ID_BLIP_TV                        = 574,
    APP_ID_BLOCKBUSTER                    = 575,
    APP_ID_BLOGGER                        = 576,
    APP_ID_BLOOMINGDALES                  = 577,
    APP_ID_BLUE_NILE                      = 578,
    APP_ID_BLUEFLY                        = 579,
    APP_ID_BOX_NET                        = 580,
    APP_ID_CAMERASDIRECT_COM_AU           = 581,
    APP_ID_CAPITAL_ONE                    = 582,
    APP_ID_CAR_AND_DRIVER                 = 583,
    APP_ID_CARMAX                         = 584,
    APP_ID_CDISCOUNT                      = 585,
    APP_ID_CHARACTER_GENERATOR            = 586,
    APP_ID_CHASE                          = 587,
    APP_ID_CHEAPTICKETS                   = 588,
    APP_ID_CHROME                         = 589,
    APP_ID_CITI                           = 590,
    APP_ID_CITY_SPORTS                    = 591,
    APP_ID_COLLABEDIT                     = 592,
    APP_ID_COSTCO                         = 593,
    APP_ID_CRAIGSLIST                     = 594,
    APP_ID_CRUTCHFIELD                    = 595,
    APP_ID_CURL                           = 596,
    APP_ID_CVS                            = 597,
    APP_ID_CVS_PSERVER                    = 598,
    APP_ID_DAAP                           = 599,
    APP_ID_DAILYMOTION                    = 600,
    APP_ID_DAVID_JONES                    = 601,
    APP_ID_DB2                            = 602,
    APP_ID_DCE_RPC                        = 603,
    APP_ID_DEALS_DIRECT                   = 604,
    APP_ID_DELICIOUS                      = 605,
    APP_ID_DELL                           = 606,
    APP_ID_DESTRUCTOID                    = 607,
    APP_ID_DEVIANTART                     = 608,
    APP_ID_DHCP                           = 609,
    APP_ID_DHCPV6_SERVER                  = 610,
    APP_ID_DICKS_SPORTING_GOODS           = 611,
    APP_ID_DIIGO                          = 612,
    APP_ID_DILLARDS                       = 613,
    APP_ID_DISCARD                        = 614,
    APP_ID_DISCOVER                       = 615,
    APP_ID_DNP3                           = 616,
    APP_ID_DNS                            = 617,
    APP_ID_DRDA                           = 618,
    APP_ID_DROPBEAR                       = 619,
    APP_ID_DRUGSTORE_COM                  = 620,
    APP_ID_E_TRADE                        = 621,
    APP_ID_EDMUNDS_COM                    = 622,
    APP_ID_EDONKEY                        = 623,
    APP_ID_EUDORA                         = 624,
    APP_ID_EUDORA_PRO                     = 625,
    APP_ID_EVOLUTION                      = 626,
    APP_ID_EXEC                           = 627,
    APP_ID_EXPEDIA                        = 628,
    APP_ID_FACEBOOK                       = 629,
    APP_ID_FACEBOOK_CHAT                  = 630,
    APP_ID_FACEBOOK_COMMENT               = 631,
    APP_ID_FACEBOOK_GAME_PREMIER_FOOTBALL = 632,
    APP_ID_FACEBOOK_READ_EMAIL            = 633,
    APP_ID_FACEBOOK_SEND_EMAIL            = 634,
    APP_ID_FACEBOOK_STATUS_UPDATE         = 635,
    APP_ID_FIDELITY                       = 636,
    APP_ID_FINGER                         = 637,
    APP_ID_FIREFOX                        = 638,
    APP_ID_FLASH_VIDEO                    = 639,
    APP_ID_FNAC                           = 640,
    APP_ID_FOXY                           = 641,
    APP_ID_FRIENDSTER                     = 642,
    APP_ID_FRYS_ELECTRONICS               = 643,
    APP_ID_FTD                            = 644,
    APP_ID_FTP                            = 645,
    APP_ID_G4                             = 646,
    APP_ID_GAME_INFORMER                  = 647,
    APP_ID_GAMESPOT                       = 648,
    APP_ID_GAMESPY                        = 649,
    APP_ID_GAMESTOP                       = 650,
    APP_ID_GAMETRAILERS                   = 651,
    APP_ID_GAWKER                         = 652,
    APP_ID_GENERIC                        = 653,
    APP_ID_GIFT                           = 654,
    APP_ID_GMAIL                          = 655,
    APP_ID_GNUCLEUS                       = 656,
    APP_ID_GNUCLEUSLAN                    = 657,
    APP_ID_GNUTELLA                       = 658,
    APP_ID_GNUTELLA2                      = 659,
    APP_ID_GOOGLE_ANALYTICS               = 660,
    APP_ID_GOOGLE_CALENDAR                = 661,
    APP_ID_GOOGLE_DESKTOP                 = 662,
    APP_ID_GOOGLE_NEWS                    = 663,
    APP_ID_GOOGLE_PRODUCT_SEARCH          = 664,
    APP_ID_GOOGLE_SAFEBROWSING            = 665,
    APP_ID_GOOGLE_TOOLBAR                 = 1146,
    APP_ID_GOPHER                         = 667,
    APP_ID_GTK_GNUTELLA                   = 668,
    APP_ID_HAIKU_LEARNING_SYSTEMS         = 669,
    APP_ID_HOME_DEPOT                     = 670,
    APP_ID_HOSTNAME_SERVER                = 671,
    APP_ID_GOOGLE_EARTH                   = 672,
    APP_ID_HOTLINE                        = 673,
    APP_ID_HOUSE_OF_FRASER                = 674,
    APP_ID_HSBC                           = 675,
    APP_ID_HTTP                           = 676,
    APP_ID_HULU                           = 677,
    APP_ID_IBM_APP                        = 678,
    APP_ID_ICQ                            = 679,
    APP_ID_IGN                            = 680,
    APP_ID_ILOVEIM                        = 681,
    APP_ID_IMAGESHACK                     = 682,
    APP_ID_IMAP                           = 683,
    APP_ID_IMGUR                          = 684,
    APP_ID_IMO_IM                         = 685,
    APP_ID_INTERNET_EXPLORER              = 686,
    APP_ID_IRCD                           = 687,
    APP_ID_ITU_H_323                      = 688,
    APP_ID_ITUNES                         = 689,
    APP_ID_J_C_PENNEY                     = 690,
    APP_ID_J_R                            = 691,
    APP_ID_JABBER                         = 692,
    APP_ID_JALOPNIK                       = 693,
    APP_ID_JAVA_RMI                       = 694,
    APP_ID_JIRA                           = 695,
    APP_ID_JOYSTIQ                        = 696,
    APP_ID_KAD                            = 697,
    APP_ID_KAY_JEWELERS                   = 698,
    APP_ID_KAZAA                          = 699,
    APP_ID_KMAIL                          = 700,
    APP_ID_KERBEROS                       = 701,
    APP_ID_KMART                          = 702,
    APP_ID_KOGAN_TECHNOLOGIES             = 703,
    APP_ID_KOHLS                          = 704,
    APP_ID_KONGREGATE                     = 705,
    APP_ID_KONQUEROR                      = 706,
    APP_ID_KOTAKU                         = 707,
    APP_ID_LAUNCHPAD                      = 708,
    APP_ID_LBPS                           = 709,
    APP_ID_LDAP                           = 710,
    APP_ID_LIMELIGHT                      = 711,
    APP_ID_LIMEWIRE                       = 712,
    APP_ID_LINKEDIN                       = 713,
    APP_ID_LINKEDIN_JOB_SEARCH            = 714,
    APP_ID_LINUXCONF                      = 715,
    APP_ID_LIVEJOURNAL                    = 716,
    APP_ID_LOGIN                          = 717,
    APP_ID_LOKALISTEN                     = 718,
    APP_ID_LORD_TAYLOR                    = 719,
    APP_ID_LOTUS_NOTES                    = 720,
    APP_ID_LOVEFILM                       = 721,
    APP_ID_LOWES                          = 722,
    APP_ID_LSH                            = 723,
    APP_ID_MANOLITO                       = 724,
    APP_ID_MEGACO                         = 725,
    APP_ID_MEGAVIDEO                      = 726,
    APP_ID_MENARDS                        = 727,
    APP_ID_METACAFE                       = 728,
    APP_ID_METAFILTER                     = 729,
    APP_ID_MGCP                           = 730,
    APP_ID_MICROSOFT_UPDATE               = 731,
    APP_ID_MICROSOFT_WINDOWS_MESSENGER    = 732,
    APP_ID_MINUS                          = 733,
    APP_ID_MIXX                           = 734,
    APP_ID_MMS                            = 735,
    APP_ID_SAFARI_MOBILE                  = 736,
    APP_ID_MODBUS                         = 737,
    APP_ID_MORGAN_STANLEY                 = 738,
    APP_ID_MORPHEUS                       = 739,
    APP_ID_MOVENETWORKS                   = 740,
    APP_ID_MP4                            = 741,
    APP_ID_MPEG                           = 742,
    APP_ID_MSN_MESSENGER                  = 743,
    APP_ID_MSN_MESSENGER_MAC              = 744,
    APP_ID_MUTE                           = 745,
    APP_ID_MUTT                           = 746,
    APP_ID_MYSQL                          = 747,
    APP_ID_MYUDUTU                        = 748,
    APP_ID_NCP                            = 749,
    APP_ID_NECKERMANN                     = 750,
    APP_ID_NEIMAN_MARCUS                  = 751,
    APP_ID_NESSUS                         = 752,
    APP_ID_NETBIOS_DGM                    = 753,
    APP_ID_NETBIOS_NS                     = 754,
    APP_ID_NETBIOS_SSN                    = 755,
    APP_ID_NETFLIX                        = 756,
    APP_ID_NETLOG                         = 757,
    APP_ID_NETVIBES                       = 758,
    APP_ID_NEWEGG                         = 759,
    APP_ID_NEWSNOW                        = 760,
    APP_ID_NEWSVINE                       = 761,
    APP_ID_NICO_NICO_DOUGA                = 762,
    APP_ID_NNTP                           = 763,
    APP_ID_NORSDTROM                      = 764,
    APP_ID_NSPLAYER                       = 765,
    APP_ID_NTALK                          = 766,
    APP_ID_NTP                            = 767,
    APP_ID_OFFICE_DEPOT                   = 768,
    APP_ID_OFFICEMAX                      = 769,
    APP_ID_OO_COM_AU                      = 770,
    APP_ID_OPENSSH                        = 771,
    APP_ID_OPERATING_SYSTEM               = 772,
    APP_ID_ORACLE_DATABASE                = 773,
    APP_ID_ORACLE_TNS                     = 774,
    APP_ID_ORBITZ                         = 775,
    APP_ID_OUTLOOK                        = 776,
    APP_ID_OUTLOOK_EXPRESS                = 777,
    APP_ID_OVERSTOCK_COM                  = 778,
    APP_ID_PANDORA                        = 779,
    APP_ID_PC_DUO                         = 780,
    APP_ID_PCANYWHERE                     = 781,
    APP_ID_PEERCAST                       = 782,
    APP_ID_PEERENABLER                    = 783,
    APP_ID_PHOTOBUCKET                    = 784,
    APP_ID_PICASA                         = 785,
    APP_ID_POCO                           = 786,
    APP_ID_POGO                           = 787,
    APP_ID_POP3                           = 788,
    APP_ID_POPCAP_GAMES                   = 789,
    APP_ID_POPURLS                        = 790,
    APP_ID_POSTGRESQL                     = 791,
    APP_ID_PRICELINE_COM                  = 792,
    APP_ID_PROFLOWERS                     = 793,
    APP_ID_PUTTY                          = 794,
    APP_ID_QUAKE                          = 795,
    APP_ID_QUICKFLIX                      = 796,
    APP_ID_QUILL_CORPORATION              = 797,
    APP_ID_QVC                            = 798,
    APP_ID_QZONE                          = 799,
    APP_ID_RADIUS                         = 800,
    APP_ID_RADIUS_ACCT                    = 801,
    APP_ID_RAPIDSHARE                     = 802,
    APP_ID_RDP                            = 803,
    APP_ID_REDDIT                         = 804,
    APP_ID_REDMINE                        = 805,
    APP_ID_REI                            = 806,
    APP_ID_REMOTE_DESKTOP_CLIENT          = 807,
    APP_ID_RENREN                         = 808,
    APP_ID_REVOLVECLOTHING                = 809,
    APP_ID_RONA                           = 810,
    APP_ID_RSS                            = 811,
    APP_ID_RTMP                           = 812,
    APP_ID_RTP                            = 813,
    APP_ID_RTSP                           = 814,
    APP_ID_SAFARI                         = 815,
    APP_ID_SAKS_FIFTH_AVENUE              = 816,
    APP_ID_SAMS_CLUB                      = 817,
    APP_ID_SCHUELERVZ                     = 818,
    APP_ID_SCHWAB                         = 819,
    APP_ID_SCOTTRADE                      = 820,
    APP_ID_SEARS                          = 821,
    APP_ID_SHAREAZA                       = 822,
    APP_ID_SHELL                          = 823,
    APP_ID_SHOCKWAVE                      = 824,
    APP_ID_SHOPLET                        = 825,
    APP_ID_SHOPNBC                        = 826,
    APP_ID_SHOPPING_HP_COM                = 827,
    APP_ID_SHOPSTYLE                      = 828,
    APP_ID_SHOUTCAST_RADIO                = 829,
    APP_ID_SHOWCLIX                       = 830,
    APP_ID_SHOWDOCUMENT                   = 831,
    APP_ID_SKYPE                          = 832,
    APP_ID_SKYPE_MAC                      = 833,
    APP_ID_SLASHDOT                       = 834,
    APP_ID_SLOW                           = 835,
    APP_ID_SMTP                           = 836,
    APP_ID_SNMP                           = 837,
    APP_ID_SNMP_TRAP                      = 838,
    APP_ID_SOCKS                          = 839,
    APP_ID_SORIBADA                       = 840,
    APP_ID_SPIN_DE                        = 841,
    APP_ID_SPORTS_AUTHORITY               = 842,
    APP_ID_SQL_SERVER                     = 843,
    APP_ID_SQUID                          = 844,
    APP_ID_SQUIRREL_EMAIL                 = 845, // deprecated
    APP_ID_SSH                            = 846,
    APP_ID_SSL                            = 847,
    APP_ID_STAPLES                        = 848,
    APP_ID_STAYFRIENDS                    = 849,
    APP_ID_STUBHUB                        = 850,
    APP_ID_STUDIVZ                        = 851,
    APP_ID_STUMBLEUPON                    = 852,
    APP_ID_STUN                           = 853,
    APP_ID_SWAROVSKI                      = 854,
    APP_ID_T_ROWE_PRICE                   = 855,
    APP_ID_TABULAR_DATA_STREAM_TDS        = 856,
    APP_ID_TALK                           = 857,
    APP_ID_TARGET                         = 858,
    APP_ID_TCHIBO                         = 859,
    APP_ID_TD_AMERITRADE                  = 860,
    APP_ID_TELNET                         = 861,
    APP_ID_TFTP                           = 862,
    APP_ID_THE_GAP                        = 863,
    APP_ID_THE_SHARPER_IMAGE              = 864,
    APP_ID_THINKGEEK                      = 865,
    APP_ID_THUNDERBIRD                    = 866,
    APP_ID_TICKETMASTER                   = 867,
    APP_ID_TICKETS_COM                    = 868,
    APP_ID_TICKETSNOW                     = 869,
    APP_ID_TIFFANY_CO                     = 870,
    APP_ID_TIGER_DIRECT                   = 871,
    APP_ID_TIMBUKTU                       = 872,
    APP_ID_TINYPIC                        = 873,
    APP_ID_TIVOLI                         = 874,
    APP_ID_TN3270                         = 875,
    APP_ID_TOC                            = 876,
    APP_ID_TOP_GEAR                       = 877,
    APP_ID_TRAC                           = 878,
    APP_ID_TRACEROUTE                     = 879,
    APP_ID_TRAVELOCITY                    = 880,
    APP_ID_TRIPADVISOR                    = 881,
    APP_ID_TWITTER                        = 882,
    APP_ID_URBAN_OUTFITTERS               = 883,
    APP_ID_USTREAM_TV                     = 884,
    APP_ID_VANGUARD                       = 885,
    APP_ID_VCOM                           = 886,
    APP_ID_VEHIX                          = 887,
    APP_ID_VENTE_PRIVEE_COM               = 888,
    APP_ID_VEOH                           = 889,
    APP_ID_VERIZON_EMAIL                  = 890,
    APP_ID_VIADEO                         = 891,
    APP_ID_VICTORIAS_SECRET               = 892,
    APP_ID_VIMEO                          = 893,
    APP_ID_VNC                            = 894,
    APP_ID_VNC_RFB                        = 895,
    APP_ID_VNC_SERVER_RFB                 = 896,
    APP_ID_VOIP_RTP                       = 897,
    APP_ID_VOIP_SIP                       = 898,
    APP_ID_VOYAGES_SNCF_COM               = 899,
    APP_ID_WACHOVIA                       = 900,
    APP_ID_WALMART                        = 901,
    APP_ID_WAV                            = 902,
    APP_ID_WEB_OF_TRUST                   = 903,
    APP_ID_WEBDAV                         = 904,
    APP_ID_WEBEX                          = 905,
    APP_ID_WEBSPHERE_MQ                   = 906,
    APP_ID_WELLS_FARGO                    = 907,
    APP_ID_WER_KENNT_WEN                  = 908,
    APP_ID_WGET                           = 909,
    APP_ID_WINDOWS_LIVE_HOTMAIL           = 910,
    APP_ID_WINDOWS_LIVE_SKYDRIVE          = 911,
    APP_ID_WINDOWS_MEDIA_PLAYER           = 912,
    APP_ID_WINMX                          = 913,
    APP_ID_WIZIQ                          = 914,
    APP_ID_WMA                            = 915,
    APP_ID_WMV                            = 916,
    APP_ID_WOOT                           = 917,
    APP_ID_WX                             = 918,
    APP_ID_X_FONT_SERVER                  = 919,
    APP_ID_X11                            = 920,
    APP_ID_XBOX                           = 921,
    APP_ID_XING                           = 922,
    APP_ID_XM_RADIO_ONLINE                = 923,
    APP_ID_XUNLEI                         = 924,
    APP_ID_XWINDOWS                       = 925,
    APP_ID_YAHOO_VOICE                    = 926,
    APP_ID_YET_ABC                        = 927,
    APP_ID_YOUSENDIT                      = 928,
    APP_ID_YOUTUBE                        = 929,
    APP_ID_ZALES                          = 930,
    APP_ID_ZAPPOS                         = 931,
    APP_ID_ZIP_CA                         = 932,
    APP_ID_ZOOOMR                         = 933,
    APP_ID_YAHOOMAIL                      = 946,
    APP_ID_YAHOO_TOOLBAR                  = 947,
    APP_ID_RSYNC                          = 1097,
    APP_ID_XSCPLS                         = 1098,
    APP_ID_ROBUST_MPA                     = 1100,
    APP_ID_VND_WAV                        = 1101,
    APP_ID_GPP                            = 1102,
    APP_ID_M4V                            = 1103,
    APP_ID_X_WAV                          = 1104,
    APP_ID_MPA                            = 1105,
    APP_ID_MP4A                           = 1106,
    APP_ID_SMTP_IMO                       = 1108,
    APP_ID_DDM_SSL                        = 1111,
    APP_ID_SMTPS                          = 1112,
    APP_ID_NNTPS                          = 1113,
    APP_ID_IMAPS                          = 1114,
    APP_ID_SSHELL                         = 1115,
    APP_ID_LDAPS                          = 1116,
    APP_ID_TELNETS                        = 1117,
    APP_ID_IRCS                           = 1118,
    APP_ID_POP3S                          = 1119,
    APP_ID_MSFT_GC_SSL                    = 1120,
    APP_ID_SF_APPLIANCE_MGMT              = 1121,
    APP_ID_HTTPS                          = 1122,
    APP_ID_SKYPE_TUNNELING                = 1126,
    APP_ID_ASPROXY                        = 1145,
    APP_ID_OPERA                          = 1288,
    APP_ID_SSL_CLIENT                     = 1296,
    APP_ID_AOL                            = 1419,
    APP_ID_MDNS                           = 1755,
    APP_ID_APPLE_CORE_MEDIA               = 2253,
    APP_ID_HTTP_TUNNEL                    = 2296,
    APP_ID_RTP_AUDIO                      = 2475,
    APP_ID_RTP_VIDEO                      = 2476,
    APP_ID_ULTRASURF                      = 2634,
    APP_ID_LYCOS                          = 2775,
    APP_ID_DOGPILE                        = 2804,
    APP_ID_SPDY                           = 2886,
    APP_ID_HTTP2                          = 2889,
    APP_ID_ANYCONNECT                     = 2921,
    APP_ID_ANYCONNECT_SSL_CLIENT          = 2922,
    APP_ID_ANYCONNECT_IPSEC_CLIENT        = 2923,
    APP_ID_ICMP                           = 3501,
    APP_ID_ICMPV6                         = 3558,
    APP_ID_HTTP_SSL_TUNNEL                = 3860,
    APP_ID_FTP_ACTIVE                     = 4002,
    APP_ID_FTP_PASSIVE                    = 4003,
    APP_ID_QUIC                           = 4023,
    APP_ID_PSIPHON                        = 4075,
    APP_ID_SMB_VERSION_1                  = 4645,
    APP_ID_SMB_VERSION_2                  = 4646,
    APP_ID_SMB_VERSION_3                  = 4647,
    APP_ID_HTTP3                          = 4667,
    APP_ID_SMB_OVER_QUIC                  = 4668,
    APP_ID_ENIP                           = 5001,
    APP_ID_CIP                            = 5002,
    APP_ID_CIP_UNKNOWN                    = 5003,
    APP_ID_CIP_MALFORMED                  = 5005,
    APP_ID_CIP_SAFETY                     = 7430,
    APP_ID_DNS_OVER_QUIC                  = 7504,
    APP_ID_DNS_OVER_HTTP3                 = 7505,
#ifdef REG_TEST
    APP_ID_DNS_OVER_TLS                   = 4615,
    APP_ID_REGTEST                        = 10000,
    APP_ID_REGTEST1                       = 10001,
    APP_ID_REGTEST2                       = 10002
#else
    APP_ID_DNS_OVER_TLS                   = 4615
#endif
};

enum AppIdType
{
    APP_ID_TYPE_SERVICE,
    APP_ID_TYPE_CLIENT,
    APP_ID_TYPE_PAYLOAD,
    APP_ID_TYPE_MAX
};

struct AppRegistryEntry
{
    AppId appId;
    uint32_t additionalInfo;
};

#endif

host_port_app_cache.h

Path = src/network_inspectors/appid/host_port_app_cache.h

#ifndef HOST_PORT_APP_CACHE_H
#define HOST_PORT_APP_CACHE_H

#include <cstring>

#include "application_ids.h"
#include "protocols/protocol_ids.h"
#include "sfip/sf_ip.h"
#include "utils/cpp_macros.h"

namespace snort
{
    struct SnortConfig;
}

class OdpContext;

PADDING_GUARD_BEGIN
struct HostPortKey
{
    HostPortKey()
    {
        ip.clear();
        port = 0;
        proto = IpProtocol::PROTO_NOT_SET;
        padding = 0;
    }

    bool operator<(const HostPortKey& right) const
    {
        return memcmp((const uint8_t*) this, (const uint8_t*) &right, sizeof(*this)) < 0;
    }

    snort::SfIp ip;
    uint16_t port;
    IpProtocol proto;
    char padding;
};
PADDING_GUARD_END

PADDING_GUARD_BEGIN
struct FirstPktkey
{
    FirstPktkey()
    {
        max_network_range.clear();
        network_address.clear();
    }

    bool operator<(const FirstPktkey& right) const
    {
        if ((htonl(right.netmask[0]) < htonl(this->netmask[0])) or
            (htonl(right.netmask[1]) < htonl(this->netmask[1])) or
            (htonl(right.netmask[2]) < htonl(this->netmask[2])) or
            (htonl(right.netmask[3]) < htonl(this->netmask[3])))
            return true;
        else
            return false;
    }

    uint32_t netmask[4] = {};
    snort::SfIp max_network_range;
    snort::SfIp network_address;
    uint16_t port = 0;
    IpProtocol proto = IpProtocol::PROTO_NOT_SET;
    char padding = 0;
};
PADDING_GUARD_END

struct HostPortVal
{
    AppId appId;
    unsigned type;
};

struct HostAppIdsVal
{
    AppId protocol_appId;
    AppId client_appId;
    AppId web_appId;
    unsigned reinspect;
};

class HostPortCache
{
public:
    HostPortVal* find(const snort::SfIp*, uint16_t port, IpProtocol, const OdpContext&);
    bool add(const snort::SnortConfig*, const snort::SfIp*, uint16_t port, IpProtocol,
        unsigned type, AppId);

    HostAppIdsVal* find_on_first_pkt(const snort::SfIp*, uint16_t port, IpProtocol, const OdpContext&);
    bool add_host(const snort::SnortConfig*, const snort::SfIp*, uint32_t* netmask, uint16_t port, IpProtocol,
        AppId, AppId, AppId, unsigned reinspect);
    void dump();

    ~HostPortCache()
    {
        cache.clear();
        cache_first_ip.clear();
        cache_first_subnet.clear();
    }

private:

    std::map<HostPortKey, HostPortVal> cache;
    std::map<HostPortKey, HostAppIdsVal> cache_first_ip;
    std::multimap<FirstPktkey, HostAppIdsVal> cache_first_subnet;
};

#endif

length_app_cache.h

Path = src/network_inspectors/appid/length_app_cache.h

#ifndef LENGTH_APP_CACHE_H
#define LENGTH_APP_CACHE_H

#include "protocols/protocol_ids.h"
#include "appid_types.h"
#include "application_ids.h"

#define LENGTH_SEQUENCE_CNT_MAX (5)

#pragma pack(1)

struct LengthSequenceEntry
{
    AppidSessionDirection direction = APP_ID_FROM_INITIATOR;
    uint16_t length = 0;       /* payload size (bytes) */
};

struct LengthKey
{
    IpProtocol proto = IpProtocol::PROTO_NOT_SET;  // IpProtocol::TCP or IpProtocol::UDP
    uint8_t sequence_cnt = 0;                      // num valid entries in sequence
    LengthSequenceEntry sequence[LENGTH_SEQUENCE_CNT_MAX];

    // Used by map where LengthKey object is the key
    bool operator<(const LengthKey& right) const
    {
        if (proto < right.proto)
            return true;
        else if (right.proto < proto)
            return false;

        if (sequence_cnt < right.sequence_cnt)
            return true;
        else if (right.sequence_cnt < sequence_cnt)
            return false;

        for (uint8_t i = 0; i < LENGTH_SEQUENCE_CNT_MAX; ++i)
        {
            if (sequence[i].direction < right.sequence[i].direction)
                return true;
            else if (right.sequence[i].direction < sequence[i].direction)
                return false;

            if (sequence[i].length < right.sequence[i].length)
                return true;
            else if (right.sequence[i].length < sequence[i].length)
                return false;
        }

        return false;
    }
};

#pragma pack()

class LengthCache
{
public:
    AppId find(const LengthKey& key)
    {
        auto entry = cache.find(key);
        if (entry == cache.end())
            return APP_ID_NONE;
        else
            return entry->second;
    }

    bool add(const LengthKey& key, AppId val)
    {
        return (cache.emplace(key, val)).second == true;
    }

private:
    std::map<LengthKey, AppId>cache;
};

#endif

lua_detector_api.h

Path = src/network_inspectors/appid/lua_detector_api.h

#ifndef LUA_DETECTOR_API_H
#define LUA_DETECTOR_API_H

// This module supports basic API towards Lua detectors.

#include <cstdint>
#include <string>

#include "appid_types.h"
#include "client_plugins/client_detector.h"
#include "service_plugins/service_detector.h"

namespace snort
{
struct Packet;
}
struct lua_State;
class AppIdSession;
class AppInfoTableEntry;

#define DETECTOR "Detector"
#define DETECTORFLOW "DetectorFlow"

#define LUA_STATE_GLOBAL_SC_ID  "snort_config"

struct DetectorPackageInfo
{
    std::string initFunctionName;
    std::string cleanFunctionName;
    std::string validateFunctionName;
    int minimum_matches = 0;
    std::string name = "NoName";
    IpProtocol proto = IpProtocol::PROTO_NOT_SET;
};

struct LuaDetectorParameters
{
    void init(AppIdDiscoveryArgs& args)
    {
        data = args.data;
        size = args.size;
        dir = args.dir;
        asd = &args.asd;
        change_bits = &args.change_bits;
        pkt = args.pkt;
    }

    const uint8_t* data = nullptr;
    uint16_t size = 0;
    AppidSessionDirection dir = APP_ID_FROM_INITIATOR;
    AppIdSession* asd = nullptr;
    AppidChangeBits* change_bits = nullptr;
    const snort::Packet* pkt = nullptr;
};

class LuaStateDescriptor
{
public:
    LuaDetectorParameters ldp;
    DetectorPackageInfo package_info;
    AppId service_id = APP_ID_UNKNOWN;
    int lua_validate(AppIdDiscoveryArgs&);
};

class LuaServiceDetector : public ServiceDetector
{
public:
    LuaServiceDetector(AppIdDiscovery* sdm, const std::string& detector_name,
        const std::string& log_name, bool is_custom, unsigned min_match, IpProtocol protocol);
    int validate(AppIdDiscoveryArgs&) override;
};

class LuaClientDetector : public ClientDetector
{
public:
    LuaClientDetector(AppIdDiscovery* cdm, const std::string& detector_name,
        const std::string& log_name, bool is_custom, unsigned min_match, IpProtocol protocol);
    int validate(AppIdDiscoveryArgs&) override;
};


class LuaObject {

public:
    LuaObject(OdpContext& odp_ctxt) : odp_ctxt(odp_ctxt) { }
    virtual ~LuaObject() = default;
    LuaObject(const LuaObject&) = delete;
    LuaObject& operator=(const LuaObject&) = delete;

    LuaStateDescriptor lsd;
    virtual AppIdDetector* get_detector() = 0;
    LuaStateDescriptor* validate_lua_state(bool packet_context);

    const std::string& get_cb_fn_name()
    { return cb_fn_name; }

    void set_cb_fn_name(const char* name)
    { cb_fn_name = name; }

    bool is_running()
    { return running; }

    void set_running(bool is_running)
    { running = is_running; }

    OdpContext& get_odp_ctxt() const
    { return odp_ctxt; }

private:
    std::string cb_fn_name;
    bool running = false;
    OdpContext& odp_ctxt;
};

class LuaServiceObject: public LuaObject
{
public:
    ServiceDetector* sd;
    LuaServiceObject(AppIdDiscovery* sdm, const std::string& detector_name,
        const std::string& log_name, bool is_custom, IpProtocol protocol, lua_State* L,
        OdpContext& odp_ctxt);
    ServiceDetector* get_detector() override
    { return sd; }
};

class LuaClientObject : public LuaObject
{
public:
    ClientDetector* cd;
    LuaClientObject(const std::string& detector_name,
        const std::string& log_name, bool is_custom, IpProtocol protocol, lua_State* L,
        OdpContext& odp_ctxt, bool& has_validate);
    ClientDetector* get_detector() override
    { return cd; }
};

typedef std::unordered_map<AppId, CHPApp*> CHPGlossary;

int register_detector(lua_State*);
void init_chp_glossary();
int init(lua_State*, int result=0);
void free_current_chp_glossary();
void free_old_chp_glossary();

void check_detector_callback(const snort::Packet& p, AppIdSession& asd, AppidSessionDirection dir,
    AppId app_id, AppidChangeBits& change_bits, AppInfoTableEntry* entry = nullptr);

#endif

lua_detector_flow_api.h

Path = src/network_inspectors/appid/lua_detector_flow_api.h

#ifndef LUA_DETECTOR_FLOW_API_H
#define LUA_DETECTOR_FLOW_API_H

// This module supports API towards Lua detectors for performing specific operations on a flow
// object.
// The flow object on Lua side is a userData.

#include "lua_detector_util.h"

struct lua_State;
class AppIdSession;

struct DetectorFlow
{
    DetectorFlow(lua_State* myLuaState, AppIdSession* asd)
    : myLuaState(myLuaState), asd(asd) { }

    ~DetectorFlow()
    {
        /*The detectorUserData itself is a userdata and therefore be freed by Lua side. */
        if (userDataRef != LUA_REFNIL)
        {
            luaL_unref(myLuaState, LUA_REGISTRYINDEX, userDataRef);
            userDataRef = LUA_REFNIL;
        }
    }

    lua_State* myLuaState;
    AppIdSession* asd;
    int userDataRef = 0;
};

int register_detector_flow_api(lua_State*);

#endif

lua_detector_module.h

Path = src/network_inspectors/appid/lua_detector_module.h

#ifndef LUA_DETECTOR_MODULE_H
#define LUA_DETECTOR_MODULE_H

#include <cstdint>
#include <list>
#include <map>
#include <string>

#include <lua.hpp>
#include <lua/lua.h>

#include "main/thread.h"
#include "main/thread_config.h"
#include "protocols/protocol_ids.h"

#include "application_ids.h"

namespace snort
{
    struct SnortConfig;
}

class AppIdContext;
class AppIdDetector;
struct DetectorFlow;
class LuaObject;

bool get_lua_field(lua_State* L, int table, const char* field, std::string& out);
bool get_lua_field(lua_State* L, int table, const char* field, int& out);
bool get_lua_field(lua_State* L, int table, const char* field, IpProtocol& out);

class LuaDetectorManager
{
public:
    LuaDetectorManager(AppIdContext&, bool);
    ~LuaDetectorManager();
    static void initialize(const snort::SnortConfig*, AppIdContext&, bool is_control=false,
        bool reload=false);
    static void init_thread_manager(const snort::SnortConfig*, const AppIdContext&);
    static void cleanup_after_swap();
    static void clear_lua_detector_mgrs();

    void set_detector_flow(DetectorFlow* df)
    {
        detector_flow = df;
    }

    DetectorFlow* get_detector_flow()
    {
        return detector_flow;
    }

    void set_ignore_chp_cleanup(bool value)
    {
        ignore_chp_cleanup = value;
    }

    void free_detector_flow();
    lua_State* L;
    bool insert_cb_detector(AppId app_id, LuaObject* ud);
    LuaObject* get_cb_detector(AppId app_id);

private:
    void initialize_lua_detectors(bool is_control, bool reload = false);
    void activate_lua_detectors(const snort::SnortConfig*);
    void list_lua_detectors();
    bool load_detector(char* detector_name, bool is_custom, bool is_control, bool reload, std::string& buf);
    void load_lua_detectors(const char* path, bool is_custom, bool is_control, bool reload = false);
    LuaObject* create_lua_detector(const char* detector_name, bool is_custom,
        const char* detector_filename, bool& has_validate);

    AppIdContext& ctxt;
    std::list<LuaObject*> allocated_objects;
    size_t num_odp_detectors = 0;
    std::map<AppId, LuaObject*> cb_detectors;
    DetectorFlow* detector_flow = nullptr;
    bool ignore_chp_cleanup = false;
};

#endif

lua_detector_util.h

Path = src/network_inspectors/appid/lua_detector_util.h

#ifndef LUA_DETECTOR_UTIL_H
#define LUA_DETECTOR_UTIL_H

// encapsulate Lua interface boilerplate to get sane, identical behavior across users

#include <cassert>
#include <lua.hpp>

template<typename T>
struct UserData
{
    T* ptr;

    T* operator->()
    { return ptr; }

    const T* operator->() const
    { return ptr; }

    operator T*() const
    { return ptr; }

    operator T*()
    { return ptr; }

    static UserData<T>* push(lua_State* L, const char* const meta, T* ptr = nullptr)
    {
        int type_size = sizeof(UserData<T>);
        auto ud = static_cast<UserData<T>*>(lua_newuserdata(L, type_size));
        assert(ud);
        ud->ptr = ptr;
        luaL_getmetatable(L, meta);
        lua_setmetatable(L, -2);
        return ud;
    }

    static UserData<T>* check(lua_State* L, const char* const meta, int n)
    {
        luaL_checktype(L, n, LUA_TUSERDATA);
        auto ud = static_cast<UserData<T>*>(luaL_checkudata(L, n, meta));
        assert(ud);
        return ud;
    }
};

#endif

service_state.h

Path = src/network_inspectors/appid/service_state.h

#ifndef SERVICE_STATE_H
#define SERVICE_STATE_H

#include <list>
#include <map>

#include "protocols/protocol_ids.h"
#include "sfip/sf_ip.h"
#include "utils/cpp_macros.h"
#include "utils/util.h"

#include "appid_peg_counts.h"
#include "service_plugins/service_discovery.h"

class ServiceDetector;

struct AppIdServiceStateKey;
class ServiceDiscoveryState;

typedef std::map<AppIdServiceStateKey, ServiceDiscoveryState*> Map_t;
typedef std::list<Map_t::iterator> Queue_t;

enum ServiceState
{
    SEARCHING_PORT_PATTERN = 0,
    SEARCHING_BRUTE_FORCE,
    FAILED,
    VALID
};

class AppIdDetectorList
{
public:
    AppIdDetectorList(IpProtocol proto, ServiceDiscovery& sd)
    {
        if (proto == IpProtocol::TCP)
            detectors = sd.get_tcp_detectors();
        else
            detectors = sd.get_udp_detectors();
        dit = detectors->begin();
    }

    ServiceDetector* next()
    {
        ServiceDetector* detector = nullptr;

        if ( dit != detectors->end())
            detector = (ServiceDetector*)(dit++)->second;
        return detector;
    }

    void reset()
    {
        dit = detectors->begin();
    }

private:
    AppIdDetectors* detectors;
    AppIdDetectorsIterator dit;
};

class ServiceDiscoveryState
{
public:
    ServiceDiscoveryState();
    ~ServiceDiscoveryState();
    ServiceDetector* select_detector_by_brute_force(IpProtocol proto, ServiceDiscovery& sd);
    void set_service_id_valid(ServiceDetector* sd);
    void set_service_id_failed(AppIdSession& asd, const snort::SfIp* client_ip,
        unsigned invalid_delta = 0);
    void update_service_incompatible(const snort::SfIp* ip);

    ServiceState get_state() const
    {
        return state;
    }

    void set_state(ServiceState state)
    {
        this->state = state;
    }

    ServiceDetector* get_service() const
    {
        return service;
    }

    void set_service(ServiceDetector* service)
    {
        this->service = service;
    }

    time_t get_reset_time() const
    {
        return reset_time;
    }

    void set_reset_time(time_t resetTime)
    {
        reset_time = resetTime;
    }

    Queue_t::iterator qptr; // Our place in service_state_queue

private:
    ServiceState state;
    ServiceDetector* service = nullptr;
    AppIdDetectorList* tcp_brute_force_mgr = nullptr;
    AppIdDetectorList* udp_brute_force_mgr = nullptr;
    unsigned valid_count = 0;
    unsigned detract_count = 0;
    snort::SfIp last_detract;

    // consecutive incompatible flows - incompatible means client packet did not match.
    unsigned invalid_client_count = 0;

    /**IP address of client in last flow that was declared incompatible. If client IP address is
     * different every time, then consecutive incompatible status indicate that flow is not using
     * specific service.
     */
    snort::SfIp last_invalid_client;
    time_t reset_time;
};

class AppIdServiceState
{
public:
    static bool initialize(size_t memcap);
    static void clean();
    static ServiceDiscoveryState* add(const snort::SfIp*, IpProtocol, uint16_t port,
        int16_t group, uint32_t asid, bool decrypted, bool do_touch = false);
    static ServiceDiscoveryState* get(const snort::SfIp*, IpProtocol, uint16_t port,
        int16_t group, uint32_t asid, bool decrypted, bool do_touch = false);
    static void remove(const snort::SfIp*, IpProtocol, uint16_t port,
        int16_t group, uint32_t asid, bool decrypted);
    static void check_reset(AppIdSession& asd, const snort::SfIp* ip, uint16_t port,
        int16_t group, uint32_t asid);
    static bool prune(size_t max_memory = 0, size_t num_items = -1u);
};


PADDING_GUARD_BEGIN
struct AppIdServiceStateKey
{
    AppIdServiceStateKey(const snort::SfIp* ip,
        IpProtocol proto, uint16_t port, int16_t group, uint32_t asid, bool decrypted) :
        ip(*ip), port(port), asid(asid), group(group), decrypted(decrypted), proto(proto)
    { }

    bool operator<(const AppIdServiceStateKey& right) const
    {
        return memcmp((const uint8_t*) this, (const uint8_t*) &right, sizeof(*this)) < 0;
    }

    snort::SfIp ip;
    uint16_t port;
    uint32_t asid;
    int16_t group;
    bool decrypted;
    IpProtocol proto;
};
PADDING_GUARD_END


extern THREAD_LOCAL AppIdStats appid_stats;

class MapList
{
public:

    MapList(size_t cap) : memcap(cap), mem_used(0) {}

    ~MapList()
    {
        for ( const auto& kv : m )
            delete kv.second;
    }

    ServiceDiscoveryState* add(const AppIdServiceStateKey& k, bool do_touch = false)
    {
        ServiceDiscoveryState* ss = nullptr;

        // Try to emplace k first, with a nullptr.
        std::pair<Map_t::iterator, bool> sit = m.emplace( std::make_pair(k, ss) );
        Map_t::iterator it = sit.first;

        if ( sit.second )
        {
            // emplace succeeded
            ss = it->second = new ServiceDiscoveryState;
            q.emplace_back(it);
            mem_used += sz;
            ss->qptr = --q.end(); // remember our place in the queue
            appid_stats.service_cache_adds++;
            appid_stats.bytes_in_use = mem_used;
            appid_stats.items_in_use = m.size();

            if ( mem_used > memcap )
                remove( q.front() );
        }
        else
        {
            ss = it->second;
            if ( do_touch )
                touch( ss->qptr );
        }

        return ss;
    }

    ServiceDiscoveryState* get(const AppIdServiceStateKey& k, bool do_touch = false)
    {
        Map_t::const_iterator it = m.find(k);
        if ( it != m.end() ) {
            if ( do_touch )
                touch(it->second->qptr);
            return it->second;
        }
        return nullptr;
    }

    bool remove(Map_t::iterator it)
    {
        if ( it != m.end() && !m.empty() )
        {
            assert( mem_used >= sz );
            mem_used -= sz;
            q.erase(it->second->qptr);  // remove from queue
            delete it->second;
            m.erase(it);                // then from cache
            appid_stats.service_cache_removes++;
            appid_stats.bytes_in_use = mem_used;
            appid_stats.items_in_use = m.size();

            return true;
        }
        return false;
    }

    bool prune(size_t max_memory = 0, size_t num_items = -1u)
    {
        if ( max_memory == 0 )
            max_memory = memcap;

        size_t i=0;
        while ( mem_used > max_memory && i++ < num_items )
            remove( q.front() );

        appid_stats.service_cache_prunes++;

        return mem_used <= max_memory;
    }

    Map_t::iterator find(const AppIdServiceStateKey& k)
    {
        return m.find(k);
    }

    void touch(Queue_t::iterator& qptr)
    {
        // If we don't already have the highest priority...
        if ( *qptr != q.back() )
        {
            q.emplace_back(*qptr);
            q.erase(qptr);
            qptr = --q.end();
        }
    }

    size_t size() const { return m.size(); }

    Queue_t::iterator newest() { return --q.end(); }
    Queue_t::iterator oldest() { return q.begin(); }
    Queue_t::iterator end() { return q.end(); }

    // how much memory we add when we put an SDS in the cache:
    static const size_t sz;

    friend class AppIdServiceState;

private:
    Map_t m;
    Queue_t q;
    size_t memcap;
    size_t mem_used;
};

#endif

tp_appid_module_api.h

Path = src/network_inspectors/appid/tp_appid_module_api.h

#ifndef TP_APPID_MODULE_API_H
#define TP_APPID_MODULE_API_H

#include <vector>
#include <string>

#include "main/thread.h"
#include "profiler/profiler_defs.h"
#include "tp_appid_types.h"

#define THIRD_PARTY_APPID_API_VERSION 6

struct TPAppidProfilerFunctions
{
    void* (*appid_malloc) (size_t);
    void (*appid_free) (void*);
};

class ThirdPartyConfig
{
public:
    uint32_t chp_body_collection_max = 0;
    bool ftp_userid_disabled = false;
    bool chp_body_collection_disabled = false;
    bool tp_allow_probes = false;
    std::string tp_appid_config;
    bool tp_appid_stats_enable = false;
    bool tp_appid_config_dump = false;
    TPAppidProfilerFunctions tp_appid_profiler_functions = {};
};

class ThirdPartyAppIdContext
{
public:
    ThirdPartyAppIdContext(uint32_t ver, const char* mname, ThirdPartyConfig& config)
        : api_version(ver), name(mname), cfg(config)
    {
        version = next_version++;
    }

    uint32_t get_version() const
    {
        return version;
    }

    virtual ~ThirdPartyAppIdContext() = default;

    uint32_t get_api_version() const { return api_version; }
    const std::string& module_name() const { return name; }

    virtual int tinit() = 0;
    virtual bool tfini(bool is_idling = false) = 0;

    virtual const ThirdPartyConfig& get_config() const { return cfg; }

    static void set_tp_reload_in_progress(bool value) { tp_reload_in_progress = value; }
    static bool get_tp_reload_in_progress() { return tp_reload_in_progress; }

    virtual const std::string& get_user_config() const = 0;

protected:
    const uint32_t api_version;
    const std::string name;
    ThirdPartyConfig cfg;

private:
    // No implicit constructor as derived classes need to provide
    // version and name.
    ThirdPartyAppIdContext() : api_version(0), name(""), version(0) { }
    uint32_t version;
    static THREAD_LOCAL bool tp_reload_in_progress;
    // Increments when a new third-party context is loaded
    SO_PUBLIC static uint32_t next_version;
};

#endif

tp_appid_session_api.h

Path = src/network_inspectors/appid/tp_appid_session_api.h

#ifndef TP_APPID_SESSION_API_H
#define TP_APPID_SESSION_API_H

#include <vector>
#include <string>
#include "appid_types.h"
#include "application_ids.h"
#include "tp_appid_types.h"

namespace snort
{
struct Packet;
}

class ThirdPartyAppIdContext;

class ThirdPartyAppIdSession
{
public:
    ThirdPartyAppIdSession(const ThirdPartyAppIdContext& ctxt)
        : appid(APP_ID_NONE), confidence(100), state(TP_STATE_INIT), ctxt(ctxt)
    {
        ctxt_version = ctxt.get_version();
    }

    virtual ~ThirdPartyAppIdSession() = default;

    virtual void reset() = 0;            // just reset state
    virtual void delete_with_ctxt() = 0;
    virtual TPState process(const snort::Packet&,
        AppidSessionDirection direction,
        std::vector<AppId>& proto_list,
        ThirdPartyAppIDAttributeData& attribute_data) = 0;

    virtual int disable_flags(uint32_t session_flags) = 0;
    virtual TPState get_state() { return state; }
    virtual void set_state(TPState) = 0;
    virtual void clear_attr(TPSessionAttr) = 0;
    virtual void set_attr(TPSessionAttr) = 0;
    virtual unsigned get_attr(TPSessionAttr) = 0;
    virtual AppId get_appid(int& conf) { conf=confidence; return appid; }
    virtual const ThirdPartyAppIdContext& get_ctxt() const
    { return ctxt; }
    uint32_t get_ctxt_version() { return ctxt_version; }

protected:
    AppId appid;
    int confidence;
    TPState state;
    const ThirdPartyAppIdContext& ctxt;
    uint32_t ctxt_version;
};

#endif

tp_appid_types.h

Path = src/network_inspectors/appid/tp_appid_types.h

#ifndef TP_APPID_TYPES_H
#define TP_APPID_TYPES_H

#include <cstdint>
#include <string>

#define MAX_ATTR_LEN 2048

using std::string;

enum TPFlags
{
    TP_SESSION_FLAG_DPI        = 0x00000001,
    TP_SESSION_FLAG_MUTABLE    = 0x00000002,
    TP_SESSION_FLAG_FUTUREFLOW = 0x00000004,
    TP_SESSION_FLAG_ATTRIBUTE  = 0x00000008,
    TP_SESSION_FLAG_TUNNELING  = 0x00000010
};

enum TPState
{
    TP_STATE_INIT,
    TP_STATE_TERMINATED,
    TP_STATE_INSPECTING,
    TP_STATE_MONITORING,
    TP_STATE_CLASSIFIED,
    TP_STATE_HA = 21
};

enum TPSessionAttr
{
    TP_ATTR_CONTINUE_MONITORING     = (1 << 0),
    TP_ATTR_COPY_RESPONSE_CONTENT   = (1 << 1),
    TP_ATTR_COPY_RESPONSE_LOCATION  = (1 << 2),
    TP_ATTR_COPY_RESPONSE_BODY      = (1 << 3),
    TP_ATTR_UNAME_KNOWN             = (1 << 4),
};

static void set_attr(string*& attr, const char* buf, size_t len, bool flush, size_t max_len)
{
    if (!attr)
        attr = new string(buf, len > max_len ? max_len : len);
    else if (flush)
    {
        delete attr;
        attr = new string(buf, len > max_len ? max_len : len);
    }
    else if (attr->size() < max_len)
    {
        size_t max_copy_len = max_len - attr->size();
        attr->append(buf, len > max_copy_len ? max_copy_len : len);
    }
    // else, skip copy because the buffer is filled up to its limit
}

#define TPAD_GET(func)                                          \
    string* func(bool caller_owns_it = false)                   \
    {                                                           \
        string* tmp = func ## _buf;                             \
        if (caller_owns_it)                                     \
            func ## _buf = nullptr;                             \
        return tmp;                                             \
    }

#define TPAD_SET(func)                                                  \
    void set_ ## func(const char* buf, size_t len, bool last_fragment, size_t max_len = MAX_ATTR_LEN) \
    {                                                                   \
        set_attr(func ## _buf, buf, len, func ## _flush, max_len);      \
        func ## _flush = last_fragment;                                 \
    }

// The ThirdPartyAppIDAttributeData class acts as a per packet cache for
// various fields coming out of that packet, that need to be analyzed
// simultaneously.
//
// Consumers of these fields should grab the pointers and avoid an extra copy,
// if speed is of the essence. Once they grab a string* pointer from
// ThirdPartyAppIDAttributeData, then they own it. That is, the consumers
// are responsible for deleting the pointers they grabbed.
//
// If you get it, you own it.
//
// However, currently tp_appid_utils.cc retrieves these, but the
// AppIdHttpSession::set() functions make copies anyway.
// Therefore, for now there is no need for any caller to own these
// pointers yet. We therefore allow callers to just peek at the string*
// buffers inside this class, without owning them. Hence the "caller_owns_it"
// flag in the get functions below.
//
class ThirdPartyAppIDAttributeData
{
    string* spdy_request_path_buf = nullptr;
    string* spdy_request_scheme_buf = nullptr;
    string* spdy_request_host_buf = nullptr;
    string* http_request_url_buf = nullptr;
    string* http_request_uri_buf = nullptr;
    string* http_request_host_buf = nullptr;
    string* http_request_cookie_buf = nullptr;
    string* http_request_via_buf = nullptr;
    string* http_response_via_buf = nullptr;
    string* http_request_user_agent_buf = nullptr;
    string* http_response_code_buf = nullptr;
    string* http_response_content_buf = nullptr;
    string* http_response_location_buf = nullptr;
    string* http_response_body_buf = nullptr;
    string* http_request_body_buf = nullptr;
    string* http_response_server_buf = nullptr;
    string* http_request_x_working_with_buf = nullptr;
    string* tls_host_buf = nullptr;
    string* tls_cname_buf = nullptr;
    string* tls_org_unit_buf = nullptr;
    string* http_request_referer_buf = nullptr;
    string* ftp_command_user_buf = nullptr;
    string* quic_sni_buf = nullptr;

    // will be set to true after last fragment for a metadata field is received
    bool spdy_request_path_flush = true;
    bool spdy_request_scheme_flush = true;
    bool spdy_request_host_flush = true;
    bool http_request_url_flush = true;
    bool http_request_uri_flush = true;
    bool http_request_host_flush = true;
    bool http_request_cookie_flush = true;
    bool http_request_via_flush = true;
    bool http_response_via_flush = true;
    bool http_request_user_agent_flush = true;
    bool http_response_code_flush = true;
    bool http_response_content_flush = true;
    bool http_response_location_flush = true;
    bool http_response_body_flush = true;
    bool http_request_body_flush = true;
    bool http_response_server_flush = true;
    bool http_request_x_working_with_flush = true;
    bool tls_host_flush = true;
    bool tls_cname_flush = true;
    bool tls_org_unit_flush = true;
    bool http_request_referer_flush = true;
    bool ftp_command_user_flush = true;
    bool quic_sni_flush = true;

    // FIXIT-L: make these private too. Figure out how these get set in tp.

public:
    ThirdPartyAppIDAttributeData() = default;

    ~ThirdPartyAppIDAttributeData()
    {
        // Only delete the pointers that we own (i.e. non null pointers).
        if (spdy_request_path_buf) delete spdy_request_path_buf;
        if (spdy_request_scheme_buf) delete spdy_request_scheme_buf;
        if (spdy_request_host_buf) delete spdy_request_host_buf;
        if (http_request_url_buf) delete http_request_url_buf;
        if (http_request_uri_buf) delete http_request_uri_buf;
        if (http_request_host_buf) delete http_request_host_buf;
        if (http_request_cookie_buf) delete http_request_cookie_buf;
        if (http_request_via_buf) delete http_request_via_buf;
        if (http_response_via_buf) delete http_response_via_buf;
        if (http_request_user_agent_buf) delete http_request_user_agent_buf;
        if (http_response_code_buf) delete http_response_code_buf;
        if (http_response_content_buf) delete http_response_content_buf;
        if (http_response_location_buf) delete http_response_location_buf;
        if (http_response_body_buf) delete http_response_body_buf;
        if (http_request_body_buf) delete http_request_body_buf;
        if (http_response_server_buf) delete http_response_server_buf;
        if (http_request_x_working_with_buf) delete http_request_x_working_with_buf;
        if (tls_host_buf) delete tls_host_buf;
        if (tls_cname_buf) delete tls_cname_buf;
        if (tls_org_unit_buf) delete tls_org_unit_buf;
        if (http_request_referer_buf) delete http_request_referer_buf;
        if (ftp_command_user_buf) delete ftp_command_user_buf;
        if (quic_sni_buf) delete quic_sni_buf;
    }

    // Note: calling these 2 times in a row, the 2nd time it returns null.
    TPAD_GET(spdy_request_path)
    TPAD_GET(spdy_request_scheme)
    TPAD_GET(spdy_request_host)
    TPAD_GET(http_request_url)
    TPAD_GET(http_request_uri)
    TPAD_GET(http_request_host)
    TPAD_GET(http_request_cookie)
    TPAD_GET(http_request_via)
    TPAD_GET(http_response_via)
    TPAD_GET(http_request_user_agent)
    TPAD_GET(http_response_code)
    TPAD_GET(http_response_content)
    TPAD_GET(http_response_location)
    TPAD_GET(http_response_body)
    TPAD_GET(http_request_body)
    TPAD_GET(http_response_server)
    TPAD_GET(http_request_x_working_with)
    TPAD_GET(tls_host)
    TPAD_GET(tls_cname)
    TPAD_GET(tls_org_unit)
    TPAD_GET(http_request_referer)
    TPAD_GET(ftp_command_user)
    TPAD_GET(quic_sni)

    // set functions
    TPAD_SET(spdy_request_path)
    TPAD_SET(spdy_request_scheme)
    TPAD_SET(spdy_request_host)
    TPAD_SET(http_request_url)
    TPAD_SET(http_request_uri)
    TPAD_SET(http_request_host)
    TPAD_SET(http_request_cookie)
    TPAD_SET(http_request_via)
    TPAD_SET(http_response_via)
    TPAD_SET(http_request_user_agent)
    TPAD_SET(http_response_code)
    TPAD_SET(http_response_content)
    TPAD_SET(http_response_location)
    TPAD_SET(http_response_body)
    TPAD_SET(http_request_body)
    TPAD_SET(http_response_server)
    TPAD_SET(http_request_x_working_with)
    TPAD_SET(tls_host)
    TPAD_SET(tls_cname)
    TPAD_SET(tls_org_unit)
    TPAD_SET(http_request_referer)
    TPAD_SET(ftp_command_user)
    TPAD_SET(quic_sni)
};

#endif

tp_appid_utils.h

Path = src/network_inspectors/appid/tp_appid_utils.h

#ifndef TP_APPID_UTILS_H
#define TP_APPID_UTILS_H

#include "protocols/packet.h"
#include "appid_types.h"

class AppIdSession;

bool do_tp_discovery(ThirdPartyAppIdContext& tp_appid_ctxt, AppIdSession&, IpProtocol, snort::Packet*,
    AppidSessionDirection&, AppidChangeBits&);

#endif

tp_lib_handler.h

Path = src/network_inspectors/appid/tp_lib_handler.h

#ifndef TP_LIB_HANDLER_H
#define TP_LIB_HANDLER_H

#include "tp_appid_module_api.h"
#include "tp_appid_session_api.h"

class AppIdConfig;
class OdpContext;

// This needs to be exported by any third party .so library.
// Must return null if it fails to create the object.
typedef ThirdPartyAppIdContext* (* TpAppIdCreateCtxt)(ThirdPartyConfig& );
typedef ThirdPartyAppIdSession* (* TpAppIdCreateSession)(ThirdPartyAppIdContext& ctxt);
typedef int (* TpAppIdPfini)();
typedef int (* TpAppIdTfini)();

// Class responsible for loading/reloading the thirdparty.so library
class TPLibHandler
{
public:
    static TPLibHandler* get()
    {
        if (self)
            return self;
        else
            return (self = new TPLibHandler());
    }

    static ThirdPartyAppIdContext* create_tp_appid_ctxt(const AppIdConfig& config,
        const OdpContext& odp_ctxt);
    static void tfini();
    static void pfini();

    TpAppIdCreateSession tpsession_factory() const
    {
        return tp_appid_create_session;
    }

private:
    TPLibHandler() = default;
    ~TPLibHandler() = default;

    static TPLibHandler* self;
    void* tp_so_handle = nullptr;   // output of dlopen(thirdparty.so)
    TpAppIdCreateCtxt tp_appid_create_ctxt = nullptr;
    TpAppIdCreateSession tp_appid_create_session = nullptr;
    TpAppIdPfini tp_appid_pfini = nullptr;
    TpAppIdTfini tp_appid_tfini = nullptr;

    bool load_callback(const char* path);
};

TPAppidProfilerFunctions get_tp_appid_profiler_functions();

#endif

network_inspectors/appid/appid_utils/

ip_funcs.h

Path = src/network_inspectors/appid/appid_utils/ip_funcs.h

#ifndef IP_FUNCS_H
#define IP_FUNCS_H

#define IPFUNCS_EXCEPT_IP                0x01
#define IPFUNCS_SECONDARY_IP             0x02
#define IPFUNCS_USER_IP                  0x04
#define IPFUNCS_HOSTS_IP                 0x08
#define IPFUNCS_APPLICATION              0x10
#define IPFUNCS_CHECKED                  0x80000000

#endif

sf_mlmp.h

Path = src/network_inspectors/appid/appid_utils/sf_mlmp.h

#ifndef SF_MULTI_PART_MPSE_H
#define SF_MULTI_PART_MPSE_H

#include <cstddef>
#include <cstdint>

struct tMlpPattern
{
    const uint8_t* pattern;
    size_t patternSize;
};

struct tMlmpPattern
{
    /*binary pattern */
    const uint8_t* pattern;

    /*binary pattern length in bytes */
    size_t patternSize;

    /**level of pattern. It should start from 0.*/
    uint32_t level;
};

struct tMlmpTree;

tMlmpTree* mlmpCreate();
int mlmpAddPattern(tMlmpTree*, const tMlmpPattern*, void* metaData);
int mlmpProcessPatterns(tMlmpTree*);
void mlmp_reload_patterns(tMlmpTree&);
void* mlmpMatchPatternUrl(tMlmpTree*, tMlmpPattern*);
void* mlmpMatchPatternGeneric(tMlmpTree*, tMlmpPattern*);
void mlmpDestroy(tMlmpTree*);
void mlmpDump(tMlmpTree*);

#endif

network_inspectors/appid/client_plugins/

client_app_bit.h

Path = src/network_inspectors/appid/client_plugins/client_app_bit.h

#ifndef CLIENT_APP_BIT_H
#define CLIENT_APP_BIT_H

#include "client_plugins/client_detector.h"

class BitClientDetector : public ClientDetector
{
public:
    BitClientDetector(ClientDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};

#endif

client_app_bit_tracker.h

Path = src/network_inspectors/appid/client_plugins/client_app_bit_tracker.h

#ifndef CLIENT_APP_BIT_TRACKER_H
#define CLIENT_APP_BIT_TRACKER_H

#include "client_plugins/client_detector.h"

class BitTrackerClientDetector : public ClientDetector
{
public:
    BitTrackerClientDetector(ClientDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};

#endif

client_app_msn.h

Path = src/network_inspectors/appid/client_plugins/client_app_msn.h

#ifndef CLIENT_APP_MSN_H
#define CLIENT_APP_MSN_H

#include "client_plugins/client_detector.h"

class MsnClientDetector : public ClientDetector
{
public:
    MsnClientDetector(ClientDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};
#endif

client_app_rtp.h

Path = src/network_inspectors/appid/client_plugins/client_app_rtp.h

#ifndef CLIENT_APP_RTP_H
#define CLIENT_APP_RTP_H

#include "client_plugins/client_detector.h"

class RtpClientDetector : public ClientDetector
{
public:
    RtpClientDetector(ClientDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};
#endif

client_app_timbuktu.h

Path = src/network_inspectors/appid/client_plugins/client_app_timbuktu.h

#ifndef CLIENT_APP_TIMBUKTU_H
#define CLIENT_APP_TIMBUKTU_H

#include "client_plugins/client_detector.h"

class TimbuktuClientDetector : public ClientDetector
{
public:
    TimbuktuClientDetector(ClientDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};
#endif

client_app_tns.h

Path = src/network_inspectors/appid/client_plugins/client_app_tns.h

#ifndef CLIENT_APP_TNS_H
#define CLIENT_APP_TNS_H

#include "client_plugins/client_detector.h"

class TnsClientDetector : public ClientDetector
{
public:
    TnsClientDetector(ClientDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};
#endif

client_app_vnc.h

Path = src/network_inspectors/appid/client_plugins/client_app_vnc.h

#ifndef CLIENT_APP_VNC_H
#define CLIENT_APP_VNC_H

#include "client_plugins/client_detector.h"

class VncClientDetector : public ClientDetector
{
public:
    VncClientDetector(ClientDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};
#endif

client_detector.h

Path = src/network_inspectors/appid/client_plugins/client_detector.h

#ifndef CLIENT_DETECTOR_H
#define CLIENT_DETECTOR_H

#include "appid_detector.h"
#include "client_discovery.h"

#define GENERIC_APP_OFFSET 2000000000

class ClientDetector : public AppIdDetector
{
public:
    ClientDetector();

    void register_appid(AppId, unsigned extractsInfo, OdpContext& odp_ctxt) override;
};
#endif

client_discovery.h

Path = src/network_inspectors/appid/client_plugins/client_discovery.h

#ifndef CLIENT_DISCOVERY_H
#define CLIENT_DISCOVERY_H

#include "flow/flow.h"

#include "appid_discovery.h"
#include "appid_types.h"

class ClientDetector;
class AppIdInspector;
class AppIdSession;

struct ClientAppMatch
{
    struct ClientAppMatch* next;
    unsigned count;
    const ClientDetector* detector = nullptr;
};

class ClientDiscovery : public AppIdDiscovery
{
public:
    void initialize(AppIdInspector&) override;
    void reload() override;

    void finalize_client_patterns();
    void reload_client_patterns();
    unsigned get_pattern_count();
    bool do_client_discovery(AppIdSession&, snort::Packet*,
        AppidSessionDirection direction, AppidChangeBits& change_bits);
    ClientDetector* get_client_detector(const std::string&) const;

private:
    void exec_client_detectors(AppIdSession&, snort::Packet*,
        AppidSessionDirection direction, AppidChangeBits& change_bits);
    ClientAppMatch* find_detector_candidates(const snort::Packet* pkt, const AppIdSession&);
    void create_detector_candidates_list(AppIdSession&, snort::Packet*);
    int get_detector_candidates_list(AppIdSession&, snort::Packet*, AppidSessionDirection direction);
};

#endif

eve_ca_patterns.h

Path = src/network_inspectors/appid/client_plugins/eve_ca_patterns.h

#ifndef EVE_CA_PATTERNS_H
#define EVE_CA_PATTERNS_H

#include <vector>

#include "search_engines/search_tool.h"
#include "application_ids.h"

struct EveCaPattern
{
    const AppId app_id;
    const std::string pattern;
    const uint8_t confidence;
    bool literal;

    EveCaPattern(AppId id, const std::string& name, uint8_t conf, bool literal = true) : app_id(id), pattern(name),
        confidence(conf), literal(literal) {}

    ~EveCaPattern() {}
};

typedef std::vector<EveCaPattern*> EveCaPatternList;

class EveCaPatternMatchers
{
public:
    ~EveCaPatternMatchers();
    AppId match_eve_ca_pattern(const std::string&, uint8_t);
    void add_eve_ca_pattern(AppId, const std::string&, uint8_t, const std::string&, bool literal = true);
    void finalize_patterns();
    void reload_patterns();
    unsigned get_pattern_count();

    const EveCaPatternList& get_eve_ca_load_list() const { return eve_ca_load_list; }

private:
    snort::SearchTool eve_ca_pattern_matcher = snort::SearchTool();
    EveCaPatternList eve_ca_load_list;
};

#endif

network_inspectors/appid/client_plugins/test/

network_inspectors/appid/detector_plugins/

cip_patterns.h

Path = src/network_inspectors/appid/detector_plugins/cip_patterns.h

#ifndef CIP_PATTERN_H
#define CIP_PATTERN_H

#include "pub_sub/cip_events.h"
#include "application_ids.h"

struct EnipCommandData
{
    AppId app_id;
    uint16_t command_id;
} ;

struct EnipCommandList
{
    EnipCommandData data;
    struct EnipCommandList* next;
} ;

struct CipPathData
{
    AppId app_id;
    uint32_t class_id;
    uint8_t service_id;
} ;

struct CipPathList
{
    CipPathData data;
    struct CipPathList* next;
} ;

struct CipSetAttributeData
{
    AppId app_id;
    uint32_t class_id;
    bool is_class_instance;
    uint32_t attribute_id;
} ;

struct CipSetAttributeList
{
    CipSetAttributeData data;
    struct CipSetAttributeList* next;
} ;

struct CipConnectionClassData
{
    AppId app_id;
    uint32_t class_id;
} ;

struct CipConnectionClassList
{
    CipConnectionClassData data;
    struct CipConnectionClassList* next;
} ;

struct CipServiceData
{
    AppId app_id;
    uint8_t service_id;
} ;

struct CipServiceList
{
    CipServiceData data;
    struct CipServiceList* next;
} ;

class CipPatternMatchers
{
public:
    ~CipPatternMatchers();
    void cip_add_enip_command(AppId app_id, uint16_t command_id);
    void cip_add_path(AppId app_id, uint32_t class_id, uint8_t service_id);
    void cip_add_set_attribute(AppId app_id, uint32_t class_id, bool is_class_instance, uint32_t attribute_id);
    void cip_add_extended_symbol_service(AppId app_id, uint8_t service_id);
    void cip_add_service(AppId app_id, uint8_t service_id);
    void cip_add_connection_class(AppId app_id, uint32_t class_id);
    AppId get_cip_payload_id(const CipEventData* event_data);

private:
    EnipCommandList* enip_command_list = nullptr;
    CipPathList* path_list = nullptr;
    CipSetAttributeList* set_attribute_list = nullptr;
    CipConnectionClassList* connection_list = nullptr;
    CipServiceList* symbol_list = nullptr;
    CipServiceList* service_list = nullptr;
} ;

#endif  // CIP_PATTERN_H

detector_dns.h

Path = src/network_inspectors/appid/detector_plugins/detector_dns.h

#ifndef DETECTOR_DNS_H
#define DETECTOR_DNS_H

#include "service_plugins/service_detector.h"

char* dns_parse_host(const uint8_t* host, uint8_t host_len);

struct DNSHeader;

class DnsValidator
{
protected:
    APPID_STATUS_CODE add_dns_query_info(AppIdSession&, uint16_t, const uint8_t*,
        uint8_t, uint16_t, uint16_t, AppidChangeBits&);
    APPID_STATUS_CODE add_dns_response_info(AppIdSession&, uint16_t, const uint8_t*,
        uint8_t, uint16_t, uint8_t, uint32_t, AppidChangeBits&);
    APPID_STATUS_CODE dns_validate_label(const uint8_t*, uint16_t&, uint16_t, uint8_t&, bool&);
    int dns_validate_query(const uint8_t*, uint16_t*, uint16_t, uint16_t, bool, AppIdSession&, AppidChangeBits&);
    int dns_validate_answer(const uint8_t*, uint16_t*, uint16_t,
        uint16_t, uint8_t, bool, AppIdSession&, AppidChangeBits&);
    int dns_validate_header(AppidSessionDirection, const DNSHeader*, bool, const AppIdSession&);
    int validate_packet(const uint8_t*, uint16_t, const int, bool, AppIdSession&, AppidChangeBits&);
};

class DnsTcpServiceDetector : public ServiceDetector, public DnsValidator
{
public:
    DnsTcpServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};

class DnsUdpServiceDetector : public ServiceDetector, public DnsValidator
{
public:
    DnsUdpServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};
#endif

detector_imap.h

Path = src/network_inspectors/appid/detector_plugins/detector_imap.h

#ifndef DETECTOR_IMAP_H
#define DETECTOR_IMAP_H

#include "client_plugins/client_detector.h"
#include "service_plugins/service_detector.h"

class AppIdSession;
struct ImapDetectorData;

class ImapClientDetector : public ClientDetector
{
public:
    ImapClientDetector(ClientDiscovery*);
    ~ImapClientDetector() override;

    void do_custom_init() override;
    void do_custom_reload() override;
    int validate(AppIdDiscoveryArgs&) override;
    ImapDetectorData* get_common_data(AppIdSession&);

private:
    snort::SearchTool* cmd_matcher = nullptr;
    unsigned longest_pattern = 0;
};

class ImapServiceDetector : public ServiceDetector
{
public:
    ImapServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
    void set_client_detector(ImapClientDetector* c)
    {
        imap_client_detector = c;
    }

private:
    ImapClientDetector* imap_client_detector = nullptr;
};

#endif

detector_kerberos.h

Path = src/network_inspectors/appid/detector_plugins/detector_kerberos.h

#ifndef DETECTOR_KERBEROS_H
#define DETECTOR_KERBEROS_H

#include "protocols/packet.h"
#include "client_plugins/client_detector.h"
#include "service_plugins/service_detector.h"

namespace snort
{
struct Packet;
}

struct KRBState;
struct KerberosDetectorData;
class KerberosServiceDetector;

class KerberosClientDetector : public ClientDetector
{
public:
    KerberosClientDetector(ClientDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
    KerberosDetectorData* get_common_data(AppIdSession&);
    void set_service_detector(KerberosServiceDetector* s)
    {
        krb_service_detector = s;
    }

    bool failed_login = false;

private:
    int krb_walk_client_packet(KRBState*, const uint8_t*, const uint8_t*,
        AppIdSession&, AppidChangeBits&);
    KerberosServiceDetector* krb_service_detector = nullptr;
};

class KerberosServiceDetector : public ServiceDetector
{
public:
    KerberosServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
    int krb_walk_server_packet(KRBState*, const uint8_t*, const uint8_t*, AppIdSession&, snort::Packet*,
        const AppidSessionDirection, const char*, AppidChangeBits&);
    void set_client_detector(KerberosClientDetector* c)
    {
        krb_client_detector = c;
    }

private:
    KerberosClientDetector* krb_client_detector = nullptr;
};

#endif

detector_pattern.h

Path = src/network_inspectors/appid/detector_plugins/detector_pattern.h

#ifndef DETECTOR_PATTERN_H
#define DETECTOR_PATTERN_H

#include "appid_api.h"
#include "client_plugins/client_detector.h"
#include "service_plugins/service_detector.h"

namespace snort
{
class SearchTool;
}

class AppIdInspector;

struct PortPatternNode
{
    AppId appId;
    IpProtocol protocol;
    unsigned short port;
    unsigned char* pattern;
    unsigned length;
    int32_t offset;
    char* detector_name;
    PortPatternNode* next;
};

struct PatternService;
struct Pattern
{
    Pattern* next;
    unsigned length;
    int offset;
    uint8_t* data;
    PatternService* ps;
};

struct PortNode
{
    PortNode* next;
    uint16_t port;
};

/**list for pattern services. Each pattern service is unique for a given uuid. */
struct PatternService
{
    PatternService* next;
    AppId id;
    Pattern* pattern;
    PortNode* port;
    IpProtocol proto;
    unsigned count;
    unsigned longest;
};

class PatternClientDetector : public ClientDetector
{
public:
    PatternClientDetector(ClientDiscovery*);
    ~PatternClientDetector() override;

    void insert_client_port_pattern(PortPatternNode*);
    void finalize_client_port_patterns(AppIdInspector&);
    void reload_client_port_patterns();
    unsigned get_pattern_count();

    int validate(AppIdDiscoveryArgs&) override;

private:
    void create_client_pattern_trees();
    void register_client_patterns();

    PortPatternNode* lua_injected_patterns = nullptr;
    PatternService* service_port_pattern = nullptr;
    snort::SearchTool* tcp_pattern_matcher = nullptr;
    snort::SearchTool* udp_pattern_matcher = nullptr;
    unsigned pattern_count = 0;
};

class PatternServiceDetector : public ServiceDetector
{
public:
    PatternServiceDetector(ServiceDiscovery*);
    ~PatternServiceDetector() override;

    void insert_service_port_pattern(PortPatternNode*);
    void finalize_service_port_patterns(AppIdInspector&);
    void reload_service_port_patterns();
    unsigned get_pattern_count();

    int validate(AppIdDiscoveryArgs&) override;

private:
    void create_service_pattern_trees();
    void register_service_patterns();
    void install_ports(PatternService*);

    PortPatternNode* lua_injected_patterns = nullptr;
    PatternService* service_port_pattern = nullptr;
    snort::SearchTool* tcp_pattern_matcher = nullptr;
    snort::SearchTool* udp_pattern_matcher = nullptr;
    snort::SearchTool* tcp_port_pattern_tree[65536] = { };
    snort::SearchTool* udp_port_pattern_tree[65536] = { };
    unsigned pattern_count = 0;
};

#endif

detector_pop3.h

Path = src/network_inspectors/appid/detector_plugins/detector_pop3.h

#ifndef DETECTOR_POP3_H
#define DETECTOR_POP3_H

#include "client_plugins/client_detector.h"
#include "service_plugins/service_detector.h"

struct POP3DetectorData;
class Pop3ServiceDetector;

class Pop3ClientDetector : public ClientDetector
{
public:
    Pop3ClientDetector(ClientDiscovery*);
    ~Pop3ClientDetector() override;

    void do_custom_init() override;
    void do_custom_reload() override;
    int validate(AppIdDiscoveryArgs&) override;
    POP3DetectorData* get_common_data(AppIdSession&);
    void set_service_detector(Pop3ServiceDetector* s)
    {
        pop3_service_detector = s;
    }

private:
    snort::SearchTool* cmd_matcher = nullptr;
    unsigned longest_pattern = 0;
    Pop3ServiceDetector* pop3_service_detector = nullptr;
};

class Pop3ServiceDetector : public ServiceDetector
{
public:
    Pop3ServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
    void set_client_detector(Pop3ClientDetector* c)
    {
        pop3_client_detector = c;
    }
    int pop3_server_validate(POP3DetectorData*, const uint8_t*, uint16_t, AppIdSession&, int, AppidChangeBits&);

private:
    Pop3ClientDetector* pop3_client_detector = nullptr;
};

#endif

detector_sip.h

Path = src/network_inspectors/appid/detector_plugins/detector_sip.h

#ifndef DETECTOR_SIP_H
#define DETECTOR_SIP_H

#include "client_plugins/client_detector.h"
#include "service_plugins/service_detector.h"
#include "framework/data_bus.h"
#include "pub_sub/sip_events.h"

#include "appid_module.h"

namespace snort
{
class Flow;
}

class AppIdInspector;
class SipEventHandler;

class SipUdpClientDetector : public ClientDetector
{
public:
    SipUdpClientDetector(ClientDiscovery*);
    ~SipUdpClientDetector() override = default;

    int validate(AppIdDiscoveryArgs&) override;
};

class SipTcpClientDetector : public ClientDetector
{
public:
    SipTcpClientDetector(ClientDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};

class SipServiceDetector : public ServiceDetector
{
public:
    SipServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
    void addFutureRtpFlows(SipEvent&, AppIdSession&);

private:
    void createRtpFlow(AppIdSession&, const snort::Packet*, const snort::SfIp* cliIp,
        uint16_t cliPort, const snort::SfIp* srvIp, uint16_t srvPort, IpProtocol);
};

class SipEventHandler : public snort::DataHandler
{
public:
    SipEventHandler(AppIdInspector& inspector) :
        DataHandler(MOD_NAME), inspector(inspector)
    { }

    void handle(snort::DataEvent&, snort::Flow*) override;

private:
    void client_handler(SipEvent&, AppIdSession&, AppidChangeBits&);
    void service_handler(SipEvent&, AppIdSession&, AppidChangeBits&);

    AppIdInspector& inspector;
};
#endif

detector_smtp.h

Path = src/network_inspectors/appid/detector_plugins/detector_smtp.h

#ifndef DETECTOR_SMTP_H
#define DETECTOR_SMTP_H

#include "client_plugins/client_detector.h"
#include "service_plugins/service_detector.h"
#include "framework/counts.h"

struct ClientSMTPData;
struct SMTPDetectorData;

class SmtpClientDetector : public ClientDetector
{
public:
    SmtpClientDetector(ClientDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
    SMTPDetectorData* get_common_data(AppIdSession&);

private:
    int extract_version_and_add_client_app(AppId, const int prefix_len,
        const uint8_t* product, const uint8_t* product_end, ClientSMTPData* const,
        AppIdSession&, AppId, AppidChangeBits&);
    int identify_client_version(ClientSMTPData* const, const uint8_t* product,
        const uint8_t* data_end, AppIdSession&, snort::Packet*, AppidChangeBits&);
};

class SmtpServiceDetector : public ServiceDetector
{
public:
    SmtpServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
    void set_client_detector(SmtpClientDetector* c)
    {
        smtp_client_detector = c;
    }

private:
    SmtpClientDetector* smtp_client_detector = nullptr;
};

#endif

dns_patterns.h

Path = src/network_inspectors/appid/detector_plugins/dns_patterns.h

#ifndef DNS_PATTERNS_H
#define DNS_PATTERNS_H

#include "search_engines/search_tool.h"
#include "application_ids.h"

struct DnsHostPattern
{
    uint8_t type;
    AppId app_id;
    uint8_t* pattern;
    int pattern_size;
};

struct DnsHostPatternList
{
    DnsHostPattern* dpattern;
    DnsHostPatternList* next;
};

struct MatchedDnsPatterns
{
    DnsHostPattern* mpattern;
    MatchedDnsPatterns* next;
};

class DnsPatternMatchers
{
public:
    ~DnsPatternMatchers();
    void add_host_pattern(uint8_t*, size_t, uint8_t, AppId);
    void finalize_patterns();
    void reload_patterns();
    unsigned get_pattern_count();
    int scan_hostname(const uint8_t*, size_t, AppId&, AppId&);

private:
    DnsHostPatternList* dns_host_pattern_list = nullptr;
    snort::SearchTool dns_host_matcher = snort::SearchTool();
    unsigned pattern_count = 0;
};

#endif

http_url_patterns.h

Path = src/network_inspectors/appid/detector_plugins/http_url_patterns.h

#ifndef HTTP_URL_PATTERNS_H
#define HTTP_URL_PATTERNS_H

#include <list>
#include <vector>

#include "flow/flow.h"
#include "search_engines/search_tool.h"
#include "trace/trace.h"
#include "utils/util.h"

#include "appid_debug.h"
#include "appid_types.h"
#include "appid_utils/sf_mlmp.h"
#include "application_ids.h"

namespace snort
{
struct Packet;
}
class AppIdHttpSession;
class AppIdContext;
struct AppIdServiceSubtype;
class OdpContext;

enum httpPatternType
{
    HTTP_PAYLOAD    = 1,
    HTTP_USER_AGENT = 2,
    HTTP_URL        = 3
};

struct HTTPHeaderIndices
{
    int start;
    int end;
};

struct UrlUserData
{
    uint32_t service_id;
    uint32_t client_id;
    uint32_t payload_id;
    AppId appId;
    tMlpPattern query;
};

struct DetectorAppUrlPattern
{
    struct
    {
        tMlpPattern host;
        tMlpPattern path;
        tMlpPattern scheme;
    } patterns;

    UrlUserData userData;
};

// These values are used in Lua code as raw numbers. Do NOT reassign new values.
enum DHPSequence
{
    SINGLE = 0,
    SKYPE_URL = 1,
    SKYPE_VERSION = 2,
    BT_ANNOUNCE = 3,
    BT_OTHER = 4,
    USER_AGENT_HEADER = 5
};

struct FieldPattern
{
    const uint8_t* data;
    HttpFieldIds patternType;
    unsigned length;
};

struct DetectorHTTPPattern
{
    bool init(const uint8_t* pat, unsigned len, DHPSequence seq, AppId service, AppId client, AppId payload, AppId app)
    {
        if( !pat )
        {
            appid_log(nullptr, TRACE_ERROR_LEVEL, "HTTP pattern string is null.");
            return false;
        }

        if (seq < SINGLE || seq > USER_AGENT_HEADER)
        {
            appid_log(nullptr, TRACE_ERROR_LEVEL, "Invalid HTTP DHP Sequence.");
            return false;
        }

        pattern_size = len;
        pattern = (const uint8_t*)snort::snort_strdup((const char*)pat);
        sequence = seq;
        service_id = service;
        client_id = client;
        payload_id = payload;
        app_id = app;

        return true;
    }

    DHPSequence sequence = SINGLE;
    AppId service_id = 0;
    AppId client_id = 0;
    AppId payload_id = 0;
    AppId app_id = 0;
    unsigned pattern_size = 0;
    const uint8_t* pattern = nullptr;
};
typedef std::vector<DetectorHTTPPattern> DetectorHTTPPatterns;

// CHP (Complex HTTP Pattern) uses more than one HTTP pattern
// to do appid detection and/or perform other actions
#define CHP_APPID_BITS_FOR_INSTANCE  7
#define CHP_APPID_INSTANCE_MAX (1 << CHP_APPID_BITS_FOR_INSTANCE)
#define CHP_APPIDINSTANCE_TO_ID(_appIdInstance) \
    ((_appIdInstance) >> CHP_APPID_BITS_FOR_INSTANCE)
#define CHP_APPIDINSTANCE_TO_INSTANCE(_appIdInstance) \
    ((_appIdInstance) & (CHP_APPID_INSTANCE_MAX-1))
/*
  NOTE: The following structures have a field called appIdInstance.
    The low-order CHP_APPID_BITS_FOR_INSTANCE bits of appIdInstance field are used
    for the instance value while the remaining bits are used for the appId, shifted left
    that same number of bits. The legacy value for older apis is generated with the
    macro below.
*/
#define CHP_APPID_SINGLE_INSTANCE(_appId) \
    (((_appId) << CHP_APPID_BITS_FOR_INSTANCE) + (CHP_APPID_INSTANCE_MAX-1))

// These values are used in Lua code as raw numbers. Do NOT reassign new values.
enum ActionType
{
    NO_ACTION,                              //0
    COLLECT_VERSION,                        //1
    EXTRACT_USER,                           //2
    REWRITE_FIELD,                          //3
    INSERT_FIELD,                           //4
    ALTERNATE_APPID,                        //5
    FUTURE_APPID_SESSION_SIP,               //6
    FUTURE_APPID_SESSION_DIP,               //7
    FUTURE_APPID_SESSION_SPORT,             //8
    FUTURE_APPID_SESSION_DPORT,             //9
    FUTURE_APPID_SESSION_PROTOCOL,          //10
    FUTURE_APPID_SESSION_CREATE,            //11
    HOLD_FLOW,                              //12
    GET_OFFSETS_FROM_REBUILT,               //13
    SEARCH_UNSUPPORTED,                     //14
    DEFER_TO_SIMPLE_DETECT,                 //15
    MAX_ACTION_TYPE = DEFER_TO_SIMPLE_DETECT,
};

struct CHPApp
{
    AppId appIdInstance; // * see note above
    unsigned app_type_flags;
    int num_matches;
    int num_scans;
    int key_pattern_count;
    int key_pattern_length_sum;
    int ptype_scan_counts[NUM_HTTP_FIELDS];
    int ptype_req_counts[NUM_HTTP_FIELDS];
    int ptype_rewrite_insert_used[NUM_HTTP_FIELDS]; // boolean
};

struct CHPAction
{
    AppId appIdInstance; // * see note above
    unsigned precedence; // order of creation
    int key_pattern;
    HttpFieldIds ptype;
    int psize;
    char* pattern;
    ActionType action;
    char* action_data;
    CHPApp* chpapp;
};

struct CHPListElement
{
    CHPAction chp_action;
    CHPListElement* next;
};

struct MatchedCHPAction
{
    CHPAction* mpattern;
    int start_match_pos;
};

// This is an array element for the dynamically growing tally below
struct CHPMatchCandidate
{
    CHPApp* chpapp;
    int key_pattern_length_sum;
    int key_pattern_countdown;
};

typedef std::vector<CHPMatchCandidate> CHPMatchTally;

class ChpMatchDescriptor
{
public:
    void sort_chp_matches()
    {
        chp_matches[cur_ptype].sort(ChpMatchDescriptor::comp_chp_actions);
    }

    HttpFieldIds cur_ptype;
    const char* buffer[NUM_HTTP_FIELDS] = { };
    uint16_t length[NUM_HTTP_FIELDS] = { };
    std::list<MatchedCHPAction> chp_matches[NUM_HTTP_FIELDS];
    CHPMatchTally match_tally;

private:
    static bool comp_chp_actions( const MatchedCHPAction& lhs, const MatchedCHPAction& rhs)
    {
        if ( ( lhs.mpattern->appIdInstance < rhs.mpattern->appIdInstance ) ||
             ( lhs.mpattern->appIdInstance == rhs.mpattern->appIdInstance
                  && lhs.mpattern->precedence < rhs.mpattern->precedence ) )
            return true;
        else
            return false;
    }
};

class HostUrlDetectorPattern
{
public:
    HostUrlDetectorPattern(const uint8_t* host_pattern, unsigned length)
    {
        host.pattern = (const uint8_t*)snort::snort_strdup((const char*)host_pattern);
        host.patternSize = length;
    }

    ~HostUrlDetectorPattern()
    {
        snort_free((void*)host.pattern);
        if (path.pattern)
            snort_free((void*)path.pattern);
        if (query.pattern)
            snort_free((void*)query.pattern);
    }

    tMlpPattern host = { nullptr, 0 };
    tMlpPattern path = { nullptr, 0 };
    tMlpPattern query = { nullptr, 0 };
    uint32_t payload_id = APP_ID_NONE;
    uint32_t service_id = APP_ID_NONE;
    uint32_t client_id = APP_ID_NONE;
    AppId appId = APP_ID_NONE;
    DHPSequence seq = SINGLE;
};

class HttpPatternMatchers
{
public:
    HttpPatternMatchers()
        : url_matcher(), client_agent_matcher(), via_matcher(), content_type_matcher()
    { }
    ~HttpPatternMatchers();

    int finalize_patterns();
    void reload_patterns();
    unsigned get_pattern_count();
    void insert_chp_pattern(CHPListElement*);
    void insert_http_pattern(enum httpPatternType, DetectorHTTPPattern&);
    void remove_http_patterns_for_id(AppId);
    void insert_content_type_pattern(DetectorHTTPPattern&);
    void insert_url_pattern(DetectorAppUrlPattern*);
    void insert_rtmp_url_pattern(DetectorAppUrlPattern*);
    void insert_app_url_pattern(DetectorAppUrlPattern*);
    int process_chp_list(CHPListElement*);
    int process_host_patterns(DetectorHTTPPatterns&);
    int process_mlmp_patterns();
    void process_http_field_patterns(FieldPattern*, size_t);

    void scan_key_chp(ChpMatchDescriptor&);
    AppId scan_chp(ChpMatchDescriptor&, char**, char**, int*, AppIdHttpSession*,
           const OdpContext&);
    AppId scan_header_x_working_with(const char*, uint32_t, char**);
    int get_appid_by_pattern(const char*, unsigned, char**);
    bool get_appid_from_url(const char*, const char*, char**, const char*, AppId*, AppId*,
        AppId*, AppId*, bool, OdpContext&);
    AppId get_appid_by_content_type(const char*, int);
    void get_server_vendor_version(const char*, int, char**, char**, AppIdServiceSubtype**);
    void identify_user_agent(const char*, int, AppId&, AppId&, char**);
    uint32_t parse_multiple_http_patterns(const char* pattern, tMlmpPattern*,
        uint32_t numPartLimit, int level);

private:
    DetectorHTTPPatterns client_agent_patterns;
    DetectorHTTPPatterns content_type_patterns;
    DetectorHTTPPatterns host_payload_patterns;
    DetectorHTTPPatterns url_patterns;
    std::vector<DetectorAppUrlPattern*> app_url_patterns;
    std::vector<DetectorAppUrlPattern*> rtmp_url_patterns;
    std::vector<HostUrlDetectorPattern*> host_url_patterns;
    CHPListElement* chpList = nullptr;

    snort::SearchTool url_matcher;
    snort::SearchTool client_agent_matcher;
    snort::SearchTool via_matcher;
    snort::SearchTool content_type_matcher;
    snort::SearchTool field_matcher;
    snort::SearchTool chp_matchers[NUM_HTTP_FIELDS];
    tMlmpTree* host_url_matcher = nullptr;
    tMlmpTree* rtmp_host_url_matcher = nullptr;
    unsigned chp_pattern_count = 0;

    void free_chp_app_elements();
    int add_mlmp_pattern(tMlmpTree* matcher, DetectorHTTPPattern& pattern );
    int add_mlmp_pattern(tMlmpTree* matcher, DetectorAppUrlPattern& pattern);

};

#endif

sip_patterns.h

Path = src/network_inspectors/appid/detector_plugins/sip_patterns.h

#ifndef SIP_PATTERNS_H
#define SIP_PATTERNS_H

#include "appid_utils/sf_mlmp.h"
#include "application_ids.h"

class OdpContext;

struct SipUaUserData
{
    AppId client_id;
    char* client_version;
};

struct DetectorAppSipPattern
{
    tMlpPattern pattern;
    SipUaUserData user_data;
    DetectorAppSipPattern* next;
};

class SipPatternMatchers
{
public:
    ~SipPatternMatchers();
    int add_ua_pattern(AppId, const char*, const char*);
    int add_server_pattern(AppId, const char*, const char*);
    int get_client_from_ua(const char*, uint32_t, AppId&, char*&);
    int get_client_from_server(const char*, uint32_t, AppId&, char*&);
    void finalize_patterns(OdpContext&);
    void reload_patterns();
    unsigned get_pattern_count();

private:
    static const int PATTERN_PART_MAX = 10;
    tMlmpPattern patterns[PATTERN_PART_MAX] = { { nullptr, 0, 0 } };
    tMlmpTree* sip_ua_matcher = nullptr;
    DetectorAppSipPattern* sip_ua_list = nullptr;
    tMlmpTree* sip_server_matcher = nullptr;
    DetectorAppSipPattern* sip_server_list = nullptr;
    unsigned pattern_count = 0;
};

#endif

ssh_patterns.h

Path = src/network_inspectors/appid/detector_plugins/ssh_patterns.h

#ifndef SSH_PATTERNS_H
#define SSH_PATTERNS_H

/*
 * SshPatternMatchers is a wrapper around an unordered_map
 * which maps strings to AppIds. SSH Client Patterns
 * are registered through a lua API, and these mappings
 * are used by AppId to identify clients.
 * An instance of the class is held by OdpContext.
 */

#include <string>
#include <unordered_map>

#include "application_ids.h"

typedef std::unordered_map<std::string, AppId> SshPatternTable;

class SshPatternMatchers
{
public:
    void add_ssh_pattern(const std::string& pattern, AppId id);
    bool has_pattern(const std::string& pattern) const;
    bool empty() const;
    AppId get_appid(const std::string& pattern) const;
    unsigned get_pattern_count();
private:
    SshPatternTable ssh_patterns;
};

#endif

ssl_patterns.h

Path = src/network_inspectors/appid/detector_plugins/ssl_patterns.h

#ifndef SSL_PATTERNS_H
#define SSL_PATTERNS_H

#include <cstring>
#include <unordered_set>
#include "search_engines/search_tool.h"
#include "application_ids.h"

struct SslPattern
{
    uint8_t type;
    AppId app_id;
    uint8_t* pattern;
    int pattern_size;
    bool is_cname;
    bool is_literal; // is not regex pattern

    bool operator==(const SslPattern& v) const
    {
        return this->type == v.type and pattern_size == v.pattern_size
            and (memcmp(pattern, v.pattern, (size_t)pattern_size) == 0);
    }
};

struct SslCacheKeyHasher
{
    size_t operator()(const SslPattern& key) const
    {
        return std::hash<std::string>{}(std::string((char*)key.pattern, key.pattern_size));
    }
};

typedef std::unordered_set<SslPattern, SslCacheKeyHasher> CnameCache;

struct MatchedSslPatterns
{
    SslPattern* mpattern;
    int match_start_pos;
    struct MatchedSslPatterns* next;
};

struct SslPatternList
{
    SslPattern* dpattern;
    SslPatternList* next;
};

class SslPatternMatchers
{
public:
    ~SslPatternMatchers();
    void add_cert_pattern(uint8_t*, size_t, uint8_t, AppId, bool, bool = true);
    void finalize_patterns();
    void reload_patterns();
    unsigned get_pattern_count();
    bool scan_hostname(const uint8_t*, size_t, AppId&, AppId&);
    bool scan_cname(const uint8_t*, size_t, AppId&, AppId&);

private:
    SslPatternList* cert_pattern_list = nullptr;
    CnameCache cert_pattern_set;
    snort::SearchTool ssl_host_matcher = snort::SearchTool();
    unsigned pattern_count = 0;
};

#endif

network_inspectors/appid/detector_plugins/test/

detector_plugins_mock.h

Path = src/network_inspectors/appid/detector_plugins/test/detector_plugins_mock.h

#ifndef DETECTOR_PLUGINS_MOCK_H
#define DETECTOR_PLUGINS_MOCK_H

#include "log/messages.h"
#include "utils/stats.h"

#include "appid_detector.h"
#include "appid_module.h"
#include "appid_peg_counts.h"

namespace snort
{
// Stubs for messages
// LCOV_EXCL_START
void ParseWarning(WarningGroup, const char*, ...) { }
// LCOV_EXCL_STOP

// Stubs for appid sessions
FlowData::FlowData(unsigned, Inspector*) : next(nullptr), prev(nullptr), handler(nullptr), id(0)
{ }
FlowData::~FlowData() = default;

// Stubs for packet
Packet::Packet(bool) { }
Packet::~Packet() = default;

Inspector::Inspector() = default;
Inspector::~Inspector() = default;
// LCOV_EXCL_START
bool Inspector::likes(Packet*) { return true; }
bool Inspector::get_buf(const char*, Packet*, InspectionBuffer&) { return true; }
class StreamSplitter* Inspector::get_splitter(bool) { return nullptr; }
// LCOV_EXCL_STOP

// Stubs for search_tool.cc
SearchTool::~SearchTool() = default;
// LCOV_EXCL_START
void SearchTool::add(const char*, unsigned, int, bool, bool) { }
void SearchTool::add(const char*, unsigned, void*, bool, bool) { }
void SearchTool::add(const uint8_t*, unsigned, int, bool, bool) { }
void SearchTool::add(const uint8_t*, unsigned, void*, bool, bool) { }
// LCOV_EXCL_STOP
void SearchTool::prep() { }

// Stubs for util.cc
char* snort_strndup(const char* src, size_t dst_size)
{
    char* dup = (char*)snort_calloc(dst_size + 1);
    if ( SnortStrncpy(dup, src, dst_size + 1) == SNORT_STRNCPY_ERROR )
    {
        snort_free(dup);
        return nullptr;
    }
    return dup;
}

char* snort_strdup(const char* str)
{
    assert(str);
    size_t n = strlen(str) + 1;
    char* p = (char*)snort_alloc(n);
    memcpy(p, str, n);
    return p;
}
}
// LCOV_EXCL_START
DiscoveryFilter::~DiscoveryFilter(){}
void show_stats(PegCount*, const PegInfo*, unsigned, const char*) { }
void show_stats(PegCount*, const PegInfo*, const IndexVec&, const char*, FILE*) { }
// LCOV_EXCL_STOP

#ifndef SIP_UNIT_TEST
class AppIdInspector : public snort::Inspector
{
public:
    AppIdInspector(AppIdModule&) { }
    ~AppIdInspector() override = default;
    bool configure(snort::SnortConfig*) override;
// LCOV_EXCL_START
    void eval(Packet*) override { }
    void show(const SnortConfig*) const override { }
    void tinit() override { }
    void tterm() override { }
// LCOV_EXCL_STOP
private:
    AppIdContext* ctxt = nullptr;
};
#endif

// Stubs for modules, config
AppIdConfig::~AppIdConfig() = default;
AppIdModule::AppIdModule()
    : Module("a", "b") { }

// LCOV_EXCL_START
bool AppIdModule::begin(const char*, int, snort::SnortConfig*)
{
    return false;
}

bool AppIdModule::set(const char*, snort::Value&, snort::SnortConfig*)
{
    return false;
}

bool AppIdModule::end(const char*, int, snort::SnortConfig*)
{
    return false;
}

const Command* AppIdModule::get_commands() const
{
    return nullptr;
}

const PegInfo* AppIdModule::get_pegs() const
{
    return nullptr;
}

PegCount* AppIdModule::get_counts() const
{
    return nullptr;
}

snort::ProfileStats* AppIdModule::get_profile(
        unsigned, const char*&, const char*&) const
{
    return nullptr;
}

void AppIdModule::set_trace(const Trace*) const { }
const TraceOption* AppIdModule::get_trace_options() const { return nullptr; }
// LCOV_EXCL_STOP

// Stubs for inspectors
unsigned AppIdSession::inspector_id = 0;
AppIdConfig stub_config;
AppIdContext stub_ctxt(stub_config);
OdpContext stub_odp_ctxt(stub_config, nullptr);
AppIdSession::AppIdSession(IpProtocol, const SfIp* ip, uint16_t, AppIdInspector& inspector,
    OdpContext& odpctxt, uint32_t, uint32_t) : snort::FlowData(inspector_id, (snort::Inspector*)&inspector),
        config(stub_config), api(*(new AppIdSessionApi(this, *ip))), odp_ctxt(odpctxt)
{
    this->set_session_flags(APPID_SESSION_DISCOVER_APP);
}
AppIdSession::~AppIdSession() { delete &api; }
AppIdHttpSession::AppIdHttpSession(AppIdSession& asd, int64_t http2_stream_id)
  : asd(asd), httpx_stream_id(http2_stream_id)
{
    for ( int i = 0; i < NUM_METADATA_FIELDS; i++)
        meta_data[i] = nullptr;
}

AppIdHttpSession::~AppIdHttpSession()
{
    for ( int i = 0; i < NUM_METADATA_FIELDS; i++)
    {
        if ( meta_data[i] )
            delete meta_data[i];
    }
}

// Stubs for AppIdPegCounts
// LCOV_EXCL_START
void AppIdPegCounts::inc_service_count(AppId) { }
void AppIdPegCounts::inc_client_count(AppId) { }
void AppIdPegCounts::inc_user_count(AppId) { }
void AppIdPegCounts::inc_payload_count(AppId) { }

THREAD_LOCAL AppIdStats appid_stats;
void AppIdModule::sum_stats(bool) { }
void AppIdModule::show_dynamic_stats() { }

// Stubs for appid_session.cc
static bool test_service_strstr_enabled = false;
const uint8_t* service_strstr(const uint8_t* p, unsigned,
    const uint8_t*, unsigned)
{
    if (test_service_strstr_enabled)
        return p;
    return nullptr;
}

// Stubs for app_info_table.cc
AppInfoTableEntry* AppInfoManager::get_app_info_entry(int)
{
    return nullptr;
}

AppInfoTableEntry* AppInfoManager::get_app_info_entry(AppId, const AppInfoTable&)
{
    return nullptr;
}

bool AppIdReloadTuner::tinit() { return false; }

bool AppIdReloadTuner::tune_resources(unsigned int)
{
    return true;
}
void ApplicationDescriptor::set_id(AppId){}
void ServiceAppDescriptor::set_id(AppId, OdpContext&){}
void ServiceDiscovery::initialize(AppIdInspector&) {}
void ServiceDiscovery::reload() {}

int ServiceDiscovery::add_service_port(AppIdDetector*, const ServiceDetectorPort&)
{ return 0; }
// LCOV_EXCL_STOP

OdpContext::OdpContext(const AppIdConfig&, snort::SnortConfig*)
{ }

THREAD_LOCAL OdpContext* pkt_thread_odp_ctxt = nullptr;
#endif

network_inspectors/appid/service_plugins/

alpn_patterns.h

Path = src/network_inspectors/appid/service_plugins/alpn_patterns.h

#ifndef ALPN_PATTERNS_H
#define ALPN_PATTERNS_H

#include <vector>

#include "search_engines/search_tool.h"
#include "application_ids.h"

struct AlpnPattern
{
    const AppId app_id;
    const std::string pattern;

    AlpnPattern(AppId id, const std::string& name) : app_id(id), pattern(name){}

    ~AlpnPattern() {}
};

typedef std::vector<AlpnPattern*> AlpnPatternList;

class AlpnPatternMatchers
{
public:
    ~AlpnPatternMatchers();
    AppId match_alpn_pattern(const std::string&);
    void add_alpn_pattern(AppId, const std::string&, const std::string&);
    void finalize_patterns();
    void reload_patterns();
    unsigned get_pattern_count();

    const AlpnPatternList& get_alpn_load_list() const { return alpn_load_list; }

private:
    snort::SearchTool alpn_pattern_matcher = snort::SearchTool();
    AlpnPatternList alpn_load_list;
};

#endif

dcerpc.h

Path = src/network_inspectors/appid/service_plugins/dcerpc.h

#ifndef DCERPC_H
#define DCERPC_H

#include <cstdint>

int dcerpc_validate(const uint8_t* data, int size);

#endif

service_bgp.h

Path = src/network_inspectors/appid/service_plugins/service_bgp.h

#ifndef SERVICE_BGP_H
#define SERVICE_BGP_H

#include "service_detector.h"

class ServiceDiscovery;

class BgpServiceDetector : public ServiceDetector
{
public:
    BgpServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};

#endif

service_bit.h

Path = src/network_inspectors/appid/service_plugins/service_bit.h

#ifndef SERVICE_BIT_H
#define SERVICE_BIT_H

#include "service_detector.h"

class ServiceDiscovery;

class BitServiceDetector : public ServiceDetector
{
public:
    BitServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};

#endif

service_bootp.h

Path = src/network_inspectors/appid/service_plugins/service_bootp.h

#ifndef SERVICE_BOOTP_H
#define SERVICE_BOOTP_H

#include "service_detector.h"

class AppIdSession;
class ServiceDiscovery;

class BootpServiceDetector : public ServiceDetector
{
public:
    BootpServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;

private:
    void add_dhcp_info(AppIdSession&, unsigned op55_len, const uint8_t* op55, unsigned
        op60_len, const uint8_t* op60, const uint8_t* mac);
    void add_new_dhcp_lease(AppIdSession&, const uint8_t* mac, uint32_t ip, uint32_t subnetmask,
        uint32_t leaseSecs, uint32_t router);
};
#endif

service_dcerpc.h

Path = src/network_inspectors/appid/service_plugins/service_dcerpc.h

#ifndef SERVICE_DCERPC_H
#define SERVICE_DCERPC_H
#include "service_detector.h"

class AppIdSession;
class ServiceDiscovery;
struct ServiceRPCData;

class DceRpcServiceDetector : public ServiceDetector
{
public:
    DceRpcServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;

private:
    int udp_validate(AppIdDiscoveryArgs&);
    int tcp_validate(AppIdDiscoveryArgs&);
};
#endif

service_detector.h

Path = src/network_inspectors/appid/service_plugins/service_detector.h

#ifndef SERVICE_DETECTOR_H
#define SERVICE_DETECTOR_H

#include "appid_detector.h"
#include "service_discovery.h"

class ServiceDetector : public AppIdDetector
{
public:
    ServiceDetector();

    void register_appid(AppId, unsigned extractsInfo, OdpContext& odp_ctxt) override;

    int service_inprocess(AppIdSession&, const snort::Packet*, AppidSessionDirection dir);

    int add_service(AppidChangeBits&, AppIdSession&, const snort::Packet*,
        AppidSessionDirection, AppId, const char* vendor = nullptr,
        const char* version = nullptr, AppIdServiceSubtype* = nullptr);

    int add_service_consume_subtype(AppIdSession&, const snort::Packet*,
        AppidSessionDirection dir, AppId, const char* vendor, const char* version,
        AppIdServiceSubtype*, AppidChangeBits&);

    int incompatible_data(AppIdSession&, const snort::Packet*, AppidSessionDirection dir);
    int fail_service(AppIdSession&, const snort::Packet*, AppidSessionDirection dir);

    void add_miscellaneous_info(AppIdSession& asd, AppId miscId)
    {
        asd.misc_app_id = miscId;
    }

private:
    int update_service_data(AppIdSession&, const snort::Packet*, AppidSessionDirection, AppId,
        const char*, const char*, AppidChangeBits&, AppIdServiceSubtype*);
};
#endif

service_discovery.h

Path = src/network_inspectors/appid/service_plugins/service_discovery.h

#ifndef SERVICE_DISCOVERY_H
#define SERVICE_DISCOVERY_H

#include "appid_discovery.h"

#include <unordered_map>
#include <vector>

#include "flow/flow.h"
#include "utils/sflsq.h"

#include "appid_types.h"

class AppIdInspector;
class AppIdSession;
class ServiceDetector;
class ServiceDiscoveryState;

namespace snort
{
struct Packet;
}

#define STATE_ID_INCONCLUSIVE_SERVICE_WEIGHT 3
#define STATE_ID_INVALID_CLIENT_THRESHOLD    9
#define STATE_ID_NEEDED_DUPE_DETRACT_COUNT   3
#define STATE_ID_MAX_VALID_COUNT 5

/* Service state stored per flow, which acts based on global ServiceState
 * at the beginning of the flow, then independently do service discovery, and
 * synchronize findings at the end of service discovery by the flow.
 */
enum SESSION_SERVICE_SEARCH_STATE
{
    START = 0,
    PORT,
    PATTERN,
    PENDING
};

class ServiceDiscovery : public AppIdDiscovery
{
public:
    ~ServiceDiscovery() override = default;
    void initialize(AppIdInspector&) override;
    void reload() override;
    void finalize_service_patterns();
    void reload_service_patterns();
    unsigned get_pattern_count();
    int add_service_port(AppIdDetector*, const ServiceDetectorPort&) override;

    AppIdDetectorsIterator get_detector_iterator(IpProtocol);
    ServiceDetector* get_next_tcp_detector(AppIdDetectorsIterator&);
    ServiceDetector* get_next_udp_detector(AppIdDetectorsIterator&);

    bool do_service_discovery(AppIdSession&, snort::Packet*, AppidSessionDirection dir,
        AppidChangeBits& change_bits);
    int identify_service(AppIdSession&, snort::Packet*, AppidSessionDirection, AppidChangeBits&);
    int fail_service(AppIdSession&, const snort::Packet*, AppidSessionDirection dir, ServiceDetector*, ServiceDiscoveryState* sds = nullptr);
    int incompatible_data(AppIdSession&, const snort::Packet*, AppidSessionDirection dir, ServiceDetector*);
    static int add_ftp_service_state(AppIdSession&);
    static void clear_ftp_service_state();
    static void set_thread_local_ftp_service();
    static void reset_thread_local_ftp_service();
    ServiceDetector* get_service_detector(const std::string&) const;

private:
    void get_next_service(const snort::Packet*, const AppidSessionDirection dir, AppIdSession&);
    void get_port_based_services(IpProtocol, uint16_t port, AppIdSession&);
    void match_by_pattern(AppIdSession&, const snort::Packet*, IpProtocol);
    static ServiceDiscovery* discovery_manager;
    std::vector<AppIdDetector*> service_detector_list;
    std::unordered_map<uint16_t, std::vector<ServiceDetector*> > tcp_services;
    std::unordered_map<uint16_t, std::vector<ServiceDetector*> > udp_services;
    std::unordered_map<uint16_t, std::vector<ServiceDetector*> > udp_reversed_services;
};

#endif

service_ftp.h

Path = src/network_inspectors/appid/service_plugins/service_ftp.h

#ifndef SERVICE_FTP_H
#define SERVICE_FTP_H

#include "service_detector.h"

class ServiceDiscovery;

class FtpServiceDetector : public ServiceDetector
{
public:
    FtpServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;

private:
    void create_expected_session(AppIdSession&, const snort::Packet*,
        const snort::SfIp* cliIp, uint16_t cliPort, const snort::SfIp* srvIp,
        uint16_t srvPort, IpProtocol, AppidSessionDirection);
};
#endif

service_irc.h

Path = src/network_inspectors/appid/service_plugins/service_irc.h

#ifndef SERVICE_IRC_H
#define SERVICE_IRC_H

#include "service_detector.h"

class ServiceDiscovery;

class IrcServiceDetector : public ServiceDetector
{
public:
    IrcServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};
#endif

service_lpr.h

Path = src/network_inspectors/appid/service_plugins/service_lpr.h

#ifndef SERVICE_LPR_H
#define SERVICE_LPR_H

#include "service_detector.h"

class ServiceDiscovery;

class LprServiceDetector : public ServiceDetector
{
public:
    LprServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};
#endif

service_mdns.h

Path = src/network_inspectors/appid/service_plugins/service_mdns.h

#ifndef SERVICE_MDNS_H
#define SERVICE_MDNS_H

#include "service_detector.h"

namespace snort
{
class SearchTool;
}
class ServiceDiscovery;
struct MatchedPatterns;

class MdnsServiceDetector : public ServiceDetector
{
public:
    MdnsServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
    void do_custom_reload() override;

private:
    MatchedPatterns* create_match_list(const char* data, uint16_t dataSize);
    void scan_matched_patterns(const char* dataPtr, uint16_t index, const char** resp_endptr,
        int* pattern_length, MatchedPatterns*& pattern_list);
    void destroy_match_list(MatchedPatterns*& pattern_list);
    int validate_reply(const uint8_t* data, uint16_t size);
    int analyze_user(AppIdSession&, const snort::Packet*, uint16_t size,
        AppidChangeBits& change_bits, MatchedPatterns*& pattern_list);
    int reference_pointer(const char* start_ptr, const char** resp_endptr, int* start_index,
        uint16_t data_size, uint8_t* user_name_len, unsigned size, MatchedPatterns*& pattern_list);

    snort::SearchTool matcher;
};
#endif

service_mysql.h

Path = src/network_inspectors/appid/service_plugins/service_mysql.h

#ifndef SERVICE_MYSQL_H
#define SERVICE_MYSQL_H

#include "service_detector.h"

class ServiceDiscovery;

class MySqlServiceDetector : public ServiceDetector
{
public:
    MySqlServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};

#endif

service_netbios.h

Path = src/network_inspectors/appid/service_plugins/service_netbios.h

#ifndef SERVICE_NETBIOS_H
#define SERVICE_NETBIOS_H

#include "service_detector.h"

class AppIdSession;
class ServiceDiscovery;

class NbnsServiceDetector : public ServiceDetector
{
public:
    NbnsServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};

class NbdgmServiceDetector : public ServiceDetector
{
public:
    NbdgmServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;

private:
    void add_smb_info(AppIdSession&, unsigned major, unsigned minor, uint32_t flags);
};
#endif

service_nntp.h

Path = src/network_inspectors/appid/service_plugins/service_nntp.h

#ifndef SERVICE_NNTP_H
#define SERVICE_NNTP_H

#include "service_detector.h"

class ServiceDiscovery;

class NntpServiceDetector : public ServiceDetector
{
public:
    NntpServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};
#endif

service_ntp.h

Path = src/network_inspectors/appid/service_plugins/service_ntp.h

#ifndef SERVICE_NTP_H
#define SERVICE_NTP_H

#include "service_detector.h"

class ServiceDiscovery;

class NtpServiceDetector : public ServiceDetector
{
public:
    NtpServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};
#endif

service_radius.h

Path = src/network_inspectors/appid/service_plugins/service_radius.h

#ifndef SERVICE_RADIUS_H
#define SERVICE_RADIUS_H

#include "service_detector.h"

class ServiceDiscovery;

class RadiusServiceDetector : public ServiceDetector
{
public:
    RadiusServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};

class RadiusAcctServiceDetector : public ServiceDetector
{
public:
    RadiusAcctServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};
#endif

service_regtest.h

Path = src/network_inspectors/appid/service_plugins/service_regtest.h

#ifndef SERVICE_REGTEST_H
#define SERVICE_REGTEST_H

#ifdef REG_TEST
#include "service_detector.h"

class ServiceDiscovery;

class RegTestServiceDetector : public ServiceDetector
{
public:
    RegTestServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};

class RegTestServiceDetector1 : public ServiceDetector
{
public:
    RegTestServiceDetector1(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};

class RegTestServiceDetector2 : public ServiceDetector
{
public:
    RegTestServiceDetector2(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};
#endif

#endif

service_rexec.h

Path = src/network_inspectors/appid/service_plugins/service_rexec.h

#ifndef SERVICE_REXEC_H
#define SERVICE_REXEC_H

#include "service_detector.h"

class ServiceDiscovery;
struct ServiceREXECData;

class RexecServiceDetector : public ServiceDetector
{
public:
    RexecServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
private:
    void rexec_bail(ServiceREXECData* rd);
};

#endif

service_rfb.h

Path = src/network_inspectors/appid/service_plugins/service_rfb.h

#ifndef SERVICE_RFB_H
#define SERVICE_RFB_H

#include "service_detector.h"

class ServiceDiscovery;

class RfbServiceDetector : public ServiceDetector
{
public:
    RfbServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};

#endif

service_rlogin.h

Path = src/network_inspectors/appid/service_plugins/service_rlogin.h

#ifndef SERVICE_RLOGIN_H
#define SERVICE_RLOGIN_H

#include "service_detector.h"

class ServiceDiscovery;

class RloginServiceDetector : public ServiceDetector
{
public:
    RloginServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};
#endif

service_rpc.h

Path = src/network_inspectors/appid/service_plugins/service_rpc.h

#ifndef SERVICE_RPC_H
#define SERVICE_RPC_H

#include "service_detector.h"

class AppIdSession;
class ServiceDiscovery;
struct ServiceRPCData;

class RpcServiceDetector : public ServiceDetector
{
public:
    RpcServiceDetector(ServiceDiscovery*);
    ~RpcServiceDetector() override;

    int validate(AppIdDiscoveryArgs&) override;

private:
    int rpc_udp_validate(AppIdDiscoveryArgs&);
    int rpc_tcp_validate(AppIdDiscoveryArgs&);
    int validate_packet(const uint8_t* data, uint16_t size, AppidSessionDirection dir, AppIdSession&,
        snort::Packet*, ServiceRPCData*, const char** pname, uint32_t* program);
};
#endif

service_rshell.h

Path = src/network_inspectors/appid/service_plugins/service_rshell.h

#ifndef SERVICE_RSHELL_H
#define SERVICE_RSHELL_H

#include "service_detector.h"

class ServiceDiscovery;
struct ServiceRSHELLData;

class RshellServiceDetector : public ServiceDetector
{
public:
    RshellServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
private:
    void rshell_bail(ServiceRSHELLData* rd);
};
#endif

service_rsync.h

Path = src/network_inspectors/appid/service_plugins/service_rsync.h

#ifndef SERVICE_RSYNC_H
#define SERVICE_RSYNC_H

#include "service_detector.h"

class ServiceDiscovery;

class RsyncServiceDetector : public ServiceDetector
{
public:
    RsyncServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};
#endif

service_rtmp.h

Path = src/network_inspectors/appid/service_plugins/service_rtmp.h

#ifndef SERVICE_RTMP_H
#define SERVICE_RTMP_H

#include "service_detector.h"

class ServiceDiscovery;

class RtmpServiceDetector : public ServiceDetector
{
public:
    RtmpServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};
#endif

service_snmp.h

Path = src/network_inspectors/appid/service_plugins/service_snmp.h

#ifndef SERVICE_SNMP_H
#define SERVICE_SNMP_H

#include "service_detector.h"

class ServiceDiscovery;

class SnmpServiceDetector : public ServiceDetector
{
public:
    SnmpServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};

#endif

service_ssl.h

Path = src/network_inspectors/appid/service_plugins/service_ssl.h

#ifndef SERVICE_SSL_H
#define SERVICE_SSL_H

#include "service_detector.h"

class OdpContext;
class ServiceDiscovery;

class SslServiceDetector : public ServiceDetector
{
public:
    SslServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};

AppId getSslServiceAppId(short srcPort);
bool is_service_over_ssl(AppId);

#endif

service_telnet.h

Path = src/network_inspectors/appid/service_plugins/service_telnet.h

#ifndef SERVICE_TELNET_H
#define SERVICE_TELNET_H

#include "service_detector.h"

class ServiceDiscovery;

class TelnetServiceDetector : public ServiceDetector
{
public:
    TelnetServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};
#endif

service_tftp.h

Path = src/network_inspectors/appid/service_plugins/service_tftp.h

#ifndef SERVICE_TFTP_H
#define SERVICE_TFTP_H

#include "service_detector.h"

class ServiceDiscovery;

class TftpServiceDetector : public ServiceDetector
{
public:
    TftpServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};
#endif

service_timbuktu.h

Path = src/network_inspectors/appid/service_plugins/service_timbuktu.h

#ifndef SERVICE_TIMBUKTU_H
#define SERVICE_TIMBUKTU_H

#include "service_detector.h"

class ServiceDiscovery;

class TimbuktuServiceDetector : public ServiceDetector
{
public:
    TimbuktuServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};

#endif

service_tns.h

Path = src/network_inspectors/appid/service_plugins/service_tns.h

#ifndef SERVICE_TNS_H
#define SERVICE_TNS_H

#include "service_detector.h"

class ServiceDiscovery;

class TnsServiceDetector : public ServiceDetector
{
public:
    TnsServiceDetector(ServiceDiscovery*);

    int validate(AppIdDiscoveryArgs&) override;
};

#endif

network_inspectors/appid/service_plugins/test/

service_plugin_mock.h

Path = src/network_inspectors/appid/service_plugins/test/service_plugin_mock.h

#ifndef SERVICE_PLUGIN_MOCK_H
#define SERVICE_PLUGIN_MOCK_H
#include "appid_detector.h"
#include "appid_module.h"
#include "appid_peg_counts.h"
#include "utils/stats.h"

#define APPID_UT_ID 1492

namespace snort
{
// Stubs for messages
void ParseWarning(WarningGroup, const char*, ...) { }

// Stubs for appid sessions
FlowData::FlowData(unsigned, Inspector*) : next(nullptr), prev(nullptr), handler(nullptr), id(0)
{ }
FlowData::~FlowData() = default;

// Stubs for packet
Packet::Packet(bool) { }
Packet::~Packet() = default;

Inspector::Inspector() = default;
Inspector::~Inspector() = default;
bool Inspector::likes(Packet*) { return true; }
bool Inspector::get_buf(const char*, Packet*, InspectionBuffer&) { return true; }
class StreamSplitter* Inspector::get_splitter(bool) { return nullptr; }

// Stubs for search_tool.cc
SearchTool::SearchTool(bool, const char*) { }
SearchTool::~SearchTool() = default;

// Stubs for util.cc
char* snort_strndup(const char* , size_t dst_size)
{
    char* dup = (char*)snort_calloc(dst_size + 1);
    return dup;
}

char* snort_strdup(const char* str)
{
    assert(str);
    size_t n = strlen(str) + 1;
    char* p = (char*)snort_alloc(n);
    memcpy(p, str, n);
    return p;
}
class InspectorManager
{
public:
SO_PUBLIC static Inspector* get_inspector(const char*, bool, SnortConfig*) {return nullptr;}
};
Module::Module(const char*, const char*) {}
Module::Module(const char*, const char*, const Parameter*, bool)
{}
PegCount Module::get_global_count(char const*) const { return 0; }
void Module::show_interval_stats(std::vector<unsigned int, std::allocator<unsigned int> >&, FILE*) {}
void Module::show_stats(){}
void Module::sum_stats(bool){}
void Module::main_accumulate_stats(){}
void Module::reset_stats() {}

AppIdSessionApi::AppIdSessionApi(const AppIdSession*, const SfIp&) :
    StashGenericObject(STASH_GENERIC_OBJECT_APPID) {}
}

AlpnPatternMatchers::~AlpnPatternMatchers() {}
EveCaPatternMatchers::~EveCaPatternMatchers() { }
SslPatternMatchers::~SslPatternMatchers() { }
SipPatternMatchers::~SipPatternMatchers() { }
HttpPatternMatchers::~HttpPatternMatchers() { }
DnsPatternMatchers::~DnsPatternMatchers() { }
CipPatternMatchers::~CipPatternMatchers() { }
void ClientDiscovery::initialize(AppIdInspector&) {}
void ClientDiscovery::reload() {}
FpSMBData* smb_data = nullptr;

int AppIdDetector::initialize(AppIdInspector&){return 0;}
void AppIdDetector::reload() { }
int AppIdDetector::data_add(AppIdSession&, void*, AppIdFreeFCN){return 0;}
void* AppIdDetector::data_get(AppIdSession&) {return nullptr;}
void AppIdDetector::add_user(AppIdSession&, const char*, AppId, bool, AppidChangeBits&){}
void AppIdDetector::add_payload(AppIdSession&, AppId){}
void AppIdDetector::add_app(const snort::Packet&, AppIdSession&, AppidSessionDirection, AppId, AppId, const char*, AppidChangeBits&){}
void ApplicationDescriptor::set_id(AppId){}
void ServiceAppDescriptor::set_id(AppId, OdpContext&){}
void ClientAppDescriptor::update_user(AppId, const char*, AppidChangeBits&){}
void AppIdDiscovery::add_pattern_data(AppIdDetector*, snort::SearchTool&, int,
        const uint8_t* const, unsigned, unsigned){}
void AppIdDiscovery::register_detector(const std::string&, AppIdDetector*,  IpProtocol){}
void AppIdDiscovery::register_tcp_pattern(AppIdDetector*, const uint8_t* const, unsigned,
    int, unsigned){}
void AppIdDiscovery::register_udp_pattern(AppIdDetector*, const uint8_t* const, unsigned,
    int, unsigned){}
int AppIdDiscovery::add_service_port(AppIdDetector*, const ServiceDetectorPort&){return 0;}
void ApplicationDescriptor::set_id(const snort::Packet&, AppIdSession&, AppidSessionDirection, AppId, AppidChangeBits&){}
int AppIdSession::add_flow_data(void*, unsigned, AppIdFreeFCN) { return 0; }
int dcerpc_validate(const uint8_t*, int){return 0; }
AppIdDiscovery::~AppIdDiscovery() { }
void show_stats(PegCount*, const PegInfo*, unsigned, const char*) { }
void show_stats(PegCount*, const PegInfo*, const IndexVec&, const char*, FILE*) { }
AppIdConfig config;
AppIdContext ctxt(config);
class AppIdInspector : public snort::Inspector
{
public:
    void eval(Packet*) override { }
    bool configure(snort::SnortConfig*) override { return true; }
};

// Stubs for modules, config
AppIdConfig::~AppIdConfig() = default;
bool AppIdModule::begin(const char*, int, snort::SnortConfig*)
{
    return false;
}

bool AppIdModule::set(const char*, snort::Value&, snort::SnortConfig*)
{
    return false;
}

bool AppIdModule::end(const char*, int, snort::SnortConfig*)
{
    return false;
}

const Command* AppIdModule::get_commands() const
{
    return nullptr;
}

const PegInfo* AppIdModule::get_pegs() const
{
    return nullptr;
}

PegCount* AppIdModule::get_counts() const
{
    return nullptr;
}

void AppIdModule::set_trace(const Trace*) const { }
const TraceOption* AppIdModule::get_trace_options() const { return nullptr; }

// Stubs for inspectors
unsigned AppIdSession::inspector_id = 0;
AppIdConfig stub_config;
AppIdContext stub_ctxt(stub_config);
static OdpContext stub_odp_ctxt(stub_config, nullptr);
OdpContext* AppIdContext::odp_ctxt = &stub_odp_ctxt;
AppIdSession::AppIdSession(IpProtocol, const SfIp* ip, uint16_t, AppIdInspector& inspector,
    OdpContext&, uint16_t) : snort::FlowData(inspector_id, (snort::Inspector*)&inspector),
    config(stub_config), api(*(new AppIdSessionApi(this, *ip))), odp_ctxt(stub_odp_ctxt) { }
AppIdSession::~AppIdSession() = default;
DiscoveryFilter::~DiscoveryFilter(){}
void AppIdSession::free_flow_data()
{
    snort_free(smb_data);
}
void* AppIdSession::get_flow_data(unsigned) const { return smb_data;}

// Stubs for AppIdPegCounts
void AppIdPegCounts::inc_service_count(AppId, bool) { }
void AppIdPegCounts::inc_client_count(AppId, bool) { }
void AppIdPegCounts::inc_payload_count(AppId, bool) { }

THREAD_LOCAL AppIdStats appid_stats;
void AppIdModule::show_dynamic_stats() { }

// Stubs for app_info_table.cc
AppInfoTableEntry* AppInfoManager::get_app_info_entry(int)
{
    return nullptr;
}

bool AppInfoManager::configured()
{
    return true;
}
ServiceDiscoveryState* AppIdServiceState::add(SfIp const*, IpProtocol,
    unsigned short, int16_t, uint16_t, bool, bool)
{
  return nullptr;
}
void ServiceDiscoveryState::set_service_id_valid(ServiceDetector*) { }

OdpContext::OdpContext(const AppIdConfig&, snort::SnortConfig*)
{ }

#endif

network_inspectors/appid/test/

appid_mock_definitions.h

Path = src/network_inspectors/appid/test/appid_mock_definitions.h

#ifndef APPID_MOCK_DEFINITIONS_H
#define APPID_MOCK_DEFINITIONS_H

#include "log/messages.h"
#include "utils/stats.h"

#include "appid_detector.h"
#include "appid_module.h"
#include "appid_peg_counts.h"
#include "service_inspectors/http_inspect/http_msg_header.h"

class ThirdPartyAppIdContext;

ThirdPartyAppIdContext* tp_appid_ctxt = nullptr;

namespace snort
{
char* snort_strndup(const char* src, size_t dst_size)
{
    char* dup = (char*)snort_calloc(dst_size + 1);
    strncpy(dup, src, dst_size + 1);
    return dup;
}

char* snort_strdup(const char* str)
{
    assert(str);
    size_t n = strlen(str) + 1;
    char* p = (char*)snort_alloc(n);
    memcpy(p, str, n);
    return p;
}

void ErrorMessage(const char*,...) { }
void WarningMessage(const char*,...) { }
void LogMessage(const char*,...) { }
void LogText(const char*, FILE*) {}

void ParseWarning(WarningGroup, const char*, ...) { }
void LogLabel(const char*, FILE*) {}

unsigned DataBus::get_id(const PubKey&) { return 0; }

SearchTool::SearchTool(bool multi, const char*) : mpsegrp(nullptr), max_len(0), multi_match(multi)
{ }
SearchTool::~SearchTool() = default;
}
DiscoveryFilter::~DiscoveryFilter(){}
void ApplicationDescriptor::set_id(AppId app_id){ my_id = app_id;}
void ServiceAppDescriptor::set_id(AppId app_id, OdpContext&){ set_id(app_id); }
void ServiceAppDescriptor::set_port_service_id(AppId app_id){ port_service_id = app_id;}
void ClientAppDescriptor::update_user(AppId app_id, const char* username, AppidChangeBits& change_bits)
{
    my_username = username;
    my_user_id = app_id;
    change_bits.set(APPID_USER_INFO_BIT);
}

AppIdDiscovery::~AppIdDiscovery() = default;
void ClientDiscovery::initialize(AppIdInspector&) { }
void ClientDiscovery::reload() { }
void AppIdDiscovery::register_detector(const string&, AppIdDetector*, IpProtocol) { }
void AppIdDiscovery::add_pattern_data(AppIdDetector*, snort::SearchTool&, int, unsigned char const*, unsigned int, unsigned int) { }
void AppIdDiscovery::register_tcp_pattern(AppIdDetector*, unsigned char const*, unsigned int, int, unsigned int) { }
void AppIdDiscovery::register_udp_pattern(AppIdDetector*, unsigned char const*, unsigned int, int, unsigned int) { }
int AppIdDiscovery::add_service_port(AppIdDetector*, ServiceDetectorPort const&) { return 0; }
DnsPatternMatchers::~DnsPatternMatchers() = default;
EveCaPatternMatchers::~EveCaPatternMatchers() = default;
HttpPatternMatchers::~HttpPatternMatchers() = default;
SipPatternMatchers::~SipPatternMatchers() = default;
SslPatternMatchers::~SslPatternMatchers() = default;
AlpnPatternMatchers::~AlpnPatternMatchers() = default;
CipPatternMatchers::~CipPatternMatchers() = default;

void Field::set(int32_t length, const uint8_t* start, bool own_the_buffer_)
{
    strt = start;
    len = length;
    own_the_buffer = own_the_buffer_;
}

Field global_field;


int ServiceDiscovery::add_ftp_service_state(AppIdSession&)
{
    return 0;
}

void ServiceDiscovery::initialize(AppIdInspector&) { }
void ServiceDiscovery::reload() { }

int ServiceDiscovery::add_service_port(AppIdDetector*, const ServiceDetectorPort&)
{ return 0; }

// Stubs for app_info_table.h
AppInfoTableEntry* AppInfoManager::get_app_info_entry(int)
{
  return nullptr;
}

bool AppInfoManager::configured()
{ return false; }

// Stubs for service_state.h
ServiceDiscoveryState* AppIdServiceState::get(SfIp const*, IpProtocol,
    unsigned short, int16_t, uint32_t, bool, bool)
{
  return nullptr;
}

ServiceDiscoveryState* AppIdServiceState::add(SfIp const*, IpProtocol,
    unsigned short, int16_t, uint32_t, bool, bool)
{
  return nullptr;
}

bool AppIdServiceState::prune(size_t, size_t)
{
    return true;
}

bool AppIdReloadTuner::tinit() { return false; }

bool AppIdReloadTuner::tune_resources(unsigned int)
{
    return true;
}

void ServiceDiscoveryState::set_service_id_valid(ServiceDetector*) { }

// Stubs for service_plugins/service_discovery.h
int ServiceDiscovery::incompatible_data(AppIdSession&, const Packet*, AppidSessionDirection, ServiceDetector*)
{
    return 0;
}

int ServiceDiscovery::fail_service(AppIdSession&, const Packet*, AppidSessionDirection, ServiceDetector*, ServiceDiscoveryState*)
{
    return 0;
}

inline void mock_init_appid_pegs()
{
    AppIdPegCounts::init_pegs();
}

inline void mock_cleanup_appid_pegs()
{
    AppIdPegCounts::cleanup_pegs();
    AppIdPegCounts::cleanup_peg_info();
}

THREAD_LOCAL AppIdStats appid_stats;

#endif

appid_mock_flow.h

Path = src/network_inspectors/appid/test/appid_mock_flow.h

#ifndef APPID_MOCK_FLOW_H
#define APPID_MOCK_FLOW_H

FlowData::FlowData(unsigned, Inspector*)
{
    next = prev = nullptr;
    handler = nullptr;
    id = 222;
}

FlowData::~FlowData() = default;

FlowData* mock_flow_data = nullptr;

typedef int32_t AppId;
Flow::~Flow() = default;

class FakeFlow : public Flow
{
};

FlowData* Flow::get_flow_data(unsigned) const
{
    return mock_flow_data;
}

int Flow::set_flow_data(FlowData* fd)
{
    mock_flow_data = fd;
    return 0;
}

FlowStash::~FlowStash() = default;

#endif

appid_mock_http_session.h

Path = src/network_inspectors/appid/test/appid_mock_http_session.h

#ifndef APPID_MOCK_HTTP_SESSION_H
#define APPID_MOCK_HTTP_SESSION_H

#include <string>

typedef AppIdHttpSession::pair_t pair_t;

AppIdHttpSession::AppIdHttpSession(AppIdSession& session, int64_t http2_stream_id)
    : asd(session), httpx_stream_id(http2_stream_id)
{
    for ( int i = 0; i < NUM_METADATA_FIELDS; i++)
        meta_data[i] = nullptr;
}

AppIdHttpSession::~AppIdHttpSession()
{
    for ( int i = 0; i < NUM_METADATA_FIELDS; i++)
        delete meta_data[i];
    if (tun_dest)
        delete tun_dest;
}

int AppIdHttpSession::process_http_packet(AppidSessionDirection, AppidChangeBits&, HttpPatternMatchers&) { return 0; }

char const* CONTENT_TYPE = "html/text";
char const* COOKIE = "this is my request cookie content";
char const* NEW_COOKIE = "request new cookie content is chocolate chip";
char const* HOST = "www.google.com";
char const* LOCATION = "abc.yahoo.com";
char const* URL = "https://www.google.com/path/to/index.html";
char const* URI = "/path/to/index.html";
char const* USERAGENT = "Mozilla/5.0 (Macintosh; Intel Mac OS X)";
char const* REFERER = "http://www.yahoo.com/search";
char const* SERVER = "Apache";
char const* X_WORKING_WITH = "working with string";
char const* VIA = "via string";
char const* RESPONSE_CODE = "301";
char const* REQ_BODY = "this is the body of the http request";
char const* RSP_BODY = "this is the body of the http response";

#define URI_OFFSET 22
#define COOKIE_OFFSET 44

void AppIdHttpSession::update_url(AppidChangeBits&)
{
    const std::string* host_test = meta_data[REQ_HOST_FID];
    const std::string* uri_test = meta_data[REQ_URI_FID];
    if (host_test and uri_test)
    {
        if (meta_data[MISC_URL_FID])
            delete meta_data[MISC_URL_FID];
        meta_data[MISC_URL_FID] = new std::string(std::string("http://") + *host_test + *uri_test);
    }
}

class MockAppIdHttpSession : public AppIdHttpSession
{
public:
    MockAppIdHttpSession(AppIdSession& asd)
        : AppIdHttpSession(asd, 0)
    {
        meta_data[REQ_AGENT_FID] = new std::string(USERAGENT);
        meta_data[REQ_HOST_FID] = new std::string(HOST);
        meta_data[REQ_REFERER_FID] = new std::string(REFERER);
        meta_data[REQ_URI_FID] = new std::string(URI);
        meta_data[REQ_COOKIE_FID] = new std::string(COOKIE);
        meta_data[REQ_BODY_FID] = new std::string(REQ_BODY);
        meta_data[RSP_CONTENT_TYPE_FID] = new std::string(CONTENT_TYPE);
        meta_data[RSP_LOCATION_FID] = new std::string(LOCATION);
        meta_data[RSP_BODY_FID] = new std::string(RSP_BODY);
        meta_data[MISC_VIA_FID] = new std::string(VIA);
        meta_data[MISC_RESP_CODE_FID] = new std::string(RESPONSE_CODE);
        meta_data[MISC_SERVER_FID] = new std::string(SERVER);
        meta_data[MISC_XWW_FID] = new std::string(X_WORKING_WITH);
        meta_data[MISC_URL_FID] = new std::string(URL);
    }

    void reset()
    {
        for ( int i = 0; i < NUM_METADATA_FIELDS; i++)
        {
            delete meta_data[i];
            meta_data[i] = nullptr;
        }
    }

    static AppIdHttpSession* init_http_session(AppIdSession& asd)
    {
        AppIdHttpSession* hsession = new MockAppIdHttpSession(asd);

        return hsession;
    }
};

#endif

appid_mock_inspector.h

Path = src/network_inspectors/appid/test/appid_mock_inspector.h

#ifndef APPID_MOCK_INSPECTOR_H
#define APPID_MOCK_INSPECTOR_H

#include "appid_inspector.h"

class Value;

namespace snort
{
Inspector::Inspector() : ref_count(nullptr)
{
    set_api(nullptr);
}

Inspector::~Inspector() = default;
bool Inspector::likes(Packet*) { return true; }
bool Inspector::get_buf(const char*, Packet*, InspectionBuffer&) { return true; }
class StreamSplitter* Inspector::get_splitter(bool) { return nullptr; }

Module::Module(char const*, char const*) {}
void Module::sum_stats(bool) {}
void Module::main_accumulate_stats() { }
void Module::show_interval_stats(std::vector<unsigned int, std::allocator<unsigned int> >&, FILE*) {}
void Module::show_stats() {}
void Module::init_stats(bool) {}
void Module::reset_stats() {}
PegCount Module::get_global_count(char const*) const { return 0; }

}

AppIdModule::AppIdModule(): snort::Module("appid_mock", "appid_mock_help") {}
void AppIdModule::sum_stats(bool) {}
void AppIdModule::show_dynamic_stats() {}
bool AppIdModule::begin(char const*, int, snort::SnortConfig*) { return true; }
bool AppIdModule::end(char const*, int, snort::SnortConfig*) { return true; }
bool AppIdModule::set(char const*, snort::Value&, snort::SnortConfig*) { return true; }
const snort::Command* AppIdModule::get_commands() const { return nullptr; }
const PegInfo* AppIdModule::get_pegs() const { return nullptr; }
PegCount* AppIdModule::get_counts() const { return nullptr; }
snort::ProfileStats* AppIdModule::get_profile(
        unsigned, const char*&, const char*& ) const { return nullptr; }
void AppIdModule::set_trace(const Trace*) const { }
const TraceOption* AppIdModule::get_trace_options() const { return nullptr; }

AppIdInspector::~AppIdInspector() = default;
void AppIdInspector::eval(snort::Packet*) { }
bool AppIdInspector::configure(snort::SnortConfig*) { return true; }
void AppIdInspector::show(const SnortConfig*) const { }
void AppIdInspector::tinit() { }
void AppIdInspector::tterm() { }
void AppIdInspector::tear_down(snort::SnortConfig*) { }
AppIdContext& AppIdInspector::get_ctxt() const { return *ctxt; }

AppIdModule appid_mod;
AppIdConfig appid_config;
AppIdContext appid_ctxt(appid_config);
THREAD_LOCAL OdpContext* pkt_thread_odp_ctxt = nullptr;
AppIdInspector dummy_appid_inspector( appid_mod );

AppIdInspector::AppIdInspector(AppIdModule& )
{
    ctxt = &appid_ctxt;
    appid_config.app_detector_dir = "/path/to/appid/detectors/";
    config = &appid_config;
}

#endif

appid_mock_session.h

Path = src/network_inspectors/appid/test/appid_mock_session.h

#ifndef APPID_MOCK_SESSION_H
#define APPID_MOCK_SESSION_H

#include "flow/ha.h"

#include "appid_dns_session.h"
#include "appid_mock_flow.h"
#include "appid_mock_http_session.h"
#include "appid_mock_inspector.h"

bool is_session_decrypted = false;

char const* APPID_UT_TLS_HOST = "vpn.topsecret.com";
char const* APPID_UT_SERVICE_IP_ADDR = "192.168.0.2";
char const* APPID_UT_INITIATOR_IP_ADDR = "192.168.0.3";

char const* APPID_ID_UT_DNS_HOST = "delphi.opendns.com";
#define APPID_UT_DNS_HOST_OFFSET 22
#define APPID_UT_DNS_PATTERN_CNAME_REC  5
#define APPID_UT_DNS_NOERROR 0
#define APPID_UT_DNS_TTL 5

const char* test_app_name = "testapp_1492";
const AppId APPID_UT_ID = 1492;
const short APPID_UT_SERVICE_PORT = 1066;
const char* APPID_UT_USERNAME = "pigpen";
const char* APPID_UT_CLIENT_VERSION = "a snorting client";
const char* APPID_UT_SERVICE = "at your service";
const char* APPID_UT_SERVICE_VENDOR = "cisco snorty";
const char* APPID_UT_SERVICE_VERSION = "Version City";

AppIdServiceSubtype APPID_UT_SERVICE_SUBTYPE = { nullptr, APPID_UT_SERVICE,
                                                 APPID_UT_SERVICE_VENDOR,
                                                 APPID_UT_SERVICE_VERSION };

unsigned AppIdSession::inspector_id = 0;
std::mutex AppIdSession::inferred_svcs_lock;
uint16_t AppIdSession::inferred_svcs_ver = 0;
uint32_t OdpContext::next_version = 0;

class MockAppIdDnsSession : public AppIdDnsSession
{
public:
    MockAppIdDnsSession()
    {
        host = (const char*) APPID_ID_UT_DNS_HOST;
        host_offset = APPID_UT_DNS_HOST_OFFSET;
        record_type = APPID_UT_DNS_PATTERN_CNAME_REC;
        response_type = APPID_UT_DNS_NOERROR;
        ttl = APPID_UT_DNS_TTL;
    }
};

AppIdConfig::~AppIdConfig() = default;
OdpContext::OdpContext(const AppIdConfig&, snort::SnortConfig*) { }

void FlowHAState::add(uint8_t) { }

static AppIdConfig stub_config;
static AppIdContext stub_ctxt(stub_config);
static OdpContext stub_odp_ctxt(stub_config, nullptr);
OdpContext* AppIdContext::odp_ctxt = &stub_odp_ctxt;
AppIdSession::AppIdSession(IpProtocol proto, const SfIp* ip, uint16_t, AppIdInspector& inspector,
    OdpContext&, uint32_t, uint32_t) : FlowData(inspector_id, &inspector), config(stub_config),
    protocol(proto), api(*(new AppIdSessionApi(this, *ip))), odp_ctxt(stub_odp_ctxt)
{
    this->set_session_flags(APPID_SESSION_DISCOVER_APP | APPID_SESSION_SPECIAL_MONITORED);
    odp_ctxt_version = odp_ctxt.get_version();
    set_service_port(APPID_UT_SERVICE_PORT);
    AppidChangeBits change_bits;

    set_client_user(APPID_UT_ID, APPID_UT_USERNAME, change_bits);
    set_client_version(APPID_UT_CLIENT_VERSION, change_bits);

    set_service_vendor(APPID_UT_SERVICE_VENDOR, change_bits);
    set_service_version(APPID_UT_SERVICE_VERSION, change_bits);
    add_service_subtype(*(new AppIdServiceSubtype(APPID_UT_SERVICE_SUBTYPE)), change_bits);

    tsession = new TlsSession;

    SfIp svc_ip;
    svc_ip.pton(AF_INET, APPID_UT_SERVICE_IP_ADDR);
    set_service_ip(svc_ip);
    api.initiator_ip.pton(AF_INET, APPID_UT_INITIATOR_IP_ADDR);

    api.dsession = new MockAppIdDnsSession;
    tp_app_id = APPID_UT_ID;
    set_service_id(APPID_UT_ID + 1, odp_ctxt);
    client_inferred_service_id = APPID_UT_ID + 2;
    set_port_service_id(APPID_UT_ID + 3);
    set_payload_id(APPID_UT_ID + 4);
    tp_payload_app_id = APPID_UT_ID + 5;
    set_client_id(APPID_UT_ID + 6);
    misc_app_id = APPID_UT_ID + 7;
}

AppIdSession::~AppIdSession()
{
    delete tsession;
}

void* AppIdSession::get_flow_data(unsigned) const
{
    return nullptr;
}

int AppIdSession::add_flow_data(void*, unsigned, AppIdFreeFCN)
{
    return 0;
}

AppId AppIdSession::pick_service_app_id() const
{
    return get_service_id();
}

AppId AppIdSession::pick_ss_misc_app_id() const
{
    return misc_app_id;
}

AppId AppIdSession::pick_ss_client_app_id() const
{
    if (get_eve_client_app_id() > APP_ID_NONE and get_client_id() <= APP_ID_NONE)
    {
        api.client.set_eve_client_app_detect_type(CLIENT_APP_DETECT_TLS_FP);
        return get_eve_client_app_id();
    }
    else
    {
        api.client.set_eve_client_app_detect_type(CLIENT_APP_DETECT_APPID);
        return get_client_id();
    }
}

AppId AppIdSession::pick_ss_payload_app_id() const
{
    return get_payload_id();
}

AppId AppIdSession::pick_ss_referred_payload_app_id() const
{
    return APPID_UT_ID;
}

AppIdHttpSession* AppIdSession::create_http_session(int64_t)
{
    AppIdHttpSession* hsession = new MockAppIdHttpSession(*this);
    AppidChangeBits change_bits;

    hsession->client.set_id(APPID_UT_ID);
    hsession->client.set_version(APPID_UT_CLIENT_VERSION);
    change_bits.set(APPID_CLIENT_INFO_BIT);
    hsession->payload.set_id(APPID_UT_ID);
    hsession->misc_app_id = APPID_UT_ID;
    hsession->referred_payload_app_id = APPID_UT_ID;
    api.hsessions.push_back(hsession);
    return hsession;
}

AppIdHttpSession* AppIdSession::get_matching_http_session(int64_t stream_id) const
{
    for (uint64_t stream_index=0; stream_index < api.hsessions.size(); stream_index++)
    {
        if (stream_id == api.hsessions[stream_index]->get_httpx_stream_id())
            return api.hsessions[stream_index];
    }
    return nullptr;
}

AppIdDnsSession* AppIdSession::create_dns_session()
{
    if ( !api.dsession )
        api.dsession = new MockAppIdDnsSession();
    return api.dsession;
}

AppIdDnsSession* AppIdSession::get_dns_session() const
{
    return api.dsession;
}

bool AppIdSession::is_tp_appid_done() const
{
    return true;
}

bool AppIdSession::is_tp_appid_available() const
{
    return true;
}

#endif

log_message_mock.h

Path = src/network_inspectors/appid/test/log_message_mock.h

#define LOG_MESSAGE_MOCK

#include <cstdarg>

using namespace std;

namespace snort
{
// Note: without SO_PUBLIC this is not being exported so tp_mock.so won't
// load because of undefined symbol error.
SO_PUBLIC void ErrorMessage(const char* format,...)
{
    va_list ap;
    va_start(ap,format);
    vfprintf(stderr, format, ap);
    va_end(ap);
}

[[noreturn]] SO_PUBLIC void FatalError(const char* format,...)
{
    va_list ap;
    va_start(ap,format);
    vfprintf(stderr, format, ap);
    va_end(ap);
    exit(1);
}


SO_PUBLIC void WarningMessage(const char* format,...)
{
    va_list ap;
    va_start(ap, format);
    vfprintf(stderr, format, ap);
    va_end(ap);
}

SO_PUBLIC void LogMessage(const char* format,...)
{
    va_list ap;
    va_start(ap, format);
    vfprintf(stdout, format, ap);
    va_end(ap);
}
}
#endif

network_inspectors/arp_spoof/

Implement a specialized inspection tool which locates and alerts on ARP protocol violations. This network inspector looks at all ARP ethernet frames and attempts to locate ARP spoofing attacks.

It alerts on source or destination address mismatch. It also alerts on an ARP request occurring on a uni-cast frame (needs to be multi-cast).

A network inspector module as it needs to examine all ethernet frames with packet type of ARP.

arp_module.h

Path = src/network_inspectors/arp_spoof/arp_module.h

#ifndef ARP_SPOOF_MODULE_H
#define ARP_SPOOF_MODULE_H

#include "framework/module.h"

#define MOD_NAME "arp_spoof"
#define MOD_HELP "detect ARP attacks and anomalies"

#define GID_ARP_SPOOF 112

#define ARPSPOOF_UNICAST_ARP_REQUEST          1
#define ARPSPOOF_ETHERFRAME_ARP_MISMATCH_SRC  2
#define ARPSPOOF_ETHERFRAME_ARP_MISMATCH_DST  3
#define ARPSPOOF_ARP_CACHE_OVERWRITE_ATTACK   4

extern THREAD_LOCAL SimpleStats asstats;
extern THREAD_LOCAL snort::ProfileStats arpPerfStats;

struct IPMacEntry
{
    uint32_t ipv4_addr;
    uint8_t mac_addr[6];
};

typedef std::vector<IPMacEntry> IPMacEntryList;

struct ArpSpoofConfig
{
    bool check_overwrite;

    IPMacEntryList ipmel;
};

class ArpSpoofModule : public snort::Module
{
public:
    ArpSpoofModule();
    ~ArpSpoofModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    ArpSpoofConfig* get_config();

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;

    unsigned get_gid() const override
    { return GID_ARP_SPOOF; }

    const snort::RuleMap* get_rules() const override;
    snort::ProfileStats* get_profile() const override;

    Usage get_usage() const override
    { return INSPECT; }

private:
    ArpSpoofConfig* config = nullptr;
    IPMacEntry host = {};
};

#endif

network_inspectors/binder/

The binder maps configuration to traffic when a flow is started (via the eval method) and when service is identified (via the exec method). Binder sets the service on the flow to that of the service inspector when it is bound to the flow.

If the binder is not explicitly configured but the wizard is configured, then the inspector manager will create a default binder with bindings for each configured service inspector (excluding wizard itself).

BinderModule creates a vector of Bindings from the Lua binder table which is moved to the Binder upon its construction. Upon start of flow, the vector is iterated in search of applicable bindings. These include:

  • stream inspector

  • service inspector

  • passive inspector

Note that although Flow contains both clouseau and gadget, only one of those is bound to the flow at a time. The wizard fills in until service is identified at which point clouseau is removed and gadget is installed.

Note that bindings are recursive. It is possible to bind a policy (config file) that has its own binder, and so on.

The implementation is not yet optimized for performance. A faster method of searching for applicable bindings should be developed.

The exec() method implements specialized Inspector::Binder functionality.

Binder subscribes to the FLOW_SERVICE_CHANGE_EVENT and if the event is received, possibly due to an inspector discovering new information about the flow’s service, then it will call Bind::exec again to reset the service-to-inspector mapping.

binding.h

Path = src/network_inspectors/binder/binding.h

#ifndef BINDING_H
#define BINDING_H

#include <string>

#include "framework/bits.h"
#include "main/policy.h"
#include "sfip/sf_ipvar.h"

namespace snort
{
class Flow;
class Inspector;
struct SnortConfig;
}

struct BindWhen
{
    enum Role
    { BR_CLIENT, BR_SERVER, BR_EITHER, BR_MAX };

    PolicyId ips_id;
    unsigned ips_id_user;
    unsigned protos;
    Role role;
    std::string svc;

    sfip_var_t* src_nets;
    sfip_var_t* dst_nets;

    VlanBitSet vlans;

    PortBitSet src_ports;
    PortBitSet dst_ports;

    std::unordered_set<int32_t> src_intfs;
    std::unordered_set<int32_t> dst_intfs;

    std::unordered_set<int16_t> src_groups;
    std::unordered_set<int16_t> dst_groups;

    std::unordered_set<uint32_t> addr_spaces;

    std::unordered_set<uint32_t> tenants;

    enum Criteria
    {
        BWC_IPS_ID =        0x0001,
        BWC_PROTO =         0x0002,
        BWC_SVC =           0x0004,
        BWC_NETS =          0x0008,
        BWC_SPLIT_NETS =    0x0010,
        BWC_VLANS =         0x0020,
        BWC_PORTS =         0x0040,
        BWC_SPLIT_PORTS =   0x0080,
        BWC_INTFS =         0x0100,
        BWC_SPLIT_INTFS =   0x0200,
        BWC_GROUPS =        0x0400,
        BWC_SPLIT_GROUPS =  0x0800,
        BWC_ADDR_SPACES =   0x1000,
        BWC_TENANTS =       0x2000
    };
    uint16_t criteria_flags;

    void add_criteria(uint16_t flags)
    { criteria_flags |= flags; }
    bool has_criteria(uint16_t flags) const
    { return (criteria_flags & flags) == flags; }
};

struct BindUse
{
    enum Action
    { BA_RESET, BA_BLOCK, BA_ALLOW, BA_INSPECT, BA_MAX };

    enum What
    { BW_NONE, BW_PASSIVE, BW_CLIENT, BW_SERVER, BW_STREAM, BW_WIZARD, BW_GADGET, BW_MAX };

    std::string svc;
    std::string type;
    std::string name;

    Action action;
    unsigned inspection_index;
    unsigned ips_index;
    What what;
    snort::Inspector* inspector;
    bool global_type;
};

struct Binding
{
    BindWhen when;
    BindUse use;

    Binding();

    void clear();
    void configure(const snort::SnortConfig* sc);

    bool check_all(const snort::Flow&, const char* = nullptr) const;
    bool check_all(const snort::Packet*) const;
    bool check_ips_policy(const snort::Flow&) const;
    bool check_ips_policy() const;
    bool check_vlan(const snort::Flow&) const;
    bool check_vlan(const snort::Packet*) const;
    bool check_addr(const snort::Flow&) const;
    bool check_addr(const snort::Packet*) const;
    bool check_split_addr(const snort::Flow&) const;
    bool check_split_addr(const snort::Packet*) const;
    bool check_proto(const snort::Flow&) const;
    bool check_proto(const snort::Packet*) const;
    bool check_port(const snort::Flow&) const;
    bool check_port(const snort::Packet*) const;
    bool check_split_port(const snort::Flow&) const;
    bool check_split_port(const snort::Packet*) const;
    bool check_intf(const snort::Flow&) const;
    bool check_intf(const snort::Packet*) const;
    bool check_split_intf(const snort::Flow&) const;
    bool check_split_intf(const snort::Packet*) const;
    bool check_group(const snort::Flow&) const;
    bool check_group(const snort::Packet*) const;
    bool check_split_group(const snort::Flow&) const;
    bool check_split_group(const snort::Packet*) const;
    bool check_address_space(const snort::Flow&) const;
    bool check_address_space(const snort::Packet*) const;
    bool check_tenant(const snort::Flow&) const;
    bool check_tenant(const snort::Packet*) const;
    bool check_service(const snort::Flow&) const;
    bool check_service(const char* service) const;
    bool check_service() const;
};

#endif

bind_module.h

Path = src/network_inspectors/binder/bind_module.h

#ifndef BIND_MODULE_H
#define BIND_MODULE_H

// binder management interface

#include "framework/module.h"
#include "binding.h"

#define BIND_NAME "binder"
#define BIND_HELP "configure processing based on CIDRs, ports, services, etc."

struct BindStats
{
    PegCount raw_packets;
    PegCount new_flows;
    PegCount rebinds;
    PegCount service_changes;
    PegCount assistant_inspectors;
    PegCount new_standby_flows;
    PegCount no_match;
    PegCount verdicts[BindUse::BA_MAX];
};

extern THREAD_LOCAL BindStats bstats;
extern THREAD_LOCAL snort::ProfileStats bindPerfStats;

class BinderModule : public snort::Module
{
public:
    BinderModule();
    ~BinderModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    // used to create default binder
    void add(const char* service, const char* type);
    void add(unsigned proto, const char* type);

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile() const override;

    std::vector<Binding>& get_bindings();
    std::vector<Binding>& get_policy_bindings();

    Usage get_usage() const override
    { return INSPECT; }

private:
    Binding binding;
    std::vector<Binding> bindings;
    std::vector<Binding> policy_bindings;
    std::string policy_filename;
    std::string policy_type;

    bool add_policy_file(const char* name, const char* type);
    void commit_binding();
    void commit_policy_binding();
};

#endif

network_inspectors/kaizen/

Snort ML is a neural network-based exploit detector for the Snort intrusion prevention system. It is designed to not only learn to detect known attacks from training data, but also learn to detect attacks it has never seen before.

Snort ML uses TensorFlow, included as LibML library.

Global configuration sets the trained network model to use. For example:

snort_ml_engine.http_param_model = { 'model.file' }

While per policy configuration sets data source and inspection depth in the selected Inspection policy. The following example enables two sources, HTTP URI and HTTP body:

snort_ml.uri_depth = -1
snort_ml.client_body_depth = 100

Trace messages are available:

  • trace.modules.snort_ml.classifier turns on messages from Snort ML

kaizen_engine.h

Path = src/network_inspectors/kaizen/kaizen_engine.h

#ifndef KAIZEN_ENGINE_H
#define KAIZEN_ENGINE_H

#include "framework/module.h"
#include "framework/inspector.h"


#define KZ_ENGINE_NAME "snort_ml_engine"
#define KZ_ENGINE_HELP "configure machine learning engine settings"

class BinaryClassifier;
struct KaizenEngineConfig
{
    std::string http_param_model_path;
};

class KaizenEngineModule : public snort::Module
{
public:
    KaizenEngineModule();

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;

    Usage get_usage() const override
    { return GLOBAL; }

    const KaizenEngineConfig& get_config()
    { return conf; }

private:
    KaizenEngineConfig conf;
};


class KaizenEngine : public snort::Inspector
{
public:
    KaizenEngine(const KaizenEngineConfig&);

    void show(const snort::SnortConfig*) const override;
    void eval(snort::Packet*) override {}

    void tinit() override;
    void tterm() override;

    void install_reload_handler(snort::SnortConfig*) override;

    static BinaryClassifier* get_classifier();

private:
    std::string read_model();
    bool validate_model();

    KaizenEngineConfig config;
    std::string http_param_model;
};


// Mock Classifier for tests if LibML absents.
// However, when REG_TEST is undefined, the entire code below won't be executed.
// Check the plugin type provided in kaizen_engine_api in the cc file
#ifndef HAVE_LIBML
class BinaryClassifier
{
public:
    bool build(const std::string& model)
    {
        pattern = model;
        return pattern != "error";
    }

    bool run(const char* ptr, size_t len, float& threshold)
    {
        std::string data(ptr, len);
        threshold = std::string::npos == data.find(pattern) ? 0.0f : 1.0f;
        return pattern != "fail";
    }

private:
    std::string pattern;
};
#endif

#endif

kaizen_inspector.h

Path = src/network_inspectors/kaizen/kaizen_inspector.h

#ifndef KAIZEN_INSPECTOR_H
#define KAIZEN_INSPECTOR_H

#include <string>
#include <utility>

#include "framework/inspector.h"

#include "kaizen_module.h"

class Kaizen : public snort::Inspector
{
public:
    Kaizen(const KaizenConfig& c) : config(c) { }

    void show(const snort::SnortConfig*) const override;
    void eval(snort::Packet*) override {}
    bool configure(snort::SnortConfig*) override;

    const KaizenConfig& get_config()
    { return config; }
private:
    KaizenConfig config;
};

#endif

kaizen_module.h

Path = src/network_inspectors/kaizen/kaizen_module.h

#ifndef KAIZEN_MODULE_H
#define KAIZEN_MODULE_H

#include "framework/module.h"
#include "main/thread.h"
#include "profiler/profiler.h"
#include "trace/trace_api.h"

#define KZ_GID 411
#define KZ_SID 1

#define KZ_NAME "snort_ml"
#define KZ_HELP "machine learning based exploit detector"

enum { TRACE_CLASSIFIER };

struct KaizenStats
{
    PegCount uri_alerts;
    PegCount client_body_alerts;
    PegCount uri_bytes;
    PegCount client_body_bytes;
    PegCount libml_calls;
};

extern THREAD_LOCAL KaizenStats kaizen_stats;
extern THREAD_LOCAL snort::ProfileStats kaizen_prof;
extern THREAD_LOCAL const snort::Trace* kaizen_trace;

struct KaizenConfig
{
    std::string http_param_model_path;
    double http_param_threshold;
    int32_t uri_depth;
    int32_t client_body_depth;
};

class KaizenModule : public snort::Module
{
public:
    KaizenModule();

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    const KaizenConfig& get_conf() const
    { return conf; }

    unsigned get_gid() const override
    { return KZ_GID; }

    const snort::RuleMap* get_rules() const override;

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;

    Usage get_usage() const override
    { return INSPECT; }

    snort::ProfileStats* get_profile() const override;

    void set_trace(const snort::Trace*) const override;
    const snort::TraceOption* get_trace_options() const override;

private:
    KaizenConfig conf = {};
};

#endif

network_inspectors/normalize/

Implement a large group of packet normalizations. See normalize.h for the entire set. If enabled/configured, examine each packet and make required modifications.

The primary utility is to repair protocol issues and provide a cleaner packet to the downstream network. This ensures that packets that Snort passes have the best likelihood of reaching their destination since otherwise attackers could craft packets that facilitate evasions. The normalizations are done before any related inspection and detection by Snort.

Process IP4, IP6, ICMP4, ICMP6, and TCP protocols. No UDP nor Ethernet level normalizations currently exist.

Maintain activity counts and print these counts as part of the Snort exit (or on-demand) counts.

If inline and able to perform packet replacement, replace the normalized packet in the output stream.

Note that TCP stream normalizations are done within the stream_tcp module. The configuration is done together with the above normalizations, however.

normalize.h

Path = src/network_inspectors/normalize/normalize.h

#ifndef NORMALIZE_H
#define NORMALIZE_H

// these control protocol specific normalizations
enum NormFlags
{
    NORM_IP4_BASE        = 0x00000001, // core ip4 norms
    //NORM_IP4_ID          = 0x00000002, // tbd:  encrypt ip id
    NORM_IP4_DF          = 0x00000004, // clear df
    NORM_IP4_RF          = 0x00000008, // clear rf
    NORM_IP4_TTL         = 0x00000010, // ensure min ttl
    NORM_IP4_TOS         = 0x00000020, // clear tos/diff-serv
    NORM_IP4_TRIM        = 0x00000040, // enforce min frame

    NORM_IP6_BASE        = 0x00000100, // core ip6 norms
    NORM_IP6_TTL         = 0x00000200, // ensure min hop limit
    NORM_ICMP4           = 0x00000400, // core icmp4 norms
    NORM_ICMP6           = 0x00000800, // core icmp6 norms

    NORM_TCP_ECN_PKT     = 0x00001000, // clear ece and cwr
    NORM_TCP_ECN_STR     = 0x00002000, // clear if not negotiated (stream)
    NORM_TCP_URP         = 0x00004000, // trim urp to dsize
    NORM_TCP_OPT         = 0x00008000, // nop over non-essential options
    NORM_TCP_IPS         = 0x00010000, // enable stream normalization/pre-ack flushing

    NORM_TCP_TRIM_SYN    = 0x00020000, // strip data from syn
    NORM_TCP_TRIM_RST    = 0x00040000, // strip data from rst
    NORM_TCP_TRIM_WIN    = 0x00080000, // trim to window
    NORM_TCP_TRIM_MSS    = 0x00100000, // trim to mss

    NORM_TCP_BLOCK       = 0x00200000, // enable tcp norms (used for normalizer indexing)
    NORM_TCP_RSV         = 0x00400000, // clear reserved bits
    NORM_TCP_PAD         = 0x00800000, // clear option padding bytes
    NORM_TCP_REQ_URG     = 0x01000000, // clear URP if URG = 0
    NORM_TCP_REQ_PAY     = 0x02000000, // clear URP/URG on no payload
    NORM_TCP_REQ_URP     = 0x04000000, // clear URG if URP is not set

    NORM_ALL             = 0xFFFFFFFF,  // all normalizations on
};

enum NormMode
{
    NORM_MODE_TEST,
    NORM_MODE_ON,
    NORM_MODE_MAX
};

bool Normalize_IsEnabled(NormFlags);
NormMode Normalize_GetMode(NormFlags);

#define NORM_IP4_ANY (0xFF)
#define NORM_IP6_ANY (NORM_IP6_BASE | NORM_IP6_TTL)
#define NORM_TCP_ANY (0xFFFF000)
#define NORM_TCP_TRIM_ANY (NORM_TCP_TRIM_SYN | NORM_TCP_TRIM_RST | \
    NORM_TCP_TRIM_WIN | NORM_TCP_TRIM_MSS)

#endif

norm.h

Path = src/network_inspectors/normalize/norm.h

#ifndef NORM_H
#define NORM_H

#include "protocols/packet_manager.h"
#include "normalize.h"

struct NormalizerConfig;
namespace snort
{
struct Packet;
}

// all normalizers look like this:
// the return is 1 if packet was changed, else 0
typedef int (* NormalFunc)( // FIXIT-L why is this exposed?
    struct NormalizerConfig*, snort::Packet*, uint8_t layer, int changes);

struct NormalizerConfig
{
    uint32_t normalizer_flags;
    uint8_t normalizer_options[32];

    // these must be in the same order PROTO_IDs are defined!
    // if entry is null, proto doesn't have normalization or it is disabled
    NormalFunc normalizers[snort::PacketManager::max_protocols()];
};

int Norm_SetConfig(NormalizerConfig*);
int Norm_Packet(NormalizerConfig*, snort::Packet*);

inline void Norm_Enable(NormalizerConfig* nc, NormFlags nf)
{
    nc->normalizer_flags |= nf;
}

inline void Norm_Disable(NormalizerConfig* nc, NormFlags nf)
{
    nc->normalizer_flags &= ~nf;
}

inline void Norm_Set(NormalizerConfig* nc, NormFlags nf, bool enable)
{
    if ( enable )
        Norm_Enable(nc, nf);
    else
        Norm_Disable(nc, nf);
}

inline bool Norm_IsEnabled(const NormalizerConfig* nc, NormFlags nf)
{
    return nc->normalizer_flags & nf;
}

inline void Norm_TcpPassOption(NormalizerConfig* nc, uint8_t opt)
{
    uint8_t byte = (opt >> 3), bit = (1 << (opt & 0x07));
    nc->normalizer_options[byte] |= bit;
}

inline void Norm_TcpDropOption(NormalizerConfig* nc, uint8_t opt)
{
    uint8_t byte = (opt >> 3), bit = (1 << (opt & 0x07));
    nc->normalizer_options[byte] &= ~bit;
}

inline bool Norm_TcpIsOptional(const NormalizerConfig* nc, uint8_t opt)
{
    uint8_t byte = (opt >> 3), bit = (1 << (opt & 0x07));
    return nc->normalizer_options[byte] & bit;
}

#endif

norm_module.h

Path = src/network_inspectors/normalize/norm_module.h

#ifndef NORM_MODULE_H
#define NORM_MODULE_H

#include "framework/module.h"
#include "norm.h"

#define NORM_NAME "normalizer"
#define NORM_HELP "packet scrubbing for inline mode"

extern THREAD_LOCAL snort::ProfileStats norm_perf_stats;

class NormalizeModule : public snort::Module
{
public:
    NormalizeModule();
    ~NormalizeModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;

    snort::ProfileStats* get_profile() const override;
    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;

    const NormalizerConfig* get_config()
    { return &config; }

    Usage get_usage() const override
    { return CONTEXT; }

private:
    bool set_ip4(const char*, const snort::Value&, snort::SnortConfig*);
    bool set_tcp(const char*, const snort::Value&, snort::SnortConfig*);

    void add_test_peg(const PegInfo&) const;

    NormalizerConfig config;

    static std::vector<const std::string*> test_text;
    static std::vector<PegInfo> test_pegs;
};

#endif

norm_stats.h

Path = src/network_inspectors/normalize/norm_stats.h

#ifndef NORM_STATS_H
#define NORM_STATS_H

#include "framework/counts.h"
#include "main/thread.h"

#include "normalize.h"

enum PegCounts
{
    PC_IP4_TRIM,
    PC_IP4_TOS,
    PC_IP4_DF,
    PC_IP4_RF,
    PC_IP4_TTL,
    PC_IP4_OPTS,
    PC_ICMP4_ECHO,
    PC_IP6_TTL,
    PC_IP6_OPTS,
    PC_ICMP6_ECHO,
    PC_TCP_SYN_OPT,
    PC_TCP_OPT,
    PC_TCP_PAD,
    PC_TCP_RSV,
    PC_TCP_NS,
    PC_TCP_URP,
    PC_TCP_ECN_PKT,
    PC_TCP_TS_ECR,
    PC_TCP_REQ_URG,
    PC_TCP_REQ_PAY,
    PC_TCP_REQ_URP,

    // These peg counts are shared with stream_tcp
    PC_TCP_TRIM_SYN,
    PC_TCP_TRIM_RST,
    PC_TCP_TRIM_WIN,
    PC_TCP_TRIM_MSS,
    PC_TCP_ECN_SSN,
    PC_TCP_TS_NOP,
    PC_TCP_IPS_DATA,
    PC_TCP_BLOCK,

    PC_MAX
};

#define TCP_PEGS_START PC_TCP_TRIM_SYN

extern THREAD_LOCAL PegCount norm_stats[PC_MAX][NORM_MODE_MAX];
extern const PegInfo norm_names[];

#endif

network_inspectors/packet_capture/

capture_module.h

Path = src/network_inspectors/packet_capture/capture_module.h

#ifndef CAPTURE_MODULE_H
#define CAPTURE_MODULE_H

#include "framework/module.h"

#define CAPTURE_NAME "packet_capture"
#define CAPTURE_HELP "raw packet dumping facility"

struct CaptureConfig
{
    bool enabled;
    int16_t group;
    std::string filter;
    std::vector<uint32_t> tenants;
};

struct CaptureStats
{
    PegCount checked;
    PegCount matched;
};

class CaptureModule : public snort::Module
{
public:
    CaptureModule();

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile() const override;
    const snort::Command* get_commands() const override;
    bool set(const char*, snort::Value&, snort::SnortConfig*) override;

    void get_config(CaptureConfig&);

    Usage get_usage() const override
    { return GLOBAL; }

private:
    CaptureConfig config;
};

extern THREAD_LOCAL CaptureStats cap_count_stats;
extern THREAD_LOCAL snort::ProfileStats cap_prof_stats;

#endif

packet_capture.h

Path = src/network_inspectors/packet_capture/packet_capture.h

#ifndef PACKET_CAPTURE_H
#define PACKET_CAPTURE_H

#include <string>

void packet_capture_enable(const std::string&, const int16_t g = -1, const std::string& t = "");
void packet_capture_disable();

#endif

network_inspectors/packet_tracer/

packet_tracer.h

Path = src/network_inspectors/packet_tracer/packet_tracer.h

#ifndef PACKET_TRACER_H
#define PACKET_TRACER_H

#include <cstdarg>
#include <cstdio>
#include <cstring>
#include <vector>

#include "framework/packet_constraints.h"
#include "main/snort_types.h"
#include "main/thread.h"
#include "protocols/ipv6.h"
#include "protocols/protocol_ids.h"
#include "sfip/sf_ip.h"
#include "time/clock_defs.h"
#include "time/stopwatch.h"

namespace snort
{
struct Packet;

class PacketTracer
{
public:
    PacketTracer() = default;
    virtual ~PacketTracer();

    typedef uint8_t TracerMute;
    static const int max_buff_size = 2048;

    // static functions
    static void set_log_file(const std::string&);
    static void thread_init();
    static void thread_term();

    static void dump(char* output_buff, unsigned int len);
    static void dump(Packet*);
    static void daq_dump(Packet*);

    static void configure(bool status, const std::string& file_name);
    static void set_constraints(const PacketConstraints* constraints);
    static void activate(const snort::Packet&);

    static SO_PUBLIC void pause();
    static SO_PUBLIC void unpause();
    static SO_PUBLIC bool is_paused();
    static SO_PUBLIC bool is_active();
    static SO_PUBLIC bool is_daq_activated();

    static SO_PUBLIC TracerMute get_mute();

    static SO_PUBLIC void log(const char* format, ...) __attribute__((format (printf, 1, 2)));
    static SO_PUBLIC void log(TracerMute, const char* format, ...) __attribute__((format (printf, 2, 3)));

    static SO_PUBLIC void daq_log(const char* format, ...) __attribute__((format (printf, 1, 2)));
    static SO_PUBLIC void pt_timer_start();

protected:


    // non-static variable
    FILE* log_fh = stdout;
    std::vector<bool> mutes;
    char buffer[max_buff_size] = {0};
    unsigned buff_len = 0;
    char daq_buffer[max_buff_size] = {0};
    unsigned daq_buff_len = 0;

    unsigned pause_count = 0;
    bool user_enabled = false;
    bool daq_activated = false;
    bool shell_enabled = false;
    bool active = false;

    std::string debugstr;
    PacketConstraints constraints;

    // static functions
    template<typename T = PacketTracer> static void _thread_init();

    // non-static functions
    void log_va(const char*, va_list, bool);
    void populate_buf(const char*, va_list, char*, uint32_t&);
    void add_ip_header_info(const snort::Packet&);
    void add_eth_header_info(const snort::Packet&);
    void add_packet_type_info(const snort::Packet&);
    void update_constraints(const PacketConstraints* constraints);
    const char *get_debug_session() { return debugstr.c_str(); }

    virtual void open_file();
    virtual void dump_to_daq(Packet*);
    virtual void reset(bool);
};

SO_PUBLIC extern THREAD_LOCAL PacketTracer* s_pkt_trace;
SO_PUBLIC extern THREAD_LOCAL Stopwatch<SnortClock>* pt_timer;

inline bool PacketTracer::is_active()
{ return s_pkt_trace ? s_pkt_trace->active : false; }

inline bool PacketTracer::is_daq_activated()
{ return s_pkt_trace ? s_pkt_trace->daq_activated : false; }

struct SO_PUBLIC PacketTracerSuspend
{
    PacketTracerSuspend()
    { PacketTracer::pause(); }

    ~PacketTracerSuspend()
    { PacketTracer::unpause(); }
};

}
#endif

packet_tracer_module.h

Path = src/network_inspectors/packet_tracer/packet_tracer_module.h

#ifndef PACKET_TRACER_MODULE_H
#define PACKET_TRACER_MODULE_H

#include "framework/module.h"

#define PACKET_TRACER_NAME "packet_tracer"
#define PACKET_TRACER_HELP "generate debug trace messages for packets"

struct PacketTracerConfig
{
    bool enabled;
    std::string file;
};

class PacketTracerModule : public snort::Module
{
public:
    PacketTracerModule();


    enum PacketTraceOutput
    {
        PACKET_TRACE_CONSOLE,
        PACKET_TRACE_FILE
    };

    const snort::Command* get_commands() const override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    Usage get_usage() const override
    { return GLOBAL; }

private:
    PacketTracerConfig* config = nullptr;
};

#endif

network_inspectors/perf_monitor/

Implement a comprehensive performance collection, processing, and logging facility.

There are two broad classes of statistics: Basic packet processing information and Flow statistics.

Basic performance statistics include items such as event counts, packet counts, stream counts, dropped packet counter, processor load, etc. The Flow statistics count flows between IP addresses.

The general data flow is:

  1. Process the relevant information from each packet as it’s being inspected. Increment local counters.

  2. At a configured interval, process the counters into the statistics.

  3. At same processing interval, Log these statistics to a file.

  4. Switch to another log file for the next cycle.

The type of statistics collection/processing/logging is controlled via configuration.

perf_monitor is implemented as a network_inspector to give it access to the per-packet processing mechanism. It doesn’t perform inspection in the broad sense, but rather collects and logs information.

Statistics gathering is performed by the PerfTracker classes. Each class acts a separate module for gathering the different forms of statistics. The PerfTracker classes pass their data into one of formatter classes, which in turn format the data for output to console or to disk.

Currently output formats are:

  1. Human-readable text

  2. CSV

  3. JSON

File Layout
Section Description

File Header

File ID and schema for parsing. This appears once at the start.

Records

This is a stream of records. There may be an unlimited number.

File Header
Field Name Size Description

Magic

4 bytes

Format identifier. Only "FLTI" exists.

Schema Size

4 bytes

Size of the included schema.

Schema

(schema size) bytes

Schema for parsing records in this file.

Record
Field Name Size Description

Timestamp

8 bytes

The time this record was written

Record Size

4 bytes

Size of the record to follow

Record

(record size) bytes

Binary record. Parse against file schema.

base_tracker.h

Path = src/network_inspectors/perf_monitor/base_tracker.h

#ifndef BASE_TRACKER_H
#define BASE_TRACKER_H

#include "perf_tracker.h"

class BaseTracker : public PerfTracker
{
public:
    BaseTracker(PerfConfig* perf);

    void process(bool) override;

private:
    std::vector<ModuleConfig> modules;
    std::vector<snort::Module*> mods_to_prep;
};

#endif

cpu_tracker.h

Path = src/network_inspectors/perf_monitor/cpu_tracker.h

#ifndef CPU_TRACKER_H
#define CPU_TRACKER_H

#include "perf_tracker.h"

class CPUTracker : public PerfTracker
{
public:
    CPUTracker(PerfConfig*);
    void reset() override;
    void process(bool) override;

protected:
    virtual void get_clocks(struct timeval& user_time,
        struct timeval& sys_time, struct timeval& wall_time);

private:
    //19 bits for microseconds
    //45 bits for seconds (out to year 1116918)
    uint64_t last_wt = 0;
    uint64_t last_ut = 0;
    uint64_t last_st = 0;

    PegCount user_stat;
    PegCount system_stat;
    PegCount wall_stat;

    void get_times(uint64_t& user, uint64_t& system, uint64_t& wall);
};

#endif

csv_formatter.h

Path = src/network_inspectors/perf_monitor/csv_formatter.h

#ifndef CSV_FORMATTER_H
#define CSV_FORMATTER_H

#include <ctime>

#include "perf_formatter.h"

class CSVFormatter : public PerfFormatter
{
public:
    CSVFormatter(const std::string& tracker_name) : PerfFormatter(tracker_name) {}

    const char* get_extension() override
    { return ".csv"; }

    void finalize_fields() override;
    void init_output(FILE*) override;
    void write(FILE*, time_t) override;

private:
    std::string header;
};

#endif

flow_ip_tracker.h

Path = src/network_inspectors/perf_monitor/flow_ip_tracker.h

#ifndef FLOW_IP_TRACKER_H
#define FLOW_IP_TRACKER_H

#include "hash/xhash.h"

#include "perf_tracker.h"

enum FlowState
{
    SFS_STATE_TCP_ESTABLISHED = 0,
    SFS_STATE_TCP_CLOSED,
    SFS_STATE_UDP_CREATED,
    SFS_STATE_MAX
};

enum FlowType
{
    SFS_TYPE_TCP = 0,
    SFS_TYPE_UDP,
    SFS_TYPE_OTHER,
    SFS_TYPE_MAX
};

struct TrafficStats
{
    PegCount  packets_a_to_b;
    PegCount  bytes_a_to_b;
    PegCount  packets_b_to_a;
    PegCount  bytes_b_to_a;
};

struct FlowStateValue
{
    TrafficStats traffic_stats[SFS_TYPE_MAX];
    PegCount total_packets;
    PegCount total_bytes;
    PegCount state_changes[SFS_STATE_MAX];
};

class FlowIPTracker : public PerfTracker
{
public:
    FlowIPTracker(PerfConfig* perf);
    ~FlowIPTracker() override;

    bool initialize(size_t new_memcap);
    void reset() override;
    void update(snort::Packet*) override;
    void process(bool) override;
    int update_state(const snort::SfIp* src_addr, const snort::SfIp* dst_addr, FlowState);
    snort::XHash* get_ip_map()
        { return ip_map; }

private:
    FlowStateValue stats;
    snort::XHash* ip_map;
    char ip_a[41], ip_b[41];
    int perf_flags;
    PerfConfig* perf_conf;
    size_t memcap;
    FlowStateValue* find_stats(const snort::SfIp* src_addr, const snort::SfIp* dst_addr, int* swapped);
    void write_stats();
    void display_stats();

};
#endif

flow_tracker.h

Path = src/network_inspectors/perf_monitor/flow_tracker.h

#ifndef FLOW_TRACKER_H
#define FLOW_TRACKER_H

#include "perf_tracker.h"

struct FlowProto
{
    std::vector<PegCount> src;
    std::vector<PegCount> dst;
    PegCount high = 0;
};

class FlowTracker : public PerfTracker
{
public:
    FlowTracker(PerfConfig* perf);

    void update(snort::Packet*) override;
    void process(bool) override;

protected:
    virtual void clear();

private:
    PegCount byte_total = 0;
    int flow_max_port_to_track;

    std::vector<PegCount> pkt_len_cnt;
    PegCount pkt_len_oversize_cnt = 0;

    FlowProto udp;
    FlowProto tcp;

    std::vector<PegCount> type_icmp;

    void update_transport_flows(int sport, int dport,
        FlowProto& proto, int len);
};

#endif

json_formatter.h

Path = src/network_inspectors/perf_monitor/json_formatter.h

#ifndef JSON_FORMATTER_H
#define JSON_FORMATTER_H

#include "perf_formatter.h"

class JSONFormatter : public PerfFormatter
{
public:
    using PerfFormatter::PerfFormatter;

    const char* get_extension() override
    { return ".json"; }

    bool allow_append() override
    { return false; }

    void init_output(FILE*) override;
    void write(FILE*, time_t) override;
    void finalize_output(FILE*) override;

private:
    bool first_write = true;
    bool initialized = false;
};

#endif

perf_formatter.h

Path = src/network_inspectors/perf_monitor/perf_formatter.h

#ifndef PERF_FORMATTER_H
#define PERF_FORMATTER_H

//
// PerfFormatter provides an API for PerfTrackers to use for reporting data.
// The basic flow from the perspective of a PerfTracker is:
//
// 1. Call register_section to create a section of stats
//
// 2. Call register_field to insert a field into the most recently created
//    section. Fields should always be pointers to stable locations in memory,
//    as they cannot be updated. Data will be pulled from these pointers when
//    writes occur.
//
// 3. Call finalize_fields to complete section and field registration. This can
//    only be called once per instance.
//
// 4. Set the values desired for fields.
//
// 5. Call write to output the current values in each field.
//
// init_output should be implemented where metadata needs to be written on
// output open.
//

#include <ctime>
#include <string>
#include <vector>

#include "framework/counts.h"

union FormatterValue
{
    PegCount* pc;
    const char* s;
    std::vector<PegCount>* ipc;
};

enum FormatterType : uint8_t
{
    FT_PEG_COUNT,
    FT_STRING,
    FT_IDX_PEG_COUNT
};

class PerfFormatter
{
public:
    PerfFormatter(const std::string& tracker_name)
    { this->tracker_name = tracker_name; }
    virtual ~PerfFormatter() = default;

    virtual bool allow_append()
    { return true; }

    virtual const char* get_extension()
    { return ""; }

    virtual std::string get_tracker_name() final
    { return tracker_name; }

    virtual void register_section(const std::string&);
    virtual void register_field(const std::string&, PegCount*);
    virtual void register_field(const std::string&, const char*);
    virtual void register_field(const std::string&, std::vector<PegCount>*);
    virtual void finalize_fields() {}
    virtual void init_output(FILE*) {}
    virtual void write(FILE*, time_t) = 0;
    virtual void finalize_output(FILE*) {}

protected:
    std::vector<std::vector<FormatterType>> types;
    std::vector<std::vector<FormatterValue>> values;

    std::vector<std::string> section_names;
    std::vector<std::vector<std::string>> field_names;

    unsigned last_section = -1;

private:
    std::string tracker_name;
};

#ifdef UNIT_TEST
#include <map>

class MockFormatter : public PerfFormatter
{
public:
    std::map<std::string, FormatterValue> public_values;

    MockFormatter(const std::string& tracker_name) : PerfFormatter(tracker_name) {}

    void write(FILE*, time_t) override
    {
        for( unsigned i = 0; i < values.size(); i++ )
            for( unsigned j = 0; j < values[i].size(); j++ )
                public_values.emplace(section_names[i] + "." + field_names[i][j], values[i][j]);
   }
};
#endif

#endif

perf_module.h

Path = src/network_inspectors/perf_monitor/perf_module.h

#ifndef PERF_MODULE_H
#define PERF_MODULE_H

#include <unordered_map>

#include "framework/module.h"

#include "perf_pegs.h"
#include "perf_reload_tuner.h"

#define PERF_NAME "perf_monitor"
#define PERF_HELP "performance monitoring and flow statistics collection"

// Perf Flags
#define PERF_BASE       0x00000001
#define PERF_CPU        0x00000002
#define PERF_FLOW       0x00000004
#define PERF_FLOWIP     0x00000008
#define PERF_SUMMARY    0x00000010

#define ROLLOVER_THRESH     512
#define MAX_PERF_FILE_SIZE  UINT64_MAX
#define MIN_PERF_FILE_SIZE  4096

enum class PerfFormat
{
    CSV,
    TEXT,
    JSON,
    MOCK
};

enum class PerfOutput
{
    TO_FILE,
    TO_CONSOLE
};

struct ModuleConfig
{
    // state optimized for run time using indices
    // can't be determined until all modules have loaded (PerfMonitor::configure)
    snort::Module* ptr;
    IndexVec pegs;

    void set_name(const std::string& name);
    void set_peg_names(snort::Value& peg_names);
    bool confirm_parse();
    bool resolve();

private:
    std::string name;
    std::unordered_map<std::string, bool> peg_names;
};

struct PerfConstraints
{
    bool flow_ip_enabled = false;
    unsigned sample_interval = 0;
    uint32_t pkt_cnt = 0;

    PerfConstraints() = default;
    PerfConstraints(bool en, unsigned interval, uint32_t cnt) :
        flow_ip_enabled(en), sample_interval(interval), pkt_cnt(cnt) { }
};

struct PerfConfig
{
    int perf_flags = 0;
    uint32_t pkt_cnt = 0;
    unsigned sample_interval = 0;
    uint64_t max_file_size = 0;
    int flow_max_port_to_track = 0;
    size_t flowip_memcap = 0;
    PerfFormat format = PerfFormat::CSV;
    PerfOutput output = PerfOutput::TO_FILE;
    std::vector<ModuleConfig> modules;
    std::vector<snort::Module*> mods_to_prep;
    PerfConstraints* constraints;

    PerfConfig() { constraints = new PerfConstraints; }
    ~PerfConfig() { delete constraints; }

    bool resolve();
};

/* The Module Class for incorporation into Snort++ */
class PerfMonModule : public snort::Module
{
public:
    PerfMonModule();
    ~PerfMonModule() override;

    const snort::Command* get_commands() const override;
    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile() const override;

    PerfConfig* get_config();
#ifdef UNIT_TEST
    void set_config(PerfConfig* ptr) { config = ptr; }
#endif

    Usage get_usage() const override
    { return GLOBAL; }

private:
    PerfConfig* config = nullptr;
};

extern THREAD_LOCAL PerfPegStats pmstats;
extern THREAD_LOCAL snort::ProfileStats perfmonStats;

#endif

perf_monitor.h

Path = src/network_inspectors/perf_monitor/perf_monitor.h

#ifndef PERF_MONITOR_H
#define PERF_MONITOR_H

#include "managers/inspector_manager.h"
#include "protocols/packet.h"

#include "base_tracker.h"
#include "cpu_tracker.h"
#include "flow_ip_tracker.h"
#include "flow_tracker.h"
#include "perf_module.h"

class FlowIPDataHandler;

class PerfMonitor : public snort::Inspector
{
public:
    PerfMonitor(PerfConfig*);
    ~PerfMonitor() override { delete config; }

    bool configure(snort::SnortConfig*) override;
    void show(const snort::SnortConfig*) const override;

    void eval(snort::Packet*) override;
    bool ready_to_process(snort::Packet* p);

    void tinit() override;
    void tterm() override;

    void rotate();

    void swap_constraints(PerfConstraints*);
    PerfConstraints* get_original_constraints();

    void enable_profiling(PerfConstraints*);
    void disable_profiling(PerfConstraints*);

    FlowIPTracker* get_flow_ip();

    inline PerfConstraints* get_constraints()
    { return config->constraints; }

    inline bool is_flow_ip_enabled()
    { return config->constraints->flow_ip_enabled; }

private:
    PerfConfig* const config;
    void disable_tracker(size_t);
};

#endif

perf_pegs.h

Path = src/network_inspectors/perf_monitor/perf_pegs.h

#ifndef PERF_PEGS_H
#define PERF_PEGS_H

#include "framework/counts.h"
#include "main/snort_types.h"

static const PegInfo perf_module_pegs[] =
{
    { CountType::SUM, "packets", "total packets processed by performance monitor" },
    { CountType::SUM, "flow_tracker_creates", "total number of flow trackers created" },
    { CountType::SUM, "flow_tracker_total_deletes", "flow trackers deleted to stay below memcap limit" },
    { CountType::SUM, "flow_tracker_reload_deletes", "flow trackers deleted due to memcap change on config reload" },
    { CountType::SUM, "flow_tracker_prunes", "flow trackers pruned for reuse by new flows" },
    { CountType::END, nullptr, nullptr },
};

struct PerfPegStats
{
    PegCount total_packets;
    PegCount flow_tracker_creates;
    PegCount flow_tracker_total_deletes;
    PegCount flow_tracker_reload_deletes;
    PegCount flow_tracker_prunes;
};

#endif

perf_reload_tuner.h

Path = src/network_inspectors/perf_monitor/perf_reload_tuner.h

#ifndef PERF_RELOAD_TUNER_H
#define PERF_RELOAD_TUNER_H

#include "main/reload_tuner.h"

class PerfMonReloadTuner : public snort::ReloadResourceTuner
{
public:
    explicit PerfMonReloadTuner(size_t memcap) : memcap(memcap) { }
    ~PerfMonReloadTuner() override = default;

    bool tinit() override;

    bool tune_idle_context() override
        { return tune_resources(max_work_idle); }

    bool tune_packet_context() override
        { return tune_resources(max_work); }

    bool tune_resources(unsigned work_limit);

private:
    size_t memcap;

};

#endif

perf_tracker.h

Path = src/network_inspectors/perf_monitor/perf_tracker.h

#ifndef PERF_TRACKER_H
#define PERF_TRACKER_H

//
// This class defines the data gathering layer of perfmon. PerfMonitor will
// create an instance of each configured class for each packet processing
// thread. Subclasses of PerfTrackers should implement or call the following
// methods, leaving the others for internal use by PerfMonitor:
//
// reset() - perform initialization after the output handle has been opened.
//
// update(Packet*) - update statistics based on the current packet.
//
// process(bool) - summarize data and report. This is called after the
// reporting thresholds have been reached.
//
// write() - tell the configured PerfFormatter to output the current stats
//

#include <cstdio>
#include <ctime>

#include "perf_formatter.h"
#include "perf_module.h"

namespace snort
{
struct Packet;
}

class PerfTracker
{
public:
    virtual ~PerfTracker();

    virtual void reset() { }
    virtual void process(bool /*summary*/) { } // FIXIT-M get rid of this step.
    virtual void update(snort::Packet*) { }
    virtual void update_time(time_t time) { cur_time = time; }
    virtual const std::string& get_name() { return tracker_name; }

    bool open(bool append);
    void close();
    bool rotate();
    bool auto_rotate();
    bool is_open() { return fh != nullptr; }

    PerfTracker(const PerfTracker&) = delete;
    PerfTracker& operator=(const PerfTracker&) = delete;

protected:
    PerfTracker(PerfConfig*, const char* tracker_name);
    virtual void write();

    uint64_t max_file_size = 0;
    PerfFormatter* formatter = nullptr;

private:
    std::string fname;
    std::string tracker_name;
    FILE* fh = nullptr;
    time_t cur_time = 0;
};
#endif

text_formatter.h

Path = src/network_inspectors/perf_monitor/text_formatter.h

#ifndef TEXT_FORMATTER_H
#define TEXT_FORMATTER_H

#include "perf_formatter.h"

class TextFormatter : public PerfFormatter
{
public:
    TextFormatter(const std::string& tracker_name) : PerfFormatter(tracker_name) {}

    const char* get_extension() override
    { return ".txt"; }

    void write(FILE*, time_t) override;
};

#endif

network_inspectors/port_scan/

A tool used to locate and alert port scanning activity.

The configuration is in two parts. The _global entity is use to provide parameters for the entire collection of port scanners. Then one can configure many port scanners with distinct characteristics.

port_scan is implemented as a network inspector module to give it access to all packets and be able to inspect the IP level information (addresses, protocols, ports, etc.)

port_scan still crafts packets so that its information can be logged. That is no longer necessary however, and when port_scan is rewritten, it will only log the relevant information as data.

The low, medium, and high thresholds and sense levels are hard-coded in ps_detect.cc.

Here are notes from the original (Snort) portscan.c:

The philosophy of portscan detection that we use is based on a generic network attack methodology: reconnaissance, network service enumeration, and service exploitation.

The reconnaissance phase determines what types of network protocols and services that a host supports. This is the traditional phase where a portscan occurs. An important requirement of this phase is that an attacker does not already know what protocols and services are supported by the destination host. If an attacker does know what services are open on the destination host then there is no need for this phase. Because of this requirement, we assume that if an attacker engages in this phase that they do not have prior knowledge to what services are open. So, the attacker will need to query the ports or protocols they are interested in. Most or at least some of these queries will be negative and take the form of either an invalid response (TCP RSTs, ICMP unreachables) or no response (in which case the host is firewalled or filtered). We detect portscans from these negative queries.

The primary goal of this portscan detection engine is to catch nmap and variant scanners. The engine tracks connection attempts on TCP, UDP, ICMP, and IP Protocols. If there is a valid response, the connection is marked as valid. If there is no response or a invalid response (TCP RST), then we track these attempts separately, so we know the number of invalid responses and the number of connection attempts that generated no response. These two values differentiate between a normal scan and a filtered scan.

We detect four different scan types, and each scan type has its own negative query characteristics. This is how we determine what type of scan we are seeing. The different scans are:

  • Portscan

  • Decoy Portscan

  • Distributed Portscan

  • Portsweep

Portscan: A portscan is a basic one host to one host scan where multiple ports are scanned on the destination host. We detect these scans by looking for a low number of hosts that contacted the destination host and a high number of unique ports and a high number of invalid responses or connections.

Distributed Portscan: A distributed portscan occurs when many hosts connect to a single destination host and multiple ports are scanned on the destination host. We detect these scans by looking for a high number of hosts that contacted the destination host and a high number of unique ports with a high number of invalid responses or connections.

Decoy Portscan: A decoy portscan is a variation on a distributed portscan, the difference being that a decoy portscan connects to a single port multiple times. This shows up in the unique port count that is tracked. There’s still many hosts connecting to the destination host.

Portsweep: A portsweep is a basic one host to many host scan where one to a few ports are scanned on each host. We detect these scans by looking at src hosts for a high number of contacted hosts and a low number of unique ports with a high number of invalid responses or connections.

Each of these scans can also be detected as a filtered portscan, or a portscan where there wasn’t invalid responses and the responses have been firewalled in some way.

ipobj.h

Path = src/network_inspectors/port_scan/ipobj.h

#ifndef IPOBJ_H
#define IPOBJ_H

/*
    IP address encapsulation interface

    This module provides encapsulation of single IP ADDRESSes as objects,
    and collections of IP ADDRESSes as objects

    Interaction with this library should be done in HOST byte order.
*/

#include "sfip/sf_cidr.h"
#include "utils/sflsq.h"

struct PORTRANGE
{
    unsigned port_lo;
    unsigned port_hi;
};

struct PORTSET
{
    SF_LIST port_list;
};

struct IP_PORT
{
    snort::SfCidr ip;
    PORTSET portset;
    char notflag;
};

struct IPSET
{
    SF_LIST ip_list;
};

/*
  IP ADDRESS SET OBJECTS

   Snort Accepts:

    IP-Address          192.168.1.1
    IP-Address/MaskBits 192.168.1.0/24
    IP-Address/Mask     192.168.1.0/255.255.255.0

   These can all be handled via the CIDR block notation : IP/MaskBits

   We use collections (lists) of cidr blocks to represent address blocks
   and individual addresses.

   For a single IPAddress the implied Mask is 32 bits,or
   255.255.255.255, or 0xffffffff, or -1.
*/
IPSET* ipset_new();
int ipset_add(IPSET* ipset, snort::SfCidr* ip, void* port, int notflag);
int ipset_contains(IPSET* ipset, const snort::SfIp* ip, void* port);
void ipset_free(IPSET* ipset);

// helper functions -- all the sets work in host order
int ipset_parse(IPSET* ipset, const char* ipstr);

#endif

ps_detect.h

Path = src/network_inspectors/port_scan/ps_detect.h

#ifndef PS_DETECT_H
#define PS_DETECT_H

#include <sys/time.h>

#include <ctime>

#include "sfip/sf_ip.h"
#include "ipobj.h"

namespace snort
{
struct Packet;
}

#define PS_OPEN_PORTS 8

#define PS_PROTO_NONE        0x00
#define PS_PROTO_TCP         0x01
#define PS_PROTO_UDP         0x02
#define PS_PROTO_ICMP        0x04
#define PS_PROTO_IP          0x08
#define PS_PROTO_ALL         0x0f

#define PS_PROTO_OPEN_PORT   0x80

#define PS_TYPE_PORTSCAN     0x01
#define PS_TYPE_PORTSWEEP    0x02
#define PS_TYPE_DECOYSCAN    0x04
#define PS_TYPE_DISTPORTSCAN 0x08
#define PS_TYPE_ALL          0x0f

#define PS_SENSE_HIGH        3
#define PS_SENSE_MEDIUM      2
#define PS_SENSE_LOW         1

#define PS_ALERT_ONE_TO_ONE                1
#define PS_ALERT_ONE_TO_ONE_DECOY          2
#define PS_ALERT_PORTSWEEP                 3
#define PS_ALERT_DISTRIBUTED               4
#define PS_ALERT_ONE_TO_ONE_FILTERED       5
#define PS_ALERT_ONE_TO_ONE_DECOY_FILTERED 6
#define PS_ALERT_DISTRIBUTED_FILTERED      7
#define PS_ALERT_PORTSWEEP_FILTERED        8

#define PS_ALERT_GENERATED                 255

//-------------------------------------------------------------------------

struct PS_ALERT_CONF
{
    short connection_count;
    short priority_count;
    short u_ip_count;
    short u_port_count;
};

struct PortscanConfig
{
    size_t memcap;

    int detect_scans;
    int detect_scan_type;
    int proto_cnt;
    int include_midstream;
    int print_tracker;

    bool alert_all;
    bool logfile;

    unsigned tcp_window;
    unsigned udp_window;
    unsigned ip_window;
    unsigned icmp_window;

    IPSET* ignore_scanners;
    IPSET* ignore_scanned;
    IPSET* watch_ip;

    PS_ALERT_CONF tcp_ports;
    PS_ALERT_CONF tcp_decoy;
    PS_ALERT_CONF tcp_sweep;
    PS_ALERT_CONF tcp_dist;

    PS_ALERT_CONF udp_ports;
    PS_ALERT_CONF udp_decoy;
    PS_ALERT_CONF udp_sweep;
    PS_ALERT_CONF udp_dist;

    PS_ALERT_CONF ip_proto;
    PS_ALERT_CONF ip_decoy;
    PS_ALERT_CONF ip_sweep;
    PS_ALERT_CONF ip_dist;

    PS_ALERT_CONF icmp_sweep;

    PortscanConfig();
    ~PortscanConfig();
};

struct PS_PROTO
{
    int connection_count;
    int priority_count;
    int u_ip_count;
    int u_port_count;

    unsigned short high_p;
    unsigned short low_p;
    unsigned short u_ports;

    snort::SfIp high_ip;
    snort::SfIp low_ip;
    snort::SfIp u_ips;

    unsigned short open_ports[PS_OPEN_PORTS];
    unsigned char open_ports_cnt;

    unsigned char alerts;

    time_t window;
};

struct PS_TRACKER
{
    int priority_node;
    int protocol;
    PS_PROTO proto;
};

struct PS_PKT
{
    snort::Packet* pkt;

    PS_TRACKER* scanner;
    PS_TRACKER* scanned;

    int proto;
    int reverse_pkt;

    PS_PKT(snort::Packet*);
};

void ps_cleanup();
void ps_reset();
void ps_update_memusage_peg();

unsigned ps_node_size();
bool ps_init_hash(unsigned long);
bool ps_prune_hash(unsigned);
int ps_detect(PS_PKT*);

#endif

ps_inspect.h

Path = src/network_inspectors/port_scan/ps_inspect.h

#ifndef PS_INSPECT_H
#define PS_INSPECT_H

// The PortScan inspector is declared here.  The public methods and
// alerting methods are defined in port_scan.cc and the detection methods
// are in ps_detect.cc.

#include "framework/inspector.h"
#include "ps_detect.h"

namespace snort
{
struct SfIp;
}
struct PS_PROTO;
struct PS_TRACKER;
struct PS_PKT;

class PortScan : public snort::Inspector
{
public:
    PortScan(class PortScanModule*);
    ~PortScan() override;

    void show(const snort::SnortConfig*) const override;
    void eval(snort::Packet*) override;

    void tinit() override;
    void tterm() override;

private:
    void ps_parse(snort::SnortConfig*, char*);

    bool ps_ignore_ip( const snort::SfIp* scanner, uint16_t scanner_port,
        const snort::SfIp* scanned, uint16_t scanned_port);

    bool ps_filter_ignore(PS_PKT*);
    int ps_get_proto(PS_PKT*, int* proto);
    int ps_detect(PS_PKT*);

    bool ps_tracker_lookup(PS_PKT*, PS_TRACKER** scanner, PS_TRACKER** scanned);
    bool ps_tracker_update(PS_PKT*, PS_TRACKER* scanner, PS_TRACKER* scanned);
    bool ps_tracker_alert(PS_PKT*, PS_TRACKER* scanner, PS_TRACKER* scanned);

    void ps_proto_update_window(unsigned window, PS_PROTO*, time_t pkt_time);

    int ps_proto_update( PS_PROTO*, int ps_cnt, int pri_cnt, unsigned window, const snort::SfIp* ip,
        unsigned short port, time_t pkt_time);

    void ps_tracker_update_ip(PS_PKT*, PS_TRACKER* scanner, PS_TRACKER* scanned);
    void ps_tracker_update_tcp(PS_PKT*, PS_TRACKER* scanner, PS_TRACKER* scanned);
    void ps_tracker_update_udp(PS_PKT*, PS_TRACKER* scanner, PS_TRACKER* scanned);
    void ps_tracker_update_icmp(PS_PKT*, PS_TRACKER* scanner, PS_TRACKER* scanned);

    void ps_alert_ip(PS_PROTO* scanner, PS_PROTO* scanned);
    void ps_alert_tcp(PS_PROTO* scanner, PS_PROTO* scanned);
    void ps_alert_udp(PS_PROTO* scanner, PS_PROTO* scanned);
    void ps_alert_icmp(PS_PROTO* scanner, PS_PROTO* scanned);

private:
    PortscanConfig* config;
};

#endif

ps_module.h

Path = src/network_inspectors/port_scan/ps_module.h

#ifndef PS_MODULE_H
#define PS_MODULE_H

#include "framework/module.h"
#include "main/reload_tuner.h"
#include "ps_detect.h"
#include "ps_pegs.h"

namespace snort
{
struct SnortConfig;
}

#define PS_NAME "port_scan"
#define PS_HELP "detect various ip, icmp, tcp, and udp port or protocol scans"

//-------------------------------------------------------------------------
// gid - sids
//-------------------------------------------------------------------------

#define GID_PORT_SCAN 122

#define PSNG_TCP_PORTSCAN                      1
#define PSNG_TCP_DECOY_PORTSCAN                2
#define PSNG_TCP_PORTSWEEP                     3
#define PSNG_TCP_DISTRIBUTED_PORTSCAN          4
#define PSNG_TCP_FILTERED_PORTSCAN             5
#define PSNG_TCP_FILTERED_DECOY_PORTSCAN       6
#define PSNG_TCP_PORTSWEEP_FILTERED            7
#define PSNG_TCP_FILTERED_DISTRIBUTED_PORTSCAN 8

#define PSNG_IP_PORTSCAN                       9
#define PSNG_IP_DECOY_PORTSCAN                 10
#define PSNG_IP_PORTSWEEP                      11
#define PSNG_IP_DISTRIBUTED_PORTSCAN           12
#define PSNG_IP_FILTERED_PORTSCAN              13
#define PSNG_IP_FILTERED_DECOY_PORTSCAN        14
#define PSNG_IP_PORTSWEEP_FILTERED             15
#define PSNG_IP_FILTERED_DISTRIBUTED_PORTSCAN  16

#define PSNG_UDP_PORTSCAN                      17
#define PSNG_UDP_DECOY_PORTSCAN                18
#define PSNG_UDP_PORTSWEEP                     19
#define PSNG_UDP_DISTRIBUTED_PORTSCAN          20
#define PSNG_UDP_FILTERED_PORTSCAN             21
#define PSNG_UDP_FILTERED_DECOY_PORTSCAN       22
#define PSNG_UDP_PORTSWEEP_FILTERED            23
#define PSNG_UDP_FILTERED_DISTRIBUTED_PORTSCAN 24

#define PSNG_ICMP_PORTSWEEP                    25
#define PSNG_ICMP_PORTSWEEP_FILTERED           26

#define PSNG_OPEN_PORT                         27

//-------------------------------------------------------------------------
// rule msgs
//-------------------------------------------------------------------------

#define PSNG_TCP_PORTSCAN_STR \
    "TCP portscan"
#define PSNG_TCP_DECOY_PORTSCAN_STR \
    "TCP decoy portscan"
#define PSNG_TCP_PORTSWEEP_STR \
    "TCP portsweep"
#define PSNG_TCP_DISTRIBUTED_PORTSCAN_STR \
    "TCP distributed portscan"
#define PSNG_TCP_FILTERED_PORTSCAN_STR \
    "TCP filtered portscan"
#define PSNG_TCP_FILTERED_DECOY_PORTSCAN_STR \
    "TCP filtered decoy portscan"
#define PSNG_TCP_FILTERED_DISTRIBUTED_PORTSCAN_STR \
    "TCP filtered distributed portscan"
#define PSNG_TCP_PORTSWEEP_FILTERED_STR \
    "TCP filtered portsweep"

#define PSNG_IP_PORTSCAN_STR \
    "IP protocol scan"
#define PSNG_IP_DECOY_PORTSCAN_STR \
    "IP decoy protocol scan"
#define PSNG_IP_PORTSWEEP_STR \
    "IP protocol sweep"
#define PSNG_IP_DISTRIBUTED_PORTSCAN_STR \
    "IP distributed protocol scan"
#define PSNG_IP_FILTERED_PORTSCAN_STR \
    "IP filtered protocol scan"
#define PSNG_IP_FILTERED_DECOY_PORTSCAN_STR \
    "IP filtered decoy protocol scan"
#define PSNG_IP_FILTERED_DISTRIBUTED_PORTSCAN_STR \
    "IP filtered distributed protocol scan"
#define PSNG_IP_PORTSWEEP_FILTERED_STR \
    "IP filtered protocol sweep"

#define PSNG_UDP_PORTSCAN_STR \
    "UDP portscan"
#define PSNG_UDP_DECOY_PORTSCAN_STR \
    "UDP decoy portscan"
#define PSNG_UDP_PORTSWEEP_STR \
    "UDP portsweep"
#define PSNG_UDP_DISTRIBUTED_PORTSCAN_STR \
    "UDP distributed portscan"
#define PSNG_UDP_FILTERED_PORTSCAN_STR \
    "UDP filtered portscan"
#define PSNG_UDP_FILTERED_DECOY_PORTSCAN_STR \
    "UDP filtered decoy portscan"
#define PSNG_UDP_FILTERED_DISTRIBUTED_PORTSCAN_STR \
    "UDP filtered distributed portscan"
#define PSNG_UDP_PORTSWEEP_FILTERED_STR \
    "UDP filtered portsweep"

#define PSNG_ICMP_PORTSWEEP_STR \
    "ICMP sweep"
#define PSNG_ICMP_PORTSWEEP_FILTERED_STR \
    "ICMP filtered sweep"

#define PSNG_OPEN_PORT_STR \
    "open port"

//-------------------------------------------------------------------------
// Reload resource tuning
//-------------------------------------------------------------------------

class PortScanReloadTuner : public snort::ReloadResourceTuner
{
public:
    explicit PortScanReloadTuner(size_t memcap) : memcap(memcap) { }
    ~PortScanReloadTuner() override = default;

    bool tinit() override
    { return ps_init_hash(memcap); }

    bool tune_idle_context() override
    { return ps_prune_hash(max_work_idle); }

    bool tune_packet_context() override
    { return ps_prune_hash(max_work); }

private:
    size_t memcap;
};

//-------------------------------------------------------------------------

extern THREAD_LOCAL PsPegStats spstats;
extern THREAD_LOCAL snort::ProfileStats psPerfStats;

struct PortscanConfig;

class PortScanModule : public snort::Module
{
public:
    PortScanModule();
    ~PortScanModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile() const override;
    const snort::RuleMap* get_rules() const override;

    unsigned get_gid() const override
    { return GID_PORT_SCAN; }

    PortscanConfig* get_data();

    // FIXIT-M this should eventually be CONTEXT.
    // Set to GLOBAL so this isn't selected away when inspection policy switches
    Usage get_usage() const override
    { return GLOBAL; }

private:
    PS_ALERT_CONF* get_alert_conf(const char* fqn);
    PortscanConfig* config;
};

#endif

ps_pegs.h

Path = src/network_inspectors/port_scan/ps_pegs.h

#ifndef PS_PEGS_H
#define PS_PEGS_H

#include "framework/counts.h"
#include "main/snort_types.h"

static const PegInfo ps_module_pegs[] =
{
    { CountType::SUM, "packets", "number of packets processed by port scan" },
    { CountType::SUM, "trackers", "number of trackers allocated by port scan" },
    { CountType::SUM, "alloc_prunes", "number of trackers pruned on allocation of new tracking" },
    { CountType::SUM, "reload_prunes", "number of trackers pruned on reload due to reduced memcap" },
    { CountType::NOW, "bytes_in_use", "number of bytes currently used by portscan" },
    { CountType::END, nullptr, nullptr },
};

struct PsPegStats
{
    PegCount packets;
    PegCount trackers;
    PegCount alloc_prunes;
    PegCount reload_prunes;
    PegCount bytes_in_use;
};

#endif

network_inspectors/reputation/

This directory contains all files related to IP Reputation inspection.

Reputation inspector provides basic IP blocklist/allowlist capabilities, to block/drop/pass traffic from IP addresses listed. In the past, we use standard Snort rules to implement Reputation-based IP blocking. This inspector will address the performance issue and make the IP reputation management easier.

This inspector also supports interfaces through manifest files. You can specify an action (BLOCK, ALLOW, MONITOR) for an IP list with interfaces. The name of manifest file is interface.info. For each line of the manifest file, the format is:

file_name, list_id, action (block, allow, monitor), [interface information]

If interface information is empty, this means all interfaces are applied

reputation_commands.h

Path = src/network_inspectors/reputation/reputation_commands.h

#ifndef REPUTATION_COMMANDS_H
#define REPUTATION_COMMANDS_H

#include "framework/module.h"

extern const snort::Command reputation_cmds[];

#endif

reputation_common.h

Path = src/network_inspectors/reputation/reputation_common.h

#ifndef REPUTATION_COMMON_H
#define REPUTATION_COMMON_H

#define REPUTATION_NAME "reputation"
#define REPUTATION_HELP "reputation inspection"

#define GID_REPUTATION 136

#define REPUTATION_EVENT_BLOCKLIST_SRC   1
#define REPUTATION_EVENT_ALLOWLIST_SRC   2
#define REPUTATION_EVENT_MONITOR_SRC     3
#define REPUTATION_EVENT_BLOCKLIST_DST   4
#define REPUTATION_EVENT_ALLOWLIST_DST   5
#define REPUTATION_EVENT_MONITOR_DST     6

#endif

reputation_config.h

Path = src/network_inspectors/reputation/reputation_config.h

#ifndef REPUTATION_CONFIG_H
#define REPUTATION_CONFIG_H

#include "framework/counts.h"
#include "main/thread.h"
#include "sfrt/sfrt.h"

#include <vector>
#include <set>
#include <string>

#define NUM_INDEX_PER_ENTRY 4

// Configuration for reputation network inspector

enum NestedIP
{
    INNER,
    OUTER,
    ALL
};

enum AllowAction
{
    DO_NOT_BLOCK,
    TRUST
};

enum IPdecision
{
    DECISION_NULL,
    BLOCKED,
    TRUSTED,
    MONITORED,
    BLOCKED_SRC,
    BLOCKED_DST,
    TRUSTED_SRC,
    TRUSTED_DST,
    TRUSTED_DO_NOT_BLOCK,
    MONITORED_SRC,
    MONITORED_DST,
    DECISION_MAX
};

#define MAX_NUM_INTFS             INT32_MAX
#define MAX_LIST_ID               UINT32_MAX

struct ListFile
{
    std::string file_name;
    int file_type;
    uint32_t list_id;
    bool all_intfs_enabled = false;
    std::set<unsigned int> intfs;
    uint8_t list_index;
    uint8_t list_type;
};

typedef std::vector<ListFile*> ListFiles;

struct ReputationConfig
{
    uint32_t memcap = 500;
    bool scanlocal = false;
    IPdecision priority = TRUSTED;
    NestedIP nested_ip = INNER;
    AllowAction allow_action = DO_NOT_BLOCK;
    std::string blocklist_path;
    std::string allowlist_path;
    std::string list_dir;
};

struct IPrepInfo
{
    char list_indexes[NUM_INDEX_PER_ENTRY];
    MEM_OFFSET next;
};

struct ReputationStats
{
    PegCount packets;
    PegCount blocked;
    PegCount trusted;
    PegCount monitored;
    PegCount memory_allocated;
    PegCount aux_ip_blocked;
    PegCount aux_ip_trusted;
    PegCount aux_ip_monitored;
};

extern const PegInfo reputation_peg_names[];
extern THREAD_LOCAL ReputationStats reputationstats;

#endif

reputation_inspect.h

Path = src/network_inspectors/reputation/reputation_inspect.h

#ifndef REPUTATION_INSPECT_H
#define REPUTATION_INSPECT_H

#include "framework/inspector.h"

#include "reputation_module.h"

struct table_flat_t;
class ReputationData
{
public:
    ReputationData() = default;
    ~ReputationData();

    ListFiles list_files;
    uint8_t* reputation_segment = nullptr;
    table_flat_t* ip_list = nullptr;
    int num_entries = 0;
    bool memcap_reached = false;
};

class Reputation : public snort::Inspector
{
public:
    explicit Reputation(ReputationConfig*);
    ~Reputation() override;

    void tinit() override;
    void tterm() override;

    void show(const snort::SnortConfig*) const override;
    void eval(snort::Packet*) override
    { }
    bool configure(snort::SnortConfig*) override;
    void install_reload_handler(snort::SnortConfig*) override;

    ReputationData& get_data()
    { return *rep_data; }
    const ReputationConfig& get_config()
    { return config; }
    ReputationData* load_data();

    void swap_thread_data(ReputationData*);
    void swap_data(ReputationData*);

private:
    ReputationConfig config;
    ReputationData* rep_data;
};

#endif

reputation_module.h

Path = src/network_inspectors/reputation/reputation_module.h

#ifndef REPUTATION_MODULE_H
#define REPUTATION_MODULE_H

// Interface to the REPUTATION network inspector

#include "framework/module.h"
#include "main/reload_tuner.h"

#include "reputation_config.h"
#include "reputation_common.h"

namespace snort
{
struct SnortConfig;
}

extern THREAD_LOCAL snort::ProfileStats reputation_perf_stats;
extern unsigned long total_duplicates;
extern unsigned long total_invalids;

class Reputation;

class ReputationReloadSwapper : public snort::ReloadSwapper
{
public:
    explicit ReputationReloadSwapper(Reputation& ins) : inspector(ins)
    { }
    ~ReputationReloadSwapper() override = default;

    void tswap() override;

private:
    Reputation& inspector;
};

class ReputationModule : public snort::Module
{
public:
    ReputationModule();
    ~ReputationModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    unsigned get_gid() const override
    { return GID_REPUTATION; }

    const snort::RuleMap* get_rules() const override;
    const snort::Command* get_commands() const override;
    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile() const override;

    ReputationConfig* get_data();

private:
    ReputationConfig* conf;
};

#endif

reputation_parse.h

Path = src/network_inspectors/reputation/reputation_parse.h

#ifndef REPUTATION_PARSE_H
#define REPUTATION_PARSE_H

#include <cstdint>

#include "sfrt/sfrt_flat.h"

struct IPrepInfo;
struct ListFile;
struct ReputationConfig;
class ReputationData;

class ReputationParser
{
public:
    static void read_manifest(const char* filename, const ReputationConfig&, ReputationData&);
    static void add_block_allow_List(const ReputationConfig&, ReputationData&);
    static void estimate_num_entries(ReputationData&);

    void load_list_file(ListFile* list_info, const ReputationConfig& config,
        ReputationData& data);
    void ip_list_init(uint32_t max_entries, const ReputationConfig&, ReputationData&);

    unsigned get_usage() const
    { return table.sfrt_flat_usage(); }

protected:
    int duplicate_info(IPrepInfo* dest_info, IPrepInfo* current_info, uint8_t* base);
    int64_t update_entry_info_impl(INFO* current, INFO new_entry, SaveDest save_dest, uint8_t* base);
    int add_ip(snort::SfCidr* ip_addr,INFO info_ptr, const ReputationConfig& config);
    int process_line(char* line, INFO info, const ReputationConfig& config);

    static int64_t update_entry_info(INFO* current, INFO new_entry, SaveDest save_dest, uint8_t* base, void* data);

    RtTable table;
};

#endif

network_inspectors/rna/

This inspector is still in experimental (work-in-progress) state.

The Real-time Network Awareness (RNA) inspector provides visibility into a network using Passive Network Discovery (PND). RNA analyzes traffic to discover hosts on the network and to detect operating system (OS) running on a host. It uses fingerprints for OS detection. It logs ip/mac addresses, ports, protocols, OS, and other information about traffic running on these hosts. It does not generate or alter traffic on its own.

RNA logs information it discovers by parsing TCP/UDP/IP/Link-layer protocols and observing data found by other inspectors (e.g., monitoring application IDs, client versions, user-agents found by appid or http inspectors). Operating systems have different signatures which are apparent in different parameters in the packets that it sends. These parameters include things like TCP window sizes, TCP options, segment sizes, etc. Such fingerprinting information is provided to RNA as input so that RNA can analyze traffic.

RNA discoveries is stored in host tracker objects, which are saved globally in an LRU cache and shared among threads. RNA memory and discovery are bounded by the memcap of cache.

Packets from untracked sessions (e.g., non-IP) are processed via the eval method as per proto-bit registrations. Packets from tracked sessions (e.g., IP, TCP, UDP, and ICMP) are processed via events as per subscriptions. Since RNA needs to see the first packet of a session published from stream trackers, these modules (e.g., stream, stream_icmp, stream_ip, stream_tcp, and stream_udp) should be enabled whenever RNA module is enabled.

Currently, RNA only supports host discovery with filtering based on IP/port/interface. To enable this (disabled by default), the config file referred by rna_conf_path can have keywords: Analyze # discover application, host, user (only host discovery is implemented) AnalyzeHostUser # discover application, host, user (same as Analyze) AnalyzeApplication # discover application AnalyzeHost # discover application, host AnalyzeUser # discover application, user portexclusion # don’t discover on this port

Format: config keyword [!]ip [interface] portexclusion dst|src|both tcp|udp port ip

Examples: config AnalyzeHost 0.0.0.0/0 -1 # discover any ipv4 on any interface config AnalyzeHost ::/0 2 # discover any ipv6 on interface 2 config AnalyzeHost !1.2.3.4/16 3 # exclude this ipv4 range on interface 3 config Analyze !cafe:feed::0/64 # exclude this ipv6 range on any interface portexclusion dst udp 53 8.8.8.8 # exclude this ip for UDP port 53 in destination direction portexclusion both tcp 4000 ::0/0 # exclude any ipv6 for TCP port 4000 in both direction

Note that exclusion has higher priority than inclusion. RNA does not support application/user discovery, fingerprint, fingerprint reader, enable_banner_grab, etc. The enable_logger config is to enable/disable sending RNA discovery events to EventManager::call_loggers. Such event logger or reader is not implemented yet. However, since RNA stores host information into host_cache, to log the discovered hosts into a file, one can 1) issue socket command: host_cache.dump(file.out), or 2) add lua config: host_cache = { dump_file = file.out}.

RNA Uses HostCacheMac, derived from LruCacheSharedMemcap, to track MAC addresses. The implementation of the MAC cache is as follows:

HostCacheMac (LruCacheSharedMemcap<uint8_t*, HostTrackerMac, HashMac>)
   |- - - - Key = uint8_t[6] (mac address)
   |- - - - Value = HostTrackerMac
   |                 | - - - - Dynamic memory growth is tracked by HostCacheAllocMac allocator
   |
   |- - - - Hash = HashMac
   |                 |
                     | - - - - Takes uint8_t[6], returns 64-bit hash of MAC. This allows us to
                     |             hash a 48 bit value (MAC) to a 64 bit key with relatively
                                   few collisions. AA:BB:CC:DD:EE:FF becomes 0xAABBCCDDEEFF

In RNAPnd discover_network_ethernet, in some scenarios, we are required to create a host tracker to be used for exclusively for logging. The call chain is the following: → discover_network_ethernet | discover_network_arp | discover_network_bpdu | discover_switch

We pass a host tracker pointer by reference (HostTracker**) from discover_network_ethernet down to the lower-level calls. These functions (discover[network_arp|network_bpdu|switch]) are responsible for setting the top level pointer to point at their own instantiated host tracker, as it needs to be preserved until discover_network_ethernet calls generate_change_vlan_update with this newly-created host tracker as an argument. This host tracker is deleted at the top level, and we must not return prior to that to avoid leaking any host trackers.

Fingerprints

Fingerprints are a sequence of features in network traffic used to identify a host’s operating system.

Tcp fingerprints are specified in lua and read by snort at configure time. Only the SYN and SYN-ACK packets are used for fingerprint matching. We refer to the SYN packet as "client" and SYN_ACK packet as "server" traffic.

A typical fingerprint looks like this:

— centos client { fpid = 110005, type = 10, uuid = "2fc04d1a-a2c2-11e2-840f-850c4648cdef", ttl = 64, tcp_window = "5712-5760", mss = "X", id = "X", topts = "2 4 8 3", ws = "7", df = false, },

This particular example identifies the host as one of CentOS Linux version 5.5,CentOS,Linux,Linux 5.5. Note that there is no reference to the actual operating system in the fingerprint, but the fpid or the uuid field can be used for this purpose.

The fields are as follows:

fpid: (int) a unique fingerprint identifier. If snort encounters another fingerprint with the same fpid, it will display a warning message and skip it. This field can be used as an index into a database or table with details in human readable form about the host. This field is not used for matching the fingerprint to the network traffic.

type: (int) the type of traffic this fingerprint should be matched against. Currently, these types are defined in rna_fingerprint.h:

enum FpType
{
    FINGERPRINT_TYPE_SERVER = 1,
    FINGERPRINT_TYPE_CLIENT = 2,
    FINGERPRINT_TYPE_SERVER6 = 10,
    FINGERPRINT_TYPE_CLIENT6 = 11,
};

uuid: (uuid string, not an arbitrary string) similar in purpose to fpid, e.g. we could store the host details in a file named 2fc04d1a-a2c2-11e2-840f-850c4648cdef. Not used in matching.

ttl: (int) time to live

df: (bool) don’t fragment flag

The remaining fields are space-separated lists of FpElement, each of FpElementType. See rna_fingerprint.h. Such a list might be "1 2-5 6" or simply "X", or "10". Snort infers the FpElementType from the input string, e.g. "2-5" will be interpreted as a FpElementType::RANGE, and "X" will be interpreted as FpElementType::DONT_CARE. See the FpElement::parse_value() function in rna_fingerprint.cc.

Here are all possible element types, with examples for each case:

FpElementType::RANGE: "1" or "20-30" FpElementType::INCREMENT: "+5" FpElementType::SYN_MATCH: "SYN" FpElementType::RANDOM: "R" FpElementType::DONT_CARE: "X" FpElementType::SYNTS: "TS"

For instance, the list

"1 2-5 SYN"

is a legal list, consisting of 3 elements: 1 (RANGE), 2-5 (RANGE) and SYN (SYN_MATCH). While the elements in this list are all valid, there are constraints on the type of the elements that make up the fingerprint fields. For instance,

mss = "2 3 X"

would result in error, because DONT_CARE ("X") is not allowed for the mss field.

Here are the remaining fingerprint fields.

tcp_window: the tcp window FpElementType::RANGE

In the packet, the tcp window is a single integer value, but in the fingerprint
we can specify a range like tcp_window = "1234-4321".
Examples:
tcp_window = "1234-4321" -- will match any packet with tcp_window in the range
tcp_window = "5678"      -- will match only packets with tcp_window = 5678
tcp_window = "R"         -- error: RANDOM is not a valid tcp_window element

mss: maximum segment size On the client: FpElementType::RANGE, FpFLementType::DONT_CARE On the server: FpElementType::RANGE, FpElementType::DONT_CARE, FpElementType::SYN_MATCH, FpElementType::SYNTS

Examples:
fptype = 2, mss = "12-34" -- client traffic (fptype = 2) with mss between 12 and 34
fptype = 2, mss = "X"     -- don't use mss for matching, so match anything from cient
fptype = 2, mss = "SYN"   -- error: client traffic (fptype = 2) but mss of type SYN_MATCH
                             only accepted for server traffic (fptyp1 = 1 or 10)
fptype = 1, mss = "TS"    -- OK: server traffic (fptype = 1) and mss of type SYNTS
mss = "+5"                -- error: mss cannot be an INCREMENT type

id: ip id FpElementType::RANGE FpElementType::RANDOM FpElementType::INCREMENT FpElementType::DONT_CARE

Example:
id = "X"

topts: tcp options FpElementType::RANGE

These are defined by TcpOptCode in src/protocols/tcp_options.h.
The ones we use for fingerprint matching are
  • MAXSEG (2)

  • WSCALE (3)

  • SACKOK (4)

  • TIMESTAMP (8)

    We match (a) whether or not an option is set, (b) what its value is and (c) the order
    of the options.
    Example:
    topts = [2 4 8 3] -- will match tcp packets with mss (2), window scale (4), sack OK (8)
                         and timestamp (3) set, in this exact order. If we swap any two
                         numbers in topts, the resulting fingerprint will no longer match.

ws: window scale FpElementType::RANGE FpElementType::DONT_CARE

Similar to the TCP fingerprints, user-agent based fingerprints loads different types of fingerprint patterns from Lua configuration, namely os (operating system), device (mobile device information), jail-broken (hacked system), and jail-broken-host (host information of the hacked system to confirm matching). During packet processing, the rna module depends on the HTTP user agent and host information found by the appid module and tries to match all parts of user-agent patterns. A sample configuration looks like this: { fpid = 1, uuid = "10000000-0000-0000-0000-111111111111", ua_type = "device", user_agent = { { substring = "CPU" }, { substring = "OS 3_0" }, { substring = "My Company" } }, device = "My Mobile", }

UDP fingerprints are of two types, namely DHCP and SMB. Currently only DHCP fingerprints are supported. Similar to TCP and user-agent based fingerprints, fingerprint patterns can be specified in the lua configuration. DHCP fingerprint matching is based on option fields present in DHCP Request packet and the updated lease information is obtained from the DHCP ACK packet. Appid module processes the DHCP request packet, parses the option 55 and option 60 fields and publishes the DHCPDataEvent. This triggers RNA fingerprint matching and event generation. While processing DHCP ACK packet, appid extracts the leased IP, netmask and lease time and publishes the DHCPInfoEvent. RNA generates the CHANGE_FULL_DHCP_INFO event with updated lease information. A sample DHCP fingerprint is shown below: { fpid = 111, uuid = "12345678-1234-1234-1234-123456789111", type = 4, dhcp55 = "1 121 3 6 15 119 252", dhcp60 = "dhcp 5.1.4", }

data_purge_cmd.h

Path = src/network_inspectors/rna/data_purge_cmd.h

#ifndef DATA_PURGE_CMD_H
#define DATA_PURGE_CMD_H

#include "main/analyzer_command.h"

#include "rna_mac_cache.h"

class DataPurgeAC : public snort::AnalyzerCommand
{
public:

    DataPurgeAC(HostCacheMac* new_cache) : mac_cache(new_cache) { }

    ~DataPurgeAC() override;

    bool execute(Analyzer&, void**) override;

    const char* stringify() override { return "DATA_PURGE"; }

private:
    HostCacheMac* mac_cache;
};

#endif

rna_app_discovery.h

Path = src/network_inspectors/rna/rna_app_discovery.h

#ifndef RNA_APP_DISCOVERY_H
#define RNA_APP_DISCOVERY_H

#include "rna_fingerprint_tcp.h"
#include "rna_fingerprint_ua.h"
#include "rna_pnd.h"

class RnaAppDiscovery
{
public:
    static void process(AppidEvent*, DiscoveryFilter&, RnaConfig*, RnaLogger&);

    static bool discover_service(const snort::Packet*, DiscoveryFilter&, RNAFlow*, IpProtocol,
        RnaConfig*, RnaLogger&, uint16_t, AppId service = APP_ID_NONE, bool is_client = false);

    static void discover_payload(const snort::Packet*, DiscoveryFilter&, RNAFlow*, IpProtocol,
        RnaConfig*, RnaLogger&, AppId service, AppId payload, AppId client);

    static void discover_client(const snort::Packet*, DiscoveryFilter&, RNAFlow*, RnaConfig*,
        RnaLogger&, const char*, AppId client, AppId service);

    static void discover_user(const snort::Packet*, DiscoveryFilter&, RNAFlow*, RnaLogger&,
        const char*, AppId, IpProtocol, RnaConfig*, bool);

    static void discover_banner(const snort::Packet*, DiscoveryFilter&, RNAFlow*, IpProtocol,
        RnaLogger&, AppId);

    static void discover_netbios_name(const snort::Packet*, DiscoveryFilter&,
        RNAFlow*, RnaLogger&, const char*);

    static RnaTracker get_server_rna_tracker(const snort::Packet*, RNAFlow*);
    static RnaTracker get_client_rna_tracker(const snort::Packet*, RNAFlow*);

private:
    static void update_service_info(const snort::Packet*, DiscoveryFilter&, RNAFlow*, IpProtocol,
        uint16_t, const char* vendor, const char* version, RnaLogger&, RnaConfig*, AppId service,
        bool is_client = false);

    static void analyze_user_agent_fingerprint(const snort::Packet*, DiscoveryFilter&, RNAFlow*,
        const char* host, const char* uagent, RnaLogger&, snort::UaFpProcessor&);

};

#endif

rna_config.h

Path = src/network_inspectors/rna/rna_config.h

#ifndef RNA_CONFIG_H
#define RNA_CONFIG_H

#include "framework/data_bus.h"

namespace snort
{
class TcpFpProcessor;
class UaFpProcessor;
class UdpFpProcessor;
class SmbFpProcessor;
}

struct RnaModuleConfig
{
    std::string rna_conf_path;
    bool enable_logger;
    bool log_when_idle;
    snort::TcpFpProcessor* tcp_processor = nullptr;
    snort::UaFpProcessor* ua_processor = nullptr;
    snort::UdpFpProcessor* udp_processor = nullptr;
    snort::SmbFpProcessor* smb_processor = nullptr;
};

// Give default values so that RNA can work even if rna_conf_path is not provided
struct RnaConfig
{
    uint32_t update_timeout = 3600;
    uint16_t max_host_client_apps = 16;
    uint16_t max_payloads = 100;
    uint16_t max_host_services = 100;
    uint16_t max_host_service_info = 16;
    bool enable_banner_grab = false;
    bool log_when_idle = false;

    static unsigned pub_id;
};

#endif

rna_cpe_os.h

Path = src/network_inspectors/rna/rna_cpe_os.h

#ifndef RNA_CPE_OS_H
#define RNA_CPE_OS_H

#include "pub_sub/external_event_ids.h"

class SO_PUBLIC CpeOsInfoEvent : public snort::DataEvent
{
public:
    CpeOsInfoEvent(const snort::Packet& p) : p(p) { }

    const snort::Packet* get_packet() const override
    {
         return &p;
    }

    void add_os(const char *name)
    {
        hash ^= std::hash<std::string>{}(name);
        os_names.emplace_back(name);
    }

    uint32_t get_hash()
    {
        return hash;
    }

    const std::vector<const char*>* get_os_names()
    {
        return &os_names;
    }
private:
    const snort::Packet& p;
    std::vector<const char*> os_names;
    uint32_t hash = 0;
};

#endif

rna_event_handler.h

Path = src/network_inspectors/rna/rna_event_handler.h

#ifndef RNA_EVENT_HANDLER_H
#define RNA_EVENT_HANDLER_H

#include "framework/data_bus.h"

#include "rna_module.h"
#include "rna_pnd.h"

inline static void update_rna_pkt_stats(const snort::Packet* p)
{
    ++rna_stats.total_packets_in_interval;
    rna_stats.total_bytes_in_interval += p->pktlen;
}

inline static void update_rna_pkt_stats(const snort::DataEvent& event)
{
    update_rna_pkt_stats(event.get_packet());
}

class RnaAppidEventHandler : public snort::DataHandler
{
public:
    RnaAppidEventHandler(RnaPnd& nd) : DataHandler(RNA_NAME), pnd(nd) { }
    void handle(snort::DataEvent&, snort::Flow*) override;
private:
    RnaPnd& pnd;
};

class RnaIcmpNewFlowEventHandler : public snort::DataHandler
{
public:
    RnaIcmpNewFlowEventHandler(RnaPnd& nd) : DataHandler(RNA_NAME), pnd(nd) { }
    void handle(snort::DataEvent&, snort::Flow*) override;
private:
    RnaPnd& pnd;
};

class RnaIcmpBidirectionalEventHandler : public snort::DataHandler
{
public:
    RnaIcmpBidirectionalEventHandler(RnaPnd& nd) : DataHandler(RNA_NAME), pnd(nd) { }
    void handle(snort::DataEvent&, snort::Flow*) override;
private:
    RnaPnd& pnd;
};

class RnaIpNewFlowEventHandler : public snort::DataHandler
{
public:
    RnaIpNewFlowEventHandler(RnaPnd& nd) : DataHandler(RNA_NAME), pnd(nd) { }
    void handle(snort::DataEvent&, snort::Flow*) override;
private:
    RnaPnd& pnd;
};


class RnaIpBidirectionalEventHandler : public snort::DataHandler
{
public:
    RnaIpBidirectionalEventHandler(RnaPnd& nd) : DataHandler(RNA_NAME), pnd(nd) { }
    void handle(snort::DataEvent&, snort::Flow*) override;
private:
    RnaPnd& pnd;
};

class RnaTcpSynEventHandler : public snort::DataHandler
{
public:
    RnaTcpSynEventHandler(RnaPnd& nd) : DataHandler(RNA_NAME), pnd(nd) { }
    void handle(snort::DataEvent&, snort::Flow*) override;
private:
    RnaPnd& pnd;
};

class RnaTcpSynAckEventHandler : public snort::DataHandler
{
public:
    RnaTcpSynAckEventHandler(RnaPnd& nd) : DataHandler(RNA_NAME), pnd(nd) { }
    void handle(snort::DataEvent&, snort::Flow*) override;
private:
    RnaPnd& pnd;
};

class RnaTcpMidstreamEventHandler : public snort::DataHandler
{
public:
    RnaTcpMidstreamEventHandler(RnaPnd& nd) : DataHandler(RNA_NAME), pnd(nd) { }
    void handle(snort::DataEvent&, snort::Flow*) override;
private:
    RnaPnd& pnd;
};

class RnaUdpNewFlowEventHandler : public snort::DataHandler
{
public:
    RnaUdpNewFlowEventHandler(RnaPnd& nd) : DataHandler(RNA_NAME), pnd(nd) { }
    void handle(snort::DataEvent&, snort::Flow*) override;
private:
    RnaPnd& pnd;
};

class RnaUdpBidirectionalEventHandler : public snort::DataHandler
{
public:
    RnaUdpBidirectionalEventHandler(RnaPnd& nd) : DataHandler(RNA_NAME), pnd(nd) { }
    void handle(snort::DataEvent&, snort::Flow*) override;
private:
    RnaPnd& pnd;
};

class RnaIdleEventHandler : public snort::DataHandler
{
public:
    RnaIdleEventHandler(RnaPnd& nd) : DataHandler(RNA_NAME), pnd(nd) { }
    void handle(snort::DataEvent&, snort::Flow*) override;
private:
    RnaPnd& pnd;
};

class RnaDHCPInfoEventHandler : public snort::DataHandler
{
public:
    RnaDHCPInfoEventHandler(RnaPnd& nd) : DataHandler(RNA_NAME), pnd(nd) { }
    void handle(snort::DataEvent&, snort::Flow*) override;
private:
    RnaPnd& pnd;
};

class RnaDHCPDataEventHandler : public snort::DataHandler
{
public:
    RnaDHCPDataEventHandler(RnaPnd& nd) : DataHandler(RNA_NAME), pnd(nd) { }
    void handle(snort::DataEvent&, snort::Flow*) override;
private:
    RnaPnd& pnd;
};

class RnaFpSMBEventHandler : public snort::DataHandler
{
public:
    RnaFpSMBEventHandler(RnaPnd& nd) : DataHandler(RNA_NAME), pnd(nd) { }
    void handle(snort::DataEvent&, snort::Flow*) override;
private:
    RnaPnd& pnd;
};

class RnaCPEOSInfoEventHandler : public snort::DataHandler
{
public:
    RnaCPEOSInfoEventHandler(RnaPnd& nd) : DataHandler(RNA_NAME), pnd(nd) { }
    void handle(snort::DataEvent&, snort::Flow*) override;
private:
    RnaPnd& pnd;
};

class RnaNetFlowEventHandler : public snort::DataHandler
{
public:
    RnaNetFlowEventHandler(RnaPnd& nd) : DataHandler(RNA_NAME), pnd(nd) { }
    void handle(snort::DataEvent&, snort::Flow*) override;
private:
    RnaPnd& pnd;
};

#endif

rna_fingerprint.h

Path = src/network_inspectors/rna/rna_fingerprint.h

#ifndef RNA_FP_H
#define RNA_FP_H

#include <cstdint>
#include <string>
#include <vector>

#include "main/snort_types.h"

#define FP_SYN_KEY          "SYN"
#define FP_RANDOM_KEY       "R"
#define FP_DONT_CARE_KEY    "X"
#define FP_SYN_TS_KEY       "TS"

#define MAXIMUM_FP_HOPS 32

namespace snort
{

class FpFingerprint
{
public:

    enum FpType
    {
        FP_TYPE_DERIVED = 0,
        FP_TYPE_SERVER = 1,
        FP_TYPE_CLIENT = 2,
        FP_TYPE_SMB = 3,
        FP_TYPE_DHCP = 4,
        FP_TYPE_USER = 5,
        FP_TYPE_SCAN = 6,
        FP_TYPE_APP = 7,
        FP_TYPE_CONFLICT = 8,
        FP_TYPE_MOBILE = 9,
        FP_TYPE_SERVER6 = 10,
        FP_TYPE_CLIENT6 = 11,
        FP_TYPE_DHCP6 = 12,
        FP_TYPE_USERAGENT = 13,
        FP_TYPE_CPE = 14,
        MAX_FP_TYPES = 15
    };

    uint32_t fpid = 0;
    uint32_t fp_type = 0;
    std::string fpuuid;
    uint8_t ttl = 0;

    virtual ~FpFingerprint() = default;

    virtual void clear()
    {
        fpid = 0;
        fp_type = 0;
        fpuuid.clear();
        ttl = 0;
    }
};

enum FpElementType
{
    INVALID = -1,
    RANGE = 1,
    INCREMENT,
    SYN_MATCH,
    RANDOM,
    DONT_CARE,
    SYNTS
};

class SO_PUBLIC FpElement
{
public:

    FpElement() = default;
    FpElement(const std::string&);

    FpElement& operator=(const FpElement& fpe) = default;
    FpElement& operator=(const std::string& str);
    bool operator==(const FpElement& y) const;

    FpElementType type;
    union
    {
        int value;
        struct
        {
            int min;
            int max;
        } range;
    } d;

private:
    void parse_value(const std::string&);
};

}

enum UserAgentInfoType
{
    OS_INFO,
    DEVICE_INFO,
    JAIL_BROKEN_INFO,
    JAIL_BROKEN_HOST
};

class RawFingerprint
{
public:

    uint32_t fpid = 0;
    uint32_t fp_type = 0;
    std::string fpuuid;
    uint8_t ttl = 0;

    std::string tcp_window;
    std::string mss;
    std::string id;
    std::string topts;
    std::string ws;
    bool df = false;

    UserAgentInfoType ua_type = OS_INFO;
    std::vector<std::string> user_agent;
    std::string host_name;
    std::string device;

    std::string dhcp55;
    std::string dhcp60;

    uint16_t smb_major = 0;
    uint16_t smb_minor = 0;
    uint32_t smb_flags = 0;

    void clear()
    {
        fpid = 0;
        fp_type = 0;
        fpuuid.clear();
        ttl = 0;
        tcp_window.clear();
        mss.clear();
        id.clear();
        topts.clear();
        ws.clear();
        df = false;
        ua_type = OS_INFO;
        user_agent.clear();
        host_name.clear();
        device.clear();
        dhcp55.clear();
        dhcp60.clear();
        smb_major = 0;
        smb_minor = 0;
        smb_flags = 0;
    }
};

#endif

rna_fingerprint_smb.h

Path = src/network_inspectors/rna/rna_fingerprint_smb.h

#ifndef RNA_FINGERPRINT_SMB_H
#define RNA_FINGERPRINT_SMB_H

#include <mutex>
#include <unordered_map>
#include <vector>

#include "main/snort_types.h"
#include "protocols/packet.h"
#include "protocols/tcp.h"
#include "sfip/sf_ip.h"

#include "rna_fingerprint.h"

class RNAFlow;

namespace snort
{

class SO_PUBLIC SmbFingerprint : public FpFingerprint
{
public:

    SmbFingerprint(unsigned maj = 0, unsigned min = 0, uint32_t f = 0);
    SmbFingerprint(const RawFingerprint& rfp);

    unsigned smb_major;
    unsigned smb_minor;
    uint32_t flags;

    bool operator==(const SmbFingerprint& y) const;
};

class SO_PUBLIC SmbFpProcessor
{
public:

    struct SmbFpHash
    {
        size_t operator()(const SmbFingerprint & key) const noexcept
        {
            std::hash<decltype(SmbFingerprint::smb_major)> hm;
            std::hash<decltype(SmbFingerprint::flags)> hf;
            auto const h = hm(key.smb_major) ^ (hm(key.smb_minor) << 1) ^ (hf(key.flags) << 2);
            return h;
        }
    };

    struct SmbEqTo
    {
        bool operator() (const SmbFingerprint& x, const SmbFingerprint& y) const
        {
            return x.smb_major == y.smb_major && x.smb_minor == y.smb_minor && x.flags == y.flags;
        }
    };

    typedef std::unordered_set<SmbFingerprint, SmbFpHash, SmbEqTo> SmbFpContainer;
    typedef SmbFpContainer::const_iterator SmbFpIter;

    bool push(const SmbFingerprint&);
    const SmbFingerprint* find(const SmbFingerprint& ) const;

private:

    SmbFpContainer smb_fps;
};

}

snort::SmbFpProcessor* get_smb_fp_processor();
SO_PUBLIC void set_smb_fp_processor(snort::SmbFpProcessor*);

#endif

rna_fingerprint_tcp.h

Path = src/network_inspectors/rna/rna_fingerprint_tcp.h

#ifndef RNA_FINGERPRINT_TCP_H
#define RNA_FINGERPRINT_TCP_H

#include <mutex>
#include <unordered_map>
#include <vector>

#include "main/snort_types.h"
#include "protocols/packet.h"
#include "protocols/tcp.h"
#include "sfip/sf_ip.h"

#include "rna_fingerprint.h"

class RNAFlow;

namespace snort
{

class SO_PUBLIC TcpFingerprint : public FpFingerprint
{
public:

    TcpFingerprint() = default;
    TcpFingerprint(const RawFingerprint& rfp);

    std::vector<FpElement> tcp_window;
    std::vector<FpElement> mss;
    std::vector<FpElement> id;
    std::vector<FpElement> topts;
    std::vector<FpElement> ws;
    bool df = false;

    void clear() override
    {
        FpFingerprint::clear();
        tcp_window.clear();
        mss.clear();
        id.clear();
        topts.clear();
        ws.clear();
        df = false;
    }

    bool operator==(const TcpFingerprint& y) const;
};

struct FpTcpKey
{
    int synmss;
    uint8_t *syn_tcpopts;
    int num_syn_tcpopts;
    int syn_timestamp;

    int tcp_window;
    int mss;
    int ws;

    int mss_pos;
    int ws_pos;
    int sackok_pos;
    int timestamp_pos;

    bool df;
    bool isIpv6;
};

class SO_PUBLIC TcpFpProcessor
{
public:

    typedef std::unordered_map<uint32_t, TcpFingerprint> TcpFpContainer;

    enum TCP_FP_MODE { SERVER, CLIENT };

    bool push(const TcpFingerprint&);

    void make_tcp_fp_tables(TCP_FP_MODE);

    const TcpFingerprint* get_tcp_fp(const FpTcpKey&, uint8_t, TCP_FP_MODE) const;

    const TcpFingerprint* get(const Packet*, RNAFlow*) const;

    const TcpFingerprint* get(uint32_t fpid) const
    {
        auto it = tcp_fps.find(fpid);
        return it != tcp_fps.end() ? &it->second : nullptr;
    }

    const TcpFpContainer& get_tcp_fps() const
    { return tcp_fps; }

private:

    // underlying container for input fingerprints
    TcpFpContainer tcp_fps;

    // table_tcp_xxx[i] contains pointers into tcp_fps to all fingerprints
    // whose tcp window range contains i
    static constexpr uint32_t table_size = TCP_MAXWIN + 1;
    std::vector<const snort::TcpFingerprint*> table_tcp_server[table_size];
    std::vector<const snort::TcpFingerprint*> table_tcp_client[table_size];
};

}

snort::TcpFpProcessor* get_tcp_fp_processor();
SO_PUBLIC void set_tcp_fp_processor(snort::TcpFpProcessor*);

struct FpFingerprintState
{
    int initial_mss = -1;
    int timestamp = -1;
    int numopts = -1;
    uint8_t tcpopts[4] = {};
    time_t timeout = -1;

    bool set(const snort::Packet*);
};

#endif

rna_fingerprint_ua.h

Path = src/network_inspectors/rna/rna_fingerprint_ua.h

#ifndef RNA_FINGERPRINT_UA_H
#define RNA_FINGERPRINT_UA_H

#include "main/snort_types.h"
#include "search_engines/search_tool.h"

#include "rna_fingerprint.h"

#define MAX_USER_AGENT_DEVICES 16

namespace snort
{

class SO_PUBLIC UaFingerprint : public FpFingerprint
{
public:
    std::string user_agent;
    std::string host_name;
    std::string device;
    uint32_t part_num = 0;
    uint32_t total_parts = 0;

    bool operator==(const UaFingerprint& y) const;
};

class SO_PUBLIC UaFpProcessor
{
public:
    ~UaFpProcessor();

    bool has_pattern()
    { return os_mpse != nullptr; }

    void make_mpse(bool priority = false);

    void match_mpse(const char*, const char*, const UaFingerprint*&, const char*&, bool&);

    void push(const RawFingerprint&);

    void push_agent(const UaFingerprint& uafp)
    { os_fps.emplace_back(uafp); }

    void push_device(const UaFingerprint& uafp)
    { device_fps.emplace_back(uafp); }

    void push_jb(const UaFingerprint& uafp)
    { jb_fps.emplace_back(uafp); }

    void push_jb_host(const UaFingerprint& uafp)
    { jb_host_fps.emplace_back(uafp); }

private:
    std::vector<UaFingerprint> os_fps;
    std::vector<UaFingerprint> device_fps;
    std::vector<UaFingerprint> jb_fps;
    std::vector<UaFingerprint> jb_host_fps;

    snort::SearchTool* os_mpse = nullptr;
    snort::SearchTool* device_mpse = nullptr;
    snort::SearchTool* jb_mpse = nullptr;
    snort::SearchTool* jb_host_mpse = nullptr;
};

} // end of namespace snort

snort::UaFpProcessor* get_ua_fp_processor();
SO_PUBLIC void set_ua_fp_processor(snort::UaFpProcessor*);

#endif

rna_fingerprint_udp.h

Path = src/network_inspectors/rna/rna_fingerprint_udp.h

#ifndef RNA_FINGERPRINT_UDP_H
#define RNA_FINGERPRINT_UDP_H

#include <string>
#include <vector>

#include "rna_fingerprint.h"

namespace snort
{
class SO_PUBLIC DHCPFingerprint : public FpFingerprint
{
public:
    DHCPFingerprint() = default;
    DHCPFingerprint(const RawFingerprint& rfp);
    std::vector<FpElement> dhcp55;
    std::string dhcp60;
};

struct FpDHCPKey
{
    unsigned dhcp55_len;
    unsigned dhcp60_len;
    const uint8_t* dhcp55;
    const uint8_t* dhcp60;
};

class SO_PUBLIC UdpFpProcessor
{
public:
    void push(const RawFingerprint& rfp);
    const DHCPFingerprint* match_dhcp_fingerprint(const FpDHCPKey&) const;

    void push_dhcp_fp(DHCPFingerprint& dhcp_fp)
    {
        dhcp_fps.emplace_back(dhcp_fp);
    }
private:
    std::vector<DHCPFingerprint> dhcp_fps;
};

}

snort::UdpFpProcessor* get_udp_fp_processor();
SO_PUBLIC void set_udp_fp_processor(snort::UdpFpProcessor*);
#endif

rna_flow.h

Path = src/network_inspectors/rna/rna_flow.h

#ifndef RNA_FLOW_H
#define RNA_FLOW_H

#include <memory>
#include <mutex>

#include "flow/flow_data.h"
#include "host_tracker/host_tracker.h"
#include "sfip/sf_ip.h"

#include "rna_fingerprint_tcp.h"

namespace snort
{
struct Packet;
}

class DiscoveryFilter;

using RnaTracker = std::shared_ptr<snort::HostTracker>;

class RNAFlow : public snort::FlowData
{
public:
    FpFingerprintState state;

    RNAFlow() : FlowData(inspector_id) { }
    ~RNAFlow() override;

    static void init();

    void clear_ht(snort::HostTracker& ht);

    static unsigned inspector_id;
    RnaTracker serverht = nullptr;
    RnaTracker clientht = nullptr;

    std::mutex rna_mutex;

    RnaTracker get_server(const snort::SfIp&);
    RnaTracker get_client(const snort::SfIp&);
    RnaTracker get_tracker(const snort::Packet*, DiscoveryFilter&);

    void set_server(const RnaTracker& ht);
    void set_client(const RnaTracker& ht);

};

#endif

rna_inspector.h

Path = src/network_inspectors/rna/rna_inspector.h

#ifndef RNA_INSPECTOR_H
#define RNA_INSPECTOR_H

#include "framework/inspector.h"

#include <string>

namespace snort
{
struct Packet;
class TcpFpProcessor;
class UaFpProcessor;
class UdpFpProcessor;
class SmbFpProcessor;
}

struct RnaConfig;
class RnaModule;
struct RnaModuleConfig;
class RnaPnd;

class SO_PUBLIC RnaInspector : public snort::Inspector
{
public:
    RnaInspector(RnaModule*);
    ~RnaInspector() override;

    bool configure(snort::SnortConfig*) override;
    void install_reload_handler(snort::SnortConfig*) override;
    void eval(snort::Packet*) override;
    void show(const snort::SnortConfig*) const override;
    void tinit() override;
    void tterm() override;

    void get_or_create_fp_processor(snort::TcpFpProcessor*&, snort::UaFpProcessor*&,
        snort::UdpFpProcessor*&, snort::SmbFpProcessor*&);
    void set_fp_processor(snort::TcpFpProcessor*, snort::UaFpProcessor*,
        snort::UdpFpProcessor*, snort::SmbFpProcessor*);

    RnaPnd* get_pnd() const { return pnd; }

private:
    void load_rna_conf();
    RnaModuleConfig* mod_conf = nullptr;
    RnaConfig* rna_conf = nullptr;
    RnaPnd* pnd = nullptr;
};

#endif

rna_logger_common.h

Path = src/network_inspectors/rna/rna_logger_common.h

#ifndef RNA_LOGGER_COMMON_H
#define RNA_LOGGER_COMMON_H

// Common definitions between rna logger and pnd modules
#define RNA_EVENT_NEW       1000
    #define NEW_HOST            1
    #define NEW_TCP_SERVICE     2
    #define NEW_NET_PROTOCOL    3
    #define NEW_XPORT_PROTOCOL  4
    #define NEW_UDP_SERVICE     6
    #define NEW_CLIENT_APP      7
    #define NEW_OS              8

#define RNA_EVENT_CHANGE    1001
    #define CHANGE_HOPS                 5
    #define CHANGE_TCP_SERVICE_INFO     6
    #define CHANGE_UDP_SERVICE_INFO    10
    #define CHANGE_MAC_INFO            13
    #define CHANGE_MAC_ADD             14
    #define CHANGE_HOST_UPDATE         15
    #define CHANGE_HOST_TYPE           16
    #define CHANGE_VLAN_TAG            18
    #define CHANGE_NETBIOS_NAME        21
    #define CHANGE_BANNER_UPDATE       24
    #define CHANGE_CLIENT_APP_UPDATE   32
    #define CHANGE_FULL_DHCP_INFO      33

#define RUA_EVENT         1004
    #define CHANGE_USER_LOGIN    2
    #define FAILED_USER_LOGIN    5

#endif

rna_logger.h

Path = src/network_inspectors/rna/rna_logger.h

#ifndef RNA_LOGGER_H
#define RNA_LOGGER_H

#include "events/event.h"
#include "host_tracker/host_cache.h"
#include "host_tracker/host_tracker.h"
#include "rna_cpe_os.h"
#include "rna_flow.h"

namespace snort
{
class Flow;
struct Packet;
class FpFingerprint;
}

struct RnaLoggerEvent : public Event
{
    RnaLoggerEvent (uint16_t t, uint16_t st, const uint8_t* mc, const RnaTracker* rt,
        const snort::HostMac* hmp, uint16_t pr, void* cv, const snort::HostApplication* hap,
        const snort::FpFingerprint* fpr, const snort::HostClient* hcp, const char* u,
        int32_t app, const char* di, bool jb, uint32_t ls, uint32_t nm,
        const struct in6_addr* rtr, const snort::Packet* p, const char* nb_name,
        const std::vector<const char*>* cpe) : type(t), subtype(st),
        mac(mc), ht(rt), hm(hmp), proto(pr), cond_var(cv), ha(hap), fp(fpr), hc(hcp),
        user(u), appid(app), device_info(di), jail_broken(jb), lease(ls), netmask(nm),
        router(rtr), pkt(p), netbios_name(nb_name), cpe_os(cpe) { }

    uint32_t event_time = 0;
    uint16_t type;
    uint16_t subtype;
    const struct in6_addr* ip = nullptr;
    const uint8_t* mac;
    const RnaTracker* ht;
    const snort::HostMac* hm;
    uint16_t proto;
    void* cond_var;
    const snort::HostApplication* ha;
    const snort::FpFingerprint* fp;
    const snort::HostClient* hc;
    const char* user;
    AppId appid;
    const char* device_info;
    bool jail_broken;
    uint32_t lease;
    uint32_t netmask;
    const struct in6_addr* router;
    const snort::Packet* pkt;
    const char* netbios_name = nullptr;
    const std::vector<const char*>* cpe_os = nullptr;
};

class RnaLogger
{
public:
    RnaLogger(const bool enable) : enabled(enable) { }

    // for host application
    void log(uint16_t type, uint16_t subtype, const snort::Packet* p, RnaTracker* ht,
        const struct in6_addr* src_ip, const uint8_t* src_mac, const snort::HostApplication* ha);

    // for host client
    void log(uint16_t type, uint16_t subtype, const snort::Packet* p, RnaTracker* ht,
        const struct in6_addr* src_ip, const uint8_t* src_mac, const snort::HostClient* hcp);

    // for host user
    void log(uint16_t type, uint16_t subtype, const snort::Packet*, RnaTracker*,
        const struct in6_addr*, const char* user, AppId appid, uint32_t event_time);

    // for cpe os info event
    void log(uint16_t type, uint16_t subtype, const snort::Packet* p, RnaTracker* ht,
        const struct in6_addr* src_ip, const uint8_t* src_mac, const snort::FpFingerprint* fp,
        const std::vector<const char*>* cpeos, uint32_t event_time);

    // for fingerprint
    void log(uint16_t type, uint16_t subtype, const snort::Packet* p, RnaTracker* ht,
        const struct in6_addr* src_ip, const uint8_t* src_mac, const snort::FpFingerprint* fp,
        uint32_t event_time, const char* device_info = nullptr, bool jail_broken = false);

    // for event time
    void log(uint16_t type, uint16_t subtype, const snort::Packet* p, RnaTracker* ht,
        const struct in6_addr* src_ip, const uint8_t* src_mac, uint32_t event_time);

    // for mac event
    void log(uint16_t type, uint16_t subtype, const snort::Packet* p, RnaTracker* ht,
        const struct in6_addr* src_ip, const uint8_t* src_mac,
        const snort::HostMac* hm = nullptr, uint32_t event_time = 0);

    // for protocol event
    void log(uint16_t type, uint16_t subtype, const snort::Packet* p, RnaTracker* ht,
        uint16_t proto, const uint8_t* mac, const struct in6_addr* ip = nullptr,
        uint32_t event_time = 0);

    // for timeout update
    void log(uint16_t type, uint16_t subtype, const snort::Packet* p, const uint8_t* src_mac,
        const struct in6_addr* src_ip, RnaTracker* ht, uint32_t event_time, void* cond_var);

    // for dhcp info event
    void log(uint16_t type, uint16_t subtype, const snort::Packet* p, RnaTracker* ht,
        const struct in6_addr* src_ip, const uint8_t* src_mac, uint32_t lease, uint32_t netmask,
        const struct in6_addr* router);

    // for all
    bool log(uint16_t type, uint16_t subtype, const struct in6_addr* src_ip,
        const uint8_t* src_mac, RnaTracker* ht, const snort::Packet* p = nullptr,
        uint32_t event_time = 0, uint16_t proto = 0, const snort::HostMac* hm = nullptr,
        const snort::HostApplication* ha = nullptr, const snort::FpFingerprint* fp = nullptr,
        void* cond_var = nullptr, const snort::HostClient* hc = nullptr,
        const char* user = nullptr, AppId appid = APP_ID_NONE, const char* device_info = nullptr,
        bool jail_broken = false, uint32_t lease = 0, uint32_t netmask = 0,
        const struct in6_addr* router = nullptr, const char* nb_name = nullptr,
        const std::vector<const char*>* cpeos = nullptr);

private:
    const bool enabled;
};

#endif

rna_mac_cache.h

Path = src/network_inspectors/rna/rna_mac_cache.h

#ifndef RNA_MAC_CACHE
#define RNA_MAC_CACHE

#include "hash/hash_key_operations.h"
// Non-standard, required to include template definition across compiler translation units
#include "host_tracker/cache_allocator.cc"
#include "host_tracker/cache_allocator.h"

#define MAC_CACHE_INITIAL_SIZE 1024 * 1024 // Default to 1 MB

template <class T>
class HostCacheAllocMac : public CacheAlloc<T>
{
public:
    template <class U>
    struct rebind
    {
        typedef HostCacheAllocMac<U> other;
    };

    using CacheAlloc<T>::lru;

    HostCacheAllocMac();
};

class HostTrackerMac
{
public:
    HostTrackerMac()
    { last_seen = (uint32_t) snort::packet_time(); }

    bool add_network_proto(const uint16_t type);
    void update_last_seen(uint32_t p_last_seen);
    void update_vlan(uint16_t vth_pri_cfi_vlan, uint16_t vth_proto);
    bool has_vlan();
    bool has_same_vlan(uint16_t);
    void get_vlan_details(uint8_t& cfi, uint8_t& priority, uint16_t& vid);

    std::vector<uint16_t, HostCacheAllocMac<uint16_t>> get_network_protos()
    {
        std::lock_guard<std::mutex> lck(host_tracker_mac_lock);
        return network_protos;
    }

    uint16_t get_vlan();

    uint32_t get_last_seen()
    {
        std::lock_guard<std::mutex> lck(host_tracker_mac_lock);
        return last_seen;
    }

    uint32_t get_last_event()
    {
        std::lock_guard<std::mutex> lck(host_tracker_mac_lock);
        return last_event;
    }

    bool delete_proto(uint16_t proto)
    {
        std::lock_guard<std::mutex> lck(host_tracker_mac_lock);

        for (auto it = network_protos.begin(); it != network_protos.end(); ++it)
        {
            if (*it == proto)
            {
                network_protos.erase(it);
                return true;
            }
        }

        return false;
    }

    void update_last_seen()
    {
        std::lock_guard<std::mutex> lck(host_tracker_mac_lock);
        last_seen = (uint32_t) snort::packet_time();
    }

    void update_last_event(uint32_t event_time)
    {
        std::lock_guard<std::mutex> lck(host_tracker_mac_lock);
        last_event = event_time ? event_time : last_seen;
    }

    // This should be updated whenever HostTrackerMac data members are changed
    void stringify(std::string& str);

    snort::HostType host_type = snort::HostType::HOST_TYPE_HOST;

private:
    std::mutex host_tracker_mac_lock;
    uint32_t last_seen = 0;
    uint32_t last_event = 0;
    bool vlan_tag_present = false;
    snort::vlan::VlanTagHdr vlan_tag = { };
    std::vector<uint16_t, HostCacheAllocMac<uint16_t>> network_protos;
};

class MacKey
{
public:
    MacKey(const uint8_t *ma)
    { memcpy(mac_addr, ma, MAC_SIZE); }

    bool operator==(const MacKey& mk) const
    {
        return !(memcmp(mac_addr, mk.mac_addr, MAC_SIZE));
    }

    uint8_t mac_addr[MAC_SIZE];
};

struct HashMac
{
    uint64_t operator()(const MacKey& mk) const
    { return snort::hash_mac(mk.mac_addr); }
};

typedef LruCacheSharedMemcap<MacKey, HostTrackerMac, HashMac> HostCacheMac;

extern HostCacheMac* get_host_cache_mac();

template <class T>
HostCacheAllocMac<T>::HostCacheAllocMac()
{
    lru = get_host_cache_mac();
}
#endif

rna_module.h

Path = src/network_inspectors/rna/rna_module.h

#ifndef RNA_MODULE_H
#define RNA_MODULE_H

#include "framework/module.h"
#include "main/reload_tuner.h"
#include "profiler/profiler.h"
#include "trace/trace_api.h"

#include "rna_config.h"
#include "rna_fingerprint.h"
#include "rna_mac_cache.h"
#include "rna_name.h"

namespace snort
{
struct SnortConfig;
}

struct RnaStats
{
    PegCount appid_change;
    PegCount cpe_os;
    PegCount icmp_bidirectional;
    PegCount icmp_new;
    PegCount ip_bidirectional;
    PegCount ip_new;
    PegCount udp_bidirectional;
    PegCount udp_new;
    PegCount tcp_syn;
    PegCount tcp_syn_ack;
    PegCount tcp_midstream;
    PegCount other_packets;
    PegCount change_host_update;
    PegCount dhcp_data;
    PegCount dhcp_info;
    PegCount smb;
    PegCount netflow_record;
    PegCount total_events_in_interval;
    PegCount total_packets_in_interval;
    PegCount total_bytes_in_interval;
};

extern THREAD_LOCAL RnaStats rna_stats;
extern THREAD_LOCAL snort::ProfileStats rna_perf_stats;
extern THREAD_LOCAL const snort::Trace* rna_trace;


// A tuner for initializing fingerprint processors during reload
class FpProcReloadTuner : public snort::ReloadResourceTuner
{
public:
    explicit FpProcReloadTuner(RnaModuleConfig& mod_conf)
        : mod_conf(mod_conf) { }
    ~FpProcReloadTuner() override = default;

    bool tinit() override;

    bool tune_packet_context() override
    { return true; }

    bool tune_idle_context() override
    { return true; }

private:
    RnaModuleConfig& mod_conf;
};

class RnaModule : public snort::Module
{
public:
    RnaModule();
    ~RnaModule() override;

    bool begin(const char*, int, snort::SnortConfig*) override;
    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;
    bool log_mac_cache(const char* outfile);

    const snort::Command* get_commands() const override;
    RnaModuleConfig* get_config();
    PegCount* get_counts() const override;
    const PegInfo* get_pegs() const override;
    snort::ProfileStats* get_profile() const override;

    Usage get_usage() const override
    { return CONTEXT; }

    void set_trace(const snort::Trace*) const override;
    const snort::TraceOption* get_trace_options() const override;

private:
    RnaModuleConfig* mod_conf = nullptr;
    const char* dump_file = nullptr;

    RawFingerprint fingerprint;

    bool is_valid_fqn(const char* fqn) const;
};

#endif

rna_name.h

Path = src/network_inspectors/rna/rna_name.h

#ifndef RNA_NAME_H
#define RNA_NAME_H

#define RNA_NAME "rna"
#define RNA_HELP "Real-time network awareness and OS fingerprinting (experimental)"

#endif

rna_pnd.h

Path = src/network_inspectors/rna/rna_pnd.h

#ifndef RNA_PND_H
#define RNA_PND_H

#include <climits>

#include "helpers/discovery_filter.h"
#include "host_tracker/host_tracker.h"
#include "protocols/eth.h"
#include "protocols/layer.h"
#include "protocols/packet.h"
#include "protocols/tcp.h"
#include "protocols/vlan.h"
#include "pub_sub/appid_events.h"
#include "pub_sub/dhcp_events.h"
#include "pub_sub/netflow_event.h"
#include "pub_sub/smb_events.h"
#include "sfip/sf_ip.h"

#include "rna_config.h"
#include "rna_logger.h"
#include "rna_mac_cache.h"

enum class TcpPacketType
{
    SYN, SYN_ACK, MIDSTREAM
};

#pragma pack(1)
struct RNA_LLC
{
    union
    {
        struct
        {
            uint8_t DSAP;
            uint8_t SSAP;
        } s;
        uint16_t proto;
    } s;
    uint8_t flags;
};
#pragma pack()

static inline bool is_eligible_packet(const snort::Packet* p)
{
    if ( p->has_ip() or
        memcmp(snort::layer::get_eth_layer(p)->ether_src, snort::zero_mac, MAC_SIZE) )
        return true;
    return false;
}

static inline bool is_eligible_ip(const snort::Packet* p)
{
    // If payload needs to be inspected ever, allow rebuilt packet when is_proxied
    if ( !p->has_ip() or p->is_rebuilt() or !p->flow )
        return false;
    return true;
}

static inline bool is_eligible_tcp(const snort::Packet* p)
{
    if ( !is_eligible_ip(p) or p->ptrs.tcph->is_rst() )
        return false;
    return true;
}

static inline bool is_eligible_udp(const snort::Packet* p)
{
    if ( !is_eligible_ip(p) )
        return false;
    if ( p->is_from_client() )
    {
        const snort::SfIp* src = p->ptrs.ip_api.get_src();
        const snort::SfIp* dst = p->ptrs.ip_api.get_dst();
        // FIXIT-M this code checking the v6 address unconditionally is almost certainly wrong,
        //          especially since it's looking for an IPv4-specific protocol
        if ( !src->is_set() and ((const uint8_t *) dst->get_ip6_ptr())[0] == 0XFF and
            p->ptrs.sp == 68 and p->ptrs.dp == 67 )
            return false; // skip BOOTP
    }
    return true;
}

static inline unsigned short rna_get_eth(const snort::Packet* p)
{
    const snort::vlan::VlanTagHdr* vh = nullptr;
    const snort::eth::EtherHdr* eh = nullptr;

    if (p->proto_bits & PROTO_BIT__VLAN)
        vh = snort::layer::get_vlan_layer(p);

    if (vh)
        return ntohs(vh->vth_proto);
    else if ((eh = snort::layer::get_eth_layer(p)))
        return ntohs(eh->ether_type);
    return USHRT_MAX;
}

class RnaPnd
{
public:

    RnaPnd(const bool en, const std::string& cp, RnaConfig* rc = nullptr);
    ~RnaPnd();

    void analyze_appid_changes(snort::DataEvent&);
    void analyze_flow_icmp(const snort::Packet*);
    void analyze_flow_ip(const snort::Packet*);
    void analyze_flow_non_ip(const snort::Packet*);
    void analyze_flow_tcp(const snort::Packet*, TcpPacketType);
    void analyze_flow_udp(const snort::Packet*);
    void analyze_dhcp_fingerprint(snort::DataEvent&);
    void add_dhcp_info(snort::DataEvent&);
    void analyze_smb_fingerprint(snort::DataEvent&);
    bool analyze_cpe_os_info(snort::DataEvent&);
    bool analyze_netflow(snort::DataEvent&);
    void analyze_netflow_host(snort::NetFlowEvent*);
    void analyze_netflow_service(snort::NetFlowEvent*);

    // generate change event for all hosts in the ip cache
    void generate_change_host_update();

    static HostCacheIp::Data find_or_create_host_tracker(const snort::SfIp&, bool&);

private:
    // generate change event for single host
    void generate_change_host_update(RnaTracker*, const snort::Packet*,
        const snort::SfIp*, const uint8_t* src_mac, const time_t&);
    void generate_change_host_update_eth(HostTrackerMac*, const snort::Packet*,
        const uint8_t* src_mac, const time_t&);

    void discover_host_types_ttl(RnaTracker&, const snort::Packet*, uint8_t pkt_ttl,
        uint32_t last_seen, const struct in6_addr*, const uint8_t* src_mac);
    int discover_host_types_icmpv6_ndp(RnaTracker& ht, const snort::Packet*, uint32_t last_seen,
        const struct in6_addr* src_ip, const uint8_t* src_mac);

    // Change vlan event related utilities
    inline void update_vlan(const snort::Packet*, HostTrackerMac&);
    void generate_change_vlan_update(RnaTracker*, const snort::Packet*,
        const uint8_t* src_mac, HostTrackerMac&, bool isnew);
    void generate_change_vlan_update(RnaTracker*, const snort::Packet*,
        const uint8_t* src_mac, const snort::SfIp*, bool isnew);

    void generate_new_host_mac(const snort::Packet*, RnaTracker, bool discover_proto = false);

    // General rna utilities not associated with flow
    void discover_network_icmp(const snort::Packet*);
    void discover_network_ip(const snort::Packet*);
    void discover_network_non_ip(const snort::Packet*);
    void discover_network_tcp(const snort::Packet*);
    void discover_network_udp(const snort::Packet*);
    void discover_network(const snort::Packet*, uint8_t ttl);

    // RNA utilities for non-IP packets
    void discover_network_ethernet(const snort::Packet*);
    int discover_network_arp(const snort::Packet*, RnaTracker*);
    int discover_network_bpdu(const snort::Packet*, const uint8_t* data, RnaTracker);
    int discover_network_cdp(const snort::Packet*, const uint8_t* data, uint16_t rlen,
        RnaTracker&);

    int discover_switch(const snort::Packet*, RnaTracker);

    RnaLogger logger;
    DiscoveryFilter filter;
    RnaConfig* conf;
    time_t update_timeout;
};

HostCacheMac* get_host_cache_mac();
void set_host_cache_mac(HostCacheMac* mac_host);

#endif

network_inspectors/rna/test/

rna_module_mock.h

Path = src/network_inspectors/rna/test/rna_module_mock.h

#ifndef RNA_MODULE_MOCK_H
#define RNA_MODULE_MOCK_H

#include "../rna_mac_cache.cc"

THREAD_LOCAL RnaStats rna_stats;
THREAD_LOCAL ProfileStats rna_perf_stats;

namespace snort
{
Module::Module(const char* s, const char*, const Parameter*, bool)
    : name(s), help(nullptr), params(nullptr), list(false)
{ }
bool TcpFingerprint::operator==(const TcpFingerprint&) const { return true; }

// inspector
class RnaInspector
{
public:

// The module gets created first, with a mod_conf and fingerprint processor,
// then, when the module is done, we take ownership of that.
RnaInspector(RnaModule* mod)
{
    mod_conf = mod->get_config();
}

~RnaInspector()
{
    if (mod_conf)
    {
        delete mod_conf->tcp_processor;
        delete mod_conf->ua_processor;
        delete mod_conf;
    }
}

TcpFpProcessor* get_fp_processor()
{
    return mod_conf->tcp_processor;
}

private:
    RnaModuleConfig* mod_conf = nullptr;
};

} // end of namespace snort

static ControlConn s_ctrlcon(1, true);
ControlConn::ControlConn(int, bool) : shell(nullptr), fd(-1), touched(0)
{}
ControlConn::~ControlConn() {}

#endif

rna_module_stubs.h

Path = src/network_inspectors/rna/test/rna_module_stubs.h

#ifndef RNA_MODULE_TEST_H
#define RNA_MODULE_TEST_H

namespace snort
{
Module* ModuleManager::get_module(const char*)
{ return nullptr; }

char* snort_strdup(const char* s)
{ return strdup(s); }

void Module::sum_stats(bool) {}
void Module::main_accumulate_stats() {}
void Module::show_stats() {}
void Module::init_stats(bool) {}
void Module::reset_stats() {}
PegCount Module::get_global_count(char const*) const
{ return 0; }
void Module::show_interval_stats(std::vector<unsigned int, std::allocator<unsigned int> >&, FILE*)
{}
void LogMessage(const char*,...) {}
void WarningMessage(const char*,...) {}
DataBus::DataBus() = default;
DataBus::~DataBus() = default;
SnortConfig::SnortConfig(const SnortConfig* const, const char*) {}
SnortConfig::~SnortConfig() = default;
time_t packet_time() { return 0; }

// tcp fingerprint functions
bool TcpFpProcessor::push(const TcpFingerprint&) { return true; }
void TcpFpProcessor::make_tcp_fp_tables(TCP_FP_MODE) { }
const TcpFingerprint* TcpFpProcessor::get_tcp_fp(const FpTcpKey&, uint8_t, TCP_FP_MODE) const
{ return nullptr; }
const TcpFingerprint* TcpFpProcessor::get(const Packet*, RNAFlow*) const
{ return nullptr; }
TcpFpProcessor* get_tcp_fp_processor() { return nullptr; }
void set_tcp_fp_processor(TcpFpProcessor*) { }

TcpFingerprint::TcpFingerprint(const RawFingerprint&) { }

UaFpProcessor::~UaFpProcessor() = default;
void UaFpProcessor::make_mpse(bool) { }
void UaFpProcessor::push(RawFingerprint const&) { }

void UdpFpProcessor::push(RawFingerprint const&) { }

SmbFingerprint::SmbFingerprint(const RawFingerprint&) { }
bool SmbFingerprint::operator==(const SmbFingerprint&) const { return true; }
bool SmbFpProcessor::push(SmbFingerprint const&) { return true; }
}

void snort::main_broadcast_command(snort::AnalyzerCommand*, ControlConn*) {}

ControlConn* ControlConn::query_from_lua(const lua_State*) { return &s_ctrlcon; }
bool ControlConn::respond(const char*, ...) { return true; }

HostCacheMac* get_host_cache_mac() { return nullptr; }

DataPurgeAC::~DataPurgeAC() = default;
bool DataPurgeAC::execute(Analyzer&, void**) { return true;}

void set_host_cache_mac(HostCacheMac*) { }

Inspector* InspectorManager::get_inspector(const char*, bool, const SnortConfig*)
{
    return nullptr;
}

void HostTracker::remove_flows() { }

#endif

packet_io/

This unit is the interface for incoming and outgoing packets and manages the DAQ.

There is one DAQ instance per active source (interface, pcap, etc.). The DAQ determines the required root decoder, instantiated upon thread initialization, and which remains the same for all packets.

The other modules use the Active interface to detain packets. A packet will not be held if it would drop the the available DAQ message pool down below the DAQ batch size. DAQ batch size (the number of packets Snort can process in batch mode) can be configured using this command line option --daq-batch-size and the pool size is obtained using a DAQ API call: daq_instance_get_msg_pool_info(DAQ_Instance_h, DAQ_MsgPoolInfo_t)

active_action.h

Path = src/packet_io/active_action.h

#ifndef ACTIVE_ACTION_H
#define ACTIVE_ACTION_H

#include "main/snort_types.h"

namespace snort
{
struct Packet;

enum ActionPriority
{
    AP_LOCAL,
    AP_MODIFY,
    AP_PROXY,
    AP_RESET,
    AP_REMOTE,
    AP_MAX
};

// These are injection actions (e.g. send a RST packet, or respond to a query
// with an "Access denied" message). Instances of this class are queued into
// the packet at inspection / detection time and executed at the end by
// Analyzer. The pure virtual exec() method can call into Active methods for
// the low-level stuff.
class SO_PUBLIC ActiveAction
{
public:
    ActiveAction(ActionPriority a = ActionPriority::AP_MAX) : action(a) {}
    virtual ~ActiveAction() = default;

    virtual void delayed_exec(Packet* ) { }

    ActionPriority get_action() const { return action; }

protected:
    ActionPriority action;

};

}

#endif

active.h

Path = src/packet_io/active.h

#ifndef ACTIVE_H
#define ACTIVE_H

// manages packet processing verdicts returned to the DAQ.  action (what to
// do) is separate from status (whether we can actually do it or not).
#include "protocols/packet_manager.h"

namespace snort
{
struct Packet;
struct SnortConfig;
class ActiveAction;

class SO_PUBLIC Active
{
public:
    Active() = default;
    ~Active() = default;

    struct Counts
    {
        PegCount injects;
        PegCount failed_injects;
        PegCount direct_injects;
        PegCount failed_direct_injects;
        PegCount holds_denied;
        PegCount holds_canceled;
        PegCount holds_allowed;
    };

    enum ActiveStatus : uint8_t
    { AST_ALLOW, AST_CANT, AST_WOULD, AST_FORCE, AST_MAX };

    enum ActiveWouldReason : uint8_t
    { WHD_NONE, WHD_INTERFACE_IDS, WHD_IPS_INLINE_TEST, WHD_NAP_INLINE_TEST, WHD_TIMEOUT, WHD_PRUNE, WHD_RELOAD, WHD_EXIT };

    enum ActiveSuspendReason : uint8_t
    { ASP_NONE, ASP_PRUNE, ASP_TIMEOUT, ASP_RELOAD, ASP_EXIT };

    // FIXIT-M: these are only used in set_delayed_action and
    // apply_delayed_action, in a big switch(action). Do away with these and
    // use the actual (Base)Action objects.
    enum ActiveActionType : uint8_t
    { ACT_TRUST, ACT_ALLOW, ACT_HOLD, ACT_RETRY, ACT_REWRITE, ACT_DROP, ACT_BLOCK, ACT_RESET, ACT_MAX };

public:

    static bool thread_init(const SnortConfig*);
    static void thread_term();

    static void suspend(ActiveSuspendReason suspend_reason)
    {
        s_suspend = true;
        s_suspend_reason = suspend_reason;
    }

    static bool is_suspended()
    { return s_suspend; }

    static void resume()
    {
        s_suspend = false;
        s_suspend_reason = ASP_NONE;
    }

    static ActiveWouldReason get_whd_reason_from_suspend_reason()
    {
        switch ( s_suspend_reason )
        {
        case ASP_NONE: return WHD_NONE;
        case ASP_PRUNE: return WHD_PRUNE;
        case ASP_TIMEOUT: return WHD_TIMEOUT;
        case ASP_RELOAD: return WHD_RELOAD;
        case ASP_EXIT: return WHD_EXIT;
        }
        return WHD_NONE;
    }

    void send_reset(Packet*, EncodeFlags);
    void send_unreach(Packet*, snort::UnreachResponse);
    uint32_t send_data(Packet*, EncodeFlags, const uint8_t* buf, uint32_t len);
    void inject_data(Packet*, EncodeFlags, const uint8_t* buf, uint32_t len);

    bool is_reset_candidate(const Packet*);
    bool is_unreachable_candidate(const Packet*);

    ActiveActionType get_action() const
    { return active_action; }

    ActiveStatus get_status() const
    { return active_status; }

    void kill_session(Packet*, EncodeFlags = ENC_FLAG_FWD);

    bool can_act() const
    { return active_status == AST_ALLOW or active_status == AST_FORCE; }

    const char* get_action_string() const
    { return act_str[active_action][active_status]; }

    const char* get_delayed_action_string() const
    { return act_str[delayed_active_action][active_status]; }

    void update_status(const Packet*, bool force = false);

    void drop_packet(const Packet*, bool force = false);
    void daq_drop_packet(const Packet*);
    void rewrite_packet(const Packet*, bool force = false);
    bool retry_packet(const Packet*);
    bool hold_packet(const Packet*);
    void cancel_packet_hold();

    void trust_session(Packet*, bool force = false);
    void block_session(Packet*, bool force = false);
    void reset_session(Packet*, bool force = false);
    void reset_session(Packet*, snort::ActiveAction* r, bool force = false);

    static void queue(snort::ActiveAction* a, snort::Packet* p);
    static void clear_queue(snort::Packet*);
    static void execute(Packet* p);

    void block_again()
    { active_action = ACT_BLOCK; }

    void reset_again()
    { active_action = ACT_RESET; }

    void set_trust()
    { active_action = ACT_TRUST; }

    void set_allow()
    { active_action = ACT_ALLOW; }

    bool packet_was_dropped() const
    { return active_action >= ACT_DROP; }

    bool packet_would_be_dropped() const
    { return active_status == AST_WOULD; }

    ActiveWouldReason get_would_be_dropped_reason() const
    { return active_would_reason; }

    bool can_partial_block_session() const
    { return active_status == AST_CANT and s_suspend_reason > ASP_NONE and s_suspend_reason != ASP_TIMEOUT; }

    bool keep_pruned_flow() const
    { return ( s_suspend_reason == ASP_PRUNE ) or ( s_suspend_reason == ASP_RELOAD ); }

    bool keep_timedout_flow() const
    { return ( s_suspend_reason == ASP_TIMEOUT ); }

    bool packet_retry_requested() const
    { return active_action == ACT_RETRY; }

    bool session_was_trusted() const
    { return active_action == ACT_TRUST; }

    bool session_was_blocked() const
    { return active_action >= ACT_BLOCK; }

    bool session_was_allowed() const
    { return active_action <= ACT_ALLOW; }

    bool packet_force_dropped() const
    { return active_status == AST_FORCE; }

    bool is_packet_held() const
    { return active_action == ACT_HOLD; }

    void set_tunnel_bypass()
    { active_tunnel_bypass++; }

    void clear_tunnel_bypass()
    { active_tunnel_bypass--; }

    bool get_tunnel_bypass() const
    { return active_tunnel_bypass > 0; }

    ActiveActionType get_delayed_action() const
    { return delayed_active_action; }

    void set_delayed_action(ActiveActionType, bool force = false);
    void set_delayed_action(ActiveActionType, ActiveAction* act, bool force = false);
    void apply_delayed_action(Packet*);

    void reset();

    static void set_default_drop_reason(uint8_t reason_id);
    static void map_drop_reason_id(const char* verdict_reason, uint8_t id);
    void set_drop_reason(const char*);
    void send_reason_to_daq(Packet&);

    const char* get_drop_reason()
    { return drop_reason; }

private:
    static bool open(const char*);
    static void close();
    static int send_eth(DAQ_Msg_h, int, const uint8_t* buf, uint32_t len);
    static int send_ip(DAQ_Msg_h, int, const uint8_t* buf, uint32_t len);

    void update_status_actionable(const Packet*);
    void daq_update_status(const Packet*);

    void block_session(const Packet*, ActiveActionType, bool force = false);

    void cant_drop();

    int get_drop_reason_id();

private:
    static const char* act_str[ACT_MAX][AST_MAX];
    static THREAD_LOCAL uint8_t s_attempts;
    static THREAD_LOCAL bool s_suspend;
    static THREAD_LOCAL ActiveSuspendReason s_suspend_reason;

    int active_tunnel_bypass = 0;
    const char* drop_reason = nullptr;

    // these can't be pkt flags because we do the handling
    // of these flags following all processing and the drop
    // or response may have been produced by a pseudopacket.
    ActiveStatus active_status = AST_MAX;
    ActiveWouldReason active_would_reason = WHD_EXIT;
    ActiveActionType active_action = ACT_MAX;
    ActiveActionType delayed_active_action = ACT_MAX;
    ActiveAction* delayed_reject = nullptr; // set with set_delayed_action()
};

struct SO_PUBLIC ActiveSuspendContext
{
    ActiveSuspendContext(Active::ActiveSuspendReason suspend_reason)
    { Active::suspend(suspend_reason); }

    ~ActiveSuspendContext()
    { Active::resume(); }
};

extern THREAD_LOCAL Active::Counts active_counts;
}
#endif

sfdaq_config.h

Path = src/packet_io/sfdaq_config.h

#ifndef SFDAQ_CONFIG_H
#define SFDAQ_CONFIG_H

#include <cstdint>
#include <string>
#include <vector>

using DaqVar = std::pair<std::string, std::string>;
using DaqVarList = std::vector<DaqVar>;

/* Module configuration */
struct SFDAQModuleConfig
{
    enum SFDAQMode
    {
        SFDAQ_MODE_UNSET,
        SFDAQ_MODE_PASSIVE,
        SFDAQ_MODE_INLINE,
        SFDAQ_MODE_READ_FILE,
    };

    SFDAQModuleConfig() = default;
    SFDAQModuleConfig(const SFDAQModuleConfig&);
    void set_variable(const char* varkvp);

    std::string name;
    SFDAQMode mode = SFDAQ_MODE_UNSET;
    DaqVarList variables;
};

/* General/base configuration */
struct SFDAQConfig
{
    SFDAQConfig();
    ~SFDAQConfig();

    void add_input(const char*);
    SFDAQModuleConfig* add_module_config(const char* module_name);
    void add_module_dir(const char*);
    void set_batch_size(uint32_t);
    void set_mru_size(int);

    uint32_t get_batch_size() const { return (batch_size == BATCH_SIZE_UNSET) ? BATCH_SIZE_DEFAULT : batch_size; }
    uint32_t get_mru_size() const { return (mru_size == SNAPLEN_UNSET) ? SNAPLEN_DEFAULT : mru_size; }

    void overlay(const SFDAQConfig*);

    /* General configuration */
    std::vector<std::string> module_dirs;
    /* Instance configuration */
    std::vector<std::string> inputs;
    uint32_t batch_size;
    int mru_size;
    unsigned int timeout;
    std::vector<SFDAQModuleConfig*> module_configs;

    /* Constants */
    static constexpr uint32_t BATCH_SIZE_UNSET = 0;
    static constexpr int SNAPLEN_UNSET = -1;
    static constexpr uint32_t BATCH_SIZE_DEFAULT = 64;
    static constexpr int SNAPLEN_DEFAULT = 1518;
    static constexpr unsigned TIMEOUT_DEFAULT = 1000;
};

#endif

sfdaq.h

Path = src/packet_io/sfdaq.h

#ifndef SFDAQ_H
#define SFDAQ_H

#include <daq_common.h>

#include <ostream>

#include "main/snort_types.h"

struct SFDAQConfig;

namespace snort
{
class SFDAQInstance;

class SFDAQ
{
public:
    static void load(const SFDAQConfig*);
    static void unload();

    static void print_types(std::ostream&);
    static const char* verdict_to_string(DAQ_Verdict verdict);
    static bool init(const SFDAQConfig*, unsigned total_instances);
    static void term();

    static bool init_instance(SFDAQInstance*, const std::string& bpf_string);

    static const char* get_input_spec(const SFDAQConfig*, unsigned instance_id);
    static const char* default_type();
    SO_PUBLIC static const DAQ_Stats_t* get_stats();

    static bool can_inject();
    static bool can_inject_raw();
    static bool can_replace();
    static bool can_run_unprivileged();
    SO_PUBLIC static bool get_tunnel_bypass(uint16_t proto);

    // FIXIT-M X Temporary thread-local instance helpers to be removed when no longer needed
    static void set_local_instance(SFDAQInstance*);

    SO_PUBLIC static SFDAQInstance* get_local_instance();
    SO_PUBLIC static const char* get_input_spec();
    SO_PUBLIC static int get_base_protocol();

    static int inject(DAQ_Msg_h, int rev, const uint8_t* buf, uint32_t len);
    SO_PUBLIC static bool forwarding_packet(const DAQ_PktHdr_t*);
};
}
#endif

sfdaq_instance.h

Path = src/packet_io/sfdaq_instance.h

#ifndef SFDAQ_INSTANCE_H
#define SFDAQ_INSTANCE_H

#include <daq_common.h>

#include <string>

#include "main/snort_types.h"
#include "protocols/protocol_ids.h"

struct SFDAQConfig;

namespace snort
{
struct Packet;
struct SfIp;

class SFDAQInstance
{
public:
    SFDAQInstance(const char* intf, unsigned id, const SFDAQConfig*);
    ~SFDAQInstance();

    bool init(DAQ_Config_h, const std::string& bpf_string);

    bool start();
    bool was_started() const;
    bool stop();
    void reload();

    DAQ_RecvStatus receive_messages(unsigned max_recv);
    DAQ_Msg_h next_message()
    {
        if (curr_batch_idx < curr_batch_size)
            return daq_msgs[curr_batch_idx++];
        return nullptr;
    }
    int finalize_message(DAQ_Msg_h msg, DAQ_Verdict verdict);
    const char* get_error();

    int get_base_protocol() const;
    uint32_t get_batch_size() const { return batch_size; }
    uint32_t get_pool_available() const { return pool_available; }
    const char* get_input_spec() const;
    const DAQ_Stats_t* get_stats();

    bool can_inject() const;
    bool can_inject_raw() const;
    bool can_replace() const;
    bool can_start_unprivileged() const;
    SO_PUBLIC bool can_whitelist() const;

    int inject(DAQ_Msg_h, int rev, const uint8_t* buf, uint32_t len);
    bool interrupt();

    SO_PUBLIC int ioctl(DAQ_IoctlCmd cmd, void *arg, size_t arglen);
    SO_PUBLIC int modify_flow_opaque(DAQ_Msg_h, uint32_t opaque);
    int set_packet_verdict_reason(DAQ_Msg_h msg, uint8_t verdict_reason);
    int set_packet_trace_data(DAQ_Msg_h, uint8_t* buff, uint32_t buff_len);
    int add_expected(const Packet* ctrlPkt, const SfIp* cliIP, uint16_t cliPort,
            const SfIp* srvIP, uint16_t srvPort, IpProtocol, unsigned timeout_ms,
            unsigned flags);
    bool get_tunnel_bypass(uint16_t proto);

private:
    void get_tunnel_capabilities();

    std::string input_spec;
    uint32_t instance_id;
    DAQ_Instance_h instance = nullptr;
    DAQ_Msg_h* daq_msgs;
    unsigned curr_batch_size = 0;
    unsigned curr_batch_idx = 0;
    uint32_t batch_size;
    uint32_t pool_size = 0;
    uint32_t pool_available = 0;
    int dlt = -1;
    DAQ_Stats_t daq_instance_stats = { };
    uint16_t daq_tunnel_mask = 0;
};
}
#endif

sfdaq_module.h

Path = src/packet_io/sfdaq_module.h

#ifndef SFDAQ_MODULE_H
#define SFDAQ_MODULE_H

#include "framework/module.h"

namespace snort
{
struct SnortConfig;
}
struct SFDAQConfig;
struct SFDAQModuleConfig;

class SFDAQModule : public snort::Module
{
public:
    SFDAQModule();

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    void prep_counts(bool dump_stats) override;
    void reset_stats() override;

    bool counts_need_prep() const override
    { return true; }

    Usage get_usage() const override
    { return GLOBAL; }

private:
    SFDAQConfig* config;
    SFDAQModuleConfig* module_config;
};

struct DAQStats
{
    PegCount pcaps;
    PegCount received;
    PegCount analyzed;
    PegCount dropped;
    PegCount filtered;
    PegCount outstanding;
    PegCount outstanding_max;
    PegCount injected;
    PegCount verdicts[MAX_DAQ_VERDICT];
    PegCount internal_blacklist;
    PegCount internal_whitelist;
    PegCount skipped;
    PegCount idle;
    PegCount rx_bytes;
    PegCount expected_flows;
    PegCount retries_queued;
    PegCount retries_dropped;
    PegCount retries_processed;
    PegCount retries_discarded;
    PegCount sof_messages;
    PegCount eof_messages;
    PegCount other_messages;
};

extern THREAD_LOCAL DAQStats daq_stats;

#endif

trough.h

Path = src/packet_io/trough.h

#ifndef TROUGH_H
#define TROUGH_H

#include <atomic>
#include <string>
#include <vector>

// Trough provides access to sources (interface, file, etc.).

class Trough
{
public:
    enum SourceType
    {
        SOURCE_FILE_LIST,  // a file containing a list of sources
        SOURCE_LIST,       // a list of sources (eg from cmd line)
        SOURCE_DIR         // a directory of sources; often used with filter
    };

    static void set_loop_count(unsigned c)
    {
        pcap_loop_count = c;
    }
    static void set_filter(const char *f);
    static void add_source(SourceType type, const char *list);
    static void setup();
    static bool has_next();
    static const char *get_next();
    static unsigned get_file_count()
    {
        return file_count;
    }
    static void clear_file_count()
    {
        file_count = 0;
    }
    static unsigned get_queue_size()
    {
        return pcap_queue.size();
    }
    static unsigned get_loop_count()
    {
        return pcap_loop_count;
    }
    static void cleanup();
private:
    struct PcapReadObject
    {
        SourceType type;
        std::string arg;
        std::string filter;
    };

    static bool add_pcaps_dir(const std::string& dirname, const std::string& filter);
    static bool add_pcaps_list_file(const std::string& list_filename, const std::string& filter);
    static bool add_pcaps_list(const std::string& list);
    static bool get_pcaps(const std::vector<struct PcapReadObject> &pol);

    static std::vector<struct PcapReadObject> pcap_object_list;
    static std::vector<std::string> pcap_queue;
    static std::vector<std::string>::const_iterator pcap_queue_iter;
    static std::string pcap_filter;

    static unsigned pcap_loop_count;
    static std::atomic<unsigned> file_count;
};

#endif

packet_io/test/

parser/

This unit supports parsing of command line args, detection rules, IP addresses, and config files. New Lua-based features are elsewhere.

Rules

A state machine parser expects a rule in the following format:

Action Proto Source Dir Destination ( Options )

Where Action is an IPS Action, Source/Destination is a source or destination IP address and port if any, Dir is a marker for uni-directional or bi-directional rule, Options is a list of IPS options including matchers and meta-data. Some of the tokens can be omitted, taking a default value.

The left part of the rule (before the opening parenthesis) is a header, which is represented by RuleTreeNode structure internally, the RTN.

The right part of the rule (in the parentheses) is a rule body, which is just a list of IPS options, delimited by a semi-colon. Internally this part is represented by OptTreeNode structure, the OTN. During parsing the options, the right IPS Option module is selected. It will parse the option and its parameters and create an instance.

RTN is a per-policy data. Within a given policy all RTNs which have the same action and 5-tuple (protocol, addresses, ports values after expanding variables) will be collapsed into a single RTN. When strict reduction is enabled, port variables with the same values will also be collapsed into single RTNs. This reduces the memory and computation efforts to evaluate RTN. Hence, RTN has a counter for OTNs referencing it. RTNs populate policies when the rule states are applied.

OTN is a global data. It is uniquely identified by gid:sid pair. So, a text rule, a rule state or stub can reference the same rule:

ips.rules = [[ reject http ( gid:1; sid:1; ) ]]
ips.states = [[ alert ( gid:1; sid:1; ) ]]

In the example above the state will affect the current policy, while the rule will be loaded and become available in all policies (with inherited or overridden state). OTN has a pointer to the right RTN for each policy.

After all the rules are read and parsed, Detection module will create rule groups and options trees to evaluate rules faster.

IP and Port Variables

IP or Port value (either a scalar or a list) can be represented by a user variable. Each IPS policy has a table which maps variable name to a specific value.

ips.variables.nets.HOME_NET = 192.168.0.0/24
ips.variables.ports.STATIC_NAT_PORT = '8080, 40000:'

RTN gets expanded values when the rule state is applied.

arg_list.h

Path = src/parser/arg_list.h

#ifndef ARG_LIST_H
#define ARG_LIST_H

#include <string>

class ArgList
{
public:
    ArgList(int c, char* v[])
    { argc = c; argv = v; reset(); }

    void reset()
    { idx = 0; arg = nullptr; }

    bool get_arg(const char*& key, const char*& val);
    void dump();

private:
    char** argv;
    int argc, idx;
    const char* arg;
    std::string buf;
};

#endif

cmd_line.h

Path = src/parser/cmd_line.h

#ifndef CMD_LINE_H
#define CMD_LINE_H

namespace snort
{
struct SnortConfig;
}

snort::SnortConfig* parse_cmd_line(int argc, char* argv[]);

#endif

config_file.h

Path = src/parser/config_file.h

#ifndef CONFIG_FILE_H
#define CONFIG_FILE_H

const char* get_snort_conf();
const char* get_snort_conf_dir();

void ConfigChecksumDrop(const char*);
void ConfigChecksumMode(const char*);

void config_conf(const char*);
void SetSnortConfDir(const char*);

#endif

parse_conf.h

Path = src/parser/parse_conf.h

#ifndef PARSE_CONF_H
#define PARSE_CONF_H

#include <string>
#include "detection/rules.h"

void parse_conf_init();
void parse_conf_term();
void parse_conf_print();

namespace snort
{
struct SnortConfig;
}

const char* get_parse_file();

// returns code or nullptr if not found, file holds abs path
// file may hold original parse path on entry
const char* get_config_file(const char* arg, std::string& file);

void parse_rules_file(snort::SnortConfig*, const char* fname);
void parse_rules_string(snort::SnortConfig*, const char* str, bool ips_policy = true);

bool ParseIpVar(const char* name, const char* value);
void parse_include(snort::SnortConfig*, const char*);

void add_service_to_otn(snort::SnortConfig*, OptTreeNode*, const char*);

ListHead* get_rule_list(snort::SnortConfig*, const char*);

#endif

parse_ip.h

Path = src/parser/parse_ip.h

#ifndef PARSE_IP_H
#define PARSE_IP_H

struct sfip_var_t;

sfip_var_t* sfip_var_from_string(const char* addr, const char* caller);

#endif

parse_ports.h

Path = src/parser/parse_ports.h

#ifndef PARSE_PORTS_H
#define PARSE_PORTS_H

#include "ports/port_var_table.h"

//-------------------------------------------------------------------------
// parser
//-------------------------------------------------------------------------

#define POPERR_NO_NAME            1
#define POPERR_NO_ENDLIST_BRACKET 2
#define POPERR_NOT_A_NUMBER       3
#define POPERR_EXTRA_BRACKET      4
#define POPERR_NO_DATA            5
#define POPERR_ADDITEM_FAILED     6
#define POPERR_MALLOC_FAILED      7
#define POPERR_INVALID_RANGE      8
#define POPERR_DUPLICATE_ENTRY    9
#define POPERR_BOUNDS             10
#define POPERR_BAD_VARIABLE       11

#define POP_MAX_BUFFER_SIZE 256

struct PortObject;

struct POParser
{
    const char* s;          /* current string pointer */
    int slen;         /* bytes left in string */
    int pos;          /* position in string of last GetChar() */
    char token[POP_MAX_BUFFER_SIZE+4];   /* single number, or range, or not flag */
    int errflag;
    /* for handling PortObject references when parsing */
    PortObject* po_ref;
    SF_LNODE* poi_pos;
    PortVarTable* pvTable;
};

PortObject* PortObjectParseString(
    PortVarTable* pvTable, POParser* pop, const char* name,  const
    char* s,int nameflag);

const char* PortObjectParseError(POParser* p);

#endif

parser.h

Path = src/parser/parser.h

#ifndef PARSER_H
#define PARSER_H

#include "detection/rules.h"
#include "detection/treenodes.h"
#include "main/policy.h"

namespace snort
{
struct SnortConfig;
}

void parser_init();
void parser_term(snort::SnortConfig*);

// line > 0 implies name is non-null
void get_parse_location(const char*& name, unsigned& line);

// use line = 0 for lua to suppress line numbers for errors or warnings
void push_parse_location(
    const char* code, const char* path, const char* name = nullptr, unsigned line = 1);

void pop_parse_location();
void inc_parse_position();

snort::SnortConfig* ParseSnortConf(const snort::SnortConfig*, const char* fname = nullptr);
void ParseRules(snort::SnortConfig*);
void ParseRulesFinish(snort::SnortConfig*);

char* ProcessFileOption(snort::SnortConfig*, const char*);
void SetRuleStates(snort::SnortConfig*);

void OrderRuleLists(snort::SnortConfig*);
void FreeRuleLists(snort::SnortConfig*);
void VarTablesFree(snort::SnortConfig*);

void parser_append_rules(const char*);
void parser_append_includes(const char*);
void parser_append_rules_special(const char* file, const char* includer);
const char* parser_get_special_includer();

int ParseBool(const char* arg);

int addRtnToOtn(snort::SnortConfig*, struct OptTreeNode*, RuleTreeNode*);
int addRtnToOtn(snort::SnortConfig*, struct OptTreeNode*, RuleTreeNode*, PolicyId);

void set_strict_rtn_reduction(bool);
bool same_headers(RuleTreeNode*, RuleTreeNode*);
RuleTreeNode* deleteRtnFromOtn(OptTreeNode*, snort::SnortConfig* sc = nullptr);
RuleTreeNode* deleteRtnFromOtn(struct OptTreeNode*, PolicyId, snort::SnortConfig* sc = nullptr, bool remove = true);

inline RuleTreeNode* getRtnFromOtn(const struct OptTreeNode* otn, PolicyId policyId)
{
    if (otn && otn->proto_nodes && (otn->proto_node_num > (unsigned)policyId))
    {
        return otn->proto_nodes[policyId];
    }
    return nullptr;
}

inline RuleTreeNode* getRtnFromOtn(const struct OptTreeNode* otn)
{
    return getRtnFromOtn(otn, snort::get_ips_policy()->policy_id);
}

RuleListNode* CreateRuleType(snort::SnortConfig* sc, const char* name, Actions::Type action_type);

void FreeRuleTreeNode(RuleTreeNode*);
void DestroyRuleTreeNode(RuleTreeNode*);

int parser_get_rule_index(unsigned gid, unsigned sid);
void parser_get_rule_ids(int index, unsigned& gid, unsigned& sid);
void rule_index_map_print_index(int index, char* buf, int);

struct RuleTreeNodeKey
{
    RuleTreeNode* rtn;
    PolicyId policyId;
};

#endif

parse_rule.h

Path = src/parser/parse_rule.h

#ifndef PARSE_RULE_H
#define PARSE_RULE_H

#include "detection/rules.h"
#include "framework/ips_option.h"

namespace snort
{
struct SnortConfig;
SO_PUBLIC int get_rule_count();
}
struct OptFpList;
struct OptTreeNode;
struct RuleTreeNode;

void parse_rule_init();
void parse_rule_term();
void parse_rule_print(unsigned fb_total, unsigned fb_unchk, unsigned fb_unset);
int get_policy_loaded_rule_count();
int get_policy_shared_rule_count();

void parse_rule_type(snort::SnortConfig*, const char*, RuleTreeNode&);
void parse_rule_proto(snort::SnortConfig*, const char*, RuleTreeNode&, bool elided = false);
void parse_rule_nets(snort::SnortConfig*, const char*, bool src, RuleTreeNode&, bool elided = false);
void parse_rule_ports(snort::SnortConfig*, const char*, bool src, RuleTreeNode&, bool elided = false);
void parse_rule_dir(snort::SnortConfig*, const char*, RuleTreeNode&, bool elided = false);
void parse_rule_opt_begin(snort::SnortConfig*, const char* key);
void parse_rule_opt_set(snort::SnortConfig*, const char* key, const char* opt, const char* val);
void parse_rule_opt_end(snort::SnortConfig*, const char* key, OptTreeNode*);

OptTreeNode* parse_rule_open(snort::SnortConfig*, RuleTreeNode&, bool stub = false);
void parse_rule_close(snort::SnortConfig*, RuleTreeNode&, OptTreeNode*);
void parse_rule_process_rtn(RuleTreeNode*);
int parse_rule_finish_ports(snort::SnortConfig*, RuleTreeNode*, OptTreeNode*);
void parse_rule_dec_head_count();

#endif

parse_so_rule.h

Path = src/parser/parse_so_rule.h

#ifndef PARSE_SO_RULE_H
#define PARSE_SO_RULE_H

// must parse out stub options for --dump-dynamic-rules
//
// only selected options are shown in so rule stubs
// all other options are stripped out of the stub
//
// assumes valid rule syntax
// deletes all # comments
// replaces each /* comment */ with a single space
// replaces each newline with a single space
// deletes more than one space between options
// return true if parsed rule body close

#include <string>

bool get_so_stub(const char* in, std::string& stub);

#endif

parse_stream.h

Path = src/parser/parse_stream.h

#ifndef PARSE_STREAM_H
#define PARSE_STREAM_H

namespace snort
{
struct SnortConfig;
}

#include <istream>

void parse_stream(std::istream&, snort::SnortConfig*);

#endif

parse_utils.h

Path = src/parser/parse_utils.h

#ifndef PARSE_UTILS_H
#define PARSE_UTILS_H

#include <string>

bool parse_byte_code(const char*, bool& negate, std::string&);
int parse_int(const char*, const char* tag, int low = -65535, int high = 65535);

#endif

var_dependency.h

Path = src/parser/var_dependency.h

#ifndef VAR_DEPENDENCY_H
#define VAR_DEPENDENCY_H

void resolve_nets();
void resolve_ports();

bool is_resolving_nets();
bool is_resolving_ports();

void push_to_weak_nets(const char* var_name, const char* val);
void push_to_weak_ports(const char* var_name, const char* val);

#endif

vars.h

Path = src/parser/vars.h

#ifndef VARS_H
#define VARS_H

#include <cstdint>
#include <string>

#include "sfip/sf_vartable.h"

namespace snort
{
struct SnortConfig;
}

//-------------------------------------------------------------------------
// var table stuff
//-------------------------------------------------------------------------

struct VarEntry
{
    char* name;
    char* value;

    unsigned char flags;
    uint32_t id;

    sfip_var_t* addrset;
    VarEntry* prev;
    VarEntry* next;
};

void ParsePathVar(const char* name, const char* value);
bool ParsePortVar(const char* name, const char* value);

VarEntry* VarAlloc();
void DeleteVars(VarEntry* var_table);

enum VarType
{
    VAR_TYPE__DEFAULT,
    VAR_TYPE__PORTVAR,
    VAR_TYPE__IPVAR
};

int VarIsIpAddr(vartable_t* ip_vartable, const char* value);
int VarIsIpList(vartable_t* ip_vartable, const char* value);
void DisallowCrossTableDuplicateVars(const char* name, VarType var_type);

const std::string VarSearch(const std::string& name);
const std::string ExpandVars(const std::string& input_str);

#endif

payload_injector/

Payload injector is a utility class intended to help other Snort components with application-protocol-level injection. The calling components decide when and what to inject. Payload injector is responsible for flow-control concerns such as transmitting RST flags to endpoints as well as blocking flows within Snort. It coordinates with the Active component of Snort to perform these functions.

Currently it is being used for HTTP/1 and HTTP/2 injections. For HTTP/2 injection triggered while server-to-client flow of traffic is in a middle of a frame is not supported. The traffic will be blocked, but the page will not be injected/displayed.

get_http2_payload supports translation of HTTP block/redirect page to HTTP2. Current implementation is limited, the constraints are specified in payload_injector_translate_page.cc.

payload_injector_config.h

Path = src/payload_injector/payload_injector_config.h

#ifndef PAYLOAD_INJECTOR_CONFIG_H
#define PAYLOAD_INJECTOR_CONFIG_H

struct PayloadInjectorConfig
{
};

#endif

payload_injector.h

Path = src/payload_injector/payload_injector.h

#ifndef PAYLOAD_INJECTOR_H
#define PAYLOAD_INJECTOR_H

#include "framework/codec.h"

namespace snort
{
struct Packet;
}

enum InjectionReturnStatus : int8_t
{
    INJECTION_SUCCESS = 1,
    ERR_INJECTOR_NOT_CONFIGURED = -1,
    ERR_STREAM_NOT_ESTABLISHED = -2,
    ERR_UNIDENTIFIED_PROTOCOL = -3,
    ERR_HTTP2_STREAM_ID_0 = -4,
    ERR_PAGE_TRANSLATION = -5,
    ERR_HTTP2_MID_FRAME = -6,
    ERR_TRANSLATED_HDRS_SIZE = -7,
    ERR_HTTP2_EVEN_STREAM_ID = -8,
    ERR_PKT_FROM_SERVER = -9,
    ERR_CONFLICTING_S2C_TRAFFIC = -10,
    // Update InjectionErrorToString when adding/removing error codes
};

struct InjectionControl
{
    const uint8_t* http_page = nullptr;
    uint32_t http_page_len = 0;
    int64_t stream_id = 0;
};

class SO_PUBLIC PayloadInjector
{
public:
    static InjectionReturnStatus inject_http_payload(snort::Packet* p, const
        InjectionControl& control);

    static const char* get_err_string(InjectionReturnStatus status);

private:
    static InjectionReturnStatus inject_http2_payload(snort::Packet* p, const
        InjectionControl& control, snort::EncodeFlags df);

#ifdef UNIT_TEST

public:
#endif
    static InjectionReturnStatus get_http2_payload(InjectionControl control,
        uint8_t*& http2_payload, uint32_t& payload_len, bool send_settings);
};

#ifdef UNIT_TEST
InjectionReturnStatus write_7_bit_prefix_int(uint32_t val, uint8_t*& out,
    uint32_t& out_free_space);
#endif

#endif

payload_injector_module.h

Path = src/payload_injector/payload_injector_module.h

#ifndef PAYLOAD_INJECTOR_MODULE_H
#define PAYLOAD_INJECTOR_MODULE_H

#include "framework/module.h"

struct PayloadInjectorCounts
{
    PegCount http_injects;
    PegCount http2_injects;
    PegCount http2_translate_err;
    PegCount http2_mid_frame;
};

extern THREAD_LOCAL PayloadInjectorCounts payload_injector_stats;

class SO_PUBLIC PayloadInjectorModule : public snort::Module
{
public:
    PayloadInjectorModule();
    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;

    Usage get_usage() const override
    { return GLOBAL; }

    bool end(const char*, int, snort::SnortConfig*) override;
};

#endif

payload_injector/test/

policy_selectors/

A set of policy selectors to select the default policies in a multi-tenant environment. The selectors can only use constraints that are known before packet decode, such as address space ID, interfaces, tenants, etc.

int_set_to_string.h

Path = src/policy_selectors/int_set_to_string.h

#ifndef INT_SET_TO_STRING_H
#define INT_SET_TO_STRING_H

// used to format configuration for output in show methods

#include <algorithm>
#include <sstream>
#include <string>
#include <vector>

template <typename T>
static std::string int_set_to_string(const std::vector<T>& v)
{
    if (v.empty())
        return "";

    std::stringstream ss;
    for (auto e : v)
        ss << e << " ";

    auto str = ss.str();
    if (!str.empty())
        str.pop_back();

    return str;
}

#endif

parse_int_set.h

Path = src/policy_selectors/parse_int_set.h

#ifndef PARSE_INT_SET_H
#define PARSE_INT_SET_H

// used to parse an int set

#include <cstdint>
#include <iomanip>
#include <limits>
#include <sstream>
#include <string>
#include <vector>

#include "framework/value.h"

template<typename T>
static bool parse_int_set(const snort::Value& v, std::vector<T>& set)
{
    assert(v.get_type() == snort::Value::VT_STR);

    set.clear();

    std::string pl = v.get_string();

    std::stringstream ss(pl);
    ss >> std::setbase(0);

    int64_t n;

    while ( ss >> n )
    {
        if ( n > std::numeric_limits<T>::max() )
            return false;

        set.emplace_back(n);
    }
    if ( !ss.eof() )
        return false;

    return true;
}

#endif

policy_selectors.h

Path = src/policy_selectors/policy_selectors.h

#ifndef POLICY_SELECTORS_H
#define POLICY_SELECTORS_H

void load_policy_selectors();

#endif

policy_selectors/address_space_selector/

The address space selector maps configuration to a packet based on the address space ID in the packet header.

AddressSpaceSelectorModule creates a vector of AddressSpaceSelections from the Lua selector table which is moved to the AddressSpaceSelector upon its construction. The PolicySelectUse portion of the AddressSpaceSelections is added to a policy map that is keyed with the tenant IDs. For each default policy selection, the policy map is queried for a match given the address space ID. The match selects the following policies:

  • network policy

  • inspection policy

  • IPS policy

address_space_selection.h

Path = src/policy_selectors/address_space_selector/address_space_selection.h

#ifndef ADDRESS_SPACE_SELECTION_H
#define ADDRESS_SPACE_SELECTION_H

// Evaluation elements for selecting policies based on address space

#include <vector>

#include "framework/policy_selector.h"

struct AddressSpaceSelection
{
    std::vector<uint32_t> addr_spaces;
    snort::PolicySelectUse use;

    AddressSpaceSelection();

    void clear();
};

#endif

address_space_selector_module.h

Path = src/policy_selectors/address_space_selector/address_space_selector_module.h

#ifndef ADDRESS_SPACE_SELECTOR_MODULE_H
#define ADDRESS_SPACE_SELECTOR_MODULE_H

// address space selector management interface

#include "framework/module.h"
#include "framework/policy_selector.h"
#include "address_space_selection.h"

#define ADDRESS_SPACE_SELECT_NAME "address_space_selector"
#define ADDRESS_SPACE_SELECT_HELP "configure traffic processing based on address space"

extern THREAD_LOCAL snort::PolicySelectStats address_space_select_stats;
extern THREAD_LOCAL snort::ProfileStats address_space_selectPerfStats;

class AddressSpaceSelectorModule : public snort::Module
{
public:
    AddressSpaceSelectorModule();
    ~AddressSpaceSelectorModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile() const override;

    std::vector<AddressSpaceSelection>& get_policy_selections();

    Usage get_usage() const override
    { return GLOBAL; }

private:
    AddressSpaceSelection selection;
    std::vector<AddressSpaceSelection> policy_selections;
    std::string policy_filename;

    void add_policy_file(const char* name);
    void commit_policy_selection();
};

#endif

policy_selectors/tenant_selector/

The tenant selector maps configuration to a packet based on the tenant ID in the packet header.

TenantSelectorModule creates a vector of TenantSelections from the Lua selector table which is moved to the TenantSelector upon its construction. The PolicySelectUse portion of the TenantSelections is added to a policy map that is keyed with the tenant IDs. For each default policy selection, the policy map is queried for a match given the tenant ID. The match selects the following policies:

  • network policy

  • inspection policy

  • IPS policy

tenant_selection.h

Path = src/policy_selectors/tenant_selector/tenant_selection.h

#ifndef TENANT_SELECTION_H
#define TENANT_SELECTION_H

// Evaluation elements for selecting policies based on tenant

#include <vector>

#include "framework/policy_selector.h"

struct TenantSelection
{
    std::vector<uint32_t> tenants;
    snort::PolicySelectUse use;

    TenantSelection();

    void clear();
};

#endif

tenant_selector_module.h

Path = src/policy_selectors/tenant_selector/tenant_selector_module.h

#ifndef TENANT_SELECTOR_MODULE_H
#define TENANT_SELECTOR_MODULE_H

// tenant selector management interface

#include <string>
#include <vector>

#include "framework/module.h"
#include "tenant_selection.h"

#define TENANT_SELECT_NAME "tenant_selector"
#define TENANT_SELECT_HELP "configure traffic processing based on tenants"

extern THREAD_LOCAL snort::PolicySelectStats tenant_select_stats;
extern THREAD_LOCAL snort::ProfileStats tenant_select_perf_stats;

class TenantSelectorModule : public snort::Module
{
public:
    TenantSelectorModule();
    ~TenantSelectorModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile() const override;

    std::vector<TenantSelection>& get_policy_selections();

    Usage get_usage() const override
    { return GLOBAL; }

private:
    TenantSelection selection;
    std::vector<TenantSelection> policy_selections;
    std::string policy_filename;

    void add_policy_file(const char* name);
    void commit_policy_selection();
};

#endif

ports/

Port groups actually refer to a groups of rules based on ports (and protocol and direction) only. The idea is that when we get a packet, say going to tcp port 80, we can select an MPSE instance for fast pattern matching. We then need to fully evaluate only rules with matching fast patterns. (Of course, any tcp destination port 80 rules w/o fast patterns must be evaluated as well, but the hope is there are very few if any of those rules.) Because there are many possible ports with related traffic, port groups are typically based on a collection of related ports, eg 80 and 8080 for HTTP.

When services came along, port groups were used for rules grouped by service as well as by ports. (Note that ports and service considerations are not intermixed.) Thus we could combine the fast patterns of all HTTP traffic to the server into one "port group" and select that instead of selecting an MPSE by port.

The following comments are from the original sfportobject.c from which ports/ is derived.

author: marc norton date: 11/05/2005

Port objects provides support for generic ports lists comprised of individual ports, port ranges, and negation of ports and port ranges.

Port lists require a somewhat more complex scheme to determine the proper grouping of rules for each port while minimizing the number of rule groups created. We can use a single group of rules in the multi-pattern detection phase, however that can have a huge impact on performance. Instead we try to create a smaller grouping of rules that might be applicable to each port.

As rules are defined using port ranges, and port lists there will be port overlaps between rules. This requires us to determine whether we should create one larger rule group to apply to all relevant ports, or to create multiple rule groups and apply the smallest applicable one to each port. In practice snort has some rules which span almost all 64K ports which might cause all rules in all port-rule groups to be merged into one set unless we apply a more complex logic than simply merging rule-port groups with common ports. This is the problem addressed by the sfportobject module.

Port list examples of acceptable usage

  • var has been overloaded, if it includes _port we add as a port-object also.

    var http_ports 80
    var http_range_ports 80:81
    var http_list_ports  [80,8080,8138]
  • portvar has been added to indicate portvariables, this form does not require _port

    portvar http 80
    portvar http_range 80:81
    portvar http_list  [80,8080,8138]
    80
    $http
    !90
    80:81
    $http_range
    !90:91
    [80,8080,8138]
    $http_list
    [$http,$http_list]
    [2001,2008,20022,8100:8150,!8121,!8123]
    [!any] - uhhh, why do people ask about this ?

Rules are defined using a port, a port-range or a list of these, we call these port objects.

As rules are loaded we generate some large rule counts on some ports and small rule counts on most ports. If for each port you build a list of rules on that port, we may end up with some ports with a large rule set that differs by the addition of a few rules on each port (relative to the group sizes) we don’t want to generate completely different rule groups for these as that would then generate multiple large state machines for the multi-pattern matching phase of the detection engine which in turn could use a lot of memory.

Turns out that one scheme, the one used herein, provides some blending of rule groups to minimize memory, and tries to minimize large group sizes to keep performance more optimal - although this is at the expense of memory.

Port Variables

  • Var has been overloaded. If it’s name includes _port as part of the var name it is added to the PortVarTable.

  • PortVar has been added. These are always added to the PortVarTable.

Loading Port Lists and Rules

  • PortTables - we support src and dst tables for tcp/udp/icmp/ip rules.

  • PortVar References - we dup the PortVar entries as needed into each table if referenced, so HTTP_PORTS for tcp and udp contain different rules.

If a rule references a PortVar we look it up in the table, if its not present we dup it from the PortVarTable, otherwise we just add the rule index to the PortVar HTTP_PORTS in the proper table. If a PortVar is not used to specify a Port entry in a rule we create a temp port-object, and check if it’s port-list is already in the table. If it’s not we make the temp port-object the permanent entry in the table. If it is, we just add the rule index to the existing entry, and delete the temp port-object. When the rules are done loading we should have a set of port-objects with port-lists that differ by at least one port. The next step handles the cases where we have multiple port-objects with at least one common port.

Merging Ports and Rules

We maintain for each port a list of port objects and their rules that apply to it. This allows us to view combining the rules associated with each port object using a few heuristics. A list of port objects applicable to each port presents rules in one of four categories:

  1. a single port object, and all rules associated with it.

  2. multiple port objects each with a small set of rules associated with it.

  3. one port object with a large rule set, and one or more port objects with a small set of rules associated with each.

  4. multiple port objects with large rule sets, and zero or more port objects each with a small set of rules associated with it.

We process these four categories as follows

  1. a single port object (large or small) do nothing, each port referencing this port object is complete.

  2. multiple small port objects merge the rules for all port objects into one virtual object, for each port in this category lookup it’s combined port object to see if it’s already defined, if not create one. This way all ports that have the same port groups point to the same virtual port object.

  3. one large port object, and one or more small port objects add the small rule groups into the large rule set, using the existing port object.

  4. multiple large port objects and zero or more small port objects merge the large port objects into a virtual port object and add all rules from both large and small sets into it’s rule set. we use the combined large group ports to form a key, so any ports referencing just these large rule groups, and some small ones will be recognized as the same. This handles cases where we have 2,3,4.etc large rule groups combined. Any time we see a n grouping of the same large rule sets we’ll look it up and point to it for that port.

    To determine when a port object has a large rule set or a small one we use
    a simple threshold value. In theory the larger this value is the more
    merging of rules in category 2 and 3 will occur. When this value is small
    category 4 should become a more prevalent situation.  However, the
    behavior of groupings for a given threshold can change as more rules are
    added to port groups.  Therefore generous statistics are printed after the
    rules and port objects are compiled into their final groupings.

Procedure for using PortLists

  1. Process Var’s as PortVar’s and standard Var’s (for now). This allows existing snort features to work, with the Var’s. Add in the PortVar support to parse the Var input into PortObjects, and add these to the PortVartable.

  2. Read Rules

    1. Read port numbers and lists

      • Dereference PortVar/Var Names if any are referenced.

    2. Create a Port Object

    3. Test if this Port object exists already,

      • If so, add the sid to it.

      • If not add it ….

Notes

All any-any port rules are managed separately, and added in to the final rules lists of each port group after this analysis. Rules defined with ranges are no longer considered any-any rules for the purpose of organizing port-rule groupings. This should help prevent some cross fertilization of rule groups with rules that are unnecessary, this causes rule group sizes to bloat and performance to slow.

Hierarchy

PortTable -> PortObject's
PortVar -> PortObject (These are pure, and are dup'ed for use in the PortTables)
PortObject -> PortObjectItems (port or port range)

port_group.h

Path = src/ports/port_group.h

#ifndef PORT_GROUP_H
#define PORT_GROUP_H

#include <cassert>
#include <vector>

#include "framework/pdu_section.h"
#include "framework/mpse_batch.h"

// RuleGroup contains a set of fast patterns in the form of an MPSE and a
// set of non-fast-pattern (nfp) rules.  when a RuleGroup is selected, the
// MPSE will run fp rules if there is a match on the associated fast
// patterns.  it will always run nfp rules since there is no way to filter
// them out.

namespace snort
{
    class IpsOption;
}

struct RULE_NODE
{
    RULE_NODE* rnNext;
    void* rnRuleData;
    int iRuleNodeID;
};

struct PatternMatcher
{
    enum Type { PMT_PKT, PMT_FILE, PMT_PDU };

    PatternMatcher(Type t, const char* s, bool r = false)
    { type = t; name = s; raw_data = r; }

    Type type;
    const char* name;
    bool raw_data;

    snort::MpseGroup group;
    snort::IpsOption* fp_opt = nullptr;
};

struct RuleGroup
{
    RuleGroup() = default;
    ~RuleGroup();

    // non-fast-pattern list
    RULE_NODE* nfp_head = nullptr;
    RULE_NODE* nfp_tail = nullptr;

    // pattern matchers
    using PmList = std::vector<PatternMatcher*>[snort::PS_MAX + 1];
    PmList pm_list;

    // detection option tree
    void* nfp_tree = nullptr;

    unsigned rule_count = 0;
    unsigned nfp_rule_count = 0;

    void add_rule();
    bool add_nfp_rule(void*);
    void delete_nfp_rules();

    PatternMatcher* get_pattern_matcher(PatternMatcher::Type, const char*, snort::PduSection sect);
};

#endif

port_item.h

Path = src/ports/port_item.h

#ifndef PORT_ITEM_H
#define PORT_ITEM_H

#include <cstdint>

#define SFPO_MAX_LPORTS 500
#define SFPO_MAX_PORTS 65536

//-------------------------------------------------------------------------
// Port Object Item supports
// port, lowport:highport (inclusive)
//
// so it indicates a single port, a consecutive range of ports, or the any
// port.  can also be negated.
//-------------------------------------------------------------------------

struct PortObjectItem
{
    bool any()
    { return !negate and lport == 0 and hport == SFPO_MAX_PORTS-1; }

    bool one()
    { return lport == hport; }

    bool negate;

    uint16_t hport;   /* hi port */
    uint16_t lport;   /* lo port */
};

PortObjectItem* PortObjectItemNew();
void PortObjectItemFree(PortObjectItem*);
PortObjectItem* PortObjectItemDup(PortObjectItem*);
int PortObjectItemsEqual(PortObjectItem* a, PortObjectItem* b);
void PortObjectItemPrint(PortObjectItem*, char* dstbuf, int bufsize);
unsigned PortObjectItemHash(PortObjectItem*, unsigned hash, unsigned scale);

#endif

port_object2.h

Path = src/ports/port_object2.h

#ifndef PORT_OBJECT2_H
#define PORT_OBJECT2_H

#include "framework/bits.h"
#include "utils/sflsq.h"

//-------------------------------------------------------------------------
// PortObject2 is similar to PortObject
//-------------------------------------------------------------------------

namespace snort
{
class GHash;
}

struct PortObject;

struct PortObject2
{
    // FIXIT-L convert char* to C++ string
    // prefix of this struct must match first 4 items in PortObject
    char* name;                 /* user name */
    int id;                     /* internal tracking - compiling sets this value */
    mutable unsigned hash = 0;

    SF_LIST* item_list;         /* list of port and port-range items */
    snort::GHash* rule_hash;         /* hash of rule (rule-indexes) in use */

    PortBitSet* port_list;      /* for collecting ports that use this object */
    struct RuleGroup* group;    /* RuleGroup based on rule_hash  */

    int port_cnt;               /* count of ports using this object */
};

PortObject2* PortObject2New(int nrules /*guess at this */);
void PortObject2Free(PortObject2*);
void PortObject2Finalize(PortObject2*);
PortObject2* PortObject2Dup(PortObject&);

typedef void (*PortObjectIterator)(int port, void*);
void PortObject2Iterate(PortObject2*, PortObjectIterator, void*);

PortObject2* PortObject2AppendPortObject(PortObject2* poa, PortObject* pob);
PortObject2* PortObject2AppendPortObject2(PortObject2* poa, PortObject2* pob);
PortObject2* PortObjectAppendEx2(PortObject2* poa, PortObject* pob);

void PortObject2PrintPorts(PortObject2* po);
void PortObject2Print(PortObject2* po);
void PortObject2PrintEx(PortObject2* po,
    void (* print_index_map)(int index, char* buf, int bufsize) );

#endif

port_object.h

Path = src/ports/port_object.h

#ifndef PORT_OBJECT_H
#define PORT_OBJECT_H

#include "utils/sflsq.h"

//-------------------------------------------------------------------------
// PortObject supports a set of PortObjectItems
// associates rules with a RuleGroup.
//-------------------------------------------------------------------------

struct RuleGroup;
struct PortObjectItem;

struct PortObject
{
    // FIXIT-L convert char* to C++ string
    char* name;                 /* user name */
    int id;                     /* internal tracking - compiling sets this value */
    mutable unsigned hash = 0;

    SF_LIST* item_list;         /* list of port and port-range items */
    SF_LIST* rule_list;         /* list of rules  */

    RuleGroup* group;           // based on rule_list - only used by any-any ports
};

PortObject* PortObjectNew();
void PortObjectFree(void*);
void PortObjectFinalize(PortObject*);

int PortObjectSetName(PortObject*, const char* name);
int PortObjectAddItem(PortObject*, PortObjectItem*, int* errflag);
int PortObjectAddPortObject(PortObject* podst, PortObject* posrc, int* errflag);
int PortObjectAddPort(PortObject*, int port);
int PortObjectAddRange(PortObject*, int lport, int hport);
int PortObjectAddRule(PortObject*, int rule);
int PortObjectAddPortAny(PortObject*);

PortObject* PortObjectDup(PortObject*);
PortObject* PortObjectDupPorts(PortObject*);

int PortObjectNormalize(PortObject*);
void PortObjectToggle(PortObject*);
bool PortObjectEqual(PortObject* poa, PortObject* pob);

int PortObjectPortCount(PortObject*);
int PortObjectHasPort(PortObject*, int port);
int PortObjectIsPureNot(PortObject*);
int PortObjectHasAny(PortObject*);

int PortObjectRemovePorts(PortObject* a,  PortObject* b);
PortObject* PortObjectAppend(PortObject* poa, PortObject* pob);

void PortObjectPrint(PortObject*);
void PortObjectPrintPortsRaw(PortObject*);

typedef void (*po_print_f)(int index, char* buf, int bufsize);
void PortObjectPrintEx(PortObject*, po_print_f);

unsigned PortObjectHash(const PortObject*, unsigned hash, unsigned scale, unsigned hardener);

#endif

port_table.h

Path = src/ports/port_table.h

#ifndef PORT_TABLE_H
#define PORT_TABLE_H

#include "ports/port_item.h"
#include "ports/port_object.h"
#include "ports/port_object2.h"
#include "utils/sflsq.h"

namespace snort
{
class GHash;
}

//-------------------------------------------------------------------------
// PortTable - provides support to analyze the Port List objects defined by
// the user as either PortVar entries or simply as inline rule port list
// declarations.
//-------------------------------------------------------------------------

struct PortTable
{
    /* turns on group optimization, better speed-but more memory
     * otherwise a single merged rule group is used.
     */
    int pt_optimize;

    /* save the users input port objects in this list
     * rules may be added after creation of a port object
     * but the ports are not modified.
     */
    SF_LIST* pt_polist;
    int pt_poid;

    /* Compiled / merged port object hash table */
    snort::GHash* pt_mpo_hash;
    snort::GHash* pt_mpxo_hash;

    /*
    * Final Port/Rule Groupings, one port object per port, or null
    */
    PortObject2* pt_port_object[SFPO_MAX_PORTS];

    int pt_lrc; /* large rule count, this many rules is a large group */

    /* Stats */
    int single_merges; /* single PortObject on a port */
    int small_merges;  /* small port objects merged into a bigger object */
    int large_single_merges; /* 1 large + some small objects */
    int large_multi_merges; /* >1 large object merged + some small objects */
    int non_opt_merges;
};

PortTable* PortTableNew();
void PortTableFree(PortTable*);
void PortTableFinalize(PortTable*);

PortObject* PortTableFindInputPortObjectPorts(PortTable*, PortObject*);

int PortTableAddObject(PortTable*, PortObject*);
int PortTableCompile(PortTable*);

typedef void (* rim_print_f)(int index, char* buf, int bufsize);

void PortTablePrintInputEx(PortTable*, rim_print_f);
int PortTablePrintCompiledEx(PortTable*, rim_print_f);

void PortTablePrintInput(PortTable*);
void PortTablePrintUserRules(PortTable*);
void PortTablePrintRuleGroups(PortTable*);

void RuleListSortUniq(SF_LIST*);
void PortTableSortUniqRules(PortTable*);

#endif

port_utils.h

Path = src/ports/port_utils.h

#ifndef PORT_UTILS_H
#define PORT_UTILS_H

#include "framework/bits.h"
#include "protocols/packet.h"
#include "utils/sflsq.h"

struct PortObject;
struct PortObjectItem;

int PortObjectBits(PortBitSet&, PortObject*);
SF_LIST* PortObjectItemListFromBits(const PortBitSet&, int n);

int* RuleListToSortedArray(SF_LIST*);

int integer_compare(const void* int1, const void* int2);

// global for printing so we don't put so many bytes on the stack
extern char po_print_buf[snort::MAX_PORTS];  // FIXIT-L delete this; replace with local stringstream

#endif

port_var_table.h

Path = src/ports/port_var_table.h

#ifndef PORT_VAR_TABLE_H
#define PORT_VAR_TABLE_H

#include "ports/port_object.h"
#include "ports/port_table.h"

namespace snort
{
class GHash;
}

//-------------------------------------------------------------------------
// PortVarTable
// port lists may be defined as 'name port-list'
// PortVars are internally stored in PortObjects
//-------------------------------------------------------------------------

typedef snort::GHash PortVarTable;

PortVarTable* PortVarTableCreate();
int PortVarTableFree(PortVarTable* pvt);
int PortVarTableAdd(PortVarTable* pvt, PortObject* po);
PortObject* PortVarTableFind(PortVarTable* pvt, const char* name, bool add_if_not_found = false);

#endif

rule_port_tables.h

Path = src/ports/rule_port_tables.h

#ifndef RULE_PORT_TABLES_H
#define RULE_PORT_TABLES_H

struct PortObject;
struct PortTable;

struct PortProto
{
    PortTable* src;
    PortTable* dst;
    PortObject* any;
    PortObject* nfp;

    PortProto();
    ~PortProto();
};

struct RulePortTables
{
    PortProto ip;
    PortProto icmp;
    PortProto tcp;
    PortProto udp;

    PortObject* svc_any;
};

RulePortTables* PortTablesNew();
void PortTablesFree(RulePortTables*);

#endif

profiler/

Performance Profiling provides facilities for evaluating the performance of individual components of snort. It manages the collection of profiling statistics, as well as the display of those statistics at shutdown.

This module provides a data structure (called ProfileStats) which stores information about memory usage, timing and entry counts. Analogous data structures exist for accumulating rule profiling and can be found in the detection/ subdirectory.

To facilitate accumulation of these statistics, this module provides a class called ProfileContext (aliased as Profile). Instantiating this class at the top of a scope allows for the automatic accumulation of statistics for a given ProfileStats struct upon scope exit.

At startup, ProfileStats structs for different modules are registered via one of the Profiler::register_* functions. Internally, this defines a tree hierarchy of statistics intended to reflect the call graph of the different modules being profiled.

Right before the packet threads are joined, ProfileStats for different thread-local statistics are consolidated in the internal tree. If configured to output statistics, this tree is traversed at shutdown and the statistics are displayed.

Rule profiling is slightly different in that instead of a tree, a flat list of evaluated rules is output at shutdown. Additionally, rule profiling uses different accumulation logic. This logic is currently shared between the detection/ and profiler/ subdirectories.

Notes: * statistics are always accumulated, regardless of whether profiler output is enabled.

  • by default, time output is sorted by total_time, and memory output is sorted by total_used.

  • if the statistics for a given module are 0 and all of its children are also 0, the statistics for that module are not output.

  • memory usage is not tracked on a per-rule basis.

active_context.h

Path = src/profiler/active_context.h

#ifndef ACTIVE_CONTEXT_H
#define ACTIVE_CONTEXT_H

template<typename T>
class ActiveContext
{
public:
    constexpr ActiveContext() : fallback(), cur(nullptr) { }

    T* set(T* ctx)
    {
        auto tmp = cur;
        cur = ctx;
        return tmp;
    }

    T* unset()
    { return set(nullptr); }

    T* get()
    { return cur; }

    const T* get() const
    { return cur; }

    T& get_default()
    { return cur ? *cur : fallback; }

    const T& get_default() const
    { return cur ? *cur : fallback; }

    T& get_fallback()
    { return fallback; }

    const T& get_fallback() const
    { return fallback; }

    bool is_set() const
    { return cur != nullptr; }

private:
    T fallback;
    T* cur;
};

#endif

json_view.h

Path = src/profiler/json_view.h

#ifndef JSON_VIEW_H
#define JSON_VIEW_H

#include <vector>

#include "main/snort_config.h"

#include "profiler_printer.h"
#include "rule_profiler.h"

void print_json_entries(ControlConn*, std::vector<rule_stats::View>&, ProfilerSorter<rule_stats::View>&, unsigned);

#endif

memory_context.h

Path = src/profiler/memory_context.h

#ifndef MEMORY_CONTEXT_H
#define MEMORY_CONTEXT_H

struct MemoryStats;

class MemoryProfiler
{
public:
    // global accumulated stats
    static const MemoryStats& get_fallthrough_stats();

    // thread local call
    static void consolidate_fallthrough_stats();
};

#endif

memory_defs.h

Path = src/profiler/memory_defs.h

#ifndef MEMORY_DEFS_H
#define MEMORY_DEFS_H

#include "main/thread.h"

struct MemoryStats
{
    uint64_t allocs;
    uint64_t deallocs;
    uint64_t allocated;
    uint64_t deallocated;

    void update_allocs(size_t n)
    { ++allocs; allocated += n; }

    void update_deallocs(size_t n)
    { ++deallocs; deallocated += n; }

    void reset();

    operator bool() const
    { return allocs || allocated; }

    bool operator==(const MemoryStats&) const;
    bool operator!=(const MemoryStats& rhs) const
    { return !(*this == rhs); }

    MemoryStats operator+(const MemoryStats&) const;
    MemoryStats& operator+=(const MemoryStats&);

    constexpr MemoryStats() :
        MemoryStats(0, 0, 0, 0) { }

    constexpr MemoryStats(uint64_t a, uint64_t d, uint64_t ab, uint64_t db) :
        allocs(a), deallocs(d), allocated(ab), deallocated(db) { }
};

inline void MemoryStats::reset()
{
    allocs = 0;
    deallocs = 0;
    allocated = 0;
    deallocated = 0;
}

inline bool MemoryStats::operator==(const MemoryStats& rhs) const
{
    return allocs == rhs.allocs &&
        deallocs == rhs.deallocs &&
        allocated == rhs.allocated &&
        deallocated == rhs.deallocated;
}

inline MemoryStats MemoryStats::operator+(const MemoryStats& rhs) const
{
    return {
        allocs + rhs.allocs,
        deallocs + rhs.deallocs,
        allocated + rhs.allocated,
        deallocated + rhs.deallocated
    };
}

inline MemoryStats& MemoryStats::operator+=(const MemoryStats& rhs)
{
    allocs += rhs.allocs;
    deallocs += rhs.deallocs;
    allocated += rhs.allocated;
    deallocated += rhs.deallocated;

    return *this;
}

struct MemoryTracker
{
    MemoryStats stats;

    void reset()
    { stats.reset(); }

    void update_allocs(size_t n)
    { stats.update_allocs(n); }

    void update_deallocs(size_t n)
    { stats.update_deallocs(n); }

    constexpr MemoryTracker() = default;
    constexpr MemoryTracker(const MemoryStats &stats) : stats(stats) { }
};

#endif

memory_profiler_active_context.h

Path = src/profiler/memory_profiler_active_context.h

#ifndef MEMORY_PROFILER_ACTIVE_CONTEXT_H
#define MEMORY_PROFILER_ACTIVE_CONTEXT_H

#include "active_context.h"
#include "memory_defs.h"

class MemoryActiveContext : public ActiveContext<MemoryTracker>
{
public:
    void update_allocs(size_t n)
    { get_default().update_allocs(n); }

    void update_deallocs(size_t n)
    { get_default().update_deallocs(n); }
};

extern THREAD_LOCAL MemoryActiveContext mp_active_context;

#endif

memory_profiler_defs.h

Path = src/profiler/memory_profiler_defs.h

#ifndef MEMORY_PROFILER_DEFS_H
#define MEMORY_PROFILER_DEFS_H

#include "main/snort_types.h"

struct MemoryTracker;

struct MemoryProfilerConfig
{
    enum Sort
    {
        SORT_NONE,
        SORT_ALLOCATIONS,
        SORT_TOTAL_USED,
        SORT_AVG_ALLOCATION
    } sort = SORT_TOTAL_USED;

    bool show = false;
    unsigned count = 0;
    int max_depth = -1;
    uint64_t dump_file_size = 0;
};

namespace snort
{
class SO_PUBLIC MemoryContext
{
public:
    MemoryContext(MemoryTracker&);
    ~MemoryContext();

    MemoryContext(const MemoryContext&) = delete;
    MemoryContext& operator=(const MemoryContext&) = delete;

private:
    MemoryTracker* saved;
};

class SO_PUBLIC MemoryExclude
{
public:
    MemoryExclude();
    ~MemoryExclude();

    MemoryExclude(const MemoryExclude&) = delete;
    MemoryExclude& operator=(const MemoryExclude&) = delete;

private:
    MemoryTracker* saved;
};
}
#endif

memory_profiler.h

Path = src/profiler/memory_profiler.h

#ifndef MEMORY_PROFILER_H
#define MEMORY_PROFILER_H

class ProfilerNodeMap;
struct MemoryProfilerConfig;

void show_memory_profiler_stats(ProfilerNodeMap&, const MemoryProfilerConfig&);

#endif

profiler_defs.h

Path = src/profiler/profiler_defs.h

#ifndef PROFILER_DEFS_H
#define PROFILER_DEFS_H

#include "main/snort_types.h"
#include "main/thread.h"
#include "memory_defs.h"
#include "memory_profiler_defs.h"
#include "rule_profiler_defs.h"
#include "time_profiler_defs.h"

namespace snort
{
#define ROOT_NODE "total"
#define FLEX_NODE "other"

enum ProfilerType
{
    PROFILER_TYPE_TIME,
    PROFILER_TYPE_MEMORY,
    PROFILER_TYPE_BOTH
};

struct ProfilerConfig
{
    TimeProfilerConfig time;
    RuleProfilerConfig rule;
    MemoryProfilerConfig memory;
};

struct SO_PUBLIC ProfileStats
{
    TimeProfilerStats time;
    MemoryTracker memory;

    void reset()
    {
        time.reset();
        memory.reset();
    }

    void reset_time()
    {
        time.reset();
    }

    bool operator==(const ProfileStats&) const;
    bool operator!=(const ProfileStats& rhs) const
    { return !(*this == rhs); }

    ProfileStats& operator+=(const ProfileStats&);
    ProfileStats& operator+=(const TimeProfilerStats&);
    ProfileStats& operator+=(const MemoryTracker&);

    constexpr ProfileStats() = default;
    constexpr ProfileStats(const TimeProfilerStats& time, const MemoryTracker& memory) :
        time(time), memory(memory) { }
};

inline bool ProfileStats::operator==(const ProfileStats& rhs) const
{ return time == rhs.time && memory.stats == rhs.memory.stats; }

inline ProfileStats& ProfileStats::operator+=(const ProfileStats& rhs)
{
    time += rhs.time;
    memory.stats += rhs.memory.stats;

    return *this;
}

inline ProfileStats& ProfileStats::operator+=(const TimeProfilerStats& rhs)
{
    time += rhs;
    return *this;
}

inline ProfileStats& ProfileStats::operator+=(const MemoryTracker& rhs)
{
    memory.stats += rhs.stats;
    return *this;
}

class SO_PUBLIC ProfileContext
{
public:
    ProfileContext(ProfileStats& stats) : time(stats.time), memory(stats.memory)
    {
        prev_time = curr_time;
        if ( prev_time )
            prev_time->pause();
        curr_time = &time;
    }

    ~ProfileContext()
    {
        if ( prev_time )
            prev_time->resume();
        curr_time = prev_time;
    }

private:
    TimeContext time;
    MemoryContext memory;
    TimeContext* prev_time;
    static THREAD_LOCAL TimeContext* curr_time;
};

using get_profile_stats_fn = ProfileStats* (*)(const char*);

class NoMemContext
{
public:
    NoMemContext(ProfileStats& stats) :
        time(stats.time) { }

private:
    TimeContext time;
};

class ProfileDisabled
{
public:
    ProfileDisabled(ProfileStats&) { }
    ProfileDisabled(TimeProfilerStats&, MemoryTracker&) { }
};

#ifdef NO_PROFILER
using Profile = ProfileDisabled;
#else
#ifndef ENABLE_MEMORY_PROFILER
using Profile = NoMemContext;
#else
using Profile = ProfileContext;
#endif
#endif

#ifndef ENABLE_RULE_PROFILER
using RuleProfile = ProfileDisabled;
#else
using RuleProfile = ProfileContext;
#endif

}
#endif

profiler.h

Path = src/profiler/profiler.h

#ifndef PROFILER_H
#define PROFILER_H

#include "main/thread.h"
#include "profiler_defs.h"

namespace snort
{
class Module;
}
class ControlConn;
class ProfilerNodeMap;
class Profiler
{
public:
    static void register_module(snort::Module*);
    static void register_module(const char*, const char*, snort::Module*);

    static void start();
    static void stop(uint64_t);

    static void consolidate_stats(snort::ProfilerType = snort::PROFILER_TYPE_BOTH);

    static void reset_stats(snort::ProfilerType = snort::PROFILER_TYPE_BOTH);
    static void prepare_stats();
    static void show_stats();
    static ProfilerNodeMap& get_profiler_nodes();
    SO_PUBLIC static void show_runtime_memory_stats();
};

extern THREAD_LOCAL snort::ProfileStats totalPerfStats;
extern THREAD_LOCAL snort::ProfileStats otherPerfStats;

#endif

profiler_module.h

Path = src/profiler/profiler_module.h

#ifndef PROFILER_MODULE_H
#define PROFILER_MODULE_H

#include "framework/module.h"

#include "profiler.h"

class ProfilerModule : public snort::Module
{
public:
    ProfilerModule();

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    snort::ProfileStats* get_profile(unsigned, const char*&, const char*&) const override;
    const snort::Command* get_commands() const override;

    Usage get_usage() const override
    { return GLOBAL; }
};

#endif

profiler_nodes.h

Path = src/profiler/profiler_nodes.h

#ifndef PROFILER_NODES_H
#define PROFILER_NODES_H

#include <memory>
#include <string>
#include <unordered_map>
#include <vector>

#include "profiler_defs.h"

namespace snort
{
class Module;
}
struct GetProfileFunctor;

class ProfilerNode
{
public:
    ProfilerNode(const std::string& key) :
        name { key } { }

    void set(snort::Module* m);
    void set(snort::get_profile_stats_fn fn);

    bool is_set() const
    { return bool(getter); }

    // thread local call
    void accumulate(snort::ProfilerType = snort::PROFILER_TYPE_BOTH);

    const snort::ProfileStats& get_stats() const
    { return stats; }

    void set_stats(const snort::ProfileStats& ps)
    { stats = ps; }

    void reset(snort::ProfilerType = snort::PROFILER_TYPE_BOTH);

    void add_child(ProfilerNode* node)
    { children.emplace_back(node); }

    const std::vector<ProfilerNode*> get_children() const
    { return children; }

    const std::string name;

    void get_local_memory_stats(FILE*);

private:
    std::vector<ProfilerNode*> children;
    std::shared_ptr<GetProfileFunctor> getter;
    snort::ProfileStats stats;
};

inline bool operator==(const ProfilerNode& lhs, const ProfilerNode& rhs)
{ return lhs.name == rhs.name; }

inline bool operator!=(const ProfilerNode& lhs, const ProfilerNode& rhs)
{ return lhs.name == rhs.name; }

class ProfilerNodeMap
{
public:
    using map_type = std::unordered_map<std::string, ProfilerNode>;

    map_type::const_iterator begin() const
    { return nodes.begin(); }

    map_type::const_iterator end() const
    { return nodes.end(); }

    void register_node(const std::string&, const char*, snort::Module*);

    void accumulate_nodes(snort::ProfilerType = snort::PROFILER_TYPE_BOTH);
    void accumulate_flex();
    void clear_flex();
    void reset_nodes(snort::ProfilerType = snort::PROFILER_TYPE_BOTH);

    void print_runtime_memory_stats();

    inline void create_new_file(std::string&, uint64_t);
    void auto_rotate(std::string&, uint64_t);
    bool rotate(std::string&, uint64_t);
    bool open(std::string&, uint64_t, bool);
    void write_header();

    const ProfilerNode& get_root();

private:
    ProfilerNode& get_node(const std::string&);

    map_type nodes;
};

#endif

profiler_printer.h

Path = src/profiler/profiler_printer.h

#ifndef PROFILER_PRINTER_H
#define PROFILER_PRINTER_H

#include <algorithm>
#include <cassert>
#include <functional>
#include <sstream>
#include <string>

#include "log/messages.h"

#include "profiler_stats_table.h"
#include "profiler_tree_builder.h"

class ControlConn;
template<typename View>
struct ProfilerSorter
{
    using Entry = typename ProfilerBuilder<View>::Entry;
    using SortFn = bool(*)(const View& rhs, const View& lhs);

    std::string name;
    SortFn sort;

    operator bool() const
    { return sort != nullptr; }

    bool operator()(const Entry& lhs, const Entry& rhs) const
    { return (*this)(lhs.view, rhs.view); }

    bool operator()(const View& lhs, const View& rhs) const
    {
        assert(sort);
        return sort(lhs, rhs);
    }
};

template<typename View>
class ProfilerPrinter
{
public:
    using Entry = typename ProfilerBuilder<View>::Entry;
    using Sorter = ProfilerSorter<View>;
    using PrintFn = std::function<void(StatsTable&, const View&)>;
    using PrintTableFn = std::function<void(const char*)>;

    ProfilerPrinter(const StatsTable::Field* fields, const PrintFn print, const Sorter& sort,
        const PrintTableFn print_t) : fields(fields), print(print), sort(sort), print_t(print_t){ }

    void print_table(const std::string& title, Entry& root, unsigned count, int max_depth = -1)
    {
        std::ostringstream ss;

        {
            StatsTable table(fields, ss);

            table << StatsTable::SEP;
            table << title;

            if ( count )
                table << " (worst " << count;
            else
                table << " (all";

            if ( max_depth >= 0 )
                table << ", depth " << max_depth;

            if ( sort )
                table << ", sorted by " << sort.name;

            table << ")\n";

            table << StatsTable::HEADER;
        }

        print_t(ss.str().c_str());

        print_recursive(root, root, 1, count, max_depth);
        print_row(root, root, 0, 0);
    }

    void print_recursive(const Entry& root, Entry& cur, int layer, unsigned count,
        int max_depth)
    {
        if ( max_depth >= 0 && max_depth < layer )
            return;

        print_children(root, cur, layer, count, max_depth);
    }

    void print_children(const Entry& root, Entry& cur, int layer, unsigned count,
        int max_depth)
    {
        auto& entries = cur.children;
        unsigned num_entries;

        if ( !count || count > entries.size() )
            num_entries = entries.size();
        else
            num_entries = count;

        if ( sort )
            std::partial_sort(entries.begin(), entries.begin() + num_entries, entries.end(), sort);

        for ( unsigned i = 0; i < num_entries; ++i )
        {
            auto& entry = entries[i];

            print_row(root, entry, layer, i + 1);
            print_recursive(root, entry, layer + 1, count, max_depth);
        }
    }

    void print_row(const Entry& root, const Entry& cur, int layer, unsigned num)
    {
        std::ostringstream ss;

        {
            StatsTable table(fields, ss);

            table << StatsTable::ROW;

            // if we're printing the root node
            if ( root == cur )
                table << "--" << root.view.name << "--";

            else
            {
                auto indent = std::string(layer, ' ') + std::to_string(num);
                table << indent << cur.view.name << layer;
            }

            // delegate to user function
            print(table, cur.view);

            // don't need to print %/caller if root
            if ( root == cur )
                table << "--" << total;

            else
            {
                float value = cur.view.pct_of(root.view.get_stats());
                table << cur.view.pct_caller() << value;

                if ( layer == 1 )
                    total += value;
            }
        }

        print_t(ss.str().c_str());
    }

private:
    const StatsTable::Field* fields;
    const PrintFn print;
    const Sorter& sort;
    float total = 0;
    const PrintTableFn print_t;
};

#endif

profiler_stats_table.h

Path = src/profiler/profiler_stats_table.h

#ifndef PROFILER_STATS_TABLE_H
#define PROFILER_STATS_TABLE_H

#include <iostream>
#include <vector>

class StatsTable
{
public:
    struct Field
    {
        const char* name;
        int width;
        char fill;
        int precision;
        std::ios_base::fmtflags flags;
    };

    StatsTable(const Field*, std::ostream&);
    ~StatsTable();

    struct Header
    { char c; };

    struct Sep
    { char c; };

    struct Row {};

    static const Header HEADER;
    static const Sep SEP;
    static const Row ROW;

    void header(char);
    void sep(char);
    void row();

    template<typename T>
    StatsTable& operator<<(T v)
    {
        if ( cur )
        {
            if ( !cur->name )
                finish();

            else
                format(*cur++);
        }

        os << v;

        return *this;
    }

    StatsTable& operator<<(Header);
    StatsTable& operator<<(Sep);
    StatsTable& operator<<(Row);

    void finish();

    std::string next()
    {
        if ( !cur || !cur->name )
            return "";

        return cur->name;
    }

private:
    void format(const Field&);

    const Field* fields;
    std::ostream& os;

    const Field* cur;
};

#endif

profiler_tree_builder.h

Path = src/profiler/profiler_tree_builder.h

#ifndef PROFILER_TREE_BUILDER_H
#define PROFILER_TREE_BUILDER_H

#include <algorithm>
#include <functional>
#include <string>
#include <vector>

#include "profiler_nodes.h"

template<typename View>
class ProfilerBuilder
{
public:

    struct Entry
    {
        View view;
        std::vector<Entry> children;

        bool operator==(const Entry& rhs) const
        { return view == rhs.view; }

        bool operator!=(const Entry& rhs) const
        { return !(*this == rhs); }

        Entry(const ProfilerNode& node, View* parent = nullptr) :
            view(node, parent) { }
    };

    using IncludeFn = std::function<bool(const ProfilerNode&)>;

    ProfilerBuilder(const IncludeFn& include) :
        include(include) { }

    Entry build(const ProfilerNode& node)
    {
        Entry root(node);
        build(root, node);
        return root;
    }

private:
    void build(Entry& entry, const ProfilerNode& cur)
    {
        for ( const auto* node : cur.get_children() )
            if ( include(*node) )
            {
                entry.children.emplace_back(*node, &entry.view);
                build(entry.children.back(), *node);
            }
    }

    const IncludeFn include;
};

#endif

rule_profiler_defs.h

Path = src/profiler/rule_profiler_defs.h

#ifndef RULE_PROFILER_DEFS_H
#define RULE_PROFILER_DEFS_H

#include "time/clock_defs.h"
#include "time/stopwatch.h"

struct dot_node_state_t;

enum OutType
{
    OUTPUT_TABLE = 0,
    OUTPUT_JSON
};

struct RuleProfilerConfig
{
    enum Sort
    {
        SORT_NONE = 0,
        SORT_CHECKS,
        SORT_AVG_CHECK,
        SORT_TOTAL_TIME,
        SORT_MATCHES,
        SORT_NO_MATCHES,
        SORT_AVG_MATCH,
        SORT_AVG_NO_MATCH
    } sort = SORT_TOTAL_TIME;

    bool show = false;
    unsigned count = 0;
};

class RuleContext
{
public:
    RuleContext(dot_node_state_t& stats) :
        stats(stats)
    { start(); }

    ~RuleContext()
    { stop(); }

    void start()
    { if ( enabled ) sw.start(); }

    void pause()
    { if ( enabled ) sw.stop(); }

    void stop(bool = false);

    bool active() const
    { return enabled and sw.active(); }

    static void set_enabled(bool b)
    { enabled = b; }

    static const struct timeval *get_start_time()
    { return &start_time; }

    static void set_start_time(const struct timeval &time);

    static const struct timeval *get_end_time()
    { return &end_time; }

    static void set_end_time(const struct timeval &time);

    static const struct timeval *get_total_time()
    { return &total_time; }

    static void count_total_time();
    static bool is_enabled()
    { return enabled; }

private:
    dot_node_state_t& stats;
    Stopwatch<SnortClock> sw;
    bool finished = false;

    static THREAD_LOCAL bool enabled;
    static struct timeval start_time;
    static struct timeval end_time;
    static struct timeval total_time;

    static void valid_start_time();
    static void valid_end_time();
};

class RulePause
{
public:
    RulePause(RuleContext& ctx) :
        ctx(ctx)
    { ctx.pause(); }

    ~RulePause()
    { ctx.start(); }

private:
    RuleContext& ctx;
};

#endif

rule_profiler.h

Path = src/profiler/rule_profiler.h

#ifndef RULE_PROFILER_H
#define RULE_PROFILER_H

#include <unordered_map>
#include <vector>

#include "detection/treenodes.h"
#include "main/snort_config.h"
#include "main/thread_config.h"

#include "rule_profiler_defs.h"

struct RuleProfilerConfig;
class ControlConn;
namespace snort
{
    class HashNode;
}

void prepare_rule_profiler_stats(std::vector<snort::HashNode*>&, std::unordered_map<SigInfo*, OtnState>&, unsigned);
void print_rule_profiler_stats(const RuleProfilerConfig&, const std::unordered_map<SigInfo*, OtnState>&,
    ControlConn* = nullptr, OutType = OutType::OUTPUT_TABLE);
void show_rule_profiler_stats(const RuleProfilerConfig&);
void reset_rule_profiler_stats();
void reset_thread_rule_profiler_stats(std::vector<snort::HashNode*>&, unsigned);

namespace rule_stats
{

struct View
{
    OtnState state;
    SigInfo sig_info;

    hr_duration elapsed() const
    { return state.elapsed; }

    hr_duration elapsed_match() const
    { return state.elapsed_match; }

    hr_duration elapsed_no_match() const
    { return elapsed() - elapsed_match(); }

    uint64_t checks() const
    { return state.checks; }

    uint64_t matches() const
    { return state.matches; }

    uint64_t no_matches() const
    { return checks() - matches(); }

    uint64_t alerts() const
    { return state.alerts; }

    uint64_t timeouts() const
    { return state.latency_timeouts; }

    uint64_t suspends() const
    { return state.latency_suspends; }

    hr_duration time_per(hr_duration d, uint64_t v) const
    {
        if ( v  == 0 )
            return CLOCK_ZERO;

        return hr_duration(d / v);
    }

    hr_duration avg_match() const
    { return time_per(elapsed_match(), matches()); }

    hr_duration avg_no_match() const
    { return time_per(elapsed_no_match(), no_matches()); }

    hr_duration avg_check() const
    { return time_per(elapsed(), checks()); }

    double rule_time_per(double total_time_usec) const
    {
        if (total_time_usec < 1.)
            return 100.0;
        return clock_usecs(TO_USECS(elapsed())) / total_time_usec * 100;
    }

    View(const OtnState& otn_state, const SigInfo* si = nullptr) :
        state(otn_state)
    {
        if ( si )
            // FIXIT-L does sig_info need to be initialized otherwise?
            sig_info = *si;
    }
};

}
#endif

table_view.h

Path = src/profiler/table_view.h

#ifndef TABLE_VIEW_H
#define TABLE_VIEW_H

#include <vector>

#include "main/snort_config.h"

#include "profiler_printer.h"
#include "rule_profiler.h"

void print_entries(ControlConn*, std::vector<rule_stats::View>&, ProfilerSorter<rule_stats::View>&, unsigned);

#endif

time_profiler_defs.h

Path = src/profiler/time_profiler_defs.h

#ifndef TIME_PROFILER_DEFS_H
#define TIME_PROFILER_DEFS_H

#include "main/snort_types.h"
#include "time/clock_defs.h"
#include "time/stopwatch.h"

struct TimeProfilerConfig
{
    enum Sort
    {
        SORT_NONE = 0,
        SORT_CHECKS,
        SORT_AVG_CHECK,
        SORT_TOTAL_TIME
    } sort = SORT_TOTAL_TIME;

    bool show = false;
    unsigned count = 0;
    int max_depth = -1;
};

namespace snort
{
struct SO_PUBLIC TimeProfilerStats
{
    hr_duration elapsed;
    uint64_t checks;
    mutable unsigned int ref_count;
    static THREAD_LOCAL bool enabled;

    static void set_enabled(bool b)
    { enabled = b; }

    static bool is_enabled()
    { return enabled; }

    void update(hr_duration delta)
    { elapsed += delta; ++checks; }

    void reset()
    { elapsed = 0_ticks; checks = 0; }

    bool is_active() const
    { return ( elapsed > CLOCK_ZERO ) || checks; }

    // reentrancy control
    bool enter() const { return ref_count++ == 0; }
    bool exit() const { return --ref_count == 0; }

    constexpr TimeProfilerStats() :
        TimeProfilerStats(0_ticks, 0, 0) { }

    constexpr TimeProfilerStats(hr_duration elapsed, uint64_t checks) :
        TimeProfilerStats(elapsed, checks, 0) { }

    constexpr TimeProfilerStats(hr_duration elapsed, uint64_t checks, unsigned int ref_count) :
        elapsed(elapsed), checks(checks), ref_count(ref_count) { }
};

inline bool operator==(const TimeProfilerStats& lhs, const TimeProfilerStats& rhs)
{ return (lhs.elapsed == rhs.elapsed) && (lhs.checks == rhs.checks); }

inline bool operator!=(const TimeProfilerStats& lhs, const TimeProfilerStats& rhs)
{ return !(lhs == rhs); }

inline TimeProfilerStats& operator+=(TimeProfilerStats& lhs, const TimeProfilerStats& rhs)
{
    lhs.elapsed += rhs.elapsed;
    lhs.checks += rhs.checks;
    return lhs;
}

class TimeContext
{
public:
    TimeContext(TimeProfilerStats& stats) :
        stats(stats)
    {
        if ( stats.is_enabled() and stats.enter() )
            sw.start();
    }

    ~TimeContext()
    {
        if ( stats.is_enabled() )
            stop();
    }

    // Use this for finer grained control of the TimeContext "lifetime"
    void stop()
    {
        if ( !stats.is_enabled() or stopped_once )
            return; // stop() should only be executed once per context

        stopped_once = true;

        // don't bother updating time if context is reentrant
        if ( stats.exit() )
            stats.update(sw.get());
    }

    void pause()
    { sw.stop(); }

    void resume()
    { sw.start(); }

    bool active() const
    { return !stopped_once; }

private:
    TimeProfilerStats& stats;
    Stopwatch<SnortClock> sw;
    bool stopped_once = false;
};

class TimeExclude
{
public:
    TimeExclude(TimeProfilerStats& stats) :
        stats(stats), ctx(tmp) { }

    TimeExclude(const TimeExclude&) = delete;
    TimeExclude& operator=(const TimeExclude&) = delete;

    ~TimeExclude()
    {
        if ( !TimeProfilerStats::is_enabled() )
            return;
        ctx.stop();
        stats.elapsed -= tmp.elapsed;
    }

private:
    TimeProfilerStats& stats;
    TimeProfilerStats tmp;
    TimeContext ctx;
};

} // namespace snort
#endif

time_profiler.h

Path = src/profiler/time_profiler.h

#ifndef TIME_PROFILER_H
#define TIME_PROFILER_H

class ProfilerNodeMap;
struct TimeProfilerConfig;
class ControlConn;

void show_time_profiler_stats(ProfilerNodeMap&, const TimeProfilerConfig&);
void print_time_profiler_stats(ProfilerNodeMap&, const TimeProfilerConfig&, ControlConn*);

#endif

protocols/

Headers for the various protocols processed by Snort. Use these headers instead of directly accessing system headers. Advantages:

  • the vagaries of required system includes are minimized and localized here

  • these headers are OO in nature and much more user friendly than the often awkward system defined structs

Packet, used throughout Snort, is defined here as well. Packet represents a unit of work, essentially a buffer associated with a flow. It does not necessarily represent a wire packet.

  • PktType indicates the general nature of the packet. This may be the transport protocol for wire packets, FILE for file data, or PDU for a reassembled buffer.

  • proto_bits indicates the protocols present in the packet.

Protocol Types:

  • IpProtocol represents the standard protocol numbers as defined by IANA.

  • ProtocolId is a superset of IpProtocol that also includes ethertype and undefined protocol values.

  • ProtocolIndex is an ordinal value that acts as an index into s_protocols and s_stats.

arp.h

Path = src/protocols/arp.h

#ifndef PROTOCOLS_ARP_H
#define PROTOCOLS_ARP_H

namespace snort
{
namespace arp
{

struct ARPHdr
{
    uint16_t ar_hrd;       /* format of hardware address   */
    uint16_t ar_pro;       /* format of protocol address   */
    uint8_t ar_hln;        /* length of hardware address   */
    uint8_t ar_pln;        /* length of protocol address   */
    uint16_t ar_op;        /* ARP opcode (command)         */
};

struct EtherARP
{
    ARPHdr ea_hdr;      /* fixed-size header */
    uint8_t arp_sha[6];    /* sender hardware address */
    union
    {
        uint8_t arp_spa[4];    /* sender protocol address */
        uint32_t arp_spa32;
    };
    uint8_t arp_tha[6];    /* target hardware address */
    uint8_t arp_tpa[4];    /* target protocol address */
} __attribute__((__packed__));

constexpr uint16_t ETHERARP_HDR_LEN = 28; /*  sizeof EtherARP != 28 */

} // namespace arp
} // namespace snort

#ifndef ARPOP_REQUEST
constexpr uint16_t ARPOP_REQUEST = 1;  /* ARP request  */
#endif

#ifndef ARPOP_REPLY
constexpr uint16_t ARPOP_REPLY = 2;    /* ARP reply    */
#endif

#ifndef ARPOP_RREQUEST
constexpr uint16_t ARPOP_RREQUEST = 3; /* RARP request */
#endif

#ifndef ARPOP_RREPLY
constexpr uint16_t ARPOP_RREPLY = 4;   /* RARP reply   */
#endif

#endif

bpdu.h

Path = src/protocols/bpdu.h

#ifndef PROTOCOLS_BPDU_H
#define PROTOCOLS_BPDU_H

namespace snort
{
namespace bpdu
{

#define BPDU_TYPE_TOPCHANGE  0x80    // Topology change type BPDU

static const uint8_t BPDU_DEST[6] = {0x01, 0x80, 0xC2, 0x00, 0x00, 0x00};

bool isBPDU(const uint8_t mac[6]);

struct BPDUData
{
    uint16_t id;
    uint8_t version;
    uint8_t type;
} __attribute__((__packed__));


bool isBPDU(const uint8_t mac[6])
{
    return (memcmp(mac, BPDU_DEST, sizeof(BPDU_DEST)) == 0);
}

} // namespace bpdu
} // namespace snort

#endif

cdp.h

Path = src/protocols/cdp.h

#ifndef PROTOCOLS_CDP_H
#define PROTOCOLS_CDP_H

#include <cstdint>
#include <cstring>

namespace snort
{
namespace cdp
{
#define CDP_HDLC_PROTOCOL_TYPE 0x2000

// CDP data type values
#define RNA_CDP_ADDRESS_TYPE 0x0002
#define RNA_CDP_CAPABILITIES_TYPE 0x0004

#define RNA_CDP_CAPABILITIES_ROUTER 0x0001
#define RNA_CDP_CAPABILITIES_SWITCH 0x000A
#define RNA_CDP_CAPABILITIES_MASK (RNA_CDP_CAPABILITIES_ROUTER | RNA_CDP_CAPABILITIES_SWITCH)

static const uint8_t CDP_DEST[6] = {0x01, 0x00, 0x0C, 0xCC, 0xCC, 0xCC};

bool is_cdp(const uint8_t mac[6]);

struct RNA_CDP
{
    uint8_t org_code[3];
    uint16_t pid;
    uint8_t version;
    uint8_t ttl;
    uint16_t checksum;
}  __attribute__((__packed__));

struct RNA_CDP_DATA
{
    uint16_t type;
    uint16_t length;
}  __attribute__((__packed__));

bool is_cdp(const uint8_t mac[6])
{ return (memcmp(mac, CDP_DEST, sizeof(CDP_DEST)) == 0); }

} // namespace cdp
} // namespace snort

#endif

cisco_meta_data.h

Path = src/protocols/cisco_meta_data.h

#ifndef PROTOCOLS_CISCO_META_DATA_H
#define PROTOCOLS_CISCO_META_DATA_H

#include <arpa/inet.h>

namespace snort
{
namespace cisco_meta_data
{
struct CiscoMetaDataHdr
{
    uint8_t version;        // MetaData header version
    uint8_t length;         // Header size in quadwords
    uint16_t opt_len_type;  // 3-bit length + 13-bit type
    uint16_t sgt;           // Security Group Tag (anything but 0xFFFF)
    uint16_t ether_type;    // Ethertype for following layer

    inline uint16_t sgt_val() const
    { return ntohs(sgt); }
};
}
}

#endif

eapol.h

Path = src/protocols/eapol.h

#ifndef PROTOCOLS_EAPOL_H
#define PROTOCOLS_EAPOL_H

namespace snort
{
namespace eapol
{
struct EtherEapol
{
    uint8_t version;   /* EAPOL proto version */
    uint8_t eaptype;   /* EAPOL Packet type */
    uint16_t len;  /* Packet body length */
};

struct EAPHdr
{
    uint8_t code;
    uint8_t id;
    uint16_t len;
};

struct EapolKey
{
    uint8_t type;
    uint8_t length[2];
    uint8_t counter[8];
    uint8_t iv[16];
    uint8_t index;
    uint8_t sig[16];
};

/* IEEE 802.1x eapol types */
#define EAPOL_TYPE_EAP      0x00      /* EAP packet */
#define EAPOL_TYPE_START    0x01      /* EAPOL start */
#define EAPOL_TYPE_LOGOFF   0x02      /* EAPOL Logoff */
#define EAPOL_TYPE_KEY      0x03      /* EAPOL Key */
#define EAPOL_TYPE_ASF      0x04      /* EAPOL Encapsulated ASF-Alert */

/* Extensible Authentication Protocol Codes RFC 2284*/
#define EAP_CODE_REQUEST    0x01
#define EAP_CODE_RESPONSE   0x02
#define EAP_CODE_SUCCESS    0x03
#define EAP_CODE_FAILURE    0x04
/* EAP Types */
#define EAP_TYPE_IDENTITY   0x01
#define EAP_TYPE_NOTIFY     0x02
#define EAP_TYPE_NAK        0x03
#define EAP_TYPE_MD5        0x04
#define EAP_TYPE_OTP        0x05
#define EAP_TYPE_GTC        0x06
#define EAP_TYPE_TLS        0x0d

/* Extensible Authentication Protocol Codes RFC 2284*/
#define EAP_CODE_REQUEST    0x01
#define EAP_CODE_RESPONSE   0x02
#define EAP_CODE_SUCCESS    0x03
#define EAP_CODE_FAILURE    0x04
/* EAP Types */
#define EAP_TYPE_IDENTITY   0x01
#define EAP_TYPE_NOTIFY     0x02
#define EAP_TYPE_NAK        0x03
#define EAP_TYPE_MD5        0x04
#define EAP_TYPE_OTP        0x05
#define EAP_TYPE_GTC        0x06
#define EAP_TYPE_TLS        0x0d
} // namespace eapol
} // namespace snort

#endif /* EAPOL_H */

eth.h

Path = src/protocols/eth.h

#ifndef PROTOCOLS_ETH_H
#define PROTOCOLS_ETH_H

#include <string>

#include <arpa/inet.h>
#include "protocols/protocol_ids.h"

#define ETHERNET_HEADER_LEN 14
#define ETHERNET_MTU        1500

namespace snort
{
namespace eth
{
constexpr uint16_t MTU_LEN = 1500;
constexpr uint16_t MAX_FRAME_LENGTH = 1500;
constexpr uint16_t ETH_HEADER_LEN = 14;

struct EtherHdr
{
    uint8_t ether_dst[6];
    uint8_t ether_src[6];
    uint16_t ether_type;

    /* return data in host byte order */
    inline ProtocolId ethertype() const
    { return (ProtocolId)ntohs(ether_type); }

    /* return data in network order */
    inline uint16_t raw_ethertype() const
    { return ether_type; }

    // <Src MAC> -> <Dst Mac> <Ether Type>
    std::string to_string() const
    {
        char str[50];

        snprintf(str, sizeof(str),
            "%02X:%02X:%02X:%02X:%02X:%02X -> %02X:%02X:%02X:%02X:%02X:%02X %04X",
            ether_src[0], ether_src[1], ether_src[2],
            ether_src[3], ether_src[4], ether_src[5],
            ether_dst[0], ether_dst[1], ether_dst[2],
            ether_dst[3], ether_dst[4], ether_dst[5],
            (uint16_t)ethertype());

        return str;
    }
};
} // namespace eth
} // namespace snort

#endif

geneve.h

Path = src/protocols/geneve.h

#ifndef PROTOCOLS_GENEVE_H
#define PROTOCOLS_GENEVE_H

namespace snort
{
namespace geneve
{

// The max size of the data portion of the option (((2 ^ 5) - 1) * 4).
#define MAX_OPT_DATA_LEN 124

// Geneve Variable-Length Option (from RFC8926):
//    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
//   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
//   |          Option Class         |      Type     |R|R|R| Length  |
//   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
//   |                                                               |
//   ~                  Variable-Length Option Data                  ~
//   |                                                               |
//   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
//
struct GeneveOpt
{
    uint16_t g_class;
    uint8_t  g_type;
    uint8_t  g_len;

    uint16_t optclass() const
    { return (ntohs(g_class)); }

    bool is_set(uint16_t which) const
    { return (g_type & which); }

    uint8_t type() const
    { return (g_type); }

    // Size of GeneveOpt header plus the variable-length data for this option.
    uint8_t olen() const
    { return (sizeof(GeneveOpt) + ((g_len & 0x1f) * 4)); }

    // Size of the variable-length data section for this option.
    uint8_t data_len() const
    { return ((g_len & 0x1f) * 4); }
};

struct GeneveOptData
{
    GeneveOptData(const GeneveOpt* g_opt, const uint8_t* opt_data, uint8_t len)
    {
        assert(len <= MAX_OPT_DATA_LEN);
        opt = *g_opt;
        memcpy(data, opt_data, len);
    }

    GeneveOpt opt;
    uint8_t data[MAX_OPT_DATA_LEN];
};

// Geneve Header (from RFC8926):
//    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
//   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
//   |Ver|  Opt Len  |O|C|    Rsvd.  |          Protocol Type        |
//   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
//   |        Virtual Network Identifier (VNI)       |    Reserved   |
//   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
//   |                                                               |
//   ~                    Variable-Length Options                    ~
//   |                                                               |
//   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
//
struct GeneveHdr
{
    uint8_t g_vl;
    uint8_t g_flags;
    uint16_t g_proto;
    uint8_t g_vni[ 3 ];
    uint8_t g_rsvd;

    // Size of header fields plus the variable-length option section.
    uint16_t hlen() const
    { return (sizeof(GeneveHdr) + ((g_vl & 0x3f) * 4)); }

    uint8_t version() const
    { return (g_vl >> 6); }

    // Size of the variable-length option section.
    uint8_t opts_len() const
    { return ((g_vl & 0x3f) * 4); }

    bool is_set(uint16_t which) const
    { return (g_flags & which); }

    uint16_t proto() const
    { return (ntohs(g_proto)); }

    uint32_t vni() const
    { return ((g_vni[0] << 16) | (g_vni[1] << 8) | g_vni[2]); }
};

struct GeneveLyr
{
    GeneveHdr hdr;      // Must be first in structure.
    uint8_t data[256];  // Max size of variable options.

    std::vector<GeneveOptData> get_opt_data() const
    {
        std::vector<GeneveOptData> options;
        const uint16_t all_opt_len = hdr.opts_len();
        uint16_t offset = 0;
        const uint8_t* dptr = data;

        while (offset < all_opt_len)
        {
            const GeneveOpt* const opt = reinterpret_cast<const GeneveOpt*>(dptr);
            const uint8_t olen = opt->olen();

            if ((offset + olen) > all_opt_len)
                break;  // Invalid opt length.

            options.emplace_back(opt, dptr + sizeof(GeneveOpt), opt->data_len());

            dptr += olen;
            offset += olen;
        }

        return options;
    }
} __attribute__((packed));

} // namespace geneve
} // namespace snort

#endif

gre.h

Path = src/protocols/gre.h

#ifndef PROTOCOLS_GRE_H
#define PROTOCOLS_GRE_H

#include <arpa/inet.h>

namespace snort
{
namespace gre
{
/* GRE related stuff */
struct GREHdr
{
    uint8_t flags;
    uint8_t version;
    uint16_t ether_type;

    inline uint8_t get_version() const
    { return version & 0x07; }

    inline ProtocolId proto() const
    { return (ProtocolId)ntohs(ether_type); }

    inline uint16_t raw_proto() const
    { return ether_type; }
};
} // namespace gre
} // namespace snort

#endif

icmp4.h

Path = src/protocols/icmp4.h

#ifndef PROTOCOLS_ICMP4_H
#define PROTOCOLS_ICMP4_H

#include <cstdint>
#include "protocols/ipv4.h" // for in_addr

namespace snort
{
namespace icmp
{
constexpr uint32_t ICMP_BASE_LEN = 4;
constexpr uint8_t ICMP_UNREACH_DATA_LEN = 8;

// do NOT add 'ICMP_' to the beginning of these const because they
// will overlap with dnet macros

//enum class IcmpType : std::uint8_t {
enum IcmpType : std::uint8_t
{
    ECHOREPLY = 0,
    DEST_UNREACH = 3,
    SOURCE_QUENCH = 4,
    REDIRECT = 5,
    ECHO_4 = 8,
    ROUTER_ADVERTISE = 9,
    ROUTER_SOLICIT = 10,
    TIME_EXCEEDED = 11,
    PARAMETERPROB = 12,
    TIMESTAMP = 13,
    TIMESTAMPREPLY = 14,
    INFO_REQUEST = 15,
    INFO_REPLY = 16,
    ADDRESS = 17,
    ADDRESSREPLY = 18,
//      NR_ICMP_TYPES = 18,
};

//enum class IcmpCode : std::uint8_t {
enum IcmpCode : std::uint8_t
{
    /* Codes for ICMP UNREACHABLES (3) */
    NET_UNREACH = 0,
    HOST_UNREACH = 1,
    PROT_UNREACH = 2,
    PORT_UNREACH = 3,
    FRAG_NEEDED = 4,
    SR_FAILED = 5,
    NET_UNKNOWN = 6,
    HOST_UNKNOWN = 7,
    HOST_ISOLATED = 8,
    PKT_FILTERED_NET = 9,
    PKT_FILTERED_HOST = 10,
    NET_UNR_TOS = 11,
    HOST_UNR_TOS = 12,
    PKT_FILTERED = 13,
    PREC_VIOLATION = 14,
    PREC_CUTOFF = 15,

    /* Code for ICMP Source Quench (4) */
    SOURCE_QUENCH_CODE = 0,

    /* Codes for an ICMP Redirect (5) */
    REDIR_NET = 0,
    REDIR_HOST = 1,
    REDIR_TOS_NET = 2,
    REDIR_TOS_HOST = 3,

    /* Codes for ICMP Echo (8) */
    ECHO_CODE = 0,

    /* Codes for ICMP time exceeded (11) */
    TIMEOUT_TRANSIT = 0,
    TIMEOUT_REASSY = 1,

    /* code for ICMP Parameter Problem (12) */
    PARAM_BADIPHDR = 0,
    PARAM_OPTMISSING = 1,
    PARAM_BAD_LENGTH = 2,
};

struct Icmp4Base
{
    IcmpType type;
    IcmpCode code;
    uint16_t csum;

    union
    {
        uint32_t opt32;
        uint16_t opt16[2];
        uint8_t opt8[4];
    };
};

struct ICMPHdr
{
    IcmpType type;
    IcmpCode code;
    uint16_t csum;

    union
    {
        struct
        {
            uint8_t pptr;
            uint8_t pres1;
            uint16_t pres2;
        } param;

        struct in_addr gwaddr;

        struct idseq
        {
            uint16_t id;
            uint16_t seq;
        } idseq;

        uint32_t sih_void;

        struct pmtu
        {
            uint16_t ipm_void;
            uint16_t nextmtu;
        } pmtu;

        struct rtradv
        {
            uint8_t num_addrs;
            uint8_t wpa;
            uint16_t lifetime;
        } rtradv;
    } icmp_hun;

#define s_icmp_pptr       icmp_hun.param.pptr
#define s_icmp_gwaddr     icmp_hun.gwaddr
#define s_icmp_id         icmp_hun.idseq.id
#define s_icmp_seq        icmp_hun.idseq.seq
#define s_icmp_void       icmp_hun.sih_void
#define s_icmp_pmvoid     icmp_hun.pmtu.ipm_void
#define s_icmp_nextmtu    icmp_hun.pmtu.nextmtu
#define s_icmp_num_addrs  icmp_hun.rtradv.num_addrs
#define s_icmp_wpa        icmp_hun.rtradv.wpa
#define s_icmp_lifetime   icmp_hun.rtradv.lifetime

    union
    {
        /* timestamp */
        struct ts
        {
            uint32_t otime;
            uint32_t rtime;
            uint32_t ttime;
        } ts;

        /* IP header for unreach */
        struct ih_ip
        {
            snort::ip::IP4Hdr* ip;
            /* options and then 64 bits of data */
        } ip;

        struct ra_addr
        {
            uint32_t addr;
            uint32_t preference;
        } radv;

        uint32_t mask;

        char data[1];
    } icmp_dun;
#define s_icmp_otime      icmp_dun.ts.otime
#define s_icmp_rtime      icmp_dun.ts.rtime
#define s_icmp_ttime      icmp_dun.ts.ttime
#define s_icmp_ip         icmp_dun.ih_ip
#define s_icmp_radv       icmp_dun.radv
#define s_icmp_mask       icmp_dun.mask
#define s_icmp_data       icmp_dun.data
};
} //namespace icmp
} // namespace snort

typedef snort::icmp::ICMPHdr ICMPHdr;

#ifndef ICMP_ECHOREPLY
constexpr uint8_t ICMP_ECHOREPLY = 0;    /* Echo Reply                   */
#endif

constexpr uint8_t ICMP_DEST_UNREACH = 3;    /* Destination Unreachable      */
constexpr uint8_t ICMP_SOURCE_QUENCH = 4;    /* Source Quench                */
#ifndef ICMP_REDIRECT
constexpr uint8_t ICMP_REDIRECT = 5;    /* Redirect (change route)      */
#endif
#ifndef ICMP_ECHO
constexpr uint8_t ICMP_ECHO = 8;    /* Echo Request                 */
#endif
constexpr uint8_t ICMP_ROUTER_ADVERTISE = 9;    /* Router Advertisement         */
constexpr uint8_t ICMP_ROUTER_SOLICIT = 10;    /* Router Solicitation          */
constexpr uint8_t ICMP_TIME_EXCEEDED = 11;    /* Time Exceeded                */
constexpr uint8_t ICMP_PARAMETERPROB = 12;    /* Parameter Problem            */
constexpr uint8_t ICMP_TIMESTAMP = 13;    /* Timestamp Request            */
constexpr uint8_t ICMP_TIMESTAMPREPLY = 14;    /* Timestamp Reply              */
constexpr uint8_t ICMP_INFO_REQUEST = 15;    /* Information Request          */
constexpr uint8_t ICMP_INFO_REPLY = 16;    /* Information Reply            */
constexpr uint8_t ICMP_ADDRESS = 17;    /* Address Mask Request         */
constexpr uint8_t ICMP_ADDRESSREPLY = 18;    /* Address Mask Reply           */
constexpr uint8_t NR_ICMP_TYPES = 18;

/* Codes for ICMP UNREACHABLES */
constexpr uint8_t ICMP_NET_UNREACH = 0;    /* Network Unreachable          */
constexpr uint8_t ICMP_HOST_UNREACH = 1;    /* Host Unreachable             */
constexpr uint8_t ICMP_PROT_UNREACH = 2;    /* Protocol Unreachable         */
constexpr uint8_t ICMP_PORT_UNREACH = 3;    /* Port Unreachable             */
constexpr uint8_t ICMP_FRAG_NEEDED = 4;    /* Fragmentation Needed/DF set  */
constexpr uint8_t ICMP_SR_FAILED = 5;    /* Source Route failed          */
constexpr uint8_t ICMP_NET_UNKNOWN = 6;
constexpr uint8_t ICMP_HOST_UNKNOWN = 7;
constexpr uint8_t ICMP_HOST_ISOLATED = 8;
constexpr uint8_t ICMP_PKT_FILTERED_NET = 9;
constexpr uint8_t ICMP_PKT_FILTERED_HOST = 10;
constexpr uint8_t ICMP_NET_UNR_TOS = 11;
constexpr uint8_t ICMP_HOST_UNR_TOS = 12;
constexpr uint8_t ICMP_PKT_FILTERED = 13;    /* Packet filtered */
constexpr uint8_t ICMP_PREC_VIOLATION = 14;    /* Precedence violation */
constexpr uint8_t ICMP_PREC_CUTOFF = 15;    /* Precedence cut off */

constexpr uint8_t ICMP_REDIR_NET = 0;
constexpr uint8_t ICMP_REDIR_HOST = 1;
constexpr uint8_t ICMP_REDIR_TOS_NET = 2;
constexpr uint8_t ICMP_REDIR_TOS_HOST = 3;

constexpr uint8_t ICMP_TIMEOUT_TRANSIT = 0;
constexpr uint8_t ICMP_TIMEOUT_REASSY = 1;

constexpr uint8_t ICMP_PARAM_BADIPHDR = 0;
constexpr uint8_t ICMP_PARAM_OPTMISSING = 1;
constexpr uint8_t ICMP_PARAM_BAD_LENGTH = 2;

#endif /* ICMP4_H */

icmp6.h

Path = src/protocols/icmp6.h

#ifndef PROTOCOLS_ICMP6_H
#define PROTOCOLS_ICMP6_H

#include <cstdint>

namespace snort
{
namespace icmp
{
constexpr uint16_t ICMP6_HEADER_MIN_LEN = 4;
constexpr uint16_t ICMP6_HEADER_NORMAL_LEN = 8;

#define ICMPv6_NS_MIN_LEN 24
#define ICMPv6_NA_MIN_LEN 24
#define ICMPv6_RS_MIN_LEN 24
#define ICMPv6_RA_MIN_LEN 16

#define ICMPV6_OPTION_SOURCE_LINKLAYER_ADDRESS 1
#define ICMPV6_OPTION_TARGET_LINKLAYER_ADDRESS 2
#define ICMPV6_OPTION_PREFIX_INFO              3
#define ICMPV6_OPTION_REDIRECT_HEADER          4
#define ICMPV6_OPTION_MTU                      5

//enum class Icmp6Types : std::uint8_t
enum Icmp6Types : std::uint8_t
{
    DESTINATION_UNREACHABLE = 1,
    PACKET_TOO_BIG = 2,
    TIME_EXCEEDED6 = 3,
    PARAMETER_PROBLEM = 4,
    ECHO_REQUEST = 128,
    ECHO_REPLY = 129,
    MULTICAST_LISTENER_QUERY = 130,
    MULTICAST_LISTENER_REPORT = 131,
    MULTICAST_LISTENER_DONE = 132,
    ROUTER_SOLICITATION = 133,
    ROUTER_ADVERTISEMENT = 134,
    NEIGHBOR_SOLICITATION = 135,
    NEIGHBOR_ADVERTISEMENT = 136,
    REDIRECT6 = 137,
    NODE_INFORMATION_QUERY = 139,
    NODE_INFORMATION_RESPONSE = 140,
    INVERSE_NEIGHBOR_DISCOVERY_SOLICITATION = 141,
    INVERSE_NEIGHBOR_DISCOVERY_ADVERTISEMENT = 142,
    VERSION_2_MULTICAST_LISTENER_REPORT = 143,
    HOME_AGENT_ADDRESS_DISCOVERY_REQUEST = 144,
    HOME_AGENT_ADDRESS_DISCOVERY_REPLY = 145,
    MOBILE_PREFIX_SOLICITATION = 146,
    MOBILE_PREFIX_ADVERTISEMENT = 147,
    CERTIFICATION_PATH_SOLICITATION = 148,
    CERTIFICATION_PATH_ADVERTISEMENT = 149,
    MULTICAST_ROUTER_ADVERTISEMENT = 151,
    MULTICAST_ROUTER_SOLICITATION = 152,
    MULTICAST_ROUTER_TERMINATION = 153,
    FMIPV6 = 154,
    RPL_CONTROL = 155,
    ILNPV6_LOCATOR_UPDATE = 156,
    DUPLICATE_ADDRESS_REQUEST = 157,
    DUPLICATE_ADDRESS_CONFIRMATION = 158,
    MPL_CONTROL = 159
};

enum class Icmp6Code : std::uint8_t
{
    /* Type == 1 */
    UNREACH_NET = 0x00,
    UNREACH_FILTER_PROHIB = 0x01,
    UNREACH_INVALID = 0x02,
    UNREACH_HOST = 0x03,
    UNREACH_PORT = 0x04,

    /* Type == ADVERTISEMENT */
    ADVERTISEMENT = 0X00,
};

struct Icmp6Hdr
{
    Icmp6Types type;
    Icmp6Code code;
    uint16_t csum;

    union
    {
        uint32_t opt32;
        uint16_t opt16[2];
        uint8_t opt8[4];
    };
};

struct ICMP6TooBig
{
    uint8_t type;
    uint8_t code;
    uint16_t csum;
    uint32_t mtu;
};

struct ICMP6RouterAdvertisement
{
    uint8_t type;
    uint8_t code;
    uint16_t csum;
    uint8_t num_addrs;
    uint8_t addr_entry_size;
    uint16_t lifetime;
    uint32_t reachable_time;
    uint32_t retrans_time;
};

struct ICMP6RouterSolicitation
{
    uint8_t type;
    uint8_t code;
    uint16_t csum;
    uint32_t reserved;
};

struct ICMP6NodeInfo
{
    uint8_t type;
    uint8_t code;
    uint16_t csum;
    uint16_t qtype;
    uint16_t flags;
    uint64_t nonce;
};
}  // namespace icmp
}  // namespace snort

#endif

ip.h

Path = src/protocols/ip.h

#ifndef PROTOCOLS_IP_H
#define PROTOCOLS_IP_H

#include <cstring>

#include "main/snort_types.h"
#include "protocols/ipv4.h"
#include "protocols/ipv6.h"
#include "sfip/sf_ip.h"

namespace snort
{
struct Packet;

// FIXIT-L can I assume api is always valid?
// i.e. if not ip4, then ipv6?
// or if not ip4, also make sure its not ip6

namespace ip
{
// keeping this as a class to avoid confusion.
class SO_PUBLIC IpApi
{
public:
    enum Type { IAT_NONE, IAT_4, IAT_6, IAT_DATA };

    // constructor and destructor MUST remain a trivial. Adding
    // any non-trivial code will cause a compilation failure.
    IpApi() = default;

    void set(const IP4Hdr* h4);
    void set(const IP6Hdr* h6);
    void set(const SfIp& src, const SfIp& dst);
    bool set(const uint8_t* raw_ip_data);
    void update(const SfIp& sip, const SfIp& dip);
    void reset();

    // return the 16 bits associated with this IP layers frag_offset/flags
    uint16_t off_w_flags() const;
    // return the frag_offset associated with this IP layers in word size.
    //   (the value is internally masked and multiplied)
    uint16_t off() const;
    // return the frag_id associated with this IP layers
    uint32_t id() const;
    const uint8_t* ip_data() const; // return a pointer to the ip layers data

    // FIXIT-L get rid of the unnecessary ones
    // returns the sum of the ip header + payload lengths in host byte order
    uint16_t dgram_len() const;
    // returns this ip layer's payload length in host byte order
    uint16_t pay_len() const;
    // return the ip_len field in host byte order
    uint16_t actual_ip_len() const;

    // overloaded == operators.
    friend bool operator==(const IpApi& lhs, const IpApi& rhs);
    friend bool operator!=(const IpApi& lhs, const IpApi& rhs);

    // returns true if this api is set.
    inline bool is_valid() const
    { return (type != IAT_NONE); }

    inline bool is_ip6() const
    { return (type == IAT_6); }

    inline bool is_ip4() const
    { return (type == IAT_4); }

    inline bool is_ip() const
    { return is_ip4() or is_ip6(); }

    inline const IP4Hdr* get_ip4h() const
    { return (type == IAT_4) ? (const IP4Hdr*)iph : nullptr; }

    inline const IP6Hdr* get_ip6h() const
    { return (type == IAT_6) ? (const IP6Hdr*)iph : nullptr; }

    inline const SfIp* get_src() const
    { return (type != IAT_NONE) ? &src : nullptr; }

    inline const SfIp* get_dst() const
    { return (type != IAT_NONE) ? &dst : nullptr; }

    // only relevant to IP4
    inline uint8_t get_ip_opt_len() const
    { return (type == IAT_4) ? ((const IP4Hdr*)iph)->get_opt_len() : 0; }

    // only relevant to IP4
    inline const uint8_t* get_ip_opt_data() const
    { return (type == IAT_4) ? reinterpret_cast<const uint8_t*>(iph) + IP4_HEADER_LEN : nullptr; }

    inline const snort_in6_addr* get_ip6_src() const
    { return (type == IAT_6) ? ((const IP6Hdr*)iph)->get_src() : nullptr; }

    inline const snort_in6_addr* get_ip6_dst() const
    { return (type == IAT_6) ? ((const IP6Hdr*)iph)->get_dst() : nullptr; }

    uint16_t tos() const;
    uint8_t ttl() const;
    IpProtocol proto() const;
    uint16_t raw_len() const;
    uint8_t hlen() const;
    uint8_t ver() const;

private:
    SfIp src = {};
    SfIp dst = {};
    const void* iph = nullptr;
    Type type = IAT_NONE;
};

} // namespace ip
} // namespace snort
#endif

ipv4.h

Path = src/protocols/ipv4.h

#ifndef PROTOCOLS_IPV4_H
#define PROTOCOLS_IPV4_H

#include <arpa/inet.h>

#include "protocols/protocol_ids.h" // include ipv4 protocol numbers

#define ETHERNET_TYPE_IP 0x0800

#ifndef IP_MAXPACKET
#define IP_MAXPACKET    65535        /* maximum packet size */
#endif

namespace snort
{
namespace ip
{
constexpr uint32_t IP4_BROADCAST = 0xffffffff;
constexpr uint8_t IP4_HEADER_LEN = 20;
constexpr uint8_t IP4_THIS_NET  = 0x00;  // msb
constexpr uint8_t IP4_MULTICAST = 0x0E;  // ms nibble
constexpr uint8_t IP4_RESERVED = 0x0F;  // ms nibble
constexpr uint8_t IP4_LOOPBACK = 0x7F;  // msb

// This must be a standard layer struct!
struct IP4Hdr
{
    uint8_t ip_verhl;      /* version & header length */
    uint8_t ip_tos;        /* type of service */
    uint16_t ip_len;       /* datagram length */
    uint16_t ip_id;        /* identification  */
    uint16_t ip_off;       /* fragment offset */
    uint8_t ip_ttl;        /* time to live field */
    IpProtocol ip_proto;      /* datagram protocol */
    uint16_t ip_csum;      /* checksum */
    uint32_t ip_src;  /* source IP */
    uint32_t ip_dst;  /* dest IP */

    /* getters */
    inline uint8_t hlen() const
    { return (uint8_t)((ip_verhl & 0x0f) << 2); }

    inline uint8_t ver() const
    { return (ip_verhl >> 4); }

    inline uint8_t tos() const
    { return ip_tos; }

    inline uint16_t len() const
    { return ntohs(ip_len); }

    inline uint8_t ttl() const
    { return ip_ttl; }

    inline IpProtocol proto() const
    { return ip_proto; }

    inline uint16_t off_w_flags() const
    { return ntohs(ip_off); }

    inline uint16_t rb() const
    { return ntohs(ip_off) & 0x8000; }

    inline uint16_t df() const
    { return ntohs(ip_off) & 0x4000; }

    inline uint16_t mf() const
    { return ntohs(ip_off) & 0x2000; }

    inline uint16_t off() const
    { return (uint16_t)((ntohs(ip_off) & 0x1FFF) << 3); }

    inline uint16_t id() const
    { return ntohs(ip_id); }

    inline uint8_t get_opt_len() const
    { return hlen() - IP4_HEADER_LEN; }

    inline uint16_t csum() const
    { return ntohs(ip_csum); }

    /* booleans */
    inline bool is_src_broadcast() const
    { return ip_src == IP4_BROADCAST; }

    inline bool is_dst_broadcast() const
    { return ip_dst == IP4_BROADCAST; }

    inline bool has_options() const
    { return hlen() > 20; }

    /* Access raw data */
    inline uint16_t raw_len() const
    { return ip_len; }

    inline uint16_t raw_id() const
    { return ip_id; }

    inline uint16_t raw_off() const
    { return ip_off; }

    inline uint16_t raw_csum() const
    { return ip_csum; }

    inline uint32_t get_src() const
    { return ip_src; }

    inline uint32_t get_dst() const
    { return ip_dst; }

    /*  setters  */
    inline void set_hlen(uint8_t value)
    { ip_verhl = (ip_verhl & 0xf0) | (value & 0x0f); }

    inline void set_proto(IpProtocol prot)
    { ip_proto = prot; }

    inline void set_ip_len(uint16_t new_len)
    { ip_len = htons(new_len); }
};

inline bool isPrivateIP(uint32_t addr)
{
    addr = ntohl(addr);
    switch (addr & 0xFF000000)
    {
    case 0x0A000000: // 10.0.0.0/8
        return true;
    case 0xA9000000: // 169.254.0.0/16
        return (addr & 0x00FF0000) == 0x00FE0000;
    case 0xAC000000: // 172.16.0.0/12
        return (addr & 0x00F00000) == 0x00100000;
    case 0xC0000000: // 192.168.0.0/16
        return (addr & 0x00FF0000) == 0x00A80000;
    }
    return false;
}
} // namespace ip
} // namespace snort
/* tcpdump shows us the way to cross platform compatibility */
#endif

ipv4_options.h

Path = src/protocols/ipv4_options.h

#ifndef PROTOCOLS_IP_OPTIONS_H
#define PROTOCOLS_IP_OPTIONS_H

#include "main/snort_types.h"

namespace snort
{
struct Packet;

namespace ip
{
struct IP4Hdr;

enum class IPOptionCodes : std::uint8_t
{
    EOL = 0x00,
    NOP = 0x01,
    RR = 0x07,
    TS = 0x44,
    SECURITY = 0x82,
    LSRR = 0x83,
    LSRR_E = 0x84,
    ESEC = 0x85,
    SATID = 0x88,
    SSRR = 0x89,
    RTRALT = 0x94,
    ANY = 0xff,
};

// FIXIT-L reduce all these classes to a simple pointer based approach
// that doesn't require any reinterpret casts (see also tcp_options.h)
struct IpOptions
{
    IPOptionCodes code;
    uint8_t len;
    uint8_t data[40];  // maximum possible

    inline uint8_t get_len() const
    { return ((uint8_t)code <= 1) ? 1 : len; }

    inline const uint8_t* get_data() const
    { return (((uint8_t)code <= 1) || (len < 2)) ? nullptr : &data[0]; }

    inline const IpOptions& next() const
    {
#ifdef __GNUC__
        //  because gcc requires strict aliasing.
        const uint8_t tmp_len = ((uint8_t)code <= 1) ? 1 : len;
        const uint8_t* const tmp = reinterpret_cast<const uint8_t*>(this);
        const IpOptions* opt = reinterpret_cast<const IpOptions*>(&tmp[tmp_len]);
        return *opt;

#else
        // ... and the legible code
        if ( (uint8_t)code <= 1 )
            return reinterpret_cast<const IpOptions&>(len);
        else
            return reinterpret_cast<const IpOptions&>(data[len -2]);
#endif
    }
};

/*
 * really creative name ... right
 * Use IpOptionIter ... this is the placeholder
 */
class SO_PUBLIC IpOptionIteratorIter
{
public:
    IpOptionIteratorIter(const IpOptions*);

    bool operator==(const IpOptionIteratorIter& rhs)
    { return opt == rhs.opt; }

    bool operator!=(const IpOptionIteratorIter& rhs)
    { return opt != rhs.opt; }

    // I'd suggest just using IpOptionIterator and completely ignoring this
    // horror of a ++ operation.
    IpOptionIteratorIter& operator++()
    {
        opt = &opt->next();
        return *this;
    }

    const IpOptions& operator*() const;

private:
    const IpOptions* opt;
};

/*
 * really creative name ... right
 * Use IP ranged for loop rather than calling this directly.
 * i.e.,
 *      IpOptionIter iter(ip4h, p)
 *      for (auto i : iter)
 *      {
 *          do_something
 *      }
 */
class SO_PUBLIC IpOptionIterator
{
public:
    // CONSTRUCTOR VALID AFTER DECODE()
    // Some options in the provided header may not be valid.
    // Provide the packet struct ensures only valid options
    // will be returned
    IpOptionIterator(const IP4Hdr* const, const Packet* const);

    // If you already know the validated option length (for instance,
    // if you are in a decode() method), then call this constructor.
    // You MUST validate all ip_options within len before using this
    // constructor
    IpOptionIterator(const IP4Hdr* const, const uint8_t valid_hdr_len);
    IpOptionIteratorIter begin() const;
    IpOptionIteratorIter end() const;

private:
    const uint8_t* end_ptr;
    const uint8_t* start_ptr;
};
} // namespace ip
} // namespace snort
#endif

ipv6.h

Path = src/protocols/ipv6.h

#ifndef PROTOCOLS_IPV6_H
#define PROTOCOLS_IPV6_H

#include <arpa/inet.h>

#include "protocols/protocol_ids.h"

namespace snort
{
namespace ip
{
constexpr uint8_t IP6_HEADER_LEN = 40;
constexpr uint32_t MIN_EXT_LEN = 8;
constexpr uint8_t IP6_MULTICAST = 0xFF;  // first/most significant octet

constexpr uint16_t IP6F_MF_MASK = 0x0001; /* more-fragments flag */
constexpr uint16_t IP6F_RES_MASK = 0x0006; /* reserved bits */

enum class MulticastScope : uint8_t
{
    RESERVED = 0x00,
    INTERFACE = 0x01,
    LINK = 0x02,
    ADMIN = 0x04,
    SITE = 0x05,
    ORG = 0x08,
    GLOBAL = 0x0E,
};

/* IPv6 address */
struct snort_in6_addr
{
    union
    {
        uint8_t u6_addr8[16];
        uint16_t u6_addr16[8];
        uint32_t u6_addr32[4];
    };
};

struct IP6Hdr
{
    uint32_t ip6_vtf;               /* 4 bits version, 8 bits TC,len
                                        20 bits flow-ID */
    uint16_t ip6_payload_len;               /* payload length */
    IpProtocol ip6_next;                 /* next header */
    uint8_t ip6_hoplim;                /* hop limit */

    snort_in6_addr ip6_src;      /* source address */
    snort_in6_addr ip6_dst;      /* destination address */

    inline uint16_t len() const
    { return ntohs(ip6_payload_len); }

    /* Same function as ipv4 */
    inline IpProtocol proto() const
    { return ip6_next; }

    inline IpProtocol next() const
    { return ip6_next; }

    inline uint8_t hop_lim() const
    { return ip6_hoplim; }

    inline uint8_t ver() const
    { return (uint8_t)(ntohl(ip6_vtf) >> 28); }

    inline uint16_t tos() const
    { return (uint16_t)((ntohl(ip6_vtf) & 0x0FF00000) >> 20); }

    inline uint32_t flow() const
    { return (ntohl(ip6_vtf) & 0x000FFFFF); }

    // because Snort expects this in terms of 32 bit words.
    inline uint8_t hlen() const
    { return IP6_HEADER_LEN; }

    inline const snort_in6_addr* get_src() const
    { return &ip6_src; }

    inline const snort_in6_addr* get_dst() const
    { return &ip6_dst; }

    inline MulticastScope get_dst_multicast_scope() const
    { return static_cast<MulticastScope>(ip6_dst.u6_addr8[1] & 0x0F); }

    /* booleans */
    inline bool is_src_multicast() const
    { return (ip6_src.u6_addr8[0] == IP6_MULTICAST); }

    inline bool is_dst_multicast() const
    { return ip6_dst.u6_addr8[0] == IP6_MULTICAST; }

    inline bool is_dst_multicast_scope_reserved() const
    { return static_cast<MulticastScope>(ip6_dst.u6_addr8[1]) == MulticastScope::RESERVED; }

    inline bool is_dst_multicast_scope_interface() const
    { return static_cast<MulticastScope>(ip6_dst.u6_addr8[1]) == MulticastScope::INTERFACE; }

    inline bool is_dst_multicast_scope_link() const
    { return static_cast<MulticastScope>(ip6_dst.u6_addr8[1]) == MulticastScope::LINK; }

    inline bool is_dst_multicast_scope_site() const
    { return (static_cast<MulticastScope>(ip6_dst.u6_addr8[1]) == MulticastScope::SITE); }

    inline bool is_dst_multicast_scope_global() const
    { return (static_cast<MulticastScope>(ip6_dst.u6_addr8[1]) == MulticastScope::GLOBAL); }

    inline bool is_valid_next_header() const
    {
        switch (ip6_next)
        {
        case IpProtocol::NONEXT:
        case IpProtocol::TCP:
        case IpProtocol::UDP:
        case IpProtocol::ICMPV6:
        case IpProtocol::HOPOPTS:
        case IpProtocol::DSTOPTS:
        case IpProtocol::ROUTING:
        case IpProtocol::FRAGMENT:
        case IpProtocol::MPLS_IP:
        case IpProtocol::GRE:
        case IpProtocol::MOBILITY_IPV6:
            return true;
        default:
            break;
        }
        return false;
    }

    /*  setters  */
    inline void set_len(uint16_t new_len)
    { ip6_payload_len = htons(new_len); }

    inline void set_proto(IpProtocol prot)
    { ip6_next = prot; }

    inline void set_raw_len(uint16_t new_len)
    { ip6_payload_len = new_len; }

    /* Access raw data */

    inline uint16_t raw_len() const
    { return ip6_payload_len; }
};

enum class HopByHopOptions : uint8_t
{
    PAD1 = 0x00,
    PADN = 0x01,
    TUNNEL_ENCAP = 0x04,
    RTALERT = 0x05,
    QUICK_START = 0x06,
    CALIPSO = 0x07,
    HOME_ADDRESS = 0xC9,
    JUMBO = 0xC2,
    ENDPOINT_IDENT = 0x8A,
};

/* to store references to IP6 Extension Headers */
struct IP6Option
{
    uint8_t type;
    const uint8_t* data;
};

/* Generic Extension Header */
struct IP6Extension
{
    IpProtocol ip6e_nxt;
    uint8_t ip6e_len;
    /* options follow */
    uint8_t ip6e_pad[6];
};

/* Fragment header */
struct IP6Frag
{
    IpProtocol ip6f_nxt;       /* next header */
    uint8_t ip6f_reserved;      /* reserved field */
    uint16_t ip6f_offlg;    /* offset, reserved, and flag */
    uint32_t ip6f_ident;    /* identification */

    inline IpProtocol next() const
    { return ip6f_nxt; }

    inline uint16_t off_w_flags() const
    { return ntohs(ip6f_offlg); }

    inline uint16_t off() const
    { return ntohs(ip6f_offlg) & 0xFFF8; }

    inline uint16_t mf() const
    { return ntohs(ip6f_offlg) & IP6F_MF_MASK; }

    inline uint16_t rb() const
    { return ntohs(ip6f_offlg) & IP6F_RES_MASK; }

    inline uint32_t id() const
    { return ntohl(ip6f_ident); }

    inline uint8_t res() const
    { return ip6f_reserved; }

    inline uint16_t raw_off_w_flags() const
    { return ip6f_offlg; }

    inline uint32_t raw_id() const
    { return ip6f_ident; }
};

// Reflects the recommended IPv6 order in RFC 2460 4.1
constexpr int IPV6_ORDER_MAX = 7;
inline int IPV6IdExtensionOrder(const ProtocolId prot_id)
{
    switch (prot_id)
    {
    case ProtocolId::HOPOPTS:   return 1;
    case ProtocolId::DSTOPTS:   return 2;
    case ProtocolId::ROUTING:   return 3;
    case ProtocolId::FRAGMENT:  return 4;
    case ProtocolId::AUTH:      return 5;
    case ProtocolId::ESP:       return 6;
    default:                   return IPV6_ORDER_MAX;
    }
}

inline int IPV6ExtensionOrder(const IpProtocol ip_proto)
{
    return IPV6IdExtensionOrder((ProtocolId)ip_proto);
}
} // namespace ip
} // namespace snort
#endif

layer.h

Path = src/protocols/layer.h

#ifndef PROTOCOLS_LAYER_H
#define PROTOCOLS_LAYER_H

#include "main/snort_types.h"
#include "protocols/protocol_ids.h"

namespace snort
{
struct Layer
{
    const uint8_t* start;
    ProtocolId prot_id;
    uint16_t length;
};

// forward declaring relevant structs. Since we're only returning a pointer,
// there is no need for the actual header files

namespace arp
{
struct EtherARP;
}

namespace cisco_meta_data
{
struct CiscoMetaDataHdr;
}

namespace eapol
{
struct EtherEapol;
}

namespace eth
{
struct EtherHdr;
}

namespace geneve
{
struct GeneveLyr;
}

namespace gre
{
struct GREHdr;
}

namespace icmp
{
struct ICMPHdr;
}

namespace ip
{
class IpApi;
struct IP6Frag;
}

namespace tcp
{
struct TCPHdr;
}

namespace udp
{
struct UDPHdr;
}

namespace vlan
{
struct VlanTagHdr;
}

namespace wlan
{
struct WifiHdr;
}

struct Packet;

namespace layer
{
//  Set by PacketManager.  Ensure you can call layer:: without a packet pointers
void set_packet_pointer(const Packet* const);

SO_PUBLIC const uint8_t* get_root_layer(const Packet* const);

SO_PUBLIC const arp::EtherARP* get_arp_layer(const Packet*);
SO_PUBLIC const cisco_meta_data::CiscoMetaDataHdr* get_cisco_meta_data_layer(const Packet* const);
SO_PUBLIC const eth::EtherHdr* get_eth_layer(const Packet*);
SO_PUBLIC const geneve::GeneveLyr* get_geneve_layer(const Packet*, bool inner);
SO_PUBLIC const gre::GREHdr* get_gre_layer(const Packet*);
SO_PUBLIC const vlan::VlanTagHdr* get_vlan_layer(const Packet*);
SO_PUBLIC const wlan::WifiHdr* get_wifi_layer(const Packet*);

/* return a pointer to the outermost UDP layer */
SO_PUBLIC const udp::UDPHdr* get_outer_udp_lyr(const Packet* const);
// return the inner ip layer's index in the p->layers array
SO_PUBLIC int get_inner_ip_lyr_index(const Packet* const);
SO_PUBLIC const Layer* get_mpls_layer(const Packet* const);

// Two versions of this because ip_defrag:: wants to call this on
// its rebuilt packet, not on the current packet.  Extra function
// header will be removed once layer is a part of the Packet struct
SO_PUBLIC const ip::IP6Frag* get_inner_ip6_frag();
SO_PUBLIC const ip::IP6Frag* get_inner_ip6_frag(const Packet* const);

// returns -1 on failure if no frag layer exists.
// else, returns zero based ip6 index
SO_PUBLIC int get_inner_ip6_frag_index(const Packet* const);

// ICMP with Embedded IP layer

// Sets the Packet's api to be the IP layer which is
// embedded inside an ICMP layer.
// RETURN:
//          true - ip layer found and api set
//          false - ip layer NOT found, api reset
SO_PUBLIC bool set_api_ip_embed_icmp(const Packet*, ip::IpApi& api);

/*
 *When a protocol is embedded in ICMP, these functions
 * will return a pointer to the layer.  Use the
 * proto_bits before calling these function to determine
 * what this layer is!
 */
SO_PUBLIC const tcp::TCPHdr* get_tcp_embed_icmp(const ip::IpApi&);
SO_PUBLIC const udp::UDPHdr* get_udp_embed_icmp(const ip::IpApi&);
SO_PUBLIC const icmp::ICMPHdr* get_icmp_embed_icmp(const ip::IpApi&);
/*
 * Starting from layer 'curr_layer', continuing looking at increasingly
 * outermost layer for another IP protocol.  If an IP protocol is found,
 * set the given ip_api to that layer.
 * PARAMS:
 *          Packet* = packet struct containing data
 *          ip::Api = ip api to be set
 *          uint8_t& next_ip_proto = The ip_protocol after the current IP
 *                              layer refer to packet get_next_ip_proto()
 *                              for more information.
 *          int8_t curr_layer = the current, zero based layer from which to
 *                              start searching inward. After the function returns,
 *                              This field will be set to the layer before
 *                              the Ip Api.  If no IP layer is found,
 *                              it will be set to -1.
 *
 *                               0 <= curr_layer < p->num_layers
 * RETURNS:
 *          true:  if the api is set
 *          false: if the api has NOT been set
 *
 * NOTE: curr_layer is zero based.  That means to get all of the ip
 *       layers (starting from the innermost layer), during the first call
 *       'curr_layer == p->num_layers'.
 *
 * NOTE: This functions is extremely useful in a loop
 *          while (set_inner_ip_api(p, api, layer)) { ... }
 */
SO_PUBLIC bool set_inner_ip_api(const Packet* const, ip::IpApi&, int8_t& curr_layer);
SO_PUBLIC bool set_inner_ip_api(const Packet* const, ip::IpApi&,
    IpProtocol& next_ip_proto, int8_t& curr_layer);

/*
 * Identical to above function except will begin searching from the
 * outermost layer until the innermost layer.
 *
 * NOTE: curr_layer is zero based.  That means to get all of the ip
 *       layers (starting from the OUTERMOST layer), during the first call
 *       'curr_layer == 0'.
 */
SO_PUBLIC bool set_outer_ip_api(const Packet* const, ip::IpApi&, int8_t& curr_layer);
SO_PUBLIC bool set_outer_ip_api(const Packet* const, ip::IpApi&,
    IpProtocol& next_ip_proto, int8_t& curr_layer);
} // namespace layer
} // namespace snort
#endif

linux_sll.h

Path = src/protocols/linux_sll.h

#ifndef PROTOCOLS_LINUX_SLL_H
#define PROTOCOLS_LINUX_SLL_H

namespace snort
{
namespace linux_sll
{
/* 'Linux cooked captures' data
 * (taken from tcpdump source).
 */

const uint8_t SLL_HDR_LEN = 16;
const uint8_t SLL_ADDRLEN = 8;

struct SLLHdr
{
    uint16_t sll_pkttype;              /* packet type */
    uint16_t sll_hatype;               /* link-layer address type */
    uint16_t sll_halen;                /* link-layer address length */
    uint8_t sll_addr[SLL_ADDRLEN];             /* link-layer address */
    uint16_t sll_protocol;             /* protocol */
};

/*
 * sll_pkttype values.
 */

#define LINUX_SLL_HOST          0
#define LINUX_SLL_BROADCAST     1
#define LINUX_SLL_MULTICAST     2
#define LINUX_SLL_OTHERHOST     3
#define LINUX_SLL_OUTGOING      4

/* sll protocol values */

#define LINUX_SLL_P_802_3       0x0001  /* Novell 802.3 frames without 802.2 LLC header */
#define LINUX_SLL_P_802_2       0x0004  /* 802.2 frames (not D/I/X Ethernet) */
} // namespace linux_sll
} // namespace snort

#endif /* LINUX_SLL_H */

mpls.h

Path = src/protocols/mpls.h

#ifndef PROTOCOLS_MPLS_H
#define PROTOCOLS_MPLS_H

#include <cstdint>

namespace snort
{
namespace mpls
{
struct MplsHdr
{
    uint32_t label;
    uint8_t tc;
    uint8_t bos;
    uint8_t ttl;
};
} // namespace mpls
}
#endif

packet.h

Path = src/protocols/packet.h

#ifndef PROTOCOLS_PACKET_H
#define PROTOCOLS_PACKET_H

#include <daq_common.h>

#include "flow/flow.h"
#include "framework/decode_data.h"
#include "framework/pdu_section.h"
#include "main/snort_types.h"
#include "protocols/geneve.h"
#include "target_based/snort_protocols.h"

namespace snort
{
class Active;
class Endianness;
class Flow;
class ActiveAction;
class IpsContext;
class Obfuscator;
class SFDAQInstance;

#define PKT_REBUILT_FRAG          0x00000001  // is a rebuilt fragment
#define PKT_REBUILT_STREAM        0x00000002  // is a rebuilt stream
#define PKT_STREAM_UNEST_UNI      0x00000004  // is from an unestablished stream and
                                         // we've only seen traffic in one direction
#define PKT_STREAM_EST            0x00000008  // is from an established stream

#define PKT_STREAM_INSERT         0x00000010  // this packet has been queued for stream reassembly
#define PKT_STREAM_TWH            0x00000020  // packet completes the 3-way handshake
#define PKT_FROM_SERVER           0x00000040  // this packet came from the server side of a connection (TCP)
#define PKT_FROM_CLIENT           0x00000080  // this packet came from the client side of a connection (TCP)

#define PKT_PDU_HEAD              0x00000100  // start of PDU
#define PKT_PDU_TAIL              0x00000200  // end of PDU
#define PKT_DETECT_LIMIT          0x00000400  // alt_dsize is valid

#define PKT_ALLOW_MULTIPLE_DETECT 0x00000800  // packet has multiple PDUs
#define PKT_PAYLOAD_OBFUSCATE     0x00001000

#define PKT_STATELESS             0x00002000  // Packet has matched a stateless rule
#define PKT_PASS_RULE             0x00004000  // this packet has matched a pass rule
#define PKT_IP_RULE               0x00008000  // this packet is being evaluated against an IP rule
#define PKT_IP_RULE_2ND           0x00010000  // this packet is being evaluated against an IP rule

#define PKT_PSEUDO                0x00020000  // is a pseudo packet
#define PKT_MODIFIED              0x00040000  // packet had normalizations, etc.
#define PKT_RESIZED               0x00080000  // packet has new size

// neither of these flags will be set for (full) retransmissions or non-data segments
// a partial overlap results in out of sequence condition
// out of sequence condition is sticky
#define PKT_STREAM_ORDER_OK       0x00100000  // this segment is in order, w/o gaps
#define PKT_STREAM_ORDER_BAD      0x00200000  // this stream had at least one gap

#define PKT_FILE_EVENT_SET        0x00400000
#define PKT_IGNORE                0x00800000  // this packet should be ignored, based on port
#define PKT_RETRANSMIT            0x01000000  // packet is a re-transmitted pkt.
#define PKT_RETRY                 0x02000000  // this packet is being re-evaluated from the internal retry queue
#define PKT_USE_DIRECT_INJECT     0x04000000  // Use ioctl when injecting.
#define PKT_HAS_PARENT            0x08000000  // derived pseudo packet from current wire packet

#define PKT_WAS_SET               0x10000000  // derived pseudo packet (PDU) from current wire packet

#define PKT_MORE_TO_FLUSH         0x20000000 // when more data is available to StreamSplitter::scan
#define PKT_FAST_PAT_EVAL         0x40000000 // temporary until IpsOption api updated

#define PKT_TCP_PSEUDO_EST        0x80000000 // A one-sided or bidirectional without LWS TCP session was detected

#define TS_PKT_OFFLOADED          0x01

#define PKT_PDU_FULL (PKT_PDU_HEAD | PKT_PDU_TAIL)

enum PseudoPacketType
{
    PSEUDO_PKT_IP,
    PSEUDO_PKT_TCP,
    PSEUDO_PKT_UDP_QUIC,
    PSEUDO_PKT_USER,
    PSEUDO_PKT_DCE_SEG,
    PSEUDO_PKT_DCE_FRAG,
    PSEUDO_PKT_SMB_SEG,
    PSEUDO_PKT_SMB_TRANS,
    PSEUDO_PKT_MAX
};

constexpr int32_t MAX_PORTS = 65536;
constexpr uint16_t NUM_IP_PROTOS = 256;
constexpr uint8_t TCP_OPTLENMAX = 40; /* (((2^4) - 1) * 4  - TCP_HEADER_LEN) */
constexpr uint8_t DEFAULT_LAYERMAX = 40;

// Packet is an abstraction describing a unit of work.  it may define a
// wire packet or it may define a cooked packet.  the latter contains
// payload data only, no headers.
struct SO_PUBLIC Packet
{
    Packet(bool packet_data = true);
    ~Packet();

    Packet(const Packet&) = delete;
    Packet& operator=(const Packet&) = delete;

    Flow* flow;   /* for session tracking */
    Endianness* endianness = nullptr;
    Obfuscator* obfuscator = nullptr;

    uint32_t packet_flags;      /* special flags for the packet */
    uint32_t xtradata_mask;
    uint32_t proto_bits;        /* protocols contained within this packet */

    uint16_t alt_dsize;         /* size for detection (iff PKT_DETECT_LIMIT) */

    uint8_t num_layers;         /* index into layers for next encap */
    // FIXIT-M Consider moving ip_proto_next below `pkth`.
    IpProtocol ip_proto_next;      /* the protocol ID after IP and all IP6 extension */
    bool disable_inspect;
    mutable FilteringState filtering_state;
    PduSection sect;

    // nothing after this point is zeroed by reset() ...
    IpsContext* context = nullptr;
    Active* active = nullptr;
    Active* active_inst;
    ActiveAction** action = nullptr;
    ActiveAction* action_inst = nullptr;

    DAQ_Msg_h daq_msg;              // DAQ message this packet came from
    SFDAQInstance* daq_instance = nullptr;  // DAQ instance the message came from

    // Everything beyond this point is set by PacketManager::decode()
    const DAQ_PktHdr_t* pkth;   // packet meta data
    const uint8_t* pkt;         // raw packet data
    uint32_t pktlen = 0;        // raw packet data length

    // These are both set before PacketManager::decode() returns
    const uint8_t* data = nullptr;  /* packet payload pointer */
    uint16_t dsize = 0;             /* packet payload size */

    DecodeData ptrs; // convenience pointers used throughout Snort++
    Layer* layers;    /* decoded encapsulations */

    PseudoPacketType pseudo_type = PSEUDO_PKT_MAX;  // valid only when PKT_PSEUDO is set

    uint64_t user_inspection_policy_id;
    uint64_t user_ips_policy_id;
    uint64_t user_network_policy_id;

    uint8_t vlan_idx;
    uint8_t ts_packet_flags; // FIXIT-M packet flags should always be thread safe

    // IP_MAXPACKET is the minimum allowable max_dsize
    // there is no requirement that all data fit into an IP datagram
    // but we do require that an IP datagram fit into Packet space
    // we can increase beyond this if needed
    static const uint32_t max_dsize = IP_MAXPACKET;

    /*  Boolean functions - general information about this packet */
    bool is_eth() const
    { return ((proto_bits & PROTO_BIT__ETH) != 0); }

    bool has_ip() const
    { return ptrs.ip_api.is_ip(); }

    bool is_ip4() const
    { return ptrs.ip_api.is_ip4(); }

    bool is_ip6() const
    { return ptrs.ip_api.is_ip6(); }

    bool is_ip() const
    { return ptrs.get_pkt_type() == PktType::IP; }

    bool is_tcp() const
    { return ptrs.get_pkt_type() == PktType::TCP; }

    bool is_udp() const
    { return ptrs.get_pkt_type() == PktType::UDP; }

    bool is_icmp() const
    { return ptrs.get_pkt_type() == PktType::ICMP; }

    bool is_data() const
    { return (ptrs.get_pkt_type() == PktType::PDU) or (ptrs.get_pkt_type() == PktType::FILE) or
        (ptrs.get_pkt_type() == PktType::USER); }

    bool is_cooked() const
    { return ((packet_flags & PKT_PSEUDO) != 0); }

    bool is_fragment() const
    { return ptrs.decode_flags & DECODE_FRAG; }

    bool is_udp_tunneled() const
    {
        if (proto_bits & PROTO_BIT__UDP_TUNNELED)
        {
            assert(ptrs.udph);
            return true;
        }

        return false;
    }

    bool has_ip_hdr() const
    { return ((proto_bits & PROTO_BIT__ANY_IP) != 0); }

    bool has_tcp_data() const
    { return (proto_bits & PROTO_BIT__TCP) and data and dsize; }

    bool has_udp_data() const
    { return (proto_bits & PROTO_BIT__UDP) and data and dsize; }

    bool has_udp_quic_data() const
    { return (is_cooked() and pseudo_type == PSEUDO_PKT_UDP_QUIC) and data and dsize; }

    /* Get general, non-boolean information */
    PktType type() const
    { return ptrs.get_pkt_type(); } // defined in codec.h

    void set_detect_limit(uint16_t n)
    {
        alt_dsize = n;
        packet_flags |= PKT_DETECT_LIMIT;
    }

    uint16_t get_detect_limit()
    { return (packet_flags & PKT_DETECT_LIMIT) ? alt_dsize : dsize; }

    const char* get_type() const;
    const char* get_pseudo_type() const;

    /* the ip_api return the protocol_ID of the protocol directly after the
     * innermost IP layer.  However, especially in IPv6, the next protocol
     * can frequently be an IP extension.  Therefore, this function
     * return the protocol ID of the first protocol after all the
     * IP layers.  For instance, if the stack is
     *     eth::ip4::udp::teredo::ip6::hop_opts::ipv6_routing::tcp
     * this function return 6 == IPPROTO_TCP == IPPROTO_ID_TCP
     */
    IpProtocol get_ip_proto_next() const
    { return ip_proto_next; }

    /* Similar to above. However, this function
     * can be called in a loop to get all of the ip_proto's.
     * NOTE: Will only return protocols of validly decoded layers.
     *
     * PARAMS:
     *          lyr - zero based layer from which to start searching inward.
     *                  will always point to the layer after an IP header or
     *                  IPv6 extension.
     *          proto - the ip_proto (read above) for the next, outermost IP layer
     * EXAMPLE:
     *
     * uint8_t ip_proto;
     * int lyr = 0
     * while ( ip_proto_next(lyr, ip_proto))
     * {
     *    ....
     * }
     */
    bool get_ip_proto_next(uint8_t& lyr, IpProtocol& proto) const;

    void reset();
    void release_helpers();

    bool is_from_client() const
    { return (packet_flags & PKT_FROM_CLIENT) != 0; }

    bool is_from_server() const
    { return (packet_flags & PKT_FROM_SERVER) != 0; }

    bool is_from_client_originally() const
    { return (!flow || flow->flags.client_initiated) ? is_from_client() : is_from_server(); }

    bool is_from_server_originally() const
    { return (!flow || flow->flags.client_initiated) ? is_from_server() : is_from_client(); }

    bool is_from_application_client() const;

    bool is_from_application_server() const;

    bool is_full_pdu() const
    { return (packet_flags & PKT_PDU_FULL) == PKT_PDU_FULL; }

    bool is_pdu_start() const
    { return (packet_flags & PKT_PDU_HEAD) != 0; }

    bool has_paf_payload() const
    { return (packet_flags & PKT_REBUILT_STREAM) or is_full_pdu(); }

    bool is_rebuilt() const
    { return (packet_flags & (PKT_REBUILT_STREAM|PKT_REBUILT_FRAG)) != 0; }

    bool is_defrag() const
    { return (packet_flags & PKT_REBUILT_FRAG) != 0; }

    bool is_retry() const
    { return (packet_flags & PKT_RETRY) != 0; }

    bool is_offloaded() const
    { return (ts_packet_flags & TS_PKT_OFFLOADED) != 0; }

    void set_offloaded()
    { ts_packet_flags |= TS_PKT_OFFLOADED; }

    void clear_offloaded()
    { ts_packet_flags &= (~TS_PKT_OFFLOADED); }

    bool has_parent() const
    { return (packet_flags & PKT_HAS_PARENT) != 0; }

    bool was_set() const
    { return (packet_flags & PKT_WAS_SET) != 0; }

    bool is_detection_enabled(bool to_server);

    bool is_inter_group_flow() const
    { return (pkth->flags & DAQ_PKT_FLAG_SIGNIFICANT_GROUPS) != 0; }

    bool test_session_flags(uint32_t);

    SnortProtocolId get_snort_protocol_id();

    void set_snort_protocol_id(SnortProtocolId proto_id)
    {
        assert( ptrs.get_pkt_type() != PktType::PDU);

        if ( flow )
            flow->ssn_state.snort_protocol_id = proto_id;
    }

    uint16_t get_flow_vlan_id() const;
    uint32_t get_flow_geneve_vni() const;
    std::vector<snort::geneve::GeneveOptData> get_geneve_options(bool inner) const;

    int16_t get_ingress_group() const
    {
        if (is_inter_group_flow())
            return pkth->ingress_group;

        return DAQ_PKTHDR_UNKNOWN;
    }

    int16_t get_egress_group() const
    {
        if (is_inter_group_flow())
            return pkth->egress_group;

        return DAQ_PKTHDR_UNKNOWN;
    }

    void set_pdu_section(PduSection pdu_sect)
    { sect = pdu_sect; }

private:
    bool allocated;
};

/* Macros to deal with sequence numbers - p810 TCP Illustrated vol 2 */
#define SEQ_LT(a,b)  ((int)((a) - (b)) <  0)
#define SEQ_LEQ(a,b) ((int)((a) - (b)) <= 0)
#define SEQ_GT(a,b)  ((int)((a) - (b)) >  0)
#define SEQ_GEQ(a,b) ((int)((a) - (b)) >= 0)
#define SEQ_EQ(a,b)  ((int)((a) - (b)) == 0)

#define BIT(i) (0x1 << ((i)-1))

inline void SetExtraData(Packet* p, const uint32_t xid) { p->xtradata_mask |= BIT(xid); }

inline uint16_t extract_16bits(const uint8_t* const p)
{ return ntohs(*(const uint16_t*)(p)); }

inline uint32_t extract_32bits(const uint8_t* p)
{
    assert(p);

    return ntohl(*(const uint32_t*)p);
}

inline uint16_t alignedNtohs(const uint16_t* ptr)
{
    uint16_t value;

    if (ptr == nullptr)
        return 0;

    value = *ptr;

#ifdef WORDS_BIGENDIAN
    return ((value & 0xff00) >> 8) | ((value & 0x00ff) << 8);
#else
    return value;
#endif
}

inline uint32_t alignedNtohl(const uint32_t* ptr)
{
    uint32_t value;

    if (ptr == nullptr)
        return 0;

    value = *ptr;

#ifdef WORDS_BIGENDIAN
    return ((value & 0xff000000) >> 24) | ((value & 0x00ff0000) >> 8)  |
           ((value & 0x0000ff00) << 8)  | ((value & 0x000000ff) << 24);
#else
    return value;
#endif
}

inline uint64_t alignedNtohq(const uint64_t* ptr)
{
    uint64_t value;

    if (ptr == nullptr)
        return 0;

    value = *ptr;

#ifdef WORDS_BIGENDIAN
    return ((value & 0xff00000000000000) >> 56) | ((value & 0x00ff000000000000) >> 40) |
           ((value & 0x0000ff0000000000) >> 24) | ((value & 0x000000ff00000000) >> 8)  |
           ((value & 0x00000000ff000000) << 8)  | ((value & 0x0000000000ff0000) << 24) |
           ((value & 0x000000000000ff00) << 40) | ((value & 0x00000000000000ff) << 56);
#else
    return value;
#endif
}
}
#endif

packet_manager.h

Path = src/protocols/packet_manager.h

#ifndef PACKET_MANAGER_H
#define PACKET_MANAGER_H

// PacketManager provides decode and encode services by leveraging Codecs.

#include <array>

#include "framework/codec.h"
#include "framework/counts.h"
#include "main/snort_types.h"
#include "managers/codec_manager.h"
#include "protocols/packet.h"

struct TextLog;

namespace snort
{
struct Packet;

enum class TcpResponse
{
    FIN,
    RST,
    PUSH,
};

enum class UnreachResponse
{
    NET,
    HOST,
    PORT,
    FWD,
};

// FIXIT-M roll the PacketManager and 'layer' namespace into the Packet
// struct

class SO_PUBLIC PacketManager
{
public:
    static void thread_init();
    static void thread_term();

    // decode this packet and set all relevant packet fields.
    static void decode(Packet*, const struct _daq_pkt_hdr*, const uint8_t* pkt,
        uint32_t pktlen, bool cooked = false, bool retry = false);

    // update the packet's checksums and length variables. Call this function
    // after Snort has changed any data in this packet
    static void encode_update(Packet*);

    //--------------------------------------------------------------------
    // FIXIT-L encode_format() should be replaced with a function that
    // does format and update in one step for packets cooked for internal
    // use only like stream_tcp and port_scan.  stream_ip packets should
    // just be decoded from last layer on.  at that point all the
    // Codec::format methods can be deleted too.  the new function should
    // be some super set of format_tcp().
    //--------------------------------------------------------------------

    // format packet for detection.  Original ttl is always used.  orig is
    // the wire pkt; clone was obtained with New()
    static int encode_format(
        EncodeFlags, const Packet* orig, Packet* clone,
        PseudoPacketType, const DAQ_PktHdr_t* = nullptr, uint32_t opaque = 0);

    static int format_tcp(
        EncodeFlags, const Packet* orig, Packet* clone, PseudoPacketType,
        const DAQ_PktHdr_t* = nullptr, uint32_t opaque = 0);

    // Send a TCP response.  TcpResponse params determined the type
    // of response. Len will be set to the response's length.
    // payload && payload_len are optional.
    static const uint8_t* encode_response(
        TcpResponse, EncodeFlags, const Packet* orig, uint32_t& len,
        const uint8_t* const payload = nullptr, uint32_t payload_len = 0);

    // Send an ICMP unreachable response!
    static const uint8_t* encode_reject(
        UnreachResponse, EncodeFlags, const Packet*, uint32_t& len);

    /* codec support and statistics */

    // get the max payload for the current packet
    static uint16_t encode_get_max_payload(const Packet*);

    // print codec information.  MUST be called after thread_term.
    static void dump_stats();

    static void reset_stats();

    // Get the name of the given protocol ID
    static const char* get_proto_name(ProtocolId);

    // Get the name of the given IP protocol
    static const char* get_proto_name(IpProtocol);

    // print this packets information, layer by layer
    static void log_protocols(TextLog* const, const Packet* const);

    /* Accessor functions -- any object in Snort++ can now convert a
     * protocol to its mapped value.
     *
     * The equivalent of Snort's PROTO_ID */
    static constexpr std::size_t max_protocols() // compile time constant
    { return CodecManager::s_protocols.size(); }

    /* If a proto was registered in a Codec's get_protocol_ids() function,
     * this function will return the 'ProtocolIndex' of the Codec to which the proto belongs.
     * If none of the loaded Codecs registered that proto, this function will
     * return zero. */
    static ProtocolIndex proto_idx(ProtocolId prot_id)
    { return CodecManager::s_proto_map[to_utype(prot_id)]; }

    static void accumulate();

private:
    static bool push_layer(Packet*, CodecData&, ProtocolId, const uint8_t* hdr_start, uint32_t len);
    static Codec* get_layer_codec(const Layer&, int idx);
    static void pop_teredo(Packet*, RawData&);
    static void handle_decode_failure(Packet*, RawData&, const CodecData&, const DecodeData&, ProtocolId);

    static bool encode(const Packet*, EncodeFlags,
        uint8_t lyr_start, IpProtocol next_prot, Buffer& buf);

    // constant offsets into the s_stats array.  Notice the stat_offset
    // constant which is used when adding a protocol specific codec
    static const uint8_t total_processed = 0;
    static const uint8_t other_codecs = 1;
    static const uint8_t discards = 2;
    static const uint8_t depth_exceeded = 3;
    static const uint8_t stat_offset = 4;

    // declared in header so it can access s_protocols
    static THREAD_LOCAL std::array<PegCount, stat_offset +
    CodecManager::s_protocols.size()> s_stats;
    // FIXIT-L gcc apparently does not consider thread_local variables to be valid in
    // constexpr expressions. As long as __thread is used instead of thread_local in gcc,
    // this is not a problem. However, if we use thread_local and gcc, the declaration
    // below will not compile.
    static std::array<PegCount, s_stats.size()> g_stats;
    static const std::array<const char*, stat_offset> stat_names;
};
}
#endif

protocol_ids.h

Path = src/protocols/protocol_ids.h

#ifndef PROTOCOLS_PROTOCOL_IDS_H
#define PROTOCOLS_PROTOCOL_IDS_H

#include <cassert>
#include <cstdint>
#include <limits>
#include <type_traits>

/*****************************************************************
 *****  NOTE:   Protocols are only included in this file when ****
 *****          their IDs are needed.                         ****
 ****************************************************************/

/*
 *  PROTOCOL ID'S By Range
 *   0    (0x0000) -   255  (0x00FF)  --> IP protocols
 *   256  (0x0100) -  1535  (0x05FF)  --> protocols without IDs (teredo, gtp)
 *  1536  (0x6000) -  65536 (0xFFFF)  --> Ethertypes
 */

//  Convert enum to a value cast to the enum's underlying type.
template<typename E>
inline constexpr typename std::underlying_type<E>::type to_utype(E enumerator)
{
    return static_cast<typename std::underlying_type<E>::type>(enumerator);
}

using ProtocolIndex = uint8_t;

/*
 * Below is a partial list of protocol numbers for the IP protocols.
 *  Defined at:
 * http://www.iana.org/assignments/protocol-numbers/protocol-numbers.xhtml
 */
enum class IpProtocol : std::uint8_t
{
    IP = 0,
    HOPOPTS = 0,
    ICMPV4 = 1,
    IGMP = 2,
    IPIP = 4,
    TCP = 6,
    UDP = 17,
    IPV6 = 41,
    ROUTING = 43,
    FRAGMENT = 44,
    GRE = 47,
    ESP = 50,
    AUTH = 51, // RFC 4302
    SWIPE = 53,
    MOBILITY = 55,
    ICMPV6 = 58,
    NONEXT = 59,
    DSTOPTS = 60,
    SUN_ND = 77,
    PIM = 103,
    PGM = 113,
    MOBILITY_IPV6 = 135,
    MPLS_IP = 137,
    /* Last updated 3/31/2016.
       Source: http://www.iana.org/assignments/protocol-numbers/protocol-numbers.xml */
    MIN_UNASSIGNED_IP_PROTO = 143,

    RESERVED = 255,       // == 0xFF
    PORT_SCAN = 255,
    PROTO_NOT_SET = 255,  // Indicates protocol has not been set.
};

//  Values up to 255 MUST be identical to those in IpProtocol.
enum class ProtocolId : std::uint16_t
{
    ETHERTYPE_NOT_SET = 0,
    IP = 0,
    HOPOPTS = 0,
    ICMPV4 = 1,
    IGMP = 2,
    IPIP = 4,
    TCP = 6,
    UDP = 17,
    IPV6 = 41,
    ROUTING = 43,
    FRAGMENT = 44,
    GRE = 47,
    ESP = 50,
    AUTH = 51, // RFC 4302
    SWIPE = 53,
    MOBILITY = 55,
    ICMPV6 = 58,
    NONEXT = 59,
    DSTOPTS = 60,
    SUN_ND = 77,
    PIM = 103,
    PGM = 113,
    MOBILITY_IPV6 = 135,
    MPLS_IP = 137,

    /* Last updated 3/31/2016.
       Source: http://www.iana.org/assignments/protocol-numbers/protocol-numbers.xml */
    MIN_UNASSIGNED_IP_PROTO = 143,

    RESERVED = 255,       // == 0xFF
    PORT_SCAN = 255,
    PROTO_NOT_SET = 255,  // Indicates protocol has not been set.

    /*
     *  Undefined Protocols!
     */
    FINISHED_DECODE = 0x0100,  // Indicates Codecs have successfully decoded packet
    TEREDO = 0x0101,
    GTP = 0x0102,
    IP_EMBEDDED_IN_ICMP4 = 0x0103,
    IP_EMBEDDED_IN_ICMP6 = 0x0104,
    ETHERNET_802_3 = 0x0105,
    ETHERNET_802_11 = 0x0106,
    ETHERNET_LLC = 0x0107,
    VXLAN = 0x0108,
    GENEVE = 0x0109,

    /*
     * Below is a partial list of ethertypes.
     *  Defined at:
     * http://www.iana.org/assignments/ieee-802-numbers/ieee-802-numbers.xhtml
     */
    ETHERTYPE_MINIMUM = 0x0600,   //  1536 - lowest ethertype value.
    ETHERTYPE_IPV4 = 0x0800,
    ETHERTYPE_ARP = 0x0806,
    ETHERTYPE_ERSPAN_TYPE3 = 0X22EB,
    ETHERTYPE_TRANS_ETHER_BRIDGING = 0x6558,
    ETHERTYPE_REVARP = 0x8035,
    ETHERTYPE_8021Q = 0x8100,
    ETHERTYPE_IPX = 0x8137,
    ETHERTYPE_IPV6 = 0x86DD,
    ETHERTYPE_PPP = 0x880B,
    ETHERTYPE_MPLS_UNICAST = 0x8847,
    ETHERTYPE_MPLS_MULTICAST = 0x8848,
    ETHERTYPE_PPPOE_DISC = 0x8863,
    ETHERTYPE_PPPOE_SESS = 0x8864,
    ETHERTYPE_EAPOL = 0x888E,
    ETHERTYPE_8021AD = 0x88A8,
    ETHERTYPE_ERSPAN_TYPE2 = 0x88BE,
    ETHERTYPE_FPATH = 0x8903,
    ETHERTYPE_CISCO_META = 0x8909,
};

static const auto num_protocol_ids =
    std::numeric_limits<std::underlying_type<ProtocolId>::type>::max() + 1;

inline bool is_ip_protocol(ProtocolId prot_id)
{
    return to_utype(prot_id) <= UINT8_MAX;
}

inline IpProtocol convert_protocolid_to_ipprotocol(const ProtocolId prot_id)
{
    assert(is_ip_protocol(prot_id));

    return IpProtocol(prot_id);
}

inline bool is_ip6_extension(const ProtocolId prot_id)
{
    if (!is_ip_protocol(prot_id))
        return false;

    switch (prot_id)
    {
    case ProtocolId::HOPOPTS:
    case ProtocolId::DSTOPTS:
    case ProtocolId::ROUTING:
    case ProtocolId::FRAGMENT:
    case ProtocolId::AUTH:
    case ProtocolId::ESP:
    case ProtocolId::MOBILITY_IPV6:
    case ProtocolId::NONEXT:
        return true;
    default:
        return false;
    }
}

#endif

ssl.h

Path = src/protocols/ssl.h

#ifndef SSL_H
#define SSL_H

#include "main/snort_types.h"

#define SSL_NO_FLAG             0x00000000

/* SSL record type flags */
#define SSL_CHANGE_CIPHER_FLAG  0x00000001
#define SSL_ALERT_FLAG          0x00000002
#define SSL_POSSIBLE_HS_FLAG    0x00000004 /* For handshakes in TLSv3 that are encrypted */
#define SSL_CLIENT_HELLO_FLAG   0x00000008
#define SSL_SERVER_HELLO_FLAG   0x00000010
#define SSL_CERTIFICATE_FLAG    0x00000020
#define SSL_SERVER_KEYX_FLAG    0x00000040
#define SSL_CLIENT_KEYX_FLAG    0x00000080
#define SSL_CIPHER_SPEC_FLAG    0x00000100
#define SSL_SFINISHED_FLAG      0x00000200
#define SSL_SAPP_FLAG           0x00000400
#define SSL_CAPP_FLAG           0x00000800
#define SSL_HS_SDONE_FLAG       0x00001000
#define SSL_HEARTBEAT_SEEN      0x00002000

/* Misc state flag */
#define SSL_POSSIBLY_ENC_FLAG   0x00004000

/* Version flags */
#define SSL_VER_SSLV2_FLAG      0x00008000
#define SSL_VER_SSLV3_FLAG      0x00010000
#define SSL_VER_TLS10_FLAG      0x00020000
#define SSL_VER_TLS11_FLAG      0x00040000
#define SSL_VER_TLS12_FLAG      0x00080000

/* Convert 3-byte lengths in TLS headers to integers. */
#define ntoh3(msb_ptr) \
    ((uint32_t)((uint32_t)(((const uint8_t*)(msb_ptr))[0] << 16) \
    + (uint32_t)(((const uint8_t*)(msb_ptr))[1] << 8) \
    + (uint32_t)(((const uint8_t*)(msb_ptr))[2])))

#define SSL_VERFLAGS \
    (SSL_VER_SSLV2_FLAG | SSL_VER_SSLV3_FLAG | \
    SSL_VER_TLS10_FLAG | SSL_VER_TLS11_FLAG | \
    SSL_VER_TLS12_FLAG)

#define SSL_V3_SERVER_HELLO(x) \
    (((x) & SSL_CUR_SERVER_HELLO_FLAG) \
    && ((x) & SSL_VERFLAGS) && (((x) & SSL_VERFLAGS) != SSL_VER_SSLV2_FLAG))

/* For rule state matching. These are only set when presently valid,
 * and do not stay set across packets. */
#define SSL_CUR_CLIENT_HELLO_FLAG   0x00100000
#define SSL_CUR_SERVER_HELLO_FLAG   0x00200000
#define SSL_CUR_SERVER_KEYX_FLAG    0x00400000
#define SSL_CUR_CLIENT_KEYX_FLAG    0x00800000
#define SSL_ENCRYPTED_FLAG          0x01000000 /* Provided for external use */
#define SSL_UNKNOWN_FLAG            0x02000000 /* Set when we decoded mostly garbage */

#define SSL_STATEFLAGS \
    (SSL_CUR_CLIENT_HELLO_FLAG | SSL_CUR_SERVER_HELLO_FLAG | \
    SSL_CUR_SERVER_KEYX_FLAG | SSL_CUR_CLIENT_KEYX_FLAG | \
    SSL_UNKNOWN_FLAG)

// Flag set when a client uses SSLv3/TLS backward compatibility and sends a
// SSLv2 Hello specifying an SSLv3/TLS version.
#define SSL_V3_BACK_COMPAT_V2   0x04000000

/* Error flags */
#define SSL_BOGUS_HS_DIR_FLAG   0x08000000 /* Record type disagrees with direction */
#define SSL_TRAILING_GARB_FLAG  0x10000000
#define SSL_BAD_TYPE_FLAG       0x20000000
#define SSL_BAD_VER_FLAG        0x40000000
#define SSL_TRUNCATED_FLAG      0x80000000
#define SSL_ARG_ERROR_FLAG      0x00000000 /* Note: overloaded with SSL_NO_FLAG */

/* The following flags are not presently of interest:
* #define SSL_CERT_URL_FLAG       (RFC 3546)
* #define SSL_CERT_STATUS_FLAG    (RFC 3546)
* #define SSL_CFINISHED_FLAG      This is contained in encrypted data
* #define SSL_HS_FINISHED_FLAG    Ignored for our purposes
*/

/* Flags for additional info */
#define SSL_ALERT_LVL_FATAL_FLAG    0x00000001

/* The constants used below are from RFC 2246 */

/* SSLv3 & TLS Record types */
#define SSL_CHANGE_CIPHER_REC 20
#define SSL_ALERT_REC 21
#define SSL_HANDSHAKE_REC 22
#define SSL_APPLICATION_REC 23
#define SSL_HEARTBEAT_REC 24

/* SSLv3 heartbeat types */
#define SSL_HEARTBEAT_REQUEST 1
#define SSL_HEARTBEAT_RESPONSE 2

/* SSLv3 & TLS handshake types */
#define SSL_HS_HELLO_REQ 0
#define SSL_HS_CHELLO    1
#define SSL_HS_SHELLO    2
#define SSL_HS_CERT      11
#define SSL_HS_SKEYX     12
#define SSL_HS_CERT_REQ  13
#define SSL_HS_SHELLO_DONE 14
#define SSL_HS_CERT_VERIFY 15
#define SSL_HS_CKEYX     16
#define SSL_HS_FINISHED  20
#define SSL_CERT_URL     21
#define SSL_CERT_STATUS  22

/* SSLv2 handshake types */
#define SSL_V2_CHELLO  1
#define SSL_V2_CKEY    2
#define SSL_V2_SHELLO  4

#pragma pack(1)

struct SSL_record_t
{
    uint8_t type;
    uint8_t major;
    uint8_t minor;
    uint16_t length;
};

#define SSL_REC_PAYLOAD_OFFSET (sizeof(uint8_t) * 5)

#define SSL_ALERT_LEVEL_WARNING 1
#define SSL_ALERT_LEVEL_FATAL   2

struct SSL_alert_t
{
    uint8_t level;
    uint8_t description;
};

struct SSL_heartbeat
{
    uint8_t type;
    uint16_t length;
};

struct SSL_handshake_t
{
    uint8_t type;
    uint8_t length[3];
};

struct SSL_handshake_hello_t
{
    uint8_t type;
    uint8_t length[3];
    uint8_t major;
    uint8_t minor;
};

// http://www.mozilla.org/projects/security/pki/nss/ssl/draft02.html
struct SSLv2_record_t
{
    uint16_t length;
    uint8_t type;
};

struct SSLv2_chello_t
{
    uint16_t length;
    uint8_t type;
    uint8_t major;
    uint8_t minor;
};

struct SSLv2_shello_t
{
    uint16_t length;
    uint8_t type;
    uint8_t ssnid;
    uint8_t certtype;
    uint8_t major;
    uint8_t minor;
};

struct SSLV3ClientHelloData
{
    ~SSLV3ClientHelloData();
    void clear();
    char* host_name = nullptr;
};

struct SSLV3ServerCertData
{
    ~SSLV3ServerCertData();
    void clear();
    /* While collecting certificates: */
    unsigned certs_len;   // (Total) length of certificate(s).
    uint8_t* certs_data = nullptr;  // Certificate(s) data (each proceeded by length (3 bytes)).
    /* Data collected from certificates afterwards: */
    char* common_name = nullptr;
    int common_name_strlen;
    char* org_unit = nullptr;
    int org_unit_strlen;
};

enum class SSLV3RecordType : uint8_t
{
    CLIENT_HELLO = 1,
    SERVER_HELLO = 2,
    CERTIFICATE = 11,
    SERVER_KEY_XCHG = 12,
    SERVER_CERT_REQ = 13,
    SERVER_HELLO_DONE = 14,
    CERTIFICATE_STATUS = 22
};

/* Usually referred to as a Certificate Handshake. */
struct ServiceSSLV3CertsRecord
{
    uint8_t type;
    uint8_t length_msb;
    uint16_t length;
    uint8_t certs_len[3];  // 3-byte length, network byte order.
    /* Certificate(s) follow.
     * For each:
     *  - Length: 3 bytes
     *  - Data  : "Length" bytes */
};

/* Usually referred to as a TLS Handshake. */
struct ServiceSSLV3Record
{
    SSLV3RecordType type;
    uint8_t length_msb;
    uint16_t length;
    uint16_t version;
    struct
    {
        uint32_t time;
        uint8_t data[28];
    } random;
};

struct ServiceSSLV3ExtensionServerName
{
    uint16_t type;
    uint16_t length;
    uint16_t list_length;
    uint8_t string_length_msb;
    uint16_t string_length;
    /* String follows. */
};

/* Extension types. */
#define SSL_EXT_SERVER_NAME 0

#define SSL_V2_MIN_LEN 5

#pragma pack()

#define SSL_HS_PAYLOAD_OFFSET (sizeof(uint8_t) * 4) /* Type and length fields */

#define SSL_BAD_HS(x) ((x) & SSL_BOGUS_HS_DIR_FLAG)

#define SSL_IS_HANDSHAKE(x) \
    ((x) & (SSL_CLIENT_HELLO_FLAG | SSL_SERVER_HELLO_FLAG | \
    SSL_CERTIFICATE_FLAG | SSL_SERVER_KEYX_FLAG | \
    SSL_CLIENT_KEYX_FLAG | SSL_CIPHER_SPEC_FLAG))

#define SSL_IS_CHELLO(x) ((x) & SSL_CLIENT_HELLO_FLAG)
#define SSL_IS_SHELLO(x) ((x) & SSL_SERVER_HELLO_FLAG)
#define SSL_IS_CKEYX(x) ((x) & SSL_CLIENT_KEYX_FLAG)
#define SSL_IS_CHANGE_CIPHER(x) ((x) & SSL_CHANGE_CIPHER_FLAG)
#define SSL_IS_APP(x) (((x) & SSL_SAPP_FLAG) || ((x) & SSL_CAPP_FLAG))
#define SSL_IS_ALERT(x) ((x) & SSL_ALERT_FLAG)
#define SSL_CLEAR_TEMPORARY_FLAGS(x) (x) &= ~SSL_STATEFLAGS

/* Verifies that the error flags haven't been triggered */
#define SSL_IS_CLEAN(x) \
    !((x) & (SSL_BOGUS_HS_DIR_FLAG | SSL_TRUNCATED_FLAG | \
    SSL_BAD_VER_FLAG | SSL_BAD_TYPE_FLAG | \
    SSL_TRAILING_GARB_FLAG | SSL_UNKNOWN_FLAG))

#define SSL_HEARTBLEED_REQUEST  0x01
#define SSL_HEARTBLEED_RESPONSE 0x02
#define SSL_HEARTBLEED_UNKNOWN  0x04

namespace snort
{
uint32_t SSL_decode(
    const uint8_t* pkt, int size, uint32_t pktflags, uint32_t prevflags,
    uint8_t* alert_flags, uint16_t* partial_rec_len, int hblen, uint32_t* info_flags = nullptr,
    SSLV3ClientHelloData* data = nullptr, SSLV3ServerCertData* server_cert_data = nullptr);

    void parse_client_hello_data(const uint8_t* pkt, uint16_t size, SSLV3ClientHelloData*);
    bool parse_server_certificates(SSLV3ServerCertData* server_cert_data);

SO_PUBLIC bool IsTlsClientHello(const uint8_t* ptr, const uint8_t* end);
SO_PUBLIC bool IsTlsServerHello(const uint8_t* ptr, const uint8_t* end);
SO_PUBLIC bool IsSSL(const uint8_t* ptr, int len, int pkt_flags);
}
#endif

tcp.h

Path = src/protocols/tcp.h

#ifndef PROTOCOLS_TCP_H
#define PROTOCOLS_TCP_H

#include <arpa/inet.h>

#include <cstdint>

namespace snort
{
namespace tcp
{
// these are bits in th_flags:
#define TH_FIN  0x01
#define TH_SYN  0x02
#define TH_RST  0x04
#define TH_PUSH 0x08
#define TH_ACK  0x10
#define TH_URG  0x20
#define TH_ECE  0x40
#define TH_CWR  0x80
#define TH_RES2 TH_ECE  // TBD TH_RES* should be deleted (see log.c)
#define TH_RES1 TH_CWR
#define TH_NORESERVED (TH_FIN|TH_SYN|TH_RST|TH_PUSH|TH_ACK|TH_URG)

// these are bits in th_offx2:
#define TH_RSV  0x0E  // reserved bits
#define TH_NS   0x01  // ECN nonce bit

/*
 * Default maximum segment size for TCP.
 * With an IP MSS of 576, this is 536,
 * but 512 is probably more convenient.
 * This should be defined as MIN(512, IP_MSS - sizeof (struct tcpiphdr)).
 */

#define TCP_MSS      512
#define TCP_MAXWIN   65535    /* largest value for (unscaled) window */
#define TCP_MAX_WINSHIFT    14    /* maximum window shift */

/*
 * User-settable options (used with setsockopt).
 */
#define TCP_NODELAY   0x01    /* don't delay send to coalesce packets */
#define TCP_MAXSEG    0x02    /* set maximum segment size */
#define SOL_TCP        6    /* TCP level */

#define GET_PKT_SEQ(p) (ntohl((p)->ptrs.tcph->th_seq))

constexpr uint8_t TCP_MIN_HEADER_LEN = 20; // this is actually the minimal TCP header length
constexpr int OPT_TRUNC = -1;
constexpr int OPT_BADLEN = -2;

struct TCPHdr
{
    uint16_t th_sport;     /* source port */
    uint16_t th_dport;     /* destination port */
    uint32_t th_seq;       /* sequence number */
    uint32_t th_ack;       /* acknowledgment number */
    uint8_t th_offx2;      /* offset and reserved */
    uint8_t th_flags;
    uint16_t th_win;       /* window */
    uint16_t th_sum;       /* checksum */
    uint16_t th_urp;       /* urgent pointer */

    /* Formatted data access and booleans */
    inline uint8_t hlen() const
    { return (th_offx2 & 0xf0) >> 2; }

    inline uint8_t off() const
    { return (th_offx2 & 0xf0) >> 2; }

    inline uint8_t options_len() const
    { return hlen() - TCP_MIN_HEADER_LEN; }

    inline uint16_t src_port() const
    { return ntohs(th_sport); }

    inline uint16_t dst_port() const
    { return ntohs(th_dport); }

    inline uint16_t win() const
    { return ntohs(th_win); }

    inline uint16_t cksum() const
    { return ntohs(th_sum); }

    inline uint16_t urp() const
    { return ntohs(th_urp); }

    inline uint32_t seq() const
    { return ntohl(th_seq); }

    inline uint32_t ack() const
    { return ntohl(th_ack); }

    inline bool has_options() const
    { return ((th_offx2 & 0xf0) > 0x50); }

    inline bool are_flags_set(uint8_t flags) const
    { return (th_flags & flags) == flags; }

    inline bool is_syn() const
    { return ((th_flags & TH_SYN) != 0); }

    inline bool is_syn_only() const
    { return (th_flags & (TH_SYN | TH_ACK)) == TH_SYN; }

    inline bool is_syn_ack() const
    { return are_flags_set(TH_SYN | TH_ACK); }

    inline bool is_ack() const
    { return ((th_flags & TH_ACK) != 0); }

    inline bool is_psh() const
    { return ((th_flags & TH_PUSH) != 0); }

    inline bool is_rst() const
    { return ((th_flags & TH_RST) != 0); }

    inline bool is_fin() const
    { return ((th_flags & TH_FIN) != 0); }

    /*  raw data access */
    inline uint16_t raw_src_port() const
    { return th_sport; }

    inline uint16_t raw_dst_port() const
    { return th_dport; }

    inline uint32_t raw_seq() const
    { return th_seq; }

    inline uint32_t raw_ack() const
    { return th_ack; }

    inline uint8_t raw_hlen() const
    { return th_offx2 >> 4; }

    inline uint16_t raw_win() const
    { return th_win; }

    inline uint16_t raw_cksum() const
    { return th_sum; }

    inline uint16_t raw_urp() const
    { return th_urp; }

    // setters
    inline void set_offset(uint8_t val)
    { th_offx2 = (uint8_t)((th_offx2 & 0x0f) | (val << 4)); }

    inline void set_urp(uint16_t new_urp)
    { th_urp = htons(new_urp); }

    inline void set_raw_urp(uint16_t new_urp)
    { th_urp = new_urp; }

    inline void set_seq(uint32_t new_seq)
    { th_seq = htonl(new_seq); }
};
}  // namespace tcp
}  // namespace snort

#endif

tcp_options.h

Path = src/protocols/tcp_options.h

#ifndef PROTOCOLS_TCP_OPTIONS_H
#define PROTOCOLS_TCP_OPTIONS_H

#include "main/snort_types.h"

namespace snort
{
struct Packet;

namespace tcp
{
struct TCPHdr;

/* http://www.iana.org/assignments/tcp-parameters
 *
 * tcp options stuff. used to be in <netinet/tcp.h> but it breaks
 * things on AIX
 */

enum class TcpOptCode : std::uint8_t
{
    EOL = 0,    /* End of Option List [RFC793] */
    NOP = 1,    /* No-Option [RFC793] */
    MAXSEG = 2, /* Maximum Segment Size [RFC793] */
    WSCALE = 3, /* Window scaling option [RFC1323] */
    SACKOK = 4, /* Experimental [RFC2018]*/
    SACK = 5,   /* Experimental [RFC2018] variable length */
    ECHO = 6,   /* Echo (obsoleted by option 8)      [RFC1072] */
    ECHOREPLY = 7,  /* Echo Reply (obsoleted by option 8)[RFC1072] */
    TIMESTAMP = 8,  /* Timestamp [RFC1323], 10 bytes */
    PARTIAL_PERM = 9,   /* Partial Order Permitted/ Experimental [RFC1693] */
    PARTIAL_SVC = 10,   /*  Partial Order Profile [RFC1693] */
    CC = 11,        /*  T/TCP Connection count  [RFC1644] */
    CC_NEW = 12,    /*  CC.NEW [RFC1644] */
    CC_ECHO = 13,   /*  CC.ECHO [RFC1644] */

    ALTCSUM = 15,   /* TCP Alternate Checksum Data [RFC1146], variable length */
    SKEETER = 16,   /* Skeeter [Knowles] */
    BUBBA = 17,     /* Bubba   [Knowles] */
    TRAILER_CSUM = 18,  /* Trailer Checksum Option [Subbu & Monroe] */
    MD5SIG = 19,    /* MD5 Signature Option [RFC2385] */

    /* Space Communications Protocol Standardization */
    SCPS = 20,  /* Capabilities [Scott] */
    SELNEGACK = 21,     /* Selective Negative Acknowledgements [Scott] */
    RECORDBOUND = 22,   /* Record Boundaries [Scott] */
    CORRUPTION = 23,    /* Corruption experienced [Scott] */
    SNAP = 24,  /* SNAP [Sukonnik] -- anyone have info?*/
    UNASSIGNED = 25,    /* Unassigned (released 12/18/00) */
    COMPRESSION = 26,   /* TCP Compression Filter [Bellovin] */
    /* http://www.research.att.com/~smb/papers/draft-bellovin-tcpcomp-00.txt*/

    AUTH = 29,  /* [RFC5925] - The TCP Authentication Option
                             Intended to replace MD5 Signature Option [RFC2385] */
};

/*  Associated lengths */
const uint8_t TCPOLEN_EOL = 1;      /* Always one byte - [RFC793]*/
const uint8_t TCPOLEN_NOP = 1;      /* Always one byte - [RFC793]*/
const uint8_t TCPOLEN_MAXSEG = 4;   /* Always 4 bytes - [RFC793] */
const uint8_t TCPOLEN_WSCALE = 3;   /* 1 byte with logarithmic values - [RFC1323]*/
const uint8_t TCPOLEN_SACKOK = 2;   /* Experimental [RFC2018]*/
const uint8_t TCPOLEN_ECHO = 6;     /* 6 bytes  - Echo (obsoleted by option 8)      [RFC1072] */
const uint8_t TCPOLEN_ECHOREPLY = 6;    /* 6 bytes  - Echo Reply (obsoleted by option 8)[RFC1072]*/
const uint8_t TCPOLEN_TIMESTAMP = 10;   /* Timestamp [RFC1323], 10 bytes */
const uint8_t TCPOLEN_PARTIAL_PERM = 2; /* Partial Order Permitted/ Experimental [RFC1693] */
const uint8_t TCPOLEN_PARTIAL_SVC = 3;  /*  3 bytes long -- Experimental - [RFC1693] */

/* at least decode T/TCP options... */
const uint8_t TCPOLEN_CC = 6;       /* page 17 of rfc1644 */
const uint8_t TCPOLEN_CC_NEW = 6;   /* page 17 of rfc1644 */
const uint8_t TCPOLEN_CC_ECHO = 6;  /* page 17 of rfc1644 */

const uint8_t TCPOLEN_TRAILER_CSUM = 3;
const uint8_t TCPOLEN_MD5SIG = 18;

// FIXIT-L reduce all these classes to a simple pointer based approach
// that doesn't require any reinterpret casts (see also ipv4_options.h)
struct TcpOption
{
    TcpOptCode code;
    uint8_t len;
    uint8_t data[40];  // maximum possible

    inline uint8_t get_len() const
    { return ((uint8_t)code <= 1) ? 1 : len; }

    inline const uint8_t* get_data() const
    { return ((uint8_t)code <= 1 || len < 2) ? nullptr : &data[0]; }

    inline const TcpOption& next() const
    {
        if ( (uint8_t)code <= 1 )
            return reinterpret_cast<const TcpOption&>(len);
        else
            return reinterpret_cast<const TcpOption&>(data[len -2]);
    }
};

class TcpOptIterator;

/*
 * Use TcpOptIterator ... this should NOT be called directly
 * unless you want to an actual iterator or some buggy code.
 */
class SO_PUBLIC TcpOptIteratorIter
{
public:
    TcpOptIteratorIter(const TcpOption*, const TcpOptIterator*);

    bool operator==(const TcpOptIteratorIter& rhs)
    { return opt == rhs.opt; }

    bool operator!=(const TcpOptIteratorIter& rhs)
    { return opt != rhs.opt; }

    const TcpOptIteratorIter& operator++();
    const TcpOption& operator*() const;

private:
    const TcpOption* opt;
    const TcpOptIterator* iter;
};

/*
 * Use IP ranged for loop rather than calling this directly.
 * i.e.,
 *      IpOptionIter iter(tcph, p)
 *      for (const TcpOption& opt : iter)
 *      {
 *          do_something
 *      }
 */
class SO_PUBLIC TcpOptIterator
{
public:
    /* CONSTRUCTOR VALID AFTER DECODE()
     * Some options in the provided header may not be valid.
     * Provide the packet struct ensures only valid options
     * will be returned
     */
    TcpOptIterator(const TCPHdr* const, const Packet* const);
    /* If you already know the validated option length (for instance,
     * if you are in a decode() method), then call this constructor.*/
    TcpOptIterator(const TCPHdr* const, const uint32_t valid_hdr_len);
    TcpOptIteratorIter begin() const;
    TcpOptIteratorIter end() const;

private:
    const uint8_t* start_ptr;
    const uint8_t* end_ptr;
};
} // namespace tcp
} // namespace snort

#endif

teredo.h

Path = src/protocols/teredo.h

#ifndef PROTOCOLS_TEREDO_H
#define PROTOCOLS_TEREDO_H

#include <cstdint>

namespace snort
{
namespace teredo
{
constexpr uint16_t TEREDO_PORT = 3544;
constexpr uint16_t INDICATOR_ORIGIN = 0x00;
constexpr uint16_t INDICATOR_ORIGIN_LEN = 8;
constexpr uint16_t INDICATOR_AUTH = 0x01;
constexpr uint16_t INDICATOR_AUTH_MIN_LEN = 13;
constexpr uint16_t MIN_HDR_LEN = 2;

inline bool is_teredo_port(uint16_t port)
{ return port == TEREDO_PORT; }
} // namespace teredo
} // namespace snort

#endif

token_ring.h

Path = src/protocols/token_ring.h

#ifndef PROTOCOLS_TOKEN_RING_H
#define PROTOCOLS_TOKEN_RING_H

#include <arpa/inet.h>

#include "protocols/protocol_ids.h"

namespace snort
{
namespace token_ring
{
/* LLC structure */
struct Trh_llc
{
    uint8_t dsap;
    uint8_t ssap;
    uint8_t protid[3];
    uint16_t ether_type;

    /* return data in byte order */
    inline ProtocolId ethertype() const
    { return (ProtocolId)ntohs(ether_type); }

    /* return data in network order */
    inline uint16_t raw_ethertype() const
    { return ether_type; }

};

/* RIF structure
 * Linux/tcpdump patch defines tokenring header in dump way, since not
 * every tokenring header with have RIF data... we define it separately, and
 * a bit more split up
 */

/* These are macros to use the bitlevel accesses in the Trh_Mr header

   they haven't been tested and they aren't used much so here is a
   listing of what used to be there

   #if defined(WORDS_BIGENDIAN)
      uint16_t bcast:3, len:5, dir:1, lf:3, res:4;
   #else
      uint16_t len:5,         length of RIF field, including RC itself
      bcast:3,       broadcast indicator
      res:4,         reserved
      lf:3,      largest frame size
      dir:1;         direction
*/

#define TRH_MR_BCAST(trhmr)  ((ntohs((trhmr)->bcast_len_dir_lf_res) & 0xe000) >> 13)
#define TRH_MR_LEN(trhmr)    ((ntohs((trhmr)->bcast_len_dir_lf_res) & 0x1F00) >> 8)
#define TRH_MR_DIR(trhmr)    ((ntohs((trhmr)->bcast_len_dir_lf_res) & 0x0080) >> 7)
#define TRH_MR_LF(trhmr)     ((ntohs((trhmr)->bcast_len_dir_lf_res) & 0x0070) >> 4)
#define TRH_MR_RES(trhmr)     ((ntohs((trhmr)->bcast_len_dir_lf_res) & 0x000F))

struct Trh_mr
{
    uint16_t bcast_len_dir_lf_res; /* broadcast/res/framesize/direction */
    uint16_t rseg[8];
};

#define TR_ALEN             6        /* octets in an Ethernet header */
#define FDDI_ALEN           6
#define IPARP_SAP           0xaa

struct Trh_hdr
{
    uint8_t ac;        /* access control field */
    uint8_t fc;        /* frame control field */
    uint8_t daddr[TR_ALEN];    /* src address */
    uint8_t saddr[TR_ALEN];    /* dst address */
};
/* End Token Ring Data Structures */

inline const Trh_mr* get_trhmr(const Trh_llc* llc)
{
    if (llc->dsap != IPARP_SAP && llc->ssap != IPARP_SAP)
        return reinterpret_cast<const Trh_mr*>(llc);

    return nullptr;
}
} // namespace token_ring
} // namespace snort

#endif

udp.h

Path = src/protocols/udp.h

#ifndef PROTOCOLS_UDP_H
#define PROTOCOLS_UDP_H

#include <arpa/inet.h>

namespace snort
{
namespace udp
{
constexpr uint8_t UDP_HEADER_LEN = 8;

struct UDPHdr
{
    uint16_t uh_sport;
    uint16_t uh_dport;
    uint16_t uh_len;
    uint16_t uh_chk;

    /*  formatted access to fields */
    inline uint16_t src_port() const
    { return ntohs(uh_sport); }

    inline uint16_t dst_port() const
    { return ntohs(uh_dport); }

    inline uint16_t len() const
    { return ntohs(uh_len); }

    inline uint16_t cksum() const
    { return ntohs(uh_chk); }

    /*  Raw access to fields */
    inline uint16_t raw_src_port() const
    { return uh_sport; }

    inline uint16_t raw_dst_port() const
    { return uh_dport; }

    inline uint16_t raw_len() const
    { return uh_len; }

    inline uint16_t raw_cksum() const
    { return uh_chk; }
};
} // namespace udp
} // namespace snort

#endif

vlan.h

Path = src/protocols/vlan.h

#ifndef PROTOCOLS_VLAN_H
#define PROTOCOLS_VLAN_H

#include <arpa/inet.h>

namespace snort
{
namespace vlan
{
struct VlanTagHdr
{
    uint16_t vth_pri_cfi_vlan;
    uint16_t vth_proto;  /* protocol field... */

    inline uint16_t priority() const
    { return ntohs(vth_pri_cfi_vlan) >> 13; }

    inline uint16_t cfi() const
    { return (ntohs(vth_pri_cfi_vlan) & 0x1000) >> 12; }

    inline uint16_t vid() const
    { return ntohs(vth_pri_cfi_vlan) & 0x0FFF; }

    inline uint16_t proto() const
    { return ntohs(vth_proto); }
};
} // namespace vlan
} // namespace snort

#endif

wlan.h

Path = src/protocols/wlan.h

#ifndef PROTOCOLS_WLAN_H
#define PROTOCOLS_WLAN_H

#include <cstdint>

namespace snort
{
namespace wlan
{
/*
 *  Wireless Header (IEEE 802.11)
 */
struct WifiHdr
{
    uint16_t frame_control;
    uint16_t duration_id;
    uint8_t addr1[6];
    uint8_t addr2[6];
    uint8_t addr3[6];
    uint16_t seq_control;
    uint8_t addr4[6];
};

/* Frame type/subtype combinations with version = 0
  ** FRAME TYPE *****  HEX ****  SUBTYPE TYPE  DESCRIPT ********/
#define WLAN_TYPE_MGMT_ASREQ   0x0      /* 0000    00  Association Req */
#define WLAN_TYPE_MGMT_ASRES   0x10     /* 0001    00  Assocaition Res */
#define WLAN_TYPE_MGMT_REREQ   0x20     /* 0010    00  Reassoc. Req.   */
#define WLAN_TYPE_MGMT_RERES   0x30     /* 0011    00  Reassoc. Resp.  */
#define WLAN_TYPE_MGMT_PRREQ   0x40     /* 0100    00  Probe Request   */
#define WLAN_TYPE_MGMT_PRRES   0x50     /* 0101    00  Probe Response  */
#define WLAN_TYPE_MGMT_BEACON  0x80     /* 1000    00  Beacon          */
#define WLAN_TYPE_MGMT_ATIM    0x90     /* 1001    00  ATIM message    */
#define WLAN_TYPE_MGMT_DIS     0xa0     /* 1010    00  Disassociation  */
#define WLAN_TYPE_MGMT_AUTH    0xb0     /* 1011    00  Authentication  */
#define WLAN_TYPE_MGMT_DEAUTH  0xc0     /* 1100    00  Deauthentication*/

#define WLAN_TYPE_CONT_PS      0xa4     /* 1010    01  Power Save      */
#define WLAN_TYPE_CONT_RTS     0xb4     /* 1011    01  Request to send */
#define WLAN_TYPE_CONT_CTS     0xc4     /* 1100    01  Clear to send   */
#define WLAN_TYPE_CONT_ACK     0xd4     /* 1101    01  Acknowledgement */
#define WLAN_TYPE_CONT_CFE     0xe4     /* 1110    01  Cont. Free end  */
#define WLAN_TYPE_CONT_CFACK   0xf4     /* 1111    01  CF-End + CF-Ack */

#define WLAN_TYPE_DATA_DATA    0x08     /* 0000    10  Data            */
#define WLAN_TYPE_DATA_DTCFACK 0x18     /* 0001    10  Data + CF-Ack   */
#define WLAN_TYPE_DATA_DTCFPL  0x28     /* 0010    10  Data + CF-Poll  */
#define WLAN_TYPE_DATA_DTACKPL 0x38     /* 0011    10  Data+CF-Ack+CF-Pl */
#define WLAN_TYPE_DATA_NULL    0x48     /* 0100    10  Null (no data)  */
#define WLAN_TYPE_DATA_CFACK   0x58     /* 0101    10  CF-Ack (no data)*/
#define WLAN_TYPE_DATA_CFPL    0x68     /* 0110    10  CF-Poll (no data)*/
#define WLAN_TYPE_DATA_ACKPL   0x78     /* 0111    10  CF-Ack+CF-Poll  */

/*** Flags for IEEE 802.11 Frame Control **
   The following are designed to be bitwise-AND-d in a uint8_t */
#define WLAN_FLAG_TODS      0x0100    /* To DS Flag   10000000 */
#define WLAN_FLAG_FROMDS    0x0200    /* From DS Flag 01000000 */
#define WLAN_FLAG_FRAG      0x0400    /* More Frag    00100000 */
#define WLAN_FLAG_RETRY     0x0800    /* Retry Flag   00010000 */
#define WLAN_FLAG_PWRMGMT   0x1000    /* Power Mgmt.  00001000 */
#define WLAN_FLAG_MOREDAT   0x2000    /* More Data    00000100 */
#define WLAN_FLAG_WEP       0x4000    /* Wep Enabled  00000010 */
#define WLAN_FLAG_ORDER     0x8000    /* Strict Order 00000001 */
} // namespace wlan
} // namespace snort

#endif

protocols/test/

protocols/test/framework/

pub_sub/

active_events.h

Path = src/pub_sub/active_events.h

#ifndef ACTIVE_EVENTS_H
#define ACTIVE_EVENTS_H

#include <framework/data_bus.h>
#include <packet_io/active.h>

namespace snort
{
struct Packet;

class SO_PUBLIC ActiveEvent : public DataEvent
{
public:
    ActiveEvent
        (const Active::ActiveActionType current, const Active::ActiveActionType previous, const Packet* p)
        : current_action(current), previous_action(previous), pkt(p)
    { }

    Active::ActiveActionType get_current_action() const
    { return current_action; }

    Active::ActiveActionType get_previous_action() const
    { return previous_action; }

    const Packet* get_packet() const override
    { return pkt; }

private:
    const Active::ActiveActionType current_action;
    const Active::ActiveActionType previous_action;
    const Packet* pkt;
};
}
#endif

appid_debug_log_event.h

Path = src/pub_sub/appid_debug_log_event.h

#ifndef APPID_DEBUG_LOG_EVENT_H
#define APPID_DEBUG_LOG_EVENT_H

#include <string>

#include "network_inspectors/appid/appid_debug.h"
#include "pub_sub/appid_event_ids.h"

class AppIdDebugLogEvent : public snort::DataEvent
{
public:
    AppIdDebugLogEvent(const AppIdDebugSessionConstraints* constraints, const char* dbg_str) :
        cs(constraints), debug_str(dbg_str) { }

    const AppIdDebugSessionConstraints* get_appid_debug_constraints() const
    {
        return cs;
    }

    const std::string& get_debug_string() const
    {
        return debug_str;
    }
private:
    const AppIdDebugSessionConstraints* cs = nullptr;
    std::string debug_str;
};

#endif

appid_event_ids.h

Path = src/pub_sub/appid_event_ids.h

#ifndef APPID_EVENT_IDS_H
#define APPID_EVENT_IDS_H

#include "framework/data_bus.h"

namespace snort
{

struct AppIdEventIds
{ enum : unsigned {

    ANY_CHANGE,
    DEBUG_LOG,
    DHCP_DATA,
    DHCP_INFO,
    FP_SMB_DATA,

    num_ids
}; };

const PubKey appid_pub_key { "appid", AppIdEventIds::num_ids };

}
#endif

appid_events.h

Path = src/pub_sub/appid_events.h

#ifndef APPID_EVENTS_H
#define APPID_EVENTS_H

// This event conveys data published by the appid module to be consumed by data bus subscribers

#include <bitset>

#include "pub_sub/appid_event_ids.h"

namespace snort
{
    class AppIdSessionApi;
}

// Events are added as needed by subscribers
// Any change here should also change change_bits_to_string()
enum AppidChangeBit
{
    APPID_CREATED_BIT = 0,
    APPID_RESET_BIT,

    // id
    APPID_SERVICE_BIT,
    APPID_CLIENT_BIT,
    APPID_PAYLOAD_BIT,
    APPID_MISC_BIT,
    APPID_REFERRED_BIT,

    // http
    APPID_HOST_BIT,
    APPID_TLSHOST_BIT,
    APPID_URL_BIT,
    APPID_USERAGENT_BIT,
    APPID_RESPONSE_BIT,
    APPID_REFERER_BIT,

    // dns
    APPID_DNS_HOST_BIT,

    // other
    APPID_SERVICE_INFO_BIT,
    APPID_CLIENT_INFO_BIT,
    APPID_USER_INFO_BIT,
    APPID_NETBIOS_NAME_BIT,
    APPID_NETBIOS_DOMAIN_BIT,
    APPID_DISCOVERY_FINISHED_BIT,

    APPID_MAX_BIT
};

typedef std::bitset<APPID_MAX_BIT> AppidChangeBits;

inline void change_bits_to_string(const AppidChangeBits& change_bits, std::string& str)
{
    size_t n = change_bits.count();

    if (change_bits.test(APPID_CREATED_BIT))
        --n? str.append("created, ") : str.append("created");
    if (change_bits.test(APPID_RESET_BIT))
        --n? str.append("reset, ") : str.append("reset");
    if (change_bits.test(APPID_SERVICE_BIT))
        --n? str.append("service, ") : str.append("service");
    if (change_bits.test(APPID_CLIENT_BIT))
        --n? str.append("client, ") : str.append("client");
    if (change_bits.test(APPID_PAYLOAD_BIT))
        --n? str.append("payload, ") : str.append("payload");
    if (change_bits.test(APPID_MISC_BIT))
        --n? str.append("misc, ") : str.append("misc");
    if (change_bits.test(APPID_REFERRED_BIT))
        --n? str.append("referred, ") : str.append("referred");
    if (change_bits.test(APPID_HOST_BIT))
        --n? str.append("host, ") : str.append("host");
    if (change_bits.test(APPID_TLSHOST_BIT))
        --n? str.append("tls-host, ") : str.append("tls-host");
    if (change_bits.test(APPID_URL_BIT))
        --n? str.append("url, ") : str.append("url");
    if (change_bits.test(APPID_USERAGENT_BIT))
        --n? str.append("user-agent, ") : str.append("user-agent");
    if (change_bits.test(APPID_RESPONSE_BIT))
        --n? str.append("response, ") : str.append("response");
    if (change_bits.test(APPID_REFERER_BIT))
        --n? str.append("referrer, ") : str.append("referrer");
    if (change_bits.test(APPID_DNS_HOST_BIT))
        --n? str.append("dns-host, ") : str.append("dns-host");
    if (change_bits.test(APPID_SERVICE_INFO_BIT))
        --n? str.append("service-info, ") : str.append("service-info");
    if (change_bits.test(APPID_CLIENT_INFO_BIT))
        --n? str.append("client-info, ") : str.append("client-info");
    if (change_bits.test(APPID_USER_INFO_BIT))
        --n? str.append("user-info, ") : str.append("user-info");
    if (change_bits.test(APPID_NETBIOS_NAME_BIT))
        --n? str.append("netbios-name, ") : str.append("netbios-name");
    if (change_bits.test(APPID_NETBIOS_DOMAIN_BIT))
        --n? str.append("netbios-domain, ") : str.append("netbios-domain");
    if (change_bits.test(APPID_DISCOVERY_FINISHED_BIT))
        --n? str.append("finished, ") : str.append("finished");
    if (n != 0) // make sure all bits from AppidChangeBit enum get translated
        str.append("change_bits_to_string error!");
}

class AppidEvent : public snort::DataEvent
{
public:
    AppidEvent(const AppidChangeBits& ac, bool is_httpx, uint32_t httpx_stream_index,
        const snort::AppIdSessionApi& api, const snort::Packet& p) :
        ac_bits(ac), is_httpx(is_httpx), httpx_stream_index(httpx_stream_index), api(api), p(p) {}

    const AppidChangeBits& get_change_bitset() const
    { return ac_bits; }

    bool get_is_httpx() const
    { return is_httpx; }

    uint32_t get_httpx_stream_index() const
    { return httpx_stream_index; }

    const snort::AppIdSessionApi& get_appid_session_api() const
    { return api; }

    const snort::Packet* get_packet() const override
    { return &p; }

private:
    const AppidChangeBits& ac_bits;
    bool is_httpx;
    uint32_t httpx_stream_index;
    const snort::AppIdSessionApi& api;
    const snort::Packet& p;
};

#endif

assistant_gadget_event.h

Path = src/pub_sub/assistant_gadget_event.h

#ifndef ASSISTANT_GADGET_EVENTS_H
#define ASSISTANT_GADGET_EVENTS_H

#include "framework/data_bus.h"
#include "pub_sub/intrinsic_event_ids.h"
#include "target_based/snort_protocols.h"
#include "utils/util.h"

// A flow is setting up assistant inspector.
// For example used by HTTP2 to set NHI as assistant inspector.

namespace snort
{
struct Packet;
}

class AssistantGadgetEvent : public snort::DataEvent
{
public:
    AssistantGadgetEvent(snort::Packet* packet, const char* _service) :
        p(packet), service(_service)
    { }

    snort::Packet* get_packet() const override
    { return p; }

    const char* get_service()
    { return service.c_str(); }

private:
    snort::Packet* p;
    std::string service;
};

#endif

auxiliary_ip_event.h

Path = src/pub_sub/auxiliary_ip_event.h

#ifndef AUXILIARY_IP_EVENT_H
#define AUXILIARY_IP_EVENT_H

#include "pub_sub/intrinsic_event_ids.h"
#include "sfip/sf_ip.h"

class AuxiliaryIpEvent : public snort::DataEvent
{
public:
   AuxiliaryIpEvent(const snort::SfIp& aux_ip) : ip(aux_ip) { }

   const snort::SfIp* get_ip()
   { return &ip; }

private:
   const snort::SfIp& ip;
};

#endif

cip_events.h

Path = src/pub_sub/cip_events.h

#ifndef CIP_EVENTS_H
#define CIP_EVENTS_H

// This event conveys data published by the CIP service inspector to be consumed
// by data bus subscribers

#include <list>

#include "framework/data_bus.h"

struct CipEventIds { enum : unsigned { DATA, num_ids }; };

const snort::PubKey cip_pub_key { "cip", CipEventIds::num_ids };

namespace snort
{
struct Packet;
struct SfIp;
}

struct CipEventData;

class SO_PUBLIC CipEvent : public snort::DataEvent
{
public:
    CipEvent(const snort::Packet*, const CipEventData*);

    const snort::Packet* get_packet() const override
    { return p; }

    const CipEventData* get_event_data()
    { return event_data; }
private:
    const snort::Packet* p;
    const CipEventData* event_data;
};

#endif

daq_message_event.h

Path = src/pub_sub/daq_message_event.h

#ifndef DAQ_MESSAGE_EVENT_H
#define DAQ_MESSAGE_EVENT_H

#include <daq.h>

#include "pub_sub/intrinsic_event_ids.h"

namespace snort
{
class SO_PUBLIC DaqMessageEvent : public snort::DataEvent
{
public:
    DaqMessageEvent(DAQ_Msg_h msg, DAQ_Verdict& v) : msg(msg), verdict(v) { }

    DAQ_Msg_h get_message()
    { return msg; }

    DAQ_MsgType get_type() const
    { return daq_msg_get_type(msg); }

    size_t get_header_length() const
    { return daq_msg_get_hdr_len(msg); }

    const void* get_header() const
    { return daq_msg_get_hdr(msg); }

    uint32_t get_data_length() const
    { return daq_msg_get_data_len(msg); }

    const uint8_t* get_data() override
    { return daq_msg_get_data(msg); }

    DAQ_Verdict get_verdict()
    { return verdict; }

    void set_verdict(DAQ_Verdict v)
    { verdict = v; }

private:
    DAQ_Msg_h msg;
    DAQ_Verdict& verdict;
};
}

#endif

data_decrypt_event.h

Path = src/pub_sub/data_decrypt_event.h

#ifndef DATA_DECRYPT_EVENT_H
#define DATA_DECRYPT_EVENT_H

#include "pub_sub/external_event_ids.h"

class DataDecryptEvent : public snort::DataEvent
{
public:

    enum StateEventType : uint16_t
    {
        DATA_DECRYPT_MONITOR_EVENT,
        DATA_DECRYPT_DO_NOT_DECRYPT_EVENT,
        DATA_DECRYPT_BLOCK_EVENT,
        DATA_DECRYPT_BLOCK_WITH_RESET_EVENT,
        DATA_DECRYPT_START_EVENT
    };

    DataDecryptEvent(const StateEventType& type)  : m_type(type)  { }
    StateEventType get_type() const { return m_type; }

private:
    StateEventType m_type;
};

#endif

dcerpc_events.h

Path = src/pub_sub/dcerpc_events.h

#ifndef DCERPC_EVENTS_H
#define DCERPC_EVENTS_H

#include "framework/data_bus.h"

struct DceTcpEventIds { enum : unsigned { EXP_SESSION, num_ids }; };

const snort::PubKey dce_tcp_pub_key { "dce_tcp", DceTcpEventIds::num_ids };

namespace snort
{
struct SfIp;
}

class DceExpectedSessionEvent : public snort::DataEvent
{
public:

    DceExpectedSessionEvent(snort::Packet* p,
        const snort::SfIp* src_ip, const uint16_t src_port,
        const snort::SfIp* dst_ip, const uint16_t dst_port,
        IpProtocol proto, SnortProtocolId protocol_id) :
            p(p), src_ip(src_ip), src_port(src_port),
            dst_ip(dst_ip), dst_port(dst_port),
            proto(proto), protocol_id(protocol_id) { }

    const snort::Packet* get_packet() const override
    { return p; }

    const snort::SfIp* get_src_ip() const
    { return src_ip; }

    const snort::SfIp* get_dst_ip() const
    { return dst_ip; }

    uint16_t get_src_port() const
    { return src_port; }

    uint16_t get_dst_port() const
    { return dst_port; }

    SnortProtocolId get_proto_id() const
    { return protocol_id; }

    IpProtocol get_ip_proto() const
    { return proto; }

private:
    const snort::Packet* p;

    const snort::SfIp* src_ip;
    uint16_t src_port;

    const snort::SfIp* dst_ip;
    uint16_t dst_port;

    IpProtocol proto;
    SnortProtocolId protocol_id;
};

#endif // DCERPC_EVENTS_H

dhcp_events.h

Path = src/pub_sub/dhcp_events.h

#ifndef DHCP_EVENTS_H
#define DHCP_EVENTS_H

#include <cstring>
#include "pub_sub/appid_event_ids.h"

#define DHCP_OP55_MAX_SIZE  64
#define DHCP_OP60_MAX_SIZE  64

namespace snort
{

class DHCPInfoEvent : public snort::DataEvent
{
public:
    DHCPInfoEvent(const snort::Packet* p, uint32_t ip_address, const uint8_t* eth,
        uint32_t subnet_mask, uint32_t lease_secs, uint32_t router) :
        pkt(p), ip_address(ip_address), subnet_mask(subnet_mask),
        lease_secs(lease_secs), router(router)
    {
        memcpy(eth_addr, eth, sizeof(eth_addr));
    }

    const snort::Packet* get_packet() const override
    { return pkt; }

    uint32_t get_ip_address() const
    { return ip_address; }

    const uint8_t* get_eth_addr() const
    { return eth_addr; }

    uint32_t get_subnet_mask() const
    { return subnet_mask; }

    uint32_t get_lease_secs() const
    { return lease_secs; }

    uint32_t get_router() const
    { return router; }

private:
    const snort::Packet* pkt;
    uint32_t ip_address;
    uint8_t eth_addr[6];
    uint32_t subnet_mask;
    uint32_t lease_secs;
    uint32_t router;
};

class DHCPDataEvent : public snort::DataEvent
{
public:
    DHCPDataEvent(const snort::Packet* p, unsigned op55_len, unsigned op60_len,
        const uint8_t* op55_val, const uint8_t* op60_val, const uint8_t* eth) : pkt(p),
        op55_len(op55_len), op60_len(op60_len)
    {
        memcpy(op55, op55_val, op55_len);
        if (op60_len)
            memcpy(op60, op60_val, op60_len);
        memcpy(eth_addr, eth, sizeof(eth_addr));
    }

    const snort::Packet* get_packet() const override
    { return pkt; }

    unsigned get_op55_len() const
    { return op55_len; }

    unsigned get_op60_len() const
    { return op60_len; }

    const uint8_t* get_op55() const
    { return op55; }

    const uint8_t* get_op60() const
    { return op60; }

    const uint8_t* get_eth_addr() const
    { return eth_addr; }

private:
    const snort::Packet* pkt;
    unsigned op55_len;
    unsigned op60_len;
    uint8_t op55[DHCP_OP55_MAX_SIZE] = {0};
    uint8_t op60[DHCP_OP60_MAX_SIZE] = {0};
    uint8_t eth_addr[6];
};

}

#endif // DHCP_EVENTS_H

dns_events.h

Path = src/pub_sub/dns_events.h

#ifndef DNS_EVENTS_H
#define DNS_EVENTS_H

// This event allows the dns service inspector to publish dns response
// for use by data bus subscribers

#include "framework/data_bus.h"
#include "sfip/sf_ip.h"

struct FqdnTtl
{
    FqdnTtl(const std::string& fqdn, uint32_t ttl) :
        fqdn(fqdn), ttl(ttl)
    {}

    std::string fqdn;
    uint32_t ttl;
};

struct IPFqdnCacheItem
{
    void add_ip(const snort::SfIp& ip);
    void add_fqdn(const FqdnTtl& fqdn_ttl);

    std::vector<snort::SfIp> ips;
    std::vector<FqdnTtl> fqdns;
};

struct DnsEventIds
{
    enum : unsigned
    {
        DNS_RESPONSE_DATA,
        num_ids
    };
};

const snort::PubKey dns_pub_key { "dns", DnsEventIds::num_ids };

class DnsResponseIp;
class DnsResponseFqdn;

namespace snort
{
class SO_PUBLIC DnsResponseDataEvents : public snort::DataEvent
{
public:
    void add_ip(const DnsResponseIp& event);
    void add_fqdn(DnsResponseFqdn& event, uint32_t ttl);
    void get_dns_data(IPFqdnCacheItem& ip_fqdn_cache_item);
    bool empty() const;

private:
    std::vector<DnsResponseIp> dns_ips;
    std::vector<DnsResponseFqdn> dns_fqdns;
};
}

#endif

eve_process_event.h

Path = src/pub_sub/eve_process_event.h

#ifndef EVE_PROCESS_EVENT_H
#define EVE_PROCESS_EVENT_H

#include <string>
#include "pub_sub/external_event_ids.h"

class EveProcessEvent : public snort::DataEvent
{
public:
    EveProcessEvent(const snort::Packet& p, const char* process, uint8_t process_conf) :
        p(p), process_name(process), process_confidence(process_conf) { }

    EveProcessEvent(const snort::Packet& p, const char* server) : p(p), server_name(server) { }

    EveProcessEvent(const snort::Packet& p) : p(p) { }

    const snort::Packet* get_packet() const override { return &p; }

    const std::string& get_process_name() const
    {
        return process_name;
    }

    uint8_t get_process_confidence() const
    {
        return process_confidence;
    }

    const std::string& get_server_name() const
    {
        return server_name;
    }

    void set_server_name(const char* server)
    {
        if (server)
            server_name = server;
    }

    const std::string& get_user_agent() const
    {
        return user_agent;
    }

    void set_user_agent(const char* u_a)
    {
        if (u_a)
            user_agent = u_a;
    }

    const std::vector<std::string> get_alpn() const
    {
        return alpn;
    }

    void set_alpn(const std::vector<std::string>& alpn_vec)
    {
        if(alpn_vec.size())
            alpn = alpn_vec;
    }

    void set_quic(bool flag)
    {
        is_quic = flag;
    }

    bool is_flow_quic() const
    {
        return is_quic;
    }

    bool is_client_process_mapping() const
    {
        return client_process_mapping;
    }

    void set_client_process_mapping(bool flag)
    {
        client_process_mapping = flag;
    }

private:
    const snort::Packet &p;
    std::string process_name;
    uint8_t process_confidence = 0;
    std::string server_name;
    std::string user_agent;
    std::vector<std::string> alpn;
    bool is_quic = false;
    bool client_process_mapping = true;
};
#endif

expect_events.h

Path = src/pub_sub/expect_events.h

#ifndef EXPECT_EVENTS_H
#define EXPECT_EVENTS_H

// This event conveys data published by the expected sessions to be consumed
// by data bus subscribers

#include <list>
#include <vector>

#include "pub_sub/intrinsic_event_ids.h"

#define EXPECT_EVENT_TYPE_EARLY_SESSION_CREATE_KEY "expect_event_type_early_session_create"

namespace snort
{
struct ExpectFlow;
class FlowData;
struct Packet;
}

class ExpectEvent : public snort::DataEvent
{
public:
    ExpectEvent(const snort::Packet* ctrl_packet, snort::ExpectFlow* ef,
        const snort::FlowData* fd)
    {
        p = ctrl_packet;
        expect_flow = ef;
        flow_data = fd;
    }

    const snort::Packet* get_packet() const override
    { return p; }

    snort::ExpectFlow* get_expect_flow()
    { return expect_flow; }

    const snort::FlowData* get_flow_data()
    { return flow_data; }

private:
    const snort::Packet* p;
    snort::ExpectFlow* expect_flow;
    const snort::FlowData* flow_data;
};

#define EXPECT_EVENT_TYPE_HANDLE_FLOWS "expect.handle_flows"

class ExpectedFlowsEvent : public snort::DataEvent
{
public:
    ExpectedFlowsEvent(std::vector<snort::ExpectFlow*>& expected_flows, const snort::Packet& p)
        : expected_flows(expected_flows), pkt(p)
    { }

    std::vector<snort::ExpectFlow*>& get_expected_flows()
    { return expected_flows; }

    const snort::Packet* get_packet() const override
    { return &pkt; }

private:
    std::vector<snort::ExpectFlow*>& expected_flows;
    const snort::Packet& pkt;
};

#endif

external_event_ids.h

Path = src/pub_sub/external_event_ids.h

#ifndef EXTERNAL_EVENT_IDS_H
#define EXTERNAL_EVENT_IDS_H

#include "framework/data_bus.h"

struct ExternalEventIds
{ enum : unsigned {

    CPE_OS_INFO,
    DATA_DECRYPT,
    EVE_PROCESS,

    num_ids
}; };

const snort::PubKey external_pub_key { "external", ExternalEventIds::num_ids };

#endif

finalize_packet_event.h

Path = src/pub_sub/finalize_packet_event.h

#ifndef FINALIZE_PACKET_EVENT_H
#define FINALIZE_PACKET_EVENT_H

// An event to indicate that the packet is about to be finalized (sent
// back to the daq).

#include <daq_common.h>

#include "pub_sub/intrinsic_event_ids.h"

namespace snort
{

class SO_PUBLIC FinalizePacketEvent : public snort::DataEvent
{
public:
    FinalizePacketEvent(const snort::Packet* p, DAQ_Verdict& v) :
        pkt(p), verdict(v)
    {
    }

    const snort::Packet* get_packet() const override
    { return pkt; }

    DAQ_Verdict& get_verdict()
    { return verdict; }

private:
    const snort::Packet* pkt;
    DAQ_Verdict& verdict;
};

}

#endif

http_event_ids.h

Path = src/pub_sub/http_event_ids.h

#ifndef HTTP_EVENT_IDS_H
#define HTTP_EVENT_IDS_H

#include "framework/data_bus.h"

namespace snort
{
// These are common values between the HTTP inspector and the subscribers.
struct HttpEventIds
{ enum : unsigned {

    REQUEST_HEADER,
    RESPONSE_HEADER,
    REQUEST_BODY,

    num_ids
}; };

const PubKey http_pub_key { "http_inspect", HttpEventIds::num_ids };

}
#endif

http_events.h

Path = src/pub_sub/http_events.h

#ifndef HTTP_EVENTS_H
#define HTTP_EVENTS_H

#include "framework/data_bus.h"
#include "pub_sub/http_event_ids.h"

class HttpMsgHeader;

namespace snort
{

class SO_PUBLIC HttpEvent : public snort::DataEvent
{
public:
    HttpEvent(HttpMsgHeader* http_msg_header_, bool httpx, int64_t stream_id) :
        http_msg_header(http_msg_header_), is_httpx(httpx), httpx_stream_id(stream_id) { }

    const uint8_t* get_content_type(int32_t &length);
    const uint8_t* get_cookie(int32_t &length);
    const uint8_t* get_authority(int32_t &length);
    const uint8_t* get_uri_host(int32_t &length);
    const uint8_t* get_uri_query(int32_t &length);
    const uint8_t* get_location(int32_t &length);
    const uint8_t* get_referer(int32_t &length);
    const uint8_t* get_server(int32_t &length);
    const uint8_t* get_trueip_addr(int32_t& length);
    const uint8_t* get_uri(int32_t &length);
    const uint8_t* get_user_agent(int32_t &length);
    const uint8_t* get_via(int32_t &length);
    const uint8_t* get_x_working_with(int32_t &length);
    int32_t get_response_code();
    bool contains_webdav_method();
    bool get_is_httpx() const;
    int64_t get_httpx_stream_id() const;

private:
    HttpMsgHeader* const http_msg_header;
    bool is_httpx = false;
    int64_t httpx_stream_id = -1;

    const uint8_t* get_header(unsigned, uint64_t, int32_t&);

};
}
#endif

http_request_body_event.h

Path = src/pub_sub/http_request_body_event.h

#ifndef HTTP_REQUEST_BODY_EVENT_H
#define HTTP_REQUEST_BODY_EVENT_H

#include "framework/data_bus.h"

#include "http_event_ids.h"

class HttpMsgBody;
class HttpFlowData;

namespace snort
{
// This event is published each time new request body data is received by http_inspect for HTTP/2
// traffic, up to the publish depth. The full request body may be sent in several pieces
class SO_PUBLIC HttpRequestBodyEvent : public snort::DataEvent
{
public:
    HttpRequestBodyEvent(HttpMsgBody* msg_body, int32_t offset, bool last, HttpFlowData* flow_data)
        : http_msg_body(msg_body), msg_offset(offset), last_piece(last), http_flow_data(flow_data)
        { }

    const uint8_t* get_request_body_data(int32_t& length, int32_t& offset);
    const uint8_t* get_client_body(int32_t& length);
    bool is_last_request_body_piece();
    bool is_mime() const;
    int64_t get_httpx_stream_id() const;

private:
    HttpMsgBody* const http_msg_body;
    const int32_t msg_offset;
    const bool last_piece;
    HttpFlowData* const http_flow_data;
};
}
#endif

intrinsic_event_ids.h

Path = src/pub_sub/intrinsic_event_ids.h

#ifndef INTRINSIC_EVENT_IDS_H
#define INTRINSIC_EVENT_IDS_H

// Common core functionality data events

#include "framework/data_bus.h"

namespace snort
{

struct IntrinsicEventIds
{ enum : unsigned {

    DAQ_SOF_MSG,
    DAQ_EOF_MSG,
    DAQ_OTHER_MSG,

    ALT_PACKET,
    PKT_WITHOUT_FLOW,
    DETAINED_PACKET,
    FINALIZE_PACKET,
    RETRY_PACKET,

    THREAD_IDLE,
    THREAD_ROTATE,

    SSL_SEARCH_ABANDONED,
    OPPORTUNISTIC_TLS,

    FLOW_STATE_CHANGE,
    FLOW_SERVICE_CHANGE,
    SERVICE_INSPECTOR_CHANGE,

    FLOW_NO_SERVICE,
    FLOW_STATE_SETUP,
    FLOW_STATE_RELOADED,
    FLOW_ASSISTANT_GADGET,

    EXPECT_HANDLE_FLOWS,
    EXPECT_EARLY_SESSION,
    AUXILIARY_IP,
    FILE_VERDICT,

    num_ids
}; };

const PubKey intrinsic_pub_key { "snort", IntrinsicEventIds::num_ids };

const unsigned intrinsic_pub_id = 1;
}
#endif

netflow_event.h

Path = src/pub_sub/netflow_event.h

#ifndef NETFLOW_EVENT_H
#define NETFLOW_EVENT_H

#include "framework/data_bus.h"
#include "service_inspectors/netflow/netflow_record.h"

namespace snort
{

struct NetFlowEventIds { enum : unsigned { DATA, num_ids }; };

const PubKey netflow_pub_key { "netflow", NetFlowEventIds::num_ids };

class NetFlowEvent : public DataEvent
{
public:
    NetFlowEvent(const snort::Packet* p, const NetFlowSessionRecord* rec,
        bool cre_host, bool cre_serv, bool swp_initiator, uint32_t s_id)
        : pkt(p), record(rec), create_host(cre_host),
          create_service(cre_serv), swapped(swp_initiator), serviceID(s_id) { }

    const Packet* get_packet() const override
    { return pkt; }

    const NetFlowSessionRecord* get_record()
    { return record; }

    bool get_create_host()
    { return create_host; }

    bool get_create_service()
    { return create_service; }

    bool is_initiator_swapped()
    { return swapped; }

    uint32_t get_service_id()
    { return serviceID; }

private:
    const Packet* pkt;
    const NetFlowSessionRecord* record;
    bool create_host;
    bool create_service;
    bool swapped;
    uint32_t serviceID = 0;
};

}

#endif

opportunistic_tls_event.h

Path = src/pub_sub/opportunistic_tls_event.h

#ifndef OPPORTUNISTIC_TLS_EVENT_H
#define OPPORTUNISTIC_TLS_EVENT_H

#include <memory>
#include <string>

#include "pub_sub/intrinsic_event_ids.h"

// An opportunistic SSL/TLS session will start from next packet

namespace snort
{

class SO_PUBLIC OpportunisticTlsEvent : public snort::DataEvent
{
public:
    OpportunisticTlsEvent(const snort::Packet* p, const char* service) :
        pkt(p), next_service(service) { }

    const snort::Packet* get_packet() const override
    { return pkt; }

    const char* get_next_service()
    { return next_service; }

private:
    const snort::Packet* pkt;
    const char* next_service;
};

}

#endif

packet_events.h

Path = src/pub_sub/packet_events.h

#ifndef PACKET_EVENTS_H
#define PACKET_EVENTS_H

#include "pub_sub/intrinsic_event_ids.h"

// A retry packet is being processed

namespace snort
{

class RetryPacketEvent : public DataEvent
{
public:
    explicit RetryPacketEvent(const Packet* p) : pkt(p)
    { }

    const Packet* get_packet() const override
    { return pkt; }

    void set_still_pending()
    { still_pending = true; }

    bool is_still_pending() const
    { return still_pending; }

private:
    const Packet* pkt;
    bool still_pending = false;
};

}

#endif

reputation_events.h

Path = src/pub_sub/reputation_events.h

#ifndef REPUTATION_EVENTS_H
#define REPUTATION_EVENTS_H

#include "framework/data_bus.h"

namespace snort
{

struct ReputationEventIds { enum : unsigned { REP_MATCHED, num_ids }; };

const PubKey reputation_pub_key { "reputation", ReputationEventIds::num_ids };

enum ReputationVerdict
{
    REP_VERDICT_BLOCKED,
    REP_VERDICT_TRUSTED,
    REP_VERDICT_MONITORED
};

class ReputationVerdictEvent : public DataEvent
{
public:
    ReputationVerdictEvent(const Packet* packet, ReputationVerdict verdict, uint32_t list_id, bool source_matched)
        : packet(packet), verdict(verdict), list_id(list_id), source_matched(source_matched)
    { }

    const Packet* get_packet() const override
    { return packet; }

    ReputationVerdict get_verdict() const
    { return verdict; }

    uint32_t get_list_id() const
    { return list_id; }

    bool get_source_matched() const
    { return source_matched; }

private:
    const Packet* packet;
    ReputationVerdict verdict;
    uint32_t list_id;
    bool source_matched;
};

}

#endif

rna_events.h

Path = src/pub_sub/rna_events.h

#ifndef RNA_EVENTS_H
#define RNA_EVENTS_H

#include "framework/data_bus.h"
#include "service_inspectors/netflow/netflow_record.h"

namespace snort
{

struct RnaEventIds { enum : unsigned { NEW_NETFLOW_CONN, num_ids }; };

const PubKey rna_pub_key { "rna", RnaEventIds::num_ids };

class RNAEvent : public DataEvent
{
public:
    RNAEvent(const snort::Packet* p, const NetFlowSessionRecord* rec, const uint32_t service)
        : pkt(p), record(rec), service_id(service) { }

    const Packet* get_packet() const override
    { return pkt; }

    const NetFlowSessionRecord* get_record()
    { return record; }

    uint32_t get_service_id()
    { return service_id; }

private:
    const Packet* pkt;
    const NetFlowSessionRecord* record;
    const uint32_t service_id;
};

}

#endif

sip_events.h

Path = src/pub_sub/sip_events.h

#ifndef SIP_EVENTS_H
#define SIP_EVENTS_H

// This event conveys data published by the SIP service inspector to be consumed
// by data bus subscribers

#include <list>

#include "framework/data_bus.h"

struct SipEventIds { enum : unsigned { DIALOG, num_ids }; };

const snort::PubKey sip_pub_key { "sip", SipEventIds::num_ids };

namespace snort
{
struct Packet;
struct SfIp;
}
struct SIPMsg;
struct SIP_DialogData;
struct SIP_MediaSession;
struct SIP_MediaData;

class SipEventMediaData
{
public:
    SipEventMediaData(SIP_MediaData* data)
    { this->data = data; }

    const snort::SfIp* get_address() const;
    uint16_t get_port() const;

private:
    SIP_MediaData* data;
};

class SipEventMediaSession
{
public:
    SipEventMediaSession(SIP_MediaSession* session)
    { this->session = session; }

    ~SipEventMediaSession();

    uint32_t get_id() const;

    void begin_media_data();
    SipEventMediaData* next_media_data();

private:
    SIP_MediaSession* session;

    std::list<SipEventMediaData*> data;
    SIP_MediaData* current_media_data = nullptr;
};

class SipEvent : public snort::DataEvent
{
public:
    SipEvent(const snort::Packet*, const SIPMsg&, const SIP_DialogData*);
    ~SipEvent() override;

    const snort::Packet* get_packet() const override
    { return p; }

    const char* get_from() const
    { return from; }

    size_t get_from_len() const
    { return from_len; }

    const char* get_user_name() const
    { return user_name; }

    size_t get_user_name_len() const
    { return user_name_len; }

    const char* get_user_agent() const
    { return user_agent; }

    size_t get_user_agent_len() const
    { return user_agent_len; }

    const char* get_server() const
    { return server; }

    size_t get_server_len() const
    { return server_len; }

    bool is_invite() const;
    bool is_media_updated() const;
    bool has_dialog() const;
    bool is_dialog_established() const;

    void begin_media_sessions();
    SipEventMediaSession* next_media_session();

private:
    const snort::Packet* p;

    const SIPMsg& msg;
    const SIP_DialogData* dialog;

    const char* from = nullptr;
    size_t from_len;
    const char* user_name = nullptr;
    size_t user_name_len;
    const char* user_agent = nullptr;
    size_t user_agent_len;
    const char* server;
    size_t server_len;

    std::list<SipEventMediaSession*> sessions;
    SIP_MediaSession* current_media_session = nullptr;
};

#endif

smb_events.h

Path = src/pub_sub/smb_events.h

#ifndef SMB_EVENTS_H
#define SMB_EVENTS_H

#include "pub_sub/appid_event_ids.h"

namespace snort
{

class FpSMBDataEvent : public snort::DataEvent
{
public:
    FpSMBDataEvent(const snort::Packet* p, unsigned major, unsigned minor,
        uint32_t flags) : pkt(p), major_version(major), minor_version(minor), flags(flags) { }

    const snort::Packet* get_packet() const override
    { return pkt; }

    unsigned get_fp_smb_major() const
    { return major_version; }

    unsigned get_fp_smb_minor() const
    { return minor_version; }

    uint32_t get_fp_smb_flags() const
    { return flags; }

private:
    const snort::Packet* pkt;
    unsigned major_version;
    unsigned minor_version;
    uint32_t flags;
};

}

#endif

ssh_events.h

Path = src/pub_sub/ssh_events.h

#ifndef SSH_EVENTS_H
#define SSH_EVENTS_H

// This event allows the SSH service inspector to publish extracted metadata
// for use by data bus subscribers

#include "framework/data_bus.h"
#include "service_inspectors/ssh/ssh.h"

struct  SshEventIds { enum : unsigned { STATE_CHANGE, num_ids }; };

const snort::PubKey ssh_pub_key { "ssh", SshEventIds::num_ids };

enum SshEventType
{
    SSH_VERSION_STRING,
    SSH_VALIDATION
};

enum SshValidationResult
{
    SSH_NOT_FINISHED,
    SSH_VALID_KEXINIT,
    SSH_INVALID_VERSION,
    SSH_INVALID_KEXINIT
};

class SshEvent : public snort::DataEvent
{
public:
    SshEvent(const SshEventType event_type, const SshValidationResult result,
        const std::string& version_str, const uint8_t direction, const snort::Packet* packet) :
        event_type(event_type), result(result), version_str(version_str), direction(direction), packet(packet)
        { }

    SshEventType get_event_type() const
    { return event_type; }

    SshValidationResult get_validation_result() const
    { return result; }

    const std::string& get_version_str() const
    { return version_str; }

    uint8_t get_direction() const
    { return direction; }

    const snort::Packet* get_packet() const override
    { return packet; }

private:
    const SshEventType event_type;
    const SshValidationResult result;
    const std::string version_str;
    const uint8_t direction;
    const snort::Packet* packet;
};

#endif

ssl_events.h

Path = src/pub_sub/ssl_events.h

#ifndef SSL_EVENTS_H
#define SSL_EVENTS_H

// This event allows the SSL service inspector to publish extracted SSL handshake client hello data
// for use by data bus subscribers

#include "framework/data_bus.h"

struct SslEventIds
{
    enum : unsigned
    {
        CHELLO_SERVER_NAME,
        SERVER_COMMON_NAME,

        num_ids
    };
};

const snort::PubKey ssl_pub_key { "ssl", SslEventIds::num_ids };

class SslClientHelloEvent : public snort::DataEvent
{
public:
    SslClientHelloEvent(const std::string& ch_server_name, const snort::Packet* packet) :
        ch_server_name(ch_server_name), packet(packet)
        { }

    const snort::Packet* get_packet() const override
    { return packet; }

    const std::string& get_host_name() const
    { return ch_server_name; }

private:
    const std::string ch_server_name;
    const snort::Packet* packet;
};

class SslServerCommonNameEvent : public snort::DataEvent
{
public:
    SslServerCommonNameEvent(const std::string& server_common_name, const snort::Packet* packet) :
        server_common_name(server_common_name), packet(packet)
        { }

    const snort::Packet* get_packet() const override
    { return packet; }

    const std::string& get_common_name() const
    { return server_common_name; }

private:
    const std::string server_common_name;
    const snort::Packet* packet;
};

#endif

stash_events.h

Path = src/pub_sub/stash_events.h

#ifndef STASH_EVENTS_H
#define STASH_EVENTS_H

#include "framework/data_bus.h"

using snort::StashItem;

class StashEvent : public snort::DataEvent
{
public:
    StashEvent(const StashItem* it = nullptr) : item(it) {}
    const StashItem* get_item() const { return item; }
private:
    const StashItem* item;
};

#endif

stream_event_ids.h

Path = src/pub_sub/stream_event_ids.h

#ifndef STREAM_EVENT_IDS_H
#define STREAM_EVENT_IDS_H

#include "framework/data_bus.h"

struct StreamEventIds
{ enum : unsigned {

    IP_NEW_FLOW,
    IP_BIDIRECTIONAL,

    ICMP_NEW_FLOW,
    ICMP_BIDIRECTIONAL,

    TCP_SYN,
    TCP_SYN_ACK,
    TCP_MIDSTREAM,
    TCP_ESTABLISHED,

    UDP_NEW_FLOW,
    UDP_BIDIRECTIONAL,

    HA_NEW_FLOW,

    num_ids
}; };

const snort::PubKey stream_pub_key { "stream", StreamEventIds::num_ids };

#endif

pub_sub/test/

search_engines/

Builtin fast pattern matching algorithms are implemented here.

  • MPSE = multi-pattern search engine

  • DFA = deterministic finite automaton

  • NFA = non-DFA

  • HFA = hybrid FA

This code has has evolved through 4 major versions:

  1. acsmx.cc: ac_std

  2. acsmx2.cc: ac_full, ac_sparse, ac_banded, ac_sparse_bands

  3. bnfa_search.cc: ac_bnfa

  4. hyperscan.cc: support of regex fast patterns

Check the comments at the start of the above files for details on the implementation.

NFAs require much less memory than DFAs, but DFAs are faster. Version 1 is a DFA. Version 2 is either NFA or DFA and supports multiple storage formats. Version 3 is NFA. Version 2 also had a TRIE based implementation which was moved to extras.

The version 2 storage formats try to reduce memory for transition storage by various schemes:

  • full - an array of 256 transitions for each state indexed by event (byte)

  • sparse - a list of valid transitions (which must be searched)

  • banded - like full except that the leading and trailing invalid transitions are not stored

  • sparse bands - a list of bands

Version 4 entails a number of refactoring changes to support regex fast patterns using hyperscan, an HFA. A key change is to return the offset of the end of match the way hyperscan does to support relative matches to fast pattern only contents and regexes.

Version 4 also includes refactoring the match queues. The *_q flavors were deleted as this aspect is orthogonal to the algorithm. Instead a match queue is always used to defer rule tree evaluation until after the full buffer is searched in order to keep the cache warm. This is a development decision based on overall performance.

Note that hyperscan essentially results in single branch detection option trees because from a client view each match state is unique - one per rule. This is a potential negative impact on performance but does not yet seem significant. Furthermore, regex based fast patterns may obviate the need for the tree. However, the tree remains as it is essential for other algorithms.

SearchTool makes it easy to use ac_bnfa. This is used by http, pop, imap, and smtp.

See "Optimizing Pattern Matching for Intrusion Detection" by Marc Norton. Available on https://snort.org/documents/.

Reference - Efficient String matching: An Aid to Bibliographic Search Alfred V Aho and Margaret J Corasick, Bell Laboratories Copyright © 1975 Association for Computing Machinery,Inc

acsmx2.h

Path = src/search_engines/acsmx2.h

#ifndef ACSMX2_H
#define ACSMX2_H

// Version 2.0

#include <cstdint>

#include "search_common.h"

namespace snort
{
struct SnortConfig;
}

#define MAX_ALPHABET_SIZE 256

/*
   FAIL STATE for 1,2,or 4 bytes for state transitions
   Uncomment this define to use 32 bit state values
   #define AC32
*/

#define AC32

#ifdef AC32

typedef unsigned int acstate_t;
#define ACSM_FAIL_STATE2  0xffffffff

#else

typedef unsigned short acstate_t;
#define ACSM_FAIL_STATE2 0xffff

#endif

struct ACSM_PATTERN2
{
    ACSM_PATTERN2* next;

    uint8_t* patrn;
    uint8_t* casepatrn;

    void* udata;
    void* rule_option_tree;
    void* neg_list;

    int n;
    int nocase;
    int negative;
};

/*
*    transition nodes  - either 8 or 12 bytes
*/
struct trans_node_t
{
    /* The character that got us here - sized to keep structure aligned on 4 bytes
     * to better the caching opportunities. A value that crosses the cache line
     * forces an expensive reconstruction, typing this as acstate_t stops that.
     */
    acstate_t key;
    acstate_t next_state;
    trans_node_t* next; /* next transition for this state */
};

/*
*   Aho-Corasick State Machine Struct - one per group of patterns
*/
struct ACSM_STRUCT2
{
    ACSM_PATTERN2* acsmPatterns;
    acstate_t* acsmFailState;
    ACSM_PATTERN2** acsmMatchList;

    /* list of transitions in each state, this is used to build the nfa & dfa
       after construction we convert to sparse or full format matrix and free
       the transition lists */
    trans_node_t** acsmTransTable;
    acstate_t** acsmNextState;
    const MpseAgent* agent;

    int acsmMaxStates;
    int acsmNumStates;

    int acsmNumTrans;
    int acsmAlphabetSize;
    int numPatterns;

    int sizeofstate;
};

/*
*   Prototypes
*/
void acsmx2_init_xlatcase();

ACSM_STRUCT2* acsmNew2(const MpseAgent*);

int acsmAddPattern2(
    ACSM_STRUCT2* p, const uint8_t* pat, unsigned n,
    bool nocase, bool negative, void* id);

int acsmCompile2(snort::SnortConfig*, ACSM_STRUCT2*);

int acsm_search_nfa(
    ACSM_STRUCT2*, const uint8_t* T, int n, MpseMatch, void* context, int* current_state);

int acsm_search_dfa_full(
    ACSM_STRUCT2*, const uint8_t* T, int n, MpseMatch, void* context, int* current_state);

int acsm_search_dfa_full_all(
    ACSM_STRUCT2*, const uint8_t* Tx, int n, MpseMatch, void* context, int* current_state);

void acsmFree2(ACSM_STRUCT2*);
int acsmPatternCount2(ACSM_STRUCT2*);

void acsmPrintInfo2(ACSM_STRUCT2* p);

int acsmPrintDetailInfo2(ACSM_STRUCT2*);
int acsmPrintSummaryInfo2();
void acsm_init_summary();

#endif

bnfa_search.h

Path = src/search_engines/bnfa_search.h

#ifndef BNFA_SEARCH_H
#define BNFA_SEARCH_H

/*
** Basic NFA based multi-pattern search using Aho_corasick construction,
** and compacted sparse storage.
**
** Version 3.0
** date:   12/21/05
*/

#include <cstdint>

#include "search_common.h"

namespace snort
{
struct SnortConfig;
}

/* debugging - allow printing the trie and nfa in list format
   #define ALLOW_LIST_PRINT */

/* debugging - enable full format
   #define ALLOW_NFA_FULL */

/*
*   DEFINES and Typedef's
*/
//#define SPARSE_FULL_STATE_0
#define BNFA_MAX_ALPHABET_SIZE          256
#define BNFA_FAIL_STATE                 0xffffffff
#define BNFA_SPARSE_LINEAR_SEARCH_LIMIT 6

#define BNFA_SPARSE_MAX_STATE           0x00ffffff
#define BNFA_SPARSE_COUNT_SHIFT         24
#define BNFA_SPARSE_VALUE_SHIFT         24

#define BNFA_SPARSE_MATCH_BIT           0x80000000
#define BNFA_SPARSE_FULL_BIT            0x40000000
#define BNFA_SPARSE_COUNT_BITS          0x3f000000
#define BNFA_SPARSE_MAX_ROW_TRANSITIONS 0x3f

typedef  unsigned int bnfa_state_t;

/*
*   Internal Pattern Representation
*/
struct bnfa_pattern_t
{
    bnfa_pattern_t* next;
    uint8_t* casepatrn;   /* case specific */
    void* userdata;       /* ptr to users pattern data/info  */

    unsigned n;           /* pattern len */
    int nocase;           /* nocase flag */
    int negative;         /* pattern is negated */
};

/*
*  List format transition node
*/
struct bnfa_trans_node_t
{
    bnfa_trans_node_t* next;
    bnfa_state_t key;
    bnfa_state_t next_state;
};

/*
*  List format patterns
*/
struct bnfa_match_node_t
{
    void* data;
    void* rule_option_tree;
    void* neg_list;
    bnfa_match_node_t* next;
};

/*
*  Final storage type for the state transitions
*/
enum
{
    BNFA_FULL,
    BNFA_SPARSE
};

enum
{
    BNFA_PER_PAT_CASE,
    BNFA_CASE,
    BNFA_NOCASE
};

/*
*   Aho-Corasick State Machine Struct
*/
struct bnfa_struct_t
{
    int bnfaMethod;
    int bnfaCaseMode;
    int bnfaFormat;
    int bnfaAlphabetSize;
    int bnfaOpt;

    unsigned bnfaPatternCnt;

    int bnfaMaxStates;
    int bnfaNumStates;
    int bnfaNumTrans;
    int bnfaMatchStates;

    bnfa_pattern_t* bnfaPatterns;
    bnfa_trans_node_t** bnfaTransTable;
    bnfa_state_t** bnfaNextState;
    bnfa_match_node_t** bnfaMatchList;
    bnfa_state_t* bnfaFailState;
    bnfa_state_t* bnfaTransList;

    const MpseAgent* agent;

    int bnfaForceFullZeroState;

    int bnfa_memory;
    int pat_memory;
    int list_memory;
    int queue_memory;
    int nextstate_memory;
    int failstate_memory;
    int matchlist_memory;
};

/*
*   Prototypes
*/
void bnfa_init_xlatcase();

bnfa_struct_t* bnfaNew(const MpseAgent*);

void bnfaFree(bnfa_struct_t* pstruct);

int bnfaAddPattern(
    bnfa_struct_t* pstruct, const uint8_t* pat, unsigned patlen,
    bool nocase, bool negative, void* userdata);

int bnfaCompile(snort::SnortConfig*, bnfa_struct_t*);

unsigned _bnfa_search_csparse_nfa(
    bnfa_struct_t * pstruct, const uint8_t* t, int tlen, MpseMatch,
    void* context, unsigned sindex, int* current_state);

int bnfaPatternCount(bnfa_struct_t* p);

void bnfaPrint(bnfa_struct_t* pstruct);   /* prints the nfa states-verbose!! */
void bnfaPrintInfo(bnfa_struct_t* pstruct);    /* print info on this search engine */

/*
 * Summary - this tracks search engine information across multiple instances of
 * search engines.  It helps in snort where we have many search engines, each using
 * rule grouping, to track total patterns, states, memory, etc...
 *
 */
void bnfaPrintInfoEx(bnfa_struct_t* p, const char* text);
void bnfaAccumInfo(bnfa_struct_t* pstruct);  // add info to summary over multiple search engines
void bnfaPrintSummary(); /* print current summary */
void bnfaInitSummary();  /* reset accumulator for global summary over multiple engines */
void bnfa_print_qinfo();
#endif

pat_stats.h

Path = src/search_engines/pat_stats.h

#ifndef PAT_STATS_H
#define PAT_STATS_H

#include "framework/counts.h"
#include "main/snort_types.h"
#include "main/thread.h"

// pattern matcher queue statistics

struct PatMatQStat
{
    PegCount max_inq;
    PegCount tot_inq_flush;
    PegCount tot_inq_inserts;
    PegCount tot_inq_overruns;
    PegCount tot_inq_uinserts;
    PegCount non_qualified_events;
    PegCount qualified_events;
    PegCount matched_bytes;
};

namespace snort
{
SO_PUBLIC extern THREAD_LOCAL PatMatQStat pmqs;
}
#endif

search_common.h

Path = src/search_engines/search_common.h

#ifndef SEARCH_COMMON_H
#define SEARCH_COMMON_H

namespace snort
{
    struct SnortConfig;
}

struct MpseAgent
{
    int (* build_tree)(snort::SnortConfig*, void* id, void** tree);
    int (* negate_list)(void* id, void** list);

    void (* user_free)(void*);
    void (* tree_free)(void**);
    void (* list_free)(void**);
};

// interface to Mpse and SearchTool

typedef int (* MpseMatch)(void* user, void* tree, int index, void* context, void* list);

#endif

search_engines.h

Path = src/search_engines/search_engines.h

#ifndef SEARCH_ENGINES_H
#define SEARCH_ENGINES_H

namespace snort
{
struct BaseApi;
}

void load_search_engines();

#endif

search_tool.h

Path = src/search_engines/search_tool.h

#ifndef SEARCH_TOOL_H
#define SEARCH_TOOL_H

#include "framework/mpse.h"

// FIXIT-L until APIs are updated, SearchTool depends on SnortConfig so it must
// not be instantiated until configure time (Inspector::configure) to ensure
// SnortConfig is fully initialized and the configured algorithm is used.

// Use hyperscan if configured with search_engine.search_method else use ac_full.
// Offload is not supported for search tool.

// We force non-hyperscan to be ac_full since the other algorithms like ac_bnfa
// don't implement search_all, which returns all patterns for a given match state.

namespace snort
{
class SO_PUBLIC SearchTool
{
public:
    SearchTool(bool multi_match = true, const char* = nullptr);
    ~SearchTool();

    const char* get_method() const;

    void add(const char* pattern, unsigned len, int s_id, bool no_case = true, bool literal = true);
    void add(const char* pattern, unsigned len, void* s_context, bool no_case = true, bool literal = true);

    void add(const uint8_t* pattern, unsigned len, int s_id, bool no_case = true, bool literal = true);
    void add(const uint8_t* pattern, unsigned len, void* s_context, bool no_case = true, bool literal = true);

    void prep();
    void reload();

    // set state to zero on first call
    int find(const char* s, unsigned s_len, MpseMatch, int& state,
        bool confine = false, void* user_data = nullptr);

    int find(const char* s, unsigned s_len, MpseMatch,
        bool confine = false, void* user_data = nullptr);

    int find_all(const char* s, unsigned s_len, MpseMatch,
        bool confine = false, void* user_data = nullptr, const SnortConfig* = nullptr);

private:
    class MpseGroup* mpsegrp;
    unsigned max_len;
    bool multi_match;
};
} // namespace snort
#endif

search_engines/test/

mpse_test_stubs.h

Path = src/search_engines/test/mpse_test_stubs.h

#ifndef TEST_STUBS_H
#define TEST_STUBS_H

#include <cassert>

#include "detection/fp_config.h"
#include "framework/base_api.h"
#include "framework/mpse.h"
#include "framework/mpse_batch.h"
#include "main/snort_config.h"
#include "managers/mpse_manager.h"
#include "utils/stats.h"

#include "search_engines/pat_stats.h"

extern std::vector<void *> s_state;
extern snort::ScratchAllocator* scratcher;

namespace snort
{
extern SnortConfig s_conf;

extern THREAD_LOCAL SnortConfig* snort_conf;
extern THREAD_LOCAL PatMatQStat pmqs;

extern unsigned parse_errors;
} // namespace snort

extern snort::Mpse* mpse;
const snort::MpseApi* get_test_api();

extern void* s_user;
extern void* s_tree;
extern void* s_list;

extern MpseAgent s_agent;

extern const snort::BaseApi* se_ac_bnfa;
extern const snort::BaseApi* se_ac_full;
extern const snort::BaseApi* se_hyperscan;

struct ExpectedMatch
{
    int id;
    int offset;
};

extern const ExpectedMatch* s_expect;
extern int s_found;

int check_mpse_match(
    void* pid, void* /*tree*/, int index, void* /*context*/, void* /*neg_list*/);

#endif

service_inspectors/

Service inspectors are the modules that process individual upper-level protocols such as FTP, Telnet, SMTP, and HTTP.

Service inspectors are often subdivided into two classes. The stream splitter accepts a stream of protocol data from the transport layer and segments it at the PDU boundaries. The inspector processes the resulting PDUs.

The wizard is a special service inspector that examines the beginning of a data stream and decides what application protocol is present.

service_inspectors.h

Path = src/service_inspectors/service_inspectors.h

#ifndef SERVICE_INSPECTORS_H
#define SERVICE_INSPECTORS_H

void load_service_inspectors();

#endif

service_inspectors/back_orifice/

Detects Back Orifice traffic by brute forcing the weak encryption of the program’s network protocol and detects the magic cookie that it’s servers and clients require to communicate with each other.

service_inspectors/cip/

CIP (Common Industrial Protocol) is a protocol used in SCADA networks. The standard is managed by ODVA (Open DeviceNet Vendor Association).

The CIP inspector decodes the CIP protocol and provides rule options to access certain protocol fields. This allows a user to write rules for CIP packets without decoding the protocol with a series of ”content” and ”byte test” options.

The inspector only evaluates PAF-flushed PDUs. If the rule options don’t check for this, they’ll fire on stale session data when the original packet goes through before flushing.

cip_definitions.h

Path = src/service_inspectors/cip/cip_definitions.h

#ifndef CIP_DEFINITIONS_H
#define CIP_DEFINITIONS_H

#include <sys/time.h>

namespace snort
{
struct Packet;
}

#define MSEC_PER_SEC (1000)
#define USEC_PER_SEC (1000000)

// CIP inspector configuration
struct CipProtoConf
{
    // Unconnected timeout, seconds.
    uint32_t unconnected_timeout;

    // Maximum number of unconnected requests per TCP connection.
    uint32_t max_unconnected_messages;

    // Maximum number of CIP connections per TCP connection.
    uint32_t max_cip_connections;

    // Custom embedded packet parameters.
    bool embedded_cip_enabled;
    uint32_t embedded_cip_class_id;
    uint8_t embedded_cip_service_id;
};

/// CIP Request/Response Management
enum CipRequestType
{
    CipRequestTypeOther = 0,
    CipRequestTypeForwardOpen,
    CipRequestTypeForwardClose,
    CipRequestTypeUnconnectedSend,
    CipRequestTypeMultipleServiceRequest,

    // Special case to represent when no request is found for a given response.
    CipRequestTypeNoMatchFound
};

struct CipStatus
{
    uint8_t general_status;
    size_t extended_status_size;
};

enum CipPacketDirection
{
    CIP_FROM_CLIENT,
    CIP_FROM_SERVER,
    CIP_FROM_UNKNOWN
};

/// EtherNet/IP encapsulation layer definitions.

// EtherNet/IP encapsulation header
struct EnipHeader
{
    uint16_t command;
    uint16_t length;
    uint32_t session_handle;
    uint32_t status;
    uint64_t sender_context;
    uint32_t options;
};

// This is an EtherNet/IP encapsulation layer common packet format item.
struct EnipCpfItem
{
    uint16_t type;
    uint16_t length;

    // Used if length > 0. Data starts after the Length field.
    const uint8_t* data;
};

// Largest number of allowed CPF items for standard EtherNet/IP commands.
#define MAX_NUM_CPF_ITEMS 4

// This is an EtherNet/IP encapsulation layer common packet format.
struct EnipCpf
{
    uint16_t item_count;

    // All CPF items in the list are valid up to and including array index item_count.
    EnipCpfItem item_list[MAX_NUM_CPF_ITEMS];
};

/// CIP layer definitions.
enum CipMessageType
{
    // Unknown CIP data type
    CipMessageTypeUnknown,

    // CIP Explicit Data
    CipMessageTypeExplicit,

    // CIP Implicit Data
    CipMessageTypeImplicit
};

enum CipSegmentType
{
    CipSegment_Type_PORT_LINK_ADDRESS,
    CipSegment_Type_PORT_LINK_ADDRESS_EXTENDED,

    CipSegment_Type_LOGICAL_CLASS,
    CipSegment_Type_LOGICAL_INSTANCE,
    CipSegment_Type_LOGICAL_MEMBER,
    CipSegment_Type_LOGICAL_CONN_POINT,
    CipSegment_Type_LOGICAL_ATTRIBUTE,
    CipSegment_Type_LOGICAL_ELECTRONIC_KEY,
    CipSegment_Type_LOGICAL_EXTENDED,
    CipSegment_Type_LOGICAL_SERVICE_ID,

    CipSegment_Type_NETWORK,
    CipSegment_Type_NETWORK_SAFETY,

    CipSegment_Type_SYMBOLIC,

    CipSegment_Type_DATA_SIMPLE,
    CipSegment_Type_DATA_EXT_SYMBOL,

    CipSegment_Type_UNKNOWN
};

#define CIP_STATUS_SUCCESS 0
#define ENIP_STATUS_SUCCESS 0

// CIP Classes
#define MESSAGE_ROUTER_CLASS_ID 0x02
#define CONNECTION_MANAGER_CLASS_ID 0x06

// CIP Services
#define SERVICE_SET_ATTRIBUTE_SINGLE 0x10
#define SERVICE_MULTIPLE_SERVICE_PACKET 0x0A

// CIP Connection Manager Services
#define CONNECTION_MANAGER_UNCONNECTED_SEND 0x52
#define CONNECTION_MANAGER_FORWARD_OPEN 0x54
#define CONNECTION_MANAGER_LARGE_FORWARD_OPEN 0x5B
#define CONNECTION_MANAGER_FORWARD_CLOSE 0x4E

#define CIP_WORD_TO_BYTES 2

struct CipSegment
{
    CipSegmentType type;

    // Total size of this segment.
    size_t size;

    // When type = CipSegment_Type_PORT_LINK_ADDRESS
    // When type = CipSegment_Type_PORT_LINK_ADDRESS_EXTENDED
    uint16_t port_id;

    // When type = CipSegment_Type_PORT_LINK_ADDRESS
    uint8_t link_address;

    // When type = CipSegment_Type_LOGICAL_CLASS
    // When type = CipSegment_Type_LOGICAL_INSTANCE
    // When type = CipSegment_Type_LOGICAL_MEMBER
    // When type = CipSegment_Type_LOGICAL_CONN_POINT
    // When type = CipSegment_Type_LOGICAL_ATTRIBUTE
    uint32_t logical_value;

    // When type = CipSegment_Type_PORT_LINK_ADDRESS_EXTENDED, this is the link address.
    // When type = CipSegment_Type_DATA_EXT_SYMBOL, this is the symbol string.
    // When type = CipSegment_Type_DATA_SIMPLE, this is the start of the data words.
    // When type = CipSegment_Type_SYMBOLIC, this is the symbol string.
    const uint8_t* data;
    size_t data_size;
};

struct CipPath
{
    // Size of the entire path.
    size_t full_path_size;

    // True if path has been decoded successfully.
    bool decoded;

    // Main segment type for this path, which drives message target.
    CipSegmentType primary_segment_type;

    bool has_class_id;
    uint32_t class_id;

    bool has_instance_id;
    uint32_t instance_id;

    bool has_attribute_id;
    uint32_t attribute_id;

    bool has_unknown_segment;
};

// Matching pair of CIP Connection IDs.
struct ConnectionIdPair
{
    uint32_t ot_connection_id;
    uint32_t to_connection_id;
};

// RPI and Network Connection Parameters from a Forward Open Request.
struct CipConnectionParameters
{
    uint32_t rpi;
    uint32_t network_connection_parameters;
    bool is_null_connection;
};

// Unique Connection Signature. This is unique to each CIP connection. This
//  tuple is unique on a given EtherNet/IP session.
struct CipConnectionSignature
{
    uint16_t connection_serial_number;
    uint16_t vendor_id;
    uint32_t originator_serial_number;
};

#define TRANSPORT_CLASS_MASK 0x0F
struct CipForwardOpenRequest
{
    // Unconnected request timeout, milliseconds.
    uint32_t timeout_ms;

    // Connection timeouts, microseconds.
    uint64_t ot_connection_timeout_us;
    uint64_t to_connection_timeout_us;

    CipConnectionSignature connection_signature;

    CipConnectionParameters ot_parameters;
    CipConnectionParameters to_parameters;
    uint8_t transport_class;

    CipPath connection_path;

    bool is_null_forward_open;

    // Timestamp for message request.
    struct timeval timestamp;
};

struct CipForwardOpenResponse
{
    // True if this was a successful Forward Open Response.
    bool success;

    // Properties for Success or Fail.
    CipConnectionSignature connection_signature;

    // Properties for a Forward Open Response Success.
    ConnectionIdPair connection_pair;

    size_t application_reply_size;

    // Timestamp for message response.
    struct timeval timestamp;
};

struct CipForwardCloseRequest
{
    // Unconnected request timeout, milliseconds.
    uint32_t timeout_ms;

    CipConnectionSignature connection_signature;

    CipPath connection_path;
};

// Used to set error flags in enip_invalid_nonfatal.
#define ENIP_INVALID_COMMAND (1 << 0)
#define ENIP_INVALID_DUPLICATE_SESSION (1 << 1)
#define ENIP_INVALID_SESSION_HANDLE (1 << 2)
#define ENIP_INVALID_INTERFACE_HANDLE (1 << 3)
#define ENIP_INVALID_CONNECTION_ID (1 << 4)
#define ENIP_INVALID_PAYLOAD_SIZE (1 << 5)
#define ENIP_INVALID_ENIP_COMMAND_CPF_MISMATCH (1 << 6)
#define ENIP_INVALID_RESERVED_FUTURE_CPF_TYPE (1 << 7)
#define ENIP_INVALID_STATUS (1 << 8)
#define ENIP_INVALID_ENIP_TCP_ONLY (1 << 9)

struct EnipSessionData
{
    // True if the ENIP Header was parsed and is valid.
    bool enip_decoded;

    // Full ENIP header.
    EnipHeader enip_header;

    // Error states for non-fatal ENIP errors. Error conditions that could trigger this:
    //  - Command code was not valid according to CIP Volume 2, Section 2-3.2.
    //  - RegisterSession attempted when a session was already active.
    //  - Session Handle did not match an active session.
    //  - Interface Handle != 0
    //  - Connection ID does not match an active connection.
    //  - Larger amount of ENIP data than specific in ENIP length.
    //  - Invalid CPF data item for a particular ENIP command.
    //  - CPF Item Type ID was found in the Reserved for future expansion range.
    //  - ENIP Status != 0, for a Request.
    //  - Attempting to send an ENIP command that is TCP only on a UDP connection.
    uint32_t enip_invalid_nonfatal;

    // True if the Common Packet Format was parsed and is valid.
    bool cpf_decoded;

    // True if the required CPF items are present for this EtherNet/IP command.
    bool required_cpf_items_present;

    // Common Packet Format data.
    EnipCpf enip_cpf;

    // Connection Class from original connection request, for connected messages.
    uint32_t connection_class_id;
};

// Used to set error flags in cip_req_invalid_nonfatal.
#define CIP_REQ_INVALID_CONNECTION_ADD_FAILED (1 << 0)
#define CIP_REQ_INVALID_UNKNOWN_SEGMENT (1 << 1)
#define CIP_REQ_INVALID_TIMEOUT_MULTIPLIER (1 << 2)
struct CipRequest
{
    // CIP Service code.
    uint8_t service;

    CipPath request_path;

    CipRequestType request_type;

    // This is only valid for Unconnected Send messages.
    CipPath route_path;

    // CIP application payload data. This starts after the Request Path.
    const uint8_t* cip_data;
    size_t cip_data_size;

    // Unconnected request timeout, milliseconds.
    bool has_timeout;
    uint32_t timeout_ms;

    // True if this request was a Forward Open Request.
    bool is_forward_open_request;
    bool is_cip_safety;
    // Class ID in the Forward Open Request Connection Path.
    // Used only when is_forward_open_request is true.
    uint32_t connection_path_class_id;

    // Error states for non-fatal CIP errors. Error conditions that could trigger this:
    //  - Forward Open Request received but couldn't add the connection to the list because a
    //      connection already existed with that signature.
    //  - Unknown segment type in request path.
    //  - Forward Open Request contained invalid Connection Timeout Multiplier.
    uint32_t cip_req_invalid_nonfatal;
};

struct CipResponse
{
    // CIP Service code. This does not include the first bit set (0x80).
    uint8_t service;

    CipStatus status;
};

struct CipMessage
{
    // True if this is a CIP request (vs response).
    bool is_cip_request;

    // Used if is_cip_request is true.
    CipRequest request;

    // Used if is_cip_request is false.
    CipResponse response;
};

struct CipCurrentData
{
    CipPacketDirection direction;

    // ENIP layer data.
    EnipSessionData enip_data;

    // CIP layer data.
    CipMessageType cip_message_type;

    // Used if cip_message_type is CipMessageTypeExplicit
    CipMessage cip_msg;

    // True if the packet was not able to be fully parsed.
    bool invalid_fatal;
};

struct EnipSession
{
    // ENIP session handle.
    uint32_t session_handle;

    // True if this session is active.
    bool active;
};

// This represents an Unconnected message request.
//  Sender Context -> Request Type
struct CipUnconnectedMessage
{
    uint64_t sender_context;

    CipRequestType request_type;

    // Unconnected request timeout, milliseconds.
    uint32_t timeout_ms;

    // Timestamp for message request.
    struct timeval timestamp;

    // True if this entry is in use.
    bool slot_active;
};

struct CipUnconnectedMessageList
{
    CipUnconnectedMessage* list;
    uint32_t list_size;
    uint32_t count;

    // True if an active request was forced to be pruned.
    bool request_pruned;
};

// This represents a CIP connection.
// This is used to:
//  a) Get the connection IDs, during a Forward Close.
//  b) Get Class ID from the Connection Path.
struct CipConnection
{
    CipConnectionSignature signature;

    ConnectionIdPair connection_id_pair;

    // Class ID from the Connection Path
    uint32_t class_id;

    // True if the connection is fully established.
    bool established;

    // Connection timeouts, seconds.
    uint32_t ot_connection_timeout_sec;
    uint32_t to_connection_timeout_sec;

    // Timestamp for last time connection was active.
    struct timeval ot_timestamp;
    struct timeval to_timestamp;

    // True if this entry is in use.
    bool slot_active;
};

struct CipConnectionList
{
    CipConnection* list;
    uint32_t list_size;
    uint32_t count;

    // True if an active connection was forced to be pruned.
    bool connection_pruned;
};

struct CipGlobalSessionData
{
    // ENIP Session for this TCP connection.
    EnipSession enip_session;

    // List of CIP connections.
    CipConnectionList connection_list;

    // List of outstanding unconnected messages (SendRRData).
    CipUnconnectedMessageList unconnected_list;

    // Current configuration for use in lower-level parsing functions.
    const CipProtoConf* config;

    snort::Packet* snort_packet;
};

// This is the overall structure used by Snort to store current and global data
//  for a particular stream.
struct CipSessionData
{
    // Current data for this packet.
    CipCurrentData current_data;

    // Overall data for this session.
    CipGlobalSessionData global_data;
};

#endif  // CIP_DEFINITIONS_H

cip.h

Path = src/service_inspectors/cip/cip.h

#ifndef CIP_H
#define CIP_H
// Implementation header with definitions, datatypes and flowdata class for CIP service inspector.

#include "flow/flow.h"
#include "framework/counts.h"
#include "framework/data_bus.h"
#include "main/thread.h"
#include "protocols/packet.h"

#include "cip_definitions.h"

namespace snort
{
struct Packet;
}

enum CipDataType
{
    CIP_DATA_TYPE_PATH_CLASS = 0,
    CIP_DATA_TYPE_PATH_EXT_SYMBOL,
    CIP_DATA_TYPE_SET_ATTRIBUTE,
    CIP_DATA_TYPE_CONNECTION,
    CIP_DATA_TYPE_IMPLICIT,
    CIP_DATA_TYPE_OTHER,
    CIP_DATA_TYPE_ENIP_COMMAND,
    CIP_DATA_TYPE_MALFORMED,
    CIP_DATA_TYPE_CONNECTION_SAFETY
};

struct CipEventData
{
    // Specify the type of CIP data.
    CipDataType type;

    // Used for:
    //  CIP_DATA_TYPE_ENIP_COMMAND
    uint16_t enip_command_id;

    // Used for:
    //  CIP_DATA_TYPE_PATH_CLASS
    //  CIP_DATA_TYPE_PATH_EXT_SYMBOL
    //  CIP_DATA_TYPE_SET_ATTRIBUTE
    uint8_t service_id;

    // Used for:
    //  CIP_DATA_TYPE_PATH_CLASS: This represents the Request Path Class.
    //  CIP_DATA_TYPE_SET_ATTRIBUTE: This represents the Request Path Class.
    //  CIP_DATA_TYPE_CONNECTION: This represents the Connection Path Class.
    //  CIP_DATA_TYPE_IMPLICIT: This represents the Connection Path Class from
    //      the original connection request, for this connection.
    uint32_t class_id;

    // Used for:
    //  CIP_DATA_TYPE_SET_ATTRIBUTE: This represents the Request Path Instance.
    uint32_t instance_id;

    // Used for:
    //  CIP_DATA_TYPE_SET_ATTRIBUTE: This represents the Request Path Attribute.
    uint32_t attribute_id;

    // Pointer to snort::Packet
    const snort::Packet* snort_packet;

    static unsigned pub_id;
};

class CipFlowData : public snort::FlowData
{
public:
    CipFlowData();
    ~CipFlowData() override;

    static void init()
    { inspector_id = snort::FlowData::create_flow_data_id(); }

public:
    static unsigned inspector_id;
    CipSessionData session;
};

CipSessionData* get_cip_session_data(const snort::Flow*);

struct CipStats
{
    PegCount packets;
    PegCount sessions;
    PegCount concurrent_sessions;
    PegCount max_concurrent_sessions;
};

extern THREAD_LOCAL CipStats cip_stats;

#endif

cip_module.h

Path = src/service_inspectors/cip/cip_module.h

#ifndef CIP_MODULE_H
#define CIP_MODULE_H

// Interface to the CIP service inspector

#include "framework/module.h"

#include "cip_definitions.h"

#define GID_CIP 148

#define CIP_MALFORMED         1
#define CIP_NON_CONFORMING    2
#define CIP_CONNECTION_LIMIT  3
#define CIP_REQUEST_LIMIT     4

#define CIP_NAME "cip"
#define CIP_HELP "cip inspection"

extern THREAD_LOCAL snort::ProfileStats cip_perf_stats;

class CipModule : public snort::Module
{
public:
    CipModule();
    ~CipModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    unsigned get_gid() const override
    { return GID_CIP; }

    const snort::RuleMap* get_rules() const override;
    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile() const override;

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

    CipProtoConf* get_data();

private:
    CipProtoConf* conf;
    std::string embedded_path;
};

#endif

cip_paf.h

Path = src/service_inspectors/cip/cip_paf.h

#ifndef CIP_PAF_H
#define CIP_PAF_H

#include "stream/stream_splitter.h"

#include "cip.h"

/* State-tracking structs */
enum cip_paf_state
{
    CIP_PAF_STATE__COMMAND_1 = 0,
    CIP_PAF_STATE__COMMAND_2,
    CIP_PAF_STATE__LENGTH_1,
    CIP_PAF_STATE__LENGTH_2,
    CIP_PAF_STATE__SET_FLUSH
};

struct cip_paf_data
{
    cip_paf_state paf_state;
    uint16_t enip_length;
};

class CipSplitter : public snort::StreamSplitter
{
public:
    CipSplitter(bool c2s);

    Status scan(snort::Packet*, const uint8_t* data, uint32_t len, uint32_t flags,
        uint32_t* fp) override;

    bool is_paf() override
    {
        return true;
    }

public:
    cip_paf_data state;
};

#endif /* CIP_PAF_H */

cip_parsing.h

Path = src/service_inspectors/cip/cip_parsing.h

#ifndef CIP_PARSING_H
#define CIP_PARSING_H

#include <cstdint>
#include <cstring>
#include "pub_sub/cip_events.h"       // For CipEventData

#include "cip_definitions.h"  // For CIP structs

namespace snort
{
struct Packet;
}

//// EtherNet/IP Parsing

// Constants - EtherNet/IP encapsulation layer
#define ENIP_HEADER_SIZE 24u

// EtherNet/IP commands.
enum EnipCommand
{
    ENIP_COMMAND_NOP = 0x0000,
    ENIP_COMMAND_LIST_SERVICES = 0x0004,
    ENIP_COMMAND_LIST_IDENTITY = 0x0063,
    ENIP_COMMAND_LIST_INTERFACES = 0x0064,
    ENIP_COMMAND_REGISTER_SESSION = 0x0065,
    ENIP_COMMAND_UNREGISTER_SESSION = 0x0066,
    ENIP_COMMAND_SEND_RR_DATA = 0x006F,
    ENIP_COMMAND_SEND_UNIT_DATA = 0x0070
};

/// EtherNet/IP data parsing functions.
bool parse_enip_layer(const uint8_t* data,
    size_t data_length,
    bool is_TCP,
    CipCurrentData* current_data,
    CipGlobalSessionData* global_data);

void pack_cip_request_event(const CipRequest* request, CipEventData* cip_event_data);

#endif  // CIP_PARSING_H

cip_session.h

Path = src/service_inspectors/cip/cip_session.h

#ifndef CIP_SESSION_H
#define CIP_SESSION_H

#include <cstdint>

#include "main/snort_config.h"
#include "main/snort_types.h"

#include "cip_definitions.h"

// Default unconnected request timeout, milliseconds.
#define DEFAULT_UNCONNECTED_REQUEST_TIMEOUT (30000)

/// ENIP Session Management.
bool enip_session_add(EnipSession* enip_session, uint32_t session_handle);
bool enip_session_remove(EnipSession* enip_session, uint32_t session_handle);

// Returns true if session_handle matches the active session.
bool enip_session_handle_valid(const EnipSession* enip_session, uint32_t session_handle);

/// CIP Connection Management.
CipConnection* cip_find_connection_by_id(
    CipConnectionList* connection_list,
    CipPacketDirection direction,
    uint32_t connection_id,
    bool established);

bool cip_add_connection_to_active(CipConnectionList* connection_list,
    const CipForwardOpenResponse* forward_open_response);
bool cip_remove_connection(CipConnectionList* connection_list,
    const CipConnectionSignature* connection_signature,
    bool established);

bool cip_add_connection_to_pending(CipConnectionList* connection_list,
    const CipForwardOpenRequest* forward_open_request);

/// CIP Request/Response Matching.
bool cip_request_add(CipUnconnectedMessageList* unconnected_list,
    const EnipSessionData* enip_data,
    const CipRequest* cip_request,
    const struct timeval* timestamp);

// Find a request in the list, and remove it.
bool cip_request_remove(CipUnconnectedMessageList* unconnected_list,
    const EnipSessionData* enip_data,
    CipRequestType* request_type);

#endif  // CIP_SESSION_H

service_inspectors/dce_rpc/

This directory contains all the files related to DCE-RPC protocol processing.

DCE is supported over TCP/IP, UDP, SMB and HTTP v1 Proxy and Server. The DCE-RPC inspector is split into four inspectors - one for each transport. This includes the configuration as well as the inspector modules.

The inspectors perform SMB desegmentation and DCE-RPC defragmentation to avoid rule evasion using these techniques. IPS rule options are supported for matching on interface, opnum and stub data.

The Snort 2x server configuration is now split between the inspectors. Options that are meaningful to all inspectors, such as policy, are copied into each inspector configuration.

The address/port mapping is handled by the binder. Autodetect functionality is replaced by wizard curses.

The Snort 2x global configuration is now rolled into server configuration.

The http_proxy and http_server functionality is implemented as two inspectors. These inspectors only serve to locate the tunnel setup content. If/when the setup content is located, the session is transferred to the DCE TCP inspector.

The SMB inspector supports version 1 and version 2 of the protocol. It processes relevant messages and inspects file transferred over this transport. For SMBv2, it supports TCP as transport and it also supports multiple File transfer on single SMB2 connection or multiple SMB2 connection from the same source IP.

The inspector eval function checks for the presence of SMB flow data in the flow, if not present, it creates an SMB flow data according to detected version, otherwise calls the respective process function. The process functions processes the commands according to respective versions and when file transfer is detected, it calls file_flow process().

dce_co.h

Path = src/service_inspectors/dce_rpc/dce_co.h

#ifndef DCE_CO_H
#define DCE_CO_H

#include "dce_common.h"
#include "dce_list.h"

#define DCE2_CO_BAD_MAJOR_VERSION           27
#define DCE2_CO_BAD_MINOR_VERSION           28
#define DCE2_CO_BAD_PDU_TYPE                29
#define DCE2_CO_FRAG_LEN_LT_HDR             30
#define DCE2_CO_REM_FRAG_LEN_LT_SIZE        31
#define DCE2_CO_NO_CTX_ITEMS_SPECFD         32
#define DCE2_CO_NO_TFER_SYNTAX_SPECFD       33
#define DCE2_CO_FRAG_LT_MAX_XMIT_FRAG       34
#define DCE2_CO_FRAG_GT_MAX_XMIT_FRAG       35
#define DCE2_CO_ALTER_CHANGE_BYTE_ORDER     36
#define DCE2_CO_FRAG_DIFF_CALL_ID           37
#define DCE2_CO_FRAG_DIFF_OPNUM             38
#define DCE2_CO_FRAG_DIFF_CTX_ID            39

/* Map response fields offsets */
#define DCE2_CO_MAP_HANDLE_OFS              20
#define DCE2_CO_MAP_NUM_TOWERS_OFS          4
#define DCE2_CO_MAP_TWR_FLOOR12_OFS         25
#define DCE2_CO_MAP_TWR_FLOOR34_OFS         7
#define DCE2_CO_MAP_TWR_LEN_OFS             4
#define DCE2_CO_MAP_FLR_COUNT_OFS           2
#define DCE2_CO_MAP_FLR_LHS_RHS_OFS         2
#define DCE2_CO_MAP_FLR_PROTO_ID_OFS        1

/* Protocol IDs. Not a full list.
 * Refer to DCE RPC 1.1 Appendix I */
#define DCE2_CO_PROTO_DOD_TCP               0x07
#define DCE2_CO_PROTO_ID_CO                 0x0b

#define DCE2_CO_BAD_MAJOR_VERSION_STR  "connection oriented DCE/RPC - invalid major version"
#define DCE2_CO_BAD_MINOR_VERSION_STR  "connection oriented DCE/RPC - invalid minor version"
#define DCE2_CO_BAD_PDU_TYPE_STR       "connection-oriented DCE/RPC - invalid PDU type"
#define DCE2_CO_FRAG_LEN_LT_HDR_STR \
    "connection-oriented DCE/RPC - fragment length less than header size"
#define DCE2_CO_REM_FRAG_LEN_LT_SIZE_STR \
    "connection-oriented DCE/RPC - remaining fragment length less than size needed"
#define DCE2_CO_NO_CTX_ITEMS_SPECFD_STR \
    "connection-oriented DCE/RPC - no context items specified"
#define DCE2_CO_NO_TFER_SYNTAX_SPECFD_STR \
    "connection-oriented DCE/RPC -no transfer syntaxes specified"
#define DCE2_CO_FRAG_LT_MAX_XMIT_FRAG_STR \
    "connection-oriented DCE/RPC - fragment length on non-last fragment less than \
maximum negotiated fragment transmit size for client"
#define DCE2_CO_FRAG_GT_MAX_XMIT_FRAG_STR \
    "connection-oriented DCE/RPC - fragment length greater than \
maximum negotiated fragment transmit size"
#define DCE2_CO_ALTER_CHANGE_BYTE_ORDER_STR \
    "connection-oriented DCE/RPC - alter context byte order different from bind"
#define DCE2_CO_FRAG_DIFF_CALL_ID_STR \
    "connection-oriented DCE/RPC - call id of non first/last fragment different \
from call id established for fragmented request"
#define DCE2_CO_FRAG_DIFF_OPNUM_STR \
    "connection-oriented DCE/RPC - opnum of non first/last fragment different \
from opnum established for fragmented request"
#define DCE2_CO_FRAG_DIFF_CTX_ID_STR \
    "connection-oriented DCE/RPC - context id of non first/last fragment different \
from context id established for fragmented request"

#define DCE2_MAX_XMIT_SIZE_FUZZ    500
#define DCE2_MOCK_HDR_LEN__CO_CLI   ((unsigned)(sizeof(DceRpcCoHdr) + sizeof(DceRpcCoRequest)))
#define DCE2_MOCK_HDR_LEN__CO_SRV   ((unsigned)(sizeof(DceRpcCoHdr) + sizeof(DceRpcCoResponse)))
#define DCE2_CO__MIN_ALLOC_SIZE     50
#define DCE2_LITTLE_ENDIAN 0x10

#pragma pack(1)

struct DceRpcCoVersion
{
    uint8_t major;
    uint8_t minor;
};

/* Connection oriented common header */
struct DceRpcCoHdr
{
    DceRpcCoVersion pversion;
    uint8_t ptype;
    uint8_t pfc_flags;
    uint8_t packed_drep[4];
    uint16_t frag_length;
    uint16_t auth_length;
    uint32_t call_id;
};

/* Bind */
struct DceRpcCoBind
{
    uint16_t max_xmit_frag;
    uint16_t max_recv_frag;
    uint32_t assoc_group_id;
    uint8_t n_context_elem;   /* number of context elements */
    uint8_t reserved;
    uint16_t reserved2;
};

struct DceRpcCoSynId
{
    Uuid if_uuid;
    uint32_t if_version;
};

struct DceRpcCoContElem
{
    uint16_t p_cont_id;
    uint8_t n_transfer_syn;  /* number of transfer syntaxes */
    uint8_t reserved;
    DceRpcCoSynId abstract_syntax;
};

struct DceRpcCoBindAck
{
    uint16_t max_xmit_frag;
    uint16_t max_recv_frag;
    uint32_t assoc_group_id;
    uint16_t sec_addr_len;
};

struct DceRpcCoContResult
{
    uint16_t result;
    uint16_t reason;
    DceRpcCoSynId transfer_syntax;
};

struct DceRpcCoAuthVerifier
{
    uint8_t auth_type;
    uint8_t auth_level;
    uint8_t auth_pad_length;
    uint8_t auth_reserved;
    uint32_t auth_context_id;
};

struct DceRpcCoRequest
{
    uint32_t alloc_hint;
    uint16_t context_id;
    uint16_t opnum;
};

struct DceRpcCoResponse
{
    uint32_t alloc_hint;
    uint16_t context_id;
    uint8_t cancel_count;
    uint8_t reserved;
};

struct DceRpcCoContResultList
{
    uint8_t n_results;
    uint8_t reserved;
    uint16_t reserved2;
};

typedef DceRpcCoBind DceRpcCoAltCtx;
typedef DceRpcCoBindAck DceRpcCoAltCtxResp;

#pragma pack()

struct DCE2_CoFragTracker
{
    DCE2_Buffer* cli_stub_buf;
    DCE2_Buffer* srv_stub_buf;

    int opnum;    /* Opnum that is ultimately used for request */
    int ctx_id;   /* Context id that is ultimately used for request */

    /* These are set on a first fragment received */
    int expected_call_id;  /* Expected call id for fragments */
    int expected_opnum;    /* Expected call id for fragments */
    int expected_ctx_id;   /* Expected call id for fragments */
};

struct DCE2_CoSeg
{
    DCE2_Buffer* buf;

    /* If there is enough data in segmentation buffer for header,
     * this will be set to the frag length in the header */
    uint16_t frag_len;
};

struct DCE2_CoTracker
{
    DCE2_List* ctx_ids;  /* splayed list so most recently used goes to front of list */
    int got_bind;        /* got an accepted bind */

    /* Queue of pending client bind or alter context request context items
     * Since the actual context id number doesn't have to occur sequentially
     * in the context list in the client packet, need to keep track to match
     * up server response since server doesn't reply with actual context id
     * numbers, but in the order they were in the client packet */
    DCE2_Queue* pending_ctx_ids;

    /* Keeps track of fragmentation buffer and frag specific data */
    DCE2_CoFragTracker frag_tracker;

    int max_xmit_frag;    /* The maximum negotiated size of a client request */
    int data_byte_order;  /* Depending on policy is from bind or request */
    int ctx_id;           /* The current context id of the request */
    int opnum;            /* The current opnum of the request */
    int call_id;          /* The current call id of the request */
    const uint8_t* stub_data;   /* Current pointer to stub data in the request */

    /* For transport segmentation */
    DCE2_CoSeg cli_seg;
    DCE2_CoSeg srv_seg;
};

/*
 * Connection oriented
 */
enum DceRpcCoPfcFlags
{
    DCERPC_CO_PFC_FLAGS__FIRST_FRAG = 0x01,
    DCERPC_CO_PFC_FLAGS__LAST_FRAG = 0x02,
    DCERPC_CO_PFC_FLAGS__PENDING_CANCEL = 0x04,
    DCERPC_CO_PFC_FLAGS__RESERVED_1 = 0x08,
    DCERPC_CO_PFC_FLAGS__CONC_MPX = 0x10,
    DCERPC_CO_PFC_FLAGS__DID_NOT_EXECUTE = 0x20,
    DCERPC_CO_PFC_FLAGS__MAYBE = 0x40,
    DCERPC_CO_PFC_FLAGS__OBJECT_UUID = 0x80
};

enum DCE2_CoCtxState
{
    DCE2_CO_CTX_STATE__ACCEPTED,
    DCE2_CO_CTX_STATE__REJECTED,
    DCE2_CO_CTX_STATE__PENDING
};

enum DCE2_CoCtxTransport
{
    /* Default 32-Bit NDR defined in DCE RPC 1.1 */
    DCE2_CO_CTX_TRANS_SYNTAX_NDR_DEF,
    /* 64-Bit NDR defined in [MS-RPCE] */
    DCE2_CO_CTX_TRANS_SYNTAX_NDR64
};

enum DCE2_CoEpmOpnum
{
    DCE2_CO_EPT_NONE = -1,
    DCE2_CO_EPT_INSERT = 0,
    DCE2_CO_EPT_DELETE,
    DCE2_CO_EPT_LOOKUP,
    DCE2_CO_EPT_MAP,
    DCE2_CO_EPT_LKUP_HANDLE_FREE,
    DCE2_CO_EPT_INQ_OBJECT,
    DCE2_CO_EPT_MGMT_DELETE
};

struct DCE2_CoCtxIdNode
{
    uint16_t ctx_id;           /* The context id */
    Uuid iface;                /* The presentation syntax uuid for the interface */
    uint16_t iface_vers_maj;   /* The major version of the interface */
    uint16_t iface_vers_min;   /* The minor version of the interface */

    /* Whether or not the server accepted or rejected the client bind/alter context
     * request.  Initially set to pending until server response */
    DCE2_CoCtxState state;
    DCE2_CoCtxTransport transport;
};

enum DceRpcCoAuthLevelType
{
    DCERPC_CO_AUTH_LEVEL__NONE = 1,
    DCERPC_CO_AUTH_LEVEL__CONNECT,
    DCERPC_CO_AUTH_LEVEL__CALL,
    DCERPC_CO_AUTH_LEVEL__PKT,
    DCERPC_CO_AUTH_LEVEL__PKT_INTEGRITY,
    DCERPC_CO_AUTH_LEVEL__PKT_PRIVACY
};

enum DceRpcCoContDefResult
{
    DCERPC_CO_CONT_DEF_RESULT__ACCEPTANCE = 0,
    DCERPC_CO_CONT_DEF_RESULT__USER_REJECTION,
    DCERPC_CO_CONT_DEF_RESULT__PROVIDER_REJECTION
};

enum DCE2_CoRpktType
{
    DCE2_CO_RPKT_TYPE__SEG,
    DCE2_CO_RPKT_TYPE__FRAG,
    DCE2_CO_RPKT_TYPE__ALL
};

inline uint8_t DceRpcCoVersMaj(const DceRpcCoHdr* co)
{
    return co->pversion.major;
}

inline uint8_t DceRpcCoVersMin(const DceRpcCoHdr* co)
{
    return co->pversion.minor;
}

inline DceRpcPduType DceRpcCoPduType(const DceRpcCoHdr* co)
{
    return (DceRpcPduType)co->ptype;
}

inline DceRpcBoFlag DceRpcCoByteOrder(const DceRpcCoHdr* co)
{
    return DceRpcByteOrder(co->packed_drep[0]);
}

inline uint16_t DceRpcCoFragLen(const DceRpcCoHdr* co)
{
    return DceRpcNtohs(&co->frag_length, DceRpcCoByteOrder(co));
}

inline uint8_t DceRpcCoNumCtxItems(const DceRpcCoBind* cob)
{
    return cob->n_context_elem;
}

inline uint16_t DceRpcCoContElemCtxId(const DceRpcCoHdr* co, const DceRpcCoContElem* coce)
{
    return DceRpcNtohs(&coce->p_cont_id, DceRpcCoByteOrder(co));
}

inline uint8_t DceRpcCoContElemNumTransSyntaxes(const DceRpcCoContElem* coce)
{
    return coce->n_transfer_syn;
}

inline const Uuid* DceRpcCoContElemIface(const DceRpcCoContElem* coce)
{
    return &coce->abstract_syntax.if_uuid;
}

inline uint16_t DceRpcCoContElemIfaceVersMaj(const DceRpcCoHdr* co, const DceRpcCoContElem* coce)
{
    return (uint16_t)(DceRpcNtohl(&coce->abstract_syntax.if_version, DceRpcCoByteOrder(co)) &
           0x0000ffff);
}

inline uint16_t DceRpcCoContElemIfaceVersMin(const DceRpcCoHdr* co, const DceRpcCoContElem* coce)
{
    return (uint16_t)(DceRpcNtohl(&coce->abstract_syntax.if_version, DceRpcCoByteOrder(co)) >> 16);
}

inline uint16_t DceRpcCoBindAckMaxRecvFrag(const DceRpcCoHdr* co, const DceRpcCoBindAck* coba)
{
    return DceRpcNtohs(&coba->max_recv_frag, DceRpcCoByteOrder(co));
}

inline uint16_t DceRpcCoSecAddrLen(const DceRpcCoHdr* co, const DceRpcCoBindAck* coba)
{
    return DceRpcNtohs(&coba->sec_addr_len, DceRpcCoByteOrder(co));
}

inline uint16_t DceRpcCoContRes(const DceRpcCoHdr* co, const DceRpcCoContResult* cocr)
{
    return DceRpcNtohs(&cocr->result, DceRpcCoByteOrder(co));
}

inline const Uuid* DceRpcCoContResTransport(const DceRpcCoContResult* cocr)
{
    return &cocr->transfer_syntax.if_uuid;
}

inline int DceRpcCoObjectFlag(const DceRpcCoHdr* co)
{
    return co->pfc_flags & DCERPC_CO_PFC_FLAGS__OBJECT_UUID;
}

inline int DceRpcCoFirstFrag(const DceRpcCoHdr* co)
{
    return co->pfc_flags & DCERPC_CO_PFC_FLAGS__FIRST_FRAG;
}

inline int DceRpcCoLastFrag(const DceRpcCoHdr* co)
{
    return co->pfc_flags & DCERPC_CO_PFC_FLAGS__LAST_FRAG;
}

inline uint16_t DceRpcCoAuthLen(const DceRpcCoHdr* co)
{
    return DceRpcNtohs(&co->auth_length, DceRpcCoByteOrder(co));
}

inline uint8_t DceRpcCoAuthLevel(const DceRpcCoAuthVerifier* coav)
{
    return coav->auth_level;
}

inline uint16_t DceRpcCoAuthPad(const DceRpcCoAuthVerifier* coav)
{
    return coav->auth_pad_length;
}

inline uint16_t DceRpcCoCtxIdResp(const DceRpcCoHdr* co, const DceRpcCoResponse* cor)
{
    return DceRpcNtohs(&cor->context_id, DceRpcCoByteOrder(co));
}

inline uint16_t DceRpcCoBindMaxXmitFrag(const DceRpcCoHdr* co, const DceRpcCoBind* cob)
{
    return DceRpcNtohs(&cob->max_xmit_frag, DceRpcCoByteOrder(co));
}

inline uint8_t DceRpcCoContNumResults(const DceRpcCoContResultList* cocrl)
{
    return cocrl->n_results;
}

inline uint32_t DceRpcCoCallId(const DceRpcCoHdr* co)
{
    return DceRpcNtohl(&co->call_id, DceRpcCoByteOrder(co));
}

inline uint16_t DceRpcCoOpnum(const DceRpcCoHdr* co, const DceRpcCoRequest* cor)
{
    return DceRpcNtohs(&cor->opnum, DceRpcCoByteOrder(co));
}

inline uint16_t DceRpcCoCtxId(const DceRpcCoHdr* co, const DceRpcCoRequest* cor)
{
    return DceRpcNtohs(&cor->context_id, DceRpcCoByteOrder(co));
}

inline int DCE2_GetNdrUint32(const uint8_t* data_ptr, uint32_t& data,
    int offset, DceRpcBoFlag bo_flag)
{
    const uint32_t* ptr;
    int align_offset = 0;

    /* Alignment */
    if (offset % 4)
    {
        align_offset = 4 - (offset % 4);
    }

    ptr = (const uint32_t*)(data_ptr + align_offset);
    data = DceRpcNtohl(ptr, bo_flag);

    return align_offset + 4;
}

inline int DCE2_GetNdrUint64(const uint8_t* data_ptr, uint64_t& data,
    int offset, DceRpcBoFlag bo_flag)
{
    const uint64_t* ptr;
    int align_offset = 0;

    /* Alignment */
    if (offset % 8)
    {
        align_offset = 8 - (offset % 8);
    }

    ptr = (const uint64_t*)(data_ptr + align_offset);
    data = DceRpcNtohl64(ptr, bo_flag);

    return align_offset + 8;
}

inline int DCE2_GetNdrUint3264(const uint8_t* data_ptr, uint64_t& data,
    int offset, DceRpcBoFlag bo_flag, DCE2_CoCtxTransport transport)
{
    if (transport == DCE2_CO_CTX_TRANS_SYNTAX_NDR64)
        return DCE2_GetNdrUint64(data_ptr, data, offset, bo_flag);

    uint32_t val = 0;
    offset = DCE2_GetNdrUint32(data_ptr, val, offset, bo_flag);
    data = val;
    return offset;
}

void DCE2_CoInitTracker(DCE2_CoTracker*);
void DCE2_CoProcess(DCE2_SsnData*, DCE2_CoTracker*,
    const uint8_t*, uint16_t);
void DCE2_CoInitRdata(uint8_t*, int);
void DCE2_CoCleanTracker(DCE2_CoTracker*);

#endif

dce_common.h

Path = src/service_inspectors/dce_rpc/dce_common.h

#ifndef DCE_COMMON_H
#define DCE_COMMON_H

#include <cassert>
#include <memory>
#include <string>

#include "detection/detection_engine.h"
#include "framework/counts.h"
#include "framework/endianness.h"
#include "framework/value.h"
#include "protocols/packet.h"

#include "dce_list.h"
#include "dce_context_data.h"

extern const snort::InspectApi dce2_smb_api;
extern const snort::InspectApi dce2_tcp_api;
extern const snort::InspectApi dce2_udp_api;
extern const snort::InspectApi dce_http_proxy_api;
extern const snort::InspectApi dce_http_server_api;
extern THREAD_LOCAL int dce2_detected;

#define GID_DCE2 133
#define DCE_RPC_SERVICE_NAME "dcerpc"

enum DCE2_Policy
{
    DCE2_POLICY__WIN2000 = 0,
    DCE2_POLICY__WINXP,
    DCE2_POLICY__WINVISTA,
    DCE2_POLICY__WIN2003,
    DCE2_POLICY__WIN2008,
    DCE2_POLICY__WIN7,
    DCE2_POLICY__SAMBA,
    DCE2_POLICY__SAMBA_3_0_37,
    DCE2_POLICY__SAMBA_3_0_22,
    DCE2_POLICY__SAMBA_3_0_20,
    DCE2_POLICY__MAX
};

struct dce2CommonStats
{
    PegCount events;
    PegCount co_pdus;
    PegCount co_bind;
    PegCount co_bind_ack;
    PegCount co_alter_ctx;
    PegCount co_alter_ctx_resp;
    PegCount co_bind_nack;
    PegCount co_request;
    PegCount co_response;
    PegCount co_cancel;
    PegCount co_orphaned;
    PegCount co_fault;
    PegCount co_auth3;
    PegCount co_shutdown;
    PegCount co_reject;
    PegCount co_ms_pdu;
    PegCount co_other_req;
    PegCount co_other_resp;
    PegCount co_req_fragments;
    PegCount co_resp_fragments;
    PegCount co_cli_max_frag_size;
    PegCount co_cli_min_frag_size;
    PegCount co_cli_seg_reassembled;
    PegCount co_cli_frag_reassembled;
    PegCount co_srv_max_frag_size;
    PegCount co_srv_min_frag_size;
    PegCount co_srv_seg_reassembled;
    PegCount co_srv_frag_reassembled;
};
#define DCE2_SARG__POLICY_WIN2000       "Win2000"
#define DCE2_SARG__POLICY_WINXP         "WinXP"
#define DCE2_SARG__POLICY_WINVISTA      "WinVista"
#define DCE2_SARG__POLICY_WIN2003       "Win2003"
#define DCE2_SARG__POLICY_WIN2008       "Win2008"
#define DCE2_SARG__POLICY_WIN7          "Win7"
#define DCE2_SARG__POLICY_SAMBA         "Samba"
#define DCE2_SARG__POLICY_SAMBA_3_0_37  "Samba-3.0.37"  /* Samba version 3.0.37 and previous */
#define DCE2_SARG__POLICY_SAMBA_3_0_22  "Samba-3.0.22"  /* Samba version 3.0.22 and previous */
#define DCE2_SARG__POLICY_SAMBA_3_0_20  "Samba-3.0.20"  /* Samba version 3.0.20 and previous */

struct dce2CommonProtoConf
{
    bool limit_alerts;
    bool disable_defrag;
    int max_frag_len;
};

struct dce2CoProtoConf
{
    dce2CommonProtoConf common; // This member must be first
    DCE2_Policy policy;
    uint16_t co_reassemble_threshold;
};

#define DCE2_DEBUG__PAF_END_MSG    "=========================================================="

enum DceRpcPduType
{
    DCERPC_PDU_TYPE__REQUEST = 0,
    DCERPC_PDU_TYPE__PING,
    DCERPC_PDU_TYPE__RESPONSE,
    DCERPC_PDU_TYPE__FAULT,
    DCERPC_PDU_TYPE__WORKING,
    DCERPC_PDU_TYPE__NOCALL,
    DCERPC_PDU_TYPE__REJECT,
    DCERPC_PDU_TYPE__ACK,
    DCERPC_PDU_TYPE__CL_CANCEL,
    DCERPC_PDU_TYPE__FACK,
    DCERPC_PDU_TYPE__CANCEL_ACK,
    DCERPC_PDU_TYPE__BIND,
    DCERPC_PDU_TYPE__BIND_ACK,
    DCERPC_PDU_TYPE__BIND_NACK,
    DCERPC_PDU_TYPE__ALTER_CONTEXT,
    DCERPC_PDU_TYPE__ALTER_CONTEXT_RESP,
    DCERPC_PDU_TYPE__AUTH3,
    DCERPC_PDU_TYPE__SHUTDOWN,
    DCERPC_PDU_TYPE__CO_CANCEL,
    DCERPC_PDU_TYPE__ORPHANED,
    DCERPC_PDU_TYPE__MICROSOFT_PROPRIETARY_OUTLOOK2003_RPC_OVER_HTTP,
    DCERPC_PDU_TYPE__MAX
};

/* Version 4 is for Connectionless
 * Version 5 is for Connection oriented */
enum DceRpcProtoMajorVers
{
    DCERPC_PROTO_MAJOR_VERS__4 = 4,
    DCERPC_PROTO_MAJOR_VERS__5 = 5
};

enum DceRpcProtoMinorVers
{
    DCERPC_PROTO_MINOR_VERS__0 = 0,
    DCERPC_PROTO_MINOR_VERS__1 = 1
};

enum DCE2_RpktType
{
    DCE2_RPKT_TYPE__NULL = 0,
    DCE2_RPKT_TYPE__SMB_SEG,
    DCE2_RPKT_TYPE__SMB_TRANS,
    DCE2_RPKT_TYPE__SMB_CO_SEG,
    DCE2_RPKT_TYPE__SMB_CO_FRAG,
    DCE2_RPKT_TYPE__TCP_CO_SEG,
    DCE2_RPKT_TYPE__TCP_CO_FRAG,
    DCE2_RPKT_TYPE__UDP_CL_FRAG,
    DCE2_RPKT_TYPE__MAX
};

struct DCE2_Roptions
{
    /* dce_iface */
    int first_frag;    /* Set to sentinel if not applicable */
    Uuid iface;
    /* For connectionless */
    uint32_t iface_vers;        /* For connectionless */

    /* For connection-oriented */
    uint16_t iface_vers_maj;
    uint16_t iface_vers_min;

    /* dce_opnum */
    int opnum;    /* Set to sentinel if not applicable */

    /* dce_stub_data */
    const uint8_t* stub_data;  /* Set to null  if not applicable */
};

enum DCE2_SsnFlag
{
    DCE2_SSN_FLAG__NONE               = 0x0000,
    DCE2_SSN_FLAG__NO_INSPECT         = 0x0001,
    DCE2_SSN_FLAG__SMB2               = 0x0002,
    DCE2_SSN_FLAG__ALL                = 0xffff
};

struct DCE2_SsnData
{
    DCE2_TransType trans;
    DCE2_Policy server_policy;
    DCE2_Policy client_policy;
    int flags;
    uint64_t alert_mask;
    DCE2_Roptions ropts;
    void* config;

    uint32_t cli_seq;
    uint32_t cli_nseq;
    uint32_t srv_seq;
    uint32_t srv_nseq;
};

class DceEndianness : public snort::Endianness
{
public:
    int hdr_byte_order;   /* Set to sentinel if not applicable */
    int data_byte_order;  /* Set to sentinel if not applicable */
    int32_t stub_data_offset;  /* Set to sentinel if not applicable */

public:
    DceEndianness();
    bool get_offset_endianness(int32_t offset, uint8_t& endian) override;
    void reset();
};

inline void DCE2_ResetRopts(DCE2_SsnData* sd, const snort::Packet* p)
{
    sd->ropts.first_frag = DCE2_SENTINEL;
    sd->ropts.opnum = DCE2_SENTINEL;
    sd->ropts.stub_data = nullptr;
    DceContextData::clear_current_ropts(p, sd->trans);
}

inline void DCE2_SsnSetNoInspect(DCE2_SsnData* sd)
{
    sd->flags |= DCE2_SSN_FLAG__NO_INSPECT;
}

inline int DCE2_SsnNoInspect(DCE2_SsnData* sd)
{
    return sd->flags & DCE2_SSN_FLAG__NO_INSPECT;
}

inline bool DCE2_GcDceDefrag(dce2CommonProtoConf* config)
{
    return (config->disable_defrag ? false : true);
}

inline bool DCE2_GcMaxFrag(dce2CommonProtoConf* config)
{
    if (config->max_frag_len != DCE2_SENTINEL)
        return true;
    return false;
}

inline uint16_t DCE2_GcMaxFragLen(dce2CommonProtoConf* config)
{
    if (DCE2_GcMaxFrag(config))
        return (uint16_t)config->max_frag_len;
    return UINT16_MAX;
}

inline DCE2_Policy DCE2_SsnGetPolicy(DCE2_SsnData* sd)
{
    assert(sd);

    if ( snort::DetectionEngine::get_current_packet()->is_from_client() )
        return sd->server_policy;
    else
        return sd->client_policy;
}

inline void DCE2_SsnSetPolicy(DCE2_SsnData* sd, DCE2_Policy policy)
{
    if ( snort::DetectionEngine::get_current_packet()->is_from_client() )
        sd->client_policy = policy;
    else
        sd->server_policy = policy;
}

/********************************************************************
 * Function: DCE2_SsnIsWindowsPolicy()
 *
 * Purpose:
 *  Convenience function to determine if policy traffic is going to
 *  is a Windows one.
 *
 * Arguments:
 *  DCE2_SsnData * - pointer to session data
 *
 * Returns:
 *  bool  -  true if Samba, false if not
 *
 ********************************************************************/
inline bool DCE2_SsnIsWindowsPolicy(DCE2_SsnData* sd)
{
    DCE2_Policy policy = DCE2_SsnGetPolicy(sd);

    switch (policy)
    {
    case DCE2_POLICY__WIN2000:
    case DCE2_POLICY__WINXP:
    case DCE2_POLICY__WINVISTA:
    case DCE2_POLICY__WIN2003:
    case DCE2_POLICY__WIN2008:
    case DCE2_POLICY__WIN7:
        return true;
    default:
        break;
    }

    return false;
}

/********************************************************************
 * Function: DCE2_SsnIsSambaPolicy()
 *
 * Purpose:
 *  Convenience function to determine if policy traffic is going to
 *  is a Samba one.
 *
 * Arguments:
 *  DCE2_SsnData * - pointer to session data
 *
 * Returns:
 *  bool  -  true if Samba, false if not
 *
 ********************************************************************/
inline bool DCE2_SsnIsSambaPolicy(DCE2_SsnData* sd)
{
    DCE2_Policy policy = DCE2_SsnGetPolicy(sd);

    switch (policy)
    {
    case DCE2_POLICY__SAMBA:
    case DCE2_POLICY__SAMBA_3_0_37:
    case DCE2_POLICY__SAMBA_3_0_22:
    case DCE2_POLICY__SAMBA_3_0_20:
        return true;
    default:
        break;
    }

    return false;
}

inline DCE2_Policy DCE2_SsnGetServerPolicy(DCE2_SsnData* sd)
{
    return sd->server_policy;
}

/********************************************************************
 * Function: DCE2_SsnIsServerSambaPolicy()
 *
 * Purpose:
 *  Convenience function to determine if server policy is a
 *  Samba one.
 *
 * Arguments:
 *  DCE2_SsnData * - pointer to session data
 *
 * Returns:
 *  bool  -  true if Samba, false if not
 *
 ********************************************************************/
inline bool DCE2_SsnIsServerSambaPolicy(DCE2_SsnData* sd)
{
    DCE2_Policy policy = DCE2_SsnGetServerPolicy(sd);

    switch (policy)
    {
    case DCE2_POLICY__SAMBA:
    case DCE2_POLICY__SAMBA_3_0_37:
    case DCE2_POLICY__SAMBA_3_0_22:
    case DCE2_POLICY__SAMBA_3_0_20:
        return true;
    default:
        break;
    }

    return false;
}

inline void dce_alert(uint32_t gid, uint32_t sid, dce2CommonStats* stats, DCE2_SsnData& sd)
{
    if ( ((dce2CommonProtoConf*)sd.config)->limit_alerts )
    {
        // Assuming the maximum sid for dce is less than 64
        if ( sd.alert_mask & ((uint64_t)1 << sid) )
            return;
        sd.alert_mask |= ((uint64_t)1 << sid);
    }
    snort::DetectionEngine::queue_event(gid,sid);
    stats->events++;
}

bool dce2_set_common_config(const snort::Value&, dce2CommonProtoConf&);
void print_dce2_common_config(const dce2CommonProtoConf&);
bool dce2_set_co_config(const snort::Value&, dce2CoProtoConf&);
void print_dce2_co_config(const dce2CoProtoConf&);
bool dce2_paf_abort(DCE2_SsnData*);
void DCE2_Detect(DCE2_SsnData*);
snort::Packet* DCE2_GetRpkt(snort::Packet*, DCE2_RpktType, const uint8_t*, uint32_t);
uint16_t DCE2_GetRpktMaxData(DCE2_RpktType);
DCE2_Ret DCE2_AddDataToRpkt(snort::Packet*, const uint8_t*, uint32_t);
DCE2_TransType get_dce2_trans_type(const snort::Packet* p);
void reset_using_rpkt();

#endif

dce_context_data.h

Path = src/service_inspectors/dce_rpc/dce_context_data.h

#ifndef DCE_CONTEXT_DATA_H
#define DCE_CONTEXT_DATA_H

#include "detection/ips_context_data.h"
#include "dce_utils.h"

struct DCE2_Roptions;
struct DCE2_SsnData;

class DceContextData : public snort::IpsContextData
{
public:
    void clear() override;

    static unsigned smb_ips_id;
    static unsigned tcp_ips_id;
    static unsigned udp_ips_id;

    DCE2_Roptions* current_ropts = nullptr;
    bool no_inspect = false;

    static void init(DCE2_TransType trans);
    static unsigned get_ips_id(DCE2_TransType trans);
    static void set_ips_id(DCE2_TransType trans, unsigned id);

    static DceContextData* get_current_data(const snort::Packet* p);
    static DCE2_Roptions* get_current_ropts(const snort::Packet* p);
    static bool is_noinspect(const snort::Packet* p);
    static void set_current_ropts(DCE2_SsnData* sd);
    static void clear_current_ropts(const snort::Packet* p, DCE2_TransType trans);
    static void clear_current_ropts(snort::IpsContext* context, DCE2_TransType trans);
};

#endif

dce_db.h

Path = src/service_inspectors/dce_rpc/dce_db.h

#ifndef DCE_DB_H
#define DCE_DB_H

#include <unordered_map>
#include <vector>
#include "dce_utils.h"

#include "main/snort_types.h"

template<typename Key, typename Value, typename Hash>
class DCE2_Db
{
public:

    virtual bool Insert(const Key& key, Value data) = 0;
    virtual Value Find(const Key& key) = 0;
    virtual void Remove(const Key& key) = 0;
    virtual int GetSize() = 0;
    virtual std::vector< std::pair<Key, Value> > get_all_entry() = 0;
};

template<typename Key, typename Value, typename Hash>
class DCE2_DbMap : public DCE2_Db<Key, Value, Hash>
{
public:

    DCE2_DbMap(bool gc = true) :   garbage_collection(gc) { }

    ~DCE2_DbMap()
    {
        auto it = Map.cbegin();
        while (it != Map.cend())
        {
            if (garbage_collection)
                delete it->second;
            it = Map.erase(it);
        }
    }

    bool Insert(const Key& key, Value data) override;
    Value Find(const Key& key) override;
    void Remove(const Key& key) override;
    int GetSize() override
    {
        return Map.size();
    }

    std::vector< std::pair<Key, Value> > get_all_entry() override;

private:
    std::unordered_map<Key, Value, Hash> Map;
    bool garbage_collection = true;
};

template<typename Key, typename Value, typename Hash>
bool DCE2_DbMap<Key, Value, Hash>::Insert(const Key& key, Value data)
{
    return Map.insert(std::make_pair(key,data)).second;
}

template<typename Key, typename Value, typename Hash>
Value DCE2_DbMap<Key, Value, Hash>::Find(const Key& key)
{
    auto elem = Map.find(key);
    if (elem != Map.end())
        return elem->second;
    return nullptr;
}

template<typename Key, typename Value, typename Hash>
void DCE2_DbMap<Key, Value, Hash>::Remove(const Key& key)
{
    auto elem = Map.find(key);
    if (elem != Map.end())
    {
        if (garbage_collection)
            delete elem->second;

        Map.erase(elem->first);
    }
}

template<typename Key, typename Value, typename Hash>
std::vector< std::pair<Key, Value> >DCE2_DbMap<Key, Value, Hash>::get_all_entry()
{
    std::vector<std::pair<Key, Value> > vec;
    std::copy(Map.begin(), Map.end(), std::back_inserter(vec));
    return vec;
}

#endif

dce_expected_session.h

Path = src/service_inspectors/dce_rpc/dce_expected_session.h

#ifndef DCE_EXPECTED_SESSION_H
#define DCE_EXPECTED_SESSION_H

#include "framework/decode_data.h"
#include "target_based/snort_protocols.h"

namespace snort
{
struct Packet;
struct SfIp;
}

struct dce2TcpProtoConf;

// Expected Session Manager
class DceExpSsnManager
{
public:
    DceExpSsnManager(IpProtocol p, PktType t) :
        proto(p), type(t) { }

    virtual ~DceExpSsnManager() = default;

    void set_proto_id(SnortProtocolId id)
    { protocol_id = id; }

    SnortProtocolId get_proto_id() const
    { return protocol_id; }

    IpProtocol get_ip_proto() const
    { return proto; }

    PktType get_pkt_type() const
    { return type; }

    static void create_expected_session(const snort::SfIp*, uint16_t, const char*);

private:
    virtual int create_expected_session_impl(snort::Packet*,
        const snort::SfIp*, uint16_t, const snort::SfIp*, uint16_t,
        PktType, IpProtocol, SnortProtocolId) = 0;

private:
    SnortProtocolId protocol_id = UNKNOWN_PROTOCOL_ID;
    IpProtocol proto;
    PktType type;
};

class DceTcpExpSsnManager : public DceExpSsnManager
{
public:
    DceTcpExpSsnManager() = delete;
    explicit DceTcpExpSsnManager(const dce2TcpProtoConf&);
    DceTcpExpSsnManager(const DceTcpExpSsnManager&) = delete;
    DceTcpExpSsnManager& operator=(const DceTcpExpSsnManager&) = delete;

private:
    int create_expected_session_impl(snort::Packet*,
        const snort::SfIp*, uint16_t, const snort::SfIp*, uint16_t,
        PktType, IpProtocol, SnortProtocolId) override;

private:
    const dce2TcpProtoConf& pc;
};

#endif // DCE_EXPECTED_SESSION_H

dce_http_proxy_module.h

Path = src/service_inspectors/dce_rpc/dce_http_proxy_module.h

#ifndef DCE2_HTTP_PROXY_MODULE_H
#define DCE2_HTTP_PROXY_MODULE_H

#include "dce_common.h"
#include "framework/counts.h"
#include "framework/module.h"
#include "main/thread.h"

class DceHttpProxyModule : public snort::Module
{
public:
    DceHttpProxyModule();

    unsigned get_gid() const override
    {
        return GID_DCE2;
    }

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }
};

#define DCE_HTTP_PROXY_NAME "dce_http_proxy"
#define DCE_HTTP_PROXY_HELP "dce over http inspection - client to/from proxy"

struct DceHttpProxyStats
{
    PegCount http_proxy_sessions;
    PegCount http_proxy_session_failures;
};

extern THREAD_LOCAL DceHttpProxyStats dce_http_proxy_stats;

#endif

dce_http_proxy_splitter.h

Path = src/service_inspectors/dce_rpc/dce_http_proxy_splitter.h

#ifndef DCE_HTTP_PROXY_SPLITTER
#define DCE_HTTP_PROXY_SPLITTER

#include "dce_common.h"
#include "stream/stream_splitter.h"

class DceHttpProxySplitter : public snort::StreamSplitter
{
public:
    DceHttpProxySplitter(bool c2s);

    Status scan(snort::Packet*, const uint8_t* data, uint32_t len,
        uint32_t flags, uint32_t* fp) override;

    const snort::StreamBuffer reassemble(snort::Flow*, unsigned, unsigned,
        const uint8_t*, unsigned, uint32_t, unsigned&) override;

    bool is_paf() override
    { return true; }

    bool cutover_inspector()
    { return cutover; }

private:
    Status match_request_head(const uint8_t* data, uint32_t& len);
    Status match_response_head(const uint8_t* data, uint32_t& len);
    Status match_response(const uint8_t* data, const uint32_t& len);

    enum DceHttpProxyState
    {
        HTTP_PROXY_INIT = 0,
        HTTP_PROXY_HEAD = 1,
        HTTP_PROXY_FIRST_NL = 2,
    };

    DceHttpProxyState match_state;
    unsigned int match_index;
    bool cutover;
};

#endif

dce_http_server_module.h

Path = src/service_inspectors/dce_rpc/dce_http_server_module.h

#ifndef DCE2_HTTP_SERVER_MODULE_H
#define DCE2_HTTP_SERVER_MODULE_H

#include "dce_common.h"
#include "framework/counts.h"
#include "framework/module.h"
#include "main/thread.h"

class DceHttpServerModule : public snort::Module
{
public:
    DceHttpServerModule();

    unsigned get_gid() const override
    {
        return GID_DCE2;
    }

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }
};

#define DCE_HTTP_SERVER_NAME "dce_http_server"
#define DCE_HTTP_SERVER_HELP "dce over http inspection - proxy to/from server"

struct DceHttpServerStats
{
    PegCount http_server_sessions;
    PegCount http_server_session_failures;
};

extern THREAD_LOCAL DceHttpServerStats dce_http_server_stats;

#endif

dce_http_server_splitter.h

Path = src/service_inspectors/dce_rpc/dce_http_server_splitter.h

#ifndef DCE_HTTP_SERVER_SPLITTER
#define DCE_HTTP_SERVER_SPLITTER

#include "dce_common.h"
#include "stream/stream_splitter.h"

class DceHttpServerSplitter : public snort::StreamSplitter
{
public:
    DceHttpServerSplitter(bool c2s);

    Status scan(snort::Packet*, const uint8_t* data, uint32_t len,
        uint32_t flags, uint32_t* fp) override;
//  FIXIT-M - Should be able to implement but framework does not permit
/*    const StreamBuffer* reassemble(Flow*, unsigned, unsigned,
        const uint8_t*, unsigned len, uint32_t, unsigned& copied)
    {
        copied = len;
        return nullptr;
    }*/
    bool is_paf() override
    {
        return true;
    }

    bool cutover_inspector()
    {
        return cutover;
    }

private:
    Status match(const uint8_t* data, uint32_t& len);
    unsigned int match_index;
    bool cutover;
};

#endif

dce_list.h

Path = src/service_inspectors/dce_rpc/dce_list.h

#ifndef DCE_LIST_H
#define DCE_LIST_H

#include "dce_utils.h"

#include "main/snort_types.h"

/********************************************************************
 * Enumerations
 ********************************************************************/
enum DCE2_ListType
{
    DCE2_LIST_TYPE__NORMAL = 0,  /* Don't do anything special */
    DCE2_LIST_TYPE__SORTED,      /* Sort list by key */
    DCE2_LIST_TYPE__SPLAYED      /* Move most recently accessed node to head */
};

enum DCE2_ListFlags
{
    DCE2_LIST_FLAG__NO_FLAG  = 0x00,   /* No flags */
    DCE2_LIST_FLAG__NO_DUPS  = 0x01,   /* No duplicate keys in list */
    DCE2_LIST_FLAG__INS_TAIL = 0x02    /* Insert at tail - default is to insert at head */
};

/********************************************************************
 * Callbacks
 ********************************************************************/
typedef void (* DCE2_ListDataFree)(void*);
typedef void (* DCE2_ListKeyFree)(void*);
typedef int (* DCE2_ListKeyCompare)(const void*, const void*);

/********************************************************************
 * Structures
 ********************************************************************/
struct DCE2_ListNode
{
    void* key;
    void* data;
    struct DCE2_ListNode* prev;
    struct DCE2_ListNode* next;
};

struct DCE2_List
{
    DCE2_ListType type;
    uint32_t num_nodes;
    DCE2_ListKeyCompare compare;
    DCE2_ListDataFree data_free;
    DCE2_ListKeyFree key_free;
    int flags;
    struct DCE2_ListNode* head;
    struct DCE2_ListNode* tail;
    struct DCE2_ListNode* current;
    struct DCE2_ListNode* next;
    struct DCE2_ListNode* prev;
};

struct DCE2_QueueNode
{
    void* data;
    DCE2_QueueNode* prev;
    DCE2_QueueNode* next;
};

typedef DCE2_ListDataFree DCE2_QueueDataFree;

struct DCE2_Queue
{
    uint32_t num_nodes;
    DCE2_QueueDataFree data_free;
    DCE2_QueueNode* current;
    DCE2_QueueNode* head;
    DCE2_QueueNode* tail;
    DCE2_QueueNode* next;
    DCE2_QueueNode* prev;
};

/********************************************************************
 * Public function prototypes
 ********************************************************************/
DCE2_List* DCE2_ListNew(DCE2_ListType, DCE2_ListKeyCompare, DCE2_ListDataFree,
    DCE2_ListKeyFree, int);
DCE2_Ret DCE2_ListInsert(DCE2_List*, void*, void*);
void* DCE2_ListFirst(DCE2_List*);
void* DCE2_ListNext(DCE2_List*);
inline bool DCE2_ListIsEmpty(DCE2_List*);
void DCE2_ListEmpty(DCE2_List*);
void DCE2_ListDestroy(DCE2_List*);
void* DCE2_ListFind(DCE2_List*, void*);
DCE2_Ret DCE2_ListFindKey(DCE2_List*, void*);
DCE2_Ret DCE2_ListRemove(DCE2_List*, void*);
void DCE2_ListRemoveCurrent(DCE2_List*);

DCE2_Queue* DCE2_QueueNew(DCE2_QueueDataFree);
DCE2_Ret DCE2_QueueEnqueue(DCE2_Queue*, void*);
void* DCE2_QueueDequeue(DCE2_Queue*);
static inline bool DCE2_QueueIsEmpty(DCE2_Queue*);
void DCE2_QueueEmpty(DCE2_Queue*);
void* DCE2_QueueFirst(DCE2_Queue*);
void* DCE2_QueueNext(DCE2_Queue*);
void DCE2_QueueDestroy(DCE2_Queue*);
void DCE2_QueueRemoveCurrent(DCE2_Queue*);
void* DCE2_QueueLast(DCE2_Queue*);

/********************************************************************
 * Function: DCE2_ListIsEmpty()
 *
 * Determines whether or not the list has any items in it
 * currently.
 *
 * Arguments:
 *  DCE2_List *
 *      A pointer to the list object.
 *
 * Returns:
 *  bool
 *      true if the list has zero nodes in it or the list object
 *          passed in is null.
 *      false if the list has one or more nodes in it.
 *
 ********************************************************************/
inline bool DCE2_ListIsEmpty(DCE2_List* list)
{
    if (list == nullptr)
        return true;
    if (list->num_nodes == 0)
        return true;
    return false;
}

/********************************************************************
 * Function: DCE2_QueueIsEmpty()
 *
 * Determines whether or not the queue has any items in it
 * currently.
 *
 ********************************************************************/
inline bool DCE2_QueueIsEmpty(DCE2_Queue* queue)
{
    if (queue == nullptr)
        return true;
    if (queue->num_nodes == 0)
        return true;
    return false;
}

#endif

dce_smb2_commands.h

Path = src/service_inspectors/dce_rpc/dce_smb2_commands.h

#ifndef DCE_SMB2_COMMANDS_H
#define DCE_SMB2_COMMANDS_H

#include "dce_smb_module.h"
#include "dce_smb_utils.h"
#include "dce_smb2_utils.h"
#include "detection/detection_util.h"
#include "file_api/file_flows.h"
#include "file_api/file_service.h"

void DCE2_Smb2Setup(DCE2_Smb2SsnData*, const Smb2Hdr*,
    const uint64_t sid, const uint8_t* smb_data, const uint8_t* end);

void DCE2_Smb2TreeConnect(DCE2_Smb2SsnData*, const Smb2Hdr*,
    const uint8_t* smb_data, const uint8_t* end, DCE2_Smb2SessionTracker* str, uint32_t tid);

void DCE2_Smb2TreeDisconnect(DCE2_Smb2SsnData*, const uint8_t* smb_data,
    const uint8_t* end);

void DCE2_Smb2Create(DCE2_Smb2SsnData*, const Smb2Hdr*,
    const uint8_t* smb_data, const uint8_t* end, uint64_t mid, uint64_t sid, uint32_t tid);

void DCE2_Smb2Read(DCE2_Smb2SsnData*, const Smb2Hdr*,
    const uint8_t* smb_data, const uint8_t* end, DCE2_Smb2SessionTracker* str,
    DCE2_Smb2TreeTracker* ttr, uint64_t mid);

void DCE2_Smb2Write(DCE2_Smb2SsnData*, const Smb2Hdr*,
    const uint8_t* smb_data, const uint8_t* end, DCE2_Smb2SessionTracker* str,
    DCE2_Smb2TreeTracker* ttr, uint64_t mid);

void DCE2_Smb2SetInfo(DCE2_Smb2SsnData*, const Smb2Hdr*,
    const uint8_t* smb_data, const uint8_t* end, DCE2_Smb2TreeTracker* ttr);

bool DCE2_Smb2ProcessFileData(DCE2_Smb2SsnData*, const uint8_t* file_data,
    uint32_t data_size);

void DCE2_Smb2CloseCmd(DCE2_Smb2SsnData*, const Smb2Hdr*,
    const uint8_t* smb_data, const uint8_t* end, DCE2_Smb2TreeTracker* ttr,
    DCE2_Smb2SessionTracker* str, uint64_t mid);

void DCE2_Smb2Logoff(DCE2_Smb2SsnData*, const uint8_t* smb_data,
    const uint64_t sid);

DCE2_Smb2TreeTracker*find_tree_for_message(DCE2_Smb2SsnData*, const uint64_t);

bool DCE2_IsSmb2DurableReconnect(const Smb2CreateRequestHdr* smb_create_hdr, const uint8_t* end, uint64_t& file_id);

void DCE2_Smb2IoctlCommand(DCE2_Smb2SsnData* ssd, const Smb2Hdr* smb_hdr,
    const uint8_t* smb_data, const uint8_t* end,
    DCE2_Smb2SessionTracker* str, DCE2_Smb2TreeTracker* ttr, uint64_t mid);

#endif

dce_smb2.h

Path = src/service_inspectors/dce_rpc/dce_smb2.h

#ifndef DCE_SMB2_H
#define DCE_SMB2_H

#include "dce_db.h"
#include "dce_smb.h"
#include "hash/lru_cache_shared.h"
#include "flow/flow_key.h"
#include "main/thread_config.h"
#include "utils/util.h"

#define GET_CURRENT_PACKET snort::DetectionEngine::get_current_packet()

struct Smb2Hdr
{
    uint8_t smb_idf[4];       /* contains 0xFE,’SMB’ */
    uint16_t structure_size;  /* This MUST be set to 64 */
    uint16_t credit_charge;   /* # of credits that this request consumes */
    uint32_t status;          /* depends */
    uint16_t command;         /* command code  */
    uint16_t credit;          /* # of credits requesting/granted */
    uint32_t flags;           /* flags */
    uint32_t next_command;    /* used for compounded request */
    uint64_t message_id;      /* identifies a message uniquely on connection */
    uint32_t process_id;      /* Reserved */
    uint32_t tree_id;         /* Tree id*/
    uint64_t session_id;      /* identifies the established session for the command*/
    uint8_t signature[16];    /* signature of the message */
};

struct Smb2ASyncHdr
{
    uint8_t smb_idf[4];       /* contains 0xFE,’SMB’ */
    uint16_t structure_size;  /* This MUST be set to 64 */
    uint16_t credit_charge;   /* # of credits that this request consumes */
    uint32_t status;          /* depends */
    uint16_t command;         /* command code  */
    uint16_t credit;          /* # of credits requesting/granted */
    uint32_t flags;           /* flags */
    uint32_t next_command;    /* used for compounded request */
    uint64_t message_id;      /* identifies a message uniquely on connection */
    uint64_t async_id;        /* handle operations asynchronously */
    uint64_t session_id;      /* identifies the established session for the command*/
    uint8_t signature[16];    /* signature of the message */
};

struct Smb2SyncHdr
{
    uint8_t smb_idf[4];       /* contains 0xFE,’SMB’ */
    uint16_t structure_size;  /* This MUST be set to 64 */
    uint16_t credit_charge;   /* # of credits that this request consumes */
    uint32_t status;          /* depends */
    uint16_t command;         /* command code  */
    uint16_t credit;          /* # of credits requesting/granted */
    uint32_t flags;           /* flags */
    uint32_t next_command;    /* used for compounded request */
    uint64_t message_id;      /* identifies a message uniquely on connection */
    uint32_t reserved;        /* reserved */
    uint32_t tree_id;         /* identifies the tree connect for the command */
    uint64_t session_id;      /* identifies the established session for the command*/
    uint8_t signature[16];    /* signature of the message */
};

struct Smb2ErrorResponseHdr
{
    uint16_t structure_size;  /* This MUST be set to 9 */
    uint16_t reserved;        /* reserved */
    uint32_t byte_count;      /* The number of bytes of error_data */
    uint8_t error_data[1];    /* If byte_count is 0, this MUST be 0*/
};

class DCE2_Smb2TreeTracker;

class DCE2_Smb2RequestTracker
{
public:

    DCE2_Smb2RequestTracker() = delete;
    DCE2_Smb2RequestTracker(const DCE2_Smb2RequestTracker& arg) = delete;
    DCE2_Smb2RequestTracker& operator=(const DCE2_Smb2RequestTracker& arg) = delete;

    explicit DCE2_Smb2RequestTracker(uint64_t file_id_v, uint64_t offset_v = 0);
    DCE2_Smb2RequestTracker(char* fname_v, uint16_t fname_len_v);
    ~DCE2_Smb2RequestTracker();

    uint64_t get_offset()
    {
        return offset;
    }

    uint64_t get_file_id()
    {
        return file_id;
    }

    void set_file_id(uint64_t fid)
    {
        file_id = fid;
    }

    void set_tree_id(uint32_t l_tree_id)
    {
        tree_id = l_tree_id;
    }

    uint32_t get_tree_id()
    {
        return tree_id;
    }

    void set_session_id(uint32_t l_session_id)
    {
        session_id = l_session_id;
    }

    uint32_t get_session_id()
    {
        return session_id;
    }

    char* fname = nullptr;
    uint16_t fname_len = 0;
    uint64_t file_id = 0;
    uint64_t offset = 0;
    uint32_t tree_id = 0;
    uint64_t session_id = 0;
};

struct DCE2_Smb2SsnData;
class DCE2_Smb2SessionTracker;

class DCE2_Smb2FileTracker
{
public:

    DCE2_Smb2FileTracker() = delete;
    DCE2_Smb2FileTracker(const DCE2_Smb2FileTracker& arg) = delete;
    DCE2_Smb2FileTracker& operator=(const DCE2_Smb2FileTracker& arg) = delete;

    DCE2_Smb2FileTracker(uint64_t file_id_v, DCE2_Smb2TreeTracker* ttr_v,
        DCE2_Smb2SessionTracker* str_v, snort::Flow* flow_v);
    ~DCE2_Smb2FileTracker();
    uint64_t max_offset = 0;
    uint64_t file_id = 0;
    uint64_t file_size = 0;
    uint64_t file_name_hash = 0;
    DCE2_Smb2TreeTracker* ttr = nullptr;
    DCE2_Smb2SessionTracker* str = nullptr;
    snort::Flow* parent_flow = nullptr;
    DCE2_CoTracker* co_tracker = nullptr;
    bool ignore : 1;
    bool upload : 1;
    bool multi_channel_file : 1;
};

class DCE2_Smb2LocalFileTracker
{
public:
    uint64_t file_offset = 0;
    DCE2_SmbPduState smb2_pdu_state = DCE2_SMB_PDU_STATE__COMMAND;
};
typedef DCE2_DbMap<uint64_t, DCE2_Smb2FileTracker*, std::hash<uint64_t> > DCE2_DbMapFtracker;
typedef DCE2_DbMap<uint64_t, DCE2_Smb2RequestTracker*, std::hash<uint64_t> > DCE2_DbMapRtracker;
class DCE2_Smb2TreeTracker
{
public:

    DCE2_Smb2TreeTracker() = delete;
    DCE2_Smb2TreeTracker(const DCE2_Smb2TreeTracker& arg) = delete;
    DCE2_Smb2TreeTracker& operator=(const DCE2_Smb2TreeTracker& arg) = delete;

    DCE2_Smb2TreeTracker (uint32_t tid_v, uint8_t share_type_v);
    ~DCE2_Smb2TreeTracker();

    // File Tracker
    DCE2_Smb2FileTracker* findFtracker(uint64_t file_id)
    {
        return file_trackers.Find(file_id);
    }

    bool insertFtracker(uint64_t file_id, DCE2_Smb2FileTracker* ftracker)
    {
        return file_trackers.Insert(file_id, ftracker);
    }

    void removeFtracker(uint64_t file_id)
    {
        file_trackers.Remove(file_id);
    }

    // common methods
    uint8_t get_share_type()
    {
        return share_type;
    }

    uint32_t get_tid()
    {
        return tid;
    }

    uint8_t share_type = 0;
    uint32_t tid = 0;
    DCE2_DbMapFtracker file_trackers;
};

PADDING_GUARD_BEGIN

struct Smb2SidHashKey
{
    uint32_t cip[4] = {};
    uint32_t sip[4] = {};
    uint32_t mplsLabel = 0;
    int16_t cgroup = 0;
    int16_t sgroup = 0;
    uint32_t addressSpaceId = 0;
    uint16_t vlan_tag = 0;
    uint16_t padding = 0;
    uint64_t sid = 0;
    uint32_t tenant_id = 0;
    uint32_t padding2 = 0;  // NOTE: If this changes, change do_hash too

    bool operator==(const Smb2SidHashKey& other) const
    {
        return( cip[0] == other.cip[0] and
               cip[1] == other.cip[1] and
               cip[2] == other.cip[2] and
               cip[3] == other.cip[3] and
               sip[0] == other.sip[0] and
               sip[1] == other.sip[1] and
               sip[2] == other.sip[2] and
               sip[3] == other.sip[3] and
               mplsLabel == other.mplsLabel and
               cgroup == other.cgroup and
               sgroup == other.sgroup and
               addressSpaceId == other.addressSpaceId and
               vlan_tag == other.vlan_tag and
               sid == other.sid and
               tenant_id == other.tenant_id );
    }
};
PADDING_GUARD_END

//The below value is taken from Hash Key class static hash hardener
#define SMB_KEY_HASH_HARDENER 133824503

struct SmbKeyHash
{
    size_t operator()(const Smb2SidHashKey& key) const
    {
        return do_hash((const uint32_t*)&key);
    }

    size_t operator()(const snort::FlowKey& key) const
    {
        return do_hash_flow_key((const uint32_t*)&key);
    }

private:
    size_t do_hash(const uint32_t* d) const
    {
        uint32_t a, b, c;
        a = b = c = SMB_KEY_HASH_HARDENER;

        a += d[0];  // IPv6 cip[0]
        b += d[1];  // IPv6 cip[1]
        c += d[2];  // IPv6 cip[2]
        mix(a, b, c);

        a += d[3];  // IPv6 cip[3]
        b += d[4];  // IPv6 sip[0]
        c += d[5];  // IPv6 sip[1]
        mix(a, b, c);

        a += d[6];  // IPv6 sip[2]
        b += d[7];  // IPv6 sip[3]
        c += d[8];  // mpls label
        mix(a, b, c);

        a += d[9];  // cgroup and sgroup
        b += d[10]; // addressSpaceId
        c += d[11]; // vlan_tag, padding
        mix(a, b, c);

        a += d[12]; // sid[0]
        b += d[13]; // sid[1]
        c += d[14]; // tenant_id

        // padding2 is ignored.
        finalize(a, b, c);

        return c;
    }

    size_t do_hash_flow_key(const uint32_t* d) const
    {
        uint32_t a, b, c;
        a = b = c = SMB_KEY_HASH_HARDENER;

        a += d[0];   // IPv6 lo[0]
        b += d[1];   // IPv6 lo[1]
        c += d[2];   // IPv6 lo[2]

        mix(a, b, c);

        a += d[3];   // IPv6 lo[3]
        b += d[4];   // IPv6 hi[0]
        c += d[5];   // IPv6 hi[1]

        mix(a, b, c);

        a += d[6];   // IPv6 hi[2]
        b += d[7];   // IPv6 hi[3]
        c += d[8];   // mpls label

        mix(a, b, c);

        a += d[9];   // addressSpaceId
        b += d[10];  // port lo & port hi
        c += d[11];  // group lo & group hi

        mix(a, b, c);

        a += d[12];  // vlan & pad
        b += d[13];  // ip_proto, pkt_type, version, flags

        finalize(a, b, c);

        return c;
    }
};

typedef DCE2_DbMap<uint32_t, DCE2_Smb2TreeTracker*, std::hash<uint32_t> > DCE2_DbMapTtracker;
typedef DCE2_DbMap<uint32_t, DCE2_Smb2SsnData*, std::hash<uint32_t> > DCE2_DbMapConntracker;
class DCE2_Smb2SessionTracker
{
public:

    DCE2_Smb2SessionTracker() = delete;
    DCE2_Smb2SessionTracker(uint64_t sid);
    ~DCE2_Smb2SessionTracker();

    void removeSessionFromAllConnection();
    void update_cache_size(int size);

    // tree tracker
    bool insertTtracker(uint32_t tree_id, DCE2_Smb2TreeTracker* ttr)
    {
        update_cache_size(sizeof(DCE2_Smb2TreeTracker));
        return tree_trackers.Insert(tree_id, ttr);
    }

    DCE2_Smb2TreeTracker* findTtracker(uint32_t tree_id)
    {
        return tree_trackers.Find(tree_id);
    }

    void removeTtracker(uint32_t tree_id)
    {
        update_cache_size(-(int)sizeof(DCE2_Smb2TreeTracker));
        tree_trackers.Remove(tree_id);
    }

    // ssd tracker
    bool insertConnectionTracker(const uint32_t key, DCE2_Smb2SsnData* ssd)
    {
        return conn_trackers.Insert(key, ssd);
    }

    DCE2_Smb2SsnData* findConnectionTracker(const uint32_t key)
    {
        return conn_trackers.Find(key);
    }

    void removeConnectionTracker(const uint32_t key)
    {
        conn_trackers.Remove(key);
    }

    int getConnTrackerSize()
    {
        return conn_trackers.GetSize();
    }

    void set_encryption_flag(bool flag)
    {
        if (flag)
            encryption_flag++;
        if (encryption_flag == 1)
            dce2_smb_stats.total_encrypted_sessions++;
    }

    bool get_encryption_flag() { return static_cast<bool>(encryption_flag); }

    DCE2_DbMapConntracker conn_trackers;
    DCE2_DbMapTtracker tree_trackers;
    uint64_t session_id = 0;
    uint8_t encryption_flag = 0;
};
struct DCE2_Smb2SsnData
{
    DCE2_SsnData sd;  // This member must be first
    uint8_t smb_id;
    DCE2_Policy policy;
    int dialect_index;
    int ssn_state_flags;
    int64_t max_file_depth; // Maximum file depth as returned from file API
    int16_t max_outstanding_requests; // Maximum number of request that can stay pending
    std::unordered_map<uint64_t, std::weak_ptr<DCE2_Smb2SessionTracker>, std::hash<uint64_t> > session_trackers;
    DCE2_Smb2FileTracker* ftracker_tcp; //To keep tab of current file being transferred over TCP
    std::unique_ptr<DCE2_Smb2LocalFileTracker> ftracker_local;
    DCE2_DbMapRtracker req_trackers;
    uint32_t flow_key;
    snort::Flow* flow = nullptr;
    DCE2_Smb2SsnData();
    ~DCE2_Smb2SsnData();
    void set_reassembled_data(uint8_t* nb_ptr, uint16_t co_len);

    std::shared_ptr<DCE2_Smb2SessionTracker> find_session_tracker(uint64_t session_id)
    {
        auto session_iter = session_trackers.find(session_id);
        if (session_iter != session_trackers.end())
        {
            return session_iter->second.lock();
        }
        return nullptr;
    }

    void remove_session_tracker(uint64_t session_id)
    {
        session_trackers.erase(session_id);
    }

    bool insert_session_tracker(uint64_t session_id, std::shared_ptr<DCE2_Smb2SessionTracker> sptr)
    {
        return session_trackers.insert(std::make_pair(session_id, sptr)).second;
    }

    // Request Tracker
    DCE2_Smb2RequestTracker* findRtracker(uint64_t mid)
    {
        return req_trackers.Find(mid);
    }

    bool insertRtracker(uint64_t message_id, DCE2_Smb2RequestTracker* rtracker)
    {
        return req_trackers.Insert(message_id, rtracker);
    }

    void removeRtracker(uint64_t message_id)
    {
        req_trackers.Remove(message_id);
    }

    int getTotalRequestsPending()
    {
        return req_trackers.GetSize();
    }

    DCE2_Smb2TreeTracker* GetTreeTrackerFromMessage(uint64_t mid)
    {
        DCE2_Smb2RequestTracker* request_tracker = findRtracker(mid);
        if (request_tracker)
        {
            auto session_tracker = find_session_tracker(request_tracker->get_session_id());
            if (session_tracker)
            {
                return session_tracker->findTtracker(request_tracker->get_tree_id());
            }
        }
        return nullptr;
    }
};

/* SMB2 command codes */
#define SMB2_COM_NEGOTIATE        0x00
#define SMB2_COM_SESSION_SETUP    0x01
#define SMB2_COM_LOGOFF           0x02
#define SMB2_COM_TREE_CONNECT     0x03
#define SMB2_COM_TREE_DISCONNECT  0x04
#define SMB2_COM_CREATE           0x05
#define SMB2_COM_CLOSE            0x06
#define SMB2_COM_FLUSH            0x07
#define SMB2_COM_READ             0x08
#define SMB2_COM_WRITE            0x09
#define SMB2_COM_LOCK             0x0A
#define SMB2_COM_IOCTL            0x0B
#define SMB2_COM_CANCEL           0x0C
#define SMB2_COM_ECHO             0x0D
#define SMB2_COM_QUERY_DIRECTORY  0x0E
#define SMB2_COM_CHANGE_NOTIFY    0x0F
#define SMB2_COM_QUERY_INFO       0x10
#define SMB2_COM_SET_INFO         0x11
#define SMB2_COM_OPLOCK_BREAK     0x12
#define SMB2_COM_MAX              0x13

struct Smb2NegotiateResponseHdr
{
    uint16_t structure_size;
    uint16_t security_mode;
    uint16_t dialect_revision;
    uint16_t negotiate_context_count;
    uint64_t servier_guid[2];
    uint32_t capabilities;
    uint32_t max_transaction_size;
    uint32_t max_read_size;
    uint32_t max_write_size;
    uint64_t system_time;
    uint64_t server_start_time;
    uint16_t security_buffer_offset;
    uint16_t security_buffer_length;
};

struct Smb2WriteRequestHdr
{
    uint16_t structure_size;  /* This MUST be set to 49 */
    uint16_t data_offset;     /* offset in bytes from the beginning of smb2 header */
    uint32_t length;          /* length of data being written in bytes */
    uint64_t offset;          /* offset in the destination file */
    uint64_t fileId_persistent;  /* fileId that is persistent */
    uint64_t fileId_volatile;    /* fileId that is volatile */
    uint32_t channel;            /* channel */
    uint32_t remaining_bytes;    /* subsequent bytes the client intends to write*/
    uint16_t write_channel_info_offset;      /* channel data info */
    uint16_t write_channel_info_length;      /* channel data info */
    uint32_t flags;      /* flags*/
};

struct Smb2WriteResponseHdr
{
    uint16_t structure_size;  /* This MUST be set to 17 */
    uint16_t reserved;        /* reserved */
    uint32_t count;           /* The number of bytes written */
    uint32_t remaining;       /* MUST be 0*/
    uint16_t write_channel_info_offset;      /* channel data info */
    uint16_t write_channel_info_length;      /* channel data info */
};

struct Smb2ReadRequestHdr
{
    uint16_t structure_size;  /* This MUST be set to 49 */
    uint8_t padding;          /* Padding */
    uint8_t flags;            /* Flags */
    uint32_t length;          /* length of data to read from the file */
    uint64_t offset;          /* offset in the destination file */
    uint64_t fileId_persistent;  /* fileId that is persistent */
    uint64_t fileId_volatile;    /* fileId that is volatile */
    uint32_t minimum_count;      /* The minimum # of bytes to be read */
    uint32_t channel;            /* channel */
    uint32_t remaining_bytes;    /* subsequent bytes the client intends to read*/
    uint16_t read_channel_info_offset;      /* channel data info */
    uint16_t read_channel_info_length;      /* channel data info */
};

struct Smb2ReadResponseHdr
{
    uint16_t structure_size; /* This MUST be set to 17 */
    uint8_t data_offset;     /* offset in bytes from beginning of smb2 header*/
    uint8_t reserved;        /* reserved */
    uint32_t length;         /* The number of bytes being returned in response */
    uint32_t remaining;      /* The number of data being sent on the channel*/
    uint32_t reserved2;      /* reserved */
};

struct Smb2SetInfoRequestHdr
{
    uint16_t structure_size;   /* This MUST be set to 33 */
    uint8_t info_type;         /* info type */
    uint8_t file_info_class;   /* file info class after header */
    uint32_t buffer_length;    /* buffer length */
    uint16_t buffer_offset;    /* buffer offset */
    uint16_t reserved;         /* reserved */
    uint32_t additional_info;  /* additional information */
    uint64_t fileId_persistent; /* fileId that is persistent */
    uint64_t fileId_volatile;  /* fileId that is volatile */
};

struct Smb2CreateRequestHdr
{
    uint16_t structure_size;          /* This MUST be set to 57 */
    uint8_t security_flags;           /* security flag, should be 0 */
    uint8_t requested_oplock_level;   /* */
    uint32_t impersonation_level;     /* */
    uint64_t smb_create_flags;        /* should be 0 */
    uint64_t reserved;                /* can be any value */
    uint32_t desired_access;          /*  */
    uint32_t file_attributes;         /* */
    uint32_t share_access;            /* READ WRITE DELETE etc */
    uint32_t create_disposition;      /* actions when file exists*/
    uint32_t create_options;          /* options for creating file*/
    uint16_t name_offset;             /* file name offset from SMB2 header */
    uint16_t name_length;             /* length of file name */
    uint32_t create_contexts_offset;  /* offset of contexts from beginning of header */
    uint32_t create_contexts_length;  /* length of contexts */
};

// file attribute for create response
#define SMB2_CREATE_RESPONSE_DIRECTORY 0x10

struct Smb2CreateResponseHdr
{
    uint16_t structure_size;          /* This MUST be set to 89 */
    uint8_t oplock_level;             /* oplock level granted, values limited */
    uint8_t flags;                    /* flags, values limited */
    uint32_t create_action;           /* action taken, values limited */
    uint64_t creation_time;           /* time created */
    uint64_t last_access_time;        /* access time */
    uint64_t last_write_time;         /* write  time */
    uint64_t change_time;             /* time modified*/
    uint64_t allocation_size;         /* size allocated */
    uint64_t end_of_file;             /* file size*/
    uint32_t file_attributes;         /* attributes of the file*/
    uint32_t reserved2;               /* */
    uint64_t fileId_persistent;       /* fileId that is persistent */
    uint64_t fileId_volatile;         /* fileId that is volatile */
    uint32_t create_contexts_offset;  /*  */
    uint32_t create_contexts_length;  /*  */
};

struct Smb2CloseRequestHdr
{
    uint16_t structure_size;          /* This MUST be set to 24 */
    uint16_t flags;                   /* flags */
    uint32_t reserved;                /* can be any value */
    uint64_t fileId_persistent;       /* fileId that is persistent */
    uint64_t fileId_volatile;         /* fileId that is volatile */
};

#define SMB2_SHARE_TYPE_DISK  0x01
#define SMB2_SHARE_TYPE_PIPE  0x02
#define SMB2_SHARE_TYPE_PRINT 0x03

struct Smb2TreeConnectResponseHdr
{
    uint16_t structure_size;          /* This MUST be set to 16 */
    uint8_t share_type;               /* type of share being accessed */
    uint8_t reserved;                 /* reserved */
    uint32_t share_flags;             /* properties for this share*/
    uint32_t capabilities;            /* various capabilities for this share */
    uint32_t maximal_access;          /* maximal access for the user */
};

struct Smb2TreeDisConnectHdr
{
    uint16_t structure_size;          /* This MUST be set to 4 */
    uint16_t reserved;                 /* reserved */
};

struct  Smb2SetupRequestHdr
{
    uint16_t structure_size;            /* This MUST be set to 25 (0x19) bytes */
    uint8_t flags;
    uint8_t security_mode;
    uint32_t capabilities;
    uint32_t channel;
    uint16_t secblob_ofs;
    uint16_t secblob_size;
    uint64_t previous_sessionid;
};

struct Smb2SetupResponseHdr
{
    uint16_t structure_size;            /* This MUST be set to 9 (0x09) bytes */
    uint16_t session_flags;
    uint16_t secblob_ofs;
    uint16_t secblob_size;
};

struct Smb2CreateContextHdr
{
    uint32_t next;            /* next context header*/
    uint16_t name_offset;     /* name offset */
    uint16_t name_length;     /* name length */
    uint16_t reserved;        /* reserved */
    uint16_t data_offset;     /* data offset */
    uint32_t data_length;     /* data length */
};

struct Smb2IoctlRequestHdr
{
    uint16_t structure_size;          /* This MUST be set to 57 */
    uint16_t reserved;
    uint32_t ctl_code;
    uint64_t fileId_persistent;       /* fileId that is persistent */
    uint64_t fileId_volatile;
    uint32_t input_offset;
    uint32_t input_count;
    uint32_t max_input_response;
    uint32_t output_offset;
    uint32_t output_count;
    uint32_t max_output_response;
    uint32_t flags;
    uint32_t reserved2;
};

struct Smb2IoctlResponseHdr
{
    uint16_t structure_size;           /* This MUST be set to 49 */
    uint16_t reserved;
    uint32_t ctl_code;
    uint64_t fileId_persistent;       /* fileId that is persistent */
    uint64_t fileId_volatile;
    uint32_t input_offset;
    uint32_t input_count;
    uint32_t output_offset;
    uint32_t output_count;
    uint32_t flags;
    uint32_t reserved2;
};

#define SMB2_HEADER_LENGTH 64
#define SMB2_GLOBAL_CAP_MULTI_CHANNEL 0x08

#define SMB2_ERROR_RESPONSE_STRUC_SIZE 9
#define SMB2_NEGOTIATE_RESPONSE_STRUC_SIZE 65

#define SMB2_CREATE_REQUEST_STRUC_SIZE 57
#define SMB2_CREATE_RESPONSE_STRUC_SIZE 89
#define SMB2_CREATE_REQUEST_DATA_OFFSET 120

#define SMB2_CLOSE_REQUEST_STRUC_SIZE 24
#define SMB2_CLOSE_RESPONSE_STRUC_SIZE 60

#define SMB2_WRITE_REQUEST_STRUC_SIZE 49
#define SMB2_WRITE_RESPONSE_STRUC_SIZE 17

#define SMB2_READ_REQUEST_STRUC_SIZE 49
#define SMB2_READ_RESPONSE_STRUC_SIZE 17

#define SMB2_SET_INFO_REQUEST_STRUC_SIZE 33
#define SMB2_SET_INFO_RESPONSE_STRUC_SIZE 2

#define SMB2_TREE_CONNECT_REQUEST_STRUC_SIZE 9
#define SMB2_TREE_CONNECT_RESPONSE_STRUC_SIZE 16
#define SMB2_TREE_DISCONNECT_REQUEST_STRUC_SIZE 4

#define SMB2_FILE_ALLOCATION_INFO 0x13
#define SMB2_FILE_ENDOFFILE_INFO 0x14

#define SMB2_SETUP_REQUEST_STRUC_SIZE 25
#define SMB2_SETUP_RESPONSE_STRUC_SIZE 9

#define SMB2_IOCTL_REQUEST_STRUC_SIZE 57
#define SMB2_IOCTL_RESPONSE_STRUC_SIZE 49

#define SMB2_LOGOFF_REQUEST_STRUC_SIZE 4

#define SMB2_FLAGS_ASYNC_COMMAND  0x00000002

#define SMB2_STATUS_PENDING  0x00000103

#define FSCTL_PIPE_WAIT 0x00110018
#define FSCTL_PIPE_TRANSCEIVE 0x0011C017
#define FSCTL_PIPE_PEEK 0x0011400C

#define SMB2_CREATE_DURABLE_RECONNECT "DHnC"
#define SMB2_CREATE_DURABLE_RECONNECT_V2 "DH2C"

extern const char* smb2_command_string[SMB2_COM_MAX];
/* Process smb2 message */
void DCE2_Smb2Process(DCE2_Smb2SsnData* ssd);

/* Check smb version based on smb header */
DCE2_SmbVersion DCE2_Smb2Version(const snort::Packet* p);

#endif  /* _DCE_SMB2_H_ */

dce_smb2_utils.h

Path = src/service_inspectors/dce_rpc/dce_smb2_utils.h

#ifndef DCE_SMB2_UTILS_H
#define DCE_SMB2_UTILS_H

#include "dce_smb.h"
#include "dce_smb2.h"
#include "file_api/file_flows.h"

Smb2SidHashKey get_key(uint64_t sid);

template<typename Key, typename Value, typename Hash>
class LruCacheLocalSmbMultiChannel : public LruCacheLocal <Key, Value, Hash>
{
public:
    using LruLocal = LruCacheLocal<Key, Value, Hash>;
    using LruLocal::current_size;
    using LruLocal::max_size;
    using LruLocal::list;
    using LruLocal::map;
    using LruLocal::stats;
    using LruMapIter = typename LruLocal::LruMapIter;

    LruCacheLocalSmbMultiChannel(const size_t sz, struct LruCacheLocalStats& st) :   LruLocal(sz, st) { }

    void update(int size)
    {
        if ( size < 0 )
            assert(current_size >= (size_t)-size);
        current_size += size;
        if ( size > 0)
        {
            // Checking 1+ size prevents crash if max_size is too low to hold even a single entry
            if ( current_size > max_size and list.size() > 1 )
                LruLocal::prune();
            if ( stats.cache_max < current_size )
                stats.cache_max = current_size;
        }
    }

    Value find(Key& key)
    {
        LruMapIter it = map.find(key);
        if (it == map.end())
            return nullptr;
        //Move entry to front of LruList
        list.splice(list.begin(), list, it->second);
        return list.begin()->second;
    }
};

typedef LruCacheLocalSmbMultiChannel<Smb2SidHashKey, std::shared_ptr<DCE2_Smb2SessionTracker>,
    SmbKeyHash> SmbSessionCache;

extern THREAD_LOCAL SmbSessionCache* smb2_session_cache;

// SMB2 Session cache manipulation functions
inline void DCE2_SmbSessionCacheInit(const size_t cache_size)
{
    smb2_session_cache = new SmbSessionCache(cache_size, dce2_smb_stats);
}

inline void DCE2_SmbSessionCacheUpdateSize(int size)
{
    smb2_session_cache->update(size);
}

inline std::shared_ptr<DCE2_Smb2SessionTracker> DCE2_SmbSessionCacheFindElseCreate(uint64_t sid)
{
    bool is_new = false;
    auto& session_ptr =  smb2_session_cache->find_else_create(get_key(sid), &is_new);
    if (is_new)
    {
        session_ptr.reset(new DCE2_Smb2SessionTracker(sid));
    }
    return session_ptr;
}

inline bool DCE2_SmbSessionCacheRemove(const Smb2SidHashKey& key)
{
    return smb2_session_cache->remove(key);
}

// SMB2 functions for fetching sid, tid, request type and so on.
inline uint64_t Smb2Sid(const Smb2Hdr* hdr)
{
    return snort::alignedNtohq(&(((const Smb2SyncHdr*)hdr)->session_id));
}

inline uint32_t Smb2Tid(const Smb2Hdr* hdr)
{
    return snort::alignedNtohl(&(((const Smb2SyncHdr*)hdr)->tree_id));
}

inline uint64_t Smb2Mid(const Smb2Hdr* hdr)
{
    return snort::alignedNtohq(&(((const Smb2SyncHdr*)hdr)->message_id));
}

inline bool Smb2Error(const Smb2Hdr* hdr)
{
    return (hdr->status != SMB_NT_STATUS__SUCCESS);
}

inline std::shared_ptr<DCE2_Smb2SessionTracker> DCE2_Smb2FindSidInSsd(DCE2_Smb2SsnData* ssd, const
    uint64_t sid)
{
    return ssd->find_session_tracker(sid);
}

inline void DCE2_Smb2InsertSidInSsd(DCE2_Smb2SsnData* ssd, const uint64_t sid,
    std::shared_ptr<DCE2_Smb2SessionTracker> stracker)
{
    // add ssd in session tracker's tcp trackers database
    if (ssd->flow_key)
    {
        stracker->insertConnectionTracker(ssd->flow_key, ssd);
    }
    ssd->insert_session_tracker(sid, stracker);
}

inline void DCE2_Smb2RemoveSidInSsd(DCE2_Smb2SsnData* ssd, const uint64_t sid)
{
    ssd->remove_session_tracker(sid);
}

DCE2_Smb2TreeTracker* DCE2_Smb2InsertTid(DCE2_Smb2SsnData*, const uint32_t tid, uint8_t share_type,
    DCE2_Smb2SessionTracker*);

DCE2_Smb2SessionTracker* DCE2_Smb2FindElseCreateSid(DCE2_Smb2SsnData*, const uint64_t sid, bool
    force_cache_update = true);

#endif

dce_smb_commands.h

Path = src/service_inspectors/dce_rpc/dce_smb_commands.h

#ifndef DCE_SMB_COMMANDS_H
#define DCE_SMB_COMMANDS_H

#include "dce_smb_utils.h"

#define SMB2_GLOBAL_CAP_MULTI_CHANNEL 0x08

DCE2_Ret DCE2_SmbOpen(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbCreate(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbClose(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbRename(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbRead(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbWrite(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbCreateNew(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbLockAndRead(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbWriteAndUnlock(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbOpenAndX(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbReadAndX(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbWriteAndX(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbSessionSetupAndX(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbNegotiate(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbTreeConnectAndX(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbTreeConnect(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbNtCreateAndX(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbTreeDisconnect(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbLogoffAndX(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbReadRaw(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbWriteRaw(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbWriteComplete(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbWriteAndClose(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);

#endif

dce_smb.h

Path = src/service_inspectors/dce_rpc/dce_smb.h

#ifndef DCE_SMB_H
#define DCE_SMB_H

#include "dce_co.h"
#include "hash/lru_cache_local.h"
#include "protocols/packet.h"
#include "profiler/profiler_defs.h"
#include "smb_common.h"
#include "smb_message.h"

#include "trace/trace_api.h"
#define SMB_DEBUG(module_name, module_id, log_level, p, ...) \
    trace_logf(log_level, module_name, module_id, p, __VA_ARGS__)

#define DCE2_SMB_NAME "dce_smb"
#define DCE2_SMB_HELP "dce over smb inspection"
#define DCE2_SMB_RPKT_TYPE_MAX 4
#define DCE2_SMB_RPKT_TYPE_START 1

#define DCE2_SMB_BAD_NBSS_TYPE 2
#define DCE2_SMB_BAD_TYPE 3
#define DCE2_SMB_BAD_ID 4
#define DCE2_SMB_BAD_WCT 5
#define DCE2_SMB_BAD_BCC 6
#define DCE2_SMB_BAD_FORM 7
#define DCE2_SMB_BAD_OFF 8
#define DCE2_SMB_TDCNT_ZE 9
#define DCE2_SMB_NB_LT_SMBHDR 10
#define DCE2_SMB_NB_LT_COM 11
#define DCE2_SMB_NB_LT_BCC 12
#define DCE2_SMB_NB_LT_DSIZE 13
#define DCE2_SMB_TDCNT_LT_DSIZE 14
#define DCE2_SMB_DSENT_GT_TDCNT 15
#define DCE2_SMB_BCC_LT_DSIZE 16
#define DCE2_SMB_INVALID_DSIZE 17
#define DCE2_SMB_EXCESSIVE_TREE_CONNECTS 18
#define DCE2_SMB_EXCESSIVE_READS 19
#define DCE2_SMB_EXCESSIVE_CHAINING 20
#define DCE2_SMB_MULT_CHAIN_SS 21
#define DCE2_SMB_MULT_CHAIN_TC 22
#define DCE2_SMB_CHAIN_SS_LOGOFF 23
#define DCE2_SMB_CHAIN_TC_TDIS 24
#define DCE2_SMB_CHAIN_OPEN_CLOSE 25
#define DCE2_SMB_INVALID_SHARE 26

#define DCE2_SMB_V1 44
#define DCE2_SMB_V2 45
#define DCE2_SMB_INVALID_BINDING 46
#define DCE2_SMB2_EXCESSIVE_COMPOUNDING 47
#define DCE2_SMB_DCNT_ZERO 48
#define DCE2_SMB_DCNT_MISMATCH 49
#define DCE2_SMB_MAX_REQS_EXCEEDED 50
#define DCE2_SMB_REQS_SAME_MID 51
#define DCE2_SMB_DEPR_DIALECT_NEGOTIATED 52
#define DCE2_SMB_DEPR_COMMAND_USED 53
#define DCE2_SMB_UNUSUAL_COMMAND_USED 54
#define DCE2_SMB_INVALID_SETUP_COUNT 55
#define DCE2_SMB_MULTIPLE_NEGOTIATIONS 56
#define DCE2_SMB_EVASIVE_FILE_ATTRS 57
#define DCE2_SMB_INVALID_FILE_OFFSET 58
#define DCE2_SMB_BAD_NEXT_COMMAND_OFFSET 59

#define DCE2_SMB_BAD_NBSS_TYPE_STR "SMB - bad NetBIOS session service session type"
#define DCE2_SMB_BAD_TYPE_STR  "SMB - bad SMB message type"
#define DCE2_SMB_BAD_ID_STR "SMB - bad SMB Id (not \\xffSMB for SMB1 or not \\xfeSMB for SMB2)"
#define DCE2_SMB_BAD_WCT_STR "SMB - bad word count or structure size"
#define DCE2_SMB_BAD_BCC_STR  "SMB - bad byte count"
#define DCE2_SMB_BAD_FORM_STR  "SMB - bad format type"
#define DCE2_SMB_BAD_OFF_STR  "SMB - bad offset"
#define DCE2_SMB_TDCNT_ZE_STR  "SMB - zero total data count"
#define DCE2_SMB_NB_LT_SMBHDR_STR "SMB - NetBIOS data length less than SMB header length"
#define DCE2_SMB_NB_LT_COM_STR   "SMB - remaining NetBIOS data length less than command length"
#define DCE2_SMB_NB_LT_BCC_STR  "SMB - remaining NetBIOS data length less than command byte count"
#define DCE2_SMB_NB_LT_DSIZE_STR \
    "SMB - remaining NetBIOS data length less than command data size"
#define DCE2_SMB_TDCNT_LT_DSIZE_STR \
    "SMB - remaining total data count less than this command data size"
#define DCE2_SMB_DSENT_GT_TDCNT_STR \
    "SMB - total data sent (STDu64) greater than command total data expected"
#define DCE2_SMB_BCC_LT_DSIZE_STR   "SMB - byte count less than command data size (STDu64)"
#define DCE2_SMB_INVALID_DSIZE_STR  "SMB - invalid command data size for byte count"
#define DCE2_SMB_EXCESSIVE_TREE_CONNECTS_STR \
    "SMB - excessive tree connect requests with pending tree connect responses"
#define DCE2_SMB_EXCESSIVE_READS_STR  "SMB - excessive read requests with pending read responses"
#define DCE2_SMB_EXCESSIVE_CHAINING_STR  "SMB - excessive command chaining"
#define DCE2_SMB_MULT_CHAIN_SS_STR   "SMB - multiple chained tree connect requests"
#define DCE2_SMB_MULT_CHAIN_TC_STR   "SMB - multiple chained tree connect requests"
#define DCE2_SMB_CHAIN_SS_LOGOFF_STR   "SMB - chained/compounded login followed by logoff"
#define DCE2_SMB_CHAIN_TC_TDIS_STR \
    "SMB - chained/compounded tree connect followed by tree disconnect"
#define DCE2_SMB_CHAIN_OPEN_CLOSE_STR \
    "SMB - chained/compounded open pipe followed by close pipe"
#define DCE2_SMB_INVALID_SHARE_STR   "SMB - invalid share access"

#define DCE2_SMB_V1_STR  "SMB - invalid SMB version 1 seen"
#define DCE2_SMB_V2_STR  "SMB - invalid SMB version 2 seen"
#define DCE2_SMB_INVALID_BINDING_STR "SMB - invalid user, tree connect, file binding"
#define DCE2_SMB2_EXCESSIVE_COMPOUNDING_STR  "SMB - excessive command compounding"
#define DCE2_SMB_DCNT_ZERO_STR   "SMB - zero data count"
#define DCE2_SMB_DCNT_MISMATCH_STR "SMB - data count mismatch in command and format"
#define DCE2_SMB_MAX_REQS_EXCEEDED_STR  "SMB - maximum number of outstanding requests exceeded"
#define DCE2_SMB_REQS_SAME_MID_STR "SMB - outstanding requests with same MID"
#define DCE2_SMB_DEPR_DIALECT_NEGOTIATED_STR  "SMB - deprecated dialect negotiated"
#define DCE2_SMB_DEPR_COMMAND_USED_STR   "SMB - deprecated command used"
#define DCE2_SMB_UNUSUAL_COMMAND_USED_STR "SMB - unusual command used"
#define DCE2_SMB_INVALID_SETUP_COUNT_STR  "SMB - invalid setup count for command"
#define DCE2_SMB_MULTIPLE_NEGOTIATIONS_STR \
    "SMB - client attempted multiple dialect negotiations on session"
#define DCE2_SMB_EVASIVE_FILE_ATTRS_STR \
    "SMB - client attempted to create or set a file's attributes to readonly/hidden/system"
#define DCE2_SMB_INVALID_FILE_OFFSET_STR \
    "SMB - file offset provided is greater than file size specified"
#define DCE2_SMB_BAD_NEXT_COMMAND_OFFSET_STR \
    "SMB - next command specified in SMB2 header is beyond payload boundary"

struct dce2SmbStats : public LruCacheLocalStats
{
    PegCount events;

    PegCount co_pdus;
    PegCount co_bind;
    PegCount co_bind_ack;
    PegCount co_alter_ctx;
    PegCount co_alter_ctx_resp;
    PegCount co_bind_nack;
    PegCount co_request;
    PegCount co_response;
    PegCount co_cancel;
    PegCount co_orphaned;
    PegCount co_fault;
    PegCount co_auth3;
    PegCount co_shutdown;
    PegCount co_reject;
    PegCount co_ms_pdu;
    PegCount co_other_req;
    PegCount co_other_resp;
    PegCount co_req_fragments;
    PegCount co_resp_fragments;
    PegCount co_cli_max_frag_size;
    PegCount co_cli_min_frag_size;
    PegCount co_cli_seg_reassembled;
    PegCount co_cli_frag_reassembled;
    PegCount co_srv_max_frag_size;
    PegCount co_srv_min_frag_size;
    PegCount co_srv_seg_reassembled;
    PegCount co_srv_frag_reassembled;

    PegCount smb_sessions;
    PegCount smb_pkts;
    PegCount smb_ignored_bytes;
    PegCount smb_cli_seg_reassembled;
    PegCount smb_srv_seg_reassembled;
    PegCount smb_max_outstanding_requests;
    //  FIXIT-M more peg count foo
    /*uint64_t smb_com_stats[2][SMB_MAX_NUM_COMS];
    uint64_t smb_chained_stats[2][SMB_ANDX_COM__MAX][SMB_MAX_NUM_COMS];
    // The +1 is for codes beyond the range of the highest valid subcommand code
    // Indicates a bogus subcommand
    uint64_t smb_trans_subcom_stats[2][TRANS_SUBCOM_MAX+1];
    uint64_t smb_trans2_subcom_stats[2][TRANS2_SUBCOM_MAX+1];
    uint64_t smb_nt_transact_subcom_stats[2][NT_TRANSACT_SUBCOM_MAX+1];
    */
    PegCount smb_files_processed;
    /* SMB2 stats */
    PegCount v2_setup;
    PegCount v2_setup_err_resp;
    PegCount v2_setup_inv_str_sz;
    PegCount v2_setup_resp_hdr_err;
    PegCount v2_tree_cnct;
    PegCount v2_tree_cnct_err_resp;
    PegCount v2_tree_cnct_ignored;
    PegCount v2_tree_cnct_inv_str_sz;
    PegCount v2_tree_cnct_resp_hdr_err;
    PegCount v2_crt;
    PegCount v2_crt_err_resp;
    PegCount v2_crt_inv_file_data;
    PegCount v2_crt_inv_str_sz;
    PegCount v2_crt_resp_hdr_err;
    PegCount v2_crt_req_hdr_err;
    PegCount v2_crt_rtrkr_misng;
    PegCount v2_crt_req_ipc;
    PegCount v2_crt_tree_trkr_misng;
    PegCount v2_wrt;
    PegCount v2_wrt_err_resp;
    PegCount v2_wrt_ignored;
    PegCount v2_wrt_inv_str_sz;
    PegCount v2_wrt_req_hdr_err;
    PegCount v2_read;
    PegCount v2_read_err_resp;
    PegCount v2_read_ignored;
    PegCount v2_read_inv_str_sz;
    PegCount v2_read_rtrkr_misng;
    PegCount v2_read_resp_hdr_err;
    PegCount v2_read_req_hdr_err;
    PegCount v2_setinfo;
    PegCount v2_stinf_err_resp;
    PegCount v2_stinf_ignored;
    PegCount v2_stinf_inv_str_sz;
    PegCount v2_stinf_req_ftrkr_misng;
    PegCount v2_stinf_req_hdr_err;
    PegCount v2_cls;
    PegCount v2_cls_err_resp;
    PegCount v2_cls_ignored;
    PegCount v2_cls_inv_str_sz;
    PegCount v2_cls_req_ftrkr_misng;
    PegCount v2_cls_req_hdr_err;
    PegCount v2_tree_discn;
    PegCount v2_tree_discn_ignored;
    PegCount v2_tree_discn_inv_str_sz;
    PegCount v2_tree_discn_req_hdr_err;
    PegCount v2_logoff;
    PegCount v2_logoff_inv_str_sz;
    PegCount v2_hdr_err;
    PegCount v2_bad_next_cmd_offset;
    PegCount v2_extra_file_data_err;
    PegCount v2_inv_file_ctx_err;
    PegCount v2_msgs_uninspected;
    PegCount v2_cmpnd_req_lt_crossed;
    PegCount concurrent_sessions;
    PegCount max_concurrent_sessions;
    PegCount total_smb1_sessions;
    PegCount total_smb2_sessions;
    PegCount total_encrypted_sessions;
    PegCount total_mc_sessions;
    PegCount v2_total_session_trackers;
    PegCount v2_total_tree_trackers;
    PegCount v2_total_file_trackers;
    PegCount v2_updated_file_flows;
    PegCount v2_ignored_file_processing;
    PegCount v2_mc_file_transfers;
    PegCount v2_ioctl;
    PegCount v2_ioctl_err_resp;
    PegCount v2_ioctl_inv_str_sz;
    PegCount v2_ioctl_ignored;
    PegCount total_sessions; // not used
};

extern THREAD_LOCAL dce2SmbStats dce2_smb_stats;
extern THREAD_LOCAL snort::ProfileStats dce2_smb_pstat_main;

enum DCE2_SmbSsnState
{
    DCE2_SMB_SSN_STATE__START         = 0x00,
    DCE2_SMB_SSN_STATE__NEGOTIATED    = 0x01,
    DCE2_SMB_SSN_STATE__FP_CLIENT     = 0x02,  // Fingerprinted client
    DCE2_SMB_SSN_STATE__FP_SERVER     = 0x04   // Fingerprinted server
};

enum DCE2_SmbDataState
{
    DCE2_SMB_DATA_STATE__NETBIOS_HEADER,
    DCE2_SMB_DATA_STATE__SMB_HEADER,
    DCE2_SMB_DATA_STATE__NETBIOS_PDU
};

enum DCE2_SmbPduState
{
    DCE2_SMB_PDU_STATE__COMMAND,
    DCE2_SMB_PDU_STATE__RAW_DATA
};

enum DCE2_SmbFileDirection
{
    DCE2_SMB_FILE_DIRECTION__UNKNOWN = 0,
    DCE2_SMB_FILE_DIRECTION__UPLOAD,
    DCE2_SMB_FILE_DIRECTION__DOWNLOAD
};

enum SmbAndXCom
{
    SMB_ANDX_COM__NONE,
    SMB_ANDX_COM__OPEN_ANDX,
    SMB_ANDX_COM__READ_ANDX,
    SMB_ANDX_COM__WRITE_ANDX,
    SMB_ANDX_COM__TREE_CONNECT_ANDX,
    SMB_ANDX_COM__SESSION_SETUP_ANDX,
    SMB_ANDX_COM__LOGOFF_ANDX,
    SMB_ANDX_COM__NT_CREATE_ANDX,
    SMB_ANDX_COM__MAX
};

struct DCE2_SmbWriteAndXRaw
{
    int remaining;  // A signed integer so it can be negative
    DCE2_Buffer* buf;
};

struct DCE2_SmbFileChunk
{
    uint64_t offset;
    uint32_t length;
    uint8_t* data;
};

enum DCE2_SmbVersion
{
    DCE2_SMB_VERSION_NULL,
    DCE2_SMB_VERSION_1,
    DCE2_SMB_VERSION_2
};

struct DCE2_SmbFileTracker
{
    struct
    {
        int file_id;   // A signed integer so it can be set to sentinel
        uint16_t u_id;
        uint16_t tree_id;
    } file_key;

    bool is_ipc;
    bool is_smb2;
    char* file_name;
    uint16_t file_name_size;
    uint64_t file_name_hash;

    union
    {
        struct
        {
            // If pipe has been set to byte mode via TRANS_SET_NMPIPE_STATE
            bool byte_mode;

            // For Windows 2000
            bool used;

            // For WriteAndX requests that use raw mode flag
            // Windows only
            DCE2_SmbWriteAndXRaw* writex_raw;

            // Connection-oriented DCE/RPC tracker
            DCE2_CoTracker* co_tracker;
        } nmpipe;

        struct
        {
            uint64_t file_size;
            uint64_t file_offset;
            uint64_t bytes_processed;
            DCE2_List* file_chunks;
            uint32_t bytes_queued;
            DCE2_SmbFileDirection file_direction;
            bool sequential_only;
        } file;
    } tracker;

    DCE2_SmbPduState smb2_pdu_state;

#define fid_v1                file_key.file_id
#define uid_v1                file_key.u_id
#define tid_v1                file_key.tree_id
#define fp_byte_mode   tracker.nmpipe.byte_mode
#define fp_used        tracker.nmpipe.used
#define fp_writex_raw  tracker.nmpipe.writex_raw
#define fp_co_tracker  tracker.nmpipe.co_tracker
#define ff_file_size          tracker.file.file_size
#define ff_file_offset        tracker.file.file_offset
#define ff_bytes_processed    tracker.file.bytes_processed
#define ff_file_direction     tracker.file.file_direction
#define ff_file_chunks        tracker.file.file_chunks
#define ff_bytes_queued       tracker.file.bytes_queued
#define ff_sequential_only    tracker.file.sequential_only
};

struct Smb2Request
{
    uint64_t message_id;   /* identifies a message uniquely on connection */
    uint64_t offset;       /* data offset */
    uint64_t file_id;      /* file id */
    struct Smb2Request* next;
    struct Smb2Request* previous;
};

struct DCE2_SmbTransactionTracker
{
    int smb_type;
    uint8_t subcom;
    bool one_way;
    bool disconnect_tid;
    bool pipe_byte_mode;
    uint32_t tdcnt;
    uint32_t dsent;
    DCE2_Buffer* dbuf;
    uint32_t tpcnt;
    uint32_t psent;
    DCE2_Buffer* pbuf;
    // For Transaction2/Query File Information
    uint16_t info_level;
};

struct DCE2_SmbRequestTracker
{
    int smb_com;

    int mid;   // A signed integer so it can be set to sentinel
    uint16_t uid;
    uint16_t tid;
    uint16_t pid;

    // For WriteRaw
    bool writeraw_writethrough;
    uint32_t writeraw_remaining;

    // For Transaction/Transaction2/NtTransact
    DCE2_SmbTransactionTracker ttracker;

    // Client can chain a write to an open.  Need to write data, but also
    // need to associate tracker with fid returned from server
    DCE2_Queue* ft_queue;

    // This is a reference to an existing file tracker
    DCE2_SmbFileTracker* ftracker;

    // Used for requests to cache data that will ultimately end up in
    // the file tracker upon response.
    char* file_name;
    uint16_t file_name_size;
    uint64_t file_size;
    uint64_t file_offset;
    bool sequential_only;

    // For TreeConnect to know whether it's to IPC
    bool is_ipc;
};

struct DCE2_SmbSsnData
{
    DCE2_SsnData sd;  // This member must be first
    DCE2_Policy policy;

    int dialect_index;
    int ssn_state_flags;

    DCE2_SmbDataState cli_data_state;
    DCE2_SmbDataState srv_data_state;

    DCE2_SmbPduState pdu_state;

    int uid;   // A signed integer so it can be set to sentinel
    int tid;   // A signed integer so it can be set to sentinel
    DCE2_List* uids;
    DCE2_List* tids;

    // For tracking files and named pipes
    DCE2_SmbFileTracker ftracker;
    DCE2_List* ftrackers;  // List of DCE2_SmbFileTracker

    // For tracking requests / responses
    DCE2_SmbRequestTracker rtracker;
    DCE2_Queue* rtrackers;
    uint16_t max_outstanding_requests;
    uint16_t outstanding_requests;

    // The current pid/mid node for this request/response
    DCE2_SmbRequestTracker* cur_rtracker;

    // Used for TCP segmentation to get full PDU
    DCE2_Buffer* cli_seg;
    DCE2_Buffer* srv_seg;

    // These are used for commands we don't need to process
    uint32_t cli_ignore_bytes;
    uint32_t srv_ignore_bytes;

    // The file API supports one concurrent upload/download per session.
    // This is a reference to a file tracker so shouldn't be freed.
    DCE2_SmbFileTracker* fapi_ftracker;

    DCE2_SmbFileTracker* fb_ftracker;
    bool block_pdus;

    // Maximum file depth as returned from file API
    int64_t max_file_depth;
};

struct DCE2_SmbFsm
{
    char input;
    int next_state;
    int fail_state;
};

class Dce2SmbFlowData : public snort::FlowData
{
public:
    Dce2SmbFlowData();
    ~Dce2SmbFlowData() override;

    static void init()
    { inspector_id = snort::FlowData::create_flow_data_id(); }

    size_t size_of() override
    { return sizeof(*this); }

public:
    static unsigned inspector_id;
    DCE2_SmbVersion smb_version;
    void* dce2_smb_session_data;
};

// Used for reassembled packets
#define DCE2_MOCK_HDR_LEN__SMB_CLI \
    ((unsigned)(sizeof(NbssHdr) + sizeof(SmbNtHdr) + sizeof(SmbWriteAndXReq)))
#define DCE2_MOCK_HDR_LEN__SMB_SRV \
    ((unsigned)(sizeof(NbssHdr) + sizeof(SmbNtHdr) + sizeof(SmbReadAndXResp)))

DCE2_SsnData* get_dce2_session_data(snort::Flow*);

const char* get_smb_com_string(uint8_t);
#endif

dce_smb_module.h

Path = src/service_inspectors/dce_rpc/dce_smb_module.h

#ifndef DCE_SMB_MODULE_H
#define DCE_SMB_MODULE_H

#include "dce_common.h"
#include "framework/module.h"
#include "dce_list.h"

namespace snort
{
class Trace;
struct SnortConfig;
}

extern THREAD_LOCAL const snort::Trace* dce_smb_trace;

#define DCE2_VALID_SMB_VERSION_FLAG_V1 1
#define DCE2_VALID_SMB_VERSION_FLAG_V2 2

enum dce2SmbFingerprintPolicy
{
    DCE2_SMB_FINGERPRINT_POLICY_NONE = 0,
    DCE2_SMB_FINGERPRINT_POLICY_CLIENT,
    DCE2_SMB_FINGERPRINT_POLICY_SERVER,
    DCE2_SMB_FINGERPRINT_POLICY_BOTH,
};

struct dce2SmbShare
{
    char* unicode_str;
    unsigned int unicode_str_len;
    char* ascii_str;
    unsigned int ascii_str_len;
};

struct dce2SmbProtoConf
{
    dce2CoProtoConf common; // This member must be first
    dce2SmbFingerprintPolicy smb_fingerprint_policy;
    uint8_t smb_max_chain;
    uint8_t smb_max_compound;
    uint16_t smb_valid_versions_mask;
    int16_t smb_file_depth;
    DCE2_List* smb_invalid_shares;
    bool legacy_mode;
    uint16_t smb_max_credit;
    size_t memcap;
};

class Dce2SmbModule : public snort::Module
{
public:
    Dce2SmbModule();
    ~Dce2SmbModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;

    unsigned get_gid() const override
    { return GID_DCE2; }

    const snort::RuleMap* get_rules() const override;
    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile() const override;
    void get_data(dce2SmbProtoConf&);

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

    void set_trace(const snort::Trace*) const override;
    const snort::TraceOption* get_trace_options() const override;

private:
    dce2SmbProtoConf config;
};

void print_dce2_smb_conf(const dce2SmbProtoConf&);

inline int64_t DCE2_ScSmbFileDepth(const dce2SmbProtoConf* sc)
{
    return sc->smb_file_depth;
}

inline uint8_t DCE2_ScSmbMaxChain(const dce2SmbProtoConf* sc)
{
    if (sc == nullptr)
        return 0;
    return sc->smb_max_chain;
}

inline DCE2_List* DCE2_ScSmbInvalidShares(const dce2SmbProtoConf* sc)
{
    if (sc == nullptr)
        return nullptr;
    return sc->smb_invalid_shares;
}

#define SMB_DEFAULT_MAX_CREDIT        8192
#define SMB_DEFAULT_MEMCAP            8388608
#define SMB_DEFAULT_MAX_COMPOUND_REQ  3

inline uint16_t DCE2_ScSmbMaxCredit(const dce2SmbProtoConf* sc)
{
    if (sc == nullptr)
        return SMB_DEFAULT_MAX_CREDIT;
    return sc->smb_max_credit;
}

inline size_t DCE2_ScSmbMemcap(const dce2SmbProtoConf* sc)
{
    if (sc == nullptr)
        return SMB_DEFAULT_MEMCAP;
    return sc->memcap;
}

inline uint16_t DCE2_ScSmbMaxCompound(const dce2SmbProtoConf* sc)
{
    if (sc == nullptr)
        return SMB_DEFAULT_MAX_COMPOUND_REQ;
    return sc->smb_max_compound;
}

inline bool DCE2_GcSmbFingerprintClient(const dce2SmbProtoConf* sc)
{
    if (sc == nullptr)
        return false;
    return (sc->smb_fingerprint_policy
           & DCE2_SMB_FINGERPRINT_POLICY_CLIENT) ? true : false;
}

inline bool DCE2_GcSmbFingerprintServer(const dce2SmbProtoConf* sc)
{
    if (sc == nullptr)
        return false;
    return (sc->smb_fingerprint_policy
           & DCE2_SMB_FINGERPRINT_POLICY_SERVER) ? true : false;
}

inline bool DCE2_GcIsLegacyMode(const dce2SmbProtoConf* sc)
{
    if (sc == nullptr)
        return false;
    return sc->legacy_mode;
}

#endif

dce_smb_paf.h

Path = src/service_inspectors/dce_rpc/dce_smb_paf.h

#ifndef DCE_SMB_PAF_H
#define DCE_SMB_PAF_H

#include "stream/stream_splitter.h"

// Enumerations for PAF states
enum DCE2_PafSmbStates
{
    DCE2_PAF_SMB_STATES__0 = 0,  // NetBIOS type
    DCE2_PAF_SMB_STATES__1,      // Added bit of NetBIOS length
    DCE2_PAF_SMB_STATES__2,      // First byte of NetBIOS length
    DCE2_PAF_SMB_STATES__3,      // Second byte of NetBIOS length
    // Junk states
    DCE2_PAF_SMB_STATES__4,      // 0xff
    DCE2_PAF_SMB_STATES__5,      // 'S'
    DCE2_PAF_SMB_STATES__6,      // 'M'
    DCE2_PAF_SMB_STATES__7       // 'B'
};

// State tracker for DCE/RPC over SMB PAF
struct DCE2_PafSmbData
{
    DCE2_PafSmbStates paf_state;
    uint64_t nb_hdr;   // Enough for NetBIOS header and 4 bytes SMB header
};

class Dce2SmbSplitter : public snort::StreamSplitter
{
public:
    explicit Dce2SmbSplitter(bool c2s);

    Status scan(snort::Packet*, const uint8_t* data, uint32_t len,
        uint32_t flags, uint32_t* fp) override;

    bool is_paf() override
    {
        return true;
    }

public:
    DCE2_PafSmbData state;
};

#endif

dce_smb_transaction.h

Path = src/service_inspectors/dce_rpc/dce_smb_transaction.h

#ifndef DCE_SMB_TRANSACTION_H
#define DCE_SMB_TRANSACTION_H

#include "dce_smb_utils.h"

DCE2_Ret DCE2_SmbTransaction(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbTransaction2(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbNtTransact(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbTransactionSecondary(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbTransaction2Secondary(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbNtTransactSecondary(DCE2_SmbSsnData*, const SmbNtHdr*,
    const DCE2_SmbComInfo*, const uint8_t*, uint32_t);

#endif

dce_smb_transaction_utils.h

Path = src/service_inspectors/dce_rpc/dce_smb_transaction_utils.h

#ifndef DCE_SMB_TRANSACTION_UTILS_H
#define DCE_SMB_TRANSACTION_UTILS_H

#include "dce_smb.h"

DCE2_Ret DCE2_SmbTransactionGetName(const uint8_t* nb_ptr,
    uint32_t nb_len, uint16_t bcc, bool unicode);

DCE2_Ret DCE2_SmbValidateTransactionFields(
    DCE2_SmbSsnData* ssd, const uint8_t* smb_hdr_ptr,
    const uint8_t* nb_ptr, const uint32_t nb_len, const uint16_t bcc,
    const uint32_t tdcnt, const uint32_t tpcnt,
    const uint32_t dcnt, const uint32_t doff, const uint32_t ddisp,
    const uint32_t pcnt, const uint32_t poff, const uint32_t pdisp);

DCE2_Ret DCE2_SmbValidateTransactionSent(
    DCE2_SmbSsnData* ssd, uint32_t dsent, uint32_t dcnt, uint32_t tdcnt,
    uint32_t psent, uint32_t pcnt, uint32_t tpcnt);

DCE2_Ret DCE2_SmbBufferTransactionData(DCE2_SmbTransactionTracker* ttracker,
    const uint8_t* data_ptr, uint16_t dcnt, uint16_t ddisp);

DCE2_Ret DCE2_SmbBufferTransactionParameters(DCE2_SmbTransactionTracker* ttracker,
    const uint8_t* param_ptr, uint16_t pcnt, uint16_t pdisp);

DCE2_Ret DCE2_SmbCheckTotalCount(DCE2_SmbSsnData* ssd, const uint32_t tcnt, const uint32_t cnt,
    const uint32_t disp);

inline bool DCE2_SmbFileUpload(DCE2_SmbFileDirection dir)
{
    return dir == DCE2_SMB_FILE_DIRECTION__UPLOAD;
}

inline bool DCE2_SmbFileDownload(DCE2_SmbFileDirection dir)
{
    return dir == DCE2_SMB_FILE_DIRECTION__DOWNLOAD;
}

#endif

dce_smb_utils.h

Path = src/service_inspectors/dce_rpc/dce_smb_utils.h

#ifndef DCE_SMB_UTILS_H
#define DCE_SMB_UTILS_H

#include "dce_smb.h"
#include "file_api/file_flows.h"

/********************************************************************
 * Enums
 ********************************************************************/
enum DCE2_SmbComError
{
    // No errors associated with the command
    DCE2_SMB_COM_ERROR__COMMAND_OK          = 0x0000,

    // An error was reported in the SMB response header
    DCE2_SMB_COM_ERROR__STATUS_ERROR        = 0x0001,

    // An invalid word count makes it unlikely any data accessed will be correct
    // and if accessed the possibility of accessing out of bounds data
    DCE2_SMB_COM_ERROR__INVALID_WORD_COUNT  = 0x0002,

    // An invalid byte count just means the byte count is not right for
    // the command processed.  The command can still be processed but
    // the byte count should not be used.  In general, the byte count
    // should not be used since Windows and Samba often times ignore it
    DCE2_SMB_COM_ERROR__INVALID_BYTE_COUNT  = 0x0004,

    // Not enough data to process command so don't try to access any
    // of the command's header or data.
    DCE2_SMB_COM_ERROR__BAD_LENGTH          = 0x0008
};

/********************************************************************
 * Structures
 ********************************************************************/
struct DCE2_SmbComInfo
{
    int smb_type;   // SMB_TYPE__REQUEST or SMB_TYPE__RESPONSE
    int cmd_error;  // mask of DCE2_SmbComError
    uint8_t smb_com;
    uint8_t word_count;
    uint16_t byte_count;
    uint16_t cmd_size;
};

// Inline accessor functions for DCE2_SmbComInfo

inline bool DCE2_ComInfoIsResponse(const DCE2_SmbComInfo* com_info)
{
    return (com_info->smb_type == SMB_TYPE__RESPONSE) ? true : false;
}

inline bool DCE2_ComInfoIsRequest(const DCE2_SmbComInfo* com_info)
{
    return (com_info->smb_type == SMB_TYPE__REQUEST) ? true : false;
}

inline uint16_t DCE2_ComInfoByteCount(const DCE2_SmbComInfo* com_info)
{
    return com_info->byte_count;
}

inline uint8_t DCE2_ComInfoSmbCom(const DCE2_SmbComInfo* com_info)
{
    return com_info->smb_com;
}

inline uint16_t DCE2_ComInfoCommandSize(const DCE2_SmbComInfo* com_info)
{
    return com_info->cmd_size;
}

inline bool DCE2_ComInfoIsStatusError(const DCE2_SmbComInfo* com_info)
{
    return (com_info->cmd_error & DCE2_SMB_COM_ERROR__STATUS_ERROR) ? true : false;
}

inline bool DCE2_ComInfoIsInvalidWordCount(const DCE2_SmbComInfo* com_info)
{
    return (com_info->cmd_error & DCE2_SMB_COM_ERROR__INVALID_WORD_COUNT) ? true : false;
}

inline bool DCE2_ComInfoIsBadLength(const DCE2_SmbComInfo* com_info)
{
    return (com_info->cmd_error & DCE2_SMB_COM_ERROR__BAD_LENGTH) ? true : false;
}

inline uint8_t DCE2_ComInfoWordCount(const DCE2_SmbComInfo* com_info)
{
    return com_info->word_count;
}

// If this returns false, the command should not be processed
inline bool DCE2_ComInfoCanProcessCommand(const DCE2_SmbComInfo* com_info)
{
    if (DCE2_ComInfoIsBadLength(com_info)
        || DCE2_ComInfoIsStatusError(com_info)
        || DCE2_ComInfoIsInvalidWordCount(com_info))
        return false;
    return true;
}

/********************************************************************
 * Function prototypes
 ********************************************************************/
bool DCE2_SmbIsTidIPC(DCE2_SmbSsnData*, const uint16_t);
char* DCE2_SmbGetFileName(const uint8_t* data, uint32_t data_len, bool unicode, uint16_t* file_name_len);
int DCE2_SmbUidTidFidCompare(const void*, const void*);
DCE2_Ret DCE2_SmbFindUid(DCE2_SmbSsnData*, const uint16_t);
void DCE2_SmbInsertUid(DCE2_SmbSsnData*, const uint16_t);
void DCE2_SmbRemoveUid(DCE2_SmbSsnData*, const uint16_t);
DCE2_SmbRequestTracker* DCE2_SmbNewRequestTracker(DCE2_SmbSsnData*, const SmbNtHdr*);
void DCE2_SmbCleanTransactionTracker(DCE2_SmbTransactionTracker*);
void DCE2_SmbCleanRequestTracker(DCE2_SmbRequestTracker*);
void DCE2_SmbRemoveRequestTracker(DCE2_SmbSsnData*, DCE2_SmbRequestTracker*);
DCE2_SmbFileTracker* DCE2_SmbGetFileTracker(DCE2_SmbSsnData*,
    const uint16_t);
DCE2_SmbFileTracker* DCE2_SmbGetTmpFileTracker(DCE2_SmbRequestTracker*);
void DCE2_SmbRemoveFileTracker(DCE2_SmbSsnData*, DCE2_SmbFileTracker*);
void DCE2_SmbCleanFileTracker(DCE2_SmbFileTracker*);
void DCE2_SmbFileTrackerDataFree(void*);
void DCE2_SmbCleanSessionFileTracker(DCE2_SmbSsnData*, DCE2_SmbFileTracker*);
void DCE2_SmbRemoveFileTrackerFromRequestTrackers(DCE2_SmbSsnData*,
    const DCE2_SmbFileTracker*);
DCE2_SmbFileTracker* DCE2_SmbDequeueTmpFileTracker(DCE2_SmbSsnData*,
    DCE2_SmbRequestTracker*, const uint16_t);
DCE2_SmbFileTracker* DCE2_SmbNewFileTracker(DCE2_SmbSsnData*,
    const uint16_t, const uint16_t, const uint16_t);
void DCE2_SmbQueueTmpFileTracker(DCE2_SmbSsnData*,
    DCE2_SmbRequestTracker*, const uint16_t, const uint16_t);
DCE2_Ret DCE2_SmbFindTid(DCE2_SmbSsnData*, const uint16_t);
void DCE2_SmbRemoveTid(DCE2_SmbSsnData*, const uint16_t);
void DCE2_SmbInsertTid(DCE2_SmbSsnData*, const uint16_t, const bool);
void DCE2_SmbInvalidShareCheck(DCE2_SmbSsnData*,
    const SmbNtHdr*, const uint8_t*, uint32_t);
DCE2_Ret DCE2_SmbInitFileTracker(DCE2_SmbSsnData*,
    DCE2_SmbFileTracker*, const bool, const uint16_t,
    const uint16_t, const int);
void DCE2_SmbRequestTrackerDataFree(void*);
DCE2_SmbFileTracker* DCE2_SmbFindFileTracker(DCE2_SmbSsnData*,
    const uint16_t, const uint16_t, const uint16_t);
DCE2_Ret DCE2_SmbProcessRequestData(DCE2_SmbSsnData*, const uint16_t,
    const uint8_t*, uint32_t, uint64_t);
DCE2_Ret DCE2_SmbProcessResponseData(DCE2_SmbSsnData*,
    const uint8_t*, uint32_t);
void DCE2_SmbInitRdata(uint8_t*, int);
void DCE2_SmbSetRdata(DCE2_SmbSsnData*, uint8_t*, uint16_t);
snort::Packet* DCE2_SmbGetRpkt(DCE2_SmbSsnData*, const uint8_t**,
    uint32_t*, DCE2_RpktType);
DCE2_Ret DCE2_SmbHandleSegmentation(DCE2_Buffer**,
    const uint8_t*, uint32_t, uint32_t);
bool DCE2_SmbIsSegBuffer(DCE2_SmbSsnData*, const uint8_t*);
void DCE2_SmbSegAlert(DCE2_SmbSsnData*, uint32_t rule_id);
void DCE2_SmbAbortFileAPI(DCE2_SmbSsnData*);
void DCE2_SmbProcessFileData(DCE2_SmbSsnData* ssd,
    DCE2_SmbFileTracker* ftracker, const uint8_t* data_ptr,
    uint32_t data_len, bool upload);
void DCE2_FileDetect();
FileVerdict DCE2_get_file_verdict();
void DCE2_SmbInitDeletePdu();
void DCE2_Update_Ftracker_from_ReqTracker(DCE2_SmbFileTracker*, DCE2_SmbRequestTracker*);

/********************************************************************
 * Inline functions
 ********************************************************************/

/********************************************************************
 * Function: DCE2_SmbType()
 *
 * Purpose:
 *  Since Windows and Samba don't seem to care or even look at the
 *  actual flag in the SMB header, make the determination based on
 *  whether from client or server.
 *
 * Arguments:
 *  DCE2_SmbSsnData * - session data structure that has the raw
 *     packet and packet flags to make determination
 *
 * Returns:
 *  SMB_TYPE__REQUEST if packet is from client
 *  SMB_TYPE__RESPONSE if packet is from server
 *
 ********************************************************************/
inline int DCE2_SmbType()
{
    if ( snort::DetectionEngine::get_current_packet()->is_from_client() )
        return SMB_TYPE__REQUEST;
    else
        return SMB_TYPE__RESPONSE;
}

inline bool SmbUnicode(const SmbNtHdr* hdr)
{
    return (snort::alignedNtohs(&hdr->smb_flg2) & SMB_FLG2__UNICODE) ? true : false;
}

inline bool SmbExtAttrReadOnly(const uint32_t ext_file_attrs)
{
    if (ext_file_attrs & SMB_EXT_FILE_ATTR_READONLY)
        return true;
    return false;
}

inline bool SmbExtAttrHidden(const uint32_t ext_file_attrs)
{
    if (ext_file_attrs & SMB_EXT_FILE_ATTR_HIDDEN)
        return true;
    return false;
}

inline bool SmbExtAttrSystem(const uint32_t ext_file_attrs)
{
    if (ext_file_attrs & SMB_EXT_FILE_ATTR_SYSTEM)
        return true;
    return false;
}

inline bool SmbEvasiveFileAttrs(const uint32_t ext_file_attrs)
{
    return (SmbExtAttrReadOnly(ext_file_attrs)
           && SmbExtAttrHidden(ext_file_attrs)
           && SmbExtAttrSystem(ext_file_attrs));
}

inline uint8_t SmbCom(const SmbNtHdr* hdr)
{
    return hdr->smb_com;
}

inline bool SmbStatusNtCodes(const SmbNtHdr* hdr)
{
    if (snort::alignedNtohs(&hdr->smb_flg2) & SMB_FLG2__NT_CODES)
        return true;
    return false;
}

inline uint32_t SmbNtStatus(const SmbNtHdr* hdr)
{
    return snort::alignedNtohl(&hdr->smb_status.nt_status);
}

inline uint8_t SmbStatusClass(const SmbNtHdr* hdr)
{
    return hdr->smb_status.smb_status.smb_class;
}

inline uint16_t SmbStatusCode(const SmbNtHdr* hdr)
{
    return snort::alignedNtohs(&hdr->smb_status.smb_status.smb_code);
}

inline uint8_t SmbNtStatusSeverity(const SmbNtHdr* hdr)
{
    return (uint8_t)(SmbNtStatus(hdr) >> 30);
}

inline uint16_t SmbPid(const SmbNtHdr* hdr)
{
    return snort::alignedNtohs(&hdr->smb_pid);
}

inline uint16_t SmbMid(const SmbNtHdr* hdr)
{
    return snort::alignedNtohs(&hdr->smb_mid);
}

inline uint16_t SmbUid(const SmbNtHdr* hdr)
{
    return snort::alignedNtohs(&hdr->smb_uid);
}

inline uint16_t SmbTid(const SmbNtHdr* hdr)
{
    return snort::alignedNtohs(&hdr->smb_tid);
}

inline bool SmbErrorInvalidDeviceRequest(const SmbNtHdr* hdr)
{
    if (SmbStatusNtCodes(hdr))
    {
        if (SmbNtStatus(hdr) == SMB_NT_STATUS__INVALID_DEVICE_REQUEST)
            return true;
    }
    else
    {
        if ((SmbStatusClass(hdr) == SMB_ERROR_CLASS__ERRSRV)
            && (SmbStatusCode(hdr) == SMB_ERRSRV__INVALID_DEVICE))
            return true;
    }

    return false;
}

inline bool SmbErrorRangeNotLocked(const SmbNtHdr* hdr)
{
    if (SmbStatusNtCodes(hdr))
    {
        if (SmbNtStatus(hdr) == SMB_NT_STATUS__RANGE_NOT_LOCKED)
            return true;
    }
    else
    {
        if ((SmbStatusClass(hdr) == SMB_ERROR_CLASS__ERRDOS)
            && (SmbStatusCode(hdr) == SMB_ERRDOS__NOT_LOCKED))
            return true;
    }

    return false;
}

// Convenience function to determine whether or not the transaction is complete
// for one side, i.e. all data and parameters sent.
inline bool DCE2_SmbIsTransactionComplete(DCE2_SmbTransactionTracker* ttracker)
{
    if ((ttracker->tdcnt == ttracker->dsent)
        && (ttracker->tpcnt == ttracker->psent))
        return true;
    return false;
}

inline DCE2_Buffer** DCE2_SmbGetSegBuffer(DCE2_SmbSsnData* ssd)
{
    if ( snort::DetectionEngine::get_current_packet()->is_from_server() )
        return &ssd->srv_seg;
    return &ssd->cli_seg;
}

#endif

dce_tcp.h

Path = src/service_inspectors/dce_rpc/dce_tcp.h

#ifndef DCE_TCP_H
#define DCE_TCP_H

#include "framework/data_bus.h"
#include "framework/inspector.h"
#include "profiler/profiler_defs.h"

#include "dce_co.h"
#include "dce_expected_session.h"
#include "dce_tcp_module.h"
#include "dce_tcp_paf.h"

#define DCE2_TCP_NAME "dce_tcp"
#define DCE2_TCP_HELP "dce over tcp inspection"

struct dce2TcpStats
{
    /* common stats - defined in dce_common.h*/
    PegCount events;

    PegCount co_pdus;
    PegCount co_bind;
    PegCount co_bind_ack;
    PegCount co_alter_ctx;
    PegCount co_alter_ctx_resp;
    PegCount co_bind_nack;
    PegCount co_request;
    PegCount co_response;
    PegCount co_cancel;
    PegCount co_orphaned;
    PegCount co_fault;
    PegCount co_auth3;
    PegCount co_shutdown;
    PegCount co_reject;
    PegCount co_ms_pdu;
    PegCount co_other_req;
    PegCount co_other_resp;
    PegCount co_req_fragments;
    PegCount co_resp_fragments;
    PegCount co_cli_max_frag_size;
    PegCount co_cli_min_frag_size;
    PegCount co_cli_seg_reassembled;
    PegCount co_cli_frag_reassembled;
    PegCount co_srv_max_frag_size;
    PegCount co_srv_min_frag_size;
    PegCount co_srv_seg_reassembled;
    PegCount co_srv_frag_reassembled;

    /*DCE TCP specific*/
    PegCount tcp_sessions;
    PegCount tcp_expected_sessions;
    PegCount tcp_expected_realized_sessions;
    PegCount tcp_pkts;
    PegCount concurrent_sessions;
    PegCount max_concurrent_sessions;
};

extern THREAD_LOCAL dce2TcpStats dce2_tcp_stats;
extern THREAD_LOCAL snort::ProfileStats dce2_tcp_pstat_main;

inline bool DCE2_TcpAutodetect(snort::Packet* p)
{
    if (p->dsize >= sizeof(DceRpcCoHdr))
    {
        const DceRpcCoHdr* co_hdr = (const DceRpcCoHdr*)p->data;

        if ((DceRpcCoVersMaj(co_hdr) == DCERPC_PROTO_MAJOR_VERS__5)
            && (DceRpcCoVersMin(co_hdr) == DCERPC_PROTO_MINOR_VERS__0)
            && ((p->is_from_client()
            && DceRpcCoPduType(co_hdr) == DCERPC_PDU_TYPE__BIND)
            || (p->is_from_server()
            && DceRpcCoPduType(co_hdr) == DCERPC_PDU_TYPE__BIND_ACK))
            && (DceRpcCoFragLen(co_hdr) >= sizeof(DceRpcCoHdr)))
        {
            return true;
        }
    }
    else if ((*p->data == DCERPC_PROTO_MAJOR_VERS__5) && p->is_from_client())
    {
        return true;
    }

    return false;
}

struct DCE2_TcpSsnData
{
    DCE2_SsnData sd;  // This member must be first
    DCE2_CoTracker co_tracker;
};

enum DCE2_TcpFlowState
{
    DCE2_TCP_FLOW__COMMON,
    DCE2_TCP_FLOW__EXPECTED,
    DCE2_TCP_FLOW__REALIZED
};

class Dce2TcpFlowData : public snort::FlowData
{
public:
    Dce2TcpFlowData();
    ~Dce2TcpFlowData() override;

    static void init()
    { inspector_id = snort::FlowData::create_flow_data_id(); }

public:
    static unsigned inspector_id;
    DCE2_TcpSsnData dce2_tcp_session = {};
    DCE2_TcpFlowState state = DCE2_TCP_FLOW__COMMON;
};

class Dce2Tcp : public snort::Inspector
{
public:
    explicit Dce2Tcp(const dce2TcpProtoConf&);

    bool configure(snort::SnortConfig*) override;
    void show(const snort::SnortConfig*) const override;
    void eval(snort::Packet*) override;
    void clear(snort::Packet*) override;

    DceTcpExpSsnManager& get_esm()
    { return esm; }

    bool is_control_channel() const override
    { return true; }

    snort::StreamSplitter* get_splitter(bool c2s) override
    { return new Dce2TcpSplitter(c2s); }

    static unsigned pub_id;

private:
    dce2TcpProtoConf config;
    DceTcpExpSsnManager esm;
};

DCE2_TcpSsnData* get_dce2_tcp_session_data(snort::Flow*);

#endif

dce_tcp_module.h

Path = src/service_inspectors/dce_rpc/dce_tcp_module.h

#ifndef DCE2_TCP_MODULE_H
#define DCE2_TCP_MODULE_H

#include "dce_common.h"
#include "framework/module.h"

namespace snort
{
struct SnortConfig;
}

struct dce2TcpProtoConf
{
    dce2CoProtoConf common = {};
};

class Dce2TcpModule : public snort::Module
{
public:
    Dce2TcpModule();

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;

    unsigned get_gid() const override
    { return GID_DCE2; }

    const snort::RuleMap* get_rules() const override;
    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile() const override;
    void get_data(dce2TcpProtoConf&);

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

private:
    dce2TcpProtoConf config = {};
};

void print_dce2_tcp_conf(const dce2TcpProtoConf& config);

#endif

dce_tcp_paf.h

Path = src/service_inspectors/dce_rpc/dce_tcp_paf.h

#ifndef DCE_TCP_PAF_H
#define DCE_TCP_PAF_H

#include "dce_common.h"
#include "stream/stream_splitter.h"

enum DCE2_PafTcpStates
{
    DCE2_PAF_TCP_STATES__0 = 0,
    DCE2_PAF_TCP_STATES__1,     // Major version
    DCE2_PAF_TCP_STATES__2,     // Minor version
    DCE2_PAF_TCP_STATES__3,     // PDU type
    DCE2_PAF_TCP_STATES__4,     // Byte order
    DCE2_PAF_TCP_STATES__5,
    DCE2_PAF_TCP_STATES__6,
    DCE2_PAF_TCP_STATES__7,
    DCE2_PAF_TCP_STATES__8,     // First byte of fragment length
    DCE2_PAF_TCP_STATES__9      // Second byte of fragment length
};

// State tracker for DCE/RPC over TCP PAF
struct DCE2_PafTcpData
{
    DCE2_PafTcpStates paf_state;
    DceRpcBoFlag byte_order;
    uint16_t frag_len;
    bool autodetected;
};

class Dce2TcpSplitter : public snort::StreamSplitter
{
public:
    explicit Dce2TcpSplitter(bool c2s);

    Status scan(snort::Packet*, const uint8_t* data, uint32_t len,
        uint32_t flags, uint32_t* fp) override;

    bool is_paf() override { return true; }

private:
    DCE2_PafTcpData state;
};

#endif

dce_udp.h

Path = src/service_inspectors/dce_rpc/dce_udp.h

#ifndef DCE_UDP_H
#define DCE_UDP_H

#include "profiler/profiler_defs.h"

#include "dce_common.h"

#define DCE2_UDP_NAME "dce_udp"
#define DCE2_UDP_HELP "dce over udp inspection"

#define DCE2_MOCK_HDR_LEN__CL  (sizeof(DceRpcClHdr))

struct dce2UdpStats
{
    /* The common stats block has to be at the beginning followed
       by the protocol specific stats */

    /*common stats -defined in common.h*/
    PegCount events;

    /*DCE UDP specific*/
    PegCount udp_sessions;
    PegCount udp_pkts;
    PegCount cl_request;
    PegCount cl_ack;
    PegCount cl_cancel;
    PegCount cl_cli_fack;
    PegCount cl_ping;
    PegCount cl_response;
    PegCount cl_reject;
    PegCount cl_cancel_ack;
    PegCount cl_srv_fack;
    PegCount cl_fault;
    PegCount cl_nocall;
    PegCount cl_working;
    PegCount cl_other_req;
    PegCount cl_other_resp;
    PegCount cl_fragments;
    PegCount cl_max_frag_size;
    PegCount cl_frag_reassembled;
    PegCount cl_max_seqnum;
    PegCount concurrent_sessions;
    PegCount max_concurrent_sessions;
};

extern THREAD_LOCAL dce2UdpStats dce2_udp_stats;
extern THREAD_LOCAL snort::ProfileStats dce2_udp_pstat_main;

struct DceRpcClHdr   /* Connectionless header */
{
    uint8_t rpc_vers;
    uint8_t ptype;
    uint8_t flags1;
    uint8_t flags2;
    uint8_t drep[3];
    uint8_t serial_hi;
    Uuid object;
    Uuid if_id;
    Uuid act_id;
    uint32_t server_boot;
    uint32_t if_vers;
    uint32_t seqnum;
    uint16_t opnum;
    uint16_t ihint;
    uint16_t ahint;
    uint16_t len;
    uint16_t fragnum;
    uint8_t auth_proto;
    uint8_t serial_lo;
};

enum DceRpcClFlags1
{
    DCERPC_CL_FLAGS1__RESERVED_01 = 0x01,
    DCERPC_CL_FLAGS1__LASTFRAG = 0x02,
    DCERPC_CL_FLAGS1__FRAG = 0x04,
    DCERPC_CL_FLAGS1__NOFACK = 0x08,
    DCERPC_CL_FLAGS1__MAYBE = 0x10,
    DCERPC_CL_FLAGS1__IDEMPOTENT = 0x20,
    DCERPC_CL_FLAGS1__BROADCAST = 0x40,
    DCERPC_CL_FLAGS1__RESERVED_80 = 0x80
};

inline uint8_t DceRpcClRpcVers(const DceRpcClHdr* cl)
{
    return cl->rpc_vers;
}

inline uint8_t DceRpcClPduType(const DceRpcClHdr* cl)
{
    return cl->ptype;
}

inline DceRpcBoFlag DceRpcClByteOrder(const DceRpcClHdr* cl)
{
    return DceRpcByteOrder(cl->drep[0]);
}

inline uint16_t DceRpcClLen(const DceRpcClHdr* cl)
{
    return DceRpcNtohs(&cl->len, DceRpcClByteOrder(cl));
}

inline uint16_t DceRpcClOpnum(const DceRpcClHdr* cl)
{
    return DceRpcNtohs(&cl->opnum, DceRpcClByteOrder(cl));
}

inline uint32_t DceRpcClSeqNum(const DceRpcClHdr* cl)
{
    return DceRpcNtohl(&cl->seqnum, DceRpcClByteOrder(cl));
}

inline const Uuid* DceRpcClIface(const DceRpcClHdr* cl)
{
    return &cl->if_id;
}

inline uint32_t DceRpcClIfaceVers(const DceRpcClHdr* cl)
{
    return DceRpcNtohl(&cl->if_vers, DceRpcClByteOrder(cl));
}

inline uint16_t DceRpcClFragNum(const DceRpcClHdr* cl)
{
    return DceRpcNtohs(&cl->fragnum, DceRpcClByteOrder(cl));
}

inline int DceRpcClFragFlag(const DceRpcClHdr* cl)
{
    return cl->flags1 & DCERPC_CL_FLAGS1__FRAG;
}

inline bool DceRpcClFirstFrag(const DceRpcClHdr* cl)
{
    return (DceRpcClFragFlag(cl) && (DceRpcClFragNum(cl) == 0));
}

inline int DceRpcClLastFrag(const DceRpcClHdr* cl)
{
    return cl->flags1 & DCERPC_CL_FLAGS1__LASTFRAG;
}

inline bool DceRpcClFrag(const DceRpcClHdr* cl)
{
    if (DceRpcClFragFlag(cl))
    {
        if (DceRpcClLastFrag(cl) && (DceRpcClFragNum(cl) == 0))
            return false;

        return true;
    }

    return false;
}

struct DCE2_ClTracker
{
    DCE2_List* act_trackers;  /* List of activity trackers */
};

struct DCE2_UdpSsnData
{
    DCE2_SsnData sd;  // This member must be first
    DCE2_ClTracker cl_tracker;
};

class Dce2UdpFlowData : public snort::FlowData
{
public:
    Dce2UdpFlowData();
    ~Dce2UdpFlowData() override;

    static void init()
    { inspector_id = snort::FlowData::create_flow_data_id(); }

    static unsigned inspector_id;
    DCE2_UdpSsnData dce2_udp_session = {};
};

DCE2_UdpSsnData* get_dce2_udp_session_data(snort::Flow*);

void DCE2_ClProcess(DCE2_SsnData* sd, DCE2_ClTracker* clt);
void DCE2_ClInitRdata(uint8_t*);

#endif

dce_udp_module.h

Path = src/service_inspectors/dce_rpc/dce_udp_module.h

#ifndef DCE2_UDP_MODULE_H
#define DCE2_UDP_MODULE_H

#include "dce_common.h"
#include "framework/module.h"

namespace snort
{
class Trace;
struct SnortConfig;
}

extern THREAD_LOCAL const snort::Trace* dce_udp_trace;

#define DCE2_CL_BAD_MAJOR_VERSION 40
#define DCE2_CL_BAD_PDU_TYPE      41
#define DCE2_CL_DATA_LT_HDR       42
#define DCE2_CL_BAD_SEQ_NUM       43

#define DCE2_CL_BAD_MAJOR_VERSION_STR "connection-less DCE/RPC - invalid major version"
#define DCE2_CL_BAD_PDU_TYPE_STR "connection-less DCE/RPC - invalid PDU type"
#define DCE2_CL_DATA_LT_HDR_STR  "connection-less DCE/RPC - data length less than header size"
#define DCE2_CL_BAD_SEQ_NUM_STR  "connection-less DCE/RPC - bad sequence number"

struct dce2UdpProtoConf
{
    dce2CommonProtoConf common = {};
};

class Dce2UdpModule : public snort::Module
{
public:
    Dce2UdpModule();

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;

    unsigned get_gid() const override
    { return GID_DCE2; }

    const snort::RuleMap* get_rules() const override;
    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile() const override;
    void get_data(dce2UdpProtoConf&);

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

    void set_trace(const snort::Trace*) const override;
    const snort::TraceOption* get_trace_options() const override;

private:
    dce2UdpProtoConf config = {};
};

void print_dce2_udp_conf(const dce2UdpProtoConf&);

#endif

dce_utils.h

Path = src/service_inspectors/dce_rpc/dce_utils.h

#ifndef DCE_UTILS_H
#define DCE_UTILS_H

#include <cctype>
#include <cstring>
#include "main/snort_types.h"

/********************************************************************
 * Macros
 ********************************************************************/
#define DCE2_SENTINEL (-1)
#define DCE2_CFG_TOK__END            '\0'

/********************************************************************
 * Enumerations
 ********************************************************************/
enum DCE2_Ret
{
    DCE2_RET__SUCCESS = 0,
    DCE2_RET__ERROR,
    DCE2_RET__MEMCAP,
    DCE2_RET__NOT_INSPECTED,
    DCE2_RET__INSPECTED,
    DCE2_RET__REASSEMBLE,
    DCE2_RET__SEG,
    DCE2_RET__FULL,
    DCE2_RET__FRAG,
    DCE2_RET__ALERTED,
    DCE2_RET__IGNORE,
    DCE2_RET__DUPLICATE
};

enum DCE2_TransType
{
    DCE2_TRANS_TYPE__NONE = 0,
    DCE2_TRANS_TYPE__SMB,
    DCE2_TRANS_TYPE__TCP,
    DCE2_TRANS_TYPE__UDP,
    DCE2_TRANS_TYPE__MAX
};

enum DCE2_IntType
{
    DCE2_INT_TYPE__INT8,
    DCE2_INT_TYPE__UINT8,
    DCE2_INT_TYPE__INT16,
    DCE2_INT_TYPE__UINT16,
    DCE2_INT_TYPE__INT32,
    DCE2_INT_TYPE__UINT32,
    DCE2_INT_TYPE__INT64,
    DCE2_INT_TYPE__UINT64
};

/* DCE/RPC byte order flag */
enum DceRpcBoFlag
{
    DCERPC_BO_FLAG__NONE,
    DCERPC_BO_FLAG__BIG_ENDIAN,
    DCERPC_BO_FLAG__LITTLE_ENDIAN
};
enum DCE2_BufType
{
    DCE2_BUF_TYPE__NULL,
    DCE2_BUF_TYPE__SEG,
    DCE2_BUF_TYPE__FRAG
};
enum DCE2_BufferMinAddFlag
{
    DCE2_BUFFER_MIN_ADD_FLAG__USE,
    DCE2_BUFFER_MIN_ADD_FLAG__IGNORE
};

/********************************************************************
 * Structures
 ********************************************************************/

struct Uuid
{
    uint32_t time_low;
    uint16_t time_mid;
    uint16_t time_high_and_version;
    uint8_t clock_seq_and_reserved;
    uint8_t clock_seq_low;
    uint8_t node[6];
};

struct DCE2_Buffer
{
    uint8_t* data;
    uint32_t len;
    uint32_t size;
    uint32_t min_add_size;
    uint32_t offset;
};

/********************************************************************
 * Inline function prototypes
 ********************************************************************/

inline bool DCE2_IsSpaceChar(const char);
inline bool DCE2_IsConfigEndChar(const char);

inline char* DCE2_PruneWhiteSpace(char*);
inline bool DCE2_IsEmptyStr(char*);

inline int DCE2_UuidCompare(const void*, const void*);

/********************************************************************
 * Public function prototypes
 ********************************************************************/
DCE2_Ret DCE2_GetValue(const char*, char*, void*, int, DCE2_IntType, uint8_t);
DCE2_Buffer* DCE2_BufferNew(uint32_t, uint32_t);
void* DCE2_ReAlloc(void*, uint32_t, uint32_t);
DCE2_Ret DCE2_BufferAddData(DCE2_Buffer*, const uint8_t*,
    uint32_t, uint32_t, DCE2_BufferMinAddFlag);
void DCE2_BufferDestroy(DCE2_Buffer* buf);

// Determines if the character passed in is a character that
// the inspector considers a to be a space character.
inline bool DCE2_IsSpaceChar(const char c)
{
    if (isspace((int)c))
        return true;
    return false;
}

// Determines if the character passed in is a character that
// the inspector considers a to be an end of configuration
// character.
inline bool DCE2_IsConfigEndChar(const char c)
{
    if (c == DCE2_CFG_TOK__END)
        return true;
    return false;
}

// Prunes whitespace surrounding string.
// String must be 0 terminated.
inline char* DCE2_PruneWhiteSpace(char* str)
{
    char* end;

    if (str == nullptr)
        return nullptr;

    /* Put end a char before null byte */
    end = str + (strlen(str) - 1);

    while (isspace((int)*str))
        str++;

    while ((end > str) && isspace((int)*end))
    {
        *end = '\0';
        end--;
    }

    return str;
}

// Checks if string is null, empty or just spaces.
// String must be 0 terminated.
inline bool DCE2_IsEmptyStr(char* str)
{
    char* end;

    if (str == nullptr)
        return true;

    end = str + strlen(str);

    while ((str < end) && isspace((int)*str))
        str++;

    if (str == end)
        return true;

    return false;
}

inline int DCE2_UuidCompare(const void* data1, const void* data2)
{
    const Uuid* uuid1 = (const Uuid*)data1;
    const Uuid* uuid2 = (const Uuid*)data2;

    if ((uuid1 == nullptr) || (uuid2 == nullptr))
        return -1;

    if ((uuid1->time_low == uuid2->time_low) &&
        (uuid1->time_mid == uuid2->time_mid) &&
        (uuid1->time_high_and_version == uuid2->time_high_and_version) &&
        (uuid1->clock_seq_and_reserved == uuid2->clock_seq_and_reserved) &&
        (uuid1->clock_seq_low == uuid2->clock_seq_low) &&
        (memcmp(uuid1->node, uuid2->node, sizeof(uuid1->node)) == 0))
    {
        return 0;
    }

    /* Just return something other than 0 */
    return -1;
}

inline DceRpcBoFlag DceRpcByteOrder(const uint8_t value)
{
    if ((value & 0x10) >> 4)
        return DCERPC_BO_FLAG__LITTLE_ENDIAN;

    return DCERPC_BO_FLAG__BIG_ENDIAN;
}

inline uint16_t DceRpcNtohs(const uint16_t* ptr, const DceRpcBoFlag bo_flag)
{
    uint16_t value;

    if (ptr == nullptr)
        return 0;

    value = *ptr;

    if (bo_flag == DCERPC_BO_FLAG__NONE)
        return value;

#ifdef WORDS_BIGENDIAN
    if (bo_flag == DCERPC_BO_FLAG__BIG_ENDIAN)
#else
    if (bo_flag == DCERPC_BO_FLAG__LITTLE_ENDIAN)
#endif  /* WORDS_BIGENDIAN */
        return value;

    return ((value & 0xff00) >> 8) | ((value & 0x00ff) << 8);
}

inline uint16_t DceRpcHtons(const uint16_t* ptr, const DceRpcBoFlag bo_flag)
{
    return DceRpcNtohs(ptr, bo_flag);
}

inline uint32_t DceRpcNtohl(const uint32_t* ptr, const DceRpcBoFlag bo_flag)
{
    uint32_t value;

    if (ptr == nullptr)
        return 0;

    value = *ptr;

    if (bo_flag == DCERPC_BO_FLAG__NONE)
        return value;

#ifdef WORDS_BIGENDIAN
    if (bo_flag == DCERPC_BO_FLAG__BIG_ENDIAN)
#else
    if (bo_flag == DCERPC_BO_FLAG__LITTLE_ENDIAN)
#endif  /* WORDS_BIGENDIAN */
        return value;

    return ((value & 0xff000000) >> 24) | ((value & 0x00ff0000) >> 8) |
           ((value & 0x0000ff00) << 8)  | ((value & 0x000000ff) << 24);
}

inline uint32_t DceRpcHtonl(const uint32_t* ptr, const DceRpcBoFlag bo_flag)
{
    return DceRpcNtohl(ptr, bo_flag);
}

inline uint64_t DceRpcNtohl64(const uint64_t* ptr, const DceRpcBoFlag bo_flag)
{
    uint64_t value;

    if (ptr == nullptr)
        return 0;

    value = *ptr;

    if (bo_flag == DCERPC_BO_FLAG__NONE)
        return value;

#ifdef WORDS_BIGENDIAN
    if (bo_flag == DCERPC_BO_FLAG__BIG_ENDIAN)
#else
    if (bo_flag == DCERPC_BO_FLAG__LITTLE_ENDIAN)
#endif  /* WORDS_BIGENDIAN */
        return value;

    return ((value & 0xff00000000000000) >> 56) | ((value & 0x00ff000000000000) >> 40) |
           ((value & 0x0000ff0000000000) >> 24) | ((value & 0x000000ff00000000) >> 8) |
           ((value & 0x00000000000000ff) << 56) | ((value & 0x000000000000ff00) << 40) |
           ((value & 0x0000000000ff0000) << 24) | ((value & 0x00000000ff000000) << 8);
}

inline uint64_t DceRpcHtonl64(const uint64_t* ptr, const DceRpcBoFlag bo_flag)
{
    return DceRpcNtohl64(ptr, bo_flag);
}

inline void DCE2_CopyUuid(Uuid* dst_uuid, const Uuid* pkt_uuid, const DceRpcBoFlag byte_order)
{
    dst_uuid->time_low = DceRpcNtohl(&pkt_uuid->time_low, byte_order);
    dst_uuid->time_mid = DceRpcNtohs(&pkt_uuid->time_mid, byte_order);
    dst_uuid->time_high_and_version = DceRpcNtohs(&pkt_uuid->time_high_and_version, byte_order);
    dst_uuid->clock_seq_and_reserved = pkt_uuid->clock_seq_and_reserved;
    dst_uuid->clock_seq_low = pkt_uuid->clock_seq_low;
    memcpy(dst_uuid->node, pkt_uuid->node, sizeof(dst_uuid->node));
}

inline int DCE2_BufferIsEmpty(DCE2_Buffer* buf)
{
    if (buf == nullptr)
        return 1;
    if ((buf->data == nullptr) || (buf->len == 0))
        return 1;
    return 0;
}

inline uint32_t DCE2_BufferLength(DCE2_Buffer* buf)
{
    if (buf == nullptr)
        return 0;
    return buf->len;
}

inline uint32_t DCE2_BufferSize(DCE2_Buffer* buf)
{
    if (buf == nullptr)
        return 0;
    return buf->size;
}

inline uint8_t* DCE2_BufferData(DCE2_Buffer* buf)
{
    if (buf == nullptr)
        return nullptr;
    return buf->data;
}

inline void DCE2_BufferEmpty(DCE2_Buffer* buf)
{
    if (buf == nullptr)
        return;
    buf->len = 0;
}

inline void dce2_move(const uint8_t*& data_ptr, uint16_t& data_len, int amount)
{
    data_len -= amount;
    data_ptr += amount;
}

inline void dce2_move(const uint8_t*& data_ptr, uint32_t& data_len, int amount)
{
    data_len -= amount;
    data_ptr += amount;
}

inline void dce2_move(const uint8_t*& data_ptr, int64_t& data_len, int amount)
{
    data_len -= amount;
    data_ptr += amount;
}

#endif

smb_common.h

Path = src/service_inspectors/dce_rpc/smb_common.h

#ifndef SMB_COMMON_H
#define SMB_COMMON_H

#include <cstdint>

#define SMB_MAX_NUM_COMS   256

#define SMB_FILE_TYPE_DISK               0x0000
#define SMB_FILE_TYPE_BYTE_MODE_PIPE     0x0001
#define SMB_FILE_TYPE_MESSAGE_MODE_PIPE  0x0002
#define SMB_FILE_TYPE_PRINTER            0x0003
#define SMB_FILE_TYPE_COMMON_DEVICE      0x0004

#define SMB_FILE_ATTRIBUTE_NORMAL       0x0000
#define SMB_FILE_ATTRIBUTE_READONLY     0x0001
#define SMB_FILE_ATTRIBUTE_HIDDEN       0x0002
#define SMB_FILE_ATTRIBUTE_SYSTEM       0x0004
#define SMB_FILE_ATTRIBUTE_VOLUME       0x0008
#define SMB_FILE_ATTRIBUTE_DIRECTORY    0x0010
#define SMB_FILE_ATTRIBUTE_ARCHIVE      0x0020
#define SMB_SEARCH_ATTRIBUTE_READONLY   0x0100
#define SMB_SEARCH_ATTRIBUTE_HIDDEN     0x0200
#define SMB_SEARCH_ATTRIBUTE_SYSTEM     0x0400
#define SMB_SEARCH_ATTRIBUTE_DIRECTORY  0x1000
#define SMB_SEARCH_ATTRIBUTE_ARCHIVE    0x2000
#define SMB_FILE_ATTRIBUTE_OTHER        0xC8C0   // Reserved

#define SMB_EXT_FILE_ATTR_READONLY    0x00000001
#define SMB_EXT_FILE_ATTR_HIDDEN      0x00000002
#define SMB_EXT_FILE_ATTR_SYSTEM      0x00000004
#define SMB_EXT_FILE_ATTR_DIRECTORY   0x00000010
#define SMB_EXT_FILE_ATTR_ARCHIVE     0x00000020
#define SMB_EXT_FILE_ATTR_NORMAL      0x00000080
#define SMB_EXT_FILE_ATTR_TEMPORARY   0x00000100
#define SMB_EXT_FILE_ATTR_COMPRESSED  0x00000800
#define SMB_EXT_FILE_POSIX_SEMANTICS  0x01000000
#define SMB_EXT_FILE_BACKUP_SEMANTICS 0x02000000
#define SMB_EXT_FILE_DELETE_ON_CLOSE  0x04000000
#define SMB_EXT_FILE_SEQUENTIAL_SCAN  0x08000000
#define SMB_EXT_FILE_RANDOM_ACCESS    0x10000000
#define SMB_EXT_FILE_NO_BUFFERING     0x20000000
#define SMB_EXT_FILE_WRITE_THROUGH    0x80000000

#define NBSS_SESSION_TYPE__MESSAGE            0x00
#define NBSS_SESSION_TYPE__REQUEST            0x81
#define NBSS_SESSION_TYPE__POS_RESPONSE       0x82
#define NBSS_SESSION_TYPE__NEG_RESPONSE       0x83
#define NBSS_SESSION_TYPE__RETARGET_RESPONSE  0x84
#define NBSS_SESSION_TYPE__KEEP_ALIVE         0x85

#define DCE2_SMB_ID   0xff534d42  /* \xffSMB */
#define DCE2_SMB2_ID  0xfe534d42  /* \xfeSMB */
#define DCE2_SMB2_TRANS_ID 0xfd534d42
#define DCE2_SMB_ID_SIZE 4

// MS-FSCC Section 2.1.5 - Pathname
#define DCE2_SMB_MAX_PATH_LEN  32760
#define DCE2_SMB_MAX_COMP_LEN    255

/* SMB command codes */
#define SMB_COM_CREATE_DIRECTORY 0x00
#define SMB_COM_DELETE_DIRECTORY 0x01
#define SMB_COM_OPEN 0x02
#define SMB_COM_CREATE 0x03
#define SMB_COM_CLOSE 0x04
#define SMB_COM_FLUSH 0x05
#define SMB_COM_DELETE 0x06
#define SMB_COM_RENAME 0x07
#define SMB_COM_QUERY_INFORMATION 0x08
#define SMB_COM_SET_INFORMATION 0x09
#define SMB_COM_READ 0x0A
#define SMB_COM_WRITE 0x0B
#define SMB_COM_LOCK_BYTE_RANGE 0x0C
#define SMB_COM_UNLOCK_BYTE_RANGE 0x0D
#define SMB_COM_CREATE_TEMPORARY 0x0E
#define SMB_COM_CREATE_NEW 0x0F
#define SMB_COM_CHECK_DIRECTORY 0x10
#define SMB_COM_PROCESS_EXIT 0x11
#define SMB_COM_SEEK 0x12
#define SMB_COM_LOCK_AND_READ 0x13
#define SMB_COM_WRITE_AND_UNLOCK 0x14
#define SMB_COM_READ_RAW 0x1A
#define SMB_COM_READ_MPX 0x1B
#define SMB_COM_READ_MPX_SECONDARY 0x1C
#define SMB_COM_WRITE_RAW 0x1D
#define SMB_COM_WRITE_MPX 0x1E
#define SMB_COM_WRITE_MPX_SECONDARY 0x1F
#define SMB_COM_WRITE_COMPLETE 0x20
#define SMB_COM_QUERY_SERVER 0x21
#define SMB_COM_SET_INFORMATION2 0x22
#define SMB_COM_QUERY_INFORMATION2 0x23
#define SMB_COM_LOCKING_ANDX 0x24
#define SMB_COM_TRANSACTION 0x25
#define SMB_COM_TRANSACTION_SECONDARY 0x26
#define SMB_COM_IOCTL 0x27
#define SMB_COM_IOCTL_SECONDARY 0x28
#define SMB_COM_COPY 0x29
#define SMB_COM_MOVE 0x2A
#define SMB_COM_ECHO 0x2B
#define SMB_COM_WRITE_AND_CLOSE 0x2C
#define SMB_COM_OPEN_ANDX 0x2D
#define SMB_COM_READ_ANDX 0x2E
#define SMB_COM_WRITE_ANDX 0x2F
#define SMB_COM_NEW_FILE_SIZE 0x30
#define SMB_COM_CLOSE_AND_TREE_DISC 0x31
#define SMB_COM_TRANSACTION2 0x32
#define SMB_COM_TRANSACTION2_SECONDARY 0x33
#define SMB_COM_FIND_CLOSE2 0x34
#define SMB_COM_FIND_NOTIFY_CLOSE 0x35
#define SMB_COM_TREE_CONNECT 0x70
#define SMB_COM_TREE_DISCONNECT 0x71
#define SMB_COM_NEGOTIATE 0x72
#define SMB_COM_SESSION_SETUP_ANDX 0x73
#define SMB_COM_LOGOFF_ANDX 0x74
#define SMB_COM_TREE_CONNECT_ANDX 0x75
#define SMB_COM_SECURITY_PACKAGE_ANDX 0x7E
#define SMB_COM_QUERY_INFORMATION_DISK 0x80
#define SMB_COM_SEARCH 0x81
#define SMB_COM_FIND 0x82
#define SMB_COM_FIND_UNIQUE 0x83
#define SMB_COM_FIND_CLOSE 0x84
#define SMB_COM_NT_TRANSACT 0xA0
#define SMB_COM_NT_TRANSACT_SECONDARY 0xA1
#define SMB_COM_NT_CREATE_ANDX 0xA2
#define SMB_COM_NT_CANCEL 0xA4
#define SMB_COM_NT_RENAME 0xA5
#define SMB_COM_OPEN_PRINT_FILE 0xC0
#define SMB_COM_WRITE_PRINT_FILE 0xC1
#define SMB_COM_CLOSE_PRINT_FILE 0xC2
#define SMB_COM_GET_PRINT_QUEUE 0xC3
#define SMB_COM_READ_BULK 0xD8
#define SMB_COM_WRITE_BULK 0xD9
#define SMB_COM_WRITE_BULK_DATA 0xDA
#define SMB_COM_INVALID 0xFE
#define SMB_COM_NO_ANDX_COMMAND 0xFF

/* Size of word count field + Word count * 2 bytes + Size of byte count field */
#define SMB_COM_SIZE(wct)  (sizeof(uint8_t) + ((wct) * sizeof(uint16_t)) + sizeof(uint16_t))

#define SMB_FLG__TYPE  0x80
#define SMB_TYPE__REQUEST   0
#define SMB_TYPE__RESPONSE  1

#define SMB_FLG2__UNICODE      0x8000
#define SMB_FLG2__NT_CODES     0x4000

#define SMB_NT_STATUS_SEVERITY__SUCCESS        0
#define SMB_NT_STATUS_SEVERITY__INFORMATIONAL  1
#define SMB_NT_STATUS_SEVERITY__WARNING        2
#define SMB_NT_STATUS_SEVERITY__ERROR          3

#define SMB_NT_STATUS__SUCCESS                0x00000000
#define SMB_NT_STATUS__INVALID_DEVICE_REQUEST 0xc0000010
#define SMB_NT_STATUS__RANGE_NOT_LOCKED       0xc000007e
#define SMB_NT_STATUS__PIPE_BROKEN            0xc000014b
#define SMB_NT_STATUS__PIPE_DISCONNECTED      0xc00000b0

#define SMB_ERROR_CLASS__SUCCESS  0x00
#define SMB_ERROR_CLASS__ERRDOS   0x01
#define SMB_ERROR_CLASS__ERRSRV   0x02
#define SMB_ERROR_CLASS__ERRHRD   0x03
#define SMB_ERROR_CLASS__ERRXOS   0x04
#define SMB_ERROR_CLASS__ERRMX1   0xe1
#define SMB_ERROR_CLASS__ERRMX2   0xe2
#define SMB_ERROR_CLASS__ERRMX3   0xe3
#define SMB_ERROR_CLASS__ERRCMD   0xff

#define SMB_ERRSRV__INVALID_DEVICE      0x0007
#define SMB_ERRDOS__NOT_LOCKED          0x009e
#define SMB_ERRDOS__BAD_PIPE            0x00e6
#define SMB_ERRDOS__PIPE_NOT_CONNECTED  0x00e9
#define SMB_ERRDOS__MORE_DATA           0x00ea

enum SmbTransactionSubcommand
{
    TRANS_UNKNOWN_0000             = 0x0000,
    TRANS_SET_NMPIPE_STATE         = 0x0001,
    TRANS_UNKNOWN_0002             = 0x0002,
    TRANS_UNKNOWN_0003             = 0x0003,
    TRANS_UNKNOWN_0004             = 0x0004,
    TRANS_UNKNOWN_0005             = 0x0005,
    TRANS_UNKNOWN_0006             = 0x0006,
    TRANS_UNKNOWN_0007             = 0x0007,
    TRANS_UNKNOWN_0008             = 0x0008,
    TRANS_UNKNOWN_0009             = 0x0009,
    TRANS_UNKNOWN_000A             = 0x000A,
    TRANS_UNKNOWN_000B             = 0x000B,
    TRANS_UNKNOWN_000C             = 0x000C,
    TRANS_UNKNOWN_000D             = 0x000D,
    TRANS_UNKNOWN_000E             = 0x000E,
    TRANS_UNKNOWN_000F             = 0x000F,
    TRANS_UNKNOWN_0010             = 0x0010,
    TRANS_RAW_READ_NMPIPE          = 0x0011,
    TRANS_UNKNOWN_0012             = 0x0012,
    TRANS_UNKNOWN_0013             = 0x0013,
    TRANS_UNKNOWN_0014             = 0x0014,
    TRANS_UNKNOWN_0015             = 0x0015,
    TRANS_UNKNOWN_0016             = 0x0016,
    TRANS_UNKNOWN_0017             = 0x0017,
    TRANS_UNKNOWN_0018             = 0x0018,
    TRANS_UNKNOWN_0019             = 0x0019,
    TRANS_UNKNOWN_001A             = 0x001A,
    TRANS_UNKNOWN_001B             = 0x001B,
    TRANS_UNKNOWN_001C             = 0x001C,
    TRANS_UNKNOWN_001D             = 0x001D,
    TRANS_UNKNOWN_001E             = 0x001E,
    TRANS_UNKNOWN_001F             = 0x001F,
    TRANS_UNKNOWN_0020             = 0x0020,
    TRANS_QUERY_NMPIPE_STATE       = 0x0021,
    TRANS_QUERY_NMPIPE_INFO        = 0x0022,
    TRANS_PEEK_NMPIPE              = 0x0023,
    TRANS_UNKNOWN_0024             = 0x0024,
    TRANS_UNKNOWN_0025             = 0x0025,
    TRANS_TRANSACT_NMPIPE          = 0x0026,
    TRANS_UNKNOWN_0027             = 0x0027,
    TRANS_UNKNOWN_0028             = 0x0028,
    TRANS_UNKNOWN_0029             = 0x0029,
    TRANS_UNKNOWN_002A             = 0x002A,
    TRANS_UNKNOWN_002B             = 0x002B,
    TRANS_UNKNOWN_002C             = 0x002C,
    TRANS_UNKNOWN_002D             = 0x002D,
    TRANS_UNKNOWN_002E             = 0x002E,
    TRANS_UNKNOWN_002F             = 0x002F,
    TRANS_UNKNOWN_0030             = 0x0030,
    TRANS_RAW_WRITE_NMPIPE         = 0x0031,
    TRANS_UNKNOWN_0032             = 0x0032,
    TRANS_UNKNOWN_0033             = 0x0033,
    TRANS_UNKNOWN_0034             = 0x0034,
    TRANS_UNKNOWN_0035             = 0x0035,
    TRANS_READ_NMPIPE              = 0x0036,
    TRANS_WRITE_NMPIPE             = 0x0037,
    TRANS_UNKNOWN_0038             = 0x0038,
    TRANS_UNKNOWN_0039             = 0x0039,
    TRANS_UNKNOWN_003A             = 0x003A,
    TRANS_UNKNOWN_003B             = 0x003B,
    TRANS_UNKNOWN_003C             = 0x003C,
    TRANS_UNKNOWN_003D             = 0x003D,
    TRANS_UNKNOWN_003E             = 0x003E,
    TRANS_UNKNOWN_003F             = 0x003F,
    TRANS_UNKNOWN_0040             = 0x0040,
    TRANS_UNKNOWN_0041             = 0x0041,
    TRANS_UNKNOWN_0042             = 0x0042,
    TRANS_UNKNOWN_0043             = 0x0043,
    TRANS_UNKNOWN_0044             = 0x0044,
    TRANS_UNKNOWN_0045             = 0x0045,
    TRANS_UNKNOWN_0046             = 0x0046,
    TRANS_UNKNOWN_0047             = 0x0047,
    TRANS_UNKNOWN_0048             = 0x0048,
    TRANS_UNKNOWN_0049             = 0x0049,
    TRANS_UNKNOWN_004A             = 0x004A,
    TRANS_UNKNOWN_004B             = 0x004B,
    TRANS_UNKNOWN_004C             = 0x004C,
    TRANS_UNKNOWN_004D             = 0x004D,
    TRANS_UNKNOWN_004E             = 0x004E,
    TRANS_UNKNOWN_004F             = 0x004F,
    TRANS_UNKNOWN_0050             = 0x0050,
    TRANS_UNKNOWN_0051             = 0x0051,
    TRANS_UNKNOWN_0052             = 0x0052,
    TRANS_WAIT_NMPIPE              = 0x0053,
    TRANS_CALL_NMPIPE              = 0x0054,
    TRANS_SUBCOM_MAX               = 0x0055
};

enum SmbTransaction2Subcommand
{
    TRANS2_OPEN2                        = 0x0000,
    TRANS2_FIND_FIRST2                  = 0x0001,
    TRANS2_FIND_NEXT2                   = 0x0002,
    TRANS2_QUERY_FS_INFORMATION         = 0x0003,
    TRANS2_SET_FS_INFORMATION           = 0x0004,
    TRANS2_QUERY_PATH_INFORMATION       = 0x0005,
    TRANS2_SET_PATH_INFORMATION         = 0x0006,
    TRANS2_QUERY_FILE_INFORMATION       = 0x0007,
    TRANS2_SET_FILE_INFORMATION         = 0x0008,
    TRANS2_FSCTL                        = 0x0009,
    TRANS2_IOCTL2                       = 0x000A,
    TRANS2_FIND_NOTIFY_FIRST            = 0x000B,
    TRANS2_FIND_NOTIFY_NEXT             = 0x000C,
    TRANS2_CREATE_DIRECTORY             = 0x000D,
    TRANS2_SESSION_SETUP                = 0x000E,
    TRANS2_UNKNOWN_000F                 = 0x000F,
    TRANS2_GET_DFS_REFERRAL             = 0x0010,
    TRANS2_REPORT_DFS_INCONSISTENCY     = 0x0011,
    TRANS2_SUBCOM_MAX                   = 0x0012
};

/********************************************************************
 * Structures and inline accessor functions
 ********************************************************************/
/* Pack the structs since we'll be laying them on top of packet data */
#pragma pack(1)

/********************************************************************
 * NetBIOS Session Service header
 ********************************************************************/
struct NbssHdr
{
    uint8_t type;
    uint8_t flags;   /* Treat flags as the upper byte to length */
    uint16_t length;
};

struct SmbNtHdr
{
    uint8_t smb_idf[4];             /* contains 0xFF, 'SMB' */
    uint8_t smb_com;                /* command code */
    union
    {
        struct
        {
            uint8_t smb_class;      /* dos error class */
            uint8_t smb_res;        /* reserved for future */
            uint16_t smb_code;      /* dos error code */
        } smb_status;
        uint32_t nt_status;         /* nt status */
    } smb_status;
    uint8_t smb_flg;                /* flags */
    uint16_t smb_flg2;              /* flags */
    uint16_t smb_pid_high;
    uint64_t smb_signature;
    uint16_t smb_res;               /* reserved for future */
    uint16_t smb_tid;               /* tree id */
    uint16_t smb_pid;               /* caller's process id */
    uint16_t smb_uid;               /* authenticated user id */
    uint16_t smb_mid;               /* multiplex id */
};

/* For server empty responses indicating client error or interim response */
struct SmbEmptyCom
{
    uint8_t smb_wct;     /* value = 0 */
    uint16_t smb_bcc;    /* value = 0 */
};

/********************************************************************
 * Common fields to all commands
 ********************************************************************/
struct SmbCommon
{
    uint8_t smb_wct;
};

inline uint8_t SmbWct(const SmbCommon* hdr)
{
    return hdr->smb_wct;
}

/* Common fields to all AndX commands */
struct SmbAndXCommon
{
    uint8_t smb_wct;
    uint8_t smb_com2;      /* secondary (X) command, 0xFF = none */
    uint8_t smb_reh2;      /* reserved (must be zero) */
    uint16_t smb_off2;     /* offset (from SMB hdr start) to next cmd (@smb_wct) */
};

struct Smb2TransformHdr
{
    uint8_t protocolid[4];
    uint8_t signature[16];
    uint8_t nonce[16];
    uint32_t orig_msg_size;
    uint16_t reserved;
    uint16_t flags;
    uint64_t session_id;
};

//NbssLen should be used by SMB1
inline uint32_t NbssLen(const NbssHdr* nb)
{
    // Treat first bit of flags as the upper byte to length
    //[MS-SMB] 2.1 Transport - Length can be maximum 0x1FFFF
    // The left operand of '&' is a garbage value
    return ((nb->flags & 0x01) << 16) | ntohs(nb->length);  // ... FIXIT-W
}

// NbssLen2 should be used by SMB2/SMB3
inline uint32_t NbssLen2(const NbssHdr* nb)
{
    // The Length is 3 bytes. [MS-SMB2] 2.1 Transport
    // The left operand of '<<' is a garbage value
    return ((nb->flags << 16) | ntohs(nb->length));  // ... FIXIT-W
}

inline uint8_t NbssType(const NbssHdr* nb)
{
    return nb->type;
}

inline uint32_t SmbTransformId(const Smb2TransformHdr* hdr)
{
    const uint8_t* id = (const uint8_t*)hdr->protocolid;
    return *id << 24 | *(id + 1) << 16 | *(id + 2) << 8 | *(id + 3);
}

inline uint32_t SmbId(const SmbNtHdr* hdr)
{
    const uint8_t* idf = (const uint8_t*)hdr->smb_idf;
    return *idf << 24 | *(idf + 1) << 16 | *(idf + 2) << 8 | *(idf + 3);
}

inline uint8_t SmbEmptyComWct(const SmbEmptyCom* ec)
{
    return ec->smb_wct;
}

inline uint16_t SmbBcc(const uint8_t* ptr, uint16_t com_size)
{
    /* com_size must be at least the size of the command encasing */
    if (com_size < sizeof(SmbEmptyCom))
        return 0;

    return snort::alignedNtohs((const uint16_t*)(ptr + com_size - sizeof(uint16_t)));
}

inline uint16_t SmbEmptyComBcc(const SmbEmptyCom* ec)
{
    return snort::alignedNtohs(&ec->smb_bcc);
}

inline int SmbType(const SmbNtHdr* hdr)
{
    // Access to field 'smb_flg' results in a dereference of a null pointer
    // (loaded from variable 'hdr')
    if (hdr->smb_flg & SMB_FLG__TYPE)  // ... FIXIT-W
        return SMB_TYPE__RESPONSE;

    return SMB_TYPE__REQUEST;
}

inline uint8_t SmbAndXCom2(const SmbAndXCommon* andx)
{
    return andx->smb_com2;
}

inline uint16_t SmbAndXOff2(const SmbAndXCommon* andx)
{
    return snort::alignedNtohs(&andx->smb_off2);
}

/* SMB formats (smb_fmt) Dialect, Pathname and ASCII are all
 * null terminated ASCII strings unless Unicode is specified
 * in the NT LM 1.0 SMB header in which case they are null
 * terminated unicode strings
 */
#define SMB_FMT__DATA_BLOCK  1
#define SMB_FMT__DIALECT     2
#define SMB_FMT__ASCII       4

inline bool SmbFmtDataBlock(const uint8_t fmt)
{
    return fmt == SMB_FMT__DATA_BLOCK ? true : false;
}

inline bool SmbFmtDialect(const uint8_t fmt)
{
    return fmt == SMB_FMT__DIALECT ? true : false;
}

inline bool SmbFmtAscii(const uint8_t fmt)
{
    return fmt == SMB_FMT__ASCII ? true : false;
}

#pragma pack()

#endif

smb_message.h

Path = src/service_inspectors/dce_rpc/smb_message.h

#ifndef SMB_MESSAGE_H
#define SMB_MESSAGE_H

#include <cstdint>

#include "protocols/packet.h"
#include "smb_common.h"

#pragma pack(1)

/********************************************************************
 * SMB_COM_OPEN
 ********************************************************************/
struct SmbOpenReq   /* smb_wct = 2 */
{
    uint8_t smb_wct;      /* count of 16-bit words that follow */
    uint16_t smb_mode;    /* r/w/share */
    uint16_t smb_attr;    /* attribute */
    uint16_t smb_bcc;     /* min = 2 */
};

struct SmbOpenResp   /* smb_wct = 7 */
{
    uint8_t smb_wct;      /* count of 16-bit words that follow */
    uint16_t smb_fid;     /* file handle */
    uint16_t smb_attr;    /* attribute */
    uint32_t smb_time;    /* time1 low */
    uint32_t smb_file_size;   /* file size low */
    uint16_t smb_access;  /* access allowed */
    uint16_t smb_bcc;     /* must be 0 */
};

#define SMB_OPEN_ACCESS_MODE__READ        0x0000
#define SMB_OPEN_ACCESS_MODE__WRITE       0x0001
#define SMB_OPEN_ACCESS_MODE__READ_WRITE  0x0002
#define SMB_OPEN_ACCESS_MODE__EXECUTE     0x0003

inline uint16_t SmbOpenRespFid(const SmbOpenResp* resp)
{
    return snort::alignedNtohs(&resp->smb_fid);
}

inline uint32_t SmbOpenRespFileSize(const SmbOpenResp* resp)
{
    return snort::alignedNtohl(&resp->smb_file_size);
}

inline uint16_t SmbOpenRespFileAttrs(const SmbOpenResp* resp)
{
    return snort::alignedNtohs(&resp->smb_attr);
}

inline bool SmbFileAttrsDirectory(const uint16_t file_attrs)
{
    if (file_attrs & SMB_FILE_ATTRIBUTE_DIRECTORY)
        return true;
    return false;
}

inline uint16_t SmbOpenRespAccessMode(const SmbOpenResp* resp)
{
    return snort::alignedNtohs(&resp->smb_access);
}

inline bool SmbOpenForWriting(const uint16_t access_mode)
{
    return access_mode == SMB_OPEN_ACCESS_MODE__WRITE;
}

/********************************************************************
 * SMB_COM_CREATE
 ********************************************************************/
struct SmbCreateReq   /* smb_wct = 3 */
{
    uint8_t smb_wct;
    uint16_t smb_file_attrs;
    uint32_t smb_creation_time;
    uint16_t smb_bcc;
};

struct SmbCreateResp   /* smb_wct = 1 */
{
    uint8_t smb_wct;
    uint16_t smb_fid;
    uint16_t smb_bcc;
};

inline uint16_t SmbCreateReqFileAttrs(const SmbCreateReq* req)
{
    return snort::alignedNtohs(&req->smb_file_attrs);
}

inline bool SmbAttrDirectory(const uint16_t file_attrs)
{
    if (file_attrs & SMB_FILE_ATTRIBUTE_DIRECTORY)
        return true;
    return false;
}

inline uint16_t SmbCreateRespFid(const SmbCreateResp* resp)
{
    return snort::alignedNtohs(&resp->smb_fid);
}

/********************************************************************
 * SMB_COM_CLOSE
 ********************************************************************/
struct SmbCloseReq   /* smb_wct = 3 */
{
    uint8_t smb_wct;      /* count of 16-bit words that follow */
    uint16_t smb_fid;     /* file handle */
    uint16_t smb_tlow;    /* time low */
    uint16_t smb_thigh;   /* time high */
    uint16_t smb_bcc;     /* must be 0 */
};

struct SmbCloseResp   /* smb_wct = 0 */
{
    uint8_t smb_wct;      /* count of 16-bit words that follow */
    uint16_t smb_bcc;     /* must be 0 */
};

inline uint16_t SmbCloseReqFid(const SmbCloseReq* req)
{
    return snort::alignedNtohs(&req->smb_fid);
}

/********************************************************************
 * SMB_COM_DELETE
 ********************************************************************/
struct SmbDeleteReq  /* smb_wct = 1 */
{
    uint8_t smb_wct;
    uint16_t smb_search_attrs;
    uint16_t smb_bcc;
};

/********************************************************************
 * SMB_COM_READ
 ********************************************************************/
struct SmbReadReq   /* smb_wct = 5 */
{
    uint8_t smb_wct;      /* count of 16-bit words that follow */
    uint16_t smb_fid;     /* file handle */
    uint16_t smb_cnt;     /* count of bytes */
    uint32_t smb_off;     /* offset */
    uint16_t smb_left;    /* count left */
    uint16_t smb_bcc;     /* must be 0 */
};

struct SmbReadResp   /* smb_wct = 5 */
{
    uint8_t smb_wct;      /* count of 16-bit words that follow */
    uint16_t smb_cnt;     /* count */
    uint16_t smb_res[4];  /* reserved (MBZ) */
    uint16_t smb_bcc;     /* length of data + 3 */
};

inline uint16_t SmbReadReqFid(const SmbReadReq* req)
{
    return snort::alignedNtohs(&req->smb_fid);
}

inline uint32_t SmbReadReqOffset(const SmbReadReq* req)
{
    return snort::alignedNtohl(&req->smb_off);
}

inline uint16_t SmbReadRespCount(const SmbReadResp* resp)
{
    return snort::alignedNtohs(&resp->smb_cnt);
}

/********************************************************************
 * SMB_COM_WRITE
 ********************************************************************/
struct SmbWriteReq   /* smb_wct = 5 */
{
    uint8_t smb_wct;      /* count of 16-bit words that follow */
    uint16_t smb_fid;     /* file handle */
    uint16_t smb_cnt;     /* count of bytes */
    uint32_t smb_offset;  /* file offset in bytes */
    uint16_t smb_left;    /* count left */
    uint16_t smb_bcc;     /* length of data + 3 */
};

struct SmbWriteResp   /* smb_wct = 1 */
{
    uint8_t smb_wct;      /* count of 16-bit words that follow */
    uint16_t smb_cnt;     /* count */
    uint16_t smb_bcc;     /* must be 0 */
};

inline uint16_t SmbWriteReqFid(const SmbWriteReq* req)
{
    return snort::alignedNtohs(&req->smb_fid);
}

inline uint16_t SmbWriteReqCount(const SmbWriteReq* req)
{
    return snort::alignedNtohs(&req->smb_cnt);
}

inline uint32_t SmbWriteReqOffset(const SmbWriteReq* req)
{
    return snort::alignedNtohl(&req->smb_offset);
}

inline uint16_t SmbWriteRespCount(const SmbWriteResp* resp)
{
    return snort::alignedNtohs(&resp->smb_cnt);
}

/********************************************************************
 * SMB_COM_CREATE_NEW
 ********************************************************************/
struct SmbCreateNewReq   /* smb_wct = 3 */
{
    uint8_t smb_wct;
    uint16_t smb_file_attrs;
    uint32_t smb_creation_time;
    uint16_t smb_bcc;
};

struct SmbCreateNewResp   /* smb_wct = 1 */
{
    uint8_t smb_wct;
    uint16_t smb_fid;
    uint16_t smb_bcc;
};

inline uint16_t SmbCreateNewReqFileAttrs(const SmbCreateNewReq* req)
{
    return snort::alignedNtohs(&req->smb_file_attrs);
}

inline uint16_t SmbCreateNewRespFid(const SmbCreateNewResp* resp)
{
    return snort::alignedNtohs(&resp->smb_fid);
}

/********************************************************************
 * SMB_COM_LOCK_AND_READ
 ********************************************************************/
struct SmbLockAndReadReq   /* smb_wct = 5 */
{
    uint8_t smb_wct;      /* count of 16-bit words that follow */
    uint16_t smb_fid;
    uint16_t smb_cnt;
    uint32_t smb_read_offset;
    uint16_t smb_remaining;
    uint16_t smb_bcc;     /* must be 0 */
};

struct SmbLockAndReadResp   /* smb_wct = 5 */
{
    uint8_t smb_wct;
    uint16_t smb_cnt;
    uint16_t reserved[4];
    uint16_t smb_bcc;
};

inline uint16_t SmbLockAndReadReqFid(const SmbLockAndReadReq* req)
{
    return snort::alignedNtohs(&req->smb_fid);
}

inline uint32_t SmbLockAndReadReqOffset(const SmbLockAndReadReq* req)
{
    return snort::alignedNtohl(&req->smb_read_offset);
}

inline uint16_t SmbLockAndReadRespCount(const SmbLockAndReadResp* resp)
{
    return snort::alignedNtohs(&resp->smb_cnt);
}

/********************************************************************
 * SMB_COM_WRITE_AND_UNLOCK
 ********************************************************************/
struct SmbWriteAndUnlockReq
{
    uint8_t smb_wct;
    uint16_t smb_fid;
    uint16_t smb_cnt;
    uint32_t smb_write_offset;
    uint16_t smb_estimate_of_remaining;
    uint16_t smb_bcc;
};

struct SmbWriteAndUnlockResp   /* smb_wct = 1 */
{
    uint8_t smb_wct;      /* count of 16-bit words that follow */
    uint16_t smb_cnt;     /* count */
    uint16_t smb_bcc;     /* must be 0 */
};

inline uint16_t SmbWriteAndUnlockReqFid(const SmbWriteAndUnlockReq* req)
{
    return snort::alignedNtohs(&req->smb_fid);
}

inline uint16_t SmbWriteAndUnlockReqCount(const SmbWriteAndUnlockReq* req)
{
    return snort::alignedNtohs(&req->smb_cnt);
}

inline uint32_t SmbWriteAndUnlockReqOffset(const SmbWriteAndUnlockReq* req)
{
    return snort::alignedNtohl(&req->smb_write_offset);
}

/********************************************************************
 * SMB_COM_OPEN_ANDX
 ********************************************************************/
struct SmbOpenAndXReq   /* smb_wct = 15 */
{
    uint8_t smb_wct;         /* count of 16-bit words that follow */
    uint8_t smb_com2;        /* secondary (X) command, 0xFF = none */
    uint8_t smb_reh2;        /* reserved (must be zero) */
    uint16_t smb_off2;       /* offset (from SMB hdr start) to next cmd (@smb_wct) */
    uint16_t smb_flags;      /* additional information:
                                bit 0 - if set, return additional information
                                bit 1 - if set, set single user total file lock (if only access)
                                bit 2 - if set, the server should notify the consumer on any
                                        action which can modify the file (delete, setattrib,
                                        rename, etc.). if not set, the server need only notify
                                        the consumer on another open request. This bit only has
                                        meaning if bit 1 is set. */
    uint16_t smb_mode;       /* file open mode */
    uint16_t smb_sattr;      /* search attributes */
    uint16_t smb_attr;       /* file attributes (for create) */
    uint32_t smb_time;       /* create time */
    uint16_t smb_ofun;       /* open function */
    uint32_t smb_size;       /* bytes to reserve on "create" or "truncate" */
    uint32_t smb_timeout;    /* max milliseconds to wait for resource to open */
    uint32_t smb_rsvd;       /* reserved (must be zero) */
    uint16_t smb_bcc;        /* minimum value = 1 */
};

struct SmbOpenAndXResp   /* smb_wct = 15 */
{
    uint8_t smb_wct;         /* count of 16-bit words that follow */
    uint8_t smb_com2;        /* secondary (X) command, 0xFF = none */
    uint8_t smb_res2;        /* reserved (pad to word) */
    uint16_t smb_off2;       /* offset (from SMB hdr start) to next cmd (@smb_wct) */
    uint16_t smb_fid;        /* file handle */
    uint16_t smb_attribute;  /* attributes of file or device */
    uint32_t smb_time;       /* last modification time */
    uint32_t smb_size;       /* current file size */
    uint16_t smb_access;     /* access permissions actually allowed */
    uint16_t smb_type;       /* file type */
    uint16_t smb_state;      /* state of IPC device (e.g. pipe) */
    uint16_t smb_action;     /* action taken */
    uint32_t smb_fileid;     /* server unique file id */
    uint16_t smb_rsvd;       /* reserved */
    uint16_t smb_bcc;        /* value = 0 */
};

inline uint32_t SmbOpenAndXReqAllocSize(const SmbOpenAndXReq* req)
{
    return snort::alignedNtohl(&req->smb_size);
}

inline uint16_t SmbOpenAndXReqFileAttrs(const SmbOpenAndXReq* req)
{
    return snort::alignedNtohs(&req->smb_attr);
}

inline uint16_t SmbOpenAndXRespFid(const SmbOpenAndXResp* resp)
{
    return snort::alignedNtohs(&resp->smb_fid);
}

inline uint16_t SmbOpenAndXRespFileAttrs(const SmbOpenAndXResp* resp)
{
    return snort::alignedNtohs(&resp->smb_attribute);
}

inline uint32_t SmbOpenAndXRespFileSize(const SmbOpenAndXResp* resp)
{
    return snort::alignedNtohl(&resp->smb_size);
}

inline uint16_t SmbOpenAndXRespResourceType(const SmbOpenAndXResp* resp)
{
    return snort::alignedNtohs(&resp->smb_type);
}

#define SMB_OPEN_RESULT__EXISTED    0x0001
#define SMB_OPEN_RESULT__CREATED    0x0002
#define SMB_OPEN_RESULT__TRUNCATED  0x0003

inline uint16_t SmbOpenAndXRespOpenResults(const SmbOpenAndXResp* resp)
{
    return snort::alignedNtohs(&resp->smb_action);
}

inline bool SmbOpenResultRead(const uint16_t open_results)
{
    return ((open_results & 0x00FF) == SMB_OPEN_RESULT__EXISTED);
}

inline bool SmbResourceTypeDisk(const uint16_t resource_type)
{
    return resource_type == SMB_FILE_TYPE_DISK;
}

/********************************************************************
 * SMB_COM_READ_ANDX
 ********************************************************************/
struct SmbReadAndXReq   /* smb_wct = 10 */
{
    uint8_t smb_wct;         /* count of 16-bit words that follow */
    uint8_t smb_com2;        /* secondary (X) command, 0xFF = none */
    uint8_t smb_reh2;        /* reserved (must be zero) */
    uint16_t smb_off2;       /* offset (from SMB hdr start) to next cmd (@smb_wct) */
    uint16_t smb_fid;        /* file handle */
    uint32_t smb_offset;     /* offset in file to begin read */
    uint16_t smb_maxcnt;     /* max number of bytes to return */
    uint16_t smb_mincnt;     /* min number of bytes to return */
    uint32_t smb_timeout;    /* number of milliseconds to wait for completion */
    uint16_t smb_countleft;  /* bytes remaining to satisfy user’s request */
    uint16_t smb_bcc;        /* value = 0 */
};

struct SmbReadAndXExtReq   /* smb_wct = 12 */
{
    uint8_t smb_wct;         /* count of 16-bit words that follow */
    uint8_t smb_com2;        /* secondary (X) command, 0xFF = none */
    uint8_t smb_reh2;        /* reserved (must be zero) */
    uint16_t smb_off2;       /* offset (from SMB hdr start) to next cmd (@smb_wct) */
    uint16_t smb_fid;        /* file handle */
    uint32_t smb_offset;     /* low offset in file to begin read */
    uint16_t smb_maxcnt;     /* max number of bytes to return */
    uint16_t smb_mincnt;     /* min number of bytes to return */
    uint32_t smb_timeout;    /* number of milliseconds to wait for completion */
    uint16_t smb_countleft;  /* bytes remaining to satisfy user’s request */
    uint32_t smb_off_high;   /* high offset in file to begin read */
    uint16_t smb_bcc;        /* value = 0 */
};

struct SmbReadAndXResp    /* smb_wct = 12 */
{
    uint8_t smb_wct;         /* count of 16-bit words that follow */
    uint8_t smb_com2;        /* secondary (X) command, 0xFF = none */
    uint8_t smb_res2;        /* reserved (pad to word) */
    uint16_t smb_off2;       /* offset (from SMB hdr start) to next cmd (@smb_wct) */
    uint16_t smb_remaining;  /* bytes remaining to be read (pipes/devices only) */
    uint32_t smb_rsvd;       /* reserved */
    uint16_t smb_dsize;      /* number of data bytes (minimum value = 0) */
    uint16_t smb_doff;       /* offset (from start of SMB hdr) to data bytes */
    uint16_t smb_dsize_high; /* high bytes of data size */
    uint32_t smb_rsvd1;      /* reserved */
    uint32_t smb_rsvd2;      /* reserved */
    uint16_t smb_bcc;        /* total bytes (including pad bytes) following */
};

inline uint16_t SmbReadAndXReqFid(const SmbReadAndXReq* req)
{
    return snort::alignedNtohs(&req->smb_fid);
}

inline uint64_t SmbReadAndXReqOffset(const SmbReadAndXExtReq* req)
{
    if (req->smb_wct == 10)
        return (uint64_t)snort::alignedNtohl(&req->smb_offset);

    return (uint64_t)snort::alignedNtohl(&req->smb_off_high) << 32
           | (uint64_t)snort::alignedNtohl(&req->smb_offset);
}

inline uint16_t SmbReadAndXRespDataOff(const SmbReadAndXResp* req)
{
    return snort::alignedNtohs(&req->smb_doff);
}

inline uint32_t SmbReadAndXRespDataCnt(const SmbReadAndXResp* resp)
{
    return (uint32_t)snort::alignedNtohs(&resp->smb_dsize_high) << 16
           | (uint32_t)snort::alignedNtohs(&resp->smb_dsize);
}

/********************************************************************
 * SMB_COM_WRITE_ANDX
 ********************************************************************/
struct SmbWriteAndXReq   /* smb_wct = 12 */
{
    uint8_t smb_wct;         /* count of 16-bit words that follow */
    uint8_t smb_com2;        /* secondary (X) command, 0xFF = none */
    uint8_t smb_reh2;        /* reserved (must be zero) */
    uint16_t smb_off2;       /* offset (from SMB hdr start) to next cmd (@smb_wct) */
    uint16_t smb_fid;        /* file handle */
    uint32_t smb_offset;     /* offset in file to begin write */
    uint32_t smb_timeout;    /* number of milliseconds to wait for completion */
    uint16_t smb_wmode;      /* write mode:
                                bit0 - complete write before return (write through)
                                bit1 - return smb_remaining (pipes/devices only)
                                bit2 - use WriteRawNamedPipe (pipes only)
                                bit3 - this is the start of a message (pipes only) */
    uint16_t smb_countleft;  /* bytes remaining to write to satisfy user’s request */
    uint16_t smb_dsize_high; /* high bytes of data size */
    uint16_t smb_dsize;      /* number of data bytes in buffer (min value = 0) */
    uint16_t smb_doff;       /* offset (from start of SMB hdr) to data bytes */
    uint16_t smb_bcc;        /* total bytes (including pad bytes) following */
};

struct SmbWriteAndXExtReq   /* smb_wct = 14 */
{
    uint8_t smb_wct;         /* count of 16-bit words that follow */
    uint8_t smb_com2;        /* secondary (X) command, 0xFF = none */
    uint8_t smb_reh2;        /* reserved (must be zero) */
    uint16_t smb_off2;       /* offset (from SMB hdr start) to next cmd (@smb_wct) */
    uint16_t smb_fid;        /* file handle */
    uint32_t smb_offset;     /* low offset in file to begin write */
    uint32_t smb_timeout;    /* number of milliseconds to wait for completion */
    uint16_t smb_wmode;      /* write mode:
                                bit0 - complete write before return (write through)
                                bit1 - return smb_remaining (pipes/devices only)
                                bit2 - use WriteRawNamedPipe (pipes only)
                                bit3 - this is the start of a message (pipes only) */
    uint16_t smb_countleft;  /* bytes remaining to write to satisfy user’s request */
    uint16_t smb_dsize_high; /* high bytes of data size */
    uint16_t smb_dsize;      /* number of data bytes in buffer (min value = 0) */
    uint16_t smb_doff;       /* offset (from start of SMB hdr) to data bytes */
    uint32_t smb_off_high;   /* high offset in file to begin write */
    uint16_t smb_bcc;        /* total bytes (including pad bytes) following */
};

struct SmbWriteAndXResp   /* smb_wct = 6 */
{
    uint8_t smb_wct;         /* count of 16-bit words that follow */
    uint8_t smb_com2;        /* secondary (X) command, 0xFF = none */
    uint8_t smb_res2;        /* reserved (pad to word) */
    uint16_t smb_off2;       /* offset (from SMB hdr start) to next cmd (@smb_wct) */
    uint16_t smb_count;      /* number of bytes written */
    uint16_t smb_remaining;  /* bytes remaining to be read (pipes/devices only) */
    uint16_t smb_count_high; /* high order bytes of data count */
    uint16_t smb_rsvd;       /* reserved */
    uint16_t smb_bcc;        /* value = 0 */
};

inline uint16_t SmbWriteAndXReqFid(const SmbWriteAndXReq* req)
{
    return snort::alignedNtohs(&req->smb_fid);
}

inline uint16_t SmbWriteAndXReqDataOff(const SmbWriteAndXReq* req)
{
    return snort::alignedNtohs(&req->smb_doff);
}

inline uint16_t SmbWriteAndXReqRemaining(const SmbWriteAndXReq* req)
{
    return snort::alignedNtohs(&req->smb_countleft);
}

inline uint64_t SmbWriteAndXReqOffset(const SmbWriteAndXExtReq* req)
{
    if (req->smb_wct == 12)
        return (uint64_t)snort::alignedNtohl(&req->smb_offset);

    return (uint64_t)snort::alignedNtohl(&req->smb_off_high) << 32
           | (uint64_t)snort::alignedNtohl(&req->smb_offset);
}

inline uint32_t SmbWriteAndXReqDataCnt(const SmbWriteAndXReq* req)
{
    return (uint32_t)snort::alignedNtohs(&req->smb_dsize_high) << 16
           | (uint32_t)snort::alignedNtohs(&req->smb_dsize);
}

inline uint16_t SmbWriteAndXReqWriteMode(const SmbWriteAndXReq* req)
{
    return snort::alignedNtohs(&req->smb_wmode);
}

inline bool SmbWriteAndXReqStartRaw(const SmbWriteAndXReq* req)
{
    return ((snort::alignedNtohs(&req->smb_wmode) & 0x000c) == 0x000c) ? true : false;
}

inline bool SmbWriteAndXReqRaw(const SmbWriteAndXReq* req)
{
    return ((snort::alignedNtohs(&req->smb_wmode) & 0x000c) == 0x0004) ? true : false;
}

inline uint16_t SmbWriteAndXRespCnt(const SmbWriteAndXResp* resp)
{
    return snort::alignedNtohs(&resp->smb_count);
}

/********************************************************************
 * SMB_COM_SESSION_SETUP_ANDX
 ********************************************************************/
struct SmbLm10_SessionSetupAndXReq   /* smb_wct = 10 */
{
    uint8_t smb_wct;       /* count of 16-bit words that follow */
    uint8_t smb_com2;      /* secondary (X) command, 0xFF = none */
    uint8_t smb_reh2;      /* reserved (must be zero) */
    uint16_t smb_off2;     /* offset (from SMB hdr start) to next cmd (@smb_wct) */
    uint16_t smb_bufsize;  /* the consumers max buffer size */
    uint16_t smb_mpxmax;   /* actual maximum multiplexed pending requests */
    uint16_t smb_vc_num;   /* 0 = first (only), non zero - additional VC number */
    uint32_t smb_sesskey;  /* Session Key (valid only if smb_vc_num != 0) */
    uint16_t smb_apasslen; /* size of account password (smb_apasswd) */
    uint32_t smb_rsvd;     /* reserved */
    uint16_t smb_bcc;      /* minimum value = 0 */
};

inline uint16_t SmbSessionSetupAndXReqMaxMultiplex(const SmbLm10_SessionSetupAndXReq* req)
{
    return snort::alignedNtohs(&req->smb_mpxmax);
}

/* Extended request as defined in NT LM 1.0 document */
struct SmbNt10_SessionSetupAndXReq   /* smb_wct = 13 */
{
    uint8_t smb_wct;              /* count of 16-bit words that follow */
    uint8_t smb_com2;             /* secondary (X) command, 0xFF = none */
    uint8_t smb_reh2;             /* reserved (must be zero) */
    uint16_t smb_off2;            /* offset (from SMB hdr start) to next cmd (@smb_wct) */
    uint16_t smb_bufsize;         /* the consumers max buffer size */
    uint16_t smb_mpxmax;          /* actual maximum multiplexed pending requests */
    uint16_t smb_vc_num;          /* 0 = first (only), non zero - additional VC number */
    uint32_t smb_sesskey;         /* Session Key (valid only if smb_vc_num != 0) */
    uint16_t smb_oem_passlen;     /* case insensitive password length */
    uint16_t smb_unicode_passlen; /* case sensitive password length */
    uint32_t smb_rsvd;            /* reserved */
    uint32_t smb_cap;             /* capabilities */
    uint16_t smb_bcc;             /* minimum value = 0 */
};

inline uint16_t SmbNt10SessionSetupAndXReqOemPassLen(const SmbNt10_SessionSetupAndXReq* req)
{
    return snort::alignedNtohs(&req->smb_oem_passlen);
}

inline uint16_t SmbNt10SessionSetupAndXReqUnicodePassLen(const SmbNt10_SessionSetupAndXReq* req)
{
    return snort::alignedNtohs(&req->smb_unicode_passlen);
}

/* Extended request for security blob */
struct SmbNt10_SessionSetupAndXExtReq   /* smb_wct = 12 */
{
    uint8_t smb_wct;          /* count of 16-bit words that follow */
    uint8_t smb_com2;         /* secondary (X) command, 0xFF = none */
    uint8_t smb_reh2;         /* reserved (must be zero) */
    uint16_t smb_off2;        /* offset (from SMB hdr start) to next cmd (@smb_wct) */
    uint16_t smb_bufsize;     /* the consumers max buffer size */
    uint16_t smb_mpxmax;      /* actual maximum multiplexed pending requests */
    uint16_t smb_vc_num;      /* 0 = first (only), non zero - additional VC number */
    uint32_t smb_sesskey;     /* Session Key (valid only if smb_vc_num != 0) */
    uint16_t smb_blob_len;    /* length of security blob */
    uint32_t smb_rsvd;        /* reserved */
    uint32_t smb_cap;         /* capabilities */
    uint16_t smb_bcc;         /* minimum value = 0 */
};

inline uint16_t SmbSessionSetupAndXReqBlobLen(const SmbNt10_SessionSetupAndXExtReq* req)
{
    return snort::alignedNtohs(&req->smb_blob_len);
}

/* Extended response for security blob */
struct SmbNt10_SessionSetupAndXExtResp   /* smb_wct = 4 */
{
    uint8_t smb_wct;         /* count of 16-bit words that follow */
    uint8_t smb_com2;        /* secondary (X) command, 0xFF = none */
    uint8_t smb_res2;        /* reserved (pad to word) */
    uint16_t smb_off2;       /* offset (from SMB hdr start) to next cmd (@smb_wct) */
    uint16_t smb_action;     /* request mode:
                                bit0 = Logged in successfully - BUT as GUEST */
    uint16_t smb_blob_len;   /* length of security blob */
    uint16_t smb_bcc;        /* min value = 0 */
};

inline uint16_t SmbSessionSetupAndXRespBlobLen(const SmbNt10_SessionSetupAndXExtResp* resp)
{
    return snort::alignedNtohs(&resp->smb_blob_len);
}

/********************************************************************
 * SMB_COM_NEGOTIATE
 ********************************************************************/
/* This is the Lanman response */
struct SmbLm10_NegotiateProtocolResp   /* smb_wct = 13 */
{
    uint8_t smb_wct;        /* count of 16-bit words that follow */
    uint16_t smb_index;     /* index identifying dialect selected */
    uint16_t smb_secmode;   /* security mode:
                               bit 0, 1 = User level, 0 = Share level
                               bit 1, 1 = encrypt passwords, 0 = do not encrypt passwords */
    uint16_t smb_maxxmt;    /* max transmit buffer size server supports, 1K min */
    uint16_t smb_maxmux;    /* max pending multiplexed requests server supports */
    uint16_t smb_maxvcs;    /* max VCs per server/consumer session supported */
    uint16_t smb_blkmode;   /* block read/write mode support:
                               bit 0, Read Block Raw supported (65535 bytes max)
                               bit 1, Write Block Raw supported (65535 bytes max) */
    uint32_t smb_sesskey;   /* Session Key (unique token identifying session) */
    uint16_t smb_srv_time;  /* server's current time (hhhhh mmmmmm xxxxx) */
    uint16_t smb_srv_tzone; /* server's current data (yyyyyyy mmmm ddddd) */
    uint32_t smb_rsvd;      /* reserved */
    uint16_t smb_bcc;       /* value = (size of smb_cryptkey) */
};

/* This is the NT response */
struct SmbNt_NegotiateProtocolResp     /* smb_wct = 17 */
{
    uint8_t smb_wct;            /* count of 16-bit words that follow */
    uint16_t smb_index;         /* index identifying dialect selected */
    uint8_t smb_secmode;        /* security mode:
                                   bit 0, 1 = User level, 0 = Share level
                                   bit 1, 1 = encrypt passwords, 0 = do not encrypt passwords */
    uint16_t smb_maxmux;        /* max pending multiplexed requests server supports */
    uint16_t smb_maxvcs;        /* max VCs per server/consumer session supported */
    uint32_t smb_maxbuf;        /* maximum buffer size supported */
    uint32_t smb_maxraw;        /* maximum raw buffer size supported */
    uint32_t smb_sesskey;       /* Session Key (unique token identifying session) */
    uint32_t smb_cap;           /* capabilities */
    struct
    {
        uint32_t low_time;
        int32_t high_time;
    } smb_srv_time;             /* server time */
    uint16_t smb_srv_tzone;     /* server's current data (yyyyyyy mmmm ddddd) */
    uint8_t smb_challenge_len;  /* Challenge length */
    uint16_t smb_bcc;           /* value = (size of smb_cryptkey) */
};

inline uint16_t SmbLm_NegotiateRespMaxMultiplex(const SmbLm10_NegotiateProtocolResp* resp)
{
    return snort::alignedNtohs(&resp->smb_maxmux);
}

inline uint16_t SmbNt_NegotiateRespMaxMultiplex(const SmbNt_NegotiateProtocolResp* resp)
{
    return snort::alignedNtohs(&resp->smb_maxmux);
}

/* This is the Core Protocol response */
struct SmbCore_NegotiateProtocolResp    /* smb_wct = 1 */
{
    uint8_t smb_wct;      /* count of 16-bit words that follow */
    uint16_t smb_index;   /* index */
    uint16_t smb_bcc;     /* must be 0 */
};

inline uint16_t SmbNegotiateRespDialectIndex(const SmbCore_NegotiateProtocolResp* resp)
{
    return snort::alignedNtohs(&resp->smb_index);
}

/*********************************************************************
 * SMB_COM_TREE_CONNECT_ANDX
 *********************************************************************/
struct SmbTreeConnectAndXReq   /* smb_wct = 4 */
{
    uint8_t smb_wct;         /* count of 16-bit words that follow */
    uint8_t smb_com2;        /* secondary (X) command, 0xFF = none */
    uint8_t smb_reh2;        /* reserved (must be zero) */
    uint16_t smb_off2;       /* offset (from SMB hdr start) to next cmd (@smb_wct) */
    uint16_t smb_flags;      /* additional information:
                                bit 0 - if set, disconnect TID in current smb_tid */
    uint16_t smb_spasslen;   /* length of smb_spasswd */
    uint16_t smb_bcc;        /* minimum value = 3 */
};

inline uint16_t SmbTreeConnectAndXReqPassLen(const SmbTreeConnectAndXReq* req)
{
    return snort::alignedNtohs(&req->smb_spasslen);
}

/********************************************************************
 * SMB_COM_NT_TRANSACT
 ********************************************************************/
#define SMB_CREATE_OPTIONS__FILE_SEQUENTIAL_ONLY     0x00000004

/********************************************************************
 * SMB_COM_NT_CREATE_ANDX
 ********************************************************************/
#define SMB_CREATE_DISPOSITSION__FILE_SUPERCEDE      0x00000000
#define SMB_CREATE_DISPOSITSION__FILE_OPEN           0x00000001
#define SMB_CREATE_DISPOSITSION__FILE_CREATE         0x00000002
#define SMB_CREATE_DISPOSITSION__FILE_OPEN_IF        0x00000003
#define SMB_CREATE_DISPOSITSION__FILE_OVERWRITE      0x00000004
#define SMB_CREATE_DISPOSITSION__FILE_OVERWRITE_IF   0x00000005

struct SmbNtCreateAndXReq   /* smb_wct = 24 */
{
    uint8_t smb_wct;            /* count of 16-bit words that follow */
    uint8_t smb_com2;           /* secondary (X) command, 0xFF = none */
    uint8_t smb_res2;           /* reserved (pad to word) */
    uint16_t smb_off2;          /* offset (from SMB hdr start) to next cmd (@smb_wct) */
    uint8_t smb_res;            /* reserved */
    uint16_t smb_name_len;      /* length of name of file */
    uint32_t smb_flags;         /* flags */
    uint32_t smb_root_fid;      /* fid for previously opened directory */
    uint32_t smb_access;        /* specifies the type of file access */
    uint64_t smb_alloc_size;    /* initial allocation size of the file */
    uint32_t smb_file_attrs;    /* specifies the file attributes for the file */
    uint32_t smb_share_access;  /* the type of share access */
    uint32_t smb_create_disp;   /* actions to take if file does or does not exist */
    uint32_t smb_create_opts;   /* options used when creating or opening file */
    uint32_t smb_impersonation_level;  /* security impersonation level */
    uint8_t smb_security_flags;   /* security flags */
    uint16_t smb_bcc;           /* byte count */
};

struct SmbNtCreateAndXResp    /* smb_wct = 34 */
{
    uint8_t smb_wct;
    uint8_t smb_com2;
    uint8_t smb_res2;
    uint16_t smb_off2;
    uint8_t smb_oplock_level;
    uint16_t smb_fid;
    uint32_t smb_create_disposition;
    uint64_t smb_creation_time;
    uint64_t smb_last_access_time;
    uint64_t smb_last_write_time;
    uint64_t smb_change_time;
    uint32_t smb_file_attrs;
    uint64_t smb_alloc_size;
    uint64_t smb_eof;
    uint16_t smb_resource_type;
    uint16_t smb_nm_pipe_state;
    uint8_t smb_directory;
    uint16_t smb_bcc;
};

// Word count is always set to 42 though there are actually 50 words
struct SmbNtCreateAndXExtResp    /* smb_wct = 42 */
{
    uint8_t smb_wct;
    uint8_t smb_com2;
    uint8_t smb_res2;
    uint16_t smb_off2;
    uint8_t smb_oplock_level;
    uint16_t smb_fid;
    uint32_t smb_create_disposition;
    uint64_t smb_creation_time;
    uint64_t smb_last_access_time;
    uint64_t smb_last_write_time;
    uint64_t smb_change_time;
    uint32_t smb_file_attrs;
    uint64_t smb_alloc_size;
    uint64_t smb_eof;
    uint16_t smb_resource_type;
    uint16_t smb_nm_pipe_state;
    uint8_t smb_directory;
    uint8_t smb_volume_guid[16];
    uint64_t smb_fileid;
    uint32_t smb_max_access_rights;
    uint32_t smb_guest_access_rights;
    uint16_t smb_bcc;
};

inline uint16_t SmbNtCreateAndXReqFileNameLen(const SmbNtCreateAndXReq* req)
{
    return snort::alignedNtohs(&req->smb_name_len);
}

inline uint32_t SmbNtCreateAndXReqCreateDisposition(const SmbNtCreateAndXReq* req)
{
    return snort::alignedNtohl(&req->smb_create_disp);
}

inline bool SmbCreateDispositionRead(const uint32_t create_disposition)
{
    return (create_disposition == SMB_CREATE_DISPOSITSION__FILE_OPEN)
           || (create_disposition > SMB_CREATE_DISPOSITSION__FILE_OVERWRITE_IF);
}

inline uint64_t SmbNtCreateAndXReqAllocSize(const SmbNtCreateAndXReq* req)
{
    return snort::alignedNtohq(&req->smb_alloc_size);
}

inline bool SmbNtCreateAndXReqSequentialOnly(const SmbNtCreateAndXReq* req)
{
    return ((snort::alignedNtohl(&req->smb_create_opts) & SMB_CREATE_OPTIONS__FILE_SEQUENTIAL_ONLY) != 0);
}

inline uint32_t SmbNtCreateAndXReqFileAttrs(const SmbNtCreateAndXReq* req)
{
    return snort::alignedNtohl(&req->smb_file_attrs);
}

inline uint16_t SmbNtCreateAndXRespFid(const SmbNtCreateAndXResp* resp)
{
    return snort::alignedNtohs(&resp->smb_fid);
}

inline uint32_t SmbNtCreateAndXRespCreateDisposition(const SmbNtCreateAndXResp* resp)
{
    return snort::alignedNtohl(&resp->smb_create_disposition);
}

inline bool SmbNtCreateAndXRespDirectory(const SmbNtCreateAndXResp* resp)
{
    return (resp->smb_directory ? true : false);
}

inline uint16_t SmbNtCreateAndXRespResourceType(const SmbNtCreateAndXResp* resp)
{
    return snort::alignedNtohs(&resp->smb_resource_type);
}

inline uint64_t SmbNtCreateAndXRespEndOfFile(const SmbNtCreateAndXResp* resp)
{
    return snort::alignedNtohq(&resp->smb_eof);
}

/********************************************************************
 * SMB_COM_TRANSACTION
 ********************************************************************/
struct SmbTransactionReq   /* smb_wct = 14 + value of smb_suwcnt */
{
    /* Note all subcommands use a setup count of 2 */
    uint8_t smb_wct;       /* count of 16-bit words that follow */
    uint16_t smb_tpscnt;   /* total number of parameter bytes being sent */
    uint16_t smb_tdscnt;   /* total number of data bytes being sent */
    uint16_t smb_mprcnt;   /* max number of parameter bytes to return */
    uint16_t smb_mdrcnt;   /* max number of data bytes to return */
    uint8_t smb_msrcnt;    /* max number of setup words to return */
    uint8_t smb_rsvd;      /* reserved (pad above to word) */
    uint16_t smb_flags;    /* additional information:
                              bit 0 - if set, also disconnect TID in smb_tid
                              bit 1 - if set, transaction is one way (no final response) */
    uint32_t smb_timeout;  /* number of milliseconds to wait for completion */
    uint16_t smb_rsvd1;    /* reserved */
    uint16_t smb_pscnt;    /* number of parameter bytes being sent this buffer */
    uint16_t smb_psoff;    /* offset (from start of SMB hdr) to parameter bytes */
    uint16_t smb_dscnt;    /* number of data bytes being sent this buffer */
    uint16_t smb_dsoff;    /* offset (from start of SMB hdr) to data bytes */
    uint8_t smb_suwcnt;    /* set up word count */
    uint8_t smb_rsvd2;     /* reserved (pad above to word) */
    uint16_t smb_setup1;   /* function (see below)
                                TRANS_SET_NM_PIPE_STATE   = 0x0001
                                TRANS_RAW_READ_NMPIPE     = 0x0011
                                TRANS_QUERY_NMPIPE_STATE  = 0x0021
                                TRANS_QUERY_NMPIPE_INFO   = 0x0022
                                TRANS_PEEK_NMPIPE         = 0x0023
                                TRANS_TRANSACT_NMPIPE     = 0x0026
                                TRANS_RAW_WRITE_NMPIPE    = 0x0031
                                TRANS_READ_NMPIPE         = 0x0036
                                TRANS_WRITE_NMPIPE        = 0x0037
                                TRANS_WAIT_NMPIPE         = 0x0053
                                TRANS_CALL_NMPIPE         = 0x0054  */
    uint16_t smb_setup2;   /* FID (handle) of pipe (if needed), or priority */
    uint16_t smb_bcc;      /* total bytes (including pad bytes) following */
};

struct SmbTransactionInterimResp    /* smb_wct = 0 */
{
    uint8_t smb_wct;        /* count of 16-bit words that follow */
    uint16_t smb_bcc;       /* must be 0 */
};

struct SmbTransactionResp   /* smb_wct = 10 + value of smb_suwcnt */
{
    /* Note all subcommands use a setup count of 0 */
    uint8_t smb_wct;       /* count of 16-bit words that follow */
    uint16_t smb_tprcnt;   /* total number of parameter bytes being returned */
    uint16_t smb_tdrcnt;   /* total number of data bytes being returned */
    uint16_t smb_rsvd;     /* reserved */
    uint16_t smb_prcnt;    /* number of parameter bytes being returned this buf */
    uint16_t smb_proff;    /* offset (from start of SMB hdr) to parameter bytes */
    uint16_t smb_prdisp;   /* byte displacement for these parameter bytes */
    uint16_t smb_drcnt;    /* number of data bytes being returned this buffer */
    uint16_t smb_droff;    /* offset (from start of SMB hdr) to data bytes */
    uint16_t smb_drdisp;   /* byte displacement for these data bytes */
    uint8_t smb_suwcnt;    /* set up return word count */
    uint8_t smb_rsvd1;     /* reserved (pad above to word) */
    uint16_t smb_bcc;      /* total bytes (including pad bytes) following */
};

inline uint16_t SmbTransactionReqSubCom(const SmbTransactionReq* req)
{
    return snort::alignedNtohs(&req->smb_setup1);
}

inline uint16_t SmbTransactionReqFid(const SmbTransactionReq* req)
{
    return snort::alignedNtohs(&req->smb_setup2);
}

inline bool SmbTransactionReqDisconnectTid(const SmbTransactionReq* req)
{
    return (snort::alignedNtohs(&req->smb_flags) & 0x0001) ? true : false;
}

inline bool SmbTransactionReqOneWay(const SmbTransactionReq* req)
{
    return (snort::alignedNtohs(&req->smb_flags) & 0x0002) ? true : false;
}

inline uint8_t SmbTransactionReqSetupCnt(const SmbTransactionReq* req)
{
    return req->smb_suwcnt;
}

inline uint16_t SmbTransactionReqTotalDataCnt(const SmbTransactionReq* req)
{
    return snort::alignedNtohs(&req->smb_tdscnt);
}

inline uint16_t SmbTransactionReqDataCnt(const SmbTransactionReq* req)
{
    return snort::alignedNtohs(&req->smb_dscnt);
}

inline uint16_t SmbTransactionReqDataOff(const SmbTransactionReq* req)
{
    return snort::alignedNtohs(&req->smb_dsoff);
}

inline uint16_t SmbTransactionReqTotalParamCnt(const SmbTransactionReq* req)
{
    return snort::alignedNtohs(&req->smb_tpscnt);
}

inline uint16_t SmbTransactionReqParamCnt(const SmbTransactionReq* req)
{
    return snort::alignedNtohs(&req->smb_pscnt);
}

inline uint16_t SmbTransactionReqParamOff(const SmbTransactionReq* req)
{
    return snort::alignedNtohs(&req->smb_psoff);
}

inline uint16_t SmbTransactionRespTotalDataCnt(const SmbTransactionResp* resp)
{
    return snort::alignedNtohs(&resp->smb_tdrcnt);
}

inline uint16_t SmbTransactionRespDataCnt(const SmbTransactionResp* resp)
{
    return snort::alignedNtohs(&resp->smb_drcnt);
}

inline uint16_t SmbTransactionRespDataOff(const SmbTransactionResp* resp)
{
    return snort::alignedNtohs(&resp->smb_droff);
}

inline uint16_t SmbTransactionRespDataDisp(const SmbTransactionResp* resp)
{
    return snort::alignedNtohs(&resp->smb_drdisp);
}

inline uint16_t SmbTransactionRespTotalParamCnt(const SmbTransactionResp* resp)
{
    return snort::alignedNtohs(&resp->smb_tprcnt);
}

inline uint16_t SmbTransactionRespParamCnt(const SmbTransactionResp* resp)
{
    return snort::alignedNtohs(&resp->smb_prcnt);
}

inline uint16_t SmbTransactionRespParamOff(const SmbTransactionResp* resp)
{
    return snort::alignedNtohs(&resp->smb_proff);
}

inline uint16_t SmbTransactionRespParamDisp(const SmbTransactionResp* resp)
{
    return snort::alignedNtohs(&resp->smb_prdisp);
}

// Flags for TRANS_SET_NMPIPE_STATE parameters
#define PIPE_STATE_NON_BLOCKING  0x8000
#define PIPE_STATE_MESSAGE_MODE  0x0100

/********************************************************************
 * SMB_COM_TRANSACTION2
 ********************************************************************/
struct SmbTransaction2Req
{
    uint8_t smb_wct;
    uint16_t smb_total_param_count;
    uint16_t smb_total_data_count;
    uint16_t smb_max_param_count;
    uint16_t smb_max_data_count;
    uint8_t smb_max_setup_count;
    uint8_t smb_res;
    uint16_t smb_flags;
    uint32_t smb_timeout;
    uint16_t smb_res2;
    uint16_t smb_param_count;
    uint16_t smb_param_offset;
    uint16_t smb_data_count;
    uint16_t smb_data_offset;
    uint8_t smb_setup_count;    /* Should be 1 for all subcommands */
    uint8_t smb_res3;
    uint16_t smb_setup;  /* This is the subcommand */
    uint16_t smb_bcc;
};

struct SmbTransaction2InterimResp
{
    uint8_t smb_wct;
    uint16_t smb_bcc;
};

struct SmbTransaction2Resp
{
    uint8_t smb_wct;
    uint16_t smb_total_param_count;
    uint16_t smb_total_data_count;
    uint16_t smb_res;
    uint16_t smb_param_count;
    uint16_t smb_param_offset;
    uint16_t smb_param_disp;
    uint16_t smb_data_count;
    uint16_t smb_data_offset;
    uint16_t smb_data_disp;
    uint16_t smb_setup_count;  /* 0 or 1 word */
    uint8_t smb_res2;
};

inline uint16_t SmbTransaction2ReqSubCom(const SmbTransaction2Req* req)
{
    return snort::alignedNtohs(&req->smb_setup);
}

inline uint16_t SmbTransaction2ReqTotalParamCnt(const SmbTransaction2Req* req)
{
    return snort::alignedNtohs(&req->smb_total_param_count);
}

inline uint16_t SmbTransaction2ReqParamCnt(const SmbTransaction2Req* req)
{
    return snort::alignedNtohs(&req->smb_param_count);
}

inline uint16_t SmbTransaction2ReqParamOff(const SmbTransaction2Req* req)
{
    return snort::alignedNtohs(&req->smb_param_offset);
}

inline uint16_t SmbTransaction2ReqTotalDataCnt(const SmbTransaction2Req* req)
{
    return snort::alignedNtohs(&req->smb_total_data_count);
}

inline uint16_t SmbTransaction2ReqDataCnt(const SmbTransaction2Req* req)
{
    return snort::alignedNtohs(&req->smb_data_count);
}

inline uint16_t SmbTransaction2ReqDataOff(const SmbTransaction2Req* req)
{
    return snort::alignedNtohs(&req->smb_data_offset);
}

inline uint8_t SmbTransaction2ReqSetupCnt(const SmbTransaction2Req* req)
{
    return req->smb_setup_count;
}

inline uint16_t SmbTransaction2RespTotalParamCnt(const SmbTransaction2Resp* resp)
{
    return snort::alignedNtohs(&resp->smb_total_param_count);
}

inline uint16_t SmbTransaction2RespParamCnt(const SmbTransaction2Resp* resp)
{
    return snort::alignedNtohs(&resp->smb_param_count);
}

inline uint16_t SmbTransaction2RespParamOff(const SmbTransaction2Resp* resp)
{
    return snort::alignedNtohs(&resp->smb_param_offset);
}

inline uint16_t SmbTransaction2RespParamDisp(const SmbTransaction2Resp* resp)
{
    return snort::alignedNtohs(&resp->smb_param_disp);
}

inline uint16_t SmbTransaction2RespTotalDataCnt(const SmbTransaction2Resp* resp)
{
    return snort::alignedNtohs(&resp->smb_total_data_count);
}

inline uint16_t SmbTransaction2RespDataCnt(const SmbTransaction2Resp* resp)
{
    return snort::alignedNtohs(&resp->smb_data_count);
}

inline uint16_t SmbTransaction2RespDataOff(const SmbTransaction2Resp* resp)
{
    return snort::alignedNtohs(&resp->smb_data_offset);
}

inline uint16_t SmbTransaction2RespDataDisp(const SmbTransaction2Resp* resp)
{
    return snort::alignedNtohs(&resp->smb_data_disp);
}

struct SmbTrans2Open2ReqParams
{
    uint16_t Flags;
    uint16_t AccessMode;
    uint16_t Reserved1;
    uint16_t FileAttributes;
    uint32_t CreationTime;
    uint16_t OpenMode;
    uint32_t AllocationSize;
    uint16_t Reserved[5];
};

typedef SmbTransaction2Req SmbTrans2Open2Req;

inline uint16_t SmbTrans2Open2ReqAccessMode(const SmbTrans2Open2ReqParams* req)
{
    return snort::alignedNtohs(&req->AccessMode);
}

inline uint16_t SmbTrans2Open2ReqFileAttrs(const SmbTrans2Open2ReqParams* req)
{
    return snort::alignedNtohs(&req->FileAttributes);
}

inline uint16_t SmbTrans2Open2ReqOpenMode(const SmbTrans2Open2ReqParams* req)
{
    return snort::alignedNtohs(&req->OpenMode);
}

inline uint32_t SmbTrans2Open2ReqAllocSize(const SmbTrans2Open2ReqParams* req)
{
    return snort::alignedNtohl(&req->AllocationSize);
}

struct SmbTrans2Open2RespParams
{
    uint16_t smb_fid;
    uint16_t file_attributes;
    uint32_t creation_time;
    uint32_t file_data_size;
    uint16_t access_mode;
    uint16_t resource_type;
    uint16_t nm_pipe_status;
    uint16_t action_taken;
    uint32_t reserved;
    uint16_t extended_attribute_error_offset;
    uint32_t extended_attribute_length;
};

inline uint16_t SmbTrans2Open2RespFid(const SmbTrans2Open2RespParams* resp)
{
    return snort::alignedNtohs(&resp->smb_fid);
}

inline uint16_t SmbTrans2Open2RespFileAttrs(const SmbTrans2Open2RespParams* resp)
{
    return snort::alignedNtohs(&resp->file_attributes);
}

inline uint32_t SmbTrans2Open2RespFileDataSize(const SmbTrans2Open2RespParams* resp)
{
    return snort::alignedNtohl(&resp->file_data_size);
}

inline uint16_t SmbTrans2Open2RespResourceType(const SmbTrans2Open2RespParams* resp)
{
    return snort::alignedNtohs(&resp->resource_type);
}

inline uint16_t SmbTrans2Open2RespActionTaken(const SmbTrans2Open2RespParams* resp)
{
    return snort::alignedNtohs(&resp->action_taken);
}

struct SmbTrans2Open2Resp
{
    uint8_t smb_wct;
    uint16_t smb_total_param_count;
    uint16_t smb_total_data_count;
    uint16_t smb_res;
    uint16_t smb_param_count;
    uint16_t smb_param_offset;
    uint16_t smb_param_disp;
    uint16_t smb_data_count;
    uint16_t smb_data_offset;
    uint16_t smb_data_disp;
    uint16_t smb_setup_count;  /* 0 */
    uint8_t smb_res2;
    uint16_t smb_bcc;
};

// See MS-CIFS Section 2.2.2.3.3
#define SMB_INFO_STANDARD               0x0001
#define SMB_INFO_QUERY_EA_SIZE          0x0002
#define SMB_INFO_QUERY_EAS_FROM_LIST    0x0003
#define SMB_INFO_QUERY_ALL_EAS          0x0004
#define SMB_INFO_IS_NAME_VALID          0x0006
#define SMB_QUERY_FILE_BASIC_INFO       0x0101
#define SMB_QUERY_FILE_STANDARD_INFO    0x0102
#define SMB_QUERY_FILE_EA_INFO          0x0103
#define SMB_QUERY_FILE_NAME_INFO        0x0104
#define SMB_QUERY_FILE_ALL_INFO         0x0107
#define SMB_QUERY_FILE_ALT_NAME_INFO    0x0108
#define SMB_QUERY_FILE_STREAM_INFO      0x0109
#define SMB_QUERY_FILE_COMPRESSION_INFO 0x010b

// See MS-SMB Section 2.2.2.3.5
// For added value, see below from MS-FSCC
#define SMB_INFO_PASSTHROUGH  0x03e8
#define SMB_INFO_PT_FILE_STANDARD_INFO  (SMB_INFO_PASSTHROUGH+5)
#define SMB_INFO_PT_FILE_ALL_INFO       (SMB_INFO_PASSTHROUGH+18)
#define SMB_INFO_PT_FILE_STREAM_INFO    (SMB_INFO_PASSTHROUGH+22)
#define SMB_INFO_PT_NETWORK_OPEN_INFO   (SMB_INFO_PASSTHROUGH+34)

struct SmbTrans2QueryFileInfoReqParams
{
    uint16_t fid;
    uint16_t information_level;
};

inline uint16_t SmbTrans2QueryFileInfoReqFid(const SmbTrans2QueryFileInfoReqParams* req)
{
    return snort::alignedNtohs(&req->fid);
}

inline uint16_t SmbTrans2QueryFileInfoReqInfoLevel(const SmbTrans2QueryFileInfoReqParams* req)
{
    return snort::alignedNtohs(&req->information_level);
}

struct SmbQueryInfoStandard
{
    uint16_t CreationDate;
    uint16_t CreationTime;
    uint16_t LastAccessDate;
    uint16_t LastAccessTime;
    uint16_t LastWriteDate;
    uint16_t LastWriteTime;
    uint32_t FileDataSize;
    uint32_t AllocationSize;
    uint16_t Attributes;
};

inline uint32_t SmbQueryInfoStandardFileDataSize(const SmbQueryInfoStandard* q)
{
    return snort::alignedNtohl(&q->FileDataSize);
}

struct SmbQueryInfoQueryEaSize
{
    uint16_t CreationDate;
    uint16_t CreationTime;
    uint16_t LastAccessDate;
    uint16_t LastAccessTime;
    uint16_t LastWriteDate;
    uint16_t LastWriteTime;
    uint32_t FileDataSize;
    uint32_t AllocationSize;
    uint16_t Attributes;
    uint32_t EaSize;
};

inline uint32_t SmbQueryInfoQueryEaSizeFileDataSize(const SmbQueryInfoQueryEaSize* q)
{
    return snort::alignedNtohl(&q->FileDataSize);
}

struct SmbQueryFileStandardInfo
{
    uint64_t AllocationSize;
    uint64_t EndOfFile;
    uint32_t NumberOfLinks;
    uint8_t DeletePending;
    uint8_t Directory;
    uint16_t Reserved;
};

inline uint64_t SmbQueryFileStandardInfoEndOfFile(const SmbQueryFileStandardInfo* q)
{
    return snort::alignedNtohq(&q->EndOfFile);
}

struct SmbQueryFileAllInfo
{
    // Basic Info
    uint64_t CreationTime;
    uint64_t LastAccessTime;
    uint64_t LastWriteTime;
    uint64_t LastChangeTime;
    uint32_t ExtFileAttributes;
    uint32_t Reserved1;
    uint64_t AllocationSize;
    uint64_t EndOfFile;
    uint32_t NumberOfLinks;
    uint8_t DeletePending;
    uint8_t Directory;
    uint16_t Reserved2;
    uint32_t EaSize;
    uint32_t FileNameLength;
};

inline uint64_t SmbQueryFileAllInfoEndOfFile(const SmbQueryFileAllInfo* q)
{
    return snort::alignedNtohq(&q->EndOfFile);
}

struct SmbQueryPTFileAllInfo
{
    // Basic Info
    uint64_t CreationTime;
    uint64_t LastAccessTime;
    uint64_t LastWriteTime;
    uint64_t LastChangeTime;
    uint32_t ExtFileAttributes;
    uint32_t Reserved1;

    // Standard Info
    uint64_t AllocationSize;
    uint64_t EndOfFile;
    uint32_t NumberOfLinks;
    uint8_t DeletePending;
    uint8_t Directory;
    uint16_t Reserved2;

    // Internal Info
    uint64_t IndexNumber;

    // EA Info
    uint32_t EaSize;

    // Access Info
    uint32_t AccessFlags;

    // Position Info
    uint64_t CurrentByteOffset;

    // Mode Info
    uint32_t Mode;

    // Alignment Info
    uint32_t AlignmentRequirement;

    // Name Info
    uint32_t FileNameLength;
};

inline uint64_t SmbQueryPTFileAllInfoEndOfFile(const SmbQueryPTFileAllInfo* q)
{
    return snort::alignedNtohq(&q->EndOfFile);
}

struct SmbQueryPTNetworkOpenInfo
{
    uint64_t CreationTime;
    uint64_t LastAccessTime;
    uint64_t LastWriteTime;
    uint64_t LastChangeTime;
    uint64_t AllocationSize;
    uint64_t EndOfFile;
    uint32_t FileAttributes;
    uint32_t Reserved;
};

inline uint64_t SmbQueryPTNetworkOpenInfoEndOfFile(const SmbQueryPTNetworkOpenInfo* q)
{
    return snort::alignedNtohq(&q->EndOfFile);
}

struct SmbQueryPTFileStreamInfo
{
    uint32_t NextEntryOffset;
    uint32_t StreamNameLength;
    uint64_t StreamSize;
    uint64_t StreamAllocationSize;
};

inline uint64_t SmbQueryPTFileStreamInfoStreamSize(const SmbQueryPTFileStreamInfo* q)
{
    return snort::alignedNtohq(&q->StreamSize);
}

struct SmbTrans2QueryFileInformationResp
{
    uint8_t smb_wct;
    uint16_t smb_total_param_count;
    uint16_t smb_total_data_count;
    uint16_t smb_res;
    uint16_t smb_param_count;
    uint16_t smb_param_offset;
    uint16_t smb_param_disp;
    uint16_t smb_data_count;
    uint16_t smb_data_offset;
    uint16_t smb_data_disp;
    uint16_t smb_setup_count;  /* 0 */
    uint8_t smb_res2;
    uint16_t smb_bcc;
};

#define SMB_INFO_SET_EAS               0x0002
#define SMB_SET_FILE_BASIC_INFO        0x0101
#define SMB_SET_FILE_DISPOSITION_INFO  0x0102
#define SMB_SET_FILE_ALLOCATION_INFO   0x0103
#define SMB_SET_FILE_END_OF_FILE_INFO  0x0104

// For added value, see above File Information Classes
#define SMB_INFO_PT_SET_FILE_BASIC_FILE_INFO   (SMB_INFO_PASSTHROUGH+4)
#define SMB_INFO_PT_SET_FILE_END_OF_FILE_INFO  (SMB_INFO_PASSTHROUGH+20)

struct SmbTrans2SetFileInfoReqParams
{
    uint16_t fid;
    uint16_t information_level;
    uint16_t reserved;
};

inline uint16_t SmbTrans2SetFileInfoReqFid(const SmbTrans2SetFileInfoReqParams* req)
{
    return snort::alignedNtohs(&req->fid);
}

inline uint16_t SmbTrans2SetFileInfoReqInfoLevel(const SmbTrans2SetFileInfoReqParams* req)
{
    return snort::alignedNtohs(&req->information_level);
}

inline bool SmbSetFileInfoEndOfFile(const uint16_t info_level)
{
    return ((info_level == SMB_SET_FILE_END_OF_FILE_INFO)
           || (info_level == SMB_INFO_PT_SET_FILE_END_OF_FILE_INFO));
}

struct SmbSetFileBasicInfo
{
    uint64_t CreationTime;
    uint64_t LastAccessTime;
    uint64_t LastWriteTime;
    uint64_t ChangeTime;
    uint32_t ExtFileAttributes;
    uint32_t Reserved;
};

inline uint32_t SmbSetFileInfoExtFileAttrs(const SmbSetFileBasicInfo* info)
{
    return snort::alignedNtohl(&info->ExtFileAttributes);
}

inline bool SmbSetFileInfoSetFileBasicInfo(const uint16_t info_level)
{
    return ((info_level == SMB_SET_FILE_BASIC_INFO)
           || (info_level == SMB_INFO_PT_SET_FILE_BASIC_FILE_INFO));
}

/********************************************************************
 * SMB_COM_NT_TRANSACT
 ********************************************************************/
#define SMB_CREATE_OPTIONS__FILE_SEQUENTIAL_ONLY     0x00000004

struct SmbNtTransactReq
{
    uint8_t smb_wct;
    uint8_t smb_max_setup_count;
    uint16_t smb_res;
    uint32_t smb_total_param_count;
    uint32_t smb_total_data_count;
    uint32_t smb_max_param_count;
    uint32_t smb_max_data_count;
    uint32_t smb_param_count;
    uint32_t smb_param_offset;
    uint32_t smb_data_count;
    uint32_t smb_data_offset;
    uint8_t smb_setup_count;
    uint16_t smb_function;
};

struct SmbNtTransactInterimResp
{
    uint8_t smb_wct;
    uint16_t smb_bcc;
};

struct SmbNtTransactResp
{
    uint8_t smb_wct;
    uint8_t smb_res[3];
    uint32_t smb_total_param_count;
    uint32_t smb_total_data_count;
    uint32_t smb_param_count;
    uint32_t smb_param_offset;
    uint32_t smb_param_disp;
    uint32_t smb_data_count;
    uint32_t smb_data_offset;
    uint32_t smb_data_disp;
    uint8_t smb_setup_count;
};

inline uint16_t SmbNtTransactReqSubCom(const SmbNtTransactReq* req)
{
    return snort::alignedNtohs(&req->smb_function);
}

inline uint8_t SmbNtTransactReqSetupCnt(const SmbNtTransactReq* req)
{
    return req->smb_setup_count;
}

inline uint32_t SmbNtTransactReqTotalParamCnt(const SmbNtTransactReq* req)
{
    return snort::alignedNtohl(&req->smb_total_param_count);
}

inline uint32_t SmbNtTransactReqParamCnt(const SmbNtTransactReq* req)
{
    return snort::alignedNtohl(&req->smb_param_count);
}

inline uint32_t SmbNtTransactReqParamOff(const SmbNtTransactReq* req)
{
    return snort::alignedNtohl(&req->smb_param_offset);
}

inline uint32_t SmbNtTransactReqTotalDataCnt(const SmbNtTransactReq* req)
{
    return snort::alignedNtohl(&req->smb_total_data_count);
}

inline uint32_t SmbNtTransactReqDataCnt(const SmbNtTransactReq* req)
{
    return snort::alignedNtohl(&req->smb_data_count);
}

inline uint32_t SmbNtTransactReqDataOff(const SmbNtTransactReq* req)
{
    return snort::alignedNtohl(&req->smb_data_offset);
}

inline uint32_t SmbNtTransactRespTotalParamCnt(const SmbNtTransactResp* resp)
{
    return snort::alignedNtohl(&resp->smb_total_param_count);
}

inline uint32_t SmbNtTransactRespParamCnt(const SmbNtTransactResp* resp)
{
    return snort::alignedNtohl(&resp->smb_param_count);
}

inline uint32_t SmbNtTransactRespParamOff(const SmbNtTransactResp* resp)
{
    return snort::alignedNtohl(&resp->smb_param_offset);
}

inline uint32_t SmbNtTransactRespParamDisp(const SmbNtTransactResp* resp)
{
    return snort::alignedNtohl(&resp->smb_param_disp);
}

inline uint32_t SmbNtTransactRespTotalDataCnt(const SmbNtTransactResp* resp)
{
    return snort::alignedNtohl(&resp->smb_total_data_count);
}

inline uint32_t SmbNtTransactRespDataCnt(const SmbNtTransactResp* resp)
{
    return snort::alignedNtohl(&resp->smb_data_count);
}

inline uint32_t SmbNtTransactRespDataOff(const SmbNtTransactResp* resp)
{
    return snort::alignedNtohl(&resp->smb_data_offset);
}

inline uint32_t SmbNtTransactRespDataDisp(const SmbNtTransactResp* resp)
{
    return snort::alignedNtohl(&resp->smb_data_disp);
}

struct SmbNtTransactCreateReqParams
{
    uint32_t flags;
    uint32_t root_dir_fid;
    uint32_t desired_access;
    uint64_t allocation_size;
    uint32_t ext_file_attributes;
    uint32_t share_access;
    uint32_t create_disposition;
    uint32_t create_options;
    uint32_t security_descriptor_length;
    uint32_t ea_length;
    uint32_t name_length;
    uint32_t impersonation_level;
    uint8_t security_flags;
};

inline uint64_t SmbNtTransactCreateReqAllocSize(const SmbNtTransactCreateReqParams* req)
{
    return snort::alignedNtohq(&req->allocation_size);
}

inline uint32_t SmbNtTransactCreateReqFileNameLength(const SmbNtTransactCreateReqParams* req)
{
    return snort::alignedNtohl(&req->name_length);
}

inline uint32_t SmbNtTransactCreateReqFileAttrs(const SmbNtTransactCreateReqParams* req)
{
    return snort::alignedNtohl(&req->ext_file_attributes);
}

inline bool SmbNtTransactCreateReqSequentialOnly(const SmbNtTransactCreateReqParams* req)
{
    return ((snort::alignedNtohl(&req->create_options) & SMB_CREATE_OPTIONS__FILE_SEQUENTIAL_ONLY) != 0);
}

struct SmbNtTransactCreateReq
{
    uint8_t smb_wct;
    uint8_t smb_max_setup_count;
    uint16_t smb_res;
    uint32_t smb_total_param_count;
    uint32_t smb_total_data_count;
    uint32_t smb_max_param_count;
    uint32_t smb_max_data_count;
    uint32_t smb_param_count;
    uint32_t smb_param_offset;
    uint32_t smb_data_count;
    uint32_t smb_data_offset;
    uint8_t smb_setup_count;    /* Must be 0x00 */
    uint16_t smb_function;      /* NT_TRANSACT_CREATE */
    uint16_t smb_bcc;
};

struct SmbNtTransactCreateRespParams
{
    uint8_t op_lock_level;
    uint8_t reserved;
    uint16_t smb_fid;
    uint32_t create_action;
    uint32_t ea_error_offset;
    uint64_t creation_time;
    uint64_t last_access_time;
    uint64_t last_write_time;
    uint64_t last_change_time;
    uint32_t ext_file_attributes;
    uint64_t allocation_size;
    uint64_t end_of_file;
    uint16_t resource_type;
    uint16_t nm_pipe_status;
    uint8_t directory;
};

inline uint16_t SmbNtTransactCreateRespFid(const SmbNtTransactCreateRespParams* resp)
{
    return snort::alignedNtohs(&resp->smb_fid);
}

inline uint32_t SmbNtTransactCreateRespCreateAction(const SmbNtTransactCreateRespParams* resp)
{
    return snort::alignedNtohl(&resp->create_action);
}

inline uint64_t SmbNtTransactCreateRespEndOfFile(const SmbNtTransactCreateRespParams* resp)
{
    return snort::alignedNtohq(&resp->end_of_file);
}

inline uint16_t SmbNtTransactCreateRespResourceType(const SmbNtTransactCreateRespParams* resp)
{
    return snort::alignedNtohs(&resp->resource_type);
}

inline bool SmbNtTransactCreateRespDirectory(const SmbNtTransactCreateRespParams* resp)
{
    return (resp->directory ? true : false);
}

struct SmbNtTransactCreateResp
{
    uint8_t smb_wct;
    uint8_t smb_res[3];
    uint32_t smb_total_param_count;
    uint32_t smb_total_data_count;
    uint32_t smb_param_count;
    uint32_t smb_param_offset;
    uint32_t smb_param_disp;
    uint32_t smb_data_count;
    uint32_t smb_data_offset;
    uint32_t smb_data_disp;
    uint8_t smb_setup_count;    /* 0x00 */
    uint16_t smb_bcc;
};

/********************************************************************
 * SMB_COM_TRANSACTION_SECONDARY
 *  Continuation command for SMB_COM_TRANSACTION requests if all
 *  data wasn't sent.
 ********************************************************************/
struct SmbTransactionSecondaryReq   /* smb_wct = 8 */
{
    uint8_t smb_wct;       /* count of 16-bit words that follow */
    uint16_t smb_tpscnt;   /* total number of parameter bytes being sent */
    uint16_t smb_tdscnt;   /* total number of data bytes being sent */
    uint16_t smb_pscnt;    /* number of parameter bytes being sent this buffer */
    uint16_t smb_psoff;    /* offset (from start of SMB hdr) to parameter bytes */
    uint16_t smb_psdisp;   /* byte displacement for these parameter bytes */
    uint16_t smb_dscnt;    /* number of data bytes being sent this buffer */
    uint16_t smb_dsoff;    /* offset (from start of SMB hdr) to data bytes */
    uint16_t smb_dsdisp;   /* byte displacement for these data bytes */
    uint16_t smb_bcc;      /* total bytes (including pad bytes) following */
};

inline uint16_t SmbTransactionSecondaryReqTotalDataCnt(const SmbTransactionSecondaryReq* req)
{
    return snort::alignedNtohs(&req->smb_tdscnt);
}

inline uint16_t SmbTransactionSecondaryReqDataCnt(const SmbTransactionSecondaryReq* req)
{
    return snort::alignedNtohs(&req->smb_dscnt);
}

inline uint16_t SmbTransactionSecondaryReqDataOff(const SmbTransactionSecondaryReq* req)
{
    return snort::alignedNtohs(&req->smb_dsoff);
}

inline uint16_t SmbTransactionSecondaryReqDataDisp(const SmbTransactionSecondaryReq* req)
{
    return snort::alignedNtohs(&req->smb_dsdisp);
}

inline uint16_t SmbTransactionSecondaryReqTotalParamCnt(const SmbTransactionSecondaryReq* req)
{
    return snort::alignedNtohs(&req->smb_tpscnt);
}

inline uint16_t SmbTransactionSecondaryReqParamCnt(const SmbTransactionSecondaryReq* req)
{
    return snort::alignedNtohs(&req->smb_pscnt);
}

inline uint16_t SmbTransactionSecondaryReqParamOff(const SmbTransactionSecondaryReq* req)
{
    return snort::alignedNtohs(&req->smb_psoff);
}

inline uint16_t SmbTransactionSecondaryReqParamDisp(const SmbTransactionSecondaryReq* req)
{
    return snort::alignedNtohs(&req->smb_psdisp);
}

/********************************************************************
 * SMB_COM_TRANSACTION2_SECONDARY
 *  Continuation command for SMB_COM_TRANSACTION2 requests if all
 *  data wasn't sent.
 ********************************************************************/
struct SmbTransaction2SecondaryReq
{
    uint8_t smb_wct;
    uint16_t smb_total_param_count;
    uint16_t smb_total_data_count;
    uint16_t smb_param_count;
    uint16_t smb_param_offset;
    uint16_t smb_param_disp;
    uint16_t smb_data_count;
    uint16_t smb_data_offset;
    uint16_t smb_data_disp;
    uint16_t smb_fid;
    uint16_t smb_bcc;
};

inline uint16_t SmbTransaction2SecondaryReqTotalParamCnt(const SmbTransaction2SecondaryReq* req)
{
    return snort::alignedNtohs(&req->smb_total_param_count);
}

inline uint16_t SmbTransaction2SecondaryReqParamCnt(const SmbTransaction2SecondaryReq* req)
{
    return snort::alignedNtohs(&req->smb_param_count);
}

inline uint16_t SmbTransaction2SecondaryReqParamOff(const SmbTransaction2SecondaryReq* req)
{
    return snort::alignedNtohs(&req->smb_param_offset);
}

inline uint16_t SmbTransaction2SecondaryReqParamDisp(const SmbTransaction2SecondaryReq* req)
{
    return snort::alignedNtohs(&req->smb_param_disp);
}

inline uint16_t SmbTransaction2SecondaryReqTotalDataCnt(const SmbTransaction2SecondaryReq* req)
{
    return snort::alignedNtohs(&req->smb_total_data_count);
}

inline uint16_t SmbTransaction2SecondaryReqDataCnt(const SmbTransaction2SecondaryReq* req)
{
    return snort::alignedNtohs(&req->smb_data_count);
}

inline uint16_t SmbTransaction2SecondaryReqDataOff(const SmbTransaction2SecondaryReq* req)
{
    return snort::alignedNtohs(&req->smb_data_offset);
}

inline uint16_t SmbTransaction2SecondaryReqDataDisp(const SmbTransaction2SecondaryReq* req)
{
    return snort::alignedNtohs(&req->smb_data_disp);
}

/********************************************************************
 * SMB_COM_NT_TRANSACT_SECONDARY
 ********************************************************************/
struct SmbNtTransactSecondaryReq
{
    uint8_t smb_wct;
    uint8_t smb_res[3];
    uint32_t smb_total_param_count;
    uint32_t smb_total_data_count;
    uint32_t smb_param_count;
    uint32_t smb_param_offset;
    uint32_t smb_param_disp;
    uint32_t smb_data_count;
    uint32_t smb_data_offset;
    uint32_t smb_data_disp;
    uint8_t smb_res2;
};

inline uint32_t SmbNtTransactSecondaryReqTotalParamCnt(const SmbNtTransactSecondaryReq* req)
{
    return snort::alignedNtohl(&req->smb_total_param_count);
}

inline uint32_t SmbNtTransactSecondaryReqParamCnt(const SmbNtTransactSecondaryReq* req)
{
    return snort::alignedNtohl(&req->smb_param_count);
}

inline uint32_t SmbNtTransactSecondaryReqParamOff(const SmbNtTransactSecondaryReq* req)
{
    return snort::alignedNtohl(&req->smb_param_offset);
}

inline uint32_t SmbNtTransactSecondaryReqParamDisp(const SmbNtTransactSecondaryReq* req)
{
    return snort::alignedNtohl(&req->smb_param_disp);
}

inline uint32_t SmbNtTransactSecondaryReqTotalDataCnt(const SmbNtTransactSecondaryReq* req)
{
    return snort::alignedNtohl(&req->smb_total_data_count);
}

inline uint32_t SmbNtTransactSecondaryReqDataCnt(const SmbNtTransactSecondaryReq* req)
{
    return snort::alignedNtohl(&req->smb_data_count);
}

inline uint32_t SmbNtTransactSecondaryReqDataOff(const SmbNtTransactSecondaryReq* req)
{
    return snort::alignedNtohl(&req->smb_data_offset);
}

inline uint32_t SmbNtTransactSecondaryReqDataDisp(const SmbNtTransactSecondaryReq* req)
{
    return snort::alignedNtohl(&req->smb_data_disp);
}

/********************************************************************
 * SMB_COM_READ_RAW
 ********************************************************************/
struct SmbReadRawReq   /* smb_wct = 8 */
{
    uint8_t smb_wct;         /* count of 16-bit words that follow */
    uint16_t smb_fid;        /* file handle */
    uint32_t smb_offset;     /* offset in file to begin read */
    uint16_t smb_maxcnt;     /* max number of bytes to return (max 65,535) */
    uint16_t smb_mincnt;     /* min number of bytes to return (normally 0) */
    uint32_t smb_timeout;    /* number of milliseconds to wait for completion */
    uint16_t smb_rsvd;       /* reserved */
    uint16_t smb_bcc;        /* value = 0 */
};

struct SmbReadRawExtReq   /* smb_wct = 10 */
{
    uint8_t smb_wct;         /* count of 16-bit words that follow */
    uint16_t smb_fid;        /* file handle */
    uint32_t smb_offset;     /* offset in file to begin read */
    uint16_t smb_maxcnt;     /* max number of bytes to return (max 65,535) */
    uint16_t smb_mincnt;     /* min number of bytes to return (normally 0) */
    uint32_t smb_timeout;    /* number of milliseconds to wait for completion */
    uint16_t smb_rsvd;       /* reserved */
    uint32_t smb_off_high;   /* high offset in file to begin write */
    uint16_t smb_bcc;        /* value = 0 */
};

/* Read Raw response is raw data wrapped in NetBIOS header */

inline uint16_t SmbReadRawReqFid(const SmbReadRawReq* req)
{
    return snort::alignedNtohs(&req->smb_fid);
}

inline uint64_t SmbReadRawReqOffset(const SmbReadRawExtReq* req)
{
    if (req->smb_wct == 8)
        return (uint64_t)snort::alignedNtohl(&req->smb_offset);

    return (uint64_t)snort::alignedNtohl(&req->smb_off_high) << 32
           | (uint64_t)snort::alignedNtohl(&req->smb_offset);
}

/********************************************************************
 * SMB_COM_WRITE_RAW
 ********************************************************************/
struct SmbWriteRawReq
{
    uint8_t smb_wct;       /* value = 12 */
    uint16_t smb_fid;      /* file handle */
    uint16_t smb_tcount;   /* total bytes (including this buf, 65,535 max ) */
    uint16_t smb_rsvd;     /* reserved */
    uint32_t smb_offset;   /* offset in file to begin write */
    uint32_t smb_timeout;  /* number of milliseconds to wait for completion */
    uint16_t smb_wmode;    /* write mode:
                              bit0 - complete write to disk and send final result response
                              bit1 - return smb_remaining (pipes/devices only) */
    uint32_t smb_rsvd2;    /* reserved */
    uint16_t smb_dsize;    /* number of data bytes this buffer (min value = 0) */
    uint16_t smb_doff;     /* offset (from start of SMB hdr) to data bytes */
    uint16_t smb_bcc;      /* total bytes (including pad bytes) following */
};

struct SmbWriteRawExtReq
{
    uint8_t smb_wct;       /* value = 14 */
    uint16_t smb_fid;      /* file handle */
    uint16_t smb_tcount;   /* total bytes (including this buf, 65,535 max ) */
    uint16_t smb_rsvd;     /* reserved */
    uint32_t smb_offset;   /* offset in file to begin write */
    uint32_t smb_timeout;  /* number of milliseconds to wait for completion */
    uint16_t smb_wmode;    /* write mode:
                              bit0 - complete write to disk and send final result response
                              bit1 - return smb_remaining (pipes/devices only) */
    uint32_t smb_rsvd2;    /* reserved */
    uint16_t smb_dsize;    /* number of data bytes this buffer (min value = 0) */
    uint16_t smb_doff;     /* offset (from start of SMB hdr) to data bytes */
    uint32_t smb_off_high; /* high offset in file to begin write */
    uint16_t smb_bcc;      /* total bytes (including pad bytes) following */
};

struct SmbWriteRawInterimResp
{
    uint8_t smb_wct;         /* value = 1 */
    uint16_t smb_remaining;  /* bytes remaining to be read (pipes/devices only) */
    uint16_t smb_bcc;        /* value = 0 */
};

inline uint16_t SmbWriteRawReqTotalCount(const SmbWriteRawReq* req)
{
    return snort::alignedNtohs(&req->smb_tcount);
}

inline bool SmbWriteRawReqWriteThrough(const SmbWriteRawReq* req)
{
    return snort::alignedNtohs(&req->smb_wmode) & 0x0001;
}

inline uint16_t SmbWriteRawReqFid(const SmbWriteRawReq* req)
{
    return snort::alignedNtohs(&req->smb_fid);
}

inline uint16_t SmbWriteRawReqDataOff(const SmbWriteRawReq* req)
{
    return snort::alignedNtohs(&req->smb_doff);
}

inline uint16_t SmbWriteRawReqDataCnt(const SmbWriteRawReq* req)
{
    return snort::alignedNtohs(&req->smb_dsize);
}

inline uint64_t SmbWriteRawReqOffset(const SmbWriteRawExtReq* req)
{
    if (req->smb_wct == 12)
        return (uint64_t)snort::alignedNtohl(&req->smb_offset);

    return (uint64_t)snort::alignedNtohl(&req->smb_off_high) << 32 |
           (uint64_t)snort::alignedNtohl(&req->smb_offset);
}

inline uint16_t SmbWriteRawInterimRespRemaining(const SmbWriteRawInterimResp* resp)
{
    return snort::alignedNtohs(&resp->smb_remaining);
}

/********************************************************************
 * SMB_COM_WRITE_COMPLETE - final response to an SMB_COM_WRITE_RAW
 ********************************************************************/
struct SmbWriteCompleteResp
{
    uint8_t smb_wct;     /* value = 1 */
    uint16_t smb_count;  /* total number of bytes written */
    uint16_t smb_bcc;    /* value = 0 */
};

inline uint16_t SmbWriteCompleteRespCount(const SmbWriteCompleteResp* resp)
{
    return snort::alignedNtohs(&resp->smb_count);
}

/********************************************************************
 * SMB_COM_WRITE_AND_CLOSE
 ********************************************************************/
struct SmbWriteAndCloseReq   /* smb_wct = 6 */
{
    uint8_t smb_wct;      /* count of 16-bit words that follow */
    uint16_t smb_fid;     /* file handle (close after write) */
    uint16_t smb_count;   /* number of bytes to write */
    uint32_t smb_offset;  /* offset in file to begin write */
    uint32_t smb_mtime;   /* modification time */
    uint16_t smb_bcc;     /* 1 (for pad) + value of smb_count */
};

struct SmbWriteAndCloseExtReq   /* smb_wct = 12 */
{
    uint8_t smb_wct;      /* count of 16-bit words that follow */
    uint16_t smb_fid;     /* file handle (close after write) */
    uint16_t smb_count;   /* number of bytes to write */
    uint32_t smb_offset;  /* offset in file to begin write */
    uint32_t smb_mtime;   /* modification time */
    uint32_t smb_rsvd1;   /* Optional */
    uint32_t smb_rsvd2;   /* Optional */
    uint32_t smb_rsvd3;   /* Optional */
    uint16_t smb_bcc;     /* 1 (for pad) + value of smb_count */
};

struct SmbWriteAndCloseResp   /* smb_wct = 1 */
{
    uint8_t smb_wct;     /* count of 16-bit words that follow */
    uint16_t smb_count;  /* number of bytes written */
    uint16_t smb_bcc;    /* must be 0 */
};

inline uint16_t SmbWriteAndCloseReqFid(const SmbWriteAndCloseReq* req)
{
    return snort::alignedNtohs(&req->smb_fid);
}

inline uint16_t SmbWriteAndCloseReqCount(const SmbWriteAndCloseReq* req)
{
    return snort::alignedNtohs(&req->smb_count);
}

inline uint32_t SmbWriteAndCloseReqOffset(const SmbWriteAndCloseReq* req)
{
    return snort::alignedNtohl(&req->smb_offset);
}

inline uint16_t SmbWriteAndCloseRespCount(const SmbWriteAndCloseResp* resp)
{
    return snort::alignedNtohs(&resp->smb_count);
}

#pragma pack()

void DCE2_SmbInitGlobals();
void DCE2_Smb1Process(struct DCE2_SmbSsnData*);
struct DCE2_SmbSsnData* dce2_create_new_smb_session(snort::Packet*, struct dce2SmbProtoConf*);
struct DCE2_Smb2SsnData* dce2_create_new_smb2_session(snort::Packet*, struct dce2SmbProtoConf*);
void DCE2_SmbDataFree(DCE2_SmbSsnData*);
void set_smb_reassembled_data(uint8_t* nb_ptr, uint16_t co_len);

#endif

service_inspectors/dnp3/

This directory contains all the files related to DNP3 protocol processing.

DNP3 is supported over both TCP/IP and UDP. The inspector provides anomaly detection and IPS rule options for matching on data, functions, indications and objects in the messages.

The protocol aware flushing (PAF) for DNP3 identifies and flushes frames at link layer. The reassembly code in dnp3_reassembly.cc handles transport layer reassembly and generates application layer PDUs for inspection. For the UDP case, packets do not go through PAF and the inspector handles link layer decoding. Since there can be multiple application level PDUs in a single UDP packet, the inspector performs detection as well.

Configuration of ports is handled by the binder. The default server port is 20000. The only other DNP3 specific configuration is CRC check enable/disable

dnp3.h

Path = src/service_inspectors/dnp3/dnp3.h

#ifndef DNP3_H
#define DNP3_H

#include "flow/flow.h"
#include "framework/counts.h"
#include "profiler/profiler_defs.h"

#define DNP3_NAME "dnp3"
#define DNP3_HELP "dnp3 inspection"

#define DNP3_BAD_CRC_STR  "DNP3 link-layer frame contains bad CRC"
#define DNP3_DROPPED_FRAME_STR "DNP3 link-layer frame is truncated or frame length is invalid"
#define DNP3_DROPPED_SEGMENT_STR "DNP3 transport-layer segment sequence number is incorrect"
#define DNP3_REASSEMBLY_BUFFER_CLEARED_STR \
    "DNP3 transport-layer segment flag violation is detected"
#define DNP3_RESERVED_ADDRESS_STR "DNP3 link-layer frame uses a reserved address"
#define DNP3_RESERVED_FUNCTION_STR "DNP3 application-layer fragment uses a reserved function code"

#define DNP3_BAD_CRC                    1
#define DNP3_DROPPED_FRAME              2
#define DNP3_DROPPED_SEGMENT            3
#define DNP3_REASSEMBLY_BUFFER_CLEARED  4
#define DNP3_RESERVED_ADDRESS           5
#define DNP3_RESERVED_FUNCTION          6

/* Packet directions */
#define DNP3_CLIENT 0
#define DNP3_SERVER 1

/* Session data flags */
#define DNP3_FUNC_RULE_FIRED    0x0001
#define DNP3_OBJ_RULE_FIRED     0x0002
#define DNP3_IND_RULE_FIRED     0x0004
#define DNP3_DATA_RULE_FIRED    0x0008

/* DNP3 minimum length: start (2 octets) + len (1 octet) */
#define DNP3_MIN_LEN 3
#define DNP3_LEN_OFFSET 2

/* Length of the rest of a DNP3 link-layer header: ctrl + src + dest */
#define DNP3_HEADER_REMAINDER_LEN 5

#define DNP3_BUFFER_SIZE 2048

#ifdef WORDS_BIGENDIAN
#define DNP3_START_BYTES       0x0564
#else
#define DNP3_START_BYTES       0x6405
#endif

#define DNP3_MIN_RESERVED_ADDR 0xFFF0
#define DNP3_MAX_RESERVED_ADDR 0xFFFB

#define DNP3_START_BYTE_1   0x05
#define DNP3_START_BYTE_2   0x64

#define DNP3_CHUNK_SIZE     16
#define DNP3_CRC_SIZE        2

/* Minimum length of DNP3 "len" field in order to get a transport header. */
#define DNP3_MIN_TRANSPORT_LEN 6
#define DNP3_MAX_TRANSPORT_LEN 250

#define DNP3_TPDU_MAX  250
#define DNP3_LPDU_MAX  292

#define DNP3_TRANSPORT_FIN(x) ((x) & 0x80)
#define DNP3_TRANSPORT_FIR(x) ((x) & 0x40)
#define DNP3_TRANSPORT_SEQ(x) ((x) & 0x3F)

/* Yep, the locations of FIR and FIN are switched at this layer... */
#define DNP3_APP_FIR(x) ((x) & 0x80)
#define DNP3_APP_FIN(x) ((x) & 0x40)
#define DNP3_APP_SEQ(x) ((x) & 0x0F)

#define DNP3_OK true
#define DNP3_FAIL false

struct Dnp3Stats
{
    PegCount total_packets;
    PegCount udp_packets;
    PegCount tcp_pdus;
    PegCount dnp3_link_layer_frames;
    PegCount dnp3_application_pdus;
    PegCount concurrent_sessions;
    PegCount max_concurrent_sessions;
};

/* DNP3 header structures */
struct dnp3_link_header_t
{
    uint16_t start;
    uint8_t len;
    uint8_t ctrl;
    uint16_t dest;
    uint16_t src;
};

struct dnp3_transport_header_t
{
    uint8_t control;
};

struct dnp3_app_request_header_t
{
    uint8_t control;
    uint8_t function;
};

struct dnp3_app_response_header_t
{
    uint8_t control;
    uint8_t function;
    uint16_t indications;
};

enum dnp3_reassembly_state_t
{
    DNP3_REASSEMBLY_STATE__IDLE = 0,
    DNP3_REASSEMBLY_STATE__ASSEMBLY,
    DNP3_REASSEMBLY_STATE__DONE
};

struct dnp3_reassembly_data_t
{
    uint8_t buffer[DNP3_BUFFER_SIZE] = {0};
    uint16_t buflen = 0;
    dnp3_reassembly_state_t state = DNP3_REASSEMBLY_STATE__IDLE;
    uint8_t last_seq = 0;
};

/* DNP3 session data */
struct dnp3_session_data_t
{
    /* Fields for rule option matching. */
    uint8_t direction = 0;
    uint8_t func = 0;
    uint8_t obj_group = 0;
    uint8_t obj_var = 0;
    uint16_t indications = 0;
    uint16_t flags = 0;

    /* Reassembly stuff */
    dnp3_reassembly_data_t client_rdata;
    dnp3_reassembly_data_t server_rdata;
};

class Dnp3FlowData : public snort::FlowData
{
public:
    Dnp3FlowData();
    ~Dnp3FlowData() override;

    static void init()
    { inspector_id = snort::FlowData::create_flow_data_id(); }

public:
    static unsigned inspector_id;
    dnp3_session_data_t dnp3_session;
};

extern THREAD_LOCAL Dnp3Stats dnp3_stats;
extern THREAD_LOCAL snort::ProfileStats dnp3_perf_stats;

#endif

dnp3_map.h

Path = src/service_inspectors/dnp3/dnp3_map.h

#ifndef DNP3_MAP__H
#define DNP3_MAP__H

#include <cstdint>

/* Check if "code" is in the function map.
 *
 * Returns: true on success, false on failure.
 */
bool dnp3_func_is_defined(uint16_t code);

/* Return the DNP3 function code corresponding to "name".
 *
 * Returns: integer
 *          -1 on failure
 */
int dnp3_func_str_to_code(const char* name);

/* Return the DNP3 indication code corresponding to "name".
 *
 * Returns: integer
 *          -1 on failure
 */
int dnp3_ind_str_to_code(const char* name);

#endif

dnp3_module.h

Path = src/service_inspectors/dnp3/dnp3_module.h

#ifndef DNP3_MODULE_H
#define DNP3_MODULE_H

#include "framework/module.h"

#define GID_DNP3  145

namespace snort
{
struct SnortConfig;
}
struct dnp3ProtoConf
{
    bool check_crc;
};

class Dnp3Module : public snort::Module
{
public:
    Dnp3Module();

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;

    unsigned get_gid() const override
    { return GID_DNP3; }

    const snort::RuleMap* get_rules() const override;
    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile() const override;

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

    void get_data(dnp3ProtoConf&);

private:
    dnp3ProtoConf config = {};
};

#endif

dnp3_paf.h

Path = src/service_inspectors/dnp3/dnp3_paf.h

#ifndef DNP3_PAF_H
#define DNP3_PAF_H

// Protocol aware flushing for DNP3.

#include "stream/stream_splitter.h"

enum dnp3_paf_state
{
    DNP3_PAF_STATE__START_1 = 0,
    DNP3_PAF_STATE__START_2,
    DNP3_PAF_STATE__LENGTH,
    DNP3_PAF_STATE__SET_FLUSH
};

struct dnp3_paf_data
{
    dnp3_paf_state paf_state;
    uint8_t dnp3_length;
    uint16_t real_length;
};

class Dnp3Splitter : public snort::StreamSplitter
{
public:
    Dnp3Splitter(bool c2s);

    Status scan(snort::Packet*, const uint8_t* data, uint32_t len,
        uint32_t flags, uint32_t* fp) override;

    bool is_paf() override
    {
        return true;
    }

public:
    dnp3_paf_data state;
};

#endif

dnp3_reassembly.h

Path = src/service_inspectors/dnp3/dnp3_reassembly.h

#ifndef DNP3_REASSEMBLY_H
#define DNP3_REASSEMBLY_H

#include "dnp3_module.h"
#include "dnp3.h"

bool dnp3_full_reassembly(const dnp3ProtoConf& config, dnp3_session_data_t* session, snort::Packet* p,
    const uint8_t* pdu_start, uint16_t pdu_length);
void dnp3_reassembly_reset(dnp3_reassembly_data_t* rdata);

#endif

service_inspectors/dns/

The DNS inspector decodes DNS Responses in DNS PDUs and can detect the following exploits: DNS Client RData Overflow, Obsolete Record Types, and Experimental Record Types.

DNS looks are DNS Response traffic over UDP and TCP and it requires Stream inspector to be enabled for TCP decoding.

dns_config.h

Path = src/service_inspectors/dns/dns_config.h

#ifndef DNS_CONFIG_H
#define DNS_CONFIG_H

class DnsConfig
{
public:
    DnsConfig() = default;
    ~DnsConfig() = default;

    void show() const;

    bool publish_response = false;
};

#endif

dns.h

Path = src/service_inspectors/dns/dns.h

#ifndef DNS_H
#define DNS_H

#include "flow/flow.h"

#include "pub_sub/dns_events.h"

// Implementation header with definitions, datatypes and flowdata class for
// DNS service inspector.

// Directional defines
#define DNS_DIR_FROM_SERVER 1
#define DNS_DIR_FROM_CLIENT 2

struct DNSHdr
{
    uint16_t id = 0;
    uint16_t flags = 0;
    uint16_t questions = 0;
    uint16_t answers = 0;
    uint16_t authorities = 0;
    uint16_t additionals = 0;
};

#define DNS_HDR_FLAG_REPLY_CODE_MASK        0x000F
#define DNS_HDR_FLAG_NON_AUTHENTICATED_OK   0x0010
#define DNS_HDR_FLAG_ANS_AUTHENTICATED      0x0020
#define DNS_HDR_FLAG_RESERVED               0x0040
#define DNS_HDR_FLAG_RECURSION_AVAIL        0x0080
#define DNS_HDR_FLAG_RECURSION_DESIRED      0x0100
#define DNS_HDR_FLAG_TRUNCATED              0x0200
#define DNS_HDR_FLAG_AUTHORITATIVE          0x0400
#define DNS_HDR_FLAG_OPCODE_MASK            0x7800
#define DNS_HDR_FLAG_RESPONSE               0x8000

struct DNSQuestion
{
    uint16_t type = 0;
    uint16_t dns_class = 0;
};

struct DNSRR
{
    uint16_t type = 0;
    uint16_t dns_class = 0;
    uint32_t ttl = 0;
    uint16_t length = 0;
};

// FIXIT-L replace alerted/relative to bool?
struct DNSNameState
{
    uint32_t txt_count = 0;
    uint32_t total_txt_len = 0;
    uint8_t txt_len = 0;
    uint8_t txt_bytes_seen = 0;
    uint8_t name_state = 0;
    uint8_t alerted = 0;
    uint16_t offset = 0;
    uint8_t relative = 0;
    std::string dns_name;

    void get_dns_name(std::string& name) const
    {
        if (dns_name.size())
            name = dns_name;
    }
};

// FIXIT-L  remove obsolete flags?
#define DNS_RR_TYPE_A                       0x0001
#define DNS_RR_TYPE_NS                      0x0002
#define DNS_RR_TYPE_MD                      0x0003 // obsolete
#define DNS_RR_TYPE_MF                      0x0004 // obsolete
#define DNS_RR_TYPE_CNAME                   0x0005
#define DNS_RR_TYPE_SOA                     0x0006
#define DNS_RR_TYPE_MB                      0x0007 // experimental
#define DNS_RR_TYPE_MG                      0x0008 // experimental
#define DNS_RR_TYPE_MR                      0x0009 // experimental
#define DNS_RR_TYPE_NULL                    0x000a // experimental
#define DNS_RR_TYPE_WKS                     0x000b
#define DNS_RR_TYPE_PTR                     0x000c
#define DNS_RR_TYPE_HINFO                   0x000d
#define DNS_RR_TYPE_MINFO                   0x000e // experimental
#define DNS_RR_TYPE_MX                      0x000f
#define DNS_RR_TYPE_TXT                     0x0010
#define DNS_RR_TYPE_AAAA                    0x001c

#define DNS_FLAG_NOT_DNS                0x01

// DNSSessionData States
#define DNS_RESP_STATE_LENGTH           0x00 // 2 bytes - TCP only
#define DNS_RESP_STATE_LENGTH_PART      0x01 // Partial length

#define DNS_RESP_STATE_HDR              0x10 // 12 bytes
#define DNS_RESP_STATE_HDR_ID           0x11 //  (2 bytes)
#define DNS_RESP_STATE_HDR_ID_PART      0x12 //  (2 bytes)
#define DNS_RESP_STATE_HDR_FLAGS        0x13 //  (2 bytes)
#define DNS_RESP_STATE_HDR_FLAGS_PART   0x14 //  (2 bytes)
#define DNS_RESP_STATE_HDR_QS           0x15 //  (2 bytes)
#define DNS_RESP_STATE_HDR_QS_PART      0x16 //  (2 bytes)
#define DNS_RESP_STATE_HDR_ANSS         0x17 //  (2 bytes)
#define DNS_RESP_STATE_HDR_ANSS_PART    0x18 //  (2 bytes)
#define DNS_RESP_STATE_HDR_AUTHS        0x19 //  (2 bytes)
#define DNS_RESP_STATE_HDR_AUTHS_PART   0x1a //  (2 bytes)
#define DNS_RESP_STATE_HDR_ADDS         0x1b //  (2 bytes)
#define DNS_RESP_STATE_HDR_ADDS_PART    0x1c //  (2 bytes)

#define DNS_RESP_STATE_QUESTION         0x20 // 4 bytes
#define DNS_RESP_STATE_Q_NAME           0x21 // (size depends on data)
#define DNS_RESP_STATE_Q_NAME_COMPLETE  0x22 // (size depends on data)
#define DNS_RESP_STATE_Q_TYPE           0x23 //  (2 bytes)
#define DNS_RESP_STATE_Q_TYPE_PART      0x24 //  (2 bytes)
#define DNS_RESP_STATE_Q_CLASS          0x25 //  (2 bytes)
#define DNS_RESP_STATE_Q_CLASS_PART     0x26 //  (2 bytes)
#define DNS_RESP_STATE_Q_COMPLETE       0x27

#define DNS_RESP_STATE_NAME_SIZE        0x31 // (1 byte)
#define DNS_RESP_STATE_NAME             0x32 // (size depends on field)
#define DNS_RESP_STATE_NAME_COMPLETE    0x33

#define DNS_RESP_STATE_ANS_RR           0x40 // (size depends on field)
#define DNS_RESP_STATE_RR_NAME_SIZE     0x41 // (1 byte)
#define DNS_RESP_STATE_RR_NAME          0x42 // (size depends on field)
#define DNS_RESP_STATE_RR_NAME_COMPLETE 0x43
#define DNS_RESP_STATE_RR_TYPE          0x44 //  (2 bytes)
#define DNS_RESP_STATE_RR_TYPE_PART     0x45 //  (2 bytes)
#define DNS_RESP_STATE_RR_CLASS         0x46 //  (2 bytes)
#define DNS_RESP_STATE_RR_CLASS_PART    0x47 //  (2 bytes)
#define DNS_RESP_STATE_RR_TTL           0x48 //  (4 bytes)
#define DNS_RESP_STATE_RR_TTL_PART      0x49 //  (4 bytes)
#define DNS_RESP_STATE_RR_RDLENGTH      0x4a //  (2 bytes)
#define DNS_RESP_STATE_RR_RDLENGTH_PART 0x4b //  (2 bytes)
#define DNS_RESP_STATE_RR_RDATA_START   0x4c // (size depends on RDLENGTH)
#define DNS_RESP_STATE_RR_RDATA_MID     0x4d // (size depends on RDLENGTH)
#define DNS_RESP_STATE_RR_COMPLETE      0x4e

#define DNS_RESP_STATE_AUTH_RR          0x50
#define DNS_RESP_STATE_ADD_RR           0x60

class DnsConfig;
struct DNSData;

class DnsResponseFqdn
{
public:
    DnsResponseFqdn()
    {}

    DnsResponseFqdn(const unsigned char* data, uint16_t bytes_unused, DNSData* dnsSessionData) :
        data(data), bytes_unused(bytes_unused), dnsSessionData(std::make_shared<DNSData>(*dnsSessionData))
    {}

    FqdnTtl get_fqdn();
    void update_ttl(uint32_t ttl);

private:
    const unsigned char* data = nullptr;
    uint16_t bytes_unused = 0;
    std::shared_ptr<DNSData> dnsSessionData;
};

// Per-session data block containing current state
// of the DNS inspector for the session.
struct DNSData
{
    uint32_t state = 0;               // The current state of the session.
    uint16_t curr_rec = 0;            // Record number for the current record
    uint16_t curr_rec_length = 0;
    uint16_t bytes_seen_curr_rec = 0;
    uint16_t length = 0;
    uint16_t bytes_unused = 0;
    uint8_t curr_rec_state = 0;
    DNSHdr hdr;                   // Copy of the data from the DNS Header
    DNSQuestion curr_q;
    DNSRR curr_rr;
    DNSNameState curr_txt;
    uint8_t flags = 0;
    std::vector<unsigned char> data;
    const DnsConfig* dns_config = nullptr;
    snort::DnsResponseDataEvents dns_events;
    DnsResponseFqdn cur_fqdn_event;

    bool publish_response() const;
    bool has_events() const;
};

class DnsResponseIp
{
public:
    DnsResponseIp()
    {}

    DnsResponseIp(const unsigned char* data, uint16_t type) :
        data(data), type(type)
    {}

    snort::SfIp get_ip();

private:
    const unsigned char* data = nullptr;
    uint16_t type = 0;
};

class DnsFlowData : public snort::FlowData
{
public:
    DnsFlowData();
    ~DnsFlowData() override;

    static void init()
    { inspector_id = snort::FlowData::create_flow_data_id(); }

public:
    static unsigned inspector_id;
    DNSData session;
};

#endif

dns_module.h

Path = src/service_inspectors/dns/dns_module.h

#ifndef DNS_MODULE_H
#define DNS_MODULE_H
//Interface to the DNS service inspector

#include "framework/bits.h"
#include "framework/module.h"
#include "main/thread.h"

namespace snort
{
struct SnortConfig;
}

#define GID_DNS 131

#define DNS_EVENT_OBSOLETE_TYPES            1
#define DNS_EVENT_EXPERIMENTAL_TYPES        2
#define DNS_EVENT_RDATA_OVERFLOW            3

#define DNS_NAME "dns"
#define DNS_HELP "dns inspection"

class DnsConfig;

struct DnsStats
{
    PegCount packets;
    PegCount requests;
    PegCount responses;
    PegCount concurrent_sessions;
    PegCount max_concurrent_sessions;
};

extern const PegInfo dns_peg_names[];
extern THREAD_LOCAL DnsStats dnsstats;
extern THREAD_LOCAL snort::ProfileStats dnsPerfStats;

class DnsModule : public snort::Module
{
public:
    DnsModule();
    ~DnsModule() override;

    bool begin(const char*, int, snort::SnortConfig*) override;
    bool set(const char*, snort::Value&, snort::SnortConfig*) override;

    unsigned get_gid() const override
    { return GID_DNS; }

    const snort::RuleMap* get_rules() const override;
    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile() const override;
    const DnsConfig* get_config();

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

private:
    DnsConfig* config = nullptr;
};

#endif

dns_splitter.h

Path = src/service_inspectors/dns/dns_splitter.h

#ifndef DNS_SPLITTER_H
#define DNS_SPLITTER_H

#include "stream/stream_splitter.h"

class DnsSplitter : public snort::StreamSplitter
{
public:
    DnsSplitter(bool c2s) : snort::StreamSplitter(c2s) { }

    Status scan(snort::Packet* p, const uint8_t* data, uint32_t len,
        uint32_t flags, uint32_t* fp) override;

    bool is_paf() override
    { return true; }

private:
    bool partial = false;
    uint16_t size = 0;
};

#endif

service_inspectors/ftp_telnet/

Telnet, FTP, and FTP-Data service inspectors. Telnet and FTP are normal inspectors, FTP-Data detected from the FTP control channel and used primarily for file processing.

FTP and Telnet inspectors share implementation because FTP uses a Telnet-based control channel.

ft_main.h

Path = src/service_inspectors/ftp_telnet/ft_main.h

#ifndef FT_MAIN_H
#define FT_MAIN_H

#include "target_based/snort_protocols.h"

#include "ftpp_ui_config.h"

#define BUF_SIZE 1024

namespace snort
{
struct Packet;
struct SnortConfig;
}

#define FTP_CLIENT_NAME "ftp_client"
#define FTP_DATA_NAME "ftp_data"
#define FTP_SERVER_NAME "ftp_server"

extern SnortProtocolId ftp_data_snort_protocol_id;

void do_detection(snort::Packet*);

void CleanupFTPServerConf(void* serverConf);
void CleanupFTPCMDConf(void* ftpCmd);
void CleanupFTPBounceTo(void* ftpBounce);

int CheckFTPServerConfigs(snort::SnortConfig*, FTP_SERVER_PROTO_CONF*);
int FTPCheckConfigs(snort::SnortConfig*, void*);

FTP_CLIENT_PROTO_CONF* get_ftp_client(snort::Packet*);
FTP_SERVER_PROTO_CONF* get_ftp_server(snort::Packet*);

#endif

ftp_bounce_lookup.h

Path = src/service_inspectors/ftp_telnet/ftp_bounce_lookup.h

#ifndef FTP_BOUNCE_LOOKUP_H
#define FTP_BOUNCE_LOOKUP_H

#include "ftpp_ui_config.h"

int ftp_bounce_lookup_init(BOUNCE_LOOKUP** BounceLookup);
int ftp_bounce_lookup_cleanup(BOUNCE_LOOKUP** BounceLookup);
int ftp_bounce_lookup_add(BOUNCE_LOOKUP* BounceLookup, const snort::SfIp* ip, FTP_BOUNCE_TO* BounceTo);

FTP_BOUNCE_TO* ftp_bounce_lookup_find(BOUNCE_LOOKUP* BounceLookup, const snort::SfIp* ip, int* iError);
FTP_BOUNCE_TO* ftp_bounce_lookup_first(BOUNCE_LOOKUP* BounceLookup, int* iError);
FTP_BOUNCE_TO* ftp_bounce_lookup_next(BOUNCE_LOOKUP* BounceLookup, int* iError);

#endif

ftp_client.h

Path = src/service_inspectors/ftp_telnet/ftp_client.h

#ifndef FTP_CLIENT_H
#define FTP_CLIENT_H

/*
 * FTP Client Module
 *
 * This file defines the client reqest structure and functions
 * to access client inspection.
 */

struct FTP_CLIENT_REQ
{
    const char* cmd_line;
    unsigned int cmd_line_size;

    const char* cmd_begin;
    const char* cmd_end;
    unsigned int cmd_size;

    const char* param_begin;
    const char* param_end;
    unsigned int param_size;

    const char* pipeline_req;
};

struct FTP_CLIENT
{
    FTP_CLIENT_REQ request;
    int (* state)(void*, unsigned char, int);
};

int ftp_client_inspection(void* session, unsigned char* data, int dsize);

#endif

ftp_cmd_lookup.h

Path = src/service_inspectors/ftp_telnet/ftp_cmd_lookup.h

#ifndef FTP_CMD_LOOKUP_H
#define FTP_CMD_LOOKUP_H

#include "ftpp_ui_config.h"

int ftp_cmd_lookup_init(CMD_LOOKUP** CmdLookup);
int ftp_cmd_lookup_cleanup(CMD_LOOKUP** CmdLookup);
int ftp_cmd_lookup_add(CMD_LOOKUP* CmdLookup, const char* cmd, int len, FTP_CMD_CONF* FTPCmd);

FTP_CMD_CONF* ftp_cmd_lookup_find(CMD_LOOKUP* CmdLookup, const char cmd[], int len, int* iError);
FTP_CMD_CONF* ftp_cmd_lookup_first(CMD_LOOKUP* CmdLookup, int* iError);
FTP_CMD_CONF* ftp_cmd_lookup_next(CMD_LOOKUP* CmdLookup, int* iError);

#endif

ftp_data.h

Path = src/service_inspectors/ftp_telnet/ftp_data.h

#ifndef FTP_DATA_H
#define FTP_DATA_H

#include "framework/inspector.h"

extern const snort::InspectApi fd_api;

#endif

ftpdata_splitter.h

Path = src/service_inspectors/ftp_telnet/ftpdata_splitter.h

#ifndef FTPDATA_SPLITTER_H
#define FTPDATA_SPLITTER_H

#include "stream/flush_bucket.h"
#include "stream/stream_splitter.h"

//---------------------------------------------------------------------------------
// FtpDataSplitter - flush when current seg size is different from previous segment
//---------------------------------------------------------------------------------
class FtpDataSplitter : public snort::StreamSplitter
{
public:
    FtpDataSplitter(bool b, uint16_t sz = 0) : snort::StreamSplitter(b)
    {
        min = sz + get_flush_bucket_size();
        restart_scan();
        expected_seg_size = 0;
    }

    Status scan(snort::Packet*, const uint8_t*, uint32_t len, uint32_t flags, uint32_t* fp ) override;
    bool finish(snort::Flow*) override;
    bool init_partial_flush(snort::Flow*) override { return true; }

    bool is_paf() override
    { return true; }

private:
    uint16_t min;
    uint16_t segs;
    uint16_t bytes;
    uint16_t expected_seg_size;

    void restart_scan();
};

#endif

ftp_module.h

Path = src/service_inspectors/ftp_telnet/ftp_module.h

#ifndef FTP_MODULE_H
#define FTP_MODULE_H

#include "framework/module.h"
#include "ftpp_ui_config.h"

#define GID_FTP 125

#define FTP_TELNET_CMD                   1
#define FTP_INVALID_CMD                  2
#define FTP_PARAMETER_LENGTH_OVERFLOW    3
#define FTP_MALFORMED_PARAMETER          4
#define FTP_PARAMETER_STR_FORMAT         5
#define FTP_RESPONSE_LENGTH_OVERFLOW     6
#define FTP_ENCRYPTED                    7
#define FTP_BOUNCE                       8
#define FTP_EVASIVE_TELNET_CMD           9

namespace snort
{
struct SnortConfig;
}

extern THREAD_LOCAL snort::ProfileStats ftpPerfStats;

//-------------------------------------------------------------------------

struct BounceTo
{
    std::string address;
    Port low;
    Port high;

    BounceTo(const std::string& address, Port lo, Port hi);
};

class FtpClientModule : public snort::Module
{
public:
    FtpClientModule();
    ~FtpClientModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    FTP_CLIENT_PROTO_CONF* get_data();
    const BounceTo* get_bounce(unsigned idx);

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

private:
    FTP_CLIENT_PROTO_CONF* conf = nullptr;
    std::vector<BounceTo*> bounce_to;

    std::string address;
    Port port = 0;
    Port last_port = 0;
};

//-------------------------------------------------------------------------

#define CMD_LEN    0x0000
#define CMD_ALLOW  0x0001
#define CMD_CHECK  0x0002
#define CMD_DATA   0x0004
#define CMD_XFER   0x0008
#define CMD_PUT    0x0010
#define CMD_GET    0x0020
#define CMD_LOGIN  0x0040
#define CMD_ENCR   0x0080
#define CMD_DIR    0x0100
#define CMD_VALID  0x0200
#define CMD_REST   0x0400
#define CMD_PROT   0x0800

struct FtpCmd
{
    std::string name;
    std::string format;

    uint32_t flags;
    unsigned number;

    FtpCmd(const std::string&, uint32_t, int);
    FtpCmd(const std::string&, const std::string&, int);
};

class FtpServerModule : public snort::Module
{
public:
    FtpServerModule();
    ~FtpServerModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    unsigned get_gid() const override
    { return GID_FTP; }

    const snort::RuleMap* get_rules() const override;
    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile() const override;

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

    FTP_SERVER_PROTO_CONF* get_data();
    const FtpCmd* get_cmd(unsigned idx);

private:
    void add_commands(snort::Value&, uint32_t flags, int num = 0);

private:
    FTP_SERVER_PROTO_CONF* conf = nullptr;
    std::vector<FtpCmd*> cmds;
    std::string names;
    std::string format;
    int number = 0;
};

#endif

ftp_parse.h

Path = src/service_inspectors/ftp_telnet/ftp_parse.h

#ifndef FTP_PARSE_H
#define FTP_PARSE_H

#include "ftpp_ui_config.h"

/*
 * These are the definitions of the parser section delimiting
 * keywords to configure FtpTelnet.  When one of these keywords
 * are seen, we begin a new section.
 */
#define TELNET        "telnet"
#define FTP           "ftp"
#define CLIENT        "client"
#define SERVER        "server"

/*
 * Data type keywords
 */
#define START_CMD_FORMAT    "<"
#define END_CMD_FORMAT      ">"
#define F_INT               "int"
#define F_NUMBER            "number"
#define F_CHAR              "char"
#define F_DATE              "date"
#define F_LITERAL           "'"
#define F_STRING            "string"
#define F_STRING_FMT        "formatted_string"
#define F_HOST_PORT         "host_port"
#define F_LONG_HOST_PORT    "long_host_port"
#define F_EXTD_HOST_PORT    "extd_host_port"

int ProcessFTPCmdValidity(
    FTP_SERVER_PROTO_CONF* ServerConf,
    const char* cmd, const char* fmt,
    char* ErrorString, int ErrStrLen);

int ProcessFTPAllowBounce(
    FTP_CLIENT_PROTO_CONF* ClientConf, const uint8_t* addr, unsigned len,
    Port low, Port high);

#endif

ftpp_return_codes.h

Path = src/service_inspectors/ftp_telnet/ftpp_return_codes.h

#ifndef FTPP_RETURN_CODES_H
#define FTPP_RETURN_CODES_H

#define FTPP_SUCCESS    0

/*
 * Non-fatal errors are positive
 */
#define FTPP_BOOL_TRUE          1
#define FTPP_NONFATAL_ERR       1
#define FTPP_OUT_OF_BOUNDS      2

#define FTPP_INVALID_PROTO      3
#define FTPP_NORMALIZED         4
#define FTPP_MALFORMED_FTP_RESPONSE  5
#define FTPP_ALERTED            6
#define FTPP_NON_DIGIT          7
#define FTPP_MALFORMED_IP_PORT  8
#define FTPP_PORT_ATTACK        9

#define FTPP_INVALID_SESSION    10

#define FTPP_OR_FOUND           100
#define FTPP_OPT_END_FOUND      101
#define FTPP_CHOICE_END_FOUND   102

/*
 * Fatal errors are negative
 */
#define FTPP_FATAL_ERR         (-1)
#define FTPP_INVALID_ARG       (-2)
#define FTPP_MEM_ALLOC_FAIL    (-3)
#define FTPP_NOT_FOUND         (-4)
#define FTPP_INVALID_FILE      (-5)

#define FTPP_ALERT             (-6)

#define FTPP_INVALID_DATE      (-100)
#define FTPP_INVALID_PARAM     (-101)

#endif

ftp_print.h

Path = src/service_inspectors/ftp_telnet/ftp_print.h

#ifndef FTP_PRINT_H
#define FTP_PRINT_H

#include "ftpp_ui_config.h"

void print_conf_client(FTP_CLIENT_PROTO_CONF*);
void print_conf_server(FTP_SERVER_PROTO_CONF*);

#endif

ftpp_si.h

Path = src/service_inspectors/ftp_telnet/ftpp_si.h

#ifndef FTPP_SI_H
#define FTPP_SI_H

#include "file_api/file_api.h"
#include "flow/flow.h"
#include "flow/flow_key.h"
#include "framework/counts.h"

#include "ftp_client.h"
#include "ftp_server.h"
#include "ftpp_ui_config.h"

/*
 * These are the defines for the different types of
 * inspection modes.  We have a server mode and a client mode.
 */
#define FTPP_SI_NO_MODE     0
#define FTPP_SI_CLIENT_MODE 1
#define FTPP_SI_SERVER_MODE 2

#define FTPP_SI_PROTO_UNKNOWN   0
#define FTPP_SI_PROTO_TELNET    1
#define FTPP_SI_PROTO_FTP       2
#define FTPP_SI_PROTO_FTP_DATA  3

#define FTPP_FILE_IGNORE    (-1)
#define FTPP_FILE_UNKNOWN    0

/* Macros for testing the type of FTP_TELNET_SESSION */
#define FTPP_SI_IS_PROTO(Ssn, Pro)      ((Ssn) && ((Ssn)->ft_ssn.proto == (Pro)))
#define PROTO_IS_FTP(ssn)               FTPP_SI_IS_PROTO(ssn, FTPP_SI_PROTO_FTP)
#define PROTO_IS_FTP_DATA(ssn)          FTPP_SI_IS_PROTO(ssn, FTPP_SI_PROTO_FTP_DATA)
#define PROTO_IS_TELNET(ssn)            FTPP_SI_IS_PROTO(ssn, FTPP_SI_PROTO_TELNET)

#define FTP_FLG_MALWARE                 0x01
#define FTP_FLG_SEARCH_ABANDONED        0x02
#define FTP_PROTP_CMD_ISSUED            0x04
#define FTP_PROTP_CMD_ACCEPT            0x08

typedef struct s_FTP_TELNET_SESSION
{
    int proto;
} FTP_TELNET_SESSION;

/*
 * The TELNET_SESSION structure contains the complete TELNET session.
 * This structure is the structure that is saved per session in the
 * Stream Interface Module.  This structure gets sent through the
 * detection engine process (Normalization, Detection).
 */
struct TELNET_SESSION
{
    FTP_TELNET_SESSION ft_ssn;

    /* The client configuration for this session if its FTP */
    TELNET_PROTO_CONF* telnet_conf;

    /* Number of consecutive are-you-there commands seen. */
    int consec_ayt;

    int encr_state;
};

class TelnetFlowData : public snort::FlowData
{
public:
    TelnetFlowData();
    ~TelnetFlowData() override;

    static void init()
    { inspector_id = snort::FlowData::create_flow_data_id(); }

public:
    static unsigned inspector_id;
    TELNET_SESSION session;
};

/*
 * These are the state values for determining the FTP data channel.
 */
#define NO_STATE                  0x00
#define LOST_STATE                0xFFFFFFFF

#define DATA_CHAN_PORT_CMD_ISSUED   0x01
#define DATA_CHAN_PORT_CMD_ACCEPT   0x02
#define DATA_CHAN_PASV_CMD_ISSUED   0x04
#define DATA_CHAN_PASV_CMD_ACCEPT   0x08
#define DATA_CHAN_XFER_CMD_ISSUED   0x10
#define DATA_CHAN_XFER_STARTED      0x20
#define DATA_CHAN_CLIENT_HELLO_SEEN 0x40
#define DATA_CHAN_REST_CMD_ISSUED   0x80

#define AUTH_TLS_CMD_ISSUED         0x01
#define AUTH_SSL_CMD_ISSUED         0x02
#define AUTH_UNKNOWN_CMD_ISSUED     0x04
#define AUTH_TLS_ENCRYPTED          0x08
#define AUTH_SSL_ENCRYPTED          0x10
#define AUTH_UNKNOWN_ENCRYPTED      0x20

/*
 * The FTP_SESSION structure contains the complete FTP session, both the
 * client and the server constructs.  This structure is the structure that
 * is saved per session in the Stream Interface Module.  This structure
 * gets sent through the detection engine process (Normalization,
 * Detection).
 */
struct FTP_SESSION
{
    FTP_TELNET_SESSION ft_ssn;

    /* The client construct contains all the info associated with a
     * client request. */
    FTP_CLIENT client;

    /* The server construct contains all the info associated with a
     * server response. */
    FTP_SERVER server;

    /* The client configuration for this session if its FTP */
    FTP_CLIENT_PROTO_CONF* client_conf;

    /* The server configuration for this session if its FTP */
    FTP_SERVER_PROTO_CONF* server_conf;

    /* The data channel info */
    int data_chan_state;
    int data_chan_index;
    int data_xfer_index;
    bool data_xfer_dir;
    snort::SfIp clientIP;
    uint16_t clientPort;
    snort::SfIp serverIP;
    uint16_t serverPort;

    /* A file is being transferred on ftp-data channel */
    char* filename;
    size_t path_hash;
    int file_xfer_info; /* -1: ignore, 0: unknown, >0: filename length */
    unsigned char flags;

    /* Command/data channel encryption */
    int encr_state;
    void *datassn;
};

void FTPFreesession(FTP_SESSION*);

class FtpFlowData : public snort::FlowData
{
public:
    FtpFlowData();
    ~FtpFlowData() override;

    static void init()
    { inspector_id = snort::FlowData::create_flow_data_id(); }

    size_t size_of() override
    { return sizeof(*this); }

public:
    static unsigned inspector_id;
    FTP_SESSION session;
};

/* FTP-Data Transfer Modes */
enum
{
    FTPP_XFER_PASSIVE = 0,
    FTPP_XFER_ACTIVE  = 1
};

struct FTP_DATA_SESSION
{
    FTP_TELNET_SESSION ft_ssn;
    snort::FlowKey ftp_key;
    char* filename;
    size_t path_hash;
    int data_chan;
    int file_xfer_info;
    FilePosition position;
    unsigned char mode;
    unsigned char packet_flags;
    bool direction;
    bool mss_changed;
};

class FtpDataFlowData : public snort::FlowData
{
public:
    FtpDataFlowData(snort::Packet*);
    ~FtpDataFlowData() override;

    static void init()
    { inspector_id = snort::FlowData::create_flow_data_id(); }

    void handle_expected(snort::Packet*) override;
    void handle_eof(snort::Packet*) override;
    size_t size_of() override
    { return sizeof(*this); }

public:
    static unsigned inspector_id;
    FTP_DATA_SESSION session;
    bool eof_handled = false;
    bool in_tls = false;
};

#define FTPDATA_FLG_REASSEMBLY_SET  (1<<0)
#define FTPDATA_FLG_FILENAME_SET    (1<<1)
#define FTPDATA_FLG_STOP            (1<<2)
#define FTPDATA_FLG_REST            (1<<3)
#define FTPDATA_FLG_FLUSH           (1<<4)

/*
 * The FTPP_SI_INPUT structure holds the information that the session
 * Inspection Module needs to determine the type of inspection mode
 * (client, server, neither) and to retrieve the appropriate server
 * configuration.
 *
 * The input is the source and destination IP addresses, and the
 * source and destination ports (since this should always be a
 * TCP packet).
 */
struct FTPP_SI_INPUT
{
    snort::SfIp sip;
    snort::SfIp dip;
    unsigned short sport;
    unsigned short dport;
    unsigned char pdir;
    unsigned char pproto;
};

int FTPGetPacketDir(snort::Packet*);

/* FTP-Data file processing */
FTP_DATA_SESSION* FTPDatasessionNew(snort::Packet* p);
void FTPDatasessionFree(void* p_ssn);

bool FTPDataDirection(snort::Packet* p, FTP_DATA_SESSION* ftpdata);

int TelnetsessionInspection(snort::Packet*, TELNET_PROTO_CONF*, TELNET_SESSION**,
    FTPP_SI_INPUT*, int* piInspectMode);

int FTPsessionInspection(snort::Packet*, FTP_SESSION**, FTPP_SI_INPUT*, int* piInspectMode);

int SetSiInput(FTPP_SI_INPUT*, snort::Packet*);

struct FtpStats
{
    PegCount total_packets;
    PegCount total_bytes;
    PegCount concurrent_sessions;
    PegCount max_concurrent_sessions;
    PegCount starttls;
    PegCount ssl_search_abandoned;
    PegCount ssl_search_abandoned_too_soon;
    PegCount total_packets_mss_changed;
    PegCount total_sessions_mss_changed;
};

struct TelnetStats
{
    PegCount total_packets;
    PegCount concurrent_sessions;
    PegCount max_concurrent_sessions;
};

extern THREAD_LOCAL FtpStats ftstats;
extern THREAD_LOCAL TelnetStats tnstats;

#endif

ftpp_ui_config.h

Path = src/service_inspectors/ftp_telnet/ftpp_ui_config.h

#ifndef FTPP_UI_CONFIG_H
#define FTPP_UI_CONFIG_H

#include "sfip/sf_ip.h"
#include "utils/kmap.h"

/*
 * Defines
 */

#define FTPP_UI_CONFIG_TELNET_DEF_AYT_THRESHOLD (-1)
#define FTPP_UI_CONFIG_FTP_DEF_RESP_MSG_MAX (-1)
#define FTPP_UI_CONFIG_FTP_DEF_CMD_PARAM_MAX 100

#define MIN_CMD 3
#define MAX_CMD 4

typedef KMAP BOUNCE_LOOKUP;

/*
 * Defines a search type for the FTP commands in the client
 * global configuration.  We want this generic so we can change
 * it easily if we change the search type.
 */
typedef KMAP CMD_LOOKUP;

typedef enum s_FTP_PARAM_TYPE
{
    e_head = 0,
    e_unrestricted,   /* The default */
    e_strformat,
    e_int,
    e_number,
    e_char,
    e_date,
    e_literal,
    e_host_port,
    e_long_host_port,
    e_extd_host_port
}  FTP_PARAM_TYPE;

/*
 * Some FTP servers accept MDTM commands to set the modification time
 * on a file.  The most common are servers accept a format using
 * YYYYMMDDHHmmss[.uuu], while others accept a format using
 * YYYYMMDDHHmmss[+|-]TZ format.  Because of this, the default syntax
 * below is for the first case (time format as specified in
 * http://www.ietf.org/internet-drafts/draft-ietf-ftpext-mlst-16.txt)
 *
 * If you need to check validity for a server that uses the TZ format,
 * use the following:
 *
 * cmd_validity MDTM < [ date nnnnnnnnnnnnnn[{+|-}n[n]] ] string >
 *
 * Format uses the following:
 *  n = digit
 *  C = character
 *  . = period (literal)
 *  + = plus (literal)
 *  - = minus (literal)
 *  [ = optional begin
 *  ] = optional end
 *  { = OR begin
 *  } = OR end
 *  | = OR separator
 *
 *  ie, nnnnnnnnnnnnnn[.n[n[n]]]  -->
 *  force conformance to YYYYMMDDHHmmss.uuu,
 *  where 1,2, or 3 microsec digits are optional.
 *
 *  ie, nnnnnnnnnnnnnn[{+|-}n[n]] -->
 *  force conformance to YYYYMMDDHHmmss+TZ,
 *  where optional +TZ is + or - one or two digit number
 */
typedef struct s_FTP_DATE_FMT
{
    char* format_string;
    int empty;
    struct s_FTP_DATE_FMT* next;
    struct s_FTP_DATE_FMT* prev;
    struct s_FTP_DATE_FMT* optional;
    struct s_FTP_DATE_FMT* next_a;
    struct s_FTP_DATE_FMT* next_b;
} FTP_DATE_FMT;

typedef struct s_FTP_PARAM_FMT
{
    FTP_PARAM_TYPE type;
    int optional;

    /* Format is only used for types listed below to specify
     * allowable values.  Other types provide no variances
     * for the format.
     */
    union u_FORMAT
    {
        uint32_t chars_allowed;     /* For type == e_char */
        FTP_DATE_FMT* date_fmt;      /* For type == e_date */
        char* literal;               /* For type == e_literal */
    } format;

    struct s_FTP_PARAM_FMT* prev_param_fmt;
    struct s_FTP_PARAM_FMT* next_param_fmt;
    struct s_FTP_PARAM_FMT* optional_fmt;
    struct s_FTP_PARAM_FMT** choices;
    int numChoices;
    int prev_optional; /* Only set if optional is set */
}  FTP_PARAM_FMT;

typedef struct s_FTP_CMD_CONF
{
    /* Maximum length for parameters for this cmd.
     * Default -1 is unlimited */
    unsigned int max_param_len;
    int max_param_len_overridden;

    bool check_validity;
    bool data_chan_cmd;
    bool data_xfer_cmd;
    bool data_rest_cmd;
    bool file_put_cmd;
    bool file_get_cmd;
    bool encr_cmd;
    bool login_cmd;
    bool prot_cmd;
    int dir_response;

    FTP_PARAM_FMT* param_format;
    char cmd_name[1];  // variable length array
}  FTP_CMD_CONF;

/*
 * This is the configuration construct that holds the specific
 * options for a FTP server.  Each unique server has it's own
 * structure and there is a global structure for servers that
 * don't have a unique configuration.
 */
struct FTP_SERVER_PROTO_CONF
{
    unsigned int def_max_param_len = FTPP_UI_CONFIG_FTP_DEF_CMD_PARAM_MAX;
    unsigned int max_cmd_len = MAX_CMD;

    bool print_commands = false;
    bool data_chan = false;
    bool check_encrypted_data = false;
    bool telnet_cmds = false;
    bool ignore_telnet_erase_cmds = false;
    bool detect_encrypted = false;

    CMD_LOOKUP* cmd_lookup;

    FTP_SERVER_PROTO_CONF();
    ~FTP_SERVER_PROTO_CONF();
};

typedef struct s_FTP_BOUNCE_TO
{
    snort::SfIp ip;
    int relevant_bits;
    unsigned short portlo;
    unsigned short porthi;
} FTP_BOUNCE_TO;

/*
 * This is the configuration construct that holds the specific
 * options for a FTP client.  Each unique client has it's own
 * structure and there is a global structure for clients that
 * don't have a unique configuration.
 */
struct FTP_CLIENT_PROTO_CONF
{
    unsigned int max_resp_len = FTPP_UI_CONFIG_FTP_DEF_RESP_MSG_MAX;

    bool bounce = false;
    bool telnet_cmds = false;
    bool ignore_telnet_erase_cmds = false;

    // allow_bounce to IP/mask port|port-range
    // FIXIT-P change this to use a quick find of IP/mask
    BOUNCE_LOOKUP* bounce_lookup = nullptr;

    FTP_CLIENT_PROTO_CONF();
    ~FTP_CLIENT_PROTO_CONF();
};

/*
 * This is the configuration construct that holds the specific
 * options for telnet.  There is a global structure for all telnet
 * connections.
 */
struct TELNET_PROTO_CONF
{
    int ayt_threshold;

    bool normalize;
    bool check_encrypted_data;
    bool detect_encrypted;

    TELNET_PROTO_CONF();
};

int ftpp_ui_config_reset_ftp_client(FTP_CLIENT_PROTO_CONF* ClientConf,
    char first);
int ftpp_ui_config_reset_ftp_server(FTP_SERVER_PROTO_CONF* ServerConf,
    char first);
void ftpp_ui_config_reset_ftp_cmd_format(FTP_PARAM_FMT* ThisFmt);
void ftpp_ui_config_reset_ftp_cmd_date_format(FTP_DATE_FMT* DateFmt);
int ftpp_ui_config_reset_ftp_cmd(FTP_CMD_CONF* FTPCmd);
int ftpp_ui_config_reset_telnet_proto(TELNET_PROTO_CONF* ClientConf);

#endif

ftp_server.h

Path = src/service_inspectors/ftp_telnet/ftp_server.h

#ifndef FTP_SERVER_H
#define FTP_SERVER_H

/*
 * FTP Server Module
 *
 * This file defines the server structure and functions to access server
 * inspection.
 */

typedef struct s_FTP_SERVER_RSP
{
    char* rsp_line;
    unsigned int rsp_line_size;

    char* rsp_begin;
    char* rsp_end;
    unsigned int rsp_size;

    char* msg_begin;
    char* msg_end;
    unsigned int msg_size;

    char* pipeline_req;
    int state;
} FTP_SERVER_RSP;

typedef struct s_FTP_SERVER
{
    FTP_SERVER_RSP response;
} FTP_SERVER;

int ftp_server_inspection(void* S, unsigned char* data, int dsize);

#endif

pp_ftp.h

Path = src/service_inspectors/ftp_telnet/pp_ftp.h

#ifndef PP_FTP_H
#define PP_FTP_H

// declares the ftp checking functions

#include "ftpp_si.h"

extern int check_ftp(FTP_SESSION* session, snort::Packet* p, int iMode);
extern int initialize_ftp(FTP_SESSION* session, snort::Packet* p, int iMode);

#endif

pp_telnet.h

Path = src/service_inspectors/ftp_telnet/pp_telnet.h

#ifndef PP_TELNET_H
#define PP_TELNET_H

// declares the telnet checking functions

/* RFC 885 defines an End of Record telnet option */
#define RFC885
/* RFC 1184 defines Abort, Suspend, and End of File telnet options */
#define RFC1184

#include "ftpp_si.h"

/* define the telnet negotiation codes (TNC) that we're interested in */
#define TNC_IAC  0xFF
#define TNC_DONT 0xFE
#define TNC_DO   0xFD
#define TNC_WONT 0xFC
#define TNC_WILL 0xFB
#define TNC_SB   0xFA
#define TNC_GA   0xF9
#define TNC_EAL  0xF8
#define TNC_EAC  0xF7
#define TNC_AYT  0xF6
#define TNC_AO   0xF5
#define TNC_IP   0xF4
#define TNC_BRK  0xF3
#define TNC_DM   0xF2
#define TNC_NOP  0xF1
#define TNC_SE   0xF0
#ifdef RFC885
#define TNC_EOR  0xEF
#endif
#ifdef RFC1184
#define TNC_ABOR 0xEE
#define TNC_SUSP 0xED
#define TNC_EOF  0xEC
#endif

#define FTPP_APPLY_TNC_ERASE_CMDS 0
#define FTPP_IGNORE_TNC_ERASE_CMDS 1

struct DataBuffer;

extern int normalize_telnet(
    TELNET_SESSION*, snort::Packet*, DataBuffer&, int iMode, char ignoreEraseCmd, bool on_ftp_channel);

void reset_telnet_buffer(snort::Packet*);

#endif

telnet.h

Path = src/service_inspectors/ftp_telnet/telnet.h

#ifndef TELNET_H
#define TELNET_H

#include "framework/inspector.h"

extern const snort::InspectApi tn_api;

#endif

telnet_module.h

Path = src/service_inspectors/ftp_telnet/telnet_module.h

#ifndef TELNET_MODULE_H
#define TELNET_MODULE_H

#include "framework/module.h"
#include "ftpp_ui_config.h"

#define GID_TELNET 126

#define TELNET_AYT_OVERFLOW   1
#define TELNET_ENCRYPTED      2
#define TELNET_SB_NO_SE       3

#define TEL_NAME "telnet"
#define TEL_HELP "telnet inspection and normalization"

namespace snort
{
struct SnortConfig;
}

extern THREAD_LOCAL snort::ProfileStats telnetPerfStats;

class TelnetModule : public snort::Module
{
public:
    TelnetModule();
    ~TelnetModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    unsigned get_gid() const override
    { return GID_TELNET; }

    const snort::RuleMap* get_rules() const override;
    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile() const override;

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

    TELNET_PROTO_CONF* get_data();

private:
    TELNET_PROTO_CONF* conf;
};

#endif

telnet_splitter.h

Path = src/service_inspectors/ftp_telnet/telnet_splitter.h

#ifndef TELNET_SPLITTER_H
#define TELNET_SPLITTER_H

#include "stream/stream_splitter.h"
#include "pp_telnet.h"

class TelnetSplitter : public snort::StreamSplitter
{
public:
    TelnetSplitter(bool c2s);

    Status scan(snort::Packet*, const uint8_t* data, uint32_t len,
        uint32_t flags, uint32_t* fp) override;

    bool is_paf() override { return true; }
private:
    enum TelnetState { TELNET_NONE, TELNET_IAC, TELNET_IAC_SB, TELNET_IAC_SB_IAC };
    TelnetState state { TELNET_NONE };
};

#endif

service_inspectors/gtp/

gtp_inspect provides anomaly detection and IPS rule options for matching on version, types, and information elements in GTP control messages. The inspector is independent of the codec used for data channel encapsulations.

At present 3 versions are supported: 0, 1, and 2. The basic configuration is provided in snort_defaults.lua. Adding a version 3 requires updating both code and lua defaults. The key code change is to MAX_GTP_VERSION_CODE, which is defined in gtp.h.

PDU processing for all versions is currently in gtp_parser.cc. A message ID is used to determine if information element data applies to the current message. This eliminates the need to reset all possible information element offsets. This data is used to support rule options.

GTP control messages do not contain sufficient information to enable automatic protocol detection, hence there is no support in the wizard. Ports, typically UDP 2123, must be configured in the binder.

gtp.h

Path = src/service_inspectors/gtp/gtp.h

#ifndef GTP_H
#define GTP_H

// Provides convenience functions for parsing and querying configuration.

#include <string>

#include "framework/counts.h"
#include "main/thread.h"

namespace snort
{
struct Packet;
}

#define MIN_GTP_VERSION_CODE   (0)
#define MAX_GTP_VERSION_CODE   (2)

#define MIN_GTP_TYPE_CODE      (0)
#define MAX_GTP_TYPE_CODE      (255)

#define MIN_GTP_IE_CODE        (0)
#define MAX_GTP_IE_CODE        (255)

struct GTP_MsgType
{
    std::string name;
};

struct GTP_InfoElement
{
    std::string name;
    uint16_t length; // 0 for variable length
};

struct GTPConfig
{
    GTP_MsgType msgv[MAX_GTP_VERSION_CODE + 1][MAX_GTP_TYPE_CODE + 1];
    GTP_InfoElement infov[MAX_GTP_VERSION_CODE + 1][MAX_GTP_IE_CODE + 1];
};

struct GTP_Stats
{
    PegCount sessions;
    PegCount concurrent_sessions;
    PegCount max_concurrent_sessions;
    PegCount events;
    PegCount unknownTypes;
    PegCount unknownIEs;

    // FIXIT-L add these stats
    //PegCount messages[MAX_GTP_VERSION_CODE + 1][MAX_GTP_TYPE_CODE + 1];

    // FIXIT-L can't put non-pegs in stats; why is this here?
    //GTP_MsgType* msgTypeTable[MAX_GTP_VERSION_CODE + 1][MAX_GTP_TYPE_CODE + 1];
};

extern THREAD_LOCAL GTP_Stats gtp_stats;

void GTPmain(const GTPConfig&, snort::Packet*);

#endif

gtp_inspect.h

Path = src/service_inspectors/gtp/gtp_inspect.h

#ifndef GTP_INSPECT_H
#define GTP_INSPECT_H

#include "flow/flow.h"

#include "gtp_parser.h"

// FIXIT-M why store per packet on flow?
struct GTP_Roptions
{
    uint8_t gtp_type;
    uint8_t gtp_version;
    const uint8_t* gtp_header;
    uint32_t msg_id;  /* used to associate to current msg */
    GTP_IEData* gtp_infoElements;
};

class GtpFlowData : public snort::FlowData
{
public:
    GtpFlowData();
    ~GtpFlowData() override;

    static void init();

public:
    static unsigned inspector_id;
    GTP_Roptions ropts;
};

namespace snort
{
struct SnortConfig;
}

int get_message_type(int version, const char* name, snort::SnortConfig*);
int get_info_type(int version, const char* name, snort::SnortConfig*);

struct GTP_IEData* get_infos();

#endif

gtp_module.h

Path = src/service_inspectors/gtp/gtp_module.h

#ifndef GTP_MODULE_H
#define GTP_MODULE_H

#include "framework/module.h"

#define GID_GTP 143

#define GTP_EVENT_BAD_MSG_LEN        (1)
#define GTP_EVENT_BAD_IE_LEN         (2)
#define GTP_EVENT_OUT_OF_ORDER_IE    (3)
#define GTP_EVENT_MISSING_TEID       (4)

#define GTP_NAME "gtp_inspect"
#define GTP_HELP "gtp control channel inspection"

namespace snort
{
class Trace;
}

extern THREAD_LOCAL snort::ProfileStats gtp_inspect_prof;
extern THREAD_LOCAL const snort::Trace* gtp_inspect_trace;

struct GtpStuff
{
    std::string name;
    int version = 0;
    int type = 0;
    int length = 0;
};

class GtpInspectModule : public snort::Module
{
public:
    GtpInspectModule();

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    unsigned get_gid() const override
    { return GID_GTP; }

    const snort::RuleMap* get_rules() const override;

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;

    snort::ProfileStats* get_profile() const override
    { return &gtp_inspect_prof; }

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

    void set_trace(const snort::Trace*) const override;
    const snort::TraceOption* get_trace_options() const override;

public:
    GtpStuff stuff;
    std::vector<GtpStuff> temp;
    std::vector<GtpStuff> config;
};

#endif

gtp_parser.h

Path = src/service_inspectors/gtp/gtp_parser.h

#ifndef GTP_PARSER_H
#define GTP_PARSER_H

#include <cstdint>

struct GTP_IEData
{
    uint64_t msg_id;  /* used to associate to current msg */
    uint16_t length;
    uint16_t shift;   /* shift relative to the header*/
};

struct GTPMsg
{
    const uint8_t* gtp_header;
    GTP_IEData* info_elements;

    uint16_t msg_length;
    uint16_t header_len;

    uint8_t version;
    uint8_t msg_type;

    /* nothing after this point is zeroed ...*/
    uint64_t msg_id; /*internal state, new msg will have a new id*/
};

int gtp_parse(const struct GTPConfig&, struct GTPMsg*, const uint8_t*, uint16_t);
void gtp_cleanInfoElements();

#endif

service_inspectors/http2_inspect/

  • HTTP/2 frame processing overview * The HTTP/2 inspector (H2I) converts HTTP/2 frames into HTTP/1.1 message sections and feeds them to the new HTTP inspector (NHI) for further processing.

The Http2StreamSplitter splits HTTP/2 traffic on frame boundaries with two exceptions. First, any Continuation frames following a Headers frame are aggregated and sent through detection as a single header block. Second, long Data frames are split into 16kb chunks for inspection as message bodies are in NHI. The Http2StreamSplitter strips the frame headers from the frame data and stores them in separate buffers. In the case of a Headers frame followed by Continuation frames, the Headers frame header is stored and the Continuation frame headers are discarded.

  • HTTP/2 Frame Classes * HTTP/2 frames are stored in frame objects for processing.

    1. Http2Frame - top level class

    2. Http2HeadersFrame : Http2Frame - common elements of frames containing headers

    3. Htt2HeadersFrameWithStartline : Http2HeadersFrame - common elements of headers with start lines

    4. Http2HeadersFrameHeader : Htt2HeadersFrameWithStartline - regular request/response headers

    5. Http2PushPromiseFrame : Htt2HeadersFrameWithStartline - server push headers

    6. Http2HeadersFrameTrailer : Http2HeadersFrame - headers frame containing trailers

    7. Http2DataFrame : Http2Frame

    8. Http2SettingsFrame : Http2Frame

    9. Http2RstStreamFrame : Http2Frame

    10. Http2WindowUpdateFrame: Http2Frame

    11. Http2PriorityFrame: Http2Frame

    12. Http2GoAwayFrame: Http2Frame

Http2Frame virtual methods: 1. bool valid_sequence(Http2Enums::StreamState state) - Returns if the current frame has been sent on a valid stream in a valid state for the frame. 2. void analyze_http1() - Contains the interface with http_inspect if necessary for the frame type 3. void clear() - Cleanup after processing is complete 4. const Field& get_buf(unsigned id) - Returns any buffers associated with this frame type 5. void update_stream_state() - Updates the state of the current stream based on the contents of the current packet. 6. bool is_detection_required() - Returns if the data in this frame be sent through the detection engine.

  • HTTP/2 Stream States * Http2 stream states are directional and take the form {<C2S state>, <S2C state>}. States can have the following values:

    1. STREAM_EXPECT_HEADERS - No frames have yet been seen on this stream in this direction. A stream with state {STREAM_EXPECT_HEADERS, STREAM_EXPECT_HEADERS} is what the HTTP/2 RFC refers to as "idle."

    2. STREAM_EXPECT_BODY - Headers are complete and we are expecting to see a Data frame

    3. STREAM_BODY - There has been at least one Data frame in this direction already, but we have not yet seen the END_STREAM flag so are still expecting more Data or Headers frames.

    4. STREAM_COMPLETE - The END_STREAM flag has been seen. No more Data or Headers frames are expected.

    5. STREAM_ERROR - An error has occurred processing this stream. This is a terminal stream state. Further frames on this stream in this direction may be checked for proper formatting, but will not impact the stream state or interact with http_inspect. For information on error processing see the section below.

  • HTTP/2 Header Decoding * HTTP/2 headers frames can come at the start of a stream and contain the header block, or at the end of a stream and contain trailers. H2I contains headers frame subclasses Http2HeadersFrameHeader and Http2HeadersFrameTrailer to support these two types of headers frames. Http2PushPromise frames begin server pushed streams and also contain the header block. Headers frames containing the header block will contain pseudo-headers that must be converted into an HTTP/1.1 start line in addition to regular HTTP/1.1 headers. The two Http2StartLine subclasses, Http2RequestLine and Http2ResponseLine perform this translation and generate the start-line, which is stored in a new buffer inside the Http2StartLine object. Trailers may only contain regular headers.

Both headers and trailers must undergo HPACK decoding before being sent to NHI for processing. To perform decoding, reassemble() makes a first copy of the encoded headers, which is stored in the frame_data buffer. The frame_data buffer is passed to the function decode_headers(), which is the main loop driving HPACK decoding. Each decoded header line is progressively written to a second decoded_headers buffer that will ultimately be sent to NHI.

The main loop in decode_headers() finds the cut point for a single header line. The line is passed to decode_header_line(), which parses the line and calls the appropriate decoding function based on the header representation type. If the type is indexed, the full header line is looked up in the table and copied to the decoded header buffer. The index may belong to either the static or the dynamic table. The static table is 61-elements defined in the HPACK RFC. The dynamic table, which starts at index 62, is specific to each direction of each flow. For the second type, literal to be indexed, the header name may be indexed or a string literal, while the value is always a literal. The resulting header line is then added to the dynamic table. The third representation type is literal not to be indexed, which is the same as literal to be indexed, except the header line is not added to the dynamic table.

  • Error Processing * H2I has two levels of failure for flow processing. Fatal errors include failures in frame splitting and errors in header decoding that compromise the HPACK dictionary. A fatal error will trigger an immediate EVENT_MISFORMATTED_HTTP2 and will cause scan() to return ABORT the next time it is called in the same direction.

Stream errors affect HTTP/1 processing within a stream. They include errors in frame sequence within a stream, errors that cause the HI stream splitter to abort, and errors that cause HI to be unable to process a frame. The individual stream will transition to STREAM_ERROR state and HI processing of that stream in that direction will end.

H2I supports the NHI test tool. See ../http_inspect/dev_notes.txt for usage instructions.

  • Memory requirements * Http2FlowData represents all H2I information in a flow. It does not account for the entries in the hpack dynamic table. The formula below estimates the size of an entry in the dynamic table: name.length() + value.length() + RFC_ENTRY_OVERHEAD (32 as defined by the RFC)

Using the formula and some sample pcaps, the average size of the dynamic table is 1645 bytes. Dynamically allocated objects related to http_inspect are considered separate and are not included. Temporary objects (frame_data and frame_header) are ignored. The remaining dynamically allocated are Http2Infractions (8 bytes * 2) and Http2EventsGen(24 bytes * 2) Therefore, the memory required by http2 per flow: sizeof(Http2FlowData) + 1645 + 16 + 48

http2_api.h

Path = src/service_inspectors/http2_inspect/http2_api.h

#ifndef HTTP2_API_H
#define HTTP2_API_H

#include "framework/inspector.h"
#include "framework/module.h"

#include "http2_flow_data.h"
#include "http2_module.h"

class Http2Api
{
public:
    static const snort::InspectApi http2_api;
    static const char* classic_buffer_names[];

private:
    Http2Api() = delete;
    static snort::Module* http2_mod_ctor() { return new Http2Module; }
    static void http2_mod_dtor(snort::Module* m) { delete m; }
    static const char* http2_my_name;
    static const char* http2_help;
    static void http2_init() { Http2FlowData::init(); }
    static void http2_term() { }
    static snort::Inspector* http2_ctor(snort::Module* mod);
    static void http2_dtor(snort::Inspector* p) { delete p; }
};

#endif

http2_data_cutter.h

Path = src/service_inspectors/http2_inspect/http2_data_cutter.h

#ifndef HTTP2_DATA_CUTTER_H
#define HTTP2_DATA_CUTTER_H

#include "service_inspectors/http_inspect/http_common.h"
#include "stream/stream_splitter.h"

#include "http2_enum.h"

class Http2FlowData;
class Http2Stream;

class Http2DataCutter
{
public:
    Http2DataCutter(Http2FlowData* flow_data, HttpCommon::SourceId src_id);
    snort::StreamSplitter::Status scan(const uint8_t* data, uint32_t length,
        uint32_t* flush_offset, uint32_t& data_offset, uint8_t frame_flags);
    void reassemble(const uint8_t* data, unsigned len);
    void discarded_frame_cleanup(Http2Stream* const stream);

private:
    bool check_http_state(Http2Stream* const stream);
    snort::StreamSplitter::Status skip_over_frame(Http2Stream* const stream, uint32_t length,
        uint32_t* flush_offset, uint32_t data_offset, uint8_t frame_flags);

    Http2FlowData* const session_data;
    const HttpCommon::SourceId source_id;

    // total per frame - scan
    uint32_t data_len = 0;
    // accumulating - scan
    uint32_t frame_bytes_seen = 0;
    uint32_t bytes_sent_http = 0;
    uint32_t data_bytes_read = 0;
    // total per frame - reassemble
    uint32_t reassemble_data_len = 0;
    // accumulating - reassemble
    uint32_t reassemble_bytes_sent = 0;
    uint32_t reassemble_hdr_bytes_read = 0;
    uint32_t reassemble_data_bytes_read = 0;

    // reassemble
    enum ReassembleState { GET_FRAME_HDR, GET_PADDING_LEN, SEND_EMPTY_DATA, SEND_DATA };
    enum ReassembleState reassemble_state = GET_FRAME_HDR;
};

#endif

http2_data_frame.h

Path = src/service_inspectors/http2_inspect/http2_data_frame.h

#ifndef HTTP2_DATA_FRAME_H
#define HTTP2_DATA_FRAME_H

#include "http2_frame.h"

class Http2Frame;
class Http2Stream;

class Http2DataFrame : public Http2Frame
{
public:
    ~Http2DataFrame() override = default;
    bool valid_sequence(Http2Enums::StreamState state) override;
    void analyze_http1(snort::Packet*) override;
    void clear(snort::Packet*) override;

    bool is_detection_required() const override { return false; }
    void update_stream_state() override;
    virtual const uint8_t* get_frame_data(uint32_t& length) const override;

    friend Http2Frame* Http2Frame::new_frame(const uint8_t*, const uint32_t, const uint8_t*,
        const uint32_t, Http2FlowData*, HttpCommon::SourceId, Http2Stream* stream);

#ifdef REG_TEST
    void print_frame(FILE* output) override;
#endif

private:
    Http2DataFrame(const uint8_t* header_buffer, const uint32_t header_len,
        const uint8_t* data_buffer_, const uint32_t data_length_, Http2FlowData* ssn_data,
        HttpCommon::SourceId src_id, Http2Stream* stream);
    uint8_t get_flags_mask() const override;

    const uint32_t data_length;
    const uint8_t* const data_buffer;
};
#endif

http2_enum.h

Path = src/service_inspectors/http2_inspect/http2_enum.h

#ifndef HTTP2_ENUM_H
#define HTTP2_ENUM_H

#include <cstdint>

namespace Http2Enums
{
static const int MAX_OCTETS = 63780;
static const int DATA_SECTION_SIZE = 16384;
static const int FRAME_HEADER_LENGTH = 9;
static const uint32_t NO_STREAM_ID = 0xFFFFFFFF;

// Perform memory allocation and deallocation tracking for Http2Stream objects in increments of 25
static const uint32_t STREAM_MEMORY_TRACKING_INCREMENT = 25;

static const uint32_t HTTP2_GID = 121;

// Frame type codes (fourth octet of frame header)
enum FrameType : uint8_t { FT_DATA=0, FT_HEADERS=1, FT_PRIORITY=2, FT_RST_STREAM=3, FT_SETTINGS=4,
    FT_PUSH_PROMISE=5, FT_PING=6, FT_GOAWAY=7, FT_WINDOW_UPDATE=8, FT_CONTINUATION=9,
    FT__NONE=255 };

// Ordered from initial state to terminal state. Do not rearrange without careful consideration.
enum StreamState { STREAM_EXPECT_HEADERS, STREAM_EXPECT_BODY, STREAM_BODY, STREAM_COMPLETE,
    STREAM_ERROR };

// Message buffers available to clients
// This enum must remain synchronized with Http2Api::classic_buffer_names[]
enum HTTP2_BUFFER { HTTP2_BUFFER_FRAME_HEADER = 1, HTTP2_BUFFER_DECODED_HEADER, HTTP2_BUFFER__MAX };

// Peg counts
// This enum must remain synchronized with Http2Module::peg_names[] in http2_tables.cc
enum PEG_COUNT { PEG_FLOW = 0, PEG_CONCURRENT_SESSIONS, PEG_MAX_CONCURRENT_SESSIONS,
    PEG_MAX_TABLE_ENTRIES, PEG_MAX_CONCURRENT_FILES, PEG_TOTAL_BYTES, PEG_MAX_CONCURRENT_STREAMS,
    PEG_FLOWS_OVER_STREAM_LIMIT, PEG_COUNT__MAX };

enum EventSid
{
    EVENT__NONE = -1,
    EVENT_INVALID_FLAG = 1,
    EVENT_INT_LEADING_ZEROS = 2,
    EVENT_INVALID_STREAM_ID = 3,
    EVENT_MISSING_CONTINUATION = 4,
    EVENT_UNEXPECTED_CONTINUATION = 5,
    EVENT_MISFORMATTED_HTTP2 = 6,
    EVENT_PREFACE_MATCH_FAILURE = 7,
    EVENT_REQUEST_WITHOUT_REQUIRED_FIELD = 8,
    EVENT_RESPONSE_WITHOUT_STATUS = 9,
    EVENT_CONNECT_WITH_SCHEME_OR_PATH = 10,
    EVENT_SETTINGS_FRAME_ERROR = 11,
    EVENT_SETTINGS_FRAME_UNKN_PARAM = 12,
    EVENT_FRAME_SEQUENCE = 13,
    EVENT_DYNAMIC_TABLE_OVERFLOW = 14,
    EVENT_INVALID_PROMISED_STREAM = 15,
    EVENT_PADDING_LEN = 16,
    EVENT_PSEUDO_HEADER_AFTER_REGULAR_HEADER = 17,
    EVENT_PSEUDO_HEADER_IN_TRAILERS = 18,
    EVENT_INVALID_PSEUDO_HEADER = 19,
    EVENT_TRAILERS_NOT_END = 20,
    EVENT_PUSH_WHEN_PROHIBITED = 21,
    EVENT_PADDING_ON_EMPTY_FRAME = 22,
    EVENT_C2S_PUSH = 23,
    EVENT_INVALID_PUSH_FRAME = 24,
    EVENT_BAD_PUSH_SEQUENCE = 25,
    EVENT_BAD_SETTINGS_VALUE = 26,
    EVENT_TOO_MANY_STREAMS = 27,
    EVENT_INVALID_RST_STREAM_FRAME = 28,
    EVENT_BAD_RST_STREAM_SEQUENCE = 29,
    EVENT_HEADER_UPPERCASE = 30,
    EVENT_INVALID_WINDOW_UPDATE_FRAME = 31,
    EVENT_WINDOW_UPDATE_FRAME_ZERO_INCREMENT = 32,
    EVENT_REQUEST_WITHOUT_METHOD = 33,
    EVENT_TABLE_SIZE_UPDATE_NOT_AT_HEADER_START = 34,
    EVENT_MORE_THAN_2_TABLE_SIZE_UPDATES = 35,
    EVENT_HPACK_TABLE_SIZE_UPDATE_EXCEEDS_MAX = 36,
    EVENT_UNEXPECTED_DATA_FRAME = 37,
    EVENT_NON_DATA_FRAME_TOO_LONG = 38,
    EVENT_LOSS_OF_SYNC = 39,
    EVENT_INVALID_PRIORITY_FRAME = 40,
    EVENT_INVALID_GOAWAY_FRAME = 41,
    EVENT_SETTINGS_QUEUE_OVERFLOW = 42,
    EVENT_SETTINGS_QUEUE_UNDERFLOW = 43,
    EVENT__MAX_VALUE
};

// All the infractions we might find while parsing and analyzing a message
enum Infraction
{
    INF__NONE = -1,
    INF_INT_EMPTY_BUFF = 0,
    INF_INT_MISSING_BYTES = 1,
    INF_INT_OVERFLOW = 2,
    INF_INT_LEADING_ZEROS = 3,
    INF_STRING_EMPTY_BUFF = 4,
    INF_STRING_MISSING_BYTES = 5,
    INF_DECODED_HEADER_BUFF_OUT_OF_SPACE = 6,
    INF_HUFFMAN_BAD_PADDING = 7,
    INF_HUFFMAN_DECODED_EOS = 8,
    INF_HUFFMAN_INCOMPLETE_CODE_PADDING = 9,
    INF_MISSING_CONTINUATION = 10,
    INF_UNEXPECTED_CONTINUATION = 11,
    INF_C2S_PUSH = 12,
    INF_INVALID_PSEUDO_HEADER = 13,
    INF_PSEUDO_HEADER_AFTER_REGULAR_HEADER = 14,
    INF_REQUEST_WITHOUT_REQUIRED_FIELD = 15,
    INF_RESPONSE_WITHOUT_STATUS = 16,
    INF_HPACK_INDEX_OUT_OF_BOUNDS = 17,
    INF_INVALID_SETTINGS_FRAME = 18,
    INF_SETTINGS_FRAME_UNKN_PARAM = 19,
    INF_FRAME_SEQUENCE = 20,
    INF_HPACK_TABLE_SIZE_UPDATE_EXCEEDS_MAX = 21,
    INF_DYNAMIC_TABLE_OVERFLOW = 22,
    INF_TABLE_SIZE_UPDATE_NOT_AT_HEADER_START = 23,
    INF_MORE_THAN_2_TABLE_SIZE_UPDATES = 24,
    INF_INVALID_STARTLINE = 25,
    INF_INVALID_HEADER = 26,
    INF_PADDING_LEN = 27,
    INF_PUSH_FRAME_TOO_SHORT = 28,
    INF_PSEUDO_HEADER_IN_TRAILERS = 29,
    INF_TRAILERS_NOT_END = 30,
    INF_CONNECT_WITH_SCHEME_OR_PATH = 31,
    INF_PADDING_ON_EMPTY_FRAME = 32,
    INF_BAD_PUSH_SEQUENCE = 33,
    INF_BAD_SETTINGS_PUSH_VALUE = 34,
    INF_PUSH_WHEN_PROHIBITED = 35,
    INF_INVALID_PROMISED_STREAM = 36,
    INF_INVALID_STREAM_ID = 37,
    INF_INVALID_FLAG = 38,
    INF_TRUNCATED_HEADER_LINE = 39,
    INF_REQUEST_WITHOUT_METHOD = 40,
    INF_CONNECT_WITHOUT_AUTHORITY = 41,
    INF_TOO_MANY_STREAMS = 42,
    INF_INVALID_RST_STREAM_FRAME = 43,
    INF_BAD_RST_STREAM_SEQUENCE = 44,
    INF_HEADER_UPPERCASE = 45,
    INF_INVALID_WINDOW_UPDATE_FRAME = 46,
    INF_WINDOW_UPDATE_FRAME_ZERO_INCREMENT = 47,
    INF_UNEXPECTED_DATA_FRAME = 48,
    INF_NON_DATA_FRAME_TOO_LONG = 49,
    INF_BAD_PRIORITY_FRAME_STREAM_ID = 50,
    INF_BAD_PRIORITY_FRAME_LENGTH = 51,
    INF_BAD_GOAWAY_FRAME_STREAM_ID = 52,
    INF_BAD_GOAWAY_FRAME_LENGTH = 53,
    INF_BAD_GOAWAY_FRAME_R_BIT = 54,
    INF_SETTINGS_QUEUE_OVERFLOW = 55,
    INF_SETTINGS_QUEUE_UNDERFLOW = 56,
    INF__MAX_VALUE
};

enum HeaderFrameFlags
{
    FLAG_ACK = 0x1,
    FLAG_END_STREAM = 0x1,
    FLAG_END_HEADERS = 0x4,
    FLAG_PADDED = 0x8,
    FLAG_PRIORITY = 0x20,
    FLAG_NO_HEADER = 0x80, //No valid flags use this bit
};

enum SettingsFrameIds
{
    SFID_HEADER_TABLE_SIZE = 1,
    SFID_ENABLE_PUSH,
    SFID_MAX_CONCURRENT_STREAMS,
    SFID_INITIAL_WINDOW_SIZE,
    SFID_MAX_FRAME_SIZE,
    SFID_MAX_HEADER_LIST_SIZE,
};

enum ScanState { SCAN_FRAME_HEADER, SCAN_PADDING_LENGTH, SCAN_DATA, SCAN_EMPTY_DATA };
} // end namespace Http2Enums

#endif

http2_flow_data.h

Path = src/service_inspectors/http2_inspect/http2_flow_data.h

#ifndef HTTP2_FLOW_DATA_H
#define HTTP2_FLOW_DATA_H

#include <queue>
#include <vector>

#include "main/snort_types.h"
#include "utils/event_gen.h"
#include "utils/infractions.h"
#include "flow/flow.h"
#include "service_inspectors/http_inspect/http_common.h"
#include "service_inspectors/http_inspect/http_field.h"
#include "stream/stream_splitter.h"

#include "http2_data_cutter.h"
#include "http2_enum.h"
#include "http2_hpack.h"
#include "http2_hpack_int_decode.h"
#include "http2_hpack_string_decode.h"
#include "http2_settings_frame.h"
#include "http2_stream.h"

using Http2Infractions = Infractions<Http2Enums::INF__MAX_VALUE, Http2Enums::INF__NONE>;

using Http2EventGen = EventGen<Http2Enums::EVENT__MAX_VALUE, Http2Enums::EVENT__NONE,
    Http2Enums::HTTP2_GID>;

class HttpFlowData;
class HttpMsgSection;
class HttpInspect;
class HttpStreamSplitter;

class SO_PUBLIC Http2FlowData : public snort::FlowData
{
public:
    Http2FlowData(snort::Flow* flow_);
    ~Http2FlowData() override;
    static unsigned inspector_id;
    static void init() { inspector_id = snort::FlowData::create_flow_data_id(); }

    // Used by http_inspect to store its stuff
    HttpFlowData* get_hi_flow_data();
    void set_hi_flow_data(HttpFlowData* flow);
    HttpMsgSection* get_hi_msg_section() const { return hi_msg_section; }
    void set_hi_msg_section(HttpMsgSection* section)
        { assert((hi_msg_section == nullptr) || (section == nullptr)); hi_msg_section = section; }

    friend class Http2Frame;
    friend class Http2DataFrame;
    friend class Http2DataCutter;
    friend class Http2GoAwayFrame;
    friend class Http2HeadersFrame;
    friend class Http2HeadersFrameHeader;
    friend class Http2HeadersFrameTrailer;
    friend class Http2HeadersFrameWithStartline;
    friend class Http2Hpack;
    friend class Http2Inspect;
    friend class Http2PriorityFrame;
    friend class Http2PushPromiseFrame;
    friend class Http2RequestLine;
    friend class Http2RstStreamFrame;
    friend class Http2SettingsFrame;
    friend class Http2StartLine;
    friend class Http2StatusLine;
    friend class Http2Stream;
    friend class Http2StreamSplitter;
    friend class Http2WindowUpdateFrame;
    friend void finish_msg_body(Http2FlowData* session_data, HttpCommon::SourceId source_id);

    Http2Stream* find_current_stream(const HttpCommon::SourceId source_id);
    uint32_t get_current_stream_id(const HttpCommon::SourceId source_id) const;
    Http2Stream* get_processing_stream(const HttpCommon::SourceId source_id, uint32_t concurrent_streams_limit);
    Http2Stream* find_processing_stream();
    uint32_t get_processing_stream_id() const;
    void set_processing_stream_id(const HttpCommon::SourceId source_id);
    bool is_processing_partial_header() const { return processing_partial_header; }

    Http2HpackDecoder* get_hpack_decoder(const HttpCommon::SourceId source_id)
    { return &hpack_decoder[source_id]; }
    Http2ConnectionSettings* get_my_connection_settings(const HttpCommon::SourceId source_id)
    { return &connection_settings[source_id]; }
    Http2ConnectionSettings* get_remote_connection_settings(const HttpCommon::SourceId source_id)
    { return &connection_settings[1 - source_id]; }

    // Used by payload injection to determine whether we are at a safe place to insert our own
    // frame into the S2C direction of an HTTP/2 flow.
    bool is_mid_frame() const;

    // Used by payload injection to determine whether we should inject S2C settings frame
    // before injecting payload
    bool was_server_settings_received() const
    { return server_settings_frame_received; }

    void set_server_settings_received()
    { server_settings_frame_received = true; }

#ifdef UNIT_TEST
    void set_mid_frame(bool); // Not implemented outside of unit tests
#endif

protected:
    snort::Flow* flow;
    HttpInspect* const hi;
    HttpStreamSplitter* hi_ss[2] = { nullptr, nullptr };

    // 0 element refers to client frame, 1 element refers to server frame

    // There are currently one infraction and one event object per flow per direction.
    Http2Infractions* const infractions[2] = { new Http2Infractions, new Http2Infractions };
    Http2EventGen* const events[2] = { new Http2EventGen, new Http2EventGen };

    // Stream ID of the frame currently being processed was sent on (i.e. the stream in the frame
    // header). This is set in scan().
    uint32_t current_stream[2] = { Http2Enums::NO_STREAM_ID, Http2Enums::NO_STREAM_ID };
    // Stream ID of the stream responsible for processing the current frame. This will be the same
    // as current_stream except when processing a push_promise frame. This is set in eval() and
    // cleared in clear().
    uint32_t processing_stream_id = Http2Enums::NO_STREAM_ID;
    // At any given time there may be different streams going in each direction. But only one of
    // them is the stream that http_inspect is actually processing at the moment.
    uint32_t stream_in_hi = Http2Enums::NO_STREAM_ID;
    HttpMsgSection* hi_msg_section = nullptr;
    bool server_settings_frame_received = false;
    bool tcp_close[2] = { false, false };

    // Reassemble() data to eval()
    uint8_t lead_frame_header[2][Http2Enums::FRAME_HEADER_LENGTH];
    const uint8_t* frame_data[2] = { nullptr, nullptr };
    uint32_t frame_data_size[2] = { 0, 0 };

    // Used in eval()
    Http2ConnectionSettings connection_settings[2];
    Http2ConnectionSettingsQueue settings_queue[2];
    Http2HpackDecoder hpack_decoder[2];
    std::list<Http2Stream> streams;
    uint32_t concurrent_files = 0;
    uint32_t concurrent_streams = 0;
    uint32_t stream_memory_allocations_tracked = Http2Enums::STREAM_MEMORY_TRACKING_INCREMENT;
    uint32_t max_stream_id[2] = {0, 0};
    bool frame_in_detection = false;
    bool delete_stream = false;

    // Internal to scan()
    bool preface[2] = { true, false };
    uint32_t preface_octets_seen = 0;
    bool continuation_expected[2] = { false, false };
    uint8_t scan_frame_header[2][Http2Enums::FRAME_HEADER_LENGTH];
    uint32_t scan_remaining_frame_octets[2] = { 0, 0 };
    uint32_t header_octets_seen[2] = { 0, 0 };
    uint8_t padding_length[2] = { 0, 0 };
    uint8_t remaining_data_padding[2] = { 0, 0 };
    Http2Enums::ScanState scan_state[2] =
        { Http2Enums::SCAN_FRAME_HEADER, Http2Enums::SCAN_FRAME_HEADER };

    // Used by scan() and reassemble()
    Http2DataCutter data_cutter[2];

    // Scan signals to reassemble()
    uint32_t bytes_scanned[2] = { 0, 0 };
    bool payload_discard[2] = { false, false };

    // Used by scan, reassemble and eval to communicate
    uint8_t frame_type[2] = { Http2Enums::FT__NONE, Http2Enums::FT__NONE };
    bool abort_flow[2] = { false, false };
    bool processing_partial_header = false;
    std::queue<uint32_t> frame_lengths[2];
    uint32_t accumulated_frame_length[2] = { 0, 0 };

    // Internal to reassemble()
    uint32_t frame_header_offset[2] = { 0, 0 };
    uint32_t frame_data_offset[2] = { 0, 0 };
    uint32_t remaining_frame_octets[2] = { 0, 0 };
    uint32_t running_total[2] = { 0, 0 };
    uint8_t remaining_padding_reassemble[2] = { 0, 0 };
    bool read_frame_header[2] = { false, false };
    bool continuation_frame[2] = { false, false };
    bool read_padding_len[2] = { false, false };
    uint8_t* frame_reassemble[2] = { nullptr, nullptr };

#ifdef REG_TEST
    static uint64_t instance_count;
    uint64_t seq_num;
#endif

private:
    Http2Stream* get_hi_stream();
    Http2Stream* find_stream(const uint32_t key);
    void delete_processing_stream();
};

class Http2FlowStreamIntf : public snort::StreamFlowIntf
{
public:
    snort::FlowData* get_stream_flow_data(const snort::Flow* flow) override;
    void set_stream_flow_data(snort::Flow* flow, snort::FlowData* flow_data) override;
    void get_stream_id(const snort::Flow* flow, int64_t& stream_id) override;
    AppId get_appid_from_stream(const snort::Flow* flow) override;
    void* get_hi_msg_section(const snort::Flow* flow) override;
    void set_hi_msg_section(snort::Flow* flow, void* section) override;
};

#endif

http2_frame.h

Path = src/service_inspectors/http2_inspect/http2_frame.h

#ifndef HTTP2_FRAME_H
#define HTTP2_FRAME_H

#include "protocols/packet.h"
#include "service_inspectors/http_inspect/http_common.h"
#include "service_inspectors/http_inspect/http_field.h"

#include "http2_enum.h"

/* This class is called Http2Frame, but an object of this class may not represent exactly one
 * HTTP/2 frame as received on the wire. For HEADERS frames, the Http2Frame object contains the
 * initial HEADERS frame plus any following CONTINUATION frames grouped together. For DATA frames,
 * the Http2Frame object represents approximately 16kb of data to be inspected. This may consist
 * of part of a larger DATA frame cut into 16kb-sized pieces, or several smaller DATA frames
 * aggregated together.
 */

class Http2FlowData;
class Http2Stream;

class Http2Frame
{
public:
    virtual ~Http2Frame() = default;
    static Http2Frame* new_frame(const uint8_t* header_buffer, const uint32_t header_len,
        const uint8_t* data_buffer, const uint32_t data_len, Http2FlowData* session_data,
        HttpCommon::SourceId source_id, Http2Stream* stream);
    virtual bool valid_sequence(Http2Enums::StreamState) { return true; }
    virtual void analyze_http1(snort::Packet*) { }
    virtual void clear(snort::Packet*) { }
    virtual const Field& get_buf(unsigned id);
    virtual bool is_detection_required() const { return true; }
    virtual void update_stream_state() { }
    const uint8_t* get_frame_pdu(uint16_t& length) const;
    virtual const uint8_t* get_frame_data(uint32_t& length) const;
#ifdef REG_TEST
    virtual void print_frame(FILE* output);
#endif

protected:
    Http2Frame(const uint8_t* header_buffer, const uint32_t header_len,
        const uint8_t* data_buffer, const uint32_t data_len, Http2FlowData* session_data,
        HttpCommon::SourceId source_id, Http2Stream* stream);
    uint8_t get_flags();
    uint32_t get_stream_id();
    virtual uint8_t get_flags_mask() const { return 0; }

    Field header;
    Field data;
    Http2FlowData* session_data;
    HttpCommon::SourceId source_id;
    Http2Stream* stream;

    const static uint8_t flags_index = 4;
    const static uint8_t stream_id_index = 5;
    const static uint32_t INVALID_STREAM_ID = 0xFFFFFFFF;
};
#endif

http2_goaway_frame.h

Path = src/service_inspectors/http2_inspect/http2_goaway_frame.h

#ifndef HTTP2_GOAWAY_FRAME_H
#define HTTP2_GOAWAY_FRAME_H

#include "http2_flow_data.h"

class Http2Frame;

class Http2GoAwayFrame : public Http2Frame
{
public:
    friend Http2Frame* Http2Frame::new_frame(const uint8_t*, const uint32_t, const uint8_t*,
        const uint32_t, Http2FlowData*, HttpCommon::SourceId, Http2Stream* stream);
    bool is_detection_required() const override { return false; }

private:
    Http2GoAwayFrame(const uint8_t* header_buffer, const uint32_t header_len,
        const uint8_t* data_buffer, const uint32_t data_len, Http2FlowData* ssn_data,
        HttpCommon::SourceId src_id, Http2Stream* _stream);
};
#endif

http2_headers_frame.h

Path = src/service_inspectors/http2_inspect/http2_headers_frame.h

#ifndef HTTP2_HEADERS_FRAME_H
#define HTTP2_HEADERS_FRAME_H

#include "http2_frame.h"

class Field;
class Http2HpackDecoder;
class Http2StartLine;
class Http2Frame;
class Http2Stream;
class HttpFlowData;

class Http2HeadersFrame : public Http2Frame
{
public:
    void clear(snort::Packet*) override;

    const Field& get_buf(unsigned id) override;
    bool is_detection_required() const override { return false; }

#ifdef REG_TEST
    void print_frame(FILE* output) override;
#endif

protected:
    Http2HeadersFrame(const uint8_t* header_buffer, const uint32_t header_len,
        const uint8_t* data_buffer, const uint32_t data_len, Http2FlowData* ssn_data,
        HttpCommon::SourceId src_id, Http2Stream* stream);
    bool decode_headers(Http2StartLine* start_line_generator, bool trailers);
    void process_decoded_headers(HttpFlowData* http_flow, HttpCommon::SourceId hi_source_id, snort::Packet* p);
    uint8_t get_flags_mask() const override;
    virtual bool in_error_state() const;

    Field http1_header;                 // finalized headers to be passed to http_inspect
    Http2HpackDecoder* hpack_decoder;
    uint8_t hpack_headers_offset = 0;
};
#endif

http2_headers_frame_header.h

Path = src/service_inspectors/http2_inspect/http2_headers_frame_header.h

#ifndef HTTP2_HEADERS_FRAME_HEADER_H
#define HTTP2_HEADERS_FRAME_HEADER_H

#include "http2_frame.h"
#include "http2_headers_frame_with_startline.h"

class Http2StartLine;

class Http2HeadersFrameHeader : public Http2HeadersFrameWithStartline
{
public:
    friend Http2Frame* Http2Frame::new_frame(const uint8_t*, const uint32_t, const uint8_t*,
        const uint32_t, Http2FlowData*, HttpCommon::SourceId, Http2Stream* stream);

    bool valid_sequence(Http2Enums::StreamState state) override;
    void analyze_http1(snort::Packet*) override;
    void update_stream_state() override;

#ifdef REG_TEST
    void print_frame(FILE* output) override;
#endif

private:
    Http2HeadersFrameHeader(const uint8_t* header_buffer, const uint32_t header_len,
        const uint8_t* data_buffer, const uint32_t data_len, Http2FlowData* ssn_data,
        HttpCommon::SourceId src_id, Http2Stream* stream);
};
#endif

http2_headers_frame_trailer.h

Path = src/service_inspectors/http2_inspect/http2_headers_frame_trailer.h

#ifndef HTTP2_HEADERS_FRAME_TRAILER_H
#define HTTP2_HEADERS_FRAME_TRAILER_H

#include "http2_frame.h"
#include "http2_headers_frame.h"

class Http2HeadersFrameTrailer : public Http2HeadersFrame
{
public:
    friend Http2Frame* Http2Frame::new_frame(const uint8_t*, const uint32_t, const uint8_t*,
        const uint32_t, Http2FlowData*, HttpCommon::SourceId, Http2Stream* stream);

    bool valid_sequence(Http2Enums::StreamState state) override;
    void analyze_http1(snort::Packet*) override;
    void update_stream_state() override;

#ifdef REG_TEST
    void print_frame(FILE* output) override;
#endif

private:
    Http2HeadersFrameTrailer(const uint8_t* header_buffer, const uint32_t header_len,
        const uint8_t* data_buffer, const uint32_t data_len, Http2FlowData* ssn_data,
        HttpCommon::SourceId src_id, Http2Stream* stream);
};
#endif

http2_headers_frame_with_startline.h

Path = src/service_inspectors/http2_inspect/http2_headers_frame_with_startline.h

#ifndef HTTP2_HEADERS_FRAME_WITH_STARTLINE_H
#define HTTP2_HEADERS_FRAME_WITH_STARTLINE_H

#include "service_inspectors/http_inspect/http_common.h"

#include "http2_enum.h"
#include "http2_frame.h"
#include "http2_headers_frame.h"

class Field;
class Http2Frame;
class Http2Stream;
class HttpFlowData;

class Http2HeadersFrameWithStartline : public Http2HeadersFrame
{
public:
    ~Http2HeadersFrameWithStartline() override;

#ifdef REG_TEST
    void print_frame(FILE* output) override;
#endif

protected:
    Http2HeadersFrameWithStartline(const uint8_t* header_buffer, const uint32_t header_len,
        const uint8_t* data_buffer, const uint32_t data_len, Http2FlowData* ssn_data,
        HttpCommon::SourceId src_id, Http2Stream* stream_) :
        Http2HeadersFrame(header_buffer, header_len, data_buffer, data_len, ssn_data, src_id,
            stream_) { }
    bool process_start_line(HttpFlowData*& http_flow, HttpCommon::SourceId hi_source_id, snort::Packet* p);
    bool are_pseudo_headers_complete();

    Http2StartLine* start_line_generator = nullptr;
    Field start_line;
};
#endif

http2_hpack_dynamic_table.h

Path = src/service_inspectors/http2_inspect/http2_hpack_dynamic_table.h

#ifndef HTTP2_HPACK_DYNAMIC_TABLE_H
#define HTTP2_HPACK_DYNAMIC_TABLE_H

#include "service_inspectors/http_inspect/http_field.h"
#include "main/snort_types.h"

#include "http2_enum.h"

#include <vector>

struct HpackTableEntry;
class Http2FlowData;

class HpackDynamicTable
{
public:
    // FIXIT-P This array can be optimized to start smaller and grow on demand
    HpackDynamicTable() : circular_buf(ARRAY_CAPACITY, nullptr) {}
    ~HpackDynamicTable();
    const HpackTableEntry* get_entry(uint32_t index) const;
    bool add_entry(const Field& name, const Field& value);
    void update_size(uint32_t new_size);
    uint32_t get_max_size() { return max_size; }

private:
    const static uint32_t RFC_ENTRY_OVERHEAD = 32;

    const static uint32_t DEFAULT_MAX_SIZE = 4096;
    const static uint32_t ARRAY_CAPACITY = 512;
    uint32_t max_size = DEFAULT_MAX_SIZE;

    uint32_t start = 0;
    uint32_t num_entries = 0;
    uint32_t rfc_table_size = 0;
    std::vector<HpackTableEntry*> circular_buf;

    void prune_to_size(uint32_t new_max_size);
};
#endif

http2_hpack.h

Path = src/service_inspectors/http2_inspect/http2_hpack.h

#ifndef HTTP2_HPACK_H
#define HTTP2_HPACK_H

#include "service_inspectors/http_inspect/http_common.h"
#include "utils/event_gen.h"
#include "utils/infractions.h"

#include "http2_hpack_int_decode.h"
#include "http2_hpack_string_decode.h"
#include "http2_hpack_table.h"

class Field;
class Http2FlowData;
class Http2StartLine;

using Http2Infractions = Infractions<Http2Enums::INF__MAX_VALUE, Http2Enums::INF__NONE>;
using Http2EventGen = EventGen<Http2Enums::EVENT__MAX_VALUE, Http2Enums::EVENT__NONE,
    Http2Enums::HTTP2_GID>;

// This class implements HPACK decompression. One instance is required in each direction for each
// HTTP/2 flow
class Http2HpackDecoder
{
public:
    Http2HpackDecoder(Http2FlowData* flow_data, HttpCommon::SourceId src_id,
        Http2EventGen* const _events, Http2Infractions* const _infractions) :
        session_data(flow_data), events(_events), infractions(_infractions), source_id(src_id),
        decode_table(flow_data) { }
    bool decode_headers(const uint8_t* encoded_headers, const uint32_t encoded_headers_length,
         Http2StartLine* start_line, bool trailers);
    bool write_decoded_headers(const uint8_t* in_buffer, const uint32_t in_length,
        uint8_t* decoded_header_buffer, uint32_t decoded_header_length, uint32_t& bytes_written);
    bool decode_header_line(const uint8_t* encoded_header_buffer,
        const uint32_t encoded_header_length, uint32_t& bytes_consumed,
        uint8_t* decoded_header_buffer, const uint32_t decoded_header_length,
        uint32_t& bytes_written);
    bool handle_dynamic_size_update(const uint8_t* encoded_header_buffer,
        const uint32_t encoded_header_length, uint32_t& bytes_consumed);
    const HpackTableEntry* get_hpack_table_entry(const uint8_t* encoded_header_buffer,
        const uint32_t encoded_header_length, const Http2HpackIntDecode& decode_int,
        uint32_t& bytes_consumed, uint64_t &index);
    bool write_header_part(const Field& header, const uint8_t* suffix, uint32_t suffix_length,
        uint8_t* decoded_header_buffer, const uint32_t decoded_header_length,
        uint32_t& bytes_written);
    bool decode_indexed_name(const uint8_t* encoded_header_buffer,
        const uint32_t encoded_header_length, const Http2HpackIntDecode& decode_int,
        uint32_t& bytes_consumed, uint8_t* decoded_header_buffer, const uint32_t decoded_header_length,
        uint32_t& bytes_written, Field& name, bool with_indexing);
    bool decode_literal_header_line(const uint8_t* encoded_header_buffer,
        const uint32_t encoded_header_length, const uint8_t name_index_mask,
        const Http2HpackIntDecode& decode_int, bool with_indexing, uint32_t& bytes_consumed,
        uint8_t* decoded_header_buffer, const uint32_t decoded_header_length, uint32_t& bytes_written,
        Field& name, Field& value);
    bool decode_indexed_header(const uint8_t* encoded_header_buffer,
        const uint32_t encoded_header_length, const Http2HpackIntDecode& decode_int,
        uint32_t &bytes_consumed, uint8_t* decoded_header_buffer, const uint32_t decoded_header_length,
        uint32_t& bytes_written, Field& name, Field& value);
    bool decode_string_literal(const uint8_t* encoded_header_buffer,
        const uint32_t encoded_header_length, uint32_t& bytes_consumed,
        uint8_t* decoded_header_buffer, const uint32_t decoded_header_length,
        uint32_t& bytes_written, Field& field);

    bool finalize_start_line();
    void set_decoded_headers(Field&);
    bool are_pseudo_headers_allowed() { return pseudo_headers_allowed; }
    void settings_table_size_update(const uint32_t size);
    void cleanup();

private:
    Http2StartLine* start_line = nullptr;
    bool pseudo_headers_allowed = false;
    uint8_t* decoded_headers = nullptr; // working buffer to store decoded headers
    Http2FlowData* session_data;
    Http2EventGen* const events;
    Http2Infractions* const infractions;
    uint32_t decoded_headers_size = 0;
    const HttpCommon::SourceId source_id;

    static Http2HpackIntDecode decode_int7;
    static Http2HpackIntDecode decode_int6;
    static Http2HpackIntDecode decode_int5;
    static Http2HpackIntDecode decode_int4;
    static Http2HpackStringDecode decode_string;

    HpackIndexTable decode_table;
    bool table_size_update_allowed = true;
    uint8_t num_table_size_updates = 0;
    bool is_trailers = false;
    bool expect_table_size_update = false;
    uint32_t min_settings_table_size_received = UINT32_MAX;
};

#endif

http2_hpack_int_decode.h

Path = src/service_inspectors/http2_inspect/http2_hpack_int_decode.h

#ifndef HTTP2_HPACK_INT_DECODE_H
#define HTTP2_HPACK_INT_DECODE_H

#include "http2_enum.h"
#include "http2_varlen_int_decode.h"

#include "main/snort_types.h"
#include "utils/event_gen.h"
#include "utils/infractions.h"

using Http2Infractions = Infractions<Http2Enums::INF__MAX_VALUE, Http2Enums::INF__NONE>;

using Http2EventGen = EventGen<Http2Enums::EVENT__MAX_VALUE, Http2Enums::EVENT__NONE,
    Http2Enums::HTTP2_GID>;

using Http2HpackIntDecode = VarLengthIntDecode<Http2EventGen, Http2Infractions>;

#endif

http2_hpack_string_decode.h

Path = src/service_inspectors/http2_inspect/http2_hpack_string_decode.h

#ifndef HTTP2_HPACK_STRING_DECODE_H
#define HTTP2_HPACK_STRING_DECODE_H

#include "http2_hpack_int_decode.h"
#include "http2_varlen_string_decode.h"

using Http2HpackStringDecode = VarLengthStringDecode<Http2HpackIntDecode, Http2EventGen, Http2Infractions>;

#endif

http2_hpack_table.h

Path = src/service_inspectors/http2_inspect/http2_hpack_table.h

#ifndef HTTP2_HPACK_TABLE_H
#define HTTP2_HPACK_TABLE_H

#include "main/snort_types.h"

#include "http2_enum.h"
#include "http2_hpack_dynamic_table.h"

class Http2FlowData;

struct HpackTableEntry
{
    HpackTableEntry(uint32_t name_len, const uint8_t* _name, uint32_t value_len,
        const uint8_t* _value) : name { static_cast<int32_t>(name_len), _name },
        value { static_cast<int32_t>(value_len), _value } { }
    HpackTableEntry(const Field& copy_name, const Field& copy_value);
    Field name;
    Field value;
};

class HpackIndexTable
{
public:
    HpackIndexTable(Http2FlowData*) { }
    const HpackTableEntry* lookup(uint64_t index) const;
    bool add_index(const Field& name, const Field& value);
    HpackDynamicTable& get_dynamic_table() { return dynamic_table; }

    const static uint8_t STATIC_MAX_INDEX = 61;

private:
    const static HpackTableEntry static_table[STATIC_MAX_INDEX + 1];
    HpackDynamicTable dynamic_table;
};
#endif

http2_huffman_state_machine.h

Path = src/service_inspectors/http2_inspect/http2_huffman_state_machine.h

#ifndef HTTP2_HUFFMAN_STATE_MACHINE_H
#define HTTP2_HUFFMAN_STATE_MACHINE_H

#include "main/snort_types.h"

enum HuffmanState
{
    HUFFMAN_LOOKUP_1 = 0,
    HUFFMAN_LOOKUP_2 = 1,
    HUFFMAN_LOOKUP_3 = 2,
    HUFFMAN_LOOKUP_4 = 3,
    HUFFMAN_LOOKUP_5 = 4,
    HUFFMAN_LOOKUP_6 = 5,
    HUFFMAN_LOOKUP_7 = 6,
    HUFFMAN_LOOKUP_8 = 7,
    HUFFMAN_LOOKUP_9 = 8,
    HUFFMAN_LOOKUP_10 = 9,
    HUFFMAN_LOOKUP_11 = 10,
    HUFFMAN_LOOKUP_12 = 11,
    HUFFMAN_LOOKUP_13 = 12,
    HUFFMAN_LOOKUP_14 = 13,
    HUFFMAN_LOOKUP_15 = 14,
    HUFFMAN_LOOKUP_MAX = HUFFMAN_LOOKUP_15,
    HUFFMAN_MATCH = HUFFMAN_LOOKUP_MAX + 1,
    HUFFMAN_FAILURE = HUFFMAN_MATCH + 1
};

struct HuffmanEntry
{
    uint8_t len;
    char symbol;
    HuffmanState state;
};

SO_PUBLIC extern const HuffmanEntry huffman_decode[][UINT8_MAX+1];

#endif

http2_inspect.h

Path = src/service_inspectors/http2_inspect/http2_inspect.h

#ifndef HTTP2_INSPECT_H
#define HTTP2_INSPECT_H

//-------------------------------------------------------------------------
// Http2Inspect class
//-------------------------------------------------------------------------

#include "log/messages.h"
#include "service_inspectors/http_inspect/http_common.h"

#include "http2_enum.h"
#include "http2_module.h"
#include "http2_stream_splitter.h"

class Http2Api;
class Http2FlowData;

class Http2Inspect : public snort::Inspector
{
public:
    Http2Inspect(const Http2ParaList* params_);
    ~Http2Inspect() override { delete params; }

    bool get_buf(snort::InspectionBuffer::Type ibt, snort::Packet* p, snort::InspectionBuffer& b) override;
    bool get_buf(unsigned id, snort::Packet* p, snort::InspectionBuffer& b) override;
    bool get_fp_buf(snort::InspectionBuffer::Type ibt, snort::Packet* p,
        snort::InspectionBuffer& b) override;
    bool configure(snort::SnortConfig*) override;
    void show(const snort::SnortConfig*) const override;
    void eval(snort::Packet* p) override;
    void clear(snort::Packet* p) override;

    Http2StreamSplitter* get_splitter(bool is_client_to_server) override
    { return &splitter[is_client_to_server ? HttpCommon::SRC_CLIENT : HttpCommon::SRC_SERVER]; }

    bool can_carve_files() const override
    { return true; }

    const uint8_t* adjust_log_packet(snort::Packet* p, uint16_t& length) override;
private:
    friend Http2Api;

    Http2StreamSplitter splitter[2] = { true, false };

    const Http2ParaList* const params;
};

bool implement_get_buf(unsigned id, Http2FlowData* session_data, HttpCommon::SourceId source_id,
    snort::InspectionBuffer& b);
void implement_eval(Http2FlowData* session_data, HttpCommon::SourceId source_id);

#endif

http2_module.h

Path = src/service_inspectors/http2_inspect/http2_module.h

#ifndef HTTP2_MODULE_H
#define HTTP2_MODULE_H

#include <string>
#include <bitset>

#include "framework/module.h"
#include "profiler/profiler.h"

#include "http2_enum.h"

#define HTTP2_NAME "http2_inspect"
#define HTTP2_HELP "HTTP/2 inspector"

struct Http2ParaList
{
public:
    uint32_t concurrent_streams_limit;
#ifdef REG_TEST

    bool test_input;
    bool test_output;
#endif
};

class Http2Module : public snort::Module
{
public:
    Http2Module() : snort::Module(HTTP2_NAME, HTTP2_HELP, http2_params) { }
    ~Http2Module() override { delete params; }
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;
    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    unsigned get_gid() const override { return Http2Enums::HTTP2_GID; }
    const snort::RuleMap* get_rules() const override { return http2_events; }
    const Http2ParaList* get_once_params()
    {
        Http2ParaList* ret_val = params;
        params = nullptr;
        return ret_val;
    }

    const PegInfo* get_pegs() const override { return peg_names; }
    PegCount* get_counts() const override { return peg_counts; }
    static void increment_peg_counts(Http2Enums::PEG_COUNT counter)
        { peg_counts[counter]++; }
    static void increment_peg_counts(Http2Enums::PEG_COUNT counter, uint64_t value)
        { peg_counts[counter] += value; }
    static void decrement_peg_counts(Http2Enums::PEG_COUNT counter)
        { peg_counts[counter]--; }
    static PegCount get_peg_counts(Http2Enums::PEG_COUNT counter)
        { return peg_counts[counter]; }

    snort::ProfileStats* get_profile() const override;

    static snort::ProfileStats& get_profile_stats()
    { return http2_profile; }

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

private:
    static const snort::Parameter http2_params[];
    static const snort::RuleMap http2_events[];
    Http2ParaList* params = nullptr;
    static const PegInfo peg_names[];
    static THREAD_LOCAL snort::ProfileStats http2_profile;
    static THREAD_LOCAL PegCount peg_counts[];
};

#endif

http2_ping_frame.h

Path = src/service_inspectors/http2_inspect/http2_ping_frame.h

#ifndef HTTP2_PING_FRAME_H
#define HTTP2_PING_FRAME_H

#include "http2_frame.h"

class Http2Frame;

class Http2PingFrame : public Http2Frame
{
public:
    friend Http2Frame* Http2Frame::new_frame(const uint8_t*, const uint32_t, const uint8_t*,
        const uint32_t, Http2FlowData*, HttpCommon::SourceId, Http2Stream* stream);
    bool is_detection_required() const override { return false; }

private:
    Http2PingFrame(const uint8_t* header_buffer, const uint32_t header_len,
        const uint8_t* data_buffer, const uint32_t data_len, Http2FlowData* ssn_data,
        HttpCommon::SourceId src_id, Http2Stream* _stream) :
        Http2Frame(header_buffer, header_len, data_buffer, data_len, ssn_data, src_id, _stream) { }

    uint8_t get_flags_mask() const override
    { return Http2Enums::FLAG_ACK; }
};
#endif

http2_priority_frame.h

Path = src/service_inspectors/http2_inspect/http2_priority_frame.h

#ifndef HTTP2_PRIORITY_FRAME_H
#define HTTP2_PRIORITY_FRAME_H

#include "http2_enum.h"
#include "http2_flow_data.h"
#include "http2_frame.h"

class Http2PriorityFrame : public Http2Frame
{
public:
    friend Http2Frame* Http2Frame::new_frame(const uint8_t*, const uint32_t, const uint8_t*,
        const uint32_t, Http2FlowData*, HttpCommon::SourceId, Http2Stream* stream);
    bool is_detection_required() const override { return false; }

private:
    Http2PriorityFrame(const uint8_t* header_buffer, const uint32_t header_len,
        const uint8_t* data_buffer, const uint32_t data_len, Http2FlowData* ssn_data,
        HttpCommon::SourceId src_id, Http2Stream* _stream) :
        Http2Frame(header_buffer, header_len, data_buffer, data_len, ssn_data, src_id, _stream)
    {
        if (get_stream_id() == 0)
        {
            *session_data->infractions[source_id] += Http2Enums::INF_BAD_PRIORITY_FRAME_STREAM_ID;
            session_data->events[source_id]->create_event(Http2Enums::EVENT_INVALID_PRIORITY_FRAME);
        }
        if (data.length() != 5)
        {
            *session_data->infractions[source_id] += Http2Enums::INF_BAD_PRIORITY_FRAME_LENGTH;
            session_data->events[source_id]->create_event(Http2Enums::EVENT_INVALID_PRIORITY_FRAME);
        }
    }
};
#endif

http2_push_promise_frame.h

Path = src/service_inspectors/http2_inspect/http2_push_promise_frame.h

#ifndef HTTP2_PUSH_PROMISE_FRAME_H
#define HTTP2_PUSH_PROMISE_FRAME_H

#include "service_inspectors/http_inspect/http_common.h"
#include "utils/event_gen.h"
#include "utils/infractions.h"

#include "http2_enum.h"
#include "http2_frame.h"
#include "http2_headers_frame_with_startline.h"

class Field;
class Http2Frame;
class Http2Stream;
class HttpFlowData;

using Http2Infractions = Infractions<Http2Enums::INF__MAX_VALUE, Http2Enums::INF__NONE>;
using Http2EventGen = EventGen<Http2Enums::EVENT__MAX_VALUE, Http2Enums::EVENT__NONE,
    Http2Enums::HTTP2_GID>;

class Http2PushPromiseFrame : public Http2HeadersFrameWithStartline
{
public:
    bool valid_sequence(Http2Enums::StreamState state) override;
    void analyze_http1(snort::Packet*) override;
    void update_stream_state() override;
    static uint32_t get_promised_stream_id(Http2EventGen* const events,
        Http2Infractions* const infractions, const uint8_t* data_buffer, uint32_t data_len);

    friend Http2Frame* Http2Frame::new_frame(const uint8_t*, const uint32_t, const uint8_t*,
        const uint32_t, Http2FlowData*, HttpCommon::SourceId, Http2Stream* stream);

#ifdef REG_TEST
    void print_frame(FILE* output) override;
#endif

private:
    Http2PushPromiseFrame(const uint8_t* header_buffer, const uint32_t header_len,
        const uint8_t* data_buffer, const uint32_t data_len, Http2FlowData* ssn_data,
        HttpCommon::SourceId src_id, Http2Stream* stream);
    uint8_t get_flags_mask() const override;

    bool in_error_state() const override;

    static const int32_t PROMISED_ID_LENGTH = 4;
};
#endif

http2_request_line.h

Path = src/service_inspectors/http2_inspect/http2_request_line.h

#ifndef HTTP2_REQUEST_LINE_H
#define HTTP2_REQUEST_LINE_H

#include "service_inspectors/http_inspect/http_common.h"
#include "service_inspectors/http_inspect/http_field.h"

#include "http2_enum.h"
#include "http2_start_line.h"

class Http2RequestLine : public Http2StartLine
{
public:
    Http2RequestLine(Http2EventGen* const evs, Http2Infractions* const infrs) :
        Http2StartLine(evs, infrs) { }
    void process_pseudo_header(const Field& name, const Field& value) override;
    bool generate_start_line(Field& start_line, bool pseudo_headers_complete) override;

private:
    Field method;
    Field path;
    Field scheme;
    Field authority;

    static const char* authority_name;
    static const uint32_t authority_name_length = 10;
    static const char* method_name;
    static const uint32_t method_name_length = 7;
    static const char* path_name;
    static const uint32_t path_name_length = 5;
    static const char* scheme_name;
    static const uint32_t scheme_name_length = 7;

    // Constants used for special checks
    static const char* method_connect;
    static const int32_t connect_length = 7;
    static const char* method_options;
    static const int32_t options_length = 7;
    static const char* scheme_http;
    static const int32_t http_length = 4;
    static const char* scheme_https;
    static const int32_t https_length = 5;

    // Account for two spaces, and trailing crlf
    static const uint8_t num_request_line_extra_chars = 4;
    // absolute form adds '://' between scheme and authority
    static const uint32_t num_absolute_form_extra_chars = 3;
};

#endif

http2_rst_stream_frame.h

Path = src/service_inspectors/http2_inspect/http2_rst_stream_frame.h

#ifndef HTTP2_RST_STREAM_FRAME_H
#define HTTP2_RST_STREAM_FRAME_H

#include "http2_frame.h"

class Http2Frame;

class Http2RstStreamFrame : public Http2Frame
{
public:
    friend Http2Frame* Http2Frame::new_frame(const uint8_t*, const uint32_t, const uint8_t*,
        const uint32_t, Http2FlowData*, HttpCommon::SourceId, Http2Stream* stream);
    bool is_detection_required() const override { return false; }
    bool valid_sequence(Http2Enums::StreamState state) override;
    void update_stream_state() override;

#ifdef REG_TEST
    void print_frame(FILE* output) override;
#endif

private:
    Http2RstStreamFrame(const uint8_t* header_buffer, const uint32_t header_len,
        const uint8_t* data_buffer, const uint32_t data_len, Http2FlowData* ssn_data,
        HttpCommon::SourceId src_id, Http2Stream* stream);
};

#endif

http2_settings_frame.h

Path = src/service_inspectors/http2_inspect/http2_settings_frame.h

#ifndef HTTP2_SETTINGS_FRAME_H
#define HTTP2_SETTINGS_FRAME_H

#include "http2_frame.h"

class Field;
class Http2ConnectionSettings;
class Http2Frame;

class Http2SettingsFrame : public Http2Frame
{
public:
    friend Http2Frame* Http2Frame::new_frame(const uint8_t*, const uint32_t, const uint8_t*,
        const uint32_t, Http2FlowData*, HttpCommon::SourceId, Http2Stream* stream);
    bool is_detection_required() const override { return false; }

#ifdef REG_TEST
    void print_frame(FILE* output) override;
#endif

private:
    Http2SettingsFrame(const uint8_t* header_buffer, const uint32_t header_len,
        const uint8_t* data_buffer, const uint32_t data_len, Http2FlowData* ssn_data,
        HttpCommon::SourceId src_id, Http2Stream* stream);

    void queue_settings();
    bool sanity_check();
    void apply_settings();
    bool handle_update(uint16_t id, uint32_t value);

    uint8_t get_flags_mask() const override
    { return Http2Enums::FLAG_ACK; }

    bool bad_frame = false;
};

class Http2ConnectionSettings
{
public:
    uint32_t get_param(uint16_t id);
    void set_param(uint16_t id, uint32_t value);

private:
    void validate_param_id(uint16_t id);

    static const uint16_t PARAMETER_COUNT = 6;
    uint32_t parameters[PARAMETER_COUNT] = {
             4096, // Header table size
                1, // Push promise
              100, // Max concurrent Streams
            65535, // Window size
            16384, // Max frame size
       4294967295  // Max header list size
    };
};

class Http2ConnectionSettingsQueue
{
public:
    ~Http2ConnectionSettingsQueue() { delete queue; }
    auto size() const { return queue ? queue->size() : 0; }
    bool extend(Http2ConnectionSettings& item) { return size() ? extend() : init(item); }
    void pop();
    auto& front() { assert(size()); return queue->front(); }
    auto& back() { assert(size()); return queue->back(); }

private:
    bool init(Http2ConnectionSettings& item);
    bool extend();

    static const uint8_t SETTINGS_QUEUE_MAX = 6;
    std::vector<Http2ConnectionSettings>* queue = nullptr;
};
#endif

http2_start_line.h

Path = src/service_inspectors/http2_inspect/http2_start_line.h

#ifndef HTTP2_START_LINE_H
#define HTTP2_START_LINE_H

#include "service_inspectors/http_inspect/http_common.h"
#include "service_inspectors/http_inspect/http_field.h"
#include "utils/event_gen.h"
#include "utils/infractions.h"

#include "http2_enum.h"

using Http2Infractions = Infractions<Http2Enums::INF__MAX_VALUE, Http2Enums::INF__NONE>;
using Http2EventGen = EventGen<Http2Enums::EVENT__MAX_VALUE, Http2Enums::EVENT__NONE,
    Http2Enums::HTTP2_GID>;

class Http2FlowData;

class Http2StartLine
{
public:
    virtual ~Http2StartLine();

    friend class Http2Hpack;

    virtual bool generate_start_line(Field& start_line, bool pseudo_headers_complete) = 0;
    virtual void process_pseudo_header(const Field& name, const Field& value) = 0;

protected:
    Http2StartLine(Http2EventGen* const events, Http2Infractions* const infractions) :
        events(events), infractions(infractions) { }

    Http2EventGen* const events;
    Http2Infractions* const infractions;
    uint32_t start_line_length = 0;
    uint8_t *start_line_buffer = nullptr;

    // Version string is HTTP/1.1
    static const char* http_version_string;
    static const uint8_t http_version_length = 8;
};

#endif

http2_status_line.h

Path = src/service_inspectors/http2_inspect/http2_status_line.h

#ifndef HTTP2_STATUS_LINE_H
#define HTTP2_STATUS_LINE_H

#include "service_inspectors/http_inspect/http_common.h"
#include "service_inspectors/http_inspect/http_field.h"

#include "http2_start_line.h"

class Http2StatusLine : public Http2StartLine
{
public:
    Http2StatusLine(Http2EventGen* const evs, Http2Infractions* const infrs) :
        Http2StartLine(evs, infrs) { }

    void process_pseudo_header(const Field& name, const Field& value) override;
    bool generate_start_line(Field& start_line, bool pseudo_headers_complete) override;

private:
    Field status;

    static const char* STATUS_NAME;
    static const uint32_t STATUS_NAME_LENGTH = 7;
    static const uint32_t RESPONSE_PSEUDO_MIN_INDEX = 8;

};

#endif

http2_stream.h

Path = src/service_inspectors/http2_inspect/http2_stream.h

#ifndef HTTP2_STREAM_H
#define HTTP2_STREAM_H

#include "service_inspectors/http_inspect/http_common.h"
#include "service_inspectors/http_inspect/http_field.h"

#include "http2_enum.h"
#include "http2_frame.h"

class Http2DataCutter;
class Http2FlowData;
class HttpFlowData;
class HttpMsgSection;

class Http2Stream
{
public:
    Http2Stream(uint32_t stream_id, Http2FlowData* session_data_);
    ~Http2Stream();
    uint32_t get_stream_id() const { return stream_id; }
    void eval_frame(const uint8_t* header_buffer, uint32_t header_len, const uint8_t* data_buffer,
        uint32_t data_len, HttpCommon::SourceId source_id, snort::Packet* p);
    void check_and_cleanup_completed();
    void clear_frame(snort::Packet* p);
    const Field& get_buf(unsigned id);
    HttpFlowData* get_hi_flow_data() const { return hi_flow_data; }
    void set_hi_flow_data(HttpFlowData* flow_data);
    Http2Frame *get_current_frame() { return current_frame; }

    void set_state(HttpCommon::SourceId source_id, Http2Enums::StreamState new_state);
    Http2Enums::StreamState get_state(HttpCommon::SourceId source_id) const
        { return state[source_id]; }
    bool is_open(HttpCommon::SourceId source_id);
    void set_end_stream_on_data_flush(HttpCommon::SourceId source_id)
        { end_stream_on_data_flush[source_id] = true; }
    bool is_end_stream_on_data_flush(HttpCommon::SourceId source_id)
        { return end_stream_on_data_flush[source_id]; }
    void finish_msg_body(HttpCommon::SourceId source_id, bool expect_trailers,
        bool clear_partial_buffer);
    void set_discard(HttpCommon::SourceId source_id)
    { discard[source_id] = true; }
    bool is_discard_set(HttpCommon::SourceId source_id)
    { return discard[source_id]; }

#ifdef REG_TEST
    void print_frame(FILE* output);
#endif

private:
    const uint32_t stream_id;
    Http2FlowData* const session_data;
    Http2Frame* current_frame = nullptr;
    HttpFlowData* hi_flow_data = nullptr;
    bool end_stream_on_data_flush[2] = { false, false };
    Http2Enums::StreamState state[2] =
        { Http2Enums::STREAM_EXPECT_HEADERS, Http2Enums::STREAM_EXPECT_HEADERS };
    bool discard[2] = { false, false };
};

#endif

http2_stream_splitter.h

Path = src/service_inspectors/http2_inspect/http2_stream_splitter.h

#ifndef HTTP2_STREAM_SPLITTER_H
#define HTTP2_STREAM_SPLITTER_H

#include "service_inspectors/http_inspect/http_common.h"
#include "stream/stream_splitter.h"

#include "http2_enum.h"
#include "http2_flow_data.h"

class Http2Inspect;

class Http2StreamSplitter : public snort::StreamSplitter
{
public:
    Http2StreamSplitter(bool is_client_to_server) :
        snort::StreamSplitter(is_client_to_server),
        source_id(is_client_to_server ? HttpCommon::SRC_CLIENT : HttpCommon::SRC_SERVER)
        { }
    Status scan(snort::Packet* pkt, const uint8_t* data, uint32_t length, uint32_t not_used,
        uint32_t* flush_offset) override;
    const snort::StreamBuffer reassemble(snort::Flow* flow, unsigned total, unsigned offset, const
        uint8_t* data, unsigned len, uint32_t flags, unsigned& copied) override;
    bool finish(snort::Flow* flow) override;
    bool is_paf() override { return true; }

    // FIXIT-M should return actual packet buffer size
    unsigned max(snort::Flow*) override { return Http2Enums::MAX_OCTETS; }
    void go_away() override {}

private:
    const HttpCommon::SourceId source_id;
    static void data_frame_header_checks(Http2FlowData* session_data,
        HttpCommon::SourceId source_id);
    static StreamSplitter::Status non_data_scan(Http2FlowData* session_data,
        uint32_t length, uint32_t* flush_offset, HttpCommon::SourceId source_id, uint8_t type,
        uint8_t frame_flags, uint32_t& data_offset);
    static snort::StreamSplitter::Status implement_scan(Http2FlowData* session_data, const uint8_t* data,
        uint32_t length, uint32_t* flush_offset, HttpCommon::SourceId source_id);
    static const snort::StreamBuffer implement_reassemble(Http2FlowData* session_data, unsigned total,
        unsigned offset, const uint8_t* data, unsigned len, uint32_t flags,
        HttpCommon::SourceId source_id);
    static bool read_frame_hdr(Http2FlowData* session_data, const uint8_t* data,
        uint32_t length, HttpCommon::SourceId source_id, uint32_t& data_offset);
};


#endif

http2_utils.h

Path = src/service_inspectors/http2_inspect/http2_utils.h

#ifndef HTTP2_UTILS_H
#define HTTP2_UTILS_H

#include "main/snort_types.h"
#include "service_inspectors/http_inspect/http_common.h"

#include "http2_flow_data.h"

// Frame header parsing utils.
// Assumption is that if input isn't null, it contains full frame header

uint32_t get_frame_length(const uint8_t* frame_header_buffer);
uint8_t get_frame_type(const uint8_t* frame_header_buffer);
uint8_t get_frame_flags(const uint8_t* frame_header_buffer);
uint32_t get_stream_id_from_header(const uint8_t* frame_header_buffer);
uint32_t get_stream_id_from_buffer(const uint8_t* buffer);

#endif

http2_varlen_int_decode.h

Path = src/service_inspectors/http2_inspect/http2_varlen_int_decode.h

#ifndef HTTP2_VARLEN_INT_DECODE_H
#define HTTP2_VARLEN_INT_DECODE_H

#include "main/snort_types.h"

template <typename EGen, typename Inf>
class VarLengthIntDecode
{
public:
    VarLengthIntDecode(uint8_t prefix);
    bool translate(const uint8_t* in_buff, const uint32_t in_len, uint32_t& bytes_consumed,
        uint64_t& result, EGen* const events, Inf* const infractions,
        bool partial_header) const;

private:
    const uint8_t prefix_mask;
};

#endif

http2_varlen_int_decode_impl.h

Path = src/service_inspectors/http2_inspect/http2_varlen_int_decode_impl.h

#ifndef HTTP2_VARLEN_INT_DECODE_IMPL_H
#define HTTP2_VARLEN_INT_DECODE_IMPL_H

#include <cassert>

#include "main/snort_types.h"

template <typename EGen, typename Inf>
VarLengthIntDecode<EGen, Inf>::VarLengthIntDecode(uint8_t prefix) : prefix_mask(((uint16_t)1 << prefix) - 1)
{
    assert ((0 < prefix) && (prefix < 9));
}

template <typename EGen, typename Inf>
bool VarLengthIntDecode<EGen, Inf>::translate(const uint8_t* in_buff, const uint32_t in_len,
    uint32_t& bytes_consumed, uint64_t& result, EGen* const events,
    Inf* const infractions, bool partial_header) const
{
    const uint8_t VAL_MASK = 0x7F;
    const uint8_t FLAG_BIT = 0x80;

    bytes_consumed = 0;
    result = 0;

    if (in_len == 0)
    {
        if (!partial_header)
            *infractions += INF_INT_EMPTY_BUFF;
        else
            *infractions += INF_TRUNCATED_HEADER_LINE;
        return false;
    }

    const uint8_t prefix_val = in_buff[bytes_consumed++] & prefix_mask;

    if (prefix_val < prefix_mask)
    {
        result = prefix_val;
        return true;
    }

    uint8_t byte = 0;
    for (uint8_t multiplier = 0; multiplier < 64; multiplier += 7)
    {
        if (bytes_consumed >= in_len)
        {
            if (!partial_header)
                *infractions += INF_INT_MISSING_BYTES;
            else
                *infractions += INF_TRUNCATED_HEADER_LINE;
            return false;
        }
        byte = in_buff[bytes_consumed++];

        // For multiplier == 63, do overflow checks
        if (multiplier == 63)
        {
            if (((byte & FLAG_BIT) != 0) || ((byte & VAL_MASK) > 1) ||
                ((result + ((uint64_t)(byte & VAL_MASK) << multiplier) + prefix_mask) < result))
            {
                *infractions += INF_INT_OVERFLOW;
                return false;
            }
        }

        result += (uint64_t)(byte & VAL_MASK) << multiplier;

        if ((byte & FLAG_BIT) == 0)
            break;
    }

    // Alert on leading 0s, allow for value 2^N-1
    if (((byte & VAL_MASK) == 0) && (bytes_consumed != 2))
    {
        *infractions += INF_INT_LEADING_ZEROS;
        events->create_event(EVENT_INT_LEADING_ZEROS);
    }

    result += prefix_mask;

    return true;
}

#endif

http2_varlen_string_decode.h

Path = src/service_inspectors/http2_inspect/http2_varlen_string_decode.h

#ifndef HTTP2_VARLEN_STRING_DECODE_H
#define HTTP2_VARLEN_STRING_DECODE_H

#include "main/snort_types.h"

template <typename IntDec, typename EGen, typename Inf>
class VarLengthStringDecode
{
public:
    // huffman_mask for string literals in HPACK is always 0x80, thus defaulting
    // to 0x80
    bool translate(const uint8_t* in_buff, const uint32_t in_len,
        IntDec& decode_int, uint32_t& bytes_consumed,
        uint8_t* out_buff, const uint32_t out_len, uint32_t& bytes_written,
        EGen* const events, Inf* const infractions,
        bool partial_header, uint8_t huffman_mask = 0x80) const;

private:
    bool get_string(const uint8_t* in_buff, const uint32_t encoded_len, uint32_t& bytes_consumed,
        uint8_t* out_buff, const uint32_t out_len, uint32_t& bytes_written,
        Inf* const infractions) const;
    bool get_huffman_string(const uint8_t* in_buff, const uint32_t encoded_len,
        uint32_t& bytes_consumed, uint8_t* out_buff, const uint32_t out_len, uint32_t&
        bytes_written, Inf* const infractions) const;
    bool get_next_byte(const uint8_t* in_buff, const uint32_t last_byte,
        uint32_t& bytes_consumed, uint8_t& cur_bit, uint8_t match_len, uint8_t& byte,
        bool& another_search) const;
};

#endif

http2_varlen_string_decode_impl.h

Path = src/service_inspectors/http2_inspect/http2_varlen_string_decode_impl.h

#ifndef HTTP2_VARLEN_STRING_DECODE_IMPL_H
#define HTTP2_VARLEN_STRING_DECODE_IMPL_H

#include "http2_huffman_state_machine.h"
#include "http2_varlen_string_decode.h"

#include <cmath>

// Minimum bit length for each lookup table
static const uint8_t min_decode_len[HUFFMAN_LOOKUP_MAX + 1] =
    {5, 2, 2, 3, 5, 1, 1, 2, 2, 2, 2, 3, 3, 3, 4};

template <typename IntDec, typename EGen, typename Inf>
bool VarLengthStringDecode<IntDec, EGen, Inf>::translate(const uint8_t* in_buff, const uint32_t in_len,
    IntDec& decode_int, uint32_t& bytes_consumed, uint8_t* out_buff,
    const uint32_t out_len, uint32_t& bytes_written,
    EGen* const events, Inf* const infractions, bool partial_header,
    uint8_t huffman_mask) const
{
    bytes_consumed = 0;
    bytes_written = 0;

    if (in_len == 0)
    {
        if (!partial_header)
            *infractions += INF_STRING_EMPTY_BUFF;
        else
            *infractions += INF_TRUNCATED_HEADER_LINE;
        return false;
    }

    const bool isHuffman = (in_buff[0] & huffman_mask) != 0;

    // Get length
    uint64_t encoded_len;
    if (!decode_int.translate(in_buff, in_len, bytes_consumed, encoded_len, events, infractions,
            partial_header))
        return false;

    if (encoded_len > (uint64_t)(in_len - bytes_consumed))
    {
        if (!partial_header)
            *infractions += INF_STRING_MISSING_BYTES;
        else
            *infractions += INF_TRUNCATED_HEADER_LINE;
        return false;
    }

    if (encoded_len == 0)
        return true;

    if (!isHuffman)
        return get_string(in_buff, encoded_len, bytes_consumed, out_buff, out_len, bytes_written,
            infractions);

    return get_huffman_string(in_buff, encoded_len, bytes_consumed, out_buff, out_len,
        bytes_written, infractions);
}

template <typename IntDec, typename EGen, typename Inf>
bool VarLengthStringDecode<IntDec, EGen, Inf>::get_string(const uint8_t* in_buff, const uint32_t encoded_len,
    uint32_t& bytes_consumed, uint8_t* out_buff, const uint32_t out_len, uint32_t& bytes_written,
    Inf* const infractions) const
{
    if (encoded_len > out_len)
    {
        *infractions += INF_DECODED_HEADER_BUFF_OUT_OF_SPACE;
        return false;
    }

    while (bytes_written < encoded_len)
        out_buff[bytes_written++] = in_buff[bytes_consumed++];

    return true;
}

// return is tail/padding
template <typename IntDec, typename EGen, typename Inf>
bool VarLengthStringDecode<IntDec, EGen, Inf>::get_next_byte(const uint8_t* in_buff, const uint32_t last_byte,
    uint32_t& bytes_consumed, uint8_t& cur_bit, uint8_t match_len, uint8_t& byte,
    bool& another_search) const
{
    another_search = true;
    cur_bit += match_len;

    if (cur_bit >= 8) // done with current bit
    {
        bytes_consumed++;
        cur_bit -= 8;
    }

    bool tail = false;
    uint8_t msb, lsb = 0xff;
    if (bytes_consumed == last_byte)
    {
        // bytes consumed must be bigger than 1 - int length is at least 1
        msb = in_buff[bytes_consumed-1];
        another_search = false;
        tail = true;
    }
    else if ((bytes_consumed + 1) == last_byte)
    {
        if (cur_bit != 0)
        {
            msb = in_buff[bytes_consumed++];
            tail = true;
        }
        else
        {
            byte = in_buff[bytes_consumed];
            return false;
        }
    }
    else
    {
        msb = in_buff[bytes_consumed];
        lsb = in_buff[bytes_consumed+1];
    }

    const uint16_t tmp = (uint16_t)(msb << 8) | lsb;
    byte = (tmp & (0xff00 >> cur_bit)) >> (8 - cur_bit);
    return tail;
}

template <typename IntDec, typename EGen, typename Inf>
bool VarLengthStringDecode<IntDec, EGen, Inf>::get_huffman_string(const uint8_t* in_buff, const uint32_t encoded_len,
    uint32_t& bytes_consumed, uint8_t* out_buff, const uint32_t out_len, uint32_t& bytes_written,
    Inf* const infractions) const
{
    const uint32_t last_encoded_byte = bytes_consumed + encoded_len;
    uint8_t byte;
    uint8_t cur_bit = 0;
    HuffmanEntry result = { 0, 0, HUFFMAN_LOOKUP_1 };
    bool another_search = false;
    HuffmanState state = HUFFMAN_LOOKUP_1;

    // Check length
    const uint32_t max_length = floor(encoded_len * 8.0/5.0);
    if (max_length > out_len)
    {
        *infractions += INF_DECODED_HEADER_BUFF_OUT_OF_SPACE;
        return false;
    }

    while (!get_next_byte(in_buff, last_encoded_byte, bytes_consumed, cur_bit, result.len, byte,
        another_search))
    {
        result = huffman_decode[state][byte];

        switch (result.state) {

        case HUFFMAN_MATCH:
            out_buff[bytes_written++] = result.symbol;
            state = HUFFMAN_LOOKUP_1;
            break;

        case HUFFMAN_LOOKUP_2:
        case HUFFMAN_LOOKUP_3:
        case HUFFMAN_LOOKUP_4:
        case HUFFMAN_LOOKUP_5:
        case HUFFMAN_LOOKUP_6:
        case HUFFMAN_LOOKUP_7:
        case HUFFMAN_LOOKUP_8:
        case HUFFMAN_LOOKUP_9:
        case HUFFMAN_LOOKUP_10:
        case HUFFMAN_LOOKUP_11:
        case HUFFMAN_LOOKUP_12:
        case HUFFMAN_LOOKUP_13:
        case HUFFMAN_LOOKUP_14:
        case HUFFMAN_LOOKUP_15:
            state = result.state;
            break;

        case HUFFMAN_FAILURE:
            *infractions += INF_HUFFMAN_DECODED_EOS;
            return false;

        default:
            break;
        }
    }

    // Tail needs 1 last lookup in case the leftover is big enough for a match.
    // Make sure match length <= available length
    uint8_t leftover_len = 8 - cur_bit;
    uint8_t old_result_len = result.len;
    HuffmanState old_result_state = result.state;

    if (another_search && (leftover_len >= min_decode_len[state]))
    {
        result = huffman_decode[state][byte];
        if ((result.state == HUFFMAN_MATCH) && (result.len <= leftover_len))
        {
            out_buff[bytes_written++] = result.symbol;
            byte = (byte << result.len) | (((uint16_t)1 << result.len) - 1);
        }
        else
        {
            // Use leftover bits for padding check if previous lookup was a match
            if (old_result_state == HUFFMAN_MATCH)
                result.len = leftover_len;
            else
                result.len = old_result_len;
        }
    }

    // Padding check
    if (result.len < 8)
    {
        if (byte != 0xff)
        {
            *infractions += INF_HUFFMAN_BAD_PADDING;
            return false;
        }
    }
    else if (result.state != HUFFMAN_MATCH)
    {
        *infractions += INF_HUFFMAN_INCOMPLETE_CODE_PADDING;
        return false;
    }

    return true;
}

#endif

http2_window_update_frame.h

Path = src/service_inspectors/http2_inspect/http2_window_update_frame.h

#ifndef HTTP2_WINDOW_UPDATE_FRAME_H
#define HTTP2_WINDOW_UPDATE_FRAME_H

#include "http2_frame.h"

class Http2Frame;

class Http2WindowUpdateFrame : public Http2Frame
{
public:
    friend Http2Frame* Http2Frame::new_frame(const uint8_t*, const uint32_t, const uint8_t*,
        const uint32_t, Http2FlowData*, HttpCommon::SourceId, Http2Stream* stream);
    bool is_detection_required() const override { return false; }
    bool valid_sequence(Http2Enums::StreamState state) override;

#ifdef REG_TEST
    void print_frame(FILE* output) override;
#endif

private:
    Http2WindowUpdateFrame(const uint8_t* header_buffer, const uint32_t header_len,
        const uint8_t* data_buffer, const uint32_t data_len, Http2FlowData* ssn_data,
        HttpCommon::SourceId src_id, Http2Stream* stream);
};

#endif

ips_http2.h

Path = src/service_inspectors/http2_inspect/ips_http2.h

#ifndef IPS_HTTP2_H
#define IPS_HTTP2_H

#include <array>

#include "profiler/profiler.h"
#include "framework/ips_option.h"
#include "framework/module.h"

#include "http2_enum.h"

enum PsIdx { PSI_FRAME_DATA, PSI_FRAME_HEADER, PSI_DECODED_HEADER, PSI_MAX };

class Http2CursorModule : public snort::Module
{
public:
    Http2CursorModule(const char* key_, const char* help, Http2Enums::HTTP2_BUFFER buffer_index_,
        snort::CursorActionType cat_, PsIdx psi_) : snort::Module(key_, help), key(key_),
        buffer_index(buffer_index_), cat(cat_), psi(psi_) {}
    snort::ProfileStats* get_profile() const override { return &http2_ps[psi]; }
    static void mod_dtor(snort::Module* m) { delete m; }
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    Usage get_usage() const override
    { return DETECT; }

private:
    friend class Http2IpsOption;
    static THREAD_LOCAL std::array<snort::ProfileStats, PsIdx::PSI_MAX> http2_ps;

    struct Http2RuleParaList
    {
    public:
        void reset();
    };

    const char* const key;
    const Http2Enums::HTTP2_BUFFER buffer_index;
    const snort::CursorActionType cat;
    const PsIdx psi;

    Http2RuleParaList para_list;
};

class Http2IpsOption : public snort::IpsOption
{
public:
    Http2IpsOption(const Http2CursorModule* cm) :
        snort::IpsOption(cm->key), key(cm->key),
        buffer_index(cm->buffer_index), cat(cm->cat), psi(cm->psi) {}
    snort::CursorActionType get_cursor_type() const override { return cat; }
    EvalStatus eval(Cursor&, snort::Packet*) override;
    uint32_t hash() const override;
    bool operator==(const snort::IpsOption& ips) const override;
    static snort::IpsOption* opt_ctor(snort::Module* m, OptTreeNode*)
        { return new Http2IpsOption((Http2CursorModule*)m); }
    static void opt_dtor(IpsOption* p) { delete p; }
private:
    const char* const key;
    const Http2Enums::HTTP2_BUFFER buffer_index;
    const snort::CursorActionType cat;
    const PsIdx psi;
};

#endif

service_inspectors/http2_inspect/test/

service_inspectors/http_inspect/

The HTTP inspector (HI) is divided into two major parts. The HttpStreamSplitter (splitter) accepts TCP payload data from Stream and subdivides it into message sections. HttpInspect (inspector) processes individual message sections.

Splitter finish() is called by Stream when the TCP connection closes (including pruning). It serves several specialized purposes in cases where the HTTP message is truncated (ends unexpectedly).

The nature of splitting allows packets to be forwarded before they are aggregated into a message section and inspected. This may lead to problems when the target consumes a partial message body even though the end of the message body was never received because Snort blocked it.

Script detection is a feature developed to solve this problem for message bodies containing Javascripts. The stream splitter scan() method searches its input for the end-of-script tag "</script>". When necessary this requires scan() to unzip the data. This is an extra unzip as storage limitations preclude saving the unzipped version of the data for subsequent reassembly.

Update: the previous sentence has been discovered to be incorrect. The memory requirements of zlib are very large. It would save a lot of memory and some processing time for script detection to unzip one time in scan() and store the result for eventual use by reassemble(). The memory lost by storing partial message sections in HI while waiting for reassemble() would be more than compensated for by not having two instances of zlib.

HttpFlowData is a data class representing all HI information relating to a flow. It serves as persistent memory between invocations of HI by the framework. It also glues together the inspector, the client-to-server splitter, and the server-to-client splitter which pass information through the flow data.

An HttpTransaction is a container that keeps all the sections of a message together and associates the request message with the response message. Transactions may be organized into pipelines when an HTTP pipeline is present. The current transaction and any pipeline live in the flow data. A transaction may have only a request because the response is not (yet) received or only a response because the corresponding request is unknown or unavailable.

The attach_my_transaction() factory method contains all the logic that makes this work. There are many corner cases. Don’t mess with it until you fully understand it.

HI implements flow depth using the request_depth and response_depth parameters. HI seeks to provide a consistent experience to detection by making flow depth independent of factors that a sender could easily manipulate, such as header length, chunking, compression, and encodings. The maximum depth is computed against normalized message body data.

Partial inspection

When the end of a script is found and the normal flush point has not been found, the current TCP segment and all previous segments for the current message section are flushed using a special procedure known as partial inspection. From the perspective of Stream (or H2I) a partial inspection is a regular flush in every respect.

scan() calls prep_partial_flush() to prepare for the partial inspection. Then it returns a normal flush point to Stream at the end of the current TCP segment. Partial inspections perform all of the functions of a regular inspection including forwarding data to file processing and detection.

The difference between a partial inspection and a regular inspection is reassemble() saves the input data for future reuse. Eventually there will be a regular full inspection of the entire message section. reassemble() will accomplish this by combining the input data for the partial inspection with later data that complete the message section.

Correct and efficient execution of a full inspection following a partial inspection requires special handling of certain functions. Unzipping is only done once in reassemble(). The stored input in reassemble() has already been through dechunking and unzipping. Data is forwarded to file processing during the partial inspection and duplicate data will not be forwarded again. Some of the message body normalization steps are done once during partial inspection with work products saved for reuse.

It is possible to do more than one partial inspection of a single message section. Each partial inspection is cumulative, covering the new data and all previous data.

Compared to just doing a full inspection, a partial inspection followed by a full inspection will not miss anything. The benefits of partial inspection are in addition to the benefits of a full inspection.

The http_inspect partial inspection mechanism is also used by http2_inspect to manage frame boundaries. When inspecting HTTP/2, a partial inspection by http_inspect may occur because script detection triggered it, because H2I wanted it, or both.

Some applications may be affected by blocks too late scenarios related to seeing part of the zero-length chunk. For example a TCP packet that ends with:

8<CR><LF>abcdefgh<CR><LF>0

might be sufficient to forward the available data ("abcdefgh") to the application even though the final <CR><LF> has not been received.

Note that the actual next bytes are uncertain here. The next packet might begin with <CR><LF>, but

100000<CR><LF>ijklmnopq ...

is another perfectly legal possibility. There is no rule against starting a nonzero chunk length with a zero character and some applications reputedly do this.

As a precaution partial inspection is performed when 1) a TCP segment ends inside a possible zero-length chunk or 2) chunk processing fails (broken chunk).

Message section

Message section is a core concept of HI. A message section is a piece of an HTTP message that is processed together. There are eight types of message section:

  1. Request line (client-to-server start line)

  2. Status line (server-to-client start line)

  3. Headers (all headers after the start line as a group)

  4. Content-Length message body (a block of message data usually not much larger than 16K from a body defined by the Content-Length header)

  5. Chunked message body (same but from a chunked body)

  6. Old message body (same but from a response body with no Content-Length header that runs to connection close)

  7. HTTP/X message body (same but content taken from an HTTP/2 or HTTP/3 Data frame)

  8. Trailers (all header lines following a chunked or HTTP/X body as a group)

Message sections are represented by message section objects that contain and process them. There are twelve message section classes that inherit as follows. An asterisk denotes a virtual class.

  1. HttpMsgSection* - top level with all common elements

  2. HttpMsgStart* : HttpMsgSection - common elements of request and status

  3. HttpMsgRequest : HttpMsgStart

  4. HttpMsgStatus : HttpMsgStart

  5. HttpMsgHeadShared* : HttpMsgSection - common elements of headers and trailers

  6. HttpMsgHeader : HttpMsgHeadShared

  7. HttpMsgTrailer : HttpMsgHeadShared

  8. HttpMsgBody* : HttpMsgSection - common elements of message body processing

  9. HttpMsgBodyCl : HttpMsgBody

  10. HttpMsgBodyChunk : HttpMsgBody

  11. HttpMsgBodyOld : HttpMsgBody

  12. HttpMsgBodyHX : HttpMsgBody

Message sections implement the Just-In-Time (JIT) principle for work products. A minimum of essential processing is done under process(). Other work products are derived and stored the first time detection or some other customer asks for them.

Field class

The Field class is an important tool for managing JIT. It consists of a pointer to a raw message field or derived work product with a length field. Various negative length values specify the status of the field. For instance STAT_NOTCOMPUTE means the item has not been computed yet, STAT_NOTPRESENT means the item does not exist, and STAT_PROBLEMATIC means an attempt to compute the item failed. Never dereference the pointer without first checking the length value.

All of these values and more are in http_enums.h which is a general repository for enumerated values in HI.

A Field is intended to represent an immutable object. It is either part of the original message section or it is a work product that has been derived from the original message section. In the former case the original message is constant and there is no reason for a Field value to change. In the latter case, once the value has been derived from the original message there is no reason to derive it again.

Once Field is set to a non-null value it should never change. The set() functions will assert if this rule is disregarded.

Partial inspections have created an exception. Fields may be used to store work products from a partial inspection that may be updated by subsequent inspections. The reset() method has been provided for this situation. It deletes any owned buffer and reinitializes the Field to null. This feature should be used with care to avoid weakening the architecture.

A Field may own the buffer containing the message or it may point to a buffer that belongs to someone else. When a Field owning a buffer is deleted the buffer is deleted as well. Ownership is determined with the Field is initially set. In general any dynamically allocated buffer should be owned by a Field. If you follow this rule you won’t need to keep track of allocated buffers or have delete[]s all over the place.

Support for custom xff type headers

HI also supports defining custom "x-forwarded-for" type headers. In a multi-vendor world, it is quite possible that the header name carrying the original client IP could be vendor-specific. This is due to the absence of standardization which would otherwise standardize the header name. In such a scenario, it is important to provide a configuration with which such x-forwarded-for type headers can be introduced to HI. The headers can be introduced with the xff_headers configuration. The default value of this configuration is "x-forwarded-for true-client-ip". The default definition introduces the two commonly known "x-forwarded-for" type headers and is preferred in the same order by the inspector as they are defined, e.g "x-forwarded-for" will be preferred than "true-client-ip" if both headers are present in the stream. Every HTTP Header is mapped to an ID internally. The custom headers are mapped to a dynamically generated ID and the mapping is appended at the end of the mapping of the known HTTP headers. Every HI instance can have its own list of custom headers and thus an instance of HTTP header mapping list is also associated with an HI instance.

URI normalization

HttpUri is the class that represents a URI. HttpMsgRequest objects have an HttpUri that is created during analyze().

URI normalization is performed during HttpUri construction in four steps.

Step 1: Identify the type of URI.

HI recognizes four types of URI:

  1. Asterisk: a lone ‘*’ character signifying that the request does not refer to any resource in particular. Often used with the OPTIONS method. This is not normalized.

  2. Authority: any URI used with the CONNECT method. The entire URI is treated as an authority.

  3. Origin: a URI that begins with a slash. Consists of only an absolute path with no scheme or authority present.

  4. Absolute: a URI which includes a scheme and a host as well as an absolute path. E.g. http://example.com/path/to/resource.

In addition there are malformed URIs that don’t meet any of the four types. These are protocol errors and will trigger an alert. Because their format is unrecognized they are not normalized.

Step 2: Decompose the URI into its up to six constituent pieces: scheme, host, port, path, query, and fragment.

Based on the URI type the overall URI is divided into scheme, authority, and absolute path. The authority is subdivided into host and port. The absolute path is subdivided into path, query, and fragment.

The raw URI pieces can be accessed via rules. For example: http_raw_uri: query; content: “foo” will only match the query portion of the URI.

Step 3: Normalize the individual pieces.

The port is not normalized. The scheme is normalized to lower case. The other four pieces are normalized in a fashion similar to 2.X with an important exception. Path-related normalizations such as eliminating directory traversals and squeezing out extra slashes are only done for the path.

The normalized URI pieces can be accessed via rules. For example: http_uri: path; content: “foo/bar”.

Step 4: Stitch the normalized pieces back together into a complete normalized URI.

This allows rules to be written against a normalized whole URI as is done in 2.X.

The procedures for normalizing the individual pieces are mostly identical to 2.X. Some points warrant mention:

  1. HI considers it to be normal for reserved characters to be percent encoded and does not generate an alert. The 119/1 alert is used only for unreserved characters that are found to be percent encoded. The ignore_unreserved configuration option allows the user to specify a list of unreserved characters that are exempt from this alert.

  2. Plus to space substitution is a configuration option. It is not currently limited to the query but that would not be a difficult feature to add.

  3. The 2.X multi_slash and directory options are combined into a single option called simplify_path.

JS normalization

HttpJsNorm class serves as a script Normalizer, and currently has two implementations: the Legacy Normalizer and the Enhanced Normalizer.

In NHI, there are three JSNorm extensions: * HttpInlineJSNorm, processes content of HTML script tags. * HttpExternalJSNorm, processes payload with JavaScript MIME type. * HttpPDFJSNorm, processes payload with PDF MIME type and PDF files in MIME attachments.

Normalization context is per transaction. It is created once js_data calls for normalized JS data, and is deleted once transaction ends. Partial inspections feed data incrementally to JS Normalizer, but the output is accumulated and concatenated in the right way, presenting more comprehensive block of data to Detection.

During message body analysis the Enhanced Normalizer does one of the following: 1. If Content-Type says its an external script then Normalizer processes the whole message body as a script text. 2. If it is an HTML-page, Normalizer searches for an opening tag and processes subsequent bytes in a stream mode, until it finds a closing tag. It proceeds and scans the entire message body for inline scripts. 3. If it is a PDF file transferred as MIME attachment or as a message body then Normalizer extracts strings assigned to /JS key, or streams referred from the same value, and normalizes their content as JS text.

Also, js_data IPS option’s buffer is a part of NHI processing in order to start the normalization.

Enhanced Normalizer supports scripts over multiple PDUs. So, if the script is not ended, Normalizer’s context is saved in HttpFlowData. The script continuation will be processed with the saved context.

In order to support Script Detection feature for inline scripts, Normalizer ensures that after reaching the script end (legitimate closing tag or bad token), it falls back to an initial state, so that the next script can be processed by the same context.

Reassembling chunked message

Algorithm for reassembling chunked message bodies:

NHI parses chunked message bodies using an algorithm based on the HTTP RFC. Chunk headers are not included in reassembled message sections and do not count against the message section length. The attacker cannot affect split points by adjusting their chunks.

Built-in alerts for chunking are generated for protocol violations and suspicious usages. Many irregularities can be compensated for but others cannot. Whenever a fatal problem occurs, NHI generates 119:213 HTTP chunk misformatted and converts to a mode very similar to run to connection close. The rest of the flow is sent to detection as is. No further attempt is made to dechunk the message body or look for the headers that begin the next message. The user should block 119:213 unless they are willing to run the risk of continuing with no real security.

In addition to 119:213 there will often be a more specific alert based on what went wrong.

From the perspective of NHI, a chunked message body is a sequence of zero or more chunks followed by a zero-length chunk. Following the zero-length chunk there will be trailers which may be empty (CRLF only).

Each chunk begins with a header and is parsed as follows:

  1. Zero or more unexpected CR or LF characters. If any are present 119:234 is generated and processing continues.

  2. Zero or more unexpected space and tab characters. If any are present 119:214 is generated. If five or more are present that is a fatal error as described above and chunk processing stops.

  3. Zero or more 0 characters. Leading zeros before other digits are meaningless and ignored. A chunk length consisting solely of zeros is the zero-length chunk. Five or more leading zeros generate 119:202 regardless of whether the chunk length eventually turns out to be zero or nonzero.

  4. The chunk length in hexadecimal format. The chunk length may be zero (see above) but it must be present. Both upper and lower case hex letters are acceptable. The 0x prefix for hex numbers is not acceptable.

    The goal here is a hexadecimal number followed by CRLF ending the chunk header. Many things may go wrong:

    • More than 8 hex digits other than the leading zeros. The number is limited by Snort to fit into 32 bits and if it does not that is a fatal error.

    • The CR may be missing, leaving a bare LF as the separator. That generates 119:235 after which processing continues normally.

    • There may be one or more trailing spaces or tabs following the number. If any are present 119:214 is generated after which processing continues normally.

    • There may be chunk options. This is legal and parsing is supported but options are so unexpected that they are suspicious. 119:210 is generated.

    • There may be a completely illegal character in the chunk length (other than those mentioned above). That is a fatal error.

    • The character following the CR may not be LF. This is a fatal error. This is different from similar bare CR errors because it does not provide a transparent data channel. An "innocent" sender that implements this error has no way to transmit chunk data that begins with LF.

  5. Following the chunk header should be a number of bytes of transparent user data equal to the chunk length. This is the part of the chunked message body that is reassembled and inspected. Everything else is discarded.

  6. Following the chunk data should be CRLF which do not count against the chunk length. These are not present for the zero length chunk. If one of the two separators is missing, 119:234 is generated and processing continues normally. If there is no separator at all that is a fatal error.

Then we return to #1 as the next chunk begins. In particular extra separators beyond the two expected are attributed to the beginning of the next chunk.

MIME inspection

MIME processing:

NHI processes request message bodies in MIME format differently from other message bodies. Message sections are forwarded to the MIME library instead of being directly input to file processing. The library parses the input into individual MIME attachments. This creates a design issue because there may be multiple attachments within a single message body section. The email inspectors solve this issue by splitting MIME attachments within their stream splitters so that there is only one attachment per reassembled packet. This attachment, if it contains a file, is the source material for the file_data rule option.

NHI stream splitter does not work this way. It does not consider MIME at all. Split points between message sections are never based on MIME or any other type of message body content.

The problem for NHI is that file_data is a singular entity and cannot accomodate multiple simultaneous files derived from a message section. NHI resolves this by accumulating the processed file attachments in a list and directly calling detection multiple times—once for each file attachment installed as file_data.

Rule options:

HttpIpsOption is the base class for http rule options. It supports the parameters field and request that are used by some rule options. HttpBufferIpsOption is a rule option that sets a buffer. It implements most of the rule options.

HI test tool usage

Test tool usage instructions:

The HI test tool consists of two features. test_output provides extensive information about the inner workings of HI. It is strongly focused on showing work products (Fields) rather than being a tracing feature. Given a problematic pcap, the developer can see what the input is, how HI interprets it, and what the output to rule options will be. Several related configuration options (see help) allow the developer to customize the output.

test_input is provided by the HttpTestInput class. It allows the developer to write tests that simulate HTTP messages split into TCP segments at specified points. The tests cover all of splitter and inspector and the impact on downstream customers such as detection and file processing. The test_input option activates a modified form of test_output. It is not necessary to also specify test_output.

The test input comes from the file http_test_msgs.txt in the current directory. Enter HTTP test message text as you want it to be presented to the StreamSplitter.

The easiest way to format is to put a blank line between message sections so that each message section is its own "paragraph". Within a paragraph the placement of single new lines does not have any effect. Format a paragraph any way you are comfortable. Extra blank lines between paragraphs also do not have any effect.

Each paragraph represents a TCP segment. The splitter can be tested by putting multiple sections in the same paragraph (splitter must split) or continuing a section in the next paragraph (splitter must search and reassemble).

Lines beginning with # are comments. Lines beginning with @ are commands. These do not apply to lines in the middle of a paragraph. Lines that begin with $ are insert commands - a special class of commands that may be used within a paragraph to insert data into the message buffer.

Commands: @break resets HTTP Inspect data structures and begins a new test. Use it liberally to prevent unrelated tests from interfering with each other. @tcpclose simulates a half-duplex TCP close. @request and @response set the message direction. Applies to subsequent paragraphs until changed. The initial direction is always request and the break command resets the direction to request. @fileset <pathname> specifies a file from which the tool will read data into the message buffer. This may be used to include a zipped or other binary file into a message body. Data is read beginning at the start of the file. The file is closed automatically whenever a new file is set or there is a break command. @fileskip <decimal number> skips over the specified number of bytes in the included file. This must be a positive number. To move backward do a new fileset and skip forward from the beginning. @<decimal number> sets the test number and hence the test output file name. Applies to subsequent sections until changed. Don’t reuse numbers.

Insert commands: $fill <decimal number> create a paragraph consisting of <number> octets of auto-fill data ABCDEFGHIJABC …. $fileread <decimal number> read the specified number of bytes from the included file into the message buffer. $h2preface creates the HTTP/2 connection preface "PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n" $h2frameheader <frame_type> <frame_length> <flags> <stream_id> generates an HTTP/2 frame header. The frame type may be the frame type name in all lowercase or the numeric frame type code: (data|headers|priority|rst_stream|settings|push_promise|ping|goaway|window_update| continuation|{0:9}) The frame length is the length of the frame payload, may be in decimal or test tool hex value (\xnn, see below under escape sequence for more details) The frame flags are represented as a single test tool hex byte (\xnn) The stream id is optional. If provided it must be a decimal number. If not included it defaults to 0.

Escape sequences begin with \. They may be used within a paragraph or to begin a paragraph. \r - carriage return \n - linefeed \t - tab \\ - backslash \# - # \@ - @ \$ - $ \xnn or \Xnn - where nn is a two-digit hexadecimal number. Insert an arbitrary 8-bit number as the next character. a-f and A-F are both acceptable.

Data are separated into segments for presentation to the splitter whenever a paragraph ends (blank line).

When the inspector aborts the connection (scan() returns StreamSplitter::ABORT) it does not expect to receive any more input from stream on that connection in that direction. Accordingly the test tool should not send it any more input. A paragraph of test input expected to result in an abort should be the last paragraph. The developer should either start a new test (@break, etc.) or at least reverse the direction and not send any more data in the original direction. Sending more data after an abort is likely to lead to confusing output that has no bearing on the test.

This test tool does not implement the feature of being hardened against bad input. If you write a badly formatted or improper test case the program may assert or crash. The responsibility is on the developer to get it right.

The test tool is designed for single-threaded operation only.

The test tool is only available when compiled with REG_TEST.

http_api.h

Path = src/service_inspectors/http_inspect/http_api.h

#ifndef HTTP_API_H
#define HTTP_API_H

#include "framework/inspector.h"
#include "framework/module.h"

#include "http_flow_data.h"
#include "http_module.h"

class HttpApi
{
public:
    static const snort::InspectApi http_api;
    static const char* classic_buffer_names[];

private:
    HttpApi() = delete;
    static snort::Module* http_mod_ctor() { return new HttpModule; }
    static void http_mod_dtor(snort::Module* m) { delete m; }
    static const char* http_my_name;
    static const char* http_help;
    static void http_init();
    static void http_term() { }
    static snort::Inspector* http_ctor(snort::Module* mod);
    static void http_dtor(snort::Inspector* p) { delete p; }
};

#endif

http_buffer_info.h

Path = src/service_inspectors/http_inspect/http_buffer_info.h

#ifndef HTTP_BUFFER_INFO_H
#define HTTP_BUFFER_INFO_H

#include "main/snort_types.h"

class HttpBufferInfo
{
public:
    HttpBufferInfo(unsigned type_, uint64_t sub_id_ = 0, uint64_t form_ = 0)
        : type(type_), sub_id(sub_id_), form(form_) {}

    uint32_t hash() const;

    bool operator==(const HttpBufferInfo& rhs) const;
    bool is_request() const;

public:
    const unsigned type;
    const uint64_t sub_id;
    const uint64_t form;
};

#endif

http_common.h

Path = src/service_inspectors/http_inspect/http_common.h

#ifndef HTTP_COMMON_H
#define HTTP_COMMON_H

#include <cstdint>

namespace HttpCommon
{
// Field status codes for when no valid value is present in length or integer value. Positive
// values are actual length or field value.
enum StatusCode { STAT_NO_SOURCE=-16, STAT_NOT_CONFIGURED=-15, STAT_NOT_COMPUTE=-14,
    STAT_PROBLEMATIC=-12, STAT_NOT_PRESENT=-11, STAT_EMPTY_STRING=0, STAT_OTHER=1 };

// Message originator--client or server
enum SourceId { SRC__NOT_COMPUTE=-14, SRC_CLIENT=0, SRC_SERVER=1 };

// Type of message section
enum SectionType { SEC_DISCARD = -19, SEC_ABORT = -18, SEC__NOT_COMPUTE=-14, SEC__NOT_PRESENT=-11,
    SEC_REQUEST = 2, SEC_STATUS, SEC_HEADER, SEC_BODY_CL, SEC_BODY_CHUNK, SEC_TRAILER,
    SEC_BODY_OLD, SEC_BODY_HX };

// Caters to all extended versions of HTTP, i.e. HTTP/2, HTTP/3
enum HXBodyState { HX_BODY_NOT_COMPLETE, HX_BODY_LAST_SEG, HX_BODY_COMPLETE,
    HX_BODY_COMPLETE_EXPECT_TRAILERS, HX_BODY_NO_BODY };

} // end namespace HttpCommon

#endif

http_context_data.h

Path = src/service_inspectors/http_inspect/http_context_data.h

#ifndef HTTP_CONTEXT_DATA_H
#define HTTP_CONTEXT_DATA_H

#include "detection/ips_context_data.h"

class HttpMsgSection;

class HttpContextData : public snort::IpsContextData
{
public:
    void clear() override
    { current_section = nullptr; }

    static void init()
    { ips_id = IpsContextData::get_ips_id(); }
    static HttpMsgSection* get_snapshot(const snort::Packet* p);
    static HttpMsgSection* get_snapshot(const snort::Flow* flow,
        snort::IpsContext* context = nullptr);
    static void save_snapshot(HttpMsgSection* section);
    static HttpMsgSection* clear_snapshot(snort::IpsContext* context);
    static unsigned ips_id;

private:
    HttpMsgSection* current_section = nullptr;
};

#endif

http_cursor_data.h

Path = src/service_inspectors/http_inspect/http_cursor_data.h

#ifndef HTTP_CURSOR_DATA_H
#define HTTP_CURSOR_DATA_H

#include "framework/cursor.h"

class HttpCursorData : public CursorData
{
public:
    HttpCursorData() : CursorData(id) {}

    static void init()
    { id = CursorData::create_cursor_data_id(); }

    HttpCursorData* clone() override
    { return new HttpCursorData(*this); }

    bool retry()
    {
        return query_index < num_query_params ||
            body_index < num_body_params;
    }

public:
    static unsigned id;
    unsigned num_query_params = 0;
    unsigned num_body_params = 0;
    unsigned query_index = 0;
    unsigned body_index = 0;
};

#endif

http_cutter.h

Path = src/service_inspectors/http_inspect/http_cutter.h

#ifndef HTTP_CUTTER_H
#define HTTP_CUTTER_H

#include <cassert>
#include <zlib.h>

#include "http_common.h"
#include "http_enum.h"
#include "http_event.h"
#include "http_module.h"

class HttpFlowData;

//-------------------------------------------------------------------------
// HttpCutter class and subclasses
//-------------------------------------------------------------------------

class HttpCutter
{
public:
    virtual ~HttpCutter() = default;
    virtual HttpEnums::ScanResult cut(const uint8_t* buffer, uint32_t length,
        HttpInfractions* infractions, HttpEventGen* events, uint32_t flow_target, bool stretch,
        HttpCommon::HXBodyState state) = 0;
    uint32_t get_num_flush() const { return num_flush; }
    uint32_t get_octets_seen() const { return octets_seen; }
    uint32_t get_num_excess() const { return num_crlf; }
    virtual uint32_t get_num_head_lines() const { return 0; }
    virtual bool get_is_broken_chunk() const { return false; }
    virtual uint32_t get_num_good_chunks() const { return 0; }
    virtual void soft_reset() {}

protected:
    // number of octets processed by previous cut() calls that returned NOT_FOUND
    uint32_t octets_seen = 0;
    uint32_t num_crlf = 0;
    uint32_t num_flush = 0;
};

class HttpStartCutter : public HttpCutter
{
public:
    HttpEnums::ScanResult cut(const uint8_t* buffer, uint32_t length,
        HttpInfractions* infractions, HttpEventGen* events, uint32_t, bool, HttpCommon::HXBodyState)
        override;

protected:
    enum ValidationResult { V_GOOD, V_BAD, V_TBD };
    bool validated = false;

    bool find_eol(uint8_t octet, uint32_t ind, HttpInfractions* infractions, HttpEventGen* events);

private:
    static const int MAX_LEADING_WHITESPACE = 20;
    virtual ValidationResult validate(uint8_t octet, HttpInfractions*, HttpEventGen*) = 0;
};

class HttpRequestCutter : public HttpStartCutter
{
private:
    uint32_t octets_checked = 0;
    bool check_h2 = true;
    ValidationResult validate(uint8_t octet, HttpInfractions*, HttpEventGen*) override;
};

class HttpStatusCutter : public HttpStartCutter
{
private:
    uint32_t octets_checked = 0;
    ValidationResult validate(uint8_t octet, HttpInfractions*, HttpEventGen*) override;
};

class HttpZeroNineCutter : public HttpStartCutter
{
public:
    HttpEnums::ScanResult cut(const uint8_t* buffer, uint32_t length,
        HttpInfractions* infractions, HttpEventGen* events, uint32_t, bool, HttpCommon::HXBodyState) override;
    static const int match_size = 5;
    static uint8_t match[match_size];

private:
    uint32_t octets_checked = 0;
    ValidationResult validate(uint8_t octet, HttpInfractions*, HttpEventGen*) override;
};

class HttpHeaderCutter : public HttpCutter
{
public:
    HttpEnums::ScanResult cut(const uint8_t* buffer, uint32_t length,
        HttpInfractions* infractions, HttpEventGen* events, uint32_t, bool, HttpCommon::HXBodyState)
        override;
    uint32_t get_num_head_lines() const override { return num_head_lines; }

private:
    enum LineEndState { ZERO, HALF, ONE, THREEHALF };
    LineEndState state = ONE;
    int32_t num_head_lines = 0;
};

class HttpBodyCutter : public HttpCutter
{
public:
    HttpBodyCutter(bool accelerated_blocking_, ScriptFinder* finder,
        HttpEnums::CompressId compression_);
    ~HttpBodyCutter() override;
    void soft_reset() override { octets_seen = 0; }

protected:
    bool need_accelerated_blocking(const uint8_t* data, uint32_t length);

private:
    bool dangerous(const uint8_t* data, uint32_t length);
    bool find_partial(const uint8_t*, uint32_t, bool);

    const bool accelerated_blocking;
    uint8_t partial_match = 0;
    HttpEnums::CompressId compression = HttpEnums::CompressId::CMP_NONE;
    bool decompress_failed = false;
    uint8_t string_length = 0;
    z_stream* compress_stream = nullptr;
    ScriptFinder* const finder;
    const uint8_t* match_string;
    const uint8_t* match_string_upper;
};

class HttpBodyClCutter : public HttpBodyCutter
{
public:
    HttpBodyClCutter(int64_t expected_length,
        bool accelerated_blocking,
        ScriptFinder* finder,
        HttpEnums::CompressId compression) :
        HttpBodyCutter(accelerated_blocking, finder, compression),
        remaining(expected_length)
        { assert(remaining > 0); }
    HttpEnums::ScanResult cut(const uint8_t*, uint32_t length, HttpInfractions*, HttpEventGen*,
        uint32_t flow_target, bool stretch, HttpCommon::HXBodyState) override;

private:
    int64_t remaining;
};

class HttpBodyOldCutter : public HttpBodyCutter
{
public:
    HttpBodyOldCutter(bool accelerated_blocking, ScriptFinder* finder,
        HttpEnums::CompressId compression) :
        HttpBodyCutter(accelerated_blocking, finder, compression)
        {}
    HttpEnums::ScanResult cut(const uint8_t*, uint32_t, HttpInfractions*, HttpEventGen*,
        uint32_t flow_target, bool stretch, HttpCommon::HXBodyState) override;
};

class HttpBodyChunkCutter : public HttpBodyCutter
{
public:
    HttpBodyChunkCutter(int64_t maximum_chunk_length_, bool accelerated_blocking,
        ScriptFinder* finder, HttpEnums::CompressId compression) :
        HttpBodyCutter(accelerated_blocking, finder, compression),
        maximum_chunk_length(maximum_chunk_length_)
        {}
    HttpEnums::ScanResult cut(const uint8_t* buffer, uint32_t length,
        HttpInfractions* infractions, HttpEventGen* events, uint32_t flow_target, bool stretch,
        HttpCommon::HXBodyState) override;
    bool get_is_broken_chunk() const override { return curr_state == HttpEnums::CHUNK_BAD; }
    uint32_t get_num_good_chunks() const override { return num_good_chunks; }
    void soft_reset() override { num_good_chunks = 0; HttpBodyCutter::soft_reset(); }

private:
    void transition_to_chunk_bad(bool& accelerate_this_packet);

    const int64_t maximum_chunk_length;

    uint32_t data_seen = 0;
    HttpEnums::ChunkState curr_state = HttpEnums::CHUNK_NEWLINES;
    uint32_t expected = 0;
    uint32_t num_leading_ws = 0;
    uint32_t num_zeros = 0;
    uint32_t digits_seen = 0;
    uint32_t num_good_chunks = 0;  // that end in the current section
    bool zero_chunk = true;
};

class HttpBodyHXCutter : public HttpBodyCutter
{
public:
    HttpBodyHXCutter(int64_t expected_length, bool accelerated_blocking, ScriptFinder* finder,
        HttpEnums::CompressId compression) :
        HttpBodyCutter(accelerated_blocking, finder, compression),
            expected_body_length(expected_length)
        {}
    HttpEnums::ScanResult cut(const uint8_t* buffer, uint32_t length, HttpInfractions*,
        HttpEventGen*, uint32_t flow_target, bool stretch, HttpCommon::HXBodyState state) override;
private:
    int64_t expected_body_length;
    uint32_t total_octets_scanned = 0;
};

#endif

http_enum.h

Path = src/service_inspectors/http_inspect/http_enum.h

#ifndef HTTP_ENUM_H
#define HTTP_ENUM_H

#include <cstdint>

namespace HttpEnums
{
static const int MAX_OCTETS = 63780;
static const int GZIP_BLOCK_SIZE = 2048;
static const int MAX_SECTION_STRETCH = 1460;
static const int REQUEST_PUBLISH_DEPTH = 2000;

static const uint32_t HTTP_GID = 119;
static const int GZIP_WINDOW_BITS = 31;
static const uint8_t GZIP_HEADER_FLAG_OFFSET = 3;
static const uint8_t GZIP_FLAG_FEXTRA = 0x4;
static const uint8_t GZIP_RESERVED_FLAGS = 0xE0; // bits 5-7
static const int DEFLATE_WINDOW_BITS = 15;
static const int MAX_FIELD_NAME_LENGTH = 100;
// Plan to support max 8 xff headers
static const uint8_t MAX_XFF_HEADERS = 8;
static const uint8_t MAX_CUSTOM_HEADERS = MAX_XFF_HEADERS;

// This can grow into a bitmap for the get_buf() form parameter
static const uint64_t FORM_REQUEST = 0x1;

// HTTP rule options.
// Lower numbered portion is message buffers available to clients.
// That part must remain synchronized with HttpApi::classic_buffer_names[]
enum HTTP_RULE_OPT { HTTP_BUFFER_CLIENT_BODY = 1, HTTP_BUFFER_COOKIE, HTTP_BUFFER_HEADER,
    HTTP_BUFFER_METHOD, HTTP_BUFFER_PARAM, HTTP_BUFFER_RAW_BODY, HTTP_BUFFER_RAW_COOKIE,
    HTTP_BUFFER_RAW_HEADER, HTTP_BUFFER_RAW_REQUEST, HTTP_BUFFER_RAW_STATUS,
    HTTP_BUFFER_RAW_TRAILER, HTTP_BUFFER_RAW_URI, HTTP_BUFFER_STAT_CODE, HTTP_BUFFER_STAT_MSG,
    HTTP_BUFFER_TRAILER, HTTP_BUFFER_TRUE_IP, HTTP_BUFFER_URI, HTTP_BUFFER_VERSION,
    BUFFER_JS_DATA, BUFFER_VBA_DATA, HTTP__BUFFER_MAX = BUFFER_VBA_DATA,
    HTTP_RANGE_NUM_HDRS, HTTP_RANGE_NUM_TRAILERS, HTTP_VERSION_MATCH,
    HTTP_HEADER_TEST, HTTP_TRAILER_TEST, HTTP_RANGE_NUM_COOKIES, HTTP_RANGE_MAX_HEADER_LINE,
    HTTP_RANGE_MAX_TRAILER_LINE, HTTP__MAX_RULE_OPTION };

// Peg counts
// This enum must remain synchronized with HttpModule::peg_names[] in http_tables.cc
enum PEG_COUNT { PEG_FLOW = 0, PEG_SCAN, PEG_REASSEMBLE, PEG_INSPECT, PEG_REQUEST, PEG_RESPONSE,
    PEG_GET, PEG_HEAD, PEG_POST, PEG_PUT, PEG_DELETE, PEG_CONNECT, PEG_OPTIONS, PEG_TRACE,
    PEG_OTHER_METHOD, PEG_REQUEST_BODY, PEG_CHUNKED, PEG_URI_NORM, PEG_URI_PATH, PEG_URI_CODING,
    PEG_CONCURRENT_SESSIONS, PEG_MAX_CONCURRENT_SESSIONS, PEG_SCRIPT_DETECTION,
    PEG_PARTIAL_INSPECT, PEG_EXCESS_PARAMS, PEG_PARAMS, PEG_CUTOVERS, PEG_SSL_SEARCH_ABND_EARLY,
    PEG_PIPELINED_FLOWS, PEG_PIPELINED_REQUESTS, PEG_TOTAL_BYTES, PEG_JS_INLINE, PEG_JS_EXTERNAL,
    PEG_JS_PDF, PEG_SKIP_MIME_ATTACH, PEG_COUNT_MAX };

// Result of scanning by splitter
enum ScanResult { SCAN_NOT_FOUND, SCAN_NOT_FOUND_ACCELERATE, SCAN_FOUND, SCAN_FOUND_PIECE,
    SCAN_DISCARD, SCAN_DISCARD_PIECE, SCAN_ABORT };

// State machine for chunk parsing
enum ChunkState { CHUNK_NEWLINES, CHUNK_ZEROS, CHUNK_LEADING_WS, CHUNK_NUMBER, CHUNK_TRAILING_WS,
    CHUNK_OPTIONS, CHUNK_HCRLF, CHUNK_DATA, CHUNK_DCRLF1, CHUNK_DCRLF2, CHUNK_BAD };

// List of possible HTTP versions.
enum VersionId { VERS__PROBLEMATIC=-12, VERS__NOT_PRESENT=-11, VERS__OTHER=1,
    VERS_1_0, VERS_1_1, VERS_2_0, VERS_3_0, VERS_0_9, VERS__MIN = VERS__PROBLEMATIC,
    VERS__MAX = VERS_0_9};

// Every request method we have ever heard of
enum MethodId {
    METH__NO_SOURCE=-16, METH__NOT_COMPUTE=-14, METH__PROBLEMATIC=-12,
    METH__NOT_PRESENT=-11, METH__OTHER=1, METH_OPTIONS, METH_GET,
    METH_HEAD, METH_POST, METH_PUT, METH_DELETE, METH_TRACE, METH_CONNECT,
    METH_VERSION_CONTROL, METH_REPORT, METH_CHECKOUT, METH_CHECKIN,
    METH_UNCHECKOUT, METH_MKWORKSPACE, METH_UPDATE, METH_LABEL, METH_MERGE,
    METH_BASELINE_CONTROL, METH_MKACTIVITY, METH_ORDERPATCH, METH_ACL,
    METH_PATCH, METH_BIND, METH_LINK, METH_MKCALENDAR, METH_MKREDIRECTREF,
    METH_REBIND, METH_UNBIND, METH_UNLINK, METH_UPDATEREDIRECTREF,

    METH__WEBDAV_LOW=1000,  // All WebDAV related methods go after this.
    METH_PROPFIND, METH_PROPPATCH, METH_MKCOL, METH_COPY, METH_MOVE,
    METH_LOCK, METH_UNLOCK, METH_SEARCH, METH_BCOPY, METH_BDELETE,
    METH_BMOVE, METH_BPROPFIND, METH_BPROPPATCH, METH_NOTIFY, METH_POLL,
    METH_SUBSCRIBE, METH_UNSUBSCRIBE, METH_X_MS_ENUMATTS,
    METH__WEBDAV_HIGH       // All WebDAV related methods go before this.
};

// URI formats
enum UriType { URI__NOT_COMPUTE=-14, URI__PROBLEMATIC=-12, URI_ASTERISK = 2, URI_AUTHORITY,
    URI_ORIGIN, URI_ABSOLUTE };

// Body compression types
enum CompressId { CMP_NONE=2, CMP_GZIP, CMP_DEFLATE };

// GZIP magic verification state
enum GzipVerificationState { GZIP_TBD, GZIP_MAGIC_BAD, GZIP_MAGIC_GOOD, GZIP_FLAGS_PROCESSED };

// Part of the URI to be provided
enum UriComponent { UC_SCHEME = 1, UC_HOST, UC_PORT, UC_PATH, UC_QUERY, UC_FRAGMENT };

// Types of character for URI scanning
enum CharAction { CHAR_NORMAL=2, CHAR_PERCENT, CHAR_PATH, CHAR_EIGHTBIT, CHAR_SUBSTIT };

// Content codings
enum Contentcoding { CONTENTCODE__OTHER=1, CONTENTCODE_GZIP, CONTENTCODE_DEFLATE,
    CONTENTCODE_COMPRESS, CONTENTCODE_EXI, CONTENTCODE_PACK200_GZIP, CONTENTCODE_X_GZIP,
    CONTENTCODE_X_COMPRESS, CONTENTCODE_IDENTITY, CONTENTCODE_CHUNKED, CONTENTCODE_BR,
    CONTENTCODE_BZIP2, CONTENTCODE_LZMA, CONTENTCODE_PEERDIST, CONTENTCODE_SDCH,
    CONTENTCODE_XPRESS, CONTENTCODE_XZ };

// Content media-types (MIME types)
enum ContentType { CT__OTHER=1, CT_APPLICATION_PDF, CT_APPLICATION_OCTET_STREAM,
    CT_APPLICATION_JAVASCRIPT, CT_APPLICATION_ECMASCRIPT, CT_APPLICATION_X_JAVASCRIPT,
    CT_APPLICATION_X_ECMASCRIPT, CT_APPLICATION_XHTML_XML, CT_TEXT_JAVASCRIPT,
    CT_TEXT_JAVASCRIPT_1_0, CT_TEXT_JAVASCRIPT_1_1, CT_TEXT_JAVASCRIPT_1_2, CT_TEXT_JAVASCRIPT_1_3,
    CT_TEXT_JAVASCRIPT_1_4, CT_TEXT_JAVASCRIPT_1_5, CT_TEXT_ECMASCRIPT, CT_TEXT_X_JAVASCRIPT,
    CT_TEXT_X_ECMASCRIPT, CT_TEXT_JSCRIPT, CT_TEXT_LIVESCRIPT, CT_TEXT_HTML };

// Transfer-Encoding header values
enum TransferEncoding { TE__OTHER=1, TE_CHUNKED, TE_IDENTITY };

// Upgrade header values
enum Upgrade { UP__OTHER=1, UP_H2C, UP_H2, UP_HTTP20 };

// Every header we have ever heard of
// Note: when making changes here also update NormalizedHeader::header_norms
enum HeaderId { HEAD__NOT_COMPUTE=-14, HEAD__PROBLEMATIC=-12, HEAD__NOT_PRESENT=-11, HEAD__OTHER=1,
    HEAD_CACHE_CONTROL, HEAD_CONNECTION, HEAD_DATE, HEAD_PRAGMA, HEAD_TRAILER, HEAD_COOKIE,
    HEAD_SET_COOKIE, HEAD_TRANSFER_ENCODING, HEAD_UPGRADE, HEAD_VIA, HEAD_WARNING, HEAD_ACCEPT,
    HEAD_ACCEPT_CHARSET, HEAD_ACCEPT_ENCODING, HEAD_ACCEPT_LANGUAGE, HEAD_AUTHORIZATION,
    HEAD_EXPECT, HEAD_FROM, HEAD_HOST, HEAD_IF_MATCH, HEAD_IF_MODIFIED_SINCE, HEAD_IF_NONE_MATCH,
    HEAD_IF_RANGE, HEAD_IF_UNMODIFIED_SINCE, HEAD_MAX_FORWARDS, HEAD_PROXY_AUTHORIZATION,
    HEAD_RANGE, HEAD_REFERER, HEAD_TE, HEAD_USER_AGENT, HEAD_ACCEPT_RANGES, HEAD_AGE, HEAD_ETAG,
    HEAD_LOCATION, HEAD_PROXY_AUTHENTICATE, HEAD_RETRY_AFTER, HEAD_SERVER, HEAD_VARY,
    HEAD_WWW_AUTHENTICATE, HEAD_ALLOW, HEAD_CONTENT_ENCODING, HEAD_CONTENT_LANGUAGE,
    HEAD_CONTENT_LENGTH, HEAD_CONTENT_LOCATION, HEAD_CONTENT_MD5, HEAD_CONTENT_RANGE,
    HEAD_CONTENT_TYPE, HEAD_EXPIRES, HEAD_LAST_MODIFIED, HEAD_X_FORWARDED_FOR, HEAD_TRUE_CLIENT_IP,
    HEAD_X_WORKING_WITH, HEAD_CONTENT_TRANSFER_ENCODING, HEAD_MIME_VERSION, HEAD_PROXY_AGENT,
    HEAD_CONTENT_DISPOSITION, HEAD_HTTP2_SETTINGS, HEAD_RESTRICT_ACCESS_TO_TENANTS,
    HEAD_RESTRICT_ACCESS_CONTEXT, HEAD__MAX_VALUE };

// All the infractions we might find while parsing and analyzing a message
enum Infraction
{
    INF__NONE = -1,
    INF_BARE_BYTE = 0,
    INF_HEAD_TOO_LONG = 1,
    INF_BAD_REQ_LINE = 2,
    INF_BAD_STAT_LINE = 3,
    INF_TOO_MANY_HEADERS = 4,
    INF_BAD_HEADER = 5,
    INF_BAD_STAT_CODE = 6,
    INF_VERSION_HIGHER_THAN_1 = 7,
    INF_BAD_VERSION = 8,
    INF_ZERO_NINE_NOT_FIRST = 9,
    INF_CODE_POINT_IN_URI = 10,
    INF_BAD_CONTENT_LENGTH = 11,
    INF_PIPELINE_OVERFLOW = 12,
    INF_BAD_CHUNK_SIZE = 13,
    INF_BAD_PHRASE = 14,
    INF_BAD_URI = 15,
    INF_ZERO_NINE_REQ = 16,
    INF_ZERO_NINE_CONTINUE = 17,
    INF_URI_PERCENT_UTF8_3B = 18,
    INF_URI_PERCENT_UNRESERVED = 19,
    INF_URI_PERCENT_UTF8_2B = 20,
    INF_URI_NEED_NORM_SCHEME = 21,
    INF_URI_PERCENT_OTHER = 22,
    INF_URI_BAD_CHAR = 23,
    INF_URI_8BIT_CHAR = 24,
    INF_URI_MULTISLASH = 25,
    INF_BACKSLASH_IN_URI = 26,
    INF_URI_SLASH_DOT = 27,
    INF_URI_SLASH_DOT_DOT = 28,
    INF_URI_ROOT_TRAV = 29,
    INF_TOO_MUCH_LEADING_WS = 30,
    INF_WS_BETWEEN_MSGS = 31,
    INF_ENDLESS_HEADER = 32,
    INF_LF_WITHOUT_CR = 33,
    INF_NOT_HTTP = 34,
    INF_NO_URI = 35,
    INF_REQUEST_WS = 36,
    INF_REQUEST_TAB = 37,
    INF_STATUS_WS = 38,
    INF_STATUS_TAB = 39,
    INF_URI_SPACE = 40,
    INF_TOO_LONG_HEADER = 41,
    INF_CR_WITHOUT_LF = 42,
    INF_CHUNK_ZEROS = 43,
    INF_CHUNK_OPTIONS = 44,
    INF_CHUNK_BAD_CHAR = 45,
    INF_CHUNK_TOO_LARGE = 46,
    INF_CHUNK_BARE_LF = 47,
    INF_CHUNK_LONE_CR = 48,
    INF_CHUNK_NO_LENGTH = 49,
    INF_CHUNK_BAD_END = 50,
    INF_PARTIAL_START = 51,
    INF_CHUNK_WHITESPACE = 52,
    INF_HEAD_NAME_WHITESPACE = 53,
    INF_GZIP_OVERRUN = 54,
    INF_GZIP_FAILURE = 55,
    INF_GZIP_EARLY_END = 56,
    INF_URI_NEED_NORM_PATH = 57,
    INF_URI_NEED_NORM_HOST = 58,
    INF_URI_NEED_NORM_QUERY = 59,
    INF_URI_NEED_NORM_FRAGMENT = 60,
    INF_URI_U_ENCODE = 61,
    INF_URI_UNKNOWN_PERCENT = 62,
    INF_URI_DOUBLE_DECODE = 63,
    INF_MULTIPLE_CONTLEN = 64,
    INF_BOTH_CL_AND_TE = 65,
    INF_BAD_CODE_BODY_HEADER = 66,
    INF_BAD_TE_HEADER = 67,
    INF_PADDED_TE_HEADER = 68,
    INF_OVERSIZE_DIR = 69,
    INF_POST_WO_BODY = 70,
    INF_UTF_NORM_FAIL = 71,
    INF_UTF7 = 72,
    INF_UNSUPPORTED_ENCODING = 73,
    INF_UNKNOWN_ENCODING = 74,
    INF_STACKED_ENCODINGS = 75,
    INF_JS_OBFUSCATION_EXCD = 76,
    INF_JS_EXCESS_WS = 77,
    INF_MIXED_ENCODINGS = 78,
    INF_RESPONSE_WO_REQUEST = 79,
    INF_SWF_ZLIB_FAILURE = 80,
    INF_SWF_LZMA_FAILURE = 81,
    INF_PDF_DEFL_FAILURE = 82,
    INF_PDF_UNSUP_COMP_TYPE = 83,
    INF_PDF_CASC_COMP = 84,
    INF_PDF_PARSE_FAILURE = 85,
    INF_FILE_DECOMPR_OVERRUN = 86,
    INF_BAD_CHAR_IN_HEADER_NAME = 87,
    INF_HEADER_WRAPPING = 88,
    INF_CHUNK_BAD_SEP = 89,
    INF_MULTIPLE_100_RESPONSES = 90,
    INF_UNEXPECTED_100_RESPONSE = 91,
    INF_UNKNOWN_1XX_STATUS = 92,
    INF_EXPECT_WITHOUT_BODY_CL0 = 93,
    INF_EXPECT_WITHOUT_BODY_NO_CL = 94,
    INF_CHUNKED_ONE_POINT_ZERO = 95,
    INF_CTE_HEADER = 96,
    INF_ILLEGAL_TRAILER = 97,
    INF_REPEATED_HEADER = 98,
    INF_CONTENT_ENCODING_CHUNKED = 99,
    INF_206_WITHOUT_RANGE = 100,
    INF_VERSION_NOT_UPPERCASE = 101,
    INF_CHUNK_LEADING_WS = 102,
    INF_BAD_HEADER_WHITESPACE = 103,
    INF_H2_NON_IDENTITY_TE = 104,
    INF_H2_DATA_OVERRUNS_CL = 105,
    INF_H2_DATA_UNDERRUNS_CL = 106,
    INF_CONNECT_REQUEST_BODY = 107,
    INF_EARLY_C2S_TRAFFIC_AFTER_CONNECT = 108,
    INF_200_CONNECT_RESP_WITH_CL = 109,
    INF_200_CONNECT_RESP_WITH_TE = 110,
    INF_100_CONNECT_RESP = 111,
    INF_EARLY_CONNECT_RESPONSE = 112,
    INF_MALFORMED_CD_FILENAME = 113,
    INF_HTTP2_IN_HI = 114,
    INF_TRUNCATED_MSG_BODY_CL = 115,
    INF_TRUNCATED_MSG_BODY_CHUNK = 116,
    INF_LONG_SCHEME = 117,
    INF_MULTIPLE_HOST_HDRS = 118,
    INF_HTTP2_SETTINGS = 119,
    INF_UPGRADE_HEADER_HTTP2 = 120,
    INF_JS_CODE_IN_EXTERNAL = 124,
    INF_JS_SHORTENED_TAG = 125,
    INF_CHUNK_OVER_MAXIMUM = 128,
    INF_LONG_HOST_VALUE = 129,
    INF_ACCEPT_ENCODING_CONSECUTIVE_COMMAS = 130,
    INF_INVALID_SUBVERSION = 133,
    INF_VERSION_0 = 134,
    INF_GZIP_FEXTRA = 135,
    INF_METHOD_NOT_ON_ALLOWED_LIST = 136,
    INF_METHOD_ON_DISALLOWED_LIST = 137,
    INF_PIPELINE_MAX = 138,
    INF_GZIP_RESERVED_FLAGS = 139,
    INF__MAX_VALUE
};

enum EventSid
{
    EVENT__NONE = -1,
    EVENT_ASCII = 1,
    EVENT_DOUBLE_DECODE = 2,
    EVENT_U_ENCODE = 3,
    EVENT_BARE_BYTE = 4,
    // EVENT_BASE_36 = 5,                        // Retired. Do not reuse this number
    EVENT_UTF_8 = 6,
    EVENT_CODE_POINT_IN_URI = 7,
    EVENT_MULTI_SLASH = 8,
    EVENT_BACKSLASH_IN_URI = 9,
    EVENT_SELF_DIR_TRAV = 10,
    EVENT_DIR_TRAV = 11,
    EVENT_APACHE_WS = 12,
    EVENT_LF_WITHOUT_CR = 13,
    EVENT_NON_RFC_CHAR = 14,
    EVENT_OVERSIZE_DIR = 15,
    EVENT_LARGE_CHUNK = 16,
    // EVENT_PROXY_USE = 17,                     // Retired. Do not reuse this number
    EVENT_WEBROOT_DIR = 18,
    EVENT_LONG_HDR = 19,
    EVENT_MAX_HEADERS = 20,
    EVENT_MULTIPLE_CONTLEN = 21,
    // EVENT_CHUNK_SIZE_MISMATCH = 22,           // Retired. Do not reuse this number
    // EVENT_INVALID_TRUEIP = 23,                // Retired. Do not reuse this number
    EVENT_MULTIPLE_HOST_HDRS = 24,
    EVENT_LONG_HOSTNAME = 25,
    // EVENT_EXCEEDS_SPACES = 26,                // Retired. Do not reuse this number
    // EVENT_CONSECUTIVE_SMALL_CHUNKS = 27,      // Retired. Do not reuse this number
    EVENT_UNBOUNDED_POST = 28,
    // EVENT_MULTIPLE_TRUEIP_IN_SESSION = 29,    // Retired. Do not reuse this number
    // EVENT_BOTH_TRUEIP_XFF_HDRS = 30,          // Retired. Do not reuse this number
    EVENT_UNKNOWN_METHOD = 31,
    EVENT_SIMPLE_REQUEST = 32,
    EVENT_UNESCAPED_SPACE_URI = 33,
    EVENT_PIPELINE_MAX = 34,

    // EVENT_ANOM_SERVER = 101,                  // Retired. Do not reuse this number
    EVENT_INVALID_STATCODE = 102,
    // EVENT_UNUSED_1 = 103,                     // Retired. Do not reuse this number
    EVENT_UTF_NORM_FAIL = 104,
    EVENT_UTF7 = 105,
    // EVENT_DECOMPR_FAILED = 106,               // Retired. Do not reuse this number
    // EVENT_CONSECUTIVE_SMALL_CHUNKS_S = 107,   // Retired. Do not reuse this number
    // EVENT_UNUSED_2 = 108,                     // Retired. Do not reuse this number
    EVENT_JS_OBFUSCATION_EXCD = 109,
    EVENT_JS_EXCESS_WS = 110,
    EVENT_MIXED_ENCODINGS = 111,
    EVENT_SWF_ZLIB_FAILURE = 112,
    EVENT_SWF_LZMA_FAILURE = 113,
    EVENT_PDF_DEFL_FAILURE = 114,
    EVENT_PDF_UNSUP_COMP_TYPE = 115,
    EVENT_PDF_CASC_COMP = 116,
    EVENT_PDF_PARSE_FAILURE = 117,

    EVENT_LOSS_OF_SYNC = 201,
    EVENT_CHUNK_ZEROS = 202,
    EVENT_WS_BETWEEN_MSGS = 203,
    EVENT_URI_MISSING = 204,
    EVENT_CTRL_IN_REASON = 205,
    EVENT_IMPROPER_WS = 206,
    EVENT_BAD_VERS = 207,
    // EVENT_VERSION_HIGHER_THAN_1 = 208,        // Retired. Do not reuse this number

    EVENT_BAD_HEADER = 209,
    EVENT_CHUNK_OPTIONS = 210,
    EVENT_URI_BAD_FORMAT = 211,
    EVENT_UNKNOWN_PERCENT = 212,
    EVENT_BROKEN_CHUNK = 213,
    EVENT_CHUNK_WHITESPACE = 214,
    EVENT_HEAD_NAME_WHITESPACE = 215,
    EVENT_GZIP_OVERRUN = 216,
    EVENT_GZIP_FAILURE = 217,
    EVENT_ZERO_NINE_CONTINUE = 218,
    EVENT_ZERO_NINE_NOT_FIRST = 219,
    EVENT_BOTH_CL_AND_TE = 220,
    EVENT_BAD_CODE_BODY_HEADER = 221,
    EVENT_BAD_TE_HEADER = 222,
    EVENT_PADDED_TE_HEADER = 223,
    // EVENT_MISFORMATTED_HTTP = 224,            // Retired. Do not reuse this number
    EVENT_UNSUPPORTED_ENCODING = 225,
    EVENT_UNKNOWN_ENCODING = 226,
    EVENT_STACKED_ENCODINGS = 227,
    EVENT_RESPONSE_WO_REQUEST = 228,
    EVENT_FILE_DECOMPR_OVERRUN = 229,
    EVENT_BAD_CHAR_IN_HEADER_NAME = 230,
    EVENT_BAD_CONTENT_LENGTH = 231,
    EVENT_HEADER_WRAPPING = 232,
    EVENT_CR_WITHOUT_LF = 233,
    EVENT_CHUNK_BAD_SEP = 234,
    EVENT_CHUNK_BARE_LF = 235,
    EVENT_MULTIPLE_100_RESPONSES = 236,
    EVENT_UNEXPECTED_100_RESPONSE = 237,
    EVENT_UNKNOWN_1XX_STATUS = 238,
    EVENT_EXPECT_WITHOUT_BODY = 239,
    EVENT_CHUNKED_ONE_POINT_ZERO = 240,
    EVENT_CTE_HEADER = 241,
    EVENT_ILLEGAL_TRAILER = 242,
    EVENT_REPEATED_HEADER = 243,
    EVENT_CONTENT_ENCODING_CHUNKED = 244,
    EVENT_206_WITHOUT_RANGE = 245,
    EVENT_VERSION_NOT_UPPERCASE = 246,
    EVENT_BAD_HEADER_WHITESPACE = 247,
    EVENT_GZIP_EARLY_END = 248,
    EVENT_EXCESS_REPEAT_PARAMS = 249,
    EVENT_H2_NON_IDENTITY_TE = 250,
    EVENT_H2_DATA_OVERRUNS_CL = 251,
    EVENT_H2_DATA_UNDERRUNS_CL = 252,
    EVENT_CONNECT_REQUEST_BODY = 253,
    EVENT_EARLY_C2S_TRAFFIC_AFTER_CONNECT = 254,
    EVENT_200_CONNECT_RESP_WITH_CL = 255,
    EVENT_200_CONNECT_RESP_WITH_TE = 256,
    EVENT_100_CONNECT_RESP = 257,
    EVENT_EARLY_CONNECT_RESPONSE = 258,
    EVENT_MALFORMED_CD_FILENAME = 259,
    EVENT_TRUNCATED_MSG_BODY_CL = 260,
    EVENT_TRUNCATED_MSG_BODY_CHUNK = 261,
    EVENT_LONG_SCHEME = 262,
    EVENT_HTTP2_UPGRADE_REQUEST = 263,
    EVENT_HTTP2_UPGRADE_RESPONSE = 264,
    // EVENT_JS_BAD_TOKEN = 265,               // Retired. Do not reuse this number
    // EVENT_JS_OPENING_TAG = 266,             // Retired. Do not reuse this number
    // EVENT_JS_CLOSING_TAG = 267,             // Retired. Do not reuse this number
    EVENT_JS_CODE_IN_EXTERNAL = 268,
    EVENT_JS_SHORTENED_TAG = 269,
    // EVENT_JS_IDENTIFIER_OVERFLOW = 270,     // Retired. Do not reuse this number
    // EVENT_JS_BRACKET_NEST_OVERFLOW = 271,   // Retired. Do not reuse this number
    EVENT_ACCEPT_ENCODING_CONSECUTIVE_COMMAS = 272,
    // EVENT_JS_DATA_LOST = 273,               // Retired. Do not reuse this number
    // EVENT_JS_SCOPE_NEST_OVERFLOW = 274,     // Retired. Do not reuse this number
    EVENT_INVALID_SUBVERSION = 275,
    EVENT_VERSION_0 = 276,
    EVENT_VERSION_HIGHER_THAN_1 = 277,
    EVENT_GZIP_FEXTRA = 278,
    EVENT_BAD_STAT_LINE = 279,
    EVENT_HEADERS_TOO_LONG = 280,
    EVENT_BAD_REQ_LINE = 281,
    EVENT_TOO_MUCH_LEADING_WS = 282,
    EVENT_STAT_TOO_LONG = 283,
    EVENT_PARTIAL_START = 284,
    EVENT_REQ_TOO_LONG = 285,
    EVENT_UNEXPECTED_H2_PREFACE = 286,
    EVENT_DISALLOWED_METHOD = 287,
    EVENT_GZIP_RESERVED_FLAGS = 288,
    EVENT__MAX_VALUE
};

extern const int8_t as_hex[256];
extern const bool token_char[256];
extern const bool scheme_char[256];
extern const bool is_sp_tab[256];
extern const bool is_cr_lf[256];
extern const bool is_sp_tab_lf[256];
extern const bool is_sp_tab_cr_lf[256];
extern const bool is_sp_tab_cr_lf_vt_ff[256];
extern const bool is_sp_tab_quote_dquote[256];
extern const bool is_print_char[256]; // printable includes SP, tab, CR, LF
extern const bool is_sp_comma[256];

} // end namespace HttpEnums

#endif

http_event.h

Path = src/service_inspectors/http_inspect/http_event.h

#ifndef HTTP_EVENT_H
#define HTTP_EVENT_H

#include "utils/event_gen.h"
#include "utils/infractions.h"
#include "utils/util_cstring.h"

#include "http_enum.h"

//-------------------------------------------------------------------------
// HTTP Event generator
//-------------------------------------------------------------------------

using HttpEventGen = EventGen<HttpEnums::EVENT__MAX_VALUE, HttpEnums::EVENT__NONE, HttpEnums::HTTP_GID>;

static const unsigned BASE_1XX_EVENTS = 100;
static const unsigned BASE_2XX_EVENTS = 200;

//-------------------------------------------------------------------------
// Http Infractions
//-------------------------------------------------------------------------

using HttpInfractions = Infractions<HttpEnums::INF__MAX_VALUE, HttpEnums::INF__NONE>;

#endif

http_field.h

Path = src/service_inspectors/http_inspect/http_field.h

#ifndef HTTP_FIELD_H
#define HTTP_FIELD_H

#include <cstdint>
#include <cstdio>
#include <cassert>

#include "main/snort_types.h"

#include "http_common.h"

// Individual pieces of the message found during parsing.
// Length values <= 0 are StatusCode values and imply that the start pointer is meaningless.
// Never use the start pointer without verifying that length > 0.
class SO_PUBLIC Field
{
public:
    static const Field FIELD_NULL;

    Field(int32_t length, const uint8_t* start, bool own_the_buffer_ = false);
    explicit Field(int32_t length) : len(length) { assert(length<=0); }
    Field() = default;

    // own_the_buffer precludes trivial copy assignment
    Field& operator=(const Field& rhs) = delete;

    ~Field() { if (own_the_buffer) delete[] strt; }
    int32_t length() const { return len; }
    const uint8_t* start() const { return strt; }
    void set(int32_t length, const uint8_t* start, bool own_the_buffer_ = false);
    void set(const Field& f);
    void set(HttpCommon::StatusCode stat_code);
    void set(int32_t length) { set(static_cast<HttpCommon::StatusCode>(length)); }
    void reset();
    void set_accumulation(bool is_accum) { was_accumulated = is_accum; }
    bool is_accumulated() const { return was_accumulated; }

#ifdef REG_TEST
    void print(FILE* output, const char* name) const;
#endif

private:
    const uint8_t* strt = nullptr;
    int32_t len = HttpCommon::STAT_NOT_COMPUTE;
    bool own_the_buffer = false;
    // FIXIT-M: find better place for the attribute, replace it with actual number of bytes processed
    bool was_accumulated = false;
};

struct MimeBufs
{
    Field file;
    Field vba;
    MimeBufs(int32_t file_len, const uint8_t* file_buf, bool file_own, int32_t vba_len, const uint8_t* vba_buf,
        bool vba_own) :
        file(file_len, file_buf, file_own),
        vba(vba_len, vba_buf, vba_own) {}
};

#endif

http_flow_data.h

Path = src/service_inspectors/http_inspect/http_flow_data.h

#ifndef HTTP_FLOW_DATA_H
#define HTTP_FLOW_DATA_H

#include <zlib.h>

#include <cstdio>
#include <list>

#include "flow/flow.h"
#include "utils/util_utf.h"
#include "decompress/file_decomp.h"

#include "http_common.h"
#include "http_enum.h"
#include "http_event.h"
#include "http_field.h"
#include "http_module.h"

class HttpTransaction;
class HttpJSNorm;
class HttpMsgSection;
class HttpCutter;
class HttpQueryParser;

namespace snort
{
class MimeSession;
}

class HttpFlowData : public snort::FlowData
{
public:
    HttpFlowData(snort::Flow* flow, const HttpParaList* params_);
    ~HttpFlowData() override;
    static unsigned inspector_id;
    static void init() { inspector_id = snort::FlowData::create_flow_data_id(); }

    friend class HttpBodyCutter;
    friend class HttpInspect;
    friend class HttpJSNorm;
    friend class HttpMsgSection;
    friend class HttpMsgStart;
    friend class HttpMsgRequest;
    friend class HttpMsgStatus;
    friend class HttpMsgHeader;
    friend class HttpMsgHeadShared;
    friend class HttpMsgTrailer;
    friend class HttpMsgBody;
    friend class HttpMsgBodyChunk;
    friend class HttpMsgBodyCl;
    friend class HttpMsgBodyHX;
    friend class HttpMsgBodyOld;
    friend class HttpQueryParser;
    friend class HttpStreamSplitter;
    friend class HttpTransaction;
#if defined(REG_TEST) || defined(UNIT_TEST)
    friend class HttpUnitTestSetup;
#endif

    HttpCommon::SectionType get_type_expected(HttpCommon::SourceId source_id) const
    { return type_expected[source_id]; }

    void finish_hx_body(HttpCommon::SourceId source_id, HttpCommon::HXBodyState state,
        bool clear_partial_buffer);

    void set_hx_body_state(HttpCommon::SourceId source_id, HttpCommon::HXBodyState state)
    { hx_body_state[source_id] = state; }

    int64_t get_hx_stream_id() const;
    bool is_for_httpx() const { return for_httpx; }

private:
    const HttpParaList* const params;

    // Convenience routines
    void half_reset(HttpCommon::SourceId source_id);
    void trailer_prep(HttpCommon::SourceId source_id);
    void garbage_collect();

    // 0 element refers to client request, 1 element refers to server response

    // *** StreamSplitter internal data - scan()
    HttpCutter* cutter[2] = { nullptr, nullptr };

    // *** StreamSplitter internal data - reassemble()
    uint8_t* section_buffer[2] = { nullptr, nullptr };
    uint32_t section_offset[2] = { 0, 0 };
    uint32_t chunk_expected_length[2] = { 0, 0 };
    uint32_t running_total[2] = { 0, 0 };
    HttpEnums::ChunkState chunk_state[2] = { HttpEnums::CHUNK_NEWLINES,
        HttpEnums::CHUNK_NEWLINES };
    uint32_t partial_raw_bytes[2] = { 0, 0 };
    uint8_t* partial_buffer[2] = { nullptr, nullptr };
    uint32_t partial_buffer_length[2] = { 0, 0 };
    uint32_t gzip_header_bytes_processed[2] = { 0, 0 };
    HttpEnums::GzipVerificationState gzip_state[2] = { HttpEnums::GZIP_TBD, HttpEnums::GZIP_TBD };
    bool gzip_header_check_done();

    // *** StreamSplitter internal data - scan() => reassemble()
    uint32_t num_excess[2] = { 0, 0 };
    uint32_t num_good_chunks[2] = { 0, 0 };
    uint32_t octets_expected[2] = { 0, 0 };
    bool is_broken_chunk[2] = { false, false };

    // *** StreamSplitter => Inspector (facts about the most recent message section)
    HttpCommon::SectionType section_type[2] = { HttpCommon::SEC__NOT_COMPUTE,
                                                HttpCommon::SEC__NOT_COMPUTE };
    int32_t octets_reassembled[2] = { HttpCommon::STAT_NOT_PRESENT, HttpCommon::STAT_NOT_PRESENT };
    int32_t num_head_lines[2] = { HttpCommon::STAT_NOT_PRESENT, HttpCommon::STAT_NOT_PRESENT };
    bool tcp_close[2] = { false, false };
    bool partial_flush[2] = { false, false };
    uint64_t last_connect_trans_w_early_traffic = 0;

    HttpInfractions* infractions[2] = { new HttpInfractions, new HttpInfractions };
    HttpEventGen* events[2] = { new HttpEventGen, new HttpEventGen };

    // Infractions are associated with a specific message and are stored in the transaction for
    // that message. But StreamSplitter splits the start line before there is a transaction and
    // needs a place to put the problems it finds. Hence infractions are created before there is a
    // transaction to associate them with and stored here until attach_my_transaction() takes them
    // away and resets these to nullptr. The accessor method hides this from StreamSplitter.
    HttpInfractions* get_infractions(HttpCommon::SourceId source_id);

    // *** Inspector => StreamSplitter (facts about the message section that is coming next)
    HttpCommon::SectionType type_expected[2] = { HttpCommon::SEC_REQUEST, HttpCommon::SEC_STATUS };
    bool last_request_was_connect = false;
    bool stretch_section_to_packet[2] = { false, false };
    bool accelerated_blocking[2] = { false, false };
    z_stream* compress_stream[2] = { nullptr, nullptr };
    uint64_t zero_nine_expected = 0;
    // length of the data from Content-Length field
    int64_t data_length[2] = { HttpCommon::STAT_NOT_PRESENT, HttpCommon::STAT_NOT_PRESENT };
    uint32_t section_size_target[2] = { 0, 0 };
    HttpEnums::CompressId compression[2] = { HttpEnums::CMP_NONE, HttpEnums::CMP_NONE };

    // *** Inspector's internal data about the current message
    struct FdCallbackContext
    {
        HttpInfractions* infractions = nullptr;
        HttpEventGen* events = nullptr;
    };
    FdCallbackContext fd_alert_context[2];
    snort::MimeSession* mime_state[2] = { nullptr, nullptr };
    snort::UtfDecodeSession* utf_state[2] = { nullptr, nullptr };
    fd_session_t* fd_state[2] = { nullptr, nullptr };
    int64_t file_depth_remaining[2] = { HttpCommon::STAT_NOT_PRESENT,
        HttpCommon::STAT_NOT_PRESENT };
    int64_t detect_depth_remaining[2] = { HttpCommon::STAT_NOT_PRESENT,
        HttpCommon::STAT_NOT_PRESENT };
    int32_t publish_depth_remaining[2] = { HttpCommon::STAT_NOT_PRESENT,
        HttpCommon::STAT_NOT_PRESENT };
    int32_t file_decomp_buffer_size_remaining[2] = { HttpCommon::STAT_NOT_PRESENT,
        HttpCommon::STAT_NOT_PRESENT };
    uint64_t expected_trans_num[2] = { 1, 1 };

    // number of user data octets seen so far (regular body or chunks)
    int64_t body_octets[2] = { HttpCommon::STAT_NOT_PRESENT, HttpCommon::STAT_NOT_PRESENT };
    // normalized octets forwarded to file or MIME processing
    int64_t file_octets[2] = { HttpCommon::STAT_NOT_PRESENT, HttpCommon::STAT_NOT_PRESENT };
    int32_t publish_octets[2] = { HttpCommon::STAT_NOT_PRESENT, HttpCommon::STAT_NOT_PRESENT };
    uint32_t partial_inspected_octets[2] = { 0, 0 };
    uint8_t* partial_detect_buffer[2] = { nullptr, nullptr };
    uint32_t partial_detect_length[2] = { 0, 0 };
    uint32_t partial_js_detect_length[2] = { 0, 0 };
    std::list<MimeBufs>* partial_mime_bufs[2] = { nullptr, nullptr };
    bool partial_mime_last_complete[2] = { true, true };
    int32_t status_code_num = HttpCommon::STAT_NOT_PRESENT;
    HttpEnums::VersionId version_id[2] = { HttpEnums::VERS__NOT_PRESENT,
                                            HttpEnums::VERS__NOT_PRESENT };
    HttpEnums::MethodId method_id = HttpEnums::METH__NOT_PRESENT;

    // *** Transaction management including pipelining
    static const int MAX_PIPELINE = 100;  // requests seen - responses seen <= MAX_PIPELINE

    HttpTransaction* transaction[2] = { nullptr, nullptr };

    // Transactions with uncleared sections awaiting deletion
    HttpTransaction* discard_list = nullptr;

    HttpTransaction** pipeline = nullptr;
    int16_t pipeline_front = 0;
    int16_t pipeline_back = 0;
    bool pipeline_overflow = false;
    bool pipeline_underflow = false;
    bool add_to_pipeline(HttpTransaction* latest);
    int pipeline_length();
    HttpTransaction* take_from_pipeline();
    void delete_pipeline();

    HttpJSNorm* js_ctx[2] = { nullptr, nullptr };
    HttpJSNorm* js_ctx_mime[2] = { nullptr, nullptr };
    bool cutover_on_clear = false;
    bool ssl_search_abandoned = false;

    // *** HTTP/X handling
    bool for_httpx = false;
    int64_t hx_stream_id = -1;
    HttpCommon::HXBodyState hx_body_state[2] = { HttpCommon::HX_BODY_NOT_COMPLETE,
        HttpCommon::HX_BODY_NOT_COMPLETE };

#ifdef REG_TEST
    static uint64_t instance_count;
    uint64_t seq_num;

    void show(FILE* out_file) const;
#endif
};

#endif

http_inspect_base.h

Path = src/service_inspectors/http_inspect/http_inspect_base.h

#ifndef HTTP_INSPECT_BASE_H
#define HTTP_INSPECT_BASE_H

#include "flow/flow.h"
#include "framework/inspector.h"
#include "main/snort_types.h"

#include "http_common.h"

class SO_PUBLIC HttpInspectBase : public snort::Inspector
{
public:
    virtual ~HttpInspectBase() override = default;

    virtual HttpCommon::SectionType get_type_expected(snort::Flow* flow, HttpCommon::SourceId source_id) const = 0;
    virtual void finish_hx_body(snort::Flow* flow, HttpCommon::SourceId source_id, HttpCommon::HXBodyState state,
        bool clear_partial_buffer) const = 0;
    virtual void set_hx_body_state(snort::Flow* flow, HttpCommon::SourceId source_id, HttpCommon::HXBodyState state) const = 0;
    virtual void eval(snort::Packet* p, HttpCommon::SourceId source_id, const uint8_t* data, uint16_t dsize) = 0;
private:
     using snort::Inspector::eval;
};

#endif

http_inspect.h

Path = src/service_inspectors/http_inspect/http_inspect.h

#ifndef HTTP_INSPECT_H
#define HTTP_INSPECT_H

//-------------------------------------------------------------------------
// HttpInspect class
//-------------------------------------------------------------------------

#include "framework/cursor.h"
#include "framework/pdu_section.h"
#include "helpers/literal_search.h"
#include "log/messages.h"

#include "http_buffer_info.h"
#include "http_common.h"
#include "http_enum.h"
#include "http_field.h"
#include "http_inspect_base.h"
#include "http_module.h"
#include "http_stream_splitter.h"

class HttpApi;
class HttpParam;
class HttpIpsOption;

class HttpInspect : public HttpInspectBase
{
public:
    HttpInspect(const HttpParaList* params_);
    ~HttpInspect() override;

    bool get_buf(snort::InspectionBuffer::Type ibt, snort::Packet* p,
        snort::InspectionBuffer& b) override;
    bool get_buf(unsigned id, snort::Packet* p, snort::InspectionBuffer& b) override;
    const Field& http_get_buf(snort::Packet* p, const HttpBufferInfo& buffer_info) const;
    const Field& http_get_param_buf(Cursor& c, snort::Packet* p,
        const HttpParam& param) const;
    int32_t http_get_num_headers(snort::Packet* p, const HttpBufferInfo& buffer_info) const;
    int32_t http_get_max_header_line(snort::Packet* p, const HttpBufferInfo& buffer_info) const;
    int32_t http_get_num_cookies(snort::Packet* p, const HttpBufferInfo& buffer_info) const;
    HttpEnums::VersionId http_get_version_id(snort::Packet* p,
        const HttpBufferInfo& buffer_info) const;
    HttpCommon::SectionType get_type_expected(snort::Flow* flow, HttpCommon::SourceId source_id) const override;
    void finish_hx_body(snort::Flow* flow, HttpCommon::SourceId source_id, HttpCommon::HXBodyState state,
        bool clear_partial_buffer) const override;
    void set_hx_body_state(snort::Flow* flow, HttpCommon::SourceId source_id, HttpCommon::HXBodyState state) const override;
    bool get_fp_buf(snort::InspectionBuffer::Type ibt, snort::Packet* p,
        snort::InspectionBuffer& b) override;
    bool configure(snort::SnortConfig*) override;
    void show(const snort::SnortConfig*) const override;
    void eval(snort::Packet* p) override;
    void eval(snort::Packet* p, HttpCommon::SourceId source_id, const uint8_t* data, uint16_t dsize) override;
    void clear(snort::Packet* p) override;

    HttpStreamSplitter* get_splitter(bool is_client_to_server) override
    { return &splitter[is_client_to_server ? HttpCommon::SRC_CLIENT : HttpCommon::SRC_SERVER]; }

    bool can_carve_files() const override
    { return true; }

    bool can_start_tls() const override
    { return true; }

    static snort::PduSection get_latest_is(const snort::Packet* p);
    static HttpCommon::SourceId get_latest_src(const snort::Packet* p);
    void disable_detection(snort::Packet* p);

    // Callbacks that provide "extra data"
    static int get_xtra_trueip(snort::Flow*, uint8_t**, uint32_t*, uint32_t*);
    static int get_xtra_uri(snort::Flow*, uint8_t**, uint32_t*, uint32_t*);
    static int get_xtra_host(snort::Flow*, uint8_t** buf, uint32_t* len, uint32_t* type);
    static int get_xtra_jsnorm(snort::Flow*, uint8_t**, uint32_t*, uint32_t*);

    unsigned get_pub_id()
    { return pub_id; }

    const uint8_t* adjust_log_packet(snort::Packet* p, uint16_t& length) override;

private:
    friend HttpApi;
    friend HttpStreamSplitter;
    friend HttpIpsOption;

    HttpStreamSplitter splitter[2] = { { true, this }, { false, this } };

    void process(const uint8_t* data, const uint16_t dsize, snort::Flow* const flow,
        HttpCommon::SourceId source_id_, bool buf_owner, snort::Packet* p) const;
    static HttpFlowData* http_get_flow_data(const snort::Flow* flow);
    static void http_set_flow_data(snort::Flow* flow, HttpFlowData* flow_data);

    const HttpParaList* const params;
    snort::LiteralSearch::Handle* s_handle = nullptr;
    ScriptFinder* script_finder = nullptr;

    // Registrations for "extra data"
    const uint32_t xtra_trueip_id;
    const uint32_t xtra_uri_id;
    const uint32_t xtra_host_id;
    const uint32_t xtra_jsnorm_id;

    unsigned pub_id = 0; // for inspection events
};

#endif

http_js_norm.h

Path = src/service_inspectors/http_inspect/http_js_norm.h

#ifndef HTTP_JS_NORM_H
#define HTTP_JS_NORM_H

#include <cstring>

#include "js_norm/js_norm.h"
#include "js_norm/js_pdf_norm.h"
#include "search_engines/search_tool.h"

#include "http_field.h"
#include "http_flow_data.h"
#include "http_event.h"
#include "http_module.h"

snort::SearchTool* js_create_mpse_open_tag();
snort::SearchTool* js_create_mpse_tag_type();
snort::SearchTool* js_create_mpse_tag_attr();

void js_normalize(const Field& input, Field& output, const HttpParaList*, HttpInfractions*, HttpEventGen*);

class HttpJSNorm
{
public:
    virtual ~HttpJSNorm() {}

    virtual snort::JSNorm& ctx() = 0;

    void link(const void* page, HttpEventGen* http_events_, HttpInfractions* infs)
    { page_start = (const uint8_t*)page; http_events = http_events_; infractions = infs; }

    uint64_t get_trans_num() const
    { return trans_num; }

protected:
    const uint8_t* page_start = nullptr;
    HttpEventGen* http_events = nullptr;
    HttpInfractions* infractions = nullptr;
    uint64_t trans_num = 0;
    bool script_continue = false;
};

class HttpInlineJSNorm : public snort::JSNorm, public HttpJSNorm
{
public:
    HttpInlineJSNorm(JSNormConfig* jsn_config, uint64_t tid, snort::SearchTool* mpse_open_tag,
        snort::SearchTool* mpse_tag_attr, uint32_t gen_id) :
        JSNorm(jsn_config, false, gen_id), mpse_otag(mpse_open_tag),
        mpse_attr(mpse_tag_attr), output_size(0), ext_ref_type(false)
    { trans_num = tid; }

    snort::JSNorm& ctx() override
    { return *this; }

protected:
    bool pre_proc() override;
    bool post_proc(int) override;

private:
    snort::SearchTool* mpse_otag;
    snort::SearchTool* mpse_attr;
    size_t output_size;
    bool ext_ref_type;
};

class HttpExternalJSNorm : public snort::JSNorm, public HttpJSNorm
{
public:
    HttpExternalJSNorm(JSNormConfig* jsn_config, uint64_t tid, uint32_t gen_id) :
        JSNorm(jsn_config, false, gen_id)
    { trans_num = tid; }

    snort::JSNorm& ctx() override
    { return *this; }

protected:
    bool pre_proc() override;
    bool post_proc(int) override;
};

class HttpPDFJSNorm : public snort::PDFJSNorm, public HttpJSNorm
{
public:
    HttpPDFJSNorm(JSNormConfig* jsn_config, uint64_t tid, uint32_t gen_id) :
        PDFJSNorm(jsn_config, gen_id)
    { trans_num = tid; }

    snort::JSNorm& ctx() override
    { return *this; }

protected:
    bool pre_proc() override;
    bool post_proc(int) override;
};

#endif

http_module.h

Path = src/service_inspectors/http_inspect/http_module.h

#ifndef HTTP_MODULE_H
#define HTTP_MODULE_H

#include <bitset>
#include <set>
#include <string>
#include <unordered_set>

#include "framework/module.h"
#include "helpers/literal_search.h"
#include "mime/file_mime_config.h"
#include "profiler/profiler.h"
#include "search_engines/search_tool.h"

#include "http_enum.h"
#include "http_str_to_code.h"

#define HTTP_NAME "http_inspect"
#define HTTP_HELP "HTTP inspector"

namespace snort
{
class Trace;
struct SnortConfig;
}

struct HttpParaList
{
public:
    ~HttpParaList();
    int64_t request_depth = -1;
    int64_t response_depth = -1;

    bool unzip = true;
    bool normalize_utf = true;
    int64_t maximum_host_length = -1;
    int64_t maximum_chunk_length = 0xFFFFFFFF;
    uint16_t maximum_header_length = 4096;
    uint16_t maximum_headers = 200;
    uint16_t maximum_pipelined_requests = 99;
    bool decompress_pdf = false;
    bool decompress_swf = false;
    bool decompress_zip = false;
    bool decompress_vba = false;
    snort::DecodeConfig* mime_decode_conf;
    uint32_t max_mime_attach = 5;
    bool script_detection = false;
    snort::LiteralSearch::Handle* script_detection_handle = nullptr;
    bool publish_request_body = true;

    struct JsNormParam
    {
        ~JsNormParam();

        void configure() const;

        bool normalize_javascript = false;
        int max_javascript_whitespaces = 200;

        mutable snort::SearchTool* mpse_otag = nullptr;
        mutable snort::SearchTool* mpse_type = nullptr;
        mutable snort::SearchTool* mpse_attr = nullptr;
    };
    JsNormParam js_norm_param;

    struct UriParam
    {
        UriParam();
        ~UriParam() { delete[] unicode_map; }

        bool percent_u = false;
        bool utf8 = true;
        bool utf8_bare_byte = false;
        int oversize_dir_length = 300;
        bool iis_unicode = false;
        std::string iis_unicode_map_file;
        int iis_unicode_code_page = 1252;
        uint8_t* unicode_map = nullptr;
        bool iis_double_decode = true;
        bool backslash_to_slash = true;
        bool plus_to_space = true;
        bool simplify_path = true;
        std::bitset<256> bad_characters;
        std::bitset<256> unreserved_char;
        HttpEnums::CharAction uri_char[256];

        static const std::bitset<256> default_unreserved_char;
    };
    UriParam uri_param;

    // This will store list of custom xff headers. These are stored in the
    // order of the header preference. The default header preference only
    // consists of known XFF Headers in the below order
    // 1. X-Forwarded-For
    // 2. True-Client-IP
    // Rest of the custom XFF Headers would be added to this list and will be
    // positioned based on the preference of the headers.
    // As of now, plan is to support a maximum of 8 xff type headers.
    StrCode xff_headers[HttpEnums::MAX_XFF_HEADERS + 1] = {};
    // The below header_list contains the list of known static header along with
    // any custom headers mapped with the their respective Header IDs.
    StrCode header_list[HttpEnums::HEAD__MAX_VALUE + HttpEnums::MAX_CUSTOM_HEADERS + 1] = {};

    std::set<std::string> allowed_methods;
    std::set<std::string> disallowed_methods;

#ifdef REG_TEST
    int64_t print_amount = 1200;

    bool test_input = false;
    bool test_output = false;
    bool print_hex = false;
    bool show_pegs = true;
    bool show_scan = false;
#endif
};

class ScriptFinder
{
public:
    ScriptFinder(snort::LiteralSearch::Handle* h) : handle(h),
        finder(snort::LiteralSearch::instantiate(h, (const uint8_t*)"</SCRIPT>", 9, true, true))
    {}

    ~ScriptFinder() { delete finder; }

    int search(const uint8_t* buf, unsigned len) const
    { return finder->search(handle, buf, len); }

private:
    snort::LiteralSearch::Handle* const handle;
    const snort::LiteralSearch* const finder;
};

class HttpModule : public snort::Module
{
public:
    HttpModule();
    ~HttpModule() override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;
    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    unsigned get_gid() const override { return HttpEnums::HTTP_GID; }
    const snort::RuleMap* get_rules() const override { return http_events; }

    const HttpParaList* get_once_params()
    {
        HttpParaList* ret_val = params;
        params = nullptr;
        return ret_val;
    }

    const PegInfo* get_pegs() const override { return peg_names; }
    PegCount* get_counts() const override { return peg_counts; }
    static void increment_peg_counts(HttpEnums::PEG_COUNT counter)
        { peg_counts[counter]++; }
    static void increment_peg_counts(HttpEnums::PEG_COUNT counter, uint64_t value)
        { peg_counts[counter] += value; }
    static void decrement_peg_counts(HttpEnums::PEG_COUNT counter)
        { peg_counts[counter]--; }
    static PegCount get_peg_counts(HttpEnums::PEG_COUNT counter)
        { return peg_counts[counter]; }

    snort::ProfileStats* get_profile() const override;

    static snort::ProfileStats& get_profile_stats()
    { return http_profile; }

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

#ifdef REG_TEST
    static const PegInfo* get_peg_names() { return peg_names; }
    static const PegCount* get_peg_counts() { return peg_counts; }
    static void reset_peg_counts()
    {
        for (unsigned k=0; k < HttpEnums::PEG_COUNT_MAX; peg_counts[k++] = 0);
    }
#endif

private:
    static const snort::Parameter http_params[];
    static const snort::RuleMap http_events[];
    HttpParaList* params = nullptr;
    static const PegInfo peg_names[];
    static THREAD_LOCAL snort::ProfileStats http_profile;
    static THREAD_LOCAL PegCount peg_counts[];
    snort::LiteralSearch::Handle* const script_detection_handle;
};

#endif

http_msg_body_chunk.h

Path = src/service_inspectors/http_inspect/http_msg_body_chunk.h

#ifndef HTTP_MSG_BODY_CHUNK_H
#define HTTP_MSG_BODY_CHUNK_H

#include "http_common.h"
#include "http_msg_body.h"

//-------------------------------------------------------------------------
// HttpMsgBodyChunk class
//-------------------------------------------------------------------------

class HttpMsgBodyChunk : public HttpMsgBody
{
public:
    HttpMsgBodyChunk(const uint8_t* buffer, const uint16_t buf_size, HttpFlowData* session_data_,
        HttpCommon::SourceId source_id_, bool buf_owner, snort::Flow* flow_,
        const HttpParaList* params_)
        : HttpMsgBody(buffer, buf_size, session_data_, source_id_, buf_owner, flow_, params_) {}
    void update_flow() override;

#ifdef REG_TEST
    void print_section(FILE* output) override;
#endif
};

#endif

http_msg_body_cl.h

Path = src/service_inspectors/http_inspect/http_msg_body_cl.h

#ifndef HTTP_MSG_BODY_CL_H
#define HTTP_MSG_BODY_CL_H

#include "http_common.h"
#include "http_field.h"
#include "http_msg_body.h"
#include "http_msg_section.h"

//-------------------------------------------------------------------------
// HttpMsgBodyCl class
//-------------------------------------------------------------------------

class HttpMsgBodyCl : public HttpMsgBody
{
public:
    HttpMsgBodyCl(const uint8_t* buffer, const uint16_t buf_size, HttpFlowData* session_data_,
        HttpCommon::SourceId source_id_, bool buf_owner, snort::Flow* flow_,
        const HttpParaList* params_)
        : HttpMsgBody(buffer, buf_size, session_data_, source_id_, buf_owner, flow_, params_) {}
    void update_flow() override;

#ifdef REG_TEST
    void print_section(FILE* output) override;
#endif
};

#endif

http_msg_body.h

Path = src/service_inspectors/http_inspect/http_msg_body.h

#ifndef HTTP_MSG_BODY_H
#define HTTP_MSG_BODY_H

#include "file_api/file_api.h"

#include <list>

#include "http_common.h"
#include "http_enum.h"
#include "http_field.h"
#include "http_msg_section.h"

//-------------------------------------------------------------------------
// HttpMsgBody class
//-------------------------------------------------------------------------

class HttpMsgBody : public HttpMsgSection
{
public:
    ~HttpMsgBody() override { delete mime_bufs; }
    void analyze() override;
    snort::PduSection get_inspection_section() const override
        { return snort::PS_BODY; }
    bool detection_required() const override { return (detect_data.length() > 0); }
    bool run_detection(snort::Packet* p) override;
    void clear() override;
    HttpMsgBody* get_body() override { return this; }
    const Field& get_classic_client_body();
    const Field& get_raw_body() { return raw_body; }
    const Field& get_decomp_vba_data();
    const Field& get_norm_js_data();
    const Field& get_detect_data() { return detect_data; }
    const Field& get_msg_text_new() const { return msg_text_new; }
    static void fd_event_callback(void* context, int event);
    bool is_first() { return first_body; }
    void publish(unsigned pub_id) override;
    int32_t get_publish_length() const;

protected:
    HttpMsgBody(const uint8_t* buffer, const uint16_t buf_size, HttpFlowData* session_data_,
        HttpCommon::SourceId source_id_, bool buf_owner, snort::Flow* flow_,
        const HttpParaList* params_);

    int64_t body_octets;
    bool first_body;

#ifdef REG_TEST
    void print_body_section(FILE* output, const char* body_type_str);
#endif

private:
    void do_file_processing(const Field& file_data);
    void do_utf_decoding(const Field& input, Field& output);
    void do_file_decompression(const Field& input, Field& output);
    void do_legacy_js_normalization(const Field& input, Field& output);
    HttpJSNorm* acquire_js_ctx();
    HttpJSNorm* acquire_js_ctx_mime();
    void clear_js_ctx_mime();

    void clean_partial(uint32_t& partial_inspected_octets, uint32_t& partial_detect_length,
        uint8_t*& partial_detect_buffer,  uint32_t& partial_js_detect_length);
    void bookkeeping_regular_flush(uint32_t& partial_detect_length,
        uint8_t*& partial_detect_buffer, uint32_t& partial_js_detect_length,
        int32_t detect_length);
    void get_file_info( FileDirection dir, const uint8_t*& filename_buffer,
        uint32_t& filename_length, const uint8_t*& uri_buffer, uint32_t& uri_length);
    void get_ole_data();

    Field msg_text_new;
    Field decoded_body;
    Field raw_body;              // request_depth or response_depth applied
    Field decompressed_file_body;
    Field cumulative_data;
    Field js_norm_body;
    Field detect_data;
    Field norm_js_data;
    Field classic_client_body;   // URI normalization applied

    // MIME buffers
    Field decompressed_vba_data;
    Field ole_data;
    std::list<MimeBufs>* mime_bufs = nullptr;
    bool last_attachment_complete = true;

    int32_t publish_length = HttpCommon::STAT_NOT_PRESENT;
};

#endif

http_msg_body_hx.h

Path = src/service_inspectors/http_inspect/http_msg_body_hx.h

#ifndef HTTP_MSG_BODY_HX_H
#define HTTP_MSG_BODY_HX_H

#include "http_common.h"
#include "http_msg_body.h"

//-------------------------------------------------------------------------
// HttpMsgBodyHX class
//-------------------------------------------------------------------------

class HttpMsgBodyHX : public HttpMsgBody
{
public:
    HttpMsgBodyHX(const uint8_t* buffer, const uint16_t buf_size, HttpFlowData* session_data_,
        HttpCommon::SourceId source_id_, bool buf_owner, snort::Flow* flow_,
        const HttpParaList* params_)
        : HttpMsgBody(buffer, buf_size, session_data_, source_id_, buf_owner, flow_, params_) {}
    void update_flow() override;

#ifdef REG_TEST
    void print_section(FILE* output) override;
#endif
};

#endif

http_msg_body_old.h

Path = src/service_inspectors/http_inspect/http_msg_body_old.h

#ifndef HTTP_MSG_BODY_OLD_H
#define HTTP_MSG_BODY_OLD_H

#include "http_common.h"
#include "http_field.h"
#include "http_msg_body.h"
#include "http_msg_section.h"

//-------------------------------------------------------------------------
// HttpMsgBodyOld class
//-------------------------------------------------------------------------

class HttpMsgBodyOld : public HttpMsgBody
{
public:
    HttpMsgBodyOld(const uint8_t* buffer, const uint16_t buf_size, HttpFlowData* session_data_,
        HttpCommon::SourceId source_id_, bool buf_owner, snort::Flow* flow_,
        const HttpParaList* params_)
        : HttpMsgBody(buffer, buf_size, session_data_, source_id_, buf_owner, flow_, params_) {}
    void update_flow() override;

#ifdef REG_TEST
    void print_section(FILE* output) override;
#endif
};

#endif

http_msg_header.h

Path = src/service_inspectors/http_inspect/http_msg_header.h

#ifndef HTTP_MSG_HEADER_H
#define HTTP_MSG_HEADER_H

#include "file_api/file_api.h"
#include "mime/file_mime_process.h"

#include "http_common.h"
#include "http_enum.h"
#include "http_field.h"
#include "http_msg_head_shared.h"

//-------------------------------------------------------------------------
// HttpMsgHeader class
//-------------------------------------------------------------------------

class HttpMsgHeader : public HttpMsgHeadShared
{
public:
    HttpMsgHeader(const uint8_t* buffer, const uint16_t buf_size, HttpFlowData* session_data_,
        HttpCommon::SourceId source_id_, bool buf_owner, snort::Flow* flow_,
        const HttpParaList* params_);
    snort::PduSection get_inspection_section() const override
        { return snort::PS_HEADER; }
    bool detection_required() const override { return true; }
    void update_flow() override;
    void gen_events() override;
    void publish(unsigned pub_id) override;
    const Field& get_true_ip();
    const Field& get_true_ip_addr();
    int32_t get_num_cookies();

    bool has_mime_boundary() const
    { return mime_boundary_found; }

    // The multi_file_processing_id is unique for each file transferred within a single connection
    // and is used by file processing to store partially processed file contexts in the flow data.
    void set_multi_file_processing_id(const uint64_t transaction_id, const uint32_t stream_id);
    uint64_t get_multi_file_processing_id() { return multi_file_processing_id; }

private:
    void prepare_body();
    void setup_mime();
    void setup_file_processing();
    void setup_encoding_decompression();
    void setup_utf_decoding();
    void setup_file_decompression();

    // Dummy configurations to support MIME processing
    snort::MailLogConfig mime_conf;

    bool mime_boundary_found = false;

    Field true_ip;
    Field true_ip_addr;
    int32_t num_cookies = HttpCommon::STAT_NOT_COMPUTE;

    uint64_t multi_file_processing_id = 0;

#ifdef REG_TEST
    void print_section(FILE* output) override;
#endif
};

#endif

http_msg_head_shared.h

Path = src/service_inspectors/http_inspect/http_msg_head_shared.h

#ifndef HTTP_MSG_HEAD_SHARED_H
#define HTTP_MSG_HEAD_SHARED_H

#include <bitset>

#include "http_common.h"
#include "http_enum.h"
#include "http_field.h"
#include "http_msg_section.h"
#include "http_normalized_header.h"
#include "http_str_to_code.h"

//-------------------------------------------------------------------------
// HttpMsgHeadShared class
//-------------------------------------------------------------------------

class HttpMsgHeadShared : public HttpMsgSection
{
public:
    void analyze() override;

    const Field& get_classic_raw_cookie();
    const Field& get_classic_norm_header();
    const Field& get_classic_norm_cookie();
    const Field& get_header_value_raw(HttpEnums::HeaderId header_id) const;
    const Field& get_all_header_values_raw(HttpEnums::HeaderId header_id);
    const Field& get_header_value_norm(HttpEnums::HeaderId header_id);
    int get_header_count(HttpEnums::HeaderId header_id) const;

    // Tables of header field names and header value names
    static const StrCode header_list[];
    static const StrCode content_code_list[];
    static const StrCode content_type_list[];
    static const StrCode charset_code_list[];
    static const StrCode charset_code_opt_list[];
    static const StrCode transfer_encoding_list[];
    static const StrCode upgrade_list[];

    // The file_cache_index is used along with the source ip and destination ip to cache file
    // verdicts.
    uint64_t get_file_cache_index();
    const Field& get_content_disposition_filename();
    int32_t get_num_headers() const { return num_headers; }
    int32_t get_max_header_line() const { return max_header_line; }
    int32_t get_content_type();

protected:
    HttpMsgHeadShared(const uint8_t* buffer, const uint16_t buf_size,
        HttpFlowData* session_data_, HttpCommon::SourceId source_id_, bool buf_owner, snort::Flow* flow_,
        const HttpParaList* params_);
    ~HttpMsgHeadShared() override;
    // Get the next item in a comma-separated header value and convert it to an enum value
    static int32_t get_next_code(const Field& field, int32_t& offset, const StrCode table[]);
    // Do a case insensitive search for "boundary=" in a Field
    static bool boundary_present(const Field& field);

    // All of these are indexed by the relative position of the header field in the message

    Field* header_line = nullptr;
    HttpEnums::HeaderId* header_name_id = nullptr;
    int32_t num_headers = HttpCommon::STAT_NOT_COMPUTE;
    int32_t max_header_line = HttpCommon::STAT_NOT_COMPUTE;

#ifdef REG_TEST
    void print_headers(FILE* output);
#endif

private:
    static const int MAX = HttpEnums::HEAD__MAX_VALUE + HttpEnums::MAX_CUSTOM_HEADERS;

    void parse_header_block();
    int32_t find_next_header(const uint8_t* buffer, int32_t length, int32_t& num_seps);
    void parse_header_lines();
    void create_norm_head_list();
    void derive_header_name_id(int index);
    const Field& get_classic_raw_header();

    Field classic_raw_header;    // raw headers with cookies spliced out
    Field classic_norm_header;   // URI normalization applied
    Field classic_norm_cookie;   // URI normalization applied to concatenated cookie values
    Field* header_name = nullptr;
    Field* header_value = nullptr;

    NormalizedHeader* get_header_node(HttpEnums::HeaderId k) const;
    NormalizedHeader* norm_heads = nullptr;

    std::bitset<MAX> headers_present = 0;

    void extract_filename_from_content_disposition();
    Field content_disposition_filename;
    uint64_t file_cache_index = 0;
    bool file_cache_index_computed = false;

    bool own_msg_buffer;
    int32_t content_type = HttpCommon::STAT_NOT_COMPUTE;
};

#endif

http_msg_request.h

Path = src/service_inspectors/http_inspect/http_msg_request.h

#ifndef HTTP_MSG_REQUEST_H
#define HTTP_MSG_REQUEST_H

#include "http_common.h"
#include "http_enum.h"
#include "http_msg_start.h"
#include "http_query_parser.h"
#include "http_str_to_code.h"
#include "http_uri.h"
#include "http_uri_norm.h"

//-------------------------------------------------------------------------
// HttpMsgRequest class
//-------------------------------------------------------------------------

class HttpMsgRequest : public HttpMsgStart
{
public:
    HttpMsgRequest(const uint8_t* buffer, const uint16_t buf_size, HttpFlowData* session_data_,
        HttpCommon::SourceId source_id_, bool buf_owner, snort::Flow* flow_,
        const HttpParaList* params_);
    ~HttpMsgRequest() override;
    bool detection_required() const override
        { return version_id == HttpEnums::VERS_0_9; }
    snort::PduSection get_inspection_section() const override
        { return snort::PS_HEADER; }
    void gen_events() override;
    void update_flow() override;
    void publish(unsigned pub_id) override;

    const Field& get_method() { return method; }
    const Field& get_uri();
    const Field& get_uri_norm_classic();
    std::string get_aux_ip();
    HttpUri* get_http_uri() { return uri; }
    ParameterMap& get_query_params();
    ParameterMap& get_body_params();

    static bool is_webdav(HttpEnums::MethodId method)
    {
        if(method > HttpEnums::MethodId::METH__WEBDAV_LOW and
           method < HttpEnums::MethodId::METH__WEBDAV_HIGH)
        {
            return true;
        }

        return false;
    }

#ifdef REG_TEST
    void print_section(FILE* output) override;
#endif

private:
    static const StrCode method_list[];

    void parse_start_line() override;
    bool http_name_nocase_ok(const uint8_t* start);
    bool handle_zero_nine();

    Field method;
    HttpUri* uri = nullptr;

    ParameterMap* query_params = nullptr;
    ParameterMap* body_params = nullptr;
};

#endif

http_msg_section.h

Path = src/service_inspectors/http_inspect/http_msg_section.h

#ifndef HTTP_MSG_SECTION_H
#define HTTP_MSG_SECTION_H

#include "detection/detection_util.h"
#include "framework/cursor.h"
#include "framework/pdu_section.h"
#include "protocols/packet.h"

#include "http_buffer_info.h"
#include "http_common.h"
#include "http_cursor_data.h"
#include "http_enum.h"
#include "http_field.h"
#include "http_flow_data.h"
#include "http_module.h"
#include "http_param.h"
#include "http_transaction.h"

//-------------------------------------------------------------------------
// HttpMsgSection class
//-------------------------------------------------------------------------

class HttpMsgSection
{
public:
    virtual ~HttpMsgSection() = default;
    virtual snort::PduSection get_inspection_section() const
        { return snort::PS_NONE; }
    virtual bool detection_required() const = 0;
    HttpCommon::SourceId get_source_id() const { return source_id; }
    HttpTransaction* get_transaction() const { return transaction; }
    const HttpParaList* get_params() const { return params; }

    HttpMsgRequest* get_request() const { return request; }
    HttpMsgStatus* get_status() const { return status; }
    HttpMsgHeader* get_header(HttpCommon::SourceId source_id) const { return header[source_id]; }
    HttpMsgTrailer* get_trailer(HttpCommon::SourceId source_id) const
        { return trailer[source_id]; }
    virtual HttpMsgBody* get_body() { return nullptr; }

    // Minimum necessary processing for every message
    virtual void analyze() = 0;

    // analyze() generates many events in the course of its work. Many other events are generated
    // by JIT normalization but only if someone asks for the item in question. gen_events()
    // addresses a third category--things that do not come up during analysis but must be
    // inspected for every message even if no one else asks about them.
    virtual void gen_events() {}

    // Manages the splitter and communication between message sections
    virtual void update_flow() = 0;

    // Publish an inspection event for other modules to consume
    virtual void publish(unsigned /*pub_id*/) {}

    // Call the detection engine to inspect the current packet
    virtual bool run_detection(snort::Packet* p);

    const Field& get_classic_buffer(unsigned id, uint64_t sub_id, uint64_t form);
    const Field& get_classic_buffer(const HttpBufferInfo& buf);
    const Field& get_param_buffer(Cursor& c, const HttpParam& param);

    HttpEnums::MethodId get_method_id() const { return method_id; }

    int32_t get_status_code_num() const { return status_code_num; }

    virtual void clear();
    bool is_clear() { return cleared; }

    uint64_t get_transaction_id() { return trans_num; }
    int32_t get_num_headers(const HttpBufferInfo& buf) const;
    int32_t get_max_header_line(const HttpBufferInfo& buf) const;
    int32_t get_num_cookies(const HttpBufferInfo& buf) const;
    HttpEnums::VersionId get_version_id(const HttpBufferInfo& buf) const;

    HttpMsgSection* next = nullptr;

#ifdef REG_TEST
    // Test tool prints all derived message parts
    virtual void print_section(FILE* output) = 0;
#endif

protected:
    HttpMsgSection(const uint8_t* buffer, const uint16_t buf_size, HttpFlowData* session_data_,
        HttpCommon::SourceId source_id_, bool buf_owner, snort::Flow* flow_, const HttpParaList*
        params_);

    void get_related_sections();

    const Field msg_text;
    HttpFlowData* const session_data;
    snort::Flow* const flow;
    const HttpParaList* const params;
    HttpTransaction* const transaction;
    uint64_t trans_num;
    int32_t status_code_num;
    const HttpCommon::SourceId source_id;
    HttpEnums::VersionId version_id;
    HttpEnums::MethodId method_id;
    const bool tcp_close;
    bool cleared = false;

    // Pointers to related message sections in the same transaction
    HttpMsgRequest* request = nullptr;
    HttpMsgStatus* status = nullptr;
    HttpMsgHeader* header[2] = {nullptr, nullptr};
    HttpMsgTrailer* trailer[2] = {nullptr, nullptr};


    // Convenience methods shared by multiple subclasses
    void add_infraction(int infraction);
    void create_event(int sid);
    void update_depth() const;
    static const Field& classic_normalize(const Field& raw, Field& norm,
        bool do_path, const HttpParaList::UriParam& uri_param);
#ifdef REG_TEST
    void print_section_title(FILE* output, const char* title) const;
    void print_section_wrapup(FILE* output) const;
    void print_peg_counts(FILE* output) const;
#endif
};

#endif

http_msg_start.h

Path = src/service_inspectors/http_inspect/http_msg_start.h

#ifndef HTTP_MSG_START_H
#define HTTP_MSG_START_H

#include "http_common.h"
#include "http_field.h"
#include "http_msg_section.h"

//-------------------------------------------------------------------------
// HttpMsgStart class
//-------------------------------------------------------------------------

class HttpMsgStart : public HttpMsgSection
{
public:
    void analyze() override;
    bool detection_required() const override { return false; }
    const Field& get_version() const { return version; }
    HttpEnums::VersionId get_version_id() const { return version_id; }

protected:
    HttpMsgStart(const uint8_t* buffer, const uint16_t buf_size, HttpFlowData* session_data_,
        HttpCommon::SourceId source_id_, bool buf_owner, snort::Flow* flow_,
        const HttpParaList* params_);
    ~HttpMsgStart() override;
    virtual void parse_start_line() = 0;
    void derive_version_id();

    Field start_line;
    Field version;
    bool own_msg_buffer;
};

#endif

http_msg_status.h

Path = src/service_inspectors/http_inspect/http_msg_status.h

#ifndef HTTP_MSG_STATUS_H
#define HTTP_MSG_STATUS_H

#include "http_common.h"
#include "http_field.h"
#include "http_msg_start.h"

//-------------------------------------------------------------------------
// HttpMsgStatus class
//-------------------------------------------------------------------------

class HttpMsgStatus : public HttpMsgStart
{
public:
    HttpMsgStatus(const uint8_t* buffer, const uint16_t buf_size, HttpFlowData* session_data_,
        HttpCommon::SourceId source_id_, bool buf_owner, snort::Flow* flow_,
        const HttpParaList* params_);
    void gen_events() override;
    void update_flow() override;

    const Field& get_status_code() { return status_code; }
    const Field& get_reason_phrase() { return reason_phrase; }

#ifdef REG_TEST
    void print_section(FILE* output) override;
#endif

private:
    void parse_start_line() override;
    void derive_status_code_num();

    Field status_code;
    Field reason_phrase;
};

#endif

http_msg_trailer.h

Path = src/service_inspectors/http_inspect/http_msg_trailer.h

#ifndef HTTP_MSG_TRAILER_H
#define HTTP_MSG_TRAILER_H

#include "http_common.h"
#include "http_enum.h"
#include "http_msg_head_shared.h"

//-------------------------------------------------------------------------
// HttpMsgTrailer class
//-------------------------------------------------------------------------

class HttpMsgTrailer : public HttpMsgHeadShared
{
public:
    HttpMsgTrailer(const uint8_t* buffer, const uint16_t buf_size, HttpFlowData* session_data_,
        HttpCommon::SourceId source_id_, bool buf_owner, snort::Flow* flow_,
        const HttpParaList* params_);
    snort::PduSection get_inspection_section() const override
        { return snort::PS_TRAILER; }
    bool detection_required() const override { return (msg_text.length() > 0); }
    void gen_events() override;
    void update_flow() override;

#ifdef REG_TEST
    void print_section(FILE* output) override;
#endif
};

#endif

http_normalized_header.h

Path = src/service_inspectors/http_inspect/http_normalized_header.h

#ifndef HTTP_NORMALIZED_HEADER_H
#define HTTP_NORMALIZED_HEADER_H

#include "http_event.h"
#include "http_field.h"

class HttpMsgHeadShared;

//-------------------------------------------------------------------------
// NormalizedHeader class
//-------------------------------------------------------------------------

class NormalizedHeader
{
public:
    NormalizedHeader(NormalizedHeader* next_, int32_t count_, HttpEnums::HeaderId id_) :
        next(next_), count(count_), id(id_) {}
    const Field& get_norm(HttpInfractions* infractions, HttpEventGen* events,
        const HttpEnums::HeaderId header_name_id[], const Field header_value[],
        const int32_t num_headers);
    const Field& get_comma_separated_raw(const HttpMsgHeadShared& msg_head, HttpInfractions* infractions,
        HttpEventGen* events, const HttpEnums::HeaderId header_name_id[], const Field header_value[],
        const int32_t num_headers);

    NormalizedHeader* next;
    int32_t count;
    const HttpEnums::HeaderId id;

private:
    // Header normalization strategies. There should be one defined for every different way we can
    // process a header field value.
    class HeaderNormalizer;
    static const HeaderNormalizer NORMALIZER_BASIC;
    static const HeaderNormalizer NORMALIZER_HOST;
    static const HeaderNormalizer NORMALIZER_CASE_INSENSITIVE;
    static const HeaderNormalizer NORMALIZER_NUMBER;
    static const HeaderNormalizer NORMALIZER_TOKEN_LIST;
    static const HeaderNormalizer NORMALIZER_METHOD_LIST;
    static const HeaderNormalizer NORMALIZER_DATE;
    static const HeaderNormalizer NORMALIZER_URI;
    static const HeaderNormalizer NORMALIZER_CONTENT_LENGTH;
    static const HeaderNormalizer NORMALIZER_CHARSET;

    // Master table of known header fields and their normalization strategies.
    static const HeaderNormalizer* const header_norms[];

    Field norm;
    Field comma_separated_raw;
};

#endif

http_normalizers.h

Path = src/service_inspectors/http_inspect/http_normalizers.h

#ifndef HTTP_NORMALIZERS_H
#define HTTP_NORMALIZERS_H

#include "http_event.h"
#include "http_field.h"
#include "http_str_to_code.h"

// There are currently no normalization functions that make header values bigger. Changes are
// required to HeaderNormalizer::normalize() to allocate more space before you can introduce a
// normalizer that may expand a header value.
//
// Normalization functions must return an output buffer with nonnegative length. Status codes are
// not acceptable.
typedef int32_t (NormFunc)(const uint8_t*, int32_t, uint8_t*, HttpInfractions*, HttpEventGen*);
NormFunc norm_to_lower;
NormFunc norm_remove_lws;
NormFunc norm_remove_quotes_lws;

// Other normalization-related utilities
void get_last_token(const Field& input, Field& last_token, char ichar);
int64_t norm_decimal_integer(const Field& input);
bool has_consecutive_commas(const Field& input);

#endif

http_param.h

Path = src/service_inspectors/http_inspect/http_param.h

#ifndef HTTP_PARAM_H
#define HTTP_PARAM_H

#include <algorithm>
#include <cassert>
#include <string>

#include "helpers/literal_search.h"

class HttpParam
{
public:
    HttpParam(const std::string& param_, bool nocase_,
        snort::LiteralSearch::Handle* search_handle_)
        : param(param_), param_upper(param_), nocase(nocase_),
          search_handle(search_handle_)
    {
        assert(param.length() > 0);

        std::transform(param_upper.begin(), param_upper.end(),
            param_upper.begin(), ::toupper);

        const uint8_t* pattern = (const uint8_t*)param_upper.c_str();
        unsigned pattern_length = param_upper.length();

        searcher = snort::LiteralSearch::instantiate(
            search_handle, pattern, pattern_length, true
        );
    }

    ~HttpParam()
    {
        delete searcher;
    }

    bool operator==(const HttpParam& rhs) const
    { return param == rhs.param && nocase == rhs.nocase; }

    const std::string& str() const
    { return param; }

    const std::string& str_upper() const
    { return param_upper; }

    const char* c_str() const
    { return param.c_str(); }

    unsigned length() const
    { return param.length(); }

    bool is_nocase() const
    { return nocase; }

    int search_nocase(const uint8_t* buffer, unsigned buffer_len) const
    {
        assert(searcher);
        return searcher->search(search_handle, buffer, buffer_len);
    }

private:
    std::string param;
    std::string param_upper;
    bool nocase = false;
    snort::LiteralSearch* searcher = nullptr;
    snort::LiteralSearch::Handle* search_handle = nullptr;
};

#endif

http_query_parser.h

Path = src/service_inspectors/http_inspect/http_query_parser.h

#ifndef HTTP_QUERY_PARSER_H
#define HTTP_QUERY_PARSER_H

#include <string>
#include <unordered_map>
#include <vector>

#include "http_common.h"
#include "http_field.h"
#include "http_flow_data.h"
#include "http_module.h"
#include "http_param.h"

struct KeyValue
{
    Field key;
    Field value;
};

typedef std::vector<KeyValue*> KeyValueVec;

class ParameterData
{
public:
    ParameterData() = default;

    ~ParameterData()
    {
        for ( KeyValue* kv : kv_vec )
            delete kv;
    }

public:
    KeyValueVec kv_vec;
    bool parsed = false;
};

typedef std::unordered_map<std::string, ParameterData> ParameterMap;

class HttpQueryParser
{
public:
    HttpQueryParser(const uint8_t* buffer_, unsigned buffer_len_,
        const uint8_t* norm_buffer_, unsigned norm_buffer_len_,
        const HttpParaList::UriParam& uri_config_,
        HttpFlowData* session_data_,
        HttpCommon::SourceId source_id_)
        : buffer(buffer_), buffer_len(buffer_len_),
          norm_buffer(norm_buffer_), norm_buffer_len(norm_buffer_len_),
          uri_config(uri_config_), session_data(session_data_),
          source_id(source_id_) {}

    void parse(const HttpParam& param, ParameterData& data);

    struct Parameter
    {
        const uint8_t* key;
        const uint8_t* value;
        unsigned key_len;
        unsigned value_len;
    };

private:
    void create_event(int sid);

    void unescape(const Field& raw, Field& norm);

    bool parse_parameter(Parameter& p);
    bool parse_key(Parameter& p);
    bool parse_value(Parameter& p);

    const uint8_t* buffer;
    unsigned buffer_len;

    const uint8_t* norm_buffer;
    unsigned norm_buffer_len;

    unsigned index = 0;

    static const unsigned MAX_REPEAT_PARAMS = 100;

    const HttpParaList::UriParam& uri_config;
    HttpFlowData* const session_data;
    const HttpCommon::SourceId source_id;
};

#endif

http_stream_splitter_base.h

Path = src/service_inspectors/http_inspect/http_stream_splitter_base.h

#ifndef HTTP_STREAM_SPLITTER_BASE_H
#define HTTP_STREAM_SPLITTER_BASE_H

#include "main/snort_types.h"
#include "stream/stream_splitter.h"

class SO_PUBLIC HttpStreamSplitterBase : public snort::StreamSplitter
{
public:
    virtual ~HttpStreamSplitterBase() override = default;

    virtual void prep_partial_flush(snort::Flow* flow, uint32_t num_flush) = 0;

    virtual Status scan(snort::Flow* flow, const uint8_t* data, uint32_t length, uint32_t* flush_offset) = 0;
protected:
    HttpStreamSplitterBase(bool c2s) : StreamSplitter(c2s) { }
private:
    using snort::StreamSplitter::scan;
};

#endif

http_stream_splitter.h

Path = src/service_inspectors/http_inspect/http_stream_splitter.h

#ifndef HTTP_STREAM_SPLITTER_H
#define HTTP_STREAM_SPLITTER_H

#include <zlib.h>

#include "http_common.h"
#include "http_enum.h"
#include "http_flow_data.h"
#include "http_stream_splitter_base.h"
#include "http_test_manager.h"

class HttpInspect;

class HttpStreamSplitter : public HttpStreamSplitterBase
{
public:
    HttpStreamSplitter(bool is_client_to_server, HttpInspect* my_inspector_) :
        HttpStreamSplitterBase(is_client_to_server),
        my_inspector(my_inspector_),
        source_id(is_client_to_server ? HttpCommon::SRC_CLIENT : HttpCommon::SRC_SERVER) {}
    Status scan(snort::Packet* pkt, const uint8_t* data, uint32_t length, uint32_t not_used,
        uint32_t* flush_offset) override;
    Status scan(snort::Flow* flow, const uint8_t* data, uint32_t length, uint32_t* flush_offset) override;
    const snort::StreamBuffer reassemble(snort::Flow* flow, unsigned total, unsigned, const
        uint8_t* data, unsigned len, uint32_t flags, unsigned& copied) override;
    bool finish(snort::Flow* flow) override;
    void prep_partial_flush(snort::Flow* flow, uint32_t num_flush) override;
    bool is_paf() override { return true; }
    static StreamSplitter::Status status_value(StreamSplitter::Status ret_val, bool http2 = false);

    // FIXIT-M should return actual packet buffer size
    unsigned max(snort::Flow*) override { return HttpEnums::MAX_OCTETS; }
    void go_away() override {}

private:
    void prepare_flush(HttpFlowData* session_data, uint32_t* flush_offset, HttpCommon::SectionType
        section_type, uint32_t num_flushed, uint32_t num_excess, int32_t num_head_lines,
        bool is_broken_chunk, uint32_t num_good_chunks, uint32_t octets_seen)
        const;
    HttpCutter* get_cutter(HttpCommon::SectionType type, HttpFlowData* session) const;
    void chunk_spray(HttpFlowData* session_data, uint8_t* buffer, const uint8_t* data,
        unsigned length) const;
    void decompress_copy(uint8_t* buffer, uint32_t& offset, const uint8_t* data,
        uint32_t length, HttpEnums::CompressId& compression, z_stream*& compress_stream,
        bool at_start, HttpInfractions* infractions, HttpEventGen* events,
        HttpFlowData* session_data) const;
    uint8_t* process_gzip_header(const uint8_t* data,
        uint32_t length, HttpFlowData* session_data) const;
    bool gzip_header_check_done(HttpFlowData* session_data) const;
    StreamSplitter::Status handle_zero_nine(snort::Flow*, HttpFlowData*, const uint8_t* data,
        uint32_t length, uint32_t* flush_offset, HttpCommon::SectionType&, HttpCutter*&);
    StreamSplitter::Status call_cutter(snort::Flow*, HttpFlowData*, const uint8_t* data,
        uint32_t length, uint32_t* flush_offset, HttpCommon::SectionType&);

    HttpInspect* const my_inspector;
    const HttpCommon::SourceId source_id;
};

#endif

http_str_to_code.h

Path = src/service_inspectors/http_inspect/http_str_to_code.h

#ifndef HTTP_STR_TO_CODE_H
#define HTTP_STR_TO_CODE_H

#include <cstdint>

struct StrCode
{
    int32_t code;
    const char* name;
};

int32_t str_to_code(const char* text, const StrCode table[]);
int32_t str_to_code(const uint8_t* text, const int32_t text_len, const StrCode table[]);
int32_t substr_to_code(const uint8_t* text, const int32_t text_len, const StrCode table[]);

// Convert the first value in a comma-separated list into a code. consumed is the number of bytes
// used from the list or -1 if there are no more list entries.
int32_t get_code_from_token_list(const uint8_t* token_list, const int32_t text_len,
    int32_t& bytes_consumed, const StrCode table[]);

#endif

http_test_input.h

Path = src/service_inspectors/http_inspect/http_test_input.h

#ifndef HTTP_TEST_INPUT_H
#define HTTP_TEST_INPUT_H

#ifdef REG_TEST

#include <cstdio>
#include <queue>

#include "http_common.h"
#include "http_enum.h"
#include "http_flow_data.h"

class HttpTestInput
{
public:
    HttpTestInput(const char* fileName);
    ~HttpTestInput();
    void scan(uint8_t*& data, uint32_t& length, HttpCommon::SourceId source_id, uint64_t seq_num);
    void flush(uint32_t num_octets);
    void reassemble(uint8_t** buffer, unsigned& length, unsigned& total, unsigned& offset,
        uint32_t& flags, HttpCommon::SourceId source_id, bool& tcp_close);
    bool finish();

private:
    FILE* test_data_file;
    // FIXIT-E Figure out how big this buf needs to be and revise value
    uint8_t msg_buf[2][2 * HttpEnums::MAX_OCTETS] = {{0}, {0}};
    std::queue<uint32_t> segments[2];
    FILE* include_file[2] = { nullptr, nullptr };

    // break command has been read and we are waiting for a new underlying flow to start
    bool need_break = false;

    // Sequence number of the underlying flow we are currently piggybacking on
    uint64_t curr_seq_num = 0;

    // data has been flushed and must be sent by reassemble() before more data may be given to
    // scan()
    bool flushed = false;

    // current direction of traffic flow. Toggled by commands in file.
    HttpCommon::SourceId last_source_id = HttpCommon::SRC_CLIENT;

    // reassemble() just completed and all flushed octets forwarded, time to resume scan()
    bool just_flushed = false;

    // TCP connection directional close
    bool tcp_closed = false;

    // number of octets that have been flushed and must be sent by reassemble
    uint32_t flush_octets = 0;

    // Number of octets sent in previous calls to reassemble()
    uint32_t reassembled_octets = 0;

    // number of characters in the buffer previously shown to splitter but not flushed yet
    uint32_t previous_offset[2] = { 0, 0 };

    // number of characters in the buffer
    uint32_t end_offset[2] = { 0, 0 };

    void generate_h2_frame_header(const char command_value[], const unsigned command_length);

    void reset();
};

#endif
#endif

http_test_manager.h

Path = src/service_inspectors/http_inspect/http_test_manager.h

#ifndef HTTP_TEST_MANAGER_H
#define HTTP_TEST_MANAGER_H

#if defined(REG_TEST) || defined(UNIT_TEST)

#include <sys/types.h>
#include <cstdio>

#include <main/snort_types.h>
//-------------------------------------------------------------------------
// HttpTestManager class
//-------------------------------------------------------------------------

class HttpTestInput;

class SO_PUBLIC HttpTestManager
{
public:
    // Bitmap: 1, 2, 4, 8, ...
    enum INPUT_TYPE { IN_NONE = 0, IN_HTTP = 0x1, IN_HTTP2 = 0x2, IN_HTTP3 = 0x3 };

    static bool use_test_input(INPUT_TYPE type) { return (type & test_input) != 0; }
    static void activate_test_input(INPUT_TYPE type);
    static void activate_test_output(INPUT_TYPE type) { test_output |= type; }
    static HttpTestInput* get_test_input_source() { return test_input_source; }
    static void update_test_number(int64_t new_test_number);
    static bool use_test_output(INPUT_TYPE type)
        { return (test_output & type) || (test_input & type); }
    static FILE* get_output_file() { return (test_out != nullptr) ? test_out : stdout; }
    static int64_t get_test_number() { return test_number; }
    static void set_print_amount(long print_amount_) { print_amount = print_amount_; }
    static long get_print_amount() { return print_amount; }
    static void set_print_hex(bool print_hex_) { print_hex = print_hex_; }
    static bool get_print_hex() { return print_hex; }
    static void set_show_pegs(bool show_pegs_) { show_pegs = show_pegs_; }
    static bool get_show_pegs() { return show_pegs; }
    static void set_show_scan(bool show_scan_) { show_scan = show_scan_; }
    static bool get_show_scan() { return show_scan; }

private:
    HttpTestManager() = delete;

    static unsigned test_input;
    static HttpTestInput* test_input_source;

    // Printing results of message processing
    static unsigned test_output;
    static const char* test_output_prefix;
    static FILE* test_out;
    static int64_t test_number;
    static long print_amount;
    static bool print_hex;
    static bool show_pegs;
    static bool show_scan;
};

#endif
#endif

http_transaction.h

Path = src/service_inspectors/http_inspect/http_transaction.h

#ifndef TRANSACTION_H
#define TRANSACTION_H

#include "http_common.h"
#include "http_enum.h"
#include "http_event.h"
#include "http_flow_data.h"

class HttpMsgRequest;
class HttpMsgStatus;
class HttpMsgHeader;
class HttpMsgTrailer;
class HttpMsgSection;
class HttpMsgBody;
class HttpMsgHeadShared;

class HttpTransaction
{
public:
    ~HttpTransaction();
    static HttpTransaction* attach_my_transaction(HttpFlowData* session_data,
        HttpCommon::SourceId source_id);
    static void delete_transaction(HttpTransaction* transaction, HttpFlowData* session_data);

    HttpMsgRequest* get_request() const { return request; }
    void set_request(HttpMsgRequest* request_) { request = request_; }

    HttpMsgStatus* get_status() const { return status; }
    void set_status(HttpMsgStatus* status_) { status = status_; }

    HttpMsgHeader* get_header(HttpCommon::SourceId source_id) const { return header[source_id]; }
    void set_header(HttpMsgHeader* header_, HttpCommon::SourceId source_id)
        { header[source_id] = header_; }

    HttpMsgTrailer* get_trailer(HttpCommon::SourceId source_id) const
        { return trailer[source_id]; }
    void set_trailer(HttpMsgTrailer* trailer_, HttpCommon::SourceId source_id)
        { trailer[source_id] = trailer_; }
    void set_body(HttpMsgBody* latest_body);

    HttpInfractions* get_infractions(HttpCommon::SourceId source_id);

    void set_one_hundred_response();
    bool final_response() const { return !second_response_expected; }

    void clear_section();
    bool is_clear() const { return active_sections == 0; }
    void garbage_collect();

    HttpTransaction* next = nullptr;

private:
    HttpTransaction(HttpFlowData* session_data_);
    void discard_section(HttpMsgSection* section);

    HttpFlowData* const session_data;

    uint64_t active_sections = 0;

    HttpMsgRequest* request = nullptr;
    HttpMsgStatus* status = nullptr;
    HttpMsgHeader* header[2] = { nullptr, nullptr };
    HttpMsgTrailer* trailer[2] = { nullptr, nullptr };
    HttpMsgBody* body_list = nullptr;
    HttpMsgSection* discard_list = nullptr;
    HttpInfractions* infractions[2];

    bool response_seen = false;
    bool one_hundred_response = false;
    bool second_response_expected = false;

    // This is a form of reference counting that prevents premature/double deletion of a
    // transaction in the fairly rare case where the request and response are received in
    // parallel.
    bool shared_ownership = false;

    // Estimates of how much memory http_inspect uses to process a transaction
    static const uint16_t small_things = 400; // minor memory costs not otherwise accounted for
    static const uint16_t transaction_memory_usage_estimate;
};

#endif

http_uri.h

Path = src/service_inspectors/http_inspect/http_uri.h

#ifndef HTTP_URI_H
#define HTTP_URI_H

#include "http_str_to_code.h"
#include "http_module.h"
#include "http_uri_norm.h"
#include "http_field.h"
#include "http_event.h"

static const int MAX_SCHEME_LENGTH = 36; // schemes longer than 36 characters are malformed
static const int LONG_SCHEME_LENGTH = 10; // schemes longer than 10 characters will alert

//-------------------------------------------------------------------------
// HttpUri class
//-------------------------------------------------------------------------

class HttpUri
{
public:
    HttpUri(const uint8_t* start, int32_t length, HttpEnums::MethodId method_id_,
        const HttpParaList::UriParam& uri_param_, HttpInfractions* infractions_,
        HttpEventGen* events_) :
        uri(length, start), infractions(infractions_), events(events_), method_id(method_id_),
        uri_param(uri_param_)
        { normalize(); }
    const Field& get_uri() const { return uri; }
    HttpEnums::UriType get_uri_type() { return uri_type; }
    const Field& get_scheme() { return scheme; }
    const Field& get_authority() { return authority; }
    const Field& get_host() { return host; }
    const Field& get_port() { return port; }
    const Field& get_abs_path() { return abs_path; }
    const Field& get_path() { return path; }
    const Field& get_query() { return query; }
    const Field& get_fragment() { return fragment; }

    const Field& get_norm_scheme();
    const Field& get_norm_host();
    const Field& get_norm_path() { return path_norm; }
    const Field& get_norm_query() { return query_norm; }
    const Field& get_norm_fragment() { return fragment_norm; }
    const Field& get_norm_classic() { return classic_norm; }

    static int32_t find_host_len(const Field& authority);

private:
    const Field uri;

    Field scheme;
    Field authority;
    Field host;
    Field port;
    Field abs_path;
    Field path;
    Field query;
    Field fragment;

    Field scheme_norm;
    Field host_norm;
    Field path_norm;
    Field query_norm;
    Field fragment_norm;
    Field classic_norm;
    HttpInfractions* const infractions;
    HttpEventGen* const events;
    HttpEnums::UriType uri_type = HttpEnums::URI__NOT_COMPUTE;
    const HttpEnums::MethodId method_id;
    const HttpParaList::UriParam& uri_param;

    void normalize();
    void parse_uri();
    void parse_authority();
    void parse_abs_path();

    void check_oversize_dir(const Field&);
};

#endif

http_uri_norm.h

Path = src/service_inspectors/http_inspect/http_uri_norm.h

#ifndef HTTP_URI_NORM_H
#define HTTP_URI_NORM_H

#include <vector>
#include <string>

#include "http_enum.h"
#include "http_field.h"
#include "http_module.h"
#include "http_event.h"

class UriNormalizer
{
public:
    static const unsigned URI_NORM_EXPANSION = 1;

    static bool need_norm(const Field& uri_component, bool do_path,
        const HttpParaList::UriParam& uri_param, HttpInfractions* infractions,
        HttpEventGen* events);
    static void normalize(const Field& input, Field& result, bool do_path, uint8_t* buffer,
        const HttpParaList::UriParam& uri_param, HttpInfractions* infractions,
        HttpEventGen* events, bool own_the_buffer = false);
    static bool classic_need_norm(const Field& uri_component, bool do_path,
        const HttpParaList::UriParam& uri_param);
    static void classic_normalize(const Field& input, Field& result, bool do_path,
        const HttpParaList::UriParam& uri_param);
    static void load_default_unicode_map(uint8_t map[65536]);
    static void load_unicode_map(uint8_t map[65536], const char* filename, int code_page);

private:
    static bool need_norm_path(const Field& uri_component,
        const HttpParaList::UriParam& uri_param);
    static bool need_norm_no_path(const Field& uri_component,
        const HttpParaList::UriParam& uri_param);
    static int32_t norm_char_clean(const Field& input, uint8_t* out_buf,
        const HttpParaList::UriParam& uri_param, HttpInfractions* infractions,
        HttpEventGen* events);
    static int32_t norm_percent_processing(const Field& input, uint8_t* out_buf,
        const HttpParaList::UriParam& uri_param, bool& utf8_needed,
        std::vector<bool>& percent_encoded, bool& double_decoding_needed,
        HttpInfractions* infractions, HttpEventGen* events);
    static int32_t norm_utf8_processing(const Field& input, uint8_t* out_buf,
        const HttpParaList::UriParam& uri_param, const std::vector<bool>& percent_encoded,
        bool& double_decoding_needed, HttpInfractions* infractions, HttpEventGen* events);
    static int32_t norm_double_decode(const Field& input, uint8_t* out_buf,
        const HttpParaList::UriParam& uri_param, HttpInfractions* infractions,
        HttpEventGen* events);
    static void norm_substitute(uint8_t* buf, int32_t length,
        const HttpParaList::UriParam& uri_param,  HttpInfractions* infractions,
        HttpEventGen* events);
    static int32_t norm_path_clean(uint8_t* buf, const int32_t in_length,
        HttpInfractions* infractions, HttpEventGen* events);
    static void detect_bad_char(const Field& uri_component,
        const HttpParaList::UriParam& uri_param, HttpInfractions* infractions,
        HttpEventGen* events);
    static uint8_t reduce_to_eight_bits(uint16_t value, const HttpParaList::UriParam& uri_param,
        HttpInfractions* infractions, HttpEventGen* events);
    static bool advance_to_code_page(FILE* file, int page_to_use);
    static bool map_code_points(FILE* file, uint8_t* map);

    static inline bool is_percent_encoding(const Field& input, int32_t index);
    static inline uint8_t extract_percent_encoding(const Field& input, int32_t index);
    static inline bool is_u_encoding(const Field& input, int32_t index);
    static inline uint16_t extract_u_encoding(const Field& input, int32_t index);
};

bool UriNormalizer::is_percent_encoding(const Field& input, int32_t index)
{
    return (index+2 < input.length()) &&
           (HttpEnums::as_hex[input.start()[index+1]] != -1) &&
           (HttpEnums::as_hex[input.start()[index+2]] != -1);
}

uint8_t UriNormalizer::extract_percent_encoding(const Field& input, int32_t index)
{
    return HttpEnums::as_hex[input.start()[index+1]] << 4 |
           HttpEnums::as_hex[input.start()[index+2]];
}

bool UriNormalizer::is_u_encoding(const Field& input, int32_t index)
{
    return (index+5 < input.length()) &&
           ((input.start()[index+1] == 'u') || (input.start()[index+1] == 'U')) &&
           (HttpEnums::as_hex[input.start()[index+2]] != -1) &&
           (HttpEnums::as_hex[input.start()[index+3]] != -1) &&
           (HttpEnums::as_hex[input.start()[index+4]] != -1) &&
           (HttpEnums::as_hex[input.start()[index+5]] != -1);
}

uint16_t UriNormalizer::extract_u_encoding(const Field& input, int32_t index)
{
    return (HttpEnums::as_hex[input.start()[index+2]] << 12) |
           (HttpEnums::as_hex[input.start()[index+3]] << 8)  |
           (HttpEnums::as_hex[input.start()[index+4]] << 4)  |
            HttpEnums::as_hex[input.start()[index+5]];
}

#endif

ips_http_buffer.h

Path = src/service_inspectors/http_inspect/ips_http_buffer.h

#ifndef IPS_HTTP_BUFFER_H
#define IPS_HTTP_BUFFER_H

#include <array>

#include "profiler/profiler.h"
#include "framework/ips_option.h"
#include "framework/module.h"

#include "ips_http.h"

enum BufferPsIdx { BUFFER_PSI_CLIENT_BODY, BUFFER_PSI_COOKIE, BUFFER_PSI_HEADER, BUFFER_PSI_METHOD,
    BUFFER_PSI_RAW_BODY, BUFFER_PSI_RAW_COOKIE, BUFFER_PSI_RAW_HEADER, BUFFER_PSI_RAW_REQUEST,
    BUFFER_PSI_RAW_STATUS, BUFFER_PSI_RAW_TRAILER, BUFFER_PSI_RAW_URI, BUFFER_PSI_STAT_CODE,
    BUFFER_PSI_STAT_MSG, BUFFER_PSI_TRAILER, BUFFER_PSI_TRUE_IP, BUFFER_PSI_URI, BUFFER_PSI_VERSION,
    BUFFER_PSI_JS_DATA, BUFFER_PSI_MAX };

class HttpBufferRuleOptModule : public HttpRuleOptModule
{
public:
    HttpBufferRuleOptModule(const char* key_, const char* help, HttpEnums::HTTP_RULE_OPT rule_opt_index_,
        snort::CursorActionType cat_, BufferPsIdx idx_)
        : HttpRuleOptModule(key_, help, rule_opt_index_, cat_), idx(idx_) {}
    HttpBufferRuleOptModule(const char* key_, const char* help, HttpEnums::HTTP_RULE_OPT rule_opt_index_,
        snort::CursorActionType cat_, BufferPsIdx idx_, const snort::Parameter params[])
        : HttpRuleOptModule(key_, help, rule_opt_index_, cat_, params), idx(idx_) {}
    snort::ProfileStats* get_profile() const override { return &http_buffer_ps[idx]; }
    static void mod_dtor(snort::Module* m) { delete m; }
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

private:
    friend class HttpBufferIpsOption;
    static THREAD_LOCAL std::array<snort::ProfileStats, BUFFER_PSI_MAX> http_buffer_ps;
    const BufferPsIdx idx;

    // URI related params. These affect the sub_id while parsed.
    // These values are saved to alert on conflicts, only used by ::end
    bool scheme = false;
    bool host = false;
    bool port = false;
    bool path = false;
    bool query = false;
    bool fragment = false;
};

class HttpBufferIpsOption : public HttpIpsOption
{
public:
    HttpBufferIpsOption(const HttpBufferRuleOptModule* cm) :
        HttpIpsOption(cm), idx(cm->idx),
        key(cm->key), fp_buffer_info(cm->rule_opt_index) {}
    EvalStatus eval(Cursor&, snort::Packet*) override;

    static IpsOption* opt_ctor(snort::Module* m, OptTreeNode*)
    { return new HttpBufferIpsOption((HttpBufferRuleOptModule*)m); }

    static void opt_dtor(snort::IpsOption* p) { delete p; }

    snort::CursorActionType get_cursor_type() const override
    { return buffer_info.is_request()? snort::CAT_SET_OTHER : cat; }

private:
    const BufferPsIdx idx;
    const char* const key;
    const HttpBufferInfo fp_buffer_info;
};

#endif

ips_http.h

Path = src/service_inspectors/http_inspect/ips_http.h

#ifndef IPS_HTTP_H
#define IPS_HTTP_H

#include <array>

#include "profiler/profiler.h"
#include "framework/pdu_section.h"
#include "framework/ips_option.h"
#include "framework/module.h"
#include "framework/range.h"

#include "http_buffer_info.h"
#include "http_enum.h"

class HttpInspect;

class HttpRuleOptModule : public snort::Module
{
public:
    HttpRuleOptModule(const char* key_, const char* help, HttpEnums::HTTP_RULE_OPT rule_opt_index_,
        snort::CursorActionType cat_)
        : snort::Module(key_, help), rule_opt_index(rule_opt_index_), key(key_),
          cat(cat_) {}
    HttpRuleOptModule(const char* key_, const char* help, HttpEnums::HTTP_RULE_OPT rule_opt_index_,
        snort::CursorActionType cat_, const snort::Parameter params[])
        : snort::Module(key_, help, params), rule_opt_index(rule_opt_index_),
        key(key_), cat(cat_) {}
    snort::ProfileStats* get_profile() const override = 0;
    static void mod_dtor(snort::Module* m) { delete m; }
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool set(const char*, snort::Value&, snort::SnortConfig*) override;

    Usage get_usage() const override
    { return DETECT; }

protected:
    snort::PduSection pdu_section = snort::PduSection::PS_NONE;
    const HttpEnums::HTTP_RULE_OPT rule_opt_index;
    const char* const key;
    uint64_t sub_id = 0;

private:
    friend class HttpIpsOption;

    const snort::CursorActionType cat;
    uint64_t form = 0;
};

class HttpIpsOption : public snort::IpsOption
{
public:
    HttpIpsOption(const HttpRuleOptModule* cm) :
        snort::IpsOption(cm->key),
        buffer_info(cm->rule_opt_index, cm->sub_id, cm->form),
        cat(cm->cat), pdu_section(cm->pdu_section) {}
    snort::CursorActionType get_cursor_type() const override { return cat; }
    EvalStatus eval(Cursor&, snort::Packet*) override = 0;
    uint32_t hash() const override;
    bool operator==(const snort::IpsOption& ips) const override;

    snort::section_flags get_pdu_section(bool) const override;

protected:
    const HttpBufferInfo buffer_info;
    const snort::CursorActionType cat;

    HttpInspect const* eval_helper(snort::Packet* p);

private:
    const snort::PduSection pdu_section;
};

#endif

ips_http_num_hdrs.h

Path = src/service_inspectors/http_inspect/ips_http_num_hdrs.h

#ifndef IPS_HTTP_NUM_HDRS_H
#define IPS_HTTP_NUM_HDRS_H

#include <array>

#include "profiler/profiler.h"
#include "framework/ips_option.h"
#include "framework/module.h"
#include "framework/range.h"

#include "http_enum.h"
#include "http_inspect.h"
#include "ips_http.h"

// Base class for all range-based rule options modules
class HttpRangeRuleOptModule : public HttpRuleOptModule
{
public:
    HttpRangeRuleOptModule(const char* key_, const char* help, HttpEnums::HTTP_RULE_OPT rule_opt_index_,
        const snort::Parameter params[], snort::ProfileStats& ps_);

    snort::ProfileStats* get_profile() const override { return &ps; }

    static void mod_dtor(snort::Module* m) { delete m; }
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool set(const char*, snort::Value&, snort::SnortConfig*) override;

private:
    snort::ProfileStats& ps;

    friend class HttpRangeIpsOption;
    const char* num_range;
    snort::RangeCheck range;
};

// Base class for all range-based rule options
class HttpRangeIpsOption : public HttpIpsOption
{
public:
    HttpRangeIpsOption(const HttpRangeRuleOptModule* cm) :
        HttpIpsOption(cm), ps(cm->ps), range(cm->range) { }
    EvalStatus eval(Cursor&, snort::Packet*) override;
    uint32_t hash() const override;
    bool operator==(const snort::IpsOption& ips) const override;

    static void opt_dtor(snort::IpsOption* p) { delete p; }

    virtual int32_t get_num(const HttpInspect* hi, snort::Packet* p) = 0;

private:
    snort::ProfileStats& ps;
    const snort::RangeCheck range;
};

// Template class for range-based rule options module
template<HttpEnums::HTTP_RULE_OPT OPT_IDX, snort::PduSection SECTION>
class HttpNumRuleOptModule : public HttpRangeRuleOptModule
{
public:
    HttpNumRuleOptModule(const char* key_, const char* help, const snort::Parameter params[])
        : HttpRangeRuleOptModule(key_, help, OPT_IDX, params, ps) { }

    bool begin(const char*, int, snort::SnortConfig*) override
    {
        HttpRangeRuleOptModule::begin(nullptr, 0, nullptr);
        pdu_section = SECTION;
        return true;
    }

private:
    static THREAD_LOCAL snort::ProfileStats ps;
};

template<HttpEnums::HTTP_RULE_OPT OPT_IDX, snort::PduSection SECTION>
THREAD_LOCAL snort::ProfileStats HttpNumRuleOptModule<OPT_IDX, SECTION>::ps;

// Template class for range-based rule options
template<int32_t (HttpInspect::* FNC)(snort::Packet*, const HttpBufferInfo&) const>
class HttpNumIpsOption : public HttpRangeIpsOption
{
public:
    using HttpRangeIpsOption::HttpRangeIpsOption;

    static IpsOption* opt_ctor(snort::Module* m, OptTreeNode*)
    { return new HttpNumIpsOption((const HttpRangeRuleOptModule*)m); }

    int32_t get_num(const HttpInspect* hi, snort::Packet* p) override
    { return (hi->*FNC)(p, buffer_info); }
};
#endif

ips_http_param.h

Path = src/service_inspectors/http_inspect/ips_http_param.h

#ifndef IPS_HTTP_PARAM_H
#define IPS_HTTP_PARAM_H

#include "profiler/profiler.h"
#include "framework/ips_option.h"
#include "framework/module.h"
#include "helpers/literal_search.h"

#include "http_param.h"
#include "ips_http.h"


class HttpParamRuleOptModule : public HttpRuleOptModule
{
public:
    HttpParamRuleOptModule(const char* key_, const char* help,
        HttpEnums::HTTP_RULE_OPT rule_opt_index_,
        snort::CursorActionType cat_, const snort::Parameter params[])
        : HttpRuleOptModule(key_, help, rule_opt_index_, cat_, params)
    { search_handle = snort::LiteralSearch::setup(); }

    ~HttpParamRuleOptModule() override
    { snort::LiteralSearch::cleanup(search_handle); }

    snort::ProfileStats* get_profile() const override { return &http_param_ps; }
    static void mod_dtor(snort::Module* m) { delete m; }
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

private:
    friend class HttpParamIpsOption;
    static THREAD_LOCAL snort::ProfileStats http_param_ps;

    std::string param;       // provide buffer containing specific parameter
    bool nocase = false;             // case insensitive match
    snort::LiteralSearch::Handle* search_handle;
};

class HttpParamIpsOption : public HttpIpsOption
{
public:
    HttpParamIpsOption(const HttpParamRuleOptModule* cm)
        : HttpIpsOption(cm), key(cm->key),
          http_param(cm->param, cm->nocase, cm->search_handle) {}
    EvalStatus eval(Cursor&, snort::Packet*) override;
    uint32_t hash() const override;
    bool operator==(const snort::IpsOption& ips) const override;

    static IpsOption* opt_ctor(snort::Module* m, OptTreeNode*)
    { return new HttpParamIpsOption((HttpParamRuleOptModule*)m); }

    static void opt_dtor(snort::IpsOption* p) { delete p; }
    bool retry(Cursor& , const Cursor&) override;

    snort::section_flags get_pdu_section(bool) const override;

private:
    const char* const key;
    const HttpParam http_param;
};

#endif

ips_http_test.h

Path = src/service_inspectors/http_inspect/ips_http_test.h

#ifndef IPS_HTTP_TEST_H
#define IPS_HTTP_TEST_H

#include <array>

#include "profiler/profiler.h"
#include "framework/ips_option.h"
#include "framework/module.h"
#include "framework/range.h"

#include "http_enum.h"
#include "http_field.h"
#include "ips_http.h"

enum TestPsIdx { TEST_PSI_HEADER_TEST, TEST_PSI_TRAILER_TEST, TEST_PSI_MAX };

enum NumericValue { NV_UNDEFINED, NV_TRUE, NV_FALSE };

class HttpTestRuleOptModule : public HttpRuleOptModule
{
public:
    HttpTestRuleOptModule(const char* key_, const char* help, HttpEnums::HTTP_RULE_OPT rule_opt_index_,
        snort::CursorActionType cat_, TestPsIdx idx_, const snort::Parameter params[])
        : HttpRuleOptModule(key_, help, rule_opt_index_, cat_, params), idx(idx_) {}

    snort::ProfileStats* get_profile() const override
    { return &http_test_ps[idx]; }

    static void mod_dtor(snort::Module* m) { delete m; }
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

private:
    friend class HttpTestIpsOption;
    static THREAD_LOCAL std::array<snort::ProfileStats, TEST_PSI_MAX> http_test_ps;
    const TestPsIdx idx;
    snort::RangeCheck check;
    enum NumericValue numeric = NV_UNDEFINED;
    bool absent = false;
};

class HttpTestIpsOption : public HttpIpsOption
{
public:
    HttpTestIpsOption(const HttpTestRuleOptModule* cm) :
        HttpIpsOption(cm), idx(cm->idx), check(cm->check),
        numeric(cm->numeric), absent(cm->absent) {}
    EvalStatus eval(Cursor&, snort::Packet*) override;
    uint32_t hash() const override;
    bool operator==(const snort::IpsOption& ips) const override;
    static IpsOption* opt_ctor(snort::Module* m, OptTreeNode*)
        { return new HttpTestIpsOption((HttpTestRuleOptModule*)m); }
    static void opt_dtor(snort::IpsOption* p) { delete p; }

private:
    const TestPsIdx idx;
    const snort::RangeCheck check;
    const enum NumericValue numeric;
    const bool absent;

    IpsOption::EvalStatus eval_header_test(const Field& http_buffer) const;
};

#endif

ips_http_version.h

Path = src/service_inspectors/http_inspect/ips_http_version.h

#ifndef IPS_HTTP_VERSION_H
#define IPS_HTTP_VERSION_H

#include "profiler/profiler.h"
#include "framework/ips_option.h"
#include "framework/module.h"

#include "http_enum.h"
#include "ips_http.h"

class HttpVersionRuleOptModule : public HttpRuleOptModule
{
public:
    HttpVersionRuleOptModule(const char* key_, const char* help, HttpEnums::HTTP_RULE_OPT rule_opt_index_,
        snort::CursorActionType cat_, const snort::Parameter params[])
        : HttpRuleOptModule(key_, help, rule_opt_index_, cat_, params){}
    snort::ProfileStats* get_profile() const override { return &http_version_ps; }
    static void mod_dtor(snort::Module* m) { delete m; }
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool set(const char*, snort::Value&, snort::SnortConfig*) override;

private:
    friend class HttpVersionIpsOption;
    static THREAD_LOCAL snort::ProfileStats http_version_ps;
    static const int version_size = HttpEnums::VERS__MAX - HttpEnums::VERS__MIN + 1;

    std::bitset<version_size> version_flags;

    bool parse_version_list(snort::Value& v);
};

class HttpVersionIpsOption : public HttpIpsOption
{
public:
    HttpVersionIpsOption(const HttpVersionRuleOptModule* cm) :
        HttpIpsOption(cm), version_flags(cm->version_flags) {}
    EvalStatus eval(Cursor&, snort::Packet*) override;
    uint32_t hash() const override;
    bool operator==(const snort::IpsOption& ips) const override;

    static IpsOption* opt_ctor(snort::Module* m, OptTreeNode*)
    { return new HttpVersionIpsOption((HttpVersionRuleOptModule*)m); }

    static void opt_dtor(snort::IpsOption* p) { delete p; }

private:
    const std::bitset<HttpVersionRuleOptModule::version_size> version_flags;
};

#endif

service_inspectors/http_inspect/test/

service_inspectors/iec104/

IEC 60870-5-104 (IEC104) is a protocol distributed by the International Electrotechnical Commission (IEC) that provides a standardized method of sending telecontrol messages between central stations and outstations, typically running on TCP port 2404.

It is used in combination with the companion specifications in the IEC 60870-5 family, most notably IEC 60870-5-101, to provide reliable transport via TCP/IP.

An IEC104 Application Protocol Data Unit (APDU) consists of one of three Application Protocol Control Information (APCI) structures, each beginning with the start byte 0x68. In the case of an Information Transfer APCI, an Application Service Data Unit (ASDU) follows the APCI.

The IEC104 inspector decodes the IEC104 protocol and provides rule options to access certain protocol fields and data content. This allows the user to write rules for IEC104 packets without decoding the protocol.

iec104_decode.h

Path = src/service_inspectors/iec104/iec104_decode.h

#ifndef IEC104_DECODE_H
#define IEC104_DECODE_H

namespace snort
{
struct Packet;
}

class Iec104FlowData;

/* IEC104 defines */
#define IEC104_START_ID 0x68

/* Need 6 bytes for Start, Length, and 4 control field octets */
#define IEC104_MIN_LEN 6

bool Iec104Decode(snort::Packet*, Iec104FlowData* iec104fd);

#endif

iec104.h

Path = src/service_inspectors/iec104/iec104.h

#ifndef IEC104_H
#define IEC104_H

#include "flow/flow.h"
#include "framework/counts.h"

#define IEC104_MIN_APCI_LEN 4        // Enough for the four control octets that are in every message

struct Iec104Stats
{
    PegCount sessions;
    PegCount frames;
    PegCount concurrent_sessions;
    PegCount max_concurrent_sessions;
};

struct Iec104SessionData
{
    uint8_t iec104_apci_type = -1;
    uint8_t iec104_asdu_func = 0;

    void session_data_reset()
    {
        iec104_apci_type = -1;
        iec104_asdu_func = 0;
    }
};

class Iec104FlowData: public snort::FlowData
{
public:
    Iec104FlowData();
    ~Iec104FlowData() override;

    static void init();

    void reset()
    { ssn_data.session_data_reset(); }

public:
    static unsigned inspector_id;
    Iec104SessionData ssn_data;
};

uint32_t get_message_type(uint32_t version, const char* name);
uint32_t get_info_type(uint32_t version, const char* name);

extern THREAD_LOCAL Iec104Stats iec104_stats;

#endif

iec104_module.h

Path = src/service_inspectors/iec104/iec104_module.h

#ifndef IEC104_MODULE_H
#define IEC104_MODULE_H

#include "framework/module.h"

#define GID_IEC104 151

#define IEC104_NAME "iec104"
#define IEC104_HELP "iec104 inspection"

extern THREAD_LOCAL snort::ProfileStats iec104_prof;

class Iec104Module: public snort::Module
{
public:
    Iec104Module();

    unsigned get_gid() const override
    {
        return GID_IEC104;
    }

    const snort::RuleMap* get_rules() const override;

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;

    snort::ProfileStats* get_profile() const override
    {
        return &iec104_prof;
    }

    Usage get_usage() const override
    {
        return INSPECT;
    }

    bool is_bindable() const override
    {
        return true;
    }

    void set_trace(const snort::Trace*) const override;
    const snort::TraceOption* get_trace_options() const override;
};

#define IEC104_BAD_LENGTH 1
#define IEC104_BAD_START 2
#define IEC104_RESERVED_ASDU_TYPE 3
#define IEC104_APCIU_RESERVED_FIELD_IN_USE 4
#define IEC104_APCIU_INVALID_MESSAGE_TYPE 5
#define IEC104_APCIS_RESERVED_FIELD_IN_USE 6
#define IEC104_APCII_NUM_ELEMENTS_SET_TO_ZERO 7
#define IEC104_APCII_INVALID_SQ_VALUE 8
#define IEC104_APCII_INVALID_NUM_ELEMENTS_VALUE 9
#define IEC104_RESERVED_COI 10
#define IEC104_RESERVED_QOI 11
#define IEC104_RESERVED_QCC 12
#define IEC104_RESERVED_QPM_KPA 13
#define IEC104_ABNORMAL_QPM_LPC 14
#define IEC104_ABNORMAL_QPM_POP 15
#define IEC104_RESERVED_QPA 16
#define IEC104_RESERVED_QOC 17
#define IEC104_RESERVED_QRP 18
#define IEC104_RESERVED_FRQ 19
#define IEC104_RESERVED_SRQ 20
#define IEC104_RESERVED_SCQ 21
#define IEC104_RESERVED_LSQ 22
#define IEC104_RESERVED_AFQ 23
#define IEC104_VSQ_ABNORMAL_SQ 24
#define IEC104_RESERVED_SIQ 25
#define IEC104_RESERVED_DIQ 26
#define IEC104_RESERVED_CAUSE_TX 27
#define IEC104_INVALID_CAUSE_TX 28
#define IEC104_INVALID_COMMON_ADDRESS 29
#define IEC104_RESERVED_QDS 30
#define IEC104_RESERVED_QDP 31
#define IEC104_RESERVED_IEEE_STD_754_NAN 32
#define IEC104_RESERVED_IEEE_STD_754_INFINITY 33
#define IEC104_RESERVED_SEP 34
#define IEC104_RESERVED_SPE 35
#define IEC104_RESERVED_OCI 36
#define IEC104_INVALID_FBP 37
#define IEC104_RESERVED_SCO 38
#define IEC104_INVALID_DCO 39
#define IEC104_RESERVED_RCO 40
#define IEC104_INVALID_MS_IN_MINUTE 41
#define IEC104_INVALID_MINS_IN_HOUR 42
#define IEC104_RESERVED_MINS_IN_HOUR 43
#define IEC104_INVALID_HOURS_IN_DAY 44
#define IEC104_RESERVED_HOURS_IN_DAY 45
#define IEC104_INVALID_DAY_OF_MONTH 46
#define IEC104_INVALID_MONTH 47
#define IEC104_RESERVED_MONTH 48
#define IEC104_INVALID_YEAR 49
#define IEC104_RESERVED_YEAR 50
#define IEC104_NULL_LOS_VALUE 51
#define IEC104_INVALID_LOS_VALUE 52
#define IEC104_RESERVED_SOF 53
#define IEC104_RESERVED_QOS 54

#define IEC104_BAD_LENGTH_STR "Length in IEC104 APCI header does not match the length needed for the given IEC104 ASDU type id"
#define IEC104_BAD_START_STR "IEC104 Start byte does not match 0x68"
#define IEC104_RESERVED_ASDU_TYPE_STR "Reserved IEC104 ASDU type id in use"
#define IEC104_APCIU_RESERVED_FIELD_IN_USE_STR "IEC104 APCI U Reserved field contains a non-default value"
#define IEC104_APCIU_INVALID_MESSAGE_TYPE_STR "IEC104 APCI U message type was set to an invalid value"
#define IEC104_APCIS_RESERVED_FIELD_IN_USE_STR "IEC104 APCI S Reserved field contains a non-default value"
#define IEC104_APCII_NUM_ELEMENTS_SET_TO_ZERO_STR "IEC104 APCI I number of elements set to zero"
#define IEC104_APCII_INVALID_SQ_VALUE_STR "IEC104 APCI I SQ bit set on an ASDU that does not support the feature"
#define IEC104_APCII_INVALID_NUM_ELEMENTS_VALUE_STR "IEC104 APCI I number of elements set to greater than one on an ASDU that does not support the feature"
#define IEC104_RESERVED_COI_STR "IEC104 APCI I Cause of Initialization set to a reserved value"
#define IEC104_RESERVED_QOI_STR "IEC104 APCI I Qualifier of Interrogation Command set to a reserved value"
#define IEC104_RESERVED_QCC_STR "IEC104 APCI I Qualifier of Counter Interrogation Command request parameter set to a reserved value"
#define IEC104_RESERVED_QPM_KPA_STR "IEC104 APCI I Qualifier of Parameter of Measured Values kind of parameter set to a reserved value"
#define IEC104_ABNORMAL_QPM_LPC_STR "IEC104 APCI I Qualifier of Parameter of Measured Values local parameter change set to a technically valid but unused value"
#define IEC104_ABNORMAL_QPM_POP_STR "IEC104 APCI I Qualifier of Parameter of Measured Values parameter option set to a technically valid but unused value"
#define IEC104_RESERVED_QPA_STR "IEC104 APCI I Qualifier of Parameter Activation set to a reserved value"
#define IEC104_RESERVED_QOC_STR "IEC104 APCI I Qualifier of Command set to a reserved value"
#define IEC104_RESERVED_QRP_STR "IEC104 APCI I Qualifier of Reset Process set to a reserved value"
#define IEC104_RESERVED_FRQ_STR "IEC104 APCI I File Ready Qualifier set to a reserved value"
#define IEC104_RESERVED_SRQ_STR "IEC104 APCI I Section Ready Qualifier set to a reserved value"
#define IEC104_RESERVED_SCQ_STR "IEC104 APCI I Select and Call Qualifier set to a reserved value"
#define IEC104_RESERVED_LSQ_STR "IEC104 APCI I Last Section or Segment Qualifier set to a reserved value"
#define IEC104_RESERVED_AFQ_STR "IEC104 APCI I Acknowledge File or Section Qualifier set to a reserved value"
#define IEC104_VSQ_ABNORMAL_SQ_STR "IEC104 APCI I Structure Qualifier set on a message where it should have no effect"
#define IEC104_RESERVED_CAUSE_TX_STR "IEC104 APCI I Cause of Transmission set to a reserved value"
#define IEC104_INVALID_CAUSE_TX_STR "IEC104 APCI I Cause of Transmission set to a value not allowed for the ASDU"
#define IEC104_INVALID_COMMON_ADDRESS_STR "IEC104 APCI I invalid two octet common address value detected"
#define IEC104_RESERVED_SIQ_STR "IEC104 APCI I Single Point Information Reserved field contains a non-default value"
#define IEC104_RESERVED_DIQ_STR "IEC104 APCI I Double Point Information Reserved field contains a non-default value"
#define IEC104_RESERVED_QDS_STR "IEC104 APCI I Quality Descriptor Structure Reserved field contains a non-default value"
#define IEC104_RESERVED_QDP_STR "IEC104 APCI I Quality Descriptor for Events of Protection Equipment Structure Reserved field contains a non-default value"
#define IEC104_RESERVED_IEEE_STD_754_NAN_STR "IEC104 APCI I IEEE STD 754 value results in NaN"
#define IEC104_RESERVED_IEEE_STD_754_INFINITY_STR "IEC104 APCI I IEEE STD 754 value results in infinity"
#define IEC104_RESERVED_SEP_STR "IEC104 APCI I Single Event of Protection Equipment Structure Reserved field contains a non-default value"
#define IEC104_RESERVED_SPE_STR "IEC104 APCI I Start Event of Protection Equipment Structure Reserved field contains a non-default value"
#define IEC104_RESERVED_OCI_STR "IEC104 APCI I Output Circuit Information Structure Reserved field contains a non-default value"
#define IEC104_INVALID_FBP_STR "IEC104 APCI I Abnormal Fixed Test Bit Pattern detected"
#define IEC104_RESERVED_SCO_STR "IEC104 APCI I Single Command Structure Reserved field contains a non-default value"
#define IEC104_INVALID_DCO_STR "IEC104 APCI I Double Command Structure contains an invalid value"
#define IEC104_RESERVED_RCO_STR "IEC104 APCI I Regulating Step Command Structure Reserved field contains a non-default value"
#define IEC104_INVALID_MS_IN_MINUTE_STR "IEC104 APCI I Time2a Millisecond set outside of the allowable range"
#define IEC104_INVALID_MINS_IN_HOUR_STR "IEC104 APCI I Time2a Minute set outside of the allowable range"
#define IEC104_RESERVED_MINS_IN_HOUR_STR "IEC104 APCI I Time2a Minute Reserved field contains a non-default value"
#define IEC104_INVALID_HOURS_IN_DAY_STR "IEC104 APCI I Time2a Hours set outside of the allowable range"
#define IEC104_RESERVED_HOURS_IN_DAY_STR "IEC104 APCI I Time2a Hours Reserved field contains a non-default value"
#define IEC104_INVALID_DAY_OF_MONTH_STR "IEC104 APCI I Time2a Day of Month set outside of the allowable range"
#define IEC104_INVALID_MONTH_STR "IEC104 APCI I Time2a Month set outside of the allowable range"
#define IEC104_RESERVED_MONTH_STR "IEC104 APCI I Time2a Month Reserved field contains a non-default value"
#define IEC104_INVALID_YEAR_STR "IEC104 APCI I Time2a Year set outside of the allowable range"
#define IEC104_NULL_LOS_VALUE_STR "IEC104 APCI I a null Length of Segment value has been detected"
#define IEC104_INVALID_LOS_VALUE_STR "IEC104 APCI I an invalid Length of Segment value has been detected"
#define IEC104_RESERVED_YEAR_STR "IEC104 APCI I Time2a Year Reserved field contains a non-default value"
#define IEC104_RESERVED_SOF_STR "IEC104 APCI I Status of File set to a reserved value"
#define IEC104_RESERVED_QOS_STR "IEC104 APCI I Qualifier of Set Point Command ql field set to a reserved value"

#endif

iec104_paf.h

Path = src/service_inspectors/iec104/iec104_paf.h

#ifndef IEC104_PAF__H
#define IEC104_PAF__H

// Protocol-Aware Flushing (PAF) code for the IEC104 inspector.

#include "stream/stream_splitter.h"

enum iec104_paf_state_t
{
    IEC104_PAF_STATE__START = 0,
    IEC104_PAF_STATE__LEN = 1,
    IEC104_PAF_STATE__SET_FLUSH = 2,
};

class Iec104Splitter: public snort::StreamSplitter
{
public:
    Iec104Splitter(bool);

    Status scan(snort::Packet*, const uint8_t* data, uint32_t len, uint32_t flags,
        uint32_t* fp) override;

    bool is_paf() override { return true; }

private:
    iec104_paf_state_t state;
    uint16_t iec104_apci_length;
};

#endif

iec104_parse_apdu.h

Path = src/service_inspectors/iec104/iec104_parse_apdu.h

#ifndef IEC104_PARSE_APCI_H
#define IEC104_PARSE_APCI_H

#include <cstdint>

void parseIec104ApciU(const struct Iec104ApciU* apci);
void parseIec104ApciS(const struct Iec104ApciS* apci);
void parseIec104ApciI(const struct Iec104ApciI* apci);


#define IEC104_ERROR -1
#define IEC104_START_BYTE 0x68
#define IEC104_APCI_HDR_LEN 2 // accounts for Start and Length
#define IEC104_APCI_TYPE_U_LEN 4
#define IEC104_APCI_TYPE_S_LEN 4
#define IEC104_APCI_TYPE_I_MIN_LEN 12
#define IEC104_APCI_TYPE_I_HDR_LEN 6
#define IEC104_CTX_NOT_ALLOWED 0
#define IEC104_CTX_ALLOWED 1


//
//
// Enums
//
//

// Definition for the different APCI headers
// This allows us to make determinations on how to handle the fields following the first 1-2 bits
enum ApciTypeEnum
{
    IEC104_NO_APCI     = -1,
    IEC104_APCI_TYPE_I = 0,
    IEC104_APCI_TYPE_S = 1,
    IEC104_APCI_TYPE_U = 2,
};

enum AsduTypeEnum
{
    IEC104_NO_ASDU         =  0,     // placeholder for an error case
    IEC104_ASDU_M_SP_NA_1  =  1,     // Single-point information
    IEC104_ASDU_M_SP_TA_1  =  2,     // Single-point information with time tag
    IEC104_ASDU_M_DP_NA_1  =  3,     // Double-point information
    IEC104_ASDU_M_DP_TA_1  =  4,     // Double-point information with time tag
    IEC104_ASDU_M_ST_NA_1  =  5,     // Step position information
    IEC104_ASDU_M_ST_TA_1  =  6,     // Step position information with time tag
    IEC104_ASDU_M_BO_NA_1  =  7,     // Bitstring of 32 bit
    IEC104_ASDU_M_BO_TA_1  =  8,     // Bitstring of 32 bit with time tag
    IEC104_ASDU_M_ME_NA_1  =  9,     // Measured value, normalized value
    IEC104_ASDU_M_ME_TA_1  =  10,    // Measured value, normalized value with time tag
    IEC104_ASDU_M_ME_NB_1  =  11,    // Measured value, scaled value
    IEC104_ASDU_M_ME_TB_1  =  12,    // Measured value, scaled value wit time tag
    IEC104_ASDU_M_ME_NC_1  =  13,    // Measured value, short floating point number
    IEC104_ASDU_M_ME_TC_1  =  14,    // Measured value, short floating point number with time tag
    IEC104_ASDU_M_IT_NA_1  =  15,    // Integrated totals
    IEC104_ASDU_M_IT_TA_1  =  16,    // Integrated totals with time tag
    IEC104_ASDU_M_EP_TA_1  =  17,    // Event of protection equipment with time tag
    IEC104_ASDU_M_EP_TB_1  =  18,    // Packed start events of protection equipment with time tag
    IEC104_ASDU_M_EP_TC_1  =  19,    // Packed output circuit information of protection equipment with time tag
    IEC104_ASDU_M_PS_NA_1  =  20,    // Packed single point information with status change detection
    IEC104_ASDU_M_ME_ND_1  =  21,    // Measured value, normalized value without quality descriptor
    // 22-29 reserved
    IEC104_ASDU_M_SP_TB_1  =  30,    // Single-point information with time tag CP56Time2a
    IEC104_ASDU_M_DP_TB_1  =  31,    // Double-point information with time tag CP56Time2a
    IEC104_ASDU_M_ST_TB_1  =  32,    // Step position information with time tag CP56Time2a
    IEC104_ASDU_M_BO_TB_1  =  33,    // Bitstring of 32 bit with time tag CP56Time2a
    IEC104_ASDU_M_ME_TD_1  =  34,    // Measured value, normalized value with time tag CP56Time2a
    IEC104_ASDU_M_ME_TE_1  =  35,    // Measured value, scaled value with time tag CP56Time2a
    IEC104_ASDU_M_ME_TF_1  =  36,    // Measured value, short floating point number with time tag CP56Time2a
    IEC104_ASDU_M_IT_TB_1  =  37,    // Integrated totals with time tag CP56Time2a
    IEC104_ASDU_M_EP_TD_1  =  38,    // Event of protection equipment with time tag CP56Time2a
    IEC104_ASDU_M_EP_TE_1  =  39,    // Packed start events of protection equipment with time tag CP56Time2a
    IEC104_ASDU_M_EP_TF_1  =  40,    // Packed output circuit information of protection equipment with time tag CP56Time2a
    // 41-44 reserved
    IEC104_ASDU_C_SC_NA_1  =  45,    // Single command
    IEC104_ASDU_C_DC_NA_1  =  46,    // Double command
    IEC104_ASDU_C_RC_NA_1  =  47,    // Regulating step command
    IEC104_ASDU_C_SE_NA_1  =  48,    // Set-point Command, normalized value
    IEC104_ASDU_C_SE_NB_1  =  49,    // Set-point Command, scaled value
    IEC104_ASDU_C_SE_NC_1  =  50,    // Set-point Command, short floating point number
    IEC104_ASDU_C_BO_NA_1  =  51,    // Bitstring 32 bit command
    // 52-57 reserved
    IEC104_ASDU_C_SC_TA_1  =  58,    // Single command with time tag CP56Time2a
    IEC104_ASDU_C_DC_TA_1  =  59,    // Double command with time tag CP56Time2a
    IEC104_ASDU_C_RC_TA_1  =  60,    // Regulating step command with time tag CP56Time2a
    IEC104_ASDU_C_SE_TA_1  =  61,    // Set-point command with time tag CP56Time2a, normalized value
    IEC104_ASDU_C_SE_TB_1  =  62,    // Set-point command with time tag CP56Time2a, scaled value
    IEC104_ASDU_C_SE_TC_1  =  63,    // Set-point command with time tag CP56Time2a, short floating point number
    IEC104_ASDU_C_BO_TA_1  =  64,    // Bitstring of 32 bit with time tag CP56Time2a
    // 65-69 reserved
    IEC104_ASDU_M_EI_NA_1  =  70,    // End of initialization
    // 71-99 reserved
    IEC104_ASDU_C_IC_NA_1  =  100,   // Interrogation command
    IEC104_ASDU_C_CI_NA_1  =  101,   // Counter interrogation command
    IEC104_ASDU_C_RD_NA_1  =  102,   // Read command
    IEC104_ASDU_C_CS_NA_1  =  103,   // Clock synchronization command
    IEC104_ASDU_C_TS_NA_1  =  104,   // Test command
    IEC104_ASDU_C_RP_NA_1  =  105,   // Reset process command
    IEC104_ASDU_C_CD_NA_1  =  106,   // Delay acquisition command
    IEC104_ASDU_C_TS_TA_1  =  107,   // Test command with time tag CP56Time2a
    // 108-109 reserved
    IEC104_ASDU_P_ME_NA_1  =  110,   // Parameter of measured values, normalized value
    IEC104_ASDU_P_ME_NB_1  =  111,   // Parameter of measured values, scaled value
    IEC104_ASDU_P_ME_NC_1  =  112,   // Parameter of measured values, short floating point number
    IEC104_ASDU_P_AC_NA_1  =  113,   // Parameter activation
    // 114-119 reserved
    IEC104_ASDU_F_FR_NA_1  =  120,   // File ready
    IEC104_ASDU_F_SR_NA_1  =  121,   // Section ready
    IEC104_ASDU_F_SC_NA_1  =  122,   // Call directory, select file, call file, call section
    IEC104_ASDU_F_LS_NA_1  =  123,   // Last section, last segment
    IEC104_ASDU_F_AF_NA_1  =  124,   // ACK file, ACK section
    IEC104_ASDU_F_SG_NA_1  =  125,   // Single information object
    IEC104_ASDU_F_DR_TA_1  =  126,   // Sequence of information elements in a single information object
    IEC104_ASDU_F_SC_NB_1  =  127,   // QueryLog – Request archive file
    // 128-256 reserved
};

// Definition for the different transmission cause codes
enum CauseOfTransmissionEnum
{
    IEC104_CAUSE_TX_NOT_USED                      = 0,     // not used
    IEC104_CAUSE_TX_PER_CYC                       = 1,     // periodic, cyclic
    IEC104_CAUSE_TX_BACK                          = 2,     // background scan3
    IEC104_CAUSE_TX_SPONT                         = 3,     // spontaneous
    IEC104_CAUSE_TX_INIT                           = 4,     // initialized
    IEC104_CAUSE_TX_REQ                            = 5,     // request or requested
    IEC104_CAUSE_TX_ACT                           = 6,     // activation
    IEC104_CAUSE_TX_ACTCON                        = 7,     // activation confirmation
    IEC104_CAUSE_TX_DEACT                         = 8,     // deactivation
    IEC104_CAUSE_TX_DEACTCON                      = 9,     // deactivation confirmation
    IEC104_CAUSE_TX_ACTTERM                       = 10,    // activation termination
    IEC104_CAUSE_TX_RETREM                        = 11,    // return information caused by a remote command
    IEC104_CAUSE_TX_RETLOC                         = 12,    // return information caused by a local command
    IEC104_CAUSE_TX_FILE                          = 13,    // file transfer
    IEC104_CAUSE_TX_RES14                         = 14,    // 14-19 reserved
    IEC104_CAUSE_TX_RES15                         = 15,    // 14-19 reserved
    IEC104_CAUSE_TX_RES16                         = 16,    // 14-19 reserved
    IEC104_CAUSE_TX_RES17                         = 17,    // 14-19 reserved
    IEC104_CAUSE_TX_RES18                         = 18,    // 14-19 reserved
    IEC104_CAUSE_TX_RES19                         = 19,    // 14-19 reserved
    IEC104_CAUSE_TX_INROGEN                       = 20,    // interrogated by station interrogation
    IEC104_CAUSE_TX_INRO1                         = 21,    // interrogated by group 1 interrogation
    IEC104_CAUSE_TX_INRO2                         = 22,    // interrogated by group 2 interrogation
    IEC104_CAUSE_TX_INRO3                         = 23,    // interrogated by group 3 interrogation
    IEC104_CAUSE_TX_INRO4                         = 24,    // interrogated by group 4 interrogation
    IEC104_CAUSE_TX_INRO5                         = 25,    // interrogated by group 5 interrogation
    IEC104_CAUSE_TX_INRO6                         = 26,    // interrogated by group 6 interrogation
    IEC104_CAUSE_TX_INRO7                         = 27,    // interrogated by group 7 interrogation
    IEC104_CAUSE_TX_INRO8                         = 28,    // interrogated by group 8 interrogation
    IEC104_CAUSE_TX_INRO9                         = 29,    // interrogated by group 9 interrogation
    IEC104_CAUSE_TX_INRO10                        = 30,    // interrogated by group 10 interrogation
    IEC104_CAUSE_TX_INRO11                        = 31,    // interrogated by group 11 interrogation
    IEC104_CAUSE_TX_INRO12                        = 32,    // interrogated by group 12 interrogation
    IEC104_CAUSE_TX_INRO13                        = 33,    // interrogated by group 13 interrogation
    IEC104_CAUSE_TX_INRO14                        = 34,    // interrogated by group 14 interrogation
    IEC104_CAUSE_TX_INRO15                        = 35,    // interrogated by group 15 interrogation
    IEC104_CAUSE_TX_INRO16                        = 36,    // interrogated by group 16 interrogation
    IEC104_CAUSE_TX_REQCOGEN                      = 37,    // requested by general counter request
    IEC104_CAUSE_TX_REQCO1                        = 38,    // requested by group 1 counter request
    IEC104_CAUSE_TX_REQCO2                        = 39,    // requested by group 2 counter request
    IEC104_CAUSE_TX_REQCO3                        = 40,    // requested by group 3 counter request
    IEC104_CAUSE_TX_REQCO4                        = 41,    // requested by group 4 counter request
    IEC104_CAUSE_TX_RES42                         = 42,    // 42-43 reserved
    IEC104_CAUSE_TX_RES43                         = 43,    // 42-43 reserved
    IEC104_CAUSE_TX_UNKNOWN_TYPE_ID               = 44,    // unknown type identification
    IEC104_CAUSE_TX_UNKNOWN_CAUSE_OF_TX           = 45,    // unknown cause of transmission
    IEC104_CAUSE_TX_UNKNOWN_COMMON_ADDR_OF_ASDU   = 46,    // unknown common address of ASDU
    IEC104_CAUSE_TX_UNKNOWN_IOA                   = 47,    // unknown information object address
    IEC104_CAUSE_TX_RES48                         = 48,    // 48-63 reserved
    IEC104_CAUSE_TX_RES49                         = 49,    // 48-63 reserved
    IEC104_CAUSE_TX_RES50                         = 50,    // 48-63 reserved
    IEC104_CAUSE_TX_RES51                         = 51,    // 48-63 reserved
    IEC104_CAUSE_TX_RES52                         = 52,    // 48-63 reserved
    IEC104_CAUSE_TX_RES53                         = 53,    // 48-63 reserved
    IEC104_CAUSE_TX_RES54                         = 54,    // 48-63 reserved
    IEC104_CAUSE_TX_RES55                         = 55,    // 48-63 reserved
    IEC104_CAUSE_TX_RES56                         = 56,    // 48-63 reserved
    IEC104_CAUSE_TX_RES57                         = 57,    // 48-63 reserved
    IEC104_CAUSE_TX_RES58                         = 58,    // 48-63 reserved
    IEC104_CAUSE_TX_RES59                         = 59,    // 48-63 reserved
    IEC104_CAUSE_TX_RES60                         = 60,    // 48-63 reserved
    IEC104_CAUSE_TX_RES61                         = 61,    // 48-63 reserved
    IEC104_CAUSE_TX_RES62                         = 62,    // 48-63 reserved
    IEC104_CAUSE_TX_RES63                         = 63,    // 48-63 reserved
};

enum StructureQualifierEnum
{
    IEC104_SQ_FALSE  = 0,
    IEC104_SQ_TRUE   = 1,
};




//
//
// Structs
//
//

//
// Generic structs
//

// struct Iec104To help determine what type of APCI is in use
struct Iec104GenericApci
{
    uint8_t start;
    uint8_t length;
    uint8_t apciTypeMajor : 1;
    uint8_t apciTypeMinor : 1;
    uint8_t reserved : 6;
}__attribute__((packed));


//
// ASDU Information Object Structs
//

struct Iec104VariableStructureQualifierType
{
    uint8_t numberOfElements : 7;
    uint8_t sq : 1;
}__attribute__((packed));

// This structure does not require the OA, but it seems to be used in all traffic seen so far
struct Iec104CauseOfTransmissionType
{
    uint8_t causeOfTransmission : 6;
    uint8_t pn : 1;
    uint8_t test : 1;
    uint8_t oa;
}__attribute__((packed));

// COI: Cause of Initialization Structure
struct Iec104CoiType
{
    uint8_t ui : 7;
    uint8_t bs : 1;
}__attribute__((packed));

// QOI: Qualifier of Interrogation Structure
struct Iec104QoiType
{
    uint8_t qoi;
}__attribute__((packed));

// QCC: Qualifier of Counter Interrogation Command Structure
struct Iec104QccType
{
    uint8_t rqt : 6;
    uint8_t frz : 2;
}__attribute__((packed));

// QPM: Qualifier of Parameter of Measured Values Structure
struct Iec104QpmType
{
    uint8_t kpa : 6;
    uint8_t lpc : 1;
    uint8_t pop : 1;
}__attribute__((packed));

// QPA: Qualifier of Parameter Activation Structure
struct Iec104QpaType
{
    uint8_t qpa;
}__attribute__((packed));

// QOC: Qualifier of Command Structure
// This doesn't add up to 8, but that is expected
// This struct gets used in fields that have 2 preceding bits
struct Iec104QocType
{
    uint8_t qu : 5;
    uint8_t se : 1;
}__attribute__((packed));

// QRP: Qualifier of Reset Process Structure
struct Iec104QrpType
{
    uint8_t qrp;
}__attribute__((packed));

// FRQ: File Ready Qualifier Structure
struct Iec104FrqType
{
    uint8_t ui : 7;
    uint8_t bs : 1;
}__attribute__((packed));

// SRQ: Section Ready Qualifier Structure
struct Iec104SrqType
{
    uint8_t ui : 7;
    uint8_t bs : 1;
}__attribute__((packed));

// SCQ: Select and Call Qualifier Structure
struct Iec104ScqType
{
    uint8_t ui1 : 4;
    uint8_t ui2 : 4;
}__attribute__((packed));

// LSQ: Last Section or Segment Qualifier Structure
struct Iec104LsqType
{
    uint8_t lsq;
}__attribute__((packed));

// AFQ: Acknowledge File or Section Qualifier Structure
struct Iec104AfqType
{
    uint8_t ui1 : 4;
    uint8_t ui2 : 4;
}__attribute__((packed));

// Common Address of ASDU Structure
// This structure does not require the high octet, but it seems to be
//  used in all traffic seen so far
struct Iec104CommonAddressOfAsduType
{
    uint16_t commonAddress;
}__attribute__((packed));

// Information Object Address One Octet Structure
struct Iec104InformationObjectAddressOneOctetType
{
    uint8_t informationObjectAddress;
}__attribute__((packed));

// Information Object Address Two Octet Structure
struct Iec104InformationObjectAddressTwoOctetType
{
    uint8_t informationObjectAddress[2];
}__attribute__((packed));

// Information Object Address Three Octet Structure
struct Iec104InformationObjectAddressThreeOctetType
{
    uint8_t informationObjectAddress[3];
}__attribute__((packed));

// SIQ: Single Point Information with Quality Descriptor Structure
struct Iec104SiqType
{
    uint8_t spi : 1;
    uint8_t reserved : 3;
    uint8_t bl : 1;
    uint8_t sb : 1;
    uint8_t nt : 1;
    uint8_t iv : 1;
}__attribute__((packed));

// DIQ: Double Point Information with Quality Descriptor Structure
struct Iec104DiqType
{
    uint8_t dpi : 2;
    uint8_t reserved : 2;
    uint8_t bl : 1;
    uint8_t sb : 1;
    uint8_t nt : 1;
    uint8_t iv : 1;
}__attribute__((packed));

// QDS: Quality Descriptor Structure
struct Iec104QdsType
{
    uint8_t ov : 1;
    uint8_t reserved : 3;
    uint8_t bl : 1;
    uint8_t sb : 1;
    uint8_t nt : 1;
    uint8_t iv : 1;
}__attribute__((packed));

// QDP: Quality Descriptor for Events of Protection Equipment Structure
struct Iec104QdpType
{
    uint8_t reserved : 3;
    uint8_t ei : 1;
    uint8_t bl : 1;
    uint8_t sb : 1;
    uint8_t nt : 1;
    uint8_t iv : 1;
}__attribute__((packed));

// VTI: Value with Transient State Indication Structure
struct Iec104VtiType
{
    uint8_t value : 7;
    uint8_t t : 1;
}__attribute__((packed));

// NVA: Normalized Value Structure
struct Iec104NvaType
{
    uint16_t value;
}__attribute__((packed));

// SVA: Scaled Value Structure
struct Iec104SvaType
{
    uint16_t value;
}__attribute__((packed));

// IEEE_STD_754: Short Floating Point Number Structure
struct Iec104IeeeStd754Type
{
    uint32_t ieeeStd754;
}__attribute__((packed));

// BCR: Binary Counter Reading Structure
struct Iec104BcrType
{
    uint32_t value;
    uint8_t sequenceNumber : 5;
    uint8_t cy : 1;
    uint8_t ca : 1;
    uint8_t iv : 1;
}__attribute__((packed));

// SEP: Single Event of Protection Equipment Structure
struct Iec104SepType
{
    uint8_t es : 2;
    uint8_t reserved : 1;
    uint8_t ei : 1;
    uint8_t bl : 1;
    uint8_t sb : 1;
    uint8_t nt : 1;
    uint8_t iv : 1;
}__attribute__((packed));

// SPE: Start Event of Protection Equipment Structure
struct Iec104SpeType
{
    uint8_t gs : 1;
    uint8_t sl1 : 1;
    uint8_t sl2 : 1;
    uint8_t sl3 : 1;
    uint8_t sie : 1;
    uint8_t srd : 1;
    uint8_t reserved : 2;
}__attribute__((packed));

// OCI: Output Circuit Information Structure
struct Iec104OciType
{
    uint8_t gc : 1;
    uint8_t cl1 : 1;
    uint8_t cl2 : 1;
    uint8_t cl3 : 1;
    uint8_t reserved : 4;
}__attribute__((packed));

// BSI: Binary State Information Structure
struct Iec104BsiType
{
    uint32_t bitstring;
}__attribute__((packed));

// FBP: Fixed Test Bit Pattern Structure
struct Iec104FbpType
{
    uint16_t fixedTestBitPattern;
}__attribute__((packed));

// SCO: Single Command Structure
struct Iec104ScoType
{
    uint8_t scs : 1;
    uint8_t reserved : 1;
    uint8_t qu : 5;
    uint8_t se : 1;
}__attribute__((packed));

// DCO: Double Command Structure
struct Iec104DcoType
{
    uint8_t dcs : 2;
    uint8_t qu : 5;
    uint8_t se : 1;
}__attribute__((packed));

// RCO: Regulating Step Command Structure
struct Iec104RcoType
{
    uint8_t rcs : 2;
    uint8_t qu : 5;
    uint8_t se : 1;
}__attribute__((packed));

// Time2a Milliseconds Structure
struct Iec104Time2aMillisecondsType
{
    uint16_t milliseconds;
}__attribute__((packed));

// Time2a IVResMinute Structure
struct Iec104Time2aIvresminuteType
{
    uint8_t minutes : 6;
    uint8_t res : 1;
    uint8_t iv : 1;
}__attribute__((packed));

// Time2a SURes2Hour Structure
struct Iec104Time2aSures2hourType
{
    uint8_t hours : 5;
    uint8_t res2 : 2;
    uint8_t su : 1;
}__attribute__((packed));

// Time2a DOWDay Structure
struct Iec104Time2aDowdayType
{
    uint8_t dayOfMonth : 5;
    uint8_t dayOfWeek : 3;
}__attribute__((packed));

// Time2a Res3Month Structure
struct Iec104Time2aRes3monthType
{
    uint8_t month : 4;
    uint8_t res3 : 4;
}__attribute__((packed));

// Time2a Res4Year Structure
struct Iec104Time2aRes4yearType
{
    uint8_t year : 7;
    uint8_t res4 : 1;
}__attribute__((packed));

// CP56Time2a Structure
struct Iec104Cp56Time2aType
{
    Iec104Time2aMillisecondsType milliseconds;
    Iec104Time2aIvresminuteType ivresminute;
    Iec104Time2aSures2hourType sures2hour;
    Iec104Time2aDowdayType dowday;
    Iec104Time2aRes3monthType res3month;
    Iec104Time2aRes4yearType res4year;
}__attribute__((packed));

// Cp24Time2a Structure
struct Iec104Cp24Time2aType
{
    Iec104Time2aMillisecondsType milliseconds;
    Iec104Time2aIvresminuteType ivresminute;
}__attribute__((packed));

// Cp16Time2a Structure
struct Iec104Cp16Time2aType
{
    Iec104Time2aMillisecondsType milliseconds;
}__attribute__((packed));

// NOF: Name of File Structure
struct Iec104NofType
{
    uint16_t nameOfFile;
}__attribute__((packed));

// NOS: Name of Section Structure
struct Iec104NosType
{
    uint8_t nameOfSection;
}__attribute__((packed));

// LOF: Length of File or Section Structure
struct Iec104LofType
{
    uint8_t lengthOfFile[3];
}__attribute__((packed));

// LOS: Length of Segment Structure
struct Iec104LosType
{
    uint8_t lengthOfSegment;
}__attribute__((packed));

// CHS: Checksum Structure
struct Iec104ChsType
{
    uint8_t chs;
}__attribute__((packed));

// SOF: Status of File Structure
// need to prepend `sof` tag on here since `for` is a reserved word
// doing it for the rest for consistency
struct Iec104SofType
{
    uint8_t sofStatus : 5;
    uint8_t sofLfd : 1;
    uint8_t sofFor : 1;
    uint8_t sofFa : 1;
}__attribute__((packed));

// QOS: Qualifier of Set Point Command Structure
struct Iec104QosType
{
    uint8_t ql : 7;
    uint8_t se : 1;
}__attribute__((packed));

// SCD: Status + Status Change Detection Structure
struct Iec104ScdType
{
    uint16_t st;
    uint16_t cd;
}__attribute__((packed));

// TSC: Test Sequence Counter
struct Iec104TscType
{
    uint16_t tsc;
}__attribute__((packed));

// Segment: Segment type
struct Iec104SegmentType
{
    uint8_t segment;
}__attribute__((packed));

// Information Element
struct Iec104InformationElementType
{
    Iec104NofType nameOfFileOrSubdirectory;
    Iec104LofType lengthOfFile;
    Iec104SofType sof;
    Iec104Cp56Time2aType creationTimeOfFile;
}__attribute__((packed));


//
//
// ASDU structs
//
//

//
// ASDUs for process information in monitor direction
//

// ASDU Type M_SP_NA_1
// Ident 1
// Single-point information

struct Iec104M_SP_NA_1_IO_Subgroup
{
    Iec104SiqType siq;
}__attribute__((packed));

struct Iec104M_SP_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_SP_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_SP_TA_1
// Ident 2
// Single-point information with time tag

struct Iec104M_SP_TA_1_IO_Subgroup
{
    Iec104SiqType siq;
    Iec104Cp24Time2aType threeOctetBinaryTime;
}__attribute__((packed));

struct Iec104M_SP_TA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_SP_TA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_DP_NA_1
// Ident 3
// Double-point information

struct Iec104M_DP_NA_1_IO_Subgroup
{
    Iec104DiqType diq;
}__attribute__((packed));

struct Iec104M_DP_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_DP_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_DP_TA_1
// Ident 4
// Double-point information with time tag

struct Iec104M_DP_TA_1_IO_Subgroup
{
    Iec104DiqType diq;
    Iec104Cp24Time2aType threeOctetBinaryTime;
}__attribute__((packed));

struct Iec104M_DP_TA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_DP_TA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_ST_NA_1
// Ident 5
// Step position information

struct Iec104M_ST_NA_1_IO_Subgroup
{
    Iec104VtiType vti;
    Iec104QdsType qds;
}__attribute__((packed));

struct Iec104M_ST_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_ST_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_ST_TA_1
// Ident 6
// Step position information with time tag

struct Iec104M_ST_TA_1_IO_Subgroup
{
    Iec104VtiType vti;
    Iec104QdsType qds;
    Iec104Cp24Time2aType threeOctetBinaryTime;
}__attribute__((packed));

struct Iec104M_ST_TA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_ST_TA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_BO_NA_1
// Ident 7
// Bitstring of 32 bit

struct Iec104M_BO_NA_1_IO_Subgroup
{
    Iec104BsiType bsi;
    Iec104QdsType qds;
}__attribute__((packed));

struct Iec104M_BO_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_BO_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_BO_TA_1
// Ident 8
// Bitstring of 32 bit with time tag

struct Iec104M_BO_TA_1_IO_Subgroup
{
    Iec104BsiType bsi;
    Iec104QdsType qds;
    Iec104Cp24Time2aType threeOctetBinaryTime;
}__attribute__((packed));

struct Iec104M_BO_TA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_BO_TA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_ME_NA_1
// Ident 9
// Measured value, normalized value

struct Iec104M_ME_NA_1_IO_Subgroup
{
    Iec104NvaType nva;
    Iec104QdsType qds;
}__attribute__((packed));

struct Iec104M_ME_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_ME_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_ME_TA_1
// Ident 10
// Measured value, normalized value with time tag

struct Iec104M_ME_TA_1_IO_Subgroup
{
    Iec104NvaType nva;
    Iec104QdsType qds;
    Iec104Cp24Time2aType threeOctetBinaryTime;
}__attribute__((packed));

struct Iec104M_ME_TA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_ME_TA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_ME_NB_1
// Ident 11
// Measured value, scaled value

struct Iec104M_ME_NB_1_IO_Subgroup
{
    Iec104SvaType sva;
    Iec104QdsType qds;
}__attribute__((packed));

struct Iec104M_ME_NB_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_ME_NB_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_ME_TB_1
// Ident 12
// Measured value, scaled value wit time tag

struct Iec104M_ME_TB_1_IO_Subgroup
{
    Iec104SvaType sva;
    Iec104QdsType qds;
    Iec104Cp24Time2aType threeOctetBinaryTime;
}__attribute__((packed));

struct Iec104M_ME_TB_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_ME_TB_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_ME_NC_1
// Ident 13
// Measured value, short floating point number

struct Iec104M_ME_NC_1_IO_Subgroup
{
    Iec104IeeeStd754Type ieeeStd754;
    Iec104QdsType qds;
}__attribute__((packed));

struct Iec104M_ME_NC_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_ME_NC_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_ME_TC_1
// Ident 14
// Measured value, short floating point number with time tag

struct Iec104M_ME_TC_1_IO_Subgroup
{
    Iec104IeeeStd754Type ieeeStd754;
    Iec104QdsType qds;
    Iec104Cp24Time2aType threeOctetBinaryTime;
}__attribute__((packed));

struct Iec104M_ME_TC_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_ME_TC_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_IT_NA_1
// Ident 15
// Integrated totals

struct Iec104M_IT_NA_1_IO_Subgroup
{
    Iec104BcrType bcr;
}__attribute__((packed));

struct Iec104M_IT_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_IT_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_IT_TA_1
// Ident 16
// Integrated totals with time tag

struct Iec104M_IT_TA_1_IO_Subgroup
{
    Iec104BcrType bcr;
    Iec104Cp24Time2aType threeOctetBinaryTime;
}__attribute__((packed));

struct Iec104M_IT_TA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_IT_TA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_EP_TA_1
// Ident 17
// Event of protection equipment with time tag

struct Iec104M_EP_TA_1_IO_Subgroup
{
    Iec104SepType sep;
    Iec104Cp16Time2aType elapsedTime;
    Iec104Cp24Time2aType threeOctetBinaryTime;
}__attribute__((packed));

struct Iec104M_EP_TA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_EP_TA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_EP_TB_1
// Ident 18
// Packed start events of protection equipment with time tag

struct Iec104M_EP_TB_1_IO_Subgroup
{
    Iec104SpeType spe;
    Iec104QdpType qdp;
    Iec104Cp16Time2aType relayDurationTime;
    Iec104Cp24Time2aType threeOctetBinaryTime;
}__attribute__((packed));

struct Iec104M_EP_TB_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_EP_TB_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_EP_TC_1
// Ident 19
// Packed output circuit information of protection equipment with time tag

struct Iec104M_EP_TC_1_IO_Subgroup
{
    Iec104OciType oci;
    Iec104QdpType qdp;
    Iec104Cp16Time2aType relayOperatingTime;
    Iec104Cp24Time2aType threeOctetBinaryTime;
}__attribute__((packed));

struct Iec104M_EP_TC_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_EP_TC_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_PS_NA_1
// Ident 20
// Packed single point information with status change detection

struct Iec104M_PS_NA_1_IO_Subgroup
{
    Iec104ScdType scd;
    Iec104QdsType qds;
}__attribute__((packed));

struct Iec104M_PS_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_PS_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_ME_ND_1
// Ident 21
// Measured value, normalized value without quality descriptor

struct Iec104M_ME_ND_1_IO_Subgroup
{
    Iec104NvaType nva;
}__attribute__((packed));

struct Iec104M_ME_ND_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_ME_ND_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_SP_TB_1
// Ident 30
// Single-point information with time tag CP56Time2a

struct Iec104M_SP_TB_1_IO_Subgroup
{
    Iec104SiqType siq;
    Iec104Cp56Time2aType sevenOctetBinaryTime;
}__attribute__((packed));

struct Iec104M_SP_TB_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_SP_TB_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_DP_TB_1
// Ident 31
// Double-point information with time tag CP56Time2a

struct Iec104M_DP_TB_1_IO_Subgroup
{
    Iec104DiqType diq;
    Iec104Cp56Time2aType sevenOctetBinaryTime;
}__attribute__((packed));

struct Iec104M_DP_TB_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_DP_TB_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_ST_TB_1
// Ident 32
// Step position information with time tag CP56Time2a

struct Iec104M_ST_TB_1_IO_Subgroup
{
    Iec104VtiType vti;
    Iec104QdsType qds;
    Iec104Cp56Time2aType sevenOctetBinaryTime;
}__attribute__((packed));

struct Iec104M_ST_TB_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_ST_TB_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_BO_TB_1
// Ident 33
// Bitstring of 32 bit with time tag CP56Time2a

struct Iec104M_BO_TB_1_IO_Subgroup
{
    Iec104BsiType bsi;
    Iec104QdsType qds;
    Iec104Cp56Time2aType sevenOctetBinaryTime;
}__attribute__((packed));

struct Iec104M_BO_TB_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_BO_TB_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_ME_TD_1
// Ident 34
// Measured value, normalized value with time tag CP56Time2a

struct Iec104M_ME_TD_1_IO_Subgroup
{
    Iec104NvaType nva;
    Iec104QdsType qds;
    Iec104Cp56Time2aType sevenOctetBinaryTime;
}__attribute__((packed));

struct Iec104M_ME_TD_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_ME_TD_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_ME_TE_1
// Ident 35
// Measured value, scaled value with time tag CP56Time2a

struct Iec104M_ME_TE_1_IO_Subgroup
{
    Iec104SvaType sva;
    Iec104QdsType qds;
    Iec104Cp56Time2aType sevenOctetBinaryTime;
}__attribute__((packed));

struct Iec104M_ME_TE_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_ME_TE_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_ME_TF_1
// Ident 36
// Measured value, short floating point number with time tag CP56Time2a

struct Iec104M_ME_TF_1_IO_Subgroup
{
    Iec104IeeeStd754Type ieeeStd754;
    Iec104QdsType qds;
    Iec104Cp56Time2aType sevenOctetBinaryTime;
}__attribute__((packed));

struct Iec104M_ME_TF_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_ME_TF_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_IT_TB_1
// Ident 37
// Integrated totals with time tag CP56Time2a

struct Iec104M_IT_TB_1_IO_Subgroup
{
    Iec104BcrType bcr;
    Iec104Cp56Time2aType sevenOctetBinaryTime;
}__attribute__((packed));

struct Iec104M_IT_TB_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_IT_TB_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_EP_TD_1
// Ident 38
// Event of protection equipment with time tag CP56Time2a

struct Iec104M_EP_TD_1_IO_Subgroup
{
    Iec104SepType sep;
    Iec104Cp16Time2aType elapsedTime;
    Iec104Cp56Time2aType sevenOctetBinaryTime;
}__attribute__((packed));

struct Iec104M_EP_TD_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_EP_TD_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_EP_TE_1
// Ident 39
// Packed start events of protection equipment with time tag CP56Time2a

struct Iec104M_EP_TE_1_IO_Subgroup
{
    Iec104SepType sep;
    Iec104QdpType qdp;
    Iec104Cp16Time2aType relayDurationTime;
    Iec104Cp56Time2aType sevenOctetBinaryTime;
}__attribute__((packed));

struct Iec104M_EP_TE_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_EP_TE_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type M_EP_TF_1
// Ident 40
// Packed output circuit information of protection equipment with time tag CP56Time2a

struct Iec104M_EP_TF_1_IO_Subgroup
{
    Iec104OciType oci;
    Iec104QdpType qdp;
    Iec104Cp16Time2aType relayDurationTime;
    Iec104Cp56Time2aType sevenOctetBinaryTime;
}__attribute__((packed));

struct Iec104M_EP_TF_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_EP_TF_1_IO_Subgroup subgroup;
}__attribute__((packed));



//
// ASDUs for process information in control direction
//

// ASDU Type C_SC_NA_1

struct Iec104C_SC_NA_1_IO_Subgroup
{
    Iec104ScoType sco;
}__attribute__((packed));

struct Iec104C_SC_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104C_SC_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type C_DC_NA_1

struct Iec104C_DC_NA_1_IO_Subgroup
{
    Iec104DcoType dco;
}__attribute__((packed));

struct Iec104C_DC_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104C_DC_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type C_RC_NA_1

struct Iec104C_RC_NA_1_IO_Subgroup
{
    Iec104RcoType rco;
}__attribute__((packed));

struct Iec104C_RC_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104C_RC_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type C_SE_NA_1

struct Iec104C_SE_NA_1_IO_Subgroup
{
    Iec104NvaType nva;
    Iec104QosType qos;
}__attribute__((packed));

struct Iec104C_SE_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104C_SE_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type C_SE_NB_1

struct Iec104C_SE_NB_1_IO_Subgroup
{
    Iec104SvaType sva;
    Iec104QosType qos;
}__attribute__((packed));

struct Iec104C_SE_NB_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104C_SE_NB_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type C_SE_NC_1

struct Iec104C_SE_NC_1_IO_Subgroup
{
    Iec104IeeeStd754Type ieeeStd754;
    Iec104QosType qos;
}__attribute__((packed));

struct Iec104C_SE_NC_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104C_SE_NC_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type C_BO_NA_1

struct Iec104C_BO_NA_1_IO_Subgroup
{
    Iec104BsiType bsi;
}__attribute__((packed));

struct Iec104C_BO_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104C_BO_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type C_SC_TA_1
// Ident 58
// Single command with time tag CP56Time2a
//   IEC-60870-5-104

struct Iec104C_SC_TA_1_IO_Subgroup
{
    Iec104ScoType sco;
    Iec104Cp56Time2aType sevenOctetBinaryTime;
}__attribute__((packed));

struct Iec104C_SC_TA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104C_SC_TA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type C_DC_TA_1
// Ident 59
// Double command with time tag CP56Time2a
//   IEC-60870-5-104

struct Iec104C_DC_TA_1_IO_Subgroup
{
    Iec104DcoType dco;
    Iec104Cp56Time2aType sevenOctetBinaryTime;
}__attribute__((packed));

struct Iec104C_DC_TA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104C_DC_TA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type C_RC_TA_1
// Ident 60
// Regulating step command with time tag CP56Time2a
//   IEC-60870-5-104

struct Iec104C_RC_TA_1_IO_Subgroup
{
    Iec104RcoType rco;
    Iec104Cp56Time2aType sevenOctetBinaryTime;
}__attribute__((packed));

struct Iec104C_RC_TA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104C_RC_TA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type C_SE_TA_1
// Ident 61
// Set-point command with time tag CP56Time2a, normalized value
//   IEC-60870-5-104

struct Iec104C_SE_TA_1_IO_Subgroup
{
    Iec104NvaType nva;
    Iec104QosType qos;
    Iec104Cp56Time2aType sevenOctetBinaryTime;
}__attribute__((packed));

struct Iec104C_SE_TA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104C_SE_TA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type C_SE_TB_1
// Ident 62
// Set-point command with time tag CP56Time2a, scaled value
//   IEC-60870-5-104

struct Iec104C_SE_TB_1_IO_Subgroup
{
    Iec104SvaType sva;
    Iec104QosType qos;
    Iec104Cp56Time2aType sevenOctetBinaryTime;
}__attribute__((packed));

struct Iec104C_SE_TB_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104C_SE_TB_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type C_SE_TC_1
// Ident 63
// Set-point command with time tag CP56Time2a, short floating point number
//   IEC-60870-5-104

struct Iec104C_SE_TC_1_IO_Subgroup
{
    Iec104IeeeStd754Type ieeeStd754;
    Iec104QosType qos;
    Iec104Cp56Time2aType sevenOctetBinaryTime;
}__attribute__((packed));

struct Iec104C_SE_TC_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104C_SE_TC_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type C_BO_TA_1
// Ident 64
// Bitstring of 32 bit with time tag CP56Time2a
//   IEC-60870-5-104

struct Iec104C_BO_TA_1_IO_Subgroup
{
    Iec104BsiType bsi;
    Iec104Cp56Time2aType sevenOctetBinaryTime;
}__attribute__((packed));

struct Iec104C_BO_TA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104C_BO_TA_1_IO_Subgroup subgroup;
}__attribute__((packed));



//
// ASDUs for system information in monitor direction
//

// ASDU Type M_EI_NA_1
// Ident 70
// End of initialization

struct Iec104M_EI_NA_1_IO_Subgroup
{
    Iec104CoiType coi;
}__attribute__((packed));

struct Iec104M_EI_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104M_EI_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));



//
// ASDUs for system information in control direction
//

// ASDU Type C_IC_NA_1
// Ident 100
// Interrogation command

struct Iec104C_IC_NA_1_IO_Subgroup
{
    Iec104QoiType qoi;
}__attribute__((packed));

struct Iec104C_IC_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104C_IC_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type C_CI_NA_1
// Ident 101
// Counter interrogation command

struct Iec104C_CI_NA_1_IO_Subgroup
{
    Iec104QccType qcc;
}__attribute__((packed));

struct Iec104C_CI_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104C_CI_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type C_RD_NA_1
// Ident 102
// Read command

struct Iec104C_RD_NA_1_IO_Subgroup
{
    // No subgroup for this type
}__attribute__((packed));

struct Iec104C_RD_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104C_RD_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type C_CS_NA_1
// Ident 103
// Clock synchronization command

struct Iec104C_CS_NA_1_IO_Subgroup
{
    Iec104Cp56Time2aType sevenOctetBinaryTime;
}__attribute__((packed));

struct Iec104C_CS_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104C_CS_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type C_TS_NA_1
// Ident 104
// Test command

struct Iec104C_TS_NA_1_IO_Subgroup
{
    Iec104FbpType fbp;
}__attribute__((packed));

struct Iec104C_TS_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104C_TS_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type C_RP_NA_1
// Ident 105
// Reset process command

struct Iec104C_RP_NA_1_IO_Subgroup
{
    Iec104QrpType qrp;
}__attribute__((packed));

struct Iec104C_RP_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104C_RP_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type C_CD_NA_1
// Ident 106
// Delay acquisition command

struct Iec104C_CD_NA_1_IO_Subgroup
{
    Iec104Cp16Time2aType msUpToSeconds;
}__attribute__((packed));

struct Iec104C_CD_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104C_CD_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type C_TS_TA_1
// Ident 107
// Test command with time tag CP56Time2a
//   IEC-60870-5-104

struct Iec104C_TS_TA_1_IO_Subgroup
{
    Iec104TscType tsc;
    Iec104Cp56Time2aType sevenOctetBinaryTime;
}__attribute__((packed));

struct Iec104C_TS_TA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104C_TS_TA_1_IO_Subgroup subgroup;
}__attribute__((packed));



//
// ASDUs for parameter in control direction
//

// ASDU Type P_ME_NA_1
// Ident 110
// Parameter of measured values, normalized value

struct Iec104P_ME_NA_1_IO_Subgroup
{
    Iec104NvaType nva;
    Iec104QpmType qpm;
}__attribute__((packed));

struct Iec104P_ME_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104P_ME_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type P_ME_NB_1
// Ident 111
// Parameter of measured values, scaled value

struct Iec104P_ME_NB_1_IO_Subgroup
{
    Iec104SvaType sva;
    Iec104QpmType qpm;
}__attribute__((packed));

struct Iec104P_ME_NB_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104P_ME_NB_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type P_ME_NC_1
// Ident 112
// Parameter of measured values, short floating point number

struct Iec104P_ME_NC_1_IO_Subgroup
{
    Iec104IeeeStd754Type ieeeStd754;
    Iec104QpmType qpm;
}__attribute__((packed));

struct Iec104P_ME_NC_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104P_ME_NC_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type P_AC_NA_1
// Ident 113
// Parameter activation

struct Iec104P_AC_NA_1_IO_Subgroup
{
    Iec104QpaType qpa;
}__attribute__((packed));

struct Iec104P_AC_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104P_AC_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));



//
// ASDUs for file transfer
//

// ASDU Type F_FR_NA_1
// Ident 120
// File ready

struct Iec104F_FR_NA_1_IO_Subgroup
{
    Iec104NofType nameOfFile;
    Iec104LofType lengthOfFile;
    Iec104FrqType frq;
}__attribute__((packed));

struct Iec104F_FR_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104F_FR_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type F_SR_NA_1
// Ident 121
// Section ready

struct Iec104F_SR_NA_1_IO_Subgroup
{
    Iec104NofType nameOfFile;
    Iec104NosType nameOfSection;
    Iec104LofType lengthOfFileOrSection;
    Iec104SrqType srq;
}__attribute__((packed));

struct Iec104F_SR_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104F_SR_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type F_SC_NA_1
// Ident 122
// Call directory, select file, call file, call section

struct Iec104F_SC_NA_1_IO_Subgroup
{
    Iec104NofType nameOfFile;
    Iec104NosType nameOfSection;
    Iec104ScqType scq;
}__attribute__((packed));

struct Iec104F_SC_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104F_SC_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type F_LS_NA_1
// Ident 123
// Last section, last segment

struct Iec104F_LS_NA_1_IO_Subgroup
{
    Iec104NofType nameOfFile;
    Iec104NosType nameOfSection;
    Iec104LsqType lsq;
    Iec104ChsType chs;
}__attribute__((packed));

struct Iec104F_LS_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104F_LS_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type F_AF_NA_1
// Ident 124
// ACK file, ACK section

struct Iec104F_AF_NA_1_IO_Subgroup
{
    Iec104NofType nameOfFile;
    Iec104NosType nameOfSection;
    Iec104AfqType afq;
}__attribute__((packed));

struct Iec104F_AF_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104F_AF_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type F_SG_NA_1
// Ident 125
// Single information object

struct Iec104F_SG_NA_1_IO_Subgroup
{
    Iec104NofType nameOfFile;
    Iec104NosType nameOfSection;
    Iec104LosType lengthOfSegment;
    Iec104SegmentType segment;
}__attribute__((packed));

struct Iec104F_SG_NA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104F_SG_NA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type F_DR_TA_1
// Ident 126
// Sequence of information elements in a single information object

struct Iec104F_DR_TA_1_IO_Subgroup
{
    Iec104NofType nameOfFileOrSubdirectory;
    Iec104LofType lengthOfFile;
    Iec104SofType sof;
    Iec104Cp56Time2aType creationTimeOfFile;
}__attribute__((packed));

struct Iec104F_DR_TA_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104F_DR_TA_1_IO_Subgroup subgroup;
}__attribute__((packed));


// ASDU Type F_SC_NB_1
// Ident 127
// QueryLog – Request archive file

struct Iec104F_SC_NB_1_IO_Subgroup
{
    Iec104NofType nameOfFile;
    Iec104Cp56Time2aType startTime;
    Iec104Cp56Time2aType stopTime;
}__attribute__((packed));

struct Iec104F_SC_NB_1_IO_Group
{
    Iec104InformationObjectAddressThreeOctetType ioa;
    Iec104F_SC_NB_1_IO_Subgroup subgroup;
}__attribute__((packed));


//
// Generic ASDU
//

struct Iec104GenericAsdu
{
    uint8_t typeId;
    Iec104VariableStructureQualifierType variableStructureQualifier;
    Iec104CauseOfTransmissionType causeOfTransmission;
    Iec104CommonAddressOfAsduType commonAddressOfAsdu;
    union
    {
        Iec104M_SP_NA_1_IO_Group m_sp_na_1;
        Iec104M_SP_TA_1_IO_Group m_sp_ta_1;
        Iec104M_DP_NA_1_IO_Group m_dp_na_1;
        Iec104M_DP_TA_1_IO_Group m_dp_ta_1;
        Iec104M_ST_NA_1_IO_Group m_st_na_1;
        Iec104M_ST_TA_1_IO_Group m_st_ta_1;
        Iec104M_BO_NA_1_IO_Group m_bo_na_1;
        Iec104M_BO_TA_1_IO_Group m_bo_ta_1;
        Iec104M_ME_NA_1_IO_Group m_me_na_1;
        Iec104M_ME_TA_1_IO_Group m_me_ta_1;
        Iec104M_ME_NB_1_IO_Group m_me_nb_1;
        Iec104M_ME_TB_1_IO_Group m_me_tb_1;
        Iec104M_ME_NC_1_IO_Group m_me_nc_1;
        Iec104M_ME_TC_1_IO_Group m_me_tc_1;
        Iec104M_IT_NA_1_IO_Group m_it_na_1;
        Iec104M_IT_TA_1_IO_Group m_it_ta_1;
        Iec104M_EP_TA_1_IO_Group m_ep_ta_1;
        Iec104M_EP_TB_1_IO_Group m_ep_tb_1;
        Iec104M_EP_TC_1_IO_Group m_ep_tc_1;
        Iec104M_PS_NA_1_IO_Group m_ps_na_1;
        Iec104M_ME_ND_1_IO_Group m_me_nd_1;
        Iec104M_SP_TB_1_IO_Group m_sp_tb_1;
        Iec104M_DP_TB_1_IO_Group m_dp_tb_1;
        Iec104M_ST_TB_1_IO_Group m_st_tb_1;
        Iec104M_BO_TB_1_IO_Group m_bo_tb_1;
        Iec104M_ME_TD_1_IO_Group m_me_td_1;
        Iec104M_ME_TE_1_IO_Group m_me_te_1;
        Iec104M_ME_TF_1_IO_Group m_me_tf_1;
        Iec104M_IT_TB_1_IO_Group m_it_tb_1;
        Iec104M_EP_TD_1_IO_Group m_ep_td_1;
        Iec104M_EP_TE_1_IO_Group m_ep_te_1;
        Iec104M_EP_TF_1_IO_Group m_ep_tf_1;
        Iec104C_SC_NA_1_IO_Group c_sc_na_1;
        Iec104C_DC_NA_1_IO_Group c_dc_na_1;
        Iec104C_RC_NA_1_IO_Group c_rc_na_1;
        Iec104C_SE_NA_1_IO_Group c_se_na_1;
        Iec104C_SE_NB_1_IO_Group c_se_nb_1;
        Iec104C_SE_NC_1_IO_Group c_se_nc_1;
        Iec104C_BO_NA_1_IO_Group c_bo_na_1;
        Iec104C_SC_TA_1_IO_Group c_sc_ta_1;
        Iec104C_DC_TA_1_IO_Group c_dc_ta_1;
        Iec104C_RC_TA_1_IO_Group c_rc_ta_1;
        Iec104C_SE_TA_1_IO_Group c_se_ta_1;
        Iec104C_SE_TB_1_IO_Group c_se_tb_1;
        Iec104C_SE_TC_1_IO_Group c_se_tc_1;
        Iec104C_BO_TA_1_IO_Group c_bo_ta_1;
        Iec104M_EI_NA_1_IO_Group m_ei_na_1;
        Iec104C_IC_NA_1_IO_Group c_ic_na_1;
        Iec104C_CI_NA_1_IO_Group c_ci_na_1;
        Iec104C_RD_NA_1_IO_Group c_rd_na_1;
        Iec104C_CS_NA_1_IO_Group c_cs_na_1;
        Iec104C_TS_NA_1_IO_Group c_ts_na_1;
        Iec104C_RP_NA_1_IO_Group c_rp_na_1;
        Iec104C_CD_NA_1_IO_Group c_cd_na_1;
        Iec104C_TS_TA_1_IO_Group c_ts_ta_1;
        Iec104P_ME_NA_1_IO_Group p_me_na_1;
        Iec104P_ME_NB_1_IO_Group p_me_nb_1;
        Iec104P_ME_NC_1_IO_Group p_me_nc_1;
        Iec104P_AC_NA_1_IO_Group p_ac_na_1;
        Iec104F_FR_NA_1_IO_Group f_fr_na_1;
        Iec104F_SR_NA_1_IO_Group f_sr_na_1;
        Iec104F_SC_NA_1_IO_Group f_sc_na_1;
        Iec104F_LS_NA_1_IO_Group f_ls_na_1;
        Iec104F_AF_NA_1_IO_Group f_af_na_1;
        Iec104F_SG_NA_1_IO_Group f_sg_na_1;
        Iec104F_DR_TA_1_IO_Group f_dr_ta_1;
        Iec104F_SC_NB_1_IO_Group f_sc_nb_1;
    };
}__attribute__((packed));


//
// APCI structs
//

// Header fields common to every APCI
struct Iec104Header
{
    uint8_t start;
    uint8_t length;
}__attribute__((packed));

// APCI Type U
struct Iec104ApciU
{
    Iec104Header header;
    uint8_t apciTypeMajor : 1;
    uint8_t apciTypeMinor : 1;
    uint8_t startdtAct : 1;
    uint8_t startdtCon : 1;
    uint8_t stopdtAct : 1;
    uint8_t stopdtCon : 1;
    uint8_t testfrAct : 1;
    uint8_t testfrCon : 1;
    uint8_t reserved1;
    uint16_t reserved2 : 1;
    uint16_t reserved3 : 15;
}__attribute__((packed));

// APCI Type S
struct Iec104ApciS
{
    Iec104Header header;
    uint16_t apciTypeMajor : 1;
    uint16_t apciTypeMinor : 1;
    uint16_t reserved1 : 14;
    uint16_t reserved2 : 1;
    uint16_t recvSeq : 15;
}__attribute__((packed));

// APCI Type I
struct Iec104ApciI
{
    Iec104Header header;
    uint16_t apciTypeMajor : 1;
    uint16_t sendSeq : 15;
    uint16_t reserved : 1;
    uint16_t recvSeq : 15;
    Iec104GenericAsdu asdu;
}__attribute__((packed));

// structs used to determine if there is an issue with the passed ASDU
struct Iec104AsduCheckCauseOfTx
{
    uint64_t percyc : 1;
    uint64_t back : 1;
    uint64_t spont : 1;
    uint64_t init : 1;
    uint64_t req : 1;
    uint64_t act : 1;
    uint64_t actcon : 1;
    uint64_t deact : 1;
    uint64_t deactcon : 1;
    uint64_t actterm : 1;
    uint64_t retrem : 1;
    uint64_t retloc : 1;
    uint64_t file : 1;
    uint64_t inrogen : 1;
    uint64_t inro1 : 1;
    uint64_t inro2 : 1;
    uint64_t inro3 : 1;
    uint64_t inro4 : 1;
    uint64_t inro5 : 1;
    uint64_t inro6 : 1;
    uint64_t inro7 : 1;
    uint64_t inro8 : 1;
    uint64_t inro9 : 1;
    uint64_t inro10 : 1;
    uint64_t inro11 : 1;
    uint64_t inro12 : 1;
    uint64_t inro13 : 1;
    uint64_t inro14 : 1;
    uint64_t inro15 : 1;
    uint64_t inro16 : 1;
    uint64_t reqcogen : 1;
    uint64_t reqco1 : 1;
    uint64_t reqco2 : 1;
    uint64_t reqco3 : 1;
    uint64_t reqco4 : 1;
    uint64_t unk_type_id : 1;
    uint64_t unk_cause_tx : 1;
    uint64_t unk_common_addr : 1;
    uint64_t unk_info_addr : 1;
};

struct Iec104AsduCheck
{
    const Iec104ApciI* apci;
    bool sq0Allowed;
    bool sq1Allowed;
    bool multipleIOAllowed;
    Iec104AsduCheckCauseOfTx checkCauseOfTx;
};

#endif

iec104_parse_information_object_elements.h

Path = src/service_inspectors/iec104/iec104_parse_information_object_elements.h

#ifndef IEC104_PARSE_INFORMATION_OBJECT_ELEMENTS_H
#define IEC104_PARSE_INFORMATION_OBJECT_ELEMENTS_H

#include "iec104_parse_apdu.h"

#define IEC104_MS_IN_MINUTE 60000
#define IEC104_MINS_IN_HOUR 60
#define IEC104_HOURS_IN_DAY 24
#define IEC104_TIME2ARES4YEAR_BASE 1900
#define IEC104_TIME2ARES4YEAR_1970 70
#define IEC104_TIME2ARES4YEAR_2027 127
#define IEC104_MAX_DAYOFMONTH_FEB_LEAPYEAR 29
#define IEC104_MAX_DAYOFMONTH_FEB_NONLEAPYEAR 28
#define IEC104_MAX_DAYOFMONTH_30 30
#define IEC104_MAX_DAYOFMONTH_31 31

void parseIec104Coi(const Iec104CoiType* coi);
void parseIec104Qoi(const Iec104QoiType* qoi);
void parseIec104Qcc(const Iec104QccType* qcc);
void parseIec104Qpm(const Iec104QpmType* qpm);
void parseIec104Qpa(const Iec104QpaType* qpa);
void parseIec104Qoc(uint8_t qu, uint8_t se);
void parseIec104Qrp(const Iec104QrpType* qrp);
void parseIec104Frq(const Iec104FrqType* frq);
void parseIec104Srq(const Iec104SrqType* srq);
void parseIec104Scq(const Iec104ScqType* scq);
void parseIec104Lsq(const Iec104LsqType* lsq);
void parseIec104Afq(const Iec104AfqType* afq);
uint32_t parseIec104Vsq(const Iec104ApciI* apci);
void parseIec104CauseOfTx(const Iec104ApciI* apci);
void parseIec104TwoOctetCommonAddress(const Iec104ApciI* apci);
void parseIec104InformationObjectAddressWithThreeOctets(
    const Iec104InformationObjectAddressThreeOctetType* ioa);
void parseIec104Siq(const Iec104SiqType* siq);
void parseIec104Diq(const Iec104DiqType* diq);
void parseIec104Qds(const Iec104QdsType* qds);
void parseIec104Qdp(const Iec104QdpType* qdp);
void parseIec104Vti(const Iec104VtiType* vti);
void parseIec104Nva(const Iec104NvaType* nva);
void parseIec104Sva(const Iec104SvaType* sva);
void parseIec104IeeeStd754(const Iec104IeeeStd754Type* ieeeStd754);
void parseIec104Bcr(const Iec104BcrType* bcr);
void parseIec104Sep(const Iec104SepType* sep);
void parseIec104Spe(const Iec104SpeType* spe);
void parseIec104Oci(const Iec104OciType* oci);
void parseIec104Bsi(const Iec104BsiType* bsi);
void parseIec104Fbp(const Iec104FbpType* fbp);
void parseIec104Sco(const Iec104ScoType* sco);
void parseIec104Dco(const Iec104DcoType* dco);
void parseIec104Rco(const Iec104RcoType* rco);
void parseIec104Time2aMilliseconds(const Iec104Time2aMillisecondsType* time2aMilliseconds);
void parseIec104Time2aIvresminute(const Iec104Time2aIvresminuteType* time2aIvresminute);
void parseIec104Time2aSures2hour(const Iec104Time2aSures2hourType* time2aSures2hour);
void parseIec104Time2aDowday(const Iec104Cp56Time2aType* sevenOctetBinaryTime);
void parseIec104Time2aRes3month(const Iec104Time2aRes3monthType* time2aRes3month);
void parseIec104Time2aRes4year(const Iec104Time2aRes4yearType* time2aRes4year);
void parseIec104Cp56Time2a(const Iec104Cp56Time2aType* sevenOctetBinaryTime);
void parseIec104Cp24Time2a(const Iec104Cp24Time2aType* threeOctetBinaryTime);
void parseIec104Cp16Time2a(const Iec104Cp16Time2aType* cp16Time2a);
void parseIec104Nof(const Iec104NofType* nof);
void parseIec104Nos(const Iec104NosType* nos);
void parseIec104Lof(const Iec104LofType* lof);
bool parseIec104Los(const Iec104LosType* los, uint16_t apduSize);
void parseIec104Chs(const Iec104ChsType* chs);
void parseIec104Sof(const Iec104SofType* sof);
void parseIec104Qos(const Iec104QosType* qos);
void parseIec104Scd(const Iec104ScdType* scd);
void parseIec104Tsc(const Iec104TscType* tsc);
void parseIec104Segment(const Iec104SegmentType* segment);

struct GenericIec104AsduIOGroup
{
    bool includeIOA;
    uint16_t apduSize;
    uint32_t asduType;
    union
    {
        const Iec104M_SP_NA_1_IO_Group* m_sp_na_1IOGroup;
        const Iec104M_SP_TA_1_IO_Group* m_sp_ta_1IOGroup;
        const Iec104M_DP_NA_1_IO_Group* m_dp_na_1IOGroup;
        const Iec104M_DP_TA_1_IO_Group* m_dp_ta_1IOGroup;
        const Iec104M_ST_NA_1_IO_Group* m_st_na_1IOGroup;
        const Iec104M_ST_TA_1_IO_Group* m_st_ta_1IOGroup;
        const Iec104M_BO_NA_1_IO_Group* m_bo_na_1IOGroup;
        const Iec104M_BO_TA_1_IO_Group* m_bo_ta_1IOGroup;
        const Iec104M_ME_NA_1_IO_Group* m_me_na_1IOGroup;
        const Iec104M_ME_TA_1_IO_Group* m_me_ta_1IOGroup;
        const Iec104M_ME_NB_1_IO_Group* m_me_nb_1IOGroup;
        const Iec104M_ME_TB_1_IO_Group* m_me_tb_1IOGroup;
        const Iec104M_ME_NC_1_IO_Group* m_me_nc_1IOGroup;
        const Iec104M_ME_TC_1_IO_Group* m_me_tc_1IOGroup;
        const Iec104M_IT_NA_1_IO_Group* m_it_na_1IOGroup;
        const Iec104M_IT_TA_1_IO_Group* m_it_ta_1IOGroup;
        const Iec104M_EP_TA_1_IO_Group* m_ep_ta_1IOGroup;
        const Iec104M_EP_TB_1_IO_Group* m_ep_tb_1IOGroup;
        const Iec104M_EP_TC_1_IO_Group* m_ep_tc_1IOGroup;
        const Iec104M_PS_NA_1_IO_Group* m_ps_na_1IOGroup;
        const Iec104M_ME_ND_1_IO_Group* m_me_nd_1IOGroup;
        const Iec104M_SP_TB_1_IO_Group* m_sp_tb_1IOGroup;
        const Iec104M_DP_TB_1_IO_Group* m_dp_tb_1IOGroup;
        const Iec104M_ST_TB_1_IO_Group* m_st_tb_1IOGroup;
        const Iec104M_BO_TB_1_IO_Group* m_bo_tb_1IOGroup;
        const Iec104M_ME_TD_1_IO_Group* m_me_td_1IOGroup;
        const Iec104M_ME_TE_1_IO_Group* m_me_te_1IOGroup;
        const Iec104M_ME_TF_1_IO_Group* m_me_tf_1IOGroup;
        const Iec104M_IT_TB_1_IO_Group* m_it_tb_1IOGroup;
        const Iec104M_EP_TD_1_IO_Group* m_ep_td_1IOGroup;
        const Iec104M_EP_TE_1_IO_Group* m_ep_te_1IOGroup;
        const Iec104M_EP_TF_1_IO_Group* m_ep_tf_1IOGroup;
        const Iec104C_SC_NA_1_IO_Group* c_sc_na_1IOGroup;
        const Iec104C_DC_NA_1_IO_Group* c_dc_na_1IOGroup;
        const Iec104C_RC_NA_1_IO_Group* c_rc_na_1IOGroup;
        const Iec104C_SE_NA_1_IO_Group* c_se_na_1IOGroup;
        const Iec104C_SE_NB_1_IO_Group* c_se_nb_1IOGroup;
        const Iec104C_SE_NC_1_IO_Group* c_se_nc_1IOGroup;
        const Iec104C_BO_NA_1_IO_Group* c_bo_na_1IOGroup;
        const Iec104C_SC_TA_1_IO_Group* c_sc_ta_1IOGroup;
        const Iec104C_DC_TA_1_IO_Group* c_dc_ta_1IOGroup;
        const Iec104C_RC_TA_1_IO_Group* c_rc_ta_1IOGroup;
        const Iec104C_SE_TA_1_IO_Group* c_se_ta_1IOGroup;
        const Iec104C_SE_TB_1_IO_Group* c_se_tb_1IOGroup;
        const Iec104C_SE_TC_1_IO_Group* c_se_tc_1IOGroup;
        const Iec104C_BO_TA_1_IO_Group* c_bo_ta_1IOGroup;
        const Iec104M_EI_NA_1_IO_Group* m_ei_na_1IOGroup;
        const Iec104C_IC_NA_1_IO_Group* c_ic_na_1IOGroup;
        const Iec104C_CI_NA_1_IO_Group* c_ci_na_1IOGroup;
        const Iec104C_RD_NA_1_IO_Group* c_rd_na_1IOGroup;
        const Iec104C_CS_NA_1_IO_Group* c_cs_na_1IOGroup;
        const Iec104C_TS_NA_1_IO_Group* c_ts_na_1IOGroup;
        const Iec104C_RP_NA_1_IO_Group* c_rp_na_1IOGroup;
        const Iec104C_CD_NA_1_IO_Group* c_cd_na_1IOGroup;
        const Iec104C_TS_TA_1_IO_Group* c_ts_ta_1IOGroup;
        const Iec104P_ME_NA_1_IO_Group* p_me_na_1IOGroup;
        const Iec104P_ME_NB_1_IO_Group* p_me_nb_1IOGroup;
        const Iec104P_ME_NC_1_IO_Group* p_me_nc_1IOGroup;
        const Iec104P_AC_NA_1_IO_Group* p_ac_na_1IOGroup;
        const Iec104F_FR_NA_1_IO_Group* f_fr_na_1IOGroup;
        const Iec104F_SR_NA_1_IO_Group* f_sr_na_1IOGroup;
        const Iec104F_SC_NA_1_IO_Group* f_sc_na_1IOGroup;
        const Iec104F_LS_NA_1_IO_Group* f_ls_na_1IOGroup;
        const Iec104F_AF_NA_1_IO_Group* f_af_na_1IOGroup;
        const Iec104F_SG_NA_1_IO_Group* f_sg_na_1IOGroup;
        const Iec104F_DR_TA_1_IO_Group* f_dr_ta_1IOGroup;
        const Iec104F_SC_NB_1_IO_Group* f_sc_nb_1IOGroup;
    };
    union
    {
        const Iec104M_SP_NA_1_IO_Subgroup* m_sp_na_1IOSubgroup;
        const Iec104M_SP_TA_1_IO_Subgroup* m_sp_ta_1IOSubgroup;
        const Iec104M_DP_NA_1_IO_Subgroup* m_dp_na_1IOSubgroup;
        const Iec104M_DP_TA_1_IO_Subgroup* m_dp_ta_1IOSubgroup;
        const Iec104M_ST_NA_1_IO_Subgroup* m_st_na_1IOSubgroup;
        const Iec104M_ST_TA_1_IO_Subgroup* m_st_ta_1IOSubgroup;
        const Iec104M_BO_NA_1_IO_Subgroup* m_bo_na_1IOSubgroup;
        const Iec104M_BO_TA_1_IO_Subgroup* m_bo_ta_1IOSubgroup;
        const Iec104M_ME_NA_1_IO_Subgroup* m_me_na_1IOSubgroup;
        const Iec104M_ME_TA_1_IO_Subgroup* m_me_ta_1IOSubgroup;
        const Iec104M_ME_NB_1_IO_Subgroup* m_me_nb_1IOSubgroup;
        const Iec104M_ME_TB_1_IO_Subgroup* m_me_tb_1IOSubgroup;
        const Iec104M_ME_NC_1_IO_Subgroup* m_me_nc_1IOSubgroup;
        const Iec104M_ME_TC_1_IO_Subgroup* m_me_tc_1IOSubgroup;
        const Iec104M_IT_NA_1_IO_Subgroup* m_it_na_1IOSubgroup;
        const Iec104M_IT_TA_1_IO_Subgroup* m_it_ta_1IOSubgroup;
        const Iec104M_EP_TA_1_IO_Subgroup* m_ep_ta_1IOSubgroup;
        const Iec104M_EP_TB_1_IO_Subgroup* m_ep_tb_1IOSubgroup;
        const Iec104M_EP_TC_1_IO_Subgroup* m_ep_tc_1IOSubgroup;
        const Iec104M_PS_NA_1_IO_Subgroup* m_ps_na_1IOSubgroup;
        const Iec104M_ME_ND_1_IO_Subgroup* m_me_nd_1IOSubgroup;
        const Iec104M_SP_TB_1_IO_Subgroup* m_sp_tb_1IOSubgroup;
        const Iec104M_DP_TB_1_IO_Subgroup* m_dp_tb_1IOSubgroup;
        const Iec104M_ST_TB_1_IO_Subgroup* m_st_tb_1IOSubgroup;
        const Iec104M_BO_TB_1_IO_Subgroup* m_bo_tb_1IOSubgroup;
        const Iec104M_ME_TD_1_IO_Subgroup* m_me_td_1IOSubgroup;
        const Iec104M_ME_TE_1_IO_Subgroup* m_me_te_1IOSubgroup;
        const Iec104M_ME_TF_1_IO_Subgroup* m_me_tf_1IOSubgroup;
        const Iec104M_IT_TB_1_IO_Subgroup* m_it_tb_1IOSubgroup;
        const Iec104M_EP_TD_1_IO_Subgroup* m_ep_td_1IOSubgroup;
        const Iec104M_EP_TE_1_IO_Subgroup* m_ep_te_1IOSubgroup;
        const Iec104M_EP_TF_1_IO_Subgroup* m_ep_tf_1IOSubgroup;
        const Iec104C_SC_NA_1_IO_Subgroup* c_sc_na_1IOSubgroup;
        const Iec104C_DC_NA_1_IO_Subgroup* c_dc_na_1IOSubgroup;
        const Iec104C_RC_NA_1_IO_Subgroup* c_rc_na_1IOSubgroup;
        const Iec104C_SE_NA_1_IO_Subgroup* c_se_na_1IOSubgroup;
        const Iec104C_SE_NB_1_IO_Subgroup* c_se_nb_1IOSubgroup;
        const Iec104C_SE_NC_1_IO_Subgroup* c_se_nc_1IOSubgroup;
        const Iec104C_BO_NA_1_IO_Subgroup* c_bo_na_1IOSubgroup;
        const Iec104C_SC_TA_1_IO_Subgroup* c_sc_ta_1IOSubgroup;
        const Iec104C_DC_TA_1_IO_Subgroup* c_dc_ta_1IOSubgroup;
        const Iec104C_RC_TA_1_IO_Subgroup* c_rc_ta_1IOSubgroup;
        const Iec104C_SE_TA_1_IO_Subgroup* c_se_ta_1IOSubgroup;
        const Iec104C_SE_TB_1_IO_Subgroup* c_se_tb_1IOSubgroup;
        const Iec104C_SE_TC_1_IO_Subgroup* c_se_tc_1IOSubgroup;
        const Iec104C_BO_TA_1_IO_Subgroup* c_bo_ta_1IOSubgroup;
        const Iec104M_EI_NA_1_IO_Subgroup* m_ei_na_1IOSubgroup;
        const Iec104C_IC_NA_1_IO_Subgroup* c_ic_na_1IOSubgroup;
        const Iec104C_CI_NA_1_IO_Subgroup* c_ci_na_1IOSubgroup;
        const Iec104C_RD_NA_1_IO_Subgroup* c_rd_na_1IOSubgroup;
        const Iec104C_CS_NA_1_IO_Subgroup* c_cs_na_1IOSubgroup;
        const Iec104C_TS_NA_1_IO_Subgroup* c_ts_na_1IOSubgroup;
        const Iec104C_RP_NA_1_IO_Subgroup* c_rp_na_1IOSubgroup;
        const Iec104C_CD_NA_1_IO_Subgroup* c_cd_na_1IOSubgroup;
        const Iec104C_TS_TA_1_IO_Subgroup* c_ts_ta_1IOSubgroup;
        const Iec104P_ME_NA_1_IO_Subgroup* p_me_na_1IOSubgroup;
        const Iec104P_ME_NB_1_IO_Subgroup* p_me_nb_1IOSubgroup;
        const Iec104P_ME_NC_1_IO_Subgroup* p_me_nc_1IOSubgroup;
        const Iec104P_AC_NA_1_IO_Subgroup* p_ac_na_1IOSubgroup;
        const Iec104F_FR_NA_1_IO_Subgroup* f_fr_na_1IOSubgroup;
        const Iec104F_SR_NA_1_IO_Subgroup* f_sr_na_1IOSubgroup;
        const Iec104F_SC_NA_1_IO_Subgroup* f_sc_na_1IOSubgroup;
        const Iec104F_LS_NA_1_IO_Subgroup* f_ls_na_1IOSubgroup;
        const Iec104F_AF_NA_1_IO_Subgroup* f_af_na_1IOSubgroup;
        const Iec104F_SG_NA_1_IO_Subgroup* f_sg_na_1IOSubgroup;
        const Iec104F_DR_TA_1_IO_Subgroup* f_dr_ta_1IOSubgroup;
        const Iec104F_SC_NB_1_IO_Subgroup* f_sc_nb_1IOSubgroup;
    };
} __attribute__((packed));

enum Iec104MonthEnum
{
    IEC104_MONTH_JAN = 1,
    IEC104_MONTH_FEB = 2,
    IEC104_MONTH_MAR = 3,
    IEC104_MONTH_APR = 4,
    IEC104_MONTH_MAY = 5,
    IEC104_MONTH_JUN = 6,
    IEC104_MONTH_JUL = 7,
    IEC104_MONTH_AUG = 8,
    IEC104_MONTH_SEP = 9,
    IEC104_MONTH_OCT = 10,
    IEC104_MONTH_NOV = 11,
    IEC104_MONTH_DEC = 12,
};

enum Iec104WeekdayEnum
{
    IEC104_DAY_WEEKDAY_UNUSED = 0,
    IEC104_DAY_MONDAY = 1,
    IEC104_DAY_TUESDAY = 2,
    IEC104_DAY_WEDNESDAY = 3,
    IEC104_DAY_THURSDAY = 4,
    IEC104_DAY_FRIDAY = 5,
    IEC104_DAY_SATURDAY = 6,
    IEC104_DAY_SUNDAY = 7,
};

enum Iec104CoiUiEnum
{
    IEC104_COI_UI_LOCPOWON = 0,
    IEC104_COI_UI_LOCMANRST = 1,
    IEC104_COI_UI_REMRST = 2,
    IEC104_COI_UI_RES3 = 3,
    IEC104_COI_UI_RES4 = 4,
    IEC104_COI_UI_RES5 = 5,
    IEC104_COI_UI_RES6 = 6,
    IEC104_COI_UI_RES7 = 7,
    IEC104_COI_UI_RES8 = 8,
    IEC104_COI_UI_RES9 = 9,
    IEC104_COI_UI_RES10 = 10,
    IEC104_COI_UI_RES11 = 11,
    IEC104_COI_UI_RES12 = 12,
    IEC104_COI_UI_RES13 = 13,
    IEC104_COI_UI_RES14 = 14,
    IEC104_COI_UI_RES15 = 15,
    IEC104_COI_UI_RES16 = 16,
    IEC104_COI_UI_RES17 = 17,
    IEC104_COI_UI_RES18 = 18,
    IEC104_COI_UI_RES19 = 19,
    IEC104_COI_UI_RES20 = 20,
    IEC104_COI_UI_RES21 = 21,
    IEC104_COI_UI_RES22 = 22,
    IEC104_COI_UI_RES23 = 23,
    IEC104_COI_UI_RES24 = 24,
    IEC104_COI_UI_RES25 = 25,
    IEC104_COI_UI_RES26 = 26,
    IEC104_COI_UI_RES27 = 27,
    IEC104_COI_UI_RES28 = 28,
    IEC104_COI_UI_RES29 = 29,
    IEC104_COI_UI_RES30 = 30,
    IEC104_COI_UI_RES31 = 31,
    IEC104_COI_UI_RES32 = 32,
    IEC104_COI_UI_RES33 = 33,
    IEC104_COI_UI_RES34 = 34,
    IEC104_COI_UI_RES35 = 35,
    IEC104_COI_UI_RES36 = 36,
    IEC104_COI_UI_RES37 = 37,
    IEC104_COI_UI_RES38 = 38,
    IEC104_COI_UI_RES39 = 39,
    IEC104_COI_UI_RES40 = 40,
    IEC104_COI_UI_RES41 = 41,
    IEC104_COI_UI_RES42 = 42,
    IEC104_COI_UI_RES43 = 43,
    IEC104_COI_UI_RES44 = 44,
    IEC104_COI_UI_RES45 = 45,
    IEC104_COI_UI_RES46 = 46,
    IEC104_COI_UI_RES47 = 47,
    IEC104_COI_UI_RES48 = 48,
    IEC104_COI_UI_RES49 = 49,
    IEC104_COI_UI_RES50 = 50,
    IEC104_COI_UI_RES51 = 51,
    IEC104_COI_UI_RES52 = 52,
    IEC104_COI_UI_RES53 = 53,
    IEC104_COI_UI_RES54 = 54,
    IEC104_COI_UI_RES55 = 55,
    IEC104_COI_UI_RES56 = 56,
    IEC104_COI_UI_RES57 = 57,
    IEC104_COI_UI_RES58 = 58,
    IEC104_COI_UI_RES59 = 59,
    IEC104_COI_UI_RES60 = 60,
    IEC104_COI_UI_RES61 = 61,
    IEC104_COI_UI_RES62 = 62,
    IEC104_COI_UI_RES63 = 63,
    IEC104_COI_UI_RES64 = 64,
    IEC104_COI_UI_RES65 = 65,
    IEC104_COI_UI_RES66 = 66,
    IEC104_COI_UI_RES67 = 67,
    IEC104_COI_UI_RES68 = 68,
    IEC104_COI_UI_RES69 = 69,
    IEC104_COI_UI_RES70 = 70,
    IEC104_COI_UI_RES71 = 71,
    IEC104_COI_UI_RES72 = 72,
    IEC104_COI_UI_RES73 = 73,
    IEC104_COI_UI_RES74 = 74,
    IEC104_COI_UI_RES75 = 75,
    IEC104_COI_UI_RES76 = 76,
    IEC104_COI_UI_RES77 = 77,
    IEC104_COI_UI_RES78 = 78,
    IEC104_COI_UI_RES79 = 79,
    IEC104_COI_UI_RES80 = 80,
    IEC104_COI_UI_RES81 = 81,
    IEC104_COI_UI_RES82 = 82,
    IEC104_COI_UI_RES83 = 83,
    IEC104_COI_UI_RES84 = 84,
    IEC104_COI_UI_RES85 = 85,
    IEC104_COI_UI_RES86 = 86,
    IEC104_COI_UI_RES87 = 87,
    IEC104_COI_UI_RES88 = 88,
    IEC104_COI_UI_RES89 = 89,
    IEC104_COI_UI_RES90 = 90,
    IEC104_COI_UI_RES91 = 91,
    IEC104_COI_UI_RES92 = 92,
    IEC104_COI_UI_RES93 = 93,
    IEC104_COI_UI_RES94 = 94,
    IEC104_COI_UI_RES95 = 95,
    IEC104_COI_UI_RES96 = 96,
    IEC104_COI_UI_RES97 = 97,
    IEC104_COI_UI_RES98 = 98,
    IEC104_COI_UI_RES99 = 99,
    IEC104_COI_UI_RES100 = 100,
    IEC104_COI_UI_RES101 = 101,
    IEC104_COI_UI_RES102 = 102,
    IEC104_COI_UI_RES103 = 103,
    IEC104_COI_UI_RES104 = 104,
    IEC104_COI_UI_RES105 = 105,
    IEC104_COI_UI_RES106 = 106,
    IEC104_COI_UI_RES107 = 107,
    IEC104_COI_UI_RES108 = 108,
    IEC104_COI_UI_RES109 = 109,
    IEC104_COI_UI_RES110 = 110,
    IEC104_COI_UI_RES111 = 111,
    IEC104_COI_UI_RES112 = 112,
    IEC104_COI_UI_RES113 = 113,
    IEC104_COI_UI_RES114 = 114,
    IEC104_COI_UI_RES115 = 115,
    IEC104_COI_UI_RES116 = 116,
    IEC104_COI_UI_RES117 = 117,
    IEC104_COI_UI_RES118 = 118,
    IEC104_COI_UI_RES119 = 119,
    IEC104_COI_UI_RES120 = 120,
    IEC104_COI_UI_RES121 = 121,
    IEC104_COI_UI_RES122 = 122,
    IEC104_COI_UI_RES123 = 123,
    IEC104_COI_UI_RES124 = 124,
    IEC104_COI_UI_RES125 = 125,
    IEC104_COI_UI_RES126 = 126,
    IEC104_COI_UI_RES127 = 127,
};

enum Iec104QoiEnum
{
    IEC104_QOI_NOTUSED = 0,
    IEC104_QOI_RES1 = 1,
    IEC104_QOI_RES2 = 2,
    IEC104_QOI_RES3 = 3,
    IEC104_QOI_RES4 = 4,
    IEC104_QOI_RES5 = 5,
    IEC104_QOI_RES6 = 6,
    IEC104_QOI_RES7 = 7,
    IEC104_QOI_RES8 = 8,
    IEC104_QOI_RES9 = 9,
    IEC104_QOI_RES10 = 10,
    IEC104_QOI_RES11 = 11,
    IEC104_QOI_RES12 = 12,
    IEC104_QOI_RES13 = 13,
    IEC104_QOI_RES14 = 14,
    IEC104_QOI_RES15 = 15,
    IEC104_QOI_RES16 = 16,
    IEC104_QOI_RES17 = 17,
    IEC104_QOI_RES18 = 18,
    IEC104_QOI_RES19 = 19,
    IEC104_QOI_INROSTAT = 20,
    IEC104_QOI_INRO1 = 21,
    IEC104_QOI_INRO2 = 22,
    IEC104_QOI_INRO3 = 23,
    IEC104_QOI_INRO4 = 24,
    IEC104_QOI_INRO5 = 25,
    IEC104_QOI_INRO6 = 26,
    IEC104_QOI_INRO7 = 27,
    IEC104_QOI_INRO8 = 28,
    IEC104_QOI_INRO9 = 29,
    IEC104_QOI_INRO10 = 30,
    IEC104_QOI_INRO11 = 31,
    IEC104_QOI_INRO12 = 32,
    IEC104_QOI_INRO13 = 33,
    IEC104_QOI_INRO14 = 34,
    IEC104_QOI_INRO15 = 35,
    IEC104_QOI_INRO16 = 36,
    IEC104_QOI_RES37 = 37,
    IEC104_QOI_RES38 = 38,
    IEC104_QOI_RES39 = 39,
    IEC104_QOI_RES40 = 40,
    IEC104_QOI_RES41 = 41,
    IEC104_QOI_RES42 = 42,
    IEC104_QOI_RES43 = 43,
    IEC104_QOI_RES44 = 44,
    IEC104_QOI_RES45 = 45,
    IEC104_QOI_RES46 = 46,
    IEC104_QOI_RES47 = 47,
    IEC104_QOI_RES48 = 48,
    IEC104_QOI_RES49 = 49,
    IEC104_QOI_RES50 = 50,
    IEC104_QOI_RES51 = 51,
    IEC104_QOI_RES52 = 52,
    IEC104_QOI_RES53 = 53,
    IEC104_QOI_RES54 = 54,
    IEC104_QOI_RES55 = 55,
    IEC104_QOI_RES56 = 56,
    IEC104_QOI_RES57 = 57,
    IEC104_QOI_RES58 = 58,
    IEC104_QOI_RES59 = 59,
    IEC104_QOI_RES60 = 60,
    IEC104_QOI_RES61 = 61,
    IEC104_QOI_RES62 = 62,
    IEC104_QOI_RES63 = 63,
    IEC104_QOI_RES64 = 64,
    IEC104_QOI_RES65 = 65,
    IEC104_QOI_RES66 = 66,
    IEC104_QOI_RES67 = 67,
    IEC104_QOI_RES68 = 68,
    IEC104_QOI_RES69 = 69,
    IEC104_QOI_RES70 = 70,
    IEC104_QOI_RES71 = 71,
    IEC104_QOI_RES72 = 72,
    IEC104_QOI_RES73 = 73,
    IEC104_QOI_RES74 = 74,
    IEC104_QOI_RES75 = 75,
    IEC104_QOI_RES76 = 76,
    IEC104_QOI_RES77 = 77,
    IEC104_QOI_RES78 = 78,
    IEC104_QOI_RES79 = 79,
    IEC104_QOI_RES80 = 80,
    IEC104_QOI_RES81 = 81,
    IEC104_QOI_RES82 = 82,
    IEC104_QOI_RES83 = 83,
    IEC104_QOI_RES84 = 84,
    IEC104_QOI_RES85 = 85,
    IEC104_QOI_RES86 = 86,
    IEC104_QOI_RES87 = 87,
    IEC104_QOI_RES88 = 88,
    IEC104_QOI_RES89 = 89,
    IEC104_QOI_RES90 = 90,
    IEC104_QOI_RES91 = 91,
    IEC104_QOI_RES92 = 92,
    IEC104_QOI_RES93 = 93,
    IEC104_QOI_RES94 = 94,
    IEC104_QOI_RES95 = 95,
    IEC104_QOI_RES96 = 96,
    IEC104_QOI_RES97 = 97,
    IEC104_QOI_RES98 = 98,
    IEC104_QOI_RES99 = 99,
    IEC104_QOI_RES100 = 100,
    IEC104_QOI_RES101 = 101,
    IEC104_QOI_RES102 = 102,
    IEC104_QOI_RES103 = 103,
    IEC104_QOI_RES104 = 104,
    IEC104_QOI_RES105 = 105,
    IEC104_QOI_RES106 = 106,
    IEC104_QOI_RES107 = 107,
    IEC104_QOI_RES108 = 108,
    IEC104_QOI_RES109 = 109,
    IEC104_QOI_RES110 = 110,
    IEC104_QOI_RES111 = 111,
    IEC104_QOI_RES112 = 112,
    IEC104_QOI_RES113 = 113,
    IEC104_QOI_RES114 = 114,
    IEC104_QOI_RES115 = 115,
    IEC104_QOI_RES116 = 116,
    IEC104_QOI_RES117 = 117,
    IEC104_QOI_RES118 = 118,
    IEC104_QOI_RES119 = 119,
    IEC104_QOI_RES120 = 120,
    IEC104_QOI_RES121 = 121,
    IEC104_QOI_RES122 = 122,
    IEC104_QOI_RES123 = 123,
    IEC104_QOI_RES124 = 124,
    IEC104_QOI_RES125 = 125,
    IEC104_QOI_RES126 = 126,
    IEC104_QOI_RES127 = 127,
    IEC104_QOI_RES128 = 128,
    IEC104_QOI_RES129 = 129,
    IEC104_QOI_RES130 = 130,
    IEC104_QOI_RES131 = 131,
    IEC104_QOI_RES132 = 132,
    IEC104_QOI_RES133 = 133,
    IEC104_QOI_RES134 = 134,
    IEC104_QOI_RES135 = 135,
    IEC104_QOI_RES136 = 136,
    IEC104_QOI_RES137 = 137,
    IEC104_QOI_RES138 = 138,
    IEC104_QOI_RES139 = 139,
    IEC104_QOI_RES140 = 140,
    IEC104_QOI_RES141 = 141,
    IEC104_QOI_RES142 = 142,
    IEC104_QOI_RES143 = 143,
    IEC104_QOI_RES144 = 144,
    IEC104_QOI_RES145 = 145,
    IEC104_QOI_RES146 = 146,
    IEC104_QOI_RES147 = 147,
    IEC104_QOI_RES148 = 148,
    IEC104_QOI_RES149 = 149,
    IEC104_QOI_RES150 = 150,
    IEC104_QOI_RES151 = 151,
    IEC104_QOI_RES152 = 152,
    IEC104_QOI_RES153 = 153,
    IEC104_QOI_RES154 = 154,
    IEC104_QOI_RES155 = 155,
    IEC104_QOI_RES156 = 156,
    IEC104_QOI_RES157 = 157,
    IEC104_QOI_RES158 = 158,
    IEC104_QOI_RES159 = 159,
    IEC104_QOI_RES160 = 160,
    IEC104_QOI_RES161 = 161,
    IEC104_QOI_RES162 = 162,
    IEC104_QOI_RES163 = 163,
    IEC104_QOI_RES164 = 164,
    IEC104_QOI_RES165 = 165,
    IEC104_QOI_RES166 = 166,
    IEC104_QOI_RES167 = 167,
    IEC104_QOI_RES168 = 168,
    IEC104_QOI_RES169 = 169,
    IEC104_QOI_RES170 = 170,
    IEC104_QOI_RES171 = 171,
    IEC104_QOI_RES172 = 172,
    IEC104_QOI_RES173 = 173,
    IEC104_QOI_RES174 = 174,
    IEC104_QOI_RES175 = 175,
    IEC104_QOI_RES176 = 176,
    IEC104_QOI_RES177 = 177,
    IEC104_QOI_RES178 = 178,
    IEC104_QOI_RES179 = 179,
    IEC104_QOI_RES180 = 180,
    IEC104_QOI_RES181 = 181,
    IEC104_QOI_RES182 = 182,
    IEC104_QOI_RES183 = 183,
    IEC104_QOI_RES184 = 184,
    IEC104_QOI_RES185 = 185,
    IEC104_QOI_RES186 = 186,
    IEC104_QOI_RES187 = 187,
    IEC104_QOI_RES188 = 188,
    IEC104_QOI_RES189 = 189,
    IEC104_QOI_RES190 = 190,
    IEC104_QOI_RES191 = 191,
    IEC104_QOI_RES192 = 192,
    IEC104_QOI_RES193 = 193,
    IEC104_QOI_RES194 = 194,
    IEC104_QOI_RES195 = 195,
    IEC104_QOI_RES196 = 196,
    IEC104_QOI_RES197 = 197,
    IEC104_QOI_RES198 = 198,
    IEC104_QOI_RES199 = 199,
    IEC104_QOI_RES200 = 200,
    IEC104_QOI_RES201 = 201,
    IEC104_QOI_RES202 = 202,
    IEC104_QOI_RES203 = 203,
    IEC104_QOI_RES204 = 204,
    IEC104_QOI_RES205 = 205,
    IEC104_QOI_RES206 = 206,
    IEC104_QOI_RES207 = 207,
    IEC104_QOI_RES208 = 208,
    IEC104_QOI_RES209 = 209,
    IEC104_QOI_RES210 = 210,
    IEC104_QOI_RES211 = 211,
    IEC104_QOI_RES212 = 212,
    IEC104_QOI_RES213 = 213,
    IEC104_QOI_RES214 = 214,
    IEC104_QOI_RES215 = 215,
    IEC104_QOI_RES216 = 216,
    IEC104_QOI_RES217 = 217,
    IEC104_QOI_RES218 = 218,
    IEC104_QOI_RES219 = 219,
    IEC104_QOI_RES220 = 220,
    IEC104_QOI_RES221 = 221,
    IEC104_QOI_RES222 = 222,
    IEC104_QOI_RES223 = 223,
    IEC104_QOI_RES224 = 224,
    IEC104_QOI_RES225 = 225,
    IEC104_QOI_RES226 = 226,
    IEC104_QOI_RES227 = 227,
    IEC104_QOI_RES228 = 228,
    IEC104_QOI_RES229 = 229,
    IEC104_QOI_RES230 = 230,
    IEC104_QOI_RES231 = 231,
    IEC104_QOI_RES232 = 232,
    IEC104_QOI_RES233 = 233,
    IEC104_QOI_RES234 = 234,
    IEC104_QOI_RES235 = 235,
    IEC104_QOI_RES236 = 236,
    IEC104_QOI_RES237 = 237,
    IEC104_QOI_RES238 = 238,
    IEC104_QOI_RES239 = 239,
    IEC104_QOI_RES240 = 240,
    IEC104_QOI_RES241 = 241,
    IEC104_QOI_RES242 = 242,
    IEC104_QOI_RES243 = 243,
    IEC104_QOI_RES244 = 244,
    IEC104_QOI_RES245 = 245,
    IEC104_QOI_RES246 = 246,
    IEC104_QOI_RES247 = 247,
    IEC104_QOI_RES248 = 248,
    IEC104_QOI_RES249 = 249,
    IEC104_QOI_RES250 = 250,
    IEC104_QOI_RES251 = 251,
    IEC104_QOI_RES252 = 252,
    IEC104_QOI_RES253 = 253,
    IEC104_QOI_RES254 = 254,
    IEC104_QOI_RES255 = 255,
};

enum Iec104QccRqtEnum
{
    IEC104_QCC_RQT_NOTUSED = 0,
    IEC104_QCC_RQT_GROUP1 = 1,
    IEC104_QCC_RQT_GROUP2 = 2,
    IEC104_QCC_RQT_GROUP3 = 3,
    IEC104_QCC_RQT_GROUP4 = 4,
    IEC104_QCC_RQT_GENCTR = 5,
    IEC104_QCC_RQT_RES6 = 6,
    IEC104_QCC_RQT_RES7 = 7,
    IEC104_QCC_RQT_RES8 = 8,
    IEC104_QCC_RQT_RES9 = 9,
    IEC104_QCC_RQT_RES10 = 10,
    IEC104_QCC_RQT_RES11 = 11,
    IEC104_QCC_RQT_RES12 = 12,
    IEC104_QCC_RQT_RES13 = 13,
    IEC104_QCC_RQT_RES14 = 14,
    IEC104_QCC_RQT_RES15 = 15,
    IEC104_QCC_RQT_RES16 = 16,
    IEC104_QCC_RQT_RES17 = 17,
    IEC104_QCC_RQT_RES18 = 18,
    IEC104_QCC_RQT_RES19 = 19,
    IEC104_QCC_RQT_RES20 = 20,
    IEC104_QCC_RQT_RES21 = 21,
    IEC104_QCC_RQT_RES22 = 22,
    IEC104_QCC_RQT_RES23 = 23,
    IEC104_QCC_RQT_RES24 = 24,
    IEC104_QCC_RQT_RES25 = 25,
    IEC104_QCC_RQT_RES26 = 26,
    IEC104_QCC_RQT_RES27 = 27,
    IEC104_QCC_RQT_RES28 = 28,
    IEC104_QCC_RQT_RES29 = 29,
    IEC104_QCC_RQT_RES30 = 30,
    IEC104_QCC_RQT_RES31 = 31,
    IEC104_QCC_RQT_RES32 = 32,
    IEC104_QCC_RQT_RES33 = 33,
    IEC104_QCC_RQT_RES34 = 34,
    IEC104_QCC_RQT_RES35 = 35,
    IEC104_QCC_RQT_RES36 = 36,
    IEC104_QCC_RQT_RES37 = 37,
    IEC104_QCC_RQT_RES38 = 38,
    IEC104_QCC_RQT_RES39 = 39,
    IEC104_QCC_RQT_RES40 = 40,
    IEC104_QCC_RQT_RES41 = 41,
    IEC104_QCC_RQT_RES42 = 42,
    IEC104_QCC_RQT_RES43 = 43,
    IEC104_QCC_RQT_RES44 = 44,
    IEC104_QCC_RQT_RES45 = 45,
    IEC104_QCC_RQT_RES46 = 46,
    IEC104_QCC_RQT_RES47 = 47,
    IEC104_QCC_RQT_RES48 = 48,
    IEC104_QCC_RQT_RES49 = 49,
    IEC104_QCC_RQT_RES50 = 50,
    IEC104_QCC_RQT_RES51 = 51,
    IEC104_QCC_RQT_RES52 = 52,
    IEC104_QCC_RQT_RES53 = 53,
    IEC104_QCC_RQT_RES54 = 54,
    IEC104_QCC_RQT_RES55 = 55,
    IEC104_QCC_RQT_RES56 = 56,
    IEC104_QCC_RQT_RES57 = 57,
    IEC104_QCC_RQT_RES58 = 58,
    IEC104_QCC_RQT_RES59 = 59,
    IEC104_QCC_RQT_RES60 = 60,
    IEC104_QCC_RQT_RES61 = 61,
    IEC104_QCC_RQT_RES62 = 62,
    IEC104_QCC_RQT_RES63 = 63,
};

enum Iec104QccFrzEnum
{
    IEC104_QCC_FRZ_READ = 0,
    IEC104_QCC_FRZ_CTRFRZWITHOUTRST = 1,
    IEC104_QCC_FRZ_CTRFRZWITHRST = 2,
    IEC104_QCC_FRZ_CTRRST = 3,
};

enum Iec104QpmKpaEnum
{
    IEC104_QPM_KPA_NOTUSED = 0,
    IEC104_QPM_KPA_THRESHVAL = 1,
    IEC104_QPM_KPA_SMOOTHFACTOR = 2,
    IEC104_QPM_KPA_LOWTXLMT = 3,
    IEC104_QPM_KPA_HIGHTXLMT = 4,
    IEC104_QPM_KPA_RES5 = 5,
    IEC104_QPM_KPA_RES6 = 6,
    IEC104_QPM_KPA_RES7 = 7,
    IEC104_QPM_KPA_RES8 = 8,
    IEC104_QPM_KPA_RES9 = 9,
    IEC104_QPM_KPA_RES10 = 10,
    IEC104_QPM_KPA_RES11 = 11,
    IEC104_QPM_KPA_RES12 = 12,
    IEC104_QPM_KPA_RES13 = 13,
    IEC104_QPM_KPA_RES14 = 14,
    IEC104_QPM_KPA_RES15 = 15,
    IEC104_QPM_KPA_RES16 = 16,
    IEC104_QPM_KPA_RES17 = 17,
    IEC104_QPM_KPA_RES18 = 18,
    IEC104_QPM_KPA_RES19 = 19,
    IEC104_QPM_KPA_RES20 = 20,
    IEC104_QPM_KPA_RES21 = 21,
    IEC104_QPM_KPA_RES22 = 22,
    IEC104_QPM_KPA_RES23 = 23,
    IEC104_QPM_KPA_RES24 = 24,
    IEC104_QPM_KPA_RES25 = 25,
    IEC104_QPM_KPA_RES26 = 26,
    IEC104_QPM_KPA_RES27 = 27,
    IEC104_QPM_KPA_RES28 = 28,
    IEC104_QPM_KPA_RES29 = 29,
    IEC104_QPM_KPA_RES30 = 30,
    IEC104_QPM_KPA_RES31 = 31,
    IEC104_QPM_KPA_RES32 = 32,
    IEC104_QPM_KPA_RES33 = 33,
    IEC104_QPM_KPA_RES34 = 34,
    IEC104_QPM_KPA_RES35 = 35,
    IEC104_QPM_KPA_RES36 = 36,
    IEC104_QPM_KPA_RES37 = 37,
    IEC104_QPM_KPA_RES38 = 38,
    IEC104_QPM_KPA_RES39 = 39,
    IEC104_QPM_KPA_RES40 = 40,
    IEC104_QPM_KPA_RES41 = 41,
    IEC104_QPM_KPA_RES42 = 42,
    IEC104_QPM_KPA_RES43 = 43,
    IEC104_QPM_KPA_RES44 = 44,
    IEC104_QPM_KPA_RES45 = 45,
    IEC104_QPM_KPA_RES46 = 46,
    IEC104_QPM_KPA_RES47 = 47,
    IEC104_QPM_KPA_RES48 = 48,
    IEC104_QPM_KPA_RES49 = 49,
    IEC104_QPM_KPA_RES50 = 50,
    IEC104_QPM_KPA_RES51 = 51,
    IEC104_QPM_KPA_RES52 = 52,
    IEC104_QPM_KPA_RES53 = 53,
    IEC104_QPM_KPA_RES54 = 54,
    IEC104_QPM_KPA_RES55 = 55,
    IEC104_QPM_KPA_RES56 = 56,
    IEC104_QPM_KPA_RES57 = 57,
    IEC104_QPM_KPA_RES58 = 58,
    IEC104_QPM_KPA_RES59 = 59,
    IEC104_QPM_KPA_RES60 = 60,
    IEC104_QPM_KPA_RES61 = 61,
    IEC104_QPM_KPA_RES62 = 62,
    IEC104_QPM_KPA_RES63 = 63,
};

enum Iec104QpaEnum
{
    IEC104_QPA_NOTUSED = 0,
    IEC104_QPA_ACTDEACTPREVPARAM = 1,
    IEC104_QPA_ACTDEACTPARAM = 2,
    IEC104_QPA_ACTDEACTCYCTX = 3,
    IEC104_QPA_RES4 = 4,
    IEC104_QPA_RES5 = 5,
    IEC104_QPA_RES6 = 6,
    IEC104_QPA_RES7 = 7,
    IEC104_QPA_RES8 = 8,
    IEC104_QPA_RES9 = 9,
    IEC104_QPA_RES10 = 10,
    IEC104_QPA_RES11 = 11,
    IEC104_QPA_RES12 = 12,
    IEC104_QPA_RES13 = 13,
    IEC104_QPA_RES14 = 14,
    IEC104_QPA_RES15 = 15,
    IEC104_QPA_RES16 = 16,
    IEC104_QPA_RES17 = 17,
    IEC104_QPA_RES18 = 18,
    IEC104_QPA_RES19 = 19,
    IEC104_QPA_RES20 = 20,
    IEC104_QPA_RES21 = 21,
    IEC104_QPA_RES22 = 22,
    IEC104_QPA_RES23 = 23,
    IEC104_QPA_RES24 = 24,
    IEC104_QPA_RES25 = 25,
    IEC104_QPA_RES26 = 26,
    IEC104_QPA_RES27 = 27,
    IEC104_QPA_RES28 = 28,
    IEC104_QPA_RES29 = 29,
    IEC104_QPA_RES30 = 30,
    IEC104_QPA_RES31 = 31,
    IEC104_QPA_RES32 = 32,
    IEC104_QPA_RES33 = 33,
    IEC104_QPA_RES34 = 34,
    IEC104_QPA_RES35 = 35,
    IEC104_QPA_RES36 = 36,
    IEC104_QPA_RES37 = 37,
    IEC104_QPA_RES38 = 38,
    IEC104_QPA_RES39 = 39,
    IEC104_QPA_RES40 = 40,
    IEC104_QPA_RES41 = 41,
    IEC104_QPA_RES42 = 42,
    IEC104_QPA_RES43 = 43,
    IEC104_QPA_RES44 = 44,
    IEC104_QPA_RES45 = 45,
    IEC104_QPA_RES46 = 46,
    IEC104_QPA_RES47 = 47,
    IEC104_QPA_RES48 = 48,
    IEC104_QPA_RES49 = 49,
    IEC104_QPA_RES50 = 50,
    IEC104_QPA_RES51 = 51,
    IEC104_QPA_RES52 = 52,
    IEC104_QPA_RES53 = 53,
    IEC104_QPA_RES54 = 54,
    IEC104_QPA_RES55 = 55,
    IEC104_QPA_RES56 = 56,
    IEC104_QPA_RES57 = 57,
    IEC104_QPA_RES58 = 58,
    IEC104_QPA_RES59 = 59,
    IEC104_QPA_RES60 = 60,
    IEC104_QPA_RES61 = 61,
    IEC104_QPA_RES62 = 62,
    IEC104_QPA_RES63 = 63,
    IEC104_QPA_RES64 = 64,
    IEC104_QPA_RES65 = 65,
    IEC104_QPA_RES66 = 66,
    IEC104_QPA_RES67 = 67,
    IEC104_QPA_RES68 = 68,
    IEC104_QPA_RES69 = 69,
    IEC104_QPA_RES70 = 70,
    IEC104_QPA_RES71 = 71,
    IEC104_QPA_RES72 = 72,
    IEC104_QPA_RES73 = 73,
    IEC104_QPA_RES74 = 74,
    IEC104_QPA_RES75 = 75,
    IEC104_QPA_RES76 = 76,
    IEC104_QPA_RES77 = 77,
    IEC104_QPA_RES78 = 78,
    IEC104_QPA_RES79 = 79,
    IEC104_QPA_RES80 = 80,
    IEC104_QPA_RES81 = 81,
    IEC104_QPA_RES82 = 82,
    IEC104_QPA_RES83 = 83,
    IEC104_QPA_RES84 = 84,
    IEC104_QPA_RES85 = 85,
    IEC104_QPA_RES86 = 86,
    IEC104_QPA_RES87 = 87,
    IEC104_QPA_RES88 = 88,
    IEC104_QPA_RES89 = 89,
    IEC104_QPA_RES90 = 90,
    IEC104_QPA_RES91 = 91,
    IEC104_QPA_RES92 = 92,
    IEC104_QPA_RES93 = 93,
    IEC104_QPA_RES94 = 94,
    IEC104_QPA_RES95 = 95,
    IEC104_QPA_RES96 = 96,
    IEC104_QPA_RES97 = 97,
    IEC104_QPA_RES98 = 98,
    IEC104_QPA_RES99 = 99,
    IEC104_QPA_RES100 = 100,
    IEC104_QPA_RES101 = 101,
    IEC104_QPA_RES102 = 102,
    IEC104_QPA_RES103 = 103,
    IEC104_QPA_RES104 = 104,
    IEC104_QPA_RES105 = 105,
    IEC104_QPA_RES106 = 106,
    IEC104_QPA_RES107 = 107,
    IEC104_QPA_RES108 = 108,
    IEC104_QPA_RES109 = 109,
    IEC104_QPA_RES110 = 110,
    IEC104_QPA_RES111 = 111,
    IEC104_QPA_RES112 = 112,
    IEC104_QPA_RES113 = 113,
    IEC104_QPA_RES114 = 114,
    IEC104_QPA_RES115 = 115,
    IEC104_QPA_RES116 = 116,
    IEC104_QPA_RES117 = 117,
    IEC104_QPA_RES118 = 118,
    IEC104_QPA_RES119 = 119,
    IEC104_QPA_RES120 = 120,
    IEC104_QPA_RES121 = 121,
    IEC104_QPA_RES122 = 122,
    IEC104_QPA_RES123 = 123,
    IEC104_QPA_RES124 = 124,
    IEC104_QPA_RES125 = 125,
    IEC104_QPA_RES126 = 126,
    IEC104_QPA_RES127 = 127,
    IEC104_QPA_RES128 = 128,
    IEC104_QPA_RES129 = 129,
    IEC104_QPA_RES130 = 130,
    IEC104_QPA_RES131 = 131,
    IEC104_QPA_RES132 = 132,
    IEC104_QPA_RES133 = 133,
    IEC104_QPA_RES134 = 134,
    IEC104_QPA_RES135 = 135,
    IEC104_QPA_RES136 = 136,
    IEC104_QPA_RES137 = 137,
    IEC104_QPA_RES138 = 138,
    IEC104_QPA_RES139 = 139,
    IEC104_QPA_RES140 = 140,
    IEC104_QPA_RES141 = 141,
    IEC104_QPA_RES142 = 142,
    IEC104_QPA_RES143 = 143,
    IEC104_QPA_RES144 = 144,
    IEC104_QPA_RES145 = 145,
    IEC104_QPA_RES146 = 146,
    IEC104_QPA_RES147 = 147,
    IEC104_QPA_RES148 = 148,
    IEC104_QPA_RES149 = 149,
    IEC104_QPA_RES150 = 150,
    IEC104_QPA_RES151 = 151,
    IEC104_QPA_RES152 = 152,
    IEC104_QPA_RES153 = 153,
    IEC104_QPA_RES154 = 154,
    IEC104_QPA_RES155 = 155,
    IEC104_QPA_RES156 = 156,
    IEC104_QPA_RES157 = 157,
    IEC104_QPA_RES158 = 158,
    IEC104_QPA_RES159 = 159,
    IEC104_QPA_RES160 = 160,
    IEC104_QPA_RES161 = 161,
    IEC104_QPA_RES162 = 162,
    IEC104_QPA_RES163 = 163,
    IEC104_QPA_RES164 = 164,
    IEC104_QPA_RES165 = 165,
    IEC104_QPA_RES166 = 166,
    IEC104_QPA_RES167 = 167,
    IEC104_QPA_RES168 = 168,
    IEC104_QPA_RES169 = 169,
    IEC104_QPA_RES170 = 170,
    IEC104_QPA_RES171 = 171,
    IEC104_QPA_RES172 = 172,
    IEC104_QPA_RES173 = 173,
    IEC104_QPA_RES174 = 174,
    IEC104_QPA_RES175 = 175,
    IEC104_QPA_RES176 = 176,
    IEC104_QPA_RES177 = 177,
    IEC104_QPA_RES178 = 178,
    IEC104_QPA_RES179 = 179,
    IEC104_QPA_RES180 = 180,
    IEC104_QPA_RES181 = 181,
    IEC104_QPA_RES182 = 182,
    IEC104_QPA_RES183 = 183,
    IEC104_QPA_RES184 = 184,
    IEC104_QPA_RES185 = 185,
    IEC104_QPA_RES186 = 186,
    IEC104_QPA_RES187 = 187,
    IEC104_QPA_RES188 = 188,
    IEC104_QPA_RES189 = 189,
    IEC104_QPA_RES190 = 190,
    IEC104_QPA_RES191 = 191,
    IEC104_QPA_RES192 = 192,
    IEC104_QPA_RES193 = 193,
    IEC104_QPA_RES194 = 194,
    IEC104_QPA_RES195 = 195,
    IEC104_QPA_RES196 = 196,
    IEC104_QPA_RES197 = 197,
    IEC104_QPA_RES198 = 198,
    IEC104_QPA_RES199 = 199,
    IEC104_QPA_RES200 = 200,
    IEC104_QPA_RES201 = 201,
    IEC104_QPA_RES202 = 202,
    IEC104_QPA_RES203 = 203,
    IEC104_QPA_RES204 = 204,
    IEC104_QPA_RES205 = 205,
    IEC104_QPA_RES206 = 206,
    IEC104_QPA_RES207 = 207,
    IEC104_QPA_RES208 = 208,
    IEC104_QPA_RES209 = 209,
    IEC104_QPA_RES210 = 210,
    IEC104_QPA_RES211 = 211,
    IEC104_QPA_RES212 = 212,
    IEC104_QPA_RES213 = 213,
    IEC104_QPA_RES214 = 214,
    IEC104_QPA_RES215 = 215,
    IEC104_QPA_RES216 = 216,
    IEC104_QPA_RES217 = 217,
    IEC104_QPA_RES218 = 218,
    IEC104_QPA_RES219 = 219,
    IEC104_QPA_RES220 = 220,
    IEC104_QPA_RES221 = 221,
    IEC104_QPA_RES222 = 222,
    IEC104_QPA_RES223 = 223,
    IEC104_QPA_RES224 = 224,
    IEC104_QPA_RES225 = 225,
    IEC104_QPA_RES226 = 226,
    IEC104_QPA_RES227 = 227,
    IEC104_QPA_RES228 = 228,
    IEC104_QPA_RES229 = 229,
    IEC104_QPA_RES230 = 230,
    IEC104_QPA_RES231 = 231,
    IEC104_QPA_RES232 = 232,
    IEC104_QPA_RES233 = 233,
    IEC104_QPA_RES234 = 234,
    IEC104_QPA_RES235 = 235,
    IEC104_QPA_RES236 = 236,
    IEC104_QPA_RES237 = 237,
    IEC104_QPA_RES238 = 238,
    IEC104_QPA_RES239 = 239,
    IEC104_QPA_RES240 = 240,
    IEC104_QPA_RES241 = 241,
    IEC104_QPA_RES242 = 242,
    IEC104_QPA_RES243 = 243,
    IEC104_QPA_RES244 = 244,
    IEC104_QPA_RES245 = 245,
    IEC104_QPA_RES246 = 246,
    IEC104_QPA_RES247 = 247,
    IEC104_QPA_RES248 = 248,
    IEC104_QPA_RES249 = 249,
    IEC104_QPA_RES250 = 250,
    IEC104_QPA_RES251 = 251,
    IEC104_QPA_RES252 = 252,
    IEC104_QPA_RES253 = 253,
    IEC104_QPA_RES254 = 254,
    IEC104_QPA_RES255 = 255,
};

enum Iec104QocQuEnum
{
    IEC104_QOC_QU_NOADDDEF = 0,
    IEC104_QOC_QU_SHORTPULSE = 1,
    IEC104_QOC_QU_LONGPULSE = 2,
    IEC104_QOC_QU_PERSIST = 3,
    IEC104_QOC_QU_RES4 = 4,
    IEC104_QOC_QU_RES5 = 5,
    IEC104_QOC_QU_RES6 = 6,
    IEC104_QOC_QU_RES7 = 7,
    IEC104_QOC_QU_RES8 = 8,
    IEC104_QOC_QU_RES9 = 9,
    IEC104_QOC_QU_RES10 = 10,
    IEC104_QOC_QU_RES11 = 11,
    IEC104_QOC_QU_RES12 = 12,
    IEC104_QOC_QU_RES13 = 13,
    IEC104_QOC_QU_RES14 = 14,
    IEC104_QOC_QU_RES15 = 15,
    IEC104_QOC_QU_RES16 = 16,
    IEC104_QOC_QU_RES17 = 17,
    IEC104_QOC_QU_RES18 = 18,
    IEC104_QOC_QU_RES19 = 19,
    IEC104_QOC_QU_RES20 = 20,
    IEC104_QOC_QU_RES21 = 21,
    IEC104_QOC_QU_RES22 = 22,
    IEC104_QOC_QU_RES23 = 23,
    IEC104_QOC_QU_RES24 = 24,
    IEC104_QOC_QU_RES25 = 25,
    IEC104_QOC_QU_RES26 = 26,
    IEC104_QOC_QU_RES27 = 27,
    IEC104_QOC_QU_RES28 = 28,
    IEC104_QOC_QU_RES29 = 29,
    IEC104_QOC_QU_RES30 = 30,
    IEC104_QOC_QU_RES31 = 31,
};

enum Iec104QrpEnum
{
    IEC104_QRP_NOTUSED = 0,
    IEC104_QRP_GENRST = 1,
    IEC104_QRP_RSTTIME = 2,
    IEC104_QRP_RES3 = 3,
    IEC104_QRP_RES4 = 4,
    IEC104_QRP_RES5 = 5,
    IEC104_QRP_RES6 = 6,
    IEC104_QRP_RES7 = 7,
    IEC104_QRP_RES8 = 8,
    IEC104_QRP_RES9 = 9,
    IEC104_QRP_RES10 = 10,
    IEC104_QRP_RES11 = 11,
    IEC104_QRP_RES12 = 12,
    IEC104_QRP_RES13 = 13,
    IEC104_QRP_RES14 = 14,
    IEC104_QRP_RES15 = 15,
    IEC104_QRP_RES16 = 16,
    IEC104_QRP_RES17 = 17,
    IEC104_QRP_RES18 = 18,
    IEC104_QRP_RES19 = 19,
    IEC104_QRP_RES20 = 20,
    IEC104_QRP_RES21 = 21,
    IEC104_QRP_RES22 = 22,
    IEC104_QRP_RES23 = 23,
    IEC104_QRP_RES24 = 24,
    IEC104_QRP_RES25 = 25,
    IEC104_QRP_RES26 = 26,
    IEC104_QRP_RES27 = 27,
    IEC104_QRP_RES28 = 28,
    IEC104_QRP_RES29 = 29,
    IEC104_QRP_RES30 = 30,
    IEC104_QRP_RES31 = 31,
    IEC104_QRP_RES32 = 32,
    IEC104_QRP_RES33 = 33,
    IEC104_QRP_RES34 = 34,
    IEC104_QRP_RES35 = 35,
    IEC104_QRP_RES36 = 36,
    IEC104_QRP_RES37 = 37,
    IEC104_QRP_RES38 = 38,
    IEC104_QRP_RES39 = 39,
    IEC104_QRP_RES40 = 40,
    IEC104_QRP_RES41 = 41,
    IEC104_QRP_RES42 = 42,
    IEC104_QRP_RES43 = 43,
    IEC104_QRP_RES44 = 44,
    IEC104_QRP_RES45 = 45,
    IEC104_QRP_RES46 = 46,
    IEC104_QRP_RES47 = 47,
    IEC104_QRP_RES48 = 48,
    IEC104_QRP_RES49 = 49,
    IEC104_QRP_RES50 = 50,
    IEC104_QRP_RES51 = 51,
    IEC104_QRP_RES52 = 52,
    IEC104_QRP_RES53 = 53,
    IEC104_QRP_RES54 = 54,
    IEC104_QRP_RES55 = 55,
    IEC104_QRP_RES56 = 56,
    IEC104_QRP_RES57 = 57,
    IEC104_QRP_RES58 = 58,
    IEC104_QRP_RES59 = 59,
    IEC104_QRP_RES60 = 60,
    IEC104_QRP_RES61 = 61,
    IEC104_QRP_RES62 = 62,
    IEC104_QRP_RES63 = 63,
    IEC104_QRP_RES64 = 64,
    IEC104_QRP_RES65 = 65,
    IEC104_QRP_RES66 = 66,
    IEC104_QRP_RES67 = 67,
    IEC104_QRP_RES68 = 68,
    IEC104_QRP_RES69 = 69,
    IEC104_QRP_RES70 = 70,
    IEC104_QRP_RES71 = 71,
    IEC104_QRP_RES72 = 72,
    IEC104_QRP_RES73 = 73,
    IEC104_QRP_RES74 = 74,
    IEC104_QRP_RES75 = 75,
    IEC104_QRP_RES76 = 76,
    IEC104_QRP_RES77 = 77,
    IEC104_QRP_RES78 = 78,
    IEC104_QRP_RES79 = 79,
    IEC104_QRP_RES80 = 80,
    IEC104_QRP_RES81 = 81,
    IEC104_QRP_RES82 = 82,
    IEC104_QRP_RES83 = 83,
    IEC104_QRP_RES84 = 84,
    IEC104_QRP_RES85 = 85,
    IEC104_QRP_RES86 = 86,
    IEC104_QRP_RES87 = 87,
    IEC104_QRP_RES88 = 88,
    IEC104_QRP_RES89 = 89,
    IEC104_QRP_RES90 = 90,
    IEC104_QRP_RES91 = 91,
    IEC104_QRP_RES92 = 92,
    IEC104_QRP_RES93 = 93,
    IEC104_QRP_RES94 = 94,
    IEC104_QRP_RES95 = 95,
    IEC104_QRP_RES96 = 96,
    IEC104_QRP_RES97 = 97,
    IEC104_QRP_RES98 = 98,
    IEC104_QRP_RES99 = 99,
    IEC104_QRP_RES100 = 100,
    IEC104_QRP_RES101 = 101,
    IEC104_QRP_RES102 = 102,
    IEC104_QRP_RES103 = 103,
    IEC104_QRP_RES104 = 104,
    IEC104_QRP_RES105 = 105,
    IEC104_QRP_RES106 = 106,
    IEC104_QRP_RES107 = 107,
    IEC104_QRP_RES108 = 108,
    IEC104_QRP_RES109 = 109,
    IEC104_QRP_RES110 = 110,
    IEC104_QRP_RES111 = 111,
    IEC104_QRP_RES112 = 112,
    IEC104_QRP_RES113 = 113,
    IEC104_QRP_RES114 = 114,
    IEC104_QRP_RES115 = 115,
    IEC104_QRP_RES116 = 116,
    IEC104_QRP_RES117 = 117,
    IEC104_QRP_RES118 = 118,
    IEC104_QRP_RES119 = 119,
    IEC104_QRP_RES120 = 120,
    IEC104_QRP_RES121 = 121,
    IEC104_QRP_RES122 = 122,
    IEC104_QRP_RES123 = 123,
    IEC104_QRP_RES124 = 124,
    IEC104_QRP_RES125 = 125,
    IEC104_QRP_RES126 = 126,
    IEC104_QRP_RES127 = 127,
    IEC104_QRP_RES128 = 128,
    IEC104_QRP_RES129 = 129,
    IEC104_QRP_RES130 = 130,
    IEC104_QRP_RES131 = 131,
    IEC104_QRP_RES132 = 132,
    IEC104_QRP_RES133 = 133,
    IEC104_QRP_RES134 = 134,
    IEC104_QRP_RES135 = 135,
    IEC104_QRP_RES136 = 136,
    IEC104_QRP_RES137 = 137,
    IEC104_QRP_RES138 = 138,
    IEC104_QRP_RES139 = 139,
    IEC104_QRP_RES140 = 140,
    IEC104_QRP_RES141 = 141,
    IEC104_QRP_RES142 = 142,
    IEC104_QRP_RES143 = 143,
    IEC104_QRP_RES144 = 144,
    IEC104_QRP_RES145 = 145,
    IEC104_QRP_RES146 = 146,
    IEC104_QRP_RES147 = 147,
    IEC104_QRP_RES148 = 148,
    IEC104_QRP_RES149 = 149,
    IEC104_QRP_RES150 = 150,
    IEC104_QRP_RES151 = 151,
    IEC104_QRP_RES152 = 152,
    IEC104_QRP_RES153 = 153,
    IEC104_QRP_RES154 = 154,
    IEC104_QRP_RES155 = 155,
    IEC104_QRP_RES156 = 156,
    IEC104_QRP_RES157 = 157,
    IEC104_QRP_RES158 = 158,
    IEC104_QRP_RES159 = 159,
    IEC104_QRP_RES160 = 160,
    IEC104_QRP_RES161 = 161,
    IEC104_QRP_RES162 = 162,
    IEC104_QRP_RES163 = 163,
    IEC104_QRP_RES164 = 164,
    IEC104_QRP_RES165 = 165,
    IEC104_QRP_RES166 = 166,
    IEC104_QRP_RES167 = 167,
    IEC104_QRP_RES168 = 168,
    IEC104_QRP_RES169 = 169,
    IEC104_QRP_RES170 = 170,
    IEC104_QRP_RES171 = 171,
    IEC104_QRP_RES172 = 172,
    IEC104_QRP_RES173 = 173,
    IEC104_QRP_RES174 = 174,
    IEC104_QRP_RES175 = 175,
    IEC104_QRP_RES176 = 176,
    IEC104_QRP_RES177 = 177,
    IEC104_QRP_RES178 = 178,
    IEC104_QRP_RES179 = 179,
    IEC104_QRP_RES180 = 180,
    IEC104_QRP_RES181 = 181,
    IEC104_QRP_RES182 = 182,
    IEC104_QRP_RES183 = 183,
    IEC104_QRP_RES184 = 184,
    IEC104_QRP_RES185 = 185,
    IEC104_QRP_RES186 = 186,
    IEC104_QRP_RES187 = 187,
    IEC104_QRP_RES188 = 188,
    IEC104_QRP_RES189 = 189,
    IEC104_QRP_RES190 = 190,
    IEC104_QRP_RES191 = 191,
    IEC104_QRP_RES192 = 192,
    IEC104_QRP_RES193 = 193,
    IEC104_QRP_RES194 = 194,
    IEC104_QRP_RES195 = 195,
    IEC104_QRP_RES196 = 196,
    IEC104_QRP_RES197 = 197,
    IEC104_QRP_RES198 = 198,
    IEC104_QRP_RES199 = 199,
    IEC104_QRP_RES200 = 200,
    IEC104_QRP_RES201 = 201,
    IEC104_QRP_RES202 = 202,
    IEC104_QRP_RES203 = 203,
    IEC104_QRP_RES204 = 204,
    IEC104_QRP_RES205 = 205,
    IEC104_QRP_RES206 = 206,
    IEC104_QRP_RES207 = 207,
    IEC104_QRP_RES208 = 208,
    IEC104_QRP_RES209 = 209,
    IEC104_QRP_RES210 = 210,
    IEC104_QRP_RES211 = 211,
    IEC104_QRP_RES212 = 212,
    IEC104_QRP_RES213 = 213,
    IEC104_QRP_RES214 = 214,
    IEC104_QRP_RES215 = 215,
    IEC104_QRP_RES216 = 216,
    IEC104_QRP_RES217 = 217,
    IEC104_QRP_RES218 = 218,
    IEC104_QRP_RES219 = 219,
    IEC104_QRP_RES220 = 220,
    IEC104_QRP_RES221 = 221,
    IEC104_QRP_RES222 = 222,
    IEC104_QRP_RES223 = 223,
    IEC104_QRP_RES224 = 224,
    IEC104_QRP_RES225 = 225,
    IEC104_QRP_RES226 = 226,
    IEC104_QRP_RES227 = 227,
    IEC104_QRP_RES228 = 228,
    IEC104_QRP_RES229 = 229,
    IEC104_QRP_RES230 = 230,
    IEC104_QRP_RES231 = 231,
    IEC104_QRP_RES232 = 232,
    IEC104_QRP_RES233 = 233,
    IEC104_QRP_RES234 = 234,
    IEC104_QRP_RES235 = 235,
    IEC104_QRP_RES236 = 236,
    IEC104_QRP_RES237 = 237,
    IEC104_QRP_RES238 = 238,
    IEC104_QRP_RES239 = 239,
    IEC104_QRP_RES240 = 240,
    IEC104_QRP_RES241 = 241,
    IEC104_QRP_RES242 = 242,
    IEC104_QRP_RES243 = 243,
    IEC104_QRP_RES244 = 244,
    IEC104_QRP_RES245 = 245,
    IEC104_QRP_RES246 = 246,
    IEC104_QRP_RES247 = 247,
    IEC104_QRP_RES248 = 248,
    IEC104_QRP_RES249 = 249,
    IEC104_QRP_RES250 = 250,
    IEC104_QRP_RES251 = 251,
    IEC104_QRP_RES252 = 252,
    IEC104_QRP_RES253 = 253,
    IEC104_QRP_RES254 = 254,
    IEC104_QRP_RES255 = 255,
};

enum Iec104FrqUiEnum
{
    IEC104_FRQ_UI_DEFAULT = 0,
    IEC104_FRQ_UI_RES1 = 1,
    IEC104_FRQ_UI_RES2 = 2,
    IEC104_FRQ_UI_RES3 = 3,
    IEC104_FRQ_UI_RES4 = 4,
    IEC104_FRQ_UI_RES5 = 5,
    IEC104_FRQ_UI_RES6 = 6,
    IEC104_FRQ_UI_RES7 = 7,
    IEC104_FRQ_UI_RES8 = 8,
    IEC104_FRQ_UI_RES9 = 9,
    IEC104_FRQ_UI_RES10 = 10,
    IEC104_FRQ_UI_RES11 = 11,
    IEC104_FRQ_UI_RES12 = 12,
    IEC104_FRQ_UI_RES13 = 13,
    IEC104_FRQ_UI_RES14 = 14,
    IEC104_FRQ_UI_RES15 = 15,
    IEC104_FRQ_UI_RES16 = 16,
    IEC104_FRQ_UI_RES17 = 17,
    IEC104_FRQ_UI_RES18 = 18,
    IEC104_FRQ_UI_RES19 = 19,
    IEC104_FRQ_UI_RES20 = 20,
    IEC104_FRQ_UI_RES21 = 21,
    IEC104_FRQ_UI_RES22 = 22,
    IEC104_FRQ_UI_RES23 = 23,
    IEC104_FRQ_UI_RES24 = 24,
    IEC104_FRQ_UI_RES25 = 25,
    IEC104_FRQ_UI_RES26 = 26,
    IEC104_FRQ_UI_RES27 = 27,
    IEC104_FRQ_UI_RES28 = 28,
    IEC104_FRQ_UI_RES29 = 29,
    IEC104_FRQ_UI_RES30 = 30,
    IEC104_FRQ_UI_RES31 = 31,
    IEC104_FRQ_UI_RES32 = 32,
    IEC104_FRQ_UI_RES33 = 33,
    IEC104_FRQ_UI_RES34 = 34,
    IEC104_FRQ_UI_RES35 = 35,
    IEC104_FRQ_UI_RES36 = 36,
    IEC104_FRQ_UI_RES37 = 37,
    IEC104_FRQ_UI_RES38 = 38,
    IEC104_FRQ_UI_RES39 = 39,
    IEC104_FRQ_UI_RES40 = 40,
    IEC104_FRQ_UI_RES41 = 41,
    IEC104_FRQ_UI_RES42 = 42,
    IEC104_FRQ_UI_RES43 = 43,
    IEC104_FRQ_UI_RES44 = 44,
    IEC104_FRQ_UI_RES45 = 45,
    IEC104_FRQ_UI_RES46 = 46,
    IEC104_FRQ_UI_RES47 = 47,
    IEC104_FRQ_UI_RES48 = 48,
    IEC104_FRQ_UI_RES49 = 49,
    IEC104_FRQ_UI_RES50 = 50,
    IEC104_FRQ_UI_RES51 = 51,
    IEC104_FRQ_UI_RES52 = 52,
    IEC104_FRQ_UI_RES53 = 53,
    IEC104_FRQ_UI_RES54 = 54,
    IEC104_FRQ_UI_RES55 = 55,
    IEC104_FRQ_UI_RES56 = 56,
    IEC104_FRQ_UI_RES57 = 57,
    IEC104_FRQ_UI_RES58 = 58,
    IEC104_FRQ_UI_RES59 = 59,
    IEC104_FRQ_UI_RES60 = 60,
    IEC104_FRQ_UI_RES61 = 61,
    IEC104_FRQ_UI_RES62 = 62,
    IEC104_FRQ_UI_RES63 = 63,
    IEC104_FRQ_UI_RES64 = 64,
    IEC104_FRQ_UI_RES65 = 65,
    IEC104_FRQ_UI_RES66 = 66,
    IEC104_FRQ_UI_RES67 = 67,
    IEC104_FRQ_UI_RES68 = 68,
    IEC104_FRQ_UI_RES69 = 69,
    IEC104_FRQ_UI_RES70 = 70,
    IEC104_FRQ_UI_RES71 = 71,
    IEC104_FRQ_UI_RES72 = 72,
    IEC104_FRQ_UI_RES73 = 73,
    IEC104_FRQ_UI_RES74 = 74,
    IEC104_FRQ_UI_RES75 = 75,
    IEC104_FRQ_UI_RES76 = 76,
    IEC104_FRQ_UI_RES77 = 77,
    IEC104_FRQ_UI_RES78 = 78,
    IEC104_FRQ_UI_RES79 = 79,
    IEC104_FRQ_UI_RES80 = 80,
    IEC104_FRQ_UI_RES81 = 81,
    IEC104_FRQ_UI_RES82 = 82,
    IEC104_FRQ_UI_RES83 = 83,
    IEC104_FRQ_UI_RES84 = 84,
    IEC104_FRQ_UI_RES85 = 85,
    IEC104_FRQ_UI_RES86 = 86,
    IEC104_FRQ_UI_RES87 = 87,
    IEC104_FRQ_UI_RES88 = 88,
    IEC104_FRQ_UI_RES89 = 89,
    IEC104_FRQ_UI_RES90 = 90,
    IEC104_FRQ_UI_RES91 = 91,
    IEC104_FRQ_UI_RES92 = 92,
    IEC104_FRQ_UI_RES93 = 93,
    IEC104_FRQ_UI_RES94 = 94,
    IEC104_FRQ_UI_RES95 = 95,
    IEC104_FRQ_UI_RES96 = 96,
    IEC104_FRQ_UI_RES97 = 97,
    IEC104_FRQ_UI_RES98 = 98,
    IEC104_FRQ_UI_RES99 = 99,
    IEC104_FRQ_UI_RES100 = 100,
    IEC104_FRQ_UI_RES101 = 101,
    IEC104_FRQ_UI_RES102 = 102,
    IEC104_FRQ_UI_RES103 = 103,
    IEC104_FRQ_UI_RES104 = 104,
    IEC104_FRQ_UI_RES105 = 105,
    IEC104_FRQ_UI_RES106 = 106,
    IEC104_FRQ_UI_RES107 = 107,
    IEC104_FRQ_UI_RES108 = 108,
    IEC104_FRQ_UI_RES109 = 109,
    IEC104_FRQ_UI_RES110 = 110,
    IEC104_FRQ_UI_RES111 = 111,
    IEC104_FRQ_UI_RES112 = 112,
    IEC104_FRQ_UI_RES113 = 113,
    IEC104_FRQ_UI_RES114 = 114,
    IEC104_FRQ_UI_RES115 = 115,
    IEC104_FRQ_UI_RES116 = 116,
    IEC104_FRQ_UI_RES117 = 117,
    IEC104_FRQ_UI_RES118 = 118,
    IEC104_FRQ_UI_RES119 = 119,
    IEC104_FRQ_UI_RES120 = 120,
    IEC104_FRQ_UI_RES121 = 121,
    IEC104_FRQ_UI_RES122 = 122,
    IEC104_FRQ_UI_RES123 = 123,
    IEC104_FRQ_UI_RES124 = 124,
    IEC104_FRQ_UI_RES125 = 125,
    IEC104_FRQ_UI_RES126 = 126,
    IEC104_FRQ_UI_RES127 = 127,
};

enum Iec104SrqUiEnum
{
    IEC104_SRQ_UI_DEFAULT = 0,
    IEC104_SRQ_UI_RES1 = 1,
    IEC104_SRQ_UI_RES2 = 2,
    IEC104_SRQ_UI_RES3 = 3,
    IEC104_SRQ_UI_RES4 = 4,
    IEC104_SRQ_UI_RES5 = 5,
    IEC104_SRQ_UI_RES6 = 6,
    IEC104_SRQ_UI_RES7 = 7,
    IEC104_SRQ_UI_RES8 = 8,
    IEC104_SRQ_UI_RES9 = 9,
    IEC104_SRQ_UI_RES10 = 10,
    IEC104_SRQ_UI_RES11 = 11,
    IEC104_SRQ_UI_RES12 = 12,
    IEC104_SRQ_UI_RES13 = 13,
    IEC104_SRQ_UI_RES14 = 14,
    IEC104_SRQ_UI_RES15 = 15,
    IEC104_SRQ_UI_RES16 = 16,
    IEC104_SRQ_UI_RES17 = 17,
    IEC104_SRQ_UI_RES18 = 18,
    IEC104_SRQ_UI_RES19 = 19,
    IEC104_SRQ_UI_RES20 = 20,
    IEC104_SRQ_UI_RES21 = 21,
    IEC104_SRQ_UI_RES22 = 22,
    IEC104_SRQ_UI_RES23 = 23,
    IEC104_SRQ_UI_RES24 = 24,
    IEC104_SRQ_UI_RES25 = 25,
    IEC104_SRQ_UI_RES26 = 26,
    IEC104_SRQ_UI_RES27 = 27,
    IEC104_SRQ_UI_RES28 = 28,
    IEC104_SRQ_UI_RES29 = 29,
    IEC104_SRQ_UI_RES30 = 30,
    IEC104_SRQ_UI_RES31 = 31,
    IEC104_SRQ_UI_RES32 = 32,
    IEC104_SRQ_UI_RES33 = 33,
    IEC104_SRQ_UI_RES34 = 34,
    IEC104_SRQ_UI_RES35 = 35,
    IEC104_SRQ_UI_RES36 = 36,
    IEC104_SRQ_UI_RES37 = 37,
    IEC104_SRQ_UI_RES38 = 38,
    IEC104_SRQ_UI_RES39 = 39,
    IEC104_SRQ_UI_RES40 = 40,
    IEC104_SRQ_UI_RES41 = 41,
    IEC104_SRQ_UI_RES42 = 42,
    IEC104_SRQ_UI_RES43 = 43,
    IEC104_SRQ_UI_RES44 = 44,
    IEC104_SRQ_UI_RES45 = 45,
    IEC104_SRQ_UI_RES46 = 46,
    IEC104_SRQ_UI_RES47 = 47,
    IEC104_SRQ_UI_RES48 = 48,
    IEC104_SRQ_UI_RES49 = 49,
    IEC104_SRQ_UI_RES50 = 50,
    IEC104_SRQ_UI_RES51 = 51,
    IEC104_SRQ_UI_RES52 = 52,
    IEC104_SRQ_UI_RES53 = 53,
    IEC104_SRQ_UI_RES54 = 54,
    IEC104_SRQ_UI_RES55 = 55,
    IEC104_SRQ_UI_RES56 = 56,
    IEC104_SRQ_UI_RES57 = 57,
    IEC104_SRQ_UI_RES58 = 58,
    IEC104_SRQ_UI_RES59 = 59,
    IEC104_SRQ_UI_RES60 = 60,
    IEC104_SRQ_UI_RES61 = 61,
    IEC104_SRQ_UI_RES62 = 62,
    IEC104_SRQ_UI_RES63 = 63,
    IEC104_SRQ_UI_RES64 = 64,
    IEC104_SRQ_UI_RES65 = 65,
    IEC104_SRQ_UI_RES66 = 66,
    IEC104_SRQ_UI_RES67 = 67,
    IEC104_SRQ_UI_RES68 = 68,
    IEC104_SRQ_UI_RES69 = 69,
    IEC104_SRQ_UI_RES70 = 70,
    IEC104_SRQ_UI_RES71 = 71,
    IEC104_SRQ_UI_RES72 = 72,
    IEC104_SRQ_UI_RES73 = 73,
    IEC104_SRQ_UI_RES74 = 74,
    IEC104_SRQ_UI_RES75 = 75,
    IEC104_SRQ_UI_RES76 = 76,
    IEC104_SRQ_UI_RES77 = 77,
    IEC104_SRQ_UI_RES78 = 78,
    IEC104_SRQ_UI_RES79 = 79,
    IEC104_SRQ_UI_RES80 = 80,
    IEC104_SRQ_UI_RES81 = 81,
    IEC104_SRQ_UI_RES82 = 82,
    IEC104_SRQ_UI_RES83 = 83,
    IEC104_SRQ_UI_RES84 = 84,
    IEC104_SRQ_UI_RES85 = 85,
    IEC104_SRQ_UI_RES86 = 86,
    IEC104_SRQ_UI_RES87 = 87,
    IEC104_SRQ_UI_RES88 = 88,
    IEC104_SRQ_UI_RES89 = 89,
    IEC104_SRQ_UI_RES90 = 90,
    IEC104_SRQ_UI_RES91 = 91,
    IEC104_SRQ_UI_RES92 = 92,
    IEC104_SRQ_UI_RES93 = 93,
    IEC104_SRQ_UI_RES94 = 94,
    IEC104_SRQ_UI_RES95 = 95,
    IEC104_SRQ_UI_RES96 = 96,
    IEC104_SRQ_UI_RES97 = 97,
    IEC104_SRQ_UI_RES98 = 98,
    IEC104_SRQ_UI_RES99 = 99,
    IEC104_SRQ_UI_RES100 = 100,
    IEC104_SRQ_UI_RES101 = 101,
    IEC104_SRQ_UI_RES102 = 102,
    IEC104_SRQ_UI_RES103 = 103,
    IEC104_SRQ_UI_RES104 = 104,
    IEC104_SRQ_UI_RES105 = 105,
    IEC104_SRQ_UI_RES106 = 106,
    IEC104_SRQ_UI_RES107 = 107,
    IEC104_SRQ_UI_RES108 = 108,
    IEC104_SRQ_UI_RES109 = 109,
    IEC104_SRQ_UI_RES110 = 110,
    IEC104_SRQ_UI_RES111 = 111,
    IEC104_SRQ_UI_RES112 = 112,
    IEC104_SRQ_UI_RES113 = 113,
    IEC104_SRQ_UI_RES114 = 114,
    IEC104_SRQ_UI_RES115 = 115,
    IEC104_SRQ_UI_RES116 = 116,
    IEC104_SRQ_UI_RES117 = 117,
    IEC104_SRQ_UI_RES118 = 118,
    IEC104_SRQ_UI_RES119 = 119,
    IEC104_SRQ_UI_RES120 = 120,
    IEC104_SRQ_UI_RES121 = 121,
    IEC104_SRQ_UI_RES122 = 122,
    IEC104_SRQ_UI_RES123 = 123,
    IEC104_SRQ_UI_RES124 = 124,
    IEC104_SRQ_UI_RES125 = 125,
    IEC104_SRQ_UI_RES126 = 126,
    IEC104_SRQ_UI_RES127 = 127,
};

enum Iec104ScqUi1Enum
{
    IEC104_SCQ_UI1_DEFAULT = 0,
    IEC104_SCQ_UI1_SELECTFILE = 1,
    IEC104_SCQ_UI1_REQUESTFILE = 2,
    IEC104_SCQ_UI1_DEACTFILE = 3,
    IEC104_SCQ_UI1_DELETEFILE = 4,
    IEC104_SCQ_UI1_SELECTSECTION = 5,
    IEC104_SCQ_UI1_REQUESTSECTION = 6,
    IEC104_SCQ_UI1_DEACTSECTION = 7,
    IEC104_SCQ_UI1_RES8 = 8,
    IEC104_SCQ_UI1_RES9 = 9,
    IEC104_SCQ_UI1_RES10 = 10,
    IEC104_SCQ_UI1_RES11 = 11,
    IEC104_SCQ_UI1_RES12 = 12,
    IEC104_SCQ_UI1_RES13 = 13,
    IEC104_SCQ_UI1_RES14 = 14,
    IEC104_SCQ_UI1_RES15 = 15,
};

enum Iec104ScqUi2Enum
{
    IEC104_SCQ_UI2_DEFAULT = 0,
    IEC104_SCQ_UI2_MEMUNAVAIL = 1,
    IEC104_SCQ_UI2_CHKSMFAILED = 2,
    IEC104_SCQ_UI2_UNEXPECTEDCOMM = 3,
    IEC104_SCQ_UI2_UNEXPECTEDNOF = 4,
    IEC104_SCQ_UI2_UNEXPECTEDNOS = 5,
    IEC104_SCQ_UI2_RES6 = 6,
    IEC104_SCQ_UI2_RES7 = 7,
    IEC104_SCQ_UI2_RES8 = 8,
    IEC104_SCQ_UI2_RES9 = 9,
    IEC104_SCQ_UI2_RES10 = 10,
    IEC104_SCQ_UI2_RES11 = 11,
    IEC104_SCQ_UI2_RES12 = 12,
    IEC104_SCQ_UI2_RES13 = 13,
    IEC104_SCQ_UI2_RES14 = 14,
    IEC104_SCQ_UI2_RES15 = 15,
};

enum Iec104LsqEnum
{
    IEC104_LSQ_NOTUSED = 0,
    IEC104_LSQ_FILETRANSFERWITHOUTDEACT = 1,
    IEC104_LSQ_FILETRANSFERWITHDEACT = 2,
    IEC104_LSQ_SECTIONTRANSFERWITHOUTDEACT = 3,
    IEC104_LSQ_SECTIONTRANSFERWITHDEACT = 4,
    IEC104_LSQ_RES5 = 5,
    IEC104_LSQ_RES6 = 6,
    IEC104_LSQ_RES7 = 7,
    IEC104_LSQ_RES8 = 8,
    IEC104_LSQ_RES9 = 9,
    IEC104_LSQ_RES10 = 10,
    IEC104_LSQ_RES11 = 11,
    IEC104_LSQ_RES12 = 12,
    IEC104_LSQ_RES13 = 13,
    IEC104_LSQ_RES14 = 14,
    IEC104_LSQ_RES15 = 15,
    IEC104_LSQ_RES16 = 16,
    IEC104_LSQ_RES17 = 17,
    IEC104_LSQ_RES18 = 18,
    IEC104_LSQ_RES19 = 19,
    IEC104_LSQ_RES20 = 20,
    IEC104_LSQ_RES21 = 21,
    IEC104_LSQ_RES22 = 22,
    IEC104_LSQ_RES23 = 23,
    IEC104_LSQ_RES24 = 24,
    IEC104_LSQ_RES25 = 25,
    IEC104_LSQ_RES26 = 26,
    IEC104_LSQ_RES27 = 27,
    IEC104_LSQ_RES28 = 28,
    IEC104_LSQ_RES29 = 29,
    IEC104_LSQ_RES30 = 30,
    IEC104_LSQ_RES31 = 31,
    IEC104_LSQ_RES32 = 32,
    IEC104_LSQ_RES33 = 33,
    IEC104_LSQ_RES34 = 34,
    IEC104_LSQ_RES35 = 35,
    IEC104_LSQ_RES36 = 36,
    IEC104_LSQ_RES37 = 37,
    IEC104_LSQ_RES38 = 38,
    IEC104_LSQ_RES39 = 39,
    IEC104_LSQ_RES40 = 40,
    IEC104_LSQ_RES41 = 41,
    IEC104_LSQ_RES42 = 42,
    IEC104_LSQ_RES43 = 43,
    IEC104_LSQ_RES44 = 44,
    IEC104_LSQ_RES45 = 45,
    IEC104_LSQ_RES46 = 46,
    IEC104_LSQ_RES47 = 47,
    IEC104_LSQ_RES48 = 48,
    IEC104_LSQ_RES49 = 49,
    IEC104_LSQ_RES50 = 50,
    IEC104_LSQ_RES51 = 51,
    IEC104_LSQ_RES52 = 52,
    IEC104_LSQ_RES53 = 53,
    IEC104_LSQ_RES54 = 54,
    IEC104_LSQ_RES55 = 55,
    IEC104_LSQ_RES56 = 56,
    IEC104_LSQ_RES57 = 57,
    IEC104_LSQ_RES58 = 58,
    IEC104_LSQ_RES59 = 59,
    IEC104_LSQ_RES60 = 60,
    IEC104_LSQ_RES61 = 61,
    IEC104_LSQ_RES62 = 62,
    IEC104_LSQ_RES63 = 63,
    IEC104_LSQ_RES64 = 64,
    IEC104_LSQ_RES65 = 65,
    IEC104_LSQ_RES66 = 66,
    IEC104_LSQ_RES67 = 67,
    IEC104_LSQ_RES68 = 68,
    IEC104_LSQ_RES69 = 69,
    IEC104_LSQ_RES70 = 70,
    IEC104_LSQ_RES71 = 71,
    IEC104_LSQ_RES72 = 72,
    IEC104_LSQ_RES73 = 73,
    IEC104_LSQ_RES74 = 74,
    IEC104_LSQ_RES75 = 75,
    IEC104_LSQ_RES76 = 76,
    IEC104_LSQ_RES77 = 77,
    IEC104_LSQ_RES78 = 78,
    IEC104_LSQ_RES79 = 79,
    IEC104_LSQ_RES80 = 80,
    IEC104_LSQ_RES81 = 81,
    IEC104_LSQ_RES82 = 82,
    IEC104_LSQ_RES83 = 83,
    IEC104_LSQ_RES84 = 84,
    IEC104_LSQ_RES85 = 85,
    IEC104_LSQ_RES86 = 86,
    IEC104_LSQ_RES87 = 87,
    IEC104_LSQ_RES88 = 88,
    IEC104_LSQ_RES89 = 89,
    IEC104_LSQ_RES90 = 90,
    IEC104_LSQ_RES91 = 91,
    IEC104_LSQ_RES92 = 92,
    IEC104_LSQ_RES93 = 93,
    IEC104_LSQ_RES94 = 94,
    IEC104_LSQ_RES95 = 95,
    IEC104_LSQ_RES96 = 96,
    IEC104_LSQ_RES97 = 97,
    IEC104_LSQ_RES98 = 98,
    IEC104_LSQ_RES99 = 99,
    IEC104_LSQ_RES100 = 100,
    IEC104_LSQ_RES101 = 101,
    IEC104_LSQ_RES102 = 102,
    IEC104_LSQ_RES103 = 103,
    IEC104_LSQ_RES104 = 104,
    IEC104_LSQ_RES105 = 105,
    IEC104_LSQ_RES106 = 106,
    IEC104_LSQ_RES107 = 107,
    IEC104_LSQ_RES108 = 108,
    IEC104_LSQ_RES109 = 109,
    IEC104_LSQ_RES110 = 110,
    IEC104_LSQ_RES111 = 111,
    IEC104_LSQ_RES112 = 112,
    IEC104_LSQ_RES113 = 113,
    IEC104_LSQ_RES114 = 114,
    IEC104_LSQ_RES115 = 115,
    IEC104_LSQ_RES116 = 116,
    IEC104_LSQ_RES117 = 117,
    IEC104_LSQ_RES118 = 118,
    IEC104_LSQ_RES119 = 119,
    IEC104_LSQ_RES120 = 120,
    IEC104_LSQ_RES121 = 121,
    IEC104_LSQ_RES122 = 122,
    IEC104_LSQ_RES123 = 123,
    IEC104_LSQ_RES124 = 124,
    IEC104_LSQ_RES125 = 125,
    IEC104_LSQ_RES126 = 126,
    IEC104_LSQ_RES127 = 127,
    IEC104_LSQ_RES128 = 128,
    IEC104_LSQ_RES129 = 129,
    IEC104_LSQ_RES130 = 130,
    IEC104_LSQ_RES131 = 131,
    IEC104_LSQ_RES132 = 132,
    IEC104_LSQ_RES133 = 133,
    IEC104_LSQ_RES134 = 134,
    IEC104_LSQ_RES135 = 135,
    IEC104_LSQ_RES136 = 136,
    IEC104_LSQ_RES137 = 137,
    IEC104_LSQ_RES138 = 138,
    IEC104_LSQ_RES139 = 139,
    IEC104_LSQ_RES140 = 140,
    IEC104_LSQ_RES141 = 141,
    IEC104_LSQ_RES142 = 142,
    IEC104_LSQ_RES143 = 143,
    IEC104_LSQ_RES144 = 144,
    IEC104_LSQ_RES145 = 145,
    IEC104_LSQ_RES146 = 146,
    IEC104_LSQ_RES147 = 147,
    IEC104_LSQ_RES148 = 148,
    IEC104_LSQ_RES149 = 149,
    IEC104_LSQ_RES150 = 150,
    IEC104_LSQ_RES151 = 151,
    IEC104_LSQ_RES152 = 152,
    IEC104_LSQ_RES153 = 153,
    IEC104_LSQ_RES154 = 154,
    IEC104_LSQ_RES155 = 155,
    IEC104_LSQ_RES156 = 156,
    IEC104_LSQ_RES157 = 157,
    IEC104_LSQ_RES158 = 158,
    IEC104_LSQ_RES159 = 159,
    IEC104_LSQ_RES160 = 160,
    IEC104_LSQ_RES161 = 161,
    IEC104_LSQ_RES162 = 162,
    IEC104_LSQ_RES163 = 163,
    IEC104_LSQ_RES164 = 164,
    IEC104_LSQ_RES165 = 165,
    IEC104_LSQ_RES166 = 166,
    IEC104_LSQ_RES167 = 167,
    IEC104_LSQ_RES168 = 168,
    IEC104_LSQ_RES169 = 169,
    IEC104_LSQ_RES170 = 170,
    IEC104_LSQ_RES171 = 171,
    IEC104_LSQ_RES172 = 172,
    IEC104_LSQ_RES173 = 173,
    IEC104_LSQ_RES174 = 174,
    IEC104_LSQ_RES175 = 175,
    IEC104_LSQ_RES176 = 176,
    IEC104_LSQ_RES177 = 177,
    IEC104_LSQ_RES178 = 178,
    IEC104_LSQ_RES179 = 179,
    IEC104_LSQ_RES180 = 180,
    IEC104_LSQ_RES181 = 181,
    IEC104_LSQ_RES182 = 182,
    IEC104_LSQ_RES183 = 183,
    IEC104_LSQ_RES184 = 184,
    IEC104_LSQ_RES185 = 185,
    IEC104_LSQ_RES186 = 186,
    IEC104_LSQ_RES187 = 187,
    IEC104_LSQ_RES188 = 188,
    IEC104_LSQ_RES189 = 189,
    IEC104_LSQ_RES190 = 190,
    IEC104_LSQ_RES191 = 191,
    IEC104_LSQ_RES192 = 192,
    IEC104_LSQ_RES193 = 193,
    IEC104_LSQ_RES194 = 194,
    IEC104_LSQ_RES195 = 195,
    IEC104_LSQ_RES196 = 196,
    IEC104_LSQ_RES197 = 197,
    IEC104_LSQ_RES198 = 198,
    IEC104_LSQ_RES199 = 199,
    IEC104_LSQ_RES200 = 200,
    IEC104_LSQ_RES201 = 201,
    IEC104_LSQ_RES202 = 202,
    IEC104_LSQ_RES203 = 203,
    IEC104_LSQ_RES204 = 204,
    IEC104_LSQ_RES205 = 205,
    IEC104_LSQ_RES206 = 206,
    IEC104_LSQ_RES207 = 207,
    IEC104_LSQ_RES208 = 208,
    IEC104_LSQ_RES209 = 209,
    IEC104_LSQ_RES210 = 210,
    IEC104_LSQ_RES211 = 211,
    IEC104_LSQ_RES212 = 212,
    IEC104_LSQ_RES213 = 213,
    IEC104_LSQ_RES214 = 214,
    IEC104_LSQ_RES215 = 215,
    IEC104_LSQ_RES216 = 216,
    IEC104_LSQ_RES217 = 217,
    IEC104_LSQ_RES218 = 218,
    IEC104_LSQ_RES219 = 219,
    IEC104_LSQ_RES220 = 220,
    IEC104_LSQ_RES221 = 221,
    IEC104_LSQ_RES222 = 222,
    IEC104_LSQ_RES223 = 223,
    IEC104_LSQ_RES224 = 224,
    IEC104_LSQ_RES225 = 225,
    IEC104_LSQ_RES226 = 226,
    IEC104_LSQ_RES227 = 227,
    IEC104_LSQ_RES228 = 228,
    IEC104_LSQ_RES229 = 229,
    IEC104_LSQ_RES230 = 230,
    IEC104_LSQ_RES231 = 231,
    IEC104_LSQ_RES232 = 232,
    IEC104_LSQ_RES233 = 233,
    IEC104_LSQ_RES234 = 234,
    IEC104_LSQ_RES235 = 235,
    IEC104_LSQ_RES236 = 236,
    IEC104_LSQ_RES237 = 237,
    IEC104_LSQ_RES238 = 238,
    IEC104_LSQ_RES239 = 239,
    IEC104_LSQ_RES240 = 240,
    IEC104_LSQ_RES241 = 241,
    IEC104_LSQ_RES242 = 242,
    IEC104_LSQ_RES243 = 243,
    IEC104_LSQ_RES244 = 244,
    IEC104_LSQ_RES245 = 245,
    IEC104_LSQ_RES246 = 246,
    IEC104_LSQ_RES247 = 247,
    IEC104_LSQ_RES248 = 248,
    IEC104_LSQ_RES249 = 249,
    IEC104_LSQ_RES250 = 250,
    IEC104_LSQ_RES251 = 251,
    IEC104_LSQ_RES252 = 252,
    IEC104_LSQ_RES253 = 253,
    IEC104_LSQ_RES254 = 254,
    IEC104_LSQ_RES255 = 255,
};

enum Iec104AfqUi1Enum
{
    IEC104_AFQ_UI1_NOTUSED = 0,
    IEC104_AFQ_UI1_POSFILEACK = 1,
    IEC104_AFQ_UI1_NEGFILEACK = 2,
    IEC104_AFQ_UI1_POSSECTIONACK = 3,
    IEC104_AFQ_UI1_NEGSECTIONACK = 4,
    IEC104_AFQ_UI1_RES5 = 5,
    IEC104_AFQ_UI1_RES6 = 6,
    IEC104_AFQ_UI1_RES7 = 7,
    IEC104_AFQ_UI1_RES8 = 8,
    IEC104_AFQ_UI1_RES9 = 9,
    IEC104_AFQ_UI1_RES10 = 10,
    IEC104_AFQ_UI1_RES11 = 11,
    IEC104_AFQ_UI1_RES12 = 12,
    IEC104_AFQ_UI1_RES13 = 13,
    IEC104_AFQ_UI1_RES14 = 14,
    IEC104_AFQ_UI1_RES15 = 15,
};

enum Iec104AfqUi2Enum
{
    IEC104_AFQ_UI2_DEFAULT = 0,
    IEC104_AFQ_UI2_MEMUNAVAIL = 1,
    IEC104_AFQ_UI2_CHKSMFAILED = 2,
    IEC104_AFQ_UI2_UNEXPECTEDCOMM = 3,
    IEC104_AFQ_UI2_UNEXPECTEDNOF = 4,
    IEC104_AFQ_UI2_UNEXPECTEDNOS = 5,
    IEC104_AFQ_UI2_RES6 = 6,
    IEC104_AFQ_UI2_RES7 = 7,
    IEC104_AFQ_UI2_RES8 = 8,
    IEC104_AFQ_UI2_RES9 = 9,
    IEC104_AFQ_UI2_RES10 = 10,
    IEC104_AFQ_UI2_RES11 = 11,
    IEC104_AFQ_UI2_RES12 = 12,
    IEC104_AFQ_UI2_RES13 = 13,
    IEC104_AFQ_UI2_RES14 = 14,
    IEC104_AFQ_UI2_RES15 = 15,
};

enum Iec104DiqDpiEnum
{
    IEC104_DIQ_DPI_INDETERMSTATE1 = 0,
    IEC104_DIQ_DPI_STATEOFF = 1,
    IEC104_DIQ_DPI_STATEON = 2,
    IEC104_DIQ_DPI_INDETERMSTATE2 = 3,
};

enum Iec104SepEsEnum
{
    IEC104_SEP_ES_INDETERMSTATE1 = 0,
    IEC104_SEP_ES_STATEOFF = 1,
    IEC104_SEP_ES_STATEON = 2,
    IEC104_SEP_ES_INDETERMSTATE2 = 3,
};

enum Iec104DcoDcsEnum
{
    IEC104_DCO_DCS_NOTPERMITTED1 = 0,
    IEC104_DCO_DCS_STATEOFF = 1,
    IEC104_DCO_DCS_STATEON = 2,
    IEC104_DCO_DCS_NOTPERMITTED2 = 3,
};

enum Iec104RcoRcsEnum
{
    IEC104_RCO_RCS_NOTPERMITTED1 = 0,
    IEC104_RCO_RCS_STEPLOWER = 1,
    IEC104_RCO_RCS_STEPHIGHER = 2,
    IEC104_RCO_RCS_NOTPERMITTED2 = 3,
};

enum Iec104SofStatusEnum
{
    IEC104_SOF_STATUS_DEFAULT = 0,
    IEC104_SOF_STATUS_RES1 = 1,
    IEC104_SOF_STATUS_RES2 = 2,
    IEC104_SOF_STATUS_RES3 = 3,
    IEC104_SOF_STATUS_RES4 = 4,
    IEC104_SOF_STATUS_RES5 = 5,
    IEC104_SOF_STATUS_RES6 = 6,
    IEC104_SOF_STATUS_RES7 = 7,
    IEC104_SOF_STATUS_RES8 = 8,
    IEC104_SOF_STATUS_RES9 = 9,
    IEC104_SOF_STATUS_RES10 = 10,
    IEC104_SOF_STATUS_RES11 = 11,
    IEC104_SOF_STATUS_RES12 = 12,
    IEC104_SOF_STATUS_RES13 = 13,
    IEC104_SOF_STATUS_RES14 = 14,
    IEC104_SOF_STATUS_RES15 = 15,
    IEC104_SOF_STATUS_RES16 = 16,
    IEC104_SOF_STATUS_RES17 = 17,
    IEC104_SOF_STATUS_RES18 = 18,
    IEC104_SOF_STATUS_RES19 = 19,
    IEC104_SOF_STATUS_RES20 = 20,
    IEC104_SOF_STATUS_RES21 = 21,
    IEC104_SOF_STATUS_RES22 = 22,
    IEC104_SOF_STATUS_RES23 = 23,
    IEC104_SOF_STATUS_RES24 = 24,
    IEC104_SOF_STATUS_RES25 = 25,
    IEC104_SOF_STATUS_RES26 = 26,
    IEC104_SOF_STATUS_RES27 = 27,
    IEC104_SOF_STATUS_RES28 = 28,
    IEC104_SOF_STATUS_RES29 = 29,
    IEC104_SOF_STATUS_RES30 = 30,
    IEC104_SOF_STATUS_RES31 = 31,
};

enum Iec104QosQlEnum
{
    IEC104_QOS_QL_DEFAULT = 0,
    IEC104_QOS_QL_RES1 = 1,
    IEC104_QOS_QL_RES2 = 2,
    IEC104_QOS_QL_RES3 = 3,
    IEC104_QOS_QL_RES4 = 4,
    IEC104_QOS_QL_RES5 = 5,
    IEC104_QOS_QL_RES6 = 6,
    IEC104_QOS_QL_RES7 = 7,
    IEC104_QOS_QL_RES8 = 8,
    IEC104_QOS_QL_RES9 = 9,
    IEC104_QOS_QL_RES10 = 10,
    IEC104_QOS_QL_RES11 = 11,
    IEC104_QOS_QL_RES12 = 12,
    IEC104_QOS_QL_RES13 = 13,
    IEC104_QOS_QL_RES14 = 14,
    IEC104_QOS_QL_RES15 = 15,
    IEC104_QOS_QL_RES16 = 16,
    IEC104_QOS_QL_RES17 = 17,
    IEC104_QOS_QL_RES18 = 18,
    IEC104_QOS_QL_RES19 = 19,
    IEC104_QOS_QL_RES20 = 20,
    IEC104_QOS_QL_RES21 = 21,
    IEC104_QOS_QL_RES22 = 22,
    IEC104_QOS_QL_RES23 = 23,
    IEC104_QOS_QL_RES24 = 24,
    IEC104_QOS_QL_RES25 = 25,
    IEC104_QOS_QL_RES26 = 26,
    IEC104_QOS_QL_RES27 = 27,
    IEC104_QOS_QL_RES28 = 28,
    IEC104_QOS_QL_RES29 = 29,
    IEC104_QOS_QL_RES30 = 30,
    IEC104_QOS_QL_RES31 = 31,
    IEC104_QOS_QL_RES32 = 32,
    IEC104_QOS_QL_RES33 = 33,
    IEC104_QOS_QL_RES34 = 34,
    IEC104_QOS_QL_RES35 = 35,
    IEC104_QOS_QL_RES36 = 36,
    IEC104_QOS_QL_RES37 = 37,
    IEC104_QOS_QL_RES38 = 38,
    IEC104_QOS_QL_RES39 = 39,
    IEC104_QOS_QL_RES40 = 40,
    IEC104_QOS_QL_RES41 = 41,
    IEC104_QOS_QL_RES42 = 42,
    IEC104_QOS_QL_RES43 = 43,
    IEC104_QOS_QL_RES44 = 44,
    IEC104_QOS_QL_RES45 = 45,
    IEC104_QOS_QL_RES46 = 46,
    IEC104_QOS_QL_RES47 = 47,
    IEC104_QOS_QL_RES48 = 48,
    IEC104_QOS_QL_RES49 = 49,
    IEC104_QOS_QL_RES50 = 50,
    IEC104_QOS_QL_RES51 = 51,
    IEC104_QOS_QL_RES52 = 52,
    IEC104_QOS_QL_RES53 = 53,
    IEC104_QOS_QL_RES54 = 54,
    IEC104_QOS_QL_RES55 = 55,
    IEC104_QOS_QL_RES56 = 56,
    IEC104_QOS_QL_RES57 = 57,
    IEC104_QOS_QL_RES58 = 58,
    IEC104_QOS_QL_RES59 = 59,
    IEC104_QOS_QL_RES60 = 60,
    IEC104_QOS_QL_RES61 = 61,
    IEC104_QOS_QL_RES62 = 62,
    IEC104_QOS_QL_RES63 = 63,
    IEC104_QOS_QL_RES64 = 64,
    IEC104_QOS_QL_RES65 = 65,
    IEC104_QOS_QL_RES66 = 66,
    IEC104_QOS_QL_RES67 = 67,
    IEC104_QOS_QL_RES68 = 68,
    IEC104_QOS_QL_RES69 = 69,
    IEC104_QOS_QL_RES70 = 70,
    IEC104_QOS_QL_RES71 = 71,
    IEC104_QOS_QL_RES72 = 72,
    IEC104_QOS_QL_RES73 = 73,
    IEC104_QOS_QL_RES74 = 74,
    IEC104_QOS_QL_RES75 = 75,
    IEC104_QOS_QL_RES76 = 76,
    IEC104_QOS_QL_RES77 = 77,
    IEC104_QOS_QL_RES78 = 78,
    IEC104_QOS_QL_RES79 = 79,
    IEC104_QOS_QL_RES80 = 80,
    IEC104_QOS_QL_RES81 = 81,
    IEC104_QOS_QL_RES82 = 82,
    IEC104_QOS_QL_RES83 = 83,
    IEC104_QOS_QL_RES84 = 84,
    IEC104_QOS_QL_RES85 = 85,
    IEC104_QOS_QL_RES86 = 86,
    IEC104_QOS_QL_RES87 = 87,
    IEC104_QOS_QL_RES88 = 88,
    IEC104_QOS_QL_RES89 = 89,
    IEC104_QOS_QL_RES90 = 90,
    IEC104_QOS_QL_RES91 = 91,
    IEC104_QOS_QL_RES92 = 92,
    IEC104_QOS_QL_RES93 = 93,
    IEC104_QOS_QL_RES94 = 94,
    IEC104_QOS_QL_RES95 = 95,
    IEC104_QOS_QL_RES96 = 96,
    IEC104_QOS_QL_RES97 = 97,
    IEC104_QOS_QL_RES98 = 98,
    IEC104_QOS_QL_RES99 = 99,
    IEC104_QOS_QL_RES100 = 100,
    IEC104_QOS_QL_RES101 = 101,
    IEC104_QOS_QL_RES102 = 102,
    IEC104_QOS_QL_RES103 = 103,
    IEC104_QOS_QL_RES104 = 104,
    IEC104_QOS_QL_RES105 = 105,
    IEC104_QOS_QL_RES106 = 106,
    IEC104_QOS_QL_RES107 = 107,
    IEC104_QOS_QL_RES108 = 108,
    IEC104_QOS_QL_RES109 = 109,
    IEC104_QOS_QL_RES110 = 110,
    IEC104_QOS_QL_RES111 = 111,
    IEC104_QOS_QL_RES112 = 112,
    IEC104_QOS_QL_RES113 = 113,
    IEC104_QOS_QL_RES114 = 114,
    IEC104_QOS_QL_RES115 = 115,
    IEC104_QOS_QL_RES116 = 116,
    IEC104_QOS_QL_RES117 = 117,
    IEC104_QOS_QL_RES118 = 118,
    IEC104_QOS_QL_RES119 = 119,
    IEC104_QOS_QL_RES120 = 120,
    IEC104_QOS_QL_RES121 = 121,
    IEC104_QOS_QL_RES122 = 122,
    IEC104_QOS_QL_RES123 = 123,
    IEC104_QOS_QL_RES124 = 124,
    IEC104_QOS_QL_RES125 = 125,
    IEC104_QOS_QL_RES126 = 126,
    IEC104_QOS_QL_RES127 = 127,
};

#endif

iec104_trace.h

Path = src/service_inspectors/iec104/iec104_trace.h

#ifndef IEC104_TRACE_H
#define IEC104_TRACE_H

// Detection trace utility

#include "main/snort_types.h"
#include "main/thread.h"

namespace snort
{
struct Packet;
class Trace;
}

extern THREAD_LOCAL const snort::Trace* iec104_trace;

enum
{
    TRACE_IEC104_IDENTIFICATION = 0,
};

#ifdef DEBUG_MSGS
#define print_debug_information(p, msg) debug_log(iec104_trace, TRACE_IEC104_IDENTIFICATION, p, msg)
#else
#define print_debug_information(...)
#endif

#endif

service_inspectors/imap/

This directory contains all files related to IMAP protocol processing. The protocol aware flushing for IMAP determines IMAP PDU and this reassembled IMAP PDU is processed by the IMAP inspector. Both IMAP requests/responses are parsed and the MIME attachments in IMAP responses are processed using the file API. The file API extracts and decodes the attachments. file_data is then set to the start of these extracted/decoded attachments. This inspector also identifies and whitelists the IMAPS traffic.

IMAP inspector uses PDFJSNorm class to extract and normalize JavaScript in PDF files. The normalized JavaScript is then available in the js_data buffer. The js_data buffer follows the JIT approach, thus, to perform the normalization, the rule with the js_data IPS option must be present as well as the js_norm module is configured.

imap_config.h

Path = src/service_inspectors/imap/imap_config.h

#ifndef IMAP_CONFIG_H
#define IMAP_CONFIG_H

// Configuration for Imap service inspector

#include "mime/file_mime_process.h"

struct IMAP_PROTO_CONF
{
    snort::DecodeConfig decode_conf;
    snort::MailLogConfig log_config;
};

struct ImapStats
{
    PegCount packets;
    PegCount sessions;
    PegCount concurrent_sessions;
    PegCount max_concurrent_sessions;
    PegCount start_tls;
    PegCount ssl_search_abandoned;
    PegCount ssl_srch_abandoned_early;
    snort::MimeStats mime_stats;
    PegCount js_pdf_scripts;
};

extern const PegInfo imap_peg_names[];
extern THREAD_LOCAL ImapStats imapstats;

#endif

imap.h

Path = src/service_inspectors/imap/imap.h

#ifndef IMAP_H
#define IMAP_H

// Implementation header with definitions, datatypes and flowdata class for
// IMAP service inspector.

#include "flow/flow.h"
#include "mime/file_mime_process.h"

// Direction packet is coming from, if we can figure it out
#define IMAP_PKT_FROM_UNKNOWN  0
#define IMAP_PKT_FROM_CLIENT   1
#define IMAP_PKT_FROM_SERVER   2

#define STATE_DATA             0    // Data state
#define STATE_TLS_CLIENT_PEND  1    // Got STARTTLS
#define STATE_TLS_SERVER_PEND  2    // Got STARTTLS
#define STATE_TLS_DATA         3    // Successful handshake, TLS encrypted data
#define STATE_COMMAND          4
#define STATE_UNKNOWN          5
#define STATE_DECRYPTION_REQ   6

// session flags
#define IMAP_FLAG_NEXT_STATE_UNKNOWN         0x00000004
#define IMAP_FLAG_GOT_NON_REBUILT            0x00000008
#define IMAP_FLAG_CHECK_SSL                  0x00000010
#define IMAP_FLAG_ABANDON_EVT                0x00000020

typedef enum _IMAPCmdEnum
{
    CMD_APPEND = 0,
    CMD_AUTHENTICATE,
    CMD_CAPABILITY,
    CMD_CHECK,
    CMD_CLOSE,
    CMD_COMPARATOR,
    CMD_COMPRESS,
    CMD_CONVERSIONS,
    CMD_COPY,
    CMD_CREATE,
    CMD_DELETE,
    CMD_DELETEACL,
    CMD_DONE,
    CMD_EXAMINE,
    CMD_EXPUNGE,
    CMD_FETCH,
    CMD_GETACL,
    CMD_GETMETADATA,
    CMD_GETQUOTA,
    CMD_GETQUOTAROOT,
    CMD_IDLE,
    CMD_LIST,
    CMD_LISTRIGHTS,
    CMD_LOGIN,
    CMD_LOGOUT,
    CMD_LSUB,
    CMD_MYRIGHTS,
    CMD_NOOP,
    CMD_NOTIFY,
    CMD_RENAME,
    CMD_SEARCH,
    CMD_SELECT,
    CMD_SETACL,
    CMD_SETMETADATA,
    CMD_SETQUOTA,
    CMD_SORT,
    CMD_STARTTLS,
    CMD_STATUS,
    CMD_STORE,
    CMD_SUBSCRIBE,
    CMD_THREAD,
    CMD_UID,
    CMD_UNSELECT,
    CMD_UNSUBSCRIBE,
    CMD_X,
    CMD_LAST
} IMAPCmdEnum;

typedef enum _IMAPRespEnum
{
    RESP_CAPABILITY = 0,
    RESP_LIST,
    RESP_LSUB,
    RESP_STATUS,
    RESP_SEARCH,
    RESP_FLAGS,
    RESP_EXISTS,
    RESP_RECENT,
    RESP_EXPUNGE,
    RESP_FETCH,
    RESP_BAD,
    RESP_BYE,
    RESP_NO,
    RESP_OK,
    RESP_PREAUTH,
    RESP_ENVELOPE,
    RESP_UID,
    RESP_LAST
} IMAPRespEnum;

typedef enum _IMAPHdrEnum
{
    HDR_CONTENT_TYPE = 0,
    HDR_CONT_TRANS_ENC,
    HDR_CONT_DISP,
    HDR_LAST
} IMAPHdrEnum;

struct IMAPSearch
{
    const char* name;
    int name_len;
};

struct IMAPToken
{
    const char* name;
    int name_len;
    int search_id;
};

struct IMAPSearchInfo
{
    int id;
    int index;
    int length;
};

class ImapMime : public snort::MimeSession
{
    using snort::MimeSession::MimeSession;
private:
    void decode_alert() override;
    void decompress_alert() override;
    void reset_state(snort::Flow* ssn) override;
    bool is_end_of_data(snort::Flow* ssn) override;
};

namespace snort
{
class PDFJSNorm;
}

struct IMAPData
{
    int state;
    int state_flags;
    int session_flags;
    uint32_t body_len;
    uint32_t body_read;
    ImapMime* mime_ssn;
    snort::PDFJSNorm* jsn;
};

class ImapFlowData : public snort::FlowData
{
public:
    ImapFlowData();
    ~ImapFlowData() override;

    static void init()
    { inspector_id = snort::FlowData::create_flow_data_id(); }

public:
    static unsigned inspector_id;
    IMAPData session;
};

#endif

imap_module.h

Path = src/service_inspectors/imap/imap_module.h

#ifndef IMAP_MODULE_H
#define IMAP_MODULE_H

// Interface to the IMAP service inspector

#include "framework/module.h"
#include "imap_config.h"

#define GID_IMAP 141

#define IMAP_UNKNOWN_CMD            1
#define IMAP_UNKNOWN_RESP           2
#define IMAP_B64_DECODING_FAILED    4
#define IMAP_QP_DECODING_FAILED     5
#define IMAP_UU_DECODING_FAILED     7
#define IMAP_FILE_DECOMP_FAILED     8

#define IMAP_NAME "imap"
#define IMAP_HELP "imap inspection"

namespace snort
{
struct SnortConfig;
}

extern THREAD_LOCAL snort::ProfileStats imapPerfStats;

class ImapModule : public snort::Module
{
public:
    ImapModule();
    ~ImapModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    unsigned get_gid() const override
    { return GID_IMAP; }

    const snort::RuleMap* get_rules() const override;
    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile() const override;

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

    IMAP_PROTO_CONF* get_data();

private:
    IMAP_PROTO_CONF* config;
};

#endif

imap_paf.h

Path = src/service_inspectors/imap/imap_paf.h

#ifndef IMAP_PAF_H
#define IMAP_PAF_H

// Protocol aware flushing for IMAP

#include "mime/file_mime_paf.h"
#include "stream/stream_splitter.h"

struct ImapDataInfo
{
    int paren_cnt;            // The open parentheses count in fetch
    const char* next_letter;  // The current command in fetch
    bool found_len;
    uint32_t length;
    bool esc_nxt_char;        // true if the next character has been escaped
};

// States for IMAP PAF
typedef enum _ImapPafState
{
    IMAP_PAF_REG_STATE,           // default state. eat until LF
    IMAP_PAF_DATA_HEAD_STATE,     // parses the fetch header
    IMAP_PAF_DATA_LEN_STATE,      // parse the literal length
    IMAP_PAF_DATA_STATE,          // search for and flush on MIME boundaries
    IMAP_PAF_FLUSH_STATE,         // flush if a termination sequence is found
    IMAP_PAF_CMD_IDENTIFIER,      // determine the line identifier ('+', '*', tag)
    IMAP_PAF_CMD_TAG,             // currently analyzing tag . identifier
    IMAP_PAF_CMD_STATUS,          // currently parsing second argument
    IMAP_PAF_CMD_SEARCH           // currently searching data for a command
} ImapPafState;

typedef enum _ImapDataEnd
{
    IMAP_PAF_DATA_END_UNKNOWN,
    IMAP_PAF_DATA_END_PAREN
} ImapDataEnd;

// State tracker for IMAP PAF
struct ImapPafData
{
    MimeDataPafInfo mime_info;    // Mime response information
    ImapPafState imap_state;      // The current IMAP paf stat
    ImapDataInfo imap_data_info;  // Used for parsing data
    ImapDataEnd data_end_state;
    bool end_of_data;
};

class ImapSplitter : public snort::StreamSplitter
{
public:
    ImapSplitter(bool c2s);

    Status scan(snort::Packet*, const uint8_t* data, uint32_t len,
        uint32_t flags, uint32_t* fp) override;

    bool is_paf() override { return true; }

public:
    ImapPafData state;
};

// Function: Check if IMAP data end is reached
bool imap_is_data_end(snort::Flow* ssn);

#endif

service_inspectors/mms/

  • MMS service inspector overview * IEC 61850 is a family of protocols distributed by the International Electrotechnical Commission (IEC) that provide a standardized method of sending service messages between various manufacturing and process control devices, typically running on TCP port 102.

It is used in combination with various parts of the OSI model, most notably the TPKT, COTP, Session, Presentation, and ACSE layers, to provide reliable transport via TCP/IP.

The MMS inspector decodes the OSI layers encapsulating the MMS protocol and provides rule options to access certain protocol fields and data content. This allows the user to write rules for MMS messages without decoding the protocol.

  • Encapsulation overview * MMS can be transported via TCP/102 when encapsulated in TPKT and the associated OSI layers. The two structures most commonly encountered are shown below:

Connection Request/Response: TCP → TPKT → COTP → OSI Session → OSI Presentation → OSI ACSE → MMS

Confirmed Request/Response: TCP → TPKT → COTP → OSI Session → OSI Presentation → MMS

Parsing functions for each of these layers can be found in util_tpkt.

  • MMS message pipelining overview * Multiple MMS messages can be sent within a single TCP stream through use of a variety of techniques. Splitting for known forms is handled as follows:

Case 1 Combined: In this case multiple messages are contained within one TCP packet. This can occur at any of the following layers: TPKT, OSI Presentation, MMS, or any combination thereof. Example layouts are shown below:

Combined at the TPKT layer: TCP → TPKT → COTP → OSI Session → OSI Presentation → MMS → TPKT → COTP → OSI Session → OSI Presentation → MMS

Combined at the OSI Presentation layer: TCP → TPKT → COTP → OSI Session → OSI Presentation → MMS → OSI Presentation → MMS

Combined at the MMS layer: TCP → TPKT → COTP → OSI Session → OSI Presentation → MMS → MMS

Due to the ability for messages to be combined at multiple layers, the starting layer of the cursor following a successful split cannot be assumed. To solve this, a set of functions have been developed in util_tpkt to make a best guess at the layer at the cursor start (isTpkt, isCotp, etc.). An example of usage can be found in the MMS Splitter. No special considerations were needed in the curse to handle this case.

Case 2 Split: In this case a single message is split across multiple TCP packets within the same stream. This can occur at any point across the packet data, not necessarily on a layer boundary. Due to this case it is necessary to buffer the incoming packet data until a flush point has been found. To handle this, the both the MMS curse and splitter need to have special considerations.

In the MMS curse, current and prior state are tracked as the potential message is processed byte by byte. The current state indicates the part of a fully encapsulated MMS message that the byte in that position would represent if it was found to contain MMS. The prior state is tracked so that the state machine knows where to pick up in the event of a message being split.

In the MMS splitter, two buffers (one for each direction) are used to track message data across packets. At the beginning of each scan, the direction is determined and the appropriate buffer is selected. The new packet data is then appended to the selected buffer. This buffer, not the current packet data, is then searched for a MMS message through use of various functions exposed in util_tpkt.

Case 3 Combined and Split: In this case, the above two techniques are combined to create a stream where one packet contains a complete MMS message as well as the start of a second, and then the packets that follow contain the remainder of the second MMS message. No additional considerations beyond those implemented for the prior two techniques were needed.

  • MMS curse * In the MMS curse, current and prior state are tracked as the potential message is processed byte by byte. The current state indicates the part of a fully encapsulated MMS message that the byte in that position would represent if it was found to contain MMS. The prior state is tracked so that the state machine knows where to pick up in the event of a message being split. Two special states exist which are used to assist in determining when to stop processing: MMS_STATEMMS and MMS_STATENOT_FOUND.

During processing only states that are required to be specific values are enforced, leaving more specific enforcement to the MMS inspector itself. This choice was made to help avoid false negatives caused by states that would not affect the inspector’s parsing. For example, the TPKT version is expected to be the value 0x03, however this does not make any material difference at this time to how the message is parsed so it is not enforced. By contrast, the COTP PDU value is enforced to be of type DT_DATA as other values indicate that MMS is most likely not in use.

If all of the encapsulation layer checks pass, a best guess is made to determine if the payload is a MMS message. This is done by searching for any of the known MMS message types. Unfortunately, this approach has the potential for some false positives to slip into the MMS inspector processing as the message type indicators are only one byte in size. This was deemed preferred as it is enough to kick out the majority of non-MMS traffic while not needing to run the more intensive parsing that occurs in the inspector.

mms_decode.h

Path = src/service_inspectors/mms/mms_decode.h

#ifndef MMS_DECODE_H
#define MMS_DECODE_H

// MMS Decode provides the final processing to determine whether or not the
// message being processed is of a known type

#include "framework/cursor.h"

namespace snort
{
struct Packet;
}

class MmsFlowData;

/* Need at least 3 bytes to cover one TLV group */
#define MMS_MIN_LEN    3

bool mms_decode(snort::Packet*, MmsFlowData* mmsfd);

#endif

mms.h

Path = src/service_inspectors/mms/mms.h

#ifndef MMS_H
#define MMS_H

// MMS adds a new service inspector designed to process Manufacturing
// Message Specification (MMS) traffic defined within the IEC-61850 family
// of protocols

#include "flow/flow.h"
#include "framework/counts.h"

struct MmsStats
{
    PegCount sessions;
    PegCount frames;
    PegCount concurrent_sessions;
    PegCount max_concurrent_sessions;
};

class MmsSessionData
{
public:
    uint32_t offset = 0;
    bool mms_found  = false;

    void session_data_reset()
    {
        offset    = 0;
        mms_found = false;
    }
};

class MmsFlowData : public snort::FlowData
{
public:
    MmsFlowData();
    ~MmsFlowData() override;

    static void init();

    void reset()
    {
        ssn_data.session_data_reset();
    }

    void set_mms_offset(uint32_t offset)
    {
        ssn_data.mms_found = true;
        ssn_data.offset    = offset;
    }

    uint32_t get_mms_offset()
    {
        return ssn_data.offset;
    }

    bool is_mms_found()
    {
        return ssn_data.mms_found;
    }

public:
    static unsigned inspector_id;
    MmsSessionData ssn_data;
};

uint32_t get_message_type(uint32_t version, const char* name);
uint32_t get_info_type(uint32_t version, const char* name);

enum MmsMsgType
{
    MMS_MSG__PDU_NOT_SET        = 0x00,
    MMS_MSG__CONFIRMED_REQUEST  = 0xA0,
    MMS_MSG__CONFIRMED_RESPONSE = 0xA1,
    MMS_MSG__CONFIRMED_ERROR    = 0xA2,
    MMS_MSG__UNCONFIRMED        = 0xA3,
    MMS_MSG__REJECT             = 0xA4,
    MMS_MSG__CANCEL_REQUEST     = 0x85,
    MMS_MSG__CANCEL_RESPONSE    = 0x86,
    MMS_MSG__CANCEL_ERROR       = 0xA7,
    MMS_MSG__INITIATE_REQUEST   = 0xA8,
    MMS_MSG__INITIATE_RESPONSE  = 0xA9,
    MMS_MSG__INITIATE_ERROR     = 0xAA,
    MMS_MSG__CONCLUDE_REQUEST   = 0x8B,
    MMS_MSG__CONCLUDE_RESPONSE  = 0x8C,
    MMS_MSG__CONCLUDE_ERROR     = 0xAD,
};

extern THREAD_LOCAL MmsStats mms_stats;

#endif

mms_module.h

Path = src/service_inspectors/mms/mms_module.h

#ifndef MMS_MODULE_H
#define MMS_MODULE_H

// MMS Module defines everything needed to integrate the MMS service
// inspector into the framework

#include "framework/module.h"

#define GID_MMS     152

#define MMS_NAME    "mms"
#define MMS_HELP    "mms inspection"

extern THREAD_LOCAL snort::ProfileStats mms_prof;

class MmsModule : public snort::Module
{
public:
    MmsModule();

    unsigned get_gid() const override
    {
        return GID_MMS;
    }

    const snort::RuleMap* get_rules() const override;

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;

    snort::ProfileStats* get_profile() const override
    {
        return &mms_prof;
    }

    Usage get_usage() const override
    {
        return INSPECT;
    }

    bool is_bindable() const override
    {
        return true;
    }
};

#endif

mms_splitter.h

Path = src/service_inspectors/mms/mms_splitter.h

#ifndef MMS_SPLITTER__H
#define MMS_SPLITTER__H

// MMS Splitter provides a means of normalizing MMS traffic. It does so by
// splitting apart or reassembling pipelined messages as necessary, and then
// parses and strips off the encapsulation layers leaving only the MMS
// message itself for analysis.

#include "stream/stream_splitter.h"

struct CurPacketContext
{
    uint8_t* data;
    uint32_t len;
    uint32_t start_offset;
    uint32_t splitter_offset;
    uint32_t exit_offset;
};

class MmsSplitter : public snort::StreamSplitter
{
public:
    MmsSplitter(bool);

    Status scan(snort::Packet*, const uint8_t* data, uint32_t len, uint32_t flags,
        uint32_t* fp) override;

    bool is_paf() override { return true; }
};

#endif

util_tpkt.h

Path = src/service_inspectors/mms/util_tpkt.h

#ifndef TPKT_H
#define TPKT_H

// The TPKT util provides an interface for processing the encapsulation
// layers associated with TPKT, COTP, OSI Session, OSI Pres, and OSI ACSE

#include "detection/detection_engine.h"
#include "flow/flow.h"
#include "framework/counts.h"
#include "framework/cursor.h"
#include "protocols/packet.h"
#include "service_inspectors/mms/mms.h"
#include "utils/util_ber.h"

namespace snort
{
struct Packet;
}

#define TPKT_PACKET_DATA_BUF_SIZE    0x10000
#define TPKT_SMALLEST_TLV_SIZE       0x03

//-------------------------------------------------------------------------
// type definitions
//-------------------------------------------------------------------------

enum OsiSessionSpduType
{
    OSI_SESSION_SPDU__GT_DT   = 0x01,
    OSI_SESSION_SPDU__CN      = 0x0D,
    OSI_SESSION_SPDU__AC      = 0x0E,
    OSI_SESSION_SPDU__NOT_SET = 0xFFFFFFFF,
};

enum ProcessAsDataTransferType
{
    OSI_SESSION_PROCESS_AS_DT__TRUE  = true,
    OSI_SESSION_PROCESS_AS_DT__FALSE = false,
};

enum TpktEncapLayerType
{
    TPKT_ENCAP_LAYER__NONE,
    TPKT_ENCAP_LAYER__TPKT,
    TPKT_ENCAP_LAYER__COTP,
    TPKT_ENCAP_LAYER__OSI_SESSION,
    TPKT_ENCAP_LAYER__OSI_PRES,
    TPKT_ENCAP_LAYER__OSI_ACSE,
    TPKT_ENCAP_LAYER__MMS,
    TPKT_ENCAP_LAYER__PARTIAL,
};

enum TpktEncapLayerSearchStateType
{
    TPKT_ENCAP_LAYER_SEARCH_STATE__EXIT,
    TPKT_ENCAP_LAYER_SEARCH_STATE__FOUND,
    TPKT_ENCAP_LAYER_SEARCH_STATE__PARTIAL,
};

enum TpktAppliSearchStateType
{
    TPKT_APPLI_SEARCH_STATE__MMS_FOUND,
    TPKT_APPLI_SEARCH_STATE__SEARCH,
    TPKT_APPLI_SEARCH_STATE__EXIT,
};

enum TpktPacketDataDirectionType
{
    TPKT_PACKET_DATA_DIRECTION__SERVER,
    TPKT_PACKET_DATA_DIRECTION__CLIENT,
};

//-------------------------------------------------------------------------
// flow stuff
//-------------------------------------------------------------------------

class TpktSessionData
{
public:
    OsiSessionSpduType cur_spdu_type = OSI_SESSION_SPDU__NOT_SET;
    bool process_as_data_transfer    = OSI_SESSION_PROCESS_AS_DT__FALSE;
    uint8_t* server_packet_data      = nullptr;
    uint32_t server_packet_len       = 0;
    uint32_t server_start_offset     = 0;
    uint32_t server_splitter_offset  = 0;
    uint32_t server_exit_offset      = 0;
    uint8_t* client_packet_data      = nullptr;
    uint32_t client_packet_len       = 0;
    uint32_t client_start_offset     = 0;
    uint32_t client_splitter_offset  = 0;
    uint32_t client_exit_offset      = 0;

    void packet_data_reset(TpktPacketDataDirectionType direction)
    {
        if (direction == TPKT_PACKET_DATA_DIRECTION__SERVER)
        {
            delete [] server_packet_data;
            server_packet_data = new uint8_t[TPKT_PACKET_DATA_BUF_SIZE];
        }
        else if (direction == TPKT_PACKET_DATA_DIRECTION__CLIENT)
        {
            delete [] client_packet_data;
            client_packet_data = new uint8_t[TPKT_PACKET_DATA_BUF_SIZE];
        }
    }

    void session_data_reset()
    {
        cur_spdu_type            = OSI_SESSION_SPDU__NOT_SET;
        process_as_data_transfer = OSI_SESSION_PROCESS_AS_DT__FALSE;

        packet_data_reset(TPKT_PACKET_DATA_DIRECTION__SERVER);
        server_packet_len      = 0;
        server_start_offset    = 0;
        server_splitter_offset = 0;
        server_exit_offset     = 0;

        packet_data_reset(TPKT_PACKET_DATA_DIRECTION__CLIENT);
        client_packet_len      = 0;
        client_start_offset    = 0;
        client_splitter_offset = 0;
        client_exit_offset     = 0;
    }
};

class TpktFlowData : public snort::FlowData
{
public:
    TpktFlowData();
    ~TpktFlowData() override;

    static void init();

    void reset()
    {
        ssn_data.session_data_reset();
    }

    void reset_packet_data(TpktPacketDataDirectionType direction)
    {
        ssn_data.packet_data_reset(direction);
    }

    size_t size_of() override
    {
        return sizeof(*this);
    }

public:
    static unsigned inspector_id;
    TpktSessionData ssn_data;
};

//-------------------------------------------------------------------------
// function definitions
//-------------------------------------------------------------------------

TpktEncapLayerType get_next_tpkt_encap_layer(snort::Packet*, Cursor*);
TpktAppliSearchStateType tpkt_search_from_tpkt_layer(Cursor*);
TpktAppliSearchStateType tpkt_search_from_cotp_layer(Cursor*);
TpktAppliSearchStateType tpkt_search_from_osi_session_layer(Cursor*, bool);
TpktAppliSearchStateType tpkt_search_from_osi_pres_layer(Cursor*);
TpktAppliSearchStateType tpkt_search_from_osi_acse_layer(Cursor*);

#endif

service_inspectors/mms/tpkt/

cotp_decode.h

Path = src/service_inspectors/mms/tpkt/cotp_decode.h

#ifndef COTP_DECODE__H
#define COTP_DECODE__H

// COTP Decode provides an interface for COTP message identification
// and parsing.
// It is intended to be used through `util_tpkt`, not directly

#include "../util_tpkt.h"

// COTP PDU Types
// other types exist, but at this time DT_DATA is the only one in use
enum CotpPduType
{
    COTP_PDU_TYPE_DT_DATA = 0x0F,
};

struct CotpHdr
{
    uint8_t length;
    uint8_t dest_ref       : 4;
    uint8_t pdu_type       : 4;
    uint8_t tpdu_num       : 7;
    uint8_t last_data_unit : 1;
}
__attribute__((packed));

TpktAppliSearchStateType tpkt_internal_search_from_cotp_layer(Cursor*);

#endif

osi_acse_decode.h

Path = src/service_inspectors/mms/tpkt/osi_acse_decode.h

#ifndef OSI_ACSE_DECODE__H
#define OSI_ACSE_DECODE__H

// OSI ACSE Decode provides an interface for ACSE message identification
// and parsing.
// It is intended to be used through `util_tpkt`, not directly

#include "../util_tpkt.h"

enum
{
    OSI_ACSE_AARQ = 0x60,
    OSI_ACSE_AARE = 0x61,
};

// function stubs
TpktAppliSearchStateType tpkt_internal_search_from_osi_acse_layer(Cursor*);

#endif

osi_pres_decode.h

Path = src/service_inspectors/mms/tpkt/osi_pres_decode.h

#ifndef OSI_PRES_DECODE__H
#define OSI_PRES_DECODE__H

// OSI PRES Decode provides an interface for PRES message identification
// and parsing.
// It is intended to be used through `util_tpkt`, not directly

#include "../util_tpkt.h"

// named identifiers for ber tag processing
enum BerTagProcessType
{
    BER_TAG__ADD_NONE               = 0x00,
    BER_TAG__ADD_HEADER_ONLY        = 0x01,
    BER_TAG__ADD_HEADER_AND_PAYLOAD = 0x02,
};

// OSI Session SPDU Types
enum OsiPresPpduModeSelectorType
{
    OSI_PRES_MODE__INVALID = 0x00,
    OSI_PRES_MODE__NORMAL  = 0x01,
};

// OSI Presentation Context Identifiers
enum OsiPresContextIdentifierType
{
    OSI_PRES_CONTEXT_ID__NULL = 0x00,
    OSI_PRES_CONTEXT_ID__ACSE = 0x01,
    OSI_PRES_CONTEXT_ID__MMS  = 0x03,
};

TpktAppliSearchStateType tpkt_internal_search_from_osi_pres_layer(Cursor*);

bool process_next_ber_tag(snort::BerReader*, snort::BerElement*, Cursor*, BerTagProcessType);

#endif

osi_session_decode.h

Path = src/service_inspectors/mms/tpkt/osi_session_decode.h

#ifndef OSI_SESSION_DECODE__H
#define OSI_SESSION_DECODE__H

// OSI Session Decode provides an interface for Session message
// identification and parsing.
// It is intended to be used through `util_tpkt`, not directly

#include "../util_tpkt.h"

#include "cotp_decode.h"

enum OsiSessionSpduParameterType
{
    OSI_SESSION_SPDU_PARAM__CN_ACCEPT_ITEM           = 0x05,
    OSI_SESSION_SPDU_PARAM__PROTOCOL_OPTIONS         = 0x13,
    OSI_SESSION_SPDU_PARAM__SESSION_REQUIREMENT      = 0x14,
    OSI_SESSION_SPDU_PARAM__VERSION_NUMBER           = 0x16,
    OSI_SESSION_SPDU_PARAM__CALLING_SESSION_SELECTOR = 0x33,
    OSI_SESSION_SPDU_PARAM__CALLED_SESSION_SELECTOR  = 0x34,
    OSI_SESSION_SPDU_PARAM__SESSION_USER_DATA        = 0xC1,
};

struct OsiSessionHdr
{
    uint8_t spdu_type;
    uint8_t length;
};

struct OsiSessionSpduParameterHdr
{
    uint8_t type;
    uint8_t length;
};

TpktAppliSearchStateType tpkt_internal_search_from_osi_session_layer(Cursor*, bool);
uint32_t get_max_loops(Cursor*);

uint32_t get_max_pos(uint32_t, uint32_t);

#endif

tpkt_decode.h

Path = src/service_inspectors/mms/tpkt/tpkt_decode.h

#ifndef TPKT_DECODE_H
#define TPKT_DECODE_H

// TPKT Decode provides an interface for TPKT message identification
// and parsing.
// It is intended to be used through `util_tpkt`, not directly

#include "../util_tpkt.h"

namespace snort
{
struct Packet;
}

// set a minimum length for TPKT to ensure it only gets processed
// when it makes sense
// 0x03 bytes for TPKT
// 0x04 bytes for COTP
// 0x02 bytes for the smallest MMS message
#define TPKT_MIN_LEN    0x09

// defined as the maximum in the spec
#define TPKT_MAX_LEN    0xFFFB

struct TpktHdr
{
    uint8_t version;
    uint8_t reserved;
    uint16_t length;
};

TpktAppliSearchStateType tpkt_internal_search_from_tpkt_layer(Cursor*);

#endif

service_inspectors/modbus/

Modbus is a protocol used in SCADA networks which is typically on TCP port 502 (aka Modbus TCP).

The Modbus inspector decodes the Modbus protocol and provides rule options to access certain protocol fields. This allows a user to write rules for Modbus packets without decoding the protocol with a series of ”content” and ”byte test” options.

The inspector only evaluates PAF-flushed PDUs. If the rule options don’t check for this, they’ll fire on stale session data when the original packet goes through before flushing.

modbus_decode.h

Path = src/service_inspectors/modbus/modbus_decode.h

#ifndef MODBUS_DECODE_H
#define MODBUS_DECODE_H

namespace snort
{
struct Packet;
}

class ModbusFlowData;

/* Need 8 bytes for MBAP Header + Function Code */
#define MODBUS_MIN_LEN 8

bool ModbusDecode(snort::Packet*, ModbusFlowData* mfd);

#endif

modbus.h

Path = src/service_inspectors/modbus/modbus.h

#ifndef MODBUS_H
#define MODBUS_H

#include "flow/flow.h"
#include "framework/counts.h"

struct ModbusStats
{
    PegCount sessions;
    PegCount frames;
    PegCount concurrent_sessions;
    PegCount max_concurrent_sessions;
};

struct modbus_session_data_t
{
    uint16_t flags;
    uint8_t func;
    uint8_t unit;
};

class ModbusFlowData : public snort::FlowData
{
public:
    ModbusFlowData();
    ~ModbusFlowData() override;

    static void init();

    void reset()
    {
        ssn_data.func = ssn_data.unit = 0;
        ssn_data.flags = 0;
    }

public:
    static unsigned inspector_id;
    modbus_session_data_t ssn_data;
};

int get_message_type(int version, const char* name);
int get_info_type(int version, const char* name);

extern THREAD_LOCAL ModbusStats modbus_stats;

#endif

modbus_module.h

Path = src/service_inspectors/modbus/modbus_module.h

#ifndef MODUBS_MODULE_H
#define MODUBS_MODULE_H

#include "framework/module.h"

#define GID_MODBUS 144

#define MODBUS_BAD_LENGTH        1
#define MODBUS_BAD_PROTO_ID      2
#define MODBUS_RESERVED_FUNCTION 3

#define MODBUS_NAME "modbus"
#define MODBUS_HELP "modbus inspection"

extern THREAD_LOCAL snort::ProfileStats modbus_prof;

class ModbusModule : public snort::Module
{
public:
    ModbusModule();

    unsigned get_gid() const override
    { return GID_MODBUS; }

    const snort::RuleMap* get_rules() const override;

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;

    snort::ProfileStats* get_profile() const override
    { return &modbus_prof; }

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }
};

#endif

modbus_paf.h

Path = src/service_inspectors/modbus/modbus_paf.h

#ifndef MODBUS_PAF__H
#define MODBUS_PAF__H

// Protocol-Aware Flushing (PAF) code for the Modbus inspector.

#include "stream/stream_splitter.h"

enum modbus_paf_state_t
{
    MODBUS_PAF_STATE__TRANS_ID_1,
    MODBUS_PAF_STATE__TRANS_ID_2,
    MODBUS_PAF_STATE__PROTO_ID_1,
    MODBUS_PAF_STATE__PROTO_ID_2,
    MODBUS_PAF_STATE__LENGTH_1,
    MODBUS_PAF_STATE__LENGTH_2,
    MODBUS_PAF_STATE__SET_FLUSH
};

class ModbusSplitter : public snort::StreamSplitter
{
public:
    ModbusSplitter(bool);

    Status scan(snort::Packet*, const uint8_t* data, uint32_t len, uint32_t flags,
        uint32_t* fp) override;

    bool is_paf() override { return true; }

private:
    modbus_paf_state_t state;
    uint16_t modbus_length;
};

#endif

service_inspectors/netflow/

The NetFlow inspector inspects Cisco NetFlow version 5 and 9 traffic. When the device running Snort3 is placed between a NetFlow collector and exporter, this allows Snort to generate RNA events based on exported NetFlow flows.

By means of a template and a record cache, Snort3 keeps track of new and existing flows, as well as their associated services - similar to host_cache. Events generated follow the same format as their corresponding RNA events, such as new_host, new_network_proto, etc.

Note that these caches are currently thread-local for performance reasons, so the occasional duplicate event is expected.

netflow_cache.h

Path = src/service_inspectors/netflow/netflow_cache.h

#ifndef NETFLOW_CACHE_H
#define NETFLOW_CACHE_H

#include <cassert>

#include "hash/lru_cache_local.h"
#include "host_tracker/cache_allocator.h"
#include "sfip/sf_ip.h"

#include "netflow_headers.h"
#include "netflow_module.h"
#include "netflow_record.h"

// Trivial derived allocator, pointing to their own cache. LruCacheAllocNetFlow has a
// CacheInterface* pointing to an lru cache. We can create different cache types by
// instantiating the lru cache using different keys and derive here allocators with
// CacheInterface* pointing to the appropriate lru cache object.
template <class T>
class LruCacheAllocNetFlow : public CacheAlloc<T>
{
public:
    // This needs to be in every derived class:
    template <class U>
    struct rebind
    {
        typedef LruCacheAllocNetFlow<U> other;
    };

    using CacheAlloc<T>::lru;
    LruCacheAllocNetFlow();
};

template<typename Key, typename Value, typename Hash>
class LruCacheLocalNetFlow : public LruCacheLocal<Key, Value, Hash>, public CacheInterface
{
public:
    using LruLocal = LruCacheLocal<Key, Value, Hash>;
    using LruLocal::current_size;
    using LruLocal::max_size;
    using LruLocal::list;

    LruCacheLocalNetFlow(const size_t sz, struct LruCacheLocalStats& st) : LruLocal(sz, st) {}

    template <class T>
    friend class LruCacheAllocNetFlow;

private:
    // Only the allocator calls this
    void update(int size) override
    {
        if ( size < 0 )
            assert( current_size >= (size_t) -size);

        // Checking 1+ size prevents crash if max_size is too low to hold even a single entry
        if ( (current_size += size) > max_size and list.size() > 1 )
            LruLocal::prune();
    }
};

template <class T>
class LruCacheAllocTemplate : public CacheAlloc<T>
{
public:
    template <class U>
    struct rebind
    {
        typedef LruCacheAllocTemplate<U> other;
    };

    using CacheAlloc<T>::lru;
    LruCacheAllocTemplate();
};

template<typename Key, typename Value, typename Hash>
class LruCacheLocalTemplate : public LruCacheLocal<Key, Value, Hash>, public CacheInterface
{
public:
    using LruLocal = LruCacheLocal<Key, Value, Hash>;
    using LruLocal::current_size;
    using LruLocal::max_size;
    using LruLocal::stats;
    using LruLocal::list;

    LruCacheLocalTemplate(const size_t sz, struct LruCacheLocalStats& st) : LruLocal(sz, st)
    {}

    bool insert(const Key& key, std::vector<NetFlow9TemplateField>& tf)
    {
        bool is_new = false;
        Value& entry = LruLocal::find_else_create(key, &is_new);

        if ( !is_new )
        {
            stats.cache_replaces++;
            entry.clear();
        }

        for ( auto& elem : tf )
            entry.emplace_back(elem.field_type, elem.field_length);

        return is_new;
    }

    template <class T>
    friend class LruCacheAllocTemplate;

private:
    void update(int size) override
    {
        if ( size < 0 )
            assert( current_size >= (size_t) -size);

        if ( (current_size += size) > max_size and list.size() > 1 )
            LruLocal::prune();
    }
};

// Used to track record for unique IP; we assume NetFlow packets coming from
// a given NetFlow device will go to the same thread
typedef LruCacheLocalNetFlow<snort::SfIp, NetFlowSessionRecord, NetFlowHash> NetFlowCache;

// Used to track NetFlow version 9 Template fields
typedef std::pair<uint16_t, snort::SfIp> TemplateFieldKey;
typedef LruCacheAllocTemplate<NetFlow9TemplateField> TemplateAllocator;
typedef std::vector<NetFlow9TemplateField, TemplateAllocator> TemplateFieldValue;
typedef LruCacheLocalTemplate<TemplateFieldKey, TemplateFieldValue, TemplateIpHash> TemplateFieldCache;

#endif

netflow.h

Path = src/service_inspectors/netflow/netflow.h

#ifndef NETFLOW_H
#define NETFLOW_H

#include <unordered_map>
#include <vector>

#include "flow/flow_data.h"
#include "profiler/profiler.h"
#include "protocols/packet.h"
#include "utils/util.h"

#include "netflow_cache.cc"
#include "netflow_record.h"

THREAD_LOCAL NetFlowStats netflow_stats;
THREAD_LOCAL snort::ProfileStats netflow_perf_stats;

// Used to ensure we fully populate the record; can't rely on the actual values being zero
struct RecordStatus
{
    bool src = false;
    bool dst = false;
    bool first = false;
    bool last = false;
    bool src_tos = false;
    bool dst_tos = false;
    bool bytes_sent = false;
    bool packets_sent = false;
};

// temporary cache required to dump the output
typedef std::pair<snort::SfIp, NetFlowSessionRecord> IpRecord;
typedef std::vector<IpRecord> DumpCache;
static DumpCache* dump_cache = nullptr;

struct IpCompare
{
    bool operator()(const IpRecord& a, const IpRecord& b)
    { return a.first.less_than(b.first); }
};

static std::unordered_map<int, int>* udp_srv_map = nullptr;
static std::unordered_map<int, int>* tcp_srv_map = nullptr;

#endif

netflow_headers.h

Path = src/service_inspectors/netflow/netflow_headers.h

#ifndef NETFLOW_HEADERS_H
#define NETFLOW_HEADERS_H

#include "flow/flow.h"

#define NETFLOW_MIN_COUNT 1
#define NETFLOW_MAX_COUNT 256
#define MAX_TIME 2145916799

enum NetFlowFieldTypes : uint16_t
{
    NETFLOW_IN_BYTES = 1,
    NETFLOW_IN_PKTS = 2,
    NETFLOW_PROTOCOL = 4,
    NETFLOW_SRC_TOS = 5,
    NETFLOW_TCP_FLAGS = 6,
    NETFLOW_SRC_PORT = 7,
    NETFLOW_SRC_IP = 8,
    NETFLOW_SRC_MASK = 9,
    NETFLOW_SNMP_IN = 10,
    NETFLOW_DST_PORT = 11,
    NETFLOW_DST_IP = 12,
    NETFLOW_DST_MASK = 13,
    NETFLOW_SNMP_OUT = 14,
    NETFLOW_IPV4_NEXT_HOP = 15,
    NETFLOW_SRC_AS = 16,
    NETFLOW_DST_AS = 17,
    NETFLOW_LAST_PKT = 21,
    NETFLOW_FIRST_PKT = 22,
    NETFLOW_SRC_IPV6 = 27,
    NETFLOW_DST_IPV6 = 28,
    NETFLOW_SRC_MASK_IPV6 = 29,
    NETFLOW_DST_MASK_IPV6 = 30,
    NETFLOW_DST_TOS = 55,
};

struct NetFlow5Hdr
{
    uint16_t version;               // NetFlow export format version number
    uint16_t flow_count;            // Number of flows exported in this packet (1-30)
    uint32_t sys_uptime;            // Current time in milliseconds since the export device booted
    uint32_t unix_secs;             // Current count of seconds since 0000 UTC 1970
    uint32_t unix_nsecs;            // Residual nanoseconds since 0000 UTC 1970
    uint32_t flow_sequence;         // Sequence counter of total flows seen
    uint8_t engine_type;            // Type of flow-switching engine
    uint8_t engine_id;              // Slot number of the flow-switching engine
    uint16_t sampling_interval;     // First two bits hold the sampling mode; remaining 14 bits hold value of sampling interval
};

struct NetFlow5RecordHdr
{
    uint32_t flow_src_addr;         // Source IP address
    uint32_t flow_dst_addr;         // Destination IP address
    uint32_t next_hop_addr;         // IP address of next hop router
    uint16_t snmp_if_in;            // SNMP index of input interface
    uint16_t snmp_if_out;           // SNMP index of output interface
    uint32_t pkt_count;             // Packets in the flow
    uint32_t bytes_sent;            // Total number of Layer 3 bytes in the packets of the flow
    uint32_t flow_first;            // System uptime at start of flow
    uint32_t flow_last;             // System uptime at the time the last packet of the flow was received
    uint16_t src_port;              // TCP/UDP source port number or equivalent
    uint16_t dst_port;              // TCP/UDP destination port number or equivalent
    uint8_t pad1;                   // Unused (zero) bytes
    uint8_t tcp_flags;              // Cumulative OR of TCP flags
    uint8_t flow_protocol;          // IP protocol type (for example, TCP = 6; UDP = 17)
    uint8_t tos;                    // IP type of service
    uint16_t src_as;                // Autonomous system number of the source, either origin or peer
    uint16_t dst_as;                // Autonomous system number of the destination, either origin or peer
    uint8_t src_mask;               // Source address prefix mask bits
    uint8_t dst_mask;               // Destination address prefix mask bits
    uint16_t pad2;                  // Unused (zero) bytes
};

struct NetFlow9Hdr
{
    uint16_t version;               // The version of netflow records exported in this packet
    uint16_t flow_count;            // Number of FlowSet records (both template and data) contained within this packet
    uint32_t sys_uptime;            // Time in milliseconds since this device was first booted
    uint32_t unix_secs;             // Seconds since 0000 Coordinated Universal Time (UTC) 1970
    uint32_t sequence_num;          // Incremental sequence counter of all export packets sent by this export device
    uint32_t source_id;             // A 32-bit value that identifies the Exporter Observation Domain
};

struct NetFlow9FlowSet
{
    uint16_t field_id;
    uint16_t field_length;
};

struct NetFlow9Template
{
    uint16_t template_id;
    uint16_t template_field_count;
};

struct NetFlow9TemplateField
{
    uint16_t field_type;
    uint16_t field_length;

    NetFlow9TemplateField(uint16_t type, uint16_t length)
        : field_type(type)
        , field_length(length)
    {}
};
#endif

netflow_module.h

Path = src/service_inspectors/netflow/netflow_module.h

#ifndef NETFLOW_MODULE_H
#define NETFLOW_MODULE_H

#include <unordered_map>

#include "flow/flow_data.h"
#include "framework/module.h"
#include "hash/lru_cache_local.h"
#include "sfip/sf_cidr.h"
#include "utils/util.h"

#define NETFLOW_NAME "netflow"
#define NETFLOW_HELP "netflow inspection"

namespace snort
{
struct SnortConfig;
}

#define NETFLOW_MAX_ZONES 1024
#define NETFLOW_ANY_ZONE (-1)

//  Used to create hash of key for indexing into cache.
struct NetFlowHash
{
    size_t operator()(const snort::SfIp& ip) const
    {
        const uint64_t* ip64 = (const uint64_t*) ip.get_ip6_ptr();
        return std::hash<uint64_t>() (ip64[0]) ^
               std::hash<uint64_t>() (ip64[1]);
    }
};

// Generate hash of template id and device ip
struct TemplateIpHash
{
    size_t operator()(const std::pair<uint16_t, snort::SfIp> &ti) const
    {
        const uint64_t* ip64 = (const uint64_t*) ti.second.get_ip6_ptr();

        return std::hash<uint64_t>() (ip64[0]) ^
               std::hash<uint64_t>() (ip64[1]) ^
               std::hash<uint64_t>() ((uint64_t)ti.first);
    }
};

struct NetFlowRule
{
    NetFlowRule() { reset(); }
    void reset()
    {
        networks.clear();
        zones.clear();
        create_host = create_service = false;
    }

    bool filter_match(const snort::SfIp* ip, const int zone) const
    {
        if ( !zones.empty()
            and std::none_of(zones.cbegin(), zones.cend(),
                [zone](int z){ return z == NETFLOW_ANY_ZONE or z == zone; }))
            return false;

        return networks.empty()
            or std::any_of(networks.cbegin(), networks.cend(),
                [ip](const snort::SfCidr& net){ return SFIP_CONTAINS == net.contains(ip);});
    }

    std::vector <snort::SfCidr> networks;
    std::vector <int> zones;
    bool create_host = false;
    bool create_service = false;
};

using NetFlowRuleList = std::vector<NetFlowRule>;
struct NetFlowRules
{
    NetFlowRuleList exclude;
    NetFlowRuleList include;
};

struct NetFlowConfig
{
    NetFlowConfig() { }
    const char* dump_file = nullptr;
    std::unordered_map <snort::SfIp, NetFlowRules, NetFlowHash> device_rule_map;
    uint32_t update_timeout = 0;
    size_t flow_memcap = 0;
    size_t template_memcap = 0;
};

struct NetFlowStats : public LruCacheLocalStats
{
    PegCount invalid_netflow_record;
    PegCount packets;
    PegCount records;
    PegCount unique_flows;
    PegCount v9_missing_template;
    PegCount v9_options_template;
    PegCount v9_templates;
    PegCount version_5;
    PegCount version_9;
    PegCount netflow_cache_bytes_in_use;
    PegCount template_cache_bytes_in_use;
};

extern THREAD_LOCAL NetFlowStats netflow_stats;
extern THREAD_LOCAL snort::ProfileStats netflow_perf_stats;

class NetFlowModule : public snort::Module
{
public:
    NetFlowModule();
    ~NetFlowModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile() const override;
    NetFlowConfig* get_data();

    void parse_service_id_file(const std::string& serv_id_file_path);

    std::unordered_map<int, int> udp_service_mappings;
    std::unordered_map<int, int> tcp_service_mappings;

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

    static unsigned module_id;
    static void init()
    { module_id = snort::FlowData::create_flow_data_id(); }

private:
    NetFlowConfig* conf = nullptr;
    NetFlowRule rule_cfg = {};
    snort::SfIp device_ip_cfg = {};
    bool is_exclude_rule = false;
};

#endif

netflow_record.h

Path = src/service_inspectors/netflow/netflow_record.h

#ifndef NETFLOW_RECORD_H
#define NETFLOW_RECORD_H

#include "sfip/sf_ip.h"

struct NetFlowSessionRecord
{
    snort::SfIp initiator_ip;
    snort::SfIp responder_ip;
    snort::SfIp next_hop_ip;
    snort::SfIp netflow_initiator_ip;
    uint8_t proto;
    uint16_t initiator_port;
    uint16_t responder_port;
    uint32_t first_pkt_second;
    uint32_t last_pkt_second;
    uint64_t initiator_pkts;
    uint64_t responder_pkts;
    uint64_t initiator_bytes;
    uint64_t responder_bytes;
    uint8_t tcp_flags;

    uint32_t nf_src_as;
    uint32_t nf_dst_as;
    uint32_t nf_snmp_in;
    uint32_t nf_snmp_out;
    uint8_t nf_src_tos;
    uint8_t nf_dst_tos;
    uint8_t nf_src_mask;
    uint8_t nf_dst_mask;
};

#endif

service_inspectors/pop/

This directory contains all files related to POP protocol processing.

The protocol aware flushing for POP determines POP PDU and this reassembled POP PDU is processed by the POP inspector. Both POP commands/responses are parsed and the MIME attachments in POP responses are processed using the file API. The file API extracts and decodes the attachments. file_data is then set to the start of these extracted/decoded attachments. This inspector also identifies and whitelists the POPS traffic.

POP inspector uses PDFJSNorm class to extract and normalize JavaScript in PDF files. The normalized JavaScript is then available in the js_data buffer. The js_data buffer follows the JIT approach, thus, to perform the normalization, the rule with the js_data IPS option must be present as well as the js_norm module is configured.

pop_config.h

Path = src/service_inspectors/pop/pop_config.h

#ifndef POP_CONFIG_H
#define POP_CONFIG_H
// Configuration for Pop service inspector

#include "mime/file_mime_process.h"

struct POP_PROTO_CONF
{
    snort::DecodeConfig decode_conf;
    snort::MailLogConfig log_config;
};

struct PopStats
{
    PegCount packets;
    PegCount total_bytes;
    PegCount sessions;
    PegCount concurrent_sessions;
    PegCount max_concurrent_sessions;
    PegCount start_tls;
    PegCount ssl_search_abandoned;
    PegCount ssl_srch_abandoned_early;
    snort::MimeStats mime_stats;
    PegCount js_pdf_scripts;
};

extern const PegInfo pop_peg_names[];
extern THREAD_LOCAL PopStats popstats;

#endif

pop.h

Path = src/service_inspectors/pop/pop.h

#ifndef POP_H
#define POP_H

// Implementation header with definitions, datatypes and flowdata class for
// POP service inspector.

#include "flow/flow.h"
#include "mime/file_mime_process.h"

// Direction packet is coming from, if we can figure it out
#define POP_PKT_FROM_UNKNOWN  0
#define POP_PKT_FROM_CLIENT   1
#define POP_PKT_FROM_SERVER   2

#define STATE_DATA             0    // Data state
#define STATE_TLS_CLIENT_PEND  1    // Got STARTTLS
#define STATE_TLS_SERVER_PEND  2    // Got STARTTLS
#define STATE_TLS_DATA         3    // Successful handshake, TLS encrypted data
#define STATE_COMMAND          4
#define STATE_UNKNOWN          5
#define STATE_DECRYPTION_REQ   6

// session flags
#define POP_FLAG_NEXT_STATE_UNKNOWN         0x00000004
#define POP_FLAG_GOT_NON_REBUILT            0x00000008
#define POP_FLAG_CHECK_SSL                  0x00000010
#define POP_FLAG_ABANDON_EVT               0x00000020

typedef enum _POPCmdEnum
{
    CMD_APOP = 0,
    CMD_AUTH,
    CMD_CAPA,
    CMD_DELE,
    CMD_LIST,
    CMD_NOOP,
    CMD_PASS,
    CMD_QUIT,
    CMD_RETR,
    CMD_RSET,
    CMD_STAT,
    CMD_STLS,
    CMD_TOP,
    CMD_UIDL,
    CMD_USER,
    CMD_LAST
} POPCmdEnum;

typedef enum _POPRespEnum
{
    RESP_OK = 1,
    RESP_ERR,
    RESP_LAST
} POPRespEnum;

typedef enum _POPHdrEnum
{
    HDR_CONTENT_TYPE = 0,
    HDR_CONT_TRANS_ENC,
    HDR_CONT_DISP,
    HDR_LAST
} POPHdrEnum;

struct POPSearch
{
    const char* name;
    int name_len;
};

struct POPToken
{
    const char* name;
    int name_len;
    int search_id;
};

struct POPSearchInfo
{
    int id;
    int index;
    int length;
};

class PopMime : public snort::MimeSession
{
    using snort::MimeSession::MimeSession;
private:
    void decode_alert() override;
    void decompress_alert() override;
    void reset_state(snort::Flow* ssn) override;
    bool is_end_of_data(snort::Flow* ssn) override;
};

namespace snort
{
class PDFJSNorm;
}

struct POPData
{
    int state;
    int prev_response;
    int state_flags;
    int session_flags;
    PopMime* mime_ssn;
    snort::PDFJSNorm* jsn;
};

class PopFlowData : public snort::FlowData
{
public:
    PopFlowData();
    ~PopFlowData() override;

    static void init()
    { inspector_id = snort::FlowData::create_flow_data_id(); }

public:
    static unsigned inspector_id;
    POPData session;
};

#endif

pop_module.h

Path = src/service_inspectors/pop/pop_module.h

#ifndef POP_MODULE_H
#define POP_MODULE_H

// Interface to the IMAP service inspector

#include "framework/module.h"
#include "pop_config.h"

#define GID_POP 142

#define POP_UNKNOWN_CMD            1
#define POP_UNKNOWN_RESP           2
#define POP_B64_DECODING_FAILED    4
#define POP_QP_DECODING_FAILED     5
#define POP_UU_DECODING_FAILED     7
#define POP_FILE_DECOMP_FAILED     8

#define POP_NAME "pop"
#define POP_HELP "pop inspection"

namespace snort
{
struct SnortConfig;
}

extern THREAD_LOCAL snort::ProfileStats popPerfStats;

class PopModule : public snort::Module
{
public:
    PopModule();
    ~PopModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    unsigned get_gid() const override
    { return GID_POP; }

    const snort::RuleMap* get_rules() const override;
    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile() const override;

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

    POP_PROTO_CONF* get_data();

private:
    POP_PROTO_CONF* config;
};

#endif

pop_paf.h

Path = src/service_inspectors/pop/pop_paf.h

#ifndef POP_PAF_H
#define POP_PAF_H

// Protocol aware flushing for POP.

#include "mime/file_mime_paf.h"
#include "stream/stream_splitter.h"

// Structure used to record expected server termination sequence
enum PopExpectedResp
{
    POP_PAF_SINGLE_LINE_STATE,      // server response will end with \r\n
    POP_PAF_MULTI_LINE_STATE,       // server response will end with \r\n.\r\n
    POP_PAF_DATA_STATE,             // Indicated MIME will be contained in response
    POP_PAF_HAS_ARG                 // Intermediate state when parsing LIST
};

enum PopParseCmdState
{
    POP_CMD_SEARCH,         // Search for Command
    POP_CMD_FIN,            // Found space. Finished parsing Command
    POP_CMD_ARG             // Parsing command with multi-line response iff arg given
};

// saves data when parsing client commands
struct PopPafParseCmd
{
    const char* next_letter;        // a pointer to the current commands data
    PopExpectedResp exp_resp;       // the expected termination sequence for this command
    PopParseCmdState status;        // whether the current has already been found
};

// State tracker for POP PAF
struct PopPafData
{
    PopExpectedResp pop_state;       // The current POP PAF state.
    PopPafParseCmd cmd_state;        // all of the command parsing data
    DataEndState end_state;          // Current termination sequence state
    MimeDataPafInfo data_info;       // Mime Information
    bool cmd_continued;              // data continued from previous packet?
    bool end_of_data;
};

class PopSplitter : public snort::StreamSplitter
{
public:
    PopSplitter(bool c2s);

    Status scan(snort::Packet*, const uint8_t* data, uint32_t len,
        uint32_t flags, uint32_t* fp) override;

    bool is_paf() override
    { return true; }

    bool sync_on_start() const override
    { return true; }

public:
    PopPafData state;
};

// Function: Callback to check if POP data end is reached
bool pop_is_data_end(snort::Flow* ssn);

#endif

service_inspectors/rpc_decode/

This directory contains all files related to RPC Decode inspector.

RPC Decode inspector normalizes the RPC requests from remote machines by converting all fragments into one continuous stream. This is very useful for doing things like defeating hostile attackers trying to stealth themselves from IDS by fragmenting the request so the string 0186A0 is broken up. RPC Decode will alert when multiple queries are in a given packet, when a complete query exceeds the packet size.

rpc_module.h

Path = src/service_inspectors/rpc_decode/rpc_module.h

#ifndef RPC_MODULE_H
#define RPC_MODULE_H
// Interface to the RPC decode service inspector

#include "framework/module.h"

#define GID_RPC_DECODE 106

#define RPC_FRAG_TRAFFIC           1
#define RPC_MULTIPLE_RECORD        2
#define RPC_LARGE_FRAGSIZE         3
#define RPC_INCOMPLETE_SEGMENT     4
#define RPC_ZERO_LENGTH_FRAGMENT   5

struct RpcStats;

extern THREAD_LOCAL RpcStats rdstats;
extern THREAD_LOCAL snort::ProfileStats rpcdecodePerfStats;

class RpcDecodeModule : public snort::Module
{
public:
    RpcDecodeModule();

    unsigned get_gid() const override
    { return GID_RPC_DECODE; }

    const snort::RuleMap* get_rules() const override;
    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile() const override;

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }
};

#endif

service_inspectors/s7commplus/

S7comm (S7 Communication)and S7commplus are Siemens proprietary protocols that runs between programmable logic controllers (PLCs) of the Siemens S7-300/400 and S7-1500/1200(v4.0) families, typically on TCP port 102.

It is used for PLC programming, exchanging data between PLCs, accessing PLC data from SCADA systems and diagnostic purposes. The S7CommPlus protocol is a new version of the original S7Comm protocol.

The S7comm and S7commplus data comes as payload of COTP data packets. The first byte is 0x32 for S7comm and 0x72 for S7commplus as protocol identifiers. The same inspector will be used for both protocols, although currently only S7commplus is supported.

The S7Commplus inspector decodes the S7Commplus protocol and provides rule options to access certain protocol fields and Data content. This allows the user to write rules for S7commplus packets without decoding the protocol.

s7comm_decode.h

Path = src/service_inspectors/s7commplus/s7comm_decode.h

#ifndef S7COMM_DECODE_H
#define S7COMM_DECODE_H

namespace snort
{
struct Packet;
}

class S7commplusFlowData;

/* S7comm defines */
#define S7COMMPLUS_PDUTYPE_CONNECT                 0x01
#define S7COMMPLUS_PDUTYPE_DATA                    0x02
#define S7COMMPLUS_PDUTYPE_DATAFW1_5               0x03
#define S7COMMPLUS_PDUTYPE_KEEPALIVE               0xFF

#define COTP_HDR_LEN_FOR_S7COMMPLUS 2
#define COTP_HDR_PDU_TYPE_DATA  0xF0

#define S7COMM_PROTOCOL_ID      0x32
#define S7COMMPLUS_PROTOCOL_ID 0x72

#define TPKT_MIN_HDR_LEN 7     /* length field in TPKT header for S7comm */
#define TPKT_MAX_HDR_LEN       /* Undecided */
#define S7COMMPLUS_MIN_HDR_LEN 4
#define HDR_VERSION_TWO 0x02
#define INTEGRITY_PART_LEN 33 /* length of Integrity part in V3 Header packets */

/* Need 8 bytes for MBAP Header + Function Code */
#define S7COMMPLUS_MIN_LEN 8       this value needs to be decided

/* GIDs, SIDs, and Strings */
#define GENERATOR_SPP_S7COMMPLUS 149   /* matches generators.h */

#define S7COMMPLUS_BAD_LENGTH 1
#define S7COMMPLUS_BAD_PROTO_ID 2
#define S7COMMPLUS_RESERVED_FUNCTION 3

#define S7COMMPLUS_BAD_LENGTH_STR \
    "(spp_s7commplus): Length in S7commplus header does not match the length needed for the given S7commplus function."
#define S7COMMPLUS_BAD_PROTO_ID_STR "(spp_s7commplus): S7commplus protocol ID is non-zero."
#define S7COMMPLUS_RESERVED_FUNCTION_STR \
    "(spp_s7commplus): Reserved S7commplus function code in use."

bool S7commplusDecode(snort::Packet*, S7commplusFlowData* mfd);

#endif

s7comm.h

Path = src/service_inspectors/s7commplus/s7comm.h

#ifndef S7COMM_H
#define S7COMM_H

#include "flow/flow.h"
#include "framework/counts.h"

struct S7commplusStats
{
    PegCount sessions;
    PegCount frames;
    PegCount concurrent_sessions;
    PegCount max_concurrent_sessions;
};

struct S7commplusSessionData
{
    uint8_t s7commplus_proto_id = 0;
    uint8_t s7commplus_proto_version = 0;
    uint16_t s7commplus_data_len = 0;
    uint8_t s7commplus_opcode = 0;
    uint16_t s7commplus_reserved_1 = 0;
    uint16_t s7commplus_function = 0;
    uint16_t s7commplus_reserved_2 = 0;

    void session_data_reset()
    {
        s7commplus_proto_id = s7commplus_proto_version = s7commplus_opcode = 0;
        s7commplus_data_len = s7commplus_function = 0;
        s7commplus_reserved_1 = s7commplus_reserved_2 = 0;
    }
};

class S7commplusFlowData : public snort::FlowData
{
public:
    S7commplusFlowData();
    ~S7commplusFlowData() override;

    static void init();

    void reset()
    {
        ssn_data.session_data_reset();
    }

public:
    static unsigned inspector_id;
    S7commplusSessionData ssn_data;
};

int get_message_type(int version, const char* name);
int get_info_type(int version, const char* name);

extern THREAD_LOCAL S7commplusStats s7commplus_stats;

#endif

s7comm_module.h

Path = src/service_inspectors/s7commplus/s7comm_module.h

#ifndef S7COMM_MODULE_H
#define S7COMM_MODULE_H

#include "framework/module.h"

#define GID_S7COMMPLUS 149

#define S7COMMPLUS_BAD_LENGTH        1
#define S7COMMPLUS_BAD_PROTO_ID      2
#define S7COMMPLUS_RESERVED_FUNCTION 3

#define S7COMMPLUS_NAME "s7commplus"
#define S7COMMPLUS_HELP "s7commplus inspection"

extern THREAD_LOCAL snort::ProfileStats s7commplus_prof;

class S7commplusModule : public snort::Module
{
public:
    S7commplusModule();

    unsigned get_gid() const override
    { return GID_S7COMMPLUS; }

    const snort::RuleMap* get_rules() const override;

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;

    snort::ProfileStats* get_profile() const override
    { return &s7commplus_prof; }

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }
};

#endif

s7comm_paf.h

Path = src/service_inspectors/s7commplus/s7comm_paf.h

#ifndef S7COMM_PAF__H
#define S7COMM_PAF__H

// Protocol-Aware Flushing (PAF) code for the S7commplus inspector.

#include "stream/stream_splitter.h"

enum s7commplus_paf_state_t
{
    S7COMMPLUS_PAF_STATE__TPKT_VER = 0,
    S7COMMPLUS_PAF_STATE__TPKT_RESERVED,
    S7COMMPLUS_PAF_STATE__TPKT_LEN_1,
    S7COMMPLUS_PAF_STATE__TPKT_LEN_2,
    S7COMMPLUS_PAF_STATE__COTP_LEN,
    S7COMMPLUS_PAF_STATE__COTP_PDU_TYPE,
    S7COMMPLUS_PAF_STATE__SET_FLUSH
};

class S7commplusSplitter : public snort::StreamSplitter
{
public:
    S7commplusSplitter(bool);

    Status scan(snort::Packet*, const uint8_t* data, uint32_t len, uint32_t flags,
        uint32_t* fp) override;

    bool is_paf() override { return true; }

private:
    s7commplus_paf_state_t state;
    uint16_t tpkt_length;
};

#endif

service_inspectors/sip/

Session Initiation Protocol (SIP) is an application-layer control (signaling) protocol for creating, modifying, and terminating sessions with one or more participants. These sessions include Internet telephone calls, multimedia distribution, and multimedia conferences. SIP inspector provides ways to tackle Common Vulnerabilities and Exposures (CVEs) related with SIP found over the past few years. It also makes detecting new attacks easier.

SIP inspector scans the message headers and tracks dialogs.

  • Alert on malformed headers such as Call-ID, Via, To, From, CSeq, Contact, Content-Type, Content-length etc.

  • SIP is based on an HTTP-like request/response transaction model. SIP inspector tracks those dialogs in a linked list. To save memory, it tracks limited number of dialogs that can be configured.

  • SIP inspector also processes SDP (message body) to track media sessions.

The media session information can help AppID identification and improves performance by ignoring those media flows.

sip_common.h

Path = src/service_inspectors/sip/sip_common.h

#ifndef SIP_COMMON_H
#define SIP_COMMON_H

#include "sfip/sf_ip.h"

// Header containing datatypes/definitions shared by SIP inspector files.

enum SIPMethodsFlag
{
    SIP_METHOD_NULL        = 0,    // 0x0000,
    SIP_METHOD_INVITE      = 1,    // 0x0001,
    SIP_METHOD_CANCEL      = 2,    // 0x0002,
    SIP_METHOD_ACK         = 3,    // 0x0004,
    SIP_METHOD_BYE         = 4,    // 0x0008,
    SIP_METHOD_REGISTER    = 5,    // 0x0010,
    SIP_METHOD_OPTIONS     = 6,    // 0x0020,
    SIP_METHOD_REFER       = 7,    // 0x0040,
    SIP_METHOD_SUBSCRIBE   = 8,    // 0x0080,
    SIP_METHOD_UPDATE      = 9,    // 0x0100,
    SIP_METHOD_JOIN        = 10,   // 0x0200,
    SIP_METHOD_INFO        = 11,   // 0x0400,
    SIP_METHOD_MESSAGE     = 12,   // 0x0800,
    SIP_METHOD_NOTIFY      = 13,   // 0x1000,
    SIP_METHOD_PRACK       = 14,   // 0x2000,
    SIP_METHOD_USER_DEFINE = 15,   // 0x4000,
    SIP_METHOD_USER_DEFINE_MAX = 32// 0x80000000,
};

struct SipHeaders
{
    const char* callid;
    const char* from;
    const char* userAgent;
    const char* server;
    const char* userName;
    uint16_t callidLen;
    uint16_t fromLen;
    uint16_t userAgentLen;
    uint16_t serverLen;
    uint16_t userNameLen;

    SIPMethodsFlag methodFlag;
};

enum SIP_DialogState
{
    SIP_DLG_CREATE = 1,   // 1
    SIP_DLG_INVITING,     // 2
    SIP_DLG_EARLY,        // 3
    SIP_DLG_AUTHENCATING, // 4
    SIP_DLG_ESTABLISHED,  // 5
    SIP_DLG_REINVITING,   // 6
    SIP_DLG_TERMINATING,  // 7
    SIP_DLG_TERMINATED    // 8
};

struct SIP_MediaData
{
    snort::SfIp maddress;  // media IP
    uint16_t mport;   // media port
    uint8_t numPort;   // number of media ports
    SIP_MediaData* nextM;
} ;

typedef SIP_MediaData* SIP_MediaDataList;

struct SIP_MediaSession
{
    uint32_t sessionID; // a hash value of the session
    int savedFlag;      // whether this data has been saved by a dialog,
                        // if savedFlag = 1, this session will be deleted after sip message is
                        // processed.
    snort::SfIp maddress_default;  // Default media IP
    SIP_MediaDataList medias; // Media list in the session
    SIP_MediaSession* nextS; // Next media session
};

typedef SIP_MediaSession* SIP_MediaList;

struct SipDialog
{
    SIP_DialogState state;
    SIP_MediaList mediaSessions;
    bool mediaUpdated;
};

#endif

sip_config.h

Path = src/service_inspectors/sip/sip_config.h

#ifndef SIP_CONFIG_H
#define SIP_CONFIG_H

// Configuration for SIP service inspector

#include "framework/counts.h"
#include "main/thread.h"
#include "sip_common.h"

#define SIP_METHOD_DEFAULT     0x003f
#define SIP_METHOD_ALL     0xffffffff

#define SIP_STATUS_CODE_LEN (3)
#define SIP_CONTENT_LEN (5)

#define METHOD_NOT_FOUND (-1)

#define SIP_SESSION_SAVED   (1)
#define SIP_SESSION_INIT    (0)

#define SIP_DEFAULT_MAX_SESSIONS            10000
#define SIP_DEFAULT_MAX_DIALOGS_IN_SESSION  4
#define SIP_DEFAULT_MAX_URI_LEN             256
#define SIP_DEFAULT_MAX_CALL_ID_LEN         256
#define SIP_DEFAULT_MAX_REQUEST_NAME_LEN    20
#define SIP_DEFAULT_MAX_FROM_LEN            256
#define SIP_DEFAULT_MAX_TO_LEN              256
#define SIP_DEFAULT_MAX_VIA_LEN             1024
#define SIP_DEFAULT_MAX_CONTACT_LEN         256
#define SIP_DEFAULT_MAX_CONTENT_LEN         1024
#define NUM_OF_RESPONSE_TYPES  10
#define NUM_OF_REQUEST_TYPES  SIP_METHOD_USER_DEFINE

struct SipStats
{
    PegCount packets;
    PegCount sessions;
    PegCount concurrent_sessions;
    PegCount max_concurrent_sessions;
    PegCount events;
    PegCount dialogs;
    PegCount ignoreChannels;
    PegCount ignoreSessions;
    PegCount requests[NUM_OF_REQUEST_TYPES];
    PegCount responses[NUM_OF_RESPONSE_TYPES];
};

extern THREAD_LOCAL SipStats sip_stats;


// Header fields and processing functions
struct SIPMethod
{
    const char* name;
    SIPMethodsFlag methodFlag;
};

extern SIPMethod StandardMethods[];

struct SIPMethodNode
{
    char* methodName;
    int methodLen;
    SIPMethodsFlag methodFlag;
    SIPMethodNode* nextm;
};

typedef SIPMethodNode* SIPMethodlist;

// SIP configuration.

struct SIP_PROTO_CONF
{
    uint32_t maxNumDialogsInSession;
    uint32_t methodsConfig;
    SIPMethodlist methods;   // Which methods to check
    uint16_t maxUriLen;      // Maximum request_URI size
    uint16_t maxCallIdLen;   // Maximum call_ID size.
    uint16_t maxRequestNameLen;  // Maximum length of request name in the CSeqID.
    uint16_t maxFromLen;     // Maximum From field size
    uint16_t maxToLen;       // Maximum To field size
    uint16_t maxViaLen;      // Maximum Via field size
    uint16_t maxContactLen;  // Maximum Contact field size
    uint16_t maxContentLen;  // Maximum Content length
    bool ignoreChannel;   // Whether to ignore media channels found by SIP PP
};

// API to parse method list
void SIP_ParseMethods(
    const char* cur_tokenp, uint32_t* methodsConfig, SIPMethodlist* pmethods);

// Sets the Default method lists
void SIP_SetDefaultMethods(SIP_PROTO_CONF* config);

// API to add a user defined method to SIP config
SIPMethodNode* SIP_AddUserDefinedMethod(
    const char* methodName, uint32_t* methodsConfig, SIPMethodlist* pmethods);

// API to delete a method from SIP config
void SIP_DeleteMethods(SIPMethodNode*);

#endif

sip_dialog.h

Path = src/service_inspectors/sip/sip_dialog.h

#ifndef SIP_DIALOG_H
#define SIP_DIALOG_H

// Dialog management for SIP call flow analysis

#include "sip_parser.h"

namespace snort
{
struct Packet;
}

#define TOTAL_RESPONSES 0
#define RESPONSE1XX     1
#define RESPONSE2XX     2
#define RESPONSE3XX     3
#define RESPONSE4XX     4
#define RESPONSE5XX     5
#define RESPONSE6XX     6
#define TOTAL_REQUESTS 0

struct SIP_DialogData
{
    SIP_DialogID dlgID;
    SIP_DialogState state;
    SIPMethodsFlag creator;
    uint16_t status_code;
    SIP_MediaList mediaSessions;
    struct SIP_DialogData* nextD;
    struct SIP_DialogData* prevD;
};

struct SIP_DialogList
{
    SIP_DialogData* head;
    uint32_t num_dialogs;
};

int SIP_updateDialog(SIPMsg& sipMsg, SIP_DialogList* dList, snort::Packet* p, SIP_PROTO_CONF*);
void sip_freeDialogs(SIP_DialogList* list);

#endif

sip.h

Path = src/service_inspectors/sip/sip.h

#ifndef SIP_H
#define SIP_H

// Implementation header with definitions, datatypes and flowdata class for SIP service inspector.

#include "flow/flow.h"
#include "framework/data_bus.h"

#include "sip_dialog.h"
#include "sip_parser.h"
#include "sip_roptions.h"

#define SIP_FLG_MISSED_PACKETS        (0x10000)

struct SIPData
{
    uint32_t state_flags;
    SIP_DialogList dialogs;
    SIP_Roptions ropts;
    SIP_PROTO_CONF *sip_config;

    static unsigned pub_id;
};

class SipFlowData : public snort::FlowData
{
public:
    SipFlowData();
    ~SipFlowData() override;

    static void init()
    { inspector_id = snort::FlowData::create_flow_data_id(); }

public:
    static unsigned inspector_id;
    SIPData session;
};

SIPData* get_sip_session_data(const snort::Flow*);
SIPMethodNode* add_sip_method(const char*);

#endif

sip_module.h

Path = src/service_inspectors/sip/sip_module.h

#ifndef SIP_MODULE_H
#define SIP_MODULE_H

// Interface to the SIP service inspector

#include "framework/module.h"
#include "sip_config.h"

#define GID_SIP 140

//#define SIP_EVENT_MAX_SESSIONS      1   // deprecated
#define SIP_EVENT_EMPTY_REQUEST_URI   2
#define SIP_EVENT_BAD_URI             3
#define SIP_EVENT_EMPTY_CALL_ID       4
#define SIP_EVENT_BAD_CALL_ID         5
#define SIP_EVENT_BAD_CSEQ_NUM        6
#define SIP_EVENT_BAD_CSEQ_NAME       7
#define SIP_EVENT_EMPTY_FROM          8
#define SIP_EVENT_BAD_FROM            9
#define SIP_EVENT_EMPTY_TO            10
#define SIP_EVENT_BAD_TO              11
#define SIP_EVENT_EMPTY_VIA           12
#define SIP_EVENT_BAD_VIA             13
#define SIP_EVENT_EMPTY_CONTACT       14
#define SIP_EVENT_BAD_CONTACT         15
#define SIP_EVENT_BAD_CONTENT_LEN     16
#define SIP_EVENT_MULTI_MSGS          17
#define SIP_EVENT_MISMATCH_CONTENT_LEN          18
#define SIP_EVENT_INVALID_CSEQ_NAME             19
#define SIP_EVENT_AUTH_INVITE_REPLAY_ATTACK     20
#define SIP_EVENT_AUTH_INVITE_DIFF_SESSION      21
#define SIP_EVENT_BAD_STATUS_CODE               22
#define SIP_EVENT_EMPTY_CONTENT_TYPE            23
#define SIP_EVENT_INVALID_VERSION               24
#define SIP_EVENT_MISMATCH_METHOD               25
#define SIP_EVENT_UNKOWN_METHOD                 26
#define SIP_EVENT_MAX_DIALOGS_IN_A_SESSION      27

#define SIP_NAME "sip"
#define SIP_HELP "sip inspection"

namespace snort
{
struct SnortConfig;
}

extern THREAD_LOCAL snort::ProfileStats sipPerfStats;

class SipModule : public snort::Module
{
public:
    SipModule();
    ~SipModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    unsigned get_gid() const override
    { return GID_SIP; }

    const snort::RuleMap* get_rules() const override;
    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile() const override;

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

    SIP_PROTO_CONF* get_data();

private:
    SIP_PROTO_CONF* conf;
    std::string sip_methods;
};

#endif

sip_parser.h

Path = src/service_inspectors/sip/sip_parser.h

#ifndef SIP_PARSER_H
#define SIP_PARSER_H

// functions for parsing and querying SIP configuration

#include "sip_config.h"

struct SIP_DialogID
{
    uint32_t callIdHash;
    uint32_t fromTagHash;
    uint32_t toTagHash;
};

struct SIPMsg
{
    uint16_t headerLen;
    uint16_t methodLen;
    SIPMethodsFlag methodFlag;
    uint16_t status_code;

    uint16_t uriLen;
    uint16_t callIdLen;
    uint16_t cseqNameLen;
    uint16_t fromLen;
    uint16_t fromTagLen;
    uint16_t toLen;
    uint16_t toTagLen;
    uint16_t viaLen;
    uint16_t contactLen;
    uint16_t bodyLen;
    uint16_t contentTypeLen;
    uint32_t content_len;
    SIP_DialogID dlgID;
    SIP_MediaSession* mediaSession;
    const char* authorization;
    const uint8_t* header;
    const uint8_t* body_data; // Set to null if not applicable
    uint64_t cseqnum;

    uint16_t userNameLen;
    uint16_t userAgentLen;
    uint16_t serverLen;
    bool mediaUpdated;

    // nothing after this point is zeroed ...  Input parameters
    unsigned char isTcp;
    const char* method;
    const char* uri;
    const char* call_id;
    const char* cseqName;
    const char* from;
    const char* from_tag;
    const char* to;
    const char* to_tag;
    const char* via;
    const char* contact;

    const char* content_type;
    const char* content_encode;

    const char* userAgent;
    const char* userName;
    const char* server;
};

#define SIPMSG_ZERO_LEN offsetof(SIPMsg, isTcp)

#define MAX_STAT_CODE      999
#define MIN_STAT_CODE      100

bool sip_parse(SIPMsg*, const char*, const char*, SIP_PROTO_CONF*);
void sip_freeMsg(SIPMsg* msg);
void sip_freeMediaSession(SIP_MediaSession*);
void sip_freeMediaList(SIP_MediaList medias);

#endif

sip_roptions.h

Path = src/service_inspectors/sip/sip_roptions.h

#ifndef SIP_ROPTIONS_H
#define SIP_ROPTIONS_H

// Definitions of sip rule option structures.

#include "sip_config.h"

#define SIP_NUM_STAT_CODE_MAX 20

struct SIP_Roptions
{
    const char* method_data;
    uint16_t method_len;

    uint16_t status_code;       // sip_stat_code data

    const uint8_t* header_data; // Set to null if not applicable
    uint16_t header_len;

    const uint8_t* body_data;   // Set to null if not applicable
    uint16_t body_len;
};

struct SipMethodRuleOptData
{
    int flags;
    int mask;
};

struct SipStatCodeRuleOptData
{
    uint16_t stat_codes[SIP_NUM_STAT_CODE_MAX];
};

#endif

sip_splitter.h

Path = src/service_inspectors/sip/sip_splitter.h

#ifndef SIP_SPLITTER_H
#define SIP_SPLITTER_H
// Protocol aware flushing for sip

#include "stream/stream_splitter.h"

enum SipPafStates
{
    SIP_PAF_START_STATE = 0,     //Default state. Continue until LF
    SIP_PAF_CONTENT_LEN_CMD, //Searching Content-Length header
    SIP_PAF_CONTENT_LEN_CONVERT,   //Parse the literal content length
    SIP_PAF_BODY_SEARCH,    //Check SIP body start
    SIP_PAF_FLUSH_STATE     //Flush if Content-Length is reached
};

//State tracker for SIP Content Length
enum SipPafDataLenStatus
{
    SIP_PAF_LENGTH_INVALID,
    SIP_PAF_LENGTH_CONTINUE,
    SIP_PAF_LENGTH_DONE
};

//State tracker for SIP Body Boundary
enum SipPafBodyStatus
{
    SIP_PAF_BODY_UNKNOWN,
    SIP_PAF_BODY_START_FIRST_CR,   //Check SIP body start - first CR
    SIP_PAF_BODY_START_FIRST_LF,   //Check SIP body start - first LF
    SIP_PAF_BODY_START_SECOND_CR,  //Check SIP body start - second CR
    SIP_PAF_BODY_START_SECOND_LF  //Check SIP body start - second LF
};

#define UNKNOWN_CONTENT_LENGTH UINT32_MAX

class SipSplitter : public snort::StreamSplitter
{
public:
    SipSplitter(bool c2s);

    Status scan(snort::Packet*, const uint8_t* data, uint32_t len,
        uint32_t flags, uint32_t* fp) override;

    bool is_paf() override
    {
        return true;
    }

private:
    void reset_states();
    bool find_data_end_single_line(const uint8_t ch);
    void process_command(const uint8_t ch);
    SipPafDataLenStatus get_length(const uint8_t c);
    bool find_body(const uint8_t ch);

    static const char content_len_key[];
    static const char content_len_key_compact[];
    SipPafStates paf_state;
    SipPafBodyStatus body_state;  //State to find sip body
    const char *next_letter;     //The current character in Content-Length
    uint32_t content_length;

#ifdef UNIT_TEST
    friend class SipSplitterUT;
#endif

};
#endif

sip_utils.h

Path = src/service_inspectors/sip/sip_utils.h

#ifndef SIP_UTILS_H
#define SIP_UTILS_H

// Utility functions for SIP inspector

#include "sip_config.h"

int SIP_TrimSP(const char*, const char*, const char**, const char**);
SIPMethodNode* SIP_FindMethod(SIPMethodlist, const char* method, unsigned int);
uint32_t strToHash(const char*, int);

#endif

service_inspectors/sip/test/

sip_splitter_test.h

Path = src/service_inspectors/sip/test/sip_splitter_test.h

#ifndef SIP_SPLITTER_TEST
#define SIP_SPLITTER_TEST
//Test for sip splitter

#include "stream/stream_splitter.h"
#include "service_inspectors/sip/sip_splitter.h"

//stubs to avoid link errors
const snort::StreamBuffer snort::StreamSplitter::reassemble(snort::Flow*, unsigned int, unsigned int,
    unsigned char const*, unsigned int, unsigned int, unsigned int &) { return {}; }
unsigned snort::StreamSplitter::max(snort::Flow *) { return 0; }

const uint8_t line_feed = '\n';
const uint8_t carriage_return = '\r';
const uint8_t no_lf_cr = '\t';

//characters recognized by isspace() as spaces
const uint8_t spaces[] = {' ', '\t', '\n', '\v', '\f', '\r'};

//character recognized by isblank() as separators
const uint8_t blanks[] = {' ', '\t' };

class SipSplitterUT
{
public:
    SipSplitterUT(const SipSplitter& ss) : ss(ss) { }

    bool splitter_is_paf()
    {
        return ss.is_paf();
    }

    void splitter_reset_states()
    {
        ss.reset_states();
    }

    SipPafStates splitter_get_paf_state()
    {
        return ss.paf_state;
    }

    SipPafBodyStatus splitter_get_body_state()
    {
        return ss.body_state;
    }

    const char* splitter_get_next_letter()
    {
        return ss.next_letter;
    }

    uint32_t splitter_get_content_length()
    {
        return ss.content_length;
    }

    const char* splitter_get_content_length_key()
    {
        return SipSplitter::content_len_key;
    }

    bool splitter_data_end_single_line(const uint8_t ch)
    {
        return ss.find_data_end_single_line(ch);
    }

    bool splitter_find_body(const uint8_t ch)
    {
        return ss.find_body(ch);
    }

    void splitter_set_content_length(uint32_t len)
    {
        ss.content_length = len;
    }

    void splitter_set_body_state(SipPafBodyStatus bstate)
    {
        ss.body_state = bstate;
    }

    void splitter_set_paf_state(SipPafStates pafstate)
    {
        ss.paf_state = pafstate;
    }

    void splitter_set_next_letter_last()
    {
        ss.next_letter = &SipSplitter::content_len_key_compact[1];
    }

    SipPafDataLenStatus splitter_get_length(const uint8_t ch)
    {
        return ss.get_length(ch);
    }

    void splitter_process_command(const uint8_t ch)
    {
        ss.process_command(ch);
    }

    snort::StreamSplitter::Status splitter_scan(snort::Packet* pkt, const uint8_t* data,
                                uint32_t len, uint32_t flags, uint32_t* fp)
    {
        return ss.scan(pkt, data, len, flags, fp);
    }

    bool is_init()
    {
        return ss.paf_state == SIP_PAF_START_STATE && ss.content_length == UNKNOWN_CONTENT_LENGTH &&
                        ss.next_letter == nullptr && ss.body_state == SIP_PAF_BODY_UNKNOWN;
    }

    const char * splitter_get_content_length_compact_key()
    {
        return SipSplitter::content_len_key_compact;
    }
private:
    SipSplitter ss;
};
#endif

service_inspectors/smtp/

This directory contains all files related to SMTP protocol processing.

The protocol aware flushing for SMTP determines SMTP PDU and this reassembled SMTP PDU is processed by the SMTP inspector. Both SMTP requests/responses are parsed and the MIME attachments in SMTP responses are processed using the file API. The file API extracts and decodes the attachments. file_data is then set to the start of these extracted/decoded attachments. This inspector also identifies and whitelists the SMTPS traffic.

SMTP inspector logs the filename, email addresses, attachment names when configured. The SMTP commands are also normalized based on the config.

SMTP inspector uses PDFJSNorm class to extract and normalize JavaScript in PDF files. The normalized JavaScript is then available in the js_data buffer. The js_data buffer follows the JIT approach, thus, to perform the normalization, the rule with the js_data IPS option must be present as well as the js_norm module is configured.

smtp_config.h

Path = src/service_inspectors/smtp/smtp_config.h

#ifndef SMTP_CONFIG_H
#define SMTP_CONFIG_H

// Configuration for SMTP inspector

#include "mime/file_mime_process.h"
#include "search_engines/search_tool.h"

enum SMTPNormType
{
    NORMALIZE_NONE = 0,
    NORMALIZE_CMDS,
    NORMALIZE_ALL
};

enum SMTPXlinkState
{
    DISABLE_XLINK2STATE = 0,
    ALERT_XLINK2STATE,
    DROP_XLINK2STATE
};

enum SMTPCmdEnum
{
    CMD_ATRN = 0,
    CMD_AUTH,
    CMD_BDAT,
    CMD_DATA,
    CMD_DEBUG,
    CMD_EHLO,
    CMD_EMAL,
    CMD_ESAM,
    CMD_ESND,
    CMD_ESOM,
    CMD_ETRN,
    CMD_EVFY,
    CMD_EXPN,
    CMD_HELO,
    CMD_HELP,
    CMD_IDENT,
    CMD_MAIL,
    CMD_NOOP,
    CMD_ONEX,
    CMD_QUEU,
    CMD_QUIT,
    CMD_RCPT,
    CMD_RSET,
    CMD_SAML,
    CMD_SEND,
    CMD_SIZE,
    CMD_STARTTLS,
    CMD_SOML,
    CMD_TICK,
    CMD_TIME,
    CMD_TURN,
    CMD_TURNME,
    CMD_VERB,
    CMD_VRFY,
    CMD_X_EXPS,
    CMD_XADR,
    CMD_XAUTH,
    CMD_XCIR,
    CMD_XEXCH50,
    CMD_XGEN,
    CMD_XLICENSE,
    CMD_X_LINK2STATE,
    CMD_XQUE,
    CMD_XSTA,
    CMD_XTRN,
    CMD_XUSR,
    CMD_ABORT,
    CMD_LAST
};

enum SMTPCmdTypeEnum
{
    SMTP_CMD_TYPE_NORMAL = 0,
    SMTP_CMD_TYPE_DATA,
    SMTP_CMD_TYPE_BDATA,
    SMTP_CMD_TYPE_AUTH,
    SMTP_CMD_TYPE_LAST
};

struct SMTPCmdConfig
{
    bool alert;
    bool normalize;     //  1 if we should normalize this command
    int max_line_len;   //  Max length of this particular command
};

struct SMTPSearch
{
    const char* name;
    int name_len;
};

struct SMTPToken
{
    const char* name;
    int name_len;
    int search_id;
    SMTPCmdTypeEnum type;
};

struct SmtpProtoConf
{
    SMTPNormType normalize;
    bool ignore_tls_data;
    int max_auth_command_line_len = 1000;
    int max_command_line_len = 0;
    int max_header_line_len = 0;
    int max_response_line_len = 0;
    SMTPXlinkState xlink2state;
    snort::MailLogConfig log_config;
    snort::DecodeConfig decode_conf;

    uint32_t xtra_filename_id;
    uint32_t xtra_mfrom_id;
    uint32_t xtra_rcptto_id;
    uint32_t xtra_ehdrs_id;

    int num_cmds;
    SMTPToken* cmds = nullptr;
    SMTPCmdConfig* cmd_config = nullptr;
    SMTPSearch* cmd_search = nullptr;
    snort::SearchTool* cmd_search_mpse = nullptr;

    void show() const;
};

struct SmtpStats
{
    PegCount packets;
    PegCount total_bytes;
    PegCount sessions;
    PegCount concurrent_sessions;
    PegCount max_concurrent_sessions;
    PegCount starttls;
    PegCount ssl_search_abandoned;
    PegCount ssl_search_abandoned_too_soon;
    snort::MimeStats mime_stats;
    PegCount js_pdf_scripts;
};

extern const PegInfo smtp_peg_names[];
extern THREAD_LOCAL SmtpStats smtpstats;

#endif

smtp.h

Path = src/service_inspectors/smtp/smtp.h

#ifndef SMTP_H
#define SMTP_H

// Implementation header with definitions, datatypes and flowdata class for
// SMTP service inspector.

#include "flow/flow.h"
#include "smtp_config.h"

// Direction packet is coming from, if we can figure it out
#define SMTP_PKT_FROM_UNKNOWN  0
#define SMTP_PKT_FROM_CLIENT   1
#define SMTP_PKT_FROM_SERVER   2

#define SEARCH_CMD       0
#define SEARCH_RESP      1
#define SEARCH_HDR       2
#define SEARCH_DATA_END  3
#define NUM_SEARCHES  4

#define BOUNDARY     0

#define STATE_CONNECT          0
#define STATE_COMMAND          1    // Command state of SMTP transaction
#define STATE_DATA             2    // Data state
#define STATE_BDATA            3    // Binary data state
#define STATE_TLS_CLIENT_PEND  4    // Got STARTTLS
#define STATE_TLS_SERVER_PEND  5    // Got STARTTLS
#define STATE_TLS_DATA         6    // Successful handshake, TLS encrypted data
#define STATE_AUTH             7
#define STATE_XEXCH50          8
#define STATE_UNKNOWN          9

#define STATE_DATA_INIT    0
#define STATE_DATA_HEADER  1    // Data header section of data state
#define STATE_DATA_BODY    2    // Data body section of data state
#define STATE_MIME_HEADER  3    // MIME header section within data section
#define STATE_DATA_UNKNOWN 4

// state flags
#define SMTP_FLAG_GOT_MAIL_CMD               0x00000001
#define SMTP_FLAG_GOT_RCPT_CMD               0x00000002
#define SMTP_FLAG_BDAT                       0x00001000
#define SMTP_FLAG_ABORT                      0x00002000
#define SMTP_FLAG_ABANDON_EVT                0x00010000

// session flags
#define SMTP_FLAG_XLINK2STATE_GOTFIRSTCHUNK  0x00000001
#define SMTP_FLAG_XLINK2STATE_ALERTED        0x00000002
#define SMTP_FLAG_NEXT_STATE_UNKNOWN         0x00000004
#define SMTP_FLAG_GOT_NON_REBUILT            0x00000008
#define SMTP_FLAG_CHECK_SSL                  0x00000010

#define SMTP_SSL_ERROR_FLAGS \
    (SSL_BOGUS_HS_DIR_FLAG | \
    SSL_BAD_VER_FLAG | \
    SSL_BAD_TYPE_FLAG | \
    SSL_UNKNOWN_FLAG)

#define MAX_AUTH_NAME_LEN  20  // Max length of SASL mechanisms, defined in RFC 4422

enum SMTPRespEnum
{
    RESP_220 = 0,
    RESP_221,
    RESP_235,
    RESP_250,
    RESP_334,
    RESP_354,
    RESP_421,
    RESP_450,
    RESP_451,
    RESP_452,
    RESP_454,
    RESP_500,
    RESP_501,
    RESP_502,
    RESP_503,
    RESP_504,
    RESP_535,
    RESP_550,
    RESP_551,
    RESP_552,
    RESP_553,
    RESP_554,
    RESP_LAST
};

enum SMTPHdrEnum
{
    HDR_CONTENT_TYPE = 0,
    HDR_CONT_TRANS_ENC,
    HDR_CONT_DISP,
    HDR_LAST
};

enum SMTPDataEndEnum
{
    DATA_END_1 = 0,
    DATA_END_2,
    DATA_END_3,
    DATA_END_4,
    DATA_END_LAST
};

enum SMTPEol
{
    EOL_NOT_SEEN,
    EOL_LF,
    EOL_CRLF,
    EOL_MIXED
};

struct SMTPSearchInfo
{
    int id;
    int index;
    int length;
};

struct SMTPAuthName
{
    int length;
    char name[MAX_AUTH_NAME_LEN];
};

class SmtpMime : public snort::MimeSession
{
public:
    using snort::MimeSession::MimeSession;
    SmtpProtoConf* config = nullptr;
#ifndef UNIT_TEST
private:
#endif
    int handle_header_line(const uint8_t* ptr, const uint8_t* eol,
        int max_header_len, snort::Packet* p) override;
    int normalize_data(const uint8_t* ptr, const uint8_t* data_end, snort::Packet* p) override;
#ifdef UNIT_TEST
private:
#endif
    void decode_alert() override;
    void decompress_alert() override;
    void reset_state(snort::Flow* ssn) override;
    bool is_end_of_data(snort::Flow* ssn) override;
};

namespace snort
{
class PDFJSNorm;
}

struct SMTPData
{
    //Initialize structure with default values
    SMTPData() : state{0},
                 state_flags{0},
                 session_flags{0},
                 dat_chunk{0},
                 mime_ssn{nullptr},
                 jsn{nullptr},
                 auth_name{nullptr},
                 client_requested_starttls{false},
                 pipelined_command_counter{0},
                 server_accepted_starttls{false},
                 client_eol{EOL_NOT_SEEN},
                 server_eol{EOL_NOT_SEEN}
    { }

    int state;
    int state_flags;
    int session_flags;
    uint32_t dat_chunk;
    SmtpMime* mime_ssn;
    snort::PDFJSNorm* jsn;
    SMTPAuthName* auth_name;
    bool client_requested_starttls;
    size_t pipelined_command_counter;
    bool server_accepted_starttls;
    SMTPEol client_eol;
    SMTPEol server_eol;
};

class SmtpFlowData : public snort::FlowData
{
public:
    SmtpFlowData();
    ~SmtpFlowData() override;

    static void init()
    { inspector_id = snort::FlowData::create_flow_data_id(); }

public:
    static unsigned inspector_id;
    SMTPData session;
};

extern THREAD_LOCAL bool smtp_normalizing;

#endif

smtp_module.h

Path = src/service_inspectors/smtp/smtp_module.h

#ifndef SMTP_MODULE_H
#define SMTP_MODULE_H

// Interface to the SMTP service inspector

#include "framework/module.h"
#include "smtp_config.h"

#define GID_SMTP 124

#define SMTP_COMMAND_OVERFLOW       1
#define SMTP_DATA_HDR_OVERFLOW      2
#define SMTP_RESPONSE_OVERFLOW      3
#define SMTP_SPECIFIC_CMD_OVERFLOW  4
#define SMTP_UNKNOWN_CMD            5
#define SMTP_ILLEGAL_CMD            6
#define SMTP_HEADER_NAME_OVERFLOW   7
#define SMTP_XLINK2STATE_OVERFLOW   8
#define SMTP_DECODE_MEMCAP_EXCEEDED 9
#define SMTP_B64_DECODING_FAILED    10
#define SMTP_QP_DECODING_FAILED     11
// Do not delete or reuse this SID. Commenting this SID as this alert is no longer valid.
//#define SMTP_BITENC_DECODING_FAILED 12
#define SMTP_UU_DECODING_FAILED     13
#define SMTP_AUTH_ABORT_AUTH        14
#define SMTP_AUTH_COMMAND_OVERFLOW  15
#define SMTP_FILE_DECOMP_FAILED     16
#define SMTP_STARTTLS_INJECTION_ATTEMPT 17
#define SMTP_LF_CRLF_MIX            18

#define SMTP_NAME "smtp"
#define SMTP_HELP "smtp inspection"

#define PCMD_LEN         0x0000
#define PCMD_ALT         0x0001
#define PCMD_AUTH        0x0002
#define PCMD_BDATA       0x0004
#define PCMD_DATA        0x0008
#define PCMD_INVALID     0x0010
#define PCMD_NORM        0x0020
#define PCMD_VALID       0x0040

namespace snort
{
struct SnortConfig;
}

extern THREAD_LOCAL snort::ProfileStats smtpPerfStats;
struct SmtpCmd
{
    std::string name;

    uint32_t flags;
    unsigned number;

    SmtpCmd(const std::string&, uint32_t, int);
    SmtpCmd(const std::string&, int);
};

class SmtpModule : public snort::Module
{
public:
    SmtpModule();
    ~SmtpModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    unsigned get_gid() const override
    { return GID_SMTP; }

    const snort::RuleMap* get_rules() const override;
    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile() const override;

    SmtpProtoConf* get_data();
    const SmtpCmd* get_cmd(unsigned idx);
    void clear_cmds();

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

private:
    void add_commands(snort::Value&, uint32_t flags);

private:
    SmtpProtoConf* config = nullptr;
    std::vector<SmtpCmd*> cmds;
    std::string names;
    int number = 0;
};

#endif

smtp_normalize.h

Path = src/service_inspectors/smtp/smtp_normalize.h

#ifndef SMTP_NORMALIZE_H
#define SMTP_NORMALIZE_H

// Implementation of normalizing SMTP traffic into the alternate buffer

#include <cstdint>

namespace snort
{
struct Packet;
}

int SMTP_NormalizeCmd(snort::Packet*, const uint8_t*, const uint8_t*, const uint8_t*);

#endif

smtp_paf.h

Path = src/service_inspectors/smtp/smtp_paf.h

#ifndef SMTP_PAF_H
#define SMTP_PAF_H

// Protocol aware flushing for SMTP

#include "mime/file_mime_paf.h"
#include "stream/stream_splitter.h"

// State tracker for SMTP PAF
enum SmtpPafState
{
    SMTP_PAF_CMD_STATE,
    SMTP_PAF_DATA_STATE
};
// State tracker for data command
typedef enum _SmtpPafCmdState
{
    SMTP_PAF_CMD_UNKNOWN,
    SMTP_PAF_CMD_START,
    SMTP_PAF_CMD_DETECT,
    SMTP_PAF_CMD_DATA_LENGTH_STATE,
    SMTP_PAF_CMD_DATA_END_STATE
} SmtpPafCmdState;

struct SmtpCmdSearchInfo
{
    SmtpPafCmdState cmd_state;
    int search_id;
    const char* search_state;
};

// State tracker for SMTP PAF
struct SmtpPafData
{
    DataEndState data_end_state;
    uint32_t length;
    SmtpPafState smtp_state;
    SmtpCmdSearchInfo cmd_info;
    MimeDataPafInfo data_info;
    bool end_of_data;
};

class SmtpSplitter : public snort::StreamSplitter
{
public:
    SmtpSplitter(bool c2s, int max_auth_cmd_line_len);

    Status scan(snort::Packet*, const uint8_t* data, uint32_t len,
        uint32_t flags, uint32_t* fp) override;

    bool is_paf() override { return true; }

public:
    SmtpPafData state;

private:
    int max_auth_command_line_len;
};

// Function: Check if IMAP data end is reached
bool smtp_is_data_end(snort::Flow* ssn);

#endif

smtp_util.h

Path = src/service_inspectors/smtp/smtp_util.h

#ifndef SMTP_UTIL_H
#define SMTP_UTIL_H

// SMTP helper functions

#include "smtp.h"
#include "smtp_config.h"

namespace snort
{
struct Packet;
}

SMTPEol SMTP_GetEOL(const uint8_t*, const uint8_t*, const uint8_t**, const uint8_t**);
void SMTP_LogFuncs(SmtpProtoConf*, snort::Packet*, snort::MimeSession*);

int SMTP_CopyToAltBuffer(snort::Packet*, const uint8_t*, int);
const uint8_t* SMTP_GetAltBuffer(snort::Packet*, unsigned& len);
void SMTP_ResetAltBuffer(snort::Packet*);

#endif

smtp_xlink2state.h

Path = src/service_inspectors/smtp/smtp_xlink2state.h

#ifndef SMTP_XLINK2STATE_H
#define SMTP_XLINK2STATE_H

// declares the API to handle the X-Link2State vulnerability

#include "smtp.h"

int ParseXLink2State(SmtpProtoConf*, snort::Packet*, SMTPData*, const uint8_t*);

#endif

service_inspectors/ssh/

This directory contains all files related to SSH protocol inspection.

The SSH inspector processes the stream reassembled packets and detects the following exploits: Challenge-Response Buffer Overflow, CRC 32, Secure CRT, and the Protocol Mismatch exploit.

Both Challenge-Response Overflow and CRC 32 attacks occur after the key exchange, and are therefore encrypted. Both attacks involve sending a large payload (20kb+) to the server immediately after the authentication challenge. To detect the attacks, the SSH inspector counts the number of bytes transmitted to the server. If those bytes exceed a pre-defined limit within a pre-define number of packets, an alert is generated. Since Challenge-Response Overflow only effects SSHv2 and CRC 32 only effects SSHv1, the SSH version string exchange is used to distinguish the attacks.

The Secure CRT and protocol mismatch exploits are observable before the key exchange.

ssh_config.h

Path = src/service_inspectors/ssh/ssh_config.h

#ifndef SSH_CONFIG_H
#define SSH_CONFIG_H

// Configuration for SSH service inspector

struct SSH_PROTO_CONF
{
    uint16_t MaxEncryptedPackets;
    uint16_t MaxClientBytes;
    uint16_t MaxServerVersionLen;
};

struct SshStats
{
    PegCount total_packets;
    PegCount total_bytes;
    PegCount concurrent_sessions;
    PegCount max_concurrent_sessions;
};

#endif

ssh.h

Path = src/service_inspectors/ssh/ssh.h

#ifndef SSH_H
#define SSH_H

// Implementation header with definitions, datatypes and flowdata class for
// SSH service inspector.

// Alert for Gobbles, CRC32, protocol mismatch (Cisco catalyst
// vulnerability), and a SecureCRT vulnerability.  Will also alert if the
// client or server traffic appears to flow the wrong direction, or if
// packets appear malformed/spoofed.

#include "flow/flow.h"
#include "protocols/packet.h"

// FIXIT-L move these to ssh.cc
// Session state flags for SSHData::state_flags
#define SSH_FLG_CLEAR           (0x0)
#define SSH_FLG_CLIENT_IDSTRING_SEEN    (0x1)
#define SSH_FLG_SERV_IDSTRING_SEEN  (0x2)
#define SSH_FLG_SERV_PKEY_SEEN      (0x4)
#define SSH_FLG_CLIENT_SKEY_SEEN    (0x8)
#define SSH_FLG_CLIENT_KEXINIT_SEEN (0x10)
#define SSH_FLG_SERV_KEXINIT_SEEN   (0x20)
#define SSH_FLG_KEXDH_INIT_SEEN     (0x40)
#define SSH_FLG_KEXDH_REPLY_SEEN    (0x80)
#define SSH_FLG_GEX_REQ_SEEN        (0x100)
#define SSH_FLG_GEX_GRP_SEEN        (0x200)
#define SSH_FLG_GEX_INIT_SEEN       (0x400)
#define SSH_FLG_GEX_REPLY_SEEN      (0x800)
#define SSH_FLG_CLIENT_NEWKEYS_SEEN (0x1000)
#define SSH_FLG_SESS_ENCRYPTED      (0x2000)
#define SSH_FLG_RESPOVERFLOW_ALERTED    (0x4000)
#define SSH_FLG_CRC32_ALERTED       (0x8000)
#define SSH_FLG_MISSED_PACKETS      (0x10000)
#define SSH_FLG_REASSEMBLY_SET      (0x20000)
#define SSH_FLG_AUTODETECTED        (0x40000)
#define SSH_FLG_SERVER_NEWKEYS_SEEN (0x80000)

// Some convenient combinations of state flags.
#define SSH_FLG_BOTH_IDSTRING_SEEN \
    (SSH_FLG_CLIENT_IDSTRING_SEEN |  \
    SSH_FLG_SERV_IDSTRING_SEEN )

#define SSH_FLG_V1_KEYEXCH_DONE \
    (SSH_FLG_SERV_PKEY_SEEN | \
    SSH_FLG_CLIENT_SKEY_SEEN )

#define SSH_FLG_V2_KEXINIT_DONE \
    (SSH_FLG_CLIENT_KEXINIT_SEEN | \
    SSH_FLG_SERV_KEXINIT_SEEN )

#define SSH_FLG_V2_DHOLD_DONE \
    (SSH_FLG_KEXDH_INIT_SEEN | \
    SSH_FLG_KEXDH_REPLY_SEEN | \
    SSH_FLG_CLIENT_NEWKEYS_SEEN )

#define SSH_FLG_V2_DHNEW_DONE \
    (SSH_FLG_GEX_REQ_SEEN | \
    SSH_FLG_GEX_GRP_SEEN | \
    SSH_FLG_GEX_INIT_SEEN | \
    SSH_FLG_GEX_REPLY_SEEN | \
    SSH_FLG_CLIENT_NEWKEYS_SEEN )

// SSH version values for SSHData::version
#define SSH_VERSION_UNKNOWN (0x0)
#define SSH_VERSION_1       (0x1)
#define SSH_VERSION_2       (0x2)

// Per-session data block containing current state
// of the SSH inspector for the session.
struct SSHData
{
    uint8_t version = SSH_VERSION_UNKNOWN; // Version of SSH detected for this session
    uint16_t num_enc_pkts;     // encrypted packets seen on this session
    uint16_t num_client_bytes; // bytes of encrypted data sent by client without a server response
    uint32_t state_flags;      // Bit vector describing the current state of the session
};

class SshFlowData : public snort::FlowData
{
public:
    SshFlowData();
    ~SshFlowData() override;

    static void init()
    { inspector_id = snort::FlowData::create_flow_data_id(); }

public:
    static unsigned inspector_id;
    SSHData session = {};
};

#define SSH_BANNER "SSH-"
// Length of SSH2 header, in bytes.
#define SSH2_HEADERLEN      (5)
// Length of SSH2 Padding, in bytes.
#define SSH2_PADDING_LEN    (1)
// Length of SSH2 packet, in bytes.
#define SSH2_PACKET_LEN    (SSH2_HEADERLEN - SSH2_PADDING_LEN)
#define SSH2_PACKET_MAX_SIZE    (256 * 1024)
#define SSH_PACKET_MAX_SIZE 35000
#define SSH_MAX_BANNER_LEN 255
#define SSH2_COOKIE_SIZE 16
#define NUM_KEXINIT_LISTS 10
#define SSH_MIN_BANNER_LEN 9 //SSH-2.0-*\n
#define SSH1_KEYX_MIN_SIZE (4 + 8 + 1) // length + padding + message

#pragma pack(1)
struct SSHKeyString
{
    uint32_t len;
    uint8_t data;
};

struct SSHMsg
{
    uint32_t len;
    uint8_t plen;
    uint8_t code;
};

struct SSH2KeyExchange
{
    SSHMsg msg;
    uint8_t cookie[16];
};

struct SSH1KeyExchangeV1
{
    uint32_t len;
    uint8_t code;
};

struct SSHKeyExchangeFinal
{
    uint8_t kex_pkt;
    uint32_t future;
};

struct SSH2Packet
{
    uint32_t packet_length; // Length not including this field or the mesg auth code (mac)
    uint8_t padding_length; // Length of padding section.
    uint8_t packet_data;    // Variable length packet payload + padding + MAC.
};
#pragma pack()

// SSH v1 message types (of interest)
#define SSH_MSG_V1_SMSG_PUBLIC_KEY  2
#define SSH_MSG_V1_CMSG_SESSION_KEY 3

// SSH v2 message types (of interest)
#define SSH_MSG_IGNORE      2
#define SSH_MSG_KEXINIT     20
#define SSH_MSG_NEWKEYS     21
#define SSH_MSG_KEXDH_INIT  30
#define SSH_MSG_KEXDH_REPLY 31
#define SSH_MSG_KEXDH_GEX_REQ   34
#define SSH_MSG_KEXDH_GEX_GRP   33
#define SSH_MSG_KEXDH_GEX_INIT  32
#define SSH_MSG_KEXDH_GEX_REPLY 31

// Direction of sent message.
#define SSH_DIR_FROM_SERVER (0x1)
#define SSH_DIR_FROM_CLIENT (0x2)

SSHData* get_session_data(const snort::Flow* flow);
SSHData* SetNewSSHData(snort::Packet* p);
#endif

ssh_module.h

Path = src/service_inspectors/ssh/ssh_module.h

#ifndef SSH_MODULE_H
#define SSH_MODULE_H

// Interface to the SSH service inspector

#include "framework/module.h"
#include "ssh_config.h"

#define GID_SSH 128

#define SSH_EVENT_RESPOVERFLOW  1
#define SSH_EVENT_CRC32         2
#define SSH_EVENT_SECURECRT     3
//#define SSH_EVENT_PROTOMISMATCH 4 (decommissioned)
#define SSH_EVENT_WRONGDIR      5
#define SSH_EVENT_PAYLOAD_SIZE  6
#define SSH_EVENT_VERSION       7

#define SSH_NAME "ssh"
#define SSH_HELP "ssh inspection"

namespace snort
{struct SnortConfig;

}

extern THREAD_LOCAL SshStats sshstats;
extern THREAD_LOCAL snort::ProfileStats sshPerfStats;

class SshModule : public snort::Module
{
public:
    SshModule();
    ~SshModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;

    unsigned get_gid() const override
    { return GID_SSH; }

    const snort::RuleMap* get_rules() const override;
    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile() const override;

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

    SSH_PROTO_CONF* get_data();

private:
    SSH_PROTO_CONF* conf;
};

#endif

ssh_splitter.h

Path = src/service_inspectors/ssh/ssh_splitter.h

#ifndef SSH_SPLITTER_H
#define SSH_SPLITTER_H

#include "protocols/packet.h"
#include "stream/stream_splitter.h"

#include "ssh.h"

enum SshPafState
{
    SSH_PAF_VER_EXCHANGE,
    SSH_PAF_KEY_EXCHANGE,
    SSH_PAF_ENCRYPTED
};

class SshSplitter : public snort::StreamSplitter
{
public:
    SshSplitter(bool c2s) : StreamSplitter(c2s)
    { }

    Status scan(snort::Packet*, const uint8_t* data, uint32_t len,
        uint32_t flags, uint32_t* fp) override;

    bool is_paf() override
    {
        return true;
    }

private:
    Status ssh2_key_exchange_scan(const uint8_t* data, uint32_t len,
        uint32_t* fp, uint32_t& remain_bytes);
    Status ssh2_scan(SSHData* sessp, const uint8_t* data, uint32_t len,
        uint32_t flags, uint32_t* fp);

    SshPafState state = SSH_PAF_VER_EXCHANGE;
    uint32_t client_remain_bytes = 0;
    uint32_t server_remain_bytes = 0;
};
#endif

service_inspectors/ssl/

This directory contains all files related to SSL inspector. The SSL inspector inspects stream reassembled SSL and TLS traffic and optionally determines if and when to stop inspection of it.

Typically, SSL is used over port 443 as HTTPS. By enabling the SSL inspector to inspect port 443, only the SSL handshake of each connection will be inspected. Once the traffic is determined to be encrypted, no further inspection of the data on the connection is made.

Each stream reassembled packet containing SSL traffic has an unencrypted portion that provides some information about the traffic itself, and the state of the connection. SSL inspector uses this information to determine whether or not a handshake is occurring or if a handshake previously occurred.

By default, SSL inspector looks for a handshake followed by encrypted traffic traveling to both sides. If one side responds with an indication that something has failed, such as the handshake, the session is not marked as encrypted. Verifying that faultless encrypted traffic is sent from both endpoints ensures two things: the last client-side handshake packet was not crafted to evade Snort, and that the traffic is legitimately encrypted.

In some cases, especially when packets may be missed, the only observed response from one endpoint will be TCP ACKs. Therefore, if a user knows that server-side encrypted data can be trusted to mark the session as encrypted, the user should use the trustservers option.

SSL inspector also inspects the heartbeat records and identifies the heartbleed evasion.

ssl_config.h

Path = src/service_inspectors/ssl/ssl_config.h

#ifndef SSL_CONFIG_H
#define SSL_CONFIG_H

// Configuration for SSL service inspector

#include "framework/counts.h"

struct SSL_PROTO_CONF
{
    bool trustservers;
    int max_heartbeat_len;
};

struct SslStats
{
    PegCount packets;
    PegCount decoded;
    PegCount hs_chello;
    PegCount hs_shello;
    PegCount hs_cert;
    PegCount hs_sdone;
    PegCount hs_ckey;
    PegCount hs_skey;
    PegCount cipher_change;
    PegCount hs_finished;
    PegCount capp;
    PegCount sapp;
    PegCount alerts;
    PegCount unrecognized;
    PegCount completed_hs;
    PegCount bad_handshakes;
    PegCount stopped;
    PegCount disabled;
    PegCount concurrent_sessions;
    PegCount max_concurrent_sessions;
};

extern const PegInfo ssl_peg_names[];
extern THREAD_LOCAL SslStats sslstats;

#endif

ssl_flow_data.h

Path = src/service_inspectors/ssl/ssl_flow_data.h

#ifndef SSL_FLOW_DATA_H
#define SSL_FLOW_DATA_H

#include "flow/flow_data.h"

#define GID_SSL 137

#define     SSL_INVALID_CLIENT_HELLO               1
#define     SSL_INVALID_SERVER_HELLO               2
#define     SSL_ALERT_HB_REQUEST                   3
#define     SSL_ALERT_HB_RESPONSE                  4

struct SSLData
{
    uint32_t ssn_flags;
    uint16_t partial_rec_len[4];
};

namespace snort
{
    class Flow;
}

class SO_PUBLIC SslBaseFlowData : public snort::FlowData
{
public:
    SslBaseFlowData() : snort::FlowData(inspector_id) {}

    virtual SSLData& get_session() = 0;

public:
    static SSLData* get_ssl_session_data(snort::Flow* flow);
    static unsigned get_ssl_inspector_id() { return inspector_id; }

protected:
    static void assign_ssl_inspector_id(unsigned u) { inspector_id = u; }

private:
    static unsigned inspector_id;
};

#endif

ssl_inspector.h

Path = src/service_inspectors/ssl/ssl_inspector.h

#ifndef SSL_INSPECTOR_H
#define SSL_INSPECTOR_H

// Implementation header with definitions, datatypes and flowdata class for SSL service inspector.

#include "flow/flow.h"
#include "ssl_flow_data.h"

class SslFlowData : public SslBaseFlowData
{
public:
    SslFlowData();
    ~SslFlowData() override;

    static void init()
    { assign_ssl_inspector_id(snort::FlowData::create_flow_data_id()); }

    SSLData& get_session() override
    { return session; }

public:
    struct {
        bool orig_flag : 1;
        bool switch_in : 1;
    } finalize_info;

private:
    SSLData session;
};

#endif

ssl_module.h

Path = src/service_inspectors/ssl/ssl_module.h

#ifndef SSL_MODULE_H
#define SSL_MODULE_H

// Interface to the SSL service inspector

#include "framework/module.h"
#include "ssl_config.h"
#include "ssl_flow_data.h"

#define SSL_NAME "ssl"
#define SSL_HELP "ssl inspection"

namespace snort
{
struct SnortConfig;
}

extern THREAD_LOCAL snort::ProfileStats sslPerfStats;

class SslModule : public snort::Module
{
public:
    SslModule();
    ~SslModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;

    unsigned get_gid() const override
    { return GID_SSL; }

    const snort::RuleMap* get_rules() const override;
    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile() const override;

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

    SSL_PROTO_CONF* get_data();

private:
    SSL_PROTO_CONF* conf;
};

#endif

ssl_splitter.h

Path = src/service_inspectors/ssl/ssl_splitter.h

#ifndef SSL_SPLITTER_H
#define SSL_SPLITTER_H

// Protocol aware flushing for SSL
// TLSPlaintext records are flushed when end-of-record meets end-of segment
// The splitter supports both sslv2 and sslv3 record format,
// it starts by checking the first byte, if it is a valid sslv3 content type,
// mark the session as sslv3; else if the MSB bit was set, marks it as sslv2,
// if this bit is not set, yet the session was marked sslv2 from prior detection,
// continue as sslv2

#include "stream/stream_splitter.h"

// Enumerations for PAF states
enum SslPafStates
{
    SSL_PAF_STATES_START = 0, //start, detect the ssl version, sslv3 type or sslv2 byte-0
    SSL_PAF_STATES_VER_MJR, // version major
    SSL_PAF_STATES_VER_MNR, // version minor
    SSL_PAF_STATES_LEN1,    // length byte-0
    SSL_PAF_STATES_LEN2,    // length byte-1
    SSL_PAF_STATES_DATA,    // fragment
    SSL_PAF_STATES_LEN2_V2, // sslv2, length byte-1
    SSL_PAF_STATES_PAD_V2,  // sslv2, padding byte if needed
};

class SslSplitter : public snort::StreamSplitter
{
public:
    SslSplitter(bool c2s);

    Status scan(snort::Packet*, const uint8_t* data, uint32_t len,
        uint32_t flags, uint32_t* fp) override;

    bool is_paf() override
    {
        return true;
    }

private:
    SslPafStates paf_state;
    uint16_t remain_len;
    uint8_t len_bytes[2]; // temporary buffer to hold 2-byte length field
    bool is_sslv2;
};

#endif

service_inspectors/wizard/

The wizard uses hexes, spells and curses to determine the most likely service on a flow. It does not determine the service with certainty; that is the job of the service inspector or appId. The goal is to get the most likely service inspector engaged as quickly as possible.

The wizard is detached from the flow upon finding a match or finding that there is no possible match.

Hexes and spells differ in the following aspects:

  • spells allow wildcards matching any number of consecutive characters whereas hexes allow a single wild char.

  • spells are case insensitive whereas hexes are case sensitive.

  • spells automatically skip leading whitespace (at very start of flow).

To match "*" symbol in traffic, put "**" in a spell.

For example: traffic "* OK" will match the pattern "** OK".

A series of asterisks is matched from left to right. * is seen as "*<glob>".

Concepts

  • MagicPage - leaf of a trie. Represents a symbol of a pattern.

  • MagicBook - trie itself. Represents a set of patterns for the wizard instance.

    • SpellBook - MagicBook implementation for spells.

    • HexBook - MagicBook implementation for hexes.

  • MagicSplitter - object related to a stream. Applies wizard logic to a stream.

  • Wand - contains state of wizard patterns for a stream.

  • CurseDetails - settings of a curse. Contains identifiers and algorithm.

  • CurseTracker - state of a curse.

  • CurseBook - contains all configured curses.

  • CurseServiceTracker - instance of a curse. Contains settings and state.

MagicSplitter

For TCP, the wizard uses a stream splitter to examine the in order data as it becomes available. If the splitter finds a match, it sets the service on the flow, which will result in a reevaluation of the bindings. If a service inspector is bound, its splitter is activated and the stream is rewound to the start.

Each flow contains two MagicSplitter objects: client-to-server and server-to-client. Each MagicSplitter contains Wand that stores pointers unique for the flow:

  1. MagicPage of Hex

  2. MagicPage of Spell

  3. Vector of all curses

Where 1 and 2 - point to the current page in pattern.

Spell matching algorithm

The spell matching algorithm is defined in SpellBook::find_spell() method. In general, MagicPage::next array is an alphabet (ASCII table), each element of which can exist or be absent. Thus, if an element exists in the position of a certain symbol, it means that there is a pattern with such a sequence of symbols.

Example:
    User configured only one pattern: "ABC"
    MagicPage(root)::next - all elements beside (int)A is nullptr.
    MagicPage(A)::next - all elements beside (int)B is nullptr.
    MagicPage(B)::next - all elements beside (int)C is nullptr.

Wizard iterates over the data from begin to end, checking at each iteration if there is a transition from the current character of the pattern to the next character of the data.

MagicPage::any reflects a glob (wildcard). If wizard transitioned to a glob of the pattern, a loop is started, in which wizard is trying to match the pattern from the current symbol of the data. If it failed to match the pattern from the current symbol of the data, it moves to the next symbol and tries again, and so either until it matches the pattern or the data runs out.

MagicPage::value is not empty only in those positions that are the ends of some pattern. Thus, if, after a complete pass through the data, the wizard have reached a position in which this field is not empty, means that it has matched the pattern.

It should be mentioned that the matching of spells is case-independent, this is implemented by converting each character to an uppercase.

Due to the fact that we want to be able to match patterns in data split into several packets, wizard saves the position of the glob into the SpellBook::glob, which is then saved to the MagicSplitter::bookmark local for the flow.

Hex matching algorithm

The algorithm for matching hexes is defined in HexBook::find_spell() and identical to the algorithm for spells, but lacks:

1) converting to an uppercase, since hexes work with raw data;
2) loops for working with the glob, since glob in hexes replaces exactly one symbol;
3) saving the position of the glob between packets.

TCP traffic processing

Execution starts from the MagicSplitter::scan().

Since we want to be able to match patterns between packets in a stream, wizard need to save the state of the pattern at the end of the processing of a particular packet. The state of the pattern is saved in the MagicSplitter::wand. However, for spells, it needs to keep the presence of a glob between packages. This is implemented with MagicSplitter::bookmark.

Spells, hexes and curses are called inside the Wizard::cast_spell(). There wizard determines the search depth and sequentially calls the processing methods.

If wizard matched the pattern in the Wizard::cast_spell(), it increments tcp_hits. If it didn’t, then it checks whether it reached the limit of max_search_depth. If wizard has reached the limit of max_search_depth and has’t matched a pattern, then it nullifies Wand::spell and Wand::hex, thus further in Wizard::finished() it’ll know that this flow can be abandoned and raise tcp_misses by 1.

UDP traffic processing

Way of processing UDP is similar to TCP but has some differences:

  1. Instead MagicSplitter::scan(), processing starts from Wizard::eval();

  2. Wizard processes only the first packet of UDP "meta-flow", so for every packet amount of previously processed bytes sets at 0;

  3. There isn’t any bookmark - UDP doesn’t support wildcard over several packets.

  4. The wizard don’t need to check Wizard::finished(), because it processes only the first packet of UDP "meta-flow". So, if it hasn’t matched anything in Wizard::cast_spell(), it increments udp_misses and unbinds itself from the flow.

Additional info

Every flow gets a context (in MagicSplitter), where wizard stores flow’s processing state. Each flow is processed independently from others.

Currently wizard cannot roll back on the pattern, so if it reaches a certain symbol of the pattern, it cannot go back. In some cases this will lead to the fact that the pattern that could be matched will not be matched.

For example:
    Patterns: "foobar", "foo*"
    Content: "foobaz"
Unfortunately, none of the available patterns will match in such case.
This is due to the fact that the symbols have a higher priority than
the glob. So from the MagicPage(O) wizard will transit to the MagicPage(B)
of the "foobar" pattern and will not process glob. Further, in MagicPage(А)::next[]
it will not find MagicPage by the symbol "z" and will consider the pattern unmatched.

Binary protocols are difficult to match with just a short stream prefix. For example suppose one has the pattern "0x12 ?" and another has "? 0x34". A match on the first doesn’t preclude a match on the second. The current implementation disregards this possibility and takes the first match.

Having the various service inspectors provide the patterns was rejected because it would have made it difficult to swap out the wizard with a new and different implementation and different pattern logic and syntax. Encapsulating everything in the wizard allows the patterns to be easily tweaked as well.

Curses are presently used for binary protocols that require more than pattern matching. They use internal algorithms to identify services, implemented with custom FSMs.

curse_book.h

Path = src/service_inspectors/wizard/curse_book.h

#ifndef CURSE_BOOK_H
#define CURSE_BOOK_H

#include <cstdint>
#include <string>
#include <vector>

#include "dce_curse.h"
#include "mms_curse.h"
#include "s7commplus_curse.h"
#include "ssl_curse.h"

class CurseTracker
{
public:
    DceTracker dce;
    MmsTracker mms;
    S7commplusTracker s7commplus;
    SslTracker ssl;
};

typedef bool (* curse_alg)(const uint8_t* data, unsigned len, CurseTracker*);

struct CurseDetails
{
    std::string name;
    const char* service;
    curse_alg alg;
    bool is_tcp;
};

class CurseBook
{
public:
    bool add_curse(const char* service);
    const std::vector<const CurseDetails*>& get_curses(bool tcp) const;

private:
    std::vector<const CurseDetails*> tcp_curses;
    std::vector<const CurseDetails*> non_tcp_curses;
    static std::vector<CurseDetails> curse_map;

    static bool dce_udp_curse(const uint8_t* data, unsigned len, CurseTracker*);
    static bool dce_tcp_curse(const uint8_t* data, unsigned len, CurseTracker*);
    static bool dce_smb_curse(const uint8_t* data, unsigned len, CurseTracker*);
    static bool mms_curse(const uint8_t* data, unsigned len, CurseTracker*);
    static bool s7commplus_curse(const uint8_t* data, unsigned len, CurseTracker*);
#ifdef CATCH_TEST_BUILD
public:
#endif
    static bool ssl_v2_curse(const uint8_t* data, unsigned len, CurseTracker*);
};

#endif

dce_curse.h

Path = src/service_inspectors/wizard/dce_curse.h

#ifndef DCE_CURSE_H
#define DCE_CURSE_H

// DCE curse helps determine if the traffic being processed is DCERPC

#include <cstdint>

enum DCE_State
{
    DCE_STATE__0 = 0,
    DCE_STATE__1,
    DCE_STATE__2,
    DCE_STATE__3,
    DCE_STATE__4,
    DCE_STATE__5,
    DCE_STATE__6,
    DCE_STATE__7,
    DCE_STATE__8,
    DCE_STATE__9,
    DCE_STATE__10
};

class DceTracker
{
public:
    DCE_State state = DCE_State::DCE_STATE__0;
    uint32_t helper;
};

#endif

magic.h

Path = src/service_inspectors/wizard/magic.h

#ifndef MAGIC_H
#define MAGIC_H

#include <cassert>
#include <cstdint>
#include <string>
#include <vector>

class MagicBook;

struct MagicPage
{
    std::string key;
    const char* value = nullptr;

    MagicPage* next[256];
    MagicPage* any;

    const MagicBook& book;

    MagicPage(const MagicBook&);
    ~MagicPage();
};

typedef std::vector<uint16_t> HexVector;

// MagicBook is a set of MagicPages implementing a trie
class MagicBook
{
public:
    virtual ~MagicBook();

    MagicBook(const MagicBook&) = delete;
    MagicBook& operator=(const MagicBook&) = delete;

    enum class ArcaneType
    {
        TCP,
        UDP,
        ANY,
        MAX = ANY
    };

    virtual bool add_spell(const char* key, const char*& val, ArcaneType proto) = 0;
    virtual const char* find_spell(const uint8_t* data, unsigned len, const MagicPage*& p,
        const MagicPage*& bookmark) const;

    const MagicPage* page1(ArcaneType proto) const
    {
        assert(proto < ArcaneType::MAX);
        return &root[(int)proto];
    }

protected:
    MagicBook();
    MagicPage* root;

    MagicPage* get_root(ArcaneType proto) const
    {
        assert(proto < ArcaneType::MAX);
        return &root[(int)proto];
    }

    virtual const MagicPage* find_spell(const uint8_t*, unsigned,
        const MagicPage*, unsigned, const MagicPage*&) const = 0;
};

//-------------------------------------------------------------------------
// spells - a sequence of case insensitive text strings with wild cards
// designated by * (indicating any number of arbitrary bytes)
//-------------------------------------------------------------------------

class SpellBook : public MagicBook
{
public:
    SpellBook();

    bool add_spell(const char*, const char*&, ArcaneType) override;

private:
    bool translate(const char*, HexVector&);
    void add_spell(const char*, const char*, const HexVector&, unsigned, MagicPage*);
    const MagicPage* find_spell(const uint8_t*, unsigned, const MagicPage*, unsigned,
        const MagicPage*&) const override;
};

//-------------------------------------------------------------------------
// hexes - a sequence of pipe delimited hex, text literals, and wild chars
// designated by '?' (indicating one arbitrary byte)
//-------------------------------------------------------------------------

class HexBook : public MagicBook
{
public:
    HexBook() = default;

    bool add_spell(const char*, const char*&, ArcaneType) override;

private:
    bool translate(const char*, HexVector&);
    void add_spell(const char*, const char*, const HexVector&, unsigned, MagicPage*);
    const MagicPage* find_spell(const uint8_t*, unsigned, const MagicPage*, unsigned,
        const MagicPage*&) const override;
};

#endif

mms_curse.h

Path = src/service_inspectors/wizard/mms_curse.h

#ifndef MMS_CURSE_H
#define MMS_CURSE_H

// MMS curse provides the ability to determine if the traffic being processed
// conforms to the Manufacturing Message Specification (MMS) traffic defined
// within the IEC-61850 family of protocols

enum MMS_State
{
    MMS_STATE__TPKT_VER = 0,
    MMS_STATE__TPKT_RES,
    MMS_STATE__TPKT_LEN1,
    MMS_STATE__TPKT_LEN2,
    MMS_STATE__COTP_LEN,
    MMS_STATE__COTP_PDU,
    MMS_STATE__COTP_TPDU_NUM,
    MMS_STATE__OSI_SESSION_SPDU,
    MMS_STATE__MMS,
    MMS_STATE__FOUND,
    MMS_STATE__SEARCH,
    MMS_STATE__NOT_FOUND,
};

class MmsTracker
{
public:
    MMS_State state = MMS_State::MMS_STATE__TPKT_VER;
    MMS_State last_state = MMS_State::MMS_STATE__TPKT_VER;
};

#endif

s7commplus_curse.h

Path = src/service_inspectors/wizard/s7commplus_curse.h

#ifndef S7COMMPLUS_CURSE_H
#define S7COMMPLUS_CURSE_H

// s7commplus_curse provides the ability to determine if the traffic being processed
// conforms to the S7CommPlus protocol used in select Siemens devices

#include <cstdint>

enum S7commplus_State
{
    S7COMMPLUS_STATE__TPKT_VER = 0,
    S7COMMPLUS_STATE__TPKT_RES,
    S7COMMPLUS_STATE__TPKT_LEN1,
    S7COMMPLUS_STATE__TPKT_LEN2,
    S7COMMPLUS_STATE__COTP_LEN,
    S7COMMPLUS_STATE__COTP_PDU,
    S7COMMPLUS_STATE__COTP_TPDU_NUM,
    S7COMMPLUS_STATE__PROTO_ID,
    S7COMMPLUS_STATE__PDU_TYPE,
    S7COMMPLUS_STATE__DATALENGTH_1,
    S7COMMPLUS_STATE__DATALENGTH_2,
    S7COMMPLUS_STATE__OPCODE,
    S7COMMPLUS_STATE__RES_1,
    S7COMMPLUS_STATE__RES_2,
    S7COMMPLUS_STATE__FUNCTION_1,
    S7COMMPLUS_STATE__FUNCTION_2,
    S7COMMPLUS_STATE__FOUND,
    S7COMMPLUS_STATE__SEARCH,
    S7COMMPLUS_STATE__NOT_FOUND,
};

class S7commplusTracker
{
public:
    S7commplus_State state = S7commplus_State::S7COMMPLUS_STATE__TPKT_VER;
    S7commplus_State last_state = S7commplus_State::S7COMMPLUS_STATE__TPKT_VER;
    uint16_t func = 0;
};

#endif

ssl_curse.h

Path = src/service_inspectors/wizard/ssl_curse.h

#ifndef SSL_CURSE_H
#define SSL_CURSE_H

// SSL curse helps determine if the traffic being processed is SSL

enum SSL_State
{
    SSL_STATE__BYTE_0_LEN_MSB = 0,
    SSL_STATE__BYTE_1_LEN_LSB,
    SSL_STATE__BYTE_2_CLIENT_HELLO,
    SSL_STATE__BYTE_3_MAX_MINOR_VER,
    SSL_STATE__BYTE_4_V3_MAJOR,
    SSL_STATE__BYTE_5_SPECS_LEN_MSB,
    SSL_STATE__BYTE_6_SPECS_LEN_LSB,
    SSL_STATE__BYTE_7_SSNID_LEN_MSB,
    SSL_STATE__BYTE_8_SSNID_LEN_LSB,
    SSL_STATE__BYTE_9_CHLNG_LEN_MSB,
    SSL_STATE__BYTE_10_CHLNG_LEN_LSB,
    SSL_STATE__SSL_FOUND,
    SSL_STATE__SSL_NOT_FOUND
};

class SslTracker
{
public:
    SSL_State state = SSL_STATE__BYTE_0_LEN_MSB;
    unsigned total_len;
    unsigned ssnid_len;
    unsigned specs_len;
    unsigned chlng_len;
};

#endif

wiz_module.h

Path = src/service_inspectors/wizard/wiz_module.h

#ifndef WIZ_MODULE_H
#define WIZ_MODULE_H

// wizard management interface

#include "framework/module.h"

#include "magic.h"

#define WIZ_NAME "wizard"
#define WIZ_HELP "inspector that implements port-independent protocol identification"

namespace snort
{
class Trace;
}

extern const PegInfo wiz_pegs[];
extern THREAD_LOCAL struct WizStats tstats;
extern THREAD_LOCAL snort::ProfileStats wizPerfStats;
extern THREAD_LOCAL const snort::Trace* wizard_trace;

class MagicBook;
class CurseBook;

class WizardModule : public snort::Module
{
public:
    WizardModule();
    ~WizardModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile() const override;

    MagicBook* get_book(bool c2s, bool hex);
    CurseBook* get_curse_book();

    uint16_t get_max_search_depth() const
    { return max_search_depth; }

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

    void set_trace(const snort::Trace*) const override;
    const snort::TraceOption* get_trace_options() const override;

private:
    std::string service;
    std::vector<std::string> c2s_patterns;
    std::vector<std::string> s2c_patterns;
    bool c2s = false;

    MagicBook* c2s_hexes = nullptr;
    MagicBook* s2c_hexes = nullptr;

    MagicBook* c2s_spells = nullptr;
    MagicBook* s2c_spells = nullptr;

    CurseBook* curses = nullptr;
    uint16_t max_search_depth = 0;

    MagicBook::ArcaneType proto = MagicBook::ArcaneType::MAX;
};

#endif

sfip/

This directory contains IP and IP variables related helper functions.

  • Supports basic IP operations and helper functions

  • Provides variable table for storing and looking up variables

  • Supports basic IP variable operations and manages a list of IP variables through variable table

sf_cidr.h

Path = src/sfip/sf_cidr.h

#ifndef SF_CIDR_H
#define SF_CIDR_H

#include "sfip/sf_ip.h"

namespace snort
{
/*
 * NOTE: As much as I'd love to make this a subclass of SfIp, member layout
 * is undefined for POD inheritance.
 */

struct SO_PUBLIC SfCidr
{
    /*
     * Constructors
     */
    SfCidr() = default;

    /*
     * Modifiers (incl. convenience ones that delegate to addr)
     */
    void clear();
    void set(const SfCidr& src);
    void set(const SfIp& src);
    SfIpRet set(const void* src, int fam);
    SfIpRet set(const char* src);
    void set_bits(uint16_t new_bits);

    /*
     * Accessors (incl. convenience ones that delegate to addr)
     */
    const SfIp* get_addr() const;
    uint16_t get_family() const;
    uint16_t get_bits() const;
    bool is_set() const;

    /*
     * Containment checks
     */
    bool fast_cont4(const SfIp& ip) const;
    bool fast_cont6(const SfIp& ip) const;
    SfIpRet contains(const SfIp* ip) const;

    const char* ntop(SfIpString) const;
    SfIpRet compare(const SfCidr&) const;

private:
    SfIp addr;
    uint16_t bits = 0;
} __attribute__((__packed__));


inline void SfCidr::clear()
{
    addr.clear();
    bits = 0;
}

inline void SfCidr::set(const SfCidr& src)
{
    addr = src.addr;
    bits = src.bits;
}

inline void SfCidr::set(const SfIp& src)
{
    addr = src;
    bits = 128;
}

inline SfIpRet SfCidr::set(const void* src, int fam)
{
    SfIpRet ret = addr.set(src, fam);
    if (ret != SFIP_SUCCESS)
        return ret;
    bits = 128;
    return SFIP_SUCCESS;
}

inline void SfCidr::set_bits(uint16_t new_bits)
{
    if (new_bits > 128)
        return;
    bits = new_bits;
}

inline const SfIp* SfCidr::get_addr() const
{
    return &addr;
}

inline uint16_t SfCidr::get_family() const
{
    return addr.get_family();
}

inline uint16_t SfCidr::get_bits() const
{
    return bits;
}

inline bool SfCidr::is_set() const
{
    return (addr.is_set() ||
            ((addr.get_family() == AF_INET || addr.get_family() == AF_INET6) &&
             bits != 128));
}

inline bool SfCidr::fast_cont4(const SfIp& ip) const
{
    uint32_t shift = 128 - bits;
    uint32_t needle = ntohl(ip.get_ip4_value());
    uint32_t haystack = ntohl(addr.get_ip4_value());

    if (haystack == 0)
        return true;

    needle >>= shift;
    needle <<= shift;

    return haystack == needle;
}

inline bool SfCidr::fast_cont6(const SfIp& ip) const
{
    uint32_t needle;
    int words = bits / 32;
    int shift, i;

    for (i = 0; i < words; i++)
    {
        if (addr.get_ip6_ptr()[i] != ip.get_ip6_ptr()[i])
            return false;
    }

    shift = 32 - (bits % 32);
    if (shift == 32)
        return true;

    needle = ntohl(ip.get_ip6_ptr()[i]);

    needle >>= shift;
    needle <<= shift;

    return ntohl(addr.get_ip6_ptr()[i]) == needle;
}

inline const char* SfCidr::ntop(SfIpString ip_str) const
{
    return addr.ntop(ip_str);
}

inline SfIpRet SfCidr::compare(const SfCidr& cidr2) const
{
    SfIpRet ret = addr.compare(*cidr2.get_addr());
    if(SFIP_EQUAL == ret)
    {
        if(bits < cidr2.get_bits()) return SFIP_LESSER;
        if(bits > cidr2.get_bits()) return SFIP_GREATER;
    }
    return ret;
}
}
#endif

sf_ip.h

Path = src/sfip/sf_ip.h

#ifndef SF_IP_H
#define SF_IP_H

#include <arpa/inet.h>
#include <sys/socket.h>

#include <cassert>
#include <cstring>
#include <sstream>

#include "main/snort_types.h"
#include "sfip/sf_returns.h"
#include "utils/cpp_macros.h"

namespace snort
{
using SfIpString = char[INET6_ADDRSTRLEN];

// INET6_ADDRSTRLEN without IPv4-mapped IPv6
#define MAX_INET6_STRLEN_NO_IPV4_MAP 40

struct SfCidr;

struct SO_PUBLIC SfIp
{
    /*
     * Constructors
     */
    SfIp() = default;
    SfIp(const void* src, int fam) { set(src, fam); }

    /*
     * Modifiers
     */
    void clear();
    SfIpRet set(const char* src, uint16_t* srcBits = nullptr);
    /* Sets to a raw source IP (4 or 16 bytes, according to family) */
    SfIpRet set(const void* src, int fam);
    /* Sets to a raw source IP, source must be a 128 bit IPv6 (detects IPv4 mapped IPv6)
     * This is specifically for conversion of DAQ_FlowStats_t ipv4 mapped ipv6 addresses */
    SfIpRet set(const void* src);
    /* Converts string IP format to an array of values. Also checks IP address format. */
    SfIpRet pton(const int fam, const char* ip);

    /*
     * Accessors
     */
    uint16_t get_family() const;
    uint32_t get_ip4_value() const;
    const uint32_t* get_ip4_ptr() const;
    const uint32_t* get_ip6_ptr() const;
    const uint32_t* get_ptr() const;
    bool is_set() const;
    bool is_ip6() const;
    bool is_ip4() const;

    /*
     *  Comparison functions
     */
    bool equals(const SfIp& rhs, bool match_unset = true) const;
    bool less_than(const SfIp& rhs) const;
    bool greater_than(const SfIp& rhs) const;
    SfIpRet compare(const SfIp& ip2, bool match_unset = true) const;
    bool fast_eq4(const SfIp& ip2) const;
    bool fast_lt6(const SfIp& ip2) const;
    bool fast_gt6(const SfIp& ip2) const;
    bool fast_eq6(const SfIp& ip2) const;
    bool fast_equals_raw(const SfIp& ip2) const;
    bool operator==(const SfIp& ip2) const;
    bool operator < (const SfIp& rhs) const;

    /*
     * Miscellaneous
     */
    /* Returns true if the IPv6 address appears mapped. */
    bool is_mapped() const;
    bool is_loopback() const;
    bool is_private() const;

    const char* ntop(char* buf, int bufsize) const;
    const char* ntop(SfIpString) const;

    void obfuscate(const SfCidr* ob);

    static bool test_features;

private:
    int cidr_mask(int val);
    bool _is_equals(const SfIp& rhs) const;
    bool _is_lesser(const SfIp& rhs) const;
    SfIpRet _ip6_cmp(const SfIp& ip2) const;

    /* Keep this union at the offset of 0 in this struct.
     * If it's not it introduces possibility of pointer misalignment when
     * pointing to its elements. This layout makes it safe to suppress this
     * warning, which is done for get_ip4_ptr(), get_ip6_ptr(), and get_ptr().
     */
    union
    {
        uint8_t ip8[16];
        uint16_t ip16[8];
        uint32_t ip32[4];
    };
    int16_t family;
} __attribute__((__packed__));


/*
 * Member function definitions
 */

inline void SfIp::clear()
{
    family = 0;
    ip32[0] = ip32[1] = ip32[2] = ip32[3] = 0;
}

inline uint16_t SfIp::get_family() const
{
    return family;
}

inline uint32_t SfIp::get_ip4_value() const
{
    return ip32[3];
}

/* Safe to ignore because ip32 is at the offset of 0 in SfIp */
PACKED_MEMBER_ADDR_BEGIN

inline const uint32_t* SfIp::get_ip4_ptr() const
{
    return &ip32[3];
}

inline const uint32_t* SfIp::get_ip6_ptr() const
{
    return ip32;
}

inline const uint32_t* SfIp::get_ptr() const
{
    if (is_ip4())
        return &ip32[3];
    return ip32;
}

PACKED_MEMBER_ADDR_END

inline bool SfIp::is_set() const
{
    return ((family == AF_INET && ip32[3]) ||
            (family == AF_INET6 &&
             (ip32[0] || ip32[1] || ip32[3] || ip16[4] ||
              (ip16[5] && ip16[5] != 0xffff))));
}

inline bool SfIp::is_ip6() const
{
    return family == AF_INET6;
}

inline bool SfIp::is_ip4() const
{
    return family == AF_INET;
}

inline bool SfIp::is_loopback() const
{
    /* Check the first 80 bits in an IPv6 address, and
        verify they're zero.  If not, it's not a loopback */
    if (ip32[0] || ip32[1] || ip16[4])
        return false;

    if (ip16[5] == 0xffff)
    {
        /* ::ffff:127.0.0.0/104 is IPv4 loopback mapped over IPv6 */
        return (ip8[12] == 0x7f);
    }

    if (!ip16[5])
    {
        /* ::7f00:0/104 is ipv4 compatible ipv6
           ::1 is the IPv6 loopback */
        return (ip32[3] == htonl(0x1) || ip8[12] == 0x7f);
    }

    return false;
}

inline bool SfIp::is_private() const
{
    /* Check the first 80 bits in an IPv6 address, and
        verify they're zero.  If not, it's not a loopback. */
    if (ip32[0] || ip32[1] || ip16[4])
        return false;

    /* (Mapped) v4 private addresses */
    if (ip16[5] == 0xffff)
    {
        /*
         * 10.0.0.0        -   10.255.255.255  (10/8 prefix)
         * 172.16.0.0      -   172.31.255.255  (172.16/12 prefix)
         * 192.168.0.0     -   192.168.255.255 (192.168/16 prefix)
         */
        return ( (ip8[12] == 10)
               || ((ip8[12] == 172) && ((ip8[13] & 0xf0) == 16))
               || ((ip8[12] == 192) && (ip8[13] == 168)) );
    }

    /* Check if the 3rd 32-bit int is zero */
    if (!ip16[5])
    {
        /* ::ipv4 compatible ipv6
           ::1 is the IPv6 loopback */
        return ( (ip8[12] == 10)
               || ((ip8[12] == 172) && ((ip8[13] & 0xf0) == 16))
               || ((ip8[12] == 192) && (ip8[13] == 168))
               || (ip32[3] == htonl(0x1)) );
    }

    return false;
}

inline bool SfIp::_is_equals(const SfIp& rhs) const
{
    if (is_ip4())
    {
        return (rhs.is_ip4()) &&
               (ip32[3] == rhs.ip32[3]);
    }
    else if (is_ip6())
    {
        return (rhs.is_ip6()) &&
               (ip32[0] == rhs.ip32[0]) &&
               (ip32[1] == rhs.ip32[1]) &&
               (ip32[2] == rhs.ip32[2]) &&
               (ip32[3] == rhs.ip32[3]);
    }
    return false;
}

inline bool SfIp::_is_lesser(const SfIp& rhs) const
{
    if (is_ip4() && rhs.is_ip4())
        return ntohl(ip32[3]) < ntohl(rhs.ip32[3]);
    uint32_t a = ntohl(ip32[0]);
    uint32_t b = ntohl(rhs.ip32[0]);
    if (a != b)
        return a < b;
    a = ntohl(ip32[1]);
    b = ntohl(rhs.ip32[1]);
    if (a != b)
        return a < b;
    a = ntohl(ip32[2]);
    b = ntohl(rhs.ip32[2]);
    if (a != b)
        return a < b;
    a = ntohl(ip32[3]);
    b = ntohl(rhs.ip32[3]);
    return a < b;
}

inline bool SfIp::equals(const SfIp& rhs, bool match_unset) const
{
    if (!is_set() || !rhs.is_set())
        return match_unset;

    return _is_equals(rhs);
}

inline bool SfIp::less_than(const SfIp& rhs) const
{
    // I'm copying and pasting.  Don't ask me why this is different then sfip_equals
    if (!is_set() || !rhs.is_set())
        return false;

    return _is_lesser(rhs);
}

inline bool SfIp::greater_than(const SfIp& rhs) const
{
    // I'm copying and pasting.  Don't ask me why this is different then sfip_equals
    if (!is_set() || !rhs.is_set())
        return false;

    return rhs._is_lesser(*this);
}

/* Support function for SfIp::compare() */
inline SfIpRet _ip4_cmp(uint32_t ip1, uint32_t ip2)
{
    uint32_t hip1 = htonl(ip1);
    uint32_t hip2 = htonl(ip2);
    if (hip1 < hip2)
        return SFIP_LESSER;
    if (hip1 > hip2)
        return SFIP_GREATER;
    return SFIP_EQUAL;
}

/* Support function for SfIp::compare() */
inline SfIpRet SfIp::_ip6_cmp(const SfIp& ip2) const
{
    SfIpRet ret;

    if ( (ret = _ip4_cmp(ip32[0], ip2.ip32[0])) != SFIP_EQUAL)
        return ret;
    if ( (ret = _ip4_cmp(ip32[1], ip2.ip32[1])) != SFIP_EQUAL)
        return ret;
    if ( (ret = _ip4_cmp(ip32[2], ip2.ip32[2])) != SFIP_EQUAL)
        return ret;
    if ( (ret = _ip4_cmp(ip32[3], ip2.ip32[3])) != SFIP_EQUAL)
        return ret;

    return ret;
}

/*
 * Returns SFIP_LESSER, SFIP_EQUAL, SFIP_GREATER, if this is less than, equal to,
 * or greater than ip2.  In the case of mismatched families, the IPv4 address
 * is converted to an IPv6 representation.
 * To support existing Snort code, an unset IP is considered to match anything
 * unless 'match_unset' is set to false.
 */
inline SfIpRet SfIp::compare(const SfIp& ip2, bool match_unset) const
{
    if (!is_set() || !ip2.is_set())
    {
        if (match_unset)
            return SFIP_EQUAL;
        return SFIP_FAILURE;
    }

    if (is_ip4() && ip2.is_ip4())
        return _ip4_cmp(get_ip4_value(), ip2.get_ip4_value());

    return _ip6_cmp(ip2);
}

inline bool SfIp::fast_eq4(const SfIp& ip2) const
{
    return get_ip4_value() == ip2.get_ip4_value();
}

inline bool SfIp::fast_lt6(const SfIp& ip2) const
{
    if (ip32[0] < ip2.ip32[0])
        return true;
    else if (ip32[0] > ip2.ip32[0])
        return false;

    if (ip32[1] < ip2.ip32[1])
        return true;
    else if (ip32[1] > ip2.ip32[1])
        return false;

    if (ip32[2] < ip2.ip32[2])
        return true;
    else if (ip32[2] > ip2.ip32[2])
        return false;

    if (ip32[3] < ip2.ip32[3])
        return true;
    else if (ip32[3] > ip2.ip32[3])
        return false;

    return false;
}

inline bool SfIp::fast_gt6(const SfIp& ip2) const
{
    if (ip32[0] > ip2.ip32[0])
        return true;
    else if (ip32[0] < ip2.ip32[0])
        return false;

    if (ip32[1] > ip2.ip32[1])
        return true;
    else if (ip32[1] < ip2.ip32[1])
        return false;

    if (ip32[2] > ip2.ip32[2])
        return true;
    else if (ip32[2] < ip2.ip32[2])
        return false;

    if (ip32[3] > ip2.ip32[3])
        return true;
    else if (ip32[3] < ip2.ip32[3])
        return false;

    return false;
}

inline bool SfIp::fast_eq6(const SfIp& ip2) const
{
    if (ip32[0] != ip2.ip32[0])
        return false;
    if (ip32[1] != ip2.ip32[1])
        return false;
    if (ip32[2] != ip2.ip32[2])
        return false;
    if (ip32[3] != ip2.ip32[3])
        return false;

    return true;
}

inline bool SfIp::fast_equals_raw(const SfIp& ip2) const
{
    int f1,f2;

    f1 = family;
    f2 = ip2.family;

    if (f1 == AF_INET)
    {
        if (f2 != AF_INET)
            return false;
        if (fast_eq4(ip2))
            return true;
    }
    else if (f1 == AF_INET6)
    {
        if (f2 != AF_INET6)
            return false;
        if (fast_eq6(ip2))
            return true;
    }
    return false;
}

inline bool SfIp::operator==(const SfIp& ip2) const
{
    return fast_equals_raw(ip2);
}

inline bool SfIp::operator < (const SfIp& rhs) const
{
    return less_than(rhs);
}

/* End of member function definitions */

/* Support functions */
// note that an ip6 address may have a trailing dotted quad form
// but that it always has at least 2 ':'s; furthermore there is
// no valid ip4 format (including mask) with 2 ':'s
// we don't have to figure out if the format is entirely legal
// we just have to be able to tell correct formats apart
static inline int sfip_str_to_fam(const char* str)
{
    const char* s;
    assert(str);
    s = strchr(str, (int)':');
    if ( s && strchr(s+1, (int)':') )
        return AF_INET6;
    if ( strchr(str, (int)'.') )
        return AF_INET;
    return AF_UNSPEC;
}

static inline bool parse_ip_from_uri(const std::string& ip_str, SfIp& ip)
{
    auto host_start = ip_str.find("://");
    if ( host_start != std::string::npos )
    {
        host_start += 3;
        if ( host_start >= ip_str.size() )
            return false;
    }
    else
        host_start = 0;

    auto host_end = host_start;
    int family = sfip_str_to_fam(ip_str.c_str() + host_start);

    if ( family == AF_INET )
    {
        while ( host_end < ip_str.size() and ip_str[host_end] != ':' and ip_str[host_end] != '/' )
            ++host_end;
    }
    else if ( family == AF_INET6 )
    {
        if ( ip_str[host_start] == '[' )
        {
            ++host_start;
            ++host_end;
        }
        while ( host_end < ip_str.size() and ip_str[host_end] != ']' and ip_str[host_end] != '/')
            ++host_end;
    }
    else
        return false;

    if ( host_end <= host_start or (host_end - host_start) > MAX_INET6_STRLEN_NO_IPV4_MAP )
        return false;

    if ( host_start != 0 or host_end != ip_str.size() )
    {
        const std::string host_str = ip_str.substr(host_start, host_end - host_start);
        if ( ip.set(host_str.c_str()) != SFIP_SUCCESS )
            return false;
    }
    else if ( ip.set(ip_str.c_str()) != SFIP_SUCCESS )
        return false;

    return true;
}

SO_PUBLIC const char* sfip_ntop(const SfIp* ip, char* buf, int bufsize);

inline std::ostream& operator<<(std::ostream& os, const SfIp* addr)
{
    SfIpString str;
    return os << addr->ntop(str);
}

// FIXIT-L X This should be in utils_net if anywhere, but that makes it way
// harder to link into unit tests
SO_PUBLIC const char* snort_inet_ntop(int family, const void* ip_raw, char* buf, int bufsize);
} // namespace snort
#endif

sf_ipvar.h

Path = src/sfip/sf_ipvar.h

#ifndef SF_IPVAR_H
#define SF_IPVAR_H

// Supports basic IP variable operations
// Manages a doubly linked list of IP variables for the variable table

/* Flags */
#define SFIP_NEGATED  1
#define SFIP_ANY      2

#include <cstdint>

#include "sfip/sf_returns.h"

namespace snort
{
struct SfIp;
struct SfCidr;
}

/* A doubly linked list of SfIp objects. */
typedef struct _ip_node
{
    snort::SfCidr* ip = nullptr;
    struct _ip_node* next = nullptr;
    int flags = 0;
    int addr_flags = 0;
                    /* Flags used exclusively by Snort */
                    /* Keeping these variables separate keeps
                     * this from stepping on Snort's toes. */
                    /* Should merge them later */
} sfip_node_t;

/* An IP variable object */
struct sfip_var_t
{
    /* Linked lists.  Switch to something faster later */
    sfip_node_t* head;
    sfip_node_t* neg_head;

    /* The mode above will select whether to use the sfip_node_t linked list
     * or the IP routing table */
//    sfrt rt;

    /* Linked list of IP variables for the variable table */
    sfip_var_t* next;

    uint32_t head_count;
    uint32_t neg_head_count;
    uint32_t id;
    char* name;
    char* value;
};

/* A variable table for storing and looking up variables
   Expand later to use a faster data structure */
struct vartable_t
{
    sfip_var_t* head;
    uint32_t id;
};

/* Deep copy of src added to dst */
SfIpRet sfvar_add(sfip_var_t* dst, sfip_var_t* src);

/* Creates a new variable that is an alias of another variable
 * Does a "deep" copy so it owns it's own pointers */
sfip_var_t* sfvar_deep_copy(const sfip_var_t*);
sfip_var_t* sfvar_create_alias(const sfip_var_t* alias_from, const char* alias_to);

/* Allocates a new variable as according to "str" */
sfip_var_t* sfvar_alloc(vartable_t* table, const char* str, SfIpRet* status);

/* Makes sure there are no IP address conflicts in the variable
   Returns SFIP_CONFLICT if so */
SfIpRet sfvar_validate(sfip_var_t* var);

/* Parses an IP list described by 'str' and saves the results in 'var'. */
SfIpRet sfvar_parse_iplist(vartable_t* table, sfip_var_t* var,
    const char* str, int negation);

/* Compares two variables.  Necessary when building RTN structure */
SfIpRet sfvar_compare(const sfip_var_t* one, const sfip_var_t* two);

/* Free an allocated variable */
void sfvar_free(sfip_var_t* var);

// returns true if both args are valid and ip is contained by var
bool sfvar_ip_in(sfip_var_t* var, const snort::SfIp* ip);

#endif

sf_returns.h

Path = src/sfip/sf_returns.h

#ifndef SF_RETURNS_H
#define SF_RETURNS_H

enum SfIpRet
{
    SFIP_SUCCESS=0,
    SFIP_FAILURE,
    SFIP_LESSER,
    SFIP_GREATER,
    SFIP_EQUAL,
    SFIP_ARG_ERR,
    SFIP_CIDR_ERR,
    SFIP_INET_PARSE_ERR,
    SFIP_INVALID_MASK,
    SFIP_ALLOC_ERR,
    SFIP_CONTAINS,
    SFIP_NOT_CONTAINS,
    SFIP_DUPLICATE,         /* Tried to add a duplicate variable name to table */
    SFIP_LOOKUP_FAILURE,    /* Failed to lookup a variable from the table */
    SFIP_UNMATCHED_BRACKET, /* IP lists that are missing a closing bracket */
    SFIP_NOT_ANY,           /* For !any */
    SFIP_CONFLICT,          /* For IP conflicts in IP lists */
    SFIP_LOOKUP_UNAVAILABLE /* var table unavailable */
};

#endif

sf_vartable.h

Path = src/sfip/sf_vartable.h

#ifndef SF_VARTABLE_H
#define SF_VARTABLE_H

// Library for implementing a variable table.

#include <cstdio>

#include "sfip/sf_returns.h"

struct sfip_var_t;
struct vartable_t;

/* Allocates new variable table */
vartable_t* sfvt_alloc_table();
void sfvt_free_table(vartable_t* table);

/* Adds the variable described by "str" to the table "table" */
SfIpRet sfvt_add_str(vartable_t* table, const char* str, sfip_var_t**);
SfIpRet sfvt_define(vartable_t* table, const char* name, const char* value);

/* Adds the variable described by "str" to the variable "dst",
 * using the vartable for looking variables used within "str" */
SfIpRet sfvt_add_to_var(vartable_t* table, sfip_var_t* dst, const char* src);

/* Looks up a variable from the table using the name as the key */
sfip_var_t* sfvt_lookup_var(vartable_t* table, const char* name);

/* Prints a table's contents */
void sfvt_print(FILE* f, vartable_t* table);

#endif

sfip/test/

sfrt/

SFRT implements two different routing table lookup methods that have been adapted to return a void pointers. Any generic information may be associated with a given IP or CIDR block.

As of this writing, the method used is a multibit-trie method similar to Gupta et-al.'s DIR-n-m.

The intended use is to associate large IP blocks with specific information.

NOTE information should only move from less specific to more specific, ie:

First insert:  1.1.0.0/16  ->  some data
Second insert: 1.1.2.3     ->  some other data

As opposed to:

First insert:  1.1.2.3     ->  some other data
Second insert: 1.1.0.0/16  ->  some data

If more general information is to overwrite existing entries, the table should be free’ed and rebuilt. This is due to the difficulty of cleaning out stale entries with the current implementation. At runtime, this won’t be a significant issue since inserts should apply to specific IP addresses and not entire blocks of IPs.

Basic Implementation

The routing tables associate an index into a "data" table with each CIDR. Each entry in the data table stores a pointer to actual data. This implementation was chosen so each routing entry only needs one word to either index the data array, or point to another table.

Inserts are performed by specifying a CIDR and a pointer to its associated data. Since a new routing table entry may overwrite previous entries, a flag selects whether the insert favors the most recent or favors the most specific. Favoring most specific should be the default behavior. If the user wishes to overwrite routing entries with more general data, the table should be flushed, rather than using favor-most-recent.

Before modifying the routing or data tables, the insert function performs a lookup on the CIDR-to-be-inserted. If no entry or an entry of differing bit length is found, the data is inserted into the data table, and its index is used for the new routing table entry. If an entry is found that is as specific as the new CIDR, the index stored points to where the new data is written into the data table.

If more specific CIDR blocks overwrote the data table, then the more general routing table entries that were not overwritten will be referencing the wrong data. Alternatively, less specific entries can only overwrite existing routing table entries if favor-most-recent inserts are used.

Because there is no quick way to clean the data-table if a user wishes to use a favor-most-recent insert for more general data, the user should flush the table with sfrt_free and create one anew. Alternatively, a small memory leak occurs with the data table, as it will be storing pointers that no routing table entry cares about.

The API calls that should be used are:

  • sfrt_new - create new table

  • sfrt_insert - insert entry

  • sfrt_lookup - lookup entry

  • sfrt_free - free table

Flat Implementation

This is based on the original implementation, but using the flat segment memory. When allocating memory, it uses memory in the segment, and returns the offset. When accessing memory, it must use the base address and offset to correctly refer to it.

sfrt_flat_dir.h

Path = src/sfrt/sfrt_flat_dir.h

#ifndef SFRT_FLAT_DIR_H
#define SFRT_FLAT_DIR_H

#include "sfrt/sfrt.h"

typedef MEM_OFFSET SUB_TABLE_PTR;
typedef MEM_OFFSET ENTRIES_PTR;

struct DIR_Entry
{
    MEM_OFFSET value;
    uint8_t length;
};

/*******************************************************************/
/* DIR-n-m data structures
 * Each table in the DIR-n-m method is represented by a
 * dir_sub_table_t.  They are managed by a dir_table_t. */
struct dir_sub_table_flat_t
{
    int num_entries; /* Number of entries in this table */
    int width;       /* width of this table. */
                     /* While one determines the other, this way fewer
                      * calculations are needed at runtime, since both
                      * are used. */
    int cur_num;     /* Present number of used nodes */

    ENTRIES_PTR entries;
};

/* Master data structure for the DIR-n-m derivative */
struct dir_table_flat_t
{
    int dimensions[20];    /* DIR-n-m will consist of any number of arbitrarily
                         * long tables. This variable keeps track of the
                         * dimensions */
    int dim_size;       /* And this variable keeps track of 'dimensions''s
                         * dimensions! */
    uint32_t mem_cap;  /* User-defined maximum memory that can be allocated
                         * for the DIR-n-m derivative */

    int cur_num;        /* Present number of used nodes */

    uint32_t allocated;

    SUB_TABLE_PTR sub_table;
};

#endif /* SFRT_FLAT_DIR_H */

sfrt_flat.h

Path = src/sfrt/sfrt_flat.h

#ifndef SFRT_FLAT_H
#define SFRT_FLAT_H

// This is based on the original sfrt.h, but using the flat segment memory.
// When allocating memory, it uses memory in the segment, and returns the offset.
// When accessing memory, it must use the base address and offset to
// correctly refer to it.

#include "sfrt/sfrt.h"
#include "sfrt/sfrt_flat_dir.h"

typedef MEM_OFFSET INFO; /* To be replaced with a pointer to a policy */
typedef MEM_OFFSET FLAT_INDEX;
typedef MEM_OFFSET TABLE_PTR;

enum SaveDest
{
    SAVE_TO_NEW,
    SAVE_TO_CURRENT
};

typedef int64_t (*updateEntryInfoFunc)(INFO* entryInfo, INFO newInfo,
    SaveDest saveDest, uint8_t* base, void* data);

struct tuple_flat_t
{
    FLAT_INDEX index;
    int length;
};

// Master table struct.  Abstracts DIR and LC-trie methods
struct table_flat_t
{
    unsigned num_ent = 0;   // Number of entries in the policy table
    unsigned max_size = 0;  // Max size of policies array
    char ip_type;           // Only IPs of this family will be used
    char table_flat_type;
    char mem_type;
    unsigned allocated;
    INFO data;              // data table. Each IP points to an entry here
    TABLE_PTR rt;           // Actual "routing" table
    TABLE_PTR rt6;          // Actual "routing" table
    TABLE_PTR list_info;    // List file information table (entry information)
};
/*******************************************************************/

GENERIC sfrt_flat_dir8x_lookup(const snort::SfIp* ip, table_flat_t* table);

struct IPLOOKUP;
struct RtTable
{
    void sfrt_flat_new(char table_flat_type, char ip_type, long data_size, uint32_t mem_cap);
    GENERIC sfrt_flat_lookup(const snort::SfIp*);

    return_codes sfrt_flat_insert(snort::SfCidr* cidr, unsigned char len, INFO ptr, int behavior,
        updateEntryInfoFunc, void* update_entry_info_data);
    unsigned sfrt_flat_usage() const;
    unsigned sfrt_flat_num_entries() const;
    table_flat_t* get_table() const
    { return table; }

    void segment_meminit(uint8_t* buff, size_t mem_cap)
    {
        base_ptr = buff;
        unused_ptr = 0;
        unused_mem = mem_cap;
    }
    MEM_OFFSET segment_snort_calloc(size_t num, size_t size);

protected:
    TABLE_PTR sfrt_dir_flat_new(uint32_t mem_cap, int count, ...);
    tuple_flat_t sfrt_dir_flat_lookup(const uint32_t* addr, int numAddrDwords, TABLE_PTR table_ptr);
    return_codes sfrt_dir_flat_insert(const uint32_t* addr, int numAddrDwords, int len, word data_index,
        int behavior, TABLE_PTR, updateEntryInfoFunc updateEntry, void* update_entry_info_data, INFO *data);
    uint32_t sfrt_dir_flat_usage(TABLE_PTR) const;
    TABLE_PTR _sub_table_flat_new(dir_table_flat_t* root, uint32_t dimension, uint32_t prefill, uint32_t bit_length);
    return_codes _dir_sub_insert(IPLOOKUP* ip, int length, int cur_len, INFO ptr,
        int current_depth, int behavior, SUB_TABLE_PTR sub_ptr, dir_table_flat_t* root_table,
        updateEntryInfoFunc updateEntry, void* update_entry_info_data, INFO* data);
    int64_t _dir_update_info(int index, int fill, word length, uint32_t val, SUB_TABLE_PTR sub_ptr,
        updateEntryInfoFunc updateEntry, void* update_entry_info_data, INFO* data);
    tuple_flat_t _dir_sub_flat_lookup(IPLOOKUP* ip, TABLE_PTR table_ptr);

    MEM_OFFSET segment_snort_alloc(size_t size);
    void segment_free(MEM_OFFSET)
    { }
    size_t segment_unusedmem() const
    { return unused_mem; }
    void* segment_basePtr() const
    { return base_ptr; }

    table_flat_t* table = nullptr;
    MEM_OFFSET unused_ptr = 0;
    size_t unused_mem = 0;
    void* base_ptr = nullptr;
};

#endif

sfrt.h

Path = src/sfrt/sfrt.h

#ifndef SFRT_H
#define SFRT_H

#include <cstdint>

#include "sfip/sf_ip.h"

// A 32-bit word is used to hold the bit patterns of the addresses.
// In IPv6 this should be 128 bits.  The following typedef is
// machine dependent.  A word must be 32 bits long!
typedef unsigned long word;

typedef void* GENERIC;  // To be replaced with a pointer to a policy
using MEM_OFFSET = uint32_t;

struct tuple_t
{
    word index;
    word length;
};

enum types
{
#if 0
    // supporting code for these types (and RT_FAVOR_* below) is
    // disabled since it is not used. not deleting in case we need
    // switch types at some point. see sfrt_*.cc.
    DIR_24_8,
    DIR_16x2,
    DIR_16_8x2,
    DIR_16_4x4,
    DIR_8x4,
    DIR_4x8,
    DIR_2x16,
    DIR_16_4x4_16x5_4x4,
    DIR_16x7_4x4,
    DIR_16x8,
#endif
    DIR_8x16,
    IPv4,
    IPv6
};

enum return_codes
{
    RT_SUCCESS=0,
    RT_INSERT_FAILURE,
    RT_POLICY_TABLE_EXCEEDED,
    DIR_INSERT_FAILURE,
    DIR_LOOKUP_FAILURE,
    MEM_ALLOC_FAILURE,
    RT_REMOVE_FAILURE
};

enum
{
#if 0
    RT_FAVOR_TIME,
    RT_FAVOR_SPECIFIC,
#endif
    RT_FAVOR_ALL
};

#endif

side_channel/

The modules/classes contained here implement the top-level interface for side channel use and control.

There are two primary classes:

1) SideChannelManager - Set of static methods to manage all SideChannel objects across all threads. The SideChannelManager::get_side_channel() method is used by SC clients to fetch one of the SC objects. Other SideChannelManager methods are used to configure the set of SC’s. SideChannelManager methods maintain one module global array of SC’s to house configuration and one module global thread- specific array of SC’s as the thread and channel specific objects. These thread/channel specific objects are instantiated from the configuration objects as part of the thread_init() method.

2) SideChannel - Set of methods for all SC communications activity. In particular: SideChannel::register_receive_handler( handler ) - register the client method to process received SC messages. SideChannel::unregister_receive_handler() SideChannel::alloc_transmit_message( length ) - Allocate a new SCMessage object of a given size to be used for transmitting. SideChannel::discard_message( SCMessage ) - Must be used in the client’s receive handler to discard the receive message. Also used to discard a transmit message if the client decides not to transmit it. SideChannel::transmit_message( SCMessage ) - transmit the message SideChannel::process( number_of_messages ) - synchronously process receive messages. Receive at most number_of_messages per invocation. Not used in asynchronous mode.

The SCPort number is the primary key used to configure SC’s. SCPort is a 16bit value. It is used in SC messages as the address/content descriptor. Conceptually one SC object can be used for a set of SCPort values. However the current implementation associates ONE SCPort value with ONE SC object. SCPort values are used by configuration entries to map side channel instances to side channel clients. The SideChannelManager::get_side_channel( SCPort ) is used by an SC client to fetch the SC associated with a given SCPort value.

The configuration entries map a side channel instance to a set of connector instances.

The SideChannel service utilizes Connector objects for communications. Connector is a plugin class that provides side channel message handling services. A SideChannel can own a transmit and/or a receive connector object. Connector objects implement a simplex channel.

Each thread (packet thread or other thread) can own multiple SideChannel objects. The SCPort is used to distinguish. The SideChannelManager::thread_init() method is called during the thread initialization to instantiate SideChannel objects for the thread.

During snort configuration, one side channel object is created to capture the common configuration elements. Then every thread that needs to utilize side channels must invoke SideChannelManager::thread_init(). This will instantiate thread specific side channel and connector objects. All side channel and connector state is local to the thread.

Side Channel messages are allocated dynamically. The SC client invokes alloc_transmit_message() to get a transmit message of a given size. The client owns the message until it either invokes transmit_message() to send it or discard_message() to discard it.

Likewise, messages received are allocated by the connected and the ownership is passed to the SC client in the receive callback. The client invokes discard_message() when the handler is done with it.

The dynamic message allocation architecture is flexible and asynchronous. Upon transmitting the message, the client transfers ownership to the connector. The connector then discards the message upon transmission.

A side channel can be bidirectional, but does not implement a request/reply paradigm. Rather it should be viewed as two simplex channels.

side_channel.h

Path = src/side_channel/side_channel.h

#ifndef SIDE_CHANNEL_H
#define SIDE_CHANNEL_H

#include <functional>

#include "framework/bits.h"
#include "framework/connector.h"

#define MAXIMUM_SC_MESSAGE_CONTENT 1024
#define DISPATCH_ALL_RECEIVE 0

class SideChannel;

typedef uint16_t SCPort;
typedef uint16_t SCSequence;

typedef std::vector<std::string> SCConnectors;

struct __attribute__((__packed__)) SCMsgHdr
{
    uint16_t port;
    uint16_t sequence;
    uint32_t time_u_seconds;
    uint64_t time_seconds;
};

struct SCMessage
{
    SideChannel* sc;
    snort::Connector* connector;
    snort::ConnectorMsgHandle* handle;
    SCMsgHdr* hdr;
    uint8_t* content;
    uint32_t content_length;
};

typedef std::function<void(SCMessage*)> SCProcessMsgFunc;

// SideChannel is instantiated for each defined SC use.
class SideChannel
{
public:
    SideChannel() = default;

    void register_receive_handler(const SCProcessMsgFunc& handler);
    void unregister_receive_handler();

    bool process(int max_messages);
    SCMessage* alloc_transmit_message(uint32_t content_length);
    bool discard_message(SCMessage* msg);
    bool transmit_message(SCMessage* msg);
    void set_message_port(SCMessage* msg, SCPort port);
    void set_default_port(SCPort port);
    snort::Connector::Direction get_direction();

    snort::Connector* connector_receive = nullptr;
    snort::Connector* connector_transmit = nullptr;

private:
    SCSequence sequence = 0;
    SCPort default_port = 0;
    SCProcessMsgFunc receive_handler = nullptr;
};

// SideChannelManager is primary interface with Snort.
class SideChannelManager
{
public:
    // Instantiate new SideChannel configuration
    static void instantiate(const SCConnectors* connectors, const PortBitSet* ports);

    // Main thread, pre-config init
    static void pre_config_init();

    // Per packet-thread startup.  Create all configured SideChannelConnector's.
    static void thread_init();

    // Per packet thread shutdown.
    static void thread_term();

    // Overall shutdown.
    static void term();

    // if configured, returns the SideChannel object associated with the specified port number.
    // Else return nullptr if none is configured.
    static SideChannel* get_side_channel(SCPort);

private:
    SideChannelManager() = delete;
};
#endif

side_channel_module.h

Path = src/side_channel/side_channel_module.h

#ifndef SIDE_CHANNEL_MODULE_H
#define SIDE_CHANNEL_MODULE_H

#include "framework/module.h"
#include "side_channel/side_channel.h"

#define SIDECHANNEL_NAME "side_channel"
#define SIDECHANNEL_HELP "implement the side-channel asynchronous messaging subsystem"

struct SideChannelConfig
{
    PortBitSet* ports = nullptr;
    SCConnectors connectors;
};

extern THREAD_LOCAL SimpleStats sc_stats;
extern THREAD_LOCAL snort::ProfileStats sc_perf_stats;

class SideChannelModule : public snort::Module
{
public:
    SideChannelModule();

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    PegCount* get_counts() const override
    { return (PegCount*)&sc_stats; }

    const PegInfo* get_pegs() const override
    { return snort::simple_pegs; }

    snort::ProfileStats* get_profile() const override;

    Usage get_usage() const override
    { return GLOBAL; }

private:
    SideChannelConfig* config = nullptr;
};

#endif

side_channel/test/

stream/

This directory contains the implementation of the Stream preprocessor components:

  • Stream constants and data types used across stream components or by clients of the stream API.

  • Prototype definitions and implementation for all stream API methods.

  • Virtual base class defining the Stream Splitter interface. Implementation of stream splitters for accumulated TCP over maximum flushing (atom splitter) and length of given segment flushing (log splitter).

  • Prototype definitions and implementation for the stream Protocol Aware Flushing API methods (PAF is now realized by stream splitter subclasses).

Major subcomponents of the Stream inspector are each implemented in a subdirectory located here. These include the following:

  • base - implements the Stream class as a subclass of Inspector, handles stream configuration, provides entry point for Stream evaluation of a Packet, stream statistics management.

  • tcp - implements module for handling Stream tcp sessions. This includes normalization protocol tracking, normalization, and reassembly.

  • udp - implements module for handling Stream udp sessions. Tracking only.

  • icmp - implements module for handling Stream icmp sessions. Tracking only.

  • ip - implements module for handling Stream ip sessions. Tracking only.

  • file - implements module for handling Stream file session. This directly sets file data and invokes file processing and detection.

  • user - implements module for handling Stream user session. This handles payload only, eg from a socket. Does splitter based reassembly like TCP.

HA functionality for each protocol (IP,ICMP,UDP, & TCP) is implemented in ./<protocol>/<protocol>_ha.*. The static class <protocol>HAManager instantiates and destroys the <protocol>HA class. <protocol>HA is a subclass of ProtocolHA and interfaces with the high level Stream HA functionality. <protocol>HA is instantiated in each packet processing thread.

Note that the lifetime of any stream splitter instance should be less than the lifetime of the corresponding inspector instance.

flush_bucket.h

Path = src/stream/flush_bucket.h

#ifndef FLUSH_BUCKET_H
#define FLUSH_BUCKET_H

// FlushBuckets manage a set of flush points for stream_tcp.

#include <cstdint>
#include <vector>

class FlushBucket
{
public:
    virtual ~FlushBucket() = default;
    virtual uint16_t get_next() = 0;

    static uint16_t get_size();
    static void set(unsigned sz);
    static void set();
    static void clear();

protected:
    FlushBucket() = default;
};

class ConstFlushBucket : public FlushBucket
{
public:
    ConstFlushBucket(uint16_t fp)
    { pt = fp; }

    uint16_t get_next() override
    { return pt; }

private:
    uint16_t pt;
};

class VarFlushBucket : public FlushBucket
{
public:
    uint16_t get_next() override;

protected:
    VarFlushBucket() = default;
    void set_next(uint16_t);

private:
    unsigned idx = 0;
    std::vector<uint16_t> flush_points;
};

class StaticFlushBucket : public VarFlushBucket
{
public:
    StaticFlushBucket();
};

class RandomFlushBucket : public VarFlushBucket
{
public:
    RandomFlushBucket();
};

#endif

paf.h

Path = src/stream/paf.h

#ifndef PAF_H
#define PAF_H

#include "main/snort_types.h"
#include "main/thread.h"
#include "profiler/profiler_defs.h"
#include "stream/stream_splitter.h"

namespace snort
{
struct Packet;
}

extern THREAD_LOCAL snort::ProfileStats pafPerfStats;

void* paf_new(unsigned max);     // create new paf config (per policy)
void paf_delete(void*);  // free config

struct SO_PUBLIC PAF_State     // per session direction
{
    uint32_t seq;    // stream cursor
    uint32_t pos;    // last flush position

    uint32_t fpt;    // current flush point
    uint32_t tot;    // total bytes flushed

    snort::StreamSplitter::Status paf;  // current scan state
};

SO_PUBLIC void paf_setup(PAF_State*);  // called at session start
void paf_reset(PAF_State*);  // called for do overs
void paf_clear(PAF_State*);  // called at session end

inline uint32_t paf_position (PAF_State* ps)
{
    return ps->seq;
}

SO_PUBLIC inline uint32_t paf_initialized (PAF_State* ps)
{
    return ( ps->paf != snort::StreamSplitter::START );
}

SO_PUBLIC inline void paf_initialize(PAF_State* ps, uint32_t seq)
{
    ps->seq = ps->pos = seq;
    ps->fpt = ps->tot = 0;
    ps->paf = snort::StreamSplitter::SEARCH;
}

inline uint32_t paf_active (PAF_State* ps)
{
    return ( ps->paf != snort::StreamSplitter::ABORT );
}

inline void paf_jump(PAF_State* ps, uint32_t n)
{
    ps->pos += n;
    ps->seq = ps->pos;
}

// called on each in order segment
SO_PUBLIC int32_t paf_check(snort::StreamSplitter* paf_config, PAF_State*, snort::Packet* p,
    const uint8_t* data, uint32_t len, uint32_t total, uint32_t seq, uint32_t* flags);

#endif

stream.h

Path = src/stream/stream.h

#ifndef STREAM_H
#define STREAM_H

// provides a common flow management interface

#include <memory>

#include <daq_common.h>

#include "flow/flow.h"

class HostAttributesDescriptor;
typedef std::shared_ptr<HostAttributesDescriptor> HostAttributesEntry;

namespace snort
{
class Flow;
struct SfIp;
class StreamSplitter;

/* traffic direction identification */
#define FROM_SERVER     0
#define FROM_CLIENT     1

#define SSN_MISSING_NONE   0x00
#define SSN_MISSING_BEFORE 0x01
#define SSN_MISSING_AFTER  0x02
#define SSN_MISSING_BOTH   (SSN_MISSING_BEFORE | SSN_MISSING_AFTER)

#define SSN_DIR_NONE           0x00
#define SSN_DIR_FROM_CLIENT    0x01
#define SSN_DIR_FROM_SERVER    0x02
#define SSN_DIR_BOTH           0x03

// sequence must match FRAG_POLICY_* enum in stream_ip.h (1-based)
#define IP_POLICIES  \
    "first | linux | bsd | bsd_right | last | windows | solaris"

// sequence must match enum StreamPolicy defines in tcp_defs.h
#define TCP_POLICIES \
    "first | last | linux | old_linux | bsd | macos | solaris | irix | " \
    "hpux11 | hpux10 | windows | win_2003 | vista | proxy"

struct AlertInfo
{
    AlertInfo() = default;
    AlertInfo(uint32_t gid, uint32_t sid, uint32_t id, uint32_t ts = 0)
        : gid(gid), sid(sid), event_id(id), event_second(ts) {}

    uint32_t gid = 0;
    uint32_t sid = 0;

    uint32_t event_id = 0;
    uint32_t event_second = 0;
};

typedef int (* LogFunction)(Flow*, uint8_t** buf, uint32_t* len, uint32_t* type);
typedef void (* LogExtraData)(Flow*, void* config, LogFunction* funcs,
    uint32_t max_count, uint32_t xtradata_mask, const AlertInfo& alert_info);

#define MAX_LOG_FN 32

//-------------------------------------------------------------------------

class SO_PUBLIC Stream
{
public:
    // for shutdown only
    static void purge_flows();

    static void handle_timeouts(bool idle);
    static bool prune_flows();
    static bool expected_flow(Flow*, Packet*);

    // Looks in the flow cache for flow session with specified key and returns
    // pointer to flow session object if found, otherwise null.
    static Flow* get_flow(const FlowKey*);

    // Allocates a flow session object from the flow cache table for the protocol
    // type of the specified key.  If no cache exists for that protocol type null is
    // returned.  If a flow already exists for the key a pointer to that session
    // object is returned.
    // If a new session object can not be allocated the program is terminated.
    static Flow* new_flow(const FlowKey*);

    // Removes the flow session object from the flow cache table and returns
    // the resources allocated to that flow to the free list.
    static void delete_flow(const FlowKey*);
    static void delete_flow(Flow*);

    // Examines the source and destination ip addresses and ports to determine if the
    // packet is from the client or server side of the flow and sets bits in the
    // packet_flags field of the Packet struct to indicate the direction determined.
    static uint32_t get_packet_direction(Packet*);

    // Set the stream normalization mode to PROXY.  In this mode wire packets go thru a proxy before snort
    // sees them.  All stream normalizations are turned off in this mode.
    static void start_proxy(Flow*);

    // Stop inspection on a flow for up to count bytes (-1 to ignore for life or until resume).
    // If response flag is set, automatically resume inspection up to count bytes when a data
    // packet in the other direction is seen.  Also marks the packet to be ignored
    // FIXIT-L stop_inspection() does not currently support the bytes/response parameters
    static void stop_inspection(Flow*, Packet*, char dir, int32_t bytes, int rspFlag);

    // Adds entry to the expected session cache with a flow key generated from the network
    // n-tuple parameters specified.  Inspection will be turned off for this expected session
    // when it arrives.
    static int ignore_flow(
        const Packet* ctrlPkt, PktType, IpProtocol, const snort::SfIp* srcIP, uint16_t srcPort,
        const snort::SfIp* dstIP, uint16_t dstPort, char direction, FlowData* fd);

    // Set Active status to force drop the current packet and set flow state to drop
    // subsequent packets arriving from the direction specified.
    static void drop_traffic(const Packet*, char dir);

    // Mark a flow as dropped, release allocated resources, and set flow state such that any
    // subsequent packets received on this flow are dropped.
    static void drop_flow(const Packet*);

    // Mark flow session as block pending. Resources will be released
    // at the end of inspection
    static void block_flow(const Packet*);

    static void flush_client(Packet*);  // flush data received by client
    static void flush_server(Packet*);  // flush data received by server

    // Add session alert - true if added
    static bool add_flow_alert(Flow*, Packet*, uint32_t gid, uint32_t sid);

    // Check session alert - true if previously alerted
    static bool check_flow_alerted(Flow*, Packet* p, uint32_t gid, uint32_t sid);

    // Set Extra Data Logging
    static int update_flow_alert(
        Flow*, Packet* p, uint32_t gid, uint32_t sid,
        uint32_t eventId, uint32_t eventSecond);

    static void disable_reassembly(Flow*);
    static char get_reassembly_direction(Flow*);

    // Returns true if stream data for the flow is in sequence, otherwise return false.
    static bool is_stream_sequenced(Flow*, uint8_t dir);

    // Get state of missing packets for the flow.
    //      SSN_MISSING_BOTH if missing before and after
    //      SSN_MISSING_BEFORE if missing before
    //      SSN_MISSING_AFTER if missing after
    //      SSN_MISSING_NONE if none missing
    static int missing_in_reassembled(Flow*, uint8_t dir);

    // Returns true if packets were missed on the stream, otherwise returns false.
    static bool missed_packets(Flow*, uint8_t dir);

    // Get the protocol identifier from a stream
    static SnortProtocolId get_snort_protocol_id(Flow*);

    // Set the protocol identifier for a stream
    static SnortProtocolId set_snort_protocol_id(Flow*, SnortProtocolId, bool is_appid_service = false);

    // initialize response count and expiration time
    static void init_active_response(const Packet*, Flow*);

    static void set_splitter(Flow*, bool toServer, StreamSplitter* = nullptr);
    static StreamSplitter* get_splitter(Flow*, bool toServer);

    // Turn off inspection for potential session. Adds session identifiers to a hash table.
    // TCP only.
    static int set_snort_protocol_id_expected(
        const Packet* ctrlPkt, PktType, IpProtocol, const snort::SfIp* srcIP, uint16_t srcPort,
        const snort::SfIp* dstIP, uint16_t dstPort, SnortProtocolId, FlowData*,
        bool swap_app_direction = false, bool expect_multi = false, bool bidirectional = false,
        bool expect_persist = false);

    // Get pointer to application data for a flow based on the lookup tuples for cases where
    // Snort does not have an active packet that is relevant.
    static FlowData* get_flow_data(
        PktType type, IpProtocol proto,
        const snort::SfIp* a1, uint16_t p1, const snort::SfIp* a2, uint16_t p2,
        uint16_t vlanId, uint32_t mplsId, uint32_t addrSpaceId, unsigned flowdata_id,
        uint32_t tenant_id, int16_t ingress_group = DAQ_PKTHDR_UNKNOWN,
        int16_t egress_group = DAQ_PKTHDR_UNKNOWN);

    // Get pointer to application data for a flow using the FlowKey as the lookup criteria
    static FlowData* get_flow_data(const FlowKey*, unsigned flowdata_id);

    // Get pointer to a session flow instance for a flow based on the lookup tuples for
    // cases where Snort does not have an active packet that is relevant.
    static Flow* get_flow(
        PktType type, IpProtocol proto,
        const snort::SfIp* a1, uint16_t p1, const snort::SfIp* a2, uint16_t p2,
        uint16_t vlanId, uint32_t mplsId, uint32_t addrSpaceId,
        uint32_t tenant_id, int16_t ingress_group = DAQ_PKTHDR_UNKNOWN,
        int16_t egress_group = DAQ_PKTHDR_UNKNOWN);

    static Flow* get_flow(
        PktType type, IpProtocol proto,
        const snort::SfIp* a1, uint16_t p1, const snort::SfIp* a2, uint16_t p2,
        uint16_t vlanId, uint32_t mplsId, const DAQ_PktHdr_t&);

    // Delete the session if it is in the closed session state.
    // Handle session block pending state
    static void check_flow_closed(Packet*);

    //  Populate a flow key from the Packet
    static void populate_flow_key(const Packet*, FlowKey*);

    static void set_snort_protocol_id_from_ha(Flow*, const SnortProtocolId);

    static bool is_midstream(Flow* flow)
    { return ((flow->ssn_state.session_flags & SSNFLAG_MIDSTREAM) != 0); }

    // Get the TTL value used at session setup
    // Set outer=false to get inner ip ttl for ip in ip; else outer=true
    static uint8_t get_flow_ttl(Flow*, char dir, bool outer);

    static bool expired_flow(Flow*, Packet*);
    static bool ignored_flow(Flow*, Packet*);
    static bool blocked_flow(Packet*);

    // extra data methods
    static void set_extra_data(Flow*, Packet*, uint32_t);
    static void log_extra_data(Flow*, uint32_t mask, const AlertInfo&);

    static uint32_t reg_xtra_data_cb(LogFunction);
    static void reg_xtra_data_log(LogExtraData, void*);
    static uint32_t get_xtra_data_map(LogFunction*&);

    // TCP-specific accessor methods.
    static uint16_t get_mss(Flow*, bool to_server);
    static uint8_t get_tcp_options_len(Flow*, bool to_server);

    static bool set_packet_action_to_hold(Packet*);
    static bool can_set_no_ack_mode(Flow*);
    static bool set_no_ack_mode(Flow*, bool);
    static void partial_flush(Flow*, bool to_server);

    static bool get_held_pkt_seq(Flow*, uint32_t&);

    static void set_pub_id();
    static unsigned get_pub_id();

private:
    static void set_ip_protocol(Flow*);
};
}
#endif

stream_inspectors.h

Path = src/stream/stream_inspectors.h

#ifndef STREAM_INSPECTORS_H
#define STREAM_INSPECTORS_H

void load_stream_inspectors();

#endif

stream_splitter.h

Path = src/stream/stream_splitter.h

#ifndef TCP_SPLITTER_H
#define TCP_SPLITTER_H

#include "main/snort_types.h"

namespace snort
{
class Flow;
struct Packet;

struct StreamBuffer
{
    const uint8_t* data;
    unsigned length;
};

//-------------------------------------------------------------------------

class SO_PUBLIC StreamSplitter
{
public:
    virtual ~StreamSplitter() = default;

    enum Status
    {
        ABORT,   // non-paf operation
        START,   // internal use only
        SEARCH,  // searching for next flush point
        FLUSH,   // flush at given offset
        LIMIT,   // flush to given offset upon reaching paf_max
        SKIP,    // skip ahead to given offset
        LIMITED, // previously did limit flush
        STOP     // stop paf scan loop
    };

    // scan(), finish(), reassemble() are called in this order:
    // (scan (reassemble)*)* finish (reassemble)*

    virtual Status scan(
        Packet*,
        const uint8_t* data,   // in order segment data as it arrives
        uint32_t len,          // length of data
        uint32_t flags,        // packet flags indicating direction of data
        uint32_t* fp           // flush point (offset) relative to data
        ) = 0;

    // finish indicates end of scanning
    // return false to discard any unflushed data
    virtual bool finish(Flow*) { return true; }
    virtual bool init_partial_flush(Flow*) { return false; }

    // the last call to reassemble() will be made with len == 0 if
    // finish() returned true as an opportunity for a final flush
    virtual const StreamBuffer reassemble(
        Flow*,
        unsigned total,        // total amount to flush (sum of iterations)
        unsigned offset,       // data offset from start of reassembly
        const uint8_t* data,   // data to reassemble
        unsigned len,          // length of data to process this iteration
        uint32_t flags,        // packet flags indicating pdu head and/or tail
        unsigned& copied       // actual data copied (1 <= copied <= len)
        );

    virtual bool sync_on_start() const { return false; }
    virtual bool is_paf() { return false; }
    virtual unsigned max(Flow* = nullptr);
    virtual void go_away() { delete this; }

    bool to_server() { return c2s; }
    bool to_client() { return !c2s; }

protected:
    StreamSplitter(bool b) : c2s(b) { }
    uint16_t get_flush_bucket_size();
    unsigned bytes_scanned = 0;

private:
    const bool c2s;
};

//-------------------------------------------------------------------------
// accumulated tcp over maximum splitter (aka footprint)

class AtomSplitter : public StreamSplitter
{
public:
    AtomSplitter(bool, uint16_t size = 0);

    Status scan(Packet*, const uint8_t*, uint32_t, uint32_t, uint32_t*) override;

private:
    void reset();

private:
    uint16_t base;
    uint16_t min;
    uint16_t segs;
};

//-------------------------------------------------------------------------
// length of given segment splitter (pass-thru)

class SO_PUBLIC LogSplitter : public StreamSplitter
{
public:
    LogSplitter(bool);

    Status scan(Packet*, const uint8_t*, uint32_t, uint32_t, uint32_t*) override;
};

//-------------------------------------------------------------------------
// stop-and-wait splitter (flush opposite direction upon data)

class StopAndWaitSplitter : public StreamSplitter
{
public:
    StopAndWaitSplitter(bool b) : StreamSplitter(b) { }

    Status scan(Packet*, const uint8_t*, uint32_t, uint32_t, uint32_t*) override;

private:
    bool saw_data()
    { return byte_count > 0; }

    void reset()
    { byte_count = 0; }

private:
    unsigned byte_count = 0;
};
}
#endif

stream/base/

This directory contains the implementation of the base Stream preprocessor module. This module is implemented as a subclass of Inspector and implements the following capabilities:

  • entry point for Stream evaluation of a packet

  • stream global configuration management

  • stream statistics management.

Each protocol (or packet source) stream supports is implemented as its own Inspector and the base class delegates the actual evaluation of a packet to the specific module.

The base stream HA processing is implemented in stream_ha.*. The session FlowHAClient is implemented as the class StreamHAClient. This provides all top-level Stream HA processing. Each stream per-protocol (../udp, ../tcp, etc.) in turn implements the per-protocol HA specifics.

A static StreamHAManager class is used to create/destroy the StreamHAClient class in each packet processing thread.

The StreamHAContent class defines the actual HA message content for this session client.

The stream_ha.* logic is linked to per-protocol logic via the ProtocolHA class. This class is declared in stream_ha.h. It is then defined and instantiated in each per-protocol HA handler.

The result is a two layer class hierarchy. StreamHAClient interfaces with the primary HA functionality as a FlowHAClient. And the stream/<protocol>'s interface with the StreamHAClient.

stream_ha.h

Path = src/stream/base/stream_ha.h

#ifndef STREAM_HA_H
#define STREAM_HA_H

#include "flow/flow.h"
#include "flow/ha.h"

class __attribute__((__packed__)) SessionHAContent
{
public:
    snort::LwState ssn_state;
    snort::Flow::FlowState flow_state;
    uint8_t flags;
    static constexpr uint8_t FLAG_LOW = 0x01; // client address / port is low in key
    static constexpr uint8_t FLAG_IP6 = 0x02; // key addresses are ip6
};

class StreamHAClient : public snort::FlowHAClient
{
public:
    StreamHAClient() : FlowHAClient(sizeof(SessionHAContent), true) { }
    bool consume(snort::Flow*&, const snort::FlowKey*, snort::HAMessage&, uint8_t size) override;
    bool produce(snort::Flow&, snort::HAMessage&) override;
    bool is_update_required(snort::Flow*) override;
};

class ProtocolHA
{
public:
    ProtocolHA(PktType);
    virtual ~ProtocolHA();
    virtual snort::Flow* create_session(const snort::FlowKey*) { return nullptr; }
    virtual void deactivate_session(snort::Flow*) { }
    virtual void process_deletion(snort::Flow&);
};

class StreamHAManager
{
public:
    static void tinit();
    static void tterm();

    static THREAD_LOCAL StreamHAClient* ha_client;
};
#endif

stream_module.h

Path = src/stream/base/stream_module.h

#ifndef STREAM_MODULE_H
#define STREAM_MODULE_H

#include "flow/flow_config.h"
#include "flow/flow_control.h"
#include "framework/module.h"
#include "main/analyzer.h"
#include "main/reload_tuner.h"

namespace snort
{
class Trace;
struct SnortConfig;
}

extern THREAD_LOCAL snort::ProfileStats s5PerfStats;
extern THREAD_LOCAL class FlowControl* flow_con;
extern THREAD_LOCAL const snort::Trace* stream_trace;

#ifdef DEBUG_MSGS
enum
{
    TRACE_BASE = 0,
    TRACE_FLOW
};
#endif

//-------------------------------------------------------------------------
// stream module
//-------------------------------------------------------------------------

#define MOD_NAME "stream"
#define MOD_HELP "common flow tracking"

struct BaseStats
{
     PegCount flows;
     PegCount prunes;
     PegCount max_flow_prunes;
     PegCount protocol_timeout_prunes;
     PegCount excess_prunes;
     PegCount uni_prunes;
     PegCount memcap_prunes;
     PegCount ha_prunes;
     PegCount stale_prunes;
     PegCount expected_flows;
     PegCount expected_realized;
     PegCount expected_pruned;
     PegCount expected_overflows;
     PegCount reload_tuning_idle;
     PegCount reload_tuning_packets;
     PegCount reload_total_adds;
     PegCount reload_total_deletes;
     PegCount reload_freelist_flow_deletes;
     PegCount reload_allowed_flow_deletes;
     PegCount reload_blocked_flow_deletes;
     PegCount reload_offloaded_flow_deletes;
     PegCount ip_timeout_prunes;
     PegCount tcp_timeout_prunes;
     PegCount udp_timeout_prunes;
     PegCount icmp_timeout_prunes;
     PegCount user_timeout_prunes;
     PegCount file_timeout_prunes;
     PegCount pdu_timeout_prunes;
     PegCount ip_memcap_prunes;
     PegCount tcp_memcap_prunes;
     PegCount udp_memcap_prunes;
     PegCount icmp_memcap_prunes;
     PegCount user_memcap_prunes;
     PegCount file_memcap_prunes;
     PegCount pdu_memcap_prunes;

     // Keep the NOW stats at the bottom as it requires special sum_stats logic
     PegCount current_flows;
     PegCount uni_flows;
     PegCount uni_ip_flows;

};

extern const PegInfo base_pegs[];

extern THREAD_LOCAL BaseStats stream_base_stats;

struct StreamModuleConfig
{
    FlowCacheConfig flow_cache_cfg;
#ifdef REG_TEST
    unsigned footprint = 0;
#endif
    uint32_t held_packet_timeout = 1000;  // in milliseconds

    void show() const;
};

class StreamReloadResourceManager : public snort::ReloadResourceTuner
{
public:
    StreamReloadResourceManager() = default;

    bool tinit() override;
    bool tune_packet_context() override;
    bool tune_idle_context() override;

    bool initialize(const StreamModuleConfig&);

private:
    bool tune_resources(unsigned work_limit);

private:
    StreamModuleConfig config;
};

class HPQReloadTuner : public snort::ReloadResourceTuner
{
public:
    explicit HPQReloadTuner(uint32_t packet_timeout) : held_packet_timeout(packet_timeout) { }
    ~HPQReloadTuner() override = default;

    bool tinit() override;
    bool tune_packet_context() override;
    bool tune_idle_context() override;

private:
    uint32_t held_packet_timeout;
};

class StreamUnloadReloadResourceManager : public snort::ReloadResourceTuner
{
public:
    StreamUnloadReloadResourceManager() = default;

    bool tinit() override;
    bool tune_packet_context() override;
    bool tune_idle_context() override;

private:
    bool tune_resources(unsigned work_limit);
};

class StreamModule : public snort::Module
{
public:
    StreamModule();

    bool begin(const char*, int, snort::SnortConfig*) override;
    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    snort::ProfileStats* get_profile() const override;
    const StreamModuleConfig* get_data();

    unsigned get_gid() const override;
    const snort::RuleMap* get_rules() const override;

    void prep_counts(bool dump_stats) override;
    void sum_stats(bool) override;
    void reset_stats() override;

    bool counts_need_prep() const override
    { return true; }

    Usage get_usage() const override
    { return GLOBAL; }

    void set_trace(const snort::Trace*) const override;
    const snort::TraceOption* get_trace_options() const override;

private:
    StreamModuleConfig config;
};

extern void base_prep();
extern void base_reset();

#endif

stream/file/

This directory contains the implementation of the Stream File flow tracking and processing functions. When the source for a flow is a file the base Stream preprocessor delegates handling of the packets on that flow to this module.

The StreamFile class is implemented as a subclass of Inspector and provides functions for loading stream file configuration and packet evaluation. The packet eval method is not used as the base Stream Inspector delegates packets directly to the file session packet processing method.

The file session packet processing function in turn delegates processing of the packet to the file_processing method of the FileAPI class.

file_module.h

Path = src/stream/file/file_module.h

#ifndef FILE_MODULE_H
#define FILE_MODULE_H

#include "framework/module.h"

namespace snort
{
struct SnortConfig;
}

extern const PegInfo file_pegs[];
extern THREAD_LOCAL struct FileStats file_stats;
extern THREAD_LOCAL snort::ProfileStats file_perf_stats;

//-------------------------------------------------------------------------
// stream_file module
//-------------------------------------------------------------------------

#define MOD_NAME "stream_file"
#define MOD_HELP "stream inspector for file flow tracking and processing"

class StreamFileModule : public snort::Module
{
public:
    StreamFileModule();

    bool begin(const char*, int, snort::SnortConfig*) override;
    bool set(const char*, snort::Value&, snort::SnortConfig*) override;

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

public:
    bool upload = false;
};

#endif

file_session.h

Path = src/stream/file/file_session.h

#ifndef FILE_SESSION_H
#define FILE_SESSION_H

#include "flow/session.h"

class FileSession : public Session
{
public:
    FileSession(snort::Flow*);
    ~FileSession() override;

    bool setup(snort::Packet*) override;
    void clear() override;
    int process(snort::Packet*) override;

private:
    void start(snort::Packet*, snort::Flow*);
    void update(snort::Packet*, snort::Flow*);
    void end(snort::Packet*, snort::Flow*);
};

#endif

stream_file.h

Path = src/stream/file/stream_file.h

#ifndef STREAM_FILE_H
#define STREAM_FILE_H

namespace snort
{
class Inspector;
}

struct StreamFileConfig
{
    bool upload;
};

StreamFileConfig* get_file_cfg(snort::Inspector*);

#endif

stream/icmp/

This directory contains the implementation of ICMP session tracking and processing functions. When the network protocol for a flow is determined to be ICMP the base Stream preprocessor will delegate handling of the packets on that flow to this module.

The StreamIcmp class is implemented as a subclass of Inspector and provides functions for loading stream ICMP configuration and packet evaluation. The packet eval method is not used as the base Stream Inspector delegates packets directly to the ICMP session packet processing method.

Currently only the ICMP Unreachable message is handled by the ICMP session module.

The module icmp_ha.cc (and icmp_ha.h) implements the per-protocol hooks into the stream logic for HA. IcmpHAManager is a static class that interfaces to a per-packet thread instance of the class IcmpHA. IcmpHA is sub-class of ProtocolHA, declared in the stream/base area. Thus each protocol within stream can have specific HA logic and interfaces.

IcmpHAManager::process_deletion() is called when an ICMP stream is being destroyed and indicates to the stream & flow HA logic that a flow deletion HA message needs to be emitted for the flow in question. Icmp streams are closed due to timeout or pruning actions which lead to the invocation of process_deletion().

IcmpHA::create_session() is called from the stream & flow HA logic and handles the creation of new flow upon receiving an HA update message.

icmp_ha.h

Path = src/stream/icmp/icmp_ha.h

#ifndef ICMP_HA_H
#define ICMP_HA_H

#include "stream/base/stream_ha.h"

//-------------------------------------------------------------------------

namespace snort
{
class Flow;
}

class IcmpHA : public ProtocolHA
{
public:
    IcmpHA() : ProtocolHA(PktType::ICMP) { }
    snort::Flow* create_session(const snort::FlowKey*) override;

private:
};

class IcmpHAManager
{
public:
    static void process_deletion(snort::Flow& flow);
    static void tinit();
    static void tterm();
    static THREAD_LOCAL IcmpHA* icmp_ha;
};
#endif

icmp_module.h

Path = src/stream/icmp/icmp_module.h

#ifndef ICMP_MODULE_H
#define ICMP_MODULE_H

#include "flow/session.h"
#include "framework/module.h"

#include "stream_icmp.h"

extern const PegInfo icmp_pegs[];
extern THREAD_LOCAL struct IcmpStats icmpStats;
extern THREAD_LOCAL snort::ProfileStats icmp_perf_stats;

namespace snort
{
struct SnortConfig;
}

struct IcmpStats
{
    SESSION_STATS;
};

//-------------------------------------------------------------------------
// stream_icmp module
//-------------------------------------------------------------------------

#define MOD_NAME "stream_icmp"
#define MOD_HELP "stream inspector for ICMP flow tracking"

class StreamIcmpModule : public snort::Module
{
public:
    StreamIcmpModule();
    ~StreamIcmpModule() override
    { delete config; }
    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    snort::ProfileStats* get_profile() const override;
    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

    StreamIcmpConfig* get_data();

private:
    StreamIcmpConfig* config = nullptr;
};

#endif

icmp_session.h

Path = src/stream/icmp/icmp_session.h

#ifndef ICMP_SESSION_H
#define ICMP_SESSION_H

#include <sys/time.h>
#include "flow/session.h"

class IcmpSession : public Session
{
public:
    IcmpSession(snort::Flow*);
    ~IcmpSession() override;

    bool setup(snort::Packet*) override;
    int process(snort::Packet*) override;
    void clear() override;

public:
    uint32_t echo_count = 0;
    struct timeval ssn_time = {};
};

void icmp_stats();
void icmp_reset();

#endif

stream_icmp.h

Path = src/stream/icmp/stream_icmp.h

#ifndef STREAM_ICMP_H
#define STREAM_ICMP_H

#include <cstdint>

namespace snort
{
class Inspector;
}

struct StreamIcmpConfig
{
    uint32_t session_timeout;

    StreamIcmpConfig();
};

StreamIcmpConfig* get_icmp_cfg(snort::Inspector*);

#endif

stream/ip/

This directory contains the implementation of IP session tracking and processing functions. When the network protocol for a flow is determined to be IP the base Stream preprocessor will delegate handling of the packets on that flow to this module.

The StreamIp class is implemented as a subclass of Inspector and provides functions for loading stream IP configuration and packet evaluation. The packet eval method is not used as the base Stream Inspector delegates packets directly to the IP session packet processing method.

The module ip_ha.cc (and ip_ha.h) implements the per-protocol hooks into the stream logic for HA. IpHAManager is a static class that interfaces to a per-packet thread instance of the class IpHA. IpHA is sub-class of ProtocolHA, declared in the stream/base area. Thus each protocol within stream can have specific HA logic and interfaces.

IpHAManager::process_deletion() is called when an IP stream is being destroyed and indicates to the stream & flow HA logic that a flow deletion HA message needs to be emitted for the flow in question. Ip streams are closed due to timeout or pruning actions which lead to the invocation of process_deletion().

IpHA::create_session() is called from the stream & flow HA logic and handles the creation of new flow upon receiving an HA update message.

ip_defrag.h

Path = src/stream/ip/ip_defrag.h

#ifndef IP_DEFRAG_H
#define IP_DEFRAG_H

// ip datagram reassembly

#include <cstdint>

struct FragEngine;
struct FragTracker;
struct Fragment;
namespace snort
{
struct Packet;
struct SnortConfig;
}

class Defrag
{
public:
    Defrag(FragEngine&);

    bool configure(snort::SnortConfig*);
    void show() const;

    void process(snort::Packet*, FragTracker*);
    void cleanup(FragTracker*);

    static void init();

private:
    int insert(snort::Packet*, FragTracker*, FragEngine*);
    int new_tracker(snort::Packet* p, FragTracker*);

    int add_frag_node(  // FIXIT-L too many args
        FragTracker* ft, FragEngine*,
        const uint8_t* fragStart, int16_t fragLength,
        char lastfrag, int16_t len,
        uint16_t slide, uint16_t trunc, uint16_t frag_offset,
        Fragment* left, Fragment** retFrag);

    int dup_frag_node(FragTracker*, Fragment* left, Fragment** retFrag);
    int expired(snort::Packet*, FragTracker*, FragEngine*);

private:
    FragEngine& engine;
    uint8_t layers;
};

#endif

ip_ha.h

Path = src/stream/ip/ip_ha.h

#ifndef IP_HA_H
#define IP_HA_H

#include "stream/base/stream_ha.h"

//-------------------------------------------------------------------------

namespace snort
{
class Flow;
}

class IpHA : public ProtocolHA
{
public:
    IpHA() : ProtocolHA(PktType::IP) { }
    snort::Flow* create_session(const snort::FlowKey*) override;

private:
};

class IpHAManager
{
public:
    static void process_deletion(snort::Flow& flow);
    static void tinit();
    static void tterm();
    static THREAD_LOCAL IpHA* ip_ha;
};
#endif

ip_module.h

Path = src/stream/ip/ip_module.h

#ifndef IP_MODULE_H
#define IP_MODULE_H

#include "flow/session.h"
#include "framework/module.h"

namespace snort
{
class Trace;
struct SnortConfig;
}

extern THREAD_LOCAL const snort::Trace* stream_ip_trace;

#define GLOBAL_KEYWORD "defrag"
#define ENGINE_KEYWORD "defrag_engine"

#define GID_DEFRAG     123

#define DEFRAG_IPOPTIONS           1
#define DEFRAG_TEARDROP            2
#define DEFRAG_SHORT_FRAG          3
#define DEFRAG_ANOMALY_OVERSIZE    4
#define DEFRAG_ANOMALY_ZERO        5
#define DEFRAG_ANOMALY_BADSIZE_SM  6
#define DEFRAG_ANOMALY_BADSIZE_LG  7
#define DEFRAG_ANOMALY_OVLP        8

/* 123:9, 123:10 are OBE w/ addition of 116:458
 * (aka DECODE_IPV6_BAD_FRAG_PKT).
 * Leave these here so they are not reused.
 * ------
#define DEFRAG_IPV6_BSD_ICMP_FRAG  9
#define DEFRAG_IPV6_BAD_FRAG_PKT  10
 * ------
*/
#define DEFRAG_MIN_TTL_EVASION    11
#define DEFRAG_EXCESSIVE_OVERLAP  12
#define DEFRAG_TINY_FRAGMENT      13

/* statistics tracking struct */
struct IpStats
{
    SESSION_STATS;
    PegCount total_bytes;        // total_ip_bytes_processed
    PegCount total;             // total_ipfragmented_packets
    PegCount current_frags;     // iCurrentFrags
    PegCount max_frags;         // iMaxFrags
    PegCount reassembles;       // total_ipreassembled_packets / iFragFlushes
    PegCount discards;
    PegCount frag_timeouts;     // iFragTimeouts
    PegCount overlaps;
    PegCount anomalies;
    PegCount alerts;
    PegCount drops;
    PegCount trackers_created;  // iFragCreates
    PegCount trackers_released;
    PegCount trackers_cleared;  // iFragDeletes - delete meant dump the frag list
    PegCount trackers_completed;// iFragComplete
    PegCount nodes_created;     // iFragInserts tracked a similar stat (# calls to insert)
    PegCount nodes_released;
    PegCount reassembled_bytes; // total_ipreassembled_bytes
    PegCount fragmented_bytes;  // total_ipfragmented_bytes
};

extern const PegInfo ip_pegs[];
extern THREAD_LOCAL snort::ProfileStats ip_perf_stats;

//-------------------------------------------------------------------------
// stream_ip module
//-------------------------------------------------------------------------

#define MOD_NAME "stream_ip"
#define MOD_HELP "stream inspector for IP flow tracking and defragmentation"

struct StreamIpConfig;

class StreamIpModule : public snort::Module
{
public:
    StreamIpModule();
    ~StreamIpModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;

    const snort::RuleMap* get_rules() const override;
    snort::ProfileStats* get_profile() const override;
    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    StreamIpConfig* get_data();

    unsigned get_gid() const override
    { return GID_DEFRAG; }

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

    void set_trace(const snort::Trace*) const override;
    const snort::TraceOption* get_trace_options() const override;

private:
    StreamIpConfig* config;
};

#endif

ip_session.h

Path = src/stream/ip/ip_session.h

#ifndef IP_SESSION_H
#define IP_SESSION_H

#include "stream/ip/ip_module.h"

struct Fragment;
struct FragEngine;

/* Only track a certain number of alerts per session */
#define MAX_FRAG_ALERTS 8

/* tracker for a fragmented packet set */
struct FragTracker
{
    IpProtocol ip_proto;      /* IP protocol */

    uint8_t ttl;           /* ttl used to detect evasions */
    uint8_t alerted;
    uint32_t frag_flags;   /* bit field */

    uint32_t frag_bytes;   /* number of fragment bytes stored, based
                             * on aligned fragment offsets/sizes
                             */

    uint32_t calculated_size; /* calculated size of reassembled pkt, based on
                                * last frag offset
                                */

    uint32_t frag_pkts;   /* number of frag pkts stored under this tracker */

    struct timeval frag_time; /* time we started tracking this frag */

    Fragment* fraglist;      /* list of fragments */
    Fragment* fraglist_tail; /* tail ptr for easy appending */
    int fraglist_count;       /* handy dandy counter */

    uint32_t alert_gid[MAX_FRAG_ALERTS]; /* flag alerts seen in a frag list  */
    uint32_t alert_sid[MAX_FRAG_ALERTS]; /* flag alerts seen in a frag list  */
    uint8_t alert_count;                 /* count alerts seen in a frag list */

    uint8_t ip_options_len;  /* length of ip options for this set of frags */
    uint8_t* ip_options_data; /* ip options from offset 0 packet */
    uint8_t copied_ip_options_len;  /* length of 'copied' ip options */

    FragEngine* engine;

    int ordinal;

    int application_protocol;
    uint32_t frag_policy;

    // Count of IP fragment overlap for each packet id.
    uint32_t overlap_count;
};

class IpSession : public Session
{
public:
    IpSession(snort::Flow*);
    ~IpSession() override;

    bool setup(snort::Packet*) override;
    int process(snort::Packet*) override;
    void clear() override;

    bool add_alert(snort::Packet*, uint32_t gid, uint32_t sid) override;
    bool check_alerted(snort::Packet*, uint32_t gid, uint32_t sid) override;

public:
    FragTracker tracker = {};
};

extern THREAD_LOCAL IpStats ip_stats;

#endif

stream_ip.h

Path = src/stream/ip/stream_ip.h

#ifndef STREAM_IP_H
#define STREAM_IP_H

#include "framework/inspector.h"

/* engine-based defragmentation policy enums */
// must update stream.h::IP_POLICIES if this changes
enum
{
    FRAG_POLICY_FIRST = 1,
    FRAG_POLICY_LINUX,
    FRAG_POLICY_BSD,
    FRAG_POLICY_BSD_RIGHT,
    FRAG_POLICY_LAST,
/* Combo of FIRST & LAST, depending on overlap situation. */
    FRAG_POLICY_WINDOWS,
/* Combo of FIRST & LAST, depending on overlap situation. */
    FRAG_POLICY_SOLARIS
};

#define FRAG_POLICY_DEFAULT FRAG_POLICY_LINUX

struct FragEngine
{
    uint32_t max_frags;
    uint32_t max_overlaps;
    uint32_t min_fragment_length;

    uint32_t frag_timeout; /* timeout for frags in this policy */
    uint16_t frag_policy;  /* policy to use for engine-based reassembly */

    uint8_t min_ttl;       /* Minimum TTL to accept */

    FragEngine();
};

struct StreamIpConfig
{
    FragEngine frag_engine;
    uint32_t session_timeout;

    StreamIpConfig();
};

StreamIpConfig* get_ip_cfg(snort::Inspector*);
class Defrag* get_defrag(snort::Inspector*);

#endif

stream/tcp/

This directory contains the implementation of TCP session tracking and processing functions. When the network protocol for a flow is determined to be TCP the base Stream preprocessor will delegate handling of the packets on that flow to this module.

The Stream TCP module is currently in process of a complete rewrite. These dev notes describe the current implementation.

The StreamTcp class is implemented as a subclass of Inspector and provides functions for loading stream TCP configuration and packet evaluation. The packet eval method is not used as the base Stream Inspector delegates packets directly to the TCP session packet processing method.

  • TCP Segment Descriptor - this class provides access to the various fields of the TCP header and payload

  • TCP Stream Tracker - this class encapsulates all the state information required for tracking one side of the TCP connection. For each flow that is tracked there will be two instances of this tracker, one for each direction.

  • TCP State Handler - abstract class interface that defines a method for handling each possible TCP event. For each TCP state a subclass of this class is created with a state specific implementation for each event handling method.

  • TCP State Machine - this class is the engine that dispatches processing to the correct event handling method of the handler for the current TCP state of the flow.

The TCP session module implements the following functions:

  • TCP segment normalization. Variations in handling normalization are configured by policy.

  • TCP connection tracking. The state of the connection from the perspective of each end of the connection is tracked.

  • TCP reassembly. Variations in handling reassembly are configured by policy.

  • Event generation for anomalies detected while processing the TCP segments.

A TcpSession object is allocated for each TCP flow being tracked. TCP session state is maintained primarily in the TcpTracker data structure. State information includes:

  • policy settings

  • list of tcp segments being reassembled

  • TCP connection state

  • paf state

  • alert history

An instance of this data structure is allocated and managed for each end of the connection.

The module tcp_ha.cc (and tcp_ha.h) implements the per-protocol hooks into the stream logic for HA. TcpHAManager is a static class that interfaces to a per-packet thread instance of the class TcpHA. TcpHA is sub-class of ProtocolHA, declared in the stream/base area. Thus each protocol within stream can have specific HA logic and interfaces.

TcpHAManager::process_deletion() is called when an TCP stream is being destroyed and indicates to the stream & flow HA logic that a flow deletion HA message needs to be emitted for the flow in question. Tcp streams are both closed internally (e.g. FIN) and externally due to cache timeout or pruning. The calls to TcpHAManager::process_deletion() in tcp/tcp_session.cc indicate a normal closure of a stream/flow.

TcpHA::create_session() is called from the stream & flow HA logic and handles the creation of new flow upon receiving an HA update message.

TcpHA::deactivate_session() is called from the stream & flow HA logic to place a session into standby mode. Upon receiving an HA Update message, the flow is first created if necessary, and is then placed into Standby state. deactivate_session() sets the TCP specific state for Standby mode.

One HA state transition is implemented within this TCP library. In TcpStreamSession::clear(), TcpHAManager::process_deletion() is invoked to cause HA to generate a Deletion message for the target flow. This handles the case where a TCP session is being removed from from the flow cache due to a timeout or pruning function. Other normal TCP stream closure actions are handled in the ../tcp/tcp_session.cc module.

held_packet_queue.h

Path = src/stream/tcp/held_packet_queue.h

#ifndef HELD_PACKET_QUEUE_H
#define HELD_PACKET_QUEUE_H

#include <daq_common.h>

#include <ctime>
#include <list>

class TcpStreamTracker;

class HeldPacket
{
public:

    HeldPacket(DAQ_Msg_h msg, uint32_t seq, const timeval& timeout, TcpStreamTracker& trk);

    bool has_expired(const timeval& cur_time)
    {
        expired = (timercmp(&cur_time, &expiration, <) == 0);
        return expired;
    }

    bool has_expired()
    { return expired; }

    TcpStreamTracker& get_tracker() const { return tracker; }
    DAQ_Msg_h get_daq_msg() const { return daq_msg; }
    uint32_t get_seq_num() const { return seq_num; }
    void adjust_expiration(const timeval& delta, bool up);

private:
    DAQ_Msg_h daq_msg;
    uint32_t seq_num;
    timeval expiration;
    TcpStreamTracker& tracker;
    bool expired;
};

class HeldPacketQueue
{
public:

    using list_t = std::list<HeldPacket>;
    using iter_t = list_t::iterator;

    iter_t append(DAQ_Msg_h msg, uint32_t seq, TcpStreamTracker& trk);
    void erase(iter_t it);

    // Return whether there still are expired packets in the queue.
    bool execute(const timeval& cur_time, int max_remove);

    void set_timeout(uint32_t ms)
    {
        timeout = { static_cast<time_t>(ms) / 1000, static_cast<suseconds_t>((ms % 1000) * 1000) };
    }

    // Return the timeout in milliseconds.
    uint32_t get_timeout() const
    { return timeout.tv_sec * 1000 + timeout.tv_usec / 1000; }

    bool empty() const
    { return q.empty(); }

    // This must be called at reload time only, with now = reload time.
    // Return true if, upon exit, there are expired packets in the queue.
    bool adjust_expiration(uint32_t new_timeout_ms, const timeval& now);

private:
    timeval timeout = {1, 0};
    list_t q;
};

#endif

segment_overlap_editor.h

Path = src/stream/tcp/segment_overlap_editor.h

#ifndef SEGMENT_OVERLAP_EDITOR_H
#define SEGMENT_OVERLAP_EDITOR_H

#include <vector>

#include "normalize/norm_stats.h"
#include "stream/paf.h"
#include "stream/stream.h"
#include "tcp_segment_node.h"

class TcpSession;
class TcpStreamTracker;

struct SegmentOverlapState
{
    TcpSession* session;
    TcpSegmentDescriptor* tsd;
    TcpSegmentNode* left;
    TcpSegmentNode* right;
    const uint8_t* rdata;

    TcpSegmentList seglist;
    uint32_t seglist_base_seq;      /* seq of first queued segment */
    uint32_t seg_count;             /* number of current queued segments */
    uint32_t seg_bytes_total;       /* total bytes currently queued */
    uint32_t seg_bytes_logical;     /* logical bytes queued (total - overlaps) */
    uint32_t total_bytes_queued;    /* total bytes queued (life of session) */
    uint32_t total_segs_queued;     /* number of segments queued (life) */
    uint32_t overlap_count;         /* overlaps encountered */

    uint32_t seq;
    uint32_t seq_end;
    uint32_t rseq;

    int32_t overlap;
    int32_t slide;
    int32_t trunc_len;

    uint16_t len;
    uint16_t rsize;
    int8_t tcp_ips_data;
    StreamPolicy reassembly_policy;

    bool keep_segment;

    ~SegmentOverlapState()
    { seglist.reset(); }

    void init_sos(TcpSession*, StreamPolicy);
    void init_soe(TcpSegmentDescriptor& tsd, TcpSegmentNode* left, TcpSegmentNode* right);
};

struct StreamAlertInfo : snort::AlertInfo
{
    StreamAlertInfo(uint32_t gid_, uint32_t sid_, uint32_t seq_num_ = 0, uint32_t id_ = 0, uint32_t ts_ = 0)
        : snort::AlertInfo(gid_, sid_, id_, ts_), seq(seq_num_)
    {}

    uint32_t seq;
};

struct TcpReassemblerState
{
    SegmentOverlapState sos;
    TcpStreamTracker* tracker;
    uint32_t flush_count;   // number of flushed queued segments
    uint32_t xtradata_mask; // extra data available to log
    std::vector<StreamAlertInfo> alerts;
    uint8_t ignore_dir;
    uint8_t packet_dir;
    bool server_side;
    PAF_State paf_state;
};

class SegmentOverlapEditor
{
protected:
    SegmentOverlapEditor() = default;
    virtual ~SegmentOverlapEditor() = default;

    void eval_left(TcpReassemblerState&);
    void eval_right(TcpReassemblerState&);

    virtual bool is_segment_retransmit(TcpReassemblerState&, bool*);
    virtual void drop_old_segment(TcpReassemblerState&);
    virtual bool zwp_data_mismatch(TcpReassemblerState&, TcpSegmentDescriptor&, uint32_t);

    virtual void left_overlap_keep_first(TcpReassemblerState&);
    virtual void left_overlap_trim_first(TcpReassemblerState&);
    virtual void left_overlap_keep_last(TcpReassemblerState&);
    virtual void right_overlap_truncate_existing(TcpReassemblerState&);
    virtual void right_overlap_truncate_new(TcpReassemblerState&);
    virtual void full_right_overlap_truncate_new(TcpReassemblerState&);
    virtual void full_right_overlap_os1(TcpReassemblerState&);
    virtual void full_right_overlap_os2(TcpReassemblerState&);
    virtual void full_right_overlap_os3(TcpReassemblerState&);
    virtual void full_right_overlap_os4(TcpReassemblerState&);
    virtual void full_right_overlap_os5(TcpReassemblerState&);

    virtual void insert_left_overlap(TcpReassemblerState&) = 0;
    virtual void insert_right_overlap(TcpReassemblerState&) = 0;
    virtual void insert_full_overlap(TcpReassemblerState&) = 0;

    virtual void add_reassembly_segment(
        TcpReassemblerState&, TcpSegmentDescriptor&, uint16_t, uint32_t,
        uint32_t, uint32_t, TcpSegmentNode*) = 0;

    virtual void dup_reassembly_segment(TcpReassemblerState&, TcpSegmentNode*, TcpSegmentNode**) = 0;
    virtual int delete_reassembly_segment(TcpReassemblerState&, TcpSegmentNode*) = 0;
    virtual void print(TcpReassemblerState&);
};

#endif

stream_tcp.h

Path = src/stream/tcp/stream_tcp.h

#ifndef STREAM_TCP_H
#define STREAM_TCP_H

namespace snort
{
class Inspector;
}
class TcpStreamConfig;

TcpStreamConfig* get_tcp_cfg(snort::Inspector*);

#endif

tcp_defs.h

Path = src/stream/tcp/tcp_defs.h

#ifndef TCP_DEFS_H
#define TCP_DEFS_H

#include "main/thread.h"

namespace snort
{
struct Packet;
}

/* actions */
#define ACTION_NOTHING               0x00000000
#define ACTION_RST                   0x00000001
#define ACTION_BAD_PKT               0x00000002
#define ACTION_LWSSN_CLOSED          0x00000004
#define ACTION_DISABLE_INSPECTION    0x00000008

#define TF_NONE             0x0000
#define TF_WSCALE           0x0001
#define TF_TSTAMP           0x0002
#define TF_TSTAMP_ZERO      0x0004
#define TF_MSS              0x0008
#define TF_FORCE_FLUSH      0x0010
#define TF_PKT_MISSED       0x0020  // sticky
#define TF_MISSING_PKT      0x0040  // used internally
#define TF_MISSING_PREV_PKT 0x0080  // reset for each reassembled

#define PAWS_WINDOW         60
#define PAWS_24DAYS         2073600         /* 24 days in seconds */

#define STREAM_UNALIGNED       0
#define STREAM_ALIGNED         1

#define MQ_NONE    0
#define MQ_BYTES   1
#define MQ_SEGS    2

#define STREAM_DEFAULT_MAX_SMALL_SEG_SIZE 0    /* disabled */
#define STREAM_DEFAULT_CONSEC_SMALL_SEGS 0     /* disabled */

#define SLAM_MAX 4

#define MAX_ZERO_WIN_PROBE_LEN 1

// target-based policy types - changes to this enum require changes to stream.h::TCP_POLICIES
enum StreamPolicy : uint8_t
{
    OS_FIRST = 0,
    OS_LAST,
    OS_LINUX,
    OS_OLD_LINUX,
    OS_BSD,
    OS_MACOS,
    OS_SOLARIS,
    OS_IRIX,
    OS_HPUX11,
    OS_HPUX10,
    OS_WINDOWS,
    OS_WINDOWS2K3,
    OS_VISTA,
    OS_PROXY,
    OS_END_OF_LIST,
    OS_DEFAULT = OS_BSD
};

// increment operator...
inline StreamPolicy& operator++(StreamPolicy& c, int)
{
    if ( c < StreamPolicy::OS_END_OF_LIST )
        c = static_cast<StreamPolicy>( static_cast<int>(c) + 1 );
    else
        c = StreamPolicy::OS_END_OF_LIST;

    return c;
}

enum FlushPolicy
{
    STREAM_FLPOLICY_IGNORE, /* ignore this traffic */
    STREAM_FLPOLICY_ON_ACK, /* protocol aware flushing (PAF) */
    STREAM_FLPOLICY_ON_DATA, /* protocol aware ips */
};

#endif

tcp_event_logger.h

Path = src/stream/tcp/tcp_event_logger.h

#ifndef TCP_EVENT_LOGGER_H
#define TCP_EVENT_LOGGER_H

#include <cstdint>

/* events */
#define EVENT_SYN_ON_EST                0x00000001
#define EVENT_DATA_ON_SYN               0x00000002
#define EVENT_DATA_ON_CLOSED            0x00000004
#define EVENT_BAD_TIMESTAMP             0x00000008
#define EVENT_WINDOW_TOO_LARGE          0x00000010
#define EVENT_DATA_AFTER_RESET          0x00000020
#define EVENT_SESSION_HIJACK_CLIENT     0x00000040
#define EVENT_SESSION_HIJACK_SERVER     0x00000080
#define EVENT_DATA_WITHOUT_FLAGS        0x00000100
#define EVENT_4WHS                      0x00000200
#define EVENT_NO_TIMESTAMP              0x00000400
#define EVENT_BAD_RST                   0x00000800
#define EVENT_BAD_FIN                   0x00001000
#define EVENT_BAD_ACK                   0x00002000
#define EVENT_DATA_AFTER_RST_RCVD       0x00004000
#define EVENT_WINDOW_SLAM               0x00008000
#define EVENT_NO_3WHS                   0x00010000
#define EVENT_BAD_SEGMENT               0x00020000
#define EVENT_EXCESSIVE_OVERLAP         0x00040000
#define EVENT_MAX_SMALL_SEGS_EXCEEDED   0x00080000
#define EVENT_MAX_QUEUED_BYTES_EXCEEDED 0x00100000
#define EVENT_MAX_QUEUED_SEGS_EXCEEDED  0x00200000

class TcpEventLogger
{
public:
    TcpEventLogger() = default;

    void clear_tcp_events()
    { tcp_events = 0; }

    void set_tcp_event(int eventcode)
    { tcp_events |= eventcode; }

    void log_tcp_events();
    void log_internal_event(uint32_t eventSid);

private:
    uint32_t tcp_events = 0;
};
#endif

tcp_ha.h

Path = src/stream/tcp/tcp_ha.h

#ifndef TCP_HA_H
#define TCP_HA_H

#include "stream/base/stream_ha.h"

//-------------------------------------------------------------------------

namespace snort
{
class Flow;
}

class TcpHA : public ProtocolHA
{
public:
    TcpHA() : ProtocolHA(PktType::TCP) { }
    snort::Flow* create_session(const snort::FlowKey*) override;
    void deactivate_session(snort::Flow*) override;

private:
};

class TcpHAManager
{
public:
    static void process_deletion(snort::Flow& flow);
    static void tinit();
    static void tterm();
    static THREAD_LOCAL TcpHA* tcp_ha;
};
#endif

tcp_module.h

Path = src/stream/tcp/tcp_module.h

#ifndef TCP_MODULE_H
#define TCP_MODULE_H

#include "flow/session.h"
#include "framework/module.h"
#include "stream/tcp/tcp_stream_config.h"

#define GID_STREAM_TCP  129

#define STREAM_TCP_SYN_ON_EST                      1
#define STREAM_TCP_DATA_ON_SYN                     2
#define STREAM_TCP_DATA_ON_CLOSED                  3
#define STREAM_TCP_BAD_TIMESTAMP                   4
#define STREAM_TCP_BAD_SEGMENT                     5
#define STREAM_TCP_WINDOW_TOO_LARGE                6
#define STREAM_TCP_EXCESSIVE_TCP_OVERLAPS          7
#define STREAM_TCP_DATA_AFTER_RESET                8
#define STREAM_TCP_SESSION_HIJACKED_CLIENT         9
#define STREAM_TCP_SESSION_HIJACKED_SERVER        10
#define STREAM_TCP_DATA_WITHOUT_FLAGS             11
#define STREAM_TCP_SMALL_SEGMENT                  12
#define STREAM_TCP_4WAY_HANDSHAKE                 13
#define STREAM_TCP_NO_TIMESTAMP                   14
#define STREAM_TCP_BAD_RST                        15
#define STREAM_TCP_BAD_FIN                        16
#define STREAM_TCP_BAD_ACK                        17
#define STREAM_TCP_DATA_AFTER_RST_RCVD            18
#define STREAM_TCP_WINDOW_SLAM                    19
#define STREAM_TCP_NO_3WHS                        20
#define STREAM_TCP_MAX_QUEUED_BYTES_EXCEEDED      21
#define STREAM_TCP_MAX_QUEUED_SEGS_EXCEEDED       22
#define STREAM_TCP_MAX_EVENTS                     32

extern const PegInfo tcp_pegs[];

extern THREAD_LOCAL const snort::Trace* stream_tcp_trace;
extern THREAD_LOCAL snort::ProfileStats s5TcpPerfStats;

struct TcpStats
{
    SESSION_STATS;
    PegCount instantiated;
    PegCount setups;
    PegCount restarts;
    PegCount resyns;
    PegCount discards;
    PegCount discards_skipped;
    PegCount invalid_seq_num;
    PegCount invalid_ack;
    PegCount no_flags_set;
    PegCount events;
    PegCount ignored;
    PegCount no_pickups;
    PegCount sessions_on_syn;
    PegCount sessions_on_syn_ack;
    PegCount sessions_on_3way;
    PegCount sessions_on_data;
    PegCount segs_queued;
    PegCount segs_released;
    PegCount segs_split;
    PegCount segs_used;
    PegCount rebuilt_packets;
    PegCount rebuilt_buffers;
    PegCount rebuilt_bytes;
    PegCount overlaps;
    PegCount gaps;
    PegCount exceeded_max_segs;
    PegCount exceeded_max_bytes;
    PegCount payload_fully_trimmed;
    PegCount internalEvents;
    PegCount client_cleanups;
    PegCount server_cleanups;
    PegCount mem_in_use;
    PegCount sessions_initializing;
    PegCount sessions_established;
    PegCount sessions_closing;
    PegCount syns;
    PegCount syn_acks;
    PegCount resets;
    PegCount fins;
    PegCount meta_acks;
    PegCount total_packets_held;
    PegCount held_packet_rexmits;
    PegCount held_packets_dropped;
    PegCount held_packets_passed;
    PegCount held_packet_timeouts;
    PegCount held_packet_purges;
    PegCount held_packet_retries;
    PegCount current_packets_held;
    PegCount max_packets_held;
    PegCount partial_flushes;
    PegCount partial_flush_bytes;
    PegCount inspector_fallbacks;
    PegCount partial_fallbacks;
    PegCount max_segs;
    PegCount max_bytes;
    PegCount zero_len_tcp_opt;
    PegCount zero_win_probes;
    PegCount proxy_mode_flows;
};

extern THREAD_LOCAL struct TcpStats tcpStats;

//-------------------------------------------------------------------------
// stream_tcp module
//-------------------------------------------------------------------------

#define STREAM_TCP_MOD_NAME "stream_tcp"
#define STREAM_TCP_MOD_HELP "stream inspector for TCP flow tracking and stream normalization and reassembly"

class StreamTcpModule : public snort::Module
{
public:
    StreamTcpModule();

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    const snort::RuleMap* get_rules() const override;

    void reset_stats() override;

    unsigned get_gid() const override
    { return GID_STREAM_TCP; }

    TcpStreamConfig* get_data();
    snort::ProfileStats* get_profile(unsigned, const char*&, const char*&) const override;
    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

    void set_trace(const snort::Trace*) const override;
    const snort::TraceOption* get_trace_options() const override;

private:
    TcpStreamConfig* config;
};

#endif

tcp_normalizer.h

Path = src/stream/tcp/tcp_normalizer.h

#ifndef TCP_NORMALIZER_H
#define TCP_NORMALIZER_H

#include "tcp_defs.h"

#include "main/thread.h"
#include "normalize/normalize.h"
#include "normalize/norm_stats.h"
#include "protocols/tcp_options.h"

class TcpStreamSession;
class TcpStreamTracker;
class TcpSegmentDescriptor;

struct TcpNormalizerState
{
    TcpStreamSession* session = nullptr;
    TcpStreamTracker* tracker = nullptr;
    TcpStreamTracker* peer_tracker = nullptr;

    StreamPolicy os_policy = StreamPolicy::OS_DEFAULT;

    int32_t paws_ts_fudge = 0;
    int tcp_ts_flags = 0;
    uint32_t zwp_seq = 0;

    int8_t trim_syn = 0;
    int8_t trim_rst = 0;
    int8_t trim_win = 0;
    int8_t trim_mss = 0;
    int8_t strip_ecn = 0;
    int8_t tcp_block = 0;
    int8_t opt_block = 0;

    bool tcp_ips_enabled = false;
    bool paws_drop_zero_ts = false;
};

class TcpNormalizer
{
public:
    using State = TcpNormalizerState;
    enum NormStatus { NORM_BAD_SEQ = -1, NORM_TRIMMED = 0, NORM_OK = 1 };

    virtual ~TcpNormalizer() = default;

    virtual void init(State&) { }

    virtual NormStatus apply_normalizations(
        State&, TcpSegmentDescriptor&, uint32_t seq, bool stream_is_inorder);
    virtual void session_blocker(State&, TcpSegmentDescriptor&);
    virtual bool packet_dropper(State&, TcpSegmentDescriptor&, NormFlags);
    virtual bool trim_syn_payload(State&, TcpSegmentDescriptor&, uint32_t max = 0);
    virtual void trim_rst_payload(State&, TcpSegmentDescriptor&, uint32_t max = 0);
    virtual void trim_win_payload(State&, TcpSegmentDescriptor&, uint32_t max = 0,
        bool force = false);
    virtual void trim_mss_payload(State&, TcpSegmentDescriptor&, uint32_t max = 0);
    virtual void ecn_tracker(State&, const snort::tcp::TCPHdr*, bool req3way);
    virtual void ecn_stripper(State&, TcpSegmentDescriptor&);
    virtual uint32_t get_zwp_seq(State&);
    virtual uint32_t get_stream_window(State&, TcpSegmentDescriptor&);
    virtual uint32_t data_inside_window(State&, TcpSegmentDescriptor&);
    virtual uint32_t get_tcp_timestamp(State&, TcpSegmentDescriptor&, bool strip);
    virtual int handle_paws(State&, TcpSegmentDescriptor&);
    virtual bool validate_rst(State&, TcpSegmentDescriptor&);
    virtual int handle_repeated_syn(State&, TcpSegmentDescriptor&) = 0;
    virtual uint16_t set_urg_offset(State&, const snort::tcp::TCPHdr* tcph, uint16_t dsize);
    virtual void set_zwp_seq(State&, uint32_t seq);
    virtual void log_drop_reason(State&, const TcpSegmentDescriptor&, bool inline_mode, const char *issuer, const std::string& log);

    static void reset_stats();

    std::string& get_name()
    { return my_name; }

protected:
    TcpNormalizer() = default;

    virtual bool trim_payload(State&, TcpSegmentDescriptor&, uint32_t, NormMode, PegCounts,
        bool force = false);
    virtual bool strip_tcp_timestamp(
        State&, TcpSegmentDescriptor&, const snort::tcp::TcpOption*, NormMode);
    virtual bool validate_rst_seq_geq(State&, TcpSegmentDescriptor&);
    virtual bool validate_rst_end_seq_geq(State&, TcpSegmentDescriptor&);
    virtual bool validate_rst_seq_eq(State&, TcpSegmentDescriptor&);

    virtual int validate_paws_timestamp(State&, TcpSegmentDescriptor&);
    virtual bool is_paws_ts_checked_required(State&, TcpSegmentDescriptor&);
    virtual int validate_paws(State&, TcpSegmentDescriptor&);
    virtual int handle_paws_no_timestamps(State&, TcpSegmentDescriptor&);

    std::string my_name;
};

#endif

tcp_normalizers.h

Path = src/stream/tcp/tcp_normalizers.h

#ifndef TCP_NORMALIZERS_H
#define TCP_NORMALIZERS_H

#include "stream/tcp/tcp_normalizer.h"

class TcpStreamSession;
class TcpStreamSession;

class TcpNormalizerFactory
{
public:
    static void initialize();
    static void term();
    static TcpNormalizer* get_instance(StreamPolicy);

private:
    TcpNormalizerFactory() = delete;

    static TcpNormalizer* normalizers[StreamPolicy::OS_END_OF_LIST];
};

class TcpNormalizerPolicy
{
public:
    TcpNormalizerPolicy() = default;
    ~TcpNormalizerPolicy() = default;

    void init(StreamPolicy os, TcpStreamSession* ssn, TcpStreamTracker* trk, TcpStreamTracker* peer);
    void reset()
    { init(StreamPolicy::OS_DEFAULT, nullptr, nullptr, nullptr); }

    TcpNormalizer::NormStatus apply_normalizations(TcpSegmentDescriptor& tsd, uint32_t seq, bool stream_is_inorder)
    { return norm->apply_normalizations(tns, tsd, seq, stream_is_inorder); }

    void session_blocker(TcpSegmentDescriptor& tsd)
    { norm->session_blocker(tns, tsd); }

    bool packet_dropper(TcpSegmentDescriptor& tsd, NormFlags nflags)
    { return norm->packet_dropper(tns, tsd, nflags); }

    bool trim_syn_payload(TcpSegmentDescriptor& tsd, uint32_t max = 0)
    { return norm->trim_syn_payload(tns, tsd, max); }

    void trim_rst_payload(TcpSegmentDescriptor& tsd, uint32_t max = 0)
    { norm->trim_rst_payload(tns, tsd, max); }

    void trim_win_payload(TcpSegmentDescriptor& tsd, uint32_t max = 0, bool force = false)
    { norm->trim_win_payload(tns, tsd, max, force); }

    void trim_mss_payload(TcpSegmentDescriptor& tsd, uint32_t max = 0)
    { norm->trim_mss_payload(tns, tsd, max); }

    void ecn_tracker(const snort::tcp::TCPHdr* tcph, bool req3way)
    { norm->ecn_tracker(tns, tcph, req3way); }

    void ecn_stripper(TcpSegmentDescriptor& tsd)
    { norm->ecn_stripper(tns, tsd); }

    uint32_t get_zwp_seq()
    { return norm->get_zwp_seq(tns); }

    uint32_t get_stream_window(TcpSegmentDescriptor& tsd)
    { return norm->get_stream_window(tns, tsd); }

    uint32_t data_inside_window(TcpSegmentDescriptor& tsd)
    { return norm->data_inside_window(tns, tsd);  }

    uint32_t get_tcp_timestamp(TcpSegmentDescriptor& tsd, bool strip)
    { return norm->get_tcp_timestamp(tns, tsd, strip); }

    int handle_paws(TcpSegmentDescriptor& tsd)
    { return norm->handle_paws(tns, tsd); }

    bool validate_rst(TcpSegmentDescriptor& tsd)
    { return norm->validate_rst(tns, tsd); }

    int handle_repeated_syn(TcpSegmentDescriptor& tsd)
    { return norm->handle_repeated_syn(tns, tsd); }

    void set_zwp_seq(uint32_t seq)
    { return norm->set_zwp_seq(tns, seq); }

    void log_drop_reason(const TcpSegmentDescriptor& tsd, bool inline_mode, const char *issuer, const std::string& log)
    { return norm->log_drop_reason(tns, tsd , inline_mode, issuer, log); }

    uint16_t set_urg_offset(const snort::tcp::TCPHdr* tcph, uint16_t dsize)
    { return norm->set_urg_offset(tns, tcph, dsize); }

    StreamPolicy get_os_policy() const
    { return tns.os_policy; }

    bool is_paws_drop_zero_ts() const
    { return tns.paws_drop_zero_ts; }

    int32_t get_paws_ts_fudge() const
    { return tns.paws_ts_fudge; }

    int8_t get_opt_block() const
    { return tns.opt_block; }

    int8_t get_strip_ecn() const
    { return tns.strip_ecn; }

    int8_t get_tcp_block() const
    { return tns.tcp_block; }

    int8_t get_trim_rst() const
    { return tns.trim_rst; }

    int8_t get_trim_syn() const
    { return tns.trim_syn; }

    int8_t get_trim_mss() const
    { return tns.trim_mss; }

    int8_t get_trim_win() const
    { return tns.trim_win; }

    bool is_tcp_ips_enabled() const
    { return tns.tcp_ips_enabled; }

    bool handling_timestamps() const
    { return tns.tcp_ts_flags != TF_NONE; }

    uint32_t get_timestamp_flags()
    { return tns.tcp_ts_flags; }

private:
    TcpNormalizer* norm = nullptr;
    TcpNormalizerState tns;
};

#endif

tcp_reassembler.h

Path = src/stream/tcp/tcp_reassembler.h

#ifndef TCP_REASSEMBLER_H
#define TCP_REASSEMBLER_H

#include "stream/stream.h"
#include "stream/tcp/segment_overlap_editor.h"

class TcpReassembler : public SegmentOverlapEditor
{
public:

    // OK means FIN seen, data scanned, flush point not found, no gaps
    enum ScanStatus {
        FINAL_FLUSH_HOLD = -2,
        FINAL_FLUSH_OK = -1
    };

    virtual void queue_packet_for_reassembly(TcpReassemblerState&, TcpSegmentDescriptor&);
    virtual void purge_segment_list(TcpReassemblerState&);
    virtual void purge_flushed_ackd(TcpReassemblerState&);
    virtual int flush_stream(
        TcpReassemblerState&, snort::Packet* p, uint32_t dir, bool final_flush = false);
    virtual void flush_queued_segments(
        TcpReassemblerState&, snort::Flow* flow, bool clear, const snort::Packet* = nullptr);
    void finish_and_final_flush(
        TcpReassemblerState&, snort::Flow* flow, bool clear, snort::Packet*);
    virtual bool is_segment_pending_flush(const TcpReassemblerState&) const;
    virtual int flush_on_data_policy(TcpReassemblerState&, snort::Packet*);
    virtual int flush_on_ack_policy(TcpReassemblerState&, snort::Packet*);
    virtual bool add_alert(TcpReassemblerState&, uint32_t gid, uint32_t sid);
    virtual bool check_alerted(TcpReassemblerState&, uint32_t gid, uint32_t sid);
    virtual int update_alert(TcpReassemblerState&, uint32_t gid, uint32_t sid,
        uint32_t event_id, uint32_t event_second);
    virtual void purge_alerts(TcpReassemblerState&);
    virtual bool segment_within_seglist_window(TcpReassemblerState&, TcpSegmentDescriptor&);
    void skip_midstream_pickup_seglist_hole(TcpReassemblerState&, TcpSegmentDescriptor&);
    void initialize_paf(TcpReassemblerState& trs)
    {
        // only initialize if we have a data segment queued
        if ( !trs.sos.seglist.head )
            return;

        if ( !paf_initialized(&trs.paf_state) or SEQ_GT(trs.paf_state.seq, trs.sos.seglist.head->i_seq) )
            paf_initialize(&trs.paf_state, trs.sos.seglist.head->i_seq);
    }

    uint32_t perform_partial_flush(TcpReassemblerState&, snort::Flow*, snort::Packet*&);

protected:
    TcpReassembler() = default;

    void add_reassembly_segment(
        TcpReassemblerState&, TcpSegmentDescriptor&, uint16_t len, uint32_t slide,
        uint32_t trunc, uint32_t seq, TcpSegmentNode* left) override;

    void dup_reassembly_segment(
        TcpReassemblerState&, TcpSegmentNode* left, TcpSegmentNode** retSeg) override;
    int delete_reassembly_segment(TcpReassemblerState&, TcpSegmentNode*) override;
    virtual void insert_segment_in_empty_seglist(TcpReassemblerState&, TcpSegmentDescriptor&);
    virtual void insert_segment_in_seglist(TcpReassemblerState&, TcpSegmentDescriptor&);
    virtual uint32_t get_pending_segment_count(const TcpReassemblerState&, unsigned max) const;
    int trim_delete_reassembly_segment(TcpReassemblerState&, TcpSegmentNode*, uint32_t flush_seq);
    void queue_reassembly_segment(TcpReassemblerState&, TcpSegmentNode* prev, TcpSegmentNode*);
    void init_overlap_editor(TcpReassemblerState&, TcpSegmentDescriptor&);
    bool is_segment_fasttrack
        (TcpReassemblerState&, TcpSegmentNode* tail, const TcpSegmentDescriptor&);
    void show_rebuilt_packet(const TcpReassemblerState&, snort::Packet*);
    int flush_data_segments(TcpReassemblerState&, uint32_t flush_len, snort::Packet* pdu);
    void prep_pdu(
        TcpReassemblerState&, snort::Flow*, snort::Packet*, uint32_t pkt_flags, snort::Packet*);
    snort::Packet* initialize_pdu(
        TcpReassemblerState&, snort::Packet*, uint32_t pkt_flags, struct timeval);
    int flush_to_seq(TcpReassemblerState&, uint32_t bytes, snort::Packet*, uint32_t pkt_flags);
    int do_zero_byte_flush(TcpReassemblerState&, snort::Packet*, uint32_t pkt_flags);
    uint32_t get_q_footprint(TcpReassemblerState&);
    uint32_t get_q_sequenced(TcpReassemblerState&);
    bool is_q_sequenced(TcpReassemblerState&);
    void final_flush(TcpReassemblerState&, snort::Packet*, uint32_t dir);
    uint32_t get_reverse_packet_dir(TcpReassemblerState&, const snort::Packet*);
    uint32_t get_forward_packet_dir(TcpReassemblerState&, const snort::Packet*);
    int32_t scan_data_pre_ack(TcpReassemblerState&, uint32_t*, snort::Packet*);
    void fallback(TcpStreamTracker&, bool server_side);
    int32_t scan_data_post_ack(TcpReassemblerState&, uint32_t* flags, snort::Packet*);
    void purge_to_seq(TcpReassemblerState&, uint32_t flush_seq);
    void purge_segments_left_of_hole(TcpReassemblerState&, const TcpSegmentNode*);

    bool next_no_gap(const TcpSegmentNode&);
    bool next_no_gap_c(const TcpSegmentNode&);
    bool next_acked_no_gap_c(const TcpSegmentNode&, const TcpReassemblerState&);
    bool fin_no_gap(const TcpSegmentNode&, const TcpReassemblerState&);
    bool fin_acked_no_gap(const TcpSegmentNode&, const TcpReassemblerState&);
    void update_next(TcpReassemblerState&, const TcpSegmentNode&);
    void update_skipped_bytes(uint32_t, TcpReassemblerState&);
    void check_first_segment_hole(TcpReassemblerState&);
    void update_rcv_nxt(TcpReassemblerState&, TcpSegmentNode&);
    bool has_seglist_hole(TcpReassemblerState&, TcpSegmentNode&, PAF_State&, uint32_t& total,
        uint32_t& flags);
    void skip_seglist_hole(TcpReassemblerState&, snort::Packet*, uint32_t flags,
        int32_t flush_amt);

    uint32_t perform_partial_flush(TcpReassemblerState&, snort::Packet*, uint32_t flushed = 0);
};

#endif

tcp_reassemblers.h

Path = src/stream/tcp/tcp_reassemblers.h

#ifndef TCP_REASSEMBLERS_H
#define TCP_REASSEMBLERS_H

#include "tcp_reassembler.h"

class TcpReassemblerFactory
{
public:
    static void initialize();
    static void term();
    static TcpReassembler* get_instance(StreamPolicy);

private:
    TcpReassemblerFactory() = delete;

    static TcpReassembler* reassemblers[StreamPolicy::OS_END_OF_LIST];
};

class TcpReassemblerPolicy
{
public:
    TcpReassemblerPolicy() = default;
    ~TcpReassemblerPolicy() = default;

    void init(TcpSession* ssn, TcpStreamTracker* trk, StreamPolicy pol, bool server);
    void reset();

    void queue_packet_for_reassembly(TcpSegmentDescriptor& tsd)
    { reassembler->queue_packet_for_reassembly(trs, tsd); }

    bool add_alert(uint32_t gid, uint32_t sid)
    { return reassembler->add_alert(trs, gid, sid); }

    bool check_alerted(uint32_t gid, uint32_t sid)
    { return reassembler->check_alerted(trs, gid, sid); }

    int update_alert(uint32_t gid, uint32_t sid, uint32_t event_id, uint32_t event_second)
    { return reassembler->update_alert(trs, gid, sid, event_id, event_second); }

    void purge_alerts()
    { reassembler->purge_alerts(trs); }

    void purge_segment_list()
    { reassembler->purge_segment_list(trs); }

    void purge_flushed_ackd()
    { return reassembler->purge_flushed_ackd(trs); }

    int flush_stream(snort::Packet* p, uint32_t dir, bool final_flush = false)
    { return reassembler->flush_stream(trs, p, dir, final_flush); }

    void finish_and_final_flush(snort::Flow* flow, bool clear, snort::Packet* p)
    { reassembler->finish_and_final_flush(trs, flow, clear, p); }

    void flush_queued_segments(snort::Flow* flow, bool clear, const snort::Packet* p = nullptr)
    { reassembler->flush_queued_segments(trs, flow, clear, p); }

    bool is_segment_pending_flush() const
    { return reassembler->is_segment_pending_flush(trs); }

    void skip_midstream_pickup_seglist_hole(TcpSegmentDescriptor& tsd)
    { reassembler->skip_midstream_pickup_seglist_hole(trs, tsd); }

    void initialize_paf()
    { reassembler->initialize_paf(trs); }

    int flush_on_data_policy(snort::Packet* p)
    { return reassembler->flush_on_data_policy(trs, p); }

    int flush_on_ack_policy(snort::Packet* p)
    { return reassembler->flush_on_ack_policy(trs, p); }

    void set_seglist_base_seq(uint32_t seglist_base_seq)
    { trs.sos.seglist_base_seq = seglist_base_seq; }

    uint32_t get_seglist_base_seq() const
    { return trs.sos.seglist_base_seq; }

    void set_xtradata_mask(uint32_t xtradata_mask)
    { trs.xtradata_mask = xtradata_mask; }

    uint32_t get_xtradata_mask() const
    { return trs.xtradata_mask; }

    bool data_was_queued() const
    { return trs.sos.total_bytes_queued > 0; }

    uint32_t get_seg_count() const
    { return trs.sos.seg_count; }

    uint32_t get_seg_bytes_total() const
    { return trs.sos.seg_bytes_total; }

    uint32_t get_overlap_count() const
    { return trs.sos.overlap_count; }

    void set_overlap_count(uint32_t overlap_count)
    { trs.sos.overlap_count = overlap_count;  }

    uint32_t get_flush_count() const
    { return trs.flush_count; }

    uint32_t get_seg_bytes_logical() const
    { return trs.sos.seg_bytes_logical; }

    StreamPolicy get_reassembly_policy() const
    { return trs.sos.reassembly_policy; }

    void set_norm_mode_test()
    { trs.sos.tcp_ips_data = NORM_MODE_TEST; }

    bool segment_within_seglist_window(TcpSegmentDescriptor& tsd)
    { return reassembler->segment_within_seglist_window(trs, tsd); }

    uint32_t perform_partial_flush(snort::Flow* flow, snort::Packet*& p)
    { return reassembler->perform_partial_flush(trs, flow, p); }

    void reset_paf()
    { paf_reset(&trs.paf_state); }

    void clear_paf()
    { paf_clear(&trs.paf_state); }

    void setup_paf()
    {
        paf_setup(&trs.paf_state);
        if ( trs.sos.seglist.cur_rseg )
            trs.sos.seglist.cur_sseg = trs.sos.seglist.cur_rseg;
        else
            trs.sos.seglist.cur_sseg = trs.sos.seglist.head;
    }

private:
    TcpReassembler* reassembler = nullptr;
    TcpReassemblerState trs = {};
    friend inline void TraceSegments(const TcpReassemblerPolicy&, const snort::Packet* p);
};
#endif

tcp_segment_descriptor.h

Path = src/stream/tcp/tcp_segment_descriptor.h

#ifndef TCP_SEGMENT_DESCRIPTOR_H
#define TCP_SEGMENT_DESCRIPTOR_H

#include <cassert>

#include <daq_common.h>

#include "flow/flow.h"
#include "detection/ips_context.h"
#include "packet_io/active.h"
#include "protocols/packet.h"
#include "protocols/tcp.h"
#include "stream/tcp/tcp_event_logger.h"

class TcpStreamTracker;

class TcpSegmentDescriptor
{
public:
    TcpSegmentDescriptor(snort::Flow*, snort::Packet*, TcpEventLogger&);
    TcpSegmentDescriptor(snort::Flow*, snort::Packet*, uint32_t meta_ack, uint16_t window);

    virtual ~TcpSegmentDescriptor() = default;

    static void setup();
    static void clear();

    bool is_ips_policy_inline()
    { return pkt->context->conf->ips_inline_mode(); }

    bool is_nap_policy_inline()
    { return pkt->context->conf->nap_inline_mode(); }

    uint32_t init_mss(uint16_t* value);
    uint32_t init_wscale(uint16_t* value);
    void set_retransmit_flag();

    snort::Flow* get_flow() const
    { return flow; }

    snort::Packet* get_pkt() const
    { return pkt; }

    const snort::tcp::TCPHdr* get_tcph() const
    { return tcph; }

    void set_seq(uint32_t seq_num)
    { seq = seq_num; }

    void update_seq(int32_t offset)
    { seq += offset; }

    uint32_t get_seq() const
    { return seq; }

    uint32_t get_ack() const
    { return ack; }

    void set_ack(uint32_t ack_num)
    { ack = ack_num; }

    void set_end_seq(uint32_t seq)
    { end_seq = seq; }

    uint32_t get_end_seq() const
    { return end_seq; }

    void set_timestamp(uint32_t timestamp)
    { timestamp_option = timestamp; }

    uint32_t get_timestamp() const
    { return timestamp_option; }

    void scale_wnd(uint16_t wscale)
    { wnd <<= wscale; }

    uint32_t get_wnd() const
    { return wnd; }

    uint16_t get_dst_port() const
    { return dst_port; }

    uint16_t get_src_port() const
    { return src_port; }

    uint8_t get_direction() const
    { return flow->ssn_state.direction; }

    uint16_t get_len() const
    { return pkt->dsize; }

    void set_len(uint16_t seg_len)
    { pkt->dsize = seg_len; }

    bool is_data_segment() const
    { return pkt->dsize > 0; }

    bool is_packet_from_client() const
    { return packet_from_client; }

    bool is_packet_from_server() const
    { return !packet_from_client; }

    void slide_segment_in_rcv_window(int32_t offset)
    {
        seq += offset;
        pkt->data += offset;
        pkt->dsize -= offset;
    }

    void set_packet_flags(uint32_t flags) const
    { pkt->packet_flags |= flags; }

    bool are_packet_flags_set(uint32_t flags) const
    { return (pkt->packet_flags & flags) == flags; }

    uint32_t get_packet_timestamp() const
    { return packet_timestamp; }

    void drop_packet() const
    {
        pkt->active->drop_packet(pkt);
        pkt->active->set_drop_reason("stream");
    }

    bool is_meta_ack_packet() const
    { return meta_ack_packet; }

    uint64_t get_packet_number() const
    { return packet_number; }

    void rewrite_payload(uint16_t offset, uint8_t* from, uint16_t length)
    {
        memcpy(const_cast<uint8_t*>(pkt->data + offset), from, length);
        set_packet_flags(PKT_MODIFIED);
    }

    void rewrite_payload(uint16_t offset, uint8_t* from)
    { rewrite_payload(offset, from, pkt->dsize); }

    TcpStreamTracker* get_listener() const
    { return listener; }

    void set_listener(TcpStreamTracker& tracker)
    { listener = &tracker; }

    TcpStreamTracker* get_talker() const
    { return talker; }

    void set_talker(TcpStreamTracker& tracker)
    { talker = &tracker; }

private:
    snort::Flow* const flow;
    snort::Packet* const pkt;
    const snort::tcp::TCPHdr* const tcph;
    TcpStreamTracker* talker = nullptr;
    TcpStreamTracker* listener = nullptr;

    const uint64_t packet_number;
    uint32_t seq;
    uint32_t ack;
    uint32_t wnd;
    uint32_t end_seq;
    uint32_t timestamp_option;
    uint16_t src_port;
    uint16_t dst_port;
    uint32_t packet_timestamp;
    bool packet_from_client;
    bool meta_ack_packet = false;
};

#endif

tcp_segment_node.h

Path = src/stream/tcp/tcp_segment_node.h

#ifndef TCP_SEGMENT_H
#define TCP_SEGMENT_H

#include "tcp_segment_descriptor.h"
#include "tcp_defs.h"

class TcpSegmentDescriptor;

//-----------------------------------------------------------------
// we make a lot of TcpSegments so it is organized by member
// size/alignment requirements to minimize unused space
// ... however, use of padding below is critical, adjust if needed
// and we use the struct hack to avoid 2 allocs per node
//-----------------------------------------------------------------

class TcpSegmentNode
{
private:
    static TcpSegmentNode* create(const struct timeval& tv, const uint8_t* segment, uint16_t len);

public:
    static TcpSegmentNode* init(const TcpSegmentDescriptor&);
    static TcpSegmentNode* init(TcpSegmentNode&);

    void term();

    static void setup();
    static void clear();

    bool is_retransmit(const uint8_t*, uint16_t size, uint32_t, uint16_t, bool*);

    uint8_t* payload()
    { return data + offset; }

    bool is_packet_missing(uint32_t to_seq)
    {
        if ( next )
            return (i_seq + i_len) != next->i_seq;
        else
            return (c_seq + c_len) < to_seq;
    }

    void update_ressembly_lengths(uint16_t bytes)
    {
        c_seq += bytes;
        c_len -= bytes;
        offset += bytes;
    }

public:
    TcpSegmentNode* prev;
    TcpSegmentNode* next;

    struct timeval tv;
    uint32_t ts;
    uint32_t i_seq;             // initial seq # of the data segment
    uint32_t c_seq;             // current seq # of data for reassembly
    uint16_t i_len;             // initial length of the data segment
    uint16_t c_len;             // length of data remaining for reassembly
    uint16_t offset;
    uint16_t size;              // actual allocated size (overlaps cause i_len to differ)
    uint8_t data[1];
};

class TcpSegmentList
{
public:
    uint32_t reset()
    {
        int i = 0;

        while ( head )
        {
            i++;
            TcpSegmentNode* dump_me = head;
            head = head->next;
            dump_me->term();
        }

        head = tail = cur_rseg = cur_sseg = nullptr;
        count = 0;
        return i;
    }

    void insert(TcpSegmentNode* prev, TcpSegmentNode* ss)
    {
        if ( prev )
        {
            ss->next = prev->next;
            ss->prev = prev;
            prev->next = ss;

            if ( ss->next )
                ss->next->prev = ss;
            else
                tail = ss;
        }
        else
        {
            ss->next = head;

            if ( ss->next )
                ss->next->prev = ss;
            else
                tail = ss;
            head = ss;
        }

        count++;
    }

    void remove(TcpSegmentNode* ss)
    {
        if ( ss->prev )
            ss->prev->next = ss->next;
        else
            head = ss->next;

        if ( ss->next )
            ss->next->prev = ss->prev;
        else
            tail = ss->prev;

        count--;
    }

    TcpSegmentNode* head = nullptr;
    TcpSegmentNode* tail = nullptr;
    TcpSegmentNode* cur_rseg = nullptr;
    TcpSegmentNode* cur_sseg = nullptr;
    uint32_t count = 0;
};

#endif

tcp_session.h

Path = src/stream/tcp/tcp_session.h

#ifndef TCP_SESSION_H
#define TCP_SESSION_H

#include "tcp_state_machine.h"
#include "tcp_stream_session.h"
#include "tcp_stream_tracker.h"

namespace snort
{
class Flow;
struct Packet;
}
class TcpEventLogger;


class TcpSession : public TcpStreamSession
{
public:
    TcpSession(snort::Flow*);
    ~TcpSession() override;

    static void sinit();
    static void sterm();

    bool setup(snort::Packet*) override;
    void restart(snort::Packet* p) override;
    void precheck(snort::Packet* p) override;
    int process(snort::Packet*) override;

    void flush() override;
    void flush_client(snort::Packet*) override;
    void flush_server(snort::Packet*) override;
    void flush_talker(snort::Packet*, bool final_flush = false) override;
    void flush_listener(snort::Packet*, bool final_flush = false) override;
    // cppcheck-suppress virtualCallInConstructor
    void clear_session(bool free_flow_data, bool flush_segments, bool restart, snort::Packet* p = nullptr) override;
    void set_extra_data(snort::Packet*, uint32_t /*flag*/) override;
    void update_perf_base_state(char new_state) override;
    TcpStreamTracker::TcpState get_talker_state(TcpSegmentDescriptor& tsd) override;
    TcpStreamTracker::TcpState get_listener_state(TcpSegmentDescriptor& tsd) override;
    void update_timestamp_tracking(TcpSegmentDescriptor&) override;
    void update_session_on_rst(TcpSegmentDescriptor&, bool) override;
    bool handle_syn_on_reset_session(TcpSegmentDescriptor&) override;
    void handle_data_on_syn(TcpSegmentDescriptor&) override;
    void update_ignored_session(TcpSegmentDescriptor&) override;
    void update_paws_timestamps(TcpSegmentDescriptor&) override;
    void check_for_repeated_syn(TcpSegmentDescriptor&) override;
    void check_for_session_hijack(TcpSegmentDescriptor&) override;
    bool check_for_window_slam(TcpSegmentDescriptor& tsd) override;
    void mark_packet_for_drop(TcpSegmentDescriptor&) override;
    void handle_data_segment(TcpSegmentDescriptor&, bool flush = true);
    bool validate_packet_established_session(TcpSegmentDescriptor&) override;

    bool is_midstream_allowed(const TcpSegmentDescriptor& tsd)
    { return tcp_config->midstream_allowed(tsd.get_pkt()); }

private:
    int process_tcp_packet(TcpSegmentDescriptor&, const snort::Packet*);
    void set_os_policy() override;
    bool flow_exceeds_config_thresholds(TcpSegmentDescriptor&);
    void update_stream_order(const TcpSegmentDescriptor&, bool aligned);
    void swap_trackers();
    void init_session_on_syn(TcpSegmentDescriptor&);
    void init_session_on_synack(TcpSegmentDescriptor&);
    void update_on_3whs_complete(TcpSegmentDescriptor&);
    bool ignore_this_packet(snort::Packet*);
    void cleanup_session_if_expired(snort::Packet*);
    void init_tcp_packet_analysis(TcpSegmentDescriptor&);
    void check_events_and_actions(const TcpSegmentDescriptor& tsd);
    void flush_tracker(TcpStreamTracker&, snort::Packet*, uint32_t dir, bool final_flush);

private:
    TcpStateMachine* tsm;
    bool splitter_init;
};

#endif

tcp_state_closed.h

Path = src/stream/tcp/tcp_state_closed.h

#ifndef TCP_STATE_CLOSED_H
#define TCP_STATE_CLOSED_H

#include "tcp_state_handler.h"

class TcpStateClosed : public TcpStateHandler
{
public:
    TcpStateClosed(TcpStateMachine&);

    bool syn_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool syn_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool ack_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool ack_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool fin_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool fin_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool rst_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;

    bool do_pre_sm_packet_actions(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool do_post_sm_packet_actions(TcpSegmentDescriptor&, TcpStreamTracker&) override;
};

#endif

tcp_state_close_wait.h

Path = src/stream/tcp/tcp_state_close_wait.h

#ifndef TCP_STATE_CLOSE_WAIT_H
#define TCP_STATE_CLOSE_WAIT_H

#include "tcp_state_handler.h"

class TcpStateCloseWait : public TcpStateHandler
{
public:
    TcpStateCloseWait(TcpStateMachine&);

    bool syn_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool ack_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool ack_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool fin_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool fin_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool rst_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;

    bool do_pre_sm_packet_actions(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool do_post_sm_packet_actions(TcpSegmentDescriptor&, TcpStreamTracker&) override;
};

#endif

tcp_state_closing.h

Path = src/stream/tcp/tcp_state_closing.h

#ifndef TCP_STATE_CLOSING_H
#define TCP_STATE_CLOSING_H

#include "tcp_state_handler.h"

class TcpStateClosing : public TcpStateHandler
{
public:
    TcpStateClosing(TcpStateMachine&);

    bool syn_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool syn_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool ack_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool ack_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool fin_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool fin_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool rst_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;

    bool do_pre_sm_packet_actions(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool do_post_sm_packet_actions(TcpSegmentDescriptor&, TcpStreamTracker&) override;
};

#endif

tcp_state_established.h

Path = src/stream/tcp/tcp_state_established.h

#ifndef TCP_STATE_ESTABLISHED_H
#define TCP_STATE_ESTABLISHED_H

#include "tcp_state_handler.h"

class TcpStateEstablished : public TcpStateHandler
{
public:
    TcpStateEstablished(TcpStateMachine&);

    bool syn_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool syn_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool syn_ack_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool ack_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool ack_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool fin_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool fin_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool rst_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;

    bool do_pre_sm_packet_actions(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool do_post_sm_packet_actions(TcpSegmentDescriptor&, TcpStreamTracker&) override;
};

#endif

tcp_state_fin_wait1.h

Path = src/stream/tcp/tcp_state_fin_wait1.h

#ifndef TCP_STATE_FIN_WAIT1_H
#define TCP_STATE_FIN_WAIT1_H

#include "tcp_state_handler.h"

class TcpStateFinWait1 : public TcpStateHandler
{
public:
    TcpStateFinWait1(TcpStateMachine&);

    bool syn_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool syn_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool syn_ack_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool ack_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool ack_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool fin_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool fin_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool rst_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;

    bool do_pre_sm_packet_actions(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool do_post_sm_packet_actions(TcpSegmentDescriptor&, TcpStreamTracker&) override;

private:
    bool check_for_window_slam(TcpSegmentDescriptor&, TcpStreamTracker&, bool* is_ack_valid = nullptr);
};

#endif

tcp_state_fin_wait2.h

Path = src/stream/tcp/tcp_state_fin_wait2.h

#ifndef TCP_STATE_FIN_WAIT2_H
#define TCP_STATE_FIN_WAIT2_H

#include "tcp_state_handler.h"

class TcpStateFinWait2 : public TcpStateHandler
{
public:
    TcpStateFinWait2(TcpStateMachine&);

    bool syn_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool syn_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool syn_ack_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool ack_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool ack_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool fin_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool rst_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;

    bool do_pre_sm_packet_actions(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool do_post_sm_packet_actions(TcpSegmentDescriptor&, TcpStreamTracker&) override;
};

#endif

tcp_state_handler.h

Path = src/stream/tcp/tcp_state_handler.h

#ifndef TCP_STATE_HANDLER_H
#define TCP_STATE_HANDLER_H

#include "protocols/tcp.h"

#include "tcp_stream_tracker.h"

class TcpSegmentDescriptor;
class TcpStateMachine;

class TcpStateHandler
{
public:
    TcpStateHandler(TcpStreamTracker::TcpState, TcpStateMachine&);
    virtual ~TcpStateHandler() = default;

    virtual bool eval(TcpSegmentDescriptor&, TcpStreamTracker&);

    virtual bool do_pre_sm_packet_actions(TcpSegmentDescriptor&, TcpStreamTracker&);
    virtual bool do_post_sm_packet_actions(TcpSegmentDescriptor&, TcpStreamTracker&);

protected:
    virtual bool syn_sent(TcpSegmentDescriptor&, TcpStreamTracker&) { return true; }
    virtual bool syn_recv(TcpSegmentDescriptor&, TcpStreamTracker&) { return true; }
    virtual bool syn_ack_sent(TcpSegmentDescriptor&, TcpStreamTracker&) { return true; }
    virtual bool syn_ack_recv(TcpSegmentDescriptor&, TcpStreamTracker&) { return true; }
    virtual bool ack_sent(TcpSegmentDescriptor&, TcpStreamTracker&) { return true; }
    virtual bool ack_recv(TcpSegmentDescriptor&, TcpStreamTracker&) { return true; }
    virtual bool data_seg_sent(TcpSegmentDescriptor&, TcpStreamTracker&) { return true; }
    virtual bool data_seg_recv(TcpSegmentDescriptor&, TcpStreamTracker&) { return true; }
    virtual bool fin_sent(TcpSegmentDescriptor&, TcpStreamTracker&) { return true; }
    virtual bool fin_recv(TcpSegmentDescriptor&, TcpStreamTracker&) { return true; }
    virtual bool rst_sent(TcpSegmentDescriptor&, TcpStreamTracker&) { return true; }
    virtual bool rst_recv(TcpSegmentDescriptor&, TcpStreamTracker&) { return true; }
    virtual bool no_flags(TcpSegmentDescriptor& tsd, TcpStreamTracker& trk)
    {
        trk.normalizer.packet_dropper(tsd, NORM_TCP_BLOCK);
        return false;
    }
};

#endif

tcp_state_last_ack.h

Path = src/stream/tcp/tcp_state_last_ack.h

#ifndef TCP_STATE_LAST_ACK_H
#define TCP_STATE_LAST_ACK_H

#include "tcp_state_handler.h"

class TcpStateLastAck : public TcpStateHandler
{
public:
    TcpStateLastAck(TcpStateMachine&);

    bool syn_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool syn_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool ack_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool ack_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool fin_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool fin_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool rst_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;

    bool do_pre_sm_packet_actions(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool do_post_sm_packet_actions(TcpSegmentDescriptor&, TcpStreamTracker&) override;
};

#endif

tcp_state_listen.h

Path = src/stream/tcp/tcp_state_listen.h

#ifndef TCP_STATE_LISTEN_H
#define TCP_STATE_LISTEN_H

#include "tcp_state_handler.h"

class TcpStateListen : public TcpStateHandler
{
public:
    TcpStateListen(TcpStateMachine&);

    bool syn_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool syn_ack_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool ack_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool fin_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool fin_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool rst_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;

    bool do_post_sm_packet_actions(TcpSegmentDescriptor&, TcpStreamTracker&) override;
};

#endif

tcp_state_machine.h

Path = src/stream/tcp/tcp_state_machine.h

#ifndef TCP_STATE_MACHINE_H
#define TCP_STATE_MACHINE_H

#include "tcp_segment_descriptor.h"
#include "tcp_state_handler.h"
#include "tcp_stream_tracker.h"

class TcpStateMachine
{
public:
    virtual ~TcpStateMachine();

    static TcpStateMachine* initialize();
    static void term();

    static TcpStateMachine* get_instance()
    { return TcpStateMachine::tsm; }

    virtual void register_state_handler(TcpStreamTracker::TcpState, TcpStateHandler&);
    virtual bool eval(TcpSegmentDescriptor&);

protected:
    TcpStateMachine();
    static TcpStateMachine* tsm;

    TcpStateHandler* tcp_state_handlers[ TcpStreamTracker::TCP_MAX_STATES ];
};

#endif

tcp_state_mid_stream_recv.h

Path = src/stream/tcp/tcp_state_mid_stream_recv.h

#ifndef TCP_STATE_MID_STREAM_RECV_H
#define TCP_STATE_MID_STREAM_RECV_H

#include "tcp_state_handler.h"

class TcpStateMidStreamRecv : public TcpStateHandler
{
public:
    TcpStateMidStreamRecv(TcpStateMachine&);

    bool syn_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool syn_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool syn_ack_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool ack_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool ack_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool fin_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool fin_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool rst_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;

    bool do_post_sm_packet_actions(TcpSegmentDescriptor&, TcpStreamTracker&) override;
};

#endif

tcp_state_mid_stream_sent.h

Path = src/stream/tcp/tcp_state_mid_stream_sent.h

#ifndef TCP_STATE_MID_STREAM_SENT_H
#define TCP_STATE_MID_STREAM_SENT_H

#include "tcp_state_handler.h"

class TcpStateMidStreamSent : public TcpStateHandler
{
public:
    TcpStateMidStreamSent(TcpStateMachine&);

    bool syn_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool syn_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool syn_ack_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool syn_ack_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool ack_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool ack_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool fin_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool fin_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool rst_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;

    bool do_post_sm_packet_actions(TcpSegmentDescriptor&, TcpStreamTracker&) override;

private:
    bool check_for_window_slam(TcpSegmentDescriptor&, TcpStreamTracker&, bool& is_ack_valid);
};

#endif

tcp_state_none.h

Path = src/stream/tcp/tcp_state_none.h

#ifndef TCP_STATE_NONE_H
#define TCP_STATE_NONE_H

#include "tcp_state_handler.h"

class TcpStateNone : public TcpStateHandler
{
public:
    TcpStateNone(TcpStateMachine&);

    bool syn_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool syn_ack_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool syn_ack_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool ack_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool fin_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool fin_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool rst_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool rst_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;

    bool do_post_sm_packet_actions(TcpSegmentDescriptor&, TcpStreamTracker&) override;
};

#endif

tcp_state_syn_recv.h

Path = src/stream/tcp/tcp_state_syn_recv.h

#ifndef TCP_STATE_SYN_RECV_H
#define TCP_STATE_SYN_RECV_H

#include "tcp_state_handler.h"

class TcpStateSynRecv : public TcpStateHandler
{
public:
    TcpStateSynRecv(TcpStateMachine&);

    bool syn_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool syn_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool syn_ack_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool syn_ack_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool ack_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool ack_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool fin_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool rst_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;

    bool do_post_sm_packet_actions(TcpSegmentDescriptor&, TcpStreamTracker&) override;
};

#endif

tcp_state_syn_sent.h

Path = src/stream/tcp/tcp_state_syn_sent.h

#ifndef TCP_STATE_SYN_SENT_H
#define TCP_STATE_SYN_SENT_H

#include "tcp_state_handler.h"

class TcpStateSynSent : public TcpStateHandler
{
public:
    TcpStateSynSent(TcpStateMachine&);

    bool syn_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool syn_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool syn_ack_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool syn_ack_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool ack_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool ack_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool fin_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool fin_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool rst_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;

    bool do_post_sm_packet_actions(TcpSegmentDescriptor&, TcpStreamTracker&) override;
};

#endif

tcp_state_time_wait.h

Path = src/stream/tcp/tcp_state_time_wait.h

#ifndef TCP_STATE_TIME_WAIT_H
#define TCP_STATE_TIME_WAIT_H

#include "tcp_state_handler.h"

class TcpStateTimeWait : public TcpStateHandler
{
public:
    TcpStateTimeWait(TcpStateMachine&);

    bool syn_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool syn_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool ack_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool data_seg_sent(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool fin_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool rst_recv(TcpSegmentDescriptor&, TcpStreamTracker&) override;

    bool do_pre_sm_packet_actions(TcpSegmentDescriptor&, TcpStreamTracker&) override;
    bool do_post_sm_packet_actions(TcpSegmentDescriptor&, TcpStreamTracker&) override;
};

#endif

tcp_stream_config.h

Path = src/stream/tcp/tcp_stream_config.h

#ifndef TCP_STREAM_CONFIG_H
#define TCP_STREAM_CONFIG_H

#include "protocols/packet.h"
#include "stream/tcp/tcp_defs.h"
#include "time/packet_time.h"

#define STREAM_CONFIG_SHOW_PACKETS             0x00000001
#define STREAM_CONFIG_NO_ASYNC_REASSEMBLY      0x00000002
#define STREAM_CONFIG_NO_REASSEMBLY            0x00000004

#define STREAM_DEFAULT_SSN_TIMEOUT  30

class TcpStreamConfig
{
public:
    TcpStreamConfig();

    bool require_3whs()
    {
        return hs_timeout >= 0;
    }

    bool midstream_allowed(snort::Packet* p)
    {
        if ( ( hs_timeout < 0 ) || ( p->pkth->ts.tv_sec - packet_first_time() < hs_timeout ) )
            return true;

        return false;
    }

    void show() const;

    StreamPolicy policy = StreamPolicy::OS_DEFAULT;

    uint16_t flags = 0;
    uint16_t flush_factor = 0;

    uint32_t session_timeout = STREAM_DEFAULT_SSN_TIMEOUT;
    uint32_t max_window = 0;
    uint32_t overlap_limit = 0;

    uint32_t max_queued_bytes = 4194304;
    uint32_t max_queued_segs = 3072;

    uint32_t max_consec_small_segs = STREAM_DEFAULT_CONSEC_SMALL_SEGS;
    uint32_t max_consec_small_seg_size = STREAM_DEFAULT_MAX_SMALL_SEG_SIZE;

    uint32_t paf_max = 16384;
    int hs_timeout = -1;

    bool no_ack = false;
    uint32_t embryonic_timeout = STREAM_DEFAULT_SSN_TIMEOUT;
    uint32_t idle_timeout = 3600;
};

#endif

tcp_stream_session.h

Path = src/stream/tcp/tcp_stream_session.h

#ifndef TCP_STREAM_SESSION_H
#define TCP_STREAM_SESSION_H

#include "detection/detection_engine.h"
#include "flow/session.h"
#include "protocols/ipv6.h"

#include "tcp_stream_config.h"
#include "tcp_stream_tracker.h"

// FIXIT-L session tracking could be split from reassembly
// into a separate module a la ip_session.cc and ip_defrag.cc
// (of course defrag should also be cleaned up)
class TcpStreamSession : public Session
{
public:
    ~TcpStreamSession() override;

    void clear() override;
    void cleanup(snort::Packet* = nullptr) override;

    void set_splitter(bool, snort::StreamSplitter*) override;
    snort::StreamSplitter* get_splitter(bool) override;

    bool is_sequenced(uint8_t dir) const override;
    bool are_packets_missing(uint8_t dir) const override;

    void disable_reassembly(snort::Flow*) override;
    uint8_t get_reassembly_direction() const override;
    uint8_t missing_in_reassembled(uint8_t dir) const override;
    bool are_client_segments_queued() const override;

    bool add_alert(snort::Packet*, uint32_t gid, uint32_t sid) override;
    bool check_alerted(snort::Packet*, uint32_t gid, uint32_t sid) override;
    int update_alert(snort::Packet*, uint32_t gid, uint32_t sid,
        uint32_t event_id, uint32_t event_second) override;

    bool set_packet_action_to_hold(snort::Packet*) override;

    uint16_t get_mss(bool to_server) const;
    uint8_t get_tcp_options_len(bool to_server) const;

    void reset();
    void start_proxy();

    void set_packet_header_foo(const TcpSegmentDescriptor&);
    void get_packet_header_foo(DAQ_PktHdr_t*, const DAQ_PktHdr_t* orig, uint32_t dir);
    bool can_set_no_ack();
    bool set_no_ack(bool);
    bool no_ack_mode_enabled() { return no_ack; }
    virtual void update_perf_base_state(char) = 0;
    virtual void clear_session(
        bool free_flow_data, bool flush_segments, bool restart, snort::Packet* p = nullptr) = 0;
    virtual TcpStreamTracker::TcpState get_talker_state(TcpSegmentDescriptor&) = 0;
    virtual TcpStreamTracker::TcpState get_listener_state(TcpSegmentDescriptor&) = 0;
    TcpStreamTracker::TcpState get_peer_state(const TcpStreamTracker* me)
    { return me == &client ? server.get_tcp_state() : client.get_tcp_state(); }

    virtual void init_new_tcp_session(TcpSegmentDescriptor&);
    virtual void update_timestamp_tracking(TcpSegmentDescriptor&) = 0;
    virtual void update_session_on_server_packet(TcpSegmentDescriptor&);
    virtual void update_session_on_client_packet(TcpSegmentDescriptor&);
    virtual void update_session_on_rst(TcpSegmentDescriptor&, bool) = 0;
    virtual bool handle_syn_on_reset_session(TcpSegmentDescriptor&) = 0;
    virtual void handle_data_on_syn(TcpSegmentDescriptor&) = 0;
    virtual void update_ignored_session(TcpSegmentDescriptor&) = 0;
    void generate_no_3whs_event()
    {
        if ( generate_3whs_alert && flow->two_way_traffic())
        {
            tel.set_tcp_event(EVENT_NO_3WHS);
            generate_3whs_alert = false;
        }
    }

    void set_pkt_action_flag(uint32_t flag)
    { pkt_action_mask |= flag; }

    void set_established(const TcpSegmentDescriptor&);
    void set_pseudo_established(snort::Packet*);
    bool check_for_one_sided_session(snort::Packet*);
    void check_for_pseudo_established(snort::Packet*);

    virtual void update_paws_timestamps(TcpSegmentDescriptor&) = 0;
    virtual void check_for_repeated_syn(TcpSegmentDescriptor&) = 0;
    virtual void check_for_session_hijack(TcpSegmentDescriptor&) = 0;
    virtual bool check_for_window_slam(TcpSegmentDescriptor&) = 0;
    virtual void mark_packet_for_drop(TcpSegmentDescriptor&) = 0;
    virtual bool validate_packet_established_session(TcpSegmentDescriptor&) = 0;

    TcpStreamTracker client;
    TcpStreamTracker server;
    bool lws_init = false;
    bool tcp_init = false;
    uint32_t pkt_action_mask = ACTION_NOTHING;
    uint32_t initiator_watermark = 0;
    int32_t ingress_index = DAQ_PKTHDR_UNKNOWN;
    int32_t egress_index = DAQ_PKTHDR_UNKNOWN;
    int16_t ingress_group = DAQ_PKTHDR_UNKNOWN;
    int16_t egress_group = DAQ_PKTHDR_UNKNOWN;
    uint32_t daq_flags = 0;
    uint32_t address_space_id = 0;
    bool generate_3whs_alert = true;
    TcpStreamConfig* tcp_config = nullptr;
    TcpEventLogger tel;
    bool cleaning = false;
    uint8_t held_packet_dir = SSN_DIR_NONE;
    uint8_t ecn = 0;

private:
    bool no_ack = false;

protected:
    TcpStreamSession(snort::Flow*);
    virtual void set_os_policy() = 0;
};

#endif

tcp_stream_tracker.h

Path = src/stream/tcp/tcp_stream_tracker.h

#ifndef TCP_STREAM_TRACKER_H
#define TCP_STREAM_TRACKER_H

#include <list>

#include "stream/paf.h"

#include "segment_overlap_editor.h"
#include "tcp_defs.h"
#include "tcp_module.h"
#include "tcp_normalizers.h"
#include "tcp_reassemblers.h"
#include "tcp_segment_descriptor.h"

extern const char* tcp_state_names[];
extern const char* tcp_event_names[];

namespace snort
{
struct Packet;
}

class HeldPacket;
class TcpReassembler;
class TcpSession;

class TcpStreamTracker
{
public:
    enum TcpState : uint8_t
    {
        TCP_LISTEN,
        TCP_SYN_SENT,
        TCP_SYN_RECV,
        TCP_ESTABLISHED,
        TCP_MID_STREAM_SENT,
        TCP_MID_STREAM_RECV,
        TCP_FIN_WAIT1,
        TCP_FIN_WAIT2,
        TCP_CLOSE_WAIT,
        TCP_CLOSING,
        TCP_LAST_ACK,
        TCP_TIME_WAIT,
        TCP_CLOSED,
        TCP_STATE_NONE,
        TCP_MAX_STATES
    };

    enum TcpEvent : uint8_t
    {
        TCP_SYN_SENT_EVENT,
        TCP_SYN_RECV_EVENT,
        TCP_SYN_ACK_SENT_EVENT,
        TCP_SYN_ACK_RECV_EVENT,
        TCP_ACK_SENT_EVENT,
        TCP_ACK_RECV_EVENT,
        TCP_DATA_SEG_SENT_EVENT,
        TCP_DATA_SEG_RECV_EVENT,
        TCP_FIN_SENT_EVENT,
        TCP_FIN_RECV_EVENT,
        TCP_RST_SENT_EVENT,
        TCP_RST_RECV_EVENT,
        TCP_NO_FLAGS_EVENT,
        TCP_MAX_EVENTS
    };

    enum PacketOrder : uint8_t { OUT_OF_SEQUENCE, IN_SEQUENCE, NONE };

    enum FinSeqNumStatus : uint8_t { FIN_NOT_SEEN, FIN_WITH_SEQ_SEEN, FIN_WITH_SEQ_ACKED };

    TcpStreamTracker(bool client);
    virtual ~TcpStreamTracker();

    bool is_client_tracker() const
    { return client_tracker; }

    bool is_server_tracker() const
    { return !client_tracker; }

    TcpState get_tcp_state() const
    { return tcp_state; }

    void set_tcp_state(TcpState tcp_state)
    { this->tcp_state = tcp_state; }

    TcpEvent get_tcp_event() const
    { return tcp_event; }

    TcpEvent set_tcp_event(const TcpSegmentDescriptor&);

    void set_tcp_event(TcpEvent tcp_event)
    { this->tcp_event = tcp_event; }

    uint32_t get_rcv_nxt() const
    { return rcv_nxt; }

    void set_rcv_nxt(uint32_t rcv_nxt)
    { this->rcv_nxt = rcv_nxt; }

    uint32_t get_rcv_wnd() const
    { return rcv_wnd; }

    void set_rcv_wnd(uint32_t rcv_wnd)
    { this->rcv_wnd = rcv_wnd; }

    uint16_t get_rcv_up() const
    { return rcv_up; }

    void set_rcv_up(uint16_t rcv_up)
    { this->rcv_up = rcv_up; }

    uint32_t get_irs() const
    { return irs; }

    void set_irs(uint32_t irs)
    { this->irs = irs; }

    uint32_t get_snd_una() const
    { return snd_una; }

    void set_snd_una(uint32_t snd_una)
    { this->snd_una = snd_una; }

    uint32_t get_snd_nxt() const
    { return snd_nxt; }

    void set_snd_nxt(uint32_t snd_nxt)
    { this->snd_nxt = snd_nxt; }

    uint16_t get_snd_up() const
    { return snd_up; }

    void set_snd_up(uint16_t snd_up)
    { this->snd_up = snd_up; }

    uint32_t get_snd_wl1() const
    { return snd_wl1; }

    void set_snd_wl1(uint32_t snd_wl1)
    { this->snd_wl1 = snd_wl1; }

    uint32_t get_snd_wl2() const
    { return snd_wl2; }

    void set_snd_wl2(uint32_t snd_wl2)
    { this->snd_wl2 = snd_wl2; }

    uint32_t get_snd_wnd() const
    { return snd_wnd; }

    void set_snd_wnd(uint32_t snd_wnd)
    { this->snd_wnd = snd_wnd; }

    uint32_t get_iss() const
    { return iss; }

    void set_iss(uint32_t iss)
    { this->iss = iss; }

    uint32_t get_fin_final_seq() const
    { return fin_final_seq + fin_seq_adjust; }

    uint32_t get_fin_seq_adjust()
    { return fin_seq_adjust; }

    uint32_t get_fin_i_seq() const
    { return fin_i_seq; }

    // set final seq # any packet rx'ed with seq > is bad
    void set_fin_seq_status_seen(const TcpSegmentDescriptor&);

    bool is_fin_seq_set() const
    { return fin_seq_set; }

    uint32_t get_ts_last_packet() const
    { return ts_last_packet; }

    void set_ts_last_packet(uint32_t ts_last_packet)
    { this->ts_last_packet = ts_last_packet; }

    bool is_ack_valid(uint32_t cur)
    {
        if ( ( snd_una == 0 ) && ( snd_nxt == 0 ) )
            return true;

        bool valid = SEQ_LEQ(cur, snd_nxt);
        if ( !valid )
            tcpStats.invalid_ack++;

        return valid;
    }

    // ack number must ack syn
    bool is_rst_valid_in_syn_sent(const TcpSegmentDescriptor& tsd) const
    { return tsd.get_ack() == snd_una; }

    uint32_t get_ts_last() const
    { return ts_last; }

    void set_ts_last(uint32_t ts_last)
    { this->ts_last = ts_last; }

    uint16_t get_tf_flags() const
    { return tf_flags; }

    void set_tf_flags(uint16_t flags)
    { this->tf_flags |= flags; }

    void clear_tf_flags(uint16_t flags)
    { this->tf_flags &= ~flags; }

    uint16_t get_wscale() const
    { return wscale; }

    void set_wscale(uint16_t wscale)
    { this->wscale = wscale; }

    uint16_t get_mss() const
    { return mss; }

    void set_mss(uint16_t mss)
    { this->mss = mss; }

    uint8_t get_tcp_options_len() const
    { return tcp_options_len; }

    void set_tcp_options_len(uint8_t tcp_options_len)
    { this->tcp_options_len = tcp_options_len; }

    void cache_mac_address(const TcpSegmentDescriptor&, uint8_t direction);
    bool compare_mac_addresses(const uint8_t eth_addr[]);

    bool is_rst_pkt_sent() const
    { return rst_pkt_sent; }

    void set_flush_policy(FlushPolicy policy)
    { flush_policy = policy; }

    FlushPolicy get_flush_policy() const
    { return flush_policy; }

    virtual void init_tcp_state();
    virtual void init_flush_policy();
    virtual void set_splitter(snort::StreamSplitter* ss);
    virtual void set_splitter(const snort::Flow* flow);

    snort::StreamSplitter* get_splitter()
    { return splitter; }

    bool is_splitter_paf() const
    { return splitter && splitter->is_paf(); }

    bool splitter_finish(snort::Flow* flow);

    bool is_reassembly_enabled() const
    { return  ( splitter and (flush_policy != STREAM_FLPOLICY_IGNORE) ); }

    virtual void init_on_syn_sent(TcpSegmentDescriptor&);
    virtual void init_on_syn_recv(TcpSegmentDescriptor&);
    virtual void init_on_synack_sent(TcpSegmentDescriptor&);
    virtual void init_on_synack_recv(TcpSegmentDescriptor&);
    virtual void init_on_data_seg_sent(TcpSegmentDescriptor&);
    virtual void init_on_data_seg_recv(TcpSegmentDescriptor&);
    virtual void finish_server_init(TcpSegmentDescriptor&);
    virtual void finish_client_init(TcpSegmentDescriptor&);

    virtual void update_tracker_ack_recv(TcpSegmentDescriptor&);
    virtual void update_tracker_ack_sent(TcpSegmentDescriptor&);
    virtual void update_tracker_no_ack_recv(TcpSegmentDescriptor&);
    virtual void update_tracker_no_ack_sent(TcpSegmentDescriptor&);
    virtual bool update_on_3whs_ack(TcpSegmentDescriptor&);
    virtual bool update_on_rst_recv(TcpSegmentDescriptor&);
    virtual void update_on_rst_sent();
    virtual bool update_on_fin_recv(TcpSegmentDescriptor&);
    virtual bool update_on_fin_sent(TcpSegmentDescriptor&);
    virtual bool is_segment_seq_valid(TcpSegmentDescriptor&);
    bool set_held_packet(snort::Packet*);
    bool is_retransmit_of_held_packet(snort::Packet*);
    void finalize_held_packet(snort::Packet*);
    void finalize_held_packet(snort::Flow*);
    void perform_fin_recv_flush(TcpSegmentDescriptor&);
    uint32_t perform_partial_flush();
    bool is_holding_packet() const { return held_packet != null_iterator; }

    // max_remove < 0 means time out all eligible packets.
    // Return whether there are more packets that need to be released.
    static bool release_held_packets(const timeval& cur_time, int max_remove);
    static void set_held_packet_timeout(const uint32_t ms);
    static bool adjust_expiration(uint32_t new_timeout_ms, const timeval& now);
    static void thread_init();
    static void thread_term();

public:
    uint32_t snd_una = 0; // SND.UNA - send unacknowledged
    uint32_t snd_nxt = 0; // SND.NXT - send next
    uint32_t snd_wnd = 0; // SND.WND - send window
    uint32_t snd_wl1 = 0; // SND.WL1 - segment sequence number used for last window update
    uint32_t snd_wl2 = 0; // SND.WL2 - segment acknowledgment number used for last window update
    uint32_t iss = 0;     // ISS     - initial send sequence number

    uint32_t rcv_nxt = 0; // RCV.NXT - receive next
    uint32_t rcv_wnd = 0; // RCV.WND - receive window
    uint32_t irs = 0;     // IRS     - initial receive sequence number

    uint16_t snd_up = 0;  // SND.UP  - send urgent pointer
    uint16_t rcv_up = 0;  // RCV.UP  - receive urgent pointer

    uint32_t held_pkt_seq = 0;
    uint32_t hole_left_edge = 0;   // First left hole
    uint32_t hole_right_edge = 0;

    TcpState tcp_state;
    TcpEvent tcp_event = TCP_MAX_EVENTS;

    bool client_tracker;
    bool require_3whs = false;
    bool rst_pkt_sent = false;
    bool midstream_initial_ack_flush = false;

// FIXIT-L make these non-public
public:
    TcpNormalizerPolicy normalizer;
    TcpReassemblerPolicy reassembler;
    TcpSession* session = nullptr;

    uint32_t r_win_base = 0; // remote side window base sequence number (the last ack we got)
    uint32_t small_seg_count = 0;
    uint32_t max_queue_seq_nxt = 0; // next expected sequence once queue limit is exceeded
    uint8_t max_queue_exceeded = MQ_NONE;
    uint8_t order = IN_SEQUENCE;
    FinSeqNumStatus fin_seq_status = TcpStreamTracker::FIN_NOT_SEEN;

protected:
    static const std::list<HeldPacket>::iterator null_iterator;
    std::list<HeldPacket>::iterator held_packet;
    snort::StreamSplitter* splitter = nullptr;
    uint32_t ts_last_packet = 0;
    uint32_t ts_last = 0;       // last timestamp (for PAWS)
    uint32_t fin_final_seq = 0;
    uint32_t fin_i_seq = 0;
    uint32_t fin_seq_adjust = 0;
    uint16_t mss = 0;           // max segment size
    uint16_t wscale = 0;        // window scale setting
    uint16_t tf_flags = 0;
    uint8_t mac_addr[6] = { };
    uint8_t tcp_options_len = 0;
    FlushPolicy flush_policy = STREAM_FLPOLICY_IGNORE;
    bool mac_addr_valid = false;
    bool fin_seq_set = false;  // FIXIT-M should be obviated by tcp state
    bool splitter_finish_flag = false;
};

// <--- note -- the 'state' parameter must be a reference
inline TcpStreamTracker::TcpState& operator++(TcpStreamTracker::TcpState& state, int)
{
    if ( state < TcpStreamTracker::TCP_MAX_STATES )
        state = static_cast<TcpStreamTracker::TcpState>( static_cast<int>(state) + 1 );
    else
        state = TcpStreamTracker::TCP_MAX_STATES;

    return state;
}

#endif

tcp_trace.h

Path = src/stream/tcp/tcp_trace.h

#ifndef TCP_TRACE_H
#define TCP_TRACE_H

#include "main/thread.h"

namespace snort
{
struct Packet;
}

class TcpSegmentDescriptor;

enum
{
    TRACE_SEGMENTS = 0,
    TRACE_STATE,
};

void S5TraceTCP(const TcpSegmentDescriptor&, const snort::Packet*);

#endif

stream/tcp/test/

stream_tcp_test_utils.h

Path = src/stream/tcp/test/stream_tcp_test_utils.h

#ifndef STREAM_TCP_TEST_UTILS_H
#define STREAM_TCP_TEST_UTILS_H

namespace snort
{
class Flow;
struct Packet;
}

snort::Packet* get_syn_packet(snort::Flow*);
snort::Packet* get_syn_ack_packet(snort::Flow*);

void release_packet(snort::Packet*);

#endif

stream/test/

stream/udp/

This directory contains the implementation of UDP session tracking and processing functions. When the network protocol for a flow is determined to be UDP the base Stream preprocessor will delegate handling of the packets on that flow to this module.

The StreamUdp class is implemented as a subclass of Inspector and provides functions for loading stream UDP configuration and packet evaluation. The packet eval method is not used as the base Stream Inspector delegates packets directly to the UDP session packet processing method.

The module udp_ha.cc (and udp_ha.h) implements the per-protocol hooks into the stream logic for HA. UdpHAManager is a static class that interfaces to a per-packet thread instance of the class UdpHA. UdpHA is sub-class of ProtocolHA, declared in the stream/base area. Thus each protocol within stream can have specific HA logic and interfaces.

UdpHAManager::process_deletion() is called when an UDP stream is being destroyed and indicates to the stream & flow HA logic that a flow deletion HA message needs to be emitted for the flow in question. Udp streams are closed due to timeout or pruning actions which lead to the invocation of process_deletion().

UdpHA::create_session() is called from the stream & flow HA logic and handles the creation of new flow upon receiving an HA update message.

stream_udp.h

Path = src/stream/udp/stream_udp.h

#ifndef STREAM_UDP_H
#define STREAM_UDP_H

#include <cstdint>

namespace snort
{
class Inspector;
}

struct StreamUdpConfig
{
    uint32_t session_timeout;

    StreamUdpConfig();
};

StreamUdpConfig* get_udp_cfg(snort::Inspector*);

#endif

udp_ha.h

Path = src/stream/udp/udp_ha.h

#ifndef UDP_HA_H
#define UDP_HA_H

#include "stream/base/stream_ha.h"

//-------------------------------------------------------------------------

namespace snort
{
class Flow;
}

class UdpHA : public ProtocolHA
{
public:
    UdpHA() : ProtocolHA(PktType::UDP) { }
    snort::Flow* create_session(const snort::FlowKey*) override;

private:
};

class UdpHAManager
{
public:
    static void process_deletion(snort::Flow& flow);
    static void tinit();
    static void tterm();
    static THREAD_LOCAL UdpHA* udp_ha;
};
#endif

udp_module.h

Path = src/stream/udp/udp_module.h

#ifndef UDP_MODULE_H
#define UDP_MODULE_H

#include "flow/session.h"
#include "framework/module.h"

#include "stream_udp.h"

namespace snort
{
struct SnortConfig;
}

struct UdpStats
{
    SESSION_STATS;
    PegCount total_bytes;
    PegCount ignored;
};

extern const PegInfo udp_pegs[];
extern THREAD_LOCAL struct UdpStats udpStats;
extern THREAD_LOCAL snort::ProfileStats udp_perf_stats;

//-------------------------------------------------------------------------
// stream_udp module
//-------------------------------------------------------------------------

#define MOD_NAME "stream_udp"
#define MOD_HELP "stream inspector for UDP flow tracking"

class StreamUdpModule : public snort::Module
{
public:
    StreamUdpModule();
    ~StreamUdpModule() override
    { delete config; }
    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    snort::ProfileStats* get_profile() const override;
    const PegInfo* get_pegs() const override;
    PegCount* get_counts() const override;
    StreamUdpConfig* get_data();

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

private:
    StreamUdpConfig* config = nullptr;
};

#endif

udp_session.h

Path = src/stream/udp/udp_session.h

#ifndef UDP_SESSION_H
#define UDP_SESSION_H

#include <sys/time.h>

#include "flow/session.h"
#include "main/snort_types.h"

class SO_PUBLIC UdpSession : public Session
{
public:
    UdpSession(snort::Flow*);
    ~UdpSession() override;

    bool setup(snort::Packet*) override;
    void update_direction(char dir, const snort::SfIp*, uint16_t port) override;
    int process(snort::Packet*) override;
    void clear() override;

public:
    struct timeval ssn_time = {};
};

void udp_stats();
void udp_reset();

#endif

stream/user/

This directory contains the implementation of user session tracking and processing functions. When the source for a flow provides a TCP payload, e.g. a socket connection, then the base Stream inspector delegates handling of the packets on that flow to this module.

The StreamUser class is implemented as a subclass of Inspector and provides functions for loading stream user configuration and packet evaluation. The packet eval method is not used as the base Stream Inspector delegates packets directly to the user session packet processing method.

The User session module implements the following functions:

  • Tracks the client and server side of the connection

  • Manages list of TCP payload segments for reassembly into PDU

  • Initializes splitter for selected type of PAF

  • Manages flushing the reassembled PDU

A UserSession object is allocated for each flow being tracked. User session state is maintained primarily in the UserTracker data structure allocated for the client and server side of the connection. State information includes:

  • list of segments being reassembled

  • PAF state

stream_user.h

Path = src/stream/user/stream_user.h

#ifndef STREAM_USER_H
#define STREAM_USER_H

#include <cstdint>

namespace snort
{
class Inspector;
}

struct StreamUserConfig
{
    uint32_t session_timeout;

    StreamUserConfig();
};

StreamUserConfig* get_user_cfg(snort::Inspector*);

#endif

user_module.h

Path = src/stream/user/user_module.h

#ifndef USER_MODULE_H
#define USER_MODULE_H

#include "framework/module.h"

namespace snort
{
class Trace;
struct SnortConfig;
}

extern THREAD_LOCAL const snort::Trace* stream_user_trace;
extern THREAD_LOCAL snort::ProfileStats user_perf_stats;

//-------------------------------------------------------------------------
// stream_user module
//-------------------------------------------------------------------------

#define MOD_NAME "stream_user"
#define MOD_HELP "stream inspector for user flow tracking and reassembly "

struct StreamUserConfig;

class StreamUserModule : public snort::Module
{
public:
    StreamUserModule();
    ~StreamUserModule() override;

    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool begin(const char*, int, snort::SnortConfig*) override;

    Usage get_usage() const override
    { return INSPECT; }

    bool is_bindable() const override
    { return true; }

    StreamUserConfig* get_data();

    void set_trace(const snort::Trace*) const override;
    const snort::TraceOption* get_trace_options() const override;

private:
    StreamUserConfig* config;
};

#endif

user_session.h

Path = src/stream/user/user_session.h

#ifndef USER_SESSION_H
#define USER_SESSION_H

#include <list>

#include "flow/session.h"
#include "stream/paf.h"

// cppcheck-suppress noConstructor
struct UserSegment
{
    static UserSegment* init(const uint8_t*, unsigned);
    static void term(UserSegment*);

    unsigned avail();
    void copy(const uint8_t*, unsigned);
    void shift(unsigned);

    unsigned get_len();
    uint8_t* get_data();

    bool unused();
    void use(unsigned);
    void reset();

    unsigned get_unused_len();
    uint8_t* get_unused_data();

private:
    unsigned len;
    unsigned offset;
    unsigned used;
    unsigned size;
    uint8_t data[1];
};

struct UserTracker
{
    UserTracker();
    ~UserTracker();

    void init();
    void term();

    void process(snort::Packet*);
    void add_data(snort::Packet*);
    int scan(snort::Packet*, uint32_t&);
    void flush(struct snort::Packet*, unsigned, uint32_t);
    void detect(const struct snort::Packet*, const struct snort::StreamBuffer&, uint32_t, snort::Packet* up);

    std::list<UserSegment*> seg_list;
    snort::StreamSplitter* splitter;
    PAF_State paf_state = {};
    unsigned total;
};

class UserSession : public Session
{
public:
    UserSession(snort::Flow*);
    ~UserSession() override;

    bool setup(snort::Packet*) override;
    void clear() override;

    int process(snort::Packet*) override;

    void set_splitter(bool c2s, snort::StreamSplitter*) override;
    snort::StreamSplitter* get_splitter(bool c2s) override;
    void restart(snort::Packet*) override;

private:
    void start(snort::Packet*, snort::Flow*);
    void update(snort::Packet*, snort::Flow*);
    void end(snort::Packet*, snort::Flow*);

    bool add_alert(snort::Packet*, uint32_t gid, uint32_t sid) override;
    bool check_alerted(snort::Packet*, uint32_t gid, uint32_t sid) override;

    int update_alert(
        snort::Packet*, uint32_t gid, uint32_t sid, uint32_t event_id, uint32_t event_second) override;

public:
    UserTracker client;
    UserTracker server;
};

#endif

target_based/

The idea of a target-based system is to model the actual targets on the network instead of merely modeling the protocols and looking for attacks within them. When TCP/IP stacks are written for different operating systems, they are usually implemented by people who read the RFCs and then their interpretation of what the RFC outlines into code. Unfortunately, there are ambiguities in the way that the RFCs define some of the edge conditions that may occur and when this happens different people implement certain aspects of their TCP/IP stacks differently. For an IDS this is a big problem.

The basic idea behind target-based IDS is that we tell the IDS information about hosts on the network so that it can avoid attacks based on information about how an individual target TCP/IP stack operates.

host_attributes.h

Path = src/target_based/host_attributes.h

#ifndef HOST_ATTRIBUTES_H
#define HOST_ATTRIBUTES_H

// Provides attribute table initialization, lookup, swap, and releasing.

#include <functional>
#include <memory>
#include <mutex>
#include <vector>

#include "framework/counts.h"
#include "sfip/sf_ip.h"
#include "target_based/snort_protocols.h"

namespace snort
{
struct SnortConfig;
}

struct HostAttributeStats
{
    PegCount total_hosts = 0;
    PegCount hosts_pruned = 0;
    PegCount dynamic_host_adds = 0;
    PegCount dynamic_service_adds = 0;
    PegCount dynamic_service_updates = 0;
    PegCount service_list_overflows = 0;
};

class HostServiceDescriptor
{
public:
    HostServiceDescriptor() = default;
    HostServiceDescriptor(uint16_t port, uint16_t protocol, SnortProtocolId spi, bool appid_service)
        : port(port), ipproto(protocol), snort_protocol_id(spi), appid_service(appid_service)
    { }

    ~HostServiceDescriptor() = default;

    void reset()
    {
        port = 0;
        ipproto = 0;
        snort_protocol_id = UNKNOWN_PROTOCOL_ID;
    }

    uint16_t port = 0;
    uint16_t ipproto = 0;
    SnortProtocolId snort_protocol_id = UNKNOWN_PROTOCOL_ID;
    bool appid_service = false;
};

struct HostPolicyDescriptor
{
    uint8_t streamPolicy = 0;
    uint8_t fragPolicy = 0;
};

struct HostAttriInfo
{
    SnortProtocolId snort_protocol_id = UNKNOWN_PROTOCOL_ID;
    uint8_t stream_policy = 0;
    uint8_t frag_policy = 0;
};

class HostAttributesDescriptor
{
public:
    HostAttributesDescriptor() = default;
    ~HostAttributesDescriptor() = default;

    bool update_service(uint16_t port, uint16_t protocol, SnortProtocolId, bool& updated,
        bool is_appid_service = false);
    void clear_appid_services();
    void get_host_attributes(uint16_t, HostAttriInfo*) const;

    // Note: the following get/set are only called from main thread on a temp LRU table
    const snort::SfIp& get_ip_addr() const
    { return ip_address; }

    void set_ip_addr(const snort::SfIp& host_ip_addr)
    {
        ip_address = host_ip_addr;
    }

    void set_frag_policy(const uint8_t frag_policy)
    {
        policies.fragPolicy = frag_policy;
    }

    void set_stream_policy(uint8_t stream_policy)
    {
        policies.streamPolicy = stream_policy;
    }

private:
    mutable std::mutex host_attributes_lock; // ensure updates to this shared object are safe
    snort::SfIp ip_address;
    HostPolicyDescriptor policies;
    std::vector<HostServiceDescriptor> services;
};

typedef std::shared_ptr<HostAttributesDescriptor> HostAttributesEntry;

#define DEFAULT_MAX_ATTRIBUTE_HOSTS 10000
#define DEFAULT_MAX_ATTRIBUTE_SERVICES_PER_HOST 100
#define DEFAULT_MAX_METADATA_SERVICES 9

// Create a hash key from an IP address stored in a SfIp object.
struct HostAttributesCacheKey
{
    size_t operator()(const snort::SfIp& ip) const
    {
        const uint64_t* ip64 = (const uint64_t*) ip.get_ip6_ptr();
        return std::hash<uint64_t>() (ip64[0]) ^
               std::hash<uint64_t>() (ip64[1]);
    }
};

class HostAttributesManager
{
public:
    static bool load_hosts_file(snort::SnortConfig*, const char* fname);
    static void activate(snort::SnortConfig*);
    static void initialize();
    static void swap_cleanup();
    static void load_failure_cleanup();
    static void term();

    static bool add_host(HostAttributesEntry, snort::SnortConfig*);
    static bool get_host_attributes(const snort::SfIp&, uint16_t, HostAttriInfo*);
    static void update_service(const snort::SfIp&, uint16_t port, uint16_t protocol,
        SnortProtocolId, bool is_appid_service = false);
    static void clear_appid_services();
    static int32_t get_num_host_entries();
    static const PegInfo* get_pegs();
    static PegCount* get_peg_counts();
};

#endif

snort_protocols.h

Path = src/target_based/snort_protocols.h

#ifndef SNORT_PROTOCOLS_H
#define SNORT_PROTOCOLS_H

#include <string>
#include <vector>
#include <unordered_map>

#include "main/snort_types.h"

using SnortProtocolId = uint16_t;

// these protocols are always defined because
// they are used as consts in switch statements
// other protos are added dynamically as used
enum SnortProtocols : SnortProtocolId
{
    //  The is_*_protocol functions depend on the order of these enums.
    SNORT_PROTO_IP = 1,
    SNORT_PROTO_ICMP,
    SNORT_PROTO_TCP,
    SNORT_PROTO_UDP,
    SNORT_PROTO_FILE,
    SNORT_PROTO_MAX
};

constexpr SnortProtocolId UNKNOWN_PROTOCOL_ID = 0;
constexpr SnortProtocolId INVALID_PROTOCOL_ID = 0xffff;

inline bool is_network_protocol(SnortProtocolId proto)
{ return (proto >= SNORT_PROTO_IP and proto <= SNORT_PROTO_UDP); }

inline bool is_builtin_protocol(SnortProtocolId proto)
{ return proto < SNORT_PROTO_MAX; }

inline bool is_service_protocol(SnortProtocolId proto)
{ return proto > SNORT_PROTO_UDP; }

// A mapping between names and IDs.
namespace snort
{
class SO_PUBLIC ProtocolReference
{
public:
    ProtocolReference();
    ~ProtocolReference();

    ProtocolReference(ProtocolReference* old_proto_ref);

    ProtocolReference(const ProtocolReference&)  = delete;
    ProtocolReference& operator=(const ProtocolReference&)  = delete;

    SnortProtocolId get_count() const;

    const char* get_name(SnortProtocolId id) const;
    const char* get_name_sorted(SnortProtocolId id);

    SnortProtocolId add(const char* protocol);
    SnortProtocolId find(const char* protocol) const;

    bool operator()(SnortProtocolId a, SnortProtocolId b);

private:
    std::vector<const char*> id_map;
    std::vector<SnortProtocolId> ind_map;
    std::unordered_map<std::string, SnortProtocolId> ref_table;

    SnortProtocolId protocol_number = 0;

    void init(const ProtocolReference* old_proto_ref);
};

void protocol_reference_global_init();
void protocol_reference_global_term();

}
#endif

target_based/test/

time/

This module provides miscellaneous utilities related to timing.

  • Periodic provides registration and execution of callbacks that should be executed periodically at certain time intervals.

  • Packet time manages the updating and reading of a clock based on time values from acquired packets.

  • Stopwatch is a timekeeping utility that can be started and paused

clock_defs.h

Path = src/time/clock_defs.h

#ifndef CLOCK_DEFS_H
#define CLOCK_DEFS_H

#ifdef USE_TSC_CLOCK
#include "time/tsc_clock.h"
using SnortClock = TscClock;
#define CLOCK_ZERO 0
#define DURA_ZERO 0
#define TO_TICKS(t) (t)
#define TO_USECS(t) (t)
#define TO_NSECS(t) (t)
#define TO_DURATION(v, t) (t)

#else
#include <chrono>
using hr_clock = std::chrono::high_resolution_clock;
using SnortClock = hr_clock;
inline long clock_scale() { return 1; }
#define CLOCK_ZERO 0_ticks
#define DURA_ZERO Clock::duration::zero()
#define TO_TICKS(t) (t.count())
#define TO_USECS(t) (std::chrono::duration_cast<std::chrono::microseconds>(t).count())
#define TO_NSECS(t) (std::chrono::duration_cast<std::chrono::nanoseconds>(t).count())
#define TO_DURATION(v, t) (std::chrono::duration_cast<decltype(v)>(std::chrono::microseconds(t)))
#endif

using hr_duration = SnortClock::duration;
using hr_time = SnortClock::time_point;

inline constexpr hr_duration operator "" _ticks (unsigned long long int v)
{ return hr_duration(v); }

template<typename Clock,
    typename Duration = typename Clock::duration,
    typename TimePoint = typename Clock::time_point,
    typename Rep = typename Clock::rep>
struct ClockTraits
{
    using duration = Duration;
    using time_point = TimePoint;
    using rep = Rep;
};

inline long clock_usecs(long ticks)
{ return ticks / clock_scale(); }

inline long clock_ticks(long usecs)
{ return usecs * clock_scale(); }

#endif

packet_time.h

Path = src/time/packet_time.h

#ifndef PACKET_TIME_H
#define PACKET_TIME_H

#include <sys/time.h>

#include "main/snort_types.h"

namespace snort
{
SO_PUBLIC void packet_gettimeofday(struct timeval* tv);
SO_PUBLIC time_t packet_time();
SO_PUBLIC int64_t timersub_ms(const struct timeval* end, const struct timeval* start);
}

void packet_time_update(const struct timeval* cur_tv);
uint32_t packet_first_time();

#endif

periodic.h

Path = src/time/periodic.h

#ifndef PERIODIC_H
#define PERIODIC_H

#include <cstdint>

#include "main/snort_types.h"

using PeriodicHook = void (*)(void*);

class SO_PUBLIC Periodic
{
public:
    // lower number is higher priority
    static void register_handler(PeriodicHook, void*, uint16_t priority, uint32_t);
    static void check();

    static void unregister_all();
};

#endif

stopwatch.h

Path = src/time/stopwatch.h

#ifndef STOPWATCH_H
#define STOPWATCH_H

template<typename Clock>
class Stopwatch
{
public:
    using duration = typename Clock::duration;
    using time_point = typename Clock::time_point;

    Stopwatch() :
        elapsed { DURA_ZERO }, running { false } { }

    void start()
    {
        if ( running )
            return;

        start_time = Clock::now();
        running = true;
    }

    void stop()
    {
        if ( !running )
            return;

        elapsed += get_delta();
        running = false;
    }

    duration get() const
    {
        if ( running )
            return elapsed + get_delta();

        return elapsed;
    }

    bool active() const
    { return running; }

    void reset()
    { running = false; elapsed = DURA_ZERO; }

    void cancel()
    { running = false; }

private:
    duration get_delta() const
    { return Clock::now() - start_time; }

    duration elapsed;
    bool running;

#ifdef USE_TSC_CLOCK
    time_point start_time = CLOCK_ZERO;
#else
    time_point start_time;
#endif
};

#endif

timersub.h

Path = src/time/timersub.h

#ifndef TIMERSUB_H
#define TIMERSUB_H

// never worry about timersub type activities again -- from GLIBC and upcased.
#define TIMERSUB(a, b, result)                                                \
    do {                                                                      \
        (result)->tv_sec = (a)->tv_sec - (b)->tv_sec;                         \
        (result)->tv_usec = (a)->tv_usec - (b)->tv_usec;                      \
        if ((result)->tv_usec < 0) {                                          \
            --(result)->tv_sec;                                               \
            (result)->tv_usec += 1000000;                                     \
        }                                                                     \
    } while (0)

#endif

tsc_clock.h

Path = src/time/tsc_clock.h

#ifndef TSC_CLOCK_H
#define TSC_CLOCK_H

// the STL chrono clocks are kinda heavy so we use the time stamp counter
// where available (x86 with rdtsc support).  this wasn't always a good
// choice on multi-core systems but most now have rdtscp, constant_tsc,
// tsc_reliable, and nonstop_tsc.  note that we don't worry about exact
// instruction sequencing.
//
// references:
// http://stackoverflow.com/questions/275004/timer-function-to-provide-time-in-nano-seconds-using-c
// http://stackoverflow.com/questions/7935518/is-clock-gettime-adequate-for-submicrosecond-timing
//
// this clock stores ticks, not actual time values.  use ticks during runtime
// convert from/to usecs at startup/shutdown.  see clock_defs.h.

#include <chrono>

struct TscClock
{
    typedef uint64_t duration;
    typedef uint64_t time_point;

    static const bool is_steady = true;

    static uint64_t counter()
    {
#if defined(__i386__) || defined(__x86_64__)
        uint32_t lo, hi;
        asm volatile("rdtsc" : "=a" (lo), "=d" (hi));
        return ((uint64_t)hi << 32) | lo;
#elif defined(__aarch64__)
        uint64_t ticks;
        asm volatile("mrs %0, CNTVCT_EL0" : "=r" (ticks));
        return ticks;
#else
#error "TscClock is not supported on this architecture!"
#endif
    }

    static time_point now() noexcept
    {
        return time_point(duration(counter()));
    }
};

long clock_scale();

#endif

time/test/

trace/

This directory contains the trace logger framework.

  • TraceLogger

    The TraceLogger encapsulates the logging method for traces. A particular thread-local logger
    is created per each packet thread and one for the main thread. The logging configuration
    happens in the module. The logger factory is used to init/cleanup loggers.
    Include "trace_logger.h" to get TraceLogger base class.
    Built-in loggers are defined in "trace_loggers.h/trace_loggers.cc".
  • TraceLoggerFactory

    The base TraceLoggerFactory is used to create a particular TraceLogger instance per each
    thread. One factory instance exists which used to init/cleanup loggers and placed
    into TraceConfig. The factory object instantiates in the module due to configuration.
    Include "trace_logger.h" to get TraceLoggerFactory base class and template function
    to create particular objects. Built-in factories are defined in "trace_loggers.h/trace_loggers.cc".
  • TraceConfig

    This is a class that contains pointer on TraceLoggerFactory object and Trace list.
    By default all Traces are off. TraceModule fills Traces from configuration.
    TraceConfig placed into "trace_config.h".
  • TraceModule

    This module provides configuration for trace logs:
        output - create a concrete logger factory based on the output value (stdout/syslog).
        constraints - set packet constraints to use for trace filtering.
        modules - set modules trace level verbosity.
        ntuple - on/off packet n-tuple info logging.
        timestamp - on/off message time stamps.
    This is a built-in module (from coreinit.lua)
    The module placed into "trace_module.h/trace_module.cc".
    TraceModule ctor should be called after all existed modules due to TraceModule
        dynamic params restriction.
  • TraceSwap

    This class extends snort::AnalyzerCommand and represents control channel CLI commands
    for setting/clearing trace configuration from the shell. Commands parameters are encapsulated in
    the TraceSwapParams class.
    Available commands:
        1) trace.set({ modules = {}, constraints = {}, ntuple = true/false, timestamp = true/false })
           -- set new modules traces, constraints, ntuple and timestamp options;
        2) trace.clear() -- clear modules traces and constraints.
  • TraceSwapParams

    This is a helper class for TraceSwap which encapsulates dynamic initialization and storing of
    snort::Parameter and snort::Command lists based on TraceModule's parameters.
  • TraceParser

    This class encapsulates module trace options and packet constraints parsing and setting logic.
  • TraceApi

    TraceApi is a facade API class used to init/reinit/term thread-local trace logger and module's
    trace pointers and to match packets and flows against trace filtering constraints.
    TraceApi placed into "trace_api.h/trace_api.cc"
  • TraceOption

    Represents a targeted module's trace option.
    The targeted module provides a list of its trace options,
    which control the verbosity level of the module's trace messages.
    Since the messages can be present in release-build and/or debug-build,
    one should pay attention which option should be in the options list.
    TraceOptionID indexes (for the module's trace options) must start from zero.
  • Trace

    This class encapsulates trace verbosity functionality.
  • Configuration override

    By default, the factory will be initialized based on Snort run mode (stdout or syslog).
    But this can be explicitly overwritten in TraceModule's configuration to specify
    which kind of logger to use.
    Changes will be applied in case of reloading too.
  • External dependencies

    Include TraceApi header somewhere in the code where it needs to make trace logs.
    Make sure that TraceApi::thread_init()/thread_term() are provided for thread where
    TraceApi::log() is going used.
    TraceConfig should be configured in SnortConfig before TraceApi init.
    To create specific TraceLogger/TraceLoggerFactory pair just inherit base classes placed
    into "trace_logger.h" and init TraceConfig with a new factory during configuration.
  • Extending the trace logger framework with TraceLogger plugins

    It's possible to create a trace logger as an inspector plugin to handle a custom logic of trace
    messages printing. The workflow here is to implement the custom logger and logger factory by
    inheriting from the Snort TraceLogger and TraceLoggerFactory classes, put them into a separate
    plugin, and call TraceApi::override_logger_factory() during the plugin configuration to
    initialize the framework with the custom logger factory.
  • Disabling packet constraints matching

    Constraints matching can be disabled by setting "trace.constraints.match = false" during
    configuration. This is effectively saying all packets don't pass the trace filter. It may be
    useful in case of using external packet filtering, such as filtering by the DAQ, or to block
    printing for all trace messages in the context of a packet.

trace_api.h

Path = src/trace/trace_api.h

#ifndef TRACE_API_H
#define TRACE_API_H

#include <cstdarg>
#include <cstdint>

#include "main/snort_types.h"
#include "protocols/packet.h"
#include "trace.h"

class TraceConfig;

namespace snort
{
struct Packet;
struct SnortConfig;

class TraceLoggerFactory;

class SO_PUBLIC TraceApi
{
public:
    static void thread_init(const TraceConfig* tc);
    static void thread_reinit(const TraceConfig* tc);
    static void thread_term();

    // This method will change an ownership of the passed TraceLoggerFactory
    // from the caller to the passed SnortConfig
    static bool override_logger_factory(SnortConfig*, TraceLoggerFactory*);

    static void log(const char* log_msg, const char* name,
        uint8_t log_level, const char* trace_option, const Packet* p);
    static void filter(const Packet& p);
    static uint8_t get_constraints_generation();
};
}

static inline bool trace_enabled(const snort::Trace* trace,
    TraceOptionID trace_option_id,
    TraceLevel log_level = DEFAULT_TRACE_LOG_LEVEL,
    const snort::Packet* p = nullptr)
{
    if ( !trace or !trace->enabled(trace_option_id, log_level) )
        return false;

    if ( !p )
        return true;

    const auto gid = snort::TraceApi::get_constraints_generation();
    if ( !p->filtering_state.was_checked(gid) )
        snort::TraceApi::filter(*p);

    return p->filtering_state.matched;
}

namespace snort
{
SO_PUBLIC void trace_vprintf(const char* name, TraceLevel log_level,
    const char* trace_option, const snort::Packet* p, const char* fmt, va_list);
}

using trace_func = void(const char*, TraceLevel, const char*, const snort::Packet*, const char*, va_list);

template <trace_func>
static inline void trace_uprintf(const snort::Trace* trace,
    TraceOptionID trace_option_id, const snort::Packet* p, const char* fmt, ...) __attribute__((format (printf, 4, 5)));

template <trace_func trace_vprintf = snort::trace_vprintf>
static inline void trace_uprintf(const snort::Trace* trace,
    TraceOptionID trace_option_id, const snort::Packet* p, const char* fmt, ...)
{
    va_list ap;
    va_start(ap, fmt);

    const char* trace_option_name = trace->option_name(trace_option_id);
    trace_vprintf(trace->module_name(), DEFAULT_TRACE_LOG_LEVEL, trace_option_name, p, fmt, ap);

    va_end(ap);
}

template <trace_func>
static inline void trace_printf(TraceLevel log_level,
    const snort::Trace* trace, TraceOptionID trace_option_id,
    const snort::Packet* p, const char* fmt, ...)
    __attribute__((format (printf, 5, 6)));

template <trace_func trace_vprintf = snort::trace_vprintf>
static inline void trace_printf(TraceLevel log_level,
    const snort::Trace* trace, TraceOptionID trace_option_id,
    const snort::Packet* p, const char* fmt, ...)
{
    if ( !trace_enabled(trace, trace_option_id, log_level, p) )
        return;

    va_list ap;
    va_start(ap, fmt);

    const char* trace_option_name = trace->option_name(trace_option_id);
    trace_vprintf(trace->module_name(), log_level, trace_option_name, p,
        fmt, ap);

    va_end(ap);
}

template <trace_func trace_vprintf = snort::trace_vprintf>
static inline void trace_printf(TraceLevel log_level, const snort::Trace* trace, TraceOptionID trace_option_id,
    const snort::Packet* p, const char* fmt, va_list ap)
{
    if ( !trace_enabled(trace, trace_option_id, log_level, p) )
        return;
    const char* trace_option_name = trace->option_name(trace_option_id);
    trace_vprintf(trace->module_name(), log_level, trace_option_name, p, fmt, ap);
}

template <trace_func>
static inline void trace_printf(TraceLevel log_level,
    const snort::Trace* trace, const snort::Packet* p,
    const char* fmt, ...) __attribute__((format (printf, 4, 5)));

template <trace_func trace_vprintf = snort::trace_vprintf>
static inline void trace_printf(TraceLevel log_level,
    const snort::Trace* trace, const snort::Packet* p,
    const char* fmt, ...)
{
    if ( !trace_enabled(trace, DEFAULT_TRACE_OPTION_ID, log_level, p) )
        return;

    va_list ap;
    va_start(ap, fmt);

    const char* trace_option_name = trace->option_name(DEFAULT_TRACE_OPTION_ID);
    trace_vprintf(trace->module_name(), log_level, trace_option_name, p,
        fmt, ap);

    va_end(ap);
}

template <trace_func>
static inline void trace_printf(const snort::Trace* trace,
    TraceOptionID trace_option_id, const snort::Packet* p,
    const char* fmt, ...) __attribute__((format (printf, 4, 5)));

template <trace_func trace_vprintf = snort::trace_vprintf>
static inline void trace_printf(const snort::Trace* trace,
    TraceOptionID trace_option_id, const snort::Packet* p, const char* fmt, ...)
{
    if ( !trace_enabled(trace, trace_option_id, DEFAULT_TRACE_LOG_LEVEL, p) )
        return;

    va_list ap;
    va_start(ap, fmt);

    const char* trace_option_name = trace->option_name(trace_option_id);
    trace_vprintf(trace->module_name(), DEFAULT_TRACE_LOG_LEVEL,
        trace_option_name, p, fmt, ap);

    va_end(ap);
}

template <trace_func>
static inline void trace_printf(const snort::Trace* trace,
    const snort::Packet* p, const char* fmt, ...)
    __attribute__((format (printf, 3, 4)));

template <trace_func trace_vprintf = snort::trace_vprintf>
static inline void trace_printf(const snort::Trace* trace,
    const snort::Packet* p, const char* fmt, ...)
{
    if ( !trace_enabled(trace, DEFAULT_TRACE_OPTION_ID, DEFAULT_TRACE_LOG_LEVEL, p) )
        return;

    va_list ap;
    va_start(ap, fmt);

    const char* trace_option_name = trace->option_name(DEFAULT_TRACE_OPTION_ID);
    trace_vprintf(trace->module_name(), DEFAULT_TRACE_LOG_LEVEL,
        trace_option_name, p, fmt, ap);

    va_end(ap);
}

template <trace_func trace_vprintf = snort::trace_vprintf>
static inline void trace_print(TraceLevel log_level,
    const snort::Trace* trace, TraceOptionID trace_option_id,
    const snort::Packet* p, const char* msg)
{
    trace_printf<trace_vprintf>(log_level, trace, trace_option_id, p,
        "%s", msg);
}

template <trace_func trace_vprintf = snort::trace_vprintf>
static inline void trace_print(const snort::Trace* trace,
    TraceOptionID trace_option_id, const snort::Packet* p, const char* msg)
{
    trace_printf<trace_vprintf>(trace, trace_option_id, p, "%s", msg);
}

template <trace_func trace_vprintf = snort::trace_vprintf>
static inline void trace_print(TraceLevel log_level,
    const snort::Trace* trace, const snort::Packet* p, const char* msg)
{
    trace_printf<trace_vprintf>(log_level, trace, p, "%s", msg);
}

template <trace_func trace_vprintf = snort::trace_vprintf>
static inline void trace_print(const snort::Trace* trace, const snort::Packet* p,
    const char* msg)
{
    trace_printf<trace_vprintf>(trace, p, "%s", msg);
}

#define trace_print trace_print<snort::trace_vprintf>
#define trace_printf trace_printf<snort::trace_vprintf>
#define trace_uprintf trace_uprintf<snort::trace_vprintf>

#define trace_log(...) trace_print(__VA_ARGS__)
#define trace_logf(...) trace_printf(__VA_ARGS__)
#define trace_ulogf(...) trace_uprintf(__VA_ARGS__)

#ifdef DEBUG_MSGS
#define debug_log trace_log
#define debug_logf trace_logf
#else
#define debug_log(...)
#define debug_logf(...)
#endif

#endif // TRACE_API_H

trace_config.h

Path = src/trace/trace_config.h

#ifndef TRACE_CONFIG_H
#define TRACE_CONFIG_H

#include "trace.h"

namespace snort
{
struct PacketConstraints;
class TraceLoggerFactory;
}

class TraceConfig
{
public:
    TraceConfig();
    TraceConfig(const TraceConfig&);
    ~TraceConfig();

    void setup_module_trace() const;
    bool set_trace(const std::string& module_name,
        const std::string& trace_option_name, uint8_t trace_level);

    void clear();
    void clear_traces();

public:
    snort::TraceLoggerFactory* logger_factory = nullptr;
    snort::PacketConstraints* constraints = nullptr;

    bool ntuple = false;
    bool timestamp = false;
    bool initialized = false;

private:
    Traces traces;
};

#endif // TRACE_CONFIG_H

trace.h

Path = src/trace/trace.h

#ifndef TRACE_H
#define TRACE_H

#include <cassert>
#include <map>
#include <vector>

#include "main/thread.h"

#define DEFAULT_TRACE_LOG_LEVEL 1
#define TRACE_CRITICAL_LEVEL 2
#define TRACE_ERROR_LEVEL 3
#define TRACE_WARNING_LEVEL 4
#define TRACE_INFO_LEVEL 6
#define TRACE_DEBUG_LEVEL 7
#define DEFAULT_TRACE_OPTION_ID 0
#define DEFAULT_TRACE_OPTION_NAME "all"

namespace snort
{
class Trace;
}

using TraceOptionID = uint8_t;
using TraceLevel = uint8_t;
using Traces = std::vector<snort::Trace>;
using ConfigTraceOptions = std::map<std::string, bool>;
using ModulesConfigTraceOptions = std::map<std::string, ConfigTraceOptions>;

namespace snort
{
class Module;

struct TraceOption
{
    TraceOption(const char* n, TraceOptionID tid, const char* h) :
        name(n), id(tid), help(h) {}

    const char* name;
    TraceOptionID id;
    const char* help;
};

class Trace
{
public:
    Trace(const Module& m);
    Trace& operator=(const Trace&);

    bool set(const std::string& option_name, uint8_t option_level);
    void set_module_trace() const;

    void clear();

    const char* module_name() const
    { return mod_name.c_str(); }

    const char* option_name(size_t index) const
    {
        assert(index < option_levels.size());
        return options[index].name;
    }

    bool enabled(TraceOptionID trace_option_id, TraceLevel log_level = DEFAULT_TRACE_LOG_LEVEL) const
    {
        assert(trace_option_id < option_levels.size());
        return option_levels[trace_option_id] >= log_level;
    }

private:
    std::string mod_name;
    const TraceOption* options;
    const Module& module;
    std::vector<TraceLevel> option_levels;
};
}

#endif

trace_logger.h

Path = src/trace/trace_logger.h

#ifndef TRACE_LOGGER_H
#define TRACE_LOGGER_H

#include <cstdint>

namespace snort
{
struct Packet;

class TraceLogger
{
public:
    virtual ~TraceLogger() = default;

    virtual void log(const char* log_msg, const char* name,
        uint8_t log_level, const char* trace_option, const Packet* p) = 0;

    void set_ntuple(bool flag)
    { ntuple = flag; }

    void set_timestamp(bool flag)
    { timestamp = flag; }

protected:
    bool ntuple = false;
    bool timestamp = false;
};

class TraceLoggerFactory
{
public:
    virtual ~TraceLoggerFactory() = default;

    virtual TraceLogger* instantiate() = 0;
};
}

#endif // TRACE_LOGGER_H

trace_loggers.h

Path = src/trace/trace_loggers.h

#ifndef TRACE_LOGGERS_H
#define TRACE_LOGGERS_H

#include "trace_logger.h"

//-----------------------------------------------
//  Logger factories
//-----------------------------------------------

class StdoutLoggerFactory : public snort::TraceLoggerFactory
{
public:
    StdoutLoggerFactory() = default;
    StdoutLoggerFactory(const StdoutLoggerFactory&) = delete;
    StdoutLoggerFactory& operator=(const StdoutLoggerFactory&) = delete;

    snort::TraceLogger* instantiate() override;
};

class SyslogLoggerFactory : public snort::TraceLoggerFactory
{
public:
    SyslogLoggerFactory() = default;
    SyslogLoggerFactory(const SyslogLoggerFactory&) = delete;
    SyslogLoggerFactory& operator=(const SyslogLoggerFactory&) = delete;

    snort::TraceLogger* instantiate() override;
};

#endif // TRACE_LOGGERS_H

trace_module.h

Path = src/trace/trace_module.h

#ifndef TRACE_MODULE_H
#define TRACE_MODULE_H

#include "framework/module.h"

class TraceParser;

class TraceModule : public snort::Module
{
private:
    enum OutputType
    {
        OUTPUT_TYPE_STDOUT = 0,
        OUTPUT_TYPE_SYSLOG,
        OUTPUT_TYPE_NO_INIT
    };

public:
    TraceModule();
    ~TraceModule() override;

    const snort::Command* get_commands() const override;
    bool begin(const char*, int, snort::SnortConfig*) override;
    bool set(const char*, snort::Value&, snort::SnortConfig*) override;
    bool end(const char*, int, snort::SnortConfig*) override;

    Usage get_usage() const override
    { return GLOBAL; }

private:
    void generate_params();

private:
    OutputType log_output_type = OUTPUT_TYPE_NO_INIT;
    bool local_syslog = false;

    std::vector<snort::Parameter> modules_params;
    std::vector<std::vector<snort::Parameter>> module_ranges;
    std::vector<std::string> modules_help;

    TraceParser* trace_parser = nullptr;
};

#endif  // TRACE_MODULE_H

trace_parser.h

Path = src/trace/trace_parser.h

#ifndef TRACE_PARSER_H
#define TRACE_PARSER_H

#include <map>
#include <string>

#include "framework/packet_constraints.h"

namespace snort
{
class Module;
class Value;
}

class TraceConfig;

class TraceParser
{
public:
    TraceParser(TraceConfig&);

    bool set_traces(const std::string& option_name, const snort::Value& val);
    bool set_constraints(const snort::Value& val);

    void finalize_constraints();
    void clear_traces();
    void clear_constraints();

    void reset_configured_trace_options();

    TraceConfig& get_trace_config() const
    { return trace_config; }

private:
    void init_configured_trace_options();

private:
    TraceConfig& trace_config;
    snort::PacketConstraints parsed_constraints{};

    static std::map<std::string, std::map<std::string, bool>> s_configured_trace_options;
};

#endif // TRACE_PARSER_H

trace_swap.h

Path = src/trace/trace_swap.h

#ifndef TRACE_SWAP_H
#define TRACE_SWAP_H

namespace snort
{
struct Command;
struct Parameter;
}

class TraceSwapParams
{
public:
    static void set_params(const snort::Parameter* params);

    static const snort::Command* get_commands();
    static const snort::Parameter* get_params();

private:
    static const snort::Command* s_commands;
    static const snort::Parameter* s_params;
};

#endif // TRACE_SWAP_H

utils/

This unit contains a mixed bag of legacy utilities that haven’t found a home in any other directory. In many cases, the STL provides better options.

On stream buffer, there are two classes inherited from std::streambuf:

  • istreambuf_glue class for reading operations

  • ostreambuf_infl class for writing operations

The input stream buffer presents a continuous sequence of bytes to the client, gathered from different sources. For example:

char* s1 = "world";
char* s2 = "!";
char* s3 = "Hello ";

These sources being fed to the stream buffer as s3, s1, s2 will form "Hello world!" sequence.

In order to do that, istreambuf_glue class represents each source as a chunk of data, which has its own position in the resulting sequence. The chunk structure contains a pointer to the source, source size, and the chunk’s offset in the resulting sequence.

Reading is done sequentially within the current chunk. When the end of chunk reached, the buffer switches to the next one, setting std::streambuf pointers.

Positioning the cursor is done in two steps:

  1. Calculate the final cursor position (absolute or by offset).

  2. Find the right chunk and local offset in it to set cursor there.

Currently, no intermediate buffering done between chunks (like alignment, prepending/appending the next chunk). The buffer doesn’t take ownership over the source’s memory.

The output stream buffer is mostly like std::stringbuf. The main purpose of it is having an extensible dynamic array, where clients could write their data, not worrying about resizing and memory management.

Aside from that, ostreambuf_infl can give away ownership over its memory, which could be useful for final consumer.

From performance perspective, ostreambuf_infl can reserve an amount of memory before actual operations. Also, memory extending is done by predefined portions of 211, 212, 213, 214, 215, 215, 215… This tries to minimize the number of memory reallocation.

boyer_moore.h

Path = src/utils/boyer_moore.h

#ifndef BOYER_MOORE_H
#define BOYER_MOORE_H

// Boyer-Moore pattern matching routines

#include "main/snort_types.h"

namespace snort
{

class SO_PUBLIC BoyerMoore
{
public:
    BoyerMoore(const uint8_t* pattern, unsigned pattern_len);

    int search(const uint8_t* buffer, unsigned buffer_len) const;
    int search_nocase(const uint8_t* buffer, unsigned buffer_len) const;

private:
    void make_skip();

    const uint8_t* pattern;
    unsigned pattern_len;
    unsigned last;

    unsigned skip[256];
};

}
#endif

cpp_macros.h

Path = src/utils/cpp_macros.h

#ifndef CPP_MACROS_H
#define CPP_MACROS_H

// Miscellaneous C preprocessor macros

// Turn x into a string literal without expanding macro definitions
// Example:
//      #define BUILD 123
//      STRINGIFY(BUILD)
// Result:
//      "BUILD"
#define STRINGIFY(x) #x

// Turn x into a string literal after macro-expanding it
// Example:
//      #define BUILD 123
//      STRINGIFY_MX(BUILD)
// Result:
//      "123"
#define STRINGIFY_MX(x) STRINGIFY(x)

// Concatenate preprocessor tokens x and y without expanding macro definitions
// Example:
//      #define ice snow
//      #define cream cone
//      PPCAT(ice, cream)
// Result:
//      icecream
#define PPCAT(x, y) x ## y

// Concatenate preprocessor tokens x and y after macro-expanding them
// Example:
//      #define ice snow
//      #define cream cone
//      PPCAT_MX(ice, cream)
// Result:
//      snowcone
#define PPCAT_MX(x, y) PPCAT(x, y)

// Pair of macros to temporarily enable and then disable warnings for structures
// being automatically padded.  Currently implemented for Clang and GCC >= 5.0.
#if defined(__clang__) && !defined(__ICC)
#  define PADDING_GUARD_BEGIN \
    _Pragma(STRINGIFY( clang diagnostic push )) \
    _Pragma(STRINGIFY( clang diagnostic warning "-Wpadded" ))
#  define PADDING_GUARD_END \
    _Pragma(STRINGIFY( clang diagnostic pop ))
#elif defined(__GNUC__) && __GNUC__ > 4 && !defined(__ICC)
#  define PADDING_GUARD_BEGIN \
    _Pragma(STRINGIFY( GCC diagnostic push )) \
    _Pragma(STRINGIFY( GCC diagnostic warning "-Wpadded" ))
#  define PADDING_GUARD_END \
    _Pragma(STRINGIFY( GCC diagnostic pop ))
#else
#  define PADDING_GUARD_BEGIN
#  define PADDING_GUARD_END
#endif

// Pair of macros to temporarily disable and then enable warnings for code
// that is accessing members of packed types with pointers
#if defined(__clang__) && __clang_major__ >= 4 && !defined(__ICC)
#  define PACKED_MEMBER_ADDR_BEGIN \
    _Pragma(STRINGIFY( clang diagnostic push )) \
    _Pragma(STRINGIFY( clang diagnostic ignored "-Waddress-of-packed-member" ))
#  define PACKED_MEMBER_ADDR_END \
    _Pragma(STRINGIFY( clang diagnostic pop ))
#elif defined(__GNUC__) && __GNUC__ >= 9 && !defined(__ICC)
#  define PACKED_MEMBER_ADDR_BEGIN \
    _Pragma(STRINGIFY( GCC diagnostic push )) \
    _Pragma(STRINGIFY( GCC diagnostic ignored "-Waddress-of-packed-member" ))
#  define PACKED_MEMBER_ADDR_END \
    _Pragma(STRINGIFY( GCC diagnostic pop ))
#else
#  define PACKED_MEMBER_ADDR_BEGIN
#  define PACKED_MEMBER_ADDR_END
#endif

#endif

dnet_header.h

Path = src/utils/dnet_header.h

#ifndef UTILS_DNET_HEADER_H
#define UTILS_DNET_HEADER_H

// Provide the correct dnet interface

#if defined(__clang__)
#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wc99-extensions"
#pragma clang diagnostic ignored "-Wflexible-array-extensions"
#endif

#if defined(__GNUC__) && !defined(__INTEL_COMPILER)
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wpedantic"
#endif

#ifdef HAVE_DUMBNET_H
#include <dumbnet.h>
#else
#include <dnet.h>
#endif

#if defined(__clang__)
#pragma clang diagnostic pop
#endif

#if defined(__GNUC__) && !defined(__INTEL_COMPILER)
#pragma GCC diagnostic pop
#endif

#endif

endian.h

Path = src/utils/endian.h

#ifndef SNORT_ENDIAN_H
#define SNORT_ENDIAN_H

# include <netinet/in.h>

#ifdef __FreeBSD__
# include <sys/endian.h>
# define bswap_16(a) bswap16(a)
# define bswap_32(a) bswap32(a)
# define bswap_64(a) bswap64(a)
#endif

#ifdef __linux__
# include <byteswap.h>
#endif

#ifndef __APPLE__
# define htonll(a) ( 1 == htonl(1) ? (a) : bswap_64(a) )
# define ntohll(a) ( 1 == ntohl(1) ? (a) : bswap_64(a) )
#endif

#if defined(WORDS_BIGENDIAN)
#define LETOHS(p)   bswap_16(*((const uint16_t*)(p)))
#define LETOHL(p)   bswap_32(*((const uint32_t*)(p)))
#else
#define LETOHS(p)   (*((const uint16_t*)(p)))
#define LETOHL(p)   (*((const uint32_t*)(p)))
#endif

#define LETOHS_UNALIGNED(p) \
    ((uint16_t)(*((const uint8_t*)(p) + 1) << 8) | \
     (uint16_t)(*((const uint8_t*)(p))))

#define LETOHL_UNALIGNED(p) \
    ((uint32_t)(*((const uint8_t*)(p) + 3) << 24) | \
     (uint32_t)(*((const uint8_t*)(p) + 2) << 16) | \
     (uint32_t)(*((const uint8_t*)(p) + 1) <<  8) | \
     (uint32_t)(*((const uint8_t*)(p))))

#define LETOHLL_UNALIGNED(p) \
    (((uint64_t)(LETOHL_UNALIGNED(p + 4)) << 32) | ((uint64_t)(LETOHL_UNALIGNED(p))))

#define BETOHS_UNALIGNED(p) \
    ((uint16_t)(*((const uint8_t*)(p)) << 8) | \
     (uint16_t)(*((const uint8_t*)(p) + 1)))

#define BETOHL_UNALIGNED(p) \
    ((uint32_t)(*((const uint8_t*)(p)) << 24) | \
     (uint32_t)(*((const uint8_t*)(p) + 1) << 16) | \
     (uint32_t)(*((const uint8_t*)(p) + 2) <<  8) | \
     (uint32_t)(*((const uint8_t*)(p) + 3)))

#define BETOHLL_UNALIGNED(p) \
    (((uint64_t)(BETOHL_UNALIGNED(p)) << 32) | ((uint64_t)(BETOHL_UNALIGNED(p + 4))))

#endif

event_gen.h

Path = src/utils/event_gen.h

#ifndef EVENT_GEN_H
#define EVENT_GEN_H

#include <bitset>
#include <cassert>

#include "detection/detection_engine.h"

//-------------------------------------------------------------------------
// Event generator class
//-------------------------------------------------------------------------

template <int EVENT_MAX, int EVENT_NONE, int GID>
class EventGen
{
public:
    EventGen() = default;

    EventGen(const std::bitset<EVENT_MAX>& suppressed) : events_suppressed(suppressed) {}

    void create_event(int sid)
    {
        if (sid == EVENT_NONE)
            return;
        assert((sid > 0) && (sid <= EVENT_MAX));
        if (!events_suppressed[sid-1])
        {
            snort::DetectionEngine::queue_event(GID, (uint32_t)sid);
            events_suppressed[sid-1] = true;
#ifdef REG_TEST
            events_generated[sid-1] = true;
#endif
        }
    }

    void suppress_event(int sid)
    {
        assert((sid > 0) && (sid <= EVENT_MAX));
        events_suppressed[sid-1] = true;
    }

#ifdef REG_TEST
    bool none_found() const { return events_generated == 0; }

    uint64_t get_raw(unsigned base) const { return
        ((events_generated >> base) & bitmask).to_ulong(); }
#endif

protected:
    std::bitset<EVENT_MAX> events_suppressed = 0;

#ifdef REG_TEST
    std::bitset<EVENT_MAX> events_generated = 0;
    const std::bitset<EVENT_MAX> bitmask = 0xFFFFFFFFFFFFFFFF;
#endif
};

#endif

grouped_list.h

Path = src/utils/grouped_list.h

#ifndef GROUPED_LIST_H
#define GROUPED_LIST_H

// The class below represents a group (double-linked list) of elements,
// where each element belongs to a sub-group.
// Sub-groups never intersect,
// but a sub-group of elements can be deleted altogether from the main group.

#include <assert.h>

namespace snort
{

template <class T>
class GroupedList
{
public:
    GroupedList();
    GroupedList(GroupedList& cont, GroupedList*& group, const T& value);
    GroupedList(GroupedList& cont, GroupedList*& group, T&& value);
    template <class... Args>
    GroupedList(GroupedList& cont, GroupedList*& group, Args&&... args);
    ~GroupedList();

    inline T& operator *();
    inline GroupedList* get_next() const;
    inline void leave_group();
    inline static unsigned erase_group(GroupedList*& group);

private:
    inline void init();
    inline void leave();
    inline void erase_all();

    GroupedList* prev;       // previous element in a container (list 1)
    GroupedList* next;       // next element in a container (list 1)
    GroupedList* mate;       // next element in a group (list 2)
    GroupedList** grp;       // pointer to this element, a group holder (list 2)
    T value;                 // value of element
    const bool holder;       // if the element is a container holder
};

template <typename T>
GroupedList<T>::GroupedList()
    : prev(this), next(this), mate(nullptr), grp(nullptr), holder(true)
{ }

template <typename T>
GroupedList<T>::GroupedList(GroupedList& cont, GroupedList*& group, const T& v)
    : prev(cont.prev), next(&cont), mate(group), grp(&group), value(v), holder(false)
{
    init();
    group = this;
}

template <typename T>
GroupedList<T>::GroupedList(GroupedList& cont, GroupedList*& group, T&& v)
    : prev(cont.prev), next(&cont), mate(group), grp(&group),value(v), holder(false)
{
    init();
    group = this;
}

template <typename T> template <class... Args>
GroupedList<T>::GroupedList(GroupedList& cont, GroupedList*& group, Args&&... args)
    : prev(cont.prev), next(&cont), mate(group), grp(&group),value{std::forward<Args>(args)...}, holder(false)
{
    init();
    group = this;
}

template <typename T>
GroupedList<T>::~GroupedList()
{
    if (holder)
        erase_all();
    else
        leave();
}

template <typename T>
T& GroupedList<T>::operator *()
{
    return value;
}

template <typename T>
GroupedList<T>* GroupedList<T>::get_next() const
{
    return next;
}

template <typename T>
inline void GroupedList<T>::leave_group()
{
    if (grp)
        *grp = *grp == this ? mate : nullptr;

    if (mate)
        mate->grp = grp;

    grp = nullptr;
    mate = nullptr;

    assert(!holder);
}

template <typename T>
inline void GroupedList<T>::init()
{
    assert(prev);
    assert(next);
    assert(next->holder);
    assert(!mate or !mate->holder);

    prev->next = this;
    next->prev = this;

    if (mate)
        mate->grp = &mate;
}

template <typename T>
void GroupedList<T>::leave()
{
    assert(!prev or prev->next == this);
    assert(!next or next->prev == this);

    if (prev)
        prev->next = next;

    if (next)
        next->prev = prev;
}

template <typename T>
void GroupedList<T>::erase_all()
{
    auto it = next;

    while (it != this)
    {
        auto el = it;
        it = it->next;

        assert(!el->holder);
        delete el;
    }
}

template <typename T>
unsigned GroupedList<T>::erase_group(GroupedList<T>*& group)
{
    unsigned cnt = 0;
    auto it = group;
    group = nullptr;

    while (it)
    {
        auto el = it;
        it = it->mate;
        ++cnt;

        assert(!el->holder);
        delete el;
    }

    return cnt;
}

}

#endif

infractions.h

Path = src/utils/infractions.h

#ifndef INFRACTIONS_H
#define INFRACTIONS_H

#include <bitset>
#include <cassert>

//-------------------------------------------------------------------------
// Infractions class
//-------------------------------------------------------------------------

template <int MAX, int NONE>
class Infractions
{
public:
    Infractions() = default;
    Infractions(int inf)
    {
        if (inf == NONE)
            return;
        assert((inf >= 0) && (inf < MAX));
        infractions[inf] = true;
    }
    Infractions& operator+=(const Infractions& rhs)
        { infractions |= rhs.infractions; return *this; }
    friend Infractions operator+(Infractions lhs, const Infractions& rhs)
        { lhs += rhs; return lhs; }
    friend bool operator&(const Infractions& lhs, const Infractions& rhs)
        { return (lhs.infractions & rhs.infractions) != 0; }

    bool none_found() const { return infractions == 0; }

    uint64_t get_raw(unsigned base) const { return
        ((infractions >> base) & std::bitset<MAX>(0xFFFFFFFFFFFFFFFF)).to_ulong(); }

private:
    std::bitset<MAX> infractions = 0;
};

#endif

kmap.h

Path = src/utils/kmap.h

#ifndef KMAP_H
#define KMAP_H

// Keyword Trie based Map Table
// The tree uses linked lists to build the finite automata
//
// MapKeyFind(): Performs a setwise strcmp() equivalent.
//
// Keys may be ascii or binary, both may be of random sizes.  Each key may
// be a different size, or all one size.  Fast dictionary lookup,
// proportional to the length of the key, and independent of the number of
// keys in the table.  May use more memory than a hash table, depends.
// Memory is allocated as needed, so none is wasted.

#include "main/snort_types.h"

#define ALPHABET_SIZE 256

typedef struct _keynode
{
    struct  _keynode* next;

    unsigned char* key;
    int nkey;
    void* userdata;          /* data associated with this pattern */
} KEYNODE;

typedef struct _kmapnode
{
    int nodechar;     /* node character */

    struct  _kmapnode* sibling;
    struct  _kmapnode* child;

    KEYNODE* knode;
} KMAPNODE;

typedef void (* KMapUserFreeFunc)(void* p);

typedef struct _kmap
{
    KMAPNODE* root[256]; /* KTrie nodes */

    KEYNODE* keylist; // list of key+data pairs
    KEYNODE* keynext; // findfirst/findnext node

    KMapUserFreeFunc userfree;

    int nchars;      // # character nodes
    bool nocase;

} KMAP;

namespace snort
{
SO_PUBLIC KMAP* KMapNew(KMapUserFreeFunc, bool nocase);
SO_PUBLIC void KMapDelete(KMAP*);

SO_PUBLIC int KMapAdd(KMAP*, void* key, int ksize, void* userdata);

SO_PUBLIC void* KMapFind(KMAP*, void* key, int ksize);
SO_PUBLIC void* KMapFindFirst(KMAP*);
SO_PUBLIC void* KMapFindNext(KMAP*);
}
#endif

memcap_allocator.h

Path = src/utils/memcap_allocator.h

#ifndef MEMCAP_ALLOCATOR_H
#define MEMCAP_ALLOCATOR_H

#include "framework/counts.h"
#include "utils/util.h"

class MemCapAllocator
{
public:
    MemCapAllocator(unsigned long capacity, unsigned long allocation_size)
        : mem_capacity(capacity), allocation_size(allocation_size)
    { }

    void* allocate()
    {
        void* data = nullptr;
        if ( is_space_available() )
        {
            data = snort_calloc(allocation_size);
            mem_allocated += allocation_size;
            ++allocation_requests;
        }
        else
            ++no_memory_available;

        return data;
    }

    void free(void* mem)
    {
        snort_free(mem);
        mem_allocated -= allocation_size;
        ++free_requests;
    }

    bool is_space_available() const
    {
        if ( !mem_capacity )
            return true;
        else
            return mem_allocated + allocation_size <= mem_capacity;
    }

    bool is_over_capacity() const
    {
        if ( !mem_capacity )
            return false;
        else
            return mem_allocated > mem_capacity;
    }

    void set_mem_capacity(unsigned long capacity)
    { mem_capacity = capacity; }

    unsigned long get_mem_capacity() const
    { return mem_capacity; }

    unsigned long get_allocation_size() const
    { return allocation_size; }

    unsigned long get_mem_allocated() const
    { return mem_allocated; }

    PegCount get_allocation_requests() const
    { return allocation_requests; }

    PegCount get_free_requests() const
    { return free_requests; }

    PegCount get_no_memory_available() const
    { return no_memory_available; }

private:
    unsigned long mem_capacity;
    unsigned long allocation_size;
    unsigned long mem_allocated = 0;

    PegCount allocation_requests = 0;
    PegCount free_requests = 0;
    PegCount no_memory_available = 0;
};

#endif

primed_allocator.h

Path = src/utils/primed_allocator.h

#ifndef PRIMED_ALLOCATOR_H
#define PRIMED_ALLOCATOR_H

// An STL list allocator that pools deallocs on a freelist but only frees memory at destruction.
// Use this for containers where the maximum size is somehow capped but where storage may not reach
// that cap. This prevents unnecessary memory allocation but still provides the speed benefit of
// not repeatedly allocating memory. Additionally, the free list may be shared among multiple
// STL lists of the same type.

namespace snort
{
template<typename T>
class PrimedAllocator
{
public:
    struct Node
    {
        Node* next;
    };

    struct State
    {
        Node* free_list = nullptr;
        unsigned ref_count = 0;

        ~State()
        {
            while ( free_list )
            {
                Node* next = free_list->next;
                delete free_list;
                free_list = next;
            }
        }
    };

    using size_type = std::size_t;
    using pointer = T*;
    using const_pointer = const T*;
    using value_type = T;

    template<typename U> struct rebind { typedef PrimedAllocator<U> other; };

    PrimedAllocator() noexcept
    { state = new State; }

    // cppcheck-suppress copyCtorPointerCopying
    PrimedAllocator(const PrimedAllocator& other) noexcept : state(other.state)
    {
        state->ref_count++;
    }

    ~PrimedAllocator()
    {
        state->ref_count--;

        if ( !state->ref_count )
            delete state;
    }

    void set_state(State* state)
    {
        this->state->ref_count--;

        if ( !this->state->ref_count )
            delete state;

        this->state = state;
        state->ref_count++;
    }

    State* get_state() const
    { return state; }

    pointer allocate(size_type, const_pointer = 0)
    {
        if ( state->free_list )
        {
            T* ret = reinterpret_cast<T*>(state->free_list);
            state->free_list = state->free_list->next;
            return ret;
        }
        return reinterpret_cast<T*>(operator new(sizeof(T)));
    }

    void deallocate(pointer p, size_type) noexcept
    {
        Node* node = reinterpret_cast<Node*>(p);
        node->next = state->free_list;
        state->free_list = node;
    }

private:
    State* state;
};
}

#endif

safec.h

Path = src/utils/safec.h

#ifndef UTILS_SAFEC_H
#define UTILS_SAFEC_H

#ifdef HAVE_SAFEC

extern "C"
{
#include <safe_mem_lib.h>
#include <safe_str_lib.h>
}

#else

#define memcpy_s(dst, dsize, src, ssize) memcpy(dst, src, ssize)
#define memmove_s(dst, dsize, src, ssize) memmove(dst, src, ssize)

#define set_mem_constraint_handler_s(x)
#define set_str_constraint_handler_s(x)

#endif

#endif

sflsq.h

Path = src/utils/sflsq.h

#ifndef SFLSQ_H
#define SFLSQ_H

#include "main/snort_types.h"

// Simple LIST interface
// Switch to STL containers ASAP
// FIXIT-M deprecated, we are going to delete this

typedef void* NODE_DATA;

typedef struct sf_lnode
{
    struct sf_lnode* next;
    struct sf_lnode* prev;
    NODE_DATA ndata;
}
SF_LNODE;

struct sf_list
{
    SF_LNODE* head, * tail;
    unsigned count;
};

typedef sf_list SF_LIST;

// -----------------------------------------------------------------------------
// Linked List Interface
// -----------------------------------------------------------------------------

namespace snort
{
SO_PUBLIC SF_LIST* sflist_new();
SO_PUBLIC void sflist_init(SF_LIST*);
SO_PUBLIC void sflist_add_tail(SF_LIST*, NODE_DATA);
SO_PUBLIC void sflist_add_head(SF_LIST*, NODE_DATA);
SO_PUBLIC void sflist_add_before(SF_LIST*, SF_LNODE*, NODE_DATA);
SO_PUBLIC NODE_DATA sflist_remove_head(SF_LIST*);
SO_PUBLIC NODE_DATA sflist_remove_tail(SF_LIST*);
SO_PUBLIC int sflist_count(SF_LIST*);
SO_PUBLIC NODE_DATA sflist_first(const SF_LIST*, SF_LNODE**);
SO_PUBLIC NODE_DATA sflist_next(SF_LNODE**);
SO_PUBLIC void sflist_free(SF_LIST*);
SO_PUBLIC void sflist_free_all(SF_LIST*, void (* free)(void*) );
SO_PUBLIC void sflist_static_free_all(SF_LIST*, void (* nfree)(void*));
}

#endif

snort_bounds.h

Path = src/utils/snort_bounds.h

#ifndef SNORT_BOUNDS_H
#define SNORT_BOUNDS_H

// bounds checking for pointers to buffers
inline bool inBounds(const uint8_t* start, const uint8_t* end, const void* p)
{
    return (start <= p) and (p < end);
}

#endif

stats.h

Path = src/utils/stats.h

#ifndef STATS_H
#define STATS_H

// Provides facilities for displaying Snort exit stats

#include <daq_common.h>
#include <vector>

#include "framework/counts.h"
#include "main/snort_types.h"
#include "main/thread.h"

using IndexVec = std::vector<unsigned>;

class ControlConn;

// FIXIT-L split this out into appropriate modules
struct PacketCount
{
    PegCount analyzed_pkts;
    PegCount hard_evals;
    PegCount raw_searches;
    PegCount cooked_searches;
    PegCount pkt_searches;
    PegCount alt_searches;
    PegCount pdu_searches;
    PegCount file_searches;
    PegCount offloads;
    PegCount alert_pkts;
    PegCount total_alert_pkts;
    PegCount log_pkts;
    PegCount pass_pkts;
    PegCount match_limit;
    PegCount queue_limit;
    PegCount log_limit;
    PegCount event_limit;
    PegCount alert_limit;
    PegCount context_stalls;
    PegCount offload_busy;
    PegCount onload_waits;
    PegCount offload_fallback;
    PegCount offload_failures;
    PegCount offload_suspends;
    PegCount pcre_match_limit;
    PegCount pcre_recursion_limit;
    PegCount pcre_error;
    PegCount cont_creations;
    PegCount cont_recalls;
    PegCount cont_flows;
    PegCount cont_evals;
    PegCount cont_matches;
    PegCount cont_mismatches;
    PegCount cont_max_num;
    PegCount cont_match_distance;
    PegCount cont_mismatch_distance;
    PegCount buf_dumps;
};

struct ProcessCount
{
    PegCount local_commands;
    PegCount remote_commands;
    PegCount signals;
    PegCount conf_reloads;
    PegCount policy_reloads;
    PegCount inspector_deletions;
    PegCount daq_reloads;
    PegCount attribute_table_reloads;
    PegCount attribute_table_hosts;     // FIXIT-D - remove when host attribute pegs updated
    PegCount attribute_table_overflow;  // FIXIT-D - remove when host attribute pegs updated
};

extern ProcessCount proc_stats;

extern const PegInfo daq_names[];
extern const PegInfo pc_names[];
extern const PegInfo proc_names[];

namespace snort
{
extern SO_PUBLIC THREAD_LOCAL PacketCount pc;

SO_PUBLIC inline PegCount get_packet_number() { return pc.analyzed_pkts; }

SO_PUBLIC void LogLabel(const char*, FILE* = stdout);
SO_PUBLIC void LogText(const char*, FILE* = stdout);
SO_PUBLIC void LogValue(const char*, const char*, FILE* = stdout);
SO_PUBLIC void LogCount(const char*, uint64_t, FILE* = stdout);

SO_PUBLIC void LogStat(const char*, uint64_t n, uint64_t tot, FILE* = stdout);
SO_PUBLIC void LogStat(const char*, double, FILE* = stdout);
}

void sum_stats(PegCount* sums, PegCount* counts, unsigned n, bool dump_stats = false);
void show_stats(PegCount*, const PegInfo*, const char* module_name = nullptr);
void show_stats(PegCount*, const PegInfo*, unsigned n, const char* module_name = nullptr);
void show_stats(PegCount*, const PegInfo*, const IndexVec&, const char* module_name, FILE*);
void show_percent_stats(PegCount*, const char*[], unsigned n, const char* module_name = nullptr);

void sum_stats(SimpleStats* sums, SimpleStats* counts);
void show_stats(SimpleStats*, const char* module_name);

double CalcPct(uint64_t, uint64_t);
void DropStats(ControlConn* ctrlcon = nullptr);
void PrintStatistics();
void TimeStart();
void TimeStop();
const struct timeval& get_time_curr();
const struct timeval& get_time_start();
const struct timeval& get_time_end();

#endif

streambuf.h

Path = src/utils/streambuf.h

#ifndef STREAMBUF
#define STREAMBUF

#include <streambuf>
#include <tuple>
#include <utility>
#include <vector>

#include "main/snort_types.h"

namespace snort
{

// an input stream over set of buffers,
// the buffer doesn't take ownership over the memory,
// no intermediate buffering between chunks
class SO_PUBLIC istreambuf_glue : public std::streambuf
{
public:
    istreambuf_glue();
    virtual ~istreambuf_glue() override = default;

    std::streamsize last_chunk_offset() const;

protected:
    // a valid s/n pair continues the chain, nullptr or zero size starts new one
    virtual std::streambuf* setbuf(char* s, std::streamsize n) override;
    virtual std::streampos seekoff(std::streamoff off, std::ios_base::seekdir way,
        std::ios_base::openmode which = std::ios_base::in | std::ios_base::out) override;
    virtual std::streampos seekpos(std::streampos sp,
        std::ios_base::openmode which = std::ios_base::in | std::ios_base::out) override;
    virtual int sync() override;
    virtual std::streamsize showmanyc() override;
    virtual std::streamsize xsgetn(char* s, std::streamsize n) override;
    virtual int underflow() override;

    bool last_chunk() const
    { return cur_idx + 1 >= chunks.size(); }

    bool last_chunk(size_t idx) const
    { return idx + 1 >= chunks.size(); }

    typedef std::tuple<char*, std::streamsize, std::streamoff> Chunk;
    typedef std::vector<Chunk> Chunks;

    Chunks chunks;
    size_t cur_idx;
    std::streamsize size;
};

// an output stream over extensible array
class SO_PUBLIC ostreambuf_infl : public std::streambuf
{
public:
    static constexpr size_t size_limit = 1 << 20;

    ostreambuf_infl();
    virtual ~ostreambuf_infl() override;

    // reserve more memory for the current buffer, keeping data
    void reserve(std::streamsize n);

    // releases the current buffer,
    // the caller takes ownership over the buffer
    const char* take_data();
    const char* take_data(std::streamsize& n);

    const char* data() const
    { return pbase(); }

    std::streamsize data_len() const
    { return pptr() - pbase(); }

protected:
    virtual std::streambuf* setbuf(char* s, std::streamsize n) override;
    virtual std::streampos seekoff(std::streamoff off, std::ios_base::seekdir way,
        std::ios_base::openmode which = std::ios_base::in | std::ios_base::out) override;
    virtual std::streampos seekpos(std::streampos sp,
        std::ios_base::openmode which = std::ios_base::in | std::ios_base::out) override;
    virtual int sync() override;
    virtual std::streamsize xsputn(const char* s, std::streamsize n) override;
    virtual std::streamsize xsgetn(char* s, std::streamsize n) override;
    virtual int overflow(int c = EOF) override;

    bool enlarge();
    bool enlarge(std::streamsize extra_len);

    struct State
    {
        const State* s;
        std::streamsize n;

        std::streamsize get_next_size()
        { auto r = n; n = s->n; s = s->s; return r; }
    };

    static const State states[];
    State gen;
};

}

#endif

util_ber.h

Path = src/utils/util_ber.h

#ifndef UTIL_BER_H
#define UTIL_BER_H

#include "main/snort_types.h"
#include "framework/cursor.h"

namespace snort
{

enum BerType
{
    BOOLEAN = 1,
    INTEGER,
    BIT_STRING,
    STRING,
};

struct BerElement
{
    uint32_t type;
    uint32_t length;
    uint64_t header_length;
    uint64_t total_length;
    const uint8_t* data;
};

class SO_PUBLIC BerReader
{
public:
    BerReader(const Cursor& c)
    {
        beg = c.buffer();
        cursor = c.start();
        end = c.endo();
    }

    bool read(const uint8_t* c, BerElement& e);

    bool convert(const BerElement& e, uint32_t& intval);
    bool extract(const uint8_t*& c, uint32_t& intval);

    bool skip(const uint8_t*& c, uint32_t type);
    bool data(const uint8_t*& c, uint32_t type);

private:
    bool read_int(uint32_t size, uint32_t& intval);

    bool read_type(uint32_t& type);
    bool read_length(uint32_t& length);

    const uint8_t* beg;
    const uint8_t* cursor;
    const uint8_t* end;
};

}
#endif

util_cstring.h

Path = src/utils/util_cstring.h

#ifndef UTIL_CSTRING_H
#define UTIL_CSTRING_H

// Utility functions and macros for interacting with and parsing C strings

#include <cctype>
#include <cerrno>
#include <cstdlib>

#include "main/snort_types.h"

namespace snort
{
#define SNORT_SNPRINTF_SUCCESS 0
#define SNORT_SNPRINTF_TRUNCATION 1
#define SNORT_SNPRINTF_ERROR (-1)

#define SNORT_STRNCPY_SUCCESS 0
#define SNORT_STRNCPY_TRUNCATION 1
#define SNORT_STRNCPY_ERROR (-1)
#define SNORT_STRNLEN_ERROR (-1)

SO_PUBLIC int safe_snprintf(char*, size_t, const char*, ... )
    __attribute__((format (printf, 3, 4)));
// these functions are deprecated; use C++ strings instead
SO_PUBLIC int SnortSnprintf(char*, size_t, const char*, ...)
    __attribute__((format (printf, 3, 4)));
SO_PUBLIC int SnortSnprintfAppend(char*, size_t, const char*, ...)
    __attribute__((format (printf, 3, 4)));

SO_PUBLIC const char* SnortStrcasestr(const char* s, int slen, const char* substr);
SO_PUBLIC const char* SnortStrnStr(const char* s, int slen, const char* searchstr);
SO_PUBLIC const char* SnortStrnPbrk(const char* s, int slen, const char* accept);

SO_PUBLIC int SnortStrncpy(char*, const char*, size_t);
SO_PUBLIC int SnortStrnlen(const char*, int);

SO_PUBLIC int sfsnprintfappend(char* dest, int dsize, const char* format, ...)
    __attribute__((format (printf, 3, 4)));

inline long SnortStrtol(const char* nptr, char** endptr, int base)
{
    long iRet;
    errno = 0;
    iRet = strtol(nptr, endptr, base);

    return iRet;
}

inline unsigned long SnortStrtoul(const char* nptr, char** endptr, int base)
{
    unsigned long iRet;
    errno = 0;
    iRet = strtoul(nptr, endptr, base);

    return iRet;
}

// Checks to make sure we're not going to evaluate a negative number for which
// strtoul() gladly accepts and parses returning an underflowed wrapped unsigned
// long without error.
// Buffer passed in MUST be '\0' terminated.
//
// Returns
//  int
//    -1 if buffer is nothing but spaces or first non-space character is a
//       negative sign.  Also if errno is EINVAL (which may be due to a bad
//       base) or there was nothing to convert.
//     0 on success
//
// Populates pointer to uint32_t value passed in which should
// only be used on a successful return from this function.
//
// Also will set errno to ERANGE on a value returned from strtoul that is
// greater than UINT32_MAX, but still return success.
//
inline int SnortStrToU32(const char* buffer, char** endptr,
    uint32_t* value, int base)
{
    unsigned long int tmp;

    if ((buffer == nullptr) || (endptr == nullptr) || (value == nullptr))
        return -1;

    // Only positive numbers should be processed and strtoul will
    // eat up white space and process '-' and '+' so move past
    // white space and check for a negative sign.
    while (isspace((int)*buffer))
        buffer++;

    // If all spaces or a negative sign is found, return error.
    // May want to exclude '+' as well.
    if ((*buffer == '\0') || (*buffer == '-'))
        return -1;

    tmp = SnortStrtoul(buffer, endptr, base);

    // The user of the function should check for ERANGE in errno since this
    // function can be used such that an ERANGE error is acceptable and
    // value gets truncated to UINT32_MAX.
    if ((errno == EINVAL) || (*endptr == buffer))
        return -1;

    // If value is greater than a UINT32_MAX set value to UINT32_MAX
    // and errno to ERANGE
    if (tmp > UINT32_MAX)
    {
        tmp = UINT32_MAX;
        errno = ERANGE;
    }

    *value = (uint32_t)tmp;

    return 0;
}
}
#endif

util.h

Path = src/utils/util.h

#ifndef UTIL_H
#define UTIL_H

// Miscellaneous functions and macros
// FIXIT-L this needs to be refactored and stripped of cruft

#if defined(__linux__)
#include <sys/syscall.h>
#endif
#include <sys/time.h>
#include <unistd.h>

#include <cerrno>
#include <cstdlib>
#include <cstring>
#include <string>
#include <vector>

#include "main/snort_types.h"

// Note: thread names > 15 chars aren't supported on most systems
#if defined(__linux__)
    #include <pthread.h>
    #define SET_THREAD_NAME(thread, name) \
    pthread_setname_np(thread, name)

#elif defined(__OpenBSD__) || defined(__FreeBSD__)
    #include <pthread_np.h>
    #define SET_THREAD_NAME(thread, name) \
    pthread_set_name_np(thread, name)

#else
    #define SET_THREAD_NAME(thread, name) \
    UNUSED(thread);\
    UNUSED(name);

#endif

#define TIMEBUF_SIZE 27

#define SECONDS_PER_DAY  86400  /* number of seconds in a day  */
#define SECONDS_PER_HOUR  3600  /* number of seconds in a hour */
#define SECONDS_PER_MIN     60     /* number of seconds in a minute */

void StoreSnortInfoStrings();
int DisplayBanner();
int gmt2local(time_t);
std::string read_infile(const char* key, const char* fname);
void CleanupProtoNames();
void CreatePidFile(pid_t);
void ClosePidFile();
bool SetUidGid(int, int);
void InitGroups(int, int);
bool EnterChroot(std::string& root_dir, std::string& log_dir);
void InitProtoNames();
unsigned int get_random_seed();
bool get_file_size(const std::string&, size_t&);

#if defined(NOCOREFILE)
void SetNoCores();
#endif

namespace
{
inline void COPY4(uint32_t* dst, const uint32_t* src)
{
    dst[0] = src[0];
    dst[1] = src[1];
    dst[2] = src[2];
    dst[3] = src[3];
}
}

inline void* snort_alloc(size_t sz)
{ return new uint8_t[sz]; }

inline void* snort_alloc(size_t num, size_t sz)
{ return snort_alloc(num * sz); }

inline void* snort_calloc(size_t num, size_t sz)
{
    sz *= num;
    auto p = snort_alloc(sz);
    memset(p, 0, sz);
    return p;
}

inline void* snort_calloc(size_t sz)
{ return snort_calloc(1, sz); }

inline void snort_free(void* p)
{ delete[] (uint8_t*)p; }

inline pid_t gettid()
{
#if defined(__linux__) && defined(SYS_gettid)
    return syscall(SYS_gettid); // cppcheck-suppress ConfigurationNotChecked
#else
    return getpid();
#endif
}

namespace snort
{
// FIXIT-M provide getter function to for standardized access into the protocol_names array
SO_PUBLIC extern char** protocol_names;

SO_PUBLIC const char* get_error(int errnum);
SO_PUBLIC char* snort_strdup(const char*);
SO_PUBLIC char* snort_strndup(const char*, size_t);
SO_PUBLIC void ts_print(const struct timeval*, char*, bool yyyymmdd = false);
void uint8_to_printable_str(const uint8_t* buff, unsigned len, std::string& print_str);
SO_PUBLIC void str_to_int_vector(const std::string& s, char delim, std::vector<uint32_t>& elems);
SO_PUBLIC std::string int_vector_to_str(const std::vector<uint32_t>& elems, char delim = ',');
}

#endif

util_jsnorm.h

Path = src/utils/util_jsnorm.h

#ifndef UTIL_JSNORM_H
#define UTIL_JSNORM_H

// Javascript Normalization

#include "main/snort_types.h"

namespace snort
{
#define ALERT_SPACES_EXCEEDED   0x1
#define ALERT_LEVELS_EXCEEDED   0x2
#define ALERT_MIXED_ENCODINGS   0x4

#define MAX_ALLOWED_OBFUSCATION 1

struct JSState
{
    int allowed_spaces;
    int allowed_levels;
    uint16_t alerts;
};

int JSNormalizeDecode(
    const char*, uint16_t, char*, uint16_t destlen, const char**, int*, JSState*, uint8_t*);
}
#endif

util_net.h

Path = src/utils/util_net.h

#ifndef UTIL_NET_H
#define UTIL_NET_H

#include <arpa/inet.h>
#include "main/snort_types.h"

namespace snort
{
struct SfCidr;
typedef char InetBuf[INET6_ADDRSTRLEN];

SO_PUBLIC char* ObfuscateIpToText(
    const struct SfIp*, const SfCidr& homenet, const SfCidr& obfuscate_net, InetBuf&);
}
#endif

util_numa.h

Path = src/utils/util_numa.h

#ifndef NUMA_UTILS_H
#define NUMA_UTILS_H

#ifdef HAVE_NUMA

#include <numa.h>
#include <numaif.h>
#include <sched.h>
#include <hwloc.h>

class NumaWrapper
{
public:
    virtual ~NumaWrapper() {}
    virtual int available()
    {
        return numa_available();
    }
    virtual int max_node()
    {
        return numa_max_node();
    }
    virtual int preferred()
    {
        return numa_preferred();
    }
    virtual int set_mem_policy(int mode, const unsigned long *nodemask,
                              unsigned long maxnode)
    {
        return set_mempolicy(mode, nodemask, maxnode);
    }
};
class HwlocWrapper
{
public:
    virtual ~HwlocWrapper() {}
    virtual unsigned get_nbobjs_by_depth(hwloc_topology_t topology, int depth)
    {
        return hwloc_get_nbobjs_by_depth(topology, depth);
    }
    virtual hwloc_obj_t get_obj_by_depth(hwloc_topology_t topology, int depth, unsigned idx)
    {
        return hwloc_get_obj_by_depth(topology, depth, idx);
    }
    virtual int get_type_depth(hwloc_topology_t topology, hwloc_obj_type_t type)
    {
        return hwloc_get_type_depth(topology, type);
    }
    virtual int bitmap_intersects(hwloc_const_cpuset_t set1, hwloc_const_cpuset_t set2)
    {
        return hwloc_bitmap_intersects(set1, set2);
    }
};
#endif // HAVE_NUMA
#endif // NUMA_UTILS_H

util_unfold.h

Path = src/utils/util_unfold.h

#ifndef UTIL_UNFOLD_H
#define UTIL_UNFOLD_H

// Utilities to deal with line endings and other whitespace. AKA "Header unfolding"

#include "main/snort_types.h"

namespace snort
{
SO_PUBLIC int sf_unfold_header(const uint8_t*, uint32_t, uint8_t*, uint32_t, uint32_t*, int, int*);
SO_PUBLIC int sf_strip_CRLF(const uint8_t*, uint32_t, uint8_t*, uint32_t, uint32_t*);
SO_PUBLIC int sf_strip_LWS(const uint8_t*, uint32_t, uint8_t*, uint32_t, uint32_t*);
}
#endif

util_utf.h

Path = src/utils/util_utf.h

#ifndef UTIL_UTF_H
#define UTIL_UTF_H

// Some UTF-{16,32}{le,be} normalization functions

#include "main/snort_types.h"

// Character set types. Used by HTTP inspectors. Update inspectors while changing this value.
enum CharsetCode
{
    CHARSET_DEFAULT=0,
    CHARSET_OTHER,
    CHARSET_UTF7,
    CHARSET_IRRELEVANT,
    CHARSET_UTF16LE,
    CHARSET_UTF16BE,
    CHARSET_UTF32LE,
    CHARSET_UTF32BE,
    CHARSET_UNKNOWN
};

// How character set was set.
enum CharsetSrc
{
    CHARSET_SET_BY_APP=0,
    CHARSET_SET_BY_BOM,
    CHARSET_SET_BY_GUESS
};

// Since payloads don't have to end on 2/4-byte boundaries, callers to
// DecodeUTF are responsible for keeping a decode_utf_state_t. This carries
// state between subsequent calls.
struct decode_utf_state_t
{
    int state = 0;
    CharsetCode charset = CHARSET_DEFAULT;
    CharsetSrc charset_src = CHARSET_SET_BY_GUESS;
};

namespace snort
{
class SO_PUBLIC UtfDecodeSession
{
public:
    UtfDecodeSession() = default;
    virtual ~UtfDecodeSession() = default;
    void set_decode_utf_state_charset(CharsetCode charset, CharsetSrc src = CHARSET_SET_BY_APP);
    CharsetCode get_decode_utf_state_charset();
    CharsetSrc get_decode_utf_charset_src();
    bool is_utf_encoding_present();
    bool decode_utf(const uint8_t* src, unsigned int src_len, uint8_t* dst, unsigned int dst_len,
        int* bytes_copied);
    static char* convert_character_encoding(const char* to_code, const char* from_code,
        char* in_buf, char* out_buf, size_t in_bytes, size_t out_bytes, size_t* out_buf_length);

private:
    decode_utf_state_t dstate;
    bool DecodeUTF16LE(const uint8_t* src, unsigned int src_len, uint8_t* dst, unsigned int dst_len, int* bytes_copied);
    bool DecodeUTF16BE(const uint8_t* src, unsigned int src_len, uint8_t* dst, unsigned int dst_len, int* bytes_copied);
    bool DecodeUTF32LE(const uint8_t* src, unsigned int src_len, uint8_t* dst, unsigned int dst_len, int* bytes_copied);
    bool DecodeUTF32BE(const uint8_t* src, unsigned int src_len, uint8_t* dst, unsigned int dst_len, int* bytes_copied);
    void determine_charset(const uint8_t** src, unsigned int* src_len);
};
}

#endif

utils/test/