nf9packet

package module
v0.0.0-...-086bda8 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 17, 2016 License: Unlicense Imports: 8 Imported by: 4

README

nf9packet

Godoc

This is golang library for NetFlow v9 packet decoding. It can be used to create NetFlow v9 packet inspection and analysis tools, NetFlow collectors or higher level libraries.

This package does only packet decoding in a single packet context. It keeps no state when decoding multiple packets. As a result Data FlowSets can not be decoded during initial packet decoding. To decode Data FlowSets user must keep track of all seen Template Records and Options Template Records and then decode Data FlowSets manually.

Most of structure names and comments are taken directly from RFC 3954. Reading the NetFlow v9 protocol specification is highly recommended before using this package.

Examples

There are three demo applications created as library usage examples:

  • nf9-packet-dump - Dumps contents of NetFlow v9 packets in plaintext or JSON. Minimal library usage example.
  • nf9-template-dump - Tool for inspecting Data Templates and Options Data Templates in NetFlow v9 streams. This tool also displays field names and descriptions. Moderate library usage example.
  • nf9-data-dump - Tool for extracting Data Flow information from NetFlow v9 streams. Extended library usage example.

Documentation

Overview

Package nf9packet provides structures and functions to decode and analyze NetFlow v9 packets.

This package does only packet decoding in a single packet context. It keeps no state when decoding multiple packets. As a result Data FlowSets can not be decoded during initial packet decoding. To decode Data FlowSets user must keep track of Template Records and Options Template Records manually.

Examples of NetFlow v9 packets: 

+--------+--------------------------------------------------------+
|        | +----------+ +---------+     +-----------+ +---------+ |
| Packet | | Template | | Data    |     | Options   | | Data    | |
| Header | | FlowSet  | | FlowSet | ... | Template  | | FlowSet | |
|        | |          | |         |     | FlowSet   | |         | |
|        | +----------+ +---------+     +-----------+ +---------+ |
+--------+--------------------------------------------------------+

+--------+----------------------------------------------+
|        | +---------+     +---------+      +---------+ |
| Packet | | Data    | ... | Data    | ...  | Data    | |
| Header | | FlowSet | ... | FlowSet | ...  | FlowSet | |
|        | +---------+     +---------+      +---------+ |
+--------+----------------------------------------------+

+--------+-------------------------------------------------+
|        | +----------+     +----------+      +----------+ |
| Packet | | Template |     | Template |      | Options  | |
| Header | | FlowSet  | ... | FlowSet  | ...  | Template | |
|        | |          |     |          |      | FlowSet  | |
|        | +----------+     +----------+      +----------+ |
+--------+-------------------------------------------------+

Example of struct hierarchy after packet decoding: 

Package
|
+--TemplateFlowSet
|  |
|  +--TemplateRecord
|  |  |
|  |  +--Field
|  |  +--...
|  |  +--Field
|  |
|  +--...
|  |
|  +--TemplateRecord
|     |
|     +--Field
|     +--...
|     +--Field
|
+--DataFlowSet
|
+--...
|
+--OptionsTemplateFlowSet
|  |
|  +--OptionsTemplateRecord
|  |  |
|  |  +--Field (scope)
|  |  +--...   (scope)
|  |  +--Field (scope)
|  |  |
|  |  +--Field (option)
|  |  +--...   (option)
|  |  +--Field (option)
|  |
|  +--...
|  |
|  +--OptionsTemplateRecord
|     |
|     +--Field (scope)
|     +--...   (scope)
|     +--Field (scope)
|     |
|     +--Field (option)
|     +--...   (option)
|     +--Field (option)
|
+--DataFlowSet

When matched with appropriate template Data FlowSet can be decoded to list of Flow Data Records or list of Options Data Records. Struct hierarchy example: 

[]FlowDataRecord
  |
  +--FlowDataRecord
  |  |
  |  +--[]byte
  |  +--...
  |  +--[]byte
  |
  +--...
  |
  +--FlowDataRecord
     |
     +--[]byte
     +--...
     +--[]byte

[]OptionsDataRecord
  |
  +--OptionsDataRecord
  |  |
  |  +--[]byte (scope)
  |  +--...    (scope)
  |  +--[]byte (scope)
  |  |
  |  +--[]byte (option)
  |  +--...    (option)
  |  +--[]byte (option)
  |
  +--...
  |
  +--OptionsDataRecord
     |
     +--[]byte
     +--...
     +--[]byte
     |
     +--[]byte (option)
     +--...    (option)
     +--[]byte (option)

Most of structure names and comments are taken directly from RFC 3954. Reading the NetFlow v9 protocol specification is highly recommended before using this package.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Dump 

func Dump(packet *Packet, w io.Writer)

Dump prints decoded packet structure, listing all structure fields and values in a simple plain text format format. It is used for debugging and simple packet inspection. Argument w can be os.Stdout or any other Writer implementation.

Types

type DataFlowSet 

type DataFlowSet struct {
	FlowSetHeader

	// Raw data bytes
	Data []byte
}

DataFlowSet is a collection of Data Records (actual NetFlow data) and Options Data Rcords (meta data).

type Field 

type Field struct {
	// A numeric value that represents the type of field.
	Type uint16

	// The length (in bytes) of the field.
	Length uint16
}

Field describes type and length of a single value in a Flow Data Record. Field does not contain the record value itself it is just a description of what record value will look like.

func (*Field) DataToString 

func (f *Field) DataToString(data []byte) string

DataToString converts field value to string representation based on field type. If used with unknow field type string "n/a" will be returned.

func (*Field) DataToUint64 

func (f *Field) DataToUint64(data []byte) uint64

DataToUint64 converts field value to uint64. This function will not generate errors if used with incompatible field types. Field value length can be up to 8 bytes, for longer values only first 8 bytes are used.

func (*Field) DefaultLength 

func (f *Field) DefaultLength() int

DefaultLength returns length of field type as specified in RFC 3954 and Cisco documentation. For variable length fields and unknown fields -1 is returned.

func (*Field) Description 

func (f *Field) Description() string

Description returns field type description based on RFC 3954 and Cisco documentation. For unkown field types string "Unknown type" will be returned.

func (*Field) Name 

func (f *Field) Name() string

Name returns a short field type identifier based on RFC 3954 and Cisco documentation. For unkown field types string "UNKNOWN_TYPE" will be returned.

func (*Field) ScopeDefaultLength 

func (f *Field) ScopeDefaultLength() int

ScopeDefaultLength is the same as DefaultLength() but should be used only for Scope Fields

func (*Field) ScopeDescription 

func (f *Field) ScopeDescription() string

ScopeDescription is the same as Description but should be used only for Scope Fields

func (*Field) ScopeName 

func (f *Field) ScopeName() string

ScopeName is the same as Name() but should be used only for Scope Fields

func (Field) String 

func (f Field) String() string

String is a convenience method for printing field name as a default field value string representation.

type FlowDataRecord 

type FlowDataRecord struct {
	// List of Flow Data Record values stored in raw format as []byte
	Values [][]byte
}

FlowDataRecord is actual NetFlow data. This structure does not contain any information about the actual data meanind. It must be combined with corresponding TemplateRecord to be decoded to a single NetFlow data row.

type FlowSetHeader 

type FlowSetHeader struct {
	// FlowSet ID:
	//    0 for TemplateFlowSet
	//    1 for OptionsTemplateFlowSet
	//    256-65535 for DataFlowSet (used as TemplateId)
	Id uint16

	// The total length of this FlowSet in bytes (including padding).
	Length uint16
}

FlowSetHeader contains fields shared by all Flow Sets (DataFlowSet, TemplateFlowSet, OptionsTemplateFlowSet).

type OptionsDataRecord 

type OptionsDataRecord struct {
	// List of Scope values stored in raw format as []byte
	ScopeValues [][]byte

	// List of Optons values stored in raw format as []byte
	OptionValues [][]byte
}

OptionsDataRecord is meta data sent alongide actual NetFlow data. Combined with OptionsTemplateRecord it can be decoded to a single data row.

type OptionsTemplateFlowSet 

type OptionsTemplateFlowSet struct {
	FlowSetHeader

	// List of Options Template Records
	Records []OptionsTemplateRecord
}

OptionsTemplateFlowSet is a collection of templates that describe structure of Options Data Records.

type OptionsTemplateRecord 

type OptionsTemplateRecord struct {
	// Template ID of this Options Template. This value is greater than 255.
	TemplateId uint16

	// The length in bytes of all Scope field definitions contained in this
	// Options Template Record.
	ScopeLength uint16

	// The length (in bytes) of all options field definitions contained in
	// this Options Template Record.
	OptionLength uint16

	// List of Scope fields in this Options Template Record.
	Scopes []Field

	// List of Option fields in this Options Template Record.
	Options []Field
}

OptionsTemplateRecord is a template that describes structure of an Options Data Record (meta data).

func (*OptionsTemplateRecord) DecodeFlowSet 

func (otpl *OptionsTemplateRecord) DecodeFlowSet(set *DataFlowSet) (list []OptionsDataRecord)

DecodeFlowSet uses current OptionsTemplateRecord to decode data in Data FlowSet to a list of Options Data Records.

type Packet 

type Packet struct {
	// Version of Flow Record format exported in this packet. The value of
	//this field is 9 for the current version.
	Version uint16

	// The total number of records in the Export Packet, which is the sum
	// of Options FlowSet records, Template FlowSet records, and Data
	// FlowSet records.
	Count uint16

	// Time in milliseconds since this device was first booted.
	SysUpTime uint32

	// Time in seconds since 0000 UTC 1970, at which the Export Packet
	// leaves the Exporter.
	UnixSecs uint32

	// Incremental sequence counter of all Export Packets sent from the
	// current Observation Domain by the Exporter.
	SequenceNumber uint32

	// A 32-bit value that identifies the Exporter Observation Domain.
	SourceId uint32

	// A slice of structs. Each element is instance of DataFlowSet or
	// TemplateFlowSet or OptionsTemplateFlowSet.
	FlowSets []interface{}
}

Packet is a decoded representation of a single NetFlow v9 UDP packet.

func Decode 

func Decode(data []byte) (*Packet, error)

Decode is the main function of this package. It converts raw packet bytes to Packet struct.

func (*Packet) DataFlowSets 

func (p *Packet) DataFlowSets() (list []DataFlowSet)

DataFlowSets generate a list of all Data FlowSets in the packet. If matched with appropriate templates Data FlowSets can be decoded to Data Records or Options Data Records.

func (*Packet) OptionsTemplateRecords 

func (p *Packet) OptionsTemplateRecords() (list []*OptionsTemplateRecord)

OptionsTemplateRecords generate a list of all Options Template Records in the packet. Options Template Records can be used to decode Data FlowSets to Options Data Records.

func (*Packet) TemplateRecords 

func (p *Packet) TemplateRecords() (list []*TemplateRecord)

TemplateRecords generate a list of all Template Records in the packet. Template Records can be used to decode Data FlowSets to Data Records.

type TemplateFlowSet 

type TemplateFlowSet struct {
	FlowSetHeader

	// List of Template Records
	Records []TemplateRecord
}

TemplateFlowSet is a collection of templates that describe structure of Data Records (actual NetFlow data).

type TemplateRecord 

type TemplateRecord struct {
	// Each of the newly generated Template Records is given a unique
	// Template ID. This uniqueness is local to the Observation Domain that
	// generated the Template ID. Template IDs of Data FlowSets are numbered
	// from 256 to 65535.
	TemplateId uint16

	// Number of fields in this Template Record. Because a Template FlowSet
	// usually contains multiple Template Records, this field allows the
	// Collector to determine the end of the current Template Record and
	// the start of the next.
	FieldCount uint16

	// List of fields in this Template Record.
	Fields []Field
}

TemplateRecord is a single template that describes structure of a Flow Record (actual Netflow data).

func (*TemplateRecord) DecodeFlowSet 

func (dtpl *TemplateRecord) DecodeFlowSet(set *DataFlowSet) (list []FlowDataRecord)

DecodeFlowSet uses current TemplateRecord to decode data in Data FlowSet to a list of Flow Data Records.

Source Files

View all Source files

Directories

Path Synopsis
examples
nf9-data-dump command
nf9-packet-dump command
nf9-template-dump command

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.