nmeta2dpae package

Submodules

nmeta2dpae.config module

This module is part of the nmeta2 suite . It represents the configuration data for a Data Plane Auxiliary Engine (DPAE). . It expects a file called “config.yaml” to be in a subdirectory called config, and this file to contain properly formed YAML

class nmeta2dpae.config.Config

Bases: object

This class is instantiated by nmeta_dpae.py and provides methods to ingest the configuration file and provides access to the keys/values that it contains. Config file is in YAML in config subdirectory and is called ‘config.yaml’

get_value(config_key)

Passed a key and see if it exists in the config YAML. If it does then return the value, if not return 0

nmeta2dpae.controlchannel module

This module is part of the nmeta2 suite . It is provides control channel services between the nmeta Data Plane Auxiliary Engine (DPAE) and the OpenFlow controller using REST API calls

class nmeta2dpae.controlchannel.ControlChannel(_nmeta2dpae, _config, if_name, dp)

Bases: object

This class is instantiated by nmeta_dpae.py and provides methods to interact with the nmeta control plane

get_policy(location)

Get the a policy from the Controller (YAML in string format)

keepalive(event_flag, location, if_name)

Do regular keepalive polls to the DPAE to check if is still available, in dedicated process. If keepalive fails, then set an event flag for parent process.

phase1(api_base, if_name)

Phase 1 (global to DPAE) connection to the control plane, as an active data plane auxiliary device

phase2(api_base, if_name, dpae2ctrl_mac, ctrl2dpae_mac, dpae_ethertype)

Phase 2 (per DPAE sniffing interface) switch/port discovery

phase3(api_base, if_name, dpae2ctrl_mac, ctrl2dpae_mac, dpae_ethertype)

Phase 3 (per DPAE sniffing interface) confirmation of sniffing packets

phase4(api_base, if_name)

Phase 4 (per DPAE sniffing interface) Negotiate what services will be run by the DPAE

tc_advise_controller(location, tc_result)

Pass Traffic Classification (TC) information to the controller via the API

tc_start(location)

Tell the Controller to start sending us packets that need traffic classification

class nmeta2dpae.controlchannel.JSON_Body(req_body)

Bases: object

Represents a JSON-encoded body of an HTTP request. Doesn’t do logging, but does set .error when things don’t go to plan with a friendly message.

decode(req_body)

Passed an allegedly JSON body and see if it decodes. Set error variable for exceptions

validate(key_list)

Passed a list of keys and check that they exist in the JSON. If they don’t return 0 and set error to description of first missing key that was found

nmeta2dpae.dp module

This module is part of the nmeta2 suite . It provides an object for data plane coordination services . Version 2.x Toulouse Code

class nmeta2dpae.dp.DP(_config)

Bases: object

This class is instantiated by nmeta2_dpae.py and provides methods to run the data plane services.

dp_discover(queue, if_name, dpae2ctrl_mac, ctrl2dpae_mac, dpae_ethertype, timeout, uuid_dpae, uuid_controller)

Data plane service for DPAE Join Discover Packet Sniffing

dp_run(interplane_queue, tc_policy, if_name)

Run Data Plane (DP) Traffic Classification for an interface

class nmeta2dpae.dp.JSON_Body(req_body)

Bases: object

Represents a JSON-encoded body of an HTTP request. Doesn’t do logging, but does set .error when things don’t go to plan with a friendly message.

decode(req_body)

Passed an allegedly JSON body and see if it decodes. Set error variable for exceptions

validate(key_list)

Passed a list of keys and check that they exist in the JSON. If they don’t return 0 and set error to description of first missing key that was found

nmeta2dpae.flow module

This module is part of the nmeta2 suite . It provides an abstraction for a TCP flow that links to a MongoDB database and changes to the context of the flow that a supplied packet belongs to . Version 2.x Toulouse Code

class nmeta2dpae.flow.Flow(logger, mongo_addr, mongo_port)

Bases: object

An object that represents a flow that we are classifying

Intended to provide an abstraction of a flow that classifiers can use to make determinations without having to understand implementations such as database lookups etc.

Be aware that this module is not very mature yet. It does not cover some basic corner cases such as packet retransmissions and out of order or missing packets.

Variables available for Classifiers (assumes class instantiated as an object called ‘flow’):

Variables for the current packet:

flow.ip_src
IP source address of latest packet in flow
flow.ip_dst
IP dest address of latest packet in flow
flow.tcp_src
TCP source port of latest packet in flow
flow.tcp_dst
TCP dest port of latest packet in flow
flow.tcp_seq
TCP sequence number of latest packet in flow
flow.tcp_acq
TCP acknowledgement number of latest packet in flow
flow.tcp_fin()
True if TCP FIN flag is set in the current packet
flow.tcp_syn()
True if TCP SYN flag is set in the current packet
flow.tcp_rst()
True if TCP RST flag is set in the current packet
flow.tcp_psh()
True if TCP PSH flag is set in the current packet
flow.tcp_ack()
True if TCP ACK flag is set in the current packet
flow.tcp_urg()
True if TCP URG flag is set in the current packet
flow.tcp_ece()
True if TCP ECE flag is set in the current packet
flow.tcp_cwr()
True if TCP CWR flag is set in the current packet
flow.payload
Payload of TCP of latest packet in flow
flow.packet_length
Length in bytes of the current packet on wire
flow.packet_direction
c2s (client to server) or s2c directionality based on first observed packet having SYN or SYN+ACK flag, otherwise client assumed as source IP of first packet and verified_direction set to 0 (i.e. don’t trust packet_direction unless verified_direction is set)

Variables for the whole flow:

flow.verified_direction
Describes how the directionality of the flow was ascertained. Values can be verified-SYN, verified-SYNACK or 0 (unverified)
flow.finalised
A classification has been made
flow.suppressed
The flow packet count number when a request was made to controller to not see further packets in this flow. 0 is not suppressed
flow.packet_count
Unique packets registered for the flow
flow.client
The IP that is the originator of the TCP session (if known, otherwise 0)
flow.server
The IP that is the destination of the TCP session session (if known, otherwise 0)

Methods available for Classifiers: (assumes class instantiated as an object called ‘flow’)

flow.max_packet_size()
Size of largest packet in the flow
flow.max_interpacket_interval()
TBD
flow.min_interpacket_interval()
TBD
Challenges:
  • duplicate packets
  • IP fragments (not handled)
  • Flow reuse - TCP source port reused (not handled - yet)
ingest_packet(pkt, pkt_receive_timestamp)

Ingest a packet and put the flow object into the context of the flow that the packet belongs to.

max_interpacket_interval()

Return the size of the largest inter-packet time interval in the flow (assessed per direction in flow). . Note: slightly inaccurate due to floating point rounding.

max_packet_size()

Return the size of the largest packet in the flow (in either direction)

min_interpacket_interval()

Return the size of the smallest inter-packet time interval in the flow (assessed per direction in flow) . Note: slightly inaccurate due to floating point rounding.

set_suppress_flow()

Set the suppressed attribute in the flow database object to the current packet count so that future suppressions of the same flow can be backed off to prevent overwhelming the controller

tcp_ack()

Does the current packet have the TCP ACK flag set?

tcp_cwr()

Does the current packet have the TCP CWR flag set?

tcp_ece()

Does the current packet have the TCP ECE flag set?

tcp_fin()

Does the current packet have the TCP FIN flag set?

tcp_psh()

Does the current packet have the TCP PSH flag set?

tcp_rst()

Does the current packet have the TCP RST flag set?

tcp_syn()

Does the current packet have the TCP SYN flag set?

tcp_urg()

Does the current packet have the TCP URG flag set?

nmeta2dpae.nmeta2dpae module

This module is part of the nmeta2 suite . nmeta Data Plane Auxiliary Engine (DPAE) Used as an auxilary data plane component for functions such as offloading packet-intensive traffic classification from the controller.

class nmeta2dpae.nmeta2dpae.DPAE

Bases: object

This class provides methods for a Data Plane Auxiliary Engine (DPAE), an auxiliary entity that provides services to nmeta.

cp_run(if_name, controlchannel, location)

Run Control Plane (CP) Traffic Classification for an interface

per_interface(if_name)

Run per interface that sniffing will run on as separate process

run()

Run the DPAE instance

nmeta2dpae.sniff module

This module is part of the nmeta2 suite . It provides packet sniffing services

class nmeta2dpae.sniff.Ifreq

Bases: _ctypes.Structure

Class used in setting Ethernet interface promiscuous mode

ifr_flags

Structure/Union member

ifr_ifrn

Structure/Union member

class nmeta2dpae.sniff.Sniff(_config, tc)

Bases: object

This class is instantiated by nmeta_dpae.py and provides methods to sniff and process inbound packets on a given interface

discover_confirm(if_name, dpae2ctrl_mac, ctrl2dpae_mac, dpae_ethertype, timeout)

This function processes sniffs for a discover confirm packet and returns 1 if seen and valid, otherwise 0 after expiry of timeout period

get_promiscuous_mode(if_name)

Return the promiscuous mode of an interface. 1 is promiscuous mode enabled 0 is promiscuous mode disabled

set_promiscuous_mode(if_name)

Set a given Ethernet interface to promiscuous mode so that it can receive packets destined for any MAC address.

sniff_run(if_name, tc, tc_policy, queue)

This function sniffs packets from a NIC. It passes the packets to the tc module for classification and returns any TC results to parent process via a queue.

In active mode it also sends the processed packet back to the switch

nmeta2dpae.sniff.mac_addr(address)

Convert a MAC address to a readable/printable string

nmeta2dpae.tc module

This module is part of the nmeta2 suite . It provides an object for traffic classification and includes ingesting the policy from YAML and checking packets against policy, calling appropriate classifiers and returning actions. . Version 2.x Toulouse Code

class nmeta2dpae.tc.TC(_config)

Bases: object

This class is instantiated by nmeta2_dpae.py and provides methods to ingest the policy as yaml and check packets against policy, calling appropriate classifiers and returning actions.

classify_dpkt(pkt, pkt_receive_timestamp, if_name)

Perform traffic classification on a packet using dpkt for packet parsing

classify_dpkt_wrapper(pkt, pkt_receive_timestamp, if_name)

Used to catch and handle exceptions in classify_dpkt otherwise it can just hang with no explaination... TBD: turn this into a decorator...

instantiate_classifiers(_classifiers)

Dynamically import and instantiate classes for any dynamic classifiers specified in the controller nmeta2 main_policy.yaml . Passed a list of tuples of classifier type / classifer name . Classifier modules live in the ‘classifiers’ subdirectory .

nmeta2dpae.tc.mac_addr(address)

Convert a MAC address to a readable/printable string

nmeta2dpae.tc_policy_dpae module

This module is part of nmeta Data Plane Auxiliary Engine (DPAE) . It is used to contain the Traffic Classification (TC) policy and provide methods and direct variables to access it . Version 2.x Toulouse Code

class nmeta2dpae.tc_policy_dpae.TCPolicy(_config)

Bases: object

This class is instantiated by nmeta2.py and provides methods to ingest the policy file main_policy.yaml and validate that it is correctly structured

get_id_flag(if_name, id_key)

Get a value for an Identity Indicator harvesting flag

get_tc_classifiers(if_name)

Return a list of traffic classifiers that should be run against ingress packets on a sniff interface. Each entry is a tuple of type (statistical or payload) and classifier name, example: [(‘statistical’, ‘statistical_qos_bandwidth_1’)]

ingest_main_policy(main_policy_text, if_name)

Turn a plain text main policy file object into a YAML object and store it as a class variable

ingest_optimised_rules(opt_rules_text, if_name)

Turn a plain optimised TC rules file object into a YAML object and store it as a class variable

tc_mode(if_name)

Return the tc mode for the policy (active or passive)

Module contents