nmeta2dpae package¶
Submodules¶
nmeta2dpae.config module¶
This module is part of the nmeta2 suite . It represents the configuration data for a Data Plane Auxiliary Engine (DPAE). . It expects a file called “config.yaml” to be in a subdirectory called config, and this file to contain properly formed YAML
-
class
nmeta2dpae.config.
Config
¶ Bases:
object
This class is instantiated by nmeta_dpae.py and provides methods to ingest the configuration file and provides access to the keys/values that it contains. Config file is in YAML in config subdirectory and is called ‘config.yaml’
-
get_value
(config_key)¶ Passed a key and see if it exists in the config YAML. If it does then return the value, if not return 0
-
nmeta2dpae.controlchannel module¶
This module is part of the nmeta2 suite . It is provides control channel services between the nmeta Data Plane Auxiliary Engine (DPAE) and the OpenFlow controller using REST API calls
-
class
nmeta2dpae.controlchannel.
ControlChannel
(_nmeta2dpae, _config, if_name, dp)¶ Bases:
object
This class is instantiated by nmeta_dpae.py and provides methods to interact with the nmeta control plane
-
get_policy
(location)¶ Get the a policy from the Controller (YAML in string format)
-
keepalive
(event_flag, location, if_name)¶ Do regular keepalive polls to the DPAE to check if is still available, in dedicated process. If keepalive fails, then set an event flag for parent process.
-
phase1
(api_base, if_name)¶ Phase 1 (global to DPAE) connection to the control plane, as an active data plane auxiliary device
-
phase2
(api_base, if_name, dpae2ctrl_mac, ctrl2dpae_mac, dpae_ethertype)¶ Phase 2 (per DPAE sniffing interface) switch/port discovery
-
phase3
(api_base, if_name, dpae2ctrl_mac, ctrl2dpae_mac, dpae_ethertype)¶ Phase 3 (per DPAE sniffing interface) confirmation of sniffing packets
-
phase4
(api_base, if_name)¶ Phase 4 (per DPAE sniffing interface) Negotiate what services will be run by the DPAE
-
tc_advise_controller
(location, tc_result)¶ Pass Traffic Classification (TC) information to the controller via the API
-
tc_start
(location)¶ Tell the Controller to start sending us packets that need traffic classification
-
-
class
nmeta2dpae.controlchannel.
JSON_Body
(req_body)¶ Bases:
object
Represents a JSON-encoded body of an HTTP request. Doesn’t do logging, but does set .error when things don’t go to plan with a friendly message.
-
decode
(req_body)¶ Passed an allegedly JSON body and see if it decodes. Set error variable for exceptions
-
validate
(key_list)¶ Passed a list of keys and check that they exist in the JSON. If they don’t return 0 and set error to description of first missing key that was found
-
nmeta2dpae.dp module¶
This module is part of the nmeta2 suite . It provides an object for data plane coordination services . Version 2.x Toulouse Code
-
class
nmeta2dpae.dp.
DP
(_config)¶ Bases:
object
This class is instantiated by nmeta2_dpae.py and provides methods to run the data plane services.
-
dp_discover
(queue, if_name, dpae2ctrl_mac, ctrl2dpae_mac, dpae_ethertype, timeout, uuid_dpae, uuid_controller)¶ Data plane service for DPAE Join Discover Packet Sniffing
-
dp_run
(interplane_queue, tc_policy, if_name)¶ Run Data Plane (DP) Traffic Classification for an interface
-
-
class
nmeta2dpae.dp.
JSON_Body
(req_body)¶ Bases:
object
Represents a JSON-encoded body of an HTTP request. Doesn’t do logging, but does set .error when things don’t go to plan with a friendly message.
-
decode
(req_body)¶ Passed an allegedly JSON body and see if it decodes. Set error variable for exceptions
-
validate
(key_list)¶ Passed a list of keys and check that they exist in the JSON. If they don’t return 0 and set error to description of first missing key that was found
-
nmeta2dpae.flow module¶
This module is part of the nmeta2 suite . It provides an abstraction for a TCP flow that links to a MongoDB database and changes to the context of the flow that a supplied packet belongs to . Version 2.x Toulouse Code
-
class
nmeta2dpae.flow.
Flow
(logger, mongo_addr, mongo_port)¶ Bases:
object
An object that represents a flow that we are classifying
Intended to provide an abstraction of a flow that classifiers can use to make determinations without having to understand implementations such as database lookups etc.
Be aware that this module is not very mature yet. It does not cover some basic corner cases such as packet retransmissions and out of order or missing packets.
Variables available for Classifiers (assumes class instantiated as an object called ‘flow’):
Variables for the current packet:
- flow.ip_src
- IP source address of latest packet in flow
- flow.ip_dst
- IP dest address of latest packet in flow
- flow.tcp_src
- TCP source port of latest packet in flow
- flow.tcp_dst
- TCP dest port of latest packet in flow
- flow.tcp_seq
- TCP sequence number of latest packet in flow
- flow.tcp_acq
- TCP acknowledgement number of latest packet in flow
- flow.tcp_fin()
- True if TCP FIN flag is set in the current packet
- flow.tcp_syn()
- True if TCP SYN flag is set in the current packet
- flow.tcp_rst()
- True if TCP RST flag is set in the current packet
- flow.tcp_psh()
- True if TCP PSH flag is set in the current packet
- flow.tcp_ack()
- True if TCP ACK flag is set in the current packet
- flow.tcp_urg()
- True if TCP URG flag is set in the current packet
- flow.tcp_ece()
- True if TCP ECE flag is set in the current packet
- flow.tcp_cwr()
- True if TCP CWR flag is set in the current packet
- flow.payload
- Payload of TCP of latest packet in flow
- flow.packet_length
- Length in bytes of the current packet on wire
- flow.packet_direction
- c2s (client to server) or s2c directionality based on first observed packet having SYN or SYN+ACK flag, otherwise client assumed as source IP of first packet and verified_direction set to 0 (i.e. don’t trust packet_direction unless verified_direction is set)
Variables for the whole flow:
- flow.verified_direction
- Describes how the directionality of the flow was ascertained. Values can be verified-SYN, verified-SYNACK or 0 (unverified)
- flow.finalised
- A classification has been made
- flow.suppressed
- The flow packet count number when a request was made to controller to not see further packets in this flow. 0 is not suppressed
- flow.packet_count
- Unique packets registered for the flow
- flow.client
- The IP that is the originator of the TCP session (if known, otherwise 0)
- flow.server
- The IP that is the destination of the TCP session session (if known, otherwise 0)
Methods available for Classifiers: (assumes class instantiated as an object called ‘flow’)
- flow.max_packet_size()
- Size of largest packet in the flow
- flow.max_interpacket_interval()
- TBD
- flow.min_interpacket_interval()
- TBD
- Challenges:
- duplicate packets
- IP fragments (not handled)
- Flow reuse - TCP source port reused (not handled - yet)
-
ingest_packet
(pkt, pkt_receive_timestamp)¶ Ingest a packet and put the flow object into the context of the flow that the packet belongs to.
-
max_interpacket_interval
()¶ Return the size of the largest inter-packet time interval in the flow (assessed per direction in flow). . Note: slightly inaccurate due to floating point rounding.
-
max_packet_size
()¶ Return the size of the largest packet in the flow (in either direction)
-
min_interpacket_interval
()¶ Return the size of the smallest inter-packet time interval in the flow (assessed per direction in flow) . Note: slightly inaccurate due to floating point rounding.
-
set_suppress_flow
()¶ Set the suppressed attribute in the flow database object to the current packet count so that future suppressions of the same flow can be backed off to prevent overwhelming the controller
-
tcp_ack
()¶ Does the current packet have the TCP ACK flag set?
-
tcp_cwr
()¶ Does the current packet have the TCP CWR flag set?
-
tcp_ece
()¶ Does the current packet have the TCP ECE flag set?
-
tcp_fin
()¶ Does the current packet have the TCP FIN flag set?
-
tcp_psh
()¶ Does the current packet have the TCP PSH flag set?
-
tcp_rst
()¶ Does the current packet have the TCP RST flag set?
-
tcp_syn
()¶ Does the current packet have the TCP SYN flag set?
-
tcp_urg
()¶ Does the current packet have the TCP URG flag set?
nmeta2dpae.nmeta2dpae module¶
This module is part of the nmeta2 suite . nmeta Data Plane Auxiliary Engine (DPAE) Used as an auxilary data plane component for functions such as offloading packet-intensive traffic classification from the controller.
-
class
nmeta2dpae.nmeta2dpae.
DPAE
¶ Bases:
object
This class provides methods for a Data Plane Auxiliary Engine (DPAE), an auxiliary entity that provides services to nmeta.
-
cp_run
(if_name, controlchannel, location)¶ Run Control Plane (CP) Traffic Classification for an interface
-
per_interface
(if_name)¶ Run per interface that sniffing will run on as separate process
-
run
()¶ Run the DPAE instance
-
nmeta2dpae.sniff module¶
This module is part of the nmeta2 suite . It provides packet sniffing services
-
class
nmeta2dpae.sniff.
Ifreq
¶ Bases:
_ctypes.Structure
Class used in setting Ethernet interface promiscuous mode
-
ifr_flags
¶ Structure/Union member
-
ifr_ifrn
¶ Structure/Union member
-
-
class
nmeta2dpae.sniff.
Sniff
(_config, tc)¶ Bases:
object
This class is instantiated by nmeta_dpae.py and provides methods to sniff and process inbound packets on a given interface
-
discover_confirm
(if_name, dpae2ctrl_mac, ctrl2dpae_mac, dpae_ethertype, timeout)¶ This function processes sniffs for a discover confirm packet and returns 1 if seen and valid, otherwise 0 after expiry of timeout period
-
get_promiscuous_mode
(if_name)¶ Return the promiscuous mode of an interface. 1 is promiscuous mode enabled 0 is promiscuous mode disabled
-
set_promiscuous_mode
(if_name)¶ Set a given Ethernet interface to promiscuous mode so that it can receive packets destined for any MAC address.
-
sniff_run
(if_name, tc, tc_policy, queue)¶ This function sniffs packets from a NIC. It passes the packets to the tc module for classification and returns any TC results to parent process via a queue.
In active mode it also sends the processed packet back to the switch
-
-
nmeta2dpae.sniff.
mac_addr
(address)¶ Convert a MAC address to a readable/printable string
nmeta2dpae.tc module¶
This module is part of the nmeta2 suite . It provides an object for traffic classification and includes ingesting the policy from YAML and checking packets against policy, calling appropriate classifiers and returning actions. . Version 2.x Toulouse Code
-
class
nmeta2dpae.tc.
TC
(_config)¶ Bases:
object
This class is instantiated by nmeta2_dpae.py and provides methods to ingest the policy as yaml and check packets against policy, calling appropriate classifiers and returning actions.
-
classify_dpkt
(pkt, pkt_receive_timestamp, if_name)¶ Perform traffic classification on a packet using dpkt for packet parsing
-
classify_dpkt_wrapper
(pkt, pkt_receive_timestamp, if_name)¶ Used to catch and handle exceptions in classify_dpkt otherwise it can just hang with no explaination... TBD: turn this into a decorator...
-
instantiate_classifiers
(_classifiers)¶ Dynamically import and instantiate classes for any dynamic classifiers specified in the controller nmeta2 main_policy.yaml . Passed a list of tuples of classifier type / classifer name . Classifier modules live in the ‘classifiers’ subdirectory .
-
-
nmeta2dpae.tc.
mac_addr
(address)¶ Convert a MAC address to a readable/printable string
nmeta2dpae.tc_policy_dpae module¶
This module is part of nmeta Data Plane Auxiliary Engine (DPAE) . It is used to contain the Traffic Classification (TC) policy and provide methods and direct variables to access it . Version 2.x Toulouse Code
-
class
nmeta2dpae.tc_policy_dpae.
TCPolicy
(_config)¶ Bases:
object
This class is instantiated by nmeta2.py and provides methods to ingest the policy file main_policy.yaml and validate that it is correctly structured
-
get_id_flag
(if_name, id_key)¶ Get a value for an Identity Indicator harvesting flag
-
get_tc_classifiers
(if_name)¶ Return a list of traffic classifiers that should be run against ingress packets on a sniff interface. Each entry is a tuple of type (statistical or payload) and classifier name, example: [(‘statistical’, ‘statistical_qos_bandwidth_1’)]
-
ingest_main_policy
(main_policy_text, if_name)¶ Turn a plain text main policy file object into a YAML object and store it as a class variable
-
ingest_optimised_rules
(opt_rules_text, if_name)¶ Turn a plain optimised TC rules file object into a YAML object and store it as a class variable
-
tc_mode
(if_name)¶ Return the tc mode for the policy (active or passive)
-