x define for size of helper name !


improvements
- if helpers had id's instead of names, lookup would be faster
- NACK handling on slave
	- accept + queue received messages that have too high seqno
- NACK handling on master
	- send retransmitted packets via unicast
x timeout for flushing pending packets in sender queue
- handle case where MASTER_ANNOUNCE of new master is lost and two masters
  are transmitting (via upsince?)
- potential race condition with helpers:
	- helper is resolved within conntrack lock
	- lock is released
	- later, new conntrack is put in hashtable
	- thus, if somebody removes the helper in between -> boom.
x put entries on ordered list for ctnetlink

protocol improvements
- exchange structure sizes / endianness at startup to ensure
  compatibility of master/slave
x add version number (8bit)
x sequence number doesn't have to be 32bit
- minseq is for?
x msgtype should be split in 'operation' and 'resource'
- if we use different structure anyway, make protocol 
  endian/typesafe.  otherwise: why seperate structure anyway?
- bitmask with SYNC packets indicating whihc fields actually changed
	- improves performance on sender + receiver
	- allows more intelligent tricks wrt expect handling
- differentiate between CREATE and UPDATE
- somehow signal when initial dump is done


- events that can happen on the master
	- creating a new conntrack
			fill in all fields
		- updating an existing conntrack:
			nat
			proto
			help
			helper
			tuplehash[REPLY]
			status
			timeout
			expecting (number of expects)
				inc if new expect is created
				dec if expect is confirmed
			master (set if we are confirming an expect)
		- deleting an old conntrack
		- creating a new expect
		- updating an existing expect
			- change expect (due to nat)
			- confirm expect
		- delete an expect

future investigation:
- what about RCU instead of spinlocks everywhere?

ctnetlink TODO
	- rename check/change functions of proto/app helpers, since they
	  are not ctnl specific
	- do we really want that check?  it's on the slave...
	- do we need to grab tcp_lock / irc_lock and others?
	- do we need the ordered list?
	- profiling: how expensive is notifier list?

- uniqueneess of conntrack ID
	- tuple is always unique in whole cluster
	- id has to be unique, too. 
	- id for a connection is determined upon creation time. it never
	  changes, even after failover and on other nodes.
	- as all IS's have corresponding conntracks on every node,
	  if ID is unique on local node, it will be unique within cluster
	- problem is after failover: how does new master generate ID's 
	  in a performant way (incrementing a new counter and checking if ID is
	  already taken is suboptimal)
		- dividing ID space between nodes
			- id only has to be unique on local node, since
			  other nodes would prefix it with 8bit node-id
	- every node has to have an ID hash for all entries :(
	- tuple is 18 bytes of size, id would be 4/8 bytes.
	- I doubt that calling the hash function on the tuple and transporting
	  a couple of extra bytes in the sync message would have worse
	  performance than yet another completely new hash table (with slab
	  cache, memory usage, locking and hashfunction).
	- thus: unique ID is not worth being used

- uniquely identifying an expectation
	- unconfirmed expectations are uniquely identified by 
		- tuple of master conntrack
		- expected tuple
	- confirmed expectations are uniquely identified by
		- tuple of master conntrack
		- tuple of sibling conntrack
	- IMPORTANT: order always has to be:
		1) create master conntrack
		2) create expect
		3) create client conntrack
		4) confirm expect (update to both master+client)

- ring buffer on sender side needs to be protected against:
	- multiple conntrack events trying to insert new data
	- single threaded sender sending packets

- ring buffer usage on sender side:

  _____________________________________________________________________
 	^          ^         ^         ^
	backlog    sent      cur       alloc

	- new packets are allocated by advancing the alloc ptr
	- once new packet is fully completed, 'cur' is advanced and the sender
	  thread is woken up
	- if alloc hits backlog, we need to malloc a new buffer and insert
	  it into the ring
	- sender thread advances 'sent' after a packet has been sent.  It also
	  advances backlog, in order to keep a fixed-length backlog

	  (sent->next, cur->prev): cts_buffs that are pending to be sent
	  (cur, alloc): cts_buffs that are allocated but not fully filled
	  (alloc->next,backlog->prev): cts_buffst that are free to be overwrt.
	  (backlog, sent): saved for retransmittion

- ring buffer usage on receiver side:

	- new packets are allocated by advancing the alloc ptr
	- the 'cur' ptr is advanced up to the last contiguously received seq
	- the 'sent' ptr is advanced every time a packet has been fully
	  processed (handed to the upper layer)
	- backlog is not needed, set to the same as 'sent' in order to 
	  male advance_alloc() happy

	  seqno: last contiguously received seqno

	  (sent->next, cur->prev): cts_buffs that are pending to be received
	  (cur, alloc): cts_buffs that are allocated but not yet filled
	  (alloc->next,sent): unused


- protocol endpoint states
	- undefined
	- slave_initial 
		- after startup in client mode
	- slave_syn_sent
		- after receiving first message from master 
	- slave_syn_recv
		- while receiving initialy sync from master
	- slave_running
		- after initial sync is complete
	- master
		- after startup in master mode
		- after being switched from client to master by /proc

	- only slave_running is eligible for becoming a master!
	- if master dies and new master comes up, master sends  


- protocol control message types:

	- master announce (master->slave)
		- master ip address (from ipv4?)
		- master up since (timestamp)
		- flags
			- new_master (just came up)
		- current seqno

	- slave_sync_req (slave->master)
		- slave ip address (from ipv4?)
		- master it is requesting from (can be any)
		- flags
			- initial sync requested
		
	- slave_nack (slave->master)
		- sequence number that is missing

- protocol data message types:

	- SYNC_MSG_UPDATE (update conntrack entry)
		- FLAG_CREATE

	- SYNC_MSG_DELETE (remove conntrack entry)

	- 

