[linux-2.6-block.git] / Documentation / networking / xfrm_sync.txt


The sync patches work is based on initial patches from
Krisztian <hidden@balabit.hu> and others and additional patches
from Jamal <hadi@cyberus.ca>.

The end goal for syncing is to be able to insert attributes + generate
events so that the SA can be safely moved from one machine to another
for HA purposes.
The idea is to synchronize the SA so that the takeover machine can do
the processing of the SA as accurate as possible if it has access to it.

We already have the ability to generate SA add/del/upd events.
These patches add ability to sync and have accurate lifetime byte (to
ensure proper decay of SAs) and replay counters to avoid replay attacks
with as minimal loss at failover time.
This way a backup stays as closely up-to-date as an active member.

Because the above items change for every packet the SA receives,
it is possible for a lot of the events to be generated.
For this reason, we also add a nagle-like algorithm to restrict
the events. i.e we are going to set thresholds to say "let me
know if the replay sequence threshold is reached or 10 secs have passed"
These thresholds are set system-wide via sysctls or can be updated
per SA.

The identified items that need to be synchronized are:
- the lifetime byte counter
note that: lifetime time limit is not important if you assume the failover
machine is known ahead of time since the decay of the time countdown
is not driven by packet arrival.
- the replay sequence for both inbound and outbound

1) Message Structure
----------------------

nlmsghdr:aevent_id:optional-TLVs.

The netlink message types are:

XFRM_MSG_NEWAE and XFRM_MSG_GETAE.

A XFRM_MSG_GETAE does not have TLVs.
A XFRM_MSG_NEWAE will have at least two TLVs (as is
discussed further below).

aevent_id structure looks like:

   struct xfrm_aevent_id {
             struct xfrm_usersa_id           sa_id;
             xfrm_address_t                  saddr;
             __u32                           flags;
             __u32                           reqid;
   };

The unique SA is identified by the combination of xfrm_usersa_id,
reqid and saddr.

flags are used to indicate different things. The possible
flags are:
        XFRM_AE_RTHR=1, /* replay threshold*/
        XFRM_AE_RVAL=2, /* replay value */
        XFRM_AE_LVAL=4, /* lifetime value */
        XFRM_AE_ETHR=8, /* expiry timer threshold */
        XFRM_AE_CR=16, /* Event cause is replay update */
        XFRM_AE_CE=32, /* Event cause is timer expiry */
        XFRM_AE_CU=64, /* Event cause is policy update */

How these flags are used is dependent on the direction of the
message (kernel<->user) as well the cause (config, query or event).
This is described below in the different messages.

The pid will be set appropriately in netlink to recognize direction
(0 to the kernel and pid = processid that created the event
when going from kernel to user space)

A program needs to subscribe to multicast group XFRMNLGRP_AEVENTS
to get notified of these events.

2) TLVS reflect the different parameters:
-----------------------------------------

a) byte value (XFRMA_LTIME_VAL)
This TLV carries the running/current counter for byte lifetime since
last event.

b)replay value (XFRMA_REPLAY_VAL)
This TLV carries the running/current counter for replay sequence since
last event.

c)replay threshold (XFRMA_REPLAY_THRESH)
This TLV carries the threshold being used by the kernel to trigger events
when the replay sequence is exceeded.

d) expiry timer (XFRMA_ETIMER_THRESH)
This is a timer value in milliseconds which is used as the nagle
value to rate limit the events.

3) Default configurations for the parameters:
----------------------------------------------

By default these events should be turned off unless there is
at least one listener registered to listen to the multicast
group XFRMNLGRP_AEVENTS.

Programs installing SAs will need to specify the two thresholds, however,
in order to not change existing applications such as racoon
we also provide default threshold values for these different parameters
in case they are not specified.

the two sysctls/proc entries are:
a) /proc/sys/net/core/sysctl_xfrm_aevent_etime
used to provide default values for the XFRMA_ETIMER_THRESH in incremental
units of time of 100ms. The default is 10 (1 second)

b) /proc/sys/net/core/sysctl_xfrm_aevent_rseqth
used to provide default values for XFRMA_REPLAY_THRESH parameter
in incremental packet count. The default is two packets.

4) Message types
----------------

a) XFRM_MSG_GETAE issued by user-->kernel.
XFRM_MSG_GETAE does not carry any TLVs.
The response is a XFRM_MSG_NEWAE which is formatted based on what
XFRM_MSG_GETAE queried for.
The response will always have XFRMA_LTIME_VAL and XFRMA_REPLAY_VAL TLVs.
*if XFRM_AE_RTHR flag is set, then XFRMA_REPLAY_THRESH is also retrieved
*if XFRM_AE_ETHR flag is set, then XFRMA_ETIMER_THRESH is also retrieved

b) XFRM_MSG_NEWAE is issued by either user space to configure
or kernel to announce events or respond to a XFRM_MSG_GETAE.

i) user --> kernel to configure a specific SA.
any of the values or threshold parameters can be updated by passing the
appropriate TLV.
A response is issued back to the sender in user space to indicate success
or failure.
In the case of success, additionally an event with
XFRM_MSG_NEWAE is also issued to any listeners as described in iii).

ii) kernel->user direction as a response to XFRM_MSG_GETAE
The response will always have XFRMA_LTIME_VAL and XFRMA_REPLAY_VAL TLVs.
The threshold TLVs will be included if explicitly requested in
the XFRM_MSG_GETAE message.

iii) kernel->user to report as event if someone sets any values or
thresholds for an SA using XFRM_MSG_NEWAE (as described in #i above).
In such a case XFRM_AE_CU flag is set to inform the user that
the change happened as a result of an update.
The message will always have XFRMA_LTIME_VAL and XFRMA_REPLAY_VAL TLVs.

iv) kernel->user to report event when replay threshold or a timeout
is exceeded.
In such a case either XFRM_AE_CR (replay exceeded) or XFRM_AE_CE (timeout
happened) is set to inform the user what happened.
Note the two flags are mutually exclusive.
The message will always have XFRMA_LTIME_VAL and XFRMA_REPLAY_VAL TLVs.

Exceptions to threshold settings
--------------------------------

If you have an SA that is getting hit by traffic in bursts such that
there is a period where the timer threshold expires with no packets
seen, then an odd behavior is seen as follows:
The first packet arrival after a timer expiry will trigger a timeout
event; i.e we don't wait for a timeout period or a packet threshold
to be reached. This is done for simplicity and efficiency reasons.

-JHS
Commit	Line	Data
b8a99520 JHS	1
	2	The sync patches work is based on initial patches from
	3	Krisztian <hidden@balabit.hu> and others and additional patches
	4	from Jamal <hadi@cyberus.ca>.
	5
	6	The end goal for syncing is to be able to insert attributes + generate
edb9a1b8	7	events so that the SA can be safely moved from one machine to another
b8a99520 JHS	8	for HA purposes.
	9	The idea is to synchronize the SA so that the takeover machine can do
	10	the processing of the SA as accurate as possible if it has access to it.
	11
	12	We already have the ability to generate SA add/del/upd events.
	13	These patches add ability to sync and have accurate lifetime byte (to
	14	ensure proper decay of SAs) and replay counters to avoid replay attacks
	15	with as minimal loss at failover time.
edb9a1b8	16	This way a backup stays as closely up-to-date as an active member.
b8a99520 JHS	17
	18	Because the above items change for every packet the SA receives,
	19	it is possible for a lot of the events to be generated.
	20	For this reason, we also add a nagle-like algorithm to restrict
	21	the events. i.e we are going to set thresholds to say "let me
	22	know if the replay sequence threshold is reached or 10 secs have passed"
	23	These thresholds are set system-wide via sysctls or can be updated
	24	per SA.
	25
	26	The identified items that need to be synchronized are:
	27	- the lifetime byte counter
	28	note that: lifetime time limit is not important if you assume the failover
	29	machine is known ahead of time since the decay of the time countdown
	30	is not driven by packet arrival.
	31	- the replay sequence for both inbound and outbound
	32
	33	1) Message Structure
	34	----------------------
	35
	36	nlmsghdr:aevent_id:optional-TLVs.
	37
	38	The netlink message types are:
	39
	40	XFRM_MSG_NEWAE and XFRM_MSG_GETAE.
	41
	42	A XFRM_MSG_GETAE does not have TLVs.
	43	A XFRM_MSG_NEWAE will have at least two TLVs (as is
	44	discussed further below).
	45
	46	aevent_id structure looks like:
	47
	48	struct xfrm_aevent_id {
	49	struct xfrm_usersa_id sa_id;
2b5f6dcc	50	xfrm_address_t saddr;
b8a99520	51	__u32 flags;
2b5f6dcc	52	__u32 reqid;
b8a99520 JHS	53	};
b8a99520 JHS	54
2b5f6dcc JHS	55	The unique SA is identified by the combination of xfrm_usersa_id,
2b5f6dcc JHS	56	reqid and saddr.
b8a99520 JHS	57
	58	flags are used to indicate different things. The possible
	59	flags are:
	60	XFRM_AE_RTHR=1, /* replay threshold*/
	61	XFRM_AE_RVAL=2, /* replay value */
	62	XFRM_AE_LVAL=4, /* lifetime value */
	63	XFRM_AE_ETHR=8, /* expiry timer threshold */
	64	XFRM_AE_CR=16, /* Event cause is replay update */
	65	XFRM_AE_CE=32, /* Event cause is timer expiry */
	66	XFRM_AE_CU=64, /* Event cause is policy update */
	67
	68	How these flags are used is dependent on the direction of the
	69	message (kernel<->user) as well the cause (config, query or event).
	70	This is described below in the different messages.
	71
	72	The pid will be set appropriately in netlink to recognize direction
	73	(0 to the kernel and pid = processid that created the event
	74	when going from kernel to user space)
	75
	76	A program needs to subscribe to multicast group XFRMNLGRP_AEVENTS
	77	to get notified of these events.
	78
	79	2) TLVS reflect the different parameters:
	80	-----------------------------------------
	81
	82	a) byte value (XFRMA_LTIME_VAL)
	83	This TLV carries the running/current counter for byte lifetime since
	84	last event.
	85
	86	b)replay value (XFRMA_REPLAY_VAL)
	87	This TLV carries the running/current counter for replay sequence since
	88	last event.
	89
	90	c)replay threshold (XFRMA_REPLAY_THRESH)
	91	This TLV carries the threshold being used by the kernel to trigger events
	92	when the replay sequence is exceeded.
	93
	94	d) expiry timer (XFRMA_ETIMER_THRESH)
	95	This is a timer value in milliseconds which is used as the nagle
	96	value to rate limit the events.
	97
	98	3) Default configurations for the parameters:
	99	----------------------------------------------
	100
	101	By default these events should be turned off unless there is
	102	at least one listener registered to listen to the multicast
	103	group XFRMNLGRP_AEVENTS.
	104
	105	Programs installing SAs will need to specify the two thresholds, however,
	106	in order to not change existing applications such as racoon
	107	we also provide default threshold values for these different parameters
	108	in case they are not specified.
	109
	110	the two sysctls/proc entries are:
	111	a) /proc/sys/net/core/sysctl_xfrm_aevent_etime
	112	used to provide default values for the XFRMA_ETIMER_THRESH in incremental
	113	units of time of 100ms. The default is 10 (1 second)
	114
	115	b) /proc/sys/net/core/sysctl_xfrm_aevent_rseqth
	116	used to provide default values for XFRMA_REPLAY_THRESH parameter
	117	in incremental packet count. The default is two packets.
	118
	119	4) Message types
	120	----------------
121
122	a) XFRM_MSG_GETAE issued by user-->kernel.
123	XFRM_MSG_GETAE does not carry any TLVs.
124	The response is a XFRM_MSG_NEWAE which is formatted based on what
125	XFRM_MSG_GETAE queried for.
126	The response will always have XFRMA_LTIME_VAL and XFRMA_REPLAY_VAL TLVs.
127	*if XFRM_AE_RTHR flag is set, then XFRMA_REPLAY_THRESH is also retrieved
128	*if XFRM_AE_ETHR flag is set, then XFRMA_ETIMER_THRESH is also retrieved
129
130	b) XFRM_MSG_NEWAE is issued by either user space to configure
131	or kernel to announce events or respond to a XFRM_MSG_GETAE.
132
133	i) user --> kernel to configure a specific SA.
134	any of the values or threshold parameters can be updated by passing the
135	appropriate TLV.
136	A response is issued back to the sender in user space to indicate success
137	or failure.
138	In the case of success, additionally an event with
139	XFRM_MSG_NEWAE is also issued to any listeners as described in iii).
140
141	ii) kernel->user direction as a response to XFRM_MSG_GETAE
142	The response will always have XFRMA_LTIME_VAL and XFRMA_REPLAY_VAL TLVs.
143	The threshold TLVs will be included if explicitly requested in
144	the XFRM_MSG_GETAE message.
145
146	iii) kernel->user to report as event if someone sets any values or
147	thresholds for an SA using XFRM_MSG_NEWAE (as described in #i above).
148	In such a case XFRM_AE_CU flag is set to inform the user that
149	the change happened as a result of an update.
150	The message will always have XFRMA_LTIME_VAL and XFRMA_REPLAY_VAL TLVs.
151
152	iv) kernel->user to report event when replay threshold or a timeout
153	is exceeded.
154	In such a case either XFRM_AE_CR (replay exceeded) or XFRM_AE_CE (timeout
155	happened) is set to inform the user what happened.
156	Note the two flags are mutually exclusive.
157	The message will always have XFRMA_LTIME_VAL and XFRMA_REPLAY_VAL TLVs.
158
159	Exceptions to threshold settings
160	--------------------------------
161
162	If you have an SA that is getting hit by traffic in bursts such that
163	there is a period where the timer threshold expires with no packets
164	seen, then an odd behavior is seen as follows:
165	The first packet arrival after a timer expiry will trigger a timeout
edb9a1b8	166	event; i.e we don't wait for a timeout period or a packet threshold
b8a99520 JHS	167	to be reached. This is done for simplicity and efficiency reasons.
	168
	169	-JHS