[linux-block.git] / Documentation / networking / ipvlan.rst

.. SPDX-License-Identifier: GPL-2.0

===================
IPVLAN Driver HOWTO
===================

Initial Release:
	Mahesh Bandewar <maheshb AT google.com>

1. Introduction:
================
This is conceptually very similar to the macvlan driver with one major
exception of using L3 for mux-ing /demux-ing among slaves. This property makes
the master device share the L2 with its slave devices. I have developed this
driver in conjunction with network namespaces and not sure if there is use case
outside of it.


2. Building and Installation:
=============================

In order to build the driver, please select the config item CONFIG_IPVLAN.
The driver can be built into the kernel (CONFIG_IPVLAN=y) or as a module
(CONFIG_IPVLAN=m).


3. Configuration:
=================

There are no module parameters for this driver and it can be configured
using IProute2/ip utility.
::

    ip link add link <master> name <slave> type ipvlan [ mode MODE ] [ FLAGS ]
       where
	 MODE: l3 (default) | l3s | l2
	 FLAGS: bridge (default) | private | vepa

e.g.

    (a) Following will create IPvlan link with eth0 as master in
	L3 bridge mode::

	  bash# ip link add link eth0 name ipvl0 type ipvlan
    (b) This command will create IPvlan link in L2 bridge mode::

	  bash# ip link add link eth0 name ipvl0 type ipvlan mode l2 bridge

    (c) This command will create an IPvlan device in L2 private mode::

	  bash# ip link add link eth0 name ipvlan type ipvlan mode l2 private

    (d) This command will create an IPvlan device in L2 vepa mode::

	  bash# ip link add link eth0 name ipvlan type ipvlan mode l2 vepa


4. Operating modes:
===================

IPvlan has two modes of operation - L2 and L3. For a given master device,
you can select one of these two modes and all slaves on that master will
operate in the same (selected) mode. The RX mode is almost identical except
that in L3 mode the slaves won't receive any multicast / broadcast traffic.
L3 mode is more restrictive since routing is controlled from the other (mostly)
default namespace.

4.1 L2 mode:
------------

In this mode TX processing happens on the stack instance attached to the
slave device and packets are switched and queued to the master device to send
out. In this mode the slaves will RX/TX multicast and broadcast (if applicable)
as well.

4.2 L3 mode:
------------

In this mode TX processing up to L3 happens on the stack instance attached
to the slave device and packets are switched to the stack instance of the
master device for the L2 processing and routing from that instance will be
used before packets are queued on the outbound device. In this mode the slaves
will not receive nor can send multicast / broadcast traffic.

4.3 L3S mode:
-------------

This is very similar to the L3 mode except that iptables (conn-tracking)
works in this mode and hence it is L3-symmetric (L3s). This will have slightly less
performance but that shouldn't matter since you are choosing this mode over plain-L3
mode to make conn-tracking work.

5. Mode flags:
==============

At this time following mode flags are available

5.1 bridge:
-----------
This is the default option. To configure the IPvlan port in this mode,
user can choose to either add this option on the command-line or don't specify
anything. This is the traditional mode where slaves can cross-talk among
themselves apart from talking through the master device.

5.2 private:
------------
If this option is added to the command-line, the port is set in private
mode. i.e. port won't allow cross communication between slaves.

5.3 vepa:
---------
If this is added to the command-line, the port is set in VEPA mode.
i.e. port will offload switching functionality to the external entity as
described in 802.1Qbg
Note: VEPA mode in IPvlan has limitations. IPvlan uses the mac-address of the
master-device, so the packets which are emitted in this mode for the adjacent
neighbor will have source and destination mac same. This will make the switch /
router send the redirect message.

6. What to choose (macvlan vs. ipvlan)?
=======================================

These two devices are very similar in many regards and the specific use
case could very well define which device to choose. if one of the following
situations defines your use case then you can choose to use ipvlan:


(a) The Linux host that is connected to the external switch / router has
    policy configured that allows only one mac per port.
(b) No of virtual devices created on a master exceed the mac capacity and
    puts the NIC in promiscuous mode and degraded performance is a concern.
(c) If the slave device is to be put into the hostile / untrusted network
    namespace where L2 on the slave could be changed / misused.


6. Example configuration:
=========================

::

  +=============================================================+
  |  Host: host1                                                |
  |                                                             |
  |   +----------------------+      +----------------------+    |
  |   |   NS:ns0             |      |  NS:ns1              |    |
  |   |                      |      |                      |    |
  |   |                      |      |                      |    |
  |   |        ipvl0         |      |         ipvl1        |    |
  |   +----------#-----------+      +-----------#----------+    |
  |              #                              #               |
  |              ################################               |
  |                              # eth0                         |
  +==============================#==============================+


(a) Create two network namespaces - ns0, ns1::

	ip netns add ns0
	ip netns add ns1

(b) Create two ipvlan slaves on eth0 (master device)::

	ip link add link eth0 ipvl0 type ipvlan mode l2
	ip link add link eth0 ipvl1 type ipvlan mode l2

(c) Assign slaves to the respective network namespaces::

	ip link set dev ipvl0 netns ns0
	ip link set dev ipvl1 netns ns1

(d) Now switch to the namespace (ns0 or ns1) to configure the slave devices

	- For ns0::

		(1) ip netns exec ns0 bash
		(2) ip link set dev ipvl0 up
		(3) ip link set dev lo up
		(4) ip -4 addr add 127.0.0.1 dev lo
		(5) ip -4 addr add $IPADDR dev ipvl0
		(6) ip -4 route add default via $ROUTER dev ipvl0

	- For ns1::

		(1) ip netns exec ns1 bash
		(2) ip link set dev ipvl1 up
		(3) ip link set dev lo up
		(4) ip -4 addr add 127.0.0.1 dev lo
		(5) ip -4 addr add $IPADDR dev ipvl1
		(6) ip -4 route add default via $ROUTER dev ipvl1
Commit	Line	Data
1dc2a785	1	.. SPDX-License-Identifier: GPL-2.0
2ad7bf36	2
1dc2a785 MCC	3	===================
	4	IPVLAN Driver HOWTO
	5	===================
2ad7bf36 MB	6
	7	Initial Release:
	8	Mahesh Bandewar <maheshb AT google.com>
	9
	10	1. Introduction:
1dc2a785 MCC	11	================
1dc2a785 MCC	12	This is conceptually very similar to the macvlan driver with one major
2ad7bf36	13	exception of using L3 for mux-ing /demux-ing among slaves. This property makes
404a5ad7	14	the master device share the L2 with its slave devices. I have developed this
edb9a1b8	15	driver in conjunction with network namespaces and not sure if there is use case
2ad7bf36 MB	16	outside of it.
	17
	18
	19	2. Building and Installation:
1dc2a785 MCC	20	=============================
	21
	22	In order to build the driver, please select the config item CONFIG_IPVLAN.
2ad7bf36 MB	23	The driver can be built into the kernel (CONFIG_IPVLAN=y) or as a module
	24	(CONFIG_IPVLAN=m).
	25
	26
	27	3. Configuration:
1dc2a785 MCC	28	=================
	29
	30	There are no module parameters for this driver and it can be configured
2ad7bf36	31	using IProute2/ip utility.
1dc2a785	32	::
2ad7bf36	33
a190d04d MB	34	ip link add link <master> name <slave> type ipvlan [ mode MODE ] [ FLAGS ]
a190d04d MB	35	where
1dc2a785 MCC	36	MODE: l3 (default) \| l3s \| l2
	37	FLAGS: bridge (default) \| private \| vepa
	38
	39	e.g.
2ad7bf36	40
a190d04d	41	(a) Following will create IPvlan link with eth0 as master in
1dc2a785 MCC	42	L3 bridge mode::
	43
	44	bash# ip link add link eth0 name ipvl0 type ipvlan
	45	(b) This command will create IPvlan link in L2 bridge mode::
	46
	47	bash# ip link add link eth0 name ipvl0 type ipvlan mode l2 bridge
	48
	49	(c) This command will create an IPvlan device in L2 private mode::
	50
	51	bash# ip link add link eth0 name ipvlan type ipvlan mode l2 private
	52
	53	(d) This command will create an IPvlan device in L2 vepa mode::
	54
	55	bash# ip link add link eth0 name ipvlan type ipvlan mode l2 vepa
2ad7bf36 MB	56
	57
	58	4. Operating modes:
1dc2a785 MCC	59	===================
	60
	61	IPvlan has two modes of operation - L2 and L3. For a given master device,
2ad7bf36 MB	62	you can select one of these two modes and all slaves on that master will
2ad7bf36 MB	63	operate in the same (selected) mode. The RX mode is almost identical except
a266ef69	64	that in L3 mode the slaves won't receive any multicast / broadcast traffic.
2ad7bf36 MB	65	L3 mode is more restrictive since routing is controlled from the other (mostly)
	66	default namespace.
	67
	68	4.1 L2 mode:
1dc2a785 MCC	69	------------
	70
	71	In this mode TX processing happens on the stack instance attached to the
2ad7bf36 MB	72	slave device and packets are switched and queued to the master device to send
	73	out. In this mode the slaves will RX/TX multicast and broadcast (if applicable)
	74	as well.
	75
	76	4.2 L3 mode:
1dc2a785 MCC	77	------------
	78
	79	In this mode TX processing up to L3 happens on the stack instance attached
2ad7bf36 MB	80	to the slave device and packets are switched to the stack instance of the
	81	master device for the L2 processing and routing from that instance will be
	82	used before packets are queued on the outbound device. In this mode the slaves
	83	will not receive nor can send multicast / broadcast traffic.
	84
4fbae7d8	85	4.3 L3S mode:
1dc2a785 MCC	86	-------------
	87
	88	This is very similar to the L3 mode except that iptables (conn-tracking)
4fbae7d8 MB	89	works in this mode and hence it is L3-symmetric (L3s). This will have slightly less
	90	performance but that shouldn't matter since you are choosing this mode over plain-L3
	91	mode to make conn-tracking work.
2ad7bf36	92
a190d04d	93	5. Mode flags:
1dc2a785 MCC	94	==============
	95
	96	At this time following mode flags are available
a190d04d MB	97
a190d04d MB	98	5.1 bridge:
1dc2a785 MCC	99	-----------
1dc2a785 MCC	100	This is the default option. To configure the IPvlan port in this mode,
a190d04d MB	101	user can choose to either add this option on the command-line or don't specify
a190d04d MB	102	anything. This is the traditional mode where slaves can cross-talk among
bb38ccce	103	themselves apart from talking through the master device.
a190d04d MB	104
a190d04d MB	105	5.2 private:
1dc2a785 MCC	106	------------
1dc2a785 MCC	107	If this option is added to the command-line, the port is set in private
bb38ccce	108	mode. i.e. port won't allow cross communication between slaves.
a190d04d	109
fe89aa6b	110	5.3 vepa:
1dc2a785 MCC	111	---------
1dc2a785 MCC	112	If this is added to the command-line, the port is set in VEPA mode.
fe89aa6b MB	113	i.e. port will offload switching functionality to the external entity as
	114	described in 802.1Qbg
	115	Note: VEPA mode in IPvlan has limitations. IPvlan uses the mac-address of the
	116	master-device, so the packets which are emitted in this mode for the adjacent
	117	neighbor will have source and destination mac same. This will make the switch /
	118	router send the redirect message.
a190d04d MB	119
a190d04d MB	120	6. What to choose (macvlan vs. ipvlan)?
1dc2a785 MCC	121	=======================================
	122
	123	These two devices are very similar in many regards and the specific use
2ad7bf36	124	case could very well define which device to choose. if one of the following
1dc2a785 MCC	125	situations defines your use case then you can choose to use ipvlan:
	126
	127
	128	(a) The Linux host that is connected to the external switch / router has
	129	policy configured that allows only one mac per port.
	130	(b) No of virtual devices created on a master exceed the mac capacity and
	131	puts the NIC in promiscuous mode and degraded performance is a concern.
	132	(c) If the slave device is to be put into the hostile / untrusted network
	133	namespace where L2 on the slave could be changed / misused.
2ad7bf36 MB	134
	135
	136	6. Example configuration:
1dc2a785 MCC	137	=========================
	138
	139	::
2ad7bf36 MB	140
	141	+=============================================================+
	142	\| Host: host1 \|
	143	\| \|
	144	\| +----------------------+ +----------------------+ \|
	145	\| \| NS:ns0 \| \| NS:ns1 \| \|
	146	\| \| \| \| \| \|
	147	\| \| \| \| \| \|
	148	\| \| ipvl0 \| \| ipvl1 \| \|
	149	\| +----------#-----------+ +-----------#----------+ \|
	150	\| # # \|
	151	\| ################################ \|
	152	\| # eth0 \|
	153	+==============================#==============================+
	154
	155
1dc2a785 MCC	156	(a) Create two network namespaces - ns0, ns1::
	157
	158	ip netns add ns0
	159	ip netns add ns1
	160
	161	(b) Create two ipvlan slaves on eth0 (master device)::
	162
	163	ip link add link eth0 ipvl0 type ipvlan mode l2
	164	ip link add link eth0 ipvl1 type ipvlan mode l2
	165
	166	(c) Assign slaves to the respective network namespaces::
	167
	168	ip link set dev ipvl0 netns ns0
	169	ip link set dev ipvl1 netns ns1
	170
	171	(d) Now switch to the namespace (ns0 or ns1) to configure the slave devices
	172
	173	- For ns0::
	174
	175	(1) ip netns exec ns0 bash
	176	(2) ip link set dev ipvl0 up
	177	(3) ip link set dev lo up
	178	(4) ip -4 addr add 127.0.0.1 dev lo
	179	(5) ip -4 addr add $IPADDR dev ipvl0
	180	(6) ip -4 route add default via $ROUTER dev ipvl0
	181
	182	- For ns1::
	183
	184	(1) ip netns exec ns1 bash
	185	(2) ip link set dev ipvl1 up
	186	(3) ip link set dev lo up
	187	(4) ip -4 addr add 127.0.0.1 dev lo
	188	(5) ip -4 addr add $IPADDR dev ipvl1
	189	(6) ip -4 route add default via $ROUTER dev ipvl1