Commit | Line | Data |
---|---|---|
06df6572 MCC |
1 | .. SPDX-License-Identifier: GPL-2.0 |
2 | ||
3 | ========================================== | |
4 | EQL Driver: Serial IP Load Balancing HOWTO | |
5 | ========================================== | |
6 | ||
1da177e4 | 7 | Simon "Guru Aleph-Null" Janes, simon@ncm.com |
06df6572 | 8 | |
1da177e4 LT |
9 | v1.1, February 27, 1995 |
10 | ||
11 | This is the manual for the EQL device driver. EQL is a software device | |
12 | that lets you load-balance IP serial links (SLIP or uncompressed PPP) | |
13 | to increase your bandwidth. It will not reduce your latency (i.e. ping | |
14 | times) except in the case where you already have lots of traffic on | |
15 | your link, in which it will help them out. This driver has been tested | |
16 | with the 1.1.75 kernel, and is known to have patched cleanly with | |
17 | 1.1.86. Some testing with 1.1.92 has been done with the v1.1 patch | |
18 | which was only created to patch cleanly in the very latest kernel | |
19 | source trees. (Yes, it worked fine.) | |
20 | ||
06df6572 MCC |
21 | 1. Introduction |
22 | =============== | |
1da177e4 LT |
23 | |
24 | Which is worse? A huge fee for a 56K leased line or two phone lines? | |
25 | It's probably the former. If you find yourself craving more bandwidth, | |
26 | and have a ISP that is flexible, it is now possible to bind modems | |
27 | together to work as one point-to-point link to increase your | |
28 | bandwidth. All without having to have a special black box on either | |
29 | side. | |
30 | ||
31 | ||
32 | The eql driver has only been tested with the Livingston PortMaster-2e | |
33 | terminal server. I do not know if other terminal servers support load- | |
34 | balancing, but I do know that the PortMaster does it, and does it | |
35 | almost as well as the eql driver seems to do it (-- Unfortunately, in | |
36 | my testing so far, the Livingston PortMaster 2e's load-balancing is a | |
37 | good 1 to 2 KB/s slower than the test machine working with a 28.8 Kbps | |
38 | and 14.4 Kbps connection. However, I am not sure that it really is | |
39 | the PortMaster, or if it's Linux's TCP drivers. I'm told that Linux's | |
40 | TCP implementation is pretty fast though.--) | |
41 | ||
42 | ||
43 | I suggest to ISPs out there that it would probably be fair to charge | |
44 | a load-balancing client 75% of the cost of the second line and 50% of | |
45 | the cost of the third line etc... | |
46 | ||
47 | ||
48 | Hey, we can all dream you know... | |
49 | ||
50 | ||
06df6572 MCC |
51 | 2. Kernel Configuration |
52 | ======================= | |
1da177e4 LT |
53 | |
54 | Here I describe the general steps of getting a kernel up and working | |
55 | with the eql driver. From patching, building, to installing. | |
56 | ||
57 | ||
06df6572 MCC |
58 | 2.1. Patching The Kernel |
59 | ------------------------ | |
1da177e4 LT |
60 | |
61 | If you do not have or cannot get a copy of the kernel with the eql | |
62 | driver folded into it, get your copy of the driver from | |
63 | ftp://slaughter.ncm.com/pub/Linux/LOAD_BALANCING/eql-1.1.tar.gz. | |
64 | Unpack this archive someplace obvious like /usr/local/src/. It will | |
06df6572 | 65 | create the following files:: |
1da177e4 | 66 | |
1da177e4 LT |
67 | -rw-r--r-- guru/ncm 198 Jan 19 18:53 1995 eql-1.1/NO-WARRANTY |
68 | -rw-r--r-- guru/ncm 30620 Feb 27 21:40 1995 eql-1.1/eql-1.1.patch | |
69 | -rwxr-xr-x guru/ncm 16111 Jan 12 22:29 1995 eql-1.1/eql_enslave | |
70 | -rw-r--r-- guru/ncm 2195 Jan 10 21:48 1995 eql-1.1/eql_enslave.c | |
1da177e4 LT |
71 | |
72 | Unpack a recent kernel (something after 1.1.92) someplace convenient | |
73 | like say /usr/src/linux-1.1.92.eql. Use symbolic links to point | |
74 | /usr/src/linux to this development directory. | |
75 | ||
76 | ||
06df6572 | 77 | Apply the patch by running the commands:: |
1da177e4 | 78 | |
1da177e4 LT |
79 | cd /usr/src |
80 | patch </usr/local/src/eql-1.1/eql-1.1.patch | |
1da177e4 LT |
81 | |
82 | ||
06df6572 MCC |
83 | 2.2. Building The Kernel |
84 | ------------------------ | |
1da177e4 LT |
85 | |
86 | After patching the kernel, run make config and configure the kernel | |
87 | for your hardware. | |
88 | ||
89 | ||
90 | After configuration, make and install according to your habit. | |
91 | ||
92 | ||
06df6572 MCC |
93 | 3. Network Configuration |
94 | ======================== | |
1da177e4 LT |
95 | |
96 | So far, I have only used the eql device with the DSLIP SLIP connection | |
97 | manager by Matt Dillon (-- "The man who sold his soul to code so much | |
98 | so quickly."--) . How you configure it for other "connection" | |
99 | managers is up to you. Most other connection managers that I've seen | |
100 | don't do a very good job when it comes to handling more than one | |
101 | connection. | |
102 | ||
103 | ||
06df6572 MCC |
104 | 3.1. /etc/rc.d/rc.inet1 |
105 | ----------------------- | |
1da177e4 LT |
106 | |
107 | In rc.inet1, ifconfig the eql device to the IP address you usually use | |
108 | for your machine, and the MTU you prefer for your SLIP lines. One | |
109 | could argue that MTU should be roughly half the usual size for two | |
110 | modems, one-third for three, one-fourth for four, etc... But going | |
111 | too far below 296 is probably overkill. Here is an example ifconfig | |
06df6572 | 112 | command that sets up the eql device:: |
1da177e4 | 113 | |
1da177e4 | 114 | ifconfig eql 198.67.33.239 mtu 1006 |
1da177e4 LT |
115 | |
116 | Once the eql device is up and running, add a static default route to | |
117 | it in the routing table using the cool new route syntax that makes | |
06df6572 | 118 | life so much easier:: |
1da177e4 | 119 | |
1da177e4 | 120 | route add default eql |
1da177e4 LT |
121 | |
122 | ||
06df6572 MCC |
123 | 3.2. Enslaving Devices By Hand |
124 | ------------------------------ | |
1da177e4 LT |
125 | |
126 | Enslaving devices by hand requires two utility programs: eql_enslave | |
127 | and eql_emancipate (-- eql_emancipate hasn't been written because when | |
128 | an enslaved device "dies", it is automatically taken out of the queue. | |
129 | I haven't found a good reason to write it yet... other than for | |
130 | completeness, but that isn't a good motivator is it?--) | |
131 | ||
132 | ||
133 | The syntax for enslaving a device is "eql_enslave <master-name> | |
06df6572 | 134 | <slave-name> <estimated-bps>". Here are some example enslavings:: |
1da177e4 | 135 | |
1da177e4 LT |
136 | eql_enslave eql sl0 28800 |
137 | eql_enslave eql ppp0 14400 | |
138 | eql_enslave eql sl1 57600 | |
1da177e4 LT |
139 | |
140 | When you want to free a device from its life of slavery, you can | |
141 | either down the device with ifconfig (eql will automatically bury the | |
142 | dead slave and remove it from its queue) or use eql_emancipate to free | |
143 | it. (-- Or just ifconfig it down, and the eql driver will take it out | |
06df6572 | 144 | for you.--):: |
1da177e4 | 145 | |
1da177e4 LT |
146 | eql_emancipate eql sl0 |
147 | eql_emancipate eql ppp0 | |
148 | eql_emancipate eql sl1 | |
1da177e4 LT |
149 | |
150 | ||
06df6572 MCC |
151 | 3.3. DSLIP Configuration for the eql Device |
152 | ------------------------------------------- | |
1da177e4 LT |
153 | |
154 | The general idea is to bring up and keep up as many SLIP connections | |
155 | as you need, automatically. | |
156 | ||
157 | ||
06df6572 MCC |
158 | 3.3.1. /etc/slip/runslip.conf |
159 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
1da177e4 | 160 | |
06df6572 | 161 | Here is an example runslip.conf:: |
1da177e4 | 162 | |
06df6572 MCC |
163 | name sl-line-1 |
164 | enabled | |
165 | baud 38400 | |
166 | mtu 576 | |
167 | ducmd -e /etc/slip/dialout/cua2-288.xp -t 9 | |
168 | command eql_enslave eql $interface 28800 | |
169 | address 198.67.33.239 | |
170 | line /dev/cua2 | |
1da177e4 | 171 | |
06df6572 MCC |
172 | name sl-line-2 |
173 | enabled | |
174 | baud 38400 | |
175 | mtu 576 | |
176 | ducmd -e /etc/slip/dialout/cua3-288.xp -t 9 | |
177 | command eql_enslave eql $interface 28800 | |
178 | address 198.67.33.239 | |
179 | line /dev/cua3 | |
1da177e4 | 180 | |
1da177e4 | 181 | |
06df6572 MCC |
182 | 3.4. Using PPP and the eql Device |
183 | --------------------------------- | |
1da177e4 LT |
184 | |
185 | I have not yet done any load-balancing testing for PPP devices, mainly | |
186 | because I don't have a PPP-connection manager like SLIP has with | |
187 | DSLIP. I did find a good tip from LinuxNET:Billy for PPP performance: | |
188 | make sure you have asyncmap set to something so that control | |
189 | characters are not escaped. | |
190 | ||
191 | ||
192 | I tried to fix up a PPP script/system for redialing lost PPP | |
193 | connections for use with the eql driver the weekend of Feb 25-26 '95 | |
194 | (Hereafter known as the 8-hour PPP Hate Festival). Perhaps later this | |
195 | year. | |
196 | ||
197 | ||
06df6572 MCC |
198 | 4. About the Slave Scheduler Algorithm |
199 | ====================================== | |
1da177e4 LT |
200 | |
201 | The slave scheduler probably could be replaced with a dozen other | |
202 | things and push traffic much faster. The formula in the current set | |
203 | up of the driver was tuned to handle slaves with wildly different | |
204 | bits-per-second "priorities". | |
205 | ||
206 | ||
207 | All testing I have done was with two 28.8 V.FC modems, one connecting | |
208 | at 28800 bps or slower, and the other connecting at 14400 bps all the | |
209 | time. | |
210 | ||
211 | ||
212 | One version of the scheduler was able to push 5.3 K/s through the | |
213 | 28800 and 14400 connections, but when the priorities on the links were | |
214 | very wide apart (57600 vs. 14400) the "faster" modem received all | |
215 | traffic and the "slower" modem starved. | |
216 | ||
217 | ||
06df6572 MCC |
218 | 5. Testers' Reports |
219 | =================== | |
1da177e4 LT |
220 | |
221 | Some people have experimented with the eql device with newer | |
222 | kernels (than 1.1.75). I have since updated the driver to patch | |
223 | cleanly in newer kernels because of the removal of the old "slave- | |
224 | balancing" driver config option. | |
225 | ||
226 | ||
06df6572 | 227 | - icee from LinuxNET patched 1.1.86 without any rejects and was able |
1da177e4 LT |
228 | to boot the kernel and enslave a couple of ISDN PPP links. |
229 | ||
06df6572 MCC |
230 | 5.1. Randolph Bentson's Test Report |
231 | ----------------------------------- | |
1da177e4 | 232 | |
06df6572 | 233 | :: |
1da177e4 | 234 | |
06df6572 MCC |
235 | From bentson@grieg.seaslug.org Wed Feb 8 19:08:09 1995 |
236 | Date: Tue, 7 Feb 95 22:57 PST | |
237 | From: Randolph Bentson <bentson@grieg.seaslug.org> | |
238 | To: guru@ncm.com | |
239 | Subject: EQL driver tests | |
1da177e4 LT |
240 | |
241 | ||
06df6572 MCC |
242 | I have been checking out your eql driver. (Nice work, that!) |
243 | Although you may already done this performance testing, here | |
244 | are some data I've discovered. | |
1da177e4 | 245 | |
06df6572 MCC |
246 | Randolph Bentson |
247 | bentson@grieg.seaslug.org | |
1da177e4 | 248 | |
06df6572 | 249 | ------------------------------------------------------------------ |
1da177e4 LT |
250 | |
251 | ||
252 | A pseudo-device driver, EQL, written by Simon Janes, can be used | |
253 | to bundle multiple SLIP connections into what appears to be a | |
254 | single connection. This allows one to improve dial-up network | |
255 | connectivity gradually, without having to buy expensive DSU/CSU | |
256 | hardware and services. | |
257 | ||
258 | I have done some testing of this software, with two goals in | |
259 | mind: first, to ensure it actually works as described and | |
260 | second, as a method of exercising my device driver. | |
261 | ||
262 | The following performance measurements were derived from a set | |
263 | of SLIP connections run between two Linux systems (1.1.84) using | |
264 | a 486DX2/66 with a Cyclom-8Ys and a 486SLC/40 with a Cyclom-16Y. | |
265 | (Ports 0,1,2,3 were used. A later configuration will distribute | |
266 | port selection across the different Cirrus chips on the boards.) | |
267 | Once a link was established, I timed a binary ftp transfer of | |
268 | 289284 bytes of data. If there were no overhead (packet headers, | |
269 | inter-character and inter-packet delays, etc.) the transfers | |
06df6572 | 270 | would take the following times:: |
1da177e4 LT |
271 | |
272 | bits/sec seconds | |
273 | 345600 8.3 | |
274 | 234600 12.3 | |
275 | 172800 16.7 | |
276 | 153600 18.8 | |
277 | 76800 37.6 | |
278 | 57600 50.2 | |
279 | 38400 75.3 | |
280 | 28800 100.4 | |
281 | 19200 150.6 | |
282 | 9600 301.3 | |
283 | ||
284 | A single line running at the lower speeds and with large packets | |
285 | comes to within 2% of this. Performance is limited for the higher | |
286 | speeds (as predicted by the Cirrus databook) to an aggregate of | |
287 | about 160 kbits/sec. The next round of testing will distribute | |
288 | the load across two or more Cirrus chips. | |
289 | ||
290 | The good news is that one gets nearly the full advantage of the | |
291 | second, third, and fourth line's bandwidth. (The bad news is | |
292 | that the connection establishment seemed fragile for the higher | |
293 | speeds. Once established, the connection seemed robust enough.) | |
294 | ||
06df6572 MCC |
295 | ====== ======== === ======== ======= ======= === |
296 | #lines speed mtu seconds theory actual %of | |
297 | kbit/sec duration speed speed max | |
298 | ====== ======== === ======== ======= ======= === | |
299 | 3 115200 900 _ 345600 | |
300 | 3 115200 400 18.1 345600 159825 46 | |
301 | 2 115200 900 _ 230400 | |
302 | 2 115200 600 18.1 230400 159825 69 | |
303 | 2 115200 400 19.3 230400 149888 65 | |
304 | 4 57600 900 _ 234600 | |
305 | 4 57600 600 _ 234600 | |
306 | 4 57600 400 _ 234600 | |
307 | 3 57600 600 20.9 172800 138413 80 | |
308 | 3 57600 900 21.2 172800 136455 78 | |
309 | 3 115200 600 21.7 345600 133311 38 | |
310 | 3 57600 400 22.5 172800 128571 74 | |
311 | 4 38400 900 25.2 153600 114795 74 | |
312 | 4 38400 600 26.4 153600 109577 71 | |
313 | 4 38400 400 27.3 153600 105965 68 | |
314 | 2 57600 900 29.1 115200 99410.3 86 | |
315 | 1 115200 900 30.7 115200 94229.3 81 | |
316 | 2 57600 600 30.2 115200 95789.4 83 | |
317 | 3 38400 900 30.3 115200 95473.3 82 | |
318 | 3 38400 600 31.2 115200 92719.2 80 | |
319 | 1 115200 600 31.3 115200 92423 80 | |
320 | 2 57600 400 32.3 115200 89561.6 77 | |
321 | 1 115200 400 32.8 115200 88196.3 76 | |
322 | 3 38400 400 33.5 115200 86353.4 74 | |
323 | 2 38400 900 43.7 76800 66197.7 86 | |
324 | 2 38400 600 44 76800 65746.4 85 | |
325 | 2 38400 400 47.2 76800 61289 79 | |
326 | 4 19200 900 50.8 76800 56945.7 74 | |
327 | 4 19200 400 53.2 76800 54376.7 70 | |
328 | 4 19200 600 53.7 76800 53870.4 70 | |
329 | 1 57600 900 54.6 57600 52982.4 91 | |
330 | 1 57600 600 56.2 57600 51474 89 | |
331 | 3 19200 900 60.5 57600 47815.5 83 | |
332 | 1 57600 400 60.2 57600 48053.8 83 | |
333 | 3 19200 600 62 57600 46658.7 81 | |
334 | 3 19200 400 64.7 57600 44711.6 77 | |
335 | 1 38400 900 79.4 38400 36433.8 94 | |
336 | 1 38400 600 82.4 38400 35107.3 91 | |
337 | 2 19200 900 84.4 38400 34275.4 89 | |
338 | 1 38400 400 86.8 38400 33327.6 86 | |
339 | 2 19200 600 87.6 38400 33023.3 85 | |
340 | 2 19200 400 91.2 38400 31719.7 82 | |
341 | 4 9600 900 94.7 38400 30547.4 79 | |
342 | 4 9600 400 106 38400 27290.9 71 | |
343 | 4 9600 600 110 38400 26298.5 68 | |
344 | 3 9600 900 118 28800 24515.6 85 | |
345 | 3 9600 600 120 28800 24107 83 | |
346 | 3 9600 400 131 28800 22082.7 76 | |
347 | 1 19200 900 155 19200 18663.5 97 | |
348 | 1 19200 600 161 19200 17968 93 | |
349 | 1 19200 400 170 19200 17016.7 88 | |
350 | 2 9600 600 176 19200 16436.6 85 | |
351 | 2 9600 900 180 19200 16071.3 83 | |
352 | 2 9600 400 181 19200 15982.5 83 | |
353 | 1 9600 900 305 9600 9484.72 98 | |
354 | 1 9600 600 314 9600 9212.87 95 | |
355 | 1 9600 400 332 9600 8713.37 90 | |
356 | ====== ======== === ======== ======= ======= === | |
357 | ||
358 | 5.2. Anthony Healy's Report | |
359 | --------------------------- | |
360 | ||
361 | :: | |
362 | ||
363 | Date: Mon, 13 Feb 1995 16:17:29 +1100 (EST) | |
364 | From: Antony Healey <ahealey@st.nepean.uws.edu.au> | |
365 | To: Simon Janes <guru@ncm.com> | |
366 | Subject: Re: Load Balancing | |
367 | ||
368 | Hi Simon, | |
1da177e4 LT |
369 | I've installed your patch and it works great. I have trialed |
370 | it over twin SL/IP lines, just over null modems, but I was | |
371 | able to data at over 48Kb/s [ISDN link -Simon]. I managed a | |
372 | transfer of up to 7.5 Kbyte/s on one go, but averaged around | |
373 | 6.4 Kbyte/s, which I think is pretty cool. :) |