rewrote libhdfs engine

[fio.git] / HOWTO

diff --git a/HOWTO b/HOWTO
index b21d27e3b15fb74325703a60de6455cef6960596..482d69651c0362c3afd5863512dc32f7d73f3733 100644 (file)
--- a/HOWTO
+++ b/HOWTO
@@ -767,20 +767,14 @@ ioengine=str      Defines how the job issues io to the file. The following
                                defines engine specific options.
 
                        libhdfs Read and write through Hadoop (HDFS).
                                defines engine specific options.
 
                        libhdfs Read and write through Hadoop (HDFS).

-                               The 'filename' option is used to specify host,
-                               port of the hdfs name-node to connect. This
-                               engine interprets offsets a little
+                               This engine interprets offsets a little

                                differently. In HDFS, files once created
                                cannot be modified. So random writes are not
                                possible. To imitate this, libhdfs engine
                                differently. In HDFS, files once created
                                cannot be modified. So random writes are not
                                possible. To imitate this, libhdfs engine

-                               expects bunch of small files to be created
-                               over HDFS, and engine will randomly pick a
-                               file out of those files based on the offset
-                               generated by fio backend. (see the example
-                               job file to create such files, use rw=write
-                               option). Please note, you might want to set
-                               necessary environment variables to work with
-                               hdfs/libhdfs properly.
+                               creates bunch of small files, and engine will
+                               pick a file out of those files based on the 
+                               offset enerated by fio backend. Each jobs uses
+                               it's own connection to HDFS.

 
                        mtd     Read, write and erase an MTD character device
                                (e.g., /dev/mtd0). Discards are treated as
 
                        mtd     Read, write and erase an MTD character device
                                (e.g., /dev/mtd0). Discards are treated as


@@ -1785,11 +1779,13 @@ that defines them is selected.
                If the job is a TCP listener or UDP reader, the hostname is not
                used and must be omitted unless it is a valid UDP multicast
                address.
                If the job is a TCP listener or UDP reader, the hostname is not
                used and must be omitted unless it is a valid UDP multicast
                address.

+[libhdfs] namenode=str The host name or IP address of a HDFS cluster namenode to contact.

 
 [netsplice] port=int
 [net] port=int The TCP or UDP port to bind to or connect to. If this is used
 with numjobs to spawn multiple instances of the same job type, then this will
 be the starting port number since fio will use a range of ports.
 
 [netsplice] port=int
 [net] port=int The TCP or UDP port to bind to or connect to. If this is used
 with numjobs to spawn multiple instances of the same job type, then this will
 be the starting port number since fio will use a range of ports.

+[libhdfs] port=int     the listening port of the HFDS cluster namenode.

 
 [netsplice] interface=str
 [net] interface=str  The IP address of the network interface used to send or
 
 [netsplice] interface=str
 [net] interface=str  The IP address of the network interface used to send or


@@ -1848,6 +1844,9 @@ be the starting port number since fio will use a range of ports.
 
 [mtd] skip_bad=bool    Skip operations against known bad blocks.
 
 
 [mtd] skip_bad=bool    Skip operations against known bad blocks.
 

+[libhdfs] hdfsdirectory        libhdfs will create chunk in this HDFS directory
+[libhdfs] chunck_size  the size of the chunck to use for each file.
+

 
 6.0 Interpreting the output
 ---------------------------
 
 6.0 Interpreting the output
 ---------------------------