From: Avritt Rohwer <avritt@google.com>
Date: Wed, 7 May 2025 21:09:13 +0000 (-0700)
Subject: Document expected filename format for s3 http engine.
X-Git-Tag: fio-3.40~18^2
X-Git-Url: https://git.kernel.dk/?a=commitdiff_plain;h=6403937e26e8658aadf8f59e5a955286b0951d5d;p=fio.git

Document expected filename format for s3 http engine.

This caused me some headache, let's add some details on how fio expects
the http_host and file to be formatted.

Signed-off-by: Avritt Rohwer avritt@google.com
---

diff --git a/HOWTO.rst b/HOWTO.rst
index a7e2f693..d15a1fd3 100644
--- a/HOWTO.rst
+++ b/HOWTO.rst
@@ -808,8 +808,11 @@ Target file/device
 	For HTTP and S3 access, specify a valid URL path or S3 key, respectively.
 	A filename for path-style S3 includes a bucket name (:file:`/bucket/k/e.y`)
 	while a virtual-hosted-style S3 filename :file:`/k/e.y` does not because 
-	its bucket name is specified in :option:`http_host`.
-
+	its bucket name is specified in :option:`http_host`. In both cases, the
+	filename should begin with a ``/``. The HTTP engine does not automatically
+	add a leading ``/`` when constructing URLs from :option:`http_host` and
+	:option:`filename`.
+ 	
 	The filename "`-`" is a reserved name, meaning *stdin* or *stdout*.  Which
 	of the two depends on the read/write direction set.
 
diff --git a/README.rst b/README.rst
index e672ce64..c4688921 100644
--- a/README.rst
+++ b/README.rst
@@ -165,6 +165,10 @@ Configure will attempt to determine the target platform automatically.
 It's possible to build fio for ESX as well, use the ``--esx`` switch to
 configure.
 
+The HTTP engine is enabled depending on if the curl and openssl shared libraries
+are detected on the system. For Ubuntu, these packages are libcurl4-openssl-dev
+and libssl-dev.
+
 
 Windows
 ~~~~~~~
diff --git a/examples/http-s3.fio b/examples/http-s3.fio
index c16e89b1..043426ba 100644
--- a/examples/http-s3.fio
+++ b/examples/http-s3.fio
@@ -9,6 +9,13 @@
 # For path-style, prefix the key with the bucket name in the filename
 # so that filename=/bucket-name/k/e.y:
 # https://s3.region-code.amazonaws.com/bucket-name/k/e.y
+#
+# IMPORTANT: filename needs to begin with a '/':
+# FIO formats the url as `"http://%s%s", o->host, object`, so if 
+# filename does not begin with a '/' DNS will fail. For example, if
+# http_host=amazonaws.com and filename=k/1, URL will be set to 
+# amazonaws.comk/1 and curl will attempt to resolve amazonaws.comk
+# which will fail.
 
 # Reference for Virtual-hosted-style vs. Path-style URLs:
 # https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html