Context Navigation

source: trunk/docs/frontends/webapi.rst

Visit:

Last change on this file was 692be00, checked in by Christopher R. Wood <chris@…>, at 2024-05-30T19:44:21Z
Document converting key to DER-encoded urlsafe b64
Property mode set to `100644`
File size: 107.1 KB

Line
1	.. -- coding: utf-8-with-signature --
2
3	==========================
4	The Tahoe REST-ful Web API
5	==========================
6
7	1. `Enabling the web-API port`_
8	2. `Basic Concepts: GET, PUT, DELETE, POST`_
9	3. `URLs`_
10
11	1. `Child Lookup`_
12
13	4. `Slow Operations, Progress, and Cancelling`_
14	5. `Programmatic Operations`_
15
16	1. `Reading a file`_
17	2. `Writing/Uploading a File`_
18	3. `Creating a New Directory`_
19	4. `Getting Information About a File Or Directory (as JSON)`_
20	5. `Attaching an Existing File or Directory by its read- or write-cap`_
21	6. `Adding Multiple Files or Directories to a Parent Directory at Once`_
22	7. `Unlinking a File or Directory`_
23
24	6. `Browser Operations: Human-Oriented Interfaces`_
25
26	1. `Viewing a Directory (as HTML)`_
27	2. `Viewing/Downloading a File`_
28	3. `Getting Information About a File Or Directory (as HTML)`_
29	4. `Creating a Directory`_
30	5. `Uploading a File`_
31	6. `Attaching an Existing File Or Directory (by URI)`_
32	7. `Unlinking a Child`_
33	8. `Renaming a Child`_
34	9. `Relinking ("Moving") a Child`_
35	10. `Other Utilities`_
36	11. `Debugging and Testing Features`_
37
38	7. `Other Useful Pages`_
39	8. `Static Files in /public_html`_
40	9. `Safety and Security Issues -- Names vs. URIs`_
41	10. `Concurrency Issues`_
42	11. `Access Blacklist`_
43
44
45	Enabling the web-API port
46	=========================
47
48	Every Tahoe node is capable of running a built-in HTTP server. To enable
49	this, just write a port number into the "[node]web.port" line of your node's
50	tahoe.cfg file. For example, writing "web.port = 3456" into the "[node]"
51	section of $NODEDIR/tahoe.cfg will cause the node to run a webserver on port
52	3456.
53
54	This string is actually a Twisted "strports" specification, meaning you can
55	get more control over the interface to which the server binds by supplying
56	additional arguments. For more details, see the documentation on
57	`twisted.application.strports`_.
58
59	Writing "tcp:3456:interface=127.0.0.1" into the web.port line does the same
60	but binds to the loopback interface, ensuring that only the programs on the
61	local host can connect. Using "ssl:3456:privateKey=mykey.pem:certKey=cert.pem"
62	runs an SSL server.
63
64	This webport can be set when the node is created by passing a --webport
65	option to the 'tahoe create-node' command. By default, the node listens on
66	port 3456, on the loopback (127.0.0.1) interface.
67
68	.. _twisted.application.strports: https://twistedmatrix.com/documents/current/api/twisted.application.strports.html
69
70
71	Basic Concepts: GET, PUT, DELETE, POST
72	======================================
73
74	As described in :doc:`../architecture`, each file and directory in a
75	Tahoe-LAFS file store is referenced by an identifier that combines the
76	designation of the object with the authority to do something with it (such as
77	read or modify the contents). This identifier is called a "read-cap" or
78	"write-cap", depending upon whether it enables read-only or read-write
79	access. These "caps" are also referred to as URIs (which may be confusing
80	because they are not currently RFC3986_-compliant URIs).
81
82	The Tahoe web-based API is "REST-ful", meaning it implements the concepts of
83	"REpresentational State Transfer": the original scheme by which the World
84	Wide Web was intended to work. Each object (file or directory) is referenced
85	by a URL that includes the read- or write- cap. HTTP methods (GET, PUT, and
86	DELETE) are used to manipulate these objects. You can think of the URL as a
87	noun, and the method as a verb.
88
89	In REST, the GET method is used to retrieve information about an object, or
90	to retrieve some representation of the object itself. When the object is a
91	file, the basic GET method will simply return the contents of that file.
92	Other variations (generally implemented by adding query parameters to the
93	URL) will return information about the object, such as metadata. GET
94	operations are required to have no side-effects.
95
96	PUT is used to upload new objects into the file store, or to replace an
97	existing link or the contents of a mutable file. DELETE is used to unlink
98	objects from directories. Both PUT and DELETE are required to be idempotent:
99	performing the same operation multiple times must have the same side-effects
100	as only performing it once.
101
102	POST is used for more complicated actions that cannot be expressed as a GET,
103	PUT, or DELETE. POST operations can be thought of as a method call: sending
104	some message to the object referenced by the URL. In Tahoe, POST is also used
105	for operations that must be triggered by an HTML form (including upload and
106	unlinking), because otherwise a regular web browser has no way to accomplish
107	these tasks. In general, everything that can be done with a PUT or DELETE can
108	also be done with a POST.
109
110	Tahoe-LAFS' web API is designed for two different kinds of consumer. The
111	first is a program that needs to manipulate the file store. Such programs are
112	expected to use the RESTful interface described above. The second is a human
113	using a standard web browser to work with the file store. This user is
114	presented with a series of HTML pages with links to download files, and forms
115	that use POST actions to upload, rename, and unlink files.
116
117	When an error occurs, the HTTP response code will be set to an appropriate
118	400-series code (like 404 Not Found for an unknown childname, or 400 Bad Request
119	when the parameters to a web-API operation are invalid), and the HTTP response
120	body will usually contain a few lines of explanation as to the cause of the
121	error and possible responses. Unusual exceptions may result in a 500 Internal
122	Server Error as a catch-all, with a default response body containing
123	a Nevow-generated HTML-ized representation of the Python exception stack trace
124	that caused the problem. CLI programs which want to copy the response body to
125	stderr should provide an "Accept: text/plain" header to their requests to get
126	a plain text stack trace instead. If the Accept header contains ``/``, or
127	``text/*``, or text/html (or if there is no Accept header), HTML tracebacks will
128	be generated.
129
130	.. _RFC3986: https://tools.ietf.org/html/rfc3986
131
132
133	URLs
134	====
135
136	Tahoe uses a variety of read- and write- caps to identify files and
137	directories. The most common of these is the "immutable file read-cap", which
138	is used for most uploaded files. These read-caps look like the following::
139
140	URI:CHK:ime6pvkaxuetdfah2p2f35pe54:4btz54xk3tew6nd4y2ojpxj4m6wxjqqlwnztgre6gnjgtucd5r4a:3:10:202
141
142	The next most common is a "directory write-cap", which provides both read and
143	write access to a directory, and look like this::
144
145	URI:DIR2:djrdkfawoqihigoett4g6auz6a:jx5mplfpwexnoqff7y5e4zjus4lidm76dcuarpct7cckorh2dpgq
146
147	There are also "directory read-caps", which start with "URI:DIR2-RO:", and
148	give read-only access to a directory. Finally there are also mutable file
149	read- and write- caps, which start with "URI:SSK", and give access to mutable
150	files.
151
152	(Later versions of Tahoe will make these strings shorter, and will remove the
153	unfortunate colons, which must be escaped when these caps are embedded in
154	URLs.)
155
156	To refer to any Tahoe object through the web API, you simply need to combine
157	a prefix (which indicates the HTTP server to use) with the cap (which
158	indicates which object inside that server to access). Since the default Tahoe
159	webport is 3456, the most common prefix is one that will use a local node
160	listening on this port::
161
162	http://127.0.0.1:3456/uri/ + $CAP
163
164	So, to access the directory named above, the URL would be::
165
166	http://127.0.0.1:3456/uri/URI%3ADIR2%3Adjrdkfawoqihigoett4g6auz6a%3Ajx5mplfpwexnoqff7y5e4zjus4lidm76dcuarpct7cckorh2dpgq/
167
168	(note that the colons in the directory-cap are url-encoded into "%3A"
169	sequences).
170
171	Likewise, to access the file named above, use::
172
173	http://127.0.0.1:3456/uri/URI%3ACHK%3Aime6pvkaxuetdfah2p2f35pe54%3A4btz54xk3tew6nd4y2ojpxj4m6wxjqqlwnztgre6gnjgtucd5r4a%3A3%3A10%3A202
174
175	In the rest of this document, we'll use "$DIRCAP" as shorthand for a read-cap
176	or write-cap that refers to a directory, and "$FILECAP" to abbreviate a cap
177	that refers to a file (whether mutable or immutable). So those URLs above can
178	be abbreviated as::
179
180	http://127.0.0.1:3456/uri/$DIRCAP/
181	http://127.0.0.1:3456/uri/$FILECAP
182
183	The operation summaries below will abbreviate these further, by eliding the
184	server prefix. They will be displayed like this::
185
186	/uri/$DIRCAP/
187	/uri/$FILECAP
188
189	/cap can be used as a synonym for /uri. If interoperability with older web-API
190	servers is required, /uri should be used.
191
192	Child Lookup
193	------------
194
195	Tahoe directories contain named child entries, just like directories in a
196	regular local filesystem. These child entries, called "dirnodes", consist of
197	a name, metadata, a write slot, and a read slot. The write and read slots
198	normally contain a write-cap and read-cap referring to the same object, which
199	can be either a file or a subdirectory. The write slot may be empty
200	(actually, both may be empty, but that is unusual).
201
202	If you have a Tahoe URL that refers to a directory, and want to reference a
203	named child inside it, just append the child name to the URL. For example, if
204	our sample directory contains a file named "welcome.txt", we can refer to
205	that file with::
206
207	http://127.0.0.1:3456/uri/$DIRCAP/welcome.txt
208
209	(or http://127.0.0.1:3456/uri/URI%3ADIR2%3Adjrdkfawoqihigoett4g6auz6a%3Ajx5mplfpwexnoqff7y5e4zjus4lidm76dcuarpct7cckorh2dpgq/welcome.txt)
210
211	Multiple levels of subdirectories can be handled this way::
212
213	http://127.0.0.1:3456/uri/$DIRCAP/tahoe-source/docs/architecture.rst
214
215	In this document, when we need to refer to a URL that references a file using
216	this child-of-some-directory format, we'll use the following string::
217
218	/uri/$DIRCAP/[SUBDIRS../]FILENAME
219
220	The "[SUBDIRS../]" part means that there are zero or more (optional)
221	subdirectory names in the middle of the URL. The "FILENAME" at the end means
222	that this whole URL refers to a file of some sort, rather than to a
223	directory.
224
225	When we need to refer specifically to a directory in this way, we'll write::
226
227	/uri/$DIRCAP/[SUBDIRS../]SUBDIR
228
229
230	Note that all components of pathnames in URLs are required to be UTF-8
231	encoded, so "resume.doc" (with an acute accent on both E's) would be accessed
232	with::
233
234	http://127.0.0.1:3456/uri/$DIRCAP/r%C3%A9sum%C3%A9.doc
235
236	Also note that the filenames inside upload POST forms are interpreted using
237	whatever character set was provided in the conventional '_charset' field, and
238	defaults to UTF-8 if not otherwise specified. The JSON representation of each
239	directory contains native Unicode strings. Tahoe directories are specified to
240	contain Unicode filenames, and cannot contain binary strings that are not
241	representable as such.
242
243	All Tahoe operations that refer to existing files or directories must include
244	a suitable read- or write- cap in the URL: the web-API server won't add one
245	for you. If you don't know the cap, you can't access the file. This allows
246	the security properties of Tahoe caps to be extended across the web-API
247	interface.
248
249
250	Slow Operations, Progress, and Cancelling
251	=========================================
252
253	Certain operations can be expected to take a long time. The "t=deep-check",
254	described below, will recursively visit every file and directory reachable
255	from a given starting point, which can take minutes or even hours for
256	extremely large directory structures. A single long-running HTTP request is a
257	fragile thing: proxies, NAT boxes, browsers, and users may all grow impatient
258	with waiting and give up on the connection.
259
260	For this reason, long-running operations have an "operation handle", which
261	can be used to poll for status/progress messages while the operation
262	proceeds. This handle can also be used to cancel the operation. These handles
263	are created by the client, and passed in as a an "ophandle=" query argument
264	to the POST or PUT request which starts the operation. The following
265	operations can then be used to retrieve status:
266
267	``GET /operations/$HANDLE?output=HTML (with or without t=status)``
268
269	``GET /operations/$HANDLE?output=JSON (same)``
270
271	These two retrieve the current status of the given operation. Each operation
272	presents a different sort of information, but in general the page retrieved
273	will indicate:
274
275	* whether the operation is complete, or if it is still running
276	* how much of the operation is complete, and how much is left, if possible
277
278	Note that the final status output can be quite large: a deep-manifest of a
279	directory structure with 300k directories and 200k unique files is about
280	275MB of JSON, and might take two minutes to generate. For this reason, the
281	full status is not provided until the operation has completed.
282
283	The HTML form will include a meta-refresh tag, which will cause a regular
284	web browser to reload the status page about 60 seconds later. This tag will
285	be removed once the operation has completed.
286
287	There may be more status information available under
288	/operations/$HANDLE/$ETC : i.e., the handle forms the root of a URL space.
289
290	``POST /operations/$HANDLE?t=cancel``
291
292	This terminates the operation, and returns an HTML page explaining what was
293	cancelled. If the operation handle has already expired (see below), this
294	POST will return a 404, which indicates that the operation is no longer
295	running (either it was completed or terminated). The response body will be
296	the same as a GET /operations/$HANDLE on this operation handle, and the
297	handle will be expired immediately afterwards.
298
299	The operation handle will eventually expire, to avoid consuming an unbounded
300	amount of memory. The handle's time-to-live can be reset at any time, by
301	passing a retain-for= argument (with a count of seconds) to either the
302	initial POST that starts the operation, or the subsequent GET request which
303	asks about the operation. For example, if a 'GET
304	/operations/$HANDLE?output=JSON&retain-for=600' query is performed, the
305	handle will remain active for 600 seconds (10 minutes) after the GET was
306	received.
307
308	In addition, if the GET includes a release-after-complete=True argument, and
309	the operation has completed, the operation handle will be released
310	immediately.
311
312	If a retain-for= argument is not used, the default handle lifetimes are:
313
314	* handles will remain valid at least until their operation finishes
315	* uncollected handles for finished operations (i.e. handles for
316	operations that have finished but for which the GET page has not been
317	accessed since completion) will remain valid for four days, or for
318	the total time consumed by the operation, whichever is greater.
319	* collected handles (i.e. the GET page has been retrieved at least once
320	since the operation completed) will remain valid for one day.
321
322	Many "slow" operations can begin to use unacceptable amounts of memory when
323	operating on large directory structures. The memory usage increases when the
324	ophandle is polled, as the results must be copied into a JSON string, sent
325	over the wire, then parsed by a client. So, as an alternative, many "slow"
326	operations have streaming equivalents. These equivalents do not use operation
327	handles. Instead, they emit line-oriented status results immediately. Client
328	code can cancel the operation by simply closing the HTTP connection.
329
330
331	Programmatic Operations
332	=======================
333
334	Now that we know how to build URLs that refer to files and directories in a
335	Tahoe-LAFS file store, what sorts of operations can we do with those URLs?
336	This section contains a catalog of GET, PUT, DELETE, and POST operations that
337	can be performed on these URLs. This set of operations are aimed at programs
338	that use HTTP to communicate with a Tahoe node. A later section describes
339	operations that are intended for web browsers.
340
341
342	Reading a File
343	--------------
344
345	``GET /uri/$FILECAP``
346
347	``GET /uri/$DIRCAP/[SUBDIRS../]FILENAME``
348
349	This will retrieve the contents of the given file. The HTTP response body
350	will contain the sequence of bytes that make up the file.
351
352	The "Range:" header can be used to restrict which portions of the file are
353	returned (see RFC 2616 section 14.35.1 "Byte Ranges"), however Tahoe only
354	supports a single "bytes" range and never provides a
355	``multipart/byteranges`` response. An attempt to begin a read past the end
356	of the file will provoke a 416 Requested Range Not Satisfiable error, but
357	normal overruns (reads which start at the beginning or middle and go beyond
358	the end) are simply truncated.
359
360	To view files in a web browser, you may want more control over the
361	Content-Type and Content-Disposition headers. Please see the next section
362	"Browser Operations", for details on how to modify these URLs for that
363	purpose.
364
365
366	Writing/Uploading a File
367	------------------------
368
369	``PUT /uri/$FILECAP``
370
371	``PUT /uri/$DIRCAP/[SUBDIRS../]FILENAME``
372
373	Upload a file, using the data from the HTTP request body, and add whatever
374	child links and subdirectories are necessary to make the file available at
375	the given location. Once this operation succeeds, a GET on the same URL will
376	retrieve the same contents that were just uploaded. This will create any
377	necessary intermediate subdirectories.
378
379	To use the /uri/$FILECAP form, $FILECAP must be a write-cap for a mutable file.
380
381	In the /uri/$DIRCAP/[SUBDIRS../]FILENAME form, if the target file is a
382	writeable mutable file, that file's contents will be overwritten
383	in-place. If it is a read-cap for a mutable file, an error will occur.
384	If it is an immutable file, the old file will be discarded, and a new
385	one will be put in its place. If the target file is a writable mutable
386	file, you may also specify an "offset" parameter -- a byte offset that
387	determines where in the mutable file the data from the HTTP request
388	body is placed. This operation is relatively efficient for MDMF mutable
389	files, and is relatively inefficient (but still supported) for SDMF
390	mutable files. If no offset parameter is specified, then the entire
391	file is replaced with the data from the HTTP request body. For an
392	immutable file, the "offset" parameter is not valid.
393
394	When creating a new file, you can control the type of file created by
395	specifying a format= argument in the query string. format=MDMF creates an
396	MDMF mutable file. format=SDMF creates an SDMF mutable file. format=CHK
397	creates an immutable file. The value of the format argument is
398	case-insensitive. If no format is specified, the newly-created file will be
399	immutable (but see below).
400
401	For compatibility with previous versions of Tahoe-LAFS, the web-API will
402	also accept a mutable=true argument in the query string. If mutable=true is
403	given, then the new file will be mutable, and its format will be the default
404	mutable file format, as configured by the [client]mutable.format option of
405	tahoe.cfg on the Tahoe-LAFS node hosting the webapi server. Use of
406	mutable=true is discouraged; new code should use format= instead of
407	mutable=true (unless it needs to be compatible with web-API servers older
408	than v1.9.0). If neither format= nor mutable=true are given, the
409	newly-created file will be immutable.
410
411	This returns the file-cap of the resulting file. If a new file was created
412	by this method, the HTTP response code (as dictated by rfc2616) will be set
413	to 201 CREATED. If an existing file was replaced or modified, the response
414	code will be 200 OK.
415
416	Note that the 'curl -T localfile http://127.0.0.1:3456/uri/$DIRCAP/foo.txt'
417	command can be used to invoke this operation.
418
419	``PUT /uri``
420
421	This uploads a file, and produces a file-cap for the contents, but does not
422	attach the file into the file store. No directories will be modified by
423	this operation. The file-cap is returned as the body of the HTTP response.
424
425	This method accepts format= and mutable=true as query string arguments, and
426	interprets those arguments in the same way as the linked forms of PUT
427	described immediately above.
428
429	Creating a New Directory
430	------------------------
431
432	``POST /uri?t=mkdir``
433
434	``PUT /uri?t=mkdir``
435
436	Create a new empty directory and return its write-cap as the HTTP response
437	body. This does not make the newly created directory visible from the
438	file store. The "PUT" operation is provided for backwards compatibility:
439	new code should use POST.
440
441	This supports a format= argument in the query string. The format=
442	argument, if specified, controls the format of the directory. format=MDMF
443	indicates that the directory should be stored as an MDMF file; format=SDMF
444	indicates that the directory should be stored as an SDMF file. The value of
445	the format= argument is case-insensitive. If no format= argument is
446	given, the directory's format is determined by the default mutable file
447	format, as configured on the Tahoe-LAFS node responding to the request.
448
449	In addition, an optional "private-key=" argument is supported which, if given,
450	specifies the underlying signing key to be used when creating the directory.
451	This value must be a DER-encoded 2048-bit RSA private key in urlsafe base64
452	encoding. (To convert an existing PEM-encoded RSA key file into the format
453	required, the following commands may be used -- assuming a modern UNIX-like
454	environment with common tools already installed:
455	``openssl rsa -in key.pem -outform der \| base64 -w 0 -i - \| tr '+/' '-_'``)
456
457	Because this key can be used to derive the write capability for the
458	associated directory, additional care should be taken to ensure that the key is
459	unique, that it is kept confidential, and that it was derived from an
460	appropriate (high-entropy) source of randomness. If this argument is omitted
461	(the default behavior), Tahoe-LAFS will generate an appropriate signing key
462	using the underlying operating system's source of entropy.
463
464	``POST /uri?t=mkdir-with-children``
465
466	Create a new directory, populated with a set of child nodes, and return its
467	write-cap as the HTTP response body. The new directory is not attached to
468	any other directory: the returned write-cap is the only reference to it.
469
470	The format of the directory can be controlled with the format= argument in
471	the query string and a signing key can be specified with the private-key=
472	argument, as described above.
473
474	Initial children are provided as the body of the POST form (this is more
475	efficient than doing separate mkdir and set_children operations). If the
476	body is empty, the new directory will be empty. If not empty, the body will
477	be interpreted as a UTF-8 JSON-encoded dictionary of children with which the
478	new directory should be populated, using the same format as would be
479	returned in the 'children' value of the t=json GET request, described below.
480	Each dictionary key should be a child name, and each value should be a list
481	of [TYPE, PROPDICT], where PROPDICT contains "rw_uri", "ro_uri", and
482	"metadata" keys (all others are ignored). For example, the PUT request body
483	could be::
484
485	{
486	"Fran\u00e7ais": [ "filenode", {
487	"ro_uri": "URI:CHK:...",
488	"metadata": {
489	"ctime": 1202777696.7564139,
490	"mtime": 1202777696.7564139,
491	"tahoe": {
492	"linkcrtime": 1202777696.7564139,
493	"linkmotime": 1202777696.7564139
494	} } } ],
495	"subdir": [ "dirnode", {
496	"rw_uri": "URI:DIR2:...",
497	"ro_uri": "URI:DIR2-RO:...",
498	"metadata": {
499	"ctime": 1202778102.7589991,
500	"mtime": 1202778111.2160511,
501	"tahoe": {
502	"linkcrtime": 1202777696.7564139,
503	"linkmotime": 1202777696.7564139
504	} } } ]
505	}
506
507	For forward-compatibility, a mutable directory can also contain caps in
508	a format that is unknown to the web-API server. When such caps are retrieved
509	from a mutable directory in a "ro_uri" field, they will be prefixed with
510	the string "ro.", indicating that they must not be decoded without
511	checking that they are read-only. The "ro." prefix must not be stripped
512	off without performing this check. (Future versions of the web-API server
513	will perform it where necessary.)
514
515	If both the "rw_uri" and "ro_uri" fields are present in a given PROPDICT,
516	and the web-API server recognizes the rw_uri as a write cap, then it will
517	reset the ro_uri to the corresponding read cap and discard the original
518	contents of ro_uri (in order to ensure that the two caps correspond to the
519	same object and that the ro_uri is in fact read-only). However this may not
520	happen for caps in a format unknown to the web-API server. Therefore, when
521	writing a directory the web-API client should ensure that the contents
522	of "rw_uri" and "ro_uri" for a given PROPDICT are a consistent
523	(write cap, read cap) pair if possible. If the web-API client only has
524	one cap and does not know whether it is a write cap or read cap, then
525	it is acceptable to set "rw_uri" to that cap and omit "ro_uri". The
526	client must not put a write cap into a "ro_uri" field.
527
528	The metadata may have a "no-write" field. If this is set to true in the
529	metadata of a link, it will not be possible to open that link for writing
530	via the SFTP frontend; see :doc:`FTP-and-SFTP` for details. Also, if the
531	"no-write" field is set to true in the metadata of a link to a mutable
532	child, it will cause the link to be diminished to read-only.
533
534	Note that the web-API-using client application must not provide the
535	"Content-Type: multipart/form-data" header that usually accompanies HTML
536	form submissions, since the body is not formatted this way. Doing so will
537	cause a server error as the lower-level code misparses the request body.
538
539	Child file names should each be expressed as a Unicode string, then used as
540	keys of the dictionary. The dictionary should then be converted into JSON,
541	and the resulting string encoded into UTF-8. This UTF-8 bytestring should
542	then be used as the POST body.
543
544	``POST /uri?t=mkdir-immutable``
545
546	Like t=mkdir-with-children above, but the new directory will be
547	deep-immutable. This means that the directory itself is immutable, and that
548	it can only contain objects that are treated as being deep-immutable, like
549	immutable files, literal files, and deep-immutable directories.
550
551	For forward-compatibility, a deep-immutable directory can also contain caps
552	in a format that is unknown to the web-API server. When such caps are retrieved
553	from a deep-immutable directory in a "ro_uri" field, they will be prefixed
554	with the string "imm.", indicating that they must not be decoded without
555	checking that they are immutable. The "imm." prefix must not be stripped
556	off without performing this check. (Future versions of the web-API server
557	will perform it where necessary.)
558
559	The cap for each child may be given either in the "rw_uri" or "ro_uri"
560	field of the PROPDICT (not both). If a cap is given in the "rw_uri" field,
561	then the web-API server will check that it is an immutable read-cap of a
562	known format, and give an error if it is not. If a cap is given in the
563	"ro_uri" field, then the web-API server will still check whether known
564	caps are immutable, but for unknown caps it will simply assume that the
565	cap can be stored, as described above. Note that an attacker would be
566	able to store any cap in an immutable directory, so this check when
567	creating the directory is only to help non-malicious clients to avoid
568	accidentally giving away more authority than intended.
569
570	A non-empty request body is mandatory, since after the directory is created,
571	it will not be possible to add more children to it.
572
573	``POST /uri/$DIRCAP/[SUBDIRS../]SUBDIR?t=mkdir``
574
575	``PUT /uri/$DIRCAP/[SUBDIRS../]SUBDIR?t=mkdir``
576
577	Create new directories as necessary to make sure that the named target
578	($DIRCAP/SUBDIRS../SUBDIR) is a directory. This will create additional
579	intermediate mutable directories as necessary. If the named target directory
580	already exists, this will make no changes to it.
581
582	If the final directory is created, it will be empty.
583
584	This accepts a format= argument in the query string, which controls the
585	format of the named target directory, if it does not already exist. format=
586	is interpreted in the same way as in the POST /uri?t=mkdir form. Note that
587	format= only controls the format of the named target directory;
588	intermediate directories, if created, are created based on the default
589	mutable type, as configured on the Tahoe-LAFS server responding to the
590	request.
591
592	This operation will return an error if a blocking file is present at any of
593	the parent names, preventing the server from creating the necessary parent
594	directory; or if it would require changing an immutable directory.
595
596	The write-cap of the new directory will be returned as the HTTP response
597	body.
598
599	``POST /uri/$DIRCAP/[SUBDIRS../]SUBDIR?t=mkdir-with-children``
600
601	Like /uri?t=mkdir-with-children, but the final directory is created as a
602	child of an existing mutable directory. This will create additional
603	intermediate mutable directories as necessary. If the final directory is
604	created, it will be populated with initial children from the POST request
605	body, as described above.
606
607	This accepts a format= argument in the query string, which controls the
608	format of the target directory, if the target directory is created as part
609	of the operation. format= is interpreted in the same way as in the POST/
610	uri?t=mkdir-with-children operation. Note that format= only controls the
611	format of the named target directory; intermediate directories, if created,
612	are created using the default mutable type setting, as configured on the
613	Tahoe-LAFS server responding to the request.
614
615	This operation will return an error if a blocking file is present at any of
616	the parent names, preventing the server from creating the necessary parent
617	directory; or if it would require changing an immutable directory; or if
618	the immediate parent directory already has a a child named SUBDIR.
619
620	``POST /uri/$DIRCAP/[SUBDIRS../]SUBDIR?t=mkdir-immutable``
621
622	Like /uri?t=mkdir-immutable, but the final directory is created as a child
623	of an existing mutable directory. The final directory will be deep-immutable,
624	and will be populated with the children specified as a JSON dictionary in
625	the POST request body.
626
627	In Tahoe 1.6 this operation creates intermediate mutable directories if
628	necessary, but that behaviour should not be relied on; see ticket #920.
629
630	This operation will return an error if the parent directory is immutable,
631	or already has a child named SUBDIR.
632
633	``POST /uri/$DIRCAP/[SUBDIRS../]?t=mkdir&name=NAME``
634
635	Create a new empty mutable directory and attach it to the given existing
636	directory. This will create additional intermediate directories as necessary.
637
638	This accepts a format= argument in the query string, which controls the
639	format of the named target directory, if it does not already exist. format=
640	is interpreted in the same way as in the POST /uri?t=mkdir form. Note that
641	format= only controls the format of the named target directory;
642	intermediate directories, if created, are created based on the default
643	mutable type, as configured on the Tahoe-LAFS server responding to the
644	request.
645
646	This operation will return an error if a blocking file is present at any of
647	the parent names, preventing the server from creating the necessary parent
648	directory, or if it would require changing any immutable directory.
649
650	The URL of this operation points to the parent of the bottommost new directory,
651	whereas the /uri/$DIRCAP/[SUBDIRS../]SUBDIR?t=mkdir operation above has a URL
652	that points directly to the bottommost new directory.
653
654	``POST /uri/$DIRCAP/[SUBDIRS../]?t=mkdir-with-children&name=NAME``
655
656	Like /uri/$DIRCAP/[SUBDIRS../]?t=mkdir&name=NAME, but the new directory will
657	be populated with initial children via the POST request body. This command
658	will create additional intermediate mutable directories as necessary.
659
660	This accepts a format= argument in the query string, which controls the
661	format of the target directory, if the target directory is created as part
662	of the operation. format= is interpreted in the same way as in the POST/
663	uri?t=mkdir-with-children operation. Note that format= only controls the
664	format of the named target directory; intermediate directories, if created,
665	are created using the default mutable type setting, as configured on the
666	Tahoe-LAFS server responding to the request.
667
668	This operation will return an error if a blocking file is present at any of
669	the parent names, preventing the server from creating the necessary parent
670	directory; or if it would require changing an immutable directory; or if
671	the immediate parent directory already has a a child named NAME.
672
673	Note that the name= argument must be passed as a queryarg, because the POST
674	request body is used for the initial children JSON.
675
676	``POST /uri/$DIRCAP/[SUBDIRS../]?t=mkdir-immutable&name=NAME``
677
678	Like /uri/$DIRCAP/[SUBDIRS../]?t=mkdir-with-children&name=NAME, but the
679	final directory will be deep-immutable. The children are specified as a
680	JSON dictionary in the POST request body. Again, the name= argument must be
681	passed as a queryarg.
682
683	In Tahoe 1.6 this operation creates intermediate mutable directories if
684	necessary, but that behaviour should not be relied on; see ticket #920.
685
686	This operation will return an error if the parent directory is immutable,
687	or already has a child named NAME.
688
689
690	Getting Information About a File Or Directory (as JSON)
691	-------------------------------------------------------
692
693	``GET /uri/$FILECAP?t=json``
694
695	``GET /uri/$DIRCAP?t=json``
696
697	``GET /uri/$DIRCAP/[SUBDIRS../]SUBDIR?t=json``
698
699	``GET /uri/$DIRCAP/[SUBDIRS../]FILENAME?t=json``
700
701	This returns a machine-parseable JSON-encoded description of the given
702	object. The JSON always contains a list, and the first element of the list is
703	always a flag that indicates whether the referenced object is a file or a
704	directory. If it is a capability to a file, then the information includes
705	file size and URI, like this::
706
707	GET /uri/$FILECAP?t=json :
708
709	[ "filenode", {
710	"ro_uri": file_uri,
711	"verify_uri": verify_uri,
712	"size": bytes,
713	"mutable": false,
714	"format": "CHK"
715	} ]
716
717	If it is a capability to a directory followed by a path from that directory
718	to a file, then the information also includes metadata from the link to the
719	file in the parent directory, like this::
720
721	GET /uri/$DIRCAP/[SUBDIRS../]FILENAME?t=json
722
723	[ "filenode", {
724	"ro_uri": file_uri,
725	"verify_uri": verify_uri,
726	"size": bytes,
727	"mutable": false,
728	"format": "CHK",
729	"metadata": {
730	"ctime": 1202777696.7564139,
731	"mtime": 1202777696.7564139,
732	"tahoe": {
733	"linkcrtime": 1202777696.7564139,
734	"linkmotime": 1202777696.7564139
735	} } } ]
736
737	If it is a directory, then it includes information about the children of
738	this directory, as a mapping from child name to a set of data about the
739	child (the same data that would appear in a corresponding GET?t=json of the
740	child itself). The child entries also include metadata about each child,
741	including link-creation- and link-change- timestamps. The output looks like
742	this::
743
744	GET /uri/$DIRCAP?t=json :
745	GET /uri/$DIRCAP/[SUBDIRS../]SUBDIR?t=json :
746
747	[ "dirnode", {
748	"rw_uri": read_write_uri,
749	"ro_uri": read_only_uri,
750	"verify_uri": verify_uri,
751	"mutable": true,
752	"format": "SDMF",
753	"children": {
754	"foo.txt": [ "filenode",
755	{
756	"ro_uri": uri,
757	"size": bytes,
758	"metadata": {
759	"ctime": 1202777696.7564139,
760	"mtime": 1202777696.7564139,
761	"tahoe": {
762	"linkcrtime": 1202777696.7564139,
763	"linkmotime": 1202777696.7564139
764	} } } ],
765	"subdir": [ "dirnode",
766	{
767	"rw_uri": rwuri,
768	"ro_uri": rouri,
769	"metadata": {
770	"ctime": 1202778102.7589991,
771	"mtime": 1202778111.2160511,
772	"tahoe": {
773	"linkcrtime": 1202777696.7564139,
774	"linkmotime": 1202777696.7564139
775	} } } ]
776	} } ]
777
778	In the above example, note how 'children' is a dictionary in which the keys
779	are child names and the values depend upon whether the child is a file or a
780	directory. The value is mostly the same as the JSON representation of the
781	child object (except that directories do not recurse -- the "children"
782	entry of the child is omitted, and the directory view includes the metadata
783	that is stored on the directory edge).
784
785	The rw_uri field will be present in the information about a directory
786	if and only if you have read-write access to that directory. The verify_uri
787	field will be present if and only if the object has a verify-cap
788	(non-distributed LIT files do not have verify-caps).
789
790	If the cap is of an unknown format, then the file size and verify_uri will
791	not be available::
792
793	GET /uri/$UNKNOWNCAP?t=json :
794
795	[ "unknown", {
796	"ro_uri": unknown_read_uri
797	} ]
798
799	GET /uri/$DIRCAP/[SUBDIRS../]UNKNOWNCHILDNAME?t=json :
800
801	[ "unknown", {
802	"rw_uri": unknown_write_uri,
803	"ro_uri": unknown_read_uri,
804	"mutable": true,
805	"metadata": {
806	"ctime": 1202777696.7564139,
807	"mtime": 1202777696.7564139,
808	"tahoe": {
809	"linkcrtime": 1202777696.7564139,
810	"linkmotime": 1202777696.7564139
811	} } } ]
812
813	As in the case of file nodes, the metadata will only be present when the
814	capability is to a directory followed by a path. The "mutable" field is also
815	not always present; when it is absent, the mutability of the object is not
816	known.
817
818	About the metadata
819	``````````````````
820
821	The value of the 'tahoe':'linkmotime' key is updated whenever a link to a
822	child is set. The value of the 'tahoe':'linkcrtime' key is updated whenever
823	a link to a child is created -- i.e. when there was not previously a link
824	under that name.
825
826	Note however, that if the edge in the Tahoe-LAFS file store points to a
827	mutable file and the contents of that mutable file is changed, then the
828	'tahoe':'linkmotime' value on that edge will not be updated, since the
829	edge itself wasn't updated -- only the mutable file was.
830
831	The timestamps are represented as a number of seconds since the UNIX epoch
832	(1970-01-01 00:00:00 UTC), with leap seconds not being counted in the long
833	term.
834
835	In Tahoe earlier than v1.4.0, 'mtime' and 'ctime' keys were populated
836	instead of the 'tahoe':'linkmotime' and 'tahoe':'linkcrtime' keys. Starting
837	in Tahoe v1.4.0, the 'linkmotime'/'linkcrtime' keys in the 'tahoe' sub-dict
838	are populated. However, prior to Tahoe v1.7beta, a bug caused the 'tahoe'
839	sub-dict to be deleted by web-API requests in which new metadata is
840	specified, and not to be added to existing child links that lack it.
841
842	From Tahoe v1.7.0 onward, the 'mtime' and 'ctime' fields are no longer
843	populated or updated (see ticket #924), except by "tahoe backup" as
844	explained below. For backward compatibility, when an existing link is
845	updated and 'tahoe':'linkcrtime' is not present in the previous metadata
846	but 'ctime' is, the old value of 'ctime' is used as the new value of
847	'tahoe':'linkcrtime'.
848
849	The reason we added the new fields in Tahoe v1.4.0 is that there is a
850	"set_children" API (described below) which you can use to overwrite the
851	values of the 'mtime'/'ctime' pair, and this API is used by the
852	"tahoe backup" command (in Tahoe v1.3.0 and later) to set the 'mtime' and
853	'ctime' values when backing up files from a local filesystem into the
854	Tahoe-LAFS file store. As of Tahoe v1.4.0, the set_children API cannot be
855	used to set anything under the 'tahoe' key of the metadata dict -- if you
856	include 'tahoe' keys in your 'metadata' arguments then it will silently
857	ignore those keys.
858
859	Therefore, if the 'tahoe' sub-dict is present, you can rely on the
860	'linkcrtime' and 'linkmotime' values therein to have the semantics described
861	above. (This is assuming that only official Tahoe clients have been used to
862	write those links, and that their system clocks were set to what you expected
863	-- there is nothing preventing someone from editing their Tahoe client or
864	writing their own Tahoe client which would overwrite those values however
865	they like, and there is nothing to constrain their system clock from taking
866	any value.)
867
868	When an edge is created or updated by "tahoe backup", the 'mtime' and
869	'ctime' keys on that edge are set as follows:
870
871	* 'mtime' is set to the timestamp read from the local filesystem for the
872	"mtime" of the local file in question, which means the last time the
873	contents of that file were changed.
874
875	* On Windows, 'ctime' is set to the creation timestamp for the file
876	read from the local filesystem. On other platforms, 'ctime' is set to
877	the UNIX "ctime" of the local file, which means the last time that
878	either the contents or the metadata of the local file was changed.
879
880	There are several ways that the 'ctime' field could be confusing:
881
882	1. You might be confused about whether it reflects the time of the creation
883	of a link in the Tahoe-LAFS file store (by a version of Tahoe < v1.7.0)
884	or a timestamp copied in by "tahoe backup" from a local filesystem.
885
886	2. You might be confused about whether it is a copy of the file creation
887	time (if "tahoe backup" was run on a Windows system) or of the last
888	contents-or-metadata change (if "tahoe backup" was run on a different
889	operating system).
890
891	3. You might be confused by the fact that changing the contents of a
892	mutable file in Tahoe doesn't have any effect on any links pointing at
893	that file in any directories, although "tahoe backup" sets the link
894	'ctime'/'mtime' to reflect timestamps about the local file corresponding
895	to the Tahoe file to which the link points.
896
897	4. Also, quite apart from Tahoe, you might be confused about the meaning
898	of the "ctime" in UNIX local filesystems, which people sometimes think
899	means file creation time, but which actually means, in UNIX local
900	filesystems, the most recent time that the file contents or the file
901	metadata (such as owner, permission bits, extended attributes, etc.)
902	has changed. Note that although "ctime" does not mean file creation time
903	in UNIX, links created by a version of Tahoe prior to v1.7.0, and never
904	written by "tahoe backup", will have 'ctime' set to the link creation
905	time.
906
907
908	Attaching an Existing File or Directory by its read- or write-cap
909	-----------------------------------------------------------------
910
911	``PUT /uri/$DIRCAP/[SUBDIRS../]CHILDNAME?t=uri``
912
913	This attaches a child object (either a file or directory) to a specified
914	location in the Tahoe-LAFS file store. The child object is referenced by its
915	read- or write- cap, as provided in the HTTP request body. This will create
916	intermediate directories as necessary.
917
918	This is similar to a UNIX hardlink: by referencing a previously-uploaded file
919	(or previously-created directory) instead of uploading/creating a new one,
920	you can create two references to the same object.
921
922	The read- or write- cap of the child is provided in the body of the HTTP
923	request, and this same cap is returned in the response body.
924
925	The default behavior is to overwrite any existing object at the same
926	location. To prevent this (and make the operation return an error instead
927	of overwriting), add a "replace=false" argument, as "?t=uri&replace=false".
928	With replace=false, this operation will return an HTTP 409 "Conflict" error
929	if there is already an object at the given location, rather than
930	overwriting the existing object. To allow the operation to overwrite a
931	file, but return an error when trying to overwrite a directory, use
932	"replace=only-files" (this behavior is closer to the traditional UNIX "mv"
933	command). Note that "true", "t", and "1" are all synonyms for "True", and
934	"false", "f", and "0" are synonyms for "False", and the parameter is
935	case-insensitive.
936
937	Note that this operation does not take its child cap in the form of
938	separate "rw_uri" and "ro_uri" fields. Therefore, it cannot accept a
939	child cap in a format unknown to the web-API server, unless its URI
940	starts with "ro." or "imm.". This restriction is necessary because the
941	server is not able to attenuate an unknown write cap to a read cap.
942	Unknown URIs starting with "ro." or "imm.", on the other hand, are
943	assumed to represent read caps. The client should not prefix a write
944	cap with "ro." or "imm." and pass it to this operation, since that
945	would result in granting the cap's write authority to holders of the
946	directory read cap.
947
948
949	Adding Multiple Files or Directories to a Parent Directory at Once
950	------------------------------------------------------------------
951
952	``POST /uri/$DIRCAP/[SUBDIRS..]?t=set_children``
953
954	``POST /uri/$DIRCAP/[SUBDIRS..]?t=set-children`` (Tahoe >= v1.6)
955
956	This command adds multiple children to a directory in a single operation.
957	It reads the request body and interprets it as a JSON-encoded description
958	of the child names and read/write-caps that should be added.
959
960	The body should be a JSON-encoded dictionary, in the same format as the
961	"children" value returned by the "GET /uri/$DIRCAP?t=json" operation
962	described above. In this format, each key is a child names, and the
963	corresponding value is a tuple of (type, childinfo). "type" is ignored, and
964	"childinfo" is a dictionary that contains "rw_uri", "ro_uri", and
965	"metadata" keys. You can take the output of "GET /uri/$DIRCAP1?t=json" and
966	use it as the input to "POST /uri/$DIRCAP2?t=set_children" to make DIR2
967	look very much like DIR1 (except for any existing children of DIR2 that
968	were not overwritten, and any existing "tahoe" metadata keys as described
969	below).
970
971	When the set_children request contains a child name that already exists in
972	the target directory, this command defaults to overwriting that child with
973	the new value (both child cap and metadata, but if the JSON data does not
974	contain a "metadata" key, the old child's metadata is preserved). The
975	command takes a boolean "overwrite=" query argument to control this
976	behavior. If you use "?t=set_children&overwrite=false", then an attempt to
977	replace an existing child will instead cause an error.
978
979	Any "tahoe" key in the new child's "metadata" value is ignored. Any
980	existing "tahoe" metadata is preserved. The metadata["tahoe"] value is
981	reserved for metadata generated by the tahoe node itself. The only two keys
982	currently placed here are "linkcrtime" and "linkmotime". For details, see
983	the section above entitled "Getting Information About a File Or Directory (as
984	JSON)", in the "About the metadata" subsection.
985
986	Note that this command was introduced with the name "set_children", which
987	uses an underscore rather than a hyphen as other multi-word command names
988	do. The variant with a hyphen is now accepted, but clients that desire
989	backward compatibility should continue to use "set_children".
990
991
992	Unlinking a File or Directory
993	-----------------------------
994
995	``DELETE /uri/$DIRCAP/[SUBDIRS../]CHILDNAME``
996
997	This removes the given name from its parent directory. CHILDNAME is the
998	name to be removed, and $DIRCAP/SUBDIRS.. indicates the directory that will
999	be modified.
1000
1001	Note that this does not actually delete the file or directory that the name
1002	points to from the tahoe grid -- it only unlinks the named reference from
1003	this directory. If there are other names in this directory or in other
1004	directories that point to the resource, then it will remain accessible
1005	through those paths. Even if all names pointing to this object are removed
1006	from their parent directories, then someone with possession of its read-cap
1007	can continue to access the object through that cap.
1008
1009	The object will only become completely unreachable once 1: there are no
1010	reachable directories that reference it, and 2: nobody is holding a read-
1011	or write- cap to the object. (This behavior is very similar to the way
1012	hardlinks and anonymous files work in traditional UNIX filesystems).
1013
1014	This operation will not modify more than a single directory. Intermediate
1015	directories which were implicitly created by PUT or POST methods will not
1016	be automatically removed by DELETE.
1017
1018	This method returns the file- or directory- cap of the object that was just
1019	removed.
1020
1021
1022	Browser Operations: Human-oriented interfaces
1023	=============================================
1024
1025	This section describes the HTTP operations that provide support for humans
1026	running a web browser. Most of these operations use HTML forms that use POST
1027	to drive the Tahoe-LAFS node. This section is intended for HTML authors who
1028	want to write web pages containing user interfaces for manipulating the
1029	Tahoe-LAFS file store.
1030
1031	Note that for all POST operations, the arguments listed can be provided
1032	either as URL query arguments or as form body fields. URL query arguments are
1033	separated from the main URL by "?", and from each other by "&". For example,
1034	"POST /uri/$DIRCAP?t=upload&mutable=true". Form body fields are usually
1035	specified by using <input type="hidden"> elements. For clarity, the
1036	descriptions below display the most significant arguments as URL query args.
1037
1038
1039	Viewing a Directory (as HTML)
1040	-----------------------------
1041
1042	``GET /uri/$DIRCAP/[SUBDIRS../]``
1043
1044	This returns an HTML page, intended to be displayed to a human by a web
1045	browser, which contains HREF links to all files and directories reachable
1046	from this directory. These HREF links do not have a t= argument, meaning
1047	that a human who follows them will get pages also meant for a human. It also
1048	contains forms to upload new files, and to unlink files and directories
1049	from their parent directory. Those forms use POST methods to do their job.
1050
1051
1052	Viewing/Downloading a File
1053	--------------------------
1054
1055	``GET /uri/$FILECAP``
1056
1057	``GET /uri/$DIRCAP/[SUBDIRS../]FILENAME``
1058
1059	``GET /named/$FILECAP/FILENAME``
1060
1061	These will retrieve the contents of the given file. The HTTP response body
1062	will contain the sequence of bytes that make up the file.
1063
1064	The ``/named/`` form is an alternative to ``/uri/$FILECAP`` which makes it
1065	easier to get the correct filename. The Tahoe server will provide the
1066	contents of the given file, with a Content-Type header derived from the
1067	given filename. This form is used to get browsers to use the "Save Link As"
1068	feature correctly, and also helps command-line tools like "wget" and "curl"
1069	use the right filename. Note that this form can only be used with file
1070	caps; it is an error to use a directory cap after the /named/ prefix.
1071
1072	URLs may also use /file/$FILECAP/FILENAME as a synonym for
1073	/named/$FILECAP/FILENAME. The use of "/file/" is deprecated in favor of
1074	"/named/" and support for "/file/" will be removed in a future release of
1075	Tahoe-LAFS.
1076
1077	If you use the first form (``/uri/$FILECAP``) and want the HTTP response to
1078	include a useful Content-Type header, add a "filename=foo" query argument,
1079	like "GET /uri/$FILECAP?filename=foo.jpg". The bare "GET /uri/$FILECAP" does
1080	not give the Tahoe node enough information to determine a Content-Type
1081	(since LAFS immutable files are merely sequences of bytes, not typed and
1082	named file objects).
1083
1084	If the URL has both filename= and "save=true" in the query arguments, then
1085	the server to add a "Content-Disposition: attachment" header, along with a
1086	filename= parameter. When a user clicks on such a link, most browsers will
1087	offer to let the user save the file instead of displaying it inline (indeed,
1088	most browsers will refuse to display it inline). "true", "t", "1", and other
1089	case-insensitive equivalents are all treated the same.
1090
1091	Character-set handling in URLs and HTTP headers is a :ref:`dubious
1092	art<urls-and-utf8>`. For maximum compatibility, Tahoe simply copies the
1093	bytes from the filename= argument into the Content-Disposition header's
1094	filename= parameter, without trying to interpret them in any particular way.
1095
1096
1097	Getting Information About a File Or Directory (as HTML)
1098	-------------------------------------------------------
1099
1100	``GET /uri/$FILECAP?t=info``
1101
1102	``GET /uri/$DIRCAP/?t=info``
1103
1104	``GET /uri/$DIRCAP/[SUBDIRS../]SUBDIR/?t=info``
1105
1106	``GET /uri/$DIRCAP/[SUBDIRS../]FILENAME?t=info``
1107
1108	This returns a human-oriented HTML page with more detail about the selected
1109	file or directory object. This page contains the following items:
1110
1111	* object size
1112	* storage index
1113	* JSON representation
1114	* raw contents (text/plain)
1115	* access caps (URIs): verify-cap, read-cap, write-cap (for mutable objects)
1116	* check/verify/repair form
1117	* deep-check/deep-size/deep-stats/manifest (for directories)
1118	* replace-contents form (for mutable files)
1119
1120
1121	Creating a Directory
1122	--------------------
1123
1124	``POST /uri?t=mkdir``
1125
1126	This creates a new empty directory, but does not attach it to any other
1127	directory in the Tahoe-LAFS file store.
1128
1129	If a "redirect_to_result=true" argument is provided, then the HTTP response
1130	will cause the web browser to be redirected to a /uri/$DIRCAP page that
1131	gives access to the newly-created directory. If you bookmark this page,
1132	you'll be able to get back to the directory again in the future. This is the
1133	recommended way to start working with a Tahoe server: create a new unlinked
1134	directory (using redirect_to_result=true), then bookmark the resulting
1135	/uri/$DIRCAP page. There is a "create directory" button on the Welcome page
1136	to invoke this action.
1137
1138	This accepts a format= argument in the query string. Refer to the
1139	documentation of the PUT /uri?t=mkdir operation in `Creating A
1140	New Directory`_ for information on the behavior of the format= argument.
1141
1142	If "redirect_to_result=true" is not provided (or is given a value of
1143	"false"), then the HTTP response body will simply be the write-cap of the
1144	new directory.
1145
1146	``POST /uri/$DIRCAP/[SUBDIRS../]?t=mkdir&name=CHILDNAME``
1147
1148	This creates a new empty directory as a child of the designated SUBDIR. This
1149	will create additional intermediate directories as necessary.
1150
1151	This accepts a format= argument in the query string. Refer to the
1152	documentation of POST /uri/$DIRCAP/[SUBDIRS../]?t=mkdir&name=CHILDNAME in
1153	`Creating a New Directory`_ for information on the behavior of the format=
1154	argument.
1155
1156	If a "when_done=URL" argument is provided, the HTTP response will cause the
1157	web browser to redirect to the given URL. This provides a convenient way to
1158	return the browser to the directory that was just modified. Without a
1159	when_done= argument, the HTTP response will simply contain the write-cap of
1160	the directory that was just created.
1161
1162
1163	Uploading a File
1164	----------------
1165
1166	``POST /uri?t=upload``
1167
1168	This uploads a file, and produces a file-cap for the contents, but does not
1169	attach the file to any directory in the Tahoe-LAFS file store. That is, no
1170	directories will be modified by this operation.
1171
1172	The file must be provided as the "file" field of an HTML encoded form body,
1173	produced in response to an HTML form like this::
1174
1175	<form action="/uri" method="POST" enctype="multipart/form-data">
1176	<input type="hidden" name="t" value="upload" />
1177	<input type="file" name="file" />
1178	<input type="submit" value="Upload Unlinked" />
1179	</form>
1180
1181	If a "when_done=URL" argument is provided, the response body will cause the
1182	browser to redirect to the given URL. If the when_done= URL has the string
1183	"%(uri)s" in it, that string will be replaced by a URL-escaped form of the
1184	newly created file-cap. (Note that without this substitution, there is no
1185	way to access the file that was just uploaded).
1186
1187	The default (in the absence of when_done=) is to return an HTML page that
1188	describes the results of the upload. This page will contain information
1189	about which storage servers were used for the upload, how long each
1190	operation took, etc.
1191
1192	This accepts format= and mutable=true query string arguments. Refer to
1193	`Writing/Uploading a File`_ for information on the behavior of format= and
1194	mutable=true.
1195
1196	``POST /uri/$DIRCAP/[SUBDIRS../]?t=upload``
1197
1198	This uploads a file, and attaches it as a new child of the given directory,
1199	which must be mutable. The file must be provided as the "file" field of an
1200	HTML-encoded form body, produced in response to an HTML form like this::
1201
1202	<form action="." method="POST" enctype="multipart/form-data">
1203	<input type="hidden" name="t" value="upload" />
1204	<input type="file" name="file" />
1205	<input type="submit" value="Upload" />
1206	</form>
1207
1208	A "name=" argument can be provided to specify the new child's name,
1209	otherwise it will be taken from the "filename" field of the upload form
1210	(most web browsers will copy the last component of the original file's
1211	pathname into this field). To avoid confusion, name= is not allowed to
1212	contain a slash.
1213
1214	If there is already a child with that name, and it is a mutable file, then
1215	its contents are replaced with the data being uploaded. If it is not a
1216	mutable file, the default behavior is to remove the existing child before
1217	creating a new one. To prevent this (and make the operation return an error
1218	instead of overwriting the old child), add a "replace=false" argument, as
1219	"?t=upload&replace=false". With replace=false, this operation will return an
1220	HTTP 409 "Conflict" error if there is already an object at the given
1221	location, rather than overwriting the existing object. Note that "true",
1222	"t", and "1" are all synonyms for "True", and "false", "f", and "0" are
1223	synonyms for "False". the parameter is case-insensitive.
1224
1225	This will create additional intermediate directories as necessary, although
1226	since it is expected to be triggered by a form that was retrieved by "GET
1227	/uri/$DIRCAP/[SUBDIRS../]", it is likely that the parent directory will
1228	already exist.
1229
1230	This accepts format= and mutable=true query string arguments. Refer to
1231	`Writing/Uploading a File`_ for information on the behavior of format= and
1232	mutable=true.
1233
1234	If a "when_done=URL" argument is provided, the HTTP response will cause the
1235	web browser to redirect to the given URL. This provides a convenient way to
1236	return the browser to the directory that was just modified. Without a
1237	when_done= argument, the HTTP response will simply contain the file-cap of
1238	the file that was just uploaded (a write-cap for mutable files, or a
1239	read-cap for immutable files).
1240
1241	``POST /uri/$DIRCAP/[SUBDIRS../]FILENAME?t=upload``
1242
1243	This also uploads a file and attaches it as a new child of the given
1244	directory, which must be mutable. It is a slight variant of the previous
1245	operation, as the URL refers to the target file rather than the parent
1246	directory. It is otherwise identical: this accepts mutable= and when_done=
1247	arguments too.
1248
1249	``POST /uri/$FILECAP?t=upload``
1250
1251	This modifies the contents of an existing mutable file in-place. An error is
1252	signalled if $FILECAP does not refer to a mutable file. It behaves just like
1253	the "PUT /uri/$FILECAP" form, but uses a POST for the benefit of HTML forms
1254	in a web browser.
1255
1256
1257	Attaching An Existing File Or Directory (by URI)
1258	------------------------------------------------
1259
1260	``POST /uri/$DIRCAP/[SUBDIRS../]?t=uri&name=CHILDNAME&uri=CHILDCAP``
1261
1262	This attaches a given read- or write- cap "CHILDCAP" to the designated
1263	directory, with a specified child name. This behaves much like the PUT t=uri
1264	operation, and is a lot like a UNIX hardlink. It is subject to the same
1265	restrictions as that operation on the use of cap formats unknown to the
1266	web-API server.
1267
1268	This will create additional intermediate directories as necessary, although
1269	since it is expected to be triggered by a form that was retrieved by "GET
1270	/uri/$DIRCAP/[SUBDIRS../]", it is likely that the parent directory will
1271	already exist.
1272
1273	This accepts the same replace= argument as POST t=upload.
1274
1275
1276	Unlinking a Child
1277	-----------------
1278
1279	``POST /uri/$DIRCAP/[SUBDIRS../]?t=delete&name=CHILDNAME``
1280
1281	``POST /uri/$DIRCAP/[SUBDIRS../]?t=unlink&name=CHILDNAME`` (Tahoe >= v1.9)
1282
1283	This instructs the node to remove a child object (file or subdirectory) from
1284	the given directory, which must be mutable. Note that the entire subtree is
1285	unlinked from the parent. Unlike deleting a subdirectory in a UNIX local
1286	filesystem, the subtree need not be empty; if it isn't, then other references
1287	into the subtree will see that the child subdirectories are not modified by
1288	this operation. Only the link from the given directory to its child is severed.
1289
1290	In Tahoe-LAFS v1.9.0 and later, t=unlink can be used as a synonym for t=delete.
1291	If interoperability with older web-API servers is required, t=delete should
1292	be used.
1293
1294
1295	Renaming a Child
1296	----------------
1297
1298	``POST /uri/$DIRCAP/[SUBDIRS../]?t=rename&from_name=OLD&to_name=NEW``
1299
1300	This instructs the node to rename a child of the given directory, which must
1301	be mutable. This has a similar effect to removing the child, then adding the
1302	same child-cap under the new name, except that it preserves metadata. This
1303	operation cannot move the child to a different directory.
1304
1305	The default behavior is to overwrite any existing link at the destination
1306	(replace=true). To prevent this (and make the operation return an error
1307	instead of overwriting), add a "replace=false" argument. With replace=false,
1308	this operation will return an HTTP 409 "Conflict" error if the destination
1309	is not the same link as the source and there is already a link at the
1310	destination, rather than overwriting the existing link. To allow the
1311	operation to overwrite a link to a file, but return an HTTP 409 error when
1312	trying to overwrite a link to a directory, use "replace=only-files" (this
1313	behavior is closer to the traditional UNIX "mv" command). Note that "true",
1314	"t", and "1" are all synonyms for "True"; "false", "f", and "0" are synonyms
1315	for "False"; and the parameter is case-insensitive.
1316
1317
1318	Relinking ("Moving") a Child
1319	----------------------------
1320
1321	``POST /uri/$DIRCAP/[SUBDIRS../]?t=relink&from_name=OLD&to_dir=$NEWDIRCAP/[NEWSUBDIRS../]&to_name=NEW``
1322	``[&replace=true\|false\|only-files]`` (Tahoe >= v1.10)
1323
1324	This instructs the node to move a child of the given source directory, into
1325	a different directory and/or to a different name. The command is named
1326	``relink`` because what it does is add a new link to the child from the new
1327	location, then remove the old link. Nothing is actually "moved": the child
1328	is still reachable through any path from which it was formerly reachable,
1329	and the storage space occupied by its ciphertext is not affected.
1330
1331	The source and destination directories must be writeable. If ``to_dir`` is
1332	not present, the child link is renamed within the same directory. If
1333	``to_name`` is not present then it defaults to ``from_name``. If the
1334	destination link (directory and name) is the same as the source link, the
1335	operation has no effect.
1336
1337	Metadata from the source directory entry is preserved. Multiple levels of
1338	descent in the source and destination paths are supported.
1339
1340	This operation will return an HTTP 404 "Not Found" error if
1341	``$DIRCAP/[SUBDIRS../]``, the child being moved, or the destination
1342	directory does not exist. It will return an HTTP 400 "Bad Request" error
1343	if any entry in the source or destination paths is not a directory.
1344
1345	The default behavior is to overwrite any existing link at the destination
1346	(replace=true). To prevent this (and make the operation return an error
1347	instead of overwriting), add a "replace=false" argument. With replace=false,
1348	this operation will return an HTTP 409 "Conflict" error if the destination
1349	is not the same link as the source and there is already a link at the
1350	destination, rather than overwriting the existing link. To allow the
1351	operation to overwrite a link to a file, but return an HTTP 409 error when
1352	trying to overwrite a link to a directory, use "replace=only-files" (this
1353	behavior is closer to the traditional UNIX "mv" command). Note that "true",
1354	"t", and "1" are all synonyms for "True"; "false", "f", and "0" are synonyms
1355	for "False"; and the parameter is case-insensitive.
1356
1357	When relinking into a different directory, for safety, the child link is
1358	not removed from the old directory until it has been successfully added to
1359	the new directory. This implies that in case of a crash or failure, the
1360	link to the child will not be lost, but it could be linked at both the old
1361	and new locations.
1362
1363	The source link should not be the same as any link (directory and child name)
1364	in the ``to_dir`` path. This restriction is not enforced, but it may be
1365	enforced in a future version. If it were violated then the result would be
1366	to create a cycle in the directory structure that is not necessarily reachable
1367	from the root of the destination path (``$NEWDIRCAP``), which could result in
1368	data loss, as described in ticket `#943`_.
1369
1370	.. _`#943`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/943
1371
1372
1373	Other Utilities
1374	---------------
1375
1376	``GET /uri?uri=$CAP``
1377
1378	This causes a redirect to /uri/$CAP, and retains any additional query
1379	arguments (like filename= or save=). This is for the convenience of web
1380	forms which allow the user to paste in a read- or write- cap (obtained
1381	through some out-of-band channel, like IM or email).
1382
1383	Note that this form merely redirects to the specific file or directory
1384	indicated by the $CAP: unlike the GET /uri/$DIRCAP form, you cannot
1385	traverse to children by appending additional path segments to the URL.
1386
1387	``GET /uri/$DIRCAP/[SUBDIRS../]?t=rename-form&name=$CHILDNAME``
1388
1389	This provides a useful facility to browser-based user interfaces. It
1390	returns a page containing a form targetting the "POST $DIRCAP t=rename"
1391	functionality described above, with the provided $CHILDNAME present in the
1392	'from_name' field of that form. I.e. this presents a form offering to
1393	rename $CHILDNAME, requesting the new name, and submitting POST rename.
1394	This same URL format can also be used with "move-form" with the expected
1395	results.
1396
1397	``GET /uri/$DIRCAP/[SUBDIRS../]CHILDNAME?t=uri``
1398
1399	This returns the file- or directory- cap for the specified object.
1400
1401	``GET /uri/$DIRCAP/[SUBDIRS../]CHILDNAME?t=readonly-uri``
1402
1403	This returns a read-only file- or directory- cap for the specified object.
1404	If the object is an immutable file, this will return the same value as
1405	t=uri.
1406
1407
1408	Debugging and Testing Features
1409	------------------------------
1410
1411	These URLs are less-likely to be helpful to the casual Tahoe user, and are
1412	mainly intended for developers.
1413
1414	``POST $URL?t=check``
1415
1416	This triggers the FileChecker to determine the current "health" of the
1417	given file or directory, by counting how many shares are available. The
1418	page that is returned will display the results. This can be used as a "show
1419	me detailed information about this file" page.
1420
1421	If a verify=true argument is provided, the node will perform a more
1422	intensive check, downloading and verifying every single bit of every share.
1423
1424	If an add-lease=true argument is provided, the node will also add (or
1425	renew) a lease to every share it encounters. Each lease will keep the share
1426	alive for a certain period of time (one month by default). Once the last
1427	lease expires or is explicitly cancelled, the storage server is allowed to
1428	delete the share.
1429
1430	If an output=JSON argument is provided, the response will be
1431	machine-readable JSON instead of human-oriented HTML. The data is a
1432	dictionary with the following keys::
1433
1434	storage-index: a base32-encoded string with the objects's storage index,
1435	or an empty string for LIT files
1436	summary: a string, with a one-line summary of the stats of the file
1437	results: a dictionary that describes the state of the file. For LIT files,
1438	this dictionary has only the 'healthy' key, which will always be
1439	True. For distributed files, this dictionary has the following
1440	keys:
1441	count-happiness: the servers-of-happiness level of the file, as
1442	defined in doc/specifications/servers-of-happiness.
1443	count-shares-good: the number of good shares that were found
1444	count-shares-needed: 'k', the number of shares required for recovery
1445	count-shares-expected: 'N', the number of total shares generated
1446	count-good-share-hosts: the number of distinct storage servers with
1447	good shares. Note that a high value does not
1448	necessarily imply good share distribution,
1449	because some of these servers may only hold
1450	duplicate shares.
1451	count-wrong-shares: for mutable files, the number of shares for
1452	versions other than the 'best' one (highest
1453	sequence number, highest roothash). These are
1454	either old, or created by an uncoordinated or
1455	not fully successful write.
1456	count-recoverable-versions: for mutable files, the number of
1457	recoverable versions of the file. For
1458	a healthy file, this will equal 1.
1459	count-unrecoverable-versions: for mutable files, the number of
1460	unrecoverable versions of the file.
1461	For a healthy file, this will be 0.
1462	count-corrupt-shares: the number of shares with integrity failures
1463	list-corrupt-shares: a list of "share locators", one for each share
1464	that was found to be corrupt. Each share locator
1465	is a list of (serverid, storage_index, sharenum).
1466	servers-responding: list of base32-encoded storage server identifiers,
1467	one for each server which responded to the share
1468	query.
1469	healthy: (bool) True if the file is completely healthy, False otherwise.
1470	Healthy files have at least N good shares. Overlapping shares
1471	do not currently cause a file to be marked unhealthy. If there
1472	are at least N good shares, then corrupt shares do not cause the
1473	file to be marked unhealthy, although the corrupt shares will be
1474	listed in the results (list-corrupt-shares) and should be manually
1475	removed to wasting time in subsequent downloads (as the
1476	downloader rediscovers the corruption and uses alternate shares).
1477	Future compatibility: the meaning of this field may change to
1478	reflect whether the servers-of-happiness criterion is met
1479	(see ticket #614).
1480	sharemap: dict mapping share identifier to list of serverids
1481	(base32-encoded strings). This indicates which servers are
1482	holding which shares. For immutable files, the shareid is
1483	an integer (the share number, from 0 to N-1). For
1484	immutable files, it is a string of the form
1485	'seq%d-%s-sh%d', containing the sequence number, the
1486	roothash, and the share number.
1487
1488	Before Tahoe-LAFS v1.11, the ``results`` dictionary also had a
1489	``needs-rebalancing`` field, but that has been removed since it was computed
1490	incorrectly.
1491
1492
1493	``POST $URL?t=start-deep-check`` (must add &ophandle=XYZ)
1494
1495	This initiates a recursive walk of all files and directories reachable from
1496	the target, performing a check on each one just like t=check. The result
1497	page will contain a summary of the results, including details on any
1498	file/directory that was not fully healthy.
1499
1500	t=start-deep-check can only be invoked on a directory. An error (400
1501	BAD_REQUEST) will be signalled if it is invoked on a file. The recursive
1502	walker will deal with loops safely.
1503
1504	This accepts the same verify= and add-lease= arguments as t=check.
1505
1506	Since this operation can take a long time (perhaps a second per object),
1507	the ophandle= argument is required (see "Slow Operations, Progress, and
1508	Cancelling" above). The response to this POST will be a redirect to the
1509	corresponding /operations/$HANDLE page (with output=HTML or output=JSON to
1510	match the output= argument given to the POST). The deep-check operation
1511	will continue to run in the background, and the /operations page should be
1512	used to find out when the operation is done.
1513
1514	Detailed check results for non-healthy files and directories will be
1515	available under /operations/$HANDLE/$STORAGEINDEX, and the HTML status will
1516	contain links to these detailed results.
1517
1518	The HTML /operations/$HANDLE page for incomplete operations will contain a
1519	meta-refresh tag, set to 60 seconds, so that a browser which uses
1520	deep-check will automatically poll until the operation has completed.
1521
1522	The JSON page (/options/$HANDLE?output=JSON) will contain a
1523	machine-readable JSON dictionary with the following keys::
1524
1525	finished: a boolean, True if the operation is complete, else False. Some
1526	of the remaining keys may not be present until the operation
1527	is complete.
1528	root-storage-index: a base32-encoded string with the storage index of the
1529	starting point of the deep-check operation
1530	count-objects-checked: count of how many objects were checked. Note that
1531	non-distributed objects (i.e. small immutable LIT
1532	files) are not checked, since for these objects,
1533	the data is contained entirely in the URI.
1534	count-objects-healthy: how many of those objects were completely healthy
1535	count-objects-unhealthy: how many were damaged in some way
1536	count-corrupt-shares: how many shares were found to have corruption,
1537	summed over all objects examined
1538	list-corrupt-shares: a list of "share identifiers", one for each share
1539	that was found to be corrupt. Each share identifier
1540	is a list of (serverid, storage_index, sharenum).
1541	list-unhealthy-files: a list of (pathname, check-results) tuples, for
1542	each file that was not fully healthy. 'pathname' is
1543	a list of strings (which can be joined by "/"
1544	characters to turn it into a single string),
1545	relative to the directory on which deep-check was
1546	invoked. The 'check-results' field is the same as
1547	that returned by t=check&output=JSON, described
1548	above.
1549	stats: a dictionary with the same keys as the t=start-deep-stats command
1550	(described below)
1551
1552	``POST $URL?t=stream-deep-check``
1553
1554	This initiates a recursive walk of all files and directories reachable from
1555	the target, performing a check on each one just like t=check. For each
1556	unique object (duplicates are skipped), a single line of JSON is emitted to
1557	the HTTP response channel (or an error indication, see below). When the walk
1558	is complete, a final line of JSON is emitted which contains the accumulated
1559	file-size/count "deep-stats" data.
1560
1561	This command takes the same arguments as t=start-deep-check.
1562
1563	A CLI tool can split the response stream on newlines into "response units",
1564	and parse each response unit as JSON. Each such parsed unit will be a
1565	dictionary, and will contain at least the "type" key: a string, one of
1566	"file", "directory", or "stats".
1567
1568	For all units that have a type of "file" or "directory", the dictionary will
1569	contain the following keys::
1570
1571	"path": a list of strings, with the path that is traversed to reach the
1572	object
1573	"cap": a write-cap URI for the file or directory, if available, else a
1574	read-cap URI
1575	"verifycap": a verify-cap URI for the file or directory
1576	"repaircap": an URI for the weakest cap that can still be used to repair
1577	the object
1578	"storage-index": a base32 storage index for the object
1579	"check-results": a copy of the dictionary which would be returned by
1580	t=check&output=json, with three top-level keys:
1581	"storage-index", "summary", and "results", and a variety
1582	of counts and sharemaps in the "results" value.
1583
1584	Note that non-distributed files (i.e. LIT files) will have values of None
1585	for verifycap, repaircap, and storage-index, since these files can neither
1586	be verified nor repaired, and are not stored on the storage servers.
1587	Likewise the check-results dictionary will be limited: an empty string for
1588	storage-index, and a results dictionary with only the "healthy" key.
1589
1590	The last unit in the stream will have a type of "stats", and will contain
1591	the keys described in the "start-deep-stats" operation, below.
1592
1593	If any errors occur during the traversal (specifically if a directory is
1594	unrecoverable, such that further traversal is not possible), an error
1595	indication is written to the response body, instead of the usual line of
1596	JSON. This error indication line will begin with the string "ERROR:" (in all
1597	caps), and contain a summary of the error on the rest of the line. The
1598	remaining lines of the response body will be a python exception. The client
1599	application should look for the ERROR: and stop processing JSON as soon as
1600	it is seen. Note that neither a file being unrecoverable nor a directory
1601	merely being unhealthy will cause traversal to stop. The line just before
1602	the ERROR: will describe the directory that was untraversable, since the
1603	unit is emitted to the HTTP response body before the child is traversed.
1604
1605
1606	``POST $URL?t=check&repair=true``
1607
1608	This performs a health check of the given file or directory, and if the
1609	checker determines that the object is not healthy (some shares are missing
1610	or corrupted), it will perform a "repair". During repair, any missing
1611	shares will be regenerated and uploaded to new servers.
1612
1613	This accepts the same verify=true and add-lease= arguments as t=check. When
1614	an output=JSON argument is provided, the machine-readable JSON response
1615	will contain the following keys::
1616
1617	storage-index: a base32-encoded string with the objects's storage index,
1618	or an empty string for LIT files
1619	repair-attempted: (bool) True if repair was attempted
1620	repair-successful: (bool) True if repair was attempted and the file was
1621	fully healthy afterwards. False if no repair was
1622	attempted, or if a repair attempt failed.
1623	pre-repair-results: a dictionary that describes the state of the file
1624	before any repair was performed. This contains exactly
1625	the same keys as the 'results' value of the t=check
1626	response, described above.
1627	post-repair-results: a dictionary that describes the state of the file
1628	after any repair was performed. If no repair was
1629	performed, post-repair-results and pre-repair-results
1630	will be the same. This contains exactly the same keys
1631	as the 'results' value of the t=check response,
1632	described above.
1633
1634	``POST $URL?t=start-deep-check&repair=true`` (must add &ophandle=XYZ)
1635
1636	This triggers a recursive walk of all files and directories, performing a
1637	t=check&repair=true on each one.
1638
1639	Like t=start-deep-check without the repair= argument, this can only be
1640	invoked on a directory. An error (400 BAD_REQUEST) will be signalled if it
1641	is invoked on a file. The recursive walker will deal with loops safely.
1642
1643	This accepts the same verify= and add-lease= arguments as
1644	t=start-deep-check. It uses the same ophandle= mechanism as
1645	start-deep-check. When an output=JSON argument is provided, the response
1646	will contain the following keys::
1647
1648	finished: (bool) True if the operation has completed, else False
1649	root-storage-index: a base32-encoded string with the storage index of the
1650	starting point of the deep-check operation
1651	count-objects-checked: count of how many objects were checked
1652
1653	count-objects-healthy-pre-repair: how many of those objects were completely
1654	healthy, before any repair
1655	count-objects-unhealthy-pre-repair: how many were damaged in some way
1656	count-objects-healthy-post-repair: how many of those objects were completely
1657	healthy, after any repair
1658	count-objects-unhealthy-post-repair: how many were damaged in some way
1659
1660	count-repairs-attempted: repairs were attempted on this many objects.
1661	count-repairs-successful: how many repairs resulted in healthy objects
1662	count-repairs-unsuccessful: how many repairs resulted did not results in
1663	completely healthy objects
1664	count-corrupt-shares-pre-repair: how many shares were found to have
1665	corruption, summed over all objects
1666	examined, before any repair
1667	count-corrupt-shares-post-repair: how many shares were found to have
1668	corruption, summed over all objects
1669	examined, after any repair
1670	list-corrupt-shares: a list of "share identifiers", one for each share
1671	that was found to be corrupt (before any repair).
1672	Each share identifier is a list of (serverid,
1673	storage_index, sharenum).
1674	list-remaining-corrupt-shares: like list-corrupt-shares, but mutable shares
1675	that were successfully repaired are not
1676	included. These are shares that need
1677	manual processing. Since immutable shares
1678	cannot be modified by clients, all corruption
1679	in immutable shares will be listed here.
1680	list-unhealthy-files: a list of (pathname, check-results) tuples, for
1681	each file that was not fully healthy. 'pathname' is
1682	relative to the directory on which deep-check was
1683	invoked. The 'check-results' field is the same as
1684	that returned by t=check&repair=true&output=JSON,
1685	described above.
1686	stats: a dictionary with the same keys as the t=start-deep-stats command
1687	(described below)
1688
1689	``POST $URL?t=stream-deep-check&repair=true``
1690
1691	This triggers a recursive walk of all files and directories, performing a
1692	t=check&repair=true on each one. For each unique object (duplicates are
1693	skipped), a single line of JSON is emitted to the HTTP response channel (or
1694	an error indication). When the walk is complete, a final line of JSON is
1695	emitted which contains the accumulated file-size/count "deep-stats" data.
1696
1697	This emits the same data as t=stream-deep-check (without the repair=true),
1698	except that the "check-results" field is replaced with a
1699	"check-and-repair-results" field, which contains the keys returned by
1700	t=check&repair=true&output=json (i.e. repair-attempted, repair-successful,
1701	pre-repair-results, and post-repair-results). The output does not contain
1702	the summary dictionary that is provied by t=start-deep-check&repair=true
1703	(the one with count-objects-checked and list-unhealthy-files), since the
1704	receiving client is expected to calculate those values itself from the
1705	stream of per-object check-and-repair-results.
1706
1707	Note that the "ERROR:" indication will only be emitted if traversal stops,
1708	which will only occur if an unrecoverable directory is encountered. If a
1709	file or directory repair fails, the traversal will continue, and the repair
1710	failure will be indicated in the JSON data (in the "repair-successful" key).
1711
1712	``POST $DIRURL?t=start-manifest`` (must add &ophandle=XYZ)
1713
1714	This operation generates a "manfest" of the given directory tree, mostly
1715	for debugging. This is a table of (path, filecap/dircap), for every object
1716	reachable from the starting directory. The path will be slash-joined, and
1717	the filecap/dircap will contain a link to the object in question. This page
1718	gives immediate access to every object in the file store subtree.
1719
1720	This operation uses the same ophandle= mechanism as deep-check. The
1721	corresponding /operations/$HANDLE page has three different forms. The
1722	default is output=HTML.
1723
1724	If output=text is added to the query args, the results will be a text/plain
1725	list. The first line is special: it is either "finished: yes" or "finished:
1726	no"; if the operation is not finished, you must periodically reload the
1727	page until it completes. The rest of the results are a plaintext list, with
1728	one file/dir per line, slash-separated, with the filecap/dircap separated
1729	by a space.
1730
1731	If output=JSON is added to the queryargs, then the results will be a
1732	JSON-formatted dictionary with six keys. Note that because large directory
1733	structures can result in very large JSON results, the full results will not
1734	be available until the operation is complete (i.e. until output["finished"]
1735	is True)::
1736
1737	finished (bool): if False then you must reload the page until True
1738	origin_si (base32 str): the storage index of the starting point
1739	manifest: list of (path, cap) tuples, where path is a list of strings.
1740	verifycaps: list of (printable) verify cap strings
1741	storage-index: list of (base32) storage index strings
1742	stats: a dictionary with the same keys as the t=start-deep-stats command
1743	(described below)
1744
1745	``POST $DIRURL?t=start-deep-size`` (must add &ophandle=XYZ)
1746
1747	This operation generates a number (in bytes) containing the sum of the
1748	filesize of all directories and immutable files reachable from the given
1749	directory. This is a rough lower bound of the total space consumed by this
1750	subtree. It does not include space consumed by mutable files, nor does it
1751	take expansion or encoding overhead into account. Later versions of the
1752	code may improve this estimate upwards.
1753
1754	The /operations/$HANDLE status output consists of two lines of text::
1755
1756	finished: yes
1757	size: 1234
1758
1759	``POST $DIRURL?t=start-deep-stats`` (must add &ophandle=XYZ)
1760
1761	This operation performs a recursive walk of all files and directories
1762	reachable from the given directory, and generates a collection of
1763	statistics about those objects.
1764
1765	The result (obtained from the /operations/$OPHANDLE page) is a
1766	JSON-serialized dictionary with the following keys (note that some of these
1767	keys may be missing until 'finished' is True)::
1768
1769	finished: (bool) True if the operation has finished, else False
1770	api-version: (int), number of deep-stats API version. Will be increased every
1771	time backwards-incompatible change is introduced.
1772	Current version is 1.
1773	count-immutable-files: count of how many CHK files are in the set
1774	count-mutable-files: same, for mutable files (does not include directories)
1775	count-literal-files: same, for LIT files (data contained inside the URI)
1776	count-files: sum of the above three
1777	count-directories: count of directories
1778	count-unknown: count of unrecognized objects (perhaps from the future)
1779	size-immutable-files: total bytes for all CHK files in the set, =deep-size
1780	size-mutable-files (TODO): same, for current version of all mutable files
1781	size-literal-files: same, for LIT files
1782	size-directories: size of directories (includes size-literal-files)
1783	size-files-histogram: list of (minsize, maxsize, count) buckets,
1784	with a histogram of filesizes, 5dB/bucket,
1785	for both literal and immutable files
1786	largest-directory: number of children in the largest directory
1787	largest-immutable-file: number of bytes in the largest CHK file
1788
1789	size-mutable-files is not implemented, because it would require extra
1790	queries to each mutable file to get their size. This may be implemented in
1791	the future.
1792
1793	Assuming no sharing, the basic space consumed by a single root directory is
1794	the sum of size-immutable-files, size-mutable-files, and size-directories.
1795	The actual disk space used by the shares is larger, because of the
1796	following sources of overhead::
1797
1798	integrity data
1799	expansion due to erasure coding
1800	share management data (leases)
1801	backend (ext3) minimum block size
1802
1803	``POST $URL?t=stream-manifest``
1804
1805	This operation performs a recursive walk of all files and directories
1806	reachable from the given starting point. For each such unique object
1807	(duplicates are skipped), a single line of JSON is emitted to the HTTP
1808	response channel (or an error indication, see below). When the walk is
1809	complete, a final line of JSON is emitted which contains the accumulated
1810	file-size/count "deep-stats" data.
1811
1812	A CLI tool can split the response stream on newlines into "response units",
1813	and parse each response unit as JSON. Each such parsed unit will be a
1814	dictionary, and will contain at least the "type" key: a string, one of
1815	"file", "directory", or "stats".
1816
1817	For all units that have a type of "file" or "directory", the dictionary will
1818	contain the following keys::
1819
1820	"path": a list of strings, with the path that is traversed to reach the
1821	object
1822	"cap": a write-cap URI for the file or directory, if available, else a
1823	read-cap URI
1824	"verifycap": a verify-cap URI for the file or directory
1825	"repaircap": an URI for the weakest cap that can still be used to repair
1826	the object
1827	"storage-index": a base32 storage index for the object
1828
1829	Note that non-distributed files (i.e. LIT files) will have values of None
1830	for verifycap, repaircap, and storage-index, since these files can neither
1831	be verified nor repaired, and are not stored on the storage servers.
1832
1833	The last unit in the stream will have a type of "stats", and will contain
1834	the keys described in the "start-deep-stats" operation, below.
1835
1836	If any errors occur during the traversal (specifically if a directory is
1837	unrecoverable, such that further traversal is not possible), an error
1838	indication is written to the response body, instead of the usual line of
1839	JSON. This error indication line will begin with the string "ERROR:" (in all
1840	caps), and contain a summary of the error on the rest of the line. The
1841	remaining lines of the response body will be a python exception. The client
1842	application should look for the ERROR: and stop processing JSON as soon as
1843	it is seen. The line just before the ERROR: will describe the directory that
1844	was untraversable, since the manifest entry is emitted to the HTTP response
1845	body before the child is traversed.
1846
1847
1848	Other Useful Pages
1849	==================
1850
1851	The portion of the web namespace that begins with "/uri" (and "/named") is
1852	dedicated to giving users (both humans and programs) access to the Tahoe-LAFS
1853	file store. The rest of the namespace provides status information about the
1854	state of the Tahoe-LAFS node.
1855
1856	``GET /`` (the root page)
1857
1858	This is the "Welcome Page", and contains a few distinct sections::
1859
1860	Node information: library versions, local nodeid, services being provided.
1861
1862	File store access forms: create a new directory, view a file/directory by
1863	URI, upload a file (unlinked), download a file by
1864	URI.
1865
1866	Grid status: introducer information, helper information, connected storage
1867	servers.
1868
1869	``GET /?t=json`` (the json welcome page)
1870
1871	This is the "json Welcome Page", and contains connectivity status
1872	of the introducer(s) and storage server(s), here's an example::
1873
1874	{
1875	"introducers": {
1876	"statuses": []
1877	},
1878	"servers": [{
1879	"nodeid": "other_nodeid",
1880	"available_space": 123456,
1881	"nickname": "George \u263b",
1882	"version": "1.0",
1883	"connection_status": "summary",
1884	"last_received_data": 1487811257
1885	}]
1886	}
1887
1888
1889	The above json ``introducers`` section includes a list of
1890	introducer connectivity status messages.
1891
1892	The above json ``servers`` section is an array with map elements. Each map
1893	has the following properties:
1894
1895	1. ``nodeid`` - an identifier derived from the node's public key
1896	2. ``available_space`` - the available space in bytes expressed as an integer
1897	3. ``nickname`` - the storage server nickname
1898	4. ``version`` - the storage server Tahoe-LAFS version
1899	5. ``connection_status`` - connectivity status
1900	6. ``last_received_data`` - the time when data was last received,
1901	expressed in seconds since epoch
1902
1903	``GET /status/``
1904
1905	This page lists all active uploads and downloads, and contains a short list
1906	of recent upload/download operations. Each operation has a link to a page
1907	that describes file sizes, servers that were involved, and the time consumed
1908	in each phase of the operation.
1909
1910	A GET of /status/?t=json will contain a machine-readable subset of the same
1911	data. It returns a JSON-encoded dictionary. The only key defined at this
1912	time is "active", with a value that is a list of operation dictionaries, one
1913	for each active operation. Once an operation is completed, it will no longer
1914	appear in data["active"] .
1915
1916	Each op-dict contains a "type" key, one of "upload", "download",
1917	"mapupdate", "publish", or "retrieve" (the first two are for immutable
1918	files, while the latter three are for mutable files and directories).
1919
1920	The "upload" op-dict will contain the following keys::
1921
1922	type (string): "upload"
1923	storage-index-string (string): a base32-encoded storage index
1924	total-size (int): total size of the file
1925	status (string): current status of the operation
1926	progress-hash (float): 1.0 when the file has been hashed
1927	progress-ciphertext (float): 1.0 when the file has been encrypted.
1928	progress-encode-push (float): 1.0 when the file has been encoded and
1929	pushed to the storage servers. For helper
1930	uploads, the ciphertext value climbs to 1.0
1931	first, then encoding starts. For unassisted
1932	uploads, ciphertext and encode-push progress
1933	will climb at the same pace.
1934
1935	The "download" op-dict will contain the following keys::
1936
1937	type (string): "download"
1938	storage-index-string (string): a base32-encoded storage index
1939	total-size (int): total size of the file
1940	status (string): current status of the operation
1941	progress (float): 1.0 when the file has been fully downloaded
1942
1943	Front-ends which want to report progress information are advised to simply
1944	average together all the progress-* indicators. A slightly more accurate
1945	value can be found by ignoring the progress-hash value (since the current
1946	implementation hashes synchronously, so clients will probably never see
1947	progress-hash!=1.0).
1948
1949	``GET /helper_status/``
1950
1951	If the node is running a helper (i.e. if [helper]enabled is set to True in
1952	tahoe.cfg), then this page will provide a list of all the helper operations
1953	currently in progress. If "?t=json" is added to the URL, it will return a
1954	JSON-formatted list of helper statistics, which can then be used to produce
1955	graphs to indicate how busy the helper is.
1956
1957	``GET /statistics/``
1958
1959	This page provides "node statistics", which are collected from a variety of
1960	sources::
1961
1962	load_monitor: every second, the node schedules a timer for one second in
1963	the future, then measures how late the subsequent callback
1964	is. The "load_average" is this tardiness, measured in
1965	seconds, averaged over the last minute. It is an indication
1966	of a busy node, one which is doing more work than can be
1967	completed in a timely fashion. The "max_load" value is the
1968	highest value that has been seen in the last 60 seconds.
1969
1970	cpu_monitor: every minute, the node uses time.clock() to measure how much
1971	CPU time it has used, and it uses this value to produce
1972	1min/5min/15min moving averages. These values range from 0%
1973	(0.0) to 100% (1.0), and indicate what fraction of the CPU
1974	has been used by the Tahoe node. Not all operating systems
1975	provide meaningful data to time.clock(): they may report 100%
1976	CPU usage at all times.
1977
1978	uploader: this counts how many immutable files (and bytes) have been
1979	uploaded since the node was started
1980
1981	downloader: this counts how many immutable files have been downloaded
1982	since the node was started
1983
1984	publishes: this counts how many mutable files (including directories) have
1985	been modified since the node was started
1986
1987	retrieves: this counts how many mutable files (including directories) have
1988	been read since the node was started
1989
1990	There are other statistics that are tracked by the node. The "raw stats"
1991	section shows a formatted dump of all of them.
1992
1993	By adding "?t=json" to the URL, the node will return a JSON-formatted
1994	dictionary of stats values, which can be used by other tools to produce
1995	graphs of node behavior. The misc/munin/ directory in the source
1996	distribution provides some tools to produce these graphs.
1997
1998	``GET /`` (introducer status)
1999
2000	For Introducer nodes, the welcome page displays information about both
2001	clients and servers which are connected to the introducer. Servers make
2002	"service announcements", and these are listed in a table. Clients will
2003	subscribe to hear about service announcements, and these subscriptions are
2004	listed in a separate table. Both tables contain information about what
2005	version of Tahoe is being run by the remote node, their advertised and
2006	outbound IP addresses, their nodeid and nickname, and how long they have
2007	been available.
2008
2009	By adding "?t=json" to the URL, the node will return a JSON-formatted
2010	dictionary of stats values, which can be used to produce graphs of connected
2011	clients over time. This dictionary has the following keys::
2012
2013	["subscription_summary"] : a dictionary mapping service name (like
2014	"storage") to an integer with the number of
2015	clients that have subscribed to hear about that
2016	service
2017	["announcement_summary"] : a dictionary mapping service name to an integer
2018	with the number of servers which are announcing
2019	that service
2020	["announcement_distinct_hosts"] : a dictionary mapping service name to an
2021	integer which represents the number of
2022	distinct hosts that are providing that
2023	service. If two servers have announced
2024	FURLs which use the same hostnames (but
2025	different ports and tubids), they are
2026	considered to be on the same host.
2027
2028
2029	Static Files in /public_html
2030	============================
2031
2032	The web-API server will take any request for a URL that starts with /static
2033	and serve it from a configurable directory which defaults to
2034	$BASEDIR/public_html . This is configured by setting the "[node]web.static"
2035	value in $BASEDIR/tahoe.cfg . If this is left at the default value of
2036	"public_html", then http://127.0.0.1:3456/static/subdir/foo.html will be
2037	served with the contents of the file $BASEDIR/public_html/subdir/foo.html .
2038
2039	This can be useful to serve a javascript application which provides a
2040	prettier front-end to the rest of the Tahoe web-API.
2041
2042
2043	Safety and Security Issues -- Names vs. URIs
2044	============================================
2045
2046	Summary: use explicit file- and dir- caps whenever possible, to reduce the
2047	potential for surprises when the file store structure is changed.
2048
2049	Tahoe-LAFS provides a mutable file store, but the ways that the store can
2050	change are limited. The only things that can change are:
2051
2052	* the mapping from child names to child objects inside mutable directories
2053	(by adding a new child, removing an existing child, or changing an
2054	existing child to point to a different object)
2055	* the contents of mutable files
2056
2057	Obviously if you query for information about the file store and then act
2058	to change it (such as by getting a listing of the contents of a mutable
2059	directory and then adding a file to the directory), then the store might
2060	have been changed after you queried it and before you acted upon it.
2061	However, if you use the URI instead of the pathname of an object when you
2062	act upon the object, then it will be the same object; only its contents
2063	can change (if it is mutable). If, on the other hand, you act upon the
2064	object using its pathname, then a different object might be in that place,
2065	which can result in more kinds of surprises.
2066
2067	For example, suppose you are writing code which recursively downloads the
2068	contents of a directory. The first thing your code does is fetch the listing
2069	of the contents of the directory. For each child that it fetched, if that
2070	child is a file then it downloads the file, and if that child is a directory
2071	then it recurses into that directory. Now, if the download and the recurse
2072	actions are performed using the child's name, then the results might be
2073	wrong, because for example a child name that pointed to a subdirectory when
2074	you listed the directory might have been changed to point to a file (in which
2075	case your attempt to recurse into it would result in an error), or a child
2076	name that pointed to a file when you listed the directory might now point to
2077	a subdirectory (in which case your attempt to download the child would result
2078	in a file containing HTML text describing the subdirectory!).
2079
2080	If your recursive algorithm uses the URI of the child instead of the name of
2081	the child, then those kinds of mistakes just can't happen. Note that both the
2082	child's name and the child's URI are included in the results of listing the
2083	parent directory, so it isn't any harder to use the URI for this purpose.
2084
2085	The read and write caps in a given directory node are separate URIs, and
2086	can't be assumed to point to the same object even if they were retrieved in
2087	the same operation (although the web-API server attempts to ensure this
2088	in most cases). If you need to rely on that property, you should explicitly
2089	verify it. More generally, you should not make assumptions about the
2090	internal consistency of the contents of mutable directories. As a result
2091	of the signatures on mutable object versions, it is guaranteed that a given
2092	version was written in a single update, but -- as in the case of a file --
2093	the contents may have been chosen by a malicious writer in a way that is
2094	designed to confuse applications that rely on their consistency.
2095
2096	In general, use names if you want "whatever object (whether file or
2097	directory) is found by following this name (or sequence of names) when my
2098	request reaches the server". Use URIs if you want "this particular object".
2099
2100
2101	Concurrency Issues
2102	==================
2103
2104	Tahoe uses both mutable and immutable files. Mutable files can be created
2105	explicitly by doing an upload with ?mutable=true added, or implicitly by
2106	creating a new directory (since a directory is just a special way to
2107	interpret a given mutable file).
2108
2109	Mutable files suffer from the same consistency-vs-availability tradeoff that
2110	all distributed data storage systems face. It is not possible to
2111	simultaneously achieve perfect consistency and perfect availability in the
2112	face of network partitions (servers being unreachable or faulty).
2113
2114	Tahoe tries to achieve a reasonable compromise, but there is a basic rule in
2115	place, known as the Prime Coordination Directive: "Don't Do That". What this
2116	means is that if write-access to a mutable file is available to several
2117	parties, then those parties are responsible for coordinating their activities
2118	to avoid multiple simultaneous updates. This could be achieved by having
2119	these parties talk to each other and using some sort of locking mechanism, or
2120	by serializing all changes through a single writer.
2121
2122	The consequences of performing uncoordinated writes can vary. Some of the
2123	writers may lose their changes, as somebody else wins the race condition. In
2124	many cases the file will be left in an "unhealthy" state, meaning that there
2125	are not as many redundant shares as we would like (reducing the reliability
2126	of the file against server failures). In the worst case, the file can be left
2127	in such an unhealthy state that no version is recoverable, even the old ones.
2128	It is this small possibility of data loss that prompts us to issue the Prime
2129	Coordination Directive.
2130
2131	Tahoe nodes implement internal serialization to make sure that a single Tahoe
2132	node cannot conflict with itself. For example, it is safe to issue two
2133	directory modification requests to a single tahoe node's web-API server at the
2134	same time, because the Tahoe node will internally delay one of them until
2135	after the other has finished being applied. (This feature was introduced in
2136	Tahoe-1.1; back with Tahoe-1.0 the web client was responsible for serializing
2137	web requests themselves).
2138
2139	For more details, please see the "Consistency vs Availability" and "The Prime
2140	Coordination Directive" sections of :doc:`../specifications/mutable`.
2141
2142
2143	Access Blacklist
2144	================
2145
2146	Gateway nodes may find it necessary to prohibit access to certain files. The
2147	web-API has a facility to block access to filecaps by their storage index,
2148	returning a 403 "Forbidden" error instead of the original file.
2149
2150	This blacklist is recorded in $NODEDIR/access.blacklist, and contains one
2151	blocked file per line. Comment lines (starting with ``#``) are ignored. Each
2152	line consists of the storage-index (in the usual base32 format as displayed
2153	by the "More Info" page, or by the "tahoe debug dump-cap" command), followed
2154	by whitespace, followed by a reason string, which will be included in the 403
2155	error message. This could hold a URL to a page that explains why the file is
2156	blocked, for example.
2157
2158	So for example, if you found a need to block access to a file with filecap
2159	``URI:CHK:n7r3m6wmomelk4sep3kw5cvduq:os7ijw5c3maek7pg65e5254k2fzjflavtpejjyhshpsxuqzhcwwq:3:20:14861``,
2160	you could do the following::
2161
2162	tahoe debug dump-cap URI:CHK:n7r3m6wmomelk4sep3kw5cvduq:os7ijw5c3maek7pg65e5254k2fzjflavtpejjyhshpsxuqzhcwwq:3:20:14861
2163	-> storage index: whpepioyrnff7orecjolvbudeu
2164	echo "whpepioyrnff7orecjolvbudeu my puppy told me to" >>$NODEDIR/access.blacklist
2165	# ... restart the node to re-read configuration ...
2166	tahoe get URI:CHK:n7r3m6wmomelk4sep3kw5cvduq:os7ijw5c3maek7pg65e5254k2fzjflavtpejjyhshpsxuqzhcwwq:3:20:14861
2167	-> error, 403 Access Prohibited: my puppy told me to
2168
2169	The ``access.blacklist`` file will be checked each time a file or directory
2170	is accessed: the file's ``mtime`` is used to decide whether it need to be
2171	reloaded. Therefore no node restart is necessary when creating the initial
2172	blacklist, nor when adding second, third, or additional entries to the list.
2173	When modifying the file, be careful to update it atomically, otherwise a
2174	request may arrive while the file is only halfway written, and the partial
2175	file may be incorrectly parsed.
2176
2177	The blacklist is applied to all access paths (including SFTP and CLI
2178	operations), not just the web-API. The blacklist also applies to directories.
2179	If a directory is blacklisted, the gateway will refuse access to both that
2180	directory and any child files/directories underneath it, when accessed via
2181	"DIRCAP/SUBDIR/FILENAME" -style URLs. Users who go directly to the child
2182	file/dir will bypass the blacklist.
2183
2184	The node will log the SI of the file being blocked, and the reason code, into
2185	the ``logs/twistd.log`` file.
2186
2187	URLs and HTTP and UTF-8
2188	=======================
2189
2190	.. _urls-and-utf8:
2191
2192	HTTP does not provide a mechanism to specify the character set used to
2193	encode non-ASCII names in URLs (`RFC3986#2.1`_). We prefer the convention
2194	that the ``filename=`` argument shall be a URL-escaped UTF-8 encoded Unicode
2195	string. For example, suppose we want to provoke the server into using a
2196	filename of "f i a n c e-acute e" (i.e. f i a n c U+00E9 e). The UTF-8
2197	encoding of this is 0x66 0x69 0x61 0x6e 0x63 0xc3 0xa9 0x65 (or
2198	"fianc\\xC3\\xA9e", as python's ``repr()`` function would show). To encode
2199	this into a URL, the non-printable characters must be escaped with the
2200	urlencode ``%XX`` mechanism, giving us "fianc%C3%A9e". Thus, the first line
2201	of the HTTP request will be "``GET
2202	/uri/CAP...?save=true&filename=fianc%C3%A9e HTTP/1.1``". Not all browsers
2203	provide this: IE7 by default uses the Latin-1 encoding, which is "fianc%E9e"
2204	(although it has a configuration option to send URLs as UTF-8).
2205
2206	The response header will need to indicate a non-ASCII filename. The actual
2207	mechanism to do this is not clear. For ASCII filenames, the response header
2208	would look like::
2209
2210	Content-Disposition: attachment; filename="english.txt"
2211
2212	If Tahoe were to enforce the UTF-8 convention, it would need to decode the
2213	URL argument into a Unicode string, and then encode it back into a sequence
2214	of bytes when creating the response header. One possibility would be to use
2215	unencoded UTF-8. Developers suggest that IE7 might accept this::
2216
2217	#1: Content-Disposition: attachment; filename="fianc\xC3\xA9e"
2218	(note, the last four bytes of that line, not including the newline, are
2219	0xC3 0xA9 0x65 0x22)
2220
2221	`RFC2231#4`_ (dated 1997): suggests that the following might work, and `some
2222	developers have reported`_ that it is supported by Firefox (but not IE7)::
2223
2224	#2: Content-Disposition: attachment; filename*=utf-8''fianc%C3%A9e
2225
2226	My reading of `RFC2616#19.5.1`_ (which defines Content-Disposition) says
2227	that the filename= parameter is defined to be wrapped in quotes (presumably
2228	to allow spaces without breaking the parsing of subsequent parameters),
2229	which would give us::
2230
2231	#3: Content-Disposition: attachment; filename*=utf-8''"fianc%C3%A9e"
2232
2233	However this is contrary to the examples in the email thread listed above.
2234
2235	Developers report that IE7 (when it is configured for UTF-8 URL encoding,
2236	which is not the default in Asian countries), will accept::
2237
2238	#4: Content-Disposition: attachment; filename=fianc%C3%A9e
2239
2240	However, for maximum compatibility, Tahoe simply copies bytes from the URL
2241	into the response header, rather than enforcing the UTF-8 convention. This
2242	means it does not try to decode the filename from the URL argument, nor does
2243	it encode the filename into the response header.
2244
2245	.. _RFC3986#2.1: https://tools.ietf.org/html/rfc3986#section-2.1
2246	.. _RFC2231#4: https://tools.ietf.org/html/rfc2231#section-4
2247	.. _some developers have reported: http://markmail.org/message/dsjyokgl7hv64ig3
2248	.. _RFC2616#19.5.1: https://tools.ietf.org/html/rfc2616#section-19.5.1

Note: See TracBrowser for help on using the repository browser.

Download in other formats: