First version, Aug 2000.
Updated Jan 2002, Oct 2004; minor tweaks at the turn of 2005-6.
Comments would still be very welcome.
Quick links to Summary and Conclusions.
A topic that keeps coming up in various contexts is the issuing of a redirection status in response to a POST transaction. I looked at this, quite some time back, and it was a mess. I've looked at it again, from time to time as the subject has been raised in discussions, and revised the content as seemed appropriate. Management summary: it's still messy: some options work reliably, other potentially-important options only work in a subset of browsers and would would be difficult or unfeasible to deploy reliably in a WWW context. We also discuss some effective workarounds.
"Redirection" in this sense refers to the HTTP status values 301, 302, 303 and 307, and the accompanying Location: header that specifies the target URL. This general topic is covered in section 10.3 of RFC2616, where it says:
This class of status code indicates that further action needs to be taken by the user agent in order to fulfill the request. The action required MAY be carried out by the user agent without interaction with the user if and only if the method used in the second request is GET or HEAD.
The implication is that the user agent implementation "may" (in practical terms, probably "should", although the RFC doesn't say that) make some provision for actioning the redirection specified by the Location: header; but that there are cases where the RFC makes it a requirement to confirm with the user before taking the indicated action.
The simplest situations on the WWW are "idempotent" transactions, i.e those which can be repeated without causing any harm. These are typically "GET" transactions, either because they are retrieval of straightforward URL references (e.g href= or src= attributes in HTML), or because they are form submissions using the GET method. Redirecting a transaction of that kind is straightforward, and no questions asked: the client receives the redirection response, including a Location: header that specifies the new URL, and the client reacts to it by re-issuing the transaction to the new URL. There's a difference between the different 30x status codes associated with these redirections in their implied cacheability, but otherwise they are basically similar (301 and 302) in response to GET requests.
POST transactions are different, since they are defined to be, in principle, non-idempotent (such as ordering a pizza, casting a vote or whatever) and mustn't be arbitrarily repeated.
The HTTP protocol specifications are designed to take this distinction into account: the GET method is defined to be inherently idempotent, whereas the POST method is defined to be, at least potentially, non-idempotent; the specifications call for a number of precautions to be taken by client agents (such as browsers) for protecting users against inadvertently (re)submitting a POST transaction which they had not intended, or submitting a POST into a context which they would not have wanted.
Note, sometimes the POST method is selected for an idempotent transaction for other reasons, e.g to avoid the size limits in implementations of GET, or to avoid the query string from being visible as part of the URL, or to get the benefits of the multipart form encoding. These usages are not wrong, but they should not distract from the basic underlying guideline of using GET for idempotent transactions and POST for non-idempotent ones.
A correspondent, commenting on this page, argued eloquently along the lines of the previous paragraph that there ought to be a third kind of form submission, similar to POST but designed to be idempotent, and not pestering the user with the precautions against re-submission. Whatever the attractions of this idea, we currently have just the two, and we have to make do with what we have.
There seem to be two fundamentally different scenarios.
The resource (script etc.) that handles the transaction hasn't moved: the "work" will be done by the originally-specified URL. But on completing the transaction, it wants to refer the client to a different URL, where the answer to the transaction can be viewed. (As we will see, this case isn't problematical.)
The resource (script etc.) that is supposed to handle the transaction (that "does the work", to put it crudely) has moved. The addressed URL invites the client to re-present the transaction to the new URL, where the "work" will actually be done. (As we will see, this has been poorly supported, and is still likely to be problematical in a WWW context: some other approach is needed.)
These distinct behaviours are not directly associated with one or other of the 30x status codes, they depend also on the details of what the scripts at the two URLs actually do with the requests. But the idempotence principle implies some restrictions on how they should properly be used.
To be explicit: a POST transaction that has been redirected to a GET transaction with the intention of carrying out the "work" at the new URL, would only be acceptable if the "work" was idempotent, by virtue of the operative transaction being a GET. A non-idempotent transaction ought not to be handled in this way.
On the other hand if the "work" is carried out at the first URL, and the only purpose of the redirection is to direct the client to a URL where they can view the answer (this might be a prepared standard thank-you page, or one of a selection of prepared static success/failure page(s) for example), then the fact that the redirection results in a GET transaction is quite appropriate: indeed a static page cannot accept a POST request.
The situation with status 302 from HTTP/1.0 was unsatisfactory, as we discuss shortly. HTTP/1.1 amended the definition of 302, and introduced two separate statuses, 303 and 307, with the intention of clarifying the behaviour.
There's a scenario that comes up in Usenet discussions, where people are trying to invoke pre-packaged CGI scripts: they actually want to invoke two different prepared scripts in succession as part of the same POST transaction. As we will see in the subsequent discussion, this is not really practical in the form in which it's posed, and would be better addressed by a different approach.
One possibility would be to write a small "glue" routine which handles the CGI transaction by invoking the two prepared scripts in succession, retrieves their responses, and constructs an appropriate overall CGI response for the whole activity. OK if both scripts are on the same server. If the scripts are on different servers, then one server can be set up to handle the transaction from the client, and to remotely invoke the transaction(s) on remote server(s) behind the scenes, before returning the result to the client. In this way the client is shielded from any details of the mechanism, and isn't required to support POST redirections in order to make it work. Details of how to do this are beyond the scope of this note (Apache can be configured to "proxy" such requests, for example).
An email correspondent contacted me about this point, with a proposal in which he wanted to use the first such script to validate the submitted form parameters, and then, after redirection with status 307, the second script was intended to action the transaction. I found this quite alarming - I had to point out that the client agent (and thus, the user, if they so choose) would get control of the submission parameters after they had been validated by the first script, and could mischievously change them before submitting them for action at the second script. I would rate it as essential to avoid any contact with the user at such a sensitive point in the proceedings!
The reader should of course make themselves familiar with the relevant part of the HTTP/1.1 specification; here I am only attempting to look at the requirements of the specification and see how they relate to possible applications and solutions in a WWW context.
Historically, in HTTP/1.0, status 302 (temporary relocation) was documented to call for the response that is now, in HTTP/1.1, documented for status 307. However, the popular browsers did not implement that specification: instead, they implemented the response that is now documented for status 303. Lynx in the distant past implemented the original specification for status 302, but later it was brought into line with the de-facto behaviour of other browsers.
HTTP/1.1 now codifies the de facto behaviour in response to status 302, but it documents two separate status codes, 303 and 307, for resolving the ambiguity.
Where there seems reason to suppose that the redirection will cause a non-idempotent transaction to be performed in a different context than the original submission implied, the HTTP specification forbids the client to silently perform the redirected transaction - instead the client must first get the user's confirmation to proceed: in relation to requests submitted with the POST method, this obviously applies to statuses 301 and 307, but also (perhaps surprisingly) to status 302. User verification isn't necessary for status 303, as should be clear from the previous discussion.
As we will see, the mandate in regard of status 302 is widely disregarded, and, since de facto 302 is practically always now treated in the way that is specified for 303, the likelihood of taking some irrevocable action that had not been intended seems to be quite small.
As for status 301, the HTTP/1.1 specification re-iterates the requirements that were already in HTTP/1.0: when redirecting a POST request it mandates clients to verify the redirection with the user, and describes the behaviour of "some existing user agents" in converting the POST into a GET as "erroneous" (for all the good that might do...).
A correspondent writes that some of the statements which I make about specified client agent behaviour in response to 3xx are not stated in so many words in RFC2616. However, if we take the statuses one by one, I think it's reasonably clear: for status 301 it rates a redirection of POST to GET as "erroneous"; for 302 and 303 it documents that redirection of POST to GET is now considered to be correct; and it refers back to the requirement in RFCs 1945 and 2068 that the client was "not allowed" to change the method on the redirected request. Therefore, it seems that the only reasonable interpretation is that unless otherwise stated (i.e for 302 and 303), the proper way to perform redirection is by repeating the original "method", i.e for our present context, POST shall be redirected to POST.
See also Common User Agent Problems, specifically 3.4 about temp redirects, and 4.1 Handling the fragment identifier.
An email correspondent writes to complain bitterly about the RFC2616 mandatory requirement to confirm status 307 with the user, which in practice is honoured by the web-compatible browsers which implement such redirection; and praising MSIE for disregarding the mandatory requirement.
As this correspondent said, most users are in no position to answer the question whether they had intended their submission to be redirected in this way, and have no basis on which to decide. True enough, if they haven't been briefed beforehand about what to expect; but presumably the authors of RFC2616 had in mind the possibilities for abuse when a non-idempotent transaction is silently redirected to somewhere else. I could only advise my correspondent that, the way things currently are, their best move was to redesign their procedure so that it was not reliant on status 307. (And also not reliant on MSIE violating a mandatory requirement of RFC2616, considering that IE has recently been tightened-up on this or that earlier disregard of web specifications.)
There are two aspects of cacheing involved:
The cacheability of the redirection response itself,
The cacheability of the resulting document, that will be returned after the redirection has been actioned.
These two issues are quite separate from each other, and should not be confused. To take a practical example: where a transaction will result in one of a small number of static responses, it can make good sense to return the result by redirecting (non-cacheably) to the appropriate (cacheable) static document.
Status 301 redirection response is inherently cacheable: a client receiving this redirection response could store the fact that the first URL was permanently redirected to the second URL, and any future request specifying the first URL could be presented immediately to the second URL, without invoking the first URL and waiting to be redirected. This behaviour is permitted, but not mandated, for status 301.
Status 303 redirection response is forbidden to be cached.
Status 307 redirection response is by default not cacheable, but it is eligible to be cached if the response contains appropriate headers to that effect.
As for the documents that are returned after a redirection response has been received, they are at least eligible to be cacheable. Under the various 30x response codes, RFC2616 contains some remarks relevant to this issue.
To address some misunderstandings that have occurred about the query string when a POST transaction is redirected as a GET: the client does not automatically affix the original POSTed parameters to the query string part of the new URL when actioning the redirection. If the intention was to supply the original query onto the GET transaction to the new URL, then it would be the job of the script that is creating the redirection, to obtain the submitted parameters from the initial POST transaction, and to include them, properly formatted, as the query part of the new URL (on the Location header) to which the redirection is being specified. (Sure: transferring the parameters from a POST to a GET transaction could very well defeat one or other of the reasons that led to POST being chosen in preference to GET in the first place - I'm only exploring the issues at this point, not making a design recommendation.)
The RFC provides for redirection transactions to be accompanied by a "short hypertext note with a hyperlink to the new URI(s)", which is intended to be used by clients that do not implement the particular redirection transaction automatically.
However, it should be obvious that such a "hyperlink" isn't directly useful if the redirected transaction is intended to be presented by the POST method, since a href= presents a GET transaction. If this fallback document was to be of any use for that purpose, it would need to contain a fresh HTML form element, specifying the new URL as its action, and (presumably) containing the parameters of the original transaction as hidden fields. (This is not an original idea - after I had realised that it was a possible solution, I found that it had already been proposed by Klaus Weide in 1997.) If such a procedure was attempted at all, then it would have to be handled very carefully by the designer: the parameter values would need to be re-validated to avoid the possibility that they had been interfered with by mischievous users before resubmission. On an operational note, users who were presented with this kind of scenario in an otherwise critical situation (e.g shopping) might well get cold feet and bale out of the whole procedure.
The tests start with an HTML form, whose form tag specifies POST submission to the first CGI script. How this form is generated is unimportant as far as the tests are concerned: it could perfectly well be a static HTML page.
One of the parameters that is submitted with the form is an indicator of which kind of 30x response is desired (the tester selects this from an options list prior to submitting the form).
The form is then submitted (by POST to the first CGI script).
The CGI script, which is coded in Perl, responds by generating the redirection response. In order to have clear control over the exact CGI response headers generated (unlike the rest of the scripting which uses CGI.pm), this part of the procedure uses straightforward print statements to print the desired headers (and body content) to the standard output (something I normally would avoid, thanks to the simplicity of using CGI.pm). However, the scripts are normal Parsed Headers scripts, they are not NPH (non-parsed headers) scripts. There didn't seem any reason to suspect that the resulting HTTP stream generated by the server (Apache 1.3.*) was in any way defective such that an NPH script could achieve better results: anyway, parsed-headers scripts are probably more realistic in terms of normal WWW applications and thus the test seemed entirely reasonable in this form.
The redirection response (at the CGI interface) that is generated by the first script consists of the Status: CGI header specifying the desired 30x status and reason text (the server of course turns this into an actual HTTP response for transmission to the client), then the Location: header specifying the URL of the second script. Then comes the fallback provision, consisting of a Content-type: text/html header, separated from the fallback HTML document by the usual blank line that separates the CGI header part of the response from the response body. For diagnostic purposes, the HTML response body includes printouts of the query parameters (CGI::dump() method) and the CGI environment. In the fallback situation, the client would be expected to display this document, showing the environment etc. as it was seen by the first script.
Finally we describe the second CGI, to which the client will be redirected and which, in the event of the client honouring the redirection rather than displaying the fallback document, will be returning the final display. This too is a diagnostic script which displays the CGI environment (i.e as it was presented to the second CGI script). This script additionally dumps the POST transaction's query name/value pairs, so as to verify correct redirection of POST transactions.
No intensive effort was put into investigating whether the various cacheability rules were being honoured. The server log was inspected to verify that the expected transactions were being fielded by the server rather than the client redisplaying cached results, but beyond that the details were not investigated.
It could very reasonably be argued that in my tests, I should not have tested the response of HTTP/1.0-type clients to status 303 and 307, since those are not part of HTTP/1.0. Nevertheless, I did so and report the results. But I can assure you that this doesn't affect the conclusions reached: as we will see, whichever way the results are interpreted, there is no reliable way, in the WWW context, of getting a POST transaction at the first URL redirected to a POST transaction at the second URL.
Extra caveat: some proxies will modify an HTTP/1.1 client request into an HTTP/1.0 request. However, it didn't appear that this effect was causing any significant difference in client/server behaviour, whether the client was working through such a proxy or not.
According to the specification, clients don't necessarily have to make any provision for following the redirection specified in the server's Location header: but if they do make such provision, then they need to conform to the requirements of the RFC. The key requirements, for POST requests, seem to be these three:
The present page concentrates on the first two of those issues; the third was not studied in any detail. Discussions at the Mozilla Bugzilla also consider what the browser's behaviour should be with regard to which URL (before or after redirection) to present to the user (e.g in the URL field), and which URL (before or after) to bookmark if the user attempts a bookmark (although I might remark that bookmarking the result of a POST transaction might not be particularly useful in practice, and is quite unlikely to produce the result that the user had hoped for). These issues are not explored further in the present page.
The notes below summarise what those requirements are. The order in which the tests are listed was chosen for convenience of back-references.
Location: header. No user confirmation is required.On several contentious points, Opera has a reputation of doing what seems to be in the short-term interest of its users, no matter what the specifications might require. And we see an example of this behaviour here, in that the specification-correct behaviour of Opera 6.* had been broken in 7.* to conform with the broken behaviour of the mass browsers. That's understandable in a commercial product, but unfortunately it also reinforces the defective behaviour to which it's attempting to accommodate. Arguably it (or any other browser offering this kind of strategy) should throw an ostentatious warning, before accepting its user's consent to proceed in any way that's contrary to the applicable specification.
This paragraph is just "thinking aloud", and not really part of the present tests. It's at least arguable that the attempt by HTTP/1.1 to specify behaviour for status 301 has proven to be a failure, just as the attempt by HTTP/1.0 to specify behaviour for status 302 failed before. By analogy, one possible way out would be to introduce a pair of new status values to replace 301, one of which would require the client agent to behave as 301 was specified to behave, i.e a cacheable counterpart to 307, and the other of which would behave in the way that 301 has been typically implemented, i.e a cacheable counterpart to 303. However, considering the dilatory implementations of status 307, it's hard to take this as something worth losing any sleep over. The "long and short" of it, as far as a web developer is concerned, is that status 301 in response to POST is practically useless, since a cacheable redirection of a POST transaction to a GET URL doesn't seem to be the kind of thing one would really need, even if one could rely on all browsers implementing it in that anti-RFC2616 way. whereas when there is a genuine requirement for POST to be redirected permanently to POST, there is no way of implementing it reliably with status 301, although (if you trust browsers to implement 307, which they haven't been particularly good at), you can get something broadly similar by using status 307 together with appropriate cacheability headers.
An email correspondent calls my attention to an MS Knowledgebase article, KB179611, which documents an "advanced" option which, if enabled, will cause all redirections to be verified with the user. My correspondent commented that he had tried it, and it seemed to have no effect. Well, now that I have tried it myself (Win IE6 6.0.2800.1106 with a bushel of security fixes applied), I have to report that it seemed to have no effect for me, either. Indeed the KB article lists the products to which it applies, and the newest browser version listed is IE5.0, so if that's true (rather than merely being out of date), it seems as if they've dropped the feature, despite there still being a checkbox for it in the advanced options.
Suffice it to say that this option would have been at its default setting in my tests which are reported here.
Netscape Gold 3.01 (mis)behaved pretty much the same as Netscape 4.7.
Mozilla 0.9.7: statuses 301 and 307 were wrongly redirected to GET. This is Bug 48202, which was still present in Mozilla 1.3.1.
The next version of Mozilla on which I reviewed this issue was 1.6, and now it seems that status 307 is behaving correctly. However, the behaviour of status 301 seems to be unchanged. The bug chain is confused: bug 48202 is titled for status 307, but it goes on to also discuss 301. What has been fixed is the behaviour for status 307, and the bug has now been marked as fixed. But meantime, the bug entry 69376 references 81636 which is asking for the (incorrect, although it doesn't say so) response to status 301 to be cached. However, this has been marked as a duplicate of bug 48202, despite the fact that 48202 has now been marked fixed without apparently addressing either of the issues (the correct response, nor the cacheability).
By the way, Mozilla bug 68423 is also somewhat related, but on closer inspection, does not impact directly on the issues discussed here.
Opera (4.10 pre-release) redirected 303 and 302 to GET, and 307 and 301 to POST, all without user verification.
Opera 6.0 (release) behaved correctly, aside from the lack of a user dialog for status 302. Statuses 301 and 307 produced identical user dialogs, by which the user was invited to consent to the POST transaction to the new URL (this was the default response), alternatively to GET the new URL or to cancel the transaction. 6.04, as seen above, is the same.
WebTV Viewer 1.1: treated all four statuses as specified for 303 (no verification with user, redirected as GET) while displaying the status message "Relocation in response to POST".
Win MSIE 3.03 16bit also treated all four statuses as specified for 303.
As did w3m(990901). As did iCab 2.1pre.
| Browser | Status codes | |||
|---|---|---|---|---|
| 303 | 302 | 307 | 301 | |
| Win IE5, 5.5, 6 | OK | OK* | OK* | Wrong |
| Mozilla 0.9.7 ... 1.3.1 | OK | OK* | Wrong | Wrong |
| Mozilla 1.6 | OK | OK* | OK | Wrong |
| Opera 6.0, 6.04 | OK | OK* | OK | OK |
| Opera 7.23 | OK | OK* | OK | Wrong |
| Netscape (Win 4.7, 3.01) | Fallback | OK* | Fallback | Wrong |
| Lynx 2.8.3 | OK | OK* | OK | OK |
| WinIE3, WebTV, w3m | OK | OK* | Wrong | Wrong |
In the table:
OK = within the HTTP/1.1 specification,
OK* = ditto, except missing the mandatory verification with the user,
Wrong = erroneous behaviour,
Fallback = displays the fallback page.
Automatic redirection of a POST transaction to GET is entirely feasible, using status 302 or 303. Although the specification for 302 mandates clients to verify the redirection with the user, this mandate seems to be generally ignored, but that can be rated mostly harmless, as previous discussion showed.
In a usenet discussion, Kevin McMurtrie commented that 303 is only specified in HTTP/1.1, so he would test the HTTP level (SERVER_PROTOCOL) in the script, and respond with 302 for HTTP/1.0 and 303 for HTTP/1.1: this does indeed seem to be a good idea in theory, and I don't know any practical reason against it, so that would now be my recommendation too.
In either case it's a good idea to supply a short HTML content body with an appropriate link, and an explanation of the situation for the benefit of any browsers which fail to follow the redirection.
Redirection of a POST transaction to another URL as POST evidently works only in a subset of browsers (namely, modern versions of the popular browsers, and a subset of minority browsers), making its use in a WWW context unreliable: this is true for status 307, and even more so for status 301. Testing the protocol for HTTP/1.0 or HTTP/1.1 doesn't resolve this issue, because it wasn't only HTTP/1.0 clients which misbehaved. Trying to rely on fallback behaviour is not a viable strategy, since several browsers take the wrong action rather than taking the fallback.
It would seem to need considerable ingenuity to cope with the various browser responses in a reliable way, and (except in non-WWW situations where the browser population is under control) I would have to recommend finding some other approach for achieving the desired result, in general. Even in those browsers which correctly implement the protocol, the user would be presented with a disturbing prompt from their browser, which might lead them to believe that all was not well and to abandon the activity at that point, which is another reason for avoiding this if you possibly can (except perhaps with a well-controlled and well-briefed group of users). If the whole server-side process cannot be re-cast to avoid the issue entirely, then the solution might involve proxy-ing some part of the transaction on the server, and merging the result into the response on the server side, so that the client side is unaware of the "seams".
One clumsy option in a WWW context to mimic the intended result would be to have the first script construct a new form with the desired POST transaction prepared, and to send it back as a normal page, with the user invited to resubmit it. Whether this could be an acceptable solution in any particular case is left as an exercise for the reader.
I think, in fairness, anyone who is wanting to do that would be better advised to look towards the proxying or relaying approach as already mentioned (and there's one practical suggestion offered below, but going into the various options in detail would take us beyond the scope of this web page).
An email correspondent reports successfully using Perl in the first CGI script, to submit the POST request "behind the scenes" to the URL of the second CGI script, capture the response, and return it to the client. This snippet of Perl does the trick, I'm told:
Using the LWP::UserAgent module, I created a new user agent and created a new request header based on the content posted to the script.
my $req = new HTTP::Request ('POST',$thisUrl,$h); $req->content_type('application/x-www-form-urlencoded'); $req->content("myData=$someData"); # Pass request to the user agent and get a response back my $res = $ua->request($req); # Check the outcome of the response if ($res->is_success) { print $query->header(-target=>'_blank'); print $res->content; exit(0); }The only caveat is that content in the response is resolved locally, and so all of the URLs on the requested page need to be absolute.
As always, this is a best-efforts production and no liability can be accepted. Corrections would be most welcome.
Original materials © Copyright 1994 - 2006 by A.J.Flavell