Hello,

Can anyone offer any clues as to what's going wrong, why, etc., when I
serve out pages that have links like <a href="/two_words"> MSIE
*sometimes* requests them as "/two+AF8-words" (server logs show this
request, but the address bar gadget doesn't show what it's really
doing, it still indicates "/two_words"). Visitors, of course, get a 404
response, and the 404 shows "/two_words" as well.

The pages are served as US-ASCII, are declared as such in the HTTP
headers, and are indeed using US-ASCII character set encoding. And
there's no scripting being used, either. It's just straight HTML 4.01,
with zero validation errors, and it really is the plain old ASCII
underscore character.

No other browser does this, with those kinds of URIs. While MSIE's
address bar shows what I want (after a failed attempt to get it, and the
browser window shows the 404 response), I can click in it, hit enter,
and it'll erroneously request the +AF8- resource again. But if I drag
the link from the page and drop it into the address bar, it'll fetch the
URI properly.

I'm almost certain this is a fault with MSIE (versions 5.5 and 6), not the
Apache webservers that I'm testing with, I can't be completely sure. MSIE
is not configured to send all URIs as UTF-8, if that makes any difference.

--
If you insist on e-mailing me, use the reply-to address (it's real but
temporary). But please reply to the group, like you're supposed to.

This message was sent without a virus, please destroy some files yourself.

Tim wrote:
> Hello,
>
> Can anyone offer any clues as to what's going wrong, why, etc., when I
> serve out pages that have links like <a href="/two_words"> MSIE
> *sometimes* requests them as "/two+AF8-words" (server logs show this
> request, but the address bar gadget doesn't show what it's really
> doing, it still indicates "/two_words"). Visitors, of course, get a 404
> response, and the 404 shows "/two_words" as well.
>
> The pages are served as US-ASCII, are declared as such in the HTTP
> headers, and are indeed using US-ASCII character set encoding. And
> there's no scripting being used, either. It's just straight HTML 4.01,
> with zero validation errors, and it really is the plain old ASCII
> underscore character.

Maybe it's this?
http://wiki.complexfission.com/twiki...eNotFoundError
Found with Google: AF8 +underscore

On Tue, 9 Aug 2005, Tim wrote:

> The pages are served as US-ASCII, are declared as such in the HTTP
> headers, and are indeed using US-ASCII character set encoding.

I've seen this happen before! MSIE is determined to know better, and
has interpreted the content as being encoded in utf-7. MSIE is
broken.

I'd suggest serving them out as iso-8859-1 or utf-8 at your choice:
us-ascii can be validly described as either, so they are perfectly
valid choices for a real web browser, and might just coax that
misbegotten thing into not being so silly.

Or you could inspect the content and maybe you'll be able to spot the
character sequence (beginning with a "+") which has triggered this
MSIE "feature"[1].

grumble

[1] In web compatibility terms this would be called a "bug", but
doubtless MS have documented its behaviour in some obscure place.

Tim wrote:

>> Can anyone offer any clues as to what's going wrong, why, etc., when I
>> serve out pages that have links like <a href="/two_words"> MSIE
>> *sometimes* requests them as "/two+AF8-words" (server logs show this
>> request, but the address bar gadget doesn't show what it's really
>> doing, it still indicates "/two_words"). Visitors, of course, get a 404
>> response, and the 404 shows "/two_words" as well.


gerryt sent:

> Maybe it's this?
> http://wiki.complexfission.com/twiki...eNotFoundError
> Found with Google: AF8 +underscore

It would appear so, and annoyingly I found that after posting, too.
Despite Googling +AF8- underscore and a few other keywords beforehand.
(Isn't that always the way?)

There doesn't appear to be a great deal of information on this phenomenon,
and I was trying to find something I could trust. I can't tell from that
page whether the author knows what they're doing, or guessing. Their
advice of "The solution is to include the character set meta-tag in every
document." really sticks in my craw, because I'm not about to edit every
page exhibiting that problem, to include something that shouldn't be done,
just because of a bug in MSIE (ignoring the HTTP headers providing the
same information), nor change all my URIs to suit its foibles.

At this stage I'm inclined to just "screw MSIE" and let it foul up on
*perfectly* *correct* pages.

--
If you insist on e-mailing me, use the reply-to address (it's real but
temporary). But please reply to the group, like you're supposed to.

This message was sent without a virus, please destroy some files yourself.

Tim wrote:

>> The pages are served as US-ASCII, are declared as such in the HTTP
>> headers, and are indeed using US-ASCII character set encoding.

Alan J. Flavell sent:

> I've seen this happen before! MSIE is determined to know better, and
> has interpreted the content as being encoded in utf-7. MSIE is
> broken.
>
> I'd suggest serving them out as iso-8859-1 or utf-8 at your choice:

I'll see if changing the HTTP headers to say iso-8859-1 instead of
US-ASCII s, but I don't hold out much hope.

> Or you could inspect the content and maybe you'll be able to spot the
> character sequence (beginning with a "+") which has triggered this
> MSIE "feature"[1].

Already had a look, and there's nothing wrong with the content, it's just
a couple of words with an underscore between them, so there's nothing that
can be done there to fix that (MSIE's) problem.

--
If you insist on e-mailing me, use the reply-to address (it's real but
temporary). But please reply to the group, like you're supposed to.

This message was sent without a virus, please destroy some files yourself.

Alan J. Flavell sent:

>> I've seen this happen before! MSIE is determined to know better, and
>> has interpreted the content as being encoded in utf-7. MSIE is
>> broken.
>>
>> I'd suggest serving them out as iso-8859-1 or utf-8 at your choice

Tim replied:

> I'll see if changing the HTTP headers to say iso-8859-1 instead of
> US-ASCII s, but I don't hold out much hope.

Just following up on my posting, local testing indicates that setting the
HTTP headers to "iso-8859-1" seems to resolve the problem. Now I've got
to test whether this works on other client and server software, then have
to remove numerous meta statements put in by HTML tidy in the past.

For some reason, HTML tidy did that with some pages, but not with others,
and I've never looked into why (other than seeing if there was some
option to insert, or not insert, meta data - I can't see any). I'd better
check it doesn't insert such things again, and cause problems with pages
saying they're US-ASCII and HTTP headers saying something else.

--
If you insist on e-mailing me, use the reply-to address (it's real but
temporary). But please reply to the group, like you're supposed to.

This message was sent without a virus, please destroy some files yourself.

On Mon, 08 Aug 2005 22:04:10 +0100, Alan J. Flavell sent:

> you could inspect the content and maybe you'll be able to spot the
> character sequence (beginning with a "+") which has triggered this
> MSIE "feature"

I could understand MSIE fouling up badly with their *BUG* if I did have
character sequences that seem similar to UTF-7, and assuming that they
were. But I don't understand what weird thinking Microsoft had that it
might be the right thing to do to convert a word with an underscore
between it and the next word *into* their idea of a UTF-7 encoding of the
same thing. The underscore is one of those ASCII characters that's
representable by 7-bits of data, it wouldn't need encoding, and seems to
be the reverse of their character encoding guessing game.

--
If you insist on e-mailing me, use the reply-to address (it's real but
temporary). But please reply to the group, like you're supposed to.

This message was sent without a virus, please destroy some files yourself.

On Tue, 9 Aug 2005, Tim wrote, quoting me:

> > Or you could inspect the content and maybe you'll be able to spot
> > the character sequence (beginning with a "+") which has triggered
> > this MSIE "feature"[1].
>
> Already had a look, and there's nothing wrong with the content, it's
> just a couple of words with an underscore between them,

Well, the URL that you cited in another followup,
http://wiki.complexfission.com/twiki...eNotFoundError
, seems to have come to the same conclusion as I did. I suppose you
wouldn't have a copy of the offending document that you could put
onlime for investigation? Naturally only if you can reproduce the
fault symptoms in that test case!

> so there's nothing that can be done there to fix that (MSIE's)
> problem.

Other than forcing a different "charset" than us-ascii, indeed.

The cited web page refers specifically to Win IE5.5. It'd be
interesting to see whether they ever got around to correcting this
misbehaviour in later versions/fixes.

cheers

"Alan J. Flavell" <flavell@ph.gla.ac.uk> posted:

> The cited web page refers specifically to Win IE5.5. It'd be
> interesting to see whether they ever got around to correcting this
> misbehaviour in later versions/fixes.

It still does it, here, on the Misery web browser version 6.0.2800.1106IS,
which is up to date with the current patches.

--
If you insist on e-mailing me, use the reply-to address (it's real but
temporary). But please reply to the group, like you're supposed to.

This message was sent without a virus, please delete some files yourself.

On Tue, 9 Aug 2005, Alan J. Flavell wrote:

> The cited web page refers specifically to Win IE5.5. It'd be
> interesting to see whether they ever got around to correcting this
> misbehaviour in later versions/fixes.

http://schneegans.de/bugs/ie-utf-7/
says "Neuere Versionen des Internet Explorers"

.... und bin so klug als wie zuvor.