[flexi-streams-devel] *substitution-char* does not suppress external-format-encoding-error

[Dmitriy Ivanov]

Hello folks,

I have bumped into the following error while playing with Hunchentoot.
(It is originated from url-decoding GET parameters with
 *hunchentoot-default-external-format*.)

(let ((flex:*substitution-char* #\?))
  (flex:octets-to-string #(#xC1 #xC2 #xC3 #xC4) :external-format :utf-8))
=> "??"

(let ((flex:*substitution-char* #\?))
  (flex:octets-to-string #(#xC0 #xC1 #xC2 #xC3 #xC4) :external-format
:utf-8))
-> signals: This sequence can't be decoded using UTF-8 as it is too short.
1
octet missing at then end.

The reason is rather "simple": the decoder invokes the following chain of calls:
  compute-number-of-chars -> check-end -> signal-encoding-error

This contrasts to the most of decoder code, which directly calls
   recover-from-encoding-error
instead of
  signal-encoding-error.
--
Sincerely,
Dmitriy Ivanov
lisp.ystok.ru