bug in host-to-vector-quad on lispworks (also host-to-vector-quad)

Thu Jul 12 21:32:24 UTC 2018

Hi Mark,

I’ve made the changes in GET-HOST-BY-NAME and GET-RANDOM-HOST-BY-NAME on usocket trunk (master) [1]. Now they only returns IPv4 addresses if the underlying hostname resolves to both IPv4 and IPv6 addresses.  I think this change is safe for all users.

I also exported these functions as I saw many users directly use them as exported symbols. This is reasonable too, as GET-RANDOM-HOST-BY-NAME actually gives the ability to do load balancing.

Any way, could you please confirm if the current code base works for you?

—Chun

[1] https://github.com/usocket/usocket <https://github.com/usocket/usocket>, a1f32274686d3749ce491dd768c9d0da1a4479c4

> Il giorno 05 lug 2018, alle ore 20:13, Mark H. David <mhd at yv.org> ha scritto:
> 
> Hi, that sounds very good to me. Thanks a lot. -Mark
> 
> ----- Original message -----
> From: "Chun Tian (binghe)" <binghe.lisp at gmail.com>
> To: "Mark H. David" <mhd at yv.org>
> Cc: usocket development <usocket-devel at common-lisp.net>
> Subject: Re: bug in host-to-vector-quad on lispworks (also host-to-vector-quad)
> Date: Thu, 5 Jul 2018 14:18:54 +0200
> 
> Hi Mark,
> 
> Now I see your points. So IPv4 addresses should have a higher priority over IPv6 when returned by USOCKET host resolving functions. I’m a little busy in this week, *next* week I’ll revise all related USOCKET functions and make the changes (also need a better way to detect IPv6 addresses in various forms). Hopefully at the end of July I can make a new release with cumulative updates in the past... more than one year.
> 
> —Chun
> 
>> Il giorno 04 lug 2018, alle ore 23:44, Mark H. David <mhd at yv.org> ha scritto:
>> 
>> Hi,
>> Kind of asking for a friend. I thought the thing to do would be to loop through the list and choose the first non-ipv6 result whereas now it simply chooses the first result.  For the random host case, it should similarly filter out ipv6 entries in the result.
>> Thank you very much!
>> -Mark
>> 
>> ----- Original message -----
>> From: "Chun Tian (binghe)" <binghe.lisp at gmail.com>
>> To: "Mark H. David" <mhd at yv.org>
>> Cc: "usocket-devel" <usocket-devel at common-lisp.net>
>> Subject: Re: bug in host-to-vector-quad on lispworks (also host-to-vector-quad)
>> Date: Wed, 4 Jul 2018 22:47:04 +0200
>> 
>> Hi Mark,
>> 
>> thanks for reporting this issue.
>> 
>> It’s obvious that #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) means IPv6 address “::1”, which is indeed a valid address mapping to “localhost”. The function HOST-TO-HBO is not an exported function, it’s used internally by some backends and deals only with integers representing IPv4 addresses. But on LispWorks (>=6.1) the function lw-hbo-to-vector-quad may pass longer vectors back to HOST-TO-HBO and then caused this issue.
>> 
>> On my Mac OS X laptop, I also got the same: (the first returned address is “::1”)
>> 
>> CL-USER 5 > (comm:get-host-entry "localhost" :fields '(:addresses))
>> (#<COMM:IPV6-ADDRESS ::1 4020187E73> 2130706433)
>> 
>> So what’s your desired behavior here?  In the quick fixes, I could do something to make get-random-host-by-name return a random IPv4 address in presence of mixed IPv4/v6 address lists. But generally speaking get-hosts-by-name should support IPv6 addresses, especially for those IPv6-only hostnames.
>> 
>> On the other side, HOST-TO-HBO is not an exported function, and it’s also not called by LispWorks backends, I don’t understand how your call chain finally touched it and let LispWorks throw an exception.  It’s hard to support IPv6 in HOST-TO-HBO.
>> 
>> Let me know what you think here, and with your help/inputs we can make usocket better.
>> 
>> Regards,
>> 
>> Chun
>> 
>>> Il giorno 04 lug 2018, alle ore 21:04, Mark H. David <mhd at yv.org> ha scritto:
>>> 
>>> Running usocket-0.7.0.1 in LispWorks 7.1.0. In my system it happens currently that
>>> 
>>> (get-hosts-by-name "localhost")
>>> 
>>> returns the following list:
>>> 
>>> (#(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) #(127 0 0 1))
>>> 
>>> Then get-host-by-name simply takes the first of that list and returns it.
>>> 
>>> Host-to-hbo expects the result of get-host-by-name to always be a vector quad, so the result of calling it with "localhost" is an error on the recursive call with the result of (get-host-by-name "localhost") => #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1)
>>> 
>>> The error message is due to falling out of the ecase:
>>> 
>>> #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) fell through ETYPECASE expression inside USOCKET::HOST-TO-HBO.
>>> 
>>> I observed that host-to-vector-quad gives the wrong result, a 16-byte instead of a 4-byte vector, due to the same root cause, but it will happen "randomly". The reason is it uses get-random-host-by-name, which chooses a random element of the same result list returned by (get-hosts-by-name "localhost").  For example, I just ran this in the REPL -- log:
>>> 
>>> USOCKET> (loop repeat 10 for r = (host-to-vector-quad "localhost") when (not (= (length r) 4)) collect r)
>>> (#(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1) #(0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1))
>>> 
>>> This seems to be LispWorks only. I tried this on SBCL and CCL, but I could not recreate similar problems. Also, not all MacOS installs seem to get this weird ipv6 localhost the way I do. I'm not sure why I'm getting this. I do not remember doing anything special or any kind of stuff with ipv6, but apparently something I've done has "infected" me with it.  However, I can say I do not notice other networking issues on my system.
>>> 
>>> Thanks,
>>> Mark
>>> 
>> 
>> Email had 1 attachment:
>> + signature.asc
>> 1k (application/pgp-signature)
> 
> Email had 1 attachment:
> + signature.asc
>  1k (application/pgp-signature)

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.common-lisp.net/pipermail/usocket-devel/attachments/20180712/614bb4b5/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 235 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <https://mailman.common-lisp.net/pipermail/usocket-devel/attachments/20180712/614bb4b5/attachment.sig>