[hunchentoot-devel] googlebot revisitation rate excessive?

[Jeff Cunningham]

Zach Beane wrote:
> On Sun, Jun 29, 2008 at 05:01:06PM -0700, Jeff Cunningham wrote:
>
>   
>> Is there something you do to indicate that the page has a fixed date? I 
>> was wondering if maybe the fact that dynamically generated pages look 
>> "changed" to Google, so they get thrown back in the queue to be rechecked.
>>     
>
> Nope, I don't do anything like that.
>   

Well, here's another hypothesis: maybe Google isn't happy with the 
response it is getting from my sites. My error log shows this sequence:

[2008-06-29 16:25:23 [INFO]] No session for session identifier 
'491:2BB8EA11136C90E6BA7D7F466951E370' (User-Agent: 'Mozilla/5.0 
(compatible; Googlebot/2.1; +http://www.google.com/bot.html)', IP: 
'127.0.0.1')
[2008-06-29 16:25:23 [WARNING]] Warning while processing connection: 
Unexpected character , after <meta
[2008-06-29 16:25:43 [ERROR]] Error while processing connection: I/O 
timeout reading #<SB-SYS:FD-STREAM for "a socket" {195190D9}>.
[2008-06-29 16:27:27 [INFO]] No session for session identifier 
'481:C9244EC27C31213FFE797F9E2ABE1535' (User-Agent: 'Mozilla/5.0 
(compatible; Googlebot/2.1; +http://www.google.com/bot.html)', IP: 
'127.0.0.1')
[2008-06-29 16:27:27 [WARNING]] Warning while processing connection: 
Unexpected character , after <meta


I've looked high and low for bad syntax involving generation of a meta 
tag, but it isn't there. I think it is an artifact of the timeout or 
something. Anyway, I'm wondering if the googlebot doesn't like the 
response my server is giving it, doesn't respond, the server waits, 
times out, then finally the googlebot gets back to it and by that time 
the session identifier is bad. Any thoughts?

--Jeff

> Zach
> _______________________________________________
> tbnl-devel site list
> tbnl-devel at common-lisp.net
> http://common-lisp.net/mailman/listinfo/tbnl-devel
>
>