[cl-pdf-devel] lww 4.3.7. version of cl-pdf can't handle big pdf file

[Chisheng Huang]

Hi,

While generating a PDF scatterplot of ~20000 circles, my LWW 4.3.7 went
belly up:
  Error: An error of type CONDITIONS::SIGNAL-HANDLER-ERROR occured, arguments : (:
  SIGNAL-CONDITION #<SIMPLE-ERROR 2A6F52FC> :REAL-CONDITION #<SIMPLE-ERROR 2A6F52D
  C>)
    1 (continue) Return from callback function
    2 Re-invoke callback function
    3 Return from callback function
    4 Re-invoke callback function
    5 (abort) Quit process.

  Type :b for backtrace, :c <option number> to proceed,  or :? for other options

  CL-USER 1 : 1 > :b

  Call to DBG::GET-CALL-FRAME
  Call to DBG::DEBUG1
  Call to INVOKE-DEBUGGER
  Call to FLI:REPLACE-FOREIGN-OBJECT
  Call to FLI::LISP-TO-C-STRING1
  Call to PDF::COMPRESS-STRING
  Call to (METHOD PDF::WRITE-OBJECT (PDF::PDF-STREAM))

I'm using cl-pdf-2.1 and the PDF::COMPRESS-STRING defined in 
contrib/zlib-lw.lisp.  According to the backtrace, it looks like I hit an 
internal upper limit in LWW 4.3.7.  To get around the problem, I hacked 
WRITE-OBJECT in pdf.lisp a bit:

(defmethod write-object ((obj pdf-stream) &optional root-level)
  (declare (ignorable root-level))
  (when (and *compress-streams* (not (no-compression obj))
	     (> (length (content obj)) *min-size-for-compression*))
    ;; Or put the HANDLER-CASE in COMPRESS-STRING ?     - cph 01-Nov-2004
    (let ((compressed (handler-case
			  (compress-string (content obj))
			(error ()
			  nil))))
      (when compressed
	(setf (content obj) compressed)
	(let ((filter (get-dict-value obj "/Filter")))
	  (if filter
	      (change-dict-value obj "/Filter" (vector "/FlateDecode" filter))
	      (add-dict-value obj "/Filter" "/FlateDecode"))))))
  ;; replaced by the above (WHEN ...).   - cph 01-Nov-2004
  #+ignore
  (when (and *compress-streams* (not (no-compression obj))
	     (> (length (content obj)) *min-size-for-compression*))
    (setf (content obj) (compress-string (content obj)))
    (let ((filter (get-dict-value obj "/Filter")))
      (if filter
	  (change-dict-value obj "/Filter" (vector "/FlateDecode" filter))
	  (add-dict-value obj "/Filter" "/FlateDecode"))))
  (call-next-method)
  (write-line "stream" *pdf-stream*)
  #+pdf-binary
  (write-sequence (content obj) *pdf-stream*)
  #-pdf-binary
  (if (stringp (content obj))
      (write-sequence (content obj) *pdf-stream*)
      (loop for c across (content obj) do
	(write-char (code-char c) *pdf-stream*)))
  (write-char #\Newline *pdf-stream*)
  (write-line "endstream" *pdf-stream*))

Maybe this hack will be useful to someones.

Best,

-cph

ps. after a bit of testing, i think the longest string that lww 4.3.7 can
    handle is between 2000000 and 2100000.  generating a scatterplot of
    ~20000 circles in pdf produces a string of length 4433794, which 
    can be happily handled by the cmucl version of cl-pdf without having
    to resort to the above hack.