Multiple fixes to allow PDFio to read more edge-case PDFs.

- Update _pdfioFileGets to allow for really long lines where it doesn't matter if we lose the end of the line. - Update "startxref" detection at the end of the file. - Refactor repair logic so that you just get a single WARNING about the repair (debug messages available for testing) - Allow whitespace after the "obj" in the object header. - Make sure to close xref stream on error. - Update predictor code to support Colors <= 32 (some implementations set Colors to the number of bytes per record in the xref stream, which prevents the predictor from doing anything...) - Allow CR CR in xref table. - Clear old trailer/root/pages/etc. objects when repairing, update existing objects that were already found in load_xref. - Don't set current object in pdfioObjectCreate/OpenStream if the stream can't be created/opened.
Clarify error callback API, and actually use the return value.
2025-06-02 03:34:23 +02:00 · 2025-04-24 11:09:54 -04:00 · 2025-04-23 14:43:14 -04:00
11 changed files with 307 additions and 137 deletions
--- a/doc/pdfio.3
+++ b/doc/pdfio.3
@ -1,4 +1,4 @@
-.TH pdfio 3 "pdf read/write library" "2025-04-13" "pdf read/write library"
+.TH pdfio 3 "pdf read/write library" "2025-04-23" "pdf read/write library"
 .SH NAME
 pdfio \- pdf read/write library
 .SH Introduction
@ -325,7 +325,7 @@ where the five arguments to the function are the filename ("myinputfile.pdf"), a
    }
 .fi
 .PP
-The error callback is called for both errors and warnings and accepts the pdfio_file_t pointer, a message string, and the callback pointer value, for example:
+The error callback is called for both errors and warnings and accepts the pdfio_file_t pointer, a message string, and the callback pointer value. It returns true to continue processing the file or false to stop, for example:
 .nf

    bool
@ -335,12 +335,15 @@ The error callback is called for both errors and warnings and accepts the pdfio_
    
      fprintf(stderr, "%s: %s\\n", pdfioFileGetName(pdf), message);
    
-      // Return false to treat warnings as errors
-      return (false);
+      // Return true for warning messages (continue) and false for errors (stop)
+      return (!strncmp(message, "WARNING:", 8));
    }
 .fi
 .PP
 The default error callback (NULL) does the equivalent of the above.
+.PP
+Note: Many errors are unrecoverable, so PDFio ignores the return value from the error callback and always stops processing the PDF file. Warning messages start with the prefix "WARNING:" while errors have no prefix.
+
 .PP
 Each PDF file contains one or more pages. The pdfioFileGetNumPages function returns the number of pages in the file while the pdfioFileGetPage function gets the specified page in the PDF file:
 .nf
@ -3910,8 +3913,9 @@ CropBox for pages in the PDF file - if \fBNULL\fR then a default "Universal" siz
 of 8.27x11in (the intersection of US Letter and ISO A4) is used.
 .PP
 The "error_cb" and "error_cbdata" arguments specify an error handler callback
-and its data pointer - if \fBNULL\fR the default error handler is used that
-writes error messages to \fBstderr\fR.
+and its data pointer - if \fBNULL\fR then the default error handler is used that
+writes error messages to \fBstderr\fR.  The error handler callback should return
+\fBtrue\fR to continue writing the PDF file or \fBfalse\fR to stop.
 .SS pdfioFileCreateArrayObj
 Create a new object in a PDF file containing an array.
 .PP
@ -4152,8 +4156,9 @@ CropBox for pages in the PDF file - if \fBNULL\fR then a default "Universal" siz
 of 8.27x11in (the intersection of US Letter and ISO A4) is used.
 .PP
 The "error_cb" and "error_cbdata" arguments specify an error handler callback
-and its data pointer - if \fBNULL\fR the default error handler is used that
-writes error messages to \fBstderr\fR.
+and its data pointer - if \fBNULL\fR then the default error handler is used that
+writes error messages to \fBstderr\fR.  The error handler callback should return
+\fBtrue\fR to continue writing the PDF file or \fBfalse\fR to stop.
 .PP
 .IP 5
 \fINote\fR: Files created using this API are slightly larger than those
@ -4392,8 +4397,18 @@ cancel the open.  If \fBNULL\fR is specified for the callback function and the
 PDF file requires a password, the open will always fail.
 .PP
 The "error_cb" and "error_cbdata" arguments specify an error handler callback
-and its data pointer - if \fBNULL\fR the default error handler is used that
-writes error messages to \fBstderr\fR.
+and its data pointer - if \fBNULL\fR then the default error handler is used that
+writes error messages to \fBstderr\fR.  The error handler callback should return
+\fBtrue\fR to continue reading the PDF file or \fBfalse\fR to stop.
+.PP
+.IP 5
+Note: Error messages starting with "WARNING:" are actually warning
+.IP 5
+messages - the callback should normally return \fBtrue\fR to allow PDFio to
+.IP 5
+try to resolve the issue.  In addition, some errors are unrecoverable and
+.IP 5
+ignore the return value of the error callback.
 .SS pdfioFileSetAuthor
 Set the author for a PDF file.
 .PP
--- a/doc/pdfio.html
+++ b/doc/pdfio.html
@ -732,7 +732,7 @@ password_cb(<span class="reserved">void</span> *data, <span class="reserved">con
  <span class="reserved">return</span> (<span class="string">&quot;Password42&quot;</span>);
 }
 </code></pre>
-<p>The error callback is called for both errors and warnings and accepts the <code>pdfio_file_t</code> pointer, a message string, and the callback pointer value, for example:</p>
+<p>The error callback is called for both errors and warnings and accepts the <code>pdfio_file_t</code> pointer, a message string, and the callback pointer value. It returns <code>true</code> to continue processing the file or <code>false</code> to stop, for example:</p>
 <pre><code class="language-c"><span class="reserved">bool</span>
 error_cb(pdfio_file_t *pdf, <span class="reserved">const</span> <span class="reserved">char</span> *message, <span class="reserved">void</span> *data)
 {
@ -740,11 +740,14 @@ error_cb(pdfio_file_t *pdf, <span class="reserved">const</span> <span class="res

  fprintf(stderr, <span class="string">&quot;%s: %s\n&quot;</span>, pdfioFileGetName(pdf), message);

-  <span class="comment">// Return false to treat warnings as errors</span>
-  <span class="reserved">return</span> (<span class="reserved">false</span>);
+  <span class="comment">// Return true for warning messages (continue) and false for errors (stop)</span>
+  <span class="reserved">return</span> (!strncmp(message, <span class="string">&quot;WARNING:&quot;</span>, <span class="number">8</span>));
 }
 </code></pre>
 <p>The default error callback (<code>NULL</code>) does the equivalent of the above.</p>
+<blockquote>
+<p>Note: Many errors are unrecoverable, so PDFio ignores the return value from the error callback and always stops processing the PDF file. Warning messages start with the prefix &quot;WARNING:&quot; while errors have no prefix.</p>
+</blockquote>
 <p>Each PDF file contains one or more pages. The <a href="#pdfioFileGetNumPages"><code>pdfioFileGetNumPages</code></a> function returns the number of pages in the file while the <a href="#pdfioFileGetPage"><code>pdfioFileGetPage</code></a> function gets the specified page in the PDF file:</p>
 <pre><code class="language-c">pdfio_file_t *pdf;   <span class="comment">// PDF file</span>
 size_t       i;      <span class="comment">// Looping var</span>
@ -4129,8 +4132,9 @@ CropBox for pages in the PDF file - if <code>NULL</code> then a default &quot;Un
 of 8.27x11in (the intersection of US Letter and ISO A4) is used.<br>
 <br>
 The &quot;error_cb&quot; and &quot;error_cbdata&quot; arguments specify an error handler callback
-and its data pointer - if <code>NULL</code> the default error handler is used that
-writes error messages to <code>stderr</code>.</p>
+and its data pointer - if <code>NULL</code> then the default error handler is used that
+writes error messages to <code>stderr</code>.  The error handler callback should return
+<code>true</code> to continue writing the PDF file or <code>false</code> to stop.</p>
 <h3 class="function"><a id="pdfioFileCreateArrayObj">pdfioFileCreateArrayObj</a></h3>
 <p class="description">Create a new object in a PDF file containing an array.</p>
 <p class="code">
@ -4434,8 +4438,9 @@ CropBox for pages in the PDF file - if <code>NULL</code> then a default &quot;Un
 of 8.27x11in (the intersection of US Letter and ISO A4) is used.<br>
 <br>
 The &quot;error_cb&quot; and &quot;error_cbdata&quot; arguments specify an error handler callback
-and its data pointer - if <code>NULL</code> the default error handler is used that
-writes error messages to <code>stderr</code>.<br>
+and its data pointer - if <code>NULL</code> then the default error handler is used that
+writes error messages to <code>stderr</code>.  The error handler callback should return
+<code>true</code> to continue writing the PDF file or <code>false</code> to stop.<br>
 <br>
 </p><blockquote>
 <em>Note</em>: Files created using this API are slightly larger than those
@ -4772,8 +4777,15 @@ cancel the open.  If <code>NULL</code> is specified for the callback function an
 PDF file requires a password, the open will always fail.<br>
 <br>
 The &quot;error_cb&quot; and &quot;error_cbdata&quot; arguments specify an error handler callback
-and its data pointer - if <code>NULL</code> the default error handler is used that
-writes error messages to <code>stderr</code>.</p>
+and its data pointer - if <code>NULL</code> then the default error handler is used that
+writes error messages to <code>stderr</code>.  The error handler callback should return
+<code>true</code> to continue reading the PDF file or <code>false</code> to stop.<br>
+<br>
+</p><blockquote>
+Note: Error messages starting with &quot;WARNING:&quot; are actually warning
+messages - the callback should normally return <code>true</code> to allow PDFio to
+try to resolve the issue.  In addition, some errors are unrecoverable and
+ignore the return value of the error callback.</blockquote>
 <h3 class="function"><a id="pdfioFileSetAuthor">pdfioFileSetAuthor</a></h3>
 <p class="description">Set the author for a PDF file.</p>
 <p class="code">
--- a/doc/pdfio.md
+++ b/doc/pdfio.md
@ -343,8 +343,8 @@ password_cb(void *data, const char *filename)
 ```

 The error callback is called for both errors and warnings and accepts the
-`pdfio_file_t` pointer, a message string, and the callback pointer value, for
-example:
+`pdfio_file_t` pointer, a message string, and the callback pointer value. It
+returns `true` to continue processing the file or `false` to stop, for example:

 ```c
 bool
@ -354,13 +354,17 @@ error_cb(pdfio_file_t *pdf, const char *message, void *data)

  fprintf(stderr, "%s: %s\n", pdfioFileGetName(pdf), message);

-  // Return false to treat warnings as errors
-  return (false);
+  // Return true for warning messages (continue) and false for errors (stop)
+  return (!strncmp(message, "WARNING:", 8));
 }
 ```

 The default error callback (`NULL`) does the equivalent of the above.

+> Note: Many errors are unrecoverable, so PDFio ignores the return value from
+> the error callback and always stops processing the PDF file.  Warning messages
+> start with the prefix "WARNING:" while errors have no prefix.
+
 Each PDF file contains one or more pages.  The [`pdfioFileGetNumPages`](@@)
 function returns the number of pages in the file while the
 [`pdfioFileGetPage`](@@) function gets the specified page in the PDF file:
--- a/pdfio-common.c
+++ b/pdfio-common.c
@ -47,7 +47,7 @@ _pdfioFileConsume(pdfio_file_t *pdf,	// I - PDF file
 // `false` to halt.
 //

-bool					// O - `false` to stop
+bool					// O - `false` to stop, `true` to continue
 _pdfioFileDefaultError(
    pdfio_file_t *pdf,			// I - PDF file
    const char   *message,		// I - Error message
@ -57,7 +57,7 @@ _pdfioFileDefaultError(

  fprintf(stderr, "%s: %s\n", pdf->filename, message);

-  return (false);
+  return (!strncmp(message, "WARNING:", 8));
 }


@ -134,19 +134,20 @@ _pdfioFileGetChar(pdfio_file_t *pdf)	// I - PDF file
 bool					// O - `true` on success, `false` on error
 _pdfioFileGets(pdfio_file_t *pdf,	// I - PDF file
               char         *buffer,	// I - Line buffer
-	       size_t       bufsize)	// I - Size of line buffer
+	       size_t       bufsize,	// I - Size of line buffer
+	       bool         discard)	// I - OK to discard excess line chars?
 {
  bool	eol = false;			// End of line?
  char	*bufptr = buffer,		// Pointer into buffer
 	*bufend = buffer + bufsize - 1;	// Pointer to end of buffer


-  PDFIO_DEBUG("_pdfioFileGets(pdf=%p, buffer=%p, bufsize=%lu) bufpos=%ld, buffer=%p, bufptr=%p, bufend=%p, offset=%lu\n", pdf, buffer, (unsigned long)bufsize, (long)pdf->bufpos, pdf->buffer, pdf->bufptr, pdf->bufend, (unsigned long)(pdf->bufpos + (pdf->bufptr - pdf->buffer)));
+  PDFIO_DEBUG("_pdfioFileGets(pdf=%p, buffer=%p, bufsize=%lu, discard=%s) bufpos=%ld, buffer=%p, bufptr=%p, bufend=%p, offset=%lu\n", pdf, buffer, (unsigned long)bufsize, discard ? "true" : "false", (long)pdf->bufpos, pdf->buffer, pdf->bufptr, pdf->bufend, (unsigned long)(pdf->bufpos + (pdf->bufptr - pdf->buffer)));

  while (!eol)
  {
    // If there are characters ready in the buffer, use them...
-    while (!eol && pdf->bufptr < pdf->bufend && bufptr < bufend)
+    while (!eol && pdf->bufptr < pdf->bufend)
    {
      char ch = *(pdf->bufptr++);	// Next character in buffer

@ -168,8 +169,10 @@ _pdfioFileGets(pdfio_file_t *pdf,	// I - PDF file
 	    pdf->bufptr ++;
 	}
      }
-      else
+      else if (bufptr < bufend)
        *bufptr++ = ch;
+      else if (!discard)
+        break;
    }

    // Fill the read buffer as needed...
--- a/pdfio-dict.c
+++ b/pdfio-dict.c
@ -643,9 +643,11 @@ _pdfioDictRead(pdfio_file_t   *pdf,	// I - PDF file
    {
      // Issue 118: Discard duplicate key/value pairs, in the future this will
      // be a warning message...
-      _pdfioFileError(pdf, "WARNING: Discarding value for duplicate dictionary key '%s'.", key + 1);
      _pdfioValueDelete(&value);
-      continue;
+      if (_pdfioFileError(pdf, "WARNING: Discarding value for duplicate dictionary key '%s'.", key + 1))
+        continue;
+      else
+        break;
    }
    else if (!_pdfioDictSetValue(dict, pdfioStringCreate(pdf, key + 1), &value))
      break;
--- a/pdfio-file.c
+++ b/pdfio-file.c
@ -188,8 +188,9 @@ pdfioFileClose(pdfio_file_t *pdf)	// I - PDF file
 // of 8.27x11in (the intersection of US Letter and ISO A4) is used.
 //
 // The "error_cb" and "error_cbdata" arguments specify an error handler callback
-// and its data pointer - if `NULL` the default error handler is used that
-// writes error messages to `stderr`.
+// and its data pointer - if `NULL` then the default error handler is used that
+// writes error messages to `stderr`.  The error handler callback should return
+// `true` to continue writing the PDF file or `false` to stop.
 //

 pdfio_file_t *				// O - PDF file or `NULL` on error
@ -426,8 +427,9 @@ _pdfioFileCreateObj(
 // of 8.27x11in (the intersection of US Letter and ISO A4) is used.
 //
 // The "error_cb" and "error_cbdata" arguments specify an error handler callback
-// and its data pointer - if `NULL` the default error handler is used that
-// writes error messages to `stderr`.
+// and its data pointer - if `NULL` then the default error handler is used that
+// writes error messages to `stderr`.  The error handler callback should return
+// `true` to continue writing the PDF file or `false` to stop.
 //
 // > *Note*: Files created using this API are slightly larger than those
 // > created using the @link pdfioFileCreate@ function since stream lengths are
@ -1019,8 +1021,14 @@ pdfioFileGetVersion(
 // PDF file requires a password, the open will always fail.
 //
 // The "error_cb" and "error_cbdata" arguments specify an error handler callback
-// and its data pointer - if `NULL` the default error handler is used that
-// writes error messages to `stderr`.
+// and its data pointer - if `NULL` then the default error handler is used that
+// writes error messages to `stderr`.  The error handler callback should return
+// `true` to continue reading the PDF file or `false` to stop.
+//
+// > Note: Error messages starting with "WARNING:" are actually warning
+// > messages - the callback should normally return `true` to allow PDFio to
+// > try to resolve the issue.  In addition, some errors are unrecoverable and
+// > ignore the return value of the error callback.
 //

 pdfio_file_t *				// O - PDF file
@ -1081,7 +1089,7 @@ pdfioFileOpen(
  }

  // Read the header from the first line...
-  if (!_pdfioFileGets(pdf, line, sizeof(line)))
+  if (!_pdfioFileGets(pdf, line, sizeof(line), true))
    goto error;

  if ((strncmp(line, "%PDF-1.", 7) && strncmp(line, "%PDF-2.", 7)) || !isdigit(line[7] & 255))
@ -1095,7 +1103,7 @@ pdfioFileOpen(
  pdf->version = strdup(line + 5);

  // Grab the last 1k of the file to find the start of the xref table...
-  if (_pdfioFileSeek(pdf, -1024, SEEK_END) < 0)
+  if (_pdfioFileSeek(pdf, 1 - sizeof(line), SEEK_END) < 0)
  {
    _pdfioFileError(pdf, "Unable to read startxref data.");
    goto error;
@ -1107,28 +1115,36 @@ pdfioFileOpen(
    goto error;
  }

+  PDFIO_DEBUG("pdfioOpen: Read %d bytes at end of file.\n", (int)bytes);
+
  line[bytes] = '\0';
  end = line + bytes - 9;

  for (ptr = line; ptr < end; ptr ++)
  {
-    if (!memcmp(ptr, "startxref", 9))
+    if (!strncmp(ptr, "startxref", 9) && !strstr(ptr + 9, "startxref") && strtol(ptr + 9, NULL, 10) > 0)
      break;
  }

  if (ptr >= end)
  {
-    _pdfioFileError(pdf, "Unable to find start of xref table.");
-    goto error;
-  }
+    if (!_pdfioFileError(pdf, "WARNING: Unable to find start of cross-reference table, will attempt to rebuild."))
+      goto error;

-  xref_offset = (off_t)strtol(ptr + 9, NULL, 10);
-
-  if (!load_xref(pdf, xref_offset, password_cb, password_cbdata))
-  {
    if (!repair_xref(pdf, password_cb, password_cbdata))
      goto error;
  }
+  else
+  {
+    PDFIO_DEBUG("pdfioFileOpen: line=%p,ptr=%p(\"%s\")\n", line, ptr, ptr);
+
+    xref_offset = (off_t)strtol(ptr + 9, NULL, 10);
+
+    PDFIO_DEBUG("pdfioFileOpen: xref_offset=%lu\n", (unsigned long)xref_offset);
+
+    if (!load_xref(pdf, xref_offset, password_cb, password_cbdata))
+      goto error;
+  }

  return (pdf);

@ -1755,7 +1771,10 @@ load_pages(pdfio_file_t *pdf,		// I - PDF file
  }

  if ((type = pdfioDictGetName(dict, "Type")) == NULL || (strcmp(type, "Pages") && strcmp(type, "Page")))
-    return (false);
+  {
+    if (!_pdfioFileError(pdf, "WARNING: No Type value for pages object."))
+      return (false);
+  }

  // If there is a Kids array, then this is a parent node and we have to look
  // at the child objects...
@ -1822,31 +1841,32 @@ load_xref(
  int		generation;		// Generation number
  _pdfio_token_t tb;			// Token buffer/stack
  off_t		line_offset;		// Offset to start of line
+  pdfio_obj_t	*pages_obj;		// Pages object


  while (!done)
  {
    if (_pdfioFileSeek(pdf, xref_offset, SEEK_SET) != xref_offset)
    {
-      _pdfioFileError(pdf, "Unable to seek to start of xref table.");
-      return (false);
+      PDFIO_DEBUG("load_xref: Unable to seek to %lu.\n", (unsigned long)xref_offset);
+      goto repair;
    }

    do
    {
      line_offset = _pdfioFileTell(pdf);

-      if (!_pdfioFileGets(pdf, line, sizeof(line)))
+      if (!_pdfioFileGets(pdf, line, sizeof(line), true))
      {
-	_pdfioFileError(pdf, "Unable to read start of xref table.");
-	return (false);
+	PDFIO_DEBUG("load_xref: Unable to read line at offset %lu.\n", (unsigned long)line_offset);
+        goto repair;
      }
    }
    while (!line[0]);

    PDFIO_DEBUG("load_xref: line_offset=%lu, line='%s'\n", (unsigned long)line_offset, line);

-    if (isdigit(line[0] & 255) && strlen(line) > 4 && (!strcmp(line + strlen(line) - 4, " obj") || ((ptr = strstr(line, " obj")) != NULL && ptr[4] == '<')))
+    if (isdigit(line[0] & 255) && strlen(line) > 4 && (!strcmp(line + strlen(line) - 4, " obj") || ((ptr = strstr(line, " obj")) != NULL && (ptr[4] == '<' || isspace(ptr[4])))))
    {
      // Cross-reference stream
      pdfio_obj_t	*obj;		// Object
@ -1868,14 +1888,14 @@ load_xref(

      if ((number = strtoimax(line, &ptr, 10)) < 1)
      {
-	_pdfioFileError(pdf, "Bad xref table header '%s'.", line);
-	return (false);
+        PDFIO_DEBUG("load_xref: Unable to scan object number.\n");
+        goto repair;
      }

      if ((generation = (int)strtol(ptr, &ptr, 10)) < 0 || (generation > 65535 && number != 0))
      {
-	_pdfioFileError(pdf, "Bad xref table header '%s'.", line);
-	return (false);
+        PDFIO_DEBUG("load_xref: Unable to scan generation number (%u).\n", (unsigned)generation);
+        goto repair;
      }

      while (isspace(*ptr & 255))
@ -1883,14 +1903,14 @@ load_xref(

      if (strncmp(ptr, "obj", 3))
      {
-	_pdfioFileError(pdf, "Bad xref table header '%s'.", line);
-	return (false);
+        PDFIO_DEBUG("load_xref: No 'obj' after object number and generation (saw '%s').\n", ptr);
+        goto repair;
      }

      if (_pdfioFileSeek(pdf, line_offset + (off_t)(ptr + 3 - line), SEEK_SET) < 0)
      {
-        _pdfioFileError(pdf, "Unable to seek to xref object %lu %u.", (unsigned long)number, (unsigned)generation);
-        return (false);
+        PDFIO_DEBUG("load_xref: Unable to seek to start of cross-reference object dictionary.\n");
+        goto repair;
      }

      PDFIO_DEBUG("load_xref: Loading object %lu %u.\n", (unsigned long)number, (unsigned)generation);
@ -1905,21 +1925,21 @@ load_xref(

      if (!_pdfioValueRead(pdf, obj, &tb, &trailer, 0))
      {
-        _pdfioFileError(pdf, "Unable to read cross-reference stream dictionary.");
-        return (false);
+        PDFIO_DEBUG("load_xref: Unable to read cross-reference object dictionary.\n");
+        goto repair;
      }
      else if (trailer.type != PDFIO_VALTYPE_DICT)
      {
-	_pdfioFileError(pdf, "Cross-reference stream does not have a dictionary.");
-	return (false);
+        PDFIO_DEBUG("load_xref: Expected dictionary for cross-reference object (type=%d).", trailer.type);
+        goto repair;
      }

      obj->value = trailer;

      if (!_pdfioTokenGet(&tb, line, sizeof(line)) || strcmp(line, "stream"))
      {
-        _pdfioFileError(pdf, "Unable to get stream after xref dictionary.");
-        return (false);
+        PDFIO_DEBUG("load_xref: No stream token after dictionary (got '%s').\n", line);
+        goto repair;
      }

      PDFIO_DEBUG("load_xref: tb.bufptr=%p, tb.bufend=%p, tb.bufptr[0]=0x%02x, tb.bufptr[0]=0x%02x\n", tb.bufptr, tb.bufend, tb.bufptr[0], tb.bufptr[1]);
@ -1937,8 +1957,8 @@ load_xref(

      if ((w_array = pdfioDictGetArray(trailer.value.dict, "W")) == NULL)
      {
-	_pdfioFileError(pdf, "Cross-reference stream does not have required W key.");
-	return (false);
+        PDFIO_DEBUG("load_xref: Missing W array in cross-reference objection dictionary.\n");
+        goto repair;
      }

      w[0]    = (size_t)pdfioArrayGetNumber(w_array, 0);
@ -1948,16 +1968,18 @@ load_xref(
      w_2     = w[0];
      w_3     = w[0] + w[1];

-      if (w[1] == 0 || w[2] > 4 || w[0] > sizeof(buffer) || w[1] > sizeof(buffer) || w[2] > sizeof(buffer) || w_total > sizeof(buffer))
+      PDFIO_DEBUG("W=[%u %u %u], w_total=%u\n", (unsigned)w[0], (unsigned)w[1], (unsigned)w[2], (unsigned)w_total);
+
+      if (pdfioArrayGetSize(w_array) > 3 || w[1] == 0 || w[2] > 4 || w[0] > sizeof(buffer) || w[1] > sizeof(buffer) || w[2] > sizeof(buffer) || w_total > sizeof(buffer))
      {
-	_pdfioFileError(pdf, "Cross-reference stream has invalid W key [%u %u %u].", (unsigned)w[0], (unsigned)w[1], (unsigned)w[2]);
-	return (false);
+        PDFIO_DEBUG("load_xref: Bad W array in cross-reference objection dictionary.\n");
+        goto repair;
      }

      if ((st = pdfioObjOpenStream(obj, true)) == NULL)
      {
-	_pdfioFileError(pdf, "Unable to open cross-reference stream.");
-	return (false);
+        PDFIO_DEBUG("load_xref: Unable to open cross-reference stream.\n");
+        goto repair;
      }

      for (index_n = 0; index_n < index_count; index_n += 2)
@ -1977,7 +1999,20 @@ load_xref(
 	{
 	  count --;

-	  PDFIO_DEBUG("load_xref: number=%u %02X%02X%02X%02X%02X\n", (unsigned)number, buffer[0], buffer[1], buffer[2], buffer[3], buffer[4]);
+#ifdef DEBUG
+          if (w_total > 5)
+	    PDFIO_DEBUG("load_xref: number=%u %02X%02X%02X%02X%02X...\n", (unsigned)number, buffer[0], buffer[1], buffer[2], buffer[3], buffer[4]);
+	  else if (w_total == 5)
+	    PDFIO_DEBUG("load_xref: number=%u %02X%02X%02X%02X%02X\n", (unsigned)number, buffer[0], buffer[1], buffer[2], buffer[3], buffer[4]);
+	  else if (w_total == 4)
+	    PDFIO_DEBUG("load_xref: number=%u %02X%02X%02X%02X\n", (unsigned)number, buffer[0], buffer[1], buffer[2], buffer[3]);
+	  else if (w_total == 3)
+	    PDFIO_DEBUG("load_xref: number=%u %02X%02X%02X\n", (unsigned)number, buffer[0], buffer[1], buffer[2]);
+	  else if (w_total == 2)
+	    PDFIO_DEBUG("load_xref: number=%u %02X%02X\n", (unsigned)number, buffer[0], buffer[1]);
+	  else
+	    PDFIO_DEBUG("load_xref: number=%u %02X\n", (unsigned)number, buffer[0]);
+#endif // DEBUG

 	  // Check whether this is an object definition...
 	  if (w[0] > 0)
@ -2059,6 +2094,7 @@ load_xref(
 	      else
 	      {
 		_pdfioFileError(pdf, "Too many object streams.");
+		pdfioStreamClose(st);
 	        return (false);
 	      }
 	    }
@ -2067,7 +2103,10 @@ load_xref(
 	  {
 	    // Add this object...
 	    if (!add_obj(pdf, (size_t)number, (unsigned short)generation, (off_t)offset))
+	    {
+	      pdfioStreamClose(st);
 	      return (false);
+	    }
 	  }

 	  number ++;
@ -2115,7 +2154,7 @@ load_xref(
 					// Offset of current line

      PDFIO_DEBUG("load_xref: Reading xref table starting at offset %lu\n", (unsigned long)trailer_offset);
-      while (_pdfioFileGets(pdf, line, sizeof(line)))
+      while (_pdfioFileGets(pdf, line, sizeof(line), false))
      {
        PDFIO_DEBUG("load_xref: '%s' at offset %lu\n", line, (unsigned long)trailer_offset);

@ -2140,8 +2179,8 @@ load_xref(

 	if (sscanf(line, "%jd%jd", &number, &num_objects) != 2)
 	{
-	  _pdfioFileError(pdf, "Malformed xref table section '%s'.", line);
-	  return (false);
+	  PDFIO_DEBUG("load_xref: Unable to scan START COUNT from line.\n");
+	  goto repair;
 	}

 	// Read this group of objects...
@ -2149,41 +2188,45 @@ load_xref(
 	{
 	  // Read a line from the file and validate it...
 	  if (_pdfioFileRead(pdf, line, 20) != 20)
-	    return (false);
+	  {
+	    PDFIO_DEBUG("load_xref: Unable to read 20 byte xref record.\n");
+	    goto repair;
+	  }

 	  line[20] = '\0';

-	  if (strcmp(line + 18, "\r\n") && strcmp(line + 18, " \n") && strcmp(line + 18, " \r"))
+	  if (strcmp(line + 18, "\r\n") && strcmp(line + 18, "\r\r") && strcmp(line + 18, " \n") && strcmp(line + 18, " \r"))
 	  {
-	    _pdfioFileError(pdf, "Malformed xref table entry '%s'.", line);
-	    return (false);
+	    PDFIO_DEBUG("load_xref: Bad end-of-line <%02X%02X>\n", line[18], line[19]);
+	    goto repair;
 	  }
+
 	  line[18] = '\0';

 	  // Parse the line
 	  if ((offset = strtoimax(line, &ptr, 10)) < 0)
 	  {
-	    _pdfioFileError(pdf, "Malformed xref table entry '%s'.", line);
-	    return (false);
+	    PDFIO_DEBUG("load_xref: Unable to scan offset.\n");
+	    goto repair;
 	  }

 	  if ((generation = (int)strtol(ptr, &ptr, 10)) < 0 || (generation > 65535 && offset != 0))
 	  {
-	    _pdfioFileError(pdf, "Malformed xref table entry '%s'.", line);
-	    return (false);
+	    PDFIO_DEBUG("load_xref: Unable to scan generation (%u).\n", (unsigned)generation);
+	    goto repair;
 	  }

 	  if (*ptr != ' ')
 	  {
-	    _pdfioFileError(pdf, "Malformed xref table entry '%s'.", line);
-	    return (false);
+	    PDFIO_DEBUG("load_xref: Missing space before type.\n");
+	    goto repair;
 	  }

 	  ptr ++;
 	  if (*ptr != 'f' && *ptr != 'n')
 	  {
-	    _pdfioFileError(pdf, "Malformed xref table entry '%s'.", line);
-	    return (false);
+	    PDFIO_DEBUG("load_xref: Bad type '%c'.\n", *ptr);
+	    goto repair;
 	  }

 	  if (*ptr == 'f')
@ -2202,21 +2245,21 @@ load_xref(

      if (strncmp(line, "trailer", 7))
      {
-	_pdfioFileError(pdf, "Missing trailer.");
-	return (false);
+        PDFIO_DEBUG("load_xref: No trailer after xref table.\n");
+	goto repair;
      }

      _pdfioTokenInit(&tb, pdf, (_pdfio_tconsume_cb_t)_pdfioFileConsume, (_pdfio_tpeek_cb_t)_pdfioFilePeek, pdf);

      if (!_pdfioValueRead(pdf, NULL, &tb, &trailer, 0))
      {
-	_pdfioFileError(pdf, "Unable to read trailer dictionary.");
-	return (false);
+        PDFIO_DEBUG("load_xref: Unable to read trailer dictionary.\n");
+	goto repair;
      }
      else if (trailer.type != PDFIO_VALTYPE_DICT)
      {
-	_pdfioFileError(pdf, "Trailer is not a dictionary.");
-	return (false);
+        PDFIO_DEBUG("load_xref: Trailer not a dictionary (type=%d).\n", trailer.type);
+	goto repair;
      }

      PDFIO_DEBUG("load_xref: Got trailer dict.\n");
@ -2238,8 +2281,7 @@ load_xref(
    }
    else
    {
-      _pdfioFileError(pdf, "Bad xref table header '%s'.", line);
-      return (false);
+      goto repair;
    }

    PDFIO_DEBUG("load_xref: Contents of trailer dictionary:\n");
@ -2268,13 +2310,31 @@ load_xref(

  if ((pdf->root_obj = pdfioDictGetObj(pdf->trailer_dict, "Root")) == NULL)
  {
-    _pdfioFileError(pdf, "Missing Root object.");
-    return (false);
+    PDFIO_DEBUG("load_xref: Missing Root object.\n");
+    goto repair;
  }

  PDFIO_DEBUG("load_xref: Root=%p(%lu)\n", pdf->root_obj, (unsigned long)pdf->root_obj->number);

-  return (load_pages(pdf, pdfioDictGetObj(pdfioObjGetDict(pdf->root_obj), "Pages"), 0));
+  if ((pages_obj = pdfioDictGetObj(pdfioObjGetDict(pdf->root_obj), "Pages")) == NULL)
+  {
+    PDFIO_DEBUG("load_xref: Missing Pages object.\n");
+    goto repair;
+  }
+
+  PDFIO_DEBUG("load_xref: Pages=%p(%lu)\n", pdf->root_obj, (unsigned long)pdf->root_obj->number);
+
+  return (load_pages(pdf, pages_obj, 0));
+
+  // If we get here the cross-reference table is busted - try repairing if the
+  // error callback says to proceed...
+
+  repair:
+
+  if (_pdfioFileError(pdf, "WARNING: Cross-reference is damaged, will attempt to rebuild."))
+    return (repair_xref(pdf, password_cb, password_data));
+  else
+    return (false);
 }


@ -2288,7 +2348,7 @@ repair_xref(
    pdfio_password_cb_t password_cb,	// I - Password callback or `NULL` for none
    void                *password_data)	// I - Password callback data, if any
 {
-  char		line[16384],		// Line from file
+  char		line[1024],		// Line from file
 		*ptr;			// Pointer into line
  off_t		line_offset;		// Offset in file
  intmax_t	number;			// Object number
@ -2296,16 +2356,22 @@ repair_xref(
  size_t	i;			// Looping var
  size_t	num_sobjs = 0;		// Number of object streams
  pdfio_obj_t	*sobjs[16384];		// Object streams to load
+  pdfio_dict_t	*backup_trailer = NULL;	// Backup trailer dictionary
+  pdfio_obj_t	*pages_obj;		// Pages object


-  // Let caller know something is wrong...
-  _pdfioFileError(pdf, "WARNING: Cross-reference table is damaged, attempting to rebuild.");
+  // Clear trailer data...
+  pdf->trailer_dict = NULL;
+  pdf->root_obj     = NULL;
+  pdf->info_obj     = NULL;
+  pdf->pages_obj    = NULL;
+  pdf->encrypt_obj  = NULL;

-  // Read from the beginning of the file, looking for
+  // Read from the beginning of the file, looking for objects...
  if ((line_offset = _pdfioFileSeek(pdf, 0, SEEK_SET)) < 0)
    return (false);

-  while (_pdfioFileGets(pdf, line, sizeof(line)))
+  while (_pdfioFileGets(pdf, line, sizeof(line), true))
  {
    // See if this is the start of an object...
    if (line[0] >= '1' && line[0] <= '9')
@ -2322,43 +2388,75 @@ repair_xref(
 	  pdfio_obj_t	*obj;		// Object
 	  _pdfio_token_t tb;		// Token buffer/stack

-          PDFIO_DEBUG("OBJECT %ld %d at offset %ld\n", (long)number, generation, (long)line_offset);
+          PDFIO_DEBUG("repair_xref: OBJECT %ld %d at offset %ld\n", (long)number, generation, (long)line_offset);

-	  if ((obj = add_obj(pdf, (size_t)number, (unsigned short)generation, line_offset)) == NULL)
+          if ((obj = pdfioFileFindObj(pdf, (size_t)number)) != NULL)
+          {
+            obj->offset = line_offset;
+          }
+          else if ((obj = add_obj(pdf, (size_t)number, (unsigned short)generation, line_offset)) == NULL)
 	  {
 	    _pdfioFileError(pdf, "Unable to allocate memory for object.");
 	    return (false);
 	  }

+          if (ptr[3])
+          {
+	    // Probably the start of the object dictionary, rewind the file so
+	    // we can read it...
+	    _pdfioFileSeek(pdf, line_offset + (ptr - line + 3), SEEK_SET);
+	  }
+
 	  _pdfioTokenInit(&tb, pdf, (_pdfio_tconsume_cb_t)_pdfioFileConsume, (_pdfio_tpeek_cb_t)_pdfioFilePeek, pdf);

 	  if (!_pdfioValueRead(pdf, obj, &tb, &obj->value, 0))
 	  {
-	    _pdfioFileError(pdf, "Unable to read cross-reference stream dictionary.");
-	    return (false);
+	    if (!_pdfioFileError(pdf, "WARNING: Unable to read object dictionary/value."))
+	      return (false);
+	    else
+	      continue;
 	  }

-	  if (_pdfioTokenGet(&tb, line, sizeof(line)) && strcmp(line, "stream"))
+	  if (_pdfioTokenGet(&tb, line, sizeof(line)))
 	  {
 	    const char *type = pdfioObjGetType(obj);
 					// Object type

 	    _pdfioTokenFlush(&tb);
-	    obj->stream_offset = _pdfioFileTell(pdf);

-	    if (type && !strcmp(type, "ObjStm") && num_sobjs < (sizeof(sobjs) / sizeof(sobjs[0])))
-	    {
-	      sobjs[num_sobjs] = obj;
-	      num_sobjs ++;
-	    }
-
-            if (type && !strcmp(type, "XRef") && !pdf->trailer_dict)
+            if (type && !strcmp(line, "stream"))
            {
-              // Save the trailer dictionary...
-	      pdf->trailer_dict = pdfioObjGetDict(obj);
-	      pdf->encrypt_obj  = pdfioDictGetObj(pdf->trailer_dict, "Encrypt");
-	      pdf->id_array     = pdfioDictGetArray(pdf->trailer_dict, "ID");
-            }
+              // Possible object or XRef stream...
+	      obj->stream_offset = _pdfioFileTell(pdf);
+
+	      if (!strcmp(type, "ObjStm") && num_sobjs < (sizeof(sobjs) / sizeof(sobjs[0])))
+	      {
+	        PDFIO_DEBUG("repair_xref: Object stream...\n");
+		sobjs[num_sobjs] = obj;
+		num_sobjs ++;
+	      }
+
+	      if (!strcmp(type, "XRef") && !pdf->trailer_dict)
+	      {
+		// Save the trailer dictionary...
+	        PDFIO_DEBUG("repair_xref: XRef stream...\n");
+		pdf->trailer_dict = pdfioObjGetDict(obj);
+		pdf->encrypt_obj  = pdfioDictGetObj(pdf->trailer_dict, "Encrypt");
+		pdf->id_array     = pdfioDictGetArray(pdf->trailer_dict, "ID");
+	      }
+	    }
+	    else if (type && !strcmp(line, "endobj"))
+	    {
+	      // Possible catalog or pages object...
+	      if (!strcmp(type, "Catalog"))
+	      {
+	        PDFIO_DEBUG("repair_xref: Catalog (root) object...\n");
+	        if (!backup_trailer)
+	          backup_trailer = pdfioDictCreate(pdf);
+
+	        pdfioDictSetObj(backup_trailer, "Root", obj);
+	      }
+	    }
 	  }
 	}
      }
@ -2369,6 +2467,8 @@ repair_xref(
      _pdfio_token_t tb;		// Token buffer/stack
      _pdfio_value_t trailer;		// Trailer

+      PDFIO_DEBUG("repair_xref: line=\"%s\"\n", line);
+
      if (line[7])
      {
 	// Probably the start of the trailer dictionary, rewind the file so
@ -2376,7 +2476,7 @@ repair_xref(
 	_pdfioFileSeek(pdf, line_offset + 7, SEEK_SET);
      }

-      PDFIO_DEBUG("TRAILER at offset %ld\n", (long)line_offset);
+      PDFIO_DEBUG("repair_xref: TRAILER at offset %ld\n", (long)line_offset);

      _pdfioTokenInit(&tb, pdf, (_pdfio_tconsume_cb_t)_pdfioFileConsume, (_pdfio_tpeek_cb_t)_pdfioFilePeek, pdf);
      if (!_pdfioValueRead(pdf, NULL, &tb, &trailer, 0))
@ -2392,10 +2492,12 @@ repair_xref(

      _pdfioTokenFlush(&tb);

-      if (!pdf->trailer_dict)
+      if (_pdfioDictGetValue(trailer.value.dict, "Root"))
      {
 	// Save the trailer dictionary and grab the root (catalog) and info
 	// objects...
+	PDFIO_DEBUG("repair_xref: Using this trailer dictionary.\n");
+
 	pdf->trailer_dict = trailer.value.dict;
 	pdf->encrypt_obj  = pdfioDictGetObj(pdf->trailer_dict, "Encrypt");
 	pdf->id_array     = pdfioDictGetArray(pdf->trailer_dict, "ID");
@ -2406,11 +2508,18 @@ repair_xref(
    line_offset = _pdfioFileTell(pdf);
  }

+  PDFIO_DEBUG("repair_xref: Stopped at line_offset=%lu\n", (unsigned long)line_offset);
+
+  if (!pdf->trailer_dict && backup_trailer)
+    pdf->trailer_dict = backup_trailer;
+
  // If the trailer contains an Encrypt key, try unlocking the file...
  if (pdf->encrypt_obj && !_pdfioCryptoUnlock(pdf, password_cb, password_data))
    return (false);

  // Load any stream objects...
+  PDFIO_DEBUG("repair_xref: Found %lu stream objects.\n", (unsigned long)num_sobjs);
+
  for (i = 0; i < num_sobjs; i ++)
  {
    if (!load_obj_stream(sobjs[i]))
@ -2429,8 +2538,16 @@ repair_xref(

  PDFIO_DEBUG("repair_xref: Root=%p(%lu)\n", pdf->root_obj, (unsigned long)pdf->root_obj->number);

+  if ((pages_obj = pdfioDictGetObj(pdfioObjGetDict(pdf->root_obj), "Pages")) == NULL)
+  {
+    _pdfioFileError(pdf, "Missing Pages object.");
+    return (false);
+  }
+
+  PDFIO_DEBUG("repair_xref: Pages=%p(%lu)\n", pages_obj, (unsigned long)pages_obj->number);
+
  // Load pages...
-  return (load_pages(pdf, pdfioDictGetObj(pdfioObjGetDict(pdf->root_obj), "Pages"), 0));
+  return (load_pages(pdf, pages_obj, 0));
 }


--- a/pdfio-object.c
+++ b/pdfio-object.c
@ -141,6 +141,7 @@ pdfioObjCreateStream(
    pdfio_obj_t    *obj,		// I - Object
    pdfio_filter_t filter)		// I - Type of compression to apply
 {
+  pdfio_stream_t *st;			// Stream
  pdfio_obj_t	*length_obj = NULL;	// Length object, if any


@ -194,11 +195,13 @@ pdfioObjCreateStream(
  if (!_pdfioFilePuts(obj->pdf, "stream\n"))
    return (NULL);

-  obj->stream_offset    = _pdfioFileTell(obj->pdf);
-  obj->pdf->current_obj = obj;
+  obj->stream_offset = _pdfioFileTell(obj->pdf);

  // Return the new stream...
-  return (_pdfioStreamCreate(obj, length_obj, 0, filter));
+  if ((st = _pdfioStreamCreate(obj, length_obj, 0, filter)) != NULL)
+    obj->pdf->current_obj = obj;
+
+  return (st);
 }


@ -534,6 +537,9 @@ pdfio_stream_t *			// O - Stream or `NULL` on error
 pdfioObjOpenStream(pdfio_obj_t *obj,	// I - Object
                   bool        decode)	// I - Decode/decompress data?
 {
+  pdfio_stream_t	*st;		// Stream
+
+
  // Range check input...
  if (!obj)
    return (NULL);
@ -556,9 +562,10 @@ pdfioObjOpenStream(pdfio_obj_t *obj,	// I - Object
    return (NULL);

  // Open the stream...
-  obj->pdf->current_obj = obj;
+  if ((st = _pdfioStreamOpen(obj, decode)) != NULL)
+    obj->pdf->current_obj = obj;

-  return (_pdfioStreamOpen(obj, decode));
+  return (st);
 }


--- a/pdfio-private.h
+++ b/pdfio-private.h
@ -385,7 +385,7 @@ extern bool		_pdfioFileError(pdfio_file_t *pdf, const char *format, ...) _PDFIO_
 extern pdfio_obj_t	*_pdfioFileFindMappedObj(pdfio_file_t *pdf, pdfio_file_t *src_pdf, size_t src_number) _PDFIO_INTERNAL;
 extern bool		_pdfioFileFlush(pdfio_file_t *pdf) _PDFIO_INTERNAL;
 extern int		_pdfioFileGetChar(pdfio_file_t *pdf) _PDFIO_INTERNAL;
-extern bool		_pdfioFileGets(pdfio_file_t *pdf, char *buffer, size_t bufsize) _PDFIO_INTERNAL;
+extern bool		_pdfioFileGets(pdfio_file_t *pdf, char *buffer, size_t bufsize, bool discard) _PDFIO_INTERNAL;
 extern ssize_t		_pdfioFilePeek(pdfio_file_t *pdf, void *buffer, size_t bytes) _PDFIO_INTERNAL;
 extern bool		_pdfioFilePrintf(pdfio_file_t *pdf, const char *format, ...) _PDFIO_INTERNAL;
 extern bool		_pdfioFilePuts(pdfio_file_t *pdf, const char *s) _PDFIO_INTERNAL;
--- a/pdfio-stream.c
+++ b/pdfio-stream.c
@ -259,7 +259,7 @@ _pdfioStreamCreate(
    {
      colors = 1;
    }
-    else if (colors < 0 || colors > 4)
+    else if (colors < 0 || colors > 32)
    {
      _pdfioFileError(st->pdf, "Unsupported Colors value %d.", colors);
      free(st);
@ -532,7 +532,7 @@ _pdfioStreamOpen(pdfio_obj_t *obj,	// I - Object
      {
        colors = 1;
      }
-      else if (colors < 0 || colors > 4)
+      else if (colors < 0 || colors > 32)
      {
        _pdfioFileError(st->pdf, "Unsupported Colors value %d.", colors);
 	goto error;
--- a/test-corpus.sh
+++ b/test-corpus.sh
@ -18,12 +18,22 @@ if test $# = 0; then
 fi

 for file in $(find "$@" -name \*.pdf -print); do
+    # Don't worry about test files containing MIME garbage...
+    (head -4 $file | grep -q Content-Type) && continue;
+
+    # Or test files containing MacBinary garbage...
+    (file $file | grep -q MacBinary) && continue;
+
+    # Don't worry about test files that Xpdf can't handle...
    pdfinfo $file >/dev/null 2>&1 || continue;

+    # Run testpdfio to test loading the file...
    ./testpdfio $file >$file.log 2>&1
    if test $? = 0; then
+    	# Passed
        rm -f $file.log
    else
+    	# Failed, preserve log and write filename to stdout...
        echo $file
    fi
 done
--- a/testpdfio.c
+++ b/testpdfio.c
@ -1333,7 +1333,7 @@ error_cb(pdfio_file_t *pdf,		// I  - PDF file
  testMessage("%s", message);

  // Continue to catch more errors...
-  return (false);
+  return (true);
 }