Multivalent Tools: PDF Compress

Compress

PDFs are compressed internally, but they might use old techology or have been generated by inefficient PDF writing software, and as a result be much larger than necessary. This tool recompresses PDFs in order to further reduce their file size. Compression ratios vary widely, from 0% to 99%.

The compression algorithm removes useless objects (duplicated or unused), strips off regenerable objects (ASCII filters, thumbnails), and replaces LZW compression with the superior Flate (even on inline images), among other optimizations. By default it retains all non-regenerable information. Further space savings are possible by setting command line options to remove various non-regenerable objects as well or to apply lossy image compression such as JPEG.

Unless the -compatible option is given, PDFs are updated to the current version of PDF, which at this time is PDF 1.5, which corresponds to Acrobat 6. Additional compression is possible in PDF 1.5 with cross reference streams and object streams. With PDF 1.5, one can usually obtain about the same compression as gzip on the whole PDF file, with the advantage that the PDF is directly readable by Acrobat. Note that if the PDF is already PDF 1.5, this option does not back convert it to and earlier version of PDF.

Compact PDF is a new format that can give an additional compression of 30 to 60% on many classes of PDF beyond what is possible in PDF 1.5. For instance, the PDF Reference 1.5 shrinks from 12.2MB as distributed by Adobe down to 4.4MB in Compact format. No information is lost in obtaining the additional compression, in contrast to some methods that throw away structural or other information or use lossy image compression. This format is not presently part of Adobe's PDF specification and cannot be directly read by Acrobat. However, it is fully supported by Multivalent: the PDF viewer in the browser and all tools including full-text search are "Compact-aware", meaning that they transparently view and manipulate Compact PDFs just as easily as standard ones. You can archive PDFs in this format and if you need to read them in non-Compact-aware PDF viewers, you can always convert them back to standard format by rerunning this tool omitting the -compact option. Technical details for developers can be found in the Compact PDF Specification.

Representative Compression Results

Compression results are highly variable, but certain classes or categories of PDFs seem compress approximately to the same degree.

Category	Representative PDF	Original Size	gzip	-compat	PDF 1.5	Compact	-max
pure text	Aida.pdf	85,437	19,187	33,184	29,661	15,477 (81% savings)	15,461
pure text	Compact format compresses all the pages together, rather than as individual pages.
TeX document	pdftex-s.pdf	329,601	197,483	240,793	188,142	93,529 (71%)	93,530
	gentlesgml.pdf	486,807	207,922	270,227	180,744	75,048 (84%)	75,040
	TeX documents that use Computer Modern fonts typically embed it as an encrypted Type 1 fonts. Compact format decrypts them to make them available to compression for the first time.
FrameMaker	Java Language Specification 2.0	4,419,906	1,622,296	3,938,673	1,534,857	829,671 (81%)	829,667
FrameMaker	FrameMaker generates many named destinations (anchors), which compress well in PDF 1.5 and Compact. FrameMaker will also sometimes write out the page template in each page rather than sharing it, and this compress out in Compact.
reference manual	PDF Reference 1.5 (draft)	12,765,416	7,399,695	10,973,652	7,247,136	4,577,057 (64%)	4,438,420
	PDF Reference 1.5 (final; in PDF 1.4)	14,171,448	8,386,007	13,184,358	8,231,991	5,356,277 (62%)	5,243,832
	PDF Reference 1.5 (final; in PDF 1.5)	9,190,216	8,377,179	8,205,628	8,205,628	5,350,532 (41%)	5,236,604
	PDFlib-manual.pdf	1,298,364	936,166	1,166,849	896,269	619,029 (52%)	588,801
	Reference manuals are typically dominated by text, which compresses better in Compact because it compresses all pages together.
converted from HTML	htmldoc	379,568	245,630	349,335	256,767	169,704 (55%)	169,708
	W3C HTML 4.0 specification	3,006,205	958,150	1,727,038	936,942	468,013 (84%)	467,999
	tcl8.4.2	8,135,892	3,784,950	6,503,767	3,648,370	1,423,353 (82%)	1,423,344
	Hyperlinks compress well with PDF 1.5 object streams and Compact.
book	Manning JDK 1.4	10,168,352	8,871,949	9,777,838	8,633,355	2,999,054 (70%)	2,857,728
	UNIX Haters Handbook	3,639,172	2,803,546	3,125,244	2,516,284	2,068,612 (43%)	2,068,607
	Real World Go Live	18,530,903	15,692,402	18,015,832	17,424,962	12,800,412 (30%)	12,800,405
	pdfxguide.pdf	10,243,070	8,325,458	5,310,326	4,897,694	4,385,093 (57%)	3,371,767
	Large books often compress well in percentage and absolute terms, saving much bandwidth for online distribution.
image dominated	p40-marshall.pdf	1,762,945	1,488,077	1,594,236	1,584,487	1,272,028 (27%)	1,272,028
	UnixTextProcessing.pdf	27,969,206	26,212,375	27,278,926	26,733,530	23,229,702 (16%)	23,229,695
	beos_osx.pdf	1,247,457	1,225,179	1,155,413	1,145,163	1,103,550 (11%)	499,557
	By default images are not recompressed. With the -jpeg option, JPEG compression is applied to raw image samples, which explains the drastic compression for beos_osx in the -max column. If you have lots of scanned images, Adobe Acrobat 6 can compress them considerably with JBIG2 or JPEG2000; for instance, UnixTextProcessing compresses down to about 9MB with JBIG2.
high quality PDF generator	JDJ 1-01.pdf	3,236,507	3,072,982	3,213,335	3,155,262	2,924,454 (9%)	2,729,700
	isaacs_installconfig.pdf	3,518,415	3,306,667	3,169,724	3,100,957	2,445,007 (30%)	2,368,293
	Sometimes only a little compression is available, but isaacs uses the latest libraries (Adobe InDesign 2.0.2, Adobe PDF Library 5.0) and still compresses well.

Options

java tool.pdf.Compress [options] PDF-file(s)

Some large PDFs need more than the 64MB that Java limits itself to by default. Additional memory can also sometimes speed up compression. Most PDFs compress in a few seconds, but a small percentage need up to a few minutes for the most advanced Compact mode. If Compress stops with an OutOfMemoryError or takes more than two minutes on a given PDF (most take less than 10 seconds), try boosting memory, as in:

java -Xmx128m tool.pdf.Compress ...

-old -- By default the PDF is updated to the latest standard, which as of this writing is PDF 1.5, which requires Multivalent or Acrobat 6.0 to view. This switch turns on compatibility mode that tries to maintain the version of PDF used by the document for use with older versions of PDF and Acrobat. Macintosh OS X 10.3 (Panther)'s Preview application cannot display PDF 1.5, and therefore requires this option.
-compact -- write PDF in Compact PDF format. When users of non-Compact-aware PDF viewers view the file, a dummy page will direct them to the site for more information about Compact PDF and to this tool which they can use to rewrite Compact PDFs into standard form.
Fonts
- -nocore14 -- Fonts can be embedded in PDF to guarantee appearance, but 14 core fonts (the families Times, Helvetica, Courier, Symbol, ZapfDingbats) are guaranteed by Acrobat. Nevertheless some PDF generators embed these fonts. This option excises them.
- -noembed -- remove all embedded font data. Viewers will rely on fonts from the operating system and if necessary font substitutions. Also useful to correct PDF generators that embed fonts for scanned paper.
- -subset -- Subset embedded Type 1 fonts to only those characters (glyphs) actually used in the document. EXPERIMENTAL.
Images
- -noalt -- A PDF can contain multiple versions of images, useful to have one smaller lower-quality version for fast on-screen manipulation and another large high-quality version for printing. This option strips away alternates.
- -jpeg -- Converts raw image samples to JPEG, which compresses better. This is useful for Macintosh OS X 10.2 which doesn't use JPEG compression; news is that OS X 10.3 (Panther) does. For technical reasons JPEG images are set to the DeviceRGB or DeviceGray color spaces.
Supplementary Data
- -nostruct -- strip structure tree, if any (not used to display the document)
- -nowebcap -- strip web capture information, if any (not used to display the document)
- -nooutline -- remove document outline aka bookmarks
- -nopagepiece -- strip page piece data, if any. Arbitrary applications can store data in a PDF, which PDF viewers ignore. During editing this information is useful for going back and forth between the application and PDF, but when a PDF document is published, this extra data is not used. Adobe Illustrator is known to store large page piece data in PDFs.
Profiles
- -web -- all the recommended options for posting on the web, that is, externally: -nopagepiece, -nocore14, -subset, -jpeg, -noalt
- -max -- all space-savings options turned on: -compact, -jpeg, -nostruct, and so on. This is useful to determine maximum possible compression, but is not recommended for general use as information is thrown out that may be useful and often not very large.
-force -- force compression of PDF, even if it has already been compressed
-inplace -- replace original PDF with compressed version. This should only be done on backup copies.
-password password -- password if PDF is encrypted
-quiet -- do not print out the changes made in rewriting the PDF

The new PDF file has the same name as the original with the addtion of of -o before the .pdf suffix.

Note: PDFs lose their "linearization" or "Fast Web View" organization. Use another tool to recompute it if desired.