Managing Gigabytes: Compressing and Indexing Documents and Images (Electrical Engineering)

Ian Witten, Alistair Moffat, and Timothy Bell have updated their original work with this even more impressive second edition. This version adds recent techniques such as block-sorting, new indexing techniques, new lossless compression strategies, and many other elements to the mix. In short, this work is a comprehensive summary of text and image compression, indexing, and querying techniques. The history of relevant algorithm development is woven well with a practical discussion of challenges, pitfalls, and specific solutions.

This title is a textbook-style exposition on the topic, with its information organized very clearly into topics such as compression, indexing, and so forth. In addition to diagrams and example text transformations, the authors use "pseudo-code" to present algorithms in a language-independent manner wherever possible. They also supplement the reading with mg--their own implementation of the techniques. The mg C language source code is freely available on the Web.

Alone, this book is an impressive collection of information. Nevertheless, the authors list numerous titles for further reading in selected topics. Whether you're in the midst of application development and need solutions fast or are merely curious about how top-notch information management is done, this hardcover is an excellent investment. --Stephen W. Plain

Topics covered: Text compression models, including Huffman, LZW, and their variants; trends in information management; index creation and compression; image compression; performance issues; and overall system implementation.

Product Description
Offers both students and professionals guidance on large-scale information systems. This resource describes a new generation of techniques for compressing, storing, and retrieving information--both machine readable text and optically scanned documents. Appropriate for information science and information retrieval courses.

All in all, this is the best computer science book I've purchased in years. I wish all CS books were written like this one: it doesn't skimp on the theory or on the implementation details.


The Wonderful Thing Is: It's the Only One   December 20, 2001
Peter Norvig (Palo Alto, CA USA)
53 out of 55 found this review helpful

This is the only book there is that will actually teach you how to build an information retrieval system (aka search engine). It discusses all the algorithms and tradeoffs, and comes with free downloadable source code to experiment with. Some of the material is standard, but covered in more implementation detail here than anywhere else. Some of the material is novel: you won't find better coverage of compression unless you hand-assemble twenty research papers, and reverse-engineer them to figure out how they're implemented. But with "Managing Gigabytes", it's all here. (Although, after a particularly envigorating discussion of how to string together a bunch of techniques to compress their corpus and save a couple 100MB, I did a check and found you could buy 512MB of RAM for less than the cost of the book. Knowledge is Power, but sometimes a little cash is more powerful.) The only negative is that this book is not called "Managing Terabytes", as the first edition promised/threatened it might be. RAM and disk are cheap, but not that cheap, and for now terabytes (and sometimes petabytes) are managed only by NASA, Google, and a few others. I can't wait to see the third edition!


Very clear, but misses some key real-world issues   August 14, 2001
Edwin Young (Seattle, WA United States)
15 out of 16 found this review helpful

As others have said, MG is a good introductory text for Information Retrieval. However I think it spends a little too much time on compression techniques and lacks a good discussion of incremental or on-line indexing. The book tends to assume that the set of texts to be searched is static - if new documents can be added or old ones deleted it makes the whole problem much harder and many of MG's techniques are no longer relevant. That said, I strongly look forward to Managing Terabytes (if it ever appears).