Favorite Books

Some well-written and self-contained introductory books for applied algorithms and DSP!

  Probability and Statistics

  • Introduction to Probability by by Dimitri P. Bertsekas and John N. Tsitsiklis [Amazon] [DirectTextbook] – A clear and concise introduction to the elements of probability.
  • Information Theory, Inference, and Learning Algorithms –  by David MacKay. [PDF] – Fun exploration of a slew of applied techniques in probability. Strong Bayesian focus.

 Signal Processing

  • Understanding Digital Signal Processing by Richard Lyons [Amazon] [DirectTextbook] [PDF] – Intuitive & visual introduction to digital signal processing by an experienced DSP consultant. Author of Signal Processing Magazine‘s Tips & Tricks column.
  • Discrete-Time Signal Processing by Alan V. Oppenheim [Amazon] [DirectTextbook] [PDF] – Classic textbook written by my Ph.D. advisor. Solid mathematical foundation if combined with supplements below.
  • Numerical Linear Algebra by Lloyd n. Trefethen and David Bau [PDF]  – Introduction to the most useful matrix methods including least-squares, norms, SVD, QR decomposition. Good foundation for vector space signal processing, wavelets, etc…
  • Foundations of Signal Processing by Martin Vetterli, Jelena Kovacevic & Vivek K Goyal [PDF] – Introduction to vector-space / linear algebra interpretation of DSP. (I also recommend Yoram Bresler’s ECE 513 notes).
  • Fourier and Wavelet Signal Processing  by Martin Vetterli, Jelena Kovacevic & Vivek K Goyal [PDF] – The textbook on wavelets. Good for multimedia, compression applications.
  • Fundamentals of Adaptive Filtering by Ali Sayed [Amazon] [DirectTextbook] [PDF]  – Lengthy classic on adaptive filtering, LMS. Useful for noise cancelling or learning quickly-varying channels with minimal computation. Also see modern literature on stochastic-gradient descent in other domains like machine learning.

Brief overview of many DSP topics: http://ccrma.stanford.edu/~jos/sasp/
Useful software tools:  http://dsp.rice.edu/software
Tips and tricks for efficient implementation: http://www.dspguru.com/dsp/tricks

Useful Papers on Special Topics

Introduction to Factor Graphs – Powerful graphical modeling tool unifying algorithms for HMMs, Vitterbi, Kalman Filters, Gaussian Belief Propogation, LDPC, and more.

On Graphical Models for Communications and Machine Learning: Algorithms, Bounds, and Analog Implementation by Justin Dauwels. Covers many topics in graphical models, multivariate-Gaussian belief propagation.

Unscented Kalman Filter – Discussion of Sigma Points — or approximating Gaussian moments through non-linear functions by passing points through the function at a few standard deviations.

Filter Banks

Multirate Digital Filters, Filter Banks, Polyphase Networks, and Applications: A Tutorial by P. P. Vaidyanathan – One of the first–but still a relevant–introduction to filter bank ideas.  (Also see Vetterli’s book above)

Adaptive Filters

Frequency-Domain and Multirate Adaptive Filtering – Classic introduction to performing adaptive filtering in the frequency domain (FDAF). This can greatly improve convergence and reduce the computation (with FFT) of adaptive filtering on channels with long impulse responses.

Lower-Latency Frequency Domain Adaptive Filtering – Adaptation in frequency domain but actual filtering in time-domain to eliminate the latency penalty of a normal FDAF.

Soft-partitioned Frequency-Domain Adaptive Filtering  – To eliminate gradient constraint step to further reduce computation of FDAF.

Dynamic Programming

Seam Carving for Content-Aware Image Resizing (Shai Avidan and Ariel Shamir, MERL)  – Cool use of DP for content-aware image resizing and object removal.


The Structure of Scientific Revolutions by Thomas Khun

Philosophical Investigations by Ludwig Wittgenstein.




Leave a Reply

Your email address will not be published. Required fields are marked *

Prove you are human *