Digital signal processing (DSP) Books

Book SynopsisIn view of the extensive development of CCS 7 and fast-paced growth of ISDN in telecommunication networks throughout the world, this valuable resource serves as a timely reference and guide. Practical and up-to-date, Engineering Networks for Synchronization, CCS 7, and ISDN provides in-depth instruction on three important and closely related elements of the modern digital network: network synchronization, CCITT Common Channel Signaling System No. 7 (CCS 7), and Narrowband ISDN.Table of ContentsSeries Editor's Note. Foreword. Preface. Introduction. Digital Network Synchronization: Basic Concepts. Planning, Testing, and Monitoring Network Synchronization. CCS 7: General Description. Introduction to ISDN. Functions of the CCS 7 Signaling Link Level. Signaling Network Functions in CCS 7. ISDN: Services and Protocols. CCS 7 ISDN User Part. CCS 7 Planning and Implementation. Testing in CCS 7. Packet and Frame Mode Services in the ISDN. Planning and Implementation the ISDN. Testing in the ISDN. Timing in SONET and SDH. Appendix 1: Ordering Information. Appendix 2: List of ISUP Messages. Index. About the Author.

Read more less

Book Synopsis

Read more less

Book SynopsisA realistic and comprehensive review of joint approaches to machine learning and signal processing algorithms, with application to communications, multimedia, and biomedical engineering systems Digital Signal Processing with Kernel Methods reviews the milestones in the mixing of classical digital signal processing models and advanced kernel machines statistical learning tools. It explains the fundamental concepts from both fields of machine learning and signal processing so that readers can quickly get up to speed in order to begin developing the concepts and application software in their own research. Digital Signal Processing with Kernel Methods provides a comprehensive overview of kernel methods in signal processing, without restriction to any application field. It also offers example applications and detailed benchmarking experiments with real and synthetic datasets throughout. Readers can find further worked examples with Matlab source code on a website developed by the authors: hTable of ContentsAbout the Authors xiii Preface xvii Acknowledgements xxi List of Abbreviations xxiii Part I Fundamentals and Basic Elements 1 1 From Signal Processing to Machine Learning 3 1.1 A New Science is Born: Signal Processing 3 1.1.1 Signal Processing Before Being Coined 3 1.1.2 1948: Birth of the Information Age 4 1.1.3 1950s: Audio Engineering Catalyzes Signal Processing 4 1.2 From Analog to Digital Signal Processing 5 1.2.1 1960s: Digital Signal Processing Begins 5 1.2.2 1970s: Digital Signal Processing Becomes Popular 6 1.2.3 1980s: Silicon Meets Digital Signal Processing 6 1.3 Digital Signal Processing Meets Machine Learning 7 1.3.1 1990s: New Application Areas 7 1.3.2 1990s: Neural Networks, Fuzzy Logic, and Genetic Optimization 7 1.4 Recent Machine Learning in Digital Signal Processing 8 1.4.1 Traditional Signal Assumptions Are No Longer Valid 8 1.4.2 Encoding Prior Knowledge 8 1.4.3 Learning and Knowledge from Data 9 1.4.4 From Machine Learning to Digital Signal Processing 9 1.4.5 From Digital Signal Processing to Machine Learning 10 2 Introduction to Digital Signal Processing 13 2.1 Outline of the Signal Processing Field 13 2.1.1 Fundamentals on Signals and Systems 14 2.1.2 Digital Filtering 21 2.1.3 Spectral Analysis 24 2.1.4 Deconvolution 28 2.1.5 Interpolation 30 2.1.6 System Identification 31 2.1.7 Blind Source Separation 36 2.2.3 Sparsity, Compressed Sensing, and Dictionary Learning 44 2.3 Multidimensional Signals and Systems 48 2.3.1 Multidimensional Signals 49 2.3.2 Multidimensional Systems 51 2.4 Spectral Analysis on Manifolds 52 2.4.1 Theoretical Fundamentals 52 2.4.2 Laplacian Matrices 54 2.5 Tutorials and Application Examples 57 2.5.1 Real and Complex Signal Processing and Representations 57 2.5.2 Convolution, Fourier Transform, and Spectrum 63 2.5.3 Continuous-Time Signals and Systems 67 2.5.4 Filtering Cardiac Signals 70 2.5.5 Nonparametric Spectrum Estimation 74 2.5.6 Parametric Spectrum Estimation 77 2.5.7 Source Separation 81 2.5.8 Time–Frequency Representations and Wavelets 84 2.5.9 Examples for Spectral Analysis on Manifolds 87 2.6 Questions and Problems 94 3 Signal Processing Models 97 3.1 Introduction 97 3.2 Vector Spaces, Basis, and Signal Models 98 3.2.1 Basic Operations for Vectors 98 3.2.2 Vector Spaces 100 3.2.3 Hilbert Spaces 101 3.2.4 Signal Models 102 3.2.5 Complex Signal Models 104 3.2.6 Standard Noise Models in Digital Signal Processing 105 3.2.7 The Role of the Cost Function 107 3.2.8 The Role of the Regularizer 109 3.3 Digital Signal Processing Models 111 3.3.1 Sinusoidal Signal Models 112 3.3.2 System Identification Signal Models 113 3.3.3 Sinc Interpolation Models 116 3.3.4 Sparse Deconvolution 120 3.3.5 Array Processing 121 3.4 Tutorials and Application Examples 122 3.4.1 Examples of Noise Models 123 3.4.2 Autoregressive Exogenous System Identification Models 132 3.4.3 Nonlinear System Identification Using Volterra Models 138 3.4.4 Sinusoidal Signal Models 140 3.4.5 Sinc-based Interpolation 144 3.4.6 Sparse Deconvolution 152 3.4.7 Array Processing 157 3.5 Questions and Problems 160 3.A MATLABsimpleInterp Toolbox Structure 161 4 Kernel Functions and Reproducing Kernel Hilbert Spaces 165 4.1 Introduction 165 4.2 Kernel Functions and Mappings 169 4.2.1 Measuring Similarity with Kernels 169 4.2.2 Positive-Definite Kernels 169 4.2.3 Reproducing Kernel in Hilbert Space and Reproducing Property 170 4.2.4 Mercer’s Theorem 173 4.3 Kernel Properties 174 4.3.1 Tikhonov’s Regularization 175 4.3.2 Representer Theorem and Regularization Properties 176 4.3.3 Basic Operations with Kernels 178 4.4 Constructing Kernel Functions 179 4.4.1 Standard Kernels 179 4.4.2 Properties of Kernels 180 4.4.3 Engineering Signal Processing Kernels 181 4.5 Complex Reproducing Kernel in Hilbert Spaces 184 4.6 Support Vector Machine Elements for Regression and Estimation 186 4.6.1 Support Vector Regression Signal Model and Cost Function 186 4.6.2 Minimizing Functional 187 4.7 Tutorials and Application Examples 191 4.7.1 Kernel Calculations and Kernel Matrices 191 4.7.2 Basic Operations with Kernels 194 4.7.3 Constructing Kernels 197 4.7.4 Complex Kernels 199 4.7.5 Application Example for Support Vector Regression Elements 202 4.8 Concluding Remarks 205 4.9 Questions and Problems 205 Part II Function Approximation and Adaptive Filtering 209 5 A Support Vector Machine Signal Estimation Framework 211 5.1 Introduction 211 5.2 A Framework for Support Vector Machine Signal Estimation 213 5.3 Primal Signal Models for Support Vector Machine Signal Processing 216 5.3.1 Nonparametric Spectrum and System Identification 218 5.3.2 Orthogonal Frequency Division Multiplexing Digital Communications 220 5.3.3 Convolutional Signal Models 222 5.3.4 Array Processing 225 5.4 Tutorials and Application Examples 227 5.4.1 Nonparametric Spectral Analysis with Primal Signal Models 227 5.4.2 System Identification with Primal Signal Model ;;-filter 228 5.4.3 Parametric Spectral Density Estimation with Primal Signal Models 230 5.4.4 Temporal Reference Array Processing with Primal Signal Models 231 5.4.5 Sinc Interpolation with Primal Signal Models 233 6 Reproducing Kernel Hilbert Space Models for Signal Processing 241 6.1 Introduction 241 6.2 Reproducing Kernel Hilbert Space Signal Models 242 6.2.1 Kernel Autoregressive Exogenous Identification 244 6.2.2 Kernel Finite Impulse Response and the ;;-Filter 247 6.2.3 Kernel Array Processing with Spatial Reference 248 6.2.4 Kernel Semiparametric Regression 249 6.3 Tutorials and Application Examples 258 6.3.1 Nonlinear System Identification with Support Vector Machine–Autoregressive and Moving Average 258 6.3.2 Nonlinear System Identification with the ;;-filter 260 6.3.3 Electric Network Modeling with Semiparametric Regression 264 6.3.4 Promotional Data 272 6.3.5 Spatial and Temporal Antenna Array Kernel Processing 275 6.4 Questions and Problems 279 7 Dual Signal Models for Signal Processing 281 7.1 Introduction 281 7.2 Dual Signal Model Elements 281 7.3 Dual Signal Model Instantiations 283 7.3.1 Dual Signal Model for Nonuniform Signal Interpolation 283 7.3.2 Dual Signal Model for Sparse Signal Deconvolution 284 7.3.3 Spectrally Adapted Mercer Kernels 285 7.4 Tutorials and Application Examples 289 7.4.1 Nonuniform Interpolation with the Dual Signal Model 290 7.4.2 Sparse Deconvolution with the Dual Signal Model 292 7.4.3 Doppler Ultrasound Processing for Fault Detection 294 7.4.4 Spectrally Adapted Mercer Kernels 296 7.4.5 Interpolation of Heart Rate Variability Signals 304 7.4.6 Denoising in Cardiac Motion-Mode Doppler Ultrasound Images 309?m 7.4.7 Indoor Location from Mobile Devices Measurements 316 7.4.8 Electroanatomical Maps in Cardiac Navigation Systems 322 7.5 Questions and Problems 331 8 Advances in Kernel Regression and Function Approximation 333 8.1 Introduction 333 8.2 Kernel-Based Regression Methods 333 8.2.1 Advances in Support Vector Regression 334 8.2.2 Multi-output Support Vector Regression 338 8.2.3 Kernel Ridge Regression 339 8.2.4 Kernel Signal-To-Noise Regression 341 8.2.5 Semisupervised Support Vector Regression 343 8.2.6 Model Selection in Kernel Regression Methods 345 8.4.1 Comparing Support Vector Regression, Relevance Vector Machines, and Gaussian Process Regression 360 8.4.2 Profile-Dependent Support Vector Regression 362 8.4.3 Multi-output Support Vector Regression 364 8.4.4 Kernel Signal-to-Noise Ratio Regression 366 8.4.5 Semisupervised Support Vector Regression 368 8.4.6 Bayesian Nonparametric Model 369 8.4.7 Gaussian Process Regression 370 8.4.8 Relevance Vector Machines 379 8.5 Concluding Remarks 382 8.6 Questions and Problems 383 9 Adaptive Kernel Learning for Signal Processing 387 9.1 Introduction 387 9.2 Linear Adaptive Filtering 387 9.2.1 Least Mean Squares Algorithm 388 9.2.2 Recursive Least-Squares Algorithm 389 9.3 Kernel Adaptive Filtering 392 9.4 Kernel Least Mean Squares 392 9.4.1 Derivation of Kernel Least Mean Squares 393 9.4.2 Implementation Challenges and Dual Formulation 394 9.5.3 Prediction of the Mackey–Glass Time Series with Kernel Recursive Least Squares 401 9.5.4 Beyond the Stationary Model 402 9.5.5 Example on Nonlinear Channel Identification and Reconvergence 405 9.6 Explicit Recursivity for Adaptive Kernel Models 406 9.6.1 Recursivity in Hilbert Spaces 406 9.6.2 Recursive Filters in Reproducing Kernel Hilbert Spaces 408 9.7 Online Sparsification with Kernels 411 9.7.1 Sparsity by Construction 411 9.7.2 Sparsity by Pruning 413 9.8 Probabilistic Approaches to Kernel Adaptive Filtering 414 9.8.1 Gaussian Processes and Kernel Ridge Regression 415 9.8.2 Online Recursive Solution for Gaussian Processes Regression 416 9.8.3 Kernel Recursive Least Squares Tracker 417 9.8.4 Probabilistic Kernel Least Mean Squares 418 9.9 Further Reading 418 9.9.1 Selection of Kernel Parameters 418 9.9.2 Multi-Kernel Adaptive Filtering 419 9.9.3 Recursive Filtering in Kernel Hilbert Spaces 419 9.10 Tutorials and Application Examples 419 9.10.1 Kernel Adaptive Filtering Toolbox 420 9.10.2 Prediction of a Respiratory Motion Time Series 421 9.10.3 Online Regression on the KIN?h?eK Dataset 423 9.10.4 The Mackey–Glass Time Series 425 9.10.5 Explicit Recursivity on Reproducing Kernel in Hilbert Space and Electroencephalogram Prediction 427 9.10.6 Adaptive Antenna Array Processing 428 9.11 Questions and Problems 430 Part III Classification, Detection, and Feature Extraction 433 10 Support Vector Machine and Kernel Classification Algorithms 435 10.1 Introduction 435 10.2 Support Vector Machine and Kernel Classifiers 435 10.2.1 Support Vector Machines 435 10.2.2 Multiclass and Multilabel Support Vector Machines 441 10.2.3 Least-Squares Support Vector Machine 447 10.2.4 Kernel Fisher’s Discriminant Analysis 448 10.3 Advances in Kernel-Based Classification 452 10.3.1 Large Margin Filtering 452 10.3.2 Semisupervised Learning 454 10.3.3 Multiple Kernel Learning 460 10.3.4 Structured-Output Learning 462 10.3.5 Active Learning 468 10.4 Large-Scale Support Vector Machines 477 10.4.1 Large-Scale Support Vector Machine Implementations 477 10.4.2 Random Fourier Features 478 10.4.3 Parallel Support Vector Machine 480 10.4.4 Outlook 483 10.5 Tutorials and Application Examples 485 10.5.1 Examples of Support Vector Machine Classification 485 10.5.2 Example of Least-Squares Support Vector Machine 492 10.5.3 Kernel-Filtering Support Vector Machine for Brain–Computer Interface Signal Classification 493 10.5.4 Example of Laplacian Support Vector Machine 494 10.5.5 Example of Graph-Based Label Propagation 498 10.5.6 Examples of Multiple Kernel Learning 498 10.6 Concluding Remarks 501 10.7 Questions and Problems 502 11 Clustering and Anomaly Detection with Kernels 503 11.1 Introduction 503 11.2 Kernel Clustering 506 11.2.1 Kernelization of the Metric 506 11.2.2 Clustering in Feature Spaces 508 11.3 Domain Description Via Support Vectors 514 11.3.1 Support Vector Domain Description 514 11.3.2 One-Class Support Vector Machine 515 11.3.3 Relationship Between Support Vector Domain Description and Density Estimation 516 11.3.4 Semisupervised One-Class Classification 517 11.4 Kernel Matched Subspace Detectors 518 11.4.1 Kernel Orthogonal Subspace Projection 518 11.4.2 Kernel Spectral Angle Mapper 520 11.5 Kernel Anomaly Change Detection 522 11.5.1 Linear Anomaly Change Detection Algorithms 522 11.5.2 Kernel Anomaly Change Detection Algorithms 523 11.6 Hypothesis Testing with Kernels 525 11.6.1 Distribution Embeddings 526 11.6.3 Maximum Mean Discrepancy 527 11.6.3 One-Class Support Measure Machine 528 11.7 Tutorials and Application Examples 529 11.7.1 Example on Kernelization of the Metric 529 11.7.2 Example on Kernel k-Means 530 11.7.3 Domain Description Examples 531 11.7.4 Kernel Spectral Angle Mapper and Kernel Orthogonal Subspace Projection Examples 534 11.7.5 Example of Kernel Anomaly Change Detection Algorithms 536 11.7.6 Example on Distribution Embeddings and Maximum Mean Discrepancy 540 11.8 Concluding Remarks 541 11.9 Questions and Problems 542 12 Kernel Feature Extraction in Signal Processing 543 12.1 Introduction 543 12.2 Multivariate Analysis in Reproducing Kernel Hilbert Spaces 545 12.2.1 Problem Statement and Notation 545 12.2.2 Linear Multivariate Analysis 546 12.2.3 Kernel Multivariate Analysis 549 12.2.4 Multivariate Analysis Experiments 551 12.3 Feature Extraction with Kernel Dependence Estimates 555 12.3.1 Feature Extraction Using Hilbert–Schmidt Independence Criterion 556 12.3.2 Blind Source Separation Using Kernels 563 12.4 Extensions for Large-Scale and Semisupervised Problems 570 12.4.2 Efficiency with the Incomplete Cholesky Decomposition 570 12.4.3 Efficiency with Random Fourier Features 570 12.4.3 Sparse Kernel Feature Extraction 571 12.4.4 Semisupervised Kernel Feature Extraction 573 12.5 Domain Adaptation with Kernels 575 12.5.1 Kernel Mean Matching 578 12.5.2 Transfer Component Analysis 579 12.5.3 Kernel Manifold Alignment 581 12.5.4 Relations between Domain Adaptation Methods 585 12.5.5 Experimental Comparison between Domain Adaptation Methods 12.6 Concluding Remarks 587 12.7 Questions and Problems 588 References 589Index 631

Read more less

Book SynopsisThe modern financial industry has been required to deal with large and diverse portfolios in a variety of asset classes often with limited market data available.Table of ContentsList of Contributors xiii Preface xv 1 Overview 1 Ali N. Akansu, Sanjeev R. Kulkarni, and Dmitry Malioutov 1.1 Introduction 1 1.2 A Bird’s-Eye View of Finance 2 1.2.1 Trading and Exchanges 4 1.2.2 Technical Themes in the Book 5 1.3 Overview of the Chapters 6 1.3.1 Chapter 2: “Sparse Markowitz Portfolios” by Christine De Mol 6 1.3.2 Chapter 3: “Mean-Reverting Portfolios: Tradeoffs between Sparsity and Volatility” by Marco Cuturi and Alexandre d’Aspremont 7 1.3.3 Chapter 4: “Temporal Causal Modeling” by Prabhanjan Kambadur, Aurélie C. Lozano, and Ronny Luss 7 1.3.4 Chapter 5: “Explicit Kernel and Sparsity of Eigen Subspace for the AR(1) Process” by Mustafa U. Torun, Onur Yilmaz and Ali N. Akansu 7 1.3.5 Chapter 6: “Approaches to High-Dimensional Covariance and Precision Matrix Estimation” by Jianqing Fan, Yuan Liao, and Han Liu 7 1.3.6 Chapter 7: “Stochastic Volatility: Modeling and Asymptotic Approaches to Option Pricing and Portfolio Selection” by Matthew Lorig and Ronnie Sircar 7 1.3.7 Chapter 8: “Statistical Measures of Dependence for Financial Data” by David S. Matteson, Nicholas A. James, and William B. Nicholson 8 1.3.8 Chapter 9: “Correlated Poisson Processes and Their Applications in Financial Modeling” by Alexander Kreinin 8 1.3.9 Chapter 10: “CVaR Minimizations in Support Vector Machines” by Junya Gotoh and Akiko Takeda 8 1.3.10 Chapter 11: “Regression Models in Risk Management” by Stan Uryasev 8 1.4 Other Topics in Financial Signal Processing and Machine Learning 9 References 9 2 Sparse Markowitz Portfolios 11 ChristineDeMol 2.1 Markowitz Portfolios 11 2.2 Portfolio Optimization as an Inverse Problem: The Need for Regularization 13 2.3 Sparse Portfolios 15 2.4 Empirical Validation 17 2.5 Variations on the Theme 18 2.5.1 Portfolio Rebalancing 18 2.5.2 Portfolio Replication or Index Tracking 19 2.5.3 Other Penalties and Portfolio Norms 19 2.6 Optimal Forecast Combination 20 Acknowlegments 21 References 21 3 Mean-Reverting Portfolios 23 Marco Cuturi and Alexandre d’Aspremont 3.1 Introduction 23 3.1.1 Synthetic Mean-Reverting Baskets 24 3.1.2 Mean-Reverting Baskets with Sufficient Volatility and Sparsity 24 3.2 Proxies for Mean Reversion 25 3.2.1 Related Work and Problem Setting 25 3.2.2 Predictability 26 3.2.3 Portmanteau Criterion 27 3.2.4 Crossing Statistics 28 3.3 Optimal Baskets 28 3.3.1 Minimizing Predictability 29 3.3.2 Minimizing the Portmanteau Statistic 29 3.3.3 Minimizing the Crossing Statistic 29 3.4 Semidefinite Relaxations and Sparse Components 30 3.4.1 A Semidefinite Programming Approach to Basket Estimation 30 3.4.2 Predictability 30 3.4.3 Portmanteau 31 3.4.4 Crossing Stats 31 3.5 Numerical Experiments 32 3.5.1 Historical Data 32 3.5.2 Mean-reverting Basket Estimators 33 3.5.3 Jurek and Yang (2007) Trading Strategy 33 3.5.4 Transaction Costs 33 3.5.5 Experimental Setup 36 3.5.6 Results 36 3.6 Conclusion 39 References 39 4 Temporal Causal Modeling 41 Prabhanjan Kambadur, Aurélie C. Lozano, and Ronny Luss 4.1 Introduction 41 4.2 TCM 46 4.2.1 Granger Causality and Temporal Causal Modeling 46 4.2.2 Grouped Temporal Causal Modeling Method 47 4.2.3 Synthetic Experiments 49 4.3 Causal Strength Modeling 51 4.4 Quantile TCM (Q-TCM) 52 4.4.1 Modifying Group OMP for Quantile Loss 52 4.4.2 Experiments 53 4.5 TCM with Regime Change Identification 55 4.5.1 Model 56 4.5.2 Algorithm 58 4.5.3 Synthetic Experiments 60 4.5.4 Application: Analyzing Stock Returns 62 4.6 Conclusions 63 References 64 5 Explicit Kernel and Sparsity of Eigen Subspace for the AR(1) Process 67 Mustafa U. Torun, Onur Yilmaz, and Ali N. Akansu 5.1 Introduction 67 5.2 Mathematical Definitions 68 5.2.1 Discrete AR(1) Stochastic Signal Model 68 5.2.2 Orthogonal Subspace 69 5.3 Derivation of Explicit KLT Kernel for a Discrete AR(1) Process 72 5.3.1 A Simple Method for Explicit Solution of a Transcendental Equation 73 5.3.2 Continuous Process with Exponential Autocorrelation 74 5.3.3 Eigenanalysis of a Discrete AR(1) Process 76 5.3.4 Fast Derivation of KLT Kernel for an AR(1) Process 79 5.4 Sparsity of Eigen Subspace 82 5.4.1 Overview of Sparsity Methods 83 5.4.2 pdf-Optimized Midtread Quantizer 84 5.4.3 Quantization of Eigen Subspace 86 5.4.4 pdf of Eigenvector 87 5.4.5 Sparse KLT Method 89 5.4.6 Sparsity Performance 91 5.5 Conclusions 97 References 97 6 Approaches to High-Dimensional Covariance and Precision Matrix Estimations 100 Jianqing Fan, Yuan Liao, and Han Liu 6.1 Introduction 100 6.2 Covariance Estimation via Factor Analysis 101 6.2.1 Known Factors 103 6.2.2 Unknown Factors 104 6.2.3 Choosing the Threshold 105 6.2.4 Asymptotic Results 105 6.2.5 A Numerical Illustration 107 6.3 Precision Matrix Estimation and Graphical Models 109 6.3.1 Column-wise Precision Matrix Estimation 110 6.3.2 The Need for Tuning-insensitive Procedures 111 6.3.3 TIGER: A Tuning-insensitive Approach for Optimal Precision Matrix Estimation 112 6.3.4 Computation 114 6.3.5 Theoretical Properties of TIGER 114 6.3.6 Applications to Modeling Stock Returns 115 6.3.7 Applications to Genomic Network 118 6.4 Financial Applications 119 6.4.1 Estimating Risks of Large Portfolios 119 6.4.2 Large Panel Test of Factor Pricing Models 121 6.5 Statistical Inference in Panel Data Models 126 6.5.1 Efficient Estimation in Pure Factor Models 126 6.5.2 Panel Data Model with Interactive Effects 127 6.5.3 Numerical Illustrations 130 6.6 Conclusions 131 References 131 7 Stochastic Volatility 135 Matthew Lorig and Ronnie Sircar 7.1 Introduction 135 7.1.1 Options and Implied Volatility 136 7.1.2 Volatility Modeling 137 7.2 Asymptotic Regimes and Approximations 141 7.2.1 Contract Asymptotics 142 7.2.2 Model Asymptotics 142 7.2.3 Implied Volatility Asymptotics 143 7.2.4 Tractable Models 145 7.2.5 Model Coefficient Polynomial Expansions 146 7.2.6 Small “Vol of Vol” Expansion 152 7.2.7 Separation of Timescales Approach 152 7.2.8 Comparison of the Expansion Schemes 154 7.3 Merton Problem with Stochastic Volatility: Model Coefficient Polynomial Expansions 155 7.3.1 Models and Dynamic Programming Equation 155 7.3.2 Asymptotic Approximation 157 7.3.3 Power Utility 159 7.4 Conclusions 160 Acknowledgements 160 References 160 8 Statistical Measures of Dependence for Financial Data 162 David S. Matteson, Nicholas A. James, and William B. Nicholson 8.1 Introduction 162 8.2 Robust Measures of Correlation and Autocorrelation 164 8.2.1 Transformations and Rank-Based Methods 166 8.2.2 Inference 169 8.2.3 Misspecification Testing 171 8.3 Multivariate Extensions 174 8.3.1 Multivariate Volatility 175 8.3.2 Multivariate Misspecification Testing 176 8.3.3 Granger Causality 176 8.3.4 Nonlinear Granger Causality 177 8.4 Copulas 179 8.4.1 Fitting Copula Models 180 8.4.2 Parametric Copulas 181 8.4.3 Extending beyond Two Random Variables 183 8.4.4 Software 185 8.5 Types of Dependence 185 8.5.1 Positive and Negative Dependence 185 8.5.2 Tail Dependence 187 References 188 9 Correlated Poisson Processes and Their Applications in Financial Modeling 191 Alexander Kreinin 9.1 Introduction 191 9.2 Poisson Processes and Financial Scenarios 193 9.2.1 Integrated Market–Credit Risk Modeling 193 9.2.2 Market Risk and Derivatives Pricing 194 9.2.3 Operational Risk Modeling 194 9.2.4 Correlation of Operational Events 195 9.3 Common Shock Model and Randomization of Intensities 196 9.3.1 Common Shock Model 196 9.3.2 Randomization of Intensities 196 9.4 Simulation of Poisson Processes 197 9.4.1 Forward Simulation 197 9.4.2 Backward Simulation 200 9.5 Extreme Joint Distribution 207 9.5.1 Reduction to Optimization Problem 207 9.5.2 Monotone Distributions 208 9.5.3 Computation of the Joint Distribution 214 9.5.4 On the Frechet–Hoeffding Theorem 215 9.5.5 Approximation of the Extreme Distributions 217 9.6 Numerical Results 219 9.6.1 Examples of the Support 219 9.6.2 Correlation Boundaries 221 9.7 Backward Simulation of the Poisson-Wiener Process 222 9.8 Concluding Remarks 227 Acknowledgments 228 Appendix A 229 A. 1 Proof of Lemmas 9.2 and 9.3 229 A.1.1 Proof of Lemma 9.2 229 A.1.2 Proof of Lemma 9.3 230 References 231 10 CVaR Minimizations in Support Vector Machines 233 Jun-ya Gotoh and Akiko Takeda 10.1 What Is CVaR? 234 10.1.1 Definition and Interpretations 234 10.1.2 Basic Properties of CVaR 238 10.1.3 Minimization of CVaR 240 10.2 Support Vector Machines 242 10.2.1 Classification 242 10.2.2 Regression 246 10.3 ν-SVMs as CVaR Minimizations 247 10.3.1 ν-SVMs as CVaR Minimizations with Homogeneous Loss 247 10.3.2 ν-SVMs as CVaR Minimizations with Nonhomogeneous Loss 251 10.3.3 Refining the ν-Property 253 10.4 Duality 256 10.4.1 Binary Classification 256 10.4.2 Geometric Interpretation of ν-SVM 257 10.4.3 Geometric Interpretation of the Range of ν for ν-SVC 258 10.4.4 Regression 259 10.4.5 One-class Classification and SVDD 259 10.5 Extensions to Robust Optimization Modelings 259 10.5.1 Distributionally Robust Formulation 259 10.5.2 Measurement-wise Robust Formulation 261 10.6 Literature Review 262 10.6.1 CVaR as a Risk Measure 263 10.6.2 From CVaR Minimization to SVM 263 10.6.3 From SVM to CVaR Minimization 263 10.6.4 Beyond CVaR 263 References 264 11 Regression Models in Risk Management 266 Stan Uryasev 11.1 Introduction 267 11.2 Error and Deviation Measures 268 11.3 Risk Envelopes and Risk Identifiers 271 11.3.1 Examples of Deviation Measures D, Corresponding Risk Envelopes Q, and Sets of Risk Identifiers QD(X) 272 11.4 Error Decomposition in Regression 273 11.5 Least-Squares Linear Regression 275 11.6 Median Regression 277 11.7 Quantile Regression and Mixed Quantile Regression 281 11.8 Special Types of Linear Regression 283 11.9 Robust Regression 284 References, Further Reading, and Bibliography 287 Index 289

Read more less

Book SynopsisFeatures inexpensive ARM Cortex-M4 microcontroller development systems available from Texas Instruments and STMicroelectronics. This book presents a hands-on approach to teaching Digital Signal Processing (DSP) with real-time examples using the ARM Cortex-M4 32-bit microprocessor. Real-time examples using analog input and output signals are provided, giving visible (using an oscilloscope) and audible (using a speaker or headphones) results. Signal generators and/or audio sources, e.g. iPods, can be used to provide experimental input signals. The text also covers the fundamental concepts of digital signal processing such as analog-to-digital and digital-to-analog conversion, FIR and IIR filtering, Fourier transforms, and adaptive filtering. Digital Signal Processing Using the ARM Cortex-M4: Uses a large number of simple example programs illustrating DSP concepts in real-time, in an electrical engineering laboratory setting Includes exTable of ContentsPreface xi 1 ARM® CORTEX® - M4 Development Systems 1 1.1 Introduction 1 1.1.1 Audio Interfaces 2 1.1.2 Texas Instruments TM4C123 LaunchPad and STM32F407 Discovery Development Kits 2 1.1.3 Hardware and Software Tools 6 Reference 7 2 Analog Input and Output 9 2.1 Introduction 9 2.1.1 Sampling, Reconstruction, and Aliasing 9 2.2 TLV320AIC3104 (AIC3104) Stereo Codec for Audio Input and Output 10 2.3 WM5102 Audio Hub Codec for Audio Input and Output 12 2.4 Programming Examples 12 2.5 Real-Time Input and Output Using Polling, Interrupts, and Direct Memory Access (DMA) 12 2.5.1 I2S Emulation on the TM4C123 15 2.5.2 Program Operation 15 2.5.3 Running the Program 16 2.5.4 Changing the Input Connection to LINE IN 16 2.5.5 Changing the Sampling Frequency 16 2.5.6 Using the Digital MEMS Microphone on the Wolfson Audio Card 20 2.5.7 Running the Program 21 2.5.8 Running the Program 23 2.5.9 DMA in the TM4C123 Processor 26 2.5.10 Running the Program 30 2.5.11 Monitoring Program Execution 30 2.5.12 Measuring the Delay Introduced by DMA-Based I/O 30 2.5.13 DMA in the STM32F407 Processor 34 2.5.14 Running the Program 35 2.5.15 Measuring the Delay Introduced by DMA-Based I/O 35 2.5.16 Running the Program 46 2.6 Real-Time Waveform Generation 46 2.6.1 Running the Program 49 2.6.2 Out-of-Band Noise in the Output of the AIC3104 Codec (tm4c123_sine48_intr.c). 49 2.6.3 Running the Program 53 2.6.4 Running the Program 62 2.6.5 Running the Program 69 2.7 Identifying the Frequency Response of the DAC Using Pseudorandom Noise 70 2.7.1 Programmable De-Emphasis in the AIC3104 Codec 72 2.7.2 Programmable Digital Effects Filters in the AIC3104 Codec 72 2.8 Aliasing 78 2.8.1 Running the Program 83 2.9 Identifying the Frequency Response of the DAC Using An Adaptive Filter 83 2.9.1 Running the Program 84 2.10 Analog Output Using the STM32F407’S 12-BIT DAC 91 References 96 3 Finite Impulse Response Filters 97 3.1 Introduction to Digital Filters 97 3.1.1 The FIR Filter 97 3.1.2 Introduction to the z-Transform 99 3.1.3 Definition of the z-Transform 100 3.1.4 Properties of the z-Transform 108 3.1.5 z-Transfer Functions 111 3.1.6 Mapping from the s-Plane to the z-Plane 111 3.1.7 Difference Equations 112 3.1.8 Frequency Response and the z-Transform 113 3.1.9 The Inverse z-Transform 114 3.2 Ideal Filter Response Classifications: LP, HP, BP, BS 114 3.2.1 Window Method of FIR Filter Design 114 3.2.2 Window Functions 116 3.2.3 Design of Ideal High-Pass Band-Pass and Band-Stop FIR Filters Using the Window Method 120 3.3 Programming Examples 123 3.3.1 Altering the Coefficients of the Moving Average Filter 132 3.3.2 Generating FIR Filter Coefficient Header Files Using MATLAB 137 4 Infinite Impulse Response Filters 163 4.1 Introduction 163 4.2 IIR Filter Structures 164 4.2.1 Direct Form I Structure 164 4.2.2 Direct Form II Structure 165 4.2.3 Direct Form II Transpose 166 4.2.4 Cascade Structure 168 4.2.5 Parallel Form Structure 169 4.3 Impulse Invariance 171 4.4 Bilinear Transformation 171 4.4.1 Bilinear Transform Design Procedure 172 4.5 Programming Examples 173 4.5.1 Design of a Simple IIR Low-Pass Filter 173 Reference 216 5 Fast Fourier Transform 217 5.1 Introduction 217 5.2 Development of the FFT Algorithm with RADIX-2 218 5.3 Decimation-in-Frequency FFT Algorithm with RADIX-2 219 5.4 Decimation-in-Time FFT Algorithm with RADIX-2 222 5.4.1 Reordered Sequences in the Radix-2 FFT and Bit-Reversed Addressing 224 5.5 Decimation-in-Frequency FFT Algorithm with RADIX-4 226 5.6 Inverse Fast Fourier Transform 227 5.7 Programming Examples 228 5.7.1 Twiddle Factors 233 5.8 Frame- or Block-Based Programming 239 5.8.1 Running the Program 242 5.8.2 Spectral Leakage 244 5.9 Fast Convolution 252 5.9.1 Running the Program 256 5.9.2 Execution Time of Fast Convolution Method of FIR Filter Implementation 256 Reference 261 6 Adaptive Filters 263 6.1 Introduction 263 6.2 Adaptive Filter Configurations 264 6.2.1 Adaptive Prediction 264 6.2.2 System Identification or Direct Modeling 265 6.2.3 Noise Cancellation 265 6.2.4 Equalization 266 6.3 Performance Function 267 6.3.1 Visualizing the Performance Function 269 6.4 Searching for the Minimum 270 6.5 Least Mean Squares Algorithm 270 6.5.1 LMS Variants 272 6.5.2 Normalized LMS Algorithm 272 6.6 Programming Examples 273 6.6.1 Using CMSIS DSP Function arm_lms_f32() 280 Index 299

Read more less

Book SynopsisAn important working resource for engineers and researchers involved in the design, development, and implementation of signal processing systems The last decade has seen a rapid expansion of the use of field programmable gate arrays (FPGAs) for a wide range of applications beyond traditional digital signal processing (DSP) systems. Written by a team of experts working at the leading edge of FPGA research and development, this second edition of FPGA-based Implementation of Signal Processing Systems has been extensively updated and revised to reflect the latest iterations of FPGA theory, applications, and technology. Written from a system-level perspective, it features expert discussions of contemporary methods and tools used in the design, optimization and implementation of DSP systems using programmable FPGA hardware. And it provides a wealth of practical insightsalong with illustrative case studies and timely real-world examplesof critical concern to engineers Table of ContentsPreface xv List of Abbreviations xxi 1 Introduction to Field Programmable Gate Arrays 1 1.1 Introduction 1 1.2 Field Programmable Gate Arrays 2 1.3 Influence of Programmability 6 1.4 Challenges of FPGAs 8 Bibliography 9 2 DSP Basics 11 2.1 Introduction 11 2.2 Definition of DSP Systems 12 2.3 DSP Transformations 16 2.4 Filters 20 2.5 Adaptive Filtering 29 2.6 Final Comments 38 Bibliography 38 3 Arithmetic Basics 41 3.1 Introduction 41 3.2 Number Representations 42 3.3 Arithmetic Operations 47 3.4 Alternative Number Representations 55 3.5 Division 59 3.6 Square Root 60 3.7 Fixed-Point versus Floating-Point 64 3.8 Conclusions 66 Bibliography 67 4 Technology Review 70 4.1 Introduction 70 4.2 Implications of Technology Scaling 71 4.3 Architecture and Programmability 72 4.4 DSP Functionality Characteristics 74 4.5 Microprocessors 76 4.6 DSP Processors 82 4.7 Graphical Processing Units 86 4.8 System-on-Chip Solutions 88 4.9 Heterogeneous Computing Platforms 91 4.10 Conclusions 92 Bibliography 92 5 Current FPGA Technologies 94 5.1 Introduction 94 5.2 Toward FPGAs 95 5.3 Altera Stratix® V and 10 FPGA Family 98 5.4 Xilinx UltrascaleTM/Virtex-7 FPGA Families 103 5.5 Xilinx Zynq FPGA Family 107 5.6 Lattice iCE40isp FPGA Family 108 5.7 MicroSemi RTG4 FPGA Family 111 5.8 Design Stratregies for FPGA-based DSP Systems 112 5.9 Conclusions 114 Bibliography 114 6 Detailed FPGA Implementation Techniques 116 6.1 Introduction 116 6.2 FPGA Functionality 117 6.3 Mapping to LUT-Based FPGA Technology 123 6.4 Fixed-Coefficient DSP 125 6.5 Distributed Arithmetic 130 6.6 Reduced-Coefficient Multiplier 133 6.7 Conclusions 137 Bibliography 138 7 Synthesis Tools for FPGAs 140 7.1 Introduction 140 7.2 High-Level Synthesis 141 7.3 Xilinx Vivado 143 7.4 Control Logic Extraction Phase Example 144 7.5 Altera SDK for OpenCL 145 7.6 Other HLS Tools 147 7.7 Conclusions 150 Bibliography 150 8 Architecture Derivation for FPGA-based DSP Systems 152 8.1 Introduction 152 8.2 DSP Algorithm Characteristics 153 8.3 DSP Algorithm Representations 157 8.4 Pipelining DSP Systems 160 8.5 Parallel Operation 170 8.6 Conclusions 178 Bibliography 179 9 Complex DSP Core Design for FPGA 180 9.1 Introduction 180 9.2 Motivation for Design for Reuse 181 9.3 Intellectual Property Cores 182 9.4 Evolution of IP Cores 184 9.5 Parameterizable (Soft) IP Cores 187 9.6 IP Core Integration 195 9.7 Current FPGA-based IP Cores 197 9.8 Watermarking IP 198 9.9 Summary 198 Bibliography 199 10 AdvancedModel-Based FPGA Accelerator Design 200 10.1 Introduction 200 10.2 Dataflow Modeling of DSP Systems 201 10.3 Architectural Synthesis of Custom Circuit Accelerators from DFGs 204 10.4 Model-Based Development of Multi-Channel Dataflow Accelerators 205 10.5 Model-Based Development for Memory-Intensive Accelerators 219 10.6 Summary 223 References 223 11 Adaptive Beamformer Example 225 11.1 Introduction to Adaptive Beamforming 226 11.2 Generic Design Process 226 11.3 Algorithm to Architecture 231 11.4 Efficient Architecture Design 235 11.5 Generic QR Architecture 240 11.6 Retiming the Generic Architecture 246 11.7 Parameterizable QR Architecture 253 11.8 Generic Control 266 11.9 Beamformer Design Example 269 11.10 Summary 271 References 271 12 FPGA Solutions for Big Data Applications 273 12.1 Introduction 273 12.2 Big Data 274 12.3 Big Data Analytics 275 12.4 Acceleration 280 12.5 k-Means Clustering FPGA Implementation 283 12.6 FPGA-Based Soft Processors 286 12.7 System Hardware 290 12.8 Conclusions 293 Bibliography 293 13 Low-Power FPGA Implementation 296 13.1 Introduction 296 13.2 Sources of Power Consumption 297 13.3 FPGA Power Consumption 300 13.4 Power Consumption Reduction Techniques 302 13.5 Dynamic Voltage Scaling in FPGAs 303 13.6 Reduction in Switched Capacitance 305 13.7 Final Comments 316 Bibliography 317 14 Conclusions 319 14.1 Introduction 319 14.2 Evolution in FPGA Design Approaches 320 14.3 Big Data and the Shift toward Computing 320 14.4 Programming Flow for FPGAs 321 14.5 Support for Floating-Point Arithmetic 322 14.6 Memory Architectures 322 Bibliography 323 Index 325

Read more less

Book SynopsisAn Innovative Approach to Multidimensional Signals and Systems Theory for Image and Video Processing In this volume, Eric Dubois further develops the theory of multi-D signal processing wherein input and output are vector-value signals. With this framework, he introduces the reader to crucial concepts in signal processing such as continuous- and discrete-domain signals and systems, discrete-domain periodic signals, sampling and reconstruction, light and color, random field models, image representation and more. While most treatments use normalized representations for non-rectangular sampling, this approach obscures much of the geometrical and scale information of the signal. In contrast, Dr. Dubois uses actual units of space-time and frequency. Basis-independent representations appear as much as possible, and the basis is introduced where needed to perform calculations or implementations. Thus, lattice theory is developed from the beginning and rectangular samplTable of ContentsAbout the Companion Website xiii 1 Introduction 1 2 Continuous-Domain Signals and Systems 5 2.1 Introduction 5 2.2 Multidimensional Signals 7 2.2.1 Zero–One Functions 7 2.2.2 Sinusoidal Signals 7 2.2.3 Real Exponential Functions 10 2.2.4 Zone Plate 10 2.2.5 Singularities 12 2.2.6 Separable and Isotropic Functions 13 2.3 Visualization of Two-Dimensional Signals 13 2.4 Signal Spaces and Systems 14 2.5 Continuous-Domain Linear Systems 15 2.5.1 Linear Systems 15 2.5.2 Linear Shift-Invariant Systems 19 2.5.3 Response of a Linear System 20 2.5.4 Response of a Linear Shift-Invariant System 20 2.5.5 Frequency Response of an LSI System 22 2.6 The Multidimensional Fourier Transform 22 2.6.1 Fourier Transform Properties 23 2.6.2 Evaluation of Multidimensional Fourier Transforms 27 2.6.3 Two-Dimensional Fourier Transform of Polygonal Zero–One Functions 30 2.6.4 Fourier Transform of a Translating Still Image 33 2.7 Further Properties of Differentiation and Related Systems 33 2.7.1 Directional Derivative 34 2.7.2 Laplacian 34 2.7.3 Filtered Derivative Systems 35 Problems 37 3 Discrete-Domain Signals and Systems 41 3.1 Introduction 41 3.2 Lattices 42 3.2.1 Basic Definitions 42 3.2.2 Properties of Lattices 44 3.2.3 Examples of 2D and 3D Lattices 44 3.3 Sampling Structures 46 3.4 Signals Defined on Lattices 47 3.5 Special Multidimensional Signals on a Lattice 48 3.5.1 Unit Sample 48 3.5.2 Sinusoidal Signals 49 3.6 Linear Systems Over Lattices 51 3.6.1 Response of a Linear System 51 3.6.2 Frequency Response 52 3.7 Discrete-Domain Fourier Transforms Over a Lattice 52 3.7.1 Definition of the Discrete-Domain Fourier Transform 52 3.7.2 Properties of the Multidimensional Fourier Transform Over a Lattice Λ 53 3.7.3 Evaluation of Forward and Inverse Discrete-Domain Fourier Transforms 57 3.8 Finite Impulse Response (FIR) Filters 59 3.8.1 Separable Filters 66 Problems 67 4 Discrete-Domain Periodic Signals 69 4.1 Introduction 69 4.2 Periodic Signals 69 4.3 Linear Shift-Invariant Systems 72 4.4 Discrete-Domain Periodic Fourier Transform 73 4.5 Properties of the Discrete-Domain Periodic Fourier Transform 77 4.6 Computation of the Discrete-Domain Periodic Fourier Transform 81 4.6.1 Direct Computation 81 4.6.2 Selection of Coset Representatives 82 4.7 Vector Space Representation of Images Based on the Discrete-Domain Periodic Fourier Transform 87 4.7.1 Vector Space Representation of Signals with Finite Extent 87 4.7.2 Block-Based Vector-Space Representation 88 Problems 90 5 Continuous-Domain Periodic Signals 93 5.1 Introduction 93 5.2 Continuous-Domain Periodic Signals 93 5.3 Linear Shift-Invariant Systems 94 5.4 Continuous-Domain Periodic Fourier Transform 96 5.5 Properties of the Continuous-Domain Periodic Fourier Transform 96 5.6 Evaluation of the Continuous-Domain Periodic Fourier Transform 100 Problems 105 6 Sampling, Reconstruction and Sampling Theorems for Multidimensional Signals 107 6.1 Introduction 107 6.2 Ideal Sampling and Reconstruction of Continuous-Domain Signals 107 6.3 Practical Sampling 110 6.4 Practical Reconstruction 112 6.5 Sampling and Periodization of Multidimensional Signals and Transforms 113 6.6 Inverse Fourier Transforms 116 6.6.1 Inverse Discrete-Domain Aperiodic Fourier Transform 117 6.6.2 Inverse Continuous-Domain Periodic Fourier Transform 118 6.6.3 Inverse Continuous-Domain Fourier Transform 119 6.7 Signals and Transforms with Finite Support 119 6.7.1 Continuous-Domain Signals with Finite Support 119 6.7.2 Discrete-Domain Aperiodic Signals with Finite Support 120 6.7.3 Band-Limited Continuous-Domain Γ-Periodic Signals 121 Problems 121 7 Light and Color Representation in Imaging Systems 125 7.1 Introduction 125 7.2 Light 125 7.3 The Space of Light Stimuli 128 7.4 The Color Vector Space 129 7.4.1 Properties of Metamerism 130 7.4.2 Algebraic Condition for Metameric Equivalence 132 7.4.3 Extension of Metameric Equivalence to A 135 7.4.4 Definition of the Color Vector Space 135 7.4.5 Bases for the Vector Space C 137 7.4.6 Transformation of Primaries 138 7.4.7 The CIE Standard Observer 140 7.4.8 Specification of Primaries 142 7.4.9 Physically Realizable Colors 144 7.5 Color Coordinate Systems 147 7.5.1 Introduction 147 7.5.2 Luminance and Chromaticity 147 7.5.3 Linear Color Representations 153 7.5.4 Perceptually Uniform Color Coordinates 155 7.5.5 Display Referred Coordinates 157 7.5.6 Luma-Color-Difference Representation 158 Problems 158 8 Processing of Color Signals 163 8.1 Introduction 163 8.2 Continuous-Domain Systems for Color Images 163 8.2.1 Continuous-Domain Color Signals 163 8.2.2 Continuous-Domain Systems for Color Signals 166 8.2.3 Frequency Response and Fourier Transform 168 8.3 Discrete-Domain Color Images 173 8.3.1 Color Signals With All Components on a Single Lattice 173 8.3.1.1 Sampling a Continuous-Domain Color Signal Using a Single Lattice 175 8.3.1.2 S-CIELAB Error Criterion 175 8.3.2 Color Signals With Different Components on Different Sampling Structures 180 8.4 Color Mosaic Displays 188 9 Random Field Models 193 9.1 Introduction 193 9.2 What is a Random Field? 194 9.3 Image Moments 195 9.3.1 Mean, Autocorrelation, Autocovariance 195 9.3.2 Properties of the Autocorrelation Function 198 9.3.3 Cross-Correlation 199 9.4 Power Density Spectrum 199 9.4.1 Properties of the Power Density Spectrum 200 9.4.2 Cross Spectrum 201 9.4.3 Spectral Density Matrix 201 9.5 Filtering and Sampling of WSS Random Fields 202 9.5.1 LSI Filtering of a Scalar WSS Random Field 202 9.5.2 Why is Sf(u) Called a Power Density Spectrum? 204 9.5.3 LSI Filtering of a WSS Color Random Field 205 9.5.4 Sampling of a WSS Continuous-Domain Random Field 206 9.6 Estimation of the Spectral Density Matrix 207 Problems 214 10 Analysis and Design of Multidimensional FIR Filters 215 10.1 Introduction 215 10.2 Moving Average Filters 215 10.3 Gaussian Filters 217 10.4 Band-pass and Band-stop Filters 220 10.5 Frequency-Domain Design of Multidimensional FIR Filters 225 10.5.1 FIR Filter Design Using Windows 226 10.5.2 FIR Filter Design Using Least-pth Optimization 229 Problems 236 11 Changing the Sampling Structure of an Image 237 11.1 Introduction 237 11.2 Sublattices 237 11.3 Upsampling 239 11.4 Downsampling 245 11.5 Arbitrary Sampling Structure Conversion 248 11.5.1 Sampling Structure Conversion Using a Common Superlattice 248 11.5.2 Polynomial Interpolation 251 Problems 254 12 Symmetry Invariant Signals and Systems 255 12.1 LSI Systems Invariant to a Group of Symmetries 255 12.1.1 Symmetries of a Lattice 255 12.1.2 Symmetry-Group Invariant Systems 258 12.1.3 Spaces of Symmetric Signals 261 12.2 Symmetry-Invariant Discrete-Domain Periodic Signals and Systems 269 12.2.1 Symmetric Discrete-Domain Periodic Signals 270 12.2.2 Discrete-Domain Periodic Symmetry-Invariant Systems 271 12.2.3 Discrete-Domain Symmetry-Invariant Periodic Fourier Transform 273 12.3 Vector-Space Representation of Images Based on the Symmetry-Invariant Periodic Fourier Transform 282 13 Lattices 289 13.1 Introduction 289 13.2 Basic Definitions 289 13.3 Properties of Lattices 293 13.4 Reciprocal Lattice 294 13.5 Sublattices 295 13.6 Cosets and the Quotient Group 296 13.7 Basis Transformations 298 13.7.1 Elementary Column Operations 299 13.7.2 Hermite Normal Form 300 13.8 Smith Normal Form 302 13.9 Intersection and Sum of Lattices 304 Appendix A: Equivalence Relations 311 Appendix B: Groups 313 Appendix C: Vector Spaces 315 Appendix D: Multidimensional Fourier Transform Properties 319 References 323 Index 329

Read more less

Book SynopsisDelivers an appropriate mix of theory and applications to help readers understand the process and problems of image and signal analysis Maintaining a comprehensive and accessible treatment of the concepts, methods, and applications of signal and image data transformation, this Second Edition of Discrete Fourier Analysis and Wavelets: Applications to Signal and Image Processing features updated and revised coverage throughout with an emphasis on key and recent developments in the field of signal and image processing. Topical coverage includes: vector spaces, signals, and images; the discrete Fourier transform; the discrete cosine transform; convolution and filtering; windowing and localization; spectrograms; frames; filter banks; lifting schemes; and wavelets. Discrete Fourier Analysis and Wavelets introduces a new chapter on framesa new technology in which signals, images, and other data are redundantly measured. This redundancy allows for more sopTable of ContentsPreface xvii Acknowledgments xxi 1 Vector Spaces, Signals, and Images 1 1.1 Overview 1 1.2 Some Common Image Processing Problems 1 1.2.1 Applications 2 1.2.1.1 Compression 2 1.2.1.2 Restoration 2 1.2.1.3 Edge Detection 3 1.2.1.4 Registration 3 1.2.2 Transform-Based Methods 3 1.3 Signals and Images 3 1.3.1 Signals 4 1.3.2 Sampling, Quantization Error, and Noise 5 1.3.3 Grayscale Images 6 1.3.4 Sampling Images 8 1.3.5 Color 9 1.3.6 Quantization and Noise for Images 9 1.4 Vector Space Models for Signals and Images 10 1.4.1 Examples—Discrete Spaces 11 1.4.2 Examples—Function Spaces 14 1.5 Basic Waveforms—The Analog Case 16 1.5.1 The One-Dimensional Waveforms 16 1.5.2 2D Basic Waveforms 19 1.6 Sampling and Aliasing 20 1.6.1 Introduction 20 1.6.2 Aliasing for Complex Exponential Waveforms 22 1.6.3 Aliasing for Sines and Cosines 23 1.6.4 The Nyquist Sampling Rate 24 1.6.5 Aliasing in Images 24 1.7 Basic Waveforms—The Discrete Case 25 1.7.1 Discrete Basic Waveforms for Finite Signals 25 1.7.2 Discrete Basic Waveforms for Images 27 1.8 Inner Product Spaces and Orthogonality 28 1.8.1 Inner Products and Norms 28 1.8.1.1 Inner Products 28 1.8.1.2 Norms 29 1.8.2 Examples 30 1.8.3 Orthogonality 33 1.8.4 The Cauchy–Schwarz Inequality 34 1.8.5 Bases and Orthogonal Decomposition 35 1.8.5.1 Bases 35 1.8.5.2 Orthogonal and Orthonormal Bases 37 1.8.5.3 Parseval’s Identity 39 1.9 Signal and Image Digitization 39 1.9.1 Quantization and Dequantization 40 1.9.1.1 The General Quantization Scheme 41 1.9.1.2 Dequantization 42 1.9.1.3 Measuring Error 42 1.9.2 Quantifying Signal and Image Distortion More Generally 43 1.10 Infinite-Dimensional Inner Product Spaces 45 1.10.1 Example: An Infinite-Dimensional Space 45 1.10.2 Orthogonal Bases in Inner Product Spaces 46 1.10.3 The Cauchy–Schwarz Inequality and Orthogonal Expansions 48 1.10.4 The Basic Waveforms and Fourier Series 49 1.10.4.1 Complex Exponential Fourier Series 49 1.10.4.2 Sines and Cosines 52 1.10.4.3 Fourier Series on Rectangles 53 1.10.5 Hilbert Spaces and L2(a, b ) 53 1.10.5.1 Expanding the Space of Functions 53 1.10.5.2 Complications 54 1.10.5.3 A Converse to Parseval 55 1.11 Matlab Project 55 Exercises 60 2 The Discrete Fourier Transform 71 2.1 Overview 71 2.2 The Time Domain and Frequency Domain 71 2.3 A Motivational Example 73 2.3.1 A Simple Signal 73 2.3.2 Decomposition into BasicWaveforms 74 2.3.3 Energy at Each Frequency 74 2.3.4 Graphing the Results 75 2.3.5 Removing Noise 77 2.4 The One-Dimensional DFT 78 2.4.1 Definition of the DFT 78 2.4.2 Sample Signal and DFT Pairs 80 2.4.2.1 An Aliasing Example 80 2.4.2.2 Square Pulses 81 2.4.2.3 Noise 82 2.4.3 Suggestions on Plotting DFTs 84 2.4.4 An Audio Example 84 2.5 Properties of the DFT 85 2.5.1 Matrix Formulation and Linearity 85 2.5.1.1 The DFT as a Matrix 85 2.5.1.2 The Inverse DFT as a Matrix 87 2.5.2 Symmetries for Real Signals 88 2.6 The Fast Fourier transform 90 2.6.1 DFT Operation Count 90 2.6.2 The FFT 91 2.6.3 The Operation Count 92 2.7 The Two-Dimensional DFT 93 2.7.1 Interpretation and Examples of the 2-D DFT 96 2.8 Matlab Project 97 2.8.1 Audio Explorations 97 2.8.2 Images 99 Exercises 101 3 The Discrete Cosine Transform 105 3.1 Motivation for the DCT—Compression 105 3.2 Other Compression Issues 106 3.3 Initial Examples—Thresholding 107 3.3.1 Compression Example 1: A Smooth Function 108 3.3.2 Compression Example 2: A Discontinuity 109 3.3.3 Compression Example 3 110 3.3.4 Observations 112 3.4 The Discrete Cosine Transform 112 3.4.1 DFT Compression Drawbacks 112 3.4.2 The Discrete Cosine Transform 113 3.4.2.1 Symmetric Reflection 113 3.4.2.2 DFT of the Extension 113 3.4.2.3 DCT/IDCT Derivation 114 3.4.2.4 Definition of the DCT and IDCT 115 3.4.3 Matrix Formulation of the DCT 116 3.5 Properties of the DCT 116 3.5.1 BasicWaveforms for the DCT 116 3.5.2 The Frequency Domain for the DCT 117 3.5.3 DCT and Compression Examples 117 3.6 The Two-Dimensional DCT 120 3.7 Block Transforms 121 3.8 JPEG Compression 123 3.8.1 Overall Outline 123 3.8.2 DCT and Quantization Details 124 3.8.3 The JPEG Dog 128 3.8.4 Sequential versus Progressive Encoding 128 3.9 Matlab Project 131 Exercises 134 4 Convolution and Filtering 139 4.1 Overview 139 4.2 One-Dimensional Convolution 139 4.2.1 Example: Low-Pass Filtering and Noise Removal 139 4.2.2 Convolution 142 4.2.2.1 Convolution Definition 142 4.2.2.2 Convolution Properties 143 4.3 Convolution Theorem and Filtering 146 4.3.1 The Convolution Theorem 146 4.3.2 Filtering and Frequency Response 147 4.3.2.1 Filtering Effect on BasicWaveforms 147 4.3.3 Filter Design 150 4.4 2D Convolution—Filtering Images 152 4.4.1 Two-Dimensional Filtering and Frequency Response 152 4.4.2 Applications of 2D Convolution and Filtering 153 4.4.2.1 Noise Removal and Blurring 153 4.4.2.2 Edge Detection 154 4.5 Infinite and Bi-Infinite Signal Models 156 4.5.1 L2(ℕ) and L2(ℤ) 158 4.5.1.1 The Inner Product Space L2(ℕ) 158 4.5.1.2 The Inner Product Space L2(ℤ) 159 4.5.2 Fourier Analysis in L2(ℤ) and L2(ℕ) 160 4.5.2.1 The Discrete Time Fourier Transform in L2(ℤ) 160 4.5.2.2 Aliasing and the Nyquist Frequency in L2(ℤ) 161 4.5.2.3 The Fourier Transform on L2(ℕ)) 163 4.5.3 Convolution and Filtering in L2(ℤ) and L2(ℕ) 163 4.5.3.1 The Convolution Theorem 164 4.5.4 The z-Transform 166 4.5.4.1 Two Points of View 166 4.5.4.2 Algebra of z-Transforms; Convolution 167 4.5.5 Convolution in ℂN versus L2(ℤ) 168 4.5.5.1 Some Notation 168 4.5.5.2 Circular Convolution and z-Transforms 169 4.5.5.3 Convolution in ℂN from Convolution in L2(ℤ) 170 4.5.6 Some Filter Terminology 171 4.5.7 The Space L2(ℤ × ℤ) 172 4.6 Matlab Project 172 4.6.1 Basic Convolution and Filtering 172 4.6.2 Audio Signals and Noise Removal 174 4.6.3 Filtering Images 175 Exercises 176 5 Windowing and Localization 185 5.1 Overview: Nonlocality of the DFT 185 5.2 Localization via Windowing 187 5.2.1 Windowing 187 5.2.2 Analysis of Windowing 188 5.2.2.1 Step 1: Relation of X and Y 189 5.2.2.2 Step 2: Effect of Index Shift 190 5.2.2.3 Step 3: N-Point versus M-Point DFT 191 5.2.3 Spectrograms 192 5.2.4 Other Types of Windows 196 5.3 Matlab Project 198 5.3.1 Windows 198 5.3.2 Spectrograms 199 Exercises 200 6 Frames 205 6.1 Introduction 205 6.2 Packet Loss 205 6.3 Frames—Using more Dot Products 208 6.4 Analysis and Synthesis with Frames 211 6.4.1 Analysis and Synthesis 211 6.4.2 Dual Frame and Perfect Reconstruction 213 6.4.3 Partial Reconstruction 214 6.4.4 Other Dual Frames 215 6.4.5 Numerical Concerns 216 6.4.5.1 Condition Number of a Matrix 217 6.5 Initial Examples of Frames 218 6.5.1 Circular Frames in ℝ2 218 6.5.2 Extended DFT Frames and Harmonic Frames 219 6.5.3 Canonical Tight Frame 221 6.5.4 Frames for Images 222 6.6 More on the Frame Operator 222 6.7 Group-Based Frames 225 6.7.1 Unitary Matrix Groups and Frames 225 6.7.2 Initial Examples of Group Frames 228 6.7.2.1 Platonic Frames 228 6.7.2.2 Symmetric Group Frames 230 6.7.2.3 Harmonic Frames 232 6.7.3 Gabor Frames 232 6.7.3.1 Flipped Gabor Frame 237 6.8 Frame Applications 237 6.8.1 Packet Loss 239 6.8.2 Redundancy and other duals 240 6.8.3 Spectrogram 241 6.9 Matlab Project 242 6.9.1 Frames and Frame Operator 243 6.9.2 Analysis and Synthesis 245 6.9.3 Condition Number 246 6.9.4 Packet Loss 246 6.9.5 Gabor Frames 246 Exercises 247 7 Filter Banks 251 7.1 Overview 251 7.2 The Haar Filter Bank 252 7.2.1 The One-Stage Two-Channel Filter Bank 252 7.2.2 Inverting the One-stage Transform 256 7.2.3 Summary of Filter Bank Operation 257 7.3 The General One-stage Two-channel Filter Bank 260 7.3.1 Formulation for Arbitrary FIR Filters 260 7.3.2 Perfect Reconstruction 261 7.3.3 Orthogonal Filter Banks 263 7.4 Multistage Filter Banks 264 7.5 Filter Banks for Finite Length Signals 267 7.5.1 Extension Strategy 267 7.5.2 Analysis of Periodic Extension 269 7.5.2.1 Adapting the Analysis Transform to Finite Length 270 7.5.2.2 Adapting the Synthesis Transform to Finite Length 272 7.5.2.3 Other Extensions 274 7.5.3 Matrix Formulation of the Periodic Case 274 7.5.4 Multistage Transforms 275 7.5.4.1 Iterating the One-stage Transform 275 7.5.4.2 Matrix Formulation of Multistage Transform 277 7.5.4.3 Reconstruction from Approximation Coefficients 278 7.5.5 Matlab Implementation of Discrete Wavelet Transforms 281 7.6 The 2D Discrete Wavelet Transform and JPEG 2000 281 7.6.1 Two-dimensional Transforms 281 7.6.2 Multistage Transforms for Two-dimensional Images 282 7.6.3 Approximations and Details for Images 286 7.6.4 JPEG 2000 288 7.7 Filter Design 289 7.7.1 Filter Banks in the z-domain 290 7.7.1.1 Downsampling and Upsampling in the z-domain 290 7.7.1.2 Filtering in the Frequency Domain 290 7.7.2 Perfect Reconstruction in the z-frequency Domain 290 7.7.3 Filter Design I: Synthesis from Analysis 292 7.7.4 Filter Design II: Product Filters 295 7.7.5 Filter Design III: More Product Filters 297 7.7.6 Orthogonal Filter Banks 299 7.7.6.1 Design Equations for an Orthogonal Bank 299 7.7.6.2 The Product Filter in the Orthogonal Case 300 7.7.6.3 Restrictions on P(z); Spectral Factorization 301 7.7.6.4 Daubechies Filters 301 7.8 Matlab Project 303 7.8.1 Basics 303 7.8.2 Audio Signals 304 7.8.3 Images 305 7.9 Alternate Matlab Project 306 7.9.1 Basics 306 7.9.2 Audio Signals 307 7.9.3 Images 307 Exercises 309 8 Lifting for Filter Banks and Wavelets 319 8.1 Overview 319 8.2 Lifting for the Haar Filter Bank 319 8.2.1 The Polyphase Analysis 320 8.2.2 Inverting the Polyphase Haar Transform 321 8.2.3 Lifting Decomposition for the Haar Transform 322 8.2.4 Inverting the Lifted Haar Transform 324 8.3 The Lifting Theorem 324 8.3.1 A Few Facts About Laurent Polynomials 325 8.3.1.1 The Width of a Laurent Polynomial 325 8.3.1.2 The Division Algorithm 325 8.3.2 The Lifting Theorem 326 8.4 Polyphase Analysis for Filter Banks 330 8.4.1 The Polyphase Decomposition and Convolution 331 8.4.2 The Polyphase Analysis Matrix 333 8.4.3 Inverting the Transform 334 8.4.4 Orthogonal Filters 338 8.5 Lifting 339 8.5.1 Relation Between the Polyphase Matrices 339 8.5.2 Factoring the Le Gall 5/3 Polyphase Matrix 341 8.5.3 Factoring the Haar Polyphase Matrix 343 8.5.4 Efficiency 345 8.5.5 Lifting to Design Transforms 346 8.6 Matlab Project 351 8.6.1 Laurent Polynomials 351 8.6.2 Lifting for CDF(2,2) 354 8.6.3 Lifting the D4 Filter Bank 356 Exercises 356 9 Wavelets 361 9.1 Overview 361 9.1.1 Chapter Outline 361 9.1.2 Continuous from Discrete 361 9.2 The Haar Basis 363 9.2.1 Haar Functions as a Basis for L2(0, 1) 364 9.2.1.1 Haar Function Definition and Graphs 364 9.2.1.2 Orthogonality 367 9.2.1.3 Completeness in L2(0, 1) 368 9.2.2 Haar Functions as an Orthonormal Basis for L2(ℝ) 372 9.2.3 Projections and Approximations 374 9.3 Haar Wavelets Versus the Haar Filter Bank 376 9.3.1 Single-stage Case 377 9.3.1.1 Functions from Sequences 377 9.3.1.2 Filter Bank Analysis/Synthesis 377 9.3.1.3 Haar Expansion and Filter Bank Parallels 378 9.3.2 Multistage Haar Filter Bank and Multiresolution 380 9.3.2.1 Some Subspaces and Bases 381 9.3.2.2 Multiresolution and Orthogonal Decomposition 381 9.3.2.3 Direct Sums 382 9.3.2.4 Connection to Multistage Haar Filter Banks 384 9.4 Orthogonal Wavelets 386 9.4.1 Essential Ingredients 386 9.4.2 Constructing a Multiresolution Analysis: The Dilation Equation 387 9.4.3 Connection to Orthogonal Filters 389 9.4.4 Computing the Scaling Function 390 9.4.5 Scaling Function Existence and Properties 394 9.4.5.1 Fixed Point Iteration and the Cascade Algorithm 394 9.4.5.2 Existence of the Scaling Function 395 9.4.5.3 The Support of the Scaling Function 397 9.4.5.4 Back to Multiresolution 399 9.4.6 Wavelets 399 9.4.7 Wavelets and the Multiresolution Analysis 404 9.4.7.1 Final Remarks on Orthogonal Wavelets 406 9.5 Biorthogonal Wavelets 407 9.5.1 Biorthogonal Scaling Functions 408 9.5.2 Biorthogonal Wavelets 409 9.5.3 Decomposition of L2(ℝ) 409 9.6 Matlab Project 411 9.6.1 Orthogonal Wavelets 411 9.6.2 Biorthogonal Wavelets 414 Exercises 414 Bibliography 421 Appendix: Solutions to Exercises 423 Index 439

Read more less

Book SynopsisA problem-solving approach to statistical signal processing for practicing engineers, technicians, and graduate students This book takes a pragmatic approach in solving a set of common problems engineers and technicians encounter when processing signals.Table of ContentsList of Figures xvii List of Tables xxiii Preface xxv List of Abbreviations xxix How to Use the Book xxxi About the Companion Website xxxiii Prerequisites xxxv Why are there so many matrixes in this book? xxxvii 1 Manipulations on Matrixes 1 1.1 Matrix Properties 1 1.1.1 Elementary Operations 2 1.2 Eigen-Decomposition 6 1.3 Eigenvectors in Everyday Life 9 1.3.1 Conversations in a Noisy Restaurant 9 1.3.2 Power Control in a Cellular System 12 1.3.3 Price Equilibrium in the Economy 14 1.4 Derivative Rules 15 1.4.1 Derivative with respect to x 16 1.4.2 Derivative with respect to x 17 1.4.3 Derivative with respect to the Matrix X 18 1.5 Quadratic Forms 19 1.6 Diagonalization of a Quadratic Form 20 1.7 Rayleigh Quotient 21 1.8 Basics of Optimization 22 1.8.1 Quadratic Function with Simple Linear Constraint (M=1) 23 1.8.2 Quadratic Function with Multiple Linear Constraints 23 Appendix A: Arithmetic vs. Geometric Mean 24 2 Linear Algebraic Systems 27 2.1 Problem Definition and Vector Spaces 27 2.1.1 Vector Spaces in Tomographic Radiometric Inversion 29 2.2 Rotations 31 2.3 Projection Matrixes and Data-Filtering 33 2.3.1 Projections and Commercial FM Radio 34 2.4 Singular Value Decomposition (SVD) and Subspaces 34 2.4.1 How to Choose the Rank of Afor Gaussian Model? 35 2.5 QR and Cholesky Factorization 36 2.6 Power Method for Leading Eigenvectors 38 2.7 Least Squares Solution of Overdetermined Linear Equations 39 2.8 Efficient Implementation of the LS Solution 41 2.9 Iterative Methods 42 3 Random Variables in Brief 45 3.1 Probability Density Function (pdf), Moments, and Other Useful Properties 45 3.2 Convexity and Jensen Inequality 49 3.3 Uncorrelatedness and Statistical Independence 49 3.4 Real-Valued Gaussian Random Variables 51 3.5 Conditional pdf for Real-Valued Gaussian Random Variables 54 3.6 Conditional pdf in Additive Noise Model 56 3.7 Complex Gaussian Random Variables 56 3.7.1 Single Complex Gaussian Random Variable 56 3.7.2 Circular Complex Gaussian Random Variable 57 3.7.3 Multivariate Complex Gaussian Random Variables 58 3.8 Sum of Square of Gaussians: Chi-Square 59 3.9 Order Statistics for N rvs 60 4 Random Processes and Linear Systems 63 4.1 Moment Characterizations and Stationarity 64 4.2 Random Processes and Linear Systems 66 4.3 Complex-Valued Random Processes 68 4.4 Pole-Zero and Rational Spectra (Discrete-Time) 69 4.4.1 Stability of LTI Systems 70 4.4.2 Rational PSD 71 4.4.3 Paley–Wiener Theorem 72 4.5 Gaussian Random Process (Discrete-Time) 73 4.6 Measuring Moments in Stochastic Processes 75 Appendix A: Transforms for Continuous-Time Signals 76 Appendix B: Transforms for Discrete-Time Signals 79 5 Models and Applications 83 5.1 Linear Regression Model 84 5.2 Linear Filtering Model 86 5.2.1 Block-Wise Circular Convolution 88 5.2.2 Discrete Fourier Transform and Circular Convolution Matrixes 89 5.2.3 Identification and Deconvolution 90 5.3 MIMO systems and Interference Models 91 5.3.1 DSL System 92 5.3.2 MIMO in Wireless Communication 92 5.4 Sinusoidal Signal 97 5.5 Irregular Sampling and Interpolation 97 5.5.1 Sampling With Jitter 100 5.6 Wavefield Sensing System 101 6 Estimation Theory 105 6.1 Historical Notes 105 6.2 Non-Bayesian vs. Bayesian 106 6.3 Performance Metrics and Bounds 107 6.3.1 Bias 107 6.3.2 Mean Square Error (MSE) 108 6.3.3 Performance Bounds 109 6.4 Statistics and Sufficient Statistics 110 6.5 MVU and BLU Estimators 111 6.6 BLUE for Linear Models 112 6.7 Example: BLUE of the Mean Value of Gaussian rvs 114 7 Parameter Estimation 117 7.1 Maximum Likelihood Estimation (MLE) 117 7.2 MLE for Gaussian Model 119 7.2.1 Additive Noise Model with 119 7.2.2 Additive Noise Model with 120 7.2.3 Additive Noise Model with Multiple Observations with Known 121 7.2.3.1 Linear Model 121 7.2.3.2 Model 122 7.2.3.3 Model 123 7.2.4 Model 123 7.2.5 Additive Noise Model with Multiple Observations with Unknown 124 7.3 Other Noise Models 125 7.4 MLE and Nuisance Parameters 126 7.5 MLE for Continuous-Time Signals 128 7.5.1 Example: Amplitude Estimation 129 7.5.2 MLE for Correlated Noise 130 7.6 MLE for Circular Complex Gaussian 131 7.7 Estimation in Phase/Frequency Modulations 131 7.7.1 MLE Phase Estimation 132 7.7.2 Phase Locked Loops 133 7.8 Least Square (LS) Estimation 135 7.8.1 Weighted LS with 136 7.8.2 LS Estimation and Linear Models 137 7.8.3 Under or Over-Parameterizing? 138 7.8.4 Constrained LS Estimation 139 7.9 Robust Estimation 140 8 Cramér–Rao Bound 143 8.1 Cramér–Rao Bound and Fisher Information Matrix 143 8.1.1 CRB for Scalar Problem (P=1) 143 8.1.2 CRB and Local Curvature of Log-Likelihood 144 8.1.3 CRB for Multiple Parameters (p 1) 144 8.2 Interpretation of CRB and Remarks 146 8.2.1 Variance of Each Parameter 146 8.2.2 Compactness of the Estimates 146 8.2.3 FIM for Known Parameters 147 8.2.4 Approximation of the Inverse of FIM 148 8.2.5 Estimation Decoupled From FIM 148 8.2.6 CRB and Nuisance Parameters 149 8.2.7 CRB for Non-Gaussian rv and Gaussian Bound 149 8.3 CRB and Variable Transformations 150 8.4 FIM for Gaussian Parametric Model 151 8.4.1 FIM for with 151 8.4.2 FIM for Continuous-Time Signals in Additive White Gaussian Noise 152 8.4.3 FIM for Circular Complex Model 152 Appendix A: Proof of CRB 154 Appendix B: FIM for Gaussian Model 156 Appendix C: Some Derivatives for MLE and CRB Computations 157 9 MLE and CRB for Some Selected Cases 159 9.1 Linear Regressions 159 9.2 Frequency Estimation 162 9.3 Estimation of Complex Sinusoid 164 9.3.1 Proper, Improper, and Non-Circular Signals 165 9.4 Time of Delay Estimation 166 9.5 Estimation of Max for Uniform pdf 170 9.6 Estimation of Occurrence Probability for Binary pdf 172 9.7 How to Optimize Histograms? 173 9.8 Logistic Regression 176 10 Numerical Analysis and Montecarlo Simulations 179 10.1 System Identification and Channel Estimation 181 10.1.1 Matlab Code and Results 184 10.2 Frequency Estimation 184 10.2.1 Variable (Coarse/Fine) Sampling 187 10.2.2 Local Parabolic Regression 189 10.2.3 Matlab Code and Results 190 10.3 Time of Delay Estimation 192 10.3.1 Granularity of Sampling in ToD Estimation 193 10.3.2 Matlab Code and Results 194 10.4 Doppler-Radar System by Frequency Estimation 196 10.4.1 EM Method 197 10.4.2 Matlab Code and Results 199 11 Bayesian Estimation 201 11.1 Additive Linear Model with Gaussian Noise 203 11.1.1 Gaussian A-priori: 204 11.1.2 Non-Gaussian A-Priori 206 11.1.3 Binary Signals: MMSE vs. MAP Estimators 207 11.1.4 Example: Impulse Noise Mitigation 210 11.2 Bayesian Estimation in Gaussian Settings 212 11.2.1 MMSE Estimator 213 11.2.2 MMSE Estimator for Linear Models 213 11.3 LMMSE Estimation and Orthogonality 215 11.4 Bayesian CRB 218 11.5 Mixing Bayesian and Non-Bayesian 220 11.5.1 Linear Model with Mixed Random/Deterministic Parameters 220 11.5.2 Hybrid CRB 222 11.6 Expectation-Maximization (EM) 223 11.6.1 EM of the Sum of Signals in Gaussian Noise 224 11.6.2 EM Method for the Time of Delay Estimation of Multiple Waveforms 227 11.6.3 Remarks 228 Appendix A: Gaussian Mixture pdf 229 12 Optimal Filtering 231 12.1 Wiener Filter 231 12.2 MMSE Deconvolution (or Equalization) 233 12.3 Linear Prediction 234 12.3.1 Yule–Walker Equations 235 12.4 LS Linear Prediction 237 12.5 Linear Prediction and AR Processes 239 12.6 Levinson Recursion and Lattice Predictors 241 13 Bayesian Tracking and Kalman Filter 245 13.1 Bayesian Tracking of State in Dynamic Systems 246 13.1.1 Evolution of the A-posteriori pdf 247 13.2 Kalman Filter (KF) 249 13.2.1 KF Equations 251 13.2.2 Remarks 253 13.3 Identification of Time-Varying Filters in Wireless Communication 255 13.4 Extended Kalman Filter (EKF) for Non-Linear Dynamic Systems 257 13.5 Position Tracking by Multi-Lateration 258 13.5.1 Positioning and Noise 260 13.5.2 Example of Position Tracking 263 13.6 Non-Gaussian Pdf and Particle Filters264 14 Spectral Analysis 267 14.1 Periodogram 268 14.1.1 Bias of the Periodogram 268 14.1.2 Variance of the Periodogram 271 14.1.3 Filterbank Interpretation 273 14.1.4 Pdf of the Periodogram (White Gaussian Process) 274 14.1.5 Bias and Resolution 275 14.1.6 Variance Reduction and WOSA 278 14.1.7 Numerical Example: Bandlimited Process and (Small) Sinusoid 280 14.2 Parametric Spectral Analysis 282 14.2.1 MLE and CRB 284 14.2.2 General Model for AR, MA, ARMA Spectral Analysis 285 14.3 AR Spectral Analysis 286 14.3.1 MLE and CRB 286 14.3.2 A Good Reason to Avoid Over-Parametrization in AR 289 14.3.3 Cramér–Rao Bound of Poles in AR Spectral Analysis 291 14.3.4 Example: Frequency Estimation by AR Spectral Analysis 293 14.4 MA Spectral Analysis 296 14.5 ARMA Spectral Analysis 298 14.5.1 Cramér–Rao Bound for ARMA Spectral Analysis 300 Appendix A: Which Sample Estimate of the Autocorrelation to Use? 302 Appendix B: Eigenvectors and Eigenvalues of Correlation Matrix 303 Appendix C: Property of Monic Polynomial 306 Appendix D: Variance of Pole in AR(1) 307 15 Adaptive Filtering 309 15.1 Adaptive Interference Cancellation 311 15.2 Adaptive Equalization in Communication Systems 313 15.2.1 Wireless Communication Systems in Brief 313 15.2.2 Adaptive Equalization 315 15.3 Steepest Descent MSE Minimization 317 15.3.1 Convergence Analysis and Step-Size 318 15.3.2 An Intuitive View of Convergence Conditions 320 15.4 From Iterative to Adaptive Filters 323 15.5 LMS Algorithm and Stochastic Gradient 324 15.6 Convergence Analysis of LMS Algorithm 325 15.6.1 Convergence in the Mean 326 15.6.2 Convergence in the Mean Square 326 15.6.3 Excess MSE 329 15.7 Learning Curve of LMS 331 15.7.1 Optimization of the Step-Size 332 15.8 NLMS Updating and Non-Stationarity 333 15.9 Numerical Example: Adaptive Identification 334 15.10 RLS Algorithm 338 15.10.1 Convergence Analysis 339 15.10.2 Learning Curve of RLS 341 15.11 Exponentially-Weighted RLS 342 15.12 LMS vs. RLS 344 Appendix A: Convergence in Mean Square 344 16 Line Spectrum Analysis 347 16.1 Model Definition 349 16.1.1 Deterministic Signals 350 16.1.2 Random Signals 350 16.1.3 Properties of Structured Covariance 351 16.2 Maximum Likelihood and Cramér–Rao Bounds 352 16.2.1 Conditional ML 353 16.2.2 Cramér–Rao Bound for Conditional Model 354 16.2.3 Unconditional ML 356 16.2.4 Cramér–Rao Bound for Unconditional Model 356 16.2.5 Conditional vs. Unconditional Model & Bounds 357 16.3 High-Resolution Methods 357 16.3.1 Iterative Quadratic ML (IQML) 358 16.3.2 Prony Method 360 16.3.3 MUSIC 360 16.3.4 ESPRIT 363 16.3.5 Model Order 365 17 Equalization in Communication Engineering 367 17.1 Linear Equalization 369 17.1.1 Zero Forcing (ZF) Equalizer 370 17.1.2 Minimum Mean Square Error (MMSE) Equalizer 371 17.1.3 Finite-Length/Finite-Block Equalizer 371 17.2 Non-Linear Equalization 372 17.2.1 ZF-DFE 373 17.2.2 MMSE–DFE 374 17.2.3 Finite-Length MMSE–DFE 375 17.2.4 Asymptotic Performance for Infinite-Length Equalizers 376 17.3 MIMO Linear Equalization 377 17.3.1 ZF MIMO Equalization 377 17.3.2 MMSE MIMO Equalization 379 17.4 MIMO–DFE Equalization 379 17.4.1 Cholesky Factorization and Min/Max Phase Decomposition 379 17.4.2 MIMO–DFE 380 18 2D Signals and Physical Filters 383 18.1 2D Sinusoids 384 18.1.1 Moiré Pattern 386 18.2 2D Filtering 388 18.2.1 2D Random Fields 390 18.2.2 Wiener Filtering 391 18.2.3 Image Acquisition and Restoration 392 18.3 Diffusion Filtering 394 18.3.1 Evolution vs. Time: Fourier Method 394 18.3.2 Extrapolation of the Density 395 18.3.3 Effect of Phase-Shift 396 18.4 Laplace Equation and Exponential Filtering 398 18.5 Wavefield Propagation 400 18.5.1 Propagation/Backpropagation 400 18.5.2 Wavefield Extrapolation and Focusing 402 18.5.3 Exploding Reflector Model 402 18.5.4 Wavefield Extrapolation 404 18.5.5 Wavefield Focusing (or Migration) 406 Appendix A: Properties of 2D Signals 406 Appendix B: Properties of 2D Fourier Transform 410 Appendix C: Finite Difference Method for PDE-Diffusion 412 19 Array Processing 415 19.1 Narrowband Model 415 19.1.1 Multiple DoAs and Multiple Sources 419 19.1.2 Sensor Spacing Design 420 19.1.3 Spatial Resolution and Array Aperture 421 19.2 Beamforming and Signal Estimation 422 19.2.1 Conventional Beamforming 425 19.2.2 Capon Beamforming (MVDR) 426 19.2.3 Multiple-Constraint Beamforming 429 19.2.4 Max-SNR Beamforming 431 19.3 DoA Estimation 432 19.3.1 ML Estimation and CRB 433 19.3.2 Beamforming and Root-MVDR 434 20 Multichannel Time of Delay Estimation 435 20.1 Model Definition for ToD 440 20.2 High Resolution Method for ToD (L=1) 441 20.2.1 ToD in the Fourier Transformed Domain 441 20.2.2 CRB and Resolution 444 20.3 Difference of ToD (DToD) Estimation 445 20.3.1 Correlation Method for DToD 445 20.3.2 Generalized Correlation Method 448 20.4 Numerical Performance Analysis of DToD 452 20.5 Wavefront Estimation: Non-Parametric Method (L=1) 454 20.5.1 Wavefront Estimation in Remote Sensing and Geophysics 456 20.5.2 Narrowband Waveforms and 2D Phase Unwrapping 457 20.5.3 2D Phase Unwrapping in Regular Grid Spacing 458 20.6 Parametric ToD Estimation and Wideband Beamforming 460 20.6.1 Delay and Sum Beamforming 462 20.6.2 Wideband Beamforming After Fourier Transform 464 20.7 Appendix A: Properties of the Sample Correlations 465 20.8 Appendix B: How to Delay a Discrete-Time Signal? 466 20.9 Appendix C: Wavefront Estimation for 2D Arrays 467 21 Tomography 467 21.1 X-ray Tomography 471 21.1.1 Discrete Model 471 21.1.2 Maximum Likelihood 473 21.1.3 Emission Tomography 473 21.2 Algebraic Reconstruction Tomography (ART) 475 21.3 Reconstruction From Projections: Fourier Method 475 21.3.1 Backprojection Algorithm 476 21.3.2 How Many Projections to Use? 479 21.4 Traveltime Tomography 480 21.5 Internet (Network) Tomography 483 21.5.1 Latency Tomography 484 21.5.2 Packet-Loss Tomography 484 22 Cooperative Estimation 487 22.1 Consensus and Cooperation 490 22.1.1 Vox Populi: The Wisdom of Crowds 490 22.1.2 Cooperative Estimation as Simple Information Consensus 490 22.1.3 Weighted Cooperative Estimation ( ) 493 22.1.4 Distributed MLE ( ) 495 22.2 Distributed Estimation for Arbitrary Linear Models (p>1) 496 22.2.1 Centralized MLE 497 22.2.2 Distributed Weighted LS 498 22.2.3 Distributed MLE 500 22.2.4 Distributed Estimation for Under-Determined Systems 501 22.2.5 Stochastic Regressor Model 503 22.2.6 Cooperative Estimation in the Internet of Things (IoT) 503 22.2.7 Example: Iterative Distributed Estimation 505 22.3 Distributed Synchronization 506 22.3.1 Synchrony-States for Analog and Discrete-Time Clocks 507 22.3.2 Coupled Clocks 510 22.3.3 Internet Synchronization and the Network Time Protocol (NTP) 512 Appendix A: Basics of Undirected Graphs 515 23 Classification and Clustering 521 23.1 Historical Notes 522 23.2 Classification 523 23.2.1 Binary Detection Theory 523 23.2.2 Binary Classification of Gaussian Distributions 528 23.3 Classification of Signals in Additive Gaussian Noise 529 23.3.1 Detection of Known Signal 531 23.3.2 Classification of Multiple Signals 532 23.3.3 Generalized Likelihood Ratio Test (GLRT) 533 23.3.4 Detection of Random Signals 535 23.4 Bayesian Classification 536 23.4.1 To Classify or Not to Classify? 537 23.4.2 Bayes Risk 537 23.5 Pattern Recognition and Machine Learning 538 23.5.1 Linear Discriminant 539 23.5.2 Least Squares Classification 540 23.5.3 Support Vectors Principle 541 23.6 Clustering 543 23.6.1 K-Means Clustering 544 23.6.2 EM Clustering 545 References 549 Index 557

Read more less

Book SynopsisA bridge between the application of subspace-based methods for parameter estimation in signal processing and subspace-based system identification in control systems Model-Based Processing: An Applied Subspace Identification Approach provides expert insight on developing models for designing model-based signal processors (MBSP) employing subspace identification techniques to achieve model-based identification (MBID) and enables readers to evaluate overall performance using validation and statistical analysis methods. Focusing on subspace approaches to system identification problems, this book teaches readers to identify models quickly and incorporate them into various processing problems including state estimation, tracking, detection, classification, controls, communications, and other applications that require reliable models that can be adapted to dynamic environments. The extraction of a model from data is vital to numerous applications, from thTable of ContentsPreface xiii Acknowledgements xxi Glossary xxiii 1 Introduction 1 1.1 Background 1 1.2 Signal Estimation 2 1.3 Model-Based Processing 8 1.4 Model-Based Identification 16 1.5 Subspace Identification 20 1.6 Notation and Terminology 22 1.7 Summary 24 MATLAB Notes 25 References 25 Problems 26 2 Random Signals and Systems 29 2.1 Introduction 29 2.2 Discrete Random Signals 32 2.3 Spectral Representation of Random Signals 36 2.4 Discrete Systems with Random Inputs 40 2.4.1 Spectral Theorems 41 2.4.2 ARMAX Modeling 42 2.5 Spectral Estimation 44 2.5.1 Classical (Nonparametric) Spectral Estimation 44 2.5.1.1 Correlation Method (Blackman–Tukey) 45 2.5.1.2 Average Periodogram Method (Welch) 46 2.5.2 Modern (Parametric) Spectral Estimation 47 2.5.2.1 Autoregressive (All-Pole) Spectral Estimation 48 2.5.2.2 Autoregressive Moving Average Spectral Estimation 51 2.5.2.3 Minimum Variance Distortionless Response (MVDR) Spectral Estimation 52 2.5.2.4 Multiple Signal Classification (MUSIC) Spectral Estimation 55 2.6 Case Study: Spectral Estimation of Bandpass Sinusoids 59 2.7 Summary 61 MATLAB Notes 61 References 62 Problems 64 3 State-Space Models for Identification 69 3.1 Introduction 69 3.2 Continuous-Time State-Space Models 69 3.3 Sampled-Data State-Space Models 73 3.4 Discrete-Time State-Space Models 74 3.4.1 Linear Discrete Time-Invariant Systems 77 3.4.2 Discrete Systems Theory 78 3.4.3 Equivalent Linear Systems 82 3.4.4 Stable Linear Systems 83 3.5 Gauss–Markov State-Space Models 83 3.5.1 Discrete-Time Gauss–Markov Models 83 3.6 Innovations Model 89 3.7 State-Space Model Structures 90 3.7.1 Time-Series Models 91 3.7.2 State-Space and Time-Series Equivalence Models 91 3.8 Nonlinear (Approximate) Gauss–Markov State-Space Models 97 3.9 Summary 101 MATLAB Notes 102 References 102 Problems 103 4 Model-Based Processors 107 4.1 Introduction 107 4.2 Linear Model-Based Processor: Kalman Filter 108 4.2.1 Innovations Approach 110 4.2.2 Bayesian Approach 114 4.2.3 Innovations Sequence 116 4.2.4 Practical Linear Kalman Filter Design: Performance Analysis 117 4.2.5 Steady-State Kalman Filter 125 4.2.6 Kalman Filter/Wiener Filter Equivalence 128 4.3 Nonlinear State-Space Model-Based Processors 129 4.3.1 Nonlinear Model-Based Processor: Linearized Kalman Filter 130 4.3.2 Nonlinear Model-Based Processor: Extended Kalman Filter 133 4.3.3 Nonlinear Model-Based Processor: Iterated–Extended Kalman Filter 138 4.3.4 Nonlinear Model-Based Processor: Unscented Kalman Filter 141 4.3.5 Practical Nonlinear Model-Based Processor Design: Performance Analysis 148 4.3.6 Nonlinear Model-Based Processor: Particle Filter 151 4.3.7 Practical Bayesian Model-Based Design: Performance Analysis 160 4.4 Case Study: 2D-Tracking Problem 166 4.5 Summary 173 MATLAB Notes 173 References 174 Problems 177 5 Parametrically Adaptive Processors 185 5.1 Introduction 185 5.2 Parametrically Adaptive Processors: Bayesian Approach 186 5.3 Parametrically Adaptive Processors: Nonlinear Kalman Filters 187 5.3.1 Parametric Models 188 5.3.2 Classical Joint State/Parametric Processors: Augmented Extended Kalman Filter 190 5.3.3 Modern Joint State/Parametric Processor: Augmented Unscented Kalman Filter 198 5.4 Parametrically Adaptive Processors: Particle Filter 201 5.4.1 Joint State/Parameter Estimation: Particle Filter 201 5.5 Parametrically Adaptive Processors: Linear Kalman Filter 208 5.6 Case Study: Random Target Tracking 214 5.7 Summary 222 MATLAB Notes 223 References 223 Problems 226 6 Deterministic Subspace Identification 231 6.1 Introduction 231 6.2 Deterministic Realization Problem 232 6.2.1 Realization Theory 233 6.2.2 Balanced Realizations 238 6.2.3 Systems Theory Summary 239 6.3 Classical Realization 241 6.3.1 Ho–Kalman Realization Algorithm 241 6.3.2 SVD Realization Algorithm 243 6.3.2.1 Realization: Linear Time-Invariant Mechanical Systems 246 6.3.3 Canonical Realization 251 6.3.3.1 Invariant System Descriptions 251 6.3.3.2 Canonical Realization Algorithm 257 6.4 Deterministic Subspace Realization: Orthogonal Projections 264 6.4.1 Subspace Realization: Orthogonal Projections 266 6.4.2 Multivariable Output Error State-Space (MOESP) Algorithm 271 6.5 Deterministic Subspace Realization: Oblique Projections 274 6.5.1 Subspace Realization: Oblique Projections 278 6.5.2 Numerical Algorithms for Subspace State-Space System Identification (N4SID) Algorithm 280 6.6 Model Order Estimation and Validation 285 6.6.1 Order Estimation: SVD Approach 286 6.6.2 Model Validation 289 6.7 Case Study: Structural Vibration Response 295 6.8 Summary 299 MATLAB Notes 300 References 300 Problems 303 7 Stochastic Subspace Identification 309 7.1 Introduction 309 7.2 Stochastic Realization Problem 312 7.2.1 Correlated Gauss–Markov Model 312 7.2.2 Gauss–Markov Power Spectrum 313 7.2.3 Gauss–Markov Measurement Covariance 314 7.2.4 Stochastic Realization Theory 315 7.3 Classical Stochastic Realization via the Riccati Equation 317 7.4 Classical Stochastic Realization via Kalman Filter 321 7.4.1 Innovations Model 321 7.4.2 Innovations Power Spectrum 322 7.4.3 Innovations Measurement Covariance 323 7.4.4 Stochastic Realization: Innovations Model 325 7.5 Stochastic Subspace Realization: Orthogonal Projections 330 7.5.1 Multivariable Output Error State-SPace (MOESP) Algorithm 334 7.6 Stochastic Subspace Realization: Oblique Projections 342 7.6.1 Numerical Algorithms for Subspace State-Space System Identification (N4SID) Algorithm 346 7.6.2 Relationship: Oblique (N4SID) and Orthogonal (MOESP) Algorithms 351 7.7 Model Order Estimation and Validation 353 7.7.1 Order Estimation: Stochastic Realization Problem 354 7.7.1.1 Order Estimation: Statistical Methods 356 7.7.2 Model Validation 362 7.7.2.1 Residual Testing 363 7.8 Case Study: Vibration Response of a Cylinder: Identification and Tracking 369 7.9 Summary 378 MATLAB NOTES 378 References 379 Problems 382 8 Subspace Processors for Physics-Based Application 391 8.1 Subspace Identification of a Structural Device 391 8.1.1 State-Space Vibrational Systems 392 8.1.1.1 State-Space Realization 394 8.1.2 Deterministic State-Space Realizations 396 8.1.2.1 Subspace Approach 396 8.1.3 Vibrational System Processing 398 8.1.4 Application: Vibrating Structural Device 400 8.1.5 Summary 404 8.2 MBID for Scintillator System Characterization 405 8.2.1 Scintillation Pulse Shape Model 407 8.2.2 Scintillator State-Space Model 409 8.2.3 Scintillator Sampled-Data State-Space Model 410 8.2.4 Gauss–Markov State-Space Model 411 8.2.5 Identification of the Scintillator Pulse Shape Model 412 8.2.6 Kalman Filter Design: Scintillation/Photomultiplier System 414 8.2.6.1 Kalman Filter Design: Scintillation/Photomultiplier Data 416 8.2.7 Summary 417 8.3 Parametrically Adaptive Detection of Fission Processes 418 8.3.1 Fission-Based Processing Model 419 8.3.2 Interarrival Distribution 420 8.3.3 Sequential Detection 422 8.3.4 Sequential Processor 422 8.3.5 Sequential Detection for Fission Processes 424 8.3.6 Bayesian Parameter Estimation 426 8.3.7 Sequential Bayesian Processor 427 8.3.8 Particle Filter for Fission Processes 429 8.3.9 SNM Detection and Estimation: Synthesized Data 430 8.3.10 Summary 433 8.4 Parametrically Adaptive Processing for Shallow Ocean Application 435 8.4.1 State-Space Propagator 436 8.4.2 State-Space Model 436 8.4.2.1 Augmented State-Space Models 438 8.4.3 Processors 441 8.4.4 Model-Based Ocean Acoustic Processing 444 8.4.4.1 Adaptive PF Design: Modal Coefficients 445 8.4.4.2 Adaptive PF Design: Wavenumbers 447 8.4.5 Summary 450 8.5 MBID for Chirp Signal Extraction 452 8.5.1 Chirp-like Signals 453 8.5.1.1 Linear Chirp 453 8.5.1.2 Frequency-Shift Key (FSK) Signal 455 8.5.2 Model-Based Identification: Linear Chirp Signals 457 8.5.2.1 Gauss–Markov State-Space Model: Linear Chirp 457 8.5.3 Model-Based Identification: FSK Signals 459 8.5.3.1 Gauss–Markov State-Space Model: FSK Signals 460 8.5.4 Summary 462 References 462 Appendix A Probability and Statistics Overview 467 A.1 Probability Theory 467 A.2 Gaussian Random Vectors 473 A.3 Uncorrelated Transformation: Gaussian Random Vectors 473 A.4 Toeplitz Correlation Matrices 474 A.5 Important Processes 474 References 476 Appendix B Projection Theory 477 B.1 Projections: Deterministic Spaces 477 B.2 Projections: Random Spaces 478 B.3 Projection: Operators 479 B.3.1 Orthogonal (Perpendicular) Projections 479 B.3.2 Oblique (Parallel) Projections 481 References 483 Appendix C Matrix Decompositions 485 C.1 Singular-Value Decomposition 485 C.2 QR-Decomposition 487 C.3 LQ-Decomposition 487 References 488 Appendix D Output-Only Subspace Identification 489 References 492 Index 495

Read more less

Book SynopsisAn Introduction to Audio Content Analysis Enables readers to understand the algorithmic analysis of musical audio signals with AI-driven approaches An Introduction to Audio Content Analysis serves as a comprehensive guide on audio content analysis explaining how signal processing and machine learning approaches can be utilized for the extraction of musical content from audio. It gives readers the algorithmic understanding to teach a computer to interpret music signals and thus allows for the design of tools for interacting with music. The work ties together topics from audio signal processing and machine learning, showing how to use audio content analysis to pick up musical characteristics automatically. A multitude of audio content analysis tasks related to the extraction of tonal, temporal, timbral, and intensity-related characteristics of the music signal are presented. Each task is introduced from both a musical and a technical perspective, detailing the algorithmic approach as well as providing practical guidance on implementation details and evaluation. To aid in reader comprehension, each task description begins with a short introduction to the most important musical and perceptual characteristics of the covered topic, followed by a detailed algorithmic model and its evaluation, and concluded with questions and exercises. For the interested reader, updated supplemental materials are provided via an accompanying website. Written by a well-known expert in the music industry, sample topics covered in Introduction to Audio Content Analysis include: Digital audio signals and their representation, common time-frequency transforms, audio featuresPitch and fundamental frequency detection, key and chordRepresentation of dynamics in music and intensity-related featuresBeat histograms, onset and tempo detection, beat histograms, and detection of structure in music, and sequence alignmentAudio fingerprinting, musical genre, mood, and instrument classification An invaluable guide for newcomers to audio signal processing and industry experts alike, An Introduction to Audio Content Analysis covers a wide range of introductory topics pertaining to music information retrieval and machine listening, allowing students and researchers to quickly gain core holistic knowledge in audio analysis and dig deeper into specific aspects of the field with the help of a large amount of references.Table of ContentsAuthor Biography xvii Preface xix Acronyms xxi List of Symbols xxv Source Code Repositories xxix 1 Introduction 1 Part I Fundamentals of Audio Content Analysis 9 2 Analysis of Audio Signals 11 3 Input Representation 17 4 Inference 91 5 Data 107 Part II Music Transcription 127 7 Tonal Analysis 129 8 Intensity217 9 Temporal Analysis 229 10 Alignment 281 Part III Music Identification, Classification, and Assessment 303 11 Audio Fingerprinting 305 12 Music Similarity Detection and Music Genre Classification 317 13 Mood Recognition 337 14 Musical Instrument Recognition 347 15 Music Performance Assessment 355 Part IV Appendices 365 Appendix A Fundamentals 367 Appendix B Fourier Transform 385 Appendix C Principal Component Analysis 405 Appendix D Linear Regression 409 Appendix E Software for Audio Analysis 411 Appendix F Datasets 417 Index 425

Read more less

Book SynopsisProvides a comprehensive resource on what has become the dominant paradigm in designing novel interaction methods, involving gestures, speech, text, touch and brain-controlled interaction, embedded in innovative and emerging human-computer interfaces.

Read more less

Book SynopsisProvides a comprehensive resource on what has become the dominant paradigm in designing novel interaction methods, involving gestures, speech, text, touch and brain-controlled interaction, embedded in innovative and emerging human-computer interfaces.

Read more less

Book SynopsisWith signal processing, we want to identify/characterize various physical processes, such as standard engineering variables (e.g. faults on bearings, dynamic properties of structures), as well as biology phenomena (e.g. genome sequencing), astronomy (e.g. measuring a black hole), social sciences (e.g. the spread of the corona virus) and the like. This book edition covers different topics from signal and information processing, including methods for signal processing, signal processing in medicine, image and video signal processing, and signal processing in engineering.Table of Contents Section 1 Methods for Signal Processing Chapter 1 Comparative Study and Analysis of Performances among RNS, DBNS, TBNS and MNS for DSP Applications Chapter 2 The Fourier Notation of the Geomagnetic Signals Informative Parameters Chapter 3 A Time Dependent Model for Image DenoisingSection 2 Methods for Signal Processing Chapter 4 An Efficient Signal Processing Algorithm for Detecting Abnormalities in EEG Signal Using CNN Chapter 5 Contribution to S-EMG Signal Compression in 1D by the Combination of the Modified Discrete Wavelet Packet Transform (MDWPT) and the Discrete Cosine Transform (DCT) Chapter 6 Improved Guided Image Fusion for Magnetic Resonance and Computed Tomography Imaging Chapter 7 A Biologically Inspired Algorithm for Low Energy Clustering Problem in Body Area NetworkSection 3 Image and Video Signal Processing Chapter 8 Segmentation of Visual Images by Sequential Extracting Homogeneous Texture Areas Chapter 9 Deep Learning in Visual Computing and Signal Processing Chapter 10 Pre-Processing Images of Public Signage for OCR Conversion Chapter 11 A Local Binary Pattern-Based Method for Color and Multicomponent Texture AnalysisSection 4 Engineering Applications of Signal Processing Chapter 12 An Adaptive EMD Technique for Induction Motor Fault Detection Chapter 13 Nondestructive Testing for Corrosion Evaluation of Metal Under Coating Chapter 14 Environment Perception Technologies for Power Transmission Line Inspection Robots Chapter 15 Feature Extraction Techniques of Non-Stationary Signals for Fault Diagnosis in Machinery Systems

Read more less

Book SynopsisThe parameter estimation and hypothesis testing are the basic tools in statistical inference. These techniques occur in many applications of data processing., and methods of Monte Carlo have become an essential tool to assess performance. For pedagogical purposes the book includes several computational problems and exercices. To prevent students from getting stuck on exercises, detailed corrections are provided.Table of ContentsPreface ix Notations and Abbreviations xi A Few Functions of Python® xiii Chapter 1 Useful Maths 1 1.1. Basic concepts on probability 1 1.2. Conditional expectation 10 1.3. Projection theorem 11 1.3.1. Conditional expectation 14 1.4. Gaussianity 14 1.4.1. Gaussian random variable 14 1.4.2. Gaussian random vectors 15 1.4.3. Gaussian conditional distribution 16 1.5. Random variable transformation 18 1.5.1. General expression 18 1.5.2. Law of the sum of two random variables 19 1.5.3. δ-method 20 1.6. Fundamental theorems of statistics 22 1.7. A few probability distributions 24 Chapter 2 Statistical Inferences 29 2.1. First step: visualizing data 29 2.1.1. Scatter plot 29 2.1.2. Histogram/boxplot 30 2.1.3. Q-Q plot 32 2.2. Reduction of dataset dimensionality 34 2.2.1. PCA 34 2.2.2. LDA 36 2.3. Some vocabulary 40 2.3.1. Statistical inference 40 2.4. Statistical model 41 2.4.1. Notation 42 2.5. Hypothesis testing 43 2.5.1. Simple hypotheses 45 2.5.2. Generalized likelihood ratio test (GLRT) 50 2.5.3. χ 2 goodness-of-fit test 57 2.6. Statistical estimation 58 2.6.1. General principles 58 2.6.2. Least squares method 62 2.6.3. Least squares method for the linear model 64 2.6.4. Method of moments 81 2.6.5. Maximum likelihood approach 84 2.6.6. Logistic regression 100 2.6.7. Non-parametric estimation of probability distribution 103 2.6.8. Bootstrap and others 107 Chapter 3 Inferences on HMM 113 3.1. Hidden Markov models (HMM) 113 3.2. Inferences on HMM 116 3.3. Filtering: general case 117 3.4. Gaussian linear case: Kalman algorithm 118 3.4.1. Kalman filter 118 3.4.2. RTS smoother 127 3.5. Discrete finite Markov case 129 3.5.1. Forward-backward formulas 130 3.5.2. Smoothing formula at one instant 133 3.5.3. Smoothing formula at two successive instants 134 3.5.4. HMM learning using the EM algorithm 135 3.5.5. The Viterbi algorithm 137 Chapter 4 Monte-Carlo Methods 141 4.1. Fundamental theorems 141 4.2. Stating the problem 141 4.3. Generating random variables 144 4.3.1. The cumulative function inversion method 144 4.3.2. The variable transformation method 147 4.3.3. Acceptance-rejection method 149 4.3.4. Sequential methods 151 4.4. Variance reduction 156 4.4.1. Importance sampling 156 4.4.2. Stratification 160 4.4.3. Antithetic variates 164 Chapter 5 Hints and Solutions 167 5.1. Useful maths 167 5.2. Statistical inferences 170 5.3. Inferences on HMM 226 5.4. Monte-Carlo methods 251 Bibliography 261 Index 263

Read more less

Book SynopsisNowadays, tensors play a central role for the representation, mining, analysis, and fusion of multidimensional, multimodal, and heterogeneous big data in numerous fields. This set on Matrices and Tensors in Signal Processing aims at giving a self-contained and comprehensive presentation of various concepts and methods, starting from fundamental algebraic structures to advanced tensor-based applications, including recently developed tensor models and efficient algorithms for dimensionality reduction and parameter estimation. Although its title suggests an orientation towards signal processing, the results presented in this set will also be of use to readers interested in other disciplines. This first book provides an introduction to matrices and tensors of higher-order based on the structures of vector space and tensor space. Some standard algebraic structures are first described, with a focus on the hilbertian approach for signal representation, and function approximation based on Fourier series and orthogonal polynomial series. Matrices and hypermatrices associated with linear, bilinear and multilinear maps are more particularly studied. Some basic results are presented for block matrices. The notions of decomposition, rank, eigenvalue, singular value, and unfolding of a tensor are introduced, by emphasizing similarities and differences between matrices and tensors of higher-order.Table of ContentsPreface xi Chapter 1. Historical Elements of Matrices and Tensors 1 Chapter 2. Algebraic Structures 9 2.1. A few historical elements 9 2.2. Chapter summary 11 2.3. Sets 12 2.3.1. Definitions 12 2.3.2. Sets of numbers 13 2.3.3. Cartesian product of sets 13 2.3.4. Set operations 14 2.3.5. De Morgan’s laws 15 2.3.6. Characteristic functions 15 2.3.7. Partitions 16 2.3.8. σ-algebras or σ-fields 16 2.3.9. Equivalence relations 16 2.3.10. Order relations 17 2.4. Maps and composition of maps 17 2.4.1. Definitions 17 2.4.2. Properties 18 2.4.3. Composition of maps 18 2.5. Algebraic structures 18 2.5.1. Laws of composition 18 2.5.2. Definition of algebraic structures 22 2.5.3. Substructures 24 2.5.4. Quotient structures 24 2.5.5. Groups 24 2.5.6. Rings 27 2.5.7. Fields 32 2.5.8. Modules 33 2.5.9. Vector spaces 33 2.5.10. Vector spaces of linear maps 38 2.5.11. Vector spaces of multilinear maps 39 2.5.12. Vector subspaces 41 2.5.13. Bases 43 2.5.14. Sum and direct sum of subspaces 45 2.5.15. Quotient vector spaces 47 2.5.16. Algebras 47 2.6. Morphisms 49 2.6.1. Group morphisms 49 2.6.2. Ring morphisms 51 2.6.3. Morphisms of vector spaces or linear maps 51 2.6.4. Algebra morphisms 56 Chapter 3. Banach and Hilbert Spaces – Fourier Series and Orthogonal Polynomials 57 3.1. Introduction and chapter summary 57 3.2. Metric spaces 59 3.2.1. Definition of distance 60 3.2.2. Definition of topology 60 3.2.3. Examples of distances 61 3.2.4. Inequalities and equivalent distances 62 3.2.5. Distance and convergence of sequences 62 3.2.6. Distance and local continuity of a function 62 3.2.7. Isometries and Lipschitzian maps 63 3.3. Normed vector spaces 63 3.3.1. Definition of norm and triangle inequalities 63 3.3.2. Examples of norms 64 3.3.3. Equivalent norms 68 3.3.4. Distance associated with a norm 69 3.4. Pre-Hilbert spaces 69 3.4.1. Real pre-Hilbert spaces 70 3.4.2. Complex pre-Hilbert spaces 70 3.4.3. Norm induced from an inner product 72 3.4.4. Distance associated with an inner product 75 3.4.5. Weighted inner products 76 3.5. Orthogonality and orthonormal bases 76 3.5.1. Orthogonal/perpendicular vectors and Pythagorean theorem 76 3.5.2. Orthogonal subspaces and orthogonal complement 77 3.5.3. Orthonormal bases 79 3.5.4. Orthogonal/unitary endomorphisms and isometries 79 3.6. Gram–Schmidt orthonormalization process 80 3.6.1. Orthogonal projection onto a subspace 80 3.6.2. Orthogonal projection and Fourier expansion 80 3.6.3. Bessel’s inequality and Parseval’s equality 82 3.6.4. Gram–Schmidt orthonormalization process 83 3.6.5. QR decomposition 85 3.6.6. Application to the orthonormalization of a set of functions 86 3.7. Banach and Hilbert spaces 88 3.7.1. Complete metric spaces 88 3.7.2. Adherence, density and separability 90 3.7.3. Banach and Hilbert spaces 91 3.7.4. Hilbert bases 93 3.8. Fourier series expansions 97 3.8.1. Fourier series, Parseval’s equality and Bessel’s inequality 97 3.8.2. Case of 2π-periodic functions from R to C 97 3.8.3. T-periodic functions from R to C 102 3.8.4. Partial Fourier sums and Bessel’s inequality 102 3.8.5. Convergence of Fourier series 103 3.8.6. Examples of Fourier series 108 3.9. Expansions over bases of orthogonal polynomials 117 Chapter 4. Matrix Algebra 123 4.1. Chapter summary 123 4.2. Matrix vector spaces 124 4.2.1. Notations and definitions 124 4.2.2. Partitioned matrices 125 4.2.3. Matrix vector spaces 126 4.3. Some special matrices 127 4.4. Transposition and conjugate transposition 128 4.5. Vectorization 130 4.6. Vector inner product, norm and orthogonality 130 4.6.1. Inner product 130 4.6.2. Euclidean/Hermitian norm 131 4.6.3. Orthogonality 131 4.7. Matrix multiplication 132 4.7.1. Definition and properties 132 4.7.2. Powers of a matrix 134 4.8. Matrix trace, inner product and Frobenius norm 137 4.8.1. Definition and properties of the trace 137 4.8.2. Matrix inner product 138 4.8.3. Frobenius norm 138 4.9. Subspaces associated with a matrix 139 4.10. Matrix rank 141 4.10.1. Definition and properties 141 4.10.2. Sum and difference rank 143 4.10.3. Subspaces associated with a matrix product 143 4.10.4. Product rank 144 4.11. Determinant, inverses and generalized inverses 145 4.11.1. Determinant 145 4.11.2. Matrix inversion 148 4.11.3. Solution of a homogeneous system of linear equations 149 4.11.4. Complex matrix inverse 150 4.11.5. Orthogonal and unitary matrices 150 4.11.6. Involutory matrices and anti-involutory matrices 151 4.11.7. Left and right inverses of a rectangular matrix 153 4.11.8. Generalized inverses 155 4.11.9. Moore–Penrose pseudo-inverse 157 4.12. Multiplicative groups of matrices 158 4.13. Matrix associated to a linear map 159 4.13.1. Matrix representation of a linear map 159 4.13.2. Change of basis 162 4.13.3. Endomorphisms 164 4.13.4. Nilpotent endomorphisms 166 4.13.5. Equivalent, similar and congruent matrices 167 4.14. Matrix associated with a bilinear/sesquilinear form 168 4.14.1. Definition of a bilinear/sesquilinear map 168 4.14.2. Matrix associated to a bilinear/sesquilinear form 170 4.14.3. Changes of bases with a bilinear form 170 4.14.4. Changes of bases with a sesquilinear form 171 4.14.5. Symmetric bilinear/sesquilinear forms 172 4.15. Quadratic forms and Hermitian forms 174 4.15.1. Quadratic forms 174 4.15.2. Hermitian forms 176 4.15.3. Positive/negative definite quadratic/Hermitian forms 177 4.15.4. Examples of positive definite quadratic forms 178 4.15.5. Cauchy–Schwarz and Minkowski inequalities 179 4.15.6. Orthogonality, rank, kernel and degeneration of a bilinear form 180 4.15.7. Gauss reduction method and Sylvester’s inertia law 181 4.16. Eigenvalues and eigenvectors 184 4.16.1. Characteristic polynomial and Cayley–Hamilton theorem 184 4.16.2. Right eigenvectors 186 4.16.3. Spectrum and regularity/singularity conditions 187 4.16.4. Left eigenvectors 188 4.16.5. Properties of eigenvectors 188 4.16.6. Eigenvalues and eigenvectors of a regularized matrix 190 4.16.7. Other properties of eigenvalues 190 4.16.8. Symmetric/Hermitian matrices 191 4.16.9. Orthogonal/unitary matrices 193 4.16.10. Eigenvalues and extrema of the Rayleigh quotient 194 4.17. Generalized eigenvalues 195 Chapter 5. Partitioned Matrices 199 5.1. Introduction 199 5.2. Submatrices 200 5.3. Partitioned matrices 201 5.4. Matrix products and partitioned matrices 202 5.4.1. Matrix products 202 5.4.2. Vector Kronecker product 202 5.4.3. Matrix Kronecker product 202 5.4.4. Khatri–Rao product 204 5.5. Special cases of partitioned matrices 205 5.5.1. Block-diagonal matrices 205 5.5.2. Signature matrices 205 5.5.3. Direct sum 205 5.5.4. Jordan forms 206 5.5.5. Block-triangular matrices 206 5.5.6. Block Toeplitz and Hankel matrices 207 5.6. Transposition and conjugate transposition 207 5.7. Trace 208 5.8. Vectorization 208 5.9. Blockwise addition 208 5.10. Blockwise multiplication 209 5.11. Hadamard product of partitioned matrices 209 5.12. Kronecker product of partitioned matrices 210 5.13. Elementary operations and elementary matrices 212 5.14. Inversion of partitioned matrices 214 5.14.1. Inversion of block-diagonal matrices 215 5.14.2. Inversion of block-triangular matrices 215 5.14.3. Block-triangularization and Schur complements 216 5.14.4. Block-diagonalization and block-factorization 216 5.14.5. Block-inversion and partitioned inverse 217 5.14.6. Other formulae for the partitioned 2 × 2 inverse 218 5.14.7. Solution of a system of linear equations 219 5.14.8. Inversion of a partitioned Gram matrix 220 5.14.9. Iterative inversion of a partitioned square matrix 220 5.14.10. Matrix inversion lemma and applications 221 5.15. Generalized inverses of 2 × 2 block matrices 222 5.16. Determinants of partitioned matrices 224 5.16.1. Determinant of block-diagonal matrices 224 5.16.2. Determinant of block-triangular matrices 225 5.16.3. Determinant of partitioned matrices with square diagonal blocks 225 5.16.4. Determinants of specific partitioned matrices 226 5.16.5. Eigenvalues of CB and BC 227 5.17. Rank of partitioned matrices 228 5.18. Levinson–Durbin algorithm 229 5.18.1. AR process and Yule–Walker equations 230 5.18.2. Levinson–Durbin algorithm 232 5.18.3. Linear prediction 237 Chapter 6. Tensor Spaces and Tensors 243 6.1. Chapter summary 243 6.2. Hypermatrices 243 6.2.1. Hypermatrix vector spaces 244 6.2.2. Hypermatrix inner product and Frobenius norm 245 6.2.3. Contraction operation and n-mode hypermatrix–matrix product 245 6.3. Outer products 249 6.4. Multilinear forms, homogeneous polynomials and hypermatrices 251 6.4.1. Hypermatrix associated to a multilinear form 251 6.4.2. Symmetric multilinear forms and symmetric hypermatrices 252 6.5. Multilinear maps and homogeneous polynomials 255 6.6. Tensor spaces and tensors 255 6.6.1. Definitions 255 6.6.2. Multilinearity and associativity 257 6.6.3. Tensors and coordinate hypermatrices 257 6.6.4. Canonical writing of tensors 258 6.6.5. Expansion of the tensor product of N vectors 260 6.6.6. Properties of the tensor product 261 6.6.7. Change of basis formula 266 6.7. Tensor rank and tensor decompositions 268 6.7.1. Matrix rank 268 6.7.2. Hypermatrix rank 268 6.7.3. Symmetric rank of a hypermatrix 269 6.7.4. Comparative properties of hypermatrices and matrices 269 6.7.5. CPD and dimensionality reduction 271 6.7.6. Tensor rank 273 6.8. Eigenvalues and singular values of a hypermatrix 274 6.9. Isomorphisms of tensor spaces 276 References 281 Index 291

Read more less

Book SynopsisMathematical morphology has developed a powerful methodology for segmenting images, based on connected filters and watersheds. We have chosen the abstract framework of node- or edge-weighted graphs for an extensive mathematical and algorithmic description of these tools. Volume 1 is devoted to watersheds. The topography of a graph appears by observing the evolution of a drop of water moving from node to node on a weighted graph, along flowing paths, until it reaches regional minima. The upstream nodes of a regional minimum constitute its catchment zone. The catchment zones may be constructed independently of each other and locally, in contrast with the traditional approach where the catchment basins have to be constructed all at the same time. Catchment zones may overlap, and thus, a new segmentation paradigm is proposed in which catchment zones cover each other according to a priority order. The resulting partition may then be corrected, by local and parallel treatments, in order to achieve the desired precision. Table of ContentsNotations xiii Introduction xxvii Part 1. Getting Started 1 Chapter 1. A Primer to Flooding, Razing and Watersheds 3 1.1. Topographic reliefs and topographic features 3 1.1.1. Images seen as topographic reliefs and inversely 3 1.1.2. Topographic features 5 1.1.3. Modeling a topographic relief as a weighted graph 8 1.2. Flooding, razing and morphological filters 10 1.2.1. The principle of duality 10 1.2.2. Dominated flooding and razing 10 1.2.3. Flooding, razing and catchment zones of a topographic relief 16 1.3. Catchment zones of flooded surfaces 18 1.3.1. Filtering and segmenting 18 1.3.2. Reducing the oversegmentation with markers 19 1.4. The waterfall hierarchy 26 1.4.1. Overflows between catchment basins 26 1.5. Size-driven hierarchies 28 1.6. Separating overlapping particles in n dimensions 31 1.7. Catchment zones and lakes of region neighborhood graphs 33 1.8. Conclusion 37 Chapter 2. Watersheds and Flooding: a Segmentation Golden Braid 39 2.1. Watersheds, offsprings and parallel branches 40 2.2. Flooding and connected operators 43 2.3. Connected operators and hierarchies 45 2.4. Hierarchical segmentation: extinction values 47 Chapter 3. Mathematical Notions 49 3.1. Summary of the chapter 49 3.2. Complete lattices 49 3.2.1. Partial order and partially ordered sets 49 3.2.2. Upper and lower bounds 50 3.2.3. Complete lattices 50 3.2.4. Dyadic relations on a complete lattice 51 3.3. Operators between complete lattices 51 3.3.1. Definition of an operator 51 3.3.2. Properties of the operators 52 3.3.3. Erosion and dilation 52 3.3.4. Opening and closing 53 3.4. The adjunction: a cornerstone of mathematical morphology 53 3.4.1. Adjoint erosions and dilations 53 3.4.2. Increasingness 53 3.4.3. Unicity 53 3.4.4. Composition 54 3.4.5. Dual operators 54 3.5. Openings and closings 54 3.5.1. Definitions 54 3.5.2. Elements with the same erosion or the same dilation 55 3.5.3. The invariants of an opening or a closing 55 3.6. Complete lattices of functions 55 3.6.1. Definitions 55 3.6.2. Infimum and supremum 56 Part 2. The Topography of Weighted Graphs 57 Chapter 4. Weighted Graphs 59 4.1. Summary of the chapter 59 4.2. Reminders on graphs 60 4.2.1. Directed and undirected graphs 60 4.3. Weight distributions on the nodes or edges of a graph 62 4.3.1. Duality 63 4.3.2. Erosions and dilations, openings, closings 63 4.3.3. Labels 66 4.4. Exploring the topography of graphs by following a drop of water 66 4.5. Node-weighted graphs 67 4.5.1. Flat zones and regional minima 67 4.5.2. Flowing paths and catchment zones 67 4.6. Edge-weighted graphs 69 4.6.1. Flat zones and regional minima 69 4.6.2. Flowing paths and catchment zones 69 4.6.3. Even zones and regional minima 71 4.7. Comparing the topography of node-weighted graphs and edge-weighted graphs 72 Chapter 5. Flowing Graphs 73 5.1. Summary of the chapter 73 5.2. Towards a convergence between node- and edge-weighted graphs 74 5.2.1. The flowing edges in a node-weighted graph G(ν, nil) 74 5.2.2. The flowing edges in an edge-weighted graph G(nil, η) 75 5.2.3. Flowing graphs 76 5.3. The flowing adjunction 76 5.4. Flowing edges under closer scrutiny 77 5.4.1. Relations between the flowing edges of G(ν, nil) and G(nil, δenν) 77 5.4.2. Relations between the flowing edges of G(nil, η) and G(εneη, nil) 78 5.4.3. Chaining the inclusions between flowing edges 78 5.4.4. Criteria characterizing flowing graphs 79 5.4.5. Transforming a node- or edge-weighted graph into a flowing graph 81 5.4.6. The invariance domains of γe and ϕn 83 5.4.7. Particular flowing graphs 87 5.5. Illustration as a hydrographic model 88 5.5.1. A hydrographic model of tanks and pipes 88 5.5.2. Associating an “edge unstable” tank network with an arbitrary node-weighted graph G(ν, nil) 90 5.5.3. Associating a “node unstable” tank network with an arbitrary edge-weighted graph G(nil, η) 91 5.5.4. Chaining the operations 92 Chapter 6. The Topography of Digraphs 97 6.1. Summary of the chapter 97 6.1.1. General digraphs 98 6.1.2. Digraphs without perpetuum mobile configurations 98 6.2. Status report 98 6.2.1. Case of node-weighted graphs 99 6.2.2. Case of edge-weighted graphs 99 6.3. The topography of unweighted digraphs 100 6.3.1. Notations 100 6.3.2. Smooth zones, dead ends, flat zones and black holes of digraphs 101 6.4. The topography of gravitational digraphs 105 6.4.1. No “perpetuum mobile” 105 6.4.2. Defining and propagating labels 107 6.4.3. A dead leaves model of catchment zones 113 6.4.4. Examples of gravitational graphs 122 6.4.5. The topography of weighted graphs interpreted in the light of the derived digraphs 122 Part 3. Reducing the Overlapping of Catchment Zones 125 Chapter 7. Measuring the Steepness of Flowing Paths 127 7.1. Summary of the chapter 127 7.2. Why do the catchment zones overlap? 128 7.2.1. Relation between the catchment zones and the flowing paths 128 7.2.2. Comparing the steepness of flowing paths 128 7.2.3. The redundancy between node and edge weights 129 7.2.4. General flow digraphs 130 7.3. The lexicographic pre-order relation of length k 131 7.3.1. Prolonging flowing paths into paths of infinite length 131 7.3.2. Comparing the steepness of two flowing paths 132 7.3.3. Properties of ∞ − steep paths 134 Chapter 8. Pruning a Flow Digraph 137 8.1. Summary of the chapter 137 8.1.1. Transforming a node- or edge-weighted graph into a node-weighted flowing digraph (reminder) 137 8.1.2. Global pruning 138 8.1.3. Local pruning 138 8.2. The pruning operator 138 8.2.1. Two operators on flow digraphs 139 8.2.2. Pruning by concatenating both operators 140 8.2.3. Properties of pruning 142 8.2.4. A variant of pruning 146 8.2.5. Local pruning 8.3. Evolution of catchment zones with pruning 147 8.3.1. Analyzing a digital elevation model 148 Chapter 9. Constructing an ∞ - steep Digraph by Flooding 155 9.1. Summary of the chapter 155 9.2. Characterization of ∞ − steep graphs 156 9.3. The core-expanding flooding algorithm 156 9.3.1. The first version of the core-expanding algorithm 157 9.3.2. The second version of the core-expanding algorithm 160 9.3.3. The third version of the core-expanding algorithm 164 9.3.4. The last version of the core-expanding algorithm, constructing a partial ∞ − steep flowing graph 167 Chapter 10. Creating Steep Watershed Partitions 169 10.1. Summary of the chapter 169 10.2. Creating watershed partitions with the core-expanding algorithm 169 10.2.1. Illustration of the HQ algorithm applied to node-weighted graphs 171 10.3. Propagating labels while pruning the digraph 172 10.3.1. Constructing a watershed partition during pruning 173 10.4. Pruning or flooding: two ways for catchment zones to grow 176 Chapter 11. An Historical Intermezzo 179 11.1. Watersheds: the early days 179 11.1.1. The level-by-level construction of watersheds 180 11.1.2. A hierarchical queue watershed algorithm 181 11.2. A watershed as the SKIZ for the topographic distance 181 11.2.1. The topographic distance 181 11.3. Convergence into a unique algorithm of three research streams 182 11.3.1. Three formulations of watershed partitions, one algorithm 182 11.3.2. Discussion 183 Part 4. Segmenting with Dead Leaves Partitions 185 Chapter 12. Intermezzo: Encoding the Digraph Associated with an Image 187 12.1. Summary of the theoretical developments seen so far 187 12.2. Summary of the chapter 188 12.3. Representing a node-weighted digraph as two images 188 12.3.1. The encoding of the digraph associated with an image 188 12.3.2. Operators acting on node-weighted digraphs 190 12.4. Defining labels 192 12.4.1. Operators on unweighted unlabeled digraphs 193 12.4.2. Operators on labeled unweighted digraphs 194 12.4.3. Operators on weighted and labeled digraphs 198 Chapter 13. Two Paradigms for Creating a Partition or a Partial Partition on a Graph 203 13.1. Summary of the chapter 203 13.2. Setting up a common stage for node- and edge-weighted graphs 203 13.3. A brief tool inventory 204 13.3.1. Operators making no use of the node weights 204 13.3.2. Operators propagating labels 204 13.3.3. Operators making use of the node weights and the graph structure 205 13.4. Dead leaves tessellations versus tilings: two paradigms 205 13.5. Extracting catchment zones containing a particular node 206 13.5.1. Core expansion versus pruning algorithms 206 13.5.2. Illustration of the pruning algorithm 207 13.6. Catchment zones versus catchment basins 209 Chapter 14. Dead Leaves Segmentation 211 14.1. Summary of the chapter 211 14.2. Segmenting with a watershed 211 14.2.1. Segmenting with watershed partitions 211 14.2.2. A crossroad of several methods 213 14.3. The evolution of a dead leaves tessellation with pruning 214 14.4. Local correction of overlapping zones 217 14.4.1. Pruning analysis 217 14.4.2. Local pruning for reducing overlapping zones 219 14.4.3. A local core-expanding algorithm for reducing overlapping zones 221 14.5. Local correction of the overlapping zones on a DEM 221 14.5.1. Local core-expanding algorithm for reducing overlapping zones 225 14.5.2. Advantage of the two-step construction of a dead leaves tessellation 227 14.6. Segmentation of some marked regions 231 14.6.1. Segmenting the domain and extracting the objects of interest 232 14.6.2. Extraction of the marked catchment zones and local correction of errors 233 Chapter 15. Propagating Segmentations 241 15.1. Summary of the chapter 241 15.2. Step-by-step segmentation 241 15.2.1. Principle of the method 241 15.2.2. Segmentation of blood cells 242 15.2.3. Segmentation of an electronic circuit 243 15.3. Marker-based segmentation 245 Appendix 247 References 259 Index 267

Read more less

Book SynopsisThis book uses MATLAB as a computing tool to explore traditional DSP topics and solve problems. This greatly expands the range and complexity of problems that students can effectively study in signal processing courses. A large number of worked examples, computer simulations and applications are provided, along with theoretical aspects that are essential in order to gain a good understanding of the main topics. Practicing engineers may also find it useful as an introductory text on the subject.Table of ContentsChapter 1. Introduction. Chapter 2. 1-D and 2-D discrete-time signals. Chapter 3. Discrete-time random processes. Chapter 4. Statistics for signal processing. Chapter 5. Convolution of 1-D and 2-D signals. Chapter 6. Discrete Fourier transform of 1-D and 2-D discrete signals. Chapter 7. Linear and invariant disctrete-time systems. Chapter 8. Infinite impulse response filters. Chapter 9. Finite impulse response filters. Chapter 10. Digital modulations. Chapter 11. Statistical signal processing: estimation. Chapter 12. Power spectrum density estimation. Chapter 13. Time-frequency spectral estimation. Chapter 14. Model-based time-frequency spectral estimation. Chapter 15. Statistical signal processing: classification. Chapter 16. Data compression. Chapter 17. Programming digital signal processors with MATLAB. List of Authors. Index.

Read more less

Book SynopsisThis book focuses on signal processing algorithms based on the timefrequency domain. Original methods and algorithms are presented which are able to extract information from non-stationary signals such as heart sounds and power electric signals. The methods proposed focus on the time-frequency domain, and most notably the Stockwell Transform for the feature extraction process and to identify signatures. For the classification method, the Adaline Neural Network is used and compared with other common classifiers. Theory enhancement, original applications and concrete implementation on FPGA for real-time processing are also covered in this book.Table of ContentsPreface ix Chapter 1. The Need for Time–Frequency Analysis 1 1.1. Introduction 1 1.2. Stationary and non-stationary concepts 2 1.2.1. Stationarity 2 1.2.2. Non-stationarity 4 1.3. Temporal representations 5 1.4. Frequency representations of signals 6 1.4.1. Fourier transform 7 1.4.2. Mean frequency, bandwidth and frequency average 10 1.5. Uncertainty principle 12 1.6. Limitation of time analysis and frequency analysis: the need for time–frequency representation 15 1.6.1. Instantaneous frequency 15 1.7. Conclusion 18 1.8. Bibliography 19 Chapter 2. Time–Frequency Analysis: The S-Transform 21 2.1. Introduction 21 2.2. Synthetic signals 22 2.3. The STFT 22 2.4. The WT 24 2.5. The Wigner–Ville distribution 25 2.5.1. The pseudo-WVD 27 2.5.2. The smoothed PWVD 27 2.6. Cohen’s class 28 2.7. The S-transform 29 2.7.1. Properties of the S-transform 30 2.7.2. The discrete S-transform 38 2.7.3. The improvement of the S-transform energy concentration 41 2.7.4. The ST-spectrogram 51 2.8. Conclusion 56 2.9. Bibliography 56 Chapter 3. Segmentation and Classification of Heart Sounds Based on the S-Transform 61 3.1. Introduction 61 3.2. Methods and materials 64 3.2.1. Data sets 64 3.2.2. Localization and segmentation of heart sounds 65 3.2.3. Classification of heart sounds 70 3.3. Results and discussion 73 3.3.1. Localization and segmentation results 73 3.3.2. S1 and S2 classification results 77 3.3.3. Murmur detection results 80 3.4. Conclusion 82 3.5. Bibliography 83 Chapter 4. Adaline for the Detection of Electrical Events in Electrical Signals 87 4.1. Introduction 87 4.2. Electric events 88 4.2.1. Power quality 88 4.2.2. Electric events 89 4.3. Adaline 90 4.4. Adaline for frequency estimation 91 4.4.1. Adaline method 91 4.4.2. Results 94 4.5. Adaline for voltage component extraction in unbalanced system 97 4.5.1. Model of the unbalanced voltage system 98 4.5.2. Extraction of the voltage components in the DQ-space 99 4.5.3. Online estimation of the instantaneous phases θd and θi 100 4.5.4. Filtering the AC components in the DQ-space 101 4.5.5. Results 104 4.6. Adaline for harmonic current identification and compensation 108 4.6.1. Adaline method 110 4.6.2. Results 115 4.7. Conclusion 117 4.8. Bibliography 118 Chapter 5. FPGA Implementation of the Adaline 121 5.1. Introduction 121 5.2. Instantaneous power theory (IPT) in the APF 122 5.3. Adaline for the computing of the IPT in the PLL 123 5.3.1. Adaline-based PLL 123 5.3.2. A multiplexing approach for hardware consumption reduction 126 5.4. Results 129 5.4.1. Simulation 129 5.4.2. FPGA implementation results 130 5.5. Conclusion 132 5.6. Bibliography 133 Index 135

Read more less

Book SynopsisThe focus of this book is on "ill-posed inverse problems". These problems cannot be solved only on the basis of observed data. The building of solutions involves the recognition of other pieces of a priori information. These solutions are then specific to the pieces of information taken into account. Clarifying and taking these pieces of information into account is necessary for grasping the domain of validity and the field of application for the solutions built. For too long, the interest in these problems has remained very limited in the signal-image community. However, the community has since recognized that these matters are more interesting and they have become the subject of much greater enthusiasm. From the application field’s point of view, a significant part of the book is devoted to conventional subjects in the field of inversion: biological and medical imaging, astronomy, non-destructive evaluation, processing of video sequences, target tracking, sensor networks and digital communications. The variety of chapters is also clear, when we examine the acquisition modalities at stake: conventional modalities, such as tomography and NMR, visible or infrared optical imaging, or more recent modalities such as atomic force imaging and polarized light imaging.Table of ContentsINTRODUCTION xiJean-Francois GIOVANNELLI and Jerome IDIER CHAPTER 1. 3D RECONSTRUCTION IN X-RAY TOMOGRAPHY: APPROACH EXAMPLE FOR CLINICAL DATA PROCESSING 1Yves GOUSSARD 1.1. Introduction 1 1.2. Problem statement 2 1.3. Method 7 1.4. Results 15 1.5. Conclusion 26 1.6. Acknowledgments 27 1.7. Bibliography 28 CHAPTER 2. ANALYSIS OF FORCE-VOLUME IMAGES IN ATOMIC FORCE MICROSCOPY USING SPARSE APPROXIMATION 31Charles SOUSSEN, David BRIE, Gregory FRANCIUS, Jerome IDIER 2.1. Introduction 31 2.2. Atomic force microscopy 32 2.3. Data processing in AFM spectroscopy 40 2.4. Sparse approximation algorithms 43 2.5. Real data processing 49 2.6. Conclusion 52 2.7. Bibliography 53 CHAPTER 3. POLARIMETRIC IMAGE RESTORATION BY NON-LOCAL MEANS 57Sylvain FAISAN, Francois ROUSSEAU, Christian HEINRICH, Jihad ZALLAT 3.1. Introduction 57 3.2. Light polarization and the Stokes–Mueller formalism 58 3.3. Estimation of the Stokes vectors 61 3.4. Results 72 3.5. Conclusion 77 3.6. Bibliography 78 CHAPTER 4. VIDEO PROCESSING AND REGULARIZED INVERSION METHODS 81Guy LE BESNERAIS, Frederic CHAMPAGNAT 4.1. Introduction 81 4.2. Three applications 82 4.3. Dense image registration 88 4.4. A few achievements based on direct formulation 92 4.5. Conclusion 104 4.6. Bibliography 106 CHAPTER 5. BAYESIAN APPROACH IN PERFORMANCE MODELING: APPLICATION TO SUPERRESOLUTION 109Frederic CHAMPAGNAT, Guy LE BESNERAIS, Caroline KULCSAR 5.1. Introduction 109 5.2. Performance modeling and Bayesian paradigm 111 5.3. Superresolution techniques behavior 113 5.4. Application examples 126 5.5. Real data processing 130 5.6. Conclusion 136 5.7. Bibliography 137 CHAPTER 6. LINE SPECTRA ESTIMATION FOR IRREGULARLY SAMPLED SIGNALS IN ASTROPHYSICS 141Sebastien BOURGUIGNON, Herve CARFANTAN 6.1. Introduction 141 6.2. Periodogram, irregular sampling, maximum likelihood 144 6.3. Line spectra models: spectral sparsity 146 6.4. Prewhitening, CLEAN and greedy approaches 151 6.5. Global approach and convex penalization 155 6.6. Probabilistic approach for sparsity 159 6.7. Conclusion 164 6.8. Bibliography 165 CHAPTER 7. JOINT DETECTION-ESTIMATION IN FUNCTIONAL MRI 169Philippe CIUCIU, Florence FORBES, Thomas VINCENT, Lotfi CHAARI 7.1. Introduction to functional neuroimaging 169 7.2. Joint detection-estimation of brain activity 171 7.3. Bayesian approach 178 7.4. Scheme for stochastic MCMC inference 183 7.5. Alternative variational inference scheme 184 7.6. Comparison of both types of solutions 190 7.7. Conclusion 194 7.8. Bibliography 195 CHAPTER 8. MCMC AND VARIATIONAL APPROACHES FOR BAYESIAN INVERSION IN DIFFRACTION IMAGING 201Hacheme AYASSO, Bernard DUCHENE, Ali MOHAMMAD-DJAFARI 8.1. Introduction 201 8.2. Measurement configuration 204 8.3. The forward model 206 8.4. Bayesian inversion approach 211 8.5. Results 220 8.6. Conclusions 220 8.7. Bibliography 222 CHAPTER 9. VARIATIONAL BAYESIAN APPROACH AND BI-MODEL FOR THE RECONSTRUCTION-SEPARATION OF ASTROPHYSICS COMPONENTS 225Thomas RODET, Aurelia FRAYSSE, Hacheme AYASSO 9.1. Introduction 225 9.2. Variational Bayesian methodology 228 9.3. Exponentiated gradient for variational Bayesian 229 9.4. Application: reconstruction-separation of astrophysical components 232 9.5. Implementation of the variational Bayesian approach 236 9.6. Results 240 9.7. Conclusion 246 9.8. Bibliography 246 CHAPTER 10. KERNEL VARIATIONAL APPROACH FOR TARGET TRACKING IN A WIRELESS SENSOR NETWORK 251Hichem SNOUSSI, Paul HONEINE, Cedric RICHARD 10.1. Introduction 251 10.2. State of the art: limitations of existing methods 252 10.3. Model-less target tracking 254 10.4. Simulation results 261 10.5. Conclusion 264 10.6. Bibliography 264 CHAPTER 11. ENTROPIES AND ENTROPIC CRITERIA 267Jean-Francois BERCHER 11.1. Introduction 267 11.2. Some entropies in information theory 268 11.3. Source coding with escort distributions and Renyi bounds 273 11.4. A simple transition model 277 11.5. Minimization of the Renyi divergence and associated entropies 281 11.6. Bibliography 289 LIST OF AUTHORS 293 INDEX 297

Read more less

Book SynopsisVolume 3 of the second edition of the fully revised and updated Digital Signal and Image Processing using MATLAB, after first two volumes on the "Fundamentals" and "Advances and Applications: The Deterministic Case", focuses on the stochastic case. It will be of particular benefit to readers who already possess a good knowledge of MATLAB, a command of the fundamental elements of digital signal processing and who are familiar with both the fundamentals of continuous-spectrum spectral analysis and who have a certain mathematical knowledge concerning Hilbert spaces. This volume is focused on applications, but it also provides a good presentation of the principles. A number of elements closer in nature to statistics than to signal processing itself are widely discussed. This choice comes from a current tendency of signal processing to use techniques from this field. More than 200 programs and functions are provided in the MATLAB language, with useful comments and guidance, to enable numerical experiments to be carried out, thus allowing readers to develop a deeper understanding of both the theoretical and practical aspects of this subject.Table of ContentsForeword ix Notations and Abbreviations xiii 1 Mathematical Concepts 1 1.1 Basic concepts on probability 1 1.2 Conditional expectation 9 1.3 Projection theorem 10 1.4 Gaussianity 13 1.5 Random variable transformation 18 1.6 Fundamental statistical theorems 21 1.7 Other important probability distributions 23 2 Statistical Inferences 25 2.1 Statistical model 25 2.2 Hypothesis tests 27 2.3 Statistical estimation 41 3 Monte-Carlo Simulation 85 3.1 Fundamental theorems 85 3.2 Stating the problem 86 3.3 Generating random variables 88 3.4 Variance reduction 99 4 Second Order Stationary Process 107 4.1 Statistics for empirical correlation 107 4.2 Linear prediction of WSS processes 111 4.3 Non-parametric spectral estimation of WSS processes 124 5 Inferences on HMM 139 5.1 Hidden Markov Models (HMM) 130 5.2 Inferences on HMM 142 5.3 Gaussian linear case: the Kalman filter 143 5.4 Discrete finite Markov case 152 6 Selected Topics 163 6.1 High resolution methods 163 6.2 Digital Communications 186 6.3 Linear equalization and the Viterbi algorithm 211 6.4 Compression 220 7 Hints and Solutions 235 H1 Mathematical concepts 235 H2 Statistical inferences 237 H3 Monte-Carlo simulation 269 H4 Second order stationary process 283 H5 Inferences on HMM 283 H6 Selected Topics 300 8 Appendices 317 A1 Miscellaneous functions 317 A2 Statistical functions 318 Bibliography 329 Index 333

Read more less

Book SynopsisThis book covers the fundamentals of digital signal processing (DSP) in a concise format, accessible to anyone with a technical background, enabling the reader for further DSP training, research, and development. The authors explore many subjects, including discrete time (digital) signals and systems, with emphasis on linear shift invariant (LSI) systems; Fourier and the z transforms; signal sampling and analog-to-digital (A/D) conversion. The book ends with examples of DSP techniques applications to practical problems from several areas.

Read more less

Book SynopsisThis textbook provides comprehensive coverage for courses in the basics of design and implementation of digital filters. The book assumes only basic knowledge in digital signal processing and covers state-of-the-art methods for digital filter design and provides a simple route for the readers to design their own filters. The advanced mathematics that is required for the filter design is minimized by providing an extensive MATLAB toolbox with over 300 files. The book presents over 200 design examples with MATLAB code and over 300 problems to be solved by the reader. The students can design and modify the code for their use. The book and the design examples cover almost all known design methods of frequency-selective digital filters as well as some of the authors’ own, unique techniques. Table of ContentsIntroduction.- Signals and Transforms.- Discrete-Time and Digital Filters.- Filter Algorithms.- Finite Wordlength Effects.- Synthesis of Fir Filters.- Realization of Fir Filters.- Synthesis of Analog Filters.- Analog Filters with Lumped and Distributed Elements.- Synthesis of IIR Filters.- Wave Digital Filters.- Ladder Wave Digital Filters.- Symmetric Wave Digital Filters.- Frequency-Response Masking Filters.- Sampling Rate Converters.- Multirate Filters.- Implementation of Digital Filters.- References.- Index.

Read more less

Book SynopsisThis comprehensive reference provides easy access to relevant information on all aspects of Computer Vision. An A-Z format of over 240 entries offers a diverse range of topics for those seeking entry into any aspect within the broad field of Computer Vision. Over 200 Authors from both industry and academia contributed to this volume.Each entry includes synonyms, a definition and discussion of the topic, and a robust bibliography. Extensive cross-references to other entries support efficient, user-friendly searches for immediate access to relevant information. Entries were peer-reviewed by a distinguished international advisory board, both scientifically and geographically diverse, ensuring balanced coverage. Over 3700 bibliographic references for further reading enable deeper exploration into any of the topics covered.The content of Computer Vision: A Reference Guide is expository and tutorial, making the book a practical resource for students who are considering entering the field, as well as professionals in other fields who need to access this vital information but may not have the time to work their way through an entire text on their topic of interest.Table of ContentsOver 240 entries organized A to Z.

Read more less

Book SynopsisThis book presents applications of cognitive management and cognitive computing in the fields of risk management, cognitive fraud detection, and in business decision making. The book provides insights on how cognitive management and cognitive computing enable businesses to quickly augment human intelligence and help humans perform tasks better. For example, the authors describe how by analyzing patterns in big data, small data, and "dark data," cognitive technologies can detect human behavior and suggest options for personalizing of products and services. The book studies companies in industries such as automotive, airline, health care, retail, wealth management, and litigation who have adopted these approaches. Presents applications of cognitive computing and cognitive management used in augmenting and empowering business decisions; Shows how to employ the Internet of Things in businesses using a cognitive management framework; Discusses technical aspects and alternatives to traditional tools, algorithms, and methodologies in cognitive computing. Table of ContentsIntroduction.- Cognitive Management.- Applications of Cognitive Computing for risk management.- Managing organizational mind by cognitive management.- Cognitive Managerial Approach towards Employee Participation in Management.- Role of Cognitive Computing in Business and Management.- The cognitive approach to Entrepreneurship.- Empowering the Internet of Things using Cognitive Management Framework.- Social Cognitive Theory in Multidisciplinary research.- Convergence of Cognitive Management and Cognitive Computing in the information society context.- Cognitive Computing and Knowledge Management.- Conclusion.

Read more less

Book SynopsisThis book is intended to be a little different from other books in its coverage. There are a great many digital signal processing (DSP) books and signals and systems books on the market. Since most undergraduate courses begin with signals and systems and then move on in later years to DSP, I felt a need to combine the two into one book that was concise yet not too overburdening. This means that students need only purchase one book instead of two and at the same time see the flow of knowledge from one subject into the next. Like the rudiments of music, it starts at the very beginning with some elementary knowledge and builds on it chapter by chapter to advanced work by chapter 15. I have been teaching now for 38 years and always think it necessary to credit the pioneers of the subjects we teach and ask the question “How did we get to this present stage in technological achievement”? Therefore, in Chapter 1 I have given a concise history trying to not sway too much away from the subject area. This is followed by the rudimentary theory in increasing complexity. It has already been taught successfully to a class at Auckland University of Technology New Zealand.Table of ContentsFrom the content: Introduction and basic signal properties.- Dynamic systems introduction.- Further introductory topics in Signals and Systems.- Frequency-domain properties of signals.- Sampling of signals and discrete mathematical methods.- Properties of discrete-time systems and signals.- A more complete picture.- FIR Filter design.

Read more less

Book SynopsisThis book celebrates binocular vision by presenting illustrations that require two eyes to see the effects of cooperation and competition between them. Pictures are flat but by printing them in different colours and viewing them through similarly coloured filters (included with the hardcover book) they are brought to life either in stereoscopic depth or in rivalry with one another. They are called anaglyphs and all those in the book display the ways in which the eyes interact. Thus, the reader is an integral element in the book and not all readers will see the same things. The history, science and art of binocular vision can be experienced in ways that are not usually available to us and with images made specifically for this book. The study of vision with two eyes was transformed by the invention of stereoscopes in the early 19th century. Anaglyphs are simple forms of stereoscopes that have three possible outcomes from viewing them – with each eye alone to see the monocular images, with both eyes to see them in stereoscopic depth or rivalry, or without the red/cyan glasses where they can have an appeal independent of the binocularity they encompass. Through the binocular pictures and the words that accompany them there will be an appreciation of just how remarkable the processes are that yield binocular singleness and depth. Moreover, the opportunities for expressing these processes are explored with many examples of truly binocular art. Table of ContentsSetting the seen.- A little history.- Binocular vision.- Stereoscopes.- Stereoscopic vision.- Binocular rivalry.- Binocular controversies.- Binocular art.- Conclusion.

Read more less

Book SynopsisThis textbook provides a compact but comprehensive treatment that provides analytical and design steps to recurrent neural networks from scratch. It provides a treatment of the general recurrent neural networks with principled methods for training that render the (generalized) backpropagation through time (BPTT). This author focuses on the basics and nuances of recurrent neural networks, providing technical and principled treatment of the subject, with a view toward using coding and deep learning computational frameworks, e.g., Python and Tensorflow-Keras. Recurrent neural networks are treated holistically from simple to gated architectures, adopting the technical machinery of adaptive non-convex optimization with dynamic constraints to leverage its systematic power in organizing the learning and training processes. This permits the flow of concepts and techniques that provide grounded support for design and training choices. The author’s approach enables strategic co-training of output layers, using supervised learning, and hidden layers, using unsupervised learning, to generate more efficient internal representations and accuracy performance. As a result, readers will be enabled to create designs tailoring proficient procedures for recurrent neural networks in their targeted applications.Table of ContentsIntroduction1. Network Architectures2. Learning Processes3. Recurrent Neural Networks (RNN)4. Gated RNN: The Long Short-Term Memory (LSTM) RNN5. Gated RNN: The Gated Recurrent Unit (GRU) RNN6. Gated RNN: The Minimal Gated Unit (MGU) RNN

Read more less

Book SynopsisThis textbook is a concise yet precise supplement to traditional books on Signals and Systems, focusing exclusively on the continuous-time case. Students can use this guide to review material, reinforce their understanding, and see how all the parts connect together in a uniform treatment focused on mathematical clarity. Readers learn the “what”, “why” and “how” about the ubiquitous Fourier and Laplace transforms encountered in the study of linear time-invariant systems in engineering: what are these transforms, why do we need them, and how do we use them? Readers will come away with an understanding of the gradual progression from time-domain analysis to frequency-domain and s-domain techniques for continuous-time linear time-invariant systems. This book reflects the author’s experience in teaching this material for over 25 years in sophomore- and junior-level required engineering courses and is ideal for undergraduate classes in electrical engineering.Table of Contents1. Introduction2. Systems3. Periodic Signals and Fourier Series4. Analysis of Stable Systems using the Fourier Transform5. Sampling and Reconstruction6. Analysis and Control of Systems using the Laplace Transform

Read more less

Book SynopsisThis SpringerBrief presents novel methods of approaching challenging problems in the reconstruction of accurate 3D models and serves as an introduction for further 3D reconstruction methods. It develops a 3D reconstruction system that produces accurate results by cascading multiple novel loop detection, sifting, and optimization methods.The authors offer a fast point cloud registration method that utilizes optimized randomness in random sample consensus for surface loop detection. The text also proposes two methods for surface-loop sifting. One is supported by a sparse-feature-based optimization graph. This graph is more robust to different scan patterns than earlier methods and can cope with tracking failure and recovery. The other is an offline algorithm that can sift loop detections based on their impact on loop optimization results and which is enabled by a dense map posterior metric for 3D reconstruction and mapping performance evaluation works without any costly ground-truth data. The methods presented in Towards Optimal Point Cloud Processing for 3D Reconstruction will be of assistance to researchers developing 3D modelling methods and to workers in the wide variety of fields that exploit such technology including metrology, geological animation and mass customization in smart manufacturing.Table of Contents1. Introduction.- 2. Preliminaries.- 3. Fractional-Order Random Sample Consensus.- 4. Online Sifting of Loop Detections for 3D Reconstruction of Caves.- 5. Dense Map Posterior: A Novel Quality Metric for 3D Reconstruction.- 6. Offline Sifting and Majorization of Loop Detections.- 7. Conclusion and Future Opportunities.- Appendix: More Information on Results Reproducibility.

Read more less

Book SynopsisThis book provides a comprehensive introduction to all major topics in digital signal processing (DSP). The book is designed to serve as a textbook for courses offered to undergraduate students enrolled in electrical, electronics, and communication engineering disciplines. The text is augmented with many illustrative examples for easy understanding of the topics covered. Every chapter contains several numerical problems with answers followed by question-and-answer type assignments. The detailed coverage and pedagogical tools make this an ideal textbook for students and researchers enrolled in electrical engineering and related programs. Table of ContentsChapter 1. Representation of Discrete Signals and Systems.- Chapter 2. Discrete and Fast Fourier Transforms (DFT and FFT).- Chapter 3. Design of IIR Digital Filters.- Chapter 4. Finite Impulse Response (FIR) Filter Design.- Chapter 5. Finite Word Length Effects.Chapter 6. Multi-rate Digital Signal Processing.

Read more less

Book SynopsisThis book provides a comprehensive guide to 3D Light-Field camera based imaging, exploring the working principles, developments and its applications in fluid mechanics and aerodynamics measurements. It begins by discussing the fundamentals of Light-Field imaging and theoretical resolution analysis, before touching upon the detailed optics design and micro-lens array assembly. Subsequently, Light-Field calibration methods that compensate for optical distortions and establish the relations between the image and real-word 3D coordinates are covered. This is followed by Light-Field 3D reconstruction algorithms which are elaborated for micrometer-scale particles and centimeter-scale physical models. Last but not least, implementations of the preceding procedures to selected fundamental and applied flow measurement scenarios are provided at the end of the book. Development and Application of Light-Field Cameras in Fluid Measurements gives an in-depth analysis of each topic discussed, making it ideal as both an introductory and reference guide for researchers and postgraduates interested in 3D flow measurements.Table of ContentsIntroduction.- Light-field camera working principles.- Volumetric calibration for light-field camera with regular and scheimpflug lens.- Light-field particle image velocimetry.- Simultaneous 3D surface geometry and pressure distribution measurement.- Light-field PIV implementation and case studies.- Future developments of Light-field based measurements.

Read more less

Book SynopsisThis book discusses the Versatile Video Coding (VVC), the ISO and ITU state-of-the-art video coding standard. VVC reaches a compression efficiency significantly higher than its predecessor standard (HEVC) and it has a high versatility for efficient use in a broad range of applications and different types of video content, including Ultra-High Definition (UHD), High-Dynamic Range (HDR), screen content, 360º videos, and resolution adaptivity. The authors introduce the novel VVC tools for block partitioning, intra-frame and inter-frames predictions, transforms, quantization, entropy coding, and in-loop filtering. The authors also present some solutions exploring VVC encoding behavior at different levels to accelerate the intra-frame prediction, applying statistical-based heuristics and machine learning (ML) techniques.Table of ContentsIntroduction.- Versatile Video Coding.- VVC Intra-Frame Prediction.- State-of-the-Art Overview.- Performance Analysis of VVC Intra-Frame Prediction.- Heuristic Based Fast Multi-Type Tree Decision Scheme for Luminance.- Light Gradient Boosting Machine Configurable Fast Block Partitioning for Luminance.- Learning-Based Fast Decision for Intra-Frame Prediction Mode Selection for Luminance.- Fast Intra-Frame Prediction Transform for Luminance Using Decision Trees.- Heuristic Based Fast Block Partitioning Scheme for Chrominance.- Conclusions.

Read more less

Book SynopsisThis book constitutes the thoroughly refereed conference proceedings of the First International Workshop on Design and Architecture for Signal and Image Processing, DASIP 2022, held in Budaypest, Hungary in June 2022. The 13 full included in the volume were carefully reviewed and selected from 32 submissions. They are organized in the following topical sections: leading signal, image and video processing and machine learning in custom embedded, edge and cloud computing architectures and systems.Table of ContentsSoftware and Architecture for Telecommunication Systems.- Towards Lightweight Deep-Learning Techniques.- Design Automation and Optimization Techniques for Embedded Hardware and Software.- Optimized Hardware and Software Implementations for Image Processing and Health Applications.

Read more less

Book SynopsisThe textbook presents basic concepts of signals and systems in a clear manner, based on the author’s 15+ years of teaching the undergraduate course for engineering students. To attain full benefit from the content, readers should have a strong knowledge of calculus and be familiar with integration, differentiation, and summation operations. The book starts with an introduction to signals and systems and continues with coverage of basic signal functions and their manipulations; energy, power, convolution, and systems; Fourier analysis of continuous time signals and digital signals; Laplace transform; and Z transforms. Practical applications are included throughout. The book is also packed with solved examples, self-study exercises, and end of chapter problems.Table of Contents1) Introduction to Signals and Systems2) BASIC SIGNAL FUNCTIONS and THEIR MANIPULATIONS3) ENERGY, POWER, CONVOLUTION, and SYSTEMS4) FOURIER ANALYSIS of DIGITAL SIGNALS5) FOURIER ANALYSIS of DIGITAL SIGNALS6) LAPLACE TRANSFORM7) Z Transform8) PRACTICAL APPLICATIONS

Read more less

Book SynopsisThis book presents the design, analysis, and application of nonlinear adaptive filters with the goal of improving efficient performance (ie the convergence speed, steady-state error, and computational complexity). The authors present a nonlinear adaptive filter, which is an important part of nonlinear system and digital signal processing and can be applied to diverse fields such as communications, control power system, radar sonar, etc. The authors also present an efficient nonlinear filter model and robust adaptive filtering algorithm based on the local cost function of optimal criterion to overcome non-Gaussian noise interference. The authors show how these achievements provide new theories and methods for robust adaptive filtering of nonlinear and non-Gaussian systems. The book is written for the scientist and engineer who are not necessarily an expert in the specific nonlinear filtering field but who want to learn about the current research and application. The book is also written to accompany a graduate/PhD course in the area of nonlinear system and adaptive signal processing.Table of Contents1) ​Adaptive filter2) Volterra adaptive filter3) FLANN adaptive filter4) Spline adaptive filter5) Kernel adaptive filters

Read more less

Book SynopsisThis book covers the theory of multidimensional signals and systems and related practical aspects. It extends the properties and mathematical tools of one-dimensional signals and systems to multiple dimensions and covers relevant timeless topics including multidimensional transformations, multidimensional sampling as well as discrete multidimensional systems. A special emphasis is placed on physical systems described by partial differential equations, the construction of suitable integral transformations and the implementation of the corresponding discrete-time algorithms. To this end, signal spaces and functional transformations are introduced at a mathematical level provided by undergraduate programs in engineering and science.The presentation takes a comprehensive, illustrative and educational approach without reference to a particular application field. Instead, the book builds a solid theoretical concept of multidimensional signals and systems and shows the application to various problems relevant for practical scenarios.Table of Contents

Read more less

Book SynopsisThis book constitutes the thoroughly refereed conference proceedings of the 16th International Workshop on Design and Architecture for Signal and Image Processing, DASIP 2023, held in Toulouse, France in January 2023.The 9 full included in the volume were carefully reviewed and selected from 17 submissions. They are organized in the following topical sections: Methods and Applications, Hardware Architectures and Implementations and others. Table of ContentsMethods and Applications.- SCAPE: HW-Aware Clustering of Data ow Actors for Tunable Scheduling Complexity.-Deep Recurrent Neural Network performing spectral recurrence on hyperspectral images for brain tissue classi cation.- Brain blood vessel segmentation in hyperspectral images through linear operators.- Neural Network Predictor for Fast Channel Change on DVB Set-Top-Boxes.- Hardware Architectures and Implementations.- AINoC: new interconnect for future Deep Neural Network accelerators.- Real-time FPGA implementation of the Semi-Global Matching stereo vision algorithm for an 4K/UHD video stream.- TaPaFuzz - An FPGA-Accelerated Framework for RISC-V IoT Graybox Fuzzing Adaptive Inference for FPGA-based 5G Automatic Modulation.- Classfication.- High-Level Online Power Monitoring of FPGA IP Based on Machine Learning.

Read more less

Book SynopsisThis book investigates the image watermarking domain, analyzing and comparing image watermarking techniques that exist in current literature. The author’s goal is to aid researchers and students in their studies in the vast and important domain of image watermarking, including its advantages and risks. The book has three chapters: image watermarking using data compression; speech modulation for image watermarking; and secure image watermarking based on LWT and SVD.In addition, this book: Investigates the image watermarking domain, analyzing and comparing current image watermarking techniques Includes detail on image encryption and mathematical tools used for image watermarking Covers image watermarking using data compression, speech modulation for image watermarking, and more Table of ContentsIntroduction.- Image watermarking using data compression.- Speech modulation for image watermarking.- Secure Image Watermarking Based on LWT and SVD.- Speech Signal Embedding into Digital Images Using Encryption and Watermarking Techniques.- Conclusion.

Read more less

Book Synopsis

Read more less

Book Synopsis

Read more less

Book SynopsisFundamentals of Inertial Navigation, Satellite-based Positioning and their Integration is an introduction to the field of Integrated Navigation Systems. It serves as an excellent reference for working engineers as well as textbook for beginners and students new to the area. The book is easy to read and understand with minimum background knowledge. The authors explain the derivations in great detail. The intermediate steps are thoroughly explained so that a beginner can easily follow the material. The book shows a step-by-step implementation of navigation algorithms and provides all the necessary details. It provides detailed illustrations for an easy comprehension. The book also demonstrates real field experiments and in-vehicle road test results with professional discussions and analysis. This work is unique in discussing the different INS/GPS integration schemes in an easy to understand and straightforward way. Those schemes include loosely vs tightly coupled, open loop vs closed loop, and many more. Table of ContentsReference Frames and Earth Geometry.- Global Positioning System.- Inertial Navigation System.- Inertial Navigation System Modeling.- Modeling INS Errors by Linear State Equations.- Kalman Filter.- INS/GPS integration.- Three Dimensional Reduced Inertial Sensor System / GPS Integration for Land-Based Vehicles.- Two Case Studies- full IMU/GPS and 3D RISS/GPS Integration.

Read more less

Book SynopsisField-Programmable Gate Arrays (FPGAs) are revolutionizing digital signal processing. The efficient implementation of front-end digital signal processing algorithms is the main goal of this book. It starts with an overview of today's FPGA technology, devices and tools for designing state-of-the-art DSP systems. A case study in the first chapter is the basis for more than 40 design examples throughout. The following chapters deal with computer arithmetic concepts, theory and the implementation of FIR and IIR filters, multirate digital signal processing systems, DFT and FFT algorithms, advanced algorithms with high future potential, and adaptive filters. Each chapter contains exercises. The VERILOG source code and a glossary are given in the appendices. This new edition incorporates Over 10 new system level case studies designed in VHDL and Verilog A new chapter on image and video processing An Altera Quartus update and new Model Sim simulations Xilinx Atlys board and ISIM simulation support Signed fixed point and floating point IEEE library examples An overview on parallel all-pass IIR filter design ICA and PCA system level designs Speech and audio coding for MP3 and ADPCM Table of ContentsComputer Arithmetic.- Finite Impulse Response (FIR) Digital Filtres.- Infinite Impulse Response (IIR) Digital Filtres.- Multirate Signal Processing.- Fourier Transforms.- Advanced Topics.- Adaptive Filtres.- Microprocessor Design.

Read more less

Book SynopsisIn der hochbitratigen optischen Nachrichtentechnik ist es wichtig, parasitäre induktive und kapazitive Einflüsse auf die Funktion von Laser- und Fotodioden zu kompensieren. Wegen des nichtlinearen Charakters der u-i-Relationen der Induktivitäten, Kapazitäten und Widerstände ist es möglich, Kompensationsverfahren gegen parasitäre Effekte zu entwickeln oder die Nichtlinearitäten gezielt zur Signalübertragung einzusetzen. Reiner Thiele beweist, dass bei Applikation der vorgestellten Kompensationsverfahren kapazitive und induktive Influenzen auf die Grundfunktion der optoelektronischen Bauelemente vermeidbar sind, das Klemmenverhalten durch die u-i-Kennlinien von Laser- oder Fotodioden komplett erfasst wird und ungünstige Einflüsse der Systemumgebung auf die optoelektronischen Schaltungen vermieden werden. Außerdem stellt er Definitionen für optoelektronische Grundstromkreise sowie ihre Berechnung für die Applikation gleichartiger Laser- oder Fotodioden als Sende- bzw. Empfangsbauelemente der optischen Nachrichtentechnik vor.Der Autor: Prof. Dr.-Ing. Reiner Thiele lehrte an der Hochschule Zittau/Görlitz und unterrichtet derzeit an der Staatlichen Studienakademie Bautzen.Table of ContentsParameter von Dioden.- Kompensation elektromagnetischer Beeinflussungen.- Optoelektronische Grundstromkreise.

Read more less

Book SynopsisThis book presents the peer-reviewed proceedings of the Sixth International Conference on Intelligent Computing and Applications (ICICA 2020), held at Government College of Engineering, Keonjhar, Odisha, India, during December 22–24, 2020. The book includes the latest research on advanced computational methodologies such as neural networks, fuzzy systems, evolutionary algorithms, hybrid intelligent systems, uncertain reasoning techniques, and other machine learning methods and their applications to decision-making and problem-solving in mobile and wireless communication networks.Table of ContentsClosed Loop Vision Based Ball Balancer.- A Novel Network Learning For Image Compressive Sensing.- COVID – 19 Severıty Predıctıons: An Analysis Usıng Correlatıon Measures.- A Novel Methodology for Comparative Analysis of Power Quality Improvement for a 3 Phase DC/AC Embedded DC/DC Converter.- Performance of Photovoltaic based ZETA Converter Water Pumping Application.- Performance Analysis of Radial Distribution System by Optimal Deployment of DG and DSTATCOM Considering Network Reconfiguration using a SAR Algorithm.

Read more less

Book SynopsisThis book provides insights into the Third International Conference on Intelligent Systems and Signal Processing (eISSP 2020) held By Electronics & Communication Engineering Department of G H Patel College of Engineering & Technology, Gujarat, India, during 28–30 December 2020. The book comprises contributions by the research scholars and academicians covering the topics in signal processing and communication engineering, applied electronics and emerging technologies, Internet of Things (IoT), robotics, machine learning, deep learning and artificial intelligence. The main emphasis of the book is on dissemination of information, experience and research results on the current topics of interest through in-depth discussions and contribution of researchers from all over world. The book is useful for research community, academicians, industrialists and postgraduate students across the globe.Table of ContentsChapter 1: Design and Analysis of Modified Split Ring Resonator Structured Multiband Antenna for WCDMA and WiMAX Applications.- Chapter 2: A Wearable Finger Exoskeleton For Motor Rehabilitation Using Mobile Application.- Chapter 3: Game theoretical approach for cluster-based routing protocol in Wireless Sensor Networks.- Chapter 4: Advance Digital Signal Processing for Interference Mitigation in Very High Throughput Satellite.- Chapter 5: Low-Power Endoscopic Image Compression Algorithms Using Modified Golomb Codes.- Chapter 6: Image Steganography Using Ridgelet Transform and SVD.- Chapter 7: HWCMA and HW-LS-CMA blind learning method for intelligent antenna system.- Chapter 8: Performance evaluation of prediction algorithm based tracking methods in a recovery of a lost target using Wireless Sensor Network.- Chapter 9: An Efficient Convolutional Neural Network for Acute Pain Recognition using HRV Features.- Chapter 10: Design and development of LSTM–RNN model for the prediction of RR intervals in ECG signals.- Chapter 11: FHSS Signals Classification by Linear Discriminant in a Multi-Signal Environment.- Chapter 12: Non-invasive Thyroid detection using thermal Imaging technique.- Chapter 13: Non Orthogonal Multiple Access Techniques for Next Generation Wireless Networks: A Review.- Chapter 14: Triple band circular patch antenna using complimentary split ring resonators.- Chapter 15: Features Analysis of Electroencephalography (EEG) for Mindfulness Meditation Effect on Cancer Patient toward Stress Level.

Read more less

Book SynopsisThis book presents select proceedings of the International Conference on Futuristic Communication and Network Technologies (CFCNT 2020) conducted at Vellore Institute of Technology, Chennai. It covers various domains in communication engineering and networking technologies. This volume comprises of recent research in areas like optical communication, optical networks, optics and optical computing, emerging trends in photonics, MEMS and sensors, active and passive RF components and devices, antenna systems and applications, RF devices and antennas for microwave emerging technologies, wireless communication for future networks, signal and image processing, machine learning/AI for networks, internet of intelligent things, network security and blockchain technologies. This book will be useful for researchers, professionals, and engineers working in the core areas of electronics and communication. Table of ContentsDeep Learning Based Image Preprocessing Techniques for Crop Disease Identification.- Performance Evaluation and Comparison Analysis of AODV and RPL using NetSim in Low Power, Lossy Networks.- Three Notched Bands Modified Hexagonal Patch Monopole Antenna.- An Internet of Things (IoT) based approach for Realtime Kitchen Monitoring using NodeMCU 1.0.- IoT Based Smart Irrigation and Monitoring System in Smart Agriculture.- Stock Price Prediction Based on Deep Learning Using Long Short Term Memory.

Read more less