diff --git a/tex/chapters/abstract.tex b/tex/chapters/abstract.tex index db7bf86..5c5c6fe 100644 --- a/tex/chapters/abstract.tex +++ b/tex/chapters/abstract.tex @@ -4,11 +4,11 @@ In many real world scenarios one is then interested in finding a \qq{best estima This is often done by means of simple parametric point estimator, providing the sample statistics. However, in complex scenarios this frequently results in a poor representation, due to multimodal densities and limited sample sizes. -Recovering the probability density function using a kernel density estimation yields a promising approach to find the \qq{real} most probable state, but comes with high computational costs. +Recovering the probability density function using a kernel density estimation yields a promising approach to solve the state estimation problem i.e. finding the \qq{real} most probable state, but comes with high computational costs. Especially in time critical and time sequential scenarios, this turns out to be impractical. Therefore, this work uses techniques from digital signal processing in the context of estimation theory, to allow rapid computations of kernel density estimates. The gains in computational efficiency are realized by substituting the Gaussian filter with an approximate filter based on the box filter. -Our approach outperforms other state of the art solutions, due to a fully linear complexity \landau{N} and a negligible overhead, even for small sample sets. +Our approach outperforms other state of the art solutions, due to a fully linear complexity and a negligible overhead, even for small sample sets. Finally, our findings are tried and tested within a real world sensor fusion system. \end{abstract} diff --git a/tex/chapters/conclusion.tex b/tex/chapters/conclusion.tex index cc1a6cc..e80aaf7 100644 --- a/tex/chapters/conclusion.tex +++ b/tex/chapters/conclusion.tex @@ -2,7 +2,7 @@ Within this paper a novel approach for rapid computation of the KDE was presented. This is achieved by considering the discrete convolution structure of the BKDE and thus elaborate its connection to digital signal processing, especially the Gaussian filter. -Using a box filter as an appropriate approximation results in an efficient computation scheme with a fully linear complexity \landau{N} and a negligible overhead, as confirmed by the utilized experiments. +Using a box filter as an appropriate approximation results in an efficient computation scheme with a fully linear complexity and a negligible overhead, as confirmed by the utilized experiments. The analysis of the error showed that the method exhibits an expected error behaviour compared to the BKDE. In terms of calculation time, our approach outperforms other state of the art implementations. diff --git a/tex/chapters/kde.tex b/tex/chapters/kde.tex index b26e5a8..4eba3fd 100644 --- a/tex/chapters/kde.tex +++ b/tex/chapters/kde.tex @@ -13,19 +13,19 @@ %In contrast, The KDE is often the preferred tool to estimate a density function from discrete data samples because of its ability to produce a continuous estimate and its flexibility. % -Given a univariate random sample $X=\{X_1, \dots, X_n\}$, where $X$ has the density function $f$ and let $w_1, \dots w_n$ be associated weights. +Given a univariate random sample $X=\{X_1, \dots, X_N\}$, where $X$ has the density function $f$ and let $w_1, \dots w_N$ be associated weights. The kernel estimator $\hat{f}$ which estimates $f$ at the point $x$ is given as \begin{equation} \label{eq:kde} -\hat{f}(x) = \frac{1}{W} \sum_{i=1}^{n} \frac{w_i}{h} K \left(\frac{x-X_i}{h}\right) +\hat{f}(x) = \frac{1}{W} \sum_{i=1}^{N} \frac{w_i}{h} K \left(\frac{x-X_i}{h}\right) \end{equation} -where $W=\sum_{i=1}^{n}w_i$ and $h\in\R^+$ is an arbitrary smoothing parameter called bandwidth. -$K$ is a kernel function such that $\int K(u) \dop{u} = 1$ \cite[138]{scott2015}. +where $W=\sum_{i=1}^{N}w_i$ and $h\in\R^+$ is an arbitrary smoothing parameter called bandwidth. +$K$ is a kernel function such that $\int K(u) \dop{u} = 1$. In general any kernel can be used, however the general advice is to chose a symmetric and low-order polynomial kernel. -Thus, several popular kernel functions are used in practice, like the Uniform, Gaussian, Epanechnikov, or Silverman kernel \cite[152.]{scott2015}. +Thus, several popular kernel functions are used in practice, like the Uniform, Gaussian, Epanechnikov, or Silverman kernel \cite{scott2015}. -While the kernel estimate inherits all the properties of the kernel, usually it is not of crucial matter if a non-optimal kernel was chosen \cite[151f.]{scott2015}. -As a matter of fact, the quality of the kernel estimate is primarily determined by the smoothing parameter $h$ \cite[145]{scott2015}. +While the kernel estimate inherits all the properties of the kernel, usually it is not of crucial matter if a non-optimal kernel was chosen. +As a matter of fact, the quality of the kernel estimate is primarily determined by the smoothing parameter $h$ \cite{scott2015}. %In theory it is possible to calculate an optimal bandwidth $h^*$ regarding to the asymptotic mean integrated squared error. %However, in order to do so the density function to be estimated needs to be known which is obviously unknown in practice. % @@ -56,12 +56,12 @@ In general, reducing the size of the sample negatively affects the accuracy of t Still, the sample size is a suitable parameter to speedup the computation. Since each single sample is combined with its adjacent samples into bins, the BKDE approximates the KDE. -Each bin represents the \qq{count} of the sample set at a given point of a equidistant grid with spacing $\delta$. +Each bin represents the count of the sample set at a given point of a equidistant grid with spacing $\delta$. A binning rule distributes a sample $x$ among the grid points $g_j=j\delta$, indexed by $j\in\Z$. % and can be represented as a set of functions $\{ w_j(x,\delta), j\in\Z \}$. Computation requires a finite grid on the interval $[a,b]$ containing the data, thus the number of grid points is $G=(b-a)/\delta+1$. -Given a binning rule $b_j$ the BKDE $\tilde{f}$ of a density $f$ computed pointwise at the grid point $g_x$ is given as +Given a binning rule $r_j$ the BKDE $\tilde{f}$ of a density $f$ computed pointwise at the grid point $g_x$ is given as \begin{equation} \label{eq:binKde} \tilde{f}(g_x) = \frac{1}{W} \sum_{j=1}^{G} \frac{C_j}{h} K \left(\frac{g_x-g_j}{h}\right) @@ -69,7 +69,7 @@ Given a binning rule $b_j$ the BKDE $\tilde{f}$ of a density $f$ computed pointw where $G$ is the number of grid points and \begin{equation} \label{eq:gridCnts} - C_j=\sum_{i=1}^{n} b_j(x_i,\delta) + C_j=\sum_{i=1}^{n} r_j(x_i,\delta) \end{equation} is the count at grid point $g_j$, such that $\sum_{j=1}^{G} C_j = W$ \cite{hall1996accuracy}. @@ -77,7 +77,7 @@ In theory, any function which determines the count at grid points is a valid bin However, for many applications it is recommend to use the simple binning rule \begin{align} \label{eq:simpleBinning} - b_j(x,\delta) &= + r_j(x,\delta) &= \begin{cases} w_j & \text{if } x \in ((j-\frac{1}{2})\delta, (j-\frac{1}{2})\delta ] \\ 0 & \text{else} @@ -86,7 +86,7 @@ However, for many applications it is recommend to use the simple binning rule or the common linear binning rule which divides the sample into two fractional weights shared by the nearest grid points \begin{align} \label{eq:linearBinning} - b_j(x,\delta) &= + r_j(x,\delta) &= \begin{cases} w_j(1-|\delta^{-1}x-j|) & \text{if } |\delta^{-1}x-j|\le1 \\ 0 & \text{else.} diff --git a/tex/chapters/mvg.tex b/tex/chapters/mvg.tex index 6432904..bf55196 100644 --- a/tex/chapters/mvg.tex +++ b/tex/chapters/mvg.tex @@ -18,7 +18,7 @@ While in both equations the constant factor of the Gaussian is removed of the in This factor is necessary to ensure that the estimate is a valid density function, i.e. that it integrates to one. Such a restriction is superfluous in the context of digital filters, so the normalization factor is omitted. -Computation of a digital filter using the a naive implementation of the discrete convolution algorithm yields $\landau{NM}$, where $N$ is the length of the input signal and $M$ is the size of the filter kernel. +Computation of a digital filter using the a naive implementation of the discrete convolution algorithm yields $\landau{NM}$, where $N$ is again the input size given by the length of the input signal and $M$ is the size of the filter kernel. In order to capture all significant values of the Gaussian function the kernel size $M$ must be adopted to the standard deviation of the Gaussian. A faster $\landau{N}$ algorithm is given by the well-known approximation scheme based on iterated box filters. While reducing the algorithmic complexity this approximation also reduces computational time significantly due to the simplistic computation scheme of the box filter. @@ -28,7 +28,7 @@ Following that, a implementation of the iterated box filter is one of the fastes \subsection{Box Filter} The box filter is a simplistic filter defined as convolution of the input signal and the box (or rectangular) function. -A discrete box filter of \qq{radius} $l\in N_0$ is given as +A discrete box filter of radius $l\in \N_0$ is given as \begin{equation} \label{eq:boxFilt} (B_L*x)(i) = \sum_{k=0}^{L-1} x(k) B_L(i-k) @@ -48,7 +48,7 @@ Such a filter clearly requires $\landau{NL}$ operations, where $N$ is the input It is well-known that a box filter can approximate a Gaussian filter by repetitive recursive computations. Given by the central limit theorem of probability, repetitive convolution of a rectangular function with itself eventually yields a Gaussian in the limit. Likewise, filtering a signal with the box filter several times approximately converges to a Gaussian filter. -In practice three or five iterations are most likely enough to obtain a reasonable close Gaussian approximation \cite{kovesi2010fast}. +In practice three to five iterations are most likely enough to obtain a reasonable close Gaussian approximation \cite{kovesi2010fast}. This allows rapid approximation of the Gaussian filter using box filter, which only requires a few addition and multiplication operations. Opposed to the Gaussian filter where several evaluations of the exponential function are necessary. @@ -83,7 +83,7 @@ Therefore, in order to approximate the Gaussian filter of a given $\sigma$ a cor Given $n$ iterations of box filters with identical sizes the ideal size $\Lideal$, as suggested by Wells~\cite{wells1986efficient}, is \begin{equation} \label{eq:boxidealwidth} - \Lideal = \sqrt{\frac{12\sigma^2}{n}+1} \text{.} + \Lideal = \sqrt{\frac{12\sigma^2}{n}+1} \quad \text{.} \end{equation} In general $\Lideal$ can be any real number, but $B_L$ in \eqref{eq:boxFx} is restricted to odd integer values. @@ -97,7 +97,7 @@ In order to reduce the rounding error Kovesi~\cite{kovesi2010fast} proposes to p \floor{\Lideal} - 1 & \text{if } \floor{\Lideal} \text{ is odd} \\ \floor{\Lideal} & \text{else } \end{cases} \\ - L_2 &= L_1 + 2 \text{.} + L_2 &= L_1 + 2 \quad \text{.} \end{split} \end{align} @@ -105,7 +105,7 @@ Given $L_1$ and $L_2$ the approximation is done by computing $m$ box filters of As all other values are known $m$ can be computed with \begin{equation} \label{eq:boxrepeatm} - m=\frac{12\sigma^2-nL_1^2-4nL_1-3n}{-4L_1-4} \text{.} + m=\frac{12\sigma^2-nL_1^2-4nL_1-3n}{-4L_1-4} \quad \text{.} \end{equation} The approximated $\sigma$ as a function of the integer width has a staircase shaped graph. diff --git a/tex/chapters/relatedwork.tex b/tex/chapters/relatedwork.tex index 6b6e9d1..96d175f 100644 --- a/tex/chapters/relatedwork.tex +++ b/tex/chapters/relatedwork.tex @@ -29,7 +29,7 @@ These algorithms reduce the number of evaluated kernels by taking the the spatia % fast multipole method & Fast Gaus Transform Another approach is to reduce the algorithmic complexity of the sum over Gaussian functions, by employing a specialized variant of the fast multipole method. -The term fast Gauss transform was coined by Greengard \cite{greengard1991fast} who suggested this approach to reduce the complexity of the KDE to \label{N+M}. +The term fast Gauss transform was coined by Greengard \cite{greengard1991fast} who suggested this approach to reduce the complexity of the KDE to \landau{N+M}. % However, the complexity grows exponentially with dimension. \cite{Improved Fast Gauss Transform and Efficient Kernel Density Estimation} % FastKDE, passed on ECF and nuFFT