fixed firsts comments from frankd
This commit is contained in:
@@ -4,11 +4,11 @@ In many real world scenarios one is then interested in finding a \qq{best estima
|
||||
This is often done by means of simple parametric point estimator, providing the sample statistics.
|
||||
However, in complex scenarios this frequently results in a poor representation, due to multimodal densities and limited sample sizes.
|
||||
|
||||
Recovering the probability density function using a kernel density estimation yields a promising approach to find the \qq{real} most probable state, but comes with high computational costs.
|
||||
Recovering the probability density function using a kernel density estimation yields a promising approach to solve the state estimation problem i.e. finding the \qq{real} most probable state, but comes with high computational costs.
|
||||
Especially in time critical and time sequential scenarios, this turns out to be impractical.
|
||||
Therefore, this work uses techniques from digital signal processing in the context of estimation theory, to allow rapid computations of kernel density estimates.
|
||||
The gains in computational efficiency are realized by substituting the Gaussian filter with an approximate filter based on the box filter.
|
||||
Our approach outperforms other state of the art solutions, due to a fully linear complexity \landau{N} and a negligible overhead, even for small sample sets.
|
||||
Our approach outperforms other state of the art solutions, due to a fully linear complexity and a negligible overhead, even for small sample sets.
|
||||
Finally, our findings are tried and tested within a real world sensor fusion system.
|
||||
\end{abstract}
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
Within this paper a novel approach for rapid computation of the KDE was presented.
|
||||
This is achieved by considering the discrete convolution structure of the BKDE and thus elaborate its connection to digital signal processing, especially the Gaussian filter.
|
||||
Using a box filter as an appropriate approximation results in an efficient computation scheme with a fully linear complexity \landau{N} and a negligible overhead, as confirmed by the utilized experiments.
|
||||
Using a box filter as an appropriate approximation results in an efficient computation scheme with a fully linear complexity and a negligible overhead, as confirmed by the utilized experiments.
|
||||
|
||||
The analysis of the error showed that the method exhibits an expected error behaviour compared to the BKDE.
|
||||
In terms of calculation time, our approach outperforms other state of the art implementations.
|
||||
|
||||
@@ -13,19 +13,19 @@
|
||||
%In contrast,
|
||||
The KDE is often the preferred tool to estimate a density function from discrete data samples because of its ability to produce a continuous estimate and its flexibility.
|
||||
%
|
||||
Given a univariate random sample $X=\{X_1, \dots, X_n\}$, where $X$ has the density function $f$ and let $w_1, \dots w_n$ be associated weights.
|
||||
Given a univariate random sample $X=\{X_1, \dots, X_N\}$, where $X$ has the density function $f$ and let $w_1, \dots w_N$ be associated weights.
|
||||
The kernel estimator $\hat{f}$ which estimates $f$ at the point $x$ is given as
|
||||
\begin{equation}
|
||||
\label{eq:kde}
|
||||
\hat{f}(x) = \frac{1}{W} \sum_{i=1}^{n} \frac{w_i}{h} K \left(\frac{x-X_i}{h}\right)
|
||||
\hat{f}(x) = \frac{1}{W} \sum_{i=1}^{N} \frac{w_i}{h} K \left(\frac{x-X_i}{h}\right)
|
||||
\end{equation}
|
||||
where $W=\sum_{i=1}^{n}w_i$ and $h\in\R^+$ is an arbitrary smoothing parameter called bandwidth.
|
||||
$K$ is a kernel function such that $\int K(u) \dop{u} = 1$ \cite[138]{scott2015}.
|
||||
where $W=\sum_{i=1}^{N}w_i$ and $h\in\R^+$ is an arbitrary smoothing parameter called bandwidth.
|
||||
$K$ is a kernel function such that $\int K(u) \dop{u} = 1$.
|
||||
In general any kernel can be used, however the general advice is to chose a symmetric and low-order polynomial kernel.
|
||||
Thus, several popular kernel functions are used in practice, like the Uniform, Gaussian, Epanechnikov, or Silverman kernel \cite[152.]{scott2015}.
|
||||
Thus, several popular kernel functions are used in practice, like the Uniform, Gaussian, Epanechnikov, or Silverman kernel \cite{scott2015}.
|
||||
|
||||
While the kernel estimate inherits all the properties of the kernel, usually it is not of crucial matter if a non-optimal kernel was chosen \cite[151f.]{scott2015}.
|
||||
As a matter of fact, the quality of the kernel estimate is primarily determined by the smoothing parameter $h$ \cite[145]{scott2015}.
|
||||
While the kernel estimate inherits all the properties of the kernel, usually it is not of crucial matter if a non-optimal kernel was chosen.
|
||||
As a matter of fact, the quality of the kernel estimate is primarily determined by the smoothing parameter $h$ \cite{scott2015}.
|
||||
%In theory it is possible to calculate an optimal bandwidth $h^*$ regarding to the asymptotic mean integrated squared error.
|
||||
%However, in order to do so the density function to be estimated needs to be known which is obviously unknown in practice.
|
||||
%
|
||||
@@ -56,12 +56,12 @@ In general, reducing the size of the sample negatively affects the accuracy of t
|
||||
Still, the sample size is a suitable parameter to speedup the computation.
|
||||
|
||||
Since each single sample is combined with its adjacent samples into bins, the BKDE approximates the KDE.
|
||||
Each bin represents the \qq{count} of the sample set at a given point of a equidistant grid with spacing $\delta$.
|
||||
Each bin represents the count of the sample set at a given point of a equidistant grid with spacing $\delta$.
|
||||
A binning rule distributes a sample $x$ among the grid points $g_j=j\delta$, indexed by $j\in\Z$.
|
||||
% and can be represented as a set of functions $\{ w_j(x,\delta), j\in\Z \}$.
|
||||
Computation requires a finite grid on the interval $[a,b]$ containing the data, thus the number of grid points is $G=(b-a)/\delta+1$.
|
||||
|
||||
Given a binning rule $b_j$ the BKDE $\tilde{f}$ of a density $f$ computed pointwise at the grid point $g_x$ is given as
|
||||
Given a binning rule $r_j$ the BKDE $\tilde{f}$ of a density $f$ computed pointwise at the grid point $g_x$ is given as
|
||||
\begin{equation}
|
||||
\label{eq:binKde}
|
||||
\tilde{f}(g_x) = \frac{1}{W} \sum_{j=1}^{G} \frac{C_j}{h} K \left(\frac{g_x-g_j}{h}\right)
|
||||
@@ -69,7 +69,7 @@ Given a binning rule $b_j$ the BKDE $\tilde{f}$ of a density $f$ computed pointw
|
||||
where $G$ is the number of grid points and
|
||||
\begin{equation}
|
||||
\label{eq:gridCnts}
|
||||
C_j=\sum_{i=1}^{n} b_j(x_i,\delta)
|
||||
C_j=\sum_{i=1}^{n} r_j(x_i,\delta)
|
||||
\end{equation}
|
||||
is the count at grid point $g_j$, such that $\sum_{j=1}^{G} C_j = W$ \cite{hall1996accuracy}.
|
||||
|
||||
@@ -77,7 +77,7 @@ In theory, any function which determines the count at grid points is a valid bin
|
||||
However, for many applications it is recommend to use the simple binning rule
|
||||
\begin{align}
|
||||
\label{eq:simpleBinning}
|
||||
b_j(x,\delta) &=
|
||||
r_j(x,\delta) &=
|
||||
\begin{cases}
|
||||
w_j & \text{if } x \in ((j-\frac{1}{2})\delta, (j-\frac{1}{2})\delta ] \\
|
||||
0 & \text{else}
|
||||
@@ -86,7 +86,7 @@ However, for many applications it is recommend to use the simple binning rule
|
||||
or the common linear binning rule which divides the sample into two fractional weights shared by the nearest grid points
|
||||
\begin{align}
|
||||
\label{eq:linearBinning}
|
||||
b_j(x,\delta) &=
|
||||
r_j(x,\delta) &=
|
||||
\begin{cases}
|
||||
w_j(1-|\delta^{-1}x-j|) & \text{if } |\delta^{-1}x-j|\le1 \\
|
||||
0 & \text{else.}
|
||||
|
||||
@@ -18,7 +18,7 @@ While in both equations the constant factor of the Gaussian is removed of the in
|
||||
This factor is necessary to ensure that the estimate is a valid density function, i.e. that it integrates to one.
|
||||
Such a restriction is superfluous in the context of digital filters, so the normalization factor is omitted.
|
||||
|
||||
Computation of a digital filter using the a naive implementation of the discrete convolution algorithm yields $\landau{NM}$, where $N$ is the length of the input signal and $M$ is the size of the filter kernel.
|
||||
Computation of a digital filter using the a naive implementation of the discrete convolution algorithm yields $\landau{NM}$, where $N$ is again the input size given by the length of the input signal and $M$ is the size of the filter kernel.
|
||||
In order to capture all significant values of the Gaussian function the kernel size $M$ must be adopted to the standard deviation of the Gaussian.
|
||||
A faster $\landau{N}$ algorithm is given by the well-known approximation scheme based on iterated box filters.
|
||||
While reducing the algorithmic complexity this approximation also reduces computational time significantly due to the simplistic computation scheme of the box filter.
|
||||
@@ -28,7 +28,7 @@ Following that, a implementation of the iterated box filter is one of the fastes
|
||||
|
||||
\subsection{Box Filter}
|
||||
The box filter is a simplistic filter defined as convolution of the input signal and the box (or rectangular) function.
|
||||
A discrete box filter of \qq{radius} $l\in N_0$ is given as
|
||||
A discrete box filter of radius $l\in \N_0$ is given as
|
||||
\begin{equation}
|
||||
\label{eq:boxFilt}
|
||||
(B_L*x)(i) = \sum_{k=0}^{L-1} x(k) B_L(i-k)
|
||||
@@ -48,7 +48,7 @@ Such a filter clearly requires $\landau{NL}$ operations, where $N$ is the input
|
||||
It is well-known that a box filter can approximate a Gaussian filter by repetitive recursive computations.
|
||||
Given by the central limit theorem of probability, repetitive convolution of a rectangular function with itself eventually yields a Gaussian in the limit.
|
||||
Likewise, filtering a signal with the box filter several times approximately converges to a Gaussian filter.
|
||||
In practice three or five iterations are most likely enough to obtain a reasonable close Gaussian approximation \cite{kovesi2010fast}.
|
||||
In practice three to five iterations are most likely enough to obtain a reasonable close Gaussian approximation \cite{kovesi2010fast}.
|
||||
|
||||
This allows rapid approximation of the Gaussian filter using box filter, which only requires a few addition and multiplication operations.
|
||||
Opposed to the Gaussian filter where several evaluations of the exponential function are necessary.
|
||||
@@ -83,7 +83,7 @@ Therefore, in order to approximate the Gaussian filter of a given $\sigma$ a cor
|
||||
Given $n$ iterations of box filters with identical sizes the ideal size $\Lideal$, as suggested by Wells~\cite{wells1986efficient}, is
|
||||
\begin{equation}
|
||||
\label{eq:boxidealwidth}
|
||||
\Lideal = \sqrt{\frac{12\sigma^2}{n}+1} \text{.}
|
||||
\Lideal = \sqrt{\frac{12\sigma^2}{n}+1} \quad \text{.}
|
||||
\end{equation}
|
||||
|
||||
In general $\Lideal$ can be any real number, but $B_L$ in \eqref{eq:boxFx} is restricted to odd integer values.
|
||||
@@ -97,7 +97,7 @@ In order to reduce the rounding error Kovesi~\cite{kovesi2010fast} proposes to p
|
||||
\floor{\Lideal} - 1 & \text{if } \floor{\Lideal} \text{ is odd} \\
|
||||
\floor{\Lideal} & \text{else }
|
||||
\end{cases} \\
|
||||
L_2 &= L_1 + 2 \text{.}
|
||||
L_2 &= L_1 + 2 \quad \text{.}
|
||||
\end{split}
|
||||
\end{align}
|
||||
|
||||
@@ -105,7 +105,7 @@ Given $L_1$ and $L_2$ the approximation is done by computing $m$ box filters of
|
||||
As all other values are known $m$ can be computed with
|
||||
\begin{equation}
|
||||
\label{eq:boxrepeatm}
|
||||
m=\frac{12\sigma^2-nL_1^2-4nL_1-3n}{-4L_1-4} \text{.}
|
||||
m=\frac{12\sigma^2-nL_1^2-4nL_1-3n}{-4L_1-4} \quad \text{.}
|
||||
\end{equation}
|
||||
|
||||
The approximated $\sigma$ as a function of the integer width has a staircase shaped graph.
|
||||
|
||||
@@ -29,7 +29,7 @@ These algorithms reduce the number of evaluated kernels by taking the the spatia
|
||||
|
||||
% fast multipole method & Fast Gaus Transform
|
||||
Another approach is to reduce the algorithmic complexity of the sum over Gaussian functions, by employing a specialized variant of the fast multipole method.
|
||||
The term fast Gauss transform was coined by Greengard \cite{greengard1991fast} who suggested this approach to reduce the complexity of the KDE to \label{N+M}.
|
||||
The term fast Gauss transform was coined by Greengard \cite{greengard1991fast} who suggested this approach to reduce the complexity of the KDE to \landau{N+M}.
|
||||
% However, the complexity grows exponentially with dimension. \cite{Improved Fast Gauss Transform and Efficient Kernel Density Estimation}
|
||||
|
||||
% FastKDE, passed on ECF and nuFFT
|
||||
|
||||
Reference in New Issue
Block a user