FHWS/Fusion2018
Archived
4
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Going thru changes

Browse Source
This commit is contained in:
Markus Bullmann
2018-03-13 15:58:41 +01:00
parent 9f098887db
commit 7c407f950e
7 changed files with 42 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
tex/bare_conf.tex
View File

@@ -121,7 +121,7 @@
\newcommand{\qq} [1]{``#1''}
\newcommand{\eg} {e.\,g.}
\newcommand{\ie} {i.\,e.}
\newcommand{\figref}[1]{Fig.~\ref{#1}}
\newcommand{\figref}[1]{fig.~\ref{#1}}
% missing math operators
\DeclareMathOperator*{\argmin}{arg\,min}
\DeclareMathOperator*{\argmax}{arg\,max}

4
tex/chapters/conclusion.tex
View File

@@ -2,13 +2,13 @@
Within this paper a novel approach for rapid approximation of the KDE was presented.
This is achieved by considering the discrete convolution structure of the BKDE and thus elaborating its connection to digital signal processing, especially the Gaussian filter.
Using a box filter as an appropriate approximation results in an efficient computation scheme with a fully linear complexity and a negligible overhead, as demonstrated by the utilized experiments.
Using a box filter as an appropriate approximation results in an efficient computation scheme with a fully linear complexity and a negligible overhead, as demonstrated by the experiments.
The analysis of the error showed that the method shows an similar error behaviour compared to the BKDE.
In terms of calculation time, our approach outperforms other state of the art implementations.
Despite being more efficient than other methods, the algorithmic complexity still increases in its exponent with an increasing number of dimensions.
%future work kurz
Finally, such a fast computation scheme makes the KDE more attractive for real time use cases.
Finally, such a fast approximation scheme makes the KDE more attractive for real time use cases.
In a sensor fusion context, the availability of a reconstructed density of the posterior enables many new approaches and techniques for finding a best estimate of the system's current state.

48
tex/chapters/experiments.tex
View File

@@ -1,29 +1,32 @@
\section{Experiments}
\subsection{Mean Integrated Squared Error}
We now empirically evaluate the feasibility of our BoxKDE method by analyzing its approximation error.
In order to evaluate the error the KDE and various approximations of it are computed and compared using the mean integrated squared error (MISE).
A synthetic sample set $\bm{X}$ with $N=1000$ obtained from a bivariate mixture normal density $f$ provides the basis of the comparison.
For each method an estimate is computed and the MISE of it relative to $f$ is calculated.
The specific structure of the underlying distribution clearly affects the error in the estimate, but only the closeness of the approximation to the KDE is of interest.
We empirically evaluate the feasibility of our BoxKDE method by analyzing its approximation error.
In order to evaluate the deviation of the estimate from the original density, the mean integrated squared error (MISE) is used.
Those errors are compared for the KDE and its various approximations.
To match the requirements of our application, a synthetic sample set $\mathcal{X}$ with $N=5000$ drawn from a bivariate mixture normal density $f$ given by \eqref{eq:normDist} provides the basis of the comparison.
For each method the estimate is computed and the MISE relative to $f$ is calculated.
The specific structure of the underlying distribution clearly affects the error in the estimate, but only the closeness of our approximation to the KDE is of interest.
Hence, $f$ is of minor importance here and was chosen rather arbitrary to highlight the behavior of the BoxKDE.
\begin{equation}
\label{eq:normDist}
\begin{split}
\bm{X} \sim & ~\G{\VecTwo{0}{0}}{0.5\bm{I}} + \G{\VecTwo{3}{0}}{\bm{I}} + \G{\VecTwo{0}{3}}{\bm{I}} \\
&+ \G{\VecTwo{-3}{0} }{\bm{I}} + \G{\VecTwo{0}{-3}}{\bm{I}} \text{,}
&+ \G{\VecTwo{-3}{0} }{\bm{I}} + \G{\VecTwo{0}{-3}}{\bm{I}}
\end{split}
\end{equation}
where the majority of the probability mass lies in the range $[-6; 6]^2$.
%where the majority of the probability mass lies in the range $[-6; 6]^2$.
\begin{figure}[t]
\input{gfx/error.tex}
\caption{MISE relative to the ground truth as a function of $h$. While the error curves of the BKDE (red) and the BoxKDE based on the extended box filter (orange dotted line) resemble the overall course of the error of the exact KDE (green), the regular BoxKDE (orange) exhibits noticeable jumps due to rounding.} \label{fig:errorBandwidth}
\caption{MISE relative to the ground truth as a function of $h$. While the error curves of the BKDE (red) and the BoxKDE based on the extended box filter (orange dotted line) resemble the overall course of the error of the KDE (green), the regular BoxKDE (orange) exhibits noticeable jumps due to rounding.} \label{fig:errorBandwidth}
\end{figure}
Four estimates are computed with varying bandwidth using the exact KDE, BKDE, BoxKDE, and ExBoxKDE, which uses the extended box filter.
Four estimates are computed with varying bandwidth using the KDE, BKDE, BoxKDE, and ExBoxKDE, which uses the extended box filter.
All estimates are calculated at $30\times 30$ equally spaced points.
%Evaluated at $50^2$ points the exact KDE is compared to the BKDE, BoxKDE, and extended box filter approximation, which are evaluated at a smaller grid with $30^2$ points.
The graphs of the MISE between $f$ and the estimates as a function of $h\in[0.15; 1.0]$ are given in \figref{fig:errorBandwidth}.
The graphs of the MISE between $f$ and the estimates as a function of $h\in[0.15, 1.0]$ are given in \figref{fig:errorBandwidth}.
A minimum error is obtained with $h=0.35$, for larger values oversmoothing occurs and the modes gradually fuse together.
Both the BKDE and the ExBoxKDE resemble the error curve of the KDE quite well and stable.
@@ -31,7 +34,7 @@ They are rather close to each other, with a tendency to diverge for larger $h$.
In contrast, the error curve of the BoxKDE has noticeable jumps at $h=\{0.25, 0.40, 0.67, 0.82\}$.
These jumps are caused by the rounding of the integer-valued box width given by \eqref{eq:boxidealwidth}.
As the extended box filter is able to approximate an exact $\sigma$, such discontinuities don't appear.
As the extended box filter is able to approximate an exact $\sigma$, such discontinuities do not appear.
Consequently, it reduces the overall error of the approximation, even though only marginal in this scenario.
The global average MISE over all values of $h$ is $0.0049$ for the regular box filter and $0.0047$ in case of the extended version.
Likewise, the maximum MISE is $0.0093$ and $0.0091$, respectively.
@@ -44,7 +47,7 @@ However, both cases do not give a deeper insight of the error behavior of our me
\begin{figure}[t]
%\includegraphics[width=\textwidth,height=6cm]{gfx/tmpPerformance.png}
\input{gfx/perf.tex}
\caption{Logarithmic plot of the runtime performance with increasing grid size $G$ and bivariate data. The weighted-average estimate (blue) performs fastest followed by the BoxKDE (orange) approximation. Both the BKDE (red) and the FastKDE (green) are magnitudes slower, especially for $G<10^3$.}\label{fig:performance}
\caption{Logarithmic plot of the runtime performance with increasing grid size $G$ and bivariate data. The weighted-average estimate (blue) performs fastest followed by the BoxKDE (orange) approximation, which is magnitudes slower, especially for $G<10^3$.}\label{fig:performance}
\end{figure}
% kde, box filter, exbox in abhänigkeit von h (bild)
@@ -54,19 +57,18 @@ However, both cases do not give a deeper insight of the error behavior of our me
\subsection{Performance}
In the following, we underpin the promising theoretical linear time complexity of our method with empirical time measurements compared to other methods.
All tests are performed on an Intel Core \mbox{i5-7600K} CPU with a frequency of \SI{4.2}{\giga\hertz}, and \SI{16}{\giga\byte} main memory.
We compare our C++ implementation of the BoxKDE approximation as shown in algorithm~\ref{alg:boxKDE} to the \texttt{ks} R package and the FastKDE Python implementation \cite{oBrien2016fast}.
The \texttt{ks} package provides a FFT-based BKDE implementation based on optimized C functions at its core.
With state estimation problems in mind, we additionally provide a C++ implementation of a weighted-average estimator.
As both methods are not using a grid, an equivalent input sample set was used for the weighted-average and the FastKDE.
All tests are performed on an Intel Core \mbox{i5-7600K} CPU @ \SI{4.2}{\giga\hertz}, and \SI{16}{\giga\byte} main memory. %, supporting the AVX2 instruction set
We compare our C++ implementation of the BoxKDE approximation as shown in algorithm~\ref{alg:boxKDE} to the R language \texttt{ks} package, which provides a FFT-based BKDE implementation based on optimized C functions at its core.
With state estimation problems in mind, we additionally provide a C++ implementation of a weighted average estimator.
An equivalently sized input sample set was used for the weighted average, as its runtime depends on the sample size and not the grid size.
The results of the performance comparison are presented in \figref{fig:performance}.
% O(N) gut erkennbar für box KDE und weighted average
The linear complexity of the BoxKDE and the weighted average is clearly visible.
% Gerade bei kleinen G bis 10^3 ist die box KDE schneller als R und FastKDE, aber das WA deutlich schneller als alle anderen
Especially for small $G$ up to $10^3$ the BoxKDE is much faster compared to BKDE and FastKDE.
Especially for small $G$ up to $10000$ grid points the BoxKDE is much faster compared to BKDE.
% Bei zunehmend größeren G wird der Abstand zwischen box KDE und WA größer.
Nevertheless, the simple weighted-average approach performs the fastest, and with increasing $G$ the distance to the BoxKDE grows constantly.
Nevertheless, the simple weighted average approach performs the fastest.
However, it is obvious that this comes with major disadvantages, like being prone to multimodalities, as discussed in section \ref{sec:intro}.
% (Das kann auch daran liegen, weil das Binning mit größeren G langsamer wird, was ich mir aber nicht erklären kann! Vlt Cache Effekte)
@@ -78,7 +80,7 @@ This behavior is caused by the underlying FFT algorithm.
% Daher wird die Laufzeit sprunghaft langsamer wenn auf eine neue power of two aufgefüllt wird, ansonsten bleibt sie konstant.
The FFT approach requires the input to be always rounded up to a power of two, what then causes a constant runtime behaviour within those boundaries and a strong performance deterioration at corresponding manifolds.
% Der Abbruch bei G=4406^2 liegt daran, weil für größere Gs eine out of memory error ausgelöst wird.
The termination of BKDE graph at $G=4406^2$ is caused by an out of memory error for even bigger $G$ in the \texttt{ks} package.
The termination of BKDE graph at $G\approx 1.9 \cdot 10^7$ is caused by an out of memory error in the \texttt{ks} package for bigger $G$.
% Der Plot für den normalen Box Filter wurde aus Gründen der Übersichtlichkeit weggelassen.
% Sowohl der box filter als auch der extended box filter haben ein sehr ähnliches Laufzeit Verhalten und somit einen sehr ähnlichen Kurvenverlauf.
@@ -87,14 +89,12 @@ Both discussed Gaussian filter approximations, namely box filter and extended bo
While the average runtime over all values of $G$ for the standard box filter is \SI{0.4092}{\second}, the extended one has an average of \SI{0.4169}{\second}.
To disambiguate \figref{fig:performance}, we only illustrated the results of the BoxKDE with the regular box filter.
The weighted-average has the great advantage of being independent of the dimensionality of the input and can be implemented effortlessly.
The weighted average has the great advantage of being independent of the dimensionality of the input and can be implemented effortlessly.
In contrast, the computation of the BoxKDE approach increases exponentially with increasing number of dimensions.
However, due to the linear time complexity and the very simple computation scheme, the overall computation time is still sufficiently fast for many applications and much smaller compared to other methods.
The BoxKDE approach presents a reasonable alternative to the weighted-average and is easily integrated into existing systems.
In addition, modern CPUs do benefit from the recursive computation scheme of the box filter, as the data exhibits a high degree of spatial locality in memory and the accesses are reliably predictable.
Furthermore, the computation is easily parallelized, as there is no data dependency between the one-dimensional filter passes in algorithm~\ref{alg:boxKDE}.
Hence, the inner loops can be parallelized using threads or SIMD instructions, but the overall speedup depends on the particular architecture and the size of the input.
\input{chapters/realworld}

4
tex/chapters/multivariate.tex
View File

@@ -9,8 +9,8 @@ In order to estimate a multivariate density using KDE or BKDE, a multivariate ke
Multivariate kernel functions can be constructed in various ways, however, a popular way is given by the product kernel.
Such a kernel is constructed by combining several univariate kernels into a product, where each kernel is applied in each dimension with a possibly different bandwidth.
Given a multivariate random variable $\bm{X}=(x_1,\dots ,x_d)$ in $d$ dimensions.
The sample set $\mathcal{X}$ is a $n\times d$ matrix \cite{scott2015}.
Given a multivariate random variable $\bm{X}=(x_1,\dots ,x_d)^T$ in $d$ dimensions.
The sample set $\mathcal{X}=(x_{i,j})=(\bm{X}_1, \dots, \bm{X}_n)$ is a $n\times d$ matrix \cite{scott2015}.
The multivariate KDE $\hat{f}$ which defines the estimate pointwise at $\bm{u}=(u_1, \dots, u_d)^T$ is given as
\begin{equation}
\label{eq:mvKDE}

13
tex/chapters/realworld.tex
View File

@@ -1,17 +1,17 @@
\subsection{Real World}
To demonstrate the real time capabilities of the proposed method a real world scenario was chosen, namely indoor localization.
To demonstrate the capabilities of the proposed method a real world scenario was chosen, namely indoor localization.
The given problem is to localize a pedestrian walking inside a building.
Ebner et al. proposed a method, which incorporates multiple sensors, \eg{} Wi-Fi, barometer, step-detection and turn-detection \cite{Ebner-15}.
At a given time $t$ the system estimates a state providing the most probable position of the pedestrian.
It is implemented using a particle filter with sample importance resampling and \SI{5000} particles.
The dynamics are modelled realistically, which constrains the movement according to walls, doors and stairs.
We arranged a \SI{223}{\meter} long walk within the first floor of a \SI{2500}{m$^2$} museum, which was built in the 13th century and therefore offers non-optimal conditions for localization.
We arranged a \SI{223}{\meter} long walk within the first floor of a \mbox{\SI{76}{} $\times$ \SI{71}{\meter}} sized museum, which was built in the 13th century and therefore offers non-optimal conditions for localization.
%The measurements for the walks were recorded using a Motorola Nexus 6 at 2.4 GHz band only.
%
Since this work only focuses on processing a given sample set, further details of the localisation system and the described scenario can be looked up in \cite{Ebner17} and \cite{Fetzer17}.
The spacing $\delta$ of the grid was set to \SI{20}{\centimeter} for $x$ and $y$-direction.
The spacing $\delta$ of the grid was set to \SI{27}{\centimeter} for $x$ and $y$-direction, resulting in a grid size of $G=74019$.
The bivariate state estimation was calculated whenever a step was recognized, about every \SI{500}{\milli \second}.
%The intention of a real world experiment is to investigate the advantages and disadvantages of the here proposed method for finding a best estimate of the pedestrian's position in the wild, compared to conventional used methods like the weighted-average or choosing the maximum weighted particle.
@@ -21,7 +21,7 @@ The bivariate state estimation was calculated whenever a step was recognized, ab
\label{fig:realWorldMulti}
\end{figure}
%
\figref{fig:realWorldMulti} illustrates a frequently occurring situation, where the particle set splits apart, due to uncertain measurements and multiple possible walking directions.
Fig.~\ref{fig:realWorldMulti} illustrates a frequently occurring situation, where the particle set splits apart, due to uncertain measurements and multiple possible walking directions.
This results in a bimodal posterior distribution, which reaches its maximum distances between the modes at \SI{13.4}{\second} (black dotted line).
Thus estimating the most probable state over time using the weighted-average results in the blue line, describing the pedestrian's position to be somewhere outside the building (light green area).
In contrast, the here proposed method (orange line) is able to retrieve a good estimate compared to the ground truth path shown by the black solid line.
@@ -37,15 +37,14 @@ The error over time for different estimation methods of the complete walk can be
It is given by calculating the distance between estimation and ground truth at a specific time $t$.
Estimates provided by simply choosing the maximum particle stand out the most.
As expected beforehand, this method provides many strong peaks through continuously jumping between single particles.
Additionally, in most real world scenarios many particles share the same weight and thus multiple highest-weighted particles exist.
\begin{figure}
\input{gfx/errorOverTime.tex}
\caption{Error development over time calculated between estimation and ground truth. Between \SI{230}{\second} and \SI{290}{\second} to pedestrian was not moving.}
\caption{Error development over time of a single Monte Carlo run of the walk calculated between estimation and ground truth. Between \SI{230}{\second} and \SI{290}{\second} to pedestrian was not moving.}
\label{fig:realWorldTime}
\end{figure}
Further investigating \figref{fig:realWorldTime}, the BoxKDE performs slightly better than the weighted-average.
Further investigating \figref{fig:realWorldTime}, the BoxKDE performs slightly better than the weighted-average in this specific Monte Carlo run.
However after deploying \SI{100} Monte Carlo runs, the difference becomes insignificant.
The main reason for this are again multimodalities caused by faulty or delayed measurements, especially when entering or leaving rooms.
Within our experiments the problem occurred due to slow and attenuated Wi-Fi signals inside thick-walled rooms.

3
tex/chapters/usage.tex
View File

@@ -9,7 +9,7 @@ Consider a set of two-dimensional samples with associated weights, \eg{} generat
The overall process for bivariate data is described in Algorithm~\ref{alg:boxKDE}.
Assuming that the given $N$ samples are stored in a sequential list, the first step is to create a grid representation.
In order to efficiently construct the grid and to allocate the required memory, the extrema of the samples need to be known in advance.
In order to efficiently construct the grid and to allocate the required memory, the extrema of the samples in each dimension need to be known in advance.
These limits might be given by the application.
For example, the position of a pedestrian within a building is limited by the physical dimensions of the building.
Such knowledge should be integrated into the system to avoid a linear search over the sample set, naturally reducing the computation time.
@@ -74,4 +74,5 @@ Depending on the required accuracy, the extended box filter algorithm can furthe
Due to its simple indexing scheme, the recursive box filter can easily be computed in parallel using SIMD operations and parallel computation cores.
Finally, the most likely state can be obtained from the filtered data, \ie{} from the estimated discrete density, by searching filtered data for its maximum value.
This last step can be integrated into the last filter operation, by recording the largest output value.

10
tex/egbib.bib
View File

@@ -1722,7 +1722,7 @@ doi={10.1109/PLANS.2008.4570051},}
title={{Multi Sensor 3D Indoor Localisation}},
year={2015},
publisher = {IEEE},
address = {Banff, Canada},
address = {},
IGNOREmonth={October},
pages={},
}
@@ -2869,7 +2869,7 @@ year = {2003}
year = {2016},
publisher = {IEEE},
pages = {},
address = {Madrid, Spain},
address = {},
issn = {}
}
@@ -3060,7 +3060,7 @@ year = {2003}
author = {Scott, David W.},
year = {2015},
title = {Multivariate Density Estimation: Theory, Practice, and Visualization},
address = {Hoboken, NJ},
address = {},
edition = {2},
publisher = {Wiley},
isbn = {978-0-471-69755-8},
@@ -3081,13 +3081,13 @@ DOI = {10.3390/ijgi6080233}
@INPROCEEDINGS{Fetzer17,
author={T. Fetzer and F. Ebner and F. Deinzer and M. Grzegorzek},
booktitle={2017 Int. Conf. on Indoor Positioning and Indoor Navigation (IPIN)},
booktitle={Int. Conf. on Indoor Positioning and Indoor Navigation (IPIN)},
title={Recovering from sample impoverishment in context of indoor localisation},
year={2017},
pages={1-8},
doi={10.1109/IPIN.2017.8115863},
ISSN={},
month={Sept},}
month={},}