FHWS/Fusion2018
Archived
4
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Captions & performance bla bla

Browse Source
This commit is contained in:
MBulli
2018-02-26 18:49:23 +01:00
parent 2f8c241313
commit 6f9d191622
2 changed files with 22 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files

43
tex/chapters/experiments.tex
View File

@@ -3,7 +3,7 @@
\subsection{Mean Integrated Squared Error}
We now empirically evaluate the accuracy of our method, using the mean integrated squared error (MISE).
We now empirically evaluate the accuracy of our boxKDE method, using the mean integrated squared error (MISE).
The ground truth is given as $N=1000$ synthetic samples drawn from a bivariate mixture normal density $f$
\begin{equation}
\begin{split}
@@ -16,19 +16,19 @@ Clearly, the structure of the ground truth affects the error in the estimate, bu
Therefore, the particular choice of the ground truth is only of minor importance here.
\begin{figure}[t]
\label{fig:evalBandwidth}
%\includegraphics[width=\columnwidth]{gfx/Eval1Bandwidth_abs.png}
\input{gfx/error.tex}
\caption{Hier kommt bandwith error plot single. da sind voll die ergebnisse zu der genaugikeit und so für voll die verfahren und weighted avg auch noch.} \label{fig:eval1GroundTruth}
\caption{MISE relative to the ground truth as a function of $h$. While the error curves of the BKDE (red) and the boxKDE based on the extended box filter (orange dotted line) resemble the overall course of the error of the exact KDE (green), the regular boxKDE (orange) exhibits noticeable jumps to rounding.} \label{fig:errorBandwidth}
\end{figure}
The exact KDE, evaluated at $50^2$ points, is compared to the BKDE, box filter, and extended box filter approximation, which are evaluated at a smaller grid with $30^2$ points.
The MISE between $f$ and the estimates as a function of $h$ are evaluated, and the resulting plot is given in figure~\ref{fig:evalBandwidth}.
Evaluated at $50^2$ points the exact KDE is compared to the BKDE, boxKDE, and extended box filter approximation, which are evaluated at a smaller grid with $30^2$ points.
The MISE between $f$ and the estimates as a function of $h$ are evaluated, and the resulting plot is given in figure~\ref{fig:errorBandwidth}.
A minimum error is obtained with $h=0.35$, for larger oversmoothing occurs and the modes gradually fuse together.
Both the BKDE and the extended box filter estimate resemble the error curve of the KDE quite well and stable.
They are rather close to each other, with a tendency to diverge for larger $h$.
In contrast, the error curve of the box filter estimate has noticeable jumps at $h=(0.4; 0.252; 0.675; 0.825)$.
In contrast, the error curve of the boxKDE has noticeable jumps at $h=(0.4; 0.252; 0.675; 0.825)$.
These jumps are caused by the rounding of the integer-valued box width given by \eqref{eq:boxidealwidth}.
As the extend box filter is able to approximate an exact $\sigma$, it lacks these discontinues.
Consequently, it reduces the overall error of the approximation, but only marginal in this scenario.
The global average MISE over all values of $h$ is $0.0049$ for the regular box filter and $0.0047$ in case of the extended version.
@@ -40,10 +40,9 @@ However, both cases do not give a deeper insight of the error behavior of our me
\begin{figure}[t]
\label{fig:performance}
%\includegraphics[width=\textwidth,height=6cm]{gfx/tmpPerformance.png}
\input{gfx/perf.tex}
\caption{Hier kommt Performance Plot 2 spaltig. Hier bitte noch ein wenig Text hinzufügen, damit da auch was steht. verstehst? vielleicht geht es sogar noch bis in die zweite zeile mit rein. mal schaun. }
\caption{Logarithmic plot of the runtime performance with increasing grid size $G$ and bivariate data. The weighted average estimate (blue) performs fastest followed by the boxKDE (orange) approximation. Both the BKDE (red), and the fastKDE (green) are magnitudes slow, especially for $G<10^4$.}\label{fig:performance}
\end{figure}
% kde, box filter, exbox in abhänigkeit von h (bild)
@@ -54,19 +53,14 @@ However, both cases do not give a deeper insight of the error behavior of our me
\subsection{Performance}
In the following, we underpin the promising theoretical linear time complexity of our method with empirical time measurements compared to other methods.
All tests are performed on a Intel Core \mbox{i5-7600K} CPU with a frequency of \SI{4.2}{\giga\hertz}, and \SI{16}{\giga\byte} main memory.
We compare our C++ implementation of the box filter based KDE approximation to the \texttt{ks} R package and the fastKDE Python implementation \cite{oBrien2016fast}.
We compare our C++ implementation of the box filter based KDE approximation based on algorithm~\ref{alg:boxKDE} to the \texttt{ks} R package and the fastKDE Python implementation \cite{oBrien2016fast}.
The \texttt{ks} packages provides a FFT-based BKDE implementation based on optimized C functions at its core.
% Vergleich zu weighted average (in c++) um unseren großen Geschwindigkeitsvorteil zu zeigen.
With state estimation problems in mind, we additionally provide a C++ implementation of a weighted average estimator.
Our C++ implementation is based on algorithm~\ref{alg:boxKDE} with three filter iterations.
No special data structures are necessary, the grid is stored as a two-dimensional array in memory.
In addition, modern CPUs do benefit from the recursive computation scheme of the box filter, as the data exhibits a high degree of spatial locality in memory and the accesses are good predictable, .
%Furthermore, the computation is easily parallelized using SIMD instructions or threads.
As both methods are not using a grid, an equivalent input sample set was used for the weighted average and the fastKDE.
The results for performance comparison are presented in plot \ref{fig:performance}.
% O(N) gut erkennbar für box KDE und weighted average
The linear complexity \landau{N} of the boxKDE and the weighted average is clearly visible.
The linear complexity of the boxKDE and the weighted average is clearly visible.
% Gerade bei kleinen G bis 10^3 ist die box KDE schneller als R und fastKDE, aber das WA deutlich schneller als alle anderen
Especially for small $G$ up to $10^3$ the boxKDE is much faster compared to BKDE and fastKDE.
% Bei zunehmend größeren G wird der Abstand zwischen box KDE und WA größer.
@@ -87,18 +81,21 @@ The termination of BKDE graph at $G=4406^2$ is caused by an out of memory error
% Der Plot für den normalen Box Filter wurde aus Gründen der Übersichtlichkeit weggelassen.
% Sowohl der box filter als auch der extended box filter haben ein sehr ähnliches Laufzeit Verhalten und somit einen sehr ähnlichen Kurvenverlauf.
% Während die durschnittliche Laufzeit über alle Werte von G beim box filter bei 0.4092s liegt, benötigte der extended box filter im Durschnitt 0.4169s.
Both discussed Gaussian filter, namely box filter and extended box filter, yield a similar runtime behavior and therefore a similar curve progression.
Both discussed Gaussian filter approximations, namely box filter and extended box filter, yield a similar runtime behavior and therefore a similar curve progression.
While the average runtime over all values of $G$ for the standard box filter is \SI{0.4092}{\second}, the extended one provides an average of \SI{0.4169}{\second}.
To keep the arrangement of fig. \ref{fig:performance} clear, we only illustrated the results of an boxKDE with the regular box filter.
To keep the arrangement of fig. \ref{fig:performance} clear, we only illustrated the results of the boxKDE with the regular box filter.
The weighted average has the great advantage of being independent of the dimensionality of the input and effortlessly implemented.
In contrast, the computation of the boxKDE approach increases exponentially with increasing number of dimensions.
However, due to the linear time complexity and the very simple computation scheme, the overall computation time is still sufficient fast for many applications and much smaller compared to other methods.
The boxKDE approach presents a reasonable alternative to the weighted average and is easily integrated into existing systems.
In addition, modern CPUs do benefit from the recursive computation scheme of the box filter, as the data exhibits a high degree of spatial locality in memory and the accesses are reliable predictable.
Furthermore, the computation is easily parallelized, as there is no data dependency between the one-dimensional filter passes in algorithm~\ref{alg:boxKDE}.
Hence, the inner loops can be parallelized using threads or SIMD instructions, but the overall speedup depends on the particular architecture and the size of the input.
\commentByToni{die grafiken haben wir ja jetzt beschrieben, aber ein wenig diskussion in die tiefe fehlt mir trotzdem noch}
\commentByFrank{Farben (blue)(green) in den Bildunterschriften stimmen nicht mehr}
\commentByFrank{Fig4 (error over time) checken ob die beiden farbigen linien jetzt richtig rum sind. NIEMALS GENERIERTE TEX GRAFIKEN DIREKT EDITIEREN}
\input{chapters/realworld}

4
tex/chapters/realworld.tex
View File

@@ -17,14 +17,14 @@ The bivariate state estimation was calculated whenever a step was recognized, ab
\begin{figure}
\input{gfx/walk.tex}
\caption{Occurring bimodal distribution, caused by uncertain measurements. After \SI{20.8}{\second}, the distribution gets unimodal. The weigted-average estimation (blue) provides an high error compared to the ground truth (solid black), while the boxKDE approach (green) does not. }
\caption{Occurring bimodal distribution, caused by uncertain measurements. After \SI{20.8}{\second}, the distribution gets unimodal. The weigted-average estimation (blue) provides an high error compared to the ground truth (solid black), while the boxKDE approach (orange) does not. }
\label{fig:realWorldMulti}
\end{figure}
%
Fig. \ref{fig:realWorldMulti} illustrates a frequently occurring situation, where the particle set splits apart, due to uncertain measurements and multiple possible walking directions.
This results in a bimodal posterior distribution, which reaches its maximum distances between the modes at \SI{13.4}{\second} (black dotted line).
Thus estimating the most probable state using the weighted-average results in the blue line, describing the pedestrian's position to be somewhere outside the building (light green area).
In contrast, the here proposed method (green line) is able to retrieve a good estimate compared the the ground truth path shown by the black solid line.
In contrast, the here proposed method (orange line) is able to retrieve a good estimate compared the the ground truth path shown by the black solid line.
Due to a right turn, the distribution gets unimodal after \SI{20.8}{\second}.
This happens since the lower red particles are walking against a wall and thus punished with a low weight.