182 lines
16 KiB
TeX
182 lines
16 KiB
TeX
\section{Experiments}
|
|
|
|
% allgemeine infos über pfade und gebäude. wo
|
|
|
|
%Gebäude
|
|
All upcoming experiments were carried out on four floors (0 to 3) of a \SI{77}{m} x \SI{55}{m} sized faculty building.
|
|
It includes several staircases and elevators and has a ceiling height of about \SI{3}{m}.
|
|
The gridded graph was generated for the complete campus and thus outdoor areas like the courtyard are also walkable.
|
|
As \docWIFI{} is attenuated by obstacles and walls, it does not provide a consistent quality over the complete building.
|
|
To get an idea about the \docWIFI{} quality, we interpolated the \docWIFI{} quality factor $q(\mObsVec_t^{\mRssiVec_\text{wifi}})$, with $l_{\text{max}} = \SI{-75}{\dBm}$ and $l_{\text{min}} = \SI{-90}{\dBm}$, between some measurements using a nearest-neighbour procedure.
|
|
In fig. \ref{fig:paths} the resulting colourized floorplan is illustrated.
|
|
Here, the intensity of red indicates a low signal strength and thus a bad quality for localising a pedestrian within this area using \docWIFI{}.
|
|
|
|
% bild: mit pfaden drauf und eventl. wifi qualität in jeweiligen bereichen? (kann frank das)
|
|
\begin{figure}
|
|
\centering
|
|
\input{gfx/eval/paths.tex}
|
|
\caption{The three paths that were part of the experiments. Starting positions are marked with black circles. The red squares illustrate the \docWIFI{} quality in this sector. The intensity of red indicates a low coverage and thus a bad quality for localisation.}
|
|
\label{fig:paths}
|
|
\end{figure}
|
|
%
|
|
%Pfade
|
|
We arranged three distinct walks (see also fig. \ref{fig:paths}).
|
|
The measurements for the walks were recorded using a Motorola Nexus 6 at 2.4 GHz band only.
|
|
The computation was done offline as described in algorithm \ref{fig:paths}.
|
|
For each walk we deployed $50$ runs using 5000 particles for each mode.
|
|
Instead of an initial position and heading, all walks start with a uniform distribution (random position and heading) as prior $q_1$.
|
|
In the graph-based transition of the dominant filter, the to-be-walked distance is given by the number of steps using a step size of \SI{70}{\centimeter} with an allowed deviation of \SI{10}{\percent}.
|
|
The deviation for the walking direction was set to \SI{25}{\degree}.
|
|
Since the simple transition randomly scatters particles within a specific range, we choose a covariance matrix that allows a variance of \SI{200}{\centimeter} in $x$- and $y$-direction for the multivariate normal distribution.
|
|
Here, floors were changed by deploying a discrete distribution for every floor level, providing a chance of \SI{27}{\percent} for changing one floor and \SI{5}{\percent} for two floors in a particular $z$-direction.
|
|
|
|
We omit any time-consuming calibration processes and therefore use the same parameters for all \docWIFI{} access-points, similar to \cite{Ebner-15}.
|
|
The position of the access-points (about five per floor) is known beforehand.
|
|
Due to legal terms, we are not allowed to depict their positions and therefore omit this information within the figures.
|
|
To model the uncertainty of the \docWIFI{}, a white Gaussian noise with $\sigma_\text{wifi} = \SI{8}{\dBm}$, growing with each measurement's age, was chosen.
|
|
The pressure change was assumed to be \SI{0.105}{$\frac{\text{\hpa}}{\text{\meter}}$}, all other barometer-parameters are determined automatically as described in \cite{Fetzer2016OMC}.
|
|
|
|
|
|
As expected before, the kernel density estimation in \eqref{eq:KDE} and the much simpler multivariate normal distribution estimation are both producing similar results in context of the divergence measure given in \eqref{equ:KLD}.
|
|
For this reason and because of the faster processing time, especially in context of calculating on a smartphone, we estimated the posteriors of the modes by approximating a multivariate normal distribution (cf. section \ref{sec:divergence}).
|
|
For calculating \eqref{equ:KLD} we set $\lambda = 0.03$ based on best practices.
|
|
Finally, the pedestrian's most likely position (state) was then estimated using the weighted arithmetic mean of the particle set.
|
|
|
|
% ground truth
|
|
The ground truth is measured by recording a timestamp at marked spots on the walking route. When passing a marker, the pedestrian clicked a button on the smartphone application.
|
|
Between two consecutive points, a constant movement speed is assumed.
|
|
Thus, the ground truth might not be \SI{100}{\percent} accurate, but fair enough for error measurements.
|
|
The approximation error is then calculated by comparing the interpolated ground truth position with
|
|
the current estimation \cite{Fetzer2016OMC}.
|
|
|
|
The here presented walks were selected because they fail in some way using a restrictive transition model and thus are well suited to represent the benefits and drawbacks of the IMMPF approach.
|
|
In this context, it should be noted that the localisation system used for the experiments is very basic and can be seen as a slimmed version of our previous works \cite{Fetzer2016OMC, Ebner-16}.
|
|
Optimizing the \docWIFI{} parameters and adding additional methods will improve the localisation results significantly.
|
|
|
|
To avoid misunderstandings in the upcoming discussions, the used terminologies for the different filter schemes are summarized as follows:
|
|
graph-based and simple filter are standard independently running particle filters (PF) using the graph-based and simple transition as described at the end of section \ref{sec:rse}.
|
|
In contrast, dominant and support filter are indeed the same filters, but used as modes within the IMMPF procedure as described in section \ref{sec:immpf}.
|
|
To allow a detailed discussion of the results shown by images, we separated the paths in different segments (seg.) of interest.
|
|
|
|
%error at the beginning always very high. about 44 meters. therefore the median is better value oder 75 quantil.
|
|
|
|
% zeigen das es stucken verhindert (eventl. hier eine andere aufnahme die mitten drinnen stecken bleibt)
|
|
% bild: stucken im raum + nicht mehr stucken im raum + kld mit anzeigen
|
|
\begin{figure}[b]
|
|
\centering
|
|
\input{gfx/eval/path3.tex}
|
|
\input{gfx/eval/path3-kld.tex}
|
|
\caption{Exemplary results on path 3 for the graph-based filter (red), the simple filter (blue) and our IMMPF approach (green). The Kullback-Leibler divergence $D_{\text{KL}}$ between the graph-based and the simple filter (purple) proves itself as a good indicator, if one filter gets stuck or loses track.}
|
|
\label{fig:path3}
|
|
\end{figure}
|
|
%
|
|
First, we discuss the results of path 3, starting at the left-hand side of the building.
|
|
Exemplary estimation results for the IMMPF, the graph-based and the simple particle filter, can be seen in fig. \ref{fig:path3}.
|
|
As mentioned above, every run of a walk starts with a uniform distribution as prior.
|
|
Due to a low \docWIFI{} coverage at the starting point in seg. 1, the pedestrian's position is falsely estimated into a room instead of the corridor.
|
|
All three filters are able to overcome this false detection.
|
|
However, the graph-based filter (red) gets then indissoluble captured within a room, because of its restrictive behaviour and the aftereffects of the initial \docWIFI{} readings (cf. fig. \ref{fig:path3} seg. 2).
|
|
It provides an \SI{75}{\percent}-quantil of $\tilde{x}_{75} = \SI{3884}{\centimeter}$ and got stuck in \SI{100}{\percent} of all runs.
|
|
As expected and discussed earlier, the simple filter (blue) is less prone to bad observations and provides not accurate, but very robust results of $\tilde{x}_{75}= \SI{809}{\centimeter}$ and a standard deviation over all results of $\bar{\sigma} = \SI{529}{\centimeter}$.
|
|
Looking at $D_{\text{KL}}$ over time confirms our assumption made in section \ref{sec:divergence}.
|
|
The graph-based filter (red) gets stuck and is not able to recover.
|
|
Starting on from this point, the Kullback-Leibler divergence $D_{\text{KL}}$ (purple) further increases due to the growing distance between both filters (blue and red) starting at seg. 2 until the end of seg. 4.
|
|
It is clearly visible, that this divergence between both filters is a very good indicator to observe, if a filter gets stuck or loses track.
|
|
Following, the IMMPF (green) results in a very natural and straight path estimation and a low $D_{\text{KL}}$ between modes.
|
|
The benefits of mixing both filtering schemes within the scenario of path 3 are thus obvious.
|
|
The IMMPF filters with an error of $\tilde{x}_{75} = \SI{667}{\centimeter}$ and $\bar{\sigma} = \SI{558}{\centimeter}$.
|
|
|
|
% zeigen das schlechtes wi-fi (zu hohe diversity) behoben wird.
|
|
% bild: lauf auf der rechten seite des gebäudes zeige mit und ohne wifi faktor (schlechtes wifi einzeichnen)
|
|
\begin{figure}[b]
|
|
\centering
|
|
\input{gfx/eval/path2.tex}
|
|
\input{gfx/eval/path2-wifi-quality.tex}
|
|
\caption{Comparison of the estimation results on path 2 with (green) and without (red) the \docWIFI{} quality factor in the Markov transition matrix. The low \docWIFI{} quality and thus high errors between the \SI{80}{th} and \SI{130}{th} second are caused by the high attenuation and low signal coverage inside the zig-zag stairwell on the building's backside.}
|
|
\label{fig:path2}
|
|
\end{figure}
|
|
%
|
|
Next, we investigate the performance of our approach by considering the scenario in path 2.
|
|
Here, the overall \docWIFI{} quality is rather low, especially in the zig-zag stairwell on the buildings back and the small entrance area at floor 1 (cf. fig. \ref{fig:paths} and fig. \ref{fig:path2} seg. 3).
|
|
Path 2 starts in the second floor, walking down the centred stairs into the first floor, then making a right turn and walking the stairs down to zeroth floor, from there we walk back to second floor using the zig-zag stairwell and after finally crossing a room we are back at the start.
|
|
This is a very challenging scenario, at first the estimation got stuck on the first floor in a room's corner for \SI{20}{\second} (see fig. \ref{fig:path2} seg. 2) and after that the \docWIFI{} is highly attenuated at the beginning and end of seg. 3.
|
|
Looking at fig. \ref{fig:path2}, one can observe the impact of the \docWIFI{} quality factor within the Markov transition matrix.
|
|
Without it, the position estimation (red) is drifting in the courtyard, missing the stairwell and producing high errors between \SIrange{80}{130}{\second} in seg. 3.
|
|
As described earlier, the bad \docWIFI{} readings are causing $D_{\text{KL}}$ to grow.
|
|
It follows that the accurate dominant filter draws new particles from the uncertain support and therefore worsen the position estimation.
|
|
Given this outcome, it is not very satisfying that the system is able to recover thanks to its high diversity during situations with uncertain measurements.
|
|
By adding the \docWIFI{} quality factor, the system is able to improve the approximated path (green) and the overall estimation results from $\tilde{x}_{75} = \SI{1278}{\centimeter}$ with $\bar{\sigma} = \SI{948}{\centimeter}$ to $\tilde{x}_{75} = \SI{953}{\centimeter}$ with $\bar{\sigma} = \SI{543}{\centimeter}$.
|
|
However, this is far from perfect and in some cases ($\sim \SI{9}{\percent}$) the quality factor was not able to prevent the estimation to drift in the courtyard.
|
|
This solely happened when particles were sampled directly onto the courtyard while changing from first to zeroth floor (cf. fig. \ref{fig:path2} seg. 3).
|
|
Those particles then received a high weight due to the attenuated measurements, causing a weight degeneracy.
|
|
Adapting the bounds $l_{\text{max}}$ and $l_{\text{min}}$ of the quality factor or optimizing the access-point's parameters can resolve this problem \cite{Ebner-17}.
|
|
|
|
% zeigen das immpf nicht viel schlechter als normaler pf (ohne stucken) ist.
|
|
% bild: er schafft es nicht die treppe rauf + er schafft es immpf + er schafft es normal filter
|
|
\begin{figure}[t]
|
|
\centering
|
|
\input{gfx/eval/path1.tex}
|
|
\input{gfx/eval/path1-time.tex}
|
|
\caption{Estimation results and error development while walking alongside path 1. In \SI{20}{\percent} of cases, the the graph-based particle filter failed to detect the first floor change. Therefore, a good (blue) and a bad (red) result are shown. The here presented approach (green) never lost track.}
|
|
\label{fig:path1}
|
|
\end{figure}
|
|
%
|
|
An exemplary result for path 1 is illustrated in fig. \ref{fig:path1}.
|
|
The path starts on the first floor and finishes on the third after walking two straight stairs.
|
|
Using the graph-based filter for localisation, we were able to obtain a very accurate path (blue) in \SI{80}{\percent} of the runs providing $\tilde{x}_{75} = \SI{526}{\centimeter}$ with $\bar{\sigma} = \SI{316}{\centimeter}$.
|
|
Due to a lack of particles near the stairs, the other \SI{20}{\percent} of the estimations given by the graph-based filter failed to detect the first floor change (red).
|
|
Using our approach (green), we were able detect all floor changes and thus never lost track.
|
|
Fig. \ref{fig:path1} shows an example, where the dominant filter also failed to change floors within seg. 2.
|
|
By looking at seg. 2 of the error plot, we can observe a more or less constant error of \SIrange{5}{6}{\meter}, which then drops rapidly at the beginning of seg. 3.
|
|
This reduction of the error is caused by the growing importance of the mixing stage, where more and more particles from the support filter are incorporated into the dominant filter.
|
|
Going on in seg. 3, the \docWIFI{} measurements suffer from an attenuation directly after leaving the stairs, what leads to a high error using the graph-based filter (blue and red).
|
|
In contrast, the IMMPF is able to compensate the false detection due to an decreasing \docWIFI{} quality and thus a highly focused posterior provided by the dominant filter.
|
|
The IMMPF then performs with $\tilde{x}_{75} = \SI{544}{\centimeter}$ and $\bar{\sigma} = \SI{281}{\centimeter}$.
|
|
Those very similar estimation results between IMMPF (green) and the graph-based filter (blue) confirm the efficiency of the mixing and how it is able to keep the accuracy while providing a higher robustness against failures.
|
|
|
|
%allgemeines abschließendes blabla?
|
|
To summarize, the here presented approach was able to recover in all situations and thus never got completely stuck within a demarcated area.
|
|
The above deployed experiments have shown, that the Markov switching process, as presented in sec. \ref{sec:immpf}, enables a reasonable mixing between two particle filters with different transition schemes.
|
|
Therefore, the quality and success of the results depend highly on the parameters used within the Markov matrix $\Pi_t$.
|
|
Those values are very sensitive and should be chosen carefully in regard to the specific system and use case.
|
|
The experiments have further shown, that faulty \docWIFI{} measurements can lead to significant errors in a very short time.
|
|
That has to do with the fact that \docWIFI{} is the only absolute information source within the state evaluation and thus plays a significant part in the mixing stage of \eqref{equ:immpMode2}.
|
|
Adding additional absolute sensors like bluetooth beacons or ultrasonics sensors, it is possible to reduce the \docWIFI{} component's significance towards other sensor models evaluating the pedestrian's possible whereabouts.
|
|
|
|
|
|
% gegenüberstellung aller pfade und werte in tabelle
|
|
% \begin{table}
|
|
% \caption{Resulting Errors for all conducted walks in meter.}
|
|
% \label{tbl:err}
|
|
% \centering
|
|
% \scalebox{0.93}{
|
|
% \begin{tabular}{|l|c|c|c|c|c|c|c|c|c|}
|
|
% \hline
|
|
% & \multicolumn{3}{c|}{Path 1} & \multicolumn{3}{c|}{Path 2} & \multicolumn{3}{c|}{Path 3}\\
|
|
% \hline
|
|
% & $\bar{x}$ & $\bar{\sigma}$ & $\tilde{x}_{75}$ & $\bar{x}$ & $\bar{\sigma}$ & $\tilde{x}_{75}$ & $\bar{x}$ & $\bar{\sigma}$ & $\tilde{x}_{75}$ \\
|
|
% \hline
|
|
% $\text{PF}_{\text{graph}}$ & $4.0$ & $3.2$ & $5.3$ & $8.2$ & $4.0$ & $10.7$ & $30.3$ & $12.8$ & $38.8$ \\
|
|
% \hline
|
|
% $\text{PF}_{\text{simple}}$ & $4.9$ & $2.8$ & $6.2$ & $7.3$ & $2.9$ & $9.4$ & $6.8$ & $5.4$ & $8.1$ \\
|
|
% \hline
|
|
% IMMPF & $4.2$ & $2.8$ & $5.4$ & $7.7$ & $5.4$ & $9.5$ & $6.3$ & $5.6$ & $6.7$ \\
|
|
% \hline
|
|
% \end{tabular}
|
|
% }
|
|
% \end{table}
|
|
|
|
%An overview of all localisation results can be seen in table \ref{tbl:err}.
|
|
|
|
%So the results of table \ref{tbl:err} should not be seen as best case localization results, but more as proofing robustness while providing room for further improvements.
|
|
|
|
%Especially, the graph-based transition model allows many optimizations and performance boosts.
|
|
%More importantly, the here presented approach was able to recover in all situations and thus never got completely stuck within a demarcated area.
|
|
%All results were similar or more accurate then the ones provided by the standalone filters, even when the localisation did not suffer from any problems.
|
|
|
|
%how the Markov transition matrix regulates the impact of the supporting filter in the right amount.
|
|
|
|
|
|
|
|
|