FHWS/OTHER2017
Archived
3
0
Fork 0
Code Issues 4 Pull Requests Releases Wiki Activity

current TeX

Browse Source 
minor code changes
This commit is contained in:
FrankE
2017-05-06 17:32:40 +02:00
parent 2438080389
commit edd41293c1
12 changed files with 650 additions and 132 deletions
Download Patch File Download Diff File Expand all files Collapse all files

184
tex/chapters/experiments.tex
View File

@@ -2,6 +2,14 @@
% intro
\todo{reihenfolge so jetzt klar?}
Within our experiments we will first have a look at model optimizations to reduce the error
between model predictions and real-world conditions.
Hereafter we examine the resulting accuracy when using the optimized models for localization
using just the \docWIFI{} component without additional sensors or assumptions.
Finally, all models are evaluated in the context of our indoor localization system \refeq{eq:recursiveDensity},
using additional smartphone sensors and the building's floorplan.
All optimizations and evaluations took place within two adjacent buildings (4 and 2 floors, respectively)
and two connected outdoor regions (entrance and inner courtyard),
\SI{110}{\meter} x \SI{60}{\meter} in size.
@@ -29,15 +37,18 @@
\subsection{Model optimization}
As the signal strength prediction model is the core of the absolute positioning component
described in section \ref{sec:system}, we start with the model parameter estimation (see \ref{sec:optimization}) for
\mTXP{}, \mPLE{} and \mWAF{} based on some reference measurements and compare the results
between various optimization strategies and a basic empiric choice of \mTXP{} = \SI{-40}{\decibel{}m} @ \SI{1}{\meter}
(defined by the usual \docAPshort{} transmit power for europe), a path loss exponent $\mPLE{} \approx $ \SI{2.5} and
$\mWAF{} \approx$ \SI{-8}{\decibel} per floor / ceiling (made of reinforced concrete) \todo{cite für werte}.
described in section \ref{sec:system}, we start with the model parameter optimization (see \ref{sec:optimization}).
\mTXP{}, \mPLE{} and \mWAF{} will be estimated based on some reference measurements using
various optimization strategies. The results of those optimization strategies are compared
with each other and an empiric parameter choice:
\mTXP{} = \SI{-40}{\decibel{}m} @ \SI{1}{\meter}
(defined by the usual \docAPshort{} transmit power for Europe), a path loss exponent $\mPLE{} = 2.5$ and
$\mWAF{} = \SI{-8}{\decibel}$ per floor/ceiling (made of reinforced concrete)
\cite{PathLossPredictionModelsForIndoor, ElectromagneticPropagation, ANewPathLossPrediction}.
\reffig{fig:referenceMeasurements} depicts the location of the used 121 reference measurements.
Each location was scanned 30 times ($\approx$ \SI{25}{\second} scan time),
non permanent \docAP{}s were removed, the values were grouped per physical transmitter (see \ref{sec:vap})
non-permanent \docAP{}s were removed, the values were grouped per physical transmitter (see \ref{sec:vap})
and aggregated to form the average signal strength per transmitter.
\begin{figure}
@@ -53,8 +64,8 @@
\begin{subfigure}[t!]{0.48\textwidth}
\input{gfx2/model-bboxes.tex}
\caption{
More than one bounding box is needed for each model to approximate the building's shape.
Each distinct floor-color denotes a single model (7 in total).
Each distinct floor-color denotes one model (7 in total) for {\em \optPerRegion{}}.
Often, more than one bounding box is needed to approximate the region's shape.
}
\label{fig:modelBBoxes}
\end{subfigure}
@@ -84,8 +95,8 @@
\input{gfx/compare-wifi-in-out.tex}
\caption{
Measurable signal strengths of a testing \docAPshort{} (black dot).
While the signal diminishes slowly along the corridor (upper rectangle)
the metallised windows (dashed outline) attenuate the signal by over \SI{30}{\decibel} (lower rectangle).
While the signal diminishes slowly along the corridor (wide rectangle)
the metallized windows (dashed outline) attenuate the signal by over \SI{30}{\decibel} (small rectangle).
}
\label{fig:wifiIndoorOutdoor}
\end{figure}
@@ -93,28 +104,42 @@
\reffig{fig:wifiIndoorOutdoor} depicts the to-be-expected issues by examining the signal strength
values of the reference measurements for one \docAP{}.
Even though the transmitter is only \SI{5}{\meter} away from the reference
measurement (small box), the metallised windows attenuate the signal as much as \SI{50}{\meter}
of corridor (wide box). The model described in section \ref{sec:sigStrengthModel} will not be able
measurement (small box), the metallized windows attenuate the signal as much as \SI{50}{\meter}
of corridor (wide rectangle). The model described in section \ref{sec:sigStrengthModel} will not be able
to match such situations, due to the lack of obstacle information.
%
We will thus look at various optimization strategies and the error between
the resulting estimation model and our reference measurements:
{\em\noOptEmpiric{}} uses the same three empiric parameters \mTXP{}, \mPLE{}, \mWAF{} for each \docAPshort{} in combination
with its position, which is well known from the floorplan.
\begin{itemize}
{\em\optParamsAllAP{}} is the same as above, except that the three parameters are optimized
using the reference measurements. However, all transmitters share the same three parameters.
{\em\optParamsEachAP{}} optimizes the three parameters per \docAP{} instead of using the same
parameters for all.
{\em\optParamsPosEachAP{}} does not need any prior knowledge and will optimize all six parameters
(3D position, \mTXP, \mPLE, \mWAF) based on the reference measurements.
{\em\optPerFloor{}} and {\em\optPerRegion{}} are just like {\em \optParamsPosEachAP{}} except that
there are several sub-models, each of which is optimized for one floor / region instead of the whole building.
The chosen bounding boxes and resulting sub-models are depicted in \reffig{fig:modelBBoxes}.
\item{
{\em\noOptEmpiric{}} uses the same three empiric parameters \mTXP{}, \mPLE{}, \mWAF{} for each \docAPshort{} in combination
with its position, which is well known from the floorplan.
}
\item{
{\em\optParamsAllAP{}} is the same as above, except that the three parameters are optimized
using the reference measurements. However, all transmitters share the same three parameters.
}
\item{
{\em\optParamsEachAP{}} optimizes the three parameters per \docAP{} instead of using the same
parameters for all.
}
\item{
{\em\optParamsPosEachAP{}} does not need any prior knowledge and will optimize all six parameters
(3D position, \mTXP, \mPLE, \mWAF) based on the reference measurements.
}
\item{
{\em\optPerFloor{}} and {\em\optPerRegion{}} are just like {\em \optParamsPosEachAP{}} except that
there are several sub-models, each of which is optimized for one floor/region instead of the whole building.
The chosen bounding boxes and resulting sub-models are depicted in \reffig{fig:modelBBoxes}.
}
\end{itemize}
\reffig{fig:wifiModelError} shows the optimization results for all strategies, which are as expected:
The estimation error is indirectly proportional to the number of optimized parameters.
@@ -154,7 +179,7 @@
\caption{
Cumulative error distribution for all optimization strategies. The error results from the (absolute) difference
between model predictions and real-world values for each reference measurement.
The higher the number of variable parameters, the better the model resembles real world conditions.
The higher the number of variable parameters, the better the model resembles real-world conditions.
}
\label{fig:wifiModelError}
\end{figure}
@@ -164,19 +189,19 @@
\begin{figure}
\begin{subfigure}{0.32\textwidth}
\centering
\input{gfx/wifiMaxErrorNN_opt0.tex}
\input{gfx2/wifiMaxErrorNN_opt0.tex}
\caption{\em \noOptEmpiric{}}
\label{fig:wifiModelErrorMaxA}
\end{subfigure}
\begin{subfigure}{0.32\textwidth}
\centering
\input{gfx/wifiMaxErrorNN_opt3.tex}
\input{gfx2/wifiMaxErrorNN_opt3.tex}
\caption{\em \optParamsPosEachAP{}}
\label{fig:wifiModelErrorMaxB}
\end{subfigure}
\begin{subfigure}{0.32\textwidth}
\centering
\input{gfx/wifiMaxErrorNN_opt5.tex}
\input{gfx2/wifiMaxErrorNN_opt5.tex}
\caption{\em \optPerRegion{}}
\label{fig:wifiModelErrorMaxC}
\end{subfigure}
@@ -277,7 +302,7 @@
Additionally we examined the impact of skipping reference measurements for difficult locations
like staircases, surrounded by steel-enforced concrete. While this slightly decreases the
estimation error for all other positions (hallway, etc) as expected, the error within the skipped locations is dramatically
estimation error for all other positions (hallway, etc.) as expected, the error within the skipped locations is dramatically
increasing (see right half of \reffig{fig:wifiNumFingerprints}). It is thus highly recommended
to also perform reference measurements for locations, that are expected to strongly deviate (signal strength)
from their surroundings.
@@ -466,6 +491,8 @@
as likely as the pedestrian's actual location, we examined various approaches.
Unfortunately, most of which did not provide a viable enhancement under all conditions for the performed walks.
\commentByFrank{ja, eig gehoert das vor in die theorie, aber da es so kurz ist und vorne immer die ueberleitung kaputt macht
oder anderen dingen vorgreifen wuerde, steht es hier}
The misclassification-rate is determined by counting the amount of (random) locations within
the building that produce a similar probability \refeq{eq:wifiProb} as the actual ground-truth
position.
@@ -514,7 +541,7 @@
for areas where a transmitter was hardly seen within the reference measurements and its optimization is thus
expected to be inaccurate.
Using a smaller $\sigma$ or a more strict exponential distribution for the model vs. scan comparison in \refeq{eq:wifiProb}
Using a smaller $\sigma$ or a stricter exponential distribution for the model vs. scan comparison in \refeq{eq:wifiProb}
had a positive effect on the misclassification error for some of the walks, but also slightly increased the overall estimation error.
%(see figure \ref{fig:normalVsExponential}).
Due to those negative side-effects, the final localization system (\refeq{eq:recursiveDensity}) is unlikely to profit from such changes.
@@ -546,13 +573,13 @@
% -------------------------------- final system -------------------------------- %
\subsection{System error using filtering}
\subsection{Filtered location estimation error}
After examining the \docWIFI{} component on its own, we will now analyze the impact of previously discussed model
optimizations on our smartphone-based indoor localization system described in section \ref{sec:system}, based on
\refeq{eq:recursiveDensity}.
Due to transition constraints from the buildings floorplan, we expect the
Due to transition constraints from the building's floorplan, we expect the
posterior density to often get stuck when the \docWIFI{} component provides erroneous estimations
due to bad signal strength predictions or observations (see \reffig{fig:wifiMultimodality}):
@@ -573,40 +600,44 @@
resulting from all executions for each walk conducted with the smartphone.
While most values represent the expected results (more optimization yields better results),
the values for {\em \noOptEmpiric{}} and {\em \optPerRegion{}} do not.
The slightly increased error for both strategies can be explained by having a closer look at the walked
the values for {\em \optParamsAllAP{}} and {\em \optPerRegion{}} do not.
The increased error for both strategies can be explained by having a closer look at the walked
paths and relates to exceptional regions like outdoors. In both cases there is some sort of model overadaption.
%
As mentioned earlier, {\em \noOptEmpiric{}} is unable to accurately model the signal strength for the whole
building, resulting in increased estimation errors for outdoor regions, where the filter fails to conclude
the walk.
As mentioned earlier, a single, simple model is unable to accurately estimate the signal strength for both
buildings and adjacent outdoor regions. Due to metallized glass (see \reffig{fig:wifiIndoorOutdoor}), in- and
outdoor conditions strongly differ. The model's optimization
builds a compromise among all locations and renders indoor places unnecessarily bad: Previous
discussions showed that outdoor regions do not provide viable \docWIFI{} signals at all. It thus makes sense
to just omit badly covered regions from the model optimization process, as the filter's evaluation will simply
omit \docWIFI{} when the quality is insufficient (see section \ref{sec:wifiQuality}).
%
While {\em \optPerRegion{}} does not suffer from such issues due to separated optimization regions for in- and outdoor,
its increased error relates to movements between such adjacent regions, as there often is a huge model difference.
While this difference is perfectly fine, as it also exists within real world conditions,
the filtering process suffers especially at such model-boundaries:
While this difference is perfectly fine, as it also exists within real-world conditions,
the filtering process suffers at such model-boundaries:
The model prevents the particles from moving e.g. from inside the building towards outdoor regions, as the
outdoor-model does not match at all. Due to sensor delays and issues with the absolute heading near in- and outdoor boundaries
outdoor-model does not yet match. Due to sensor delays and issues with the absolute heading near in- and outdoor boundaries
(metal-framed doors) the error is slightly increased and retained for some time until the density stabilizes itself.
Such situations should be mitigated by the smartphone's GPS sensor. However, within our testing walks, the GPS
did rarely provide accurate measurements, as the outdoor-time was too short for the sensor to receive a valid
fix and the accuracy indicated by the GPS usually was \SI{50}{\meter} and above.
Especially for {\em path 1}, the particle-filter often got stuck within the upper right outdoor area between both buildings
(see \reffig{fig:allWalks}). Using the empirical parameters, \SI{40}{\percent} of all runs for this path got stuck at this location.
{\em \optParamsAllAP{}} already reduced the risk to \SI{20}{\percent} and all other optimization strategies did not get stuck at all.
Additionally increasing the number of particles from 5000 to 10000 indicated only a minor increase in accuracy and slightly decreased
the risk of getting stuck. For battery- and performance-constrained use-cases on the smartphone 5000 thus seems to be a sufficient.
The same effect holds for all other conducted walks: The better the model optimization, the lower the risk of getting stuck somewhere along the path.
Varying the number of particles between 5000 and 10000 indicated only a minor increase in accuracy and slightly decreased the risk of getting stuck.
Comparing the error results within \reffig{fig:modelPerformance} and \reffig{fig:overallSystemError}, one can
denote the positive impact of fusioning multiple sensors with a transition model based on the building's
actual floorplan. Outdoor regions indicated a very low signal quality (see section \ref{sec:wifiQuality}).
By omitting \docWIFI{} from the system's evaluation step, the IMU was able to
keep the pedestrian's current heading until the signal quality reached sane levels again.
Issues while moving from the inside out, or vice versa, should also be mitigated by incorporating the smartphone's GPS sensor.
However, within our testing walks, the GPS did rarely provide accurate measurements, as the outdoor-time often was too short
for the sensor to receive a valid fix. The accuracy indicated by the GPS usually was $\ge \SI{50}{\meter}$ and thus
did not provide usefull information.
However, comparing the error results within \reffig{fig:modelPerformance} and \reffig{fig:overallSystemError}, one can
denote the positive impact of fusing multiple sensors with a transition model based on the building's
actual floorplan. Even within outdoor regions and staircases that suffer from erroneous \docWIFI{} estimations due to a bad
signal strength coverage. The quality metric described in section \ref{sec:wifiQuality} was able to detect such
cases and \docWIFI{} was temporarily ignored. The remaining sensors, like the IMU, and the floorplan were able to
keep the pedestrian's heading until the signal quality reached sane levels again.
\begin{figure}
\begin{subfigure}{0.49\textwidth}
@@ -640,11 +671,52 @@
%
\caption{
Cumulative error distribution for each model when used within the final localization system from \refeq{eq:recursiveDensity}.
Despite some discussed exceptions, highly optimized models lead to lower localization errors.
Especially {\em \optParamsAllAP{}} suffered from overadaption and thus provided worse results. Compared to just using \docWIFI{}
(\reffig{fig:modelPerformance}) the error difference between the models now is much more pronounced.
Starting from {\em \optParamsEachAP{}} the system rarely gets stuck and provides a viable accuracy.
}
\label{fig:overallSystemError}
\end{figure}
Finally, \reffig{fig:final} depicts all of the previously discussed improvements and issues by examining {\em path 1}
from \reffig{fig:allWalks}.
For better visibility within path- and error-plots, the non filtered estimations were smoothed using a moving average of
ten consecutive values ($\approx \SI{7}{\second}$). As can be seen, optimizing the \docWIFI{} model yields an improvement
for indoor situations, as the estimation is closer to the ground truth, and the starting position (indicated by the rectangle)
is more accurate.
For the depicted walk, the error outdoors is increased, as the likeliest position is shifted. Adding
the particle filter (\refeq{eq:recursiveDensity}) on top of the optimized model fixes this issue. What cannot be seen
within the images: while the likeliest position is deteriorated by the optimization, the likelihood of the region around
the pedestrian's ground truth actually is increased. Thus, combined with transition model and other sensors, the system
is able to stay right on track. The filter fails for {\em \noOptEmpiric}, as one \docAPshort{} near the entry of the second
building prevents the density from entering due to a very high difference between model and real-world conditions.
\begin{figure}
\begin{subfigure}{0.49\textwidth}
\centering
\input{gfx/final3D.tex}
\end{subfigure}
\begin{subfigure}{0.49\textwidth}
\centering
\input{gfx/final2D.tex}
\end{subfigure}
\\
\begin{subfigure}{0.99\textwidth}
\input{gfx/final-error.tex}
\end{subfigure}
\caption{
Detailed analysis of the \docWIFI{} error for {\em \noOptEmpiric{}} (unoptimized) and {\em \optPerFloor{}}
using {\em path 1} (see \reffig{fig:allWalks}). While optimization reduces the error indoors, the error outdoors
is increased (bold line). A particle filter (PF, \refeq{eq:recursiveDensity}) on top of the optimized model
takes \SI{5}{\second} to initialize the starting-position (rectangles), fixes the outdoor-issue and
improves indoor situations. A filter on top of {\em \noOptEmpiric{}} got stuck right before
entering the 2nd building.
}
\label{fig:final}
\end{figure}
% results
% 5000 particles
%