FHWS/OTHER2017
Archived
3
0
Fork 0
Code Issues 4 Pull Requests Releases Wiki Activity

current TeX and Code

Browse Source
This commit is contained in:
FrankE
2017-05-05 14:33:13 +02:00
parent 1b24977a95
commit e3ba39c5a2
17 changed files with 621 additions and 114 deletions
Download Patch File Download Diff File Expand all files Collapse all files

244
tex/chapters/experiments.tex
View File

@@ -35,7 +35,7 @@
(defined by the usual \docAPshort{} transmit power for europe), a path loss exponent $\mPLE{} \approx $ \SI{2.5} and
$\mWAF{} \approx$ \SI{-8}{\decibel} per floor / ceiling (made of reinforced concrete) \todo{cite für werte}.
Figure \ref{fig:referenceMeasurements} depicts the location of the used 121 reference measurements.
\reffig{fig:referenceMeasurements} depicts the location of the used 121 reference measurements.
Each location was scanned 30 times ($\approx$ \SI{25}{\second} scan time),
non permanent \docAP{}s were removed, the values were grouped per physical transmitter (see \ref{sec:vap})
and aggregated to form the average signal strength per transmitter.
@@ -90,7 +90,7 @@
\label{fig:wifiIndoorOutdoor}
\end{figure}
Figure \ref{fig:wifiIndoorOutdoor} depicts the to-be-expected issues by examining the signal strength
\reffig{fig:wifiIndoorOutdoor} depicts the to-be-expected issues by examining the signal strength
values of the reference measurements for one \docAP{}.
Even though the transmitter is only \SI{5}{\meter} away from the reference
measurement (small box), the metallised windows attenuate the signal as much as \SI{50}{\meter}
@@ -114,12 +114,13 @@
{\em\optPerFloor{}} and {\em\optPerRegion{}} are just like {\em \optParamsPosEachAP{}} except that
there are several sub-models, each of which is optimized for one floor / region instead of the whole building.
The chosen bounding boxes and resulting sub-models are depicted in figure \ref{fig:modelBBoxes}.
The chosen bounding boxes and resulting sub-models are depicted in \reffig{fig:modelBBoxes}.
Figure \ref{fig:wifiModelError} shows the optimization results for all strategies, which are as expected:
\reffig{fig:wifiModelError} shows the optimization results for all strategies, which are as expected:
The estimation error is indirectly proportional to the number of optimized parameters.
However, even with {\em \optPerRegion{}} the maximal error is relatively high due to some locations that do
not fit the model at all, which is shown in figure \ref{fig:wifiModelErrorB}.
However, while median- and average-errors are fine, maximal errors sometimes are relatively high.
As depicted in \reffig{fig:wifiModelErrorMax}, even with {\em \optPerRegion{}} some locations simply do not fit the model,
and thus lead to high (local) errors.
%
Looking at the optimization results for \mTXP{}, \mPLE{} and \mWAF{} supports
this finding. While the median for those values based on all optimized transmitters is totally sane
@@ -130,30 +131,62 @@
For \SI{68}{\percent} of all installed transmitters, the estimated floor-number matched the real location.
\begin{figure}
% cumulative error density
\begin{subfigure}{0.52\textwidth}
\input{gfx2/wifi_model_error_0_95.tex}
\end{subfigure}
\begin{subfigure}{0.23\textwidth}
\input{gfx/wifiMaxErrorNN_opt0.tex}
\caption{\em \noOptEmpiric{}}
\label{fig:wifiModelErrorA}
\end{subfigure}
%\begin{subfigure}{0.25\textwidth}
% \input{gfx/wifiMaxErrorNN_opt3.tex}
%\end{subfigure}
\begin{subfigure}{0.23\textwidth}
\input{gfx/wifiMaxErrorNN_opt5.tex}
\caption{\em \optPerRegion{}}
\label{fig:wifiModelErrorB}
% table
\begin{subfigure}{0.47\textwidth}
\smaller
\centering
\begin{tabular}{|l|c|c|c|c|}
\hline
& 25 \% & median & 75 \% & avg \\\hline
\noOptEmpiric{} & \SI{2.5}{\decibel} & \SI{5.6}{\decibel} & \SI{9.3}{\decibel} & \SI{6.5}{\decibel} \\\hline
\optParamsAllAP{} & \SI{2.0}{\decibel} & \SI{4.3}{\decibel} & \SI{7.5}{\decibel} & \SI{5.4}{\decibel} \\\hline
\optParamsEachAP{} & \SI{1.6}{\decibel} & \SI{3.3}{\decibel} & \SI{6.2}{\decibel} & \SI{4.4}{\decibel} \\\hline
\optParamsPosEachAP{} & \SI{1.5}{\decibel} & \SI{3.0}{\decibel} & \SI{5.5}{\decibel} & \SI{3.8}{\decibel} \\\hline
\optPerFloor{} & \SI{0.7}{\decibel} & \SI{1.6}{\decibel} & \SI{3.3}{\decibel} & \SI{2.6}{\decibel} \\\hline
\optPerRegion{} & \SI{0.6}{\decibel} & \SI{1.4}{\decibel} & \SI{3.1}{\decibel} & \SI{2.4}{\decibel} \\\hline
\end{tabular}
\vspace{9mm}
\end{subfigure}
\caption{
Comparison between different optimization strategies by examining the error (in \decibel) at each reference measurement.
Cumulative error distribution for all optimization strategies. The error results from the (absolute) difference
between model predictions and real-world values for each reference measurement.
The higher the number of variable parameters, the better the model resembles real world conditions.
Both figures on the right depict the highest error for each reference measurement, where full red means $\ge$ \SI{20}{\decibel}.
}
\label{fig:wifiModelError}
\end{figure}
\begin{figure}
\begin{subfigure}{0.32\textwidth}
\centering
\input{gfx/wifiMaxErrorNN_opt0.tex}
\caption{\em \noOptEmpiric{}}
\label{fig:wifiModelErrorMaxA}
\end{subfigure}
\begin{subfigure}{0.32\textwidth}
\centering
\input{gfx/wifiMaxErrorNN_opt3.tex}
\caption{\em \optParamsPosEachAP{}}
\label{fig:wifiModelErrorMaxB}
\end{subfigure}
\begin{subfigure}{0.32\textwidth}
\centering
\input{gfx/wifiMaxErrorNN_opt5.tex}
\caption{\em \optPerRegion{}}
\label{fig:wifiModelErrorMaxC}
\end{subfigure}
\caption{
Local maximum error between model estimation and reference measurements among all known transmitters.
While optimization is able to reduce such errors, some local maxima remain due to overadaption.
}
\label{fig:wifiModelErrorMax}
\end{figure}
%\begin{figure}
@@ -173,7 +206,7 @@
%Pos: cnt(34) min(3.032438) max(26.767128) range(23.734690) med(7.342710) avg(8.571227) stdDev(4.801449)
While {\em \optPerRegion{}} is able to overcome the indoor vs. outdoor issues depicted in
figure \ref{fig:wifiIndoorOutdoor}, by using a separate bounding box just for the outdoor area,
\reffig{fig:wifiIndoorOutdoor}, by using a separate bounding box just for the outdoor area,
it obviously requires a profound prior knowledge to correctly select the individual regions for the sub-model.
%Such issues can only be fixed using more appropriate models that consider walls and other obstacles.
@@ -229,7 +262,7 @@
% \label{fig:wifiNumFingerprints}%
%\end{figure}
Figure \ref{fig:wifiNumFingerprints} depicts the impact of reducing the number of reference measurements
\reffig{fig:wifiNumFingerprints} depicts the impact of reducing the number of reference measurements
during the optimization process for the {\em \optPerRegion{}} strategy.
The error is determined by using the (absolute) difference between expected signal strength and
the optimized model's corresponding prediction for all of the 121 reference measurements.
@@ -245,7 +278,7 @@
Additionally we examined the impact of skipping reference measurements for difficult locations
like staircases, surrounded by steel-enforced concrete. While this slightly decreases the
estimation error for all other positions (hallway, etc) as expected, the error within the skipped locations is dramatically
increasing (see right half of figure \ref{fig:wifiNumFingerprints}). It is thus highly recommended
increasing (see right half of \reffig{fig:wifiNumFingerprints}). It is thus highly recommended
to also perform reference measurements for locations, that are expected to strongly deviate (signal strength)
from their surroundings.
@@ -286,15 +319,20 @@
% -------------------------------- wifi walk error -------------------------------- %
\subsection{Location estimation error}
\todo{uebergang holprig?}
\subsection{\docWIFI{} location estimation error}
\todo{uebergang jetzt besser?}
Having optimized several signal strength prediction models, we can now examine the resulting localization
accuracy for each one. For now, this will just cover the \docWIFI{} component itself.
The impact of adding additional sensors and a transition model will be evaluated later.
%Using the optimized model setups and the measurements $\mRssiVec$ determined by scanning for nearby \docAPshort{}s,
%we can directly perform a location estimation by rewriting \refeq{eq:wifiProb}:
For each of the discussed optimization strategies we can now determine the resulting localization accuracy.
The position within the building that best fits some signal strength measurements $\mRssiVec$ received by the smartphone
is the one that maximizes $p(\mPosVec \mid \mRssiVec)$ and can be rewritten as:
%For each of the discussed optimization strategies we can now determine the resulting localization accuracy.
The position $\mPosVec{}$ within the building that best fits some \docWIFI{} signal strength measurements $\mRssiVec$ received by the smartphone
is the one that maximizes $p(\mPosVec \mid \mRssiVec)$.
Omitting prior knowledge and normalization, this can be rewritten as:
\begin{equation}
p(\mPosVec \mid \mRssiVec) =
@@ -312,12 +350,13 @@
\mPosVec^* = \argmax_{\mPosVec}
\prod_{\mRssi_{i} \in \mRssiVec{}}
\mathcal{N}(\mRssi_i \mid \mu_{i,\mPosVec}, \sigma^2)
\enskip.
\label{eq:bestWiFiPos}
\end{equation}
where $\mu_{i,\mPosVec}$ is the signal strength for \docAP{} $i$
at location $\mPosVec$ returned from the to-be-examined prediction model.
For all comparisons we use a constant uncertainty $\sigma = \SI{8}{\decibel}$.
Within \refeq{eq:bestWiFiPos} $\mu_{i,\mPosVec}$ is the signal strength for \docAP{} $i$,
installed at location $\mPosVec$, returned from the to-be-examined prediction model.
For all comparisons we use a constant uncertainty of $\sigma = \SI{8}{\decibel}$.
The quality of the estimated location is determined by using the Euclidean distance between estimation
$\mPosVec^*$ and the pedestrian's ground truth position at the time the scan $\mRssiVec$
@@ -327,7 +366,7 @@
We therefore conducted 13 walks on 5 different paths within our building,
each of which is defined by connecting marker points at well known positions
(see figure \ref{fig:allWalks}).
(see \reffig{fig:allWalks}).
Whenever the pedestrian reached such a marker, the current time was recorded.
Due to constant walking speeds, the ground-truth for any timestamp can be approximated
using linear interpolation between adjacent markers.
@@ -335,7 +374,7 @@
% walked paths
\begin{figure}
\centering
\input{gfx/all_walks.tex}
\input{gfx2/all_walks.tex}
\caption{
Overview of all conducted paths.
Outdoor areas are marked in green.
@@ -344,12 +383,40 @@
\end{figure}
\begin{figure}
\input{gfx/modelPerformance_meter.tex}
\caption{
Cumulative error distribution between ground truth and location estimation using \refeq{eq:bestWiFiPos} depending
on the underlying signal strength prediction model.
Extremely high errors between the \SIrange{90}{100}{\percent} quartile are related to bad \docWIFI{}
coverage within outdoor areas (see figure \ref{fig:wifiIndoorOutdoor}).
% error gfx
\begin{subfigure}{0.52\textwidth}
\centering
\input{gfx2/modelPerformance_meter.tex}
\end{subfigure}
% table
%5.98767 9.23025 14.4272 11.9649
%6.53764 9.01424 12.8797 12.0121
%6.85665 9.82203 13.8528 12.9988
%5.35629 8.5921 14.8037 11.9996
%4.30191 6.91534 14.0746 11.948
%4.26189 6.35975 11.5646 10.7466
\begin{subfigure}{0.47\textwidth}
\smaller
\centering
\begin{tabular}{|l|c|c|c|c|}
\hline
& \SI{25}{\percent} & median & \SI{75}{\percent} & avg \\\hline
\noOptEmpiric{} & \SI{6.0}{\meter} & \SI{9.2}{\meter} & \SI{14.4}{\meter} & \SI{11.9}{\meter} \\\hline
\optParamsAllAP{} & \SI{6.5}{\meter} & \SI{9.0}{\meter} & \SI{12.8}{\meter} & \SI{12.0}{\meter} \\\hline
\optParamsEachAP{} & \SI{6.8}{\meter} & \SI{9.8}{\meter} & \SI{13.8}{\meter} & \SI{13.0}{\meter} \\\hline
\optParamsPosEachAP{} & \SI{5.4}{\meter} & \SI{8.6}{\meter} & \SI{14.8}{\meter} & \SI{12.0}{\meter} \\\hline
\optPerFloor{} & \SI{4.3}{\meter} & \SI{6.9}{\meter} & \SI{14.0}{\meter} & \SI{11.9}{\meter} \\\hline
\optPerRegion{} & \SI{4.2}{\meter} & \SI{6.5}{\meter} & \SI{11.6}{\meter} & \SI{10.7}{\meter} \\\hline
\end{tabular}
\vspace{9mm}
\end{subfigure}
\caption {
Cumulative error distribution between walked ground truth and \docWIFI{}-only location estimation using \refeq{eq:bestWiFiPos}.
%depending on the signal strength prediction model.
All models suffer from several (extremely) high errors that relate to bad \docWIFI{}
coverage e.g. within outdoor areas (see \reffig{fig:wifiIndoorOutdoor}). This negatively affects the average and 75th
percentile. The strategies {\em \optParamsAllAP{}} and {\em \optParamsEachAP{}} sometimes suffered from overadaption,
indicated by increased error values for the 25th percentile.
}
\label{fig:modelPerformance}
\end{figure}
@@ -359,11 +426,11 @@
%against the corresponding ground-truth, which indicates the absolute 3D error in meter.
The position estimation for each \docWIFI{} measurement within the recorded walks (3756 scans in total)
is compared against its corresponding ground-truth, indicating the 3D distance error.
The resulting cumulative error distribution can be seen in figure \ref{fig:modelPerformance}.
The resulting cumulative error distribution can be seen in \reffig{fig:modelPerformance}.
The quality of the location estimation directly scales with the quality of the signal strength prediction model.
However, as discussed earlier, the maximal estimation error might increase for some setups.
%
This is either due to multimodalities, where more than one area is possible based on the recent
This is either due to multimodalities, where more than one area matches the recent
\docWIFI{} observation, or optimization yielded an overadaption where the average signal
strength prediction error is small, but the maximum error is dramatically increased for some regions.
@@ -372,17 +439,18 @@
% -------------------------------- plots indicating walk issues -------------------------------- %
\begin{figure}
\input{gfx/wifiMultimodality.tex}
\input{gfx2/wifiMultimodality.tex}
\caption{
Location probability \refeq{eq:bestWiFiPos} for three scans. Higher color intensities are more likely.
Ideally, places near the black ground truth are highly highly probable (green).
Often, other locations are just as likely as the ground truth (blue),
or the location with the highest probability does not match at all (red).
\docWIFI{}-only location probability for three distinct scans where
higher color intensities denote a higher likelihood for \refeq{eq:bestWiFiPos}.
The first scan (left, green) depicts a best-case scenario, where the region around the ground truth (black rectangle) is highly probable.
Often, other locations are just as likely as the ground truth (2nd scan, blue),
or the location with the highest probability is far from the actual ground truth (3rd scan, right, red).
}
\label{fig:wifiMultimodality}
\end{figure}
Figure \ref{fig:wifiMultimodality} depicts aforementioned issues of multimodal (blue) or wrong (red) location
\reffig{fig:wifiMultimodality} depicts aforementioned issues of multimodal (blue) or wrong (red) location
estimations. Filtering (\refeq{eq:recursiveDensity}) thus is highly recommended, as minor errors are compensated
using other sensors or a movement model that prevents the estimation from leaping within the building.
However, if wrong sensor values are observed for longer time periods, even filtering will produce erroneous
@@ -410,23 +478,30 @@
%as the Smartphone did not see this \docAPshort{} the other location can be ruled out.
While this works in theory, evaluations revealed several issues:
There is a chance that even a nearby \docAPshort{} is unseen during a scan due to packet collisions or
temporal effects within the surrounding. It thus might make sense to opt-out other locations
only, if at least two \docAPshort{}s are missing. On the other hand, this obviously demands for (at least)
two \docAPshort{}s to actually be different between the two locations, and requires a lot of permanently
installed transmitters to work out.
\begin{itemize}
\item{
There is a chance that even a nearby \docAPshort{} is unseen during a scan due to packet collisions or
temporal effects within the surrounding. It thus might make sense to opt-out other locations
only, if at least two \docAPshort{}s are missing. On the other hand, this obviously demands for (at least)
two \docAPshort{}s to actually be different between the two locations, and requires a lot of permanently
installed transmitters to work out.
}
\item{
Furthermore, this requires the signal strength prediction model to be fairly accurate. Within our testing
walks, several places are surrounded by concrete walls, which cause a harsh, local drop in signal strength.
The models used within this work will not accurately predict the signal strength for such locations.
%%Including \docAPshort{}s unseen by the Smartphone thus often increases the estimation error instead
%%of fixing the multimodality.
}
\end{itemize}
Furthermore, this requires the signal strength prediction model to be fairly accurate. Within our testing
walks, several places are surrounded by concrete walls, which cause a harsh, local drop in signal strength.
The models used within this work will not accurately predict the signal strength for such locations.
%%Including \docAPshort{}s unseen by the Smartphone thus often increases the estimation error instead
%%of fixing the multimodality.
To sum up, while some situations, e.g. outdoors, could be improved,
many other situations are deteriorated, especially when some transmitters are (temporarily)
attenuated by ambient conditions like concrete walls.
We therefore examined variations of the probability calculation from \refeq{eq:wifiProb}.
Despite the results show in \cite{PotentialRisks}, removing weak \docAPshort{}s from $\mRssiVec{}$
yielded similar results. While some estimations were improved, the overall error increased
@@ -471,15 +546,16 @@
% -------------------------------- final system -------------------------------- %
\subsection{Overall system error}
\subsection{System error using filtering}
After examining the \docWIFI{} component on its own, we will now analyze the impact of aforementioned model
optimizations on our smartphone-based indoor localization system described in section \ref{sec:system}.
After examining the \docWIFI{} component on its own, we will now analyze the impact of previously discussed model
optimizations on our smartphone-based indoor localization system described in section \ref{sec:system}, based on
\refeq{eq:recursiveDensity}.
Due to transition constraints from the buildings floorplan, we expect the
posterior density to often get stuck when the \docWIFI{} component provides erroneous estimations
due to bad signal strength predictions:
%
due to bad signal strength predictions or observations (see \reffig{fig:wifiMultimodality}):
A pedestrian walks along a hallway, but bad model values indicate that his most likely position
is within a room right next to the hallway.
If the density (described by the particles) is dragged (completely) into this room,
@@ -493,7 +569,7 @@
we calculated each combination of the {\em 13 walks and six optimization strategies},
25 times, using 5000, 7500 and 10000 particles resulting in 75 runs per walk, 975 per strategy and 5850 in total.
%
Figure \ref{fig:overallSystemError} depicts the cumulative error distribution per optimization strategy,
\reffig{fig:overallSystemError} depicts the cumulative error distribution per optimization strategy,
resulting from all executions for each walk conducted with the smartphone.
While most values represent the expected results (more optimization yields better results),
@@ -518,14 +594,14 @@
fix and the accuracy indicated by the GPS usually was \SI{50}{\meter} and above.
Especially for {\em path 1}, the particle-filter often got stuck within the upper right outdoor area between both buildings
(see figure \ref{fig:allWalks}). Using the empirical parameters, \SI{40}{\percent} of all runs for this path got stuck at this location.
(see \reffig{fig:allWalks}). Using the empirical parameters, \SI{40}{\percent} of all runs for this path got stuck at this location.
{\em \optParamsAllAP{}} already reduced the risk to \SI{20}{\percent} and all other optimization strategies did not get stuck at all.
The same effect holds for all other conducted walks: The better the model optimization, the lower the risk of getting stuck somewhere along the path.
Varying the number of particles between 5000 and 10000 indicated only a minor increase in accuracy and slightly decreased the risk of getting stuck.
Comparing the error results within figure \ref{fig:modelPerformance} and \ref{fig:overallSystemError}, one can
Comparing the error results within \reffig{fig:modelPerformance} and \reffig{fig:overallSystemError}, one can
denote the positive impact of fusioning multiple sensors with a transition model based on the building's
actual floorplan. Outdoor regions indicated a very low signal quality (see section \ref{sec:wifiQuality}).
By omitting \docWIFI{} from the system's evaluation step, the IMU was able to
@@ -533,7 +609,35 @@
\begin{figure}
\input{gfx/overall-system-error.tex}
\begin{subfigure}{0.49\textwidth}
\input{gfx2/overall-system-error.tex}
\end{subfigure}
%
\begin{subfigure}{0.50\textwidth}
%OVERALL:2.62158 5.13701 11.1822 9.00261
%OVERALL:2.92524 6.00231 12.4425 10.6983
%OVERALL:1.98318 3.99259 7.92429 5.81281
%OVERALL:1.8647 3.86918 7.10482 5.62054
%OVERALL:1.60847 3.15739 6.13963 4.79148
%OVERALL:1.63617 3.34828 6.5379 5.12281
\footnotesize
\centering
\setlength{\tabcolsep}{0.25em} % for the horizontal padding
\begin{tabular}{|l|c|c|c|c|c|}
\hline
& \SI{25}{\percent} & median & \SI{75}{\percent} & avg & stuck \\\hline
\noOptEmpiric{} & \SI{2.6}{\meter} & \SI{5.1}{\meter} & \SI{11.2}{\meter} & \SI{9.0}{\meter} & \SI{22}{\percent} \\\hline
\optParamsAllAP{} & \SI{2.9}{\meter} & \SI{6.0}{\meter} & \SI{12.4}{\meter} & \SI{10.7}{\meter} & \SI{15}{\percent} \\\hline
\optParamsEachAP{} & \SI{1.9}{\meter} & \SI{4.0}{\meter} & \SI{7.9}{\meter} & \SI{5.8}{\meter} & \SI{5}{\percent} \\\hline
\optParamsPosEachAP{} & \SI{1.9}{\meter} & \SI{3.9}{\meter} & \SI{7.1}{\meter} & \SI{5.6}{\meter} & \SI{5}{\percent} \\\hline
\optPerFloor{} & \SI{1.6}{\meter} & \SI{3.2}{\meter} & \SI{6.1}{\meter} & \SI{4.8}{\meter} & \SI{4}{\percent} \\\hline
\optPerRegion{} & \SI{1.6}{\meter} & \SI{3.3}{\meter} & \SI{6.5}{\meter} & \SI{5.0}{\meter} & \SI{4}{\percent} \\\hline
\end{tabular}
\setlength{\tabcolsep}{1.0em} % reset the horizontal padding
\vspace{11.5mm}
\end{subfigure}
%
\caption{
Cumulative error distribution for each model when used within the final localization system from \refeq{eq:recursiveDensity}.
Despite some discussed exceptions, highly optimized models lead to lower localization errors.