FHWS/OTHER2017
Archived
3
0
Fork 0
Code Issues 4 Pull Requests Releases Wiki Activity

current TeX

Browse Source
This commit is contained in:
FrankE
2017-05-02 18:31:36 +02:00
parent 47def9ad90
commit 1b24977a95
3 changed files with 72 additions and 61 deletions
Download Patch File Download Diff File Expand all files Collapse all files

109
tex/chapters/experiments.tex
View File

@@ -112,16 +112,18 @@
{\em\optParamsPosEachAP{}} does not need any prior knowledge and will optimize all six parameters
(3D position, \mTXP, \mPLE, \mWAF) based on the reference measurements.
{\em\optPerFloor{}} and {\em\optPerRegion{}} are just like \optParamsPosEachAP{} except that
there are several sub-models that are optimized for one floor / region instead of the whole building.
{\em\optPerFloor{}} and {\em\optPerRegion{}} are just like {\em \optParamsPosEachAP{}} except that
there are several sub-models, each of which is optimized for one floor / region instead of the whole building.
The chosen bounding boxes and resulting sub-models are depicted in figure \ref{fig:modelBBoxes}.
Figure \ref{fig:wifiModelError} shows the optimization results for all strategies, which are as expected:
The estimation error is indirectly proportional to the number of optimized parameters.
However, even with {\em \optPerRegion{}} the maximal error is relatively high due to some locations that do
not fit the model at all. Looking at the optimization results for \mTXP{}, \mPLE{} and \mWAF{} supports
not fit the model at all, which is shown in figure \ref{fig:wifiModelErrorB}.
%
Looking at the optimization results for \mTXP{}, \mPLE{} and \mWAF{} supports
this finding. While the median for those values based on all optimized transmitters is totally sane
(\SI{-42}{\decibel{}m}, \SI{2.4}, \SI{-6.0}{\decibel}), the minimum and maximum values are clearly outside of the physically possible range.
(\SI{-42}{\decibel{}m}, \SI{2.4}, \SI{-6.0}{\decibel}), the minimum and maximum values are far beyond the physically possible range.
The same holds for the estimated transmitter position when using {\em \optParamsPosEachAP{}}: The median
distance between estimated and real position is $\sim$\SI{8}{\meter} and the maximum $\sim$\SI{27}{\meter}.
@@ -134,6 +136,7 @@
\begin{subfigure}{0.23\textwidth}
\input{gfx/wifiMaxErrorNN_opt0.tex}
\caption{\em \noOptEmpiric{}}
\label{fig:wifiModelErrorA}
\end{subfigure}
%\begin{subfigure}{0.25\textwidth}
% \input{gfx/wifiMaxErrorNN_opt3.tex}
@@ -141,6 +144,7 @@
\begin{subfigure}{0.23\textwidth}
\input{gfx/wifiMaxErrorNN_opt5.tex}
\caption{\em \optPerRegion{}}
\label{fig:wifiModelErrorB}
\end{subfigure}
\caption{
Comparison between different optimization strategies by examining the error (in \decibel) at each reference measurement.
@@ -169,8 +173,8 @@
%Pos: cnt(34) min(3.032438) max(26.767128) range(23.734690) med(7.342710) avg(8.571227) stdDev(4.801449)
While {\em \optPerRegion{}} is able to overcome the indoor vs. outdoor issues depicted in
figure \ref{fig:wifiIndoorOutdoor} e.g. by using a separate bounding box just for the outdoor area,
it obviously requires a profound prior knowledge when selecting the individual regions for the sub-model.
figure \ref{fig:wifiIndoorOutdoor}, by using a separate bounding box just for the outdoor area,
it obviously requires a profound prior knowledge to correctly select the individual regions for the sub-model.
%Such issues can only be fixed using more appropriate models that consider walls and other obstacles.
% das ist wohl zu viel
@@ -183,16 +187,18 @@
% -------------------------------- number of fingerprints -------------------------------- %
\hspace{3mm} % HACK...
As we try to minimize the system's setup time as much as possible, we need to determine
the amount of necessary reference measurements for the optimization to produce viable model parameters.
Depending on the chosen model and thus the number of to-be-optimized parameters, more measurements are required.
the amount of necessary reference measurements for the optimization to produce robust model parameters.
Depending on the chosen model, and thus the number of to-be-optimized parameters, more measurements will be required.
While there was almost no difference between using 121 or 30 reference measurements for
{\em \optParamsAllAP{}} and {\em \optParamsEachAP{}}
(average \SIrange{5.3}{5.4}{\decibel} and \SIrange{4.5}{5.0}{\decibel}),
(average error changed from \SIrange{5.3}{5.4}{\decibel} and \SIrange{4.5}{5.0}{\decibel}, respectively),
{\em \optPerRegion{}} is highly affected
(average \SIrange{2.0}{6.2}{\decibel}), as it needs at least a certain number of measurements for each
of its regions for the optimization to converge.
(average error changed from \SIrange{2.0}{6.2}{\decibel}), as it needs at least a certain number of measurements within each
region for the optimization to converge.
\begin{figure}
\begin{subfigure}{0.49\textwidth}
@@ -228,17 +234,18 @@
The error is determined by using the (absolute) difference between expected signal strength and
the optimized model's corresponding prediction for all of the 121 reference measurements.
%
Considering only 60 of the 121 scans (\SI{50}{\percent}) yields a slightly increasing model error and still provides good results.
Considering only 60 of the 121 scans (\SI{50}{\percent}) yields a slightly increasing model error but still provides good results.
While using only \SI{25}{\percent} of the reference measurements increases the error rapidly,
for \SI{75}{\percent} of the 121 considered cases the estimation is still better than using just empiric values without optimization.
The extremely large outlier depicted in the lower half of figure \ref{fig:wifiNumFingerprints} (red line) relates to one
sub-model with only one assigned reference measurement, where the optimized result is unable to predict values
for the rest of the sub-model's region. \todo{versteht man das?}
for \SI{75}{\percent} of the 121 considered error-values, the estimation is still better than using just empiric values without optimization.
%The extremely large outlier depicted in the right half of figure \ref{fig:wifiNumFingerprints} (red line) relates to one
%sub-model with only one assigned reference measurement, where the optimized result is unable to predict values
%for the rest of the sub-model's region. \todo{versteht man das?}
Additionally we examined the impact of skipping reference measurements for difficult locations
like staircases, surrounded by steel-enforced concrete. While this slightly decreases the
estimation error for all other positions (hallway, etc) as expected, the error within the skipped locations is dramatically
increasing (see lower half of figure \ref{fig:wifiNumFingerprints}). It is thus highly recommended
increasing (see right half of figure \ref{fig:wifiNumFingerprints}). It is thus highly recommended
to also perform reference measurements for locations, that are expected to strongly deviate (signal strength)
from their surroundings.
@@ -310,7 +317,7 @@
where $\mu_{i,\mPosVec}$ is the signal strength for \docAP{} $i$
at location $\mPosVec$ returned from the to-be-examined prediction model.
For all comparisons we use a constant uncertainty $\sigma = $\SI{8}{\decibel}.
For all comparisons we use a constant uncertainty $\sigma = \SI{8}{\decibel}$.
The quality of the estimated location is determined by using the Euclidean distance between estimation
$\mPosVec^*$ and the pedestrian's ground truth position at the time the scan $\mRssiVec$
@@ -326,22 +333,20 @@
using linear interpolation between adjacent markers.
% walked paths
\begin{figure}[t]
{
\centering
\input{gfx/all_walks.tex}
}
\label{fig:allWalks}
\begin{figure}
\centering
\input{gfx/all_walks.tex}
\caption{
Overview of all conducted paths.
Outdoor areas are marked in green.
}
\label{fig:allWalks}
\end{figure}
\begin{figure}[b]
\begin{figure}
\input{gfx/modelPerformance_meter.tex}
\caption{
Error between ground truth and estimation using \refeq{eq:bestWiFiPos} depending
Cumulative error distribution between ground truth and location estimation using \refeq{eq:bestWiFiPos} depending
on the underlying signal strength prediction model.
Extremely high errors between the \SIrange{90}{100}{\percent} quartile are related to bad \docWIFI{}
coverage within outdoor areas (see figure \ref{fig:wifiIndoorOutdoor}).
@@ -353,7 +358,7 @@
%for each \docWIFI{} measurement within the recorded paths (3756 \docAPshort{} scans in total)
%against the corresponding ground-truth, which indicates the absolute 3D error in meter.
The position estimation for each \docWIFI{} measurement within the recorded walks (3756 scans in total)
is compared against its corresponding ground-truth, indicating the 3D error.
is compared against its corresponding ground-truth, indicating the 3D distance error.
The resulting cumulative error distribution can be seen in figure \ref{fig:modelPerformance}.
The quality of the location estimation directly scales with the quality of the signal strength prediction model.
However, as discussed earlier, the maximal estimation error might increase for some setups.
@@ -366,11 +371,11 @@
% -------------------------------- plots indicating walk issues -------------------------------- %
\begin{figure}[t]
\begin{figure}
\input{gfx/wifiMultimodality.tex}
\caption{
Location probability \refeq{eq:bestWiFiPos} for three scans. Higher color intensities are more likely.
Ideally, places near the ground truth (black) are highly highly probable (green).
Ideally, places near the black ground truth are highly highly probable (green).
Often, other locations are just as likely as the ground truth (blue),
or the location with the highest probability does not match at all (red).
}
@@ -379,33 +384,35 @@
Figure \ref{fig:wifiMultimodality} depicts aforementioned issues of multimodal (blue) or wrong (red) location
estimations. Filtering (\refeq{eq:recursiveDensity}) thus is highly recommended, as minor errors are compensated
using other sensors and/or a movement model that prevents the estimation from leaping within the building.
However, if wrong sensor values (red) are observed for longer time periods, even filtering will produce erroneous
using other sensors or a movement model that prevents the estimation from leaping within the building.
However, if wrong sensor values are observed for longer time periods, even filtering will produce erroneous
results and might get stranded (density is trapped e.g. within a room),
as the movement model is constrained by the actual floorplan.
% -------------------------------- other distributions, unseen APs, etc -------------------------------- %
\hspace{3mm}%hack
To reduce the amount such of misclassifications, where other locations within the building are
as likely as the pedestrians actual location, we examined various approaches.
Unfortunately, most of which did not provided a viable enhancement under all conditions for the performed walks.
as likely as the pedestrian's actual location, we examined various approaches.
Unfortunately, most of which did not provide a viable enhancement under all conditions for the performed walks.
The misclassification-rate is determined by counting the amount of (random) locations within
the building that produce a similar probability \refeq{eq:wifiProb} as the actual ground-truth
position.
One possibility to dissolve such an equal \docWIFI{}-likelihood between two (or more) locations is,
to not only consider the \docAPshort{}s seen by the Smartphone, but also the \docAPshort{}s not seen
by the Smartphone. This additional information can be used to rule out all locations where this
\docAP{} should be received (high signal strength from the prediction model).
to not only consider the \docAPshort{}s seen by the smartphone, but also the \docAPshort{}s not seen
by the smartphone. This additional information can be used to rule out all locations where this unseen
\docAP{} should have be received (high signal strength from the prediction model).
% There might be an \docAP{} that should be visible at the other locations. However,
%as the Smartphone did not see this \docAPshort{} the other location can be ruled out.
While this works in theory, evaluations revealed several issues:
There is a chance that even a nearby \docAPshort{} is unseen during a scan due to packet collisions or
temporal effects within the surrounding. It thus might make sense to opt-out other locations
only, if at least two \docAPshort{}s are missing. On the other hand, this obviously requires (at least)
only, if at least two \docAPshort{}s are missing. On the other hand, this obviously demands for (at least)
two \docAPshort{}s to actually be different between the two locations, and requires a lot of permanently
installed transmitters to work out.
@@ -415,14 +422,14 @@
%%Including \docAPshort{}s unseen by the Smartphone thus often increases the estimation error instead
%%of fixing the multimodality.
To sum up, while some situations, e.g. outdoors, could greatly be improved,
To sum up, while some situations, e.g. outdoors, could be improved,
many other situations are deteriorated, especially when some transmitters are (temporarily)
attenuated by ambient conditions like concrete walls.
We therefore examined variations of the probability calculation from \refeq{eq:wifiProb}.
Despite the results show in \cite{PotentialRisks}, removing weak \docAPshort{}s from $\mRssiVec{}$
yielded similar results. While some estimations were improved, the overall estimation error increased
yielded similar results. While some estimations were improved, the overall error increased
for our walks, as there are many situations where only a handful \docAP{}s can be seen.
Removing this (valid) information will highly increase the error for such situations.
@@ -469,7 +476,7 @@
After examining the \docWIFI{} component on its own, we will now analyze the impact of aforementioned model
optimizations on our smartphone-based indoor localization system described in section \ref{sec:system}.
Due to other sensors and the transition constraints from the buildings floorplan, we expect the
Due to transition constraints from the buildings floorplan, we expect the
posterior density to often get stuck when the \docWIFI{} component provides erroneous estimations
due to bad signal strength predictions:
%
@@ -479,13 +486,14 @@
the IMU indicates no change in direction (pedestrian walks straight),
and the room has only one single door, the density is trapped within this room.
%
Such problems can often be solved by simply using more particles to describe the posterior.
While such problems can often be solved by simply using more particles to describe the posterior,
smartphone use-cases are usually performance- and battery limited.
As particle filtering from \refeq{eq:recursiveDensity} is a random process with varying output,
we calculated each combination of the {\em 13 walks and optimization strategy},
we calculated each combination of the {\em 13 walks and six optimization strategies},
25 times, using 5000, 7500 and 10000 particles resulting in 75 runs per walk, 975 per strategy and 5850 in total.
%
Figure \ref{fig:overallSystemError} depicts the error distribution per optimization strategy,
Figure \ref{fig:overallSystemError} depicts the cumulative error distribution per optimization strategy,
resulting from all executions for each walk conducted with the smartphone.
While most values represent the expected results (more optimization yields better results),
@@ -506,21 +514,22 @@
(metal-framed doors) the error is slightly increased and retained for some time until the density stabilizes itself.
Such situations should be mitigated by the smartphone's GPS sensor. However, within our testing walks, the GPS
did rarely provide accurate measurements, as the outdoor-time was to short for the sensor to receive a valid
fix. The accuracy indicated by the GPS usually was \SI{50}{\meter} and above.
did rarely provide accurate measurements, as the outdoor-time was too short for the sensor to receive a valid
fix and the accuracy indicated by the GPS usually was \SI{50}{\meter} and above.
Especially for {\em path 1}, the particle-filter often got stuck within the upper right outdoor area between both buildings
(see figure \ref{fig:allWalks}). Using the empirical parameters, \SI{40}{\percent} of all runs for this path got stuck at this location.
While {\em \optParamsAllAP{}} already reduced the risk to \SI{20}{\percent}, all other optimization strategies did not get stuck at all.
{\em \optParamsAllAP{}} already reduced the risk to \SI{20}{\percent} and all other optimization strategies did not get stuck at all.
The same effect holds for all other conducted walks: The better the model optimization, the lower the risk of getting stuck somewhere along the path.
Varying the number of particles between 5000 and 10000 indicated only a minor increase in accuracy and slightly decreased the risk of getting stuck.
Comparing the results of figure \ref{fig:modelPerformance} and \ref{fig:overallSystemError} one can
denote the positive impact of fusioning multiple sensors with a transition model based on the buildings
actual floorplan. Especially the outdoor regions, or other areas with disabled \docWIFI{} component highly
profit from the data provided by the smartphones IMU, which prevents the estimation from getting lost.
Comparing the error results within figure \ref{fig:modelPerformance} and \ref{fig:overallSystemError}, one can
denote the positive impact of fusioning multiple sensors with a transition model based on the building's
actual floorplan. Outdoor regions indicated a very low signal quality (see section \ref{sec:wifiQuality}).
By omitting \docWIFI{} from the system's evaluation step, the IMU was able to
keep the pedestrian's current heading until the signal quality reached sane levels again.
\begin{figure}