Georeferencing of IP-Addresses

This chapter explains the functionality of the Python-Script which georeferences IP-Addresses out of the statistic data provided by the WebOffice application.

The Script is using the statistics which are generated by the WebOffice application. Those include the IP-Addresses of every user that called the application, from which the coordinates can be estimated. The in the process generated CSV-File can be used for further analysis. It is possible for example to generate a heat map of the access locations using ArcGIS Online or ArcGIS Pro.

 

icon_comment

The estimation of the coordinates to the IP-Addresses is no concern of data protection because the data is still anonymous and no individual is identified.

To configured user tracking (saving of personal data) use the application configuration in the WebOffice author standalone.

 

System requirements

All requirements are met if ArcMap is already installed. If not Python 2.7 as well as the Python-Libraries numpy and pandas need to be installed.

 

Program use

Pick a date for the analysis

Pick a date for the analysis

 

First of all the time span for the analysis needs to be identified.

The following entries are possible:

D => Analysis for a day

M => Analysis for a month

Q => Analysis for a quarter

Y => Analysis for a year

The required entry which is required next will differ depending of the chosen time span. The next step was performed for the analysis of a day (Entry => D")

 

Date entry and folder selection

Date entry and folder selection

 

The entered date needs to be confirmed with Y". If the wrong date was entered the script can be started over by typing N instead.

 

Folder confirmation

Folder confirmation

 

Next, the path for the statistics folder needs to be entered. If the data is located at the recommended default path which is C:\Tomcat\webapps\WebOffice\WEB-INF\work\statistics, only D for default needs to be entered. Otherwise the absolute path to the statistics folder needs to be entered. Furthermore, the output file will be named after the WebOffice application.

 

Reading of the statistic data

Reading of the statistic data

 

If the entered path is correct, the statistics data will be imported. This can take up to a few minutes depending on the time span chosen for the analysis. The progress can be viewed in the command prompt. After the importing process is finished, the total number of the found entries are displayed. Now the filtering process can be carried out. If there is no project name entered before pressing enter, all projects will be included in the analysis.

The following entries are possible:

Projectname

ProjectnameA,ProjektnameB,ProjectnameC, ...

No entry (all projects will be analysed)

 

Locating of the IP-Addresses

Locating of the IP-Addresses

 

After generating a definite IP-List, the total number of all entries are displayed. Now there is the possibility to locate the remaining IP-Addresses. For that there simply needs to be entered a Y for the next query.

 

Locating of the IP-Addresses done

Locating of the IP-Addresses done

 

The locating of the IP-Addresses can take from a few minutes up to an hour. Due to technical limitations only 150 IP-Addresses per minute can be located. After a package of 150 IP-Addresses was calculated, there is a one minute break before the next package can be estimated.

The CSV-File which was created during the process can now be further processed with ESRI products.

 

 

Inputfiles

As mentioned before, the path of the statistics folder is requested by the script. The default path therefore is C:\Tomcat\webapps\WebOffice\WEB-INF\work\statistics.

 

 

Outputfiles

The now described sub folders are stored at the same location where the script is located.

 

 Output

All output files will be stored in the the output folder.

The naming is as follows:

Day: <Name of wO-Application>_YYMMDD

Month: <Name of wO-Application>_YYMM

Quarter: <Name of wO-Application>_YY_Q

Year: <Name of wO-Application>_YY

 

 Log

The sub folder log contains the log file and the processed IP-Lists without Geocoding.

The naming is as follows:

Day: <Name of wO-Application>_LOG_YYMMDD

Month: <Name of wO-Application>_LOG_YYMM

Quarter: <Name of wO-Application>_LOG_YY_Q

Year: <Name of wO-Application>_LOG_YY

 

 

Further processing of the CSV-Files

After creating the IP-Address list with the corresponding coordinates, it is now possible to visualize that data in form of a map. Therefore various options are feasible which will be shown in the following sub chapters.

 

ArcGIS Online

Using ArcGIS Online it is possible with just a few steps to create a heat map. For this purpose ArcGIS Online needs to be opened in a Webbrowser and a new map must be created by clicking on Map.

 

Create a new map

Create a new map

 

The CSV-File can be inserted via drag&drop in the opened map view.

 

Drag&Drop of the CSV-File

Drag&Drop of the CSV-File

 

After the data is integrated in ArcGIS Online, different drawing styles can be chosen. When choosing the drawing style Heat Map and clicking Done, a heat map from the provided data will be created. That map can now be shared online and edited further.

 

Create heatmap

Create heatmap

 

ArcGIS Pro

Using ArcGIS Pro also gives the possibility to edit the CSV-File further. First of all a Folder Connection to the CSV-File needs to be created. After that the file can be dragged from the Catalog right into Contents. Clicking right on the inserted file opens a context menu.

 

Display XY Data in ArcGIS Pro

Display XY Data in ArcGIS Pro

 

Choosing Display XY Data... opens a dialog field in the functional area for creating a feature class.

 

Create feature class

Create feature class

 

If the desired coordinate system is chosen, the feature class can be created by clicking Run.

To create a heat map out of the generated feature class, the feature class needs to be selected first in the Contents area.

 

Select feature class

Select feature class

 

Now it is possible to edit the display of the signatures in the Symbology tab which can be found in the function area.

 

Display of signatures

Display of signatures

 

Clicking on Single Symbol opens a drop down menu where the entry Heat Map can be chosen.

 

Symbology Heat Map

Symbology Heat Map

 

The now created heat map is now ready to be published and shared on ArcGIS Online.

 

Heat Map in ArcGIS Pro

Heat Map in ArcGIS Pro

 

ArcMap 10.x

ArcMap also holds the possibility to edit the CSV-Data further. First of all a Folder Connection needs to be established to access the CSV-File. Once displayed in the Catalog the file can be dragged in the Table of Contents.

Clicking right on the inserted file opens a context menu.

 

Display XY Data in ArcMap

Display XY Data in ArcMap

 

Clicking on Display XY Data... opens a dialog window to create a layer.

 

Create Layer

Create Layer

 

A coordinate system needs to be selected if there is none already preselected. This can be done by clicking on the button Edit...

 

Select coordinate system

Select coordinate system

 

The choice of the desired coordinate system is confirmed by clicking OK.

 

Create Layer with desired coordinate system

Create Layer with desired coordinate system

 

The field description should now show information regarding the chosen coordinate system. If that is the case, the dialog can be affirmed by clicking OK which creates the layer.

 

The resulting layer is now ready to be further edited.