Table of Contents Author Guidelines Submit a Manuscript
The Scientific World Journal
Volume 2014, Article ID 616030, 6 pages
Research Article

Behavior Identification Based on Geotagged Photo Data Set

Software College, Northeastern University, Shenyang 110819, China

Received 6 December 2013; Accepted 22 January 2014; Published 24 February 2014

Academic Editors: Y. Cai and J. Zhang

Copyright © 2014 Guo-qi Liu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


The popularity of mobile devices has produced a set of image data with geographic information, time information, and text description information, which is called geotagged photo data set. The division of this kind of data by its behavior and the location not only can identify the user’s important location and daily behavior, but also helps users to sort the huge image data. This paper proposes a method to build an index based on multiple classification result, which can divide the data set multiple times and distribute labels to the data to build index according to the estimated probability of classification results in order to accomplish the identification of users’ important location and daily behaviors. This paper collects 1400 discrete sets of data as experimental data to verify the method proposed in this paper. The result of the experiment shows that the index and actual tagging results have a high inosculation.

1. Introduction

With the popularization and development of mobile devices, the speed of producing photos by people has become faster and faster. Now people often use Flickr, Instagram, or other photo sharing softwares to manage and store these photos. Because these photos are fragmented, users do not sort or classify them so that people always waste time searching the photo storage when browsing. Therefore, how to automatically classify enormous photos is an urgent problem to be solved. And at the same time people want to figure out user’s behavior rules according to the relationship between the user’s behavior and important location in the classification results. This is one of the heated issues of social computing researches at present.

Now many scholars take part in social computing researching on identification of user’s behavior rules and important location. At the initial stage, most of the researchers use the automatic positioning and sensing equipment to collect the user’s mobile information [1, 2] or infer the user’s mobile route from the signal of wireless network [3, 4]. With the automatic and continuous way of data collection, identification of important location and behavior needs a long time to analyze the movement of the user with high cost and ignores the protection of user’s privacy which leads to some restrictions to the follow-up research. Relatively, the gaining of the identification through discrete geotagged photo is more convenient with great quantity. This paper focuses on the photos offered by some photo sharing software similar to Instagram and Flickr. The photos have some data information such as time stamp, location information, and text label, which are called geotagged photo [5] data set.

Geotagged photo data in this paper are collected individually for 16 months ( containing photos, shooting position information, and time stamp. According to aforementioned characteristics, this paper proposes a multiple classification method. First, dividing the data set into several sets based on the different shooting cities; second, further dividing the data set based on crucial shooting spots; finally, classifying the result according to users’ behavior, thus a three-layer structure like “city-place-behavior” is figured out. Considering the overlap of the behaviors’ meaning and the complexity of the photos’ information, some tags as an index are given to the data with high possibility from kinds of classification results; in this way, the goal of sorting the data set is achieved.

This paper is organized as follows. Section 2 describes related work. Section 3 explains the multiple classification methods and related definitions. Section 4 presents each phase of the experiment process and result. Section 5 is the conclusion.

2. Related Work

Now, in the social computing research area, there are some researchers that choose the continuity mobile data to get human’s behavior and movement mode. Reference [6] uses DBSCAN algorithm to extract the areas where users upload photos frequently, which are defined as appealing areas in which some significant research will be conducted. Reference [7] uses support vector machine (SVM) to train data to get the photos which can represent the areas. Reference [8] uses an algorithm based on density to divide the area into grids to get the dense connected region. Reference [9] proposes a multidisciplinary method for the recognition of physical activity with the emphasis on feature extraction and selection processes, which are considered to be the most critical stages in identifying the main unknown activity discriminate elements. References [10, 11] identify users’ important places and behaviors through their long-term continuous GPS data (Rekimoto et al.). Reference [4] used the continuous location information uploaded by wireless network to get users’ location and movement mode. However, the high cost, the limited development of application, and disrupting users’ privacy are the fatal disadvantages of the method. Reference [12] tries to analyze the photos from mobile and compared to the known scenic then predict the user location and accomplish the route tracking according to the time and spatial information. And the geotagged photo data set used in this paper is uploaded by users themselves with a diversity of information, which is more conducive to research.

The collected data need to be divided into multiple classifications based on users’ places and behaviors. Now, the SVM classification algorithm [13] and RBF neural network [14] are usually used. We use LIBSVM [15] and the neural network tool supplied by Matlab to classify the data.

3. Solving Method and Related Definitions

The data this paper collected is uploaded by the users, so filtering the noise for the data set is not necessary. Classification algorithms are used to divide the data set, and the class labels consisted of users’ important places and behaviors, so we can identify user’s important places and behaviors by the classification result.

3.1. Classification Process

Figure 1 shows the classification process in this paper, it has 3 steps: “city” classification, “important places” classification, and “behavior” classification.

Figure 1: Classification flow chart.

“City” Classification. We can infer user’s behavior by the city where he is. The behaviors may be different for the difference between the city he lives and the city where he is traveling to or on business in. So we can divide the data set into 2 parts: one is the living part, and the other is the traveling part, then we can divide the living part in detail.

“Important Place” Classification. This step divides the living part by user’s important shooting places, like office, home, and restaurant. This work differentiates the same behaviors happening in different places.

“Behavior” Classification. This step divides the data set by user’s behaviors. The data set only contains time stamp and location information that can be used by classification algorithm, so we cannot divide the data set into many classes. There are only a few branches like “work,” “eating,” “go out,” and so on.

Through the above 3 steps, the data set is divided into several classes which consisted of “city,” “important place,” and “behavior”. The overall classification result is shown in Figure 2.

Figure 2: Classification chart.
3.2. Label Allocation

In general classification method, the classification result of each data is unique, but the unique result will lose some important information in the classification of geotagged photo data set. For instance, if the user goes out to do some work and then has a meal with clients, the data recording this event should contain both going out for a meal and going out for work; however, no matter which class this data is to be divided into, the other behavior information will be lost. So in order to keep the information intact, this paper adopts a method of the probability estimation. If the probability is higher in one classification than the threshold, we assign this class a label. As a result, each datum may have several labels.

When all data is labeled, an index is built. The classification of the data set and getting locations’ information and behaviors’ information together are achieved. It is more accurate to assume the users’ behaviors when they are uploading new data.

3.3. The Definitions

This section formally defines the concepts in this paper.

Definition 1 (city). It means the city where a user takes the photo. This information can be got from the GPS information of our application.

Definition 2 (place). It means the location where a user takes a photo, such as school, home. It also can be automatically accessed from our application.

Definition 3 (behavior). It means the behavior that the photo records.

Definition 4 (label). It consists of the three definitions mentioned above:

Each label consists of “city,” “place,” and “behavior”; for example, the user has a meal in a restaurant in a city.

Definition 5 (probabilitylabel). It means that the estimated probability of the data belongs to the class, if one considers this label of the data effective, and it is assigned to the label set of data.

Definition 6 (LabelSetphoto). It consists of the labels which are assigned to the data:

4. The Classification Process and Result

4.1. Data Acquisition and Explanation

The data in this paper is collected by the iPhone app developed by our lab. This app has photographed function and positioning function. When user takes a photo the app will automatically combine the photo, time information, and location information into a geotagged photo data. Users can upload the data into our lab’s server. This data set has more than 1400 records, collecting for 16 months. The collection process and data form are shown in Figure 3.

Figure 3: Collection process and data form.
4.2. Experimental Result
4.2.1. Initial Experimental Result

At the beginning of the experiment, data are divided into 6 classes according to the analysis of data set. They are “work,” “meal,” “go out,” “home,” “trip,” and “hometown”. We give each record a label to define which class it belongs to. The number of each class is shown in Figure 4.

Figure 4: Number of each class.

In the process of training, we use 5 cross validations to reduce errors and the accuracy is the average of the 5 results. The classification algorithms we used are SVM and RBF neural network. The accuracy of SVM is 77.59%, and the accuracy of RBF neural network is 71.72%. Based on the results, the accuracy of SVM is higher. So we choose to analyze the result of SVM classification result. As shown in Figure 5, the accuracy of “trip” and “hometown” is 100%. It is because we first divide the data set by the city information before the classification. The accuracy of “meal” and “home” is low, because the data of “home” is few and the data of “meal” is discrete in position, like “go out”.

Figure 5: The accuracy of classification.

We can know that the above method has some problems. The classes both have “home” which means place and “meal” and “work” which means behavior; the classification algorithm cannot work well, so we decide to use the multiple classifications method.

4.2.2. Multiple Classification

The structure of multiple classifications and the number of record of each class are shown in Figure 6.

Figure 6: The structure of multiple classifications.

The accuracy of the multiple classifications is shown in Figure 7.

Figure 7: The accuracy of the multiple classifications.

By using the multiple classifications method, the accuracy of the data in Shenyang is 71.7%, and the accuracy of the data in Siping is 82.4%, not including the data of “trip” class. It is clearly shown that the whole accuracy has been improved. But there are still some problems. We find that just one class’s label cannot show all the significance of a data, and it may lose some important information. So we consider assigning labels for each data.

4.2.3. The Result of Label Assigned

This paper uses the probability estimate of classification algorithm to compute the probability of the data belonging to the class; if the probability is higher than the threshold we assign this class’s label to the data. The process is shown in Algorithm 1.

Algorithm 1: Label assigned.

After we assigned labels to each data, this paper uses the labels to divide the data set and build the index. The structure is shown in Figure 6. In this way, this paper has sorted the data set and users can search and browse the data by the index. By analyzing this result we research the relationship between the user’s important places and behaviors. The actual application scene of this method is shown in Figure 8.

Figure 8: Application screenshots.

5. Conclusions

In our study, we propose the method called geotagged photo data set based classification algorithm, which can build an index to quickly sort the data. This method has practical significance for the mobile terminal users to manage the photos. The result of classification can identify the user’s important places and behaviors, so the study has a good research prospect.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.


This research was supported by the National Natural Science Foundation of China (Grant nos. 61374178 and 61202085), the Liaoning Provincial Natural Science Foundation of China (Grant no. 201202076), the Specialized Research Fund for the Doctoral Program of Higher Education (Grant no. 20120042120010) and the Ph.D. Start-up Foundation of Liaoning Province, China (Grant nos. 20111001, 20121001, and 20121002).


  1. Y. Zheng, X. Xie, and W.-Y. Ma, “Mining interesting locations and travel sequences from GPS trajectories,” in Proceedings of the 18th international conference on World Wide Web, ACM, 2009.
  2. B. McKercher and G. Lau, “Movement patterns of tourists within a destination,” Tourism Geographies, vol. 10, no. 3, pp. 355–374, 2008. View at Publisher · View at Google Scholar · View at Scopus
  3. J. Krumm and E. Horvitz, “LOCADIO: inferring motion and location from Wi-Fi signal strengths,” in Proceedings of the 1st Annual International Conference on Mobile and Ubiquitous Systems: Networking and Services (MOBIQUITOUS '04), pp. 4–13, August 2004. View at Publisher · View at Google Scholar · View at Scopus
  4. J. Rekimoto, T. Miyaki, and T. Ishizawa, “LifeTag: wiFi-based continuous location logging for life pattern analysis,” Location- and Context-Awareness, vol. 4718, pp. 35–49, 2007. View at Google Scholar · View at Scopus
  5. T. Rattenbury, N. Good, and M. Naaman, “Towards automatic extraction of event and place semantics from flickr tags,” in Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '07), pp. 103–110, ACM, July 2007. View at Publisher · View at Google Scholar · View at Scopus
  6. Y.-T. Zheng, Z.-J. Zha, and T.-S. Chua, “Mining travel patterns from geotagged photos,” ACM Transactions on Intelligent Systems and Technology (TIST), vol. 3.3, article 56, 2012. View at Google Scholar
  7. K. Yanai, H. Kawakubo, and B. Qiu, “A visual analysis of the relationship between word concepts and geographical locations,” in Proceedings of the ACM International Conference on Image and Video Retrieval (CIVR '09), pp. 92–99, ACM, July 2009. View at Publisher · View at Google Scholar · View at Scopus
  8. F. Giannotti, M. Nanni, F. Pinelli, and D. Pedreschi, “Trajectory pattern mining,” in Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '07), pp. 330–339, ACM, August 2007. View at Publisher · View at Google Scholar · View at Scopus
  9. O. Banos, M. Damas, H. Pomares, A. Prieto, and I. Rojas, “Daily living activity recognition based on statistical feature quality group selection,” Expert Systems with Applications, vol. 39, no. 9, pp. 8013–8021, 2012. View at Publisher · View at Google Scholar · View at Scopus
  10. V. W. Zheng, Y. Zheng, X. Xie, and Q. Yang, “Collaborative location and activity recommendations with GPS history data,” in Proceedings of the 19th International World Wide Web Conference (WWW '10), pp. 1029–1038, ACM, April 2010. View at Publisher · View at Google Scholar · View at Scopus
  11. L. Mingxia, S. Hu, L. Cuihua, J. Taisong, and Z. Quan, “Location and route tracking in university from photos without GPS information,” Advances in Multimedia Information Processing, vol. 7674, pp. 697–706, 2012. View at Google Scholar
  12. L. Liao, Location-based activity recognition [Ph.D. thesis], University of Washington, 2006.
  13. C. Cortes and V. Vapnik, “Support-vector networks,” Machine Learning, vol. 20, no. 3, pp. 273–297, 1995. View at Publisher · View at Google Scholar · View at Scopus
  14. D. Lowe and D. Broomhead, “Multivariable functional interpolation and adaptive networks,” Complex Systems, vol. 2, pp. 321–355, 1988. View at Google Scholar
  15. C.-C. Chang and C.-J. Lin, “LIBSVM: a Library for support vector machines,” ACM Transactions on Intelligent Systems and Technology, vol. 2, no. 3, article 27, 2011. View at Publisher · View at Google Scholar · View at Scopus