Table of Contents
ISRN Machine Vision
Volume 2012, Article ID 872687, 11 pages
http://dx.doi.org/10.5402/2012/872687
Research Article

Spatiotemporal Relations and Modeling Motion Classes by Combined Topological and Directional Relations Method

1Department of Mathematics, MIA Laboratory, University of La Rochelle, 17000 La Rochelle, France
2MIA Laboratory and Computer Science Department, University of La Rochelle, 17000 La Rochelle, France

Received 29 December 2011; Accepted 23 January 2012

Academic Editor: J. Alvarez-Borrego

Copyright © 2012 Nadeem Salamat and El-hadi Zahzah. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Defining spatiotemporal relations and modeling motion events are emerging issues of current research. Motion events are the subclasses of spatiotemporal relations, where stable and unstable spatio-temporal topological relations and temporal order of occurrence of a primitive event play an important role. In this paper, we proposed a theory of spatio-temporal relations based on topological and orientation perspective. This theory characterized the spatiotemporal relations into different classes according to the application domain and topological stability. This proposes a common sense reasoning and modeling motion events in diverse application with the motion classes as primitives, which describe change in orientation and topological relations model. Orientation information is added to remove the locative symmetry of topological relations from motion events, and these events are defined as a systematic way. This will help to improve the understanding of spatial scenario in spatiotemporal applications.

1. Introduction

Automatic event detection is gaining more and more attention in computer vision and video researchers community. Visual scene description takes into account ontological viewpoint of relative object positions. It is sufficient to emphasize the model of moving object’s spatial relations such as modeling video events [1, 2]. Modeling spatiotemporal relations between moving objects involves the modeling of motion events such as durative events. These events are the union of primitive events, which hold for each snapshot during the interval with a particular temporal order.

Spatiotemporal features could be used for modeling the spatiotemporal events [3]. Defining spatiotemporal relations have two main domains of research: spatiotemporal object and spatiotemporal relations modeling. Cuboid object approximation or three-dimensional geometry is used to model the former and for lateral two-dimensional objects occupy different spatial locations at different time points [4]. Several types of logics for mechanizing the spatiotemporal relations and reasoning process are used like interval temporal logic [5, 6], point temporal logic [7], and propositional model logic [8].

The point temporal logic supports instantaneous snapshots of the world. A snapshot represents the current situation, and a spatiotemporal relation is defined if a particular spatial relation holds for every snapshot during that interval. It is considered that time and space are bounded to each other. Spatiotemporal relations are modeled between moving objects by taking transaction from one snapshot to the next snapshot.

Dimiter Vakarelov, in [9, 10] provided strong mathematical and logical bases for defining the general spatiotemporal topological relations and divided them into two categories: (i) stable spatiotemporal topological relations and (ii) unstable spatiotemporal topological relations. A stable spatiotemporal topological relation is a relation which holds for every frame or snapshot in the interval and unstable spatiotemporal topological relations are those which hold at least for one snapshot during the temporal interval. Some spatiotemporal relations are strictly stable such as disjoint (D), nontangent proper part (NTPP), nontangent proper part inverse (NTPPI), equal (EQ), and others may be stable or unstable like meet (M), partially overlap (PO), tangent proper part (TPP), and tangent proper part inverse (TPPI). This provides a way to use the spatiotemporal relations in linguistics and their use as motion events modeling.

Most of the existing theories of spatiotemporal relations are domain-based, where domain knowledge imposes the conditions to a spatiotemporal relation to be topologically stable or unstable. A domain where the spatiotemporal relation is topologically stable, there the directional or distance relations are unstable such as spatiotemporal relations on the road networks. In defining motion events or verbs which represents the transitive movement, stability or unstability of topological relations plays important role and directional relations along with the topological relations remove certain symmetries and helps the user to understand real-scene situation. Consider the following two examples.(1)Mr. John (object, name of a person) crosses (relation) the football ground (object);(2)Mr. John (object, name of a person) crosses (relation) the football ground (object) from north to south.

In proposition 1, there is no confusion about the topological relation that object 𝐴 (John) has certain topological relation with object 𝐵 (football ground). But there is a symmetry about the directions, and user did not know the exact direction from object 𝐴 to object 𝐵 before and after the occurrence of cross event. But in proposition 2, when directional constraints are added, they remove the confusion about directions and symmetry of topological relations that object 𝐴 (john) crosses the object 𝐵 (ground) from north to south. It justifies that how the topological relations in two objects change, and what was the temporal order of occurrence the primitive events.

In our approach, we used a method of combined topological and directional (CTD) relations [11], more suitable for reasoning about the moving objects and developing the motion events. Situation is represented by relationship between the considered entities. It is natural to represent the information using relations. Events can be expressed by interpreting collective behavior of physical objects over a certain period of time. The main focus of this work is to formalize the spatiotemporal relations and the spatiotemporal events in a systematic way.

This paper is arranged as follows. Related work is discussed in Section 2 and Section 3 composed of preliminary definitions. Section 4 explains the combined topological and directional relations method, spatiotemporal relations are defined in Sections 5 and 6 compose on geometric representation of some motion events and these motion events are defined in Section 7. Section 8 concludes the paper.

2. Related Work

A moving object occupies different positions at different time instants. Relative motion means that the object changes its position with respect to another object. This relative motion can be studied through different aspects of space, and spatial relations are one of them. A set of spatial relations which hold for one snapshot is considered as a primitive event, then spatial relations between moving objects for an interval are characterized as spatiotemporal relations or spatiotemporal events.

Commonly adopted approaches are qualitative and domain-based such as qualitative trajectory calculus (QTC) [12, 13]. This describes relation between moving point objects. Hornsby and Egenhofer [14] modeled the different spatiotemporal relations between moving objects on road network. All these relations represent certain class of motion and objects are approximated as point objects. When objects are under motion, especially on road networks, the relations are purely directional relations, where the objects change their position, but do not change the topological structure of scene.

A mereotopological approach is extend to define spatiotemporal relations and a notation of temporal slice is used, where temporal slice is called an episode of history for a given interval [15, 16]. The primitive events can be defined using the Allen’s temporal logic and defining relation 𝑜𝑙𝑑𝑠(P,i) (property P 𝑜𝑙𝑑𝑠 during the time interval i) [17]. In this method, interval temporal logic is used, and a primitive temporal interval is defined, the smallest interval where the relation does not change. For composite events another property “occurs,” defined as occurs(e,i) = event 𝑒, occurs during the interval 𝑖, and different hold predicates are combined through logical connectors in a sequential order.

Ma and Mc Kevitt [18] described a method based on continuous transection from one state to another state. In this approach, topological relations are computed by the 9-intersection method [19]. This method supports instantaneous point temporal logic, which detects only changes in topological structure of scene at different instants of time. This method is based on point set topology and uses the snapshot model for spatiotemporal data [2022]. This model of topological relations is used by Muller [23, 24] to model the motion events or motion classes. They model the different motion events which involve the topological changes at each analysis frame. Spatiotemporal relations between moving objects are also effected by the environment regarding its application domain such as modeling the relation cross, enter, leave shows that one object is only on concept level, that is, a region of interest. They are defined for a network, visual tracking, image understanding and activity recognition, or freely moving objects like modeling movement behavior of animals.

In [10] provided the strong mathematical and logical bases for defining the general spatiotemporal topological relations and divided them into two categories, namely, stable and unstable spatiotemporal relations. Both stable and unstable spatiotemporal relations play an important role in modeling the spatiotemporal events. World is represented as situation (a primitive event), and action is simply a function from one situation to another. A single snapshot represents a primitive event at an instant 𝑡. Events are embedded in time, they have temporal boundaries, they have their relationship to time. They do not occupy space, but they are related to space.

Spatiotemporal events are defined as composite events, how their different parts (primitive events) are interrelated. A property 𝑜𝑙𝑑𝑠_𝑎𝑡(P,t) (property P holds at time instant 𝑡) is used along with the instantaneous temporal logic. The primitive events are defined for each snapshot during an interval 𝑇 using the Allen’s temporal logic and defining the relation 𝑜𝑙𝑑𝑠(P,T) (property P holds during the time interval T) [17], and a relationship between holds and 𝑜𝑙𝑑𝑠_at can be represented as 𝑜𝑙𝑑𝑠(𝑃,𝑇)=forall𝑡𝑇, 𝑜𝑙𝑑𝑠_𝑎𝑡(𝑃,𝑡). This provides us that a property 𝑃 holds for an interval 𝑇 if it holds for every point during the interval. In this method, interval temporal logic is used, and a primitive temporal interval is defined, if 𝑇 is a zero duration interval, then it represents a snapshot.

Motion events are the subclasses of spatiotemporal relations with a temporal ordering in a primitive actions. Motion events, they do not formulate the necessitate of a calculus, they are only logical representation and temporal ordering of existence of different primitive events. Modeling the motion events, where property (P) changes at each instant, it is more suitable to use the sequential logic and use relation, 𝑠𝑒𝑞_𝑒𝑣𝑒(𝑡,𝑒1,𝑒2,𝑠) (event 𝑒1 occurs before 𝑒2 in S during time 𝑡). Composite events are the initial conditions dependant, when an initial primitive event occurs at a certain time point 𝑡0, it set up the superclass and name of the possible composite event to be happening.

Topological relations have a certain type of locative symmetries, they do not explain the symmetric location of path and motion direction of argument objects. To remove the symmetry of spatial relations about the locative perspective, relevant spatial orientation is added. In language semantics, motion events are divided into three classes: an initial, the median, and terminal [25]. Some sentences can be explained with the help of a single directional relation such as enter, release, touch, and some need two directions like cross and graze.

We used CTD method [11] to develop such motion events, where topological components play role for defining the motion events, directional components are used to overcome the locative symmetries and locative perspective and for other class of motion events, topological components can be used for controlling variables, and directional components become important. We hope this paper will create a bridge between the two approaches of modeling the spatiotemporal events, approach based on interval logic, and that of point logic.

3. Preliminary Definitions

In this section, we recall some basic definitions which are frequently used throughout the remainder of the paper.Fuzzy set: a fuzzy set 𝐴 in a set 𝑋 is a set of pairs(𝑋,𝜇𝐴(𝑥))suchthat𝐴=𝑥,𝜇𝐴(𝑥)𝑥𝑋.(1)Fuzzy membership function: a membership function 𝜇 in a set 𝑋 is a function 𝜇𝑋[0,1]. Different fuzzy membership functions are proposed according to the requirements of the applications. For instance, Trapezoidal membership function is defined as𝜇(𝑥;𝛼,𝛽,𝛾,𝛿)=maxmin𝑥𝛼𝛽𝛼,1,𝛿𝑥𝛿𝛾,0,(2) it is written as 𝜇(𝛼,𝛽,𝛾,𝛿)(𝑥), where 𝑥,𝛼,𝛽,𝛾,𝛿𝛼<𝛽𝛾<𝛿.Force histogram: the force histogram attaches a weight to the argument object 𝐴 that this lies after 𝐵 in direction 𝜃, it is defined as 𝐅𝐴𝐵(𝜃)=+𝐹𝜃,𝐴𝜃(𝑣),𝐵𝜃(𝑣)𝑑𝑣.(3) The definition of force histogram 𝐅𝐴𝐵(𝜃), directly depends upon the definition of real-valued functions 𝜙,𝑓, and 𝐹 used for the treatment of points, segments, and longitudinal sections, respectively [26]. These functions are defined as𝜙𝑟1(𝑦)=𝑦𝑟𝑓𝑥if𝑦>00otherwise,𝐼,𝑦𝜃𝐼𝐽,𝑧𝐽=𝑥𝐼+𝑦𝜃𝐼𝐽+𝑧𝐽𝑥𝐼+𝑦𝜃𝐼𝐽𝑧𝐽0𝐹𝜙(𝑢𝑤)𝑑𝑤𝑑𝑢,𝜃,𝐴𝜃(𝑣),𝐵𝜃=(𝑣)𝑖=1𝑛,𝑗=1𝑚𝑓𝑥𝐼𝑖,𝑦𝜃𝐼𝑖𝐽𝑗,𝑧𝐽𝑗,(4) where 𝑛and 𝑚 represent the number of segments of object 𝐴 and object 𝐵, respectively, and variables (x, y, z) are explained in Figure 1. These are the definitions of Force histograms, directly depending upon the definition of function 𝜙. 𝐅𝐴𝐵(𝜃) is actually a real-valued function.

872687.fig.001
Figure 1: Oriented line Δ𝜃(𝑣), segment as in case of object 𝐵, longitudinal section as in case of object 𝐴[27].

4. Combined Topological and Directional Relations Method

In this section, we explain different steps of the combined topological and directional relations method. This explains different terms used in computation of combined topological and directional relations.

4.1. Oriented Lines, Segments, and Longitudinal Sections

Let 𝐴 and 𝐵 be two spatial objects and (𝑣,𝜃), where 𝑣 is any real number and 𝜃[0,2𝜋]. Δ𝜃(𝑣) is an oriented line at orientation angle 𝜃, and 𝐴Δ𝜃(𝑣) is the intersection of object 𝐴 and oriented line Δ𝜃(𝑣). It is denoted by 𝐴𝜃(𝑣), called segment of object 𝐴 and length of its projection interval on x-axis is 𝑥. Similarly for object 𝐵, where 𝐵Δ𝜃(𝑣)=𝐵𝜃(𝑣) is segment and length of its projection interval on x-axis is 𝑧, 𝑦 is the difference between the minimum of projection points of 𝐴Δ𝜃(𝑣) and maximum value of projection points of 𝐵Δ𝜃(𝑣)(for details [27]).

In case of polygonal object approximation (𝑥,𝑦,𝑧) can be calculated from intersecting points of line and object boundary, oriented lines are considered which passes through at least one vertex of two polygons. If there exist more than one segment, then it is called longitudinal section as in case of 𝐴𝜃(𝑣) in Figure 1.

4.2. Allen Temporal Relations in Spatial Domain and Fuzziness

Allen [5] introduced the 13 jointly exhaustive and pairwise disjoint (JEPD) interval relations. These relations are 𝒜={<,𝑚,𝑜,𝑠,𝑓,𝑑,𝑒𝑞,𝑑𝑖,𝑓𝑖,𝑠𝑖,𝑜𝑖,𝑚𝑖,>} with meanings before, meet, overlap, start, finish, during, equal, during_by, finish_by, start_by, overlap_by, meet_by, and after. All the Allen relations in space are conceptually illustrated in Figure 2.

872687.fig.002
Figure 2: Black segment represents the reference object, and gray segment represents argument object.

These relations have a rich support for the topological relations and represent the eight topological relations in one-dimensional spatial domain. Fuzzy Allen relations are used to represent the fuzzy topological relations, where vagueness or fuzziness is represented at the relation’s level.

Fuzzification process of Allen relations do not depend upon particular choice of fuzzy membership function. Trapezoidal membership function is used due to flexibility in shape. Let 𝑟(𝐼,𝐽) be an Allen relation between segments 𝐼 (segment of an argument object) and 𝐽 (segment of an reference object), 𝑟 is the distance between 𝑟(𝐼,𝐽) and its conceptional neighborhood. We consider a fuzzy membership function 𝜇𝑟[01]. The fuzzy Allen relations defined in [28] as𝑓<(𝐼,𝐽)=𝜇(,,𝑏3𝑎/2,𝑏𝑎)𝑓(𝑦)>(𝐼,𝐽)=𝜇(0,𝑎/2,,)𝑓(𝑦)𝑚(𝐼,𝐽)=𝜇(𝑏3𝑎/2,𝑏𝑎,𝑏𝑎,𝑏𝑎/2)𝑓(𝑦)𝑚𝑖(𝐼,𝐽)=𝜇(𝑎/2,0,0,𝑎/2)𝑓(𝑦)𝑂(𝐼,𝐽)=𝜇(𝑏𝑎,𝑏𝑎/2,𝑏𝑎/2,𝑏)𝑓(𝑦)𝑜𝑖(𝐼,𝐽)=𝜇(𝑎,𝑎/2,𝑎/2,0)𝑓(𝑦)𝑓(𝜇𝐼,𝐽)=min((𝑏+𝑎)/2,𝑎,𝑎,+)(𝜇𝑦),(3𝑎/2,𝑎,𝑎,𝑎/2)𝜇(𝑦),(,,𝑧/2,𝑧)𝑓(𝑥)𝑓𝑖𝜇(𝐼,𝐽)=min𝑏𝑎/2,𝑏,𝑏,𝑏+𝑎/2𝜇(𝑦),(,,𝑏,(𝑏+𝑎)/2)𝜇(𝑦),(𝑧,2𝑧,+,+)𝑓(𝑥)𝑠𝜇(𝐼,𝐽)=min𝑏𝑎/2,𝑏,𝑏,𝑏+𝑎/2𝜇(𝑦),(,𝑏,(𝑏+𝑎)/2)𝜇(𝑦),(,,𝑧/2,𝑧)𝑓(𝑥)𝑠𝑖𝜇(𝐼,𝐽)=min((𝑏+𝑎)/2,𝑎,𝑎,+)𝜇(𝑦),(3𝑎/2,𝑎,𝑎,𝑎/2)𝜇(𝑦),(𝑧,2𝑧,+,+)𝑓(𝑥)𝑑𝜇(𝐼,𝐽)=min(𝑏,𝑏+𝑎/2,3𝑎/2,𝑎)𝜇(𝑦),(,,𝑧/2,𝑧)𝑓(𝑥)𝑑𝑖𝜇(𝐼,𝐽)=min(𝑏,𝑏+𝑎/2,3𝑎/2,𝑎)𝜇(𝑦),(𝑧,2𝑧,+,+),(𝑥)(5) where 𝑎=min(𝑥,𝑧),𝑏=max(𝑥,𝑧), 𝑥 is the length of segment (𝐼), 𝑧 is the length of segment (𝐽), and (𝑥,𝑦,𝑧) are computed as described in Section 4.1.

Most of relations are defined by one membership like 𝑑(𝑑𝑢𝑟𝑖𝑛𝑔), 𝑑𝑖(𝑑𝑢𝑟𝑖𝑛𝑔_𝑏𝑦), 𝑓(𝑓𝑖𝑛𝑖𝑠), and 𝑓𝑖(𝑓𝑖𝑛𝑖𝑠𝑒𝑑_𝑏𝑦). In fuzzy set theory, sum of all the relations is one, this gives the definition for fuzzy relation 𝑒𝑞𝑢𝑎𝑙. These are the topological relations which represent the fuzziness at relation’s level, for example, here Meet topological relation is represented based on nearness, and length of the smaller interval defines the smooth transition between the Meet(Meet_by) and before(after) relation. In spatial domain, before(after) are called the disjoint topological relations. These relations have the following properties:𝑓<(𝜃)=𝑓>(𝜃+𝜋),𝑓𝑚(𝜃)=𝑓𝑚𝑖𝑓(𝜃+𝜋),𝑜(𝜃)=𝑓𝑜𝑖(𝜃+𝜋),𝑓𝑓(𝜃)=𝑓𝑠𝑓(𝜃+𝜋),𝑓𝑖(𝜃)=𝑓𝑠𝑖(𝜃+𝜋),𝑓𝑑(𝜃)=𝑓𝑑𝑓(𝜃+𝜋),𝑑𝑖(𝜃)=𝑓di(𝜃+𝜋),𝑓=(𝜃)=𝑓=(𝜃+𝜋).(6)

Eight relations are possible combination of eight independent Allen relations in one-dimensional spatial domain. These relations and their reorientation show that the whole 2D space can be explored with the help of 1D Allen relations using the oriented lines varying from (0, 𝜋).

4.3. Combining Topological and Directional Relations

Eight topological relations represented in point set topology or point less topology between object pair are represented in one-dimensional space by the Allen’s temporal relations in spatial domain. We extend these Allen relations for two-dimensional objects through logical implication, where a 2D object is decomposed into parallel segments of a 1D lines in a given direction, and the relation between each pair of line segments is computed.

The process of object decomposition is repeated for each direction varying from 0 to 𝜋, two-dimensional topological relations are then defined as it provides us with information about how the objects are relatively distributed. These relations are not jointly exhaustive and pairwise disjoint (JEPD), to obtain JEPD set of topological and directional relations an algorithm was advocated in [11], it provides us with the JEPD set of relations. Objects are approximated through the polygon object approximation. Different steps of computing the combined topological and directional relations are(i)fix angle 𝜃 and draw lines passing through the vertices of polygons representing the objects;(ii)for each line, compute the variables (𝑥,𝑦,𝑧) as depicted in Section 4.1 and compute Allen relation for each segment as given by (5). In case of longitudinal sections, use fuzzy operators to integrate the information, usually the disjunction operators are suitable. These relations are computed for each line in a direction, then obtained information is integrated into a single value. Normalize these relations for a direction 𝜃 by dividing sum of all Allen relations to each Allen relation;(iii)these normalized fuzzy Allen relation is then multiplied to a fuzzy directional set to find the degree of an Allen relation in a direction;(iv)for qualitative directions, this information is summarized, and different topological relations with directional contents are defined, such as 𝑓𝐸=𝜋/4𝜃=0𝒜𝑟2×cos2(2𝜃)+𝜋𝜃=3𝜋/4𝒜𝑟1×cos2(2𝜃),(7) where 𝑓 represents a topological relation, and 𝐸 represents the direction 𝐴𝑟1 is the reorientation of 𝐴𝑟2; (v)this information is represented in a matrix, this matrix represents fuzzy spatial information;(vi)these fuzzy spatial relations are defuzzified by an algorithm, this provides us with final topological and directional relations between the object pair. These topological and directional relations are JEPD.

This model describes well the possible topological relations between every sort of objects.

5. Spatiotemporal Relations

Spatiotemporal relations can be defined as spatial relation holds for an interval, that is, relation holds for a certain time interval, and it does not change. In spatiotemporal object theory it is defined as (P) a spatial relation is a relation holding between all temporal slices of two entities during the relevant period. All eight spatiotemporal relations are defined in terms of theorems.

5.1. Spatiotemporal Relation

Theorem 1. A spatiotemporal disjoint relation between object pair (𝑋,𝑌) holds during the interval 𝑇, that is, 𝐷(𝑋𝑌,𝑇)forall𝑡𝑇,𝐷(𝑋𝑌,𝑡) holds.

Proof. () A spatiotemporal 𝑑𝑖𝑠𝑗𝑜𝑖𝑛𝑡𝐷(𝑋𝑌,𝑇) relation is defined as object pair (𝑋,𝑌) are disjoint during the interval 𝑇 if 𝑋𝑡𝑌(𝑋 is temporally equivalent to 𝑌). Now let us consider the partition of interval 𝑇=[𝑡𝑎𝑡𝑏], then its partition can be taken as 𝑡𝑎=𝑡1<𝑡2<𝑡3<𝑡𝑛=𝑡𝑏. Each 𝑡𝑖𝑇,𝑖=1,2,,𝑛 represents discrete points of interval 𝑇, and this representation is equivalent to a snapshot. Typically a snapshot is a sampling process, which represents zero duration temporal slice of a spatiotemporal object. There are 𝑛 snapshots in interval 𝑇, as a result a disjoint topological relation exists for each snapshot separately. Thus, for all 𝑡𝑇,𝐷(𝑋𝑌,𝑡) holds.
() Let us consider 𝑛 snapshots where the temporal ordering holds, that is, 𝑡1,𝑡2,,𝑡𝑛 such that 𝑡1<𝑡2<𝑡3<𝑡𝑛 and all these points form partition of an interval 𝑇. If the disjoint topological relation holds at discrete points, it means that 𝐷(𝑋𝑌,𝑡1)𝐷(𝑋𝑌,𝑡2)𝐷(𝑋𝑌,[𝑡1𝑡2]). If the disjoint topological relation holds between object pair, it means that both the objects are temporally equivalent (𝑥𝑡𝑦). Hence, 𝐷(𝑋𝑌,𝑇) holds during the whole interval 𝑇.

Theorem 2. A spatiotemporal relation Meet M(XY,T) holds 𝑡𝑖𝑇 such that 𝑀(𝑋𝑌,𝑡𝑖)for all 𝑡𝑗𝑇𝑡𝑗𝑡𝑖𝐷(𝑋𝑌,𝑡𝑗) holds.

Proof. () A spatiotemporal relation meet 𝑀(𝑋𝑌,𝑇) holds between object pair (𝑋,𝑌) over interval 𝑇, where 𝑋𝑡𝑌. Let 𝑡𝑎=𝑡1<𝑡2<𝑡3<<𝑡𝑛=𝑡𝑏 be partition of interval 𝑇=[𝑡𝑎𝑡𝑏], if forall𝑡𝑖𝑇,𝑀(𝑋𝑌,𝑡𝑖) holds, then a stable topological relation 𝑀(𝑋𝑌,𝑇) holds. We consider on contrary, that 𝑡𝑗, where the topological relation 𝑀(𝑋𝑌,𝑡𝑗) does not hold, but it holds at 𝑀(𝑋𝑌,𝑡𝑗1), then according to the temporal logic and continuity of topological relations (𝑀(𝑋𝑌,𝑡𝑗1))(𝐷(𝑋𝑌,𝑡𝑗)𝑀(𝑋𝑌,𝑡𝑗)PO(𝑋𝑌,𝑡𝑗). This shows that any of the three relations is possible (stands for future position). If PO(𝑋𝑌,𝑡𝑗) holds, then the whole spatiotemporal relation is changed, and it becomes the spatiotemporal partial overlap relation. This possibility is ruled out. In other case, spatiotemporal relation remains meet, as 𝑗 is an arbitrary variable, this shows the minimum condition. Hence, 𝑡𝑇,s.t.𝑀(𝑋𝑌,𝑡) holds.
() Let us consider that there are 𝑛 snapshots in an order, which construct an interval 𝑇. Now consider that there exists at least one snapshot during whole interval, where spatial meet relation holds, and for all other snapshots, the spatial relation is disjoint. This shows that during temporal interval 𝑇, the unstable spatiotemporal meet relation holds. It satisfies the minimum conditions for a spatiotemporal meet relation, hence 𝑀(𝑋𝑌,𝑇) holds during interval 𝑇.

Theorem 3. A spatiotemporal partial overlap (PO) relation holds over interval 𝑇, that is, PO(𝑋𝑌,𝑇)𝑡𝑇,s.t.PO(𝑋𝑌,𝑡).

Proof. Spatiotemporal relations have the spatial and temporal boundaries. A stable spatiotemporal relation holds during the temporal slice, If it holds at every point of the interval. As temporal slice is the union of finite points of temporal domain, spatiotemporal partial overlap holds during the whole slice, if this relation holds at least one sampling point (snapshot), at remaining points any of the spatial relation may exist. Hence𝑡𝑇,s.t.,PO(𝑋𝑌,𝑡)(𝑡1𝑡,CO(𝑋𝑌,𝑡1)𝑀(𝑋𝑌,𝑡1)𝐷(𝑋𝑌,𝑡1)) (CO(𝑋𝑌) stands for complete overlap of objects (𝑋𝑌), s.t., CO(𝑋𝑌)=TPP(𝑋𝑌)NTPP(𝑋𝑌)TPPI(𝑋𝑌)NTPPI(𝑋𝑌)). If there does not exist such a 𝑡1, then the relation holds for every 𝑡𝑇, which shows that a stable PO topological relation holds.
() We suppose on contrary that 𝑡𝑇,s.t.PO(𝑋𝑌,𝑡) holds. It means that at all points either the binary topological relations are complete overlap or disjoint and meet. If the relations are complete overlap, that is, for all 𝑡𝑇,CO(𝑋𝑌,𝑡) holds, then the spatiotemporal relation will be a part of complete overlap. In case of other choice that 𝑡𝑇, s.t.,𝑀(𝑋𝑌,𝑡) or for all 𝑡𝑇,s.t.,𝑀(𝑋𝑌,𝑡) holds, then the spatiotemporal topological relation will be stable or unstable meet and for case for all 𝑡𝑇,s.t.,𝐷(𝑋𝑌,𝑡), the topological relation will be disjoint. The choice, that 𝑡1𝑇,s.t.,𝑀(𝑋𝑌,𝑡) and 𝑡2𝑇,s.t.,CO(𝑋𝑌,𝑡2 holds is impossible because in a such a case 𝑡𝑇𝑡1<𝑡<𝑡2, s.t.,PO(𝑋𝑌,𝑡) holds (continuity of spatial relations).

Theorem 4. A spatiotemporal tangent proper part (TPP) relation holds over interval 𝑇, that is,TPP(𝑋𝑌,𝑇)𝑡1𝑇,such thatTPP(𝑋𝑌,𝑡1)for all 𝑡2𝑡2𝑡1, NTPP(𝑋𝑌,𝑡2) holds.

Proof. () A spatiotemporal topological relation TPP(𝑋𝑌) holds between the object pair 𝑋,𝑌 during the interval 𝑇. Now let us consider that interval 𝑇 consist of n snapshots, if this relation holds for every snapshot then a spatiotemporal stable topological relation holds. In other case, there are two possibilities that for all 𝑡1𝑡2, there exists a topological relation TPP(𝑋𝑌,𝑡1), and for 𝑡2 either the topological relation is PO(𝑋𝑌,𝑡2) or NTPP(𝑋𝑌,𝑡2) due to continuity of topological relations between moving objects. For PO(𝑋𝑌,𝑡2), the spatiotemporal topological relation is changed, and it becomes the spatiotemporal PO topological relation, this possibility is ruled out. It remains that NTPP(𝑋𝑌,𝑡2), if this relation holds and 𝑡2 is an arbitrary point, so the relation becomes the unstable spatiotemporal TPP(𝑋𝑌,𝑇).
() Consider that 𝑡𝑖 such that TPP(𝑋𝑌,𝑡𝑖) holds. We consider on contrary that 𝑡𝑖+1𝑡𝑖1 such that NTPP(𝑋𝑌,𝑡𝑖1) or NTPP(𝑋𝑌,𝑡𝑖+1) does not holds. Then, possible topological relations at 𝑡𝑖1 are TPP(𝑋𝑌,𝑡𝑖1),PO(𝑋𝑌,𝑡𝑖1) similarly for 𝑡𝑖+1. Other possibilities are ruled out due to the continuity of topological relations, and EQ(𝑋𝑌,𝑡𝑖1) does not hold because objects are considered under motion, and expansion or zooming of one object is not allowed.
In case the topological relation PO(𝑋𝑌,𝑡𝑖1) holds, then the whole spatiotemporal relation over the interval 𝑇 becomes partial overlap. Similarly for instant 𝑡𝑖+1 and 𝑖 is an arbitrary point, so this is impossible for whole the interval 𝑇. For the topological relation TPP(𝑋𝑌,𝑡𝑖1), the spatiotemporal relation becomes the stable spatiotemporal TPP.

Theorem 5. A spatiotemporal nontangent proper part (NTPP) relation holds over interval 𝑇, that is, NTPP(𝑋𝑌,𝑇) for all𝑡𝑇,NTPP(𝑋𝑌,𝑡) holds.

Proof. () Let us suppose on contrary that 𝑡𝑖𝑇s.t.NTPP(𝑋𝑌,𝑡𝑖) does not hold, and at temporal points 𝑡𝑖1,𝑡𝑖+1 the relation NTPP(𝑋𝑌,𝑡𝑖1),NTPP(𝑋𝑌,𝑡𝑖+1), holds. Then continuity of spatial relations forces the existence of TPP(𝑋𝑌,𝑡𝑖) or EQ(𝑋𝑌,𝑡𝑖) spatial relations. This contradicts the existence of the spatiotemporal NTPP(𝑋𝑌,𝑇) relation. Hence, forall𝑡𝑇,NTPP(𝑋𝑌,𝑡) holds.
() It is given that forall𝑡𝑇,NTPP(𝑋𝑌,𝑡) holds. If a spatial relation between object pair holds at every point of the interval, then it means that it holds throughout the interval, that is, NTPP(𝑋𝑌,𝑇) holds.

Theorem 6. A spatiotemporal tangent proper part inverse (TPP) relation holds over interval 𝑇, that is, TPPI(𝑋𝑌,𝑇)𝑡𝑇,s.t.TPPI(𝑋𝑌,𝑡)forall𝑡1𝑡,NTPPI(𝑋𝑌,𝑡1) holds.

Proof. Proof is similar to the TPP(𝑋𝑌,𝑇), just replace TPP by TPPI and NTPP by NTPPI.

Theorem 7. A spatiotemporal nontangent proper part inverse (NTPPI) relation holds over temporal interval 𝑇, that is, NTPPI(𝑋𝑌,𝑇)forall𝑡𝑇andNTPPI(𝑋𝑌,𝑡) holds.

Proof. Proof is similar to the NTPP(𝑋𝑌,𝑇).

Theorem 8. A spatiotemporal relation equal(EQ) holds between the object pair 𝑋𝑌, EQ(𝑋𝑌,𝑇)forall𝑡𝑇, s.t.,EQ(𝑋𝑌,𝑡) holds.

Proof. () We suppose on contrary that there exists a 𝑡𝑇, where the EQ(𝑋𝑌,𝑡) relation does not hold. It shows that there are two possibilities that either the relation at 𝑡 is a complete overlap or partial overlap. If the relation at 𝑡 is complete overlap, then the spatiotemporal relation becomes TPP or TPPI. In the second case, the spatiotemporal relation becomes the PO(𝑋𝑌,𝑇) during the whole interval. Thus, both cases prove the contrary conditions, hence 𝑡s.t.EQ(𝑋𝑌,𝑡) does not hold, that is, for all𝑡𝑇,EQ(𝑋𝑌,𝑡) holds.
() Converse of this proof is very simple and straight forward. Let 𝑇 be the interval for which we have to define the spatiotemporal relation, both the objects are temporally comparable (𝑋𝑡𝑌). Let 𝑡𝑇 be an arbitrary point of the interval and relation EQ(𝑋𝑌,𝑡) holds. Since 𝑡 is an arbitrary point so, the relation holds throughout the interval 𝑇, that is, EQ(𝑋𝑌,𝑇) holds.

6. Visual Interpretation: A Three-Dimensional View

Geometrical figures can better elaborate concepts, a moving object changes its position at each instant 𝑡. These objects in a spatiotemporal domain can be represented by their envelops, a two-dimensional object becomes volume. Here, spatiotemporal meet and partially overlap relations are represented by their envelops in Figures 3(a)3(d) and 4(a)4(h). These are possible representation of motion events. Spatial relations between moving objects are used in modeling the motion verbs or motion events in natural language processing. A set of motion relations is introduced that capture semantic between pairs of moving objects. This information is useful about reasoning the moving objects.

fig3
Figure 3: Spatiotemporal Meet relation (unstable meet).
fig4
Figure 4: Spatiotemporal partial_Overlap relation (unstable overlap).

7. Modeling Motion Classes

Visual images may illustrate cases of a definition, giving us a more visual grasp of its applications. They may help us understanding the description of a mathematical situation or steps in reasoning. These relations can be defined as the transection of relations at time 𝑡1 to 𝑡2. This change may be in topological or metric relations, and different classes of spatial relations, between moving objects have been defined [12, 13, 29, 30]. Motion classes based on intuitive logics or motion verbs have been defined in [31] by Phillipe Muller and Ralf H. Güting and Markus Schneider used in database. We define in this paper only the motion events, where topological relations capture changes between situations. These motion events can be defined using the predicates 𝑜𝑙𝑑𝑠_𝑎𝑡, 𝑜𝑙𝑑𝑠,𝑜𝑐𝑐𝑢𝑟𝑠_𝑎𝑡,   𝑜𝑐𝑐𝑢𝑟𝑠 and 𝑠𝑒𝑞𝑢𝑒𝑛𝑐𝑒. In next section, 𝑠𝑒𝑞_𝑒𝑣𝑒 represents the sequence event.

7.1. Unstable Meet Spatiotemporal Relation

Unstable spatiotemporal relation is a relation where objects changes their states at each time instant. A spatiotemporal meet relation is characterized by different motion events depending upon the logical and temporal order of different states or primitive events. Touch(XY,T): A spatiotemporal meet relation can be characterized as a motion event Touch, s.t. 𝑡1,𝑡2,𝑡3,𝑇and𝑡1<𝑡2<𝑡3, where primitive events occur in an order and defined as𝐷𝑇𝑜𝑢𝑐(𝑋𝑌,𝑇)=𝑠𝑒𝑞_𝑒𝑣𝑒𝑜𝑙𝑑𝑠𝑋𝑌,𝑡1𝑀𝑜𝑙𝑑𝑠𝑋𝑌,𝑡2𝐷𝑜𝑙𝑑𝑠𝑋𝑌,𝑡3.(8)

An institutive view of this spatiotemporal relation is shown in Figure 3(a). This relation can be expressed by a single direction, where a meet topological relation holds. It means𝐷𝑖𝑟(𝑇𝑜𝑢𝑐(𝑋𝑌,𝑇))=𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡2.(9)Snap(XY,T): A spatiotemporal meet relation is called a Snap if 𝑡1,𝑡2𝑇and𝑡1<𝑡2 such that𝐷𝑆𝑛𝑎𝑝(𝑋𝑌,𝑇)=𝑠𝑒𝑞_𝑒𝑣𝑒𝑜𝑙𝑑𝑠𝑋𝑌,𝑡1𝑀𝑜𝑙𝑑𝑠𝑋𝑌,𝑡2.(10)

A geometric representation is shown in Figure 3(c). This relation can be expressed by a single direction, where a meet topological relation holds. It means𝐷𝑖𝑟(𝑆𝑛𝑎𝑝(𝑋𝑌,𝑇))=𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡2.(11)Release(XY,T): A spatiotemporal 𝑀𝑒𝑒𝑡(𝑋𝑌,𝑇) is called 𝑅𝑒𝑙𝑒𝑎𝑠𝑒(𝑋𝑌,𝑇), read as 𝑋 releases 𝑌 during interval 𝑇 if it has a certain temporal ordering, 𝑡1,𝑡2𝑇and𝑡1<𝑡2such that𝐷𝑅𝑒𝑙𝑒𝑎𝑠𝑒(𝑋𝑌,𝑇)=𝑠𝑒𝑞_𝑒𝑣𝑒𝑜𝑙𝑑𝑠𝑋𝑌,𝑡1𝑀𝑜𝑙𝑑𝑠𝑋𝑌,𝑡1.(12)

A three-dimensional geometric view of this relation is shown in Figure 3(d). This relation can be expressed by a single direction, which is the destination direction. For example, object 𝑋  𝑟𝑒𝑙𝑒𝑎𝑠𝑒𝑠 (motion event) object 𝑌 towards East (destination direction). Direction for such a relation is defined as𝐷𝑖𝑟(𝑅𝑒𝑙𝑒𝑎𝑠𝑒(𝑋𝑌,𝑇))=𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡2.(13)Bypass(XY,T): A spatiotemporal 𝑀𝑒𝑒𝑡(𝑋𝑌,𝑇) is called a 𝐵𝑦𝑝𝑎𝑠𝑠(𝑋𝑌,𝑇), read as 𝑋 bypasses 𝑌 during interval 𝑇 if it has a certain temporal ordering, that is, 𝑡1,𝑡2,𝑡3,𝑡4𝑇 such that𝑡1<𝑡2<𝑡3<t4𝐷𝐵𝑦𝑝𝑎𝑠𝑠(𝑋𝑌,𝑇)=𝑠𝑒𝑞_𝑒𝑣𝑒𝑜𝑙𝑑𝑠𝑋𝑌,𝑡1𝑀𝑜𝑙𝑑𝑠𝑋𝑌,𝑡2𝑀𝑜𝑙𝑑𝑠𝑋𝑌,𝑡3𝐷𝑜𝑙𝑑𝑠𝑋𝑌,𝑡4.(14)

This relation can be expressed by a single direction, where a meet topological relation holds. It means𝐷𝑖𝑟(𝑇𝑜𝑢𝑐(𝑋𝑌,𝑇))=𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡2.(15)Excurse(XY,T): A spatiotemporal Meet(XY,T) is called a 𝐸𝑥𝑐𝑢𝑟𝑠𝑒(𝑋𝑌,𝑇), read as 𝑋 excurse 𝑌 during interval 𝑇 if it has a certain temporal ordering, an intuitive view of this relation is shown in Figure 3(b). 𝑡1,𝑡2,𝑡3𝑇,s.t.,𝑡1<𝑡2<𝑡3𝑀𝐸𝑥𝑐𝑢𝑟𝑠𝑒(𝑋𝑌,𝑇)=𝑠𝑒𝑞_𝑒𝑣𝑒𝑜𝑙𝑑𝑠𝑋𝑌,𝑡1𝐷𝑜𝑙𝑑𝑠𝑋𝑌,𝑡2𝑀𝑜𝑙𝑑𝑠𝑋𝑌,𝑡3.(16)

This relation is expressed by an initial and destination directions, the direction for this relation can be defined as𝐷𝑖𝑟(𝐸𝑥𝑐𝑢𝑟𝑠𝑒(𝑋𝑌,𝑇))=𝑠𝑒𝑞_𝑒𝑣𝑒𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡1𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡3.(17)

7.2. Unstable Overlap Spatiotemporal Relation
Enter(XY,T): An unstable spatiotemporal overlap relation is called Enter, generally denoted by Enter(𝑋𝑌,𝑇) and read as “𝑋 enters in 𝑌 during interval 𝑇.” If 𝑡1,𝑡2,𝑡3,𝑡4𝑇suchthat𝑡1<𝑡2<𝑡3<𝑡4, then relation is defined as

𝐷𝐸𝑛𝑡𝑒𝑟(𝑋𝑌,𝑇)=𝑠𝑒𝑞_𝑒𝑣𝑒𝑜𝑙𝑑𝑠𝑋𝑌,𝑡1𝑀𝑜𝑙𝑑𝑠𝑋𝑌,𝑡2𝑜𝑙𝑑𝑠𝑃𝑂𝑋𝑌,𝑡3𝑜𝑙𝑑𝑠TPP𝑋𝑌,𝑡4.(18)

An intuitive view of this relation is shown in Figure 4(a). This relation can be expressed by a single direction because the destination point is inside and can be expressed without direction, a direction for the Enter spatiotemporal event is the direction where a meet topological relation holds, that is,𝐷𝑖𝑟(𝐸𝑛𝑡𝑒𝑟(𝑋𝑌,𝑇))=𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡2.(19)Leave(𝑋𝑌,𝑇): A spatiotemporal partial overlap relation is called Leave, denoted as Leave(XY,T) “𝑋 leaves 𝑌 during interval 𝑇”. If 𝑡1,𝑡2,𝑡3,𝑡4𝑇suchthat𝑡1<𝑡2<𝑡3<𝑡4, then relation is defined as𝐿𝑒𝑎𝑣𝑒(𝑋𝑌,𝑇)=𝑠𝑒𝑞_𝑒𝑣𝑒𝑜𝑙𝑑𝑠𝑁𝑇𝑃𝑃𝑋𝑌,𝑡1𝑜𝑙𝑑𝑠𝑇𝑃𝑃𝑋𝑌,𝑡2𝑜𝑙𝑑𝑠𝑃𝑂𝑋𝑌,𝑡3𝑀𝑜𝑙𝑑𝑠𝑋𝑌,𝑡4𝐷𝑜𝑙𝑑𝑠𝑋𝑌,𝑡5.(20)

An intuitive view of this relation is shown in Figure 4(b). This relation can be expressed by a single direction which is the destination point, that is𝐷𝑖𝑟(𝐿𝑒𝑎𝑣𝑒(𝑋𝑌,𝑇))=𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡4.(21)Cross(XY,T): A spatiotemporal relation Cross(XY,T)𝑋 crosses 𝑌 during the interval 𝑇.” Its geometric view is given in Figure 4(c). If 𝑡1,𝑡2,𝑡3,,𝑡9𝑇suchthat𝑡1<𝑡2<<𝑡9, then relation is defined as𝐷𝐶𝑟𝑜𝑠𝑠(𝑋𝑌,𝑇)=𝑠𝑒𝑞_𝑒𝑣𝑒𝑜𝑙𝑑𝑠𝑋𝑌,𝑡1𝑀𝑜𝑙𝑑𝑠𝑋𝑌,𝑡2𝑜𝑙𝑑𝑠𝑃𝑂𝑋𝑌,𝑡3𝑜𝑙𝑑𝑠𝑇𝑃𝑃𝑋𝑌,𝑡4𝑜𝑙𝑑𝑠𝑁𝑇𝑃𝑃𝑋𝑌,𝑡5𝑜𝑙𝑑𝑠𝑇𝑃𝑃𝑋𝑌,𝑡6𝑜𝑙𝑑𝑠𝑃𝑂𝑋𝑌,𝑡7𝑀𝑜𝑙𝑑𝑠𝑋𝑌,𝑡8𝐷𝑜𝑙𝑑𝑠𝑋𝑌,𝑡9.(22)

This spatiotemporal relation is expressed by a initial as well as destination direction such as object 𝑋 crosses (motion event) object𝑌 from north (direction) towards east (direction) during the interval 𝑇:𝐷𝑖𝑟(𝐶𝑟𝑜𝑠𝑠(𝑋𝑌,𝑇))=𝑠𝑒𝑞_𝑒𝑣𝑒𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡1𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡9.(23)Into(𝑋𝑌,𝑇): A spatiotemporal relation𝐼𝑛𝑡𝑜(XY,T) read as “𝑋 get into 𝑌 during the interval 𝑇.” If 𝑡1,𝑡2,𝑡3𝑇suchthat𝑡1<𝑡2<𝑡3, then relation is defined as𝑀𝐼𝑛𝑡𝑜(𝑋𝑌,𝑇)=𝑠𝑒𝑞_𝑒𝑣𝑒𝑜𝑙𝑑𝑠𝑋𝑌,𝑡1𝑜𝑙𝑑𝑠𝑃𝑂𝑋𝑌,𝑡2𝑜𝑙𝑑𝑠𝑇𝑃𝑃𝑋𝑌,𝑡3.(24)

Its three-dimensional geometric view is given in Figure 4(e). This relation can be expressed by a single direction in language semantics, where a meet topological relation holds. For example, object 𝐴 get into (spatiotemporal event) object 𝐵 from north (direction). It means.𝐷𝑖𝑟(𝐼𝑛𝑡𝑜(𝑋𝑌,𝑇))=𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡1.(25)𝑂𝑢𝑡_𝑜𝑓(𝑋𝑌,𝑇): A spatiotemporal relation 𝑂𝑢𝑡𝑜𝑓(XY,T) read as “𝑋 comes out of 𝑌 during the interval 𝑇,” its intuitive view is considered in Figure 4(f). If 𝑡1,𝑡2,𝑡3,𝑡4𝑇suchthat𝑡1<𝑡2<𝑡3<𝑡4, then relation is defined as𝑂𝑢𝑡_𝑜𝑓(𝑋𝑌,𝑇)=𝑠𝑒𝑞_𝑒𝑣𝑒𝑜𝑙𝑑𝑠𝑇𝑃𝑃𝑋𝑌,𝑡1𝑜𝑙𝑑𝑠𝑃𝑂𝑋𝑌,𝑡2𝐷𝑜𝑙𝑑𝑠𝑋𝑌,𝑡3.(26)

This relation can be expressed by a single direction. Object 𝑋 go out_of (motion event) object 𝑌 towards east (direction), where a meet topological relation holds. It means 𝐷𝑖𝑟(𝑜𝑢𝑡_𝑜𝑓(𝑋𝑌,𝑇))=𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡3.(27)Melt(𝑋𝑌,𝑇): A spatiotemporal relation 𝑀𝑒𝑙𝑡(XY,T) read as “𝑋,𝑌 melts during the interval 𝑇”. If 𝑡1,𝑡2,𝑡3,𝑡4𝑇suchthat𝑡1<𝑡2<𝑡3<𝑡4, then relation is defined as𝐷𝑀𝑒𝑙𝑡(𝑋𝑌,𝑇)=𝑠𝑒𝑞_𝑒𝑣𝑒𝑜𝑙𝑑𝑠𝑋𝑌,𝑡1𝑀𝑜𝑙𝑑𝑠𝑋𝑌,𝑡2𝑜𝑙𝑑𝑠𝑃𝑂𝑋𝑌,𝑡3𝑜𝑙𝑑𝑠𝐸𝑄𝑋𝑌,𝑡4.(28)

An intuitive view of this relation is shown in Figure 4(g). This relation can be expressed by a single direction because its destination point is dimensionless. This can be its direction, where initial spatial relation holds:𝐷𝑖𝑟(𝑀𝑒𝑙𝑡(𝑋𝑌,𝑇))=𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡1.(29)Spring(𝑋𝑌,𝑇): A spatiotemporal relation Spring (𝑋𝑌,𝑇) also called Separate(𝑋𝑌,𝑇) read as “𝑋 separates 𝑌 during the interval 𝑇.” If 𝑡1,𝑡2,𝑡3,𝑡4𝑇suchthat𝑡1<𝑡2<𝑡3<𝑡4, then relation is defined as𝑆𝑝𝑟𝑖𝑛𝑔(𝑋𝑌,𝑇)=𝑠𝑒𝑞_𝑒𝑣𝑒𝑜𝑙𝑑𝑠𝐸𝑄𝑋𝑌,𝑡1𝑜𝑙𝑑𝑠𝑃𝑂𝑋𝑌,𝑡2𝑀𝑜𝑙𝑑𝑠𝑋𝑌,𝑡3𝐷𝑜𝑙𝑑𝑠𝑋𝑌,𝑡4.(30)

Its three-dimensional geometric view is given in Figure 4(h). This relation can be expressed by a single direction because its destination point is dimensionless. This can be its direction, where terminal spatial relation holds:𝐷𝑖𝑟(𝑆𝑝𝑟𝑖𝑛𝑔(𝑋𝑌,𝑇))=𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡4.(31)Graze(𝑋𝑌,𝑇): A spatiotemporal relation Graze(𝑋𝑌,𝑇) read as “𝑋 grazes 𝑌 during the interval 𝑇.” If 𝑡1,𝑡2,𝑡3,𝑡4,𝑡5𝑇suchthat𝑡1<𝑡2<𝑡3<𝑡4<𝑡5, then relation is defined as𝐷𝐺𝑟𝑎𝑧𝑒(𝑋𝑌,𝑇)=𝑠𝑒𝑞_𝑒𝑣𝑒𝑜𝑙𝑑𝑠𝑋𝑌,𝑡1𝑀𝑜𝑙𝑑𝑠𝑋𝑌,𝑡2𝑜𝑙𝑑𝑠𝑃𝑂𝑋𝑌,𝑡3𝑀𝑜𝑙𝑑𝑠𝑋𝑌,𝑡4𝐷𝑜𝑙𝑑𝑠𝑋𝑌,𝑡5.(32)

This relation is represented in a three-dimensional perspective in Figure 4(d). This spatiotemporal relation is expressed by an initial as well as destination direction such as object 𝑋 grazes (motion event) object 𝑌 from north (direction) toward east (direction):𝐷𝑖𝑟(𝐺𝑟𝑎𝑧𝑒(𝑋𝑌,𝑇))=𝑠𝑒𝑞_𝑒𝑣𝑒𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡1𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡4.(33)

8. Conclusion and Future Work

In this paper, we define spatiotemporal relations, where the discrete time space is used. These spatiotemporal relations are topologically stable or unstable. Motion events represent the subclass of spatiotemporal relations, and certain number of motion events represent the class of a topologically unstable and stable spatiotemporal relations. In these spatiotemporal relations temporal order of holding a primitive event is more important, and this order has a pivotal role in natural language semantics. Topological relations have a locative symmetries, to remove these symmetries we add a directional components. In this paper, CTD method [11] is used to model the motion events, where topological and directional information for a snapshot are captured at the same abstract level. Hopefully this work will bring a significant change in video understanding, modeling video events, and other related areas of research.

References

  1. R. Adaikkalavan, “Formalization and detection of events using interval-based semantics,” in Proceedings of the International Conference on Management of Data, pp. 58–60, 2005.
  2. J. F. Allen, “Maintaining knowledge about temporal intervals,” Communications of the ACM, vol. 26, no. 11, pp. 832–843, 1983. View at Publisher · View at Google Scholar · View at Scopus
  3. J. F. Allen and G. Ferguson, “Actions and events in interval temporal logic,” Journal of Logic and Computation, vol. 4, no. 5, pp. 531–579, 1994. View at Publisher · View at Google Scholar · View at Scopus
  4. T. Bittner, M. Donnelly, and B. Smith, “A spatio-temporal ontology for geographic information integration,” International Journal of Geographical Information Science, vol. 23, no. 6, pp. 765–798, 2009. View at Publisher · View at Google Scholar · View at Scopus
  5. N. van de Weghe, A. G. Cohn, G. De Tré, and P. De Maeyer, “A qualitative trajectory calculus as a basis for representing moving objects in geographical information systems,” Control and Cybernetics, vol. 35, no. 1, pp. 97–119, 2006. View at Google Scholar · View at Scopus
  6. M. J. Egenhofer and K. K. Al-Taha, “Reasoning about gradual changes of topological relationships,” in Proceedings of the International Conference On GIS—From Space to Territory, pp. 196–219, Springer, London, UK, 1992.
  7. M. J. Egenhofer, J. Sharma, and D. M. Mark, “A critical comparison of the 4-intersection and 9-intersection models for spatial relations: formal analysis,” AutoCarto, vol. 11, pp. 1–12, 1993. View at Google Scholar
  8. M. Erwig and M. Schneider, “A visual language for the evolution of spatial relationships and its translation into a spatio-temporal calculus,” Journal of Visual Languages and Computing, vol. 14, no. 2, pp. 181–211, 2003. View at Publisher · View at Google Scholar · View at Scopus
  9. J. Fernyhough, A. G. Cohn, and D. C. Hogg, “Constructing qualitative event models automatically from video input,” Image and Vision Computing, vol. 18, no. 2, pp. 81–103, 2000. View at Publisher · View at Google Scholar · View at Scopus
  10. A. Galton, “A generalized topological view of motion in discrete space,” Theoretical Computer Science, vol. 305, no. 1–3, pp. 111–134, 2003. View at Publisher · View at Google Scholar · View at Scopus
  11. A. Galton and J. C. Augusto, “Two approaches to event definition,” in Proceedings of the International Conference on Database and Expert Systems Applications, pp. 547–556, Springer, 2002.
  12. R. H. Güting and M. Schneider, Moving Objects Databases, Morgan Kaufmann, 2005.
  13. J. Y. Halpern and Y. Shoham, “A propositional modal logic of time intervals,” Journal of the ACM, vol. 38, no. 4, pp. 935–962, 1991. View at Google Scholar
  14. K. S. Hornsby and M. J. Egenhofer, “Modeling moving objects over multiple granularities,” Annals of Mathematics and Artificial Intelligence, vol. 36, no. 1-2, pp. 177–194, 2002. View at Publisher · View at Google Scholar · View at Scopus
  15. K. S. Hornsby and K. King, “Modeling motion relations for moving objects on road networks,” GeoInformatica, vol. 12, no. 4, pp. 477–495, 2008. View at Publisher · View at Google Scholar · View at Scopus
  16. Z. M. Ibrahim and A. Y. Tawfik, “An abstract theory and ontology of motion based on the regions connection calculus,” in 7th International Symposium on Abstraction, Reformulation, and Approximation , SARA 2007, pp. 230–242, can, July 2007. View at Scopus
  17. J. Z. Li, M. T. Ozsu, and D. Szafron, “Modeling of moving objects in a video database,” in Proceedings of the 1997 IEEE International Conference on Multimedia Computing and Systems, ICMCS, pp. 336–343, June 1997. View at Scopus
  18. M. Ma and P. Mc Kevitt, “Visual semantics and ontology of eventive verbs,” in Proceedings of the 1st International Joint Conference on Natural Language Processing (IJCNLP '04), pp. 187–196, chn, March 2004. View at Scopus
  19. N. van de Weghe, A. G. Cohn, P. De Maeyer, and F. Witlox, “Representing moving objects in computer-based expert systems: the overtake event example,” Expert Systems with Applications, vol. 29, no. 4, pp. 977–983, 2005. View at Publisher · View at Google Scholar
  20. R. P. Markus Schneider, “A universal abstract model for future movement of moving objects,” in Lecture Notes in Geoinformation and Cartography Part 3, pp. 111–120, Springer, Berlin, Germany, 2007. View at Google Scholar
  21. P. Matsakis and D. Nikitenko, “Combined extraction of directional and topological relationship information from 2D concave objects,” in Fuzzy Modeling with Spatial Information for Geographic Problems, pp. 15–40, Springer, New York, NY, USA, 2005. View at Google Scholar
  22. P. Matsakis and L. Wendling, “A new way to represent the relative position between areal objects,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 7, pp. 634–643, 1999. View at Google Scholar · View at Scopus
  23. P. Muller, “A qualitative theory of motion based on spatio-temporal primitives,” in Proceedings of the Proceedings of the Sixth International Conference (KR '98), A. G. Cohn, L. K. Schubert, and S. C. Shapiro, Eds., pp. 131–143, 1998.
  24. P. Muller, “Topological spatio-temporal reasoning and representation,” Computational Intelligence, vol. 18, no. 3, pp. 420–450, 2002. View at Google Scholar · View at Scopus
  25. P. Muller and L. Sarda, “The semantics of french transitive movement verbs and the ontological nature of their objects,” in Proceedings of the International Colloquium of Cognitive Science (ICCS ’97), May 1997.
  26. Y. Nenov and D. Vakarelov, “Modal logics for mereotopological relations,” in Advances in Modal Logic, pp. 249–272, 2008. View at Google Scholar
  27. N. Salamat and E. Zahzah, “2D fuzzy spatial relations: new way of computing and representation,” Advances in Fuzzy Systems, 2012.
  28. N. Salamat and E. Zahzah, “On the improvement of combined topological and directional relations information,” Pattern Recognition, vol. 45, no. 4, pp. 1559–1568, 2012. View at Google Scholar
  29. D. Vakarelov, “Dynamic mereotopology: a point-free theory of changing regions. I. Stable and unstable mereotopological relations,” Fundamenta Informaticae, vol. 100, no. 1–4, pp. 159–180, 2010. View at Publisher · View at Google Scholar · View at Scopus
  30. M. Worboys, “Event-oriented approaches to geographic phenomena,” International Journal of Geographical Information Science, vol. 19, no. 1, pp. 1–28, 2005. View at Publisher · View at Google Scholar · View at Scopus
  31. A. K. Zaidi and L. W. Wagenhals, “Planning temporal events using point-interval logic,” Mathematical and Computer Modelling, vol. 43, no. 9-10, pp. 1229–1253, 2006. View at Publisher · View at Google Scholar · View at Scopus