Spatiotemporal Relations and Modeling Motion Classes by Combined Topological and Directional Relations Method

Abstract

Defining spatiotemporal relations and modeling motion events are emerging issues of current research. Motion events are the subclasses of spatiotemporal relations, where stable and unstable spatio-temporal topological relations and temporal order of occurrence of a primitive event play an important role. In this paper, we proposed a theory of spatio-temporal relations based on topological and orientation perspective. This theory characterized the spatiotemporal relations into different classes according to the application domain and topological stability. This proposes a common sense reasoning and modeling motion events in diverse application with the motion classes as primitives, which describe change in orientation and topological relations model. Orientation information is added to remove the locative symmetry of topological relations from motion events, and these events are defined as a systematic way. This will help to improve the understanding of spatial scenario in spatiotemporal applications.

1. Introduction

Automatic event detection is gaining more and more attention in computer vision and video researchers community. Visual scene description takes into account ontological viewpoint of relative object positions. It is sufficient to emphasize the model of moving object’s spatial relations such as modeling video events [1, 2]. Modeling spatiotemporal relations between moving objects involves the modeling of motion events such as durative events. These events are the union of primitive events, which hold for each snapshot during the interval with a particular temporal order.

Spatiotemporal features could be used for modeling the spatiotemporal events [3]. Defining spatiotemporal relations have two main domains of research: spatiotemporal object and spatiotemporal relations modeling. Cuboid object approximation or three-dimensional geometry is used to model the former and for lateral two-dimensional objects occupy different spatial locations at different time points [4]. Several types of logics for mechanizing the spatiotemporal relations and reasoning process are used like interval temporal logic [5, 6], point temporal logic [7], and propositional model logic [8].

The point temporal logic supports instantaneous snapshots of the world. A snapshot represents the current situation, and a spatiotemporal relation is defined if a particular spatial relation holds for every snapshot during that interval. It is considered that time and space are bounded to each other. Spatiotemporal relations are modeled between moving objects by taking transaction from one snapshot to the next snapshot.

Dimiter Vakarelov, in [9, 10] provided strong mathematical and logical bases for defining the general spatiotemporal topological relations and divided them into two categories: (i) stable spatiotemporal topological relations and (ii) unstable spatiotemporal topological relations. A stable spatiotemporal topological relation is a relation which holds for every frame or snapshot in the interval and unstable spatiotemporal topological relations are those which hold at least for one snapshot during the temporal interval. Some spatiotemporal relations are strictly stable such as disjoint (D), nontangent proper part (NTPP), nontangent proper part inverse (NTPPI), equal (EQ), and others may be stable or unstable like meet (M), partially overlap (PO), tangent proper part (TPP), and tangent proper part inverse (TPPI). This provides a way to use the spatiotemporal relations in linguistics and their use as motion events modeling.

Most of the existing theories of spatiotemporal relations are domain-based, where domain knowledge imposes the conditions to a spatiotemporal relation to be topologically stable or unstable. A domain where the spatiotemporal relation is topologically stable, there the directional or distance relations are unstable such as spatiotemporal relations on the road networks. In defining motion events or verbs which represents the transitive movement, stability or unstability of topological relations plays important role and directional relations along with the topological relations remove certain symmetries and helps the user to understand real-scene situation. Consider the following two examples.(1)Mr. John (object, name of a person) crosses (relation) the football ground (object);(2)Mr. John (object, name of a person) crosses (relation) the football ground (object) from north to south.

In proposition 1, there is no confusion about the topological relation that object 𝐴 (John) has certain topological relation with object 𝐵 (football ground). But there is a symmetry about the directions, and user did not know the exact direction from object 𝐴 to object 𝐵 before and after the occurrence of cross event. But in proposition 2, when directional constraints are added, they remove the confusion about directions and symmetry of topological relations that object 𝐴 (john) crosses the object 𝐵 (ground) from north to south. It justifies that how the topological relations in two objects change, and what was the temporal order of occurrence the primitive events.

In our approach, we used a method of combined topological and directional (CTD) relations [11], more suitable for reasoning about the moving objects and developing the motion events. Situation is represented by relationship between the considered entities. It is natural to represent the information using relations. Events can be expressed by interpreting collective behavior of physical objects over a certain period of time. The main focus of this work is to formalize the spatiotemporal relations and the spatiotemporal events in a systematic way.

This paper is arranged as follows. Related work is discussed in Section 2 and Section 3 composed of preliminary definitions. Section 4 explains the combined topological and directional relations method, spatiotemporal relations are defined in Sections 5 and 6 compose on geometric representation of some motion events and these motion events are defined in Section 7. Section 8 concludes the paper.

2. Related Work

A moving object occupies different positions at different time instants. Relative motion means that the object changes its position with respect to another object. This relative motion can be studied through different aspects of space, and spatial relations are one of them. A set of spatial relations which hold for one snapshot is considered as a primitive event, then spatial relations between moving objects for an interval are characterized as spatiotemporal relations or spatiotemporal events.

Commonly adopted approaches are qualitative and domain-based such as qualitative trajectory calculus (QTC) [12, 13]. This describes relation between moving point objects. Hornsby and Egenhofer [14] modeled the different spatiotemporal relations between moving objects on road network. All these relations represent certain class of motion and objects are approximated as point objects. When objects are under motion, especially on road networks, the relations are purely directional relations, where the objects change their position, but do not change the topological structure of scene.

A mereotopological approach is extend to define spatiotemporal relations and a notation of temporal slice is used, where temporal slice is called an episode of history for a given interval [15, 16]. The primitive events can be defined using the Allen’s temporal logic and defining relation ℎ𝑜𝑙𝑑𝑠(P,i) (property P ℎ𝑜𝑙𝑑𝑠 during the time interval i) [17]. In this method, interval temporal logic is used, and a primitive temporal interval is defined, the smallest interval where the relation does not change. For composite events another property “occurs,” defined as occurs(e,i) = event 𝑒, occurs during the interval 𝑖, and different hold predicates are combined through logical connectors in a sequential order.

Ma and Mc Kevitt [18] described a method based on continuous transection from one state to another state. In this approach, topological relations are computed by the 9-intersection method [19]. This method supports instantaneous point temporal logic, which detects only changes in topological structure of scene at different instants of time. This method is based on point set topology and uses the snapshot model for spatiotemporal data [20–22]. This model of topological relations is used by Muller [23, 24] to model the motion events or motion classes. They model the different motion events which involve the topological changes at each analysis frame. Spatiotemporal relations between moving objects are also effected by the environment regarding its application domain such as modeling the relation cross, enter, leave shows that one object is only on concept level, that is, a region of interest. They are defined for a network, visual tracking, image understanding and activity recognition, or freely moving objects like modeling movement behavior of animals.

In [10] provided the strong mathematical and logical bases for defining the general spatiotemporal topological relations and divided them into two categories, namely, stable and unstable spatiotemporal relations. Both stable and unstable spatiotemporal relations play an important role in modeling the spatiotemporal events. World is represented as situation (a primitive event), and action is simply a function from one situation to another. A single snapshot represents a primitive event at an instant 𝑡. Events are embedded in time, they have temporal boundaries, they have their relationship to time. They do not occupy space, but they are related to space.

Spatiotemporal events are defined as composite events, how their different parts (primitive events) are interrelated. A property ℎ𝑜𝑙𝑑𝑠_𝑎𝑡(P,t) (property P holds at time instant 𝑡) is used along with the instantaneous temporal logic. The primitive events are defined for each snapshot during an interval 𝑇 using the Allen’s temporal logic and defining the relation ℎ𝑜𝑙𝑑𝑠(P,T) (property P holds during the time interval T) [17], and a relationship between holds and ℎ𝑜𝑙𝑑𝑠_at can be represented as ℎ𝑜𝑙𝑑𝑠(𝑃,𝑇)=forall𝑡∈𝑇, ℎ𝑜𝑙𝑑𝑠_𝑎𝑡(𝑃,𝑡). This provides us that a property 𝑃 holds for an interval 𝑇 if it holds for every point during the interval. In this method, interval temporal logic is used, and a primitive temporal interval is defined, if 𝑇 is a zero duration interval, then it represents a snapshot.

Motion events are the subclasses of spatiotemporal relations with a temporal ordering in a primitive actions. Motion events, they do not formulate the necessitate of a calculus, they are only logical representation and temporal ordering of existence of different primitive events. Modeling the motion events, where property (P) changes at each instant, it is more suitable to use the sequential logic and use relation, 𝑠𝑒𝑞_𝑒𝑣𝑒(𝑡,𝑒₁,𝑒₂,𝑠) (event 𝑒₁ occurs before 𝑒₂ in S during time 𝑡). Composite events are the initial conditions dependant, when an initial primitive event occurs at a certain time point 𝑡₀, it set up the superclass and name of the possible composite event to be happening.

Topological relations have a certain type of locative symmetries, they do not explain the symmetric location of path and motion direction of argument objects. To remove the symmetry of spatial relations about the locative perspective, relevant spatial orientation is added. In language semantics, motion events are divided into three classes: an initial, the median, and terminal [25]. Some sentences can be explained with the help of a single directional relation such as enter, release, touch, and some need two directions like cross and graze.

We used CTD method [11] to develop such motion events, where topological components play role for defining the motion events, directional components are used to overcome the locative symmetries and locative perspective and for other class of motion events, topological components can be used for controlling variables, and directional components become important. We hope this paper will create a bridge between the two approaches of modeling the spatiotemporal events, approach based on interval logic, and that of point logic.

3. Preliminary Definitions

In this section, we recall some basic definitions which are frequently used throughout the remainder of the paper.Fuzzy set: a fuzzy set 𝐴 in a set 𝑋 is a set of pairs(𝑋,𝜇_𝐴(𝑥))suchthat𝐴=𝑥,𝜇_𝐴(𝑥)∣𝑥∈𝑋.(1)Fuzzy membership function: a membership function 𝜇 in a set 𝑋 is a function 𝜇∶𝑋→[0,1]. Different fuzzy membership functions are proposed according to the requirements of the applications. For instance, Trapezoidal membership function is defined as𝜇(𝑥;𝛼,𝛽,𝛾,𝛿)=maxmin𝑥−𝛼𝛽−𝛼,1,𝛿𝛿−−𝑥𝛾,0,(2) it is written as 𝜇_{(𝛼,𝛽,𝛾,𝛿)}(𝑥), where 𝑥,𝛼,𝛽,𝛾,𝛿∈ℝ∧𝛼<𝛽≤𝛾<𝛿.Force histogram: the force histogram attaches a weight to the argument object 𝐴 that this lies after 𝐵 in direction 𝜃, it is defined as 𝐅^𝐴𝐵(𝜃)=^+∞−∞𝐹𝜃,𝐴_𝜃(𝑣),𝐵_𝜃(𝑣)𝑑𝑣.(3) The definition of force histogram 𝐅^𝐴𝐵(𝜃), directly depends upon the definition of real-valued functions 𝜙,𝑓, and 𝐹 used for the treatment of points, segments, and longitudinal sections, respectively [26]. These functions are defined as𝜙_𝑟(𝑦)=𝑦1^𝑟𝑓𝑥0oitfhe𝑦rw>is0e,_𝐼,𝑦^𝜃𝐼𝐽,𝑧_𝐽=^{𝑥𝐼+𝑦𝜃𝐼𝐽+𝑧𝐽}_{𝑥𝐼+𝑦𝜃𝐼𝐽}^𝑧𝐽₀𝐹𝜃,𝐴_𝜃𝜙(𝑢−𝑤)𝑑𝑤𝑑𝑢,(𝑣),𝐵_𝜃(𝑣)=_{𝑖=1⋯𝑛,𝑗=1⋯𝑚}𝑓𝑥_𝐼𝑖,𝑦^{𝜃𝐼𝑖𝐽𝑗},𝑧_𝐽𝑗,(4) where 𝑛and 𝑚 represent the number of segments of object 𝐴 and object 𝐵, respectively, and variables (x, y, z) are explained in Figure 1. These are the definitions of Force histograms, directly depending upon the definition of function 𝜙. 𝐅^𝐴𝐵(𝜃) is actually a real-valued function.

Figure 1 

Oriented line Δ𝜃(𝑣), segment as in case of object 𝐵, longitudinal section as in case of object 𝐴[27].

4. Combined Topological and Directional Relations Method

In this section, we explain different steps of the combined topological and directional relations method. This explains different terms used in computation of combined topological and directional relations.

4.1. Oriented Lines, Segments, and Longitudinal Sections

Let 𝐴 and 𝐵 be two spatial objects and (𝑣,𝜃)∈ℝ, where 𝑣 is any real number and 𝜃∈[0,2𝜋]. Δ_𝜃(𝑣) is an oriented line at orientation angle 𝜃, and 𝐴∩Δ_𝜃(𝑣) is the intersection of object 𝐴 and oriented line Δ_𝜃(𝑣). It is denoted by 𝐴_𝜃(𝑣), called segment of object 𝐴 and length of its projection interval on x-axis is 𝑥. Similarly for object 𝐵, where 𝐵∩Δ_𝜃(𝑣)=𝐵_𝜃(𝑣) is segment and length of its projection interval on x-axis is 𝑧, 𝑦 is the difference between the minimum of projection points of 𝐴∩Δ_𝜃(𝑣) and maximum value of projection points of 𝐵∩Δ_𝜃(𝑣)(for details [27]).

In case of polygonal object approximation (𝑥,𝑦,𝑧) can be calculated from intersecting points of line and object boundary, oriented lines are considered which passes through at least one vertex of two polygons. If there exist more than one segment, then it is called longitudinal section as in case of 𝐴_𝜃(𝑣) in Figure 1.

4.2. Allen Temporal Relations in Spatial Domain and Fuzziness

Allen [5] introduced the 13 jointly exhaustive and pairwise disjoint (JEPD) interval relations. These relations are 𝒜={<,𝑚,𝑜,𝑠,𝑓,𝑑,𝑒𝑞,𝑑_𝑖,𝑓_𝑖,𝑠_𝑖,𝑜_𝑖,𝑚_𝑖,>} with meanings before, meet, overlap, start, finish, during, equal, during_by, finish_by, start_by, overlap_by, meet_by, and after. All the Allen relations in space are conceptually illustrated in Figure 2.

Figure 2 

Black segment represents the reference object, and gray segment represents argument object.

These relations have a rich support for the topological relations and represent the eight topological relations in one-dimensional spatial domain. Fuzzy Allen relations are used to represent the fuzzy topological relations, where vagueness or fuzziness is represented at the relation’s level.

Fuzzification process of Allen relations do not depend upon particular choice of fuzzy membership function. Trapezoidal membership function is used due to flexibility in shape. Let 𝑟(𝐼,𝐽) be an Allen relation between segments 𝐼 (segment of an argument object) and 𝐽 (segment of an reference object), 𝑟′ is the distance between 𝑟(𝐼,𝐽) and its conceptional neighborhood. We consider a fuzzy membership function 𝜇∶𝑟′→[01]. The fuzzy Allen relations defined in [28] as𝑓_<(𝐼𝑓,𝐽)=𝜇_{(−∞,−∞,−𝑏−3𝑎/2,−𝑏−𝑎)}(𝑦)_>𝑓(𝐼,𝐽)=𝜇_{(0,𝑎/2,∞,∞)}(𝑦)_𝑚(𝐼,𝐽𝑓)=𝜇_{(−𝑏−3𝑎/2,−𝑏−𝑎,−𝑏−𝑎,−𝑏−𝑎/2)}(𝑦)_𝑚𝑖𝑓(𝐼,𝐽)=𝜇_{(−𝑎/2,0,0,𝑎/2)}(𝑦)_𝑂𝑓(𝐼,𝐽)=𝜇_{(−𝑏−𝑎,−𝑏−𝑎/2,−𝑏−𝑎/2,−𝑏)}(𝑦)_𝑜𝑖𝑓(𝐼,𝐽)=𝜇_{(−𝑎,−𝑎/2,−𝑎/2,0)}(𝑦)_𝑓(𝐼,𝐽)=mi𝜇n𝜇_{(−(𝑏+𝑎)/2,−𝑎,−𝑎,+∞)}(𝑦),_{(−3𝑎/2,−𝑎,−𝑎,−𝑎/2)}𝜇(𝑦),_{(−∞,−∞,𝑧/2,𝑧)}𝑓(𝑥)_𝑓𝑖(𝐼,𝐽)=m𝜇in𝜇_{−𝑏−𝑎/2,−𝑏,−𝑏,−𝑏+𝑎/2}(𝑦),_{(−∞,−∞,−𝑏,−(𝑏+𝑎)/2)}𝜇(𝑦),_{(𝑧,2𝑧,+∞,+∞)}𝑓(𝑥)_𝑠(𝐼,𝐽)=m𝜇in𝜇_{−𝑏−𝑎/2,−𝑏,−𝑏,−𝑏+𝑎/2}(𝑦),_{(−∞,−∞−𝑏,−(𝑏+𝑎)/2)}𝜇(𝑦),_{(−∞,−∞,𝑧/2,𝑧)}𝑓(𝑥)_𝑠𝑖(𝐼,𝐽)=mi𝜇n𝜇_{(−(𝑏+𝑎)/2,−𝑎,−𝑎,+∞)}(𝑦),_{(−3𝑎/2,−𝑎,−𝑎,−𝑎/2)}𝜇(𝑦),_{(𝑧,2𝑧,+∞,+∞)}𝑓(𝑥)_𝑑(𝐼,𝐽)=min𝜇𝜇_{(−𝑏,−𝑏+𝑎/2,−3𝑎/2,−𝑎)}(𝑦),_{(−∞,−∞,𝑧/2,𝑧)}𝑓(𝑥)_𝑑𝑖(𝐼,𝐽)=min𝜇𝜇_{(−𝑏,−𝑏+𝑎/2,−3𝑎/2,−𝑎)}(𝑦),_{(𝑧,2𝑧,+∞,+∞)}(𝑥),(5) where 𝑎=min(𝑥,𝑧),𝑏=max(𝑥,𝑧), 𝑥 is the length of segment (𝐼), 𝑧 is the length of segment (𝐽), and (𝑥,𝑦,𝑧) are computed as described in Section 4.1.

Most of relations are defined by one membership like 𝑑(𝑑𝑢𝑟𝑖𝑛𝑔), 𝑑_𝑖(𝑑𝑢𝑟𝑖𝑛𝑔_𝑏𝑦), 𝑓(𝑓𝑖𝑛𝑖𝑠ℎ), and 𝑓_𝑖(𝑓𝑖𝑛𝑖𝑠ℎ𝑒𝑑_𝑏𝑦). In fuzzy set theory, sum of all the relations is one, this gives the definition for fuzzy relation 𝑒𝑞𝑢𝑎𝑙. These are the topological relations which represent the fuzziness at relation’s level, for example, here Meet topological relation is represented based on nearness, and length of the smaller interval defines the smooth transition between the Meet(Meet_by) and before(after) relation. In spatial domain, before(after) are called the disjoint topological relations. These relations have the following properties:𝑓_<𝑓(𝜃)=𝑓_>(𝜃+𝜋),𝑓_𝑚(𝜃)=𝑓_𝑚𝑖(𝜃+𝜋),_𝑜𝑓(𝜃)=𝑓_𝑜𝑖(𝜃+𝜋),𝑓_𝑓(𝜃)=𝑓_𝑠(𝜃+𝜋),_𝑓𝑖𝑓(𝜃)=𝑓_𝑠𝑖(𝜃+𝜋),𝑓_𝑑(𝜃)=𝑓_𝑑(𝜃+𝜋),_𝑑𝑖(𝜃)=𝑓_di(𝜃+𝜋),𝑓=(𝜃)=𝑓=(𝜃+𝜋).(6)

Eight relations are possible combination of eight independent Allen relations in one-dimensional spatial domain. These relations and their reorientation show that the whole 2D space can be explored with the help of 1D Allen relations using the oriented lines varying from (0, 𝜋).

4.3. Combining Topological and Directional Relations

Eight topological relations represented in point set topology or point less topology between object pair are represented in one-dimensional space by the Allen’s temporal relations in spatial domain. We extend these Allen relations for two-dimensional objects through logical implication, where a 2D object is decomposed into parallel segments of a 1D lines in a given direction, and the relation between each pair of line segments is computed.

The process of object decomposition is repeated for each direction varying from 0 to 𝜋, two-dimensional topological relations are then defined as it provides us with information about how the objects are relatively distributed. These relations are not jointly exhaustive and pairwise disjoint (JEPD), to obtain JEPD set of topological and directional relations an algorithm was advocated in [11], it provides us with the JEPD set of relations. Objects are approximated through the polygon object approximation. Different steps of computing the combined topological and directional relations are(i)fix angle 𝜃 and draw lines passing through the vertices of polygons representing the objects;(ii)for each line, compute the variables (𝑥,𝑦,𝑧) as depicted in Section 4.1 and compute Allen relation for each segment as given by (5). In case of longitudinal sections, use fuzzy operators to integrate the information, usually the disjunction operators are suitable. These relations are computed for each line in a direction, then obtained information is integrated into a single value. Normalize these relations for a direction 𝜃 by dividing sum of all Allen relations to each Allen relation;(iii)these normalized fuzzy Allen relation is then multiplied to a fuzzy directional set to find the degree of an Allen relation in a direction;(iv)for qualitative directions, this information is summarized, and different topological relations with directional contents are defined, such as 𝑓_𝐸=^𝜋/4_𝜃=0𝒜_𝑟2×cos²(2𝜃)+^𝜋_𝜃=_3𝜋/4𝒜_𝑟1×cos²(2𝜃),(7) where 𝑓 represents a topological relation, and 𝐸 represents the direction 𝐴_𝑟1 is the reorientation of 𝐴_𝑟2; (v)this information is represented in a matrix, this matrix represents fuzzy spatial information;(vi)these fuzzy spatial relations are defuzzified by an algorithm, this provides us with final topological and directional relations between the object pair. These topological and directional relations are JEPD.

This model describes well the possible topological relations between every sort of objects.

5. Spatiotemporal Relations

Spatiotemporal relations can be defined as spatial relation holds for an interval, that is, relation holds for a certain time interval, and it does not change. In spatiotemporal object theory it is defined as (P) a spatial relation is a relation holding between all temporal slices of two entities during the relevant period. All eight spatiotemporal relations are defined in terms of theorems.

5.1. Spatiotemporal Relation

Theorem 1. A spatiotemporal disjoint relation between object pair (𝑋,𝑌) holds during the interval 𝑇, that is, 𝐷(𝑋𝑌,𝑇)⇔forall𝑡∈𝑇,𝐷(𝑋𝑌,𝑡) holds.

Proof. (⇒) A spatiotemporal 𝑑𝑖𝑠𝑗𝑜𝑖𝑛𝑡𝐷(𝑋𝑌,𝑇) relation is defined as object pair (𝑋,𝑌) are disjoint during the interval 𝑇 if 𝑋≡_𝑡𝑌(𝑋 is temporally equivalent to 𝑌). Now let us consider the partition of interval 𝑇=[𝑡_𝑎𝑡_𝑏], then its partition can be taken as 𝑡_𝑎=𝑡₁<𝑡₂<𝑡₃⋯<𝑡_𝑛=𝑡_𝑏. Each 𝑡_𝑖∈𝑇,𝑖=1,2,…,𝑛 represents discrete points of interval 𝑇, and this representation is equivalent to a snapshot. Typically a snapshot is a sampling process, which represents zero duration temporal slice of a spatiotemporal object. There are 𝑛 snapshots in interval 𝑇, as a result a disjoint topological relation exists for each snapshot separately. Thus, for all 𝑡∈𝑇,𝐷(𝑋𝑌,𝑡) holds.
(⇐) Let us consider 𝑛 snapshots where the temporal ordering holds, that is, 𝑡₁,𝑡₂,⋯,𝑡_𝑛 such that 𝑡₁<𝑡₂<𝑡₃⋯<𝑡_𝑛 and all these points form partition of an interval 𝑇. If the disjoint topological relation holds at discrete points, it means that 𝐷(𝑋𝑌,𝑡₁)∧𝐷(𝑋𝑌,𝑡₂)→𝐷(𝑋𝑌,[𝑡₁𝑡₂]). If the disjoint topological relation holds between object pair, it means that both the objects are temporally equivalent (𝑥≡_𝑡𝑦). Hence, 𝐷(𝑋𝑌,𝑇) holds during the whole interval 𝑇.

Theorem 2. A spatiotemporal relation Meet M(XY,T) holds ⇔∃𝑡_𝑖∈𝑇 such that 𝑀(𝑋𝑌,𝑡_𝑖)∧for all 𝑡_𝑗∈𝑇∧𝑡_𝑗≠𝑡_𝑖⇒𝐷(𝑋𝑌,𝑡_𝑗) holds.

Proof. (⇒) A spatiotemporal relation meet 𝑀(𝑋𝑌,𝑇) holds between object pair (𝑋,𝑌) over interval 𝑇, where 𝑋≡_𝑡𝑌. Let 𝑡_𝑎=𝑡₁<𝑡₂<𝑡₃<⋯<𝑡_𝑛=𝑡_𝑏 be partition of interval 𝑇=[𝑡_𝑎𝑡_𝑏], if forall𝑡_𝑖∈𝑇,𝑀(𝑋𝑌,𝑡_𝑖) holds, then a stable topological relation 𝑀(𝑋𝑌,𝑇) holds. We consider on contrary, that ∃𝑡_𝑗, where the topological relation 𝑀(𝑋𝑌,𝑡_𝑗) does not hold, but it holds at 𝑀(𝑋𝑌,𝑡_𝑗−1), then according to the temporal logic and continuity of topological relations ○(𝑀(𝑋𝑌,𝑡_𝑗−1))⇒(𝐷(𝑋𝑌,𝑡_𝑗)∨𝑀(𝑋𝑌,𝑡_𝑗)∨PO(𝑋𝑌,𝑡_𝑗). This shows that any of the three relations is possible (○stands for future position). If PO(𝑋𝑌,𝑡_𝑗) holds, then the whole spatiotemporal relation is changed, and it becomes the spatiotemporal partial overlap relation. This possibility is ruled out. In other case, spatiotemporal relation remains meet, as 𝑗 is an arbitrary variable, this shows the minimum condition. Hence, ∃𝑡∈𝑇,s.t.𝑀(𝑋𝑌,𝑡) holds.
(⇐) Let us consider that there are 𝑛 snapshots in an order, which construct an interval 𝑇. Now consider that there exists at least one snapshot during whole interval, where spatial meet relation holds, and for all other snapshots, the spatial relation is disjoint. This shows that during temporal interval 𝑇, the unstable spatiotemporal meet relation holds. It satisfies the minimum conditions for a spatiotemporal meet relation, hence 𝑀(𝑋𝑌,𝑇) holds during interval 𝑇.

Theorem 3. A spatiotemporal partial overlap (PO) relation holds over interval 𝑇, that is, PO(𝑋𝑌,𝑇)⇔∃𝑡∈𝑇,s.t.PO(𝑋𝑌,𝑡).

Proof. Spatiotemporal relations have the spatial and temporal boundaries. A stable spatiotemporal relation holds during the temporal slice, If it holds at every point of the interval. As temporal slice is the union of finite points of temporal domain, spatiotemporal partial overlap holds during the whole slice, if this relation holds at least one sampling point (snapshot), at remaining points any of the spatial relation may exist. Hence∃𝑡∈𝑇,s.t.,PO(𝑋𝑌,𝑡)∧(𝑡₁≠𝑡,CO(𝑋𝑌,𝑡₁)∨𝑀(𝑋𝑌,𝑡₁)∨𝐷(𝑋𝑌,𝑡₁)) (CO(𝑋𝑌) stands for complete overlap of objects (𝑋𝑌), s.t., CO(𝑋𝑌)=TPP(𝑋𝑌)∨NTPP(𝑋𝑌)∨TPPI(𝑋𝑌)∨NTPPI(𝑋𝑌)). If there does not exist such a 𝑡₁, then the relation holds for every 𝑡∈𝑇, which shows that a stable PO topological relation holds.
(⇐) We suppose on contrary that ∄𝑡∈𝑇,s.t.PO(𝑋𝑌,𝑡) holds. It means that at all points either the binary topological relations are complete overlap or disjoint and meet. If the relations are complete overlap, that is, for all 𝑡∈𝑇,CO(𝑋𝑌,𝑡) holds, then the spatiotemporal relation will be a part of complete overlap. In case of other choice that ∃𝑡∈𝑇, s.t.,𝑀(𝑋𝑌,𝑡) or for all 𝑡∈𝑇,s.t.,𝑀(𝑋𝑌,𝑡) holds, then the spatiotemporal topological relation will be stable or unstable meet and for case for all 𝑡∈𝑇,s.t.,𝐷(𝑋𝑌,𝑡), the topological relation will be disjoint. The choice, that ∃𝑡₁∈𝑇,s.t.,𝑀(𝑋𝑌,𝑡) and ∃𝑡₂∈𝑇,s.t.,CO(𝑋𝑌,𝑡₂ holds is impossible because in a such a case ∃𝑡∈𝑇∧𝑡₁<𝑡<𝑡₂, s.t.,PO(𝑋𝑌,𝑡) holds (continuity of spatial relations).

Theorem 4. A spatiotemporal tangent proper part (TPP) relation holds over interval 𝑇, that is,TPP(𝑋𝑌,𝑇)⇔∃𝑡₁∈𝑇,such thatTPP(𝑋𝑌,𝑡₁)for all 𝑡₂∧𝑡₂≠𝑡₁, NTPP(𝑋𝑌,𝑡₂) holds.

Proof. (⇒) A spatiotemporal topological relation TPP(𝑋𝑌) holds between the object pair 𝑋,𝑌 during the interval 𝑇. Now let us consider that interval 𝑇 consist of n snapshots, if this relation holds for every snapshot then a spatiotemporal stable topological relation holds. In other case, there are two possibilities that for all 𝑡₁≠𝑡₂, there exists a topological relation TPP(𝑋𝑌,𝑡₁), and for 𝑡₂ either the topological relation is PO(𝑋𝑌,𝑡₂) or NTPP(𝑋𝑌,𝑡₂) due to continuity of topological relations between moving objects. For PO(𝑋𝑌,𝑡₂), the spatiotemporal topological relation is changed, and it becomes the spatiotemporal PO topological relation, this possibility is ruled out. It remains that NTPP(𝑋𝑌,𝑡₂), if this relation holds and 𝑡₂ is an arbitrary point, so the relation becomes the unstable spatiotemporal TPP(𝑋𝑌,𝑇).
(⇐) Consider that ∃𝑡_𝑖 such that TPP(𝑋𝑌,𝑡_𝑖) holds. We consider on contrary that ∃𝑡_𝑖+1∨𝑡_𝑖−1 such that NTPP(𝑋𝑌,𝑡_𝑖−1) or NTPP(𝑋𝑌,𝑡_𝑖+1) does not holds. Then, possible topological relations at 𝑡_𝑖−1 are TPP(𝑋𝑌,𝑡_𝑖−1),PO(𝑋𝑌,𝑡_𝑖−1) similarly for 𝑡_𝑖+1. Other possibilities are ruled out due to the continuity of topological relations, and EQ(𝑋𝑌,𝑡_𝑖−1) does not hold because objects are considered under motion, and expansion or zooming of one object is not allowed.
In case the topological relation PO(𝑋𝑌,𝑡_𝑖−1) holds, then the whole spatiotemporal relation over the interval 𝑇 becomes partial overlap. Similarly for instant 𝑡_𝑖+1 and 𝑖 is an arbitrary point, so this is impossible for whole the interval 𝑇. For the topological relation TPP(𝑋𝑌,𝑡_𝑖−1), the spatiotemporal relation becomes the stable spatiotemporal TPP.

Theorem 5. A spatiotemporal nontangent proper part (NTPP) relation holds over interval 𝑇, that is, NTPP(𝑋𝑌,𝑇)⇔ for all𝑡∈𝑇,NTPP(𝑋𝑌,𝑡) holds.

Proof. (⇒) Let us suppose on contrary that ∃𝑡_𝑖∈𝑇s.t.NTPP(𝑋𝑌,𝑡_𝑖) does not hold, and at temporal points 𝑡_𝑖−1,𝑡_𝑖+1 the relation NTPP(𝑋𝑌,𝑡_𝑖−1),NTPP(𝑋𝑌,𝑡_𝑖+1), holds. Then continuity of spatial relations forces the existence of TPP(𝑋𝑌,𝑡_𝑖) or EQ(𝑋𝑌,𝑡_𝑖) spatial relations. This contradicts the existence of the spatiotemporal NTPP(𝑋𝑌,𝑇) relation. Hence, forall𝑡∈𝑇,NTPP(𝑋𝑌,𝑡) holds.
(⇐) It is given that forall𝑡∈𝑇,NTPP(𝑋𝑌,𝑡) holds. If a spatial relation between object pair holds at every point of the interval, then it means that it holds throughout the interval, that is, NTPP(𝑋𝑌,𝑇) holds.

Theorem 6. A spatiotemporal tangent proper part inverse (TPP) relation holds over interval 𝑇, that is, TPPI(𝑋𝑌,𝑇)⇔∃𝑡∈𝑇,s.t.TPPI(𝑋𝑌,𝑡)∧forall𝑡₁≠𝑡,NTPPI(𝑋𝑌,𝑡₁) holds.

Proof. Proof is similar to the TPP(𝑋𝑌,𝑇), just replace TPP by TPPI and NTPP by NTPPI.

Theorem 7. A spatiotemporal nontangent proper part inverse (NTPPI) relation holds over temporal interval 𝑇, that is, NTPPI(𝑋𝑌,𝑇)⇔forall𝑡∈𝑇andNTPPI(𝑋𝑌,𝑡) holds.

Proof. Proof is similar to the NTPP(𝑋𝑌,𝑇).

Theorem 8. A spatiotemporal relation equal(EQ) holds between the object pair 𝑋𝑌, EQ(𝑋𝑌,𝑇)⇔forall𝑡∈𝑇, s.t.,EQ(𝑋𝑌,𝑡) holds.

Proof. (⇒) We suppose on contrary that there exists a 𝑡∈𝑇, where the EQ(𝑋𝑌,𝑡) relation does not hold. It shows that there are two possibilities that either the relation at 𝑡 is a complete overlap or partial overlap. If the relation at 𝑡 is complete overlap, then the spatiotemporal relation becomes TPP or TPPI. In the second case, the spatiotemporal relation becomes the PO(𝑋𝑌,𝑇) during the whole interval. Thus, both cases prove the contrary conditions, hence ∄𝑡s.t.EQ(𝑋𝑌,𝑡) does not hold, that is, for all𝑡∈𝑇,EQ(𝑋𝑌,𝑡) holds.
(⇐) Converse of this proof is very simple and straight forward. Let 𝑇 be the interval for which we have to define the spatiotemporal relation, both the objects are temporally comparable (𝑋≡_𝑡𝑌). Let 𝑡∈𝑇 be an arbitrary point of the interval and relation EQ(𝑋𝑌,𝑡) holds. Since 𝑡 is an arbitrary point so, the relation holds throughout the interval 𝑇, that is, EQ(𝑋𝑌,𝑇) holds.

6. Visual Interpretation: A Three-Dimensional View

Geometrical figures can better elaborate concepts, a moving object changes its position at each instant 𝑡. These objects in a spatiotemporal domain can be represented by their envelops, a two-dimensional object becomes volume. Here, spatiotemporal meet and partially overlap relations are represented by their envelops in Figures 3(a)–3(d) and 4(a)–4(h). These are possible representation of motion events. Spatial relations between moving objects are used in modeling the motion verbs or motion events in natural language processing. A set of motion relations is introduced that capture semantic between pairs of moving objects. This information is useful about reasoning the moving objects.

(a) touch

(b) excurse

(c) snap

(d) release

Figure 3 

Spatiotemporal Meet relation (unstable meet).

(a) enter

(b) leave

(c) cross

(d) graze

(e) into

(f) out of

(g) melt

(h) spring

Figure 4 

Spatiotemporal partial_Overlap relation (unstable overlap).

7. Modeling Motion Classes

Visual images may illustrate cases of a definition, giving us a more visual grasp of its applications. They may help us understanding the description of a mathematical situation or steps in reasoning. These relations can be defined as the transection of relations at time 𝑡₁ to 𝑡₂. This change may be in topological or metric relations, and different classes of spatial relations, between moving objects have been defined [12, 13, 29, 30]. Motion classes based on intuitive logics or motion verbs have been defined in [31] by Phillipe Muller and Ralf H. Güting and Markus Schneider used in database. We define in this paper only the motion events, where topological relations capture changes between situations. These motion events can be defined using the predicates ℎ𝑜𝑙𝑑𝑠_𝑎𝑡, ℎ𝑜𝑙𝑑𝑠,𝑜𝑐𝑐𝑢𝑟𝑠_𝑎𝑡,   𝑜𝑐𝑐𝑢𝑟𝑠 and 𝑠𝑒𝑞𝑢𝑒𝑛𝑐𝑒. In next section, 𝑠𝑒𝑞_𝑒𝑣𝑒 represents the sequence event.

7.1. Unstable Meet Spatiotemporal Relation

Unstable spatiotemporal relation is a relation where objects changes their states at each time instant. A spatiotemporal meet relation is characterized by different motion events depending upon the logical and temporal order of different states or primitive events. Touch(XY,T): A spatiotemporal meet relation can be characterized as a motion event Touch, s.t. ∃𝑡₁,𝑡₂,𝑡₃,∈𝑇and𝑡₁<𝑡₂<𝑡₃, where primitive events occur in an order and defined as𝑇𝑜𝑢𝑐=ℎ(𝑠𝑋𝑒𝑞𝑌_,𝑇𝑒𝑣)𝑒ℎ𝑜𝑙𝑑𝑠𝐷𝑋𝑌,𝑡₁∧ℎ𝑜𝑙𝑑𝑠𝐷𝑋𝑌,𝑡₃∧ℎ𝑜𝑙𝑑𝑠𝑀𝑋𝑌,𝑡₂.(8)

An institutive view of this spatiotemporal relation is shown in Figure 3(a). This relation can be expressed by a single direction, where a meet topological relation holds. It means𝐷𝑖𝑟(𝑇𝑜𝑢𝑐ℎ(𝑋𝑌,𝑇))=ℎ𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡₂.(9)Snap(XY,T): A spatiotemporal meet relation is called a Snap if ∃𝑡₁,𝑡₂∈𝑇and𝑡₁<𝑡₂ such that𝑆𝑛𝑎𝑝=(𝑋𝑠𝑒𝑌𝑞,_𝑇𝑒)𝑣𝑒ℎ𝑜𝑙𝑑𝑠𝐷𝑋𝑌,𝑡₁∧ℎ𝑜𝑙𝑑𝑠𝑀𝑋𝑌,𝑡₂.(10)

A geometric representation is shown in Figure 3(c). This relation can be expressed by a single direction, where a meet topological relation holds. It means𝐷𝑖𝑟(𝑆𝑛𝑎𝑝(𝑋𝑌,𝑇))=ℎ𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡₂.(11)Release(XY,T): A spatiotemporal 𝑀𝑒𝑒𝑡(𝑋𝑌,𝑇) is called 𝑅𝑒𝑙𝑒𝑎𝑠𝑒(𝑋𝑌,𝑇), read as 𝑋 releases 𝑌 during interval 𝑇 if it has a certain temporal ordering, ∃𝑡₁,𝑡₂∈𝑇and𝑡₁<𝑡₂such that𝑅𝑒𝑙𝑒=𝑎𝑠𝑠𝑒𝑒(𝑞𝑋_𝑌𝑒,𝑣𝑇𝑒)ℎ𝑜𝑙𝑑𝑠𝐷𝑋𝑌,𝑡₁∧ℎ𝑜𝑙𝑑𝑠𝑀𝑋𝑌,𝑡₁.(12)

A three-dimensional geometric view of this relation is shown in Figure 3(d). This relation can be expressed by a single direction, which is the destination direction. For example, object 𝑋  𝑟𝑒𝑙𝑒𝑎𝑠𝑒𝑠 (motion event) object 𝑌 towards East (destination direction). Direction for such a relation is defined as𝐷𝑖𝑟(𝑅𝑒𝑙𝑒𝑎𝑠𝑒(𝑋𝑌,𝑇))=ℎ𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡₂.(13)Bypass(XY,T): A spatiotemporal 𝑀𝑒𝑒𝑡(𝑋𝑌,𝑇) is called a 𝐵𝑦𝑝𝑎𝑠𝑠(𝑋𝑌,𝑇), read as 𝑋 bypasses 𝑌 during interval 𝑇 if it has a certain temporal ordering, that is, ∃𝑡₁,𝑡₂,𝑡₃,𝑡₄∈𝑇 such that𝑡₁<𝑡₂<𝑡₃<t₄𝐵𝑦𝑝𝑎=𝑠𝑠𝑠(𝑒𝑋𝑞_𝑌𝑒,𝑣𝑇𝑒)ℎ𝑜𝑙𝑑𝑠𝐷𝑋𝑌,𝑡₁∧ℎ𝑜𝑙𝑑𝑠𝑀𝑋𝑌,𝑡₂∧ℎ𝑜𝑙𝑑𝑠𝑀𝑋𝑌,𝑡₃∧ℎ𝑜𝑙𝑑𝑠𝐷𝑋𝑌,𝑡₄.(14)

This relation can be expressed by a single direction, where a meet topological relation holds. It means𝐷𝑖𝑟(𝑇𝑜𝑢𝑐ℎ(𝑋𝑌,𝑇))=ℎ𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡₂.(15)Excurse(XY,T): A spatiotemporal Meet(XY,T) is called a 𝐸𝑥𝑐𝑢𝑟𝑠𝑒(𝑋𝑌,𝑇), read as 𝑋 excurse 𝑌 during interval 𝑇 if it has a certain temporal ordering, an intuitive view of this relation is shown in Figure 3(b). ∃𝑡₁,𝑡₂,𝑡₃∈𝑇,s.t.,𝑡₁<𝑡₂<𝑡₃𝐸𝑥𝑐𝑢=𝑟𝑠𝑠𝑒𝑒(𝑞𝑋_𝑒𝑌𝑣,𝑒𝑇)ℎ𝑜𝑙𝑑𝑠𝑀𝑋𝑌,𝑡₁∧ℎ𝑜𝑙𝑑𝑠𝑀𝑋𝑌,𝑡₃∧ℎ𝑜𝑙𝑑𝑠𝐷𝑋𝑌,𝑡₂.(16)

This relation is expressed by an initial and destination directions, the direction for this relation can be defined as𝐷𝑖𝑟(=𝐸𝑥𝑠𝑐𝑒𝑢𝑞𝑟_𝑠𝑒𝑒𝑣(𝑒𝑋𝑌ℎ,𝑜𝑇𝑙𝑑)𝑠)𝐷𝑖𝑟𝑋𝑌,𝑡₁∧ℎ𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡₃.(17)

7.2. Unstable Overlap Spatiotemporal Relation

Enter(XY,T): An unstable spatiotemporal overlap relation is called Enter, generally denoted by Enter(𝑋𝑌,𝑇) and read as “𝑋 enters in 𝑌 during interval 𝑇.” If ∃𝑡₁,𝑡₂,𝑡₃,𝑡₄∈𝑇suchthat𝑡₁<𝑡₂<𝑡₃<𝑡₄, then relation is defined as

𝐸𝑛𝑡𝑒=𝑟(𝑠𝑋𝑒𝑌𝑞,_𝑇𝑒𝑣)𝑒ℎ𝑜𝑙𝑑𝑠𝐷𝑋𝑌,𝑡₁∧ℎ𝑜𝑙𝑑𝑠𝑃𝑂𝑋𝑌,𝑡₃∧ℎ𝑜𝑙𝑑𝑠𝑀𝑋𝑌,𝑡₂∧ℎ𝑜𝑙𝑑𝑠TPP𝑋𝑌,𝑡₄.(18)

An intuitive view of this relation is shown in Figure 4(a). This relation can be expressed by a single direction because the destination point is inside and can be expressed without direction, a direction for the Enter spatiotemporal event is the direction where a meet topological relation holds, that is,𝐷𝑖𝑟(𝐸𝑛𝑡𝑒𝑟(𝑋𝑌,𝑇))=ℎ𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡₂.(19)Leave(𝑋𝑌,𝑇): A spatiotemporal partial overlap relation is called Leave, denoted as Leave(XY,T) “𝑋 leaves 𝑌 during interval 𝑇”. If ∃𝑡₁,𝑡₂,𝑡₃,𝑡₄∈𝑇suchthat𝑡₁<𝑡₂<𝑡₃<𝑡₄, then relation is defined as𝐿𝑒𝑎=𝑣𝑒𝑠(𝑒𝑋𝑞𝑌_𝑒,𝑣𝑇𝑒)ℎ𝑜𝑙𝑑𝑠𝑁𝑇𝑃𝑃𝑋𝑌,𝑡₁∧ℎ𝑜𝑙𝑑𝑠𝑃𝑂𝑋𝑌,𝑡₃∧ℎ𝑜𝑙𝑑𝑠𝑇𝑃𝑃𝑋𝑌,𝑡₂∧ℎ𝑜𝑙𝑑𝑠𝐷𝑋𝑌,𝑡₅∧ℎ𝑜𝑙𝑑𝑠𝑀𝑋𝑌,𝑡₄.(20)

An intuitive view of this relation is shown in Figure 4(b). This relation can be expressed by a single direction which is the destination point, that is𝐷𝑖𝑟(𝐿𝑒𝑎𝑣𝑒(𝑋𝑌,𝑇))=ℎ𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡₄.(21)Cross(XY,T): A spatiotemporal relation Cross(XY,T) “𝑋 crosses 𝑌 during the interval 𝑇.” Its geometric view is given in Figure 4(c). If ∃𝑡₁,𝑡₂,𝑡₃,…,𝑡₉∈𝑇suchthat𝑡₁<𝑡₂<⋯<𝑡₉, then relation is defined as𝐶𝑟𝑜𝑠=𝑠(𝑠𝑋𝑒𝑌𝑞_,𝑇𝑒𝑣)𝑒ℎ𝑜𝑙𝑑𝑠𝐷𝑋𝑌,𝑡₁∧ℎ𝑜𝑙𝑑𝑠𝑃𝑂𝑋𝑌,𝑡₃∧ℎ𝑜𝑙𝑑𝑠𝑀𝑋𝑌,𝑡₂∧ℎ𝑜𝑙𝑑𝑠𝑁𝑇𝑃𝑃𝑋𝑌,𝑡₅∧ℎ𝑜𝑙𝑑𝑠𝑇𝑃𝑃𝑋𝑌,𝑡₄∧ℎ𝑜𝑙𝑑𝑠𝑃𝑂𝑋𝑌,𝑡₇∧ℎ𝑜𝑙𝑑𝑠𝑇𝑃𝑃𝑋𝑌,𝑡₆∧ℎ𝑜𝑙𝑑𝑠𝐷𝑋𝑌,𝑡₉∧ℎ𝑜𝑙𝑑𝑠𝑀𝑋𝑌,𝑡₈.(22)

This spatiotemporal relation is expressed by a initial as well as destination direction such as object 𝑋 crosses (motion event) object𝑌 from north (direction) towards east (direction) during the interval 𝑇:𝐷𝑖𝑟(=𝐶𝑟𝑠𝑜𝑒𝑠𝑞𝑠_(𝑒𝑋𝑣𝑒𝑌,𝑇ℎ𝑜))𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡₁∧ℎ𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡₉.(23)Into(𝑋𝑌,𝑇): A spatiotemporal relation𝐼𝑛𝑡𝑜(XY,T) read as “𝑋 get into 𝑌 during the interval 𝑇.” If ∃𝑡₁,𝑡₂,𝑡₃∈𝑇suchthat𝑡₁<𝑡₂<𝑡₃, then relation is defined as𝐼𝑛𝑡𝑜=(𝑋𝑠𝑌𝑒,𝑞𝑇_)𝑒𝑣𝑒ℎ𝑜𝑙𝑑𝑠𝑀𝑋𝑌,𝑡₁∧ℎ𝑜𝑙𝑑𝑠𝑇𝑃𝑃𝑋𝑌,𝑡₃∧ℎ𝑜𝑙𝑑𝑠𝑃𝑂𝑋𝑌,𝑡₂.(24)

Its three-dimensional geometric view is given in Figure 4(e). This relation can be expressed by a single direction in language semantics, where a meet topological relation holds. For example, object 𝐴 get into (spatiotemporal event) object 𝐵 from north (direction). It means.𝐷𝑖𝑟(𝐼𝑛𝑡𝑜(𝑋𝑌,𝑇))=ℎ𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡₁.(25)𝑂𝑢𝑡_𝑜𝑓(𝑋𝑌,𝑇): A spatiotemporal relation 𝑂𝑢𝑡𝑜𝑓(XY,T) read as “𝑋 comes out of 𝑌 during the interval 𝑇,” its intuitive view is considered in Figure 4(f). If ∃𝑡₁,𝑡₂,𝑡₃,𝑡₄∈𝑇suchthat𝑡₁<𝑡₂<𝑡₃<𝑡₄, then relation is defined as𝑂𝑢𝑡_=𝑜𝑓𝑠𝑒(𝑋𝑞_𝑌𝑒,𝑣𝑇𝑒)ℎ𝑜𝑙𝑑𝑠𝑇𝑃𝑃𝑋𝑌,𝑡₁∧ℎ𝑜𝑙𝑑𝑠𝐷𝑋𝑌,𝑡₃∧ℎ𝑜𝑙𝑑𝑠𝑃𝑂𝑋𝑌,𝑡₂.(26)

This relation can be expressed by a single direction. Object 𝑋 go out_of (motion event) object 𝑌 towards east (direction), where a meet topological relation holds. It means 𝐷𝑖𝑟(𝑜𝑢𝑡_𝑜𝑓(𝑋𝑌,𝑇))=ℎ𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡₃.(27)Melt(𝑋𝑌,𝑇): A spatiotemporal relation 𝑀𝑒𝑙𝑡(XY,T) read as “𝑋,𝑌 melts during the interval 𝑇”. If ∃𝑡₁,𝑡₂,𝑡₃,𝑡₄∈𝑇suchthat𝑡₁<𝑡₂<𝑡₃<𝑡₄, then relation is defined as𝑀𝑒𝑙𝑡=(𝑋𝑠𝑒𝑌𝑞,_𝑇𝑒)𝑣𝑒ℎ𝑜𝑙𝑑𝑠𝐷𝑋𝑌,𝑡₁∧ℎ𝑜𝑙𝑑𝑠𝑃𝑂𝑋𝑌,𝑡₃∧ℎ𝑜𝑙𝑑𝑠𝑀𝑋𝑌,𝑡₂∧ℎ𝑜𝑙𝑑𝑠𝐸𝑄𝑋𝑌,𝑡₄.(28)

An intuitive view of this relation is shown in Figure 4(g). This relation can be expressed by a single direction because its destination point is dimensionless. This can be its direction, where initial spatial relation holds:𝐷𝑖𝑟(𝑀𝑒𝑙𝑡(𝑋𝑌,𝑇))=ℎ𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡₁.(29)Spring(𝑋𝑌,𝑇): A spatiotemporal relation Spring (𝑋𝑌,𝑇) also called Separate(𝑋𝑌,𝑇) read as “𝑋 separates 𝑌 during the interval 𝑇.” If ∃𝑡₁,𝑡₂,𝑡₃,𝑡₄∈𝑇suchthat𝑡₁<𝑡₂<𝑡₃<𝑡₄, then relation is defined as𝑆𝑝𝑟𝑖𝑛=𝑔𝑠(𝑒𝑋𝑞𝑌_,𝑒𝑇𝑣𝑒)ℎ𝑜𝑙𝑑𝑠𝐸𝑄𝑋𝑌,𝑡₁∧ℎ𝑜𝑙𝑑𝑠𝑀𝑋𝑌,𝑡₃∧ℎ𝑜𝑙𝑑𝑠𝑃𝑂𝑋𝑌,𝑡₂∧ℎ𝑜𝑙𝑑𝑠𝐷𝑋𝑌,𝑡₄.(30)

Its three-dimensional geometric view is given in Figure 4(h). This relation can be expressed by a single direction because its destination point is dimensionless. This can be its direction, where terminal spatial relation holds:𝐷𝑖𝑟(𝑆𝑝𝑟𝑖𝑛𝑔(𝑋𝑌,𝑇))=ℎ𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡₄.(31)Graze(𝑋𝑌,𝑇): A spatiotemporal relation Graze(𝑋𝑌,𝑇) read as “𝑋 grazes 𝑌 during the interval 𝑇.” If ∃𝑡₁,𝑡₂,𝑡₃,𝑡₄,𝑡₅∈𝑇suchthat𝑡₁<𝑡₂<𝑡₃<𝑡₄<𝑡₅, then relation is defined as𝐺𝑟𝑎𝑧=𝑒(𝑠𝑋𝑒𝑞𝑌_,𝑒𝑇𝑣)𝑒ℎ𝑜𝑙𝑑𝑠𝐷𝑋𝑌,𝑡₁∧ℎ𝑜𝑙𝑑𝑠𝑃𝑂𝑋𝑌,𝑡₃∧ℎ𝑜𝑙𝑑𝑠𝑀𝑋𝑌,𝑡₂∧ℎ𝑜𝑙𝑑𝑠𝐷𝑋𝑌,𝑡₅∧ℎ𝑜𝑙𝑑𝑠𝑀𝑋𝑌,𝑡₄.(32)

This relation is represented in a three-dimensional perspective in Figure 4(d). This spatiotemporal relation is expressed by an initial as well as destination direction such as object 𝑋 grazes (motion event) object 𝑌 from north (direction) toward east (direction):𝐷𝑖𝑟(=𝐺𝑟𝑠𝑎𝑒𝑧𝑞𝑒_𝑒(𝑋𝑣𝑒𝑌,ℎ𝑇𝑜)𝑙)𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡₁∧ℎ𝑜𝑙𝑑𝑠𝐷𝑖𝑟𝑋𝑌,𝑡₄.(33)

8. Conclusion and Future Work

In this paper, we define spatiotemporal relations, where the discrete time space is used. These spatiotemporal relations are topologically stable or unstable. Motion events represent the subclass of spatiotemporal relations, and certain number of motion events represent the class of a topologically unstable and stable spatiotemporal relations. In these spatiotemporal relations temporal order of holding a primitive event is more important, and this order has a pivotal role in natural language semantics. Topological relations have a locative symmetries, to remove these symmetries we add a directional components. In this paper, CTD method [11] is used to model the motion events, where topological and directional information for a snapshot are captured at the same abstract level. Hopefully this work will bring a significant change in video understanding, modeling video events, and other related areas of research.