Abstract

A new framework is presented for multiparticipant coordination of over-the-horizon maneuvering processes. In this framework, geographical information is decentralizedly augmented via the multitude of annotation processes: landmark localization by map builder, connection generation by planner, and GPS tracking by probe vehicles. By integrating the augmentation process on a common satellite image, the subscriber participants reuse the geographics within specific maneuvering context. Based on graph theoretic representation of the multiparticipant augment process, an interactive geographics annotation system was developed and verified within the context of interactive rendezvous and cooperative monitoring.

1. Introductory Remarks

Recent advancements of space technology combined with large scale information networks provide physical-geometric basis for over-the-horizon maneuvering process; for instance, current global positioning system (GPS) yields effective information for dynamic localization of vehicles along roadway areas [1]; the behaviors of such vehicles are matched with geometric representation of local terrains for planning [2], regulating [3], and operating [4] vehicle control processes equipped with self-reliant intelligence. In many practical situations, the perspective from each vehicle is confined within the roadway area. This implies that the decision making by each advanced vehicles must be supported by the information to be gathered finally via on-vehicle sensing systems, including machine visions. To expand the virtual scope of each vehicle, the on-vehicle control systems should be dynamically networked as air traffic control systems [5]. However, in contrast with the aviation systems, it is not easy to figure out the multitude of vehicle's trajectories satisfying specific constraints arising in complex local terrain.

By networking GPS satellites and vehicle control systems with the earth observation systems, we can articulate the multitude of maneuvering processes within a bird's eye view of local terrains. Figure 1 displays a current implementation of the geographics annotation system where a vehicle is dispatched to probe the topological and geometric structure of a district [7]. A time series of the longitude-latitude estimate is generated based on pseudoranges from GPS satellites including QEZZ; the estimate is transferred to the annotation system via data relay site as GPS track along a roadway; adding to it, the probe vehicle captures the scene images via the on-vehicle camera to upload in association with the GPS track data. In the geographics annotation system, the gathered information by the probe vehicle is matched with a cut of satellite image provided by earth observation system, ALOS, for example, The scene image at the origin is matched with a small section of satellite image to identify the roadway segment around the GPS track date; the roadway segment is extended in the bird's eye view to adapt the GPS track data to a feasible path connecting the origin and the destination. Thus, we have an as-is articulation of roadway pattern incident to origin and destination landmarks; the images of the landmarks are captured by the on-vehicle camera and localized on the bird's eye view in association with the segments of the GPS tracks. The multitude of the vehicles, thus, can be exploited as probes for feasible segmentation of roadway patters connecting landmarks. Such a look-and-feel representation provides the virtual scope of the as-is geography as the basis for over-the-horizon coordination of the maneuvering processes; the multitude of the vehicles are exploited as probes for other participant vehicles; the origin-destination patterns by the probe vehicles are archived and reused as verified roadway patters to be followed.

We can augment the information arising in the local terrain in terms of the articulated maneuvering processes; the origin-destination landmarks and associated GPS tracks are linked to supplement the topological and geometric descriptions of the bird's eye view, respectively. Such augmented annotations can be utilized by participant vehicles as an a priori information to be gathered along the path connecting origin-destination landmarks. This implies that the annotated geographics can be exploited as the basis of spontaneous coordination of the multitude of the vehicles participating the maneuvering processes. Furthermore, simultaneous augmentation of geographics annotation with reference to common landmarks provides computational foundation for the over-the-horizon cooperation of the multitude of the maneuvering processes. For instance, at-a-glance presentation of landmarks within a common satellite imagery makes it possible for a participant to design a specific path as a chain of expected landmarks towards a preassigned destination.

Symbolically, each maneuvering process can be identified in terms of a graph spanning a specific selection of the landmark symbols. Through the design steps, hence, the annotation is augmented via the landmark association within a participant specific context. Noticing the uniqueness of the landmark allocation in the bird's eye views, the common understanding of the landmark selection on the ongoing geographics annotation yields a preestablished mental space [8] for facilitating linguistic communication among the participant designers; the real-time feedback of ongoing geographics annotation processes, simultaneously, activates not-yet-explicated “programming mechanisms” for unifying spatial and logical mathematical aspects of human's inherent capability [9]. In reference to such dynamic visualization of on-going geographics annotation, thus, participants can match design steps to cooperatively generate maneuvering processes towards a rendezvous point by mutual consent within the vista of the satellite image. Adding to it, the augmented annotation through the maneuvering processes can be exploited as a prediction of the scene for participant vehicles.

Despite the simultaneous visualization of mechanical annotations by probe vehicles with symbolic design by human designers, it is not easy to associate the chain of the landmark symbols with a part of a vehicles trajectory nor to identify the GPS track as a variation of symbolic design within the complex constraints arising in the local terrain. To expand the virtual scope for interactive coordination of the over-the-horizon maneuvering processes on the geographics annotation, in this paper, we consider dynamic integration of mechanical and human aspects of geographics annotation processes. The problem is to identify the symbolic representations of the maneuvering process as versions of graph minors of a physical entity: a GPS track adapted to the geometric representation of the local terrain.

2. Multiparticipant Design of Maneuvering Processes

To integrate the annotation augmentation processes by probe vehicles and human designers, the chain of landmark symbols and verified GPS tracks are required to be identified as two aspects of a physical entity. Following the multiaspect concept for complex systems design [10, 11], the geographical information should be manipulated within the framework of physical symbol system [12]; the landmark-path association should be reactively manipulated under physical-geometric constraints governing the real world. By using the geographics annotation system shown in Figure 1, we can simulate a GPS track connecting origin-destination landmarks as illustrated in Figure 2; the scene images captured by probing vehicles are retrieved for sampling chromatic diversity as features of roadway area; through precise matching of the chromatic diversity with the local image around the GPS data, the roadway pattern is identified and extended towards a possible destination. In many practical scenes, we can generate such “future trajectories” beyond the physical-geometric perspective [6]; as shown in the “GPS track window” of Figure 2, the future trajectories well-simulate the GPS tracks to be observed in subsequent maneuvering process. This implies that we have an augmented version of geographics by using the simulated GPS track as a feasible connection spanning selected landmarks.

Through the satellite-roadway-vehicle network, furthermore, the participant vehicle can download local representation of the annotated geographics along the future trajectories within common understanding of the landmark allocation. The consistency and smoothness of simulated GPS tracks implies that the participant vehicles observe essential structure of the landmark scene along the future trajectory; the anticipatively observed scenes images can be exploited for “presetting” vehicle control systems prior to physical access; the participant vehicles can control the focus of on-vehicle vision along a segment of the future trajectory as shown in Figure 3.

To extend the scope of such an over-the-horizon probing and planning scheme to the multitude of maneuvering processes, the annotated geographics should be reusable apart from specific context of planner participants; scriber-participants are charged with the reorganization of the design steps according to their own maneuvering context. In spite of unique localization of the landmarks and GPS tracks as well, the scope of the landmark selection should be supervenient on the participant specific context within associated local geographics. For supporting such a multicontext computation, the spontaneous synchronization scheme is required to have a mutual access path of multiscope representation installed among the multitude of the participants; the common set of landmarks with geometric association in terms of GPS tracks should be reorganized and indicated within participant specific scope of the geographics annotation. To establish the multitude of human access path to the spontaneous synchronization scheme, in what follows, we introduce a new computational framework for visualizing and programming participant specific design steps based on dynamic geographics annotation processes.

3. Multiparticipant Graph Generation

Physically, a maneuvering process is represented as a trajectory within a bird's eye view of the local terrain. Linguistically, each maneuvering process can be identified in terms of a graph spanning a specific selection of the landmark symbols. Computationally, thus, the multiparticipant design of the maneuvering processes is represented by a cooperative geographics annotation process as illustrated in Figure 4 where the multitude of participants are collectively in charge of asynchronous augmentation of the annotation for possible decision makings; the landmarks may be located by a map builder in a cut of the satellite image on which a path planner may generate symbolic connections within a specific maneuvering context; based on the information gathering through a session of local probing, on the other hand, a feasible version of landmark connection may be complied to augment the ongoing geographics annotation.

As the basis of the multiparticipant annotation, let be an image plane coordinated in terms of (longitude, latitude) and consider the information representing the dynamic evolution of the geographics within a bird's eye view . In what follows, we identify the image plane with the contents: a photocopy of local terrain filling up the image plane. Suppose that the participant specific representations of the world are visualized within the multitude of local sections , , supporting the participant's scope , . We introduce the relation by . The totality of the local sections is designated by With look-and-feel order induced by , such information provides the multitude of the participant specific restriction within the entire geographics . By using the class , we can induce an ordered family of local terrains , where is the support of the local section .

Within the specific local section , the publisher participant localizes a landmark at ; is called the grounding of landmark ; the grounding of is designated by . The totality of such landmarks, specifies the multiparticipant extension of the scope into a common geographics of locally generated landmarks; for each subdescription, we have the following image-based reclustering of : Publisher participants introduce symbolic associations among the landmarks anywhere within their own scope of the geographics . This implies that a set of origin-destination pairs are induced in the common bird's eye view . The totality of the origin-destination pairs is called connections. By sharing the world image , subscriber participants expand the horizon of their own maneuvering plan to a chain of possible connections on ; by invoking the maneuvering records of probing vehicles, the possible connections can be gathered and compiled in the entire geographics .

Suppose that the scope of decision making is confined within the landmark set on local geographics , that is, and consider the totality of induced connections given by Equation (6) implies that the scope of the connectioning is extended to the adjacent landmarks with respect to the geographics . Such a minimal expansion makes it possible for the participants to associate the connections with global information and local restriction as well.

On a scope , we define the route by a graph structure embedded in the geographics ; for to be associated with , is generated within the expansion of the common bird's eye view . The graph structure provides a basis for the computation of the path connecting the origin and destination landmarks in terms of the chain of the connections within the graph structure Since there is a unique sequence , satisfying the path can be identified with an ordered set of landmarks, that is,

Following graph theory, we can index the sufficiency of the geographics annotation prior to the design of path . If the graph include at least one spanning tree, the connections should satisfy the following evaluation: where denotes the size of set . Noticing Euler-Poincaré's formula, on the other hand, the geometric distribution of feasible connections in a sufficiently complex geographics should be bounded by the maximum connection in a planar graph. This implies the following 2D upper boundary: Hence, we have the following index for evaluating the a priori sufficiency of the connections spanning the geographics: By invoking the evaluation (10), (11), it follows that where denotes the average degree of the graph structure . Thus, we have the following evaluation asymptotically. This implies that we can a priori evaluate the sufficiency of the annotated geographics with respect to the design of a connected path. Being given the index , in turn, we can estimate a landmark-wise complexity for selecting a next connection by .

4. Geographics Sensitive Graph Manipulation

Within the annotated geographics , a maneuvering process is symbolically manipulated as illustrated in Figure 5. To implement physical-geometric maneuvering process within the geometric constraints arising in the geographics , the origin-destination connection should be paraphrased into a chain of feasible connections on the bird's eye view . This implies that the physically feasible route is generated as subdivisions of the graph structure ; in turn, the linguistically simplified connection is computed as a topological minor [13] of the physical representation.

In many practical situations, the origin-destination connection is consisting of sufficiently rich gateway networks and a few set of turnpikes linking the gateways. In such a small world [14], the subdivision process is implemented by the following multiscope decomposition within the scope of geographics information : In (15a), the route connecting the landmark symbols is subdivided into a chain of stopover landmarks. This implies that, in early stage of planning, the route should be represented in terms of the topological minor . Some landmarks in the topological minor can be recompiled into a graph structure within local geographics . Thus, we have a multiscope subdivision of the original description as indicated in (15b); in the section , the subdivision (15a) with respect to is a topological minor of the graph structure . In the local geographics , on the other hand, a feasible connection is added to yield a connected graph called reachable expansion; generally, the reachable expansion is not included within the local section. In many naturally complex scenes, we can generate additional connections between the geophysical subdivision and the reachable graph.

In the spontaneous synchronization scheme, publisher subscriber participants are required to computationally transfer individual representation of maneuvering processes to each other. Noticing consistent reprogrammability (15a), (15b), we can exploit the dynamic graph structure on increasing family of landmark set , and associated connections as the computational basis for the spontaneous synchronization scheme. Through such graph reprogramming, we have a uniquely scoped graph structure for supporting a maneuvering process. Since and , a path should be selected as a multistage decision steps satisfying the following criterion: with respect to a properly specified “Markovian cost” determined by information . Along such a conventional optimal path, the dynamics of the minimal cost is known to satisfy the following principle of optimality: To adapt to the multiparticipant maneuvering processes, we can paraphrase the principle of the optimality on the increasing family of the landmark sets as follows: where denotes the family of annotated geographics supporting the design steps with respect to , . The halting condition is given by . The existence of the criterion (18) implies that we can design an optimal plan in accordance with a monotone family of ongoing geographics annotation confined by a fixed representation .

5. Probing-Based Connection Generation

We can utilize GPS track data as an ordered series of groundings along a really existing path. By articulating the time series within the restriction , we can augment the annotation in terms of associated connections.

On the image plane supporting the geographical information , the GPS track is identified with a sequence of (longitude, latitude) coordinate , that is, Let a roadway pattern be detected within a small circle confined in terms of GPS residual as indicated in Figure 6; the local geometry of the roadway pattern is matched with the segment with origin and direction . Suppose that smoothness of the GPS track is evaluated in terms of the backward consistency index given by Then, we can select a sequence of segments, called GPS path, as follows: where denotes a preassigned regularity level. In practical situations, the regularity level is adjusted to the resolution of the segmentation, that is, , for sufficiently small GPS sampling time. By definition, stands for an estimate of feasible trajectory based on the observation .

Consider the following ordered set called GPS trajectory Within the fluctuation of GPS track, we can exploit the information as a geometrically consistent path connecting the landmarks within the satellite image. For preassigned landmark resolution , a landmark with grounding is said to be a -contact if the following condition is satisfied:

The GPS trajectory adapted to the local terrain is associated with the annotated geographics as illustrated in Figure 7; the geometrically consistent entity is partitioned into a sequence of segments , , along a design of the path ; simultaneously, is decomposed into the chain of trajectory points and to be associated with the topological minor directory connecting final origin-destination pair as well.

For a set of landmarks , we have the following partitioning of the set with respect to : where Define the following order in the class : Then we have the following ordered partitioning: with the forward restriction given by

Consider a landmark satisfying By selecting a chain of such a -landmarks with respect to the GPS-trajectory , we can induce a connection of the following form: , . Thus, we have a GPS based connection generation algorithm as follows:

Algorithm 1. Step 0. For given GPS track , detect landmark satisfying set , then continue.Step 1. Select satisfying Step 2. Set Step 3. If then exit;
else update by the landmark satisfying
and set then return to Step 1.

As the result, we have an augmented version of annotated geographics as described in Figure 7. Supported by such a probing-based augmentation, we can reuse the participant specific design steps within the ongoing geographics annotation. For instance, we can utilize the GPS trajectory based on vehicle specific track data for extending the space of the connection ; to this end, the -partitioning process (25) is applied to the trajectory to reorganize the -ordered track (28) into a geographics sensitive representation: a -ordered partitioning (28) satisfying the monotone condition (27). Thus, we have an augmentation of connection with respect to on the landmark annotation , that is, When a new landmark is defined by a map builder, we can invoke the nearest -contact detection (24) to yield the -forward restriction (29). Such scheme is implemented by forward and backward application of the connection generation Algorithm 1 to expand the space with a feasible set of connections. This yields an augmentation process with respect to along the trajectories : Such probing-based augmentation of the annotated geographics can be immediately visualized to a user participant to design a maneuvering plan. To this end, the user selects a chain via the path planning process (7) within the visualization of the annotation consisting of . In this case, it is sufficient for the map builder to specify a topological minor of the route graph in reference to the complexity index ; via the dynamic probing-geographics association, the topological minor is to be subdivided into a feasible set of primitive connections as context-free information to be reused by participant users.

The existence of cooperative geographics augmentation process (35), (36), (37) implies that we can exploit GPS trajectories for expanding the space of connections within the resolution of the local geographics . Since , in many practical situations, we can adjust the GPS-based connectioning system in terms of the geographics resolution .

6. Cooperative Design of Interactive Rendezvous Process

Suppose that a map builder has filled in major landmarks with key connections in a cut of satellite image and consider multiparticipant design of a cooperative maneuvering process. The problem is to reuse the basic description of the geographics within a new context of maneuvering process: cooperative decision making on a rendezvous point in accordance with the maneuvering process in reference to ongoing geographics annotation. Through interactive geographics annotation, the participants can cooperatively design individual maneuvering process towards a landmark to be determined. The schematics of such an interactive rendezvous process is illustrated in Figure 8 where a set of landmarks with associated connections is generated on a satellite image provided by the earth observation system shown in Figure 1. The maneuvering plan is cooperatively designed to select a route graph and indicated on the satellite image; the route graph is continuously verified by the expansion of the GPS track uploaded through the maneuvering process. Simultaneously, the GPS track is matched with the annotated geographics to compute the next landmark as shown in Figure 2. Based on such annotated geographics, participants commonly understand the goal of the current design step.

Figures 9 and 10 illustrate an example of annotated geographics where a set of landmarks and associated connections are indicated in a cut of satellite image of 640 480 resolution; in this digital image, the positioning error is supposed to be confined within 20 m × 20 m area in which the GPS residuals were verified to be corrected via the adaptive segmentation scheme as shown in Figure 6.

Figure 9 displays an overview of a priori located landmarks with feasible connections on a satellite image spanning a downtown area and a campus; to support the multitude of the participant with individual intention, the paths are subdivided by the annotation system with respect to indicated landmarks via subdivision algorithm (15a), (15b). In this case, the resulted the a priori sufficiency index was evaluated by ; by definition, there are defined twice connections of the minimal requirement, that is, . Simultaneously, the complexity of graph structure at a landmark is estimated by ; in average, the network topology at each landmark is identified with a crossroad. In this annotated geographics, a set of local scene images are attached to each landmark as shown in Figure 10; a sequence of landmarks along the designed path can be visualized prior to physical access.

Figures 1116 illustrate a typical performance of the interactive rendezvous process; the goal of the geographics annotation is to cooperatively navigate a guest participant from the junction station in the downtown to a rendezvous point to be determined in the campus area. To design a maneuvering process, the guest-host participants invoke the annotated geographics (Figure 9) with initial representation of the geographics by the host participant as illustrated in Figure 11 where initial selection of the landmarks are associated via a path connecting the station and the campus.

In response to the initial design, the guest participant manipulates the geographics in the following three steps. First, the guest invokes a local section focusing the bus pool at the station to get a service to the campus area as illustrated in Figure 12; in this figure, the connections to the bus stop to be selected are marked with the GPS track uploaded by the guest. Next, the guest retrieves another local section as displayed in Figure 13 where a local network of transportation services are displayed. On board the selected service, finally, the guest is notified of the status of the maneuvering process as exhibited in Figure 14; the GPS track uploaded by the moving guest is displayed to the guest and host participants to verify the implementation of the designed process.

Monitoring the transition of the GPS track shown in Figure 15, the host participant makes reference to the local view including the campus area as shown in Figure 16. In this figure, the planned route of the selected service is indicated with online verification by the GPS track. As the result of the interactive geographical annotation process mentioned above, the host confirms the rendezvous point under simultaneous understanding of the guest participant.

Throughout the multiscope annotation augmentation, the support geographics was fixed to visualized in Figure 9. Following multiscope transition, the restrictions (5) and (6) were updated to design the maneuvering process satisfying the multiscope optimization condition (18). In accordance with the extension of the GPS track, the -partitioning was updated to identify the nearest subsequent landmark satisfying (30). As the result, a landmark scene is selected in the visualizations indicated in Figure 10 as the prediction of the current destination. Thus, the system was demonstrated to support cooperative design of an over-the-horizon maneuvering process through interactive generation of geographics annotation by the guest and host participants.

Due to geometric complexity, GPS tracks are often deviated from symbolic connections of origin-destination landmarks; the maximum deviation in Figure 15 amounts to 320 m which is two or three times of landmark granularity in downtown and campus area. Despite such geometric-symbolic discrepancy, the interactive rendezvous system can identify the maneuvering process within the multiscope graph structure to indicate the one-step prediction of landmark images throughout the design and verification steps.

7. Saliency Transfer for Cooperative Monitoring

Let a segment of the future trajectory be downloaded as a priori information of a scene. Following empirical knowledge of ecological optics [15] and inherent preference [16], combined with recent advancements in machine perception [17] and emotional perception [18], the generic structure of the scene can be described in terms of a set of fractal codes specifying a roadway area and an aggregation of boundary objects. Noting this, the randomness of the scale information is extracted from the scene image shown in Figure 3 to identify the ground-object structure as shown in Figure 17 [7]; guided by the downloaded segment, a fractal code is designed to recognize a connected open space confined by the distribution of boundary objects. Despite infinite diversity of appearance, natural scenes exhibit environment specific sign patterns to be identified by inherent vision system within individual intention of viewers. By simulating not-yet-explicated mechanism of inherent vision, we can a priori control the focus of on-vehicle vision system to the sign patterns to be captured in subsequent scenes.

Noticing that inherent vision has developed efficient mechanism for identifying the chromatic diversity in terms of primary, we can simulate the capturing process of sign patterns as shown in Figure 18. In this figure, the fluctuation of colors in the scene image (Figure 3) is displayed in the upper subwindow; the distribution of the colors is identified with a fractal attractor to specify a scene specific deviation of the primary as indicated in lower subwindow. By matching the deviated primary with the scene image, we have a random distribution of sign patterns as displayed in the main window of Figure 18. This result implies that we can exploit such deviated primary to control the focus of the on-vehicle vision. In fact, we can design a saliency-based mechanism for selectively scanning the sign patterns as demonstrated in Figure 19. By matching the distribution of detected sign patterns with the ground-object structure shown in Figure 17, we have a contextual visualization of the scene as demonstrated in Figure 20 where a signal to be focused is separated from distractive sign boards and other signals confused in noisy background. This implies that we can utilize landmark images along the future trajectory for presetting the on-vehicle vision.

Figures 2122 illustrate another example of experimental results; saliency patterns are selectively scanned (Figure 21) to visualize hazardous vehicle within the context of the ground-object structure (Figure 22). In this case, the on-vehicle vision detects the distribution of the deviated primary to localize the image of a vehicle wrapped by a warning mark.

Noticing the simulated capability for scanning and visualization of sign patterns, the annotated geographics yields a computational basis for the implementation of on-vehicle vision in cooperation with human's inherent perception in naturally complex scenes. Through experimental studies, it has been demonstrated that the distributions of local scale fluctuation and chromatic diversity jointly yield a robust representation of image features arising in naturally designed scenes. This implies that we can exploit the deviated primary annotated by probing vehicles to “preset” on-vehicle vision prior to physical access. As demonstrated in Figures 1722, the saliency-based image analysis exhibits sufficient robustness for the restoration of geometric ambiguity arising in GPS track and/or landmark allocation. This implies that the network integration of on-vehicle vision systems is crucial for precautious hazard avoidance via the expansion of virtual scope as well.

8. Concluding Remarks

A multiparticipant graph generation scheme was implemented for articulating GPS tracks with respect to landmark allocation. In this scheme, initial representation of the geographics including landmark localization with turnpike connections is combined with GPS tracks to successively augment participant specific annotation; through the visualization of on-going augmentation steps on a common satellite image, simultaneously, the multitude of the participants recognize the totality of maneuvering process along individual path to be designed. By formulating the augmentation process in terms of a monotone expansion of graph structure, the on-going geographics annotation yields sufficient information for successive decision steps. Through experimental studies, it has been demonstrated that the geographics annotation system is effective as the support of over-the-horizon maneuvering process; the multiscope transition of the interactive rendezvous process is cooperatively designed and verified by on-going GPS track with anticipative scene images; the virtual scope of on-vehicle vision is extended within the geographics annotation via cooperative monitoring of scene specific saliency.