An Efficient Approach for Web Indexing of Big Data through Hyperlinks in Web Crawling

Algorithm 1

Input: A graph of Web Pages and a vertex page qt of

Output: All pages reached from vertex page qt labelled as discovered

crawling_algo(W,  qt)

Input:

Step  1.    Assign all and retrieve from disk, query term qt at time

Step  2.   While disk not empty do

Step  3.      Assign

Step  4.      For all in CS

Step  5.           If → leftnode exists

Step  6.                → leftnode

Step  7.               Repeat for all nodes recursively

Step  8.           else if → rightnode exists

Step  9.                → rightnode

Step  10.             Repeat for all nodes recursively

Step  11.          else

Step  12.             Set boolean of as true

Step  13.             Add to corresponding RS

Step  14.         Return RS

: query node, qt: First query term, boolean: flag variable,

CS: Category Set, RS: Result Set, : Count.