Research Article

Large-Scale Coarse-to-Fine Object Retrieval Ontology and Deep Local Multitask Learning

Algorithm 2

Online-phase (retrieval-phase) CFOR system algorithm applied in the fashion field.
Procedure: ONLINE_CFOR
Goals:
  Detect objects when having an input image
  Extract information of input objects such as regions, categories, and attributes
  Return retrieval results
Input:
Onto //establish ontology of fashion data
Database //constructed data of all extracted information and features
classficationModels (state, onto) //trained classification model for all possible concept states.
multitaskModel //trained attribute multitask classification model for all possible states.
states:list of all states which appear in ontology.
imgQuery //input query image
Output:
retrived_info //semantic information of all retrieved results.
imgList //retrieval image results
BEGIN
obj_list ⟵ detector(imgQuery)   //extract objects which appear in input query image
for obj in obj_list:
begin
   //extract information of input objects. Objects move through ontology to extract the region and then the category and finally attributes.
   infor ⟵ infor_extract(states, obj, onto, classifyModels, multitaskModel)
    //extract features of input object
   feat ⟵ feat_extract(states, obj, onto, classifyModels, multitaskModel)
end
if expand is True:
begin
   //expand query and optional query based on extracted attribute information
   infor, feat ⟵ query_expansion(infor, feat)
   //compute similarity score between input and database. In this function, the score computed based on the similarity of input features and database image features will be changed when passed through ontology. The score is then added with the matching attribute score computed by matching the input attribute with those in the database. To index the database, nonexhaustive search will be applied using GPU to speed up indexing time.
   score_list ⟵ compute_sim_score(database, infor, feat)
   indexes, socre_list ⟵ ranking(score_list, database, top_k, GPU_search = True)   //top-k ranking by using GPU nonexhaustive search
   retrieved_info, imgList ⟵ retrieval(indexes, score_list, database, GPU_search = True)   //retrieval task
end
END