摘要:
Disclosed is a computer-implemented method of determining smarty between first and second elements of an electronic document. The method uses a computer to calculate a plurality of measures of similarity between the first and second elements in at least two representations of the electronic document. A computer program product and system implementing this method are also disclosed.
摘要:
A system and method for an adaptive threshold Web Page segmenting is disclosed. In one embodiment, a method performed by a physical computing system having one or more processors for segmenting a Web page including a plurality of nodes includes parsing content in the Web page into the plurality of nodes using the physical computing system, obtaining feature values between each pair of nodes using the physical computing system, estimating an adaptive threshold value using the obtained feature values using the physical computing system, and segmenting the Web page by comparing the feature values associated with each pair of nodes with the estimated adaptive threshold value.
摘要:
A system and method of detecting separator lines in a web page may include determining coordinates of visible web elements on a web page, generating an edge image of the web page based on the coordinates of the web elements, filtering edges belonging to non-separator line elements within the edge image, detecting horizontal lines within the edge image, detecting vertical lines within the edge image, and filtering short lines within the edge image. A system for detecting separator lines in a web page may include a memory device, and a processor communicatively coupled to the memory, in which the processor determines coordinates of visible web elements on a web page, generates an edge image of the web page based on the coordinates of the web elements, filters edges belonging to non-separator line elements within the edge image, detects horizontal lines within the edge image, detects vertical lines within the edge image, and filters short lines within the edge image.
摘要:
Presented is a method of extracting named entities from a large-scale document corpus. The method includes identifying named entities in the corpus and forming a set of seed entities manually or automatically using some existing resources, constructing a named entity graph to discover same-type probability between any given pair of named entities, expanding the set of seed entities and performing a confidence propagation of the seed entities on the named entity graph.
摘要:
A computer-implemented method for obtaining the rendering co-ordinates of visible text elements on a web page is disclosed. The web page is represented by an input data structure comprising a plurality of text nodes, each of which represents a text element on the web page. The method comprises the following steps: a) using a computer device, wrapping each of the plurality of text nodes in a pair of mark-up language tags; b) using said computer device, obtaining the co-ordinates of a bounding rectangle for each text node using the mark-up language tags; c) using said computer device, attaching an attribute specifying the co-ordinates of the bounding rectangle to each text node; and d) using said computer device, determining whether each text node is invisible, and if it is, excluding it from an output data structure comprising the plurality of text nodes and attached attributes.
摘要:
Disclosed is a method of generating a model representation of product information. The method obtains a list of products from a source of product information. A hierarchical tree is then constructed from the obtained list of products, wherein each hierarchical layer of the tree corresponds to a different category of product information.
摘要:
A method for detection of visual separators in web pages using code analysis includes receiving a web page and its associated web code by a web page analysis device and analyzing the web code to detect visual separators in the web page. A web page analysis device for visual separator detection in web pages is also provided.
摘要:
A method for producing web page content includes identifying blocks within a web page. The blocks are selectively assembled into sections. The sections are selectively assembled into article candidates. An article candidate that includes article content is distinguished from article candidates that do not include article content. Content is produced only from the article candidate distinguished as including article content.
摘要:
A system and method are provided for extracting main content from a web page. Web page segmentation is performed on a web page to provide affinity-grouped segments. Descriptive features of at least one of the affinity-grouped segments are computed. At least one of the affinity-grouped segments is classified as a main body segment based on the computed descriptive features. Additional affinity-grouped segments are classified as to a document function based on the computed descriptive features. Classified affinity-grouped segments are assembled according to their classified document functions to provide the main content.
摘要:
Systems and methods for seed set expansion are provided. A context-based extractor (22) generates a set of context-based candidate members of a seed set from a set of web pages associated with an organization as words connected with a seed set member by a contextual pattern and a context confidence value for each candidate member. A list-based extractor (24) generates a set of list-based candidate members from elements within a plurality of lists in the set of web pages and a list confidence value associated with each candidate member. A confidence arbitrator (26) determines an intersection set of candidate members present in both sets of candidate members and determines a final confidence value for each of the intersection set of candidate members based on their respective context confidence value and list confidence value. A candidate selector (28) selects a candidate member for inclusion in a seed set (21).