Data · dataset · 2014
Design and Construction of Distributed JS Parsing System
Listed in DataCite
This paper has two major research directions: First, to design an algorithm which can extract JS scripts in HTMLs and parse them; Second, to analyse task scheduling algorithm and design a task scheduling algorithm of this system by using Hadoop distributed computing technology.
Description
Through the research of the rules of JavaScript grammer and its existence in HTMLs, this paper designed the process and algorithm of JavaScript extraction based on JavaScript parsing engine.
This is the first module. Through doing research on the Map/Reduce task scheduling algorithm, according to the characteristics of JavaScript parsing task and distributed environment, this paper also figured out the most suitable Map/Reduce task scheduling algorithm for the system to support reasonable operation of JavaScript parsing task. And then a distributed JS parsing system was constructed.
Read the rest (2 more)
In order to check the accuracy and the performance of the system, this paper had a test on the system, summarizes deficiencies and suggests improvements at last. The distributed JS parsing system can parse a large number of JS scripts in HTMLs efficiently and quickly. The experimental results show that this system can extract the text messages and urls contained in JS scripts efficiently.
Thereby the research of this paper can provide a reliable technical support for the field of search engine, public opinion analysis and data acquisition.
Links
Where it is published
- Repository landing page figshare.com/articles/dataset/Design_and_Construction_of_Distributed_JS_Par… ↗
landing page · from DataCite
- DOI doi.org/10.6084/m9.figshare.988756 ↗
DOI / persistent id · from DataCite
Documentation and papers
Catalogue records · 2
- DataCite API api.datacite.org/dois/10.6084/m9.figshare.988756 ↗
metadata API · from DataCite
- DataCite Commons commons.datacite.org/doi.org/10.6084/m9.figshare.988756 ↗
catalogue entry · from DataCite
Topics
- Stated by source
- Computer and information sciences
- Inferred from text
- Text 75%
Provenance · 1 source records, 10 field assertions
| Source | Key | Last seen | Raw |
|---|---|---|---|
| DataCite | 10.6084/m9.figshare.988756 | 14 d ago | JSON v1 |
| Field | Assertion | Extractor | Evidence |
|---|---|---|---|
| access_level | source · DataCite | connector:datacite@1.0.0 | /data/attributes/rightsList |
| byte_size | source · DataCite | connector:datacite@1.0.0 | |
| concepts[field].fos:computer-and-information-sciences | source · DataCite | connector:datacite@1.0.0 | |
| concepts[modality].local:modality:text | enrichment · DataCite | keyword-concept-rules@1.0.0 | title+description (75%) |
| created_date | source · DataCite | connector:datacite@1.0.0 | |
| description | source · DataCite | connector:datacite@1.0.0 | /data/attributes/descriptions |
| license | source · DataCite | connector:datacite@1.0.0 | /data/attributes/rightsList |
| publication_date | source · DataCite | connector:datacite@1.0.0 | /data/attributes/dates |
| title | source · DataCite | connector:datacite@1.0.0 | /data/attributes/titles/0/title |
| updated_date | source · DataCite | connector:datacite@1.0.0 |