← Search

Text · dataset · 2021

Evolution of Web search engine interfaces through SERP screenshots and HTML complete pages for 20 years

Listed in INESC TEC Research Data Repository

This dataset was extracted for a study on the evolution of Web search engine interfaces since their appearance.

Description

The well-known list of “10 blue links” has evolved into richer interfaces, often personalized to the search query, the user, and other aspects. We used the most searched queries by year to extract a representative sample of SERP from the Internet Archive.

The Internet Archive has been keeping snapshots and the respective HTML version of webpages over time and tts collection contains more than 50 billion webpages. We used Python and Selenium Webdriver, for browser automation, to visit each capture online, check if the capture is valid, save the HTML version, and generate a full screenshot. The dataset contains all the extracted captures.

Read the rest (4 more)

Each capture is represented by a screenshot, an HTML file, and a files' folder. We concatenate the initial of the search engine (G) with the capture's timestamp for file naming. The filename ends with a sequential integer "-N" if the timestamp is repeated.

For example, "G20070330145203-1" identifies a second capture from Google by March 30, 2007. The first is identified by "G20070330145203". Using this dataset, we analyzed how SERP evolved in terms of content, layout, design (e.g., color scheme, text styling, graphics), navigation, and file size.

We have registered the appearance of SERP features and analyzed the design patterns involved in each SERP component. We found that the number of elements in SERP has been rising over the years, demanding a more extensive interface area and larger files. This systematic analysis portrays evolution trends in search engine user interfaces and, more generally, web design.

We expect this work will trigger other, more specific studies that can take advantage of the dataset we provide here. This graphic represents the diversity of captures by year and search engine (Google and Bing).

Links

Topics

Stated by source
CS: Computer Science
Inferred from text
Text 75%
Provenance · 1 source records, 11 field assertions
SourceKeyLast seenRaw
INESC TEC Research Data Repository4f8782a7-e9b0-46ae-beca-b8641f24c9e813 d agoJSON v1
FieldAssertionExtractorEvidence
access_levelsource · rdm inesctec ptconnector:rdm_inesctec_pt@1.0.0
concepts[field].local:field:computer-science-aimapping · rdm inesctec ptconnector:rdm_inesctec_pt@1.0.0
concepts[field].local:field:engineeringmapping · rdm inesctec ptconnector:rdm_inesctec_pt@1.0.0
concepts[modality].local:modality:textenrichment · rdm inesctec ptkeyword-concept-rules@1.0.0title+description (75%)
concepts[topic].ckan_group:rdm_inesctec_pt:cs-computer-sciencesource · rdm inesctec ptconnector:rdm_inesctec_pt@1.0.0
created_datesource · rdm inesctec ptconnector:rdm_inesctec_pt@1.0.0
descriptionsource · rdm inesctec ptconnector:rdm_inesctec_pt@1.0.0/notes
license_textsource · rdm inesctec ptconnector:rdm_inesctec_pt@1.0.0
publication_datesource · rdm inesctec ptconnector:rdm_inesctec_pt@1.0.0
titlesource · rdm inesctec ptconnector:rdm_inesctec_pt@1.0.0/title
updated_datesource · rdm inesctec ptconnector:rdm_inesctec_pt@1.0.0