Googlescraper (Lippmannian Device)

Launch tool

Batch queries Google. Query the resonance of a particular term, or a series of terms, in a set of Websites.

Instructions

The Googlescraper (also known as the Lippmannian Device) queries Google and makes the results available for further analysis. In the top text box, place the source set, in this case a list of URLs. In the bottom text box, place key words. Google will be asked if each keyword occurs in each URL. Results are displayed as a tag cloud and an html table. They also are written to a text file which you can access at the bottom or through previous results.

Harvester feature: In the top box, you may also place a combination of URLs and text, and the URLs will be fetched out of the text and queried for the key words placed in the bottom box. Detailed instructions of use and use cases are available.

Sample project

The Googlescraper can be used for a number of specific research projects, including censorship research, and source distance research. The most common use of the tool is researching the presence as well as the ranking of particular sources within Google engine results. A sample project is this tag cloud, which visually presents unique hosts from top 100 URLs returned from the query of "synthetic biology." The hosts are sized by occurences of "Venter" on each site. The method for this project: 1) Search google for "synthetic biology". 2) Paste top 100 results in the top box, and enter Venter in the bottom box.

Course units using this tool

The Spheres
The Spheres Spheres as way of thinking about the Web Thinking of the Web in terms of spheres refers initially to the name of one of the most well known, the blogosphere ...

Other projects using this tool

Climate Change Skeptics
Introduction To what extent are climate change 'skeptics' present in the climate change spaces on the Web? The question is posed in order to gain insight into whether ...

Dmi Protocols
Protocols devised by the DMI This page is being replaced gradually by our new research protocols and methods page. Hyperlink Analysis Perfom an issue crawl ...

Issue Image Analysis
Issue Animals Research With climate change, animals become endangered. Global warming as well as global cooling threatens the habitat of species, as animals migrate ...

Methods By Theme
Digital Methods Summer School 2007: New Objects of Study 2010 2009 2008 2007 How does one do research online? What are the new objects of study, and how do they ...

Nofollow
Nofollow / Indexing Issues in the Blogosphere Introduction: Indexing and Ranking Search engine critiques generally focus on either the allocation of pages to be searched ...

Protocol Redistributed Content Discovery
1) Derive issue related sites known to be blocked in a country. 2) Query list of sites for a controversial subject matter or name in Google Scraper. Retain teaser ...

Protocol Surfer Rerouting
1) Familiarize oneself with the content of a set of blocked Websites, e.g., women's issues sites blocked in China.2) Query Web for the key words or issue language ...

Test Home
DMI Tools Digital Methods Project Overview FAQ Tag Cloud Introduction The Digital Methods Initiative is a contribution to doing research into the "natively ...

Tool Issue Network Cloud How To
Enter the URL of an Issuecrawler xml file. The xml source file URL looks like this: http://www.issuecrawler.net/files/inm 316224.xml The xml source file URL is located ...

I	Attachment	Action	Size	Date	Who
png	google_scraper.png	manage	9.2 K	05 Dec 2008 - 14:41	Anne Helmond
png	syn_bio_venter.png	manage	15.9 K	12 Dec 2008 - 14:42	Esther Weltevrede
png	syn_bio_venter_def.png	manage	51.2 K	12 Dec 2008 - 15:15	Esther Weltevrede

Topic revision: r15 - 05 Feb 2010 - 19:20:00 - Richard Rogers