The DMI toolbar is a Firefox extension that provides extra functionality to the DMI tools. Currently it provides off-loading of HTTP requests to the client browser. This means that the requests do not happen through the DMI server but from the computer using the Firefox extension.
This extension and the HTTP off-loading are an experimental way of performing digital methods research. Use caution when contemplating the results. To avoid researching personalized results, make sure to follow the 'research browser' guidelines below before each scrape
Newer versions of Firefox, starting with Firefox 57 (Quantum) require you to re-install the extension. Automatic updates do not work.
- Install and start Firefox
- Within Firefox, click on and install the DMI Firefox extension and restart Firefox. You also can right-click and 'save as' to desktop, and drag and drop the file, dmitools.xpi, into Firefox.
- Make sure to allow pop-ups from tools.digitalmethods.net
The research browser
Depending on your research needs, use a research browser to avoid bias by stored cookies, personalisation settings and other factors that may change the results returned.
The DMI toolbar works in combination with the Google Scraper
or Lippmannian Device
, amongst other tools. Here we explain how the researcher can be disentangled from Google.
When using the Google scraper or Lippmannian Device with our Firefox toolbar, the researcher needs to take a few steps to ensure that day to day activities do not interfere with research.
Traditionally, all our scrapers would connect to websites from DMI's servers. As some services such as Google do not like automated requests, they would temporarily block all requests coming from the IP-address performing thoses requests (i.e. DMI's servers). Before Google blocks an IP-address for a longer time, it will offer a CAPTCHA
to the user in order to verify that the requests are made by a human. This is not something our servers could confirm.
By offloading the requests to the researcher's browser 1) less requests originate from the same IP, and 2) if Google finds there are too many requests, the CAPTCHA can be filled in by the researcher.
Can I get blocked?
We have tested the toolbar for many requests and were confronted with multiple CAPTCHAs. Using this technique our IP-addresses were never blocked. Experience learns that Google only blocks IP-addresses temporarily.
Nothing happens after filling in the CAPTCHA returned by the Google Scraper.
Make sure to close the tab where you filled in the CAPTCHA
Tell me more about how Google personalizes my results.
Please note that this plugin is provided as is, without any guarantee about its functionality or proper use. This plugin is only provided for use in academic research.