Article Downloader

Download content of scraped results

One of the advanced applications of Midesk is a multi-step scraping where you first identify URLs in the first step and download the underlying content in the second.

For example, you may point a monitoring job to a job board or a blog and monitor the latest entries. Once a new entry is found, you may define that such an entry shall be saved to your system, for example, to a specificNews Channel.

Midesk is clever enough to download only new entries (preventing duplicates). It also allows you to choose what to do with entries once they disappeared from the target website.