WebFetch


WebFetch is a set of Perl 5 modules written by Ian Kluft from 1998 through 2009 that provides a framework for retrieving information feeds from the Web, or posting a feed on a web site. It is intended to be run on a Unix or Linux server automatically, such as from cron. The current version 0.13 was released September 20, 2009. It is Open Source software, licensed under the GNU General Public License.
WebFetch can collect various types of inputs via plugin modules and send or store them to various outputs also via plugin modules. Inputs include RSS, Atom, local news feed files, and perl data structures. Outputs include perl data structures, the Template Toolkit and pages in TWiki systems.

History

Before there were RSS and Atom aggregation, WebFetch started with modules to retrieve news in ad hoc feed formats of various web sites, including Slashdot.
WebFetch is available from the Comprehensive Perl Archive Network. The following Perl modules are included with it.
The following Perl modules were based on ad hoc news formats from before RSS, and in some cases web scraping. They used to be provided with WebFetch in the past, but were deprecated and removed.