RDF Crawler

RDF Crawler appears to be a rather dormant project, the last release is dated 27 November 2000. Although the system is targetted at RDF data, it is in an early state of development and does not handle robot.txt exclusion policies for instance. There may be implementation details this project could benefit from.

There does not appear to be any licence statement for RDF Crawler, it depends upon the following packages:

• GNU Regular Expressions, released under the GPL licence.
• Apache Xerces, released under the Apache Software Licence
• IBM Alphaworks' XML4J, now part of Apache Xerces, as above.
• RDF API by Sergey Melnik at Stanford University has no explicit licence terms. It also depends on the public domain SAX package, and W3C DOM and RDF packages under the W3C® Software Notice and License.