Is there a unit somewhere that can extract links from html pages? I want to be able to recursively add pages to a chm archive.

Thanks,

Andrew Haines

Andrew Haines wrote

Is there a unit somewhere that can extract links from html pages? I want to be able to recursively add pages to a chm archive.

I think this from l505 will work:
http://lists.freepascal.org/lists/fpc-announce/2007-February/000536.html

but are there any units that come default with fpc that can do this?

Andrew

None that I am aware of.

Michael.

Andrew Haines wrote

Is there a unit somewhere that can extract links from html pages? I want to be able to recursively add pages to a chm archive.

I created a program called GetLinks in a couple minutes: http://opensvn.csie.org/pspcgi/general-utilities/parser/html/demo/getlinks.pas

Latest html parser files in progress:
http://opensvn.csie.org/pspcgi/general-utilities/parser/html/

p.s. It is a little heavy handed to create and free two classes just to do a simple task like get some urls from a web page. Some day I'll make a simpler even lighter version of that parser for quick and dirty usage, but still keep the old one maybe. But whatever works, works!

This is great stuff thanks :)

I've just made a chm sitemap reader using your units. Next a writer.

Regards,

Andrew

> Is there a unit somewhere that can extract links from html pages? I want > to be able to recursively add pages to a chm archive. I created a program called GetLinks in a couple minutes:

Updated the files and changed the htmlutil functions a bit.

Also, created a recursive example that uses Synapse.. and grabs web links *infinitely* until it finds no more links (using nested pascal function for the recursion, Torvalds hates those).

The recursive demo probably won't work with file:// style links since it invokes Synapse, and only simple http relative paths work as is. (not sure if CHM files use file:// style links, guessing).

Latest download includes recursive extractor and getlinks demo: http://z505.com/download/pascal/html/fast-html-parser-jul-02-2007.zip http://sourceforge.net/project/showfiles.php?group_id=145841&package_id=212708&release_id=520417