Friday, December 21, 2018

web crawler - Scan and map website and log all links that have "particular-string" in them




Is there a tool that would scan a website and log all links that have a particular string in them?



I see Teleport Pro, but it seems like it does make a copy of the site, but I only need links.


Answer



You'll need some kind of offline browser in order to automatically traverse all of the links on the site. I haven't used Teleport Pro but HTTrack is a free offline browser that's pretty powerful. They provide an example of how you can traverse a site looking for a particular keyword.


No comments:

Post a Comment

hard drive - Leaving bad sectors in unformatted partition?

Laptop was acting really weird, and copy and seek times were really slow, so I decided to scan the hard drive surface. I have a couple hundr...