Latest Register Log In

+ Advanced Search

Scrape URLs From Page

Scrape URLs From Page
08/20/07 (Edited 02/02/14)

At Admin Panel -> Listings -> Add Listings you have a "Page Link Scraper" area. This allows you to scrape URLs from a web page to add them to your directory. Type a URL here and it'll place all URLs from that page into your "bulk addition" box except:

  • Relative URLs / internal links (URLs which contain the domain name of the page being scraped, or no domain at all), if you check the box saying to exclude them.
  • If you've selected to not allow people to submit duplicates, all duplicates (URLs which area already in your directory) will be excluded.

You'll be able to review the list and remove those links you don't actually want to add. For those you do want, it can retrieve the titles and meta descriptions automatically. This can help you build your directory quickly, if you know a page (say a wikipedia list) which lists the sites you'll want.

Description Importing from a URL.
Views 1225 views. Averaging 0 views per day.