WSN Directory

Loading Presupplied Data Sets

Sun, 04 Dec 2016 16:54:59 GMT

When installing any WSN script, the "Pre-Load Data" section in the installer gives you the option to give your site the head start of using one of the provided data sets. These data sets provide commonly-desired category structures, or in some cases even listing data. The datasets currently available as of this writing (note some of these options may not show for some of the scripts where they're not really relevent):

Business categories: categories and subcategories for types of businesses, ideal for starting a business directory
Business categories by state/province: same as above but with the top level categories being USA states and Canadian provinces, with the business categories nested below that
USA cities by county by state: all USA cities and towns with more than a few thousand people, organized into county subcategories and state top level categories
USA counties by state: all USA counties as subcategories organized by state top level categories
USA states: the 50 USA states as top level categories
Counties of single state: you'll be prompted to select a USA state, and the counties of that state will be the top level categories
Cities by county of single state: you'll be prompted to select a USA state, and the state's counties will be top level categories while the state's towns will be subcategories.
PAD categories: the official software categories from the PAD shareware format
PAD listings (60K): tens of thousands of software listings organized by category
PAD listings (16K): a more carefully cultivated set of software listings organized by category
Web scripts categories: categories appropriate for a directory of web scripts

It's best if you make the correct selection during installation, but sometimes you may want to change your data set after you've already installed. This is possible. First, you must delete your current site data (categories, listings and associated info) and be sure that you really want to do that. Go to Admin Panel -> Miscellaneous -> Advanced Options and place the following in the "Run SQL" box:

TRUNCATE TABLE {PREFIX}categories;
TRUNCATE TABLE {PREFIX}links;
TRUNCATE TABLE {PREFIX}comments;
TRUNCATE TABLE {PREFIX}ratings;

Run that. Now your next step is to load one of the pre-loadable data sets. You'll do this by visiting a special URL in your web browser while logged in as an administrator. The URL varies slightly depending on which data you want to load:

Business categories: youradminpanelurl/loaddata.php?filled=1&datasource[0]=businesscategories&dataindex=0
Business categories by state/province: youradminpanelurl/loaddata.php?filled=1&datasource[0]=businesscategoriesbystate&dataindex=0
USA cities by county by state: youradminpanelurl/loaddata.php?filled=1&datasource[0]=cities&dataindex=0
USA counties by state: youradminpanelurl/loaddata.php?filled=1&datasource[0]=counties&dataindex=0
USA states: youradminpanelurl/loaddata.php?filled=1&datasource[0]=states&dataindex=0
Counties of single state: youradminpanelurl/loaddata.php?filled=1&datasource[0]=statecounties&dataindex=0&state=California
Cities by county of single state: youradminpanelurl/loaddata.php?filled=1&datasource[0]=statecities&dataindex=0&state=California
PAD categories: youradminpanelurl/loaddata.php?filled=1&datasource[0]=pad&dataindex=0
PAD listings (60K): youradminpanelurl/loaddata.php?filled=1&datasource[0]=padlistings&dataindex=0
PAD listings (16K): youradminpanelurl/loaddata.php?filled=1&datasource[0]=padlistingsalt&dataindex=0
Web scripts categories: youradminpanelurl/loaddata.php?filled=1&datasource[0]=scriptdirectory&dataindex=0

For the single state ones, be sure to change California to your desired state at the end of the URL.

MLS Listings

Sun, 04 Sep 2011 16:13:22 GMT

Though not available to the general public, real estate agents have access to an MLS IDX database containing real estate listings. WSN can import these listings. Go to Admin Panel -> Listings -> Add Listings -> MLS listings import and enter the FTP info to your MLS database in order to run an import. WSN will automatically fetch the latest files from the MLS ftp site once a day, ensuring your listings are always up to date.

Before you import MLS data, you'll need to get relevant real estate fields (like number of bedrooms and bathrooms) added. Simply go to Admin Panel -> Themes -> Add Themes and install the "Real Estate" theme (it's recommended that you check the boxes to select it for all and delete all other themes). When you install this theme it'll automatically add all the MLS fields to your database and template. You can then remove the fields you don't want to display by editing the custom fields submit/edit and display templates at Admin Panel -> Themes -> Manage Templates.

There are several tweaks relating to MLS data processing. The mlstextrate tweak allows you to process more or fewer listings per page load, which may be necessary to get the automatic import to finish faster on a rarely-visited site. filtermlsfield specifies a field that must be filled with filtermlsvalue in order for a listing to be imported -- for example, if you only want listings from El Dorado county you'd set filtermlsfield to county and filtermlsvalue to El Dorado.

Pre-Supplied Software Data Set

Thu, 03 Mar 2011 15:48:19 GMT

WSN Software Directory comes with the option of installing 60,000+ software listings from the official PAD file repository. Just select this from the pre-load data options during setup.

If you'd like to replace the current listings on your site with a new default dataset, you can do this by following these steps:
1) Bulk-delete your current data by doing to Admin Panel -> Miscellenous -> Advanced Options and running the queries
TRUNCATE TABLE {PREFIX}categories;
TRUNCATE TABLE {PREFIX}links;
TRUNCATE TABLE {PREFIX}comments;
TRUNCATE TABLE {PREFIX}ratings;
in the run SQL queries box.
2) Go to yourwsnurl/admin/loaddata.php?filled=1&datasource[0]=padlistings&dataindex=0 and let that run to conclusion (will take a few hours).

CSV Imports

Sat, 08 Jan 2011 10:52:49 GMT

CSV is a simple database format which popular spreadsheet software can read and write. If you have or want to create data in this format, WSN can read it and load articles from it.

Go to Admin Panel -> Listings -> Add Listings and scroll to CSV Import. Select your CSV, submit it, and you'll reach a screen which asks you what each CSV field is for. Carefully make the appropriate selections.

Since a CSV can't contain the actual data of a file, if you want to attach files you'll need to use one of two methods:
(a) Pre-upload the files to the attachments subdirectory, specify the filename (without the path) in one (or more) of the CSV fields, and select to associate the appropriate part to file #1 (or #2, etc if multiple) when importing your CSV.
(b) Specify the URL of a file which is on the internet in one (or more) of the CSV fields, and select to associate the appropriate part to file #1 (or #2, etc if multiple) when importing your CSV. This can be a full URL from any site that allows hotlinking. Note that the files will slowly import over time, or when you regenerate attachments.

If you include secondary categories in your data, you can separate the categories in the list with a pipe, semicolon or comma and you can use either id/names of existing categories or names for new categories to create. Example valid formats: "13|3897" or "Business;Recreation" or "Sacramento, San Francisco" or many other combinations.

Note: If your CSV file is especially large or your server memory limits are especially small, you may get a timeout or memory error and need to split your CSV into smaller parts to process one at a time with CSV-splitting software like this.

Note: If you want to do a CSV import of categories (rather than listings), you can use Admin Panel -> Categories -> Add Categories -> Flat-File Import to import a CSV of categories.

Scrape URLs From Page

Tue, 21 Aug 2007 01:06:47 GMT

At Admin Panel -> Listings -> Add Listings you have a "Page Link Scraper" area. This allows you to scrape URLs from a web page to add them to your directory. Type a URL here and it'll place all URLs from that page into your "bulk addition" box except:

Relative URLs / internal links (URLs which contain the domain name of the page being scraped, or no domain at all), if you check the box saying to exclude them.
If you've selected to not allow people to submit duplicates, all duplicates (URLs which area already in your directory) will be excluded.

You'll be able to review the list and remove those links you don't actually want to add. For those you do want, it can retrieve the titles and meta descriptions automatically. This can help you build your directory quickly, if you know a page (say a wikipedia list) which lists the sites you'll want.