Latest Register Log In

+ Advanced Search

Issues with Exceptionally Large Sites

Issues with Exceptionally Large Sites
By
06/02/04 (Edited 05/06/14)

Sites With Huge Amounts of Data

If you have thousands of forums/subforums, the regular forum selector on the submission/edit pages will fail because the full selector options can't be generated due to memory or time limitations. Switch on the level-based forum selector option at Admin Panel -> Settings -> Switches -- the level-based selector will work with millions of forums. Then reduce the number of levels in the submit selector to 0 at Admin -> Forums -> Forum Settings.

If you don't have any forums set to be invisible to specific usergroups, go to Admin Panel -> Settings -> Tweaks Editor and turn on the 'noinvisiblegroups' tweak. This will speed up many queries as it no longer has to check if the topic is in a visible forum for the viewing member.

If you have huge numbers of members, you may need to switch off the "keyword subscriptions" at Admin -> Settings -> Switches to reduce slow queries of the members table. You may also want to disable instant subscriptions there, as daily digests create far fewer emails (and thus far less server load) than instant subscriptions.

If you have hundreds of thousands of topics, the topic details page may become slow due to previous/next links. In WSN 9.0 or later, switch off the "Previous/Next links" switch at Admin -> Settings -> Switches... in earlier versions you'll need to remove this code from the topic details template instead:

<p class="center">
<IF {PREVIOUSEXISTS}><a href="{PREVIOUSURL}">{LANG_DETAILS_PREVIOUS}</a></IF>
<IF {PREVIOUSEXISTS} and {NEXTEXISTS}> | </IF>
<IF {NEXTEXISTS}><a href="{NEXTURL}">{LANG_DETAILS_NEXT}</a></IF>
</p>


Switching off the 'secondary forums' switch will speed things up on the details page by making the views update query faster, and will also speed the submit/edit pages. Retrievals may also become slow, and you can speed these up by switching off expirations and effective times.

With large numbers of topics in a directory, the effective time and expiration switches slow things down due to their need to get a live running total of topics that haven't expired or have become effective -- turn them off if you don't need them.

If the 'show topics in forum' page is slow, try removing
<IF {SWITCH_TOPICTYPES}>
<!-- BEGIN TYPE TOTALS -->
<br ><span class="minilabels">{TYPEDISPLAYNAME}:</span> {TYPETOTAL}
<!-- END TYPE TOTALS -->
</IF>
from that template (the live totals of each topic type become very slow with a large topics table).

If you have millions of topics, insertions (the addition of a new link) may become impractically slow to the point of breakage. To fix this you need to switch off 'similar topics' and use phpmyadmin to delete the fulltext index from the _links table of the database. You may then also convert the table from MyISAM to InnoDB for a further performance boost.

To speed up search results, you can enable the 'nosearchsort' tweak at Admin -> Settings -> Tweaks Editor. If combined with setting mix topic types to 'yes' in the type manager, this will stop all sorting of search results, which typically improves speed very much, though decreases relevance (note it doesn't apply to distance searches).

You may also need to be careful about the member rating forumla and topic importance formula you use. The default formula is a simple one that won't be a problem, but if you've changed it to something server-intensive (or originally installed a very old version which had a more intensive formula):
1) Go to Admin Panel -> Members -> Settings and change the rating formula to something simple.
2) Go to Admin Panel -> Topics -> Topic Settings and change the importance formula to something simple.

If searches are running slow, go to Admin -> Settings -> Search and turn the "Faster Any/All Words Search" option on. Then selected fewer searchable fields, if possible. The mysql fulltext search type will be the fastest but only for titles and descriptions. Of the others, 'contains' should be fastest.

Sites With Heavy Traffic

If your load levels have grown excessively high due to your large amount of traffic, consider whether or not you can use the cache system for guests. The cache system will make repeated loads of the same page by different people drastically faster.

If pages take too long to load sites should ensure that they do not have an excessive number of toplists on a page. If server load is an issue you may also want to consider turning on caching for the guest usergroup (Admin -> Settings -> General), perhaps triggered by a server load over 3.

If excessive RAM is being used, turning off some switches (such as private messaging and instant messaging) helps. Upgrading to PHP 5.4 will also save memory.

Admin Panel -> Settings -> System Configuration also offers an emergency load control mechanism, and Admin Panel -> Settings -> SEO allows you to throw 404s to any rogue bots or denial-of-service attackers.

Hung Queries

If hung queries are causing high server load and snowballing poor performance on your server, you can place a file (let's name it mysqlkill.php) in WSN's modifications directory to terminate all processes which have been hung for more than 60 seconds:
<?php
if (rand(1, 10) == 5)
{ // this bit autoterminates mysql processes that have been hung for more than a minute, to keep the server responsive:
$result = mysql_query("SHOW FULL PROCESSLIST");
while ($row=mysql_fetch_array($result)) { if ($row["Time"] > 60 ) mysql_query("KILL ".$row["Id"]); }
}
?>




Description Tips on changes you may need to make for good performace when you have many thousands of forums.
Rating
Views 3866 views. Averaging 1 view per day.