Counting Records


This is the development page for a new core method getCount().

There are numerous places in Wikka where we need to know "how many" of a certain thing there are in the database: how many pages, how many comments, how many pages owned by the logged-in user, ... etc. And there will be more.

Currently all these counts are retrieved via different methods: sometimes building a query and getting the count directly (the most efficient way); sometimes by using the LoadSingle() method after building a query, which is highly inefficient, as it effectively first asks MySQL to assign a name to the count and return records in an associative array, then retrieves the first (only!) item from the array of records, and finally retrieves the number from the associative array by name.
 

A single method


To avoid all this inconsistency and inefficiency, here is a simple method that does nothing but retrieve a count and return it as an integer.

The getCount() method


Implemented as beta feature as of 2005-07-19.
The following method is added to libs\Wakka.class.php right after the LoadAll() method:
 

    /**
     * Generic 'count' query.
     *
     * Get a count of the number of records in a given table that would be matched
     * by the given (optional) WHERE criteria. Only a single table can be queried.
     *
     * @access  public
     * @uses    Query()
     *
     * @param   string  $table  required: (logical) table name to query;
     *                          prefix will be automatically added
     * @param   string  $where  optional: criteria to be specified for a WHERE clause;
     *                          do not include WHERE
     * @return  integer number of matches returned by MySQL
     */

    function getCount($table,$where='')                         # JW 2005-07-16
    {
        // build query
        $where = ('' != $where) ? ' WHERE '.$where : '';
        $query = 'SELECT COUNT(*) FROM '.$this->config['table_prefix'].$table.$where;

        // get and return the count as an integer
        return (int)mysql_result($this->Query($query),0);
    }


As can be seen from the comment, all we pass to the method is the logical table name to query, and optionally the criteria to be used in a WHERE clause. The method builds a query adding the missing bits and returns the result (making sure it is an integer).

Doing it consistently


Now, with the getCount() method we have the tool to consistently and efficiently retrieve record counts where needed. Let's first look at where this can be used in the current 1.1.6.0 release.

Wikka Core


There is one method in the Wikka Core where a count of records is used, so we can aply the new method here.

ExistsPage() method
Current code:
  1.     /**
  2.      * Check by name if a page exists.
  3.      *
  4.      * @author      {@link http://wikka.jsnx.com/JavaWoman JavaWoman}
  5.      * @copyright   Copyright © 2004, Marjolein Katsma
  6.      * @license     http://www.gnu.org/copyleft/lesser.html GNU Lesser General Public License
  7.      * @version     1.0
  8.      *
  9.      * @access      public
  10.      * @uses        Query()
  11.      *
  12.      * @param       string  $page  page name to check
  13.      * @return      boolean  TRUE if page exists, FALSE otherwise
  14.      */
  15.     function ExistsPage($page)
  16.     {
  17.         $count = 0;
  18.         $query =    "SELECT COUNT(tag)
  19.                     FROM ".$this->config['table_prefix']."pages
  20.                     WHERE tag='".mysql_real_escape_string($page)."'";
  21.         if ($r = $this->Query($query))
  22.         {
  23.             $count = mysql_result($r,0);
  24.             mysql_free_result($r);
  25.         }
  26.         return ($count > 0) ? TRUE : FALSE;
  27.     }


Modified code implemented as beta feature as of 2005-07-19.
This can now be rewritten as:
 

    /**
     * Check by name if a page exists.
     *
     * @author      {@link http://wikka.jsnx.com/JavaWoman JavaWoman}
     * @copyright   Copyright © 2004, Marjolein Katsma
     * @license     http://www.gnu.org/copyleft/lesser.html GNU Lesser General Public License
     * @version     1.1
     *
     * @access      public
     * @uses        getCount()
     *
     * @param       string  $page  page name to check
     * @return      boolean  TRUE if page exists, FALSE otherwise
     */

    function ExistsPage($page)
    {
        $where = "`tag` = '".mysql_real_escape_string($page)."'";
        $count = $this->getCount('pages',$where);
        return ($count > 0);
    }


Actions


actions/countcomments.php
Current code:
  1. <?php
  2. /**
  3.  * Print total number of comments in this wiki.
  4.  */
  5. $commentsdata = $this->LoadSingle("SELECT count(*) as num FROM ".$this->config["table_prefix"]."comments");
  6. echo $commentsdata["num"];
  7. ?>


This can now be rewritten as:
  1. <?php
  2. /**
  3.  * Print total number of comments in this wiki.
  4.  */
  5. echo $this->getCount('comments');
  6. ?>


actions/countowned.php
Current code:
  1. <?php
  2. /**
  3.  * Print number of pages owned by the current user.
  4.  */
  5. $str = 'SELECT COUNT(*) FROM '.$this->config["table_prefix"].'pages WHERE `owner` ';
  6. $str .= "= '" . $this->GetUserName() . "' AND `latest` = 'Y'";
  7. $countquery = $this->Query($str);
  8. $count  = mysql_result($countquery, 0);
  9. echo $this->Link('MyPages', '', $count,'','','Display a list of the pages you currently own');
  10.  
  11. ?>


This can now be rewritten as:
  1. <?php
  2. /**
  3.  * Print number of pages owned by the current user.
  4.  */
  5. $where = "`owner` = ".$this->GetUserName()." AND `latest` = 'Y'";
  6. $count = $this->getCount('pages',$where);
  7. echo $this->Link('MyPages', '',$count,'','','Display a list of the pages you currently own');
  8. ?>


actions/countpages.php
Current code:
  1. <?php
  2. /**
  3.  * Print the total number of pages in this wiki.
  4.  */
  5. $pagedata = $this->LoadSingle("SELECT count(*) as num FROM ".$this->config["table_prefix"]."pages WHERE latest = 'Y'");
  6. echo $this->Link('PageIndex', '', $pagedata['num'],'','','Display an alphabetical page index');
  7.  
  8. ?>


This can now be rewritten as:
  1. <?php
  2. /**
  3.  * Print the total number of pages in this wiki.
  4.  */
  5. $where = "`latest` = 'Y'";
  6. $count = $this->getCount('pages',$where);
  7. echo $this->Link('PageIndex', '',$count,'','','Display an alphabetical page index');
  8. ?>


actions/countusers.php
Current code:
  1. <?php
  2. /**
  3.  * Print number of registered users.
  4.  */
  5. $userdata = $this->LoadSingle("SELECT count(*) as num FROM ".$this->config["table_prefix"]."users ");
  6. echo $userdata["num"];
  7. ?>


This can now be rewritten as:
  1. <?php
  2. /**
  3.  * Print number of registered users.
  4.  */
  5. echo $this->getCount('users');
  6. ?>


actions/highscores.php
This uses the following code to get a 'total' count:
  1.     $str = 'SELECT COUNT(*) FROM '.$this->config["table_prefix"].'pages WHERE `latest` = \'Y\' ';
  2.     $totalQuery = $this->Query( $str );
  3.     $total  = mysql_result($totalQuery, 0);


This can now be rewritten as:
  1.     $where = "`latest` = 'Y'";
  2.     $total = $this->getCount('pages',$where);


While this is actually the same query, we'd use the getCount() method here for consistency (and hiding complexity).

actions/lastusers.php
This prints the number of pages owned by a particular (new) user as follows:
  1. if ($stat!=="0") $num = $this->LoadSingle("select count(*) as n from ".$this->config["table_prefix"]."pages where owner='".$user["name"]."' AND latest = 'Y'");
  2.  $htmlout .= "    <td>".$this->Link($user["name"])."</td>\n    <td>".($stat!=="0"?" . . . . . (".$num["n"].")":"")."</td>\n    <td>(".$user["signuptime"].")</td>\n";


This can now be (better) rewritten as:
  1.     if ($stat !== "0")
  2.     {
  3.         $where = "`owner` = '".$user['name']."' AND `latest` = 'Y'";
  4.         $htmlout .= "    <td>".$this->Link($user['name'])."</td>\n    <td>"." . . . . . (".$this->getCount('pages',$where).")"."</td>\n    <td>(".$user['signuptime'].")</td>\n";
  5.     }
  6.     else
  7.     {
  8.         $htmlout .= "    <td>".$this->Link($user['name'])."</td>\n    <td></td>\n    <td>(".$user['signuptime'].")</td>\n";
  9.     }

There are other ways in which this action can (and should) be improved but we're looking only at counting records here.

actions/ownedpages.php
This uses two counts. Current code:
  1.    $str = 'SELECT COUNT(*) FROM '.$this->config["table_prefix"].'pages WHERE `owner` ';
  2.     $str .= "= '" . $this->GetUserName() . "' AND `latest` = 'Y'";
  3.     $countQuery = $this->Query( $str );
  4.    
  5.     # get the total # of pages
  6.     $str = 'SELECT COUNT(*) FROM '.$this->config["table_prefix"].'pages WHERE `latest` = \'Y\' ';
  7.     $totalQuery = $this->Query( $str );    
  8.    
  9.     $count  = mysql_result($countQuery, 0);
  10.     $total  = mysql_result($totalQuery, 0);


This can now be rewritten as:
  1.     $whereOwner = "`owner` = '" . $this->GetUserName() . "' AND `latest` = 'Y'";
  2.     $whereTotal   = "`latest` = 'Y'";
  3.  
  4.     $count = $this->getCount('pages',$whereOwner);
  5.     $total = $this->getCount('pages',$whereTotal);


As with the highscores action, this uses actually the same queries, but we use the getCount() method for the same reasons.

Beta features and proposed code


A number of current beta features and other code proposed for a next release can also make good use of the getCount() method. We'll look at them here.

Actions


actions/pageadmin.php
The pageadmin action has now been fixed using modifications somewhat along the lines indicated below (though details differ); see PageAdminAction for the details.
See also the PageAdminAction development page and WikkaBetaFeatures.
 

This action uses a lot of counts but (in version 0.3.1) was retrieving them very inefficiently, in most cases by actually retrieving all records and then counting those, without ever using the records themselves. We'll simply show some "before-and-after" code illustrating how the getCount() method can help here. Line numbers are as they were in beta version 0.3.1 as installed on this site.

Applying the new getCount() method applied as outlined below not only solves the problem of the (now) missing LoadReferrers() method but makes the whole pageadmin action a lot more efficient. See also WikkaBetaFeatures.

Before (version 0.3.1):
  1.         // restrict MySQL query by search string
  2.         $where = 'WHERE tag LIKE "%'.$q.'%"and latest = "Y"';
  3.  
  4.         // get total number of pages
  5.         $pages = $this->LoadSingle('SELECT count(*) as n FROM '.$this->config['table_prefix'].'pages '.$where);
  6.         $numpages = $pages['n'];

After:
  1.         // restrict MySQL query by search string
  2.         $where = "`tag` LIKE '%".$q."%' AND `latest` = 'Y'";
  3.         // get total number of pages
  4.         $numpages = $this->getCount('pages',$where);


Before:
  1.                 // get page revisions and create revision link if needed
  2.                 $revisions = $this->LoadRevisions($page['tag']);
  3.                 $rv = count($revisions);

After:
  1.                 // get page revisions and create revision link if needed
  2.                 $where = "`tag` = '".$page['tag']."'";
  3.                 $rv = $this->getCount('pages',$where);


Before:
  1.                 // get page comments and create comments link if needed
  2.                 $comments = $this->LoadComments($page['tag']);         
  3.                 $cn = count($comments);

After:
  1.                 // get page comments and create comments link if needed
  2.                 $where = "`page_tag` = '".$page['tag']."'";
  3.                 $cn = $this->getCount('comments',$where);


Before:
  1.                 // get page backlinks and create backlinks link
  2.                 $backlinks = $this->LoadPagesLinkingTo($page['tag']);          
  3.                 $bn = count($backlinks);

After:
  1.                 // get page backlinks and create backlinks link
  2.                 $where = "`to_tag` = '".$page['tag']."'";
  3.                 $bn = $this->getCount('links',$where);


Before:
  1.                 // get page referrers and create referrer link
  2.                 $referrers = $this->LoadReferrers($page['tag']);           
  3.                 $rn = count($referrers);

After:
  1.                 // get page referrers and create referrer link
  2.                 $where = "`page_tag` = '".$page['tag']."'";
  3.                 $rn = $this->getCount('referrers',$where);

The $where variable is the same here as for the comments count, so line 193 is actually superfluous! So rather than building the $where variables just before each getCount() call, it would be clearer to build both in advance.

actions/useradmin.php
See also the UserAdminAction development page and WikkaBetaFeatures.

Not surprisingly, this Admin action also uses several counts. But in one case it's pre-building a WHERE clause to be used in two different queries, only one of which is fo counting, so we need to adapt the code for both. Three blocks of "before" and three "after" will show how to handle this:

Before:
  1.     // search results
  2.     $where = ($_POST["search"])? "WHERE name LIKE \"%".$_POST["search"]."%\"" : "";

  1.     // 1. Get total number of users
  2.     $users = $this->LoadSingle("select count(*) as n FROM ".$this->config["table_prefix"]."users ".$where);
  3.     $numusers = $users['n'];

  1.     // get user data
  2.     $userdata = $this->LoadAll("SELECT name, email, signuptime FROM ".$this->config["table_prefix"]."users ".
  3.     $where." ORDER BY ".$sort." ".$d." limit ".$s.", ".$l);


After:
  1.     // search results
  2.     $where = ($_POST['search']) ? '`name` LIKE "%'.$_POST['search'].'%"' : '';

  1.     // 1. Get total number of users
  2.     $numusers = $this->getCount('users',$where);

  1.     // get user data
  2.     $userdata = $this->LoadAll("SELECT name, email, signuptime FROM ".$this->config["table_prefix"]."users ".
  3.         (($where != '') ? "WHERE ".$where : '')." ORDER BY ".$sort." ".$d." limit ".$s.", ".$l);


Farther down, three more counts are used:

Before:
  1.             $num = $this->LoadSingle("SELECT COUNT(*) AS n FROM ".$this->config["table_prefix"].
  2.             "pages WHERE owner='".$user["name"]."' AND latest = 'Y'");
  3.             $numchanges = $this->LoadSingle("SELECT COUNT(*) AS m FROM ".$this->config["table_prefix"].
  4.             "pages WHERE user='".$user["name"]."'");
  5.             $numcomments = $this->LoadSingle("SELECT COUNT(*) AS q FROM ".$this->config["table_prefix"].
  6.             "comments WHERE user='".$user["name"]."'");


After:
  1.             $whereOwner = "`owner` ='".$user['name']."' AND latest = 'Y'";
  2.             $whereUser = "`user` = '".$user['name']."'";
  3.             $num = $this->getCount('pages',$whereOwner);
  4.             $numchanges = $this->getCount('pages',$whereUser);
  5.             $numcomments = $thiis->getCount('comments',$whereUser);



Handlers


While the rewritten referrers handler uses a number of counts, these are part of larger queries: no rewrite necessary here. The new review blacklist handler can use the getCount() method though:

handlers/page/review_blacklist.php
See also the AdvancedReferrersHandler development page and WikkaBetaFeatures.

The new review blacklist handler does a total count as well which can be simplified now.
Before:
  1.     // get total number of domains in blacklist
  2.     $query_refcount  = 'SELECT COUNT(spammer) AS total';
  3.     $query_refcount .= ' FROM '.$pre.'referrer_blacklist';

  1.     $totalrefs = $this->LoadSingle($query_refcount);


After (the first block at line 179 is no longer needed at all!):
  1.     $totalrefs = $this->getCount('referrer_blacklist');



Todo


The new getCount() method itself and its implementation in ExistsPage() have been implemented as beta on this site now. A new version of the (beta) pageadmin action now uses it as well (solving an outstanding bug due to the AdvancedReferrersHandler). However:
  1. All other possible code changes outlined here still need to be tested.
  2. The method could be extended somewhat to make it more capable; for instance:
    • allow the table parameter to be an array, and build the FROM clause using the list of table names

Comments?


As always, comments and suggestions more than welcome.


CategoryDevelopmentCore CategoryDevelopmentActions CategoryDevelopmentHandlers
There are 2 comments on this page. [Show comments]
Valid XHTML :: Valid CSS: :: Powered by WikkaWiki