Jonathan Holland

A RedditClone in a some unmeasured amount of time and ASP.NET MVC

2/7/2010

TLDR :Full code is here. Running Sandbox is here.

Writing a crappy, subpar clone of Reddit seems to have become the latest FizzBuzz fad.

In the last week, I've seen:

And the list goes. All of these implement simply implement posting and voting, and with the exception of the PHP one, don't use any form of persistance (The perl one might, I can't read it).

I wanted to play, but I also wanted to provide a little more features, as I don't believe any of those actually count as a "Reddit Clone". I decided on the following features:

View Links ordered by score
Submit Links
Vote on links
Comment on links in typical nested reddit fashion
Vote on comments
Order comments by score
Use a real database as persistance

These features make my project a bit more ambitious than the others, but also makes it a nice technical showcase for how concise a program can be.

Since Java was already done, I decided to use C# and ASP.NET MVC as the language and framework for my clone.

The first order of busines is building the data model. Because I am going for simplicity I am going to define the two tables I need using C# classes and Linq to SQL attributes. Using this Linq to SQL will generate the database for me the first time the application is ran.

My model is simple:

Submissions
Comments

The submissions class:

    1     [Table(Name = "Submissions")]
    2     public class Submission
    3     {
    4         private EntitySet<Comment> _comments
    5            = new EntitySet<Comment>();
    6 
    7         public Submission()
    8         {
    9             this.Score = 1;
   10         }
   11 
   12         [Column
   13             (IsPrimaryKey = true,
   14             IsDbGenerated = true,
   15             DbType = "INT NOT NULL IDENTITY")]
   16         public int ID { get; set; }
   17 
   18         [Column]
   19         public string Title { get; set; }
   20 
   21         [Column]
   22         public string Link { get; set; }
   23 
   24         [Column]
   25         public int Score { get; set; }
   26 
   27         [Association(Storage = "_comments", OtherKey = "SubmissionID")]
   28         public EntitySet<Comment> Comments
   29         {
   30               get { return this._comments; }
   31               set { this._comments.Assign(value); }
   32         }       
   33     }

And the Comment class:

    1     public class Comment
    2     {
    3         private EntitySet<Comment> _childComments
    4           = new EntitySet<Comment>();
    5 
    6         public Comment()
    7         {
    8             this.Score = 1;
    9         }
   10 
   11         [Column
   12             (IsPrimaryKey = true,
   13             IsDbGenerated = true,
   14             DbType = "INT NOT NULL IDENTITY")]
   15         public int ID { get; set; }
   16 
   17         [Column]
   18         public Nullable<int> ParentCommentID { get; set; }
   19 
   20         [Column]
   21         public int SubmissionID { get; set; }
   22 
   23         [Column]
   24         public string Text { get; set; }
   25 
   26         [Column]
   27         public int Score { get; set; }
   28 
   29         [Association(Storage = "_childComments", OtherKey="ParentCommentID", ThisKey = "ID")]
   30         public EntitySet<Comment> ChildComments
   31         {
   32             get { return this._childComments; }
   33             set { this._childComments.Assign(value); }
   34         }
   35     }

With those two classes defined, all I need to do now is create a DataContext:

    1     [Database(Name = "RedditClone")]
    2     public class RedditModel : DataContext
    3     {
    4         private static string _connString = 
    5             ConfigurationManager.ConnectionStrings["RedditClone"].ConnectionString;
    6 
    7         public RedditModel() : base(_connString) 
    8         {            
    9             if (!this.DatabaseExists())            
   10                 this.CreateDatabase();            
   11         }
   12 
   13         public Table<Submission> Submissions
   14         {
   15             get { return this.GetTable<Submission>();}
   16         }
   17 
   18         public Table<Comment> Comments
   19         {
   20             get { return this.GetTable<Comment>(); }
   21         }
   22     }

The first time a RedditModel class is instanciated, Linq to SQL will build a database with the following schema:

This approach works very well for a simple project, as it makes installation very simple. All you have to do is drop the files onto a server and browse to it and you have the full database built and ready.

At this point, I have a full persistance layer completely finished and have used roughly 50 lines of code...Not bad.

However, I can't do much with a persistance layer without a frontend. So I need to define some MVC Controllers to handle requests. My design calls for two controllers:

SubmissionController
CommentController

SubmissionController will perform the following tasks:

List all submissions
Load a single submission
Submit a submission
Vote (Up and Down) on submissions

CommentController will perform the following tasks:

Create a commment
Vote (Up and down) on comments

Because of how I structured the data model, I don't need a action to request comments, they lazily load when you access a submission.

I'll start with the "Index" action on SubmissionController. This action responds to GET requests to the Submissions controller and will return a list of submisssions ordered by score:

    1         public ActionResult Index()
    2         {
    3             RedditModel model = new RedditModel();
    4             return View(
    5                 "Index",
    6                 model.Submissions
    7                     .OrderByDescending(s => s.Score)
    8                     .ToList());
    9         }

As you can see, the heavily lifting of data manipulation is easily abstracted into the model. All this actually has to do is grab the data and it needs and pass it to the view (Which I have yet to define).

The "View" action is equally simple:

    1         public ActionResult View(int id)
    2         {
    3             RedditModel model = new RedditModel();
    4             return View(
    5                     "Comments", 
    6                     model.Submissions
    7                         .Where(s => s.ID == id)
    8                         .FirstOrDefault());
    9         }

With these two actions defined, I have handled the read-only aspect of my Reddit Clone. However, I want the user to be able to submit new links, as well as vote. So I'll add three more actions:

    1         [ValidateInput(false)]
    2         [AcceptVerbs(HttpVerbs.Post)]
    3         public ActionResult Submit(FormCollection collection)
    4         {
    5             RedditModel model = new RedditModel();
    6             Submission newSub = new Submission();
    7 
    8             UpdateModel(newSub, new[] { "Title", "Link" });
    9 
   10             model.Submissions.InsertOnSubmit(newSub);
   11             model.SubmitChanges();
   12 
   13             return RedirectToAction("Index");
   14         }

The "Submit" action responds to a POST verb and creates a new submission. Due to the nice MVC UpdateModel method, it is very easy to map a POST to my model class.

Finally, I need to add voting. To do this I define a "VoteUp" action and a "VoteDown" action. Both of these call a private method that adjusts the vote count for the submission:

    1         public ActionResult VoteUp(int id)
    2         {
    3             vote(id, 1);
    4             return RedirectToAction("Index");
    5         }
    6 
    7         public ActionResult VoteDown(int id)
    8         {
    9             vote(id, -1);
   10             return RedirectToAction("Index");
   11         }
   12 
   13         private void vote(int id, int direction)
   14         {
   15             RedditModel model = new RedditModel();
   16 
   17             Submission voteSub = model.Submissions.Where(s => s.ID == id).FirstOrDefault();
   18             voteSub.Score += direction;
   19 
   20             model.SubmitChanges();
   21         }

With this done, I'm at about 100 lines of C#, and I have a more functional RedditClone than the other examples.

I still need to add the CommentsController, which is very similar to the SubmissionController:

    1     public class CommentsController : Controller
    2     {
    3         [ValidateInput(false)]
    4         [AcceptVerbs(HttpVerbs.Post)]
    5         public ActionResult Create(FormCollection collection)
    6         {
    7             RedditModel model = new RedditModel();
    8             Comment newComment = new Comment();            
    9 
   10             UpdateModel(newComment, new[] { "SubmissionID", "ParentCommentID", "Text" });            
   11 
   12             model.Comments.InsertOnSubmit(newComment);
   13             model.SubmitChanges();
   14 
   15             return RedirectToRoute(new { controller = "Submissions", action = "View", id = newComment.SubmissionID });   
   16         }
   17 
   18         public ActionResult VoteUp(int id, int submissionID)
   19         {
   20             vote(id, 1);
   21             return RedirectToRoute(new { controller = "Submissions", action = "View", id = submissionID });
   22         }
   23 
   24         public ActionResult VoteDown(int id, int submissionID)
   25         {
   26             vote(id, -1);
   27             return RedirectToRoute(new { controller = "Submissions", action = "View", id = submissionID });
   28         }
   29 
   30         private void vote(int id, int direction)
   31         {
   32             RedditModel model = new RedditModel();
   33 
   34             Comment voteComment = model.Comments.Where(c => c.ID == id).FirstOrDefault();
   35             voteComment.Score += direction;
   36 
   37             model.SubmitChanges();
   38         }
   39     }

This wraps up all of the code needed. All that is left is to create views that actually display the data.

I'm a fan of the opensource project Spark, and will use it instead of the conventional ASP.NET view engine. I find the syntax from Spark a lot easier to read and less "tag-soup" than the ASPX views.

Because I have two different views (All submissions, and comments for a submission), I'll implement a master layout that they both share. Spark makes this easy:

    1 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
    2 <html xmlns="http://www.w3.org/1999/xhtml">
    3     <head>
    4         <title>REDDIT CLONE!</title>        
    5     </head>
    6     <body>
    7         <h1><a href="/">REDDIT CLONE!</a></h1>
    8         <use:view />
    9     </body>
   10 </html>

Next I'll define my "Index" view:

    1 <viewdata model="List<RedditClone.Models.Submission>" />
    2 <use master="Application" />
    3 
    4 <SubmitLink />
    5 
    6 <ol>
    7     <li each='var sub in ViewData.Model'>                  
    8         <Submission submission='sub' />        
    9         <br /><br />
   10     </li>               
   11 </ol>

You'll notice the two tags "SubmitLink" and "Submission". These are spark partial views that I used so that I can reuse those templates in other places.

SubmitLink:

    1 <h2>Submit a link:</h2>
    2 <% using (Html.BeginForm("Submit","Submissions")) { %>       
    3     Title: 
    4     !{Html.TextBox("Title")} <br />
    5     Link: 
    6     !{Html.TextBox("Link")}<br />    
    7     <button type="submit" value="Submit">Submit Link</button>
    8 <% } %>

Submission:

    1 (${submission.Score}) -
    2 <a href="${submission.Link}">
    3     ${submission.Title}
    4 </a>
    5 <br />
    6 
    7 !{Html.ActionLink(submission.Comments.Count + " Comments", "View", 
    8         new { submission.ID} )} -
    9 !{Html.ActionLink("Vote Up", "VoteUp", 
   10         new { submission.ID} )} -
   11 !{Html.ActionLink("Vote Down", "VoteDown", 
   12         new { submission.ID} )} 

I now have a fully functional reddit page, however I don't have a way to view / post comments. I need to define the comments view:

    1 <viewdata model="RedditClone.Models.Submission" />
    2 <use master="Application" />
    3 
    4 <Submission submission='ViewData.Model' />
    5 
    6 <hr /> 
    7 <h3>Submit Comment:</h3>
    8 <%  using (Html.BeginForm("Create", "Comments")) {%>                
    9     !{Html.Hidden("SubmissionID", ViewData.Model.ID)}
   10     !{Html.TextArea("Text")}
   11     <button type="submit" value="Submit">Post Comment</button>
   12 <% } %>    
   13 
   14 <%            
   15     var rootComments = 
   16             ViewData.Model
   17                 .Comments
   18                 .Where(c => !c.ParentCommentID.HasValue)
   19                 .OrderByDescending(c => c.Score)
   20                 .ToList();
   21     Html.RenderPartial("_renderComment", new { comments = rootComments }); 
   22 %>

This view is straight forward, at the top it uses the Submission partial view to show the submission information and links. Below that it has a form to submit a new comment. Finally, it grabs the root comments for the link and passes them to the _renderComment partial.

You may wonder why I am using Html.RenderPartial instead of the spark tag format. The reasoning will be easier to explain when you see the partial:

    1 <viewdata comments="List<RedditClone.Models.Comment>" />
    2 
    3 <ul>     
    4      <li each= 'var comment in comments.OrderByDescending(c =]] c.Score)'>
    5         ( ${comment.Score} ) - ${comment.Text}
    6         <br />                 
    7         !{Html.ActionLink("Vote Up", "VoteUp", "Comments", new {comment.ID, comment.SubmissionID}, null )} -
    8         !{Html.ActionLink("Vote Down", "VoteDown", "Comments", new {comment.ID, comment.SubmissionID}, null )}
    9 
   10         <% using (Html.BeginForm("Create", "Comments")) {%>
   11             !{Html.Hidden("ParentCommentID", comment.ID)}
   12             !{Html.Hidden("SubmissionID", comment.SubmissionID)}
   13             !{Html.TextArea("Text")}
   14             <button type="submit" value="Submit">Reply</button>
   15         <% } %>
   16 
   17         <test if='comment.ChildComments.Any()'>
   18             <% Html.RenderPartial("_renderComment", new { comments = comment.ChildComments.ToList() }); %>
   19         </test>            
   20      </li>     
   21 </ul>

The partial renders an entire comment tree by calling itself recursively. Unfortunantly Spark cannot handle recursive partials, so I get around that limitation by invoking it using the default ASP.NET MVC call Html.RenderPartial. Hopefully Spark gains this ability int the future as it will significantly improve readability.

I've now finished my entire reddit clone. It is able to do all of the features I wanted, and weighs in at around 120 lines of C#, and 100 lines of Spark templates. While this is heavier than some of the others, it does have commenting and a real database. Had I not done this, the entire app (templates and all) would have easily fit into 100 lines.

I think this helps dismiss the notion that ASP.NET apps are heavy and require enormous amounts of boilerplate code. It also shows that you have full control of the HTML outputted, which was a common complaint on those who use webforms.

The RedditClone is running here: http://redditClone.gfilter.net.

The complete source code is on my Codeplex Repository: http://redditClone.codeplex.com.

Remember, this was just for fun, and is missing a lot of features that would make it robust (Input validation for one!). However, it is a nice proof of concept on how fast you can make an application using ASP.NET MVC.

Working around Androids quirks

12/6/2009

The more I develop for the Android OS, the more strange quirks I find in the platform.

Overall, it is not a bad platform at all, but there are a handful of decisions made by Google that I just do not understand.

In this case, I found myself fighting the activity life cycle (Directly linked from Google, I think they can afford the bandwidth):

The first thing to note about this diagram is that onPause is completely misleading, and will not fire when your application loses focus. It only fires if a activity partially obscures your app.

However my issue was that my game is made up of several activities, and the user needed to be able to move through them seamlessly without feeling like they are separate applications. This means that the background music should continue to play, and not stop and restart when changing activities.

To solve this, the music was separated out of the activities, and into a global application class.

However, that caused a bigger problem. When the application loses focus (Perhaps the user hit "Home", perhaps a phone call came in...etc), the music did not stop.

"Easy!" I thought, "I'll just hook into the onStop handler of the activities and pause the music".

That "easy" job ended up taking me over 24 hours to figure out.

The problem is that onStop is not only called when your application loses focus, it is also called when you change activities from within your application, and there is no obvious way to tell the difference between an external application changing focus and an internal application changing focus.

However, after trying in vain for a few days to figure out how to tell the difference, I found out about the DroidFu library written by Matthias Käppler.

This library included a helper that would tell you if the activity was currently the top activity, but more importantly, it demonstrated how to access the running tasks using the ActivityManager (Something that I could not find in the Android docs).

While I didn't particularly like how he implemented the method (as a static helper method that did a lot of unneeded things), I wanted to credit him with showing me the proper API call that I needed.

I decided the proper approach was to add a new event into the Activity Life-cycle. This event is "onLostFocus", and will fire before onStop is invoked. This is implemented by creating a custom super class for all activities:

 
import android.app.Activity;
import android.app.ActivityManager;
import android.content.Context;
import android.os.Bundle;
 
/**
 * Base class for all Activities
 */
public class ActivityBase extends Activity {
 
    /**
     * Called when the activity is stopped 
     * and a new package is running
     */
    public void onLostFocus()
    {        
    }
 
    @Override
    public void onStop()
    {
        if (wasFocusLost())
        {
            this.onLostFocus();
        }
        super.onStop();
    }
 
    /**
     * Checks if the activity is below another application     
     */
    public boolean wasFocusLost()
    {              
         ActivityManager am  = (ActivityManager)
              this.getSystemService(Context.ACTIVITY_SERVICE);
 
        return !(am.getRunningTasks(1)
                     .get(0)
                     .topActivity
                     .getPackageName()
                     .equals(this.getPackageName()));
    }      
}

Using this super class, all of my activities can hook into onLostFocus() just like they hook onCreate() or onStop().

Why Google did not include this out of the box is a complete mystery to me, as it is an important part of the activity life-cycle.

Hopefully this super class will save someone the pain and frustration of trying to solve this problem in the future.

First impressions are important

11/28/2009

I've been working on what is going to be my first entry into the Android market.

It is a simple game, I hope to ship it with less than 80 total hours spent in development.

Yesterday, I thought I'd post a screenshot to a android dedicated website and gather responses:

Out of 13 comments, the majority of the responders complained about the (placeholder) block art.

It was an interesting lesson learned, because while I had initially intended to redo the block sprites with higher quality images, after spending the last 40 hours staring at those, they were starting to grow on me.

Getting feedback brought me back down to earth, and reminded me that if I want this game to be a success, it needs to look as nice as it plays.

Understanding the performance difference between dynamic and weak typing

9/28/2009

It seems like every few months someone throws out an argument such as “Dynamic typed languages can never be as fast as statically typed languages”. This particular statement came from an otherwise well written rant from one of the core Mono developers.

even if you implement the same VM technology in something like Parrot as you have in Mono, dynamic typing will always be slower;

While in practice, I do not disagree with this assertion, I think that it is a dangerous if left unqualified and I thought I’d clear up some “Static v Dynamic” misconceptions. The main issue preventing the creation of a fast runtime is not dynamic typing, but is largely an issue of weak versus strong typing.

Dynamic languages can have strong typing. Common Lisp and Python are fairly good examples of this. However dynamic languages are more frequently considered to have weak typing (ala, JavaScript or PHP).

In a strong typed language, the type may still be dynamic, but the runtime enforces type rules. This means that the following python code will throw a error:

  >>> 1 + "1"
  TypeError: unsupported operand type(s) for +: 'int' and 'str'

In a weak typed language, instead of enforcing type rules, the runtime will instead attempt to find the common denominator between the two types. For example, the same code ran under JavaScript:

>>> 1 + "1"
 "11"

It is in this weak typing that the major performance issues come from. To explain why, I’ll need to explain how language virtual machines(VM’s) work.

Every virtual machine worth its salt performance-wise uses a technique called Just In Time compilation (JIT).

In very simple terms, when a VM executes a block of code, it first passes this code to a JIT compiler. The job of the JIT compiler is to convert the code into native platform specific code. A pointer to the new compiled code is then stored so that the VM knows to invoke the native code directly the next time the function is called.

A JITing VM is drastically faster than an interpreted VM for any operation that is repeated multiple times. However, weak typing throws a huge monkey wrench into this trick.

Take this C# example:

int addInts(int a, int b) {
    return a + b;
}

The JIT will see that both “a” and “b” are integer types and will convert that function into a single x86 opcode for adding two integers. However, the equivalent JavaScript is not so lucky:

function add(var a, var b) {
     return a + b;
}

In this situation, the types of “a” and “b” are not known. A naive runtime has no idea what add operator to invoke, so what ends up generated is something along the lines of this pseudo-code:

if(a is a string and b is a string)
       return addStrings(a,b);
else if (a is a integer and b is a integer)
       return addInts(a,b);
else if (a is a string and b is a integer OR b is a string and a is integer) {
       a = ConvertToString(a)
       b = ConvertToString(b)
       return addStrings(a,b)
}
// Etc handing floating points etc.

The outputted code for this function ends up being 98% type checks, and 1% actual code. Hundreds of CPU operations compared to the single CPU operation that the C# code needed.

Because of this overhead, even with a JIT compiler, the outputted code is so bloated with type checks you will never see the performance of a strong typed language, and this is where the basis of the initial quote I mentioned above came from.

However, it is still possibly to get reasonable performance out of a weak typed language, thanks to exceedingly clever VM designs.

In a weak typed language it is impossible to know the types of variables ahead of time...However it is possible to observe the types of variables AFTER the first execution of a function.

Any computer program, even fancy object oriented or functional ones, break down into a sequence of instructions that the VM/CPU execute. Regardless of the number of branches in the code, the execution will only go down one path at a time.

A tracing VM observes these execution paths and builds what is known as “traces”, essentially records of the discovered variable types and execution flow during each run.

It is easiest to understand using code:

var a = 5;
for (var b = 0; b < 100; b++) {
    a += b;
}

In this specific JavaScript snippet, the produced JIT code within the loop will initially be the verbose type checking example I stated above, however at the same time the tracing engine will note that it has discovered that both “a” and “b” are integer types.

On the second iteration of the loop the VM now has two options:

Continue to run the loop contents, complete with the very verbose type checking code.
Re-compile the inner contents of the loop into a single ADD instruction, much like a static compiler would.

The VM will look at how “hot” the code is, to decide what action to take. If the code is going to be repeated over a certain threshold, the cost of recompilation is estimated to be less than the total execution cost, and the VM will recompile the code using the discovered knowledge of the types.

However, if the code is not very hot, i.e. the loop is only a few iterations; then the VM may choose to not waste time recompiling and continue executing the inefficient code. This tuning is a fine balancing act.

Tracing cannot solve all problems, real code is rarely as simple as that loop, and much like function inlining, there are some things that have too many external inputs for the VM to reliably trace.

For example, what if a third variable was introduced in the loop and this variable received its state from a function call? If the VM cannot trace the function to ensure that it will always consistently return the same type, then it cannot perform any optimization on the loop.

However, even with these limitations, tracing VM’s are incredibly fast compared to standard naïve runtimes. Googles V8 JavaScript engine and Mozillas TraceMonkey are finally fast enough for JavaScript to be considered a worthy application language even though they are still extremely slow compared to a strong typed language.

To summarize, a weak typed language, not a dynamic language, will always be slower than a strong typed language, and this unfortunately won’t change until we invent quantum computers :-)

Why free software shouldn't depend on Cocoa or GNUStep

6/28/2009

The Free Software Foundations decision to develop and push GnuStep leads the community in a risky direction. It is dangerous to depend on Cocoa, so we need to discourage its use.

The problem is not unique to GnuStep; any free implementation of NextStep would raise the same issue. The danger is that Apple is probably planning to force all free NextStep implementations underground some day using software patents. This is a serious danger, and only fools would ignore it until the day it actually happens. We need to take precautions now to protect ourselves from this future danger.

This is not to say that implementing NextStep is a bad thing. Free NextStep implementations permit users to run their Cocoa programs on free platforms, which is good. Ideally we want to provide free implementations for all languages that programmers have used.

The problem is not in the NextStep implementations, but rather in applications written in Cocoa. If we lose the use of GNUStep, we will lose them too. That doesn't make them unethical, but it means that writing them and using them is taking a gratuitous risk.

We should systematically arrange to depend on the free NextStep implementations as little as possible. In other words, we should discourage people from writing programs that use Cocoa. Therefore, we should not include NextStep implementations in the default installation of GNU/Linux distributions, and we should distribute and recommend non-Cocoa applications rather than comparable cocoa applications whenever possible

For those who don't get the joke, this is a response to this hypocritical garbage.

Trees in relational databases

6/6/2009

I recently posted an explanation on how to use a CTE to traverse a tree in a relational database on Reddit.

This started a lot of conversation, and people were wondering on the performance aspect of using recursive queries.

Well, I decided to take an extreme look. I'll pretend that Reddit had a very popular thread that had 5,000 comments on it. And, to really tax this query, I'll make it so that it is an extremely unbalanced tree 1000 comments deep.

For those following along, I'm going to use T-SQL, and everything in this post can be dropped directly into SQLServer 2005+ and ran.

I'm going to build a tree that looks like this:

  Root
      \
      Comment 1
      Comment 2
      Comment 3
      Comment 4
      Comment 5
               \
               Comment 6
               Comment 7
               Comment 8
               Comment 9
               Comment 10
                         \
                         Comment 11
                         Comment 12
                         Comment 13
                         Comment 14
                         Comment 15 (This repeats to depth of 1,000)

This is the absolute worst case scenario, because I have a horrible unbalanced tree, and will have to recursively walk the entire thing.

So, to create the schema:

  USE [Tree]
  CREATE TABLE [dbo].[Tree]
  (
    [Id] [int] IDENTITY(1,1) NOT NULL,
        [ParentID] [int] NULL,
        CONSTRAINT [PK_Tree] PRIMARY KEY CLUSTERED 
        (
           [Id] ASC
        )WITH (PAD_INDEX  = OFF, IGNORE_DUP_KEY = OFF) ON [PRIMARY]
  ) ON [PRIMARY]

  GO
  ALTER TABLE [dbo].[Tree]  WITH CHECK ADD  CONSTRAINT [FK_Tree_Tree] FOREIGN KEY([ParentID])
         REFERENCES [dbo].[Tree] ([Id])
  GO
  ALTER TABLE [dbo].[Tree] CHECK CONSTRAINT [FK_Tree_Tree]

This is a simple table with the following attributes:

Two Columns (Id, ParentId)
Id is an identity column and the primary key
ParentId is Nullable, and is a Foreign Key back to Id

Now, I'm going to insert my comment thread of doom into this table, I'm using some pretty tacky T-SQL to do this, but I'm lazy:

USE Tree;

DECLARE @depthCounter int;
DECLARE @lastNode int;
DECLARE @i int;

SET @DepthCounter = 0;

INSERT INTO Tree (ParentID)
	VALUES
	(
		NULL
	);

SET @LastNode = SCOPE_IDENTITY();

WHILE @DepthCounter < 1000
BEGIN	
	SET @i = 0;
	WHILE @i < 5
		BEGIN
			INSERT INTO Tree (ParentID)
				VALUES
				(
					@LastNode
				);	
			SET @i = @i + 1;
		END
	SET @LastNode = SCOPE_IDENTITY();	
	SET @DepthCounter = @DepthCounter + 1;
END

With my tree of doom inserted, I'll run the query starting from the root:

Use Tree;

WITH CommentTree (ParentID, ID, Level)
 AS
 (
	SELECT ParentID, ID, 0 as [Level]
		FROM Tree
		WHERE ParentID IS NULL
	UNION ALL
		SELECT c.ParentID, c.ID, ct.Level + 1
			FROM Tree c
			JOIN CommentTree ct ON ct.ID = c.ParentID
  )

  SELECT ParentID, ID, Level FROM CommentTree 
	 ORDER BY ParentID, Level, ID OPTION (MAXRECURSION 5000)

On my home laptop (A modest 2ghz) running Microsoft SQLServer 2005 EXPRESS I get the following result to generate this dataset:

  SQL Server Execution Times:
  CPU time = 10998 ms,  elapsed time = 11209 ms.

11 Seconds is an eternity, but this is a the worlds worst case. Now to try the same test data, but instead walking only 100 levels deep (500 comments):

Use Tree;

WITH CommentTree (ParentID, ID, Level)
 AS
 (
	SELECT ParentID, ID, 0 as [Level]
		FROM Tree
		WHERE ParentID = 44339
	UNION ALL
		SELECT c.ParentID, c.ID, ct.Level + 1
			FROM Tree c
			JOIN CommentTree ct ON ct.ID = c.ParentID
  )

  SELECT ParentID, ID, Level FROM CommentTree 
	 ORDER BY ParentID, Level, ID OPTION (MAXRECURSION 5000)

This returned 500 rows and the following speed stats:

  SQL Server Execution Times:
    CPU time = 1217 ms,  elapsed time = 1223 ms.

This is still a unusual dataset, because it is still a very off balance tree.

So, my final benchmark will be an approximation of a regular reddit thread. 300 comments total, 100 root comments, with 2 replies each.

The setup script:

USE Tree;

DECLARE @depthCounter int;
DECLARE @lastNode int;
DECLARE @firstNode int;

SET @DepthCounter = 0;

INSERT INTO Tree (ParentID)
	VALUES
	(
		NULL
	);

SET @FirstNode = SCOPE_IDENTITY();

WHILE @DepthCounter < 100
BEGIN	
	INSERT INTO Tree (ParentID)
		VALUES
		(
			@FirstNode
		);
	SET @LastNode = SCOPE_IDENTITY();

	INSERT INTO Tree (ParentID)
		VALUES
		(
			@LastNode
		);
	INSERT INTO Tree (ParentID)
		VALUES
		(
			@LastNode
		);
	SET @DepthCounter = @DepthCounter + 1;
END

Now, when ran:

(301 row(s) affected)
SQL Server Execution Times:
    CPU time = 47 ms,  elapsed time = 51 ms.

That's right, in the real world, this query took 51ms.

I think this shows that CTE (even running on commodity hardware) are perfectly adequate to handle nested trees on social forums.

Code Snippet of the Day: Recursive File Finder

5/28/2009

.NET doesn't have a recursive file finding class, so I've used this helper a couple of times:

    /// <summary>
    /// Finds a list of files that match the specified critiria
    /// </summary>
    public class FileFinder
    {
        private Dictionary<FileInfo,int> files;
 
        /// <summary>
        /// Returns all files that match MatchType that are children of StartDir
        /// </summary>
        /// <param name="startDir"></param>
        /// <param name="matchType"></param>
        /// <returns></returns>
        public List<FileInfo> Find(string startDir, string matchType)
        {
            files = new Dictionary<FileInfo, int>();
            findFiles(startDir, matchType);
 
            return files.Keys.ToList();
        }
 
        private void findFiles(string startDir, string matchType)
        {
            DirectoryInfo d = new DirectoryInfo(startDir);
            foreach (FileInfo f in d.GetFiles(matchType))
            {
                if (!files.ContainsKey(f))
                {
                    files.Add(f, 0);
                }
            }
 
            if (d.GetDirectories().Length > 0)
            {
                foreach (DirectoryInfo subD in d.GetDirectories())
                {
                    findFiles(subD.FullName, matchType);
                }
            }
        }
 
    }

To use:

FileFinder fFinder = new FileFinder();
List<FileInfo> files = fFinder.Find("C:\\","*.dll");

In retrospect, maybe we were wrong

5/8/2009

In a formal paper from Sun, oh so many years ago:

The newest version of the Microsoft Visual J++ development environment supports a language construct called delegates or bound method references. This construct, and the new keywords delegate and multicast introduced to support it, are not a part of the JavaTM programming language, which is specified by the Java Language Specification and amended by the Inner Classes Specification included in the documentation for the JDKTM 1.1 software.
It is unlikely that the Java programming language will ever include this construct. ... We believe bound method references are unnecessary because another design alternative, inner classes, provides equal or superior functionality

Fast forward a decade, and Java is still in the stone-ages, meanwhile Microsoft gave up on J++/JVM due to Suns refusal to innovate and gave us a language with not only "Bound Method References", but anonymous closures and lambda expressions.

Looking back, I wonder where Java would be now if their pissing match with MS didn't exist, and they had actually adopted delegates.

Hindsight is 20/20.

First Look: Windows 7

5/6/2009

I upgraded Windows Vista to the Windows 7 Release Candidate tonight. I had previously stuck an early beta of 7 on a separate partition, but 7 seemed stable enough to become my main OS for a while.

Before the install, my primary drive had 15 gigs of free space.

After the install it had 19 gigs of free space.

I cannot remember a single upgrade that was EVER smaller than its predecessor, especially considering this was an upgrade, and not a clean reinstall.

Aside from that, everything feels incredibly snappy...the GDI lock contention overhaul appears to have made an enormous change to how responsive the OS feels in general and I'm digging the UI changes.

Kudos Microsoft. You have a winner here.

Making Document.Write Actually Work

4/29/2009

I think most people know better than to use document.write to emit content into a web page, but if not here are a couple reasons:

document.write does not work in XHTML.
If document.write is executed after the page has finished loading, it will overwrite the entire page.
document.write executes when it is encountered, it cannot inject a given node point.

The bottom line is that document.write sucks, and nobody should use it. However, almost every 3rd party analytic's vendor uses document.write in their tracking scripts.

These are usually iframes or script references you embed in your code, and then the vendors inject whatever they feel they need into your page.

I recently made some modifications to a page to dynamically load these 3rd party references after the rest of the page finished. However, doing that caused the embedded document.write calls to overwrite the entire page.

Luckily, with JavaScript, it is easy to replace browser implemented functions with your own function, and document.write is no exception:

document.write = function(args) {
    var dwNode = document.createElement('DIV'); 
    dwNode.innerHTML = args;
 
    document.body.appendChild(dwNode);
 
    // Script nodes in innerHTML won't run unless we eval them
    var runScripts = function(e) {
        if (e.nodeType != 1)
            return;
 
        if (e.tagName.toLowerCase() == 'script') {
            eval(e.text);
        }
        else {
            var n = e.firstChild;
            while (n) {
                if (n.nodeType == 1)
                    runScripts(n);
 
                n = n.nextSibling;
            }
        }
    };
 
    runScripts(dwNode);
};

This replacement document.write is called whenever the vendor code calls document.write. This code actually creates and injects a DIV into the DOM, and then eval's any embedded script tags. The script eval code was found on the internet after I discovered that script tags within innerHTML do not work.

This works well (Tested in IE6 and FireFox), and I was able to defer the loading of the vendor code until after the page has finished rendering, causing a nice improvement in the end-user experience.

Just make sure you place this in an include file, or in your page before any code that will make calls to document.write run.