In this article we're going to cover some basic concepts on what you can do in order to make your ASP.NET application as spider and search engine friendly as possible.

If you're developing for the Web then you should familiarize yourself with some Search Engine Optimization or SEO concepts. The idea here is to make your ASP.NET application as friendly as possible for spiders, and the specific spider we're talking about is Google.

In this article we're going to cover some basic concepts on what you can do in order to make your ASP.NET application as spider and search engine friendly as possible.

Postbacks

Your biggest gain in the search engine world is going to avoid the use of postbacks. For example, say you have content within an ASP panel. But in order to display that content you use a button and capture a click event in the code behind, then you change the property of the panel to visible=true once the button is clicked. This will not work with spiders since they don't "click buttons" so to speak. The way to write the page so the spider will work with it is to use a link, and then pass a parameter via a URL, this could be a link back to the page if you want, but then in the Page_Load event check for the parameters values to determine what panel or content to display in your page.

I can't understate the importance of eliminating postbacks when it comes to the Internet. Links are much better when dealing with spiders, avoid the postback, spiders simple can't do them.

Friendly URLs

Another thing to look into is the use of a URL rewriter in order to create spider friendly URLs. There are many examples on the Web for creating a URL rewriter. What a URL rewriter does is translate the parameters over to a directory like structure. For example, mypager.aspx?param1=1&param2=2 becomes something like: /mypage/1/2/default.aspx. This will enhance the spiders efficiency in spidering your site and potentially increase the frequency of a spider doing a deep crawl through your site. You can read evidence of this fact via the Google FAQ:

"Your pages are dynamically generated. We're able to index dynamically generated pages. However, because our web crawler could overwhelm and crash sites that serve dynamic content, we limit the number of dynamic pages we index. In addition, our crawlers may suspect that a URL with many dynamic parameters might be the same page as another URL with different parameters. For that reason, we recommend using fewer parameters if possible. Typically, URLs with 1-2 parameters are more easily crawlable than those with many parameters. Also, you can help us find your dynamic URLs by submitting them to Google Sitemaps. "

There are plenty of resources on the Web for URL rewriting:

• http://www.codeproject.com/aspnet/URLRewriter.asp
• http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnaspp/html/urlrewriting.asp
• http://www.15seconds.com/Issue/030522.htm

Titles and Meta Tags

Another issue when generating dynamic pages, and using the same page but either posting back or linking back via a URL. Be sure to change the Title of the page and Meta description. If you do not do this then Google is going to "think" that it is the same page, and the results will not be displayed as high as you would like. It will definitely affect your search results. One way of tacking this is to convert both the title, and meta description tags to HTML controls and then change the inner text, or dynamically generate the text for the tags when displaying different content.

Here's a sample block of code for dynamically changing the title tag of your Web page:

First modify the page in order to make the title tag a control you can modify:

<TITLE id="PageTitle" runat="server">

Then in your code you first declare the control as an HTMLGenericControl and set the properties:

Protected WithEvents PageTitle As System.Web.UI.HtmlControls.HtmlGenericControl

As mentioned the meta description tag is also important. For example, say you do a search on keyword contained within the title tag of your document, but not in the body of the document. Google will display the meta description of your site in the results. So if every page within your application has the same meta description value, all the pages are going to look the same, and may not appear to be relevant to the person doing the search.

You can remedy this using the method above or simply populating the value and outputting it to the form:

<META NAME="DESCRIPTION" CONTENT="<%= MetaDescription %>">

Then in the code behind just set the value of MetaDescription

Public MetaDescription As String

MetaDescription = "My meta value...."

Viewstate

Viewstate can be another thing that adversely affects the indexing of your site. For example, if you view the source of an ASP.NET application you may see something like the following:

<input type="hidden" name="__VIEWSTATE" value="dDwtMjA3MTUw...=" />

And the value of this field can continue on for a long time. I've seen cases where the viewstate is over 100k or more. The problem this has with search engines is many times a search engine will rank your page based on where a keyword occurs in the document. For example, say you're searching on ASP.NET and you first have 100k of viewstate and then your keyword appears within the HTML document. This could affect how your page ranks for that keyword since many search algorithms base relavancy on where the keyword appears or how close to the top of the document it appears.

One way, and one I recommend is to remove the viewstate entirely from the source of the HTML page. Not only will this benefit your search engine results, but it will also reduce the download time of the page since your reduce the size of the page.

The following article is shows you how to remove the viewstate, it focuses on DotNetNuke, but you can use the same functions within any ASP.NET application.
http://www.wwwcoder.com/main/parentid/224/site/3507/68/default.aspx

Closing

There are many issues that cover SEO for Websites, but in this article we're covering what you as an ASP.NET developer can address. For more information check out the many sites on SEO by searching on the topic. Good luck in your Web site promotional efforts!

Author Info

Written by Patrick Santry, ASP.NET MVP, MCSE, recognized speaker, and author of several books and magazine articles, and owner of WWWCoder.com. Patrick has over 11 years of Web application development and management. You can visit his blog at http://blogs.wwwcoder.com/psantry/.