Monday, September 07, 2009

I'm sure it's been done before and I'm sure it will be done again.  Sometimes habits work to your advantage and sometimes they don't.

Here's a case where a habit bit me recently.

static void Main(string[] args)
{
    try
    {
        object dbConnection = GetDbConnectionWithRetry(0);
    }
    catch (Exception oneException)
    {
        Console.WriteLine("oneException=[" + oneException.ToString() + "]");
    }
}

private
static object GetDbConnectionWithRetry(int recursionCount)
{
    Console.WriteLine("GetDbConnectionWithRetry recursionCount=[" + recursionCount + "]");
    object dbConnection = null;
    if (recursionCount < 10)
    {
        dbConnection = OpenDbConnection();
        if (dbConnection == null)
        {
            dbConnection = GetDbConnectionWithRetry(recursionCount++);
        }
    }
    return dbConnection;
}

private
static object OpenDbConnection()
{
    // This is just a dummy method for sample code purposes
    return null;
}

It's humorous in retrospect.  Can you spot the bug?  What habit should I break myself of?

9/7/2009 3:22:02 AM (Central Standard Time, UTC-06:00)  #    Disclaimer  |  Comments [1]  | 
 Saturday, August 22, 2009

Quite a while ago, I had a fairly bad experience at Stack Overflow and wrote a blog post about it:

StackOverflow Impressions - Too Many Rough Edges

I have now returned to being a contributor at Stack Overflow.  But why?

More and more Google searches are leading me to Stack Overflow for answers.  The answers are often useful.  Stack Overflow has no meaningful direct competition.

I'm a pragmatic person and I appreciate that the service provided by Stack Overflow is useful.

Since I'm gaining value from the site on a regular basis now, it makes sense to try to help out and "give back" to the community by providing answers when possible.

Stack Overflow is "good enough" to support.

Also, while I believe the benefits of putting a Stack Overflow reputation number on a resume are overstated (and include some small amount of risk), I do think asking and answering questions on Stack Overflow is an important indirect marketing opportunity for individuals.  As one example, if a blog post I've written answers a question some one asked, answering the question on Stack Overflow with a pointer to my blog post is helpful and increases the useful life/value of that blog post.  I think blog posts are a great way to expand the programmer knowledge base, but not all blog posts will be easily found through Google searches (Providing links to your blog from Stack Overflow will not increase your blog's page rank, but people will click through none the less).

8/22/2009 6:59:00 AM (Central Standard Time, UTC-06:00)  #    Disclaimer  |  Comments [0]  | 
 Sunday, August 16, 2009

After a number of small releases focused on application infrastructure, this release is a semi-major user focused feature release.

The biggest new feature is treating function parameters as obfuscatable variables.

A user requested that function/variable names not grow in size as part of obfuscation, so function/variable names now stay the same size.  This may continue to be refined over time depending on what people think.

Since this obfuscator was originally written for my own use, it made some assumptions about coding style, especially in regards to whitespace.  A number of those types of assumptions have now been removed based on testing against code submitted by users.

While some substring type issues still remain, it is now possible to have local variables which are substrings of one another like "counter" and "counter1" without the obfuscator missing a beat.

It used to be possible to have a variable named "funct" (which is a substring of the lua reserved word "function"), but this is no longer allowed.  More work is needed in this area (not mangling lua reserved words) going forward.

The latest version of the Capprime Lua Obfuscator is available here:

http://www.capprime.com/CapprimeLuaObfuscator/CapprimeLuaObfuscator.aspx

8/16/2009 5:30:13 AM (Central Standard Time, UTC-06:00)  #    Disclaimer  |  Comments [0]  | 
 Monday, July 27, 2009

This release increases the number of characters that can be obfuscated at one time from 1000 to 3000.

The daily character obfuscation limit has also been increased from 3000 to 6000.

These limits may change again in the future depending on usage and feedback.

The Capprime Lua Obfuscator is available here:

http://www.capprime.com/CapprimeLuaObfuscator/CapprimeLuaObfuscator.aspx

7/27/2009 3:19:07 AM (Central Standard Time, UTC-06:00)  #    Disclaimer  |  Comments [0]  | 
 Saturday, July 18, 2009

Disclaimer: I am not an ORM expert.  These tools have long learning curves and very few people could effectively provide adequate unbiased guidance.  That said, I am going to try to provide guidance based on the research I have done.  Your mileage may vary.  This information is very time sensitive, these tools change rapidly and this information will likely be significantly out of date by the end of 2009.  I have no vested interest in any of these projects.

First off, what is ORM?  ORM stands for Object Relational Mapping.

For any given project that updates data in a database:

Should you use an ORM - Yes!
Should you build your own custom ORM from scratch - No!
Should you customize the ORM solution you choose - Probably

These aren’t even really controversial opinions.  There was a time when ORMs weren’t mature enough to be “safe”, but that time has passed.  The time saved by using an ORM is a no brainer decision.

There is even a pretty straight-forward answer to what .NET ORM you should be using:

NHibernate

The major advantages NHibernate has are:

  • It's free (and open source)
  • It can handle enterprise class ORM problems
  • It has an active community surrounding it providing things it doesn't necessarily have “out of the box” like code generation and "convention over configuration"
  • The majority of developer opinions regarding .NET ORM you will find on the Internet mention NHibernate
  • It supports Linq (although not necessarily cleanly out of the box yet)

That isn’t to say that NHibernate doesn’t have disadvantages.  In particular, the API can be difficult to work with.  In reality, I believe any ORM capable of handling enterprise level ORM problems will have to expose a complex API and I believe that justifies NHibernate’s complex API to some extent.  That said, there are projects surrounding NHibernate to expose simpler interfaces into NHibernate (more on a couple of those projects included below).

In the end, unfortunately in my opinion, simplicity is not something you really want in an ORM (I’m a simplicity nut, so this particular issue is very hard for me to cope with personally).  The object relational impedance mismatch is a deceptively complex problem.  Database performance is usually a very critical component of any non-trivial application.  If you opt for a “simpler” ORM than NHibernate today, you may regret that decision tomorrow by underestimating your needs for an ORM.

That said, if you aren’t an experienced programmer, or you just can’t make it over the NHibernate learning curve, or you just don’t have enough time to work through the NHibernate ramp up process right now, there is a good alternative worth recommending.

Subsonic is a free, open source, "Microsoft subsidized" ORM that aims for simplicity of the API and definitely delivers on that front.  It can actually handle many of the common use cases that are required of an ORM, but at some point you will fall off that “feature” cliff (for example: having to deal with a poorly designed database that you cannot change).  You can be productive with Subsonic very quickly and it is very powerful.

Also, if you can’t use open source or would prefer a commercial product for whatever reason, LLBLGen Pro is worth recommending.  It’s harder to find information about LLBLGen Pro since it’s not free, but it appears to be a quality solution.  There is a fully functional 30 day demo available.

Microsoft has entered the ORM market recently with two products: LinqToSql and EntityFramework.  While Linq itself is a wonderful step forward for pretty much all .NET developers, Microsoft’s ORM products are a major disappointment.  While Microsoft has promised that EntityFramework will improve significantly with .NET 4.0, the evidence to back that up so far is not compelling.  NHibernate seems to be way ahead of EntityFramework and as I mentioned above, it’s pretty risky to gamble on a “partial ORM” solution.

Thankfully, NHibernate and Subsonic support Linq, so you don't have to settle for Microsoft’s ORM tools (although I’m sure many projects will be stuck using Microsoft’s ORM, because it’s from Microsoft - I’ve been there personally and I’m sure I will be there again).

I mentioned above that you should probably customize/wrap your ORM of choice.  Especially if you don’t choose NHibernate, I would work to isolate your ORM from the rest of your code base so you can swap in NHibernate later if necessary.  While it’s possible that there are unit test generators for ORMs out there, I haven’t seen any and that seems like an obvious feature to have (generate basic unit tests for your basic data and business level objects based on the models exposed by the ORM).  I like to have auto generated unit tests for my data and business layers as a confidence builder and as a base upon which to build more complex unit tests quickly.  There are many other good reasons to wrap your ORM in a simpler, domain specific API.

The biggest issue with integrating an ORM is to make sure you have very clear separation of concerns.  Your data layer should be the only layer with any exposure to a connection string.  Your UI/Presentation Layer should not have any exposure to your data layer.

A very common temptation with ORMs is to use the “entity” objects they generate from the database model as your “business layer”, when that is really your data layer and you need to avoid mixing the two.  Please resist that temptation.  Unfortunately, there is a truck load of sample code available on the Internet that blatantly violates the separation of concerns concept.

People new to NHibernate (and ORMs in general) should probably look into using one of the following “helper” projects along with NHibernate to help with the ramp up.

(1) Fluent NHibernate

This will help generate the XML mapping files between your objects and database so you don’t have to write a bunch of XML by hand.

(2) Castle ActiveRecord

I recommend this project, but have reservations at the same time.  The Active Record pattern can really help beginners, especially people new to ORM, get their head around how ORM works and why it’s beneficial.

It can also trap you in a box that is difficult to get out of, so I would recommend not getting too aggressive with the adoption of ActiveRecord until you have a good feeling for its limitations.

I would say start with ActiveRecord if you must, but get out of that box as soon as you can and once you have enough experience with the alternatives, you can more safely judge when ActiveRecord is the right answer.

While there are many alternatives to ActiveRecord, one of the more common alternative patterns is the Unit of Work pattern.

I share that just to provide additional information, you will probably not be able to apply either pattern as documented without bumping into their limitations eventually.


I spent quite a bit of time reading about many other .NET ORMs before posting this article, but for various reasons they weren’t included in the article.  Some of the primary reasons for non-inclusion were:

(1) Lack of Linq support or lack of stated intent to provide Linq support
(2) Lack of recommendations from real developers while doing various Google searches

These seem like two pretty easy to apply criteria to separate “good” ORM from “bad” ORM (and there are a lot of bad ORMs - and I would hope most people try to code their own simple ORM sometime to help them understand how easy it is to create a bad ORM).

I would like to thank the following four bloggers specifically:

Ayende Rahien/Oren Eini, Rob Conery, Frans Bouma, and David Hayden

for their extensive writing on .NET ORM who helped frame much of what I have recommended in this article.  Blogging about your ORM helps give confidence to those who want to use or recommend your ORM, so you will see a correlation between these bloggers and the tools I recommended.

 

7/18/2009 6:48:28 AM (Central Standard Time, UTC-06:00)  #    Disclaimer  |  Comments [2]  | 
 Saturday, July 04, 2009

This release includes fixes for two uncommon server errors:

1) Error occurred when Lua code contained apparent HTML tags.
2) Error occurred when Lua variable name was missing.

These two scenarios are now handled properly without errors being generated.


Since the Capprime Lua Obfuscator was first publically released two months ago, the web application has been visited by approximately 70 unique (non-bot) IP addresses.


Feedback is welcome and encouraged!

7/4/2009 10:43:20 AM (Central Standard Time, UTC-06:00)  #    Disclaimer  |  Comments [0]  | 
 Wednesday, June 24, 2009

First, create the application setup described in my WCF - Establishing a sample app baseline blog entry.

We will then modify the client program so that it encounters some errors while communicating with the WCF Service.

Next, add a new method to ConsoleApp1:

        private static string GenerateStringOfCertainLength(int stringLength)
        {
            string returnValue = String.Empty;
            int numTens = stringLength / 10;
            int remainder = stringLength % 10;

            for (int counter = 0; counter < numTens; counter++)
            {
                returnValue += "0123456789";
            }
            for (int counter = 0; counter < remainder; counter++)
            {
                returnValue += counter.ToString();
            }

            return returnValue;
        }

Replace the contents of the try block in ConsoleApp1 with:

                ServiceReference1.Service1Client oneService1Client = new ServiceReference1.Service1Client();
                ServiceReference1.CompositeType oneCompositeType = new ServiceReference1.CompositeType();
                oneCompositeType.StringValue = GenerateStringOfCertainLength(8193);
                oneCompositeType = oneService1Client.GetDataUsingDataContract(oneCompositeType);

Ctrl-F5 (Debug -> Start Without Debugging)

This may crash Cassini (the VS Web Server) and will result in the following exception:

--
oneException=[System.ServiceModel.FaultException: The formatter threw an exception while trying to deserialize the message: There was an error while trying to deserialize parameter http://tempuri.org/:composite. The InnerException message was 'There was an error deserializing the object of type WcfApp1.CompositeType. The maximum string content length quota (8192) has been exceeded while reading XML data. This quota may be increased by changing the MaxStringContentLength property on the XmlDictionaryReaderQuotas object used when creating the XML reader. Line 1, position 8652.'.  Please see InnerException for more details.

Server stack trace:
   at System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime operation, ProxyRpc& rpc)
   at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
   at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs)
   at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
   at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]:
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at ConsoleApp1.ServiceReference1.IService1.GetDataUsingDataContract(CompositeType composite)
   at ConsoleApp1.ServiceReference1.Service1Client.GetDataUsingDataContract(CompositeType composite) in C:\dev\Prototyping\WCF\WcfApp1\ConsoleApp1\Service References\ServiceReference1\Reference.cs:line 120
   at ConsoleApp1.Program.Main(String[] args) in C:\dev\Prototyping\WCF\WcfApp1\ConsoleApp1\Program.cs:line 17]
--


If we send 50000 instead of 8193, we get a different exception:

--
oneException=[System.ServiceModel.ProtocolException: The remote server returned an unexpected response: (400) Bad Request. ---> System.Net.WebException: The remote server returned an error: (400) Bad Request.
   at System.Net.HttpWebRequest.GetResponse()
   at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
   --- End of inner exception stack trace ---

Server stack trace:
   at System.ServiceModel.Channels.HttpChannelUtilities.ValidateRequestReplyResponse(HttpWebRequest request, HttpWebResponse response, HttpChannelFactory factory, WebException responseException)
   at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
   at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
   at System.ServiceModel.Channels.ClientReliableChannelBinder`1.RequestClientReliableChannelBinder`1.OnRequest(TRequestChannel channel, Message message, TimeSpan timeout, MaskingMode maskingMode)
   at System.ServiceModel.Channels.ClientReliableChannelBinder`1.Request(Message message, TimeSpan timeout, MaskingMode maskingMode)
   at System.ServiceModel.Channels.ClientReliableChannelBinder`1.Request(Message message, TimeSpan timeout)
   at System.ServiceModel.Security.SecuritySessionClientSettings`1.SecurityRequestSessionChannel.Request(Message message, TimeSpan timeout)
   at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)
   at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
   at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs)
   at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
   at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]:
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at ConsoleApp1.ServiceReference1.IService1.GetDataUsingDataContract(CompositeType composite)
   at ConsoleApp1.ServiceReference1.Service1Client.GetDataUsingDataContract(CompositeType composite) in C:\dev\Prototyping\WCF\WcfApp1\ConsoleApp1\Service References\ServiceReference1\Reference.cs:line 120
   at ConsoleApp1.Program.Main(String[] args) in C:\dev\Prototyping\WCF\WcfApp1\ConsoleApp1\Program.cs:line 17]
--


The out of the box wizard generated service has a couple of limitations in string lengths that it will support.

These are essentially client side errors and can't be usefully debugged on the server side or using tools like Fiddler.

Let's go back to the 8193 character string and solve that issue first.  We can do this by modifying the web.config for the WCF web service and the app.config for the console client application.

There is a tool in Visual Studio that gives us a property editor for WCF configuration.  You may or may not find it preferable to use the tool over modifying the xml files directly, but that is how I am going to describe the changes we need to make.

If you right click on the web.config in the Solution Explorer and don't see a menu option called "Edit WCF Configuration", do the following to get it added:

  In Visual Studio 2008, go to the tools menu, then select "WCF Service Configuration Editor"
    File -> Exit

Now, right click web.config in Solution Explorer -> Edit WCF Configuration

  Select Bindings -> New Binding Configuration...
    Please select a binding type: wsHttpBinding
  Select Bindings -> NewBinding0 (wsHttpBinding)
    Set Configuration -> Name to: CustomWsHttpBindingConfiguration
    Set ReaderQuotas Properties -> MaxStringContentLength to: 9000

  Select Services -> WcfApp1.Service1 -> Endpoints -> (Empty Name) [the first one - not mexHttpBinding]
    Select Endpoint Properties -> Binding Configuration: CustomWsHttpBindingConfiguration

  File -> Save
  File -> Exit

Any time we update the web.config file, we should update the service reference, even though in this case it is likely unnecessary:

  Solution Explorer
    Right Click ConsoleApp1 -> Service References -> ServiceReference1
      Update Service Reference
      OK

We need to make the same change on the client side as we made on the server side:

  Right Click app.config in Solution Explorer -> Edit WCF Configuration
    Select Bindings -> WSHttpBinding_IService1 (wsHttpBinding)
      Set ReaderQuotas Properties -> MaxStringContentLength to: 9000
    Save & Exit


If we run the application now using 8193, the exception is gone.  So, it is now clear how to send strings larger than 8192 characters to a WCF web service (the exception helped point us in the right direction).

Now, let's go back to the 50000 scenario.  The above changes are not enough to fix it as we are now exceeding a different character limit.

Right click web.config in Solution Explorer -> Edit WCF Configuration
  Select Bindings -> CustomWsHttpBindingConfiguration (wsHttpBinding)
    Set General -> MaxReceivedMessageSize to: 75000
    Set ReaderQuotas Properties -> MaxStringContentLength to: 50000
  Save & Exit

Right Click app.config in Solution Explorer -> Edit WCF Configuration
  Select Bindings -> WSHttpBinding_IService1 (wsHttpBinding)
    Set General -> MaxReceivedMessageSize to: 75000
    Set ReaderQuotas Properties -> MaxStringContentLength to: 50000
  Save & Exit

Again, just for best practice reasons, we update the service reference:

  Solution Explorer
    Right Click ConsoleApp1 -> Service References -> ServiceReference1
      Update Service Reference
      OK

How did I come up with those numbers?  Trial and error.  The value for MaxStringContentLength logically matches the length of the string we are trying to send.  MaxReceivedMessageSize seems to include extra characters, but it's not clear to me what exactly those extra characters are.

If you increase the values of these two config settings on the server, but not the client, the client starts giving useful error messages (like: The maximum message size quota for incoming messages (70000) has been exceeded.) instead of the mostly useless "(400) Bad Request." which basically means that the server rejected your request and is pretty much unwilling to tell you why.

Please note that increasing the size of these config settings makes your web service more vulnerable to denial of service attacks.

It's pretty easy to exceed the wizard generated values however if you are, for example, trying to send the contents of a text file as a string parameter to a WCF method.

So, now you know how to configure WCF to allow sending larger strings back and forth.

6/24/2009 7:49:23 AM (Central Standard Time, UTC-06:00)  #    Disclaimer  |  Comments [1]  | 
 Monday, June 15, 2009

I had restored a backup of a SQL 2000 database into SQL Server 2008 Express.  When I created a second database in the same SQL Server 2008 Express instance, and created a user for that database, that user would get this error when trying to expand the Object Explorer->Databases node:

--
Failed to retrieve data for this request. (Microsoft.SqlServer.Management.Sdk.Sfc)

For help, click: http://go.microsoft.com/fwlink?ProdName=Microsoft+SQL+Server&LinkId=20476

An exception occurred while executing a Transact-SQL statement or batch. (Microsoft.SqlServer.ConnectionInfo)

The server principal "username" is not able to access the database "dbname" under the current security context. (Microsoft SQL Server, Error: 916)

For help, click: http://go.microsoft.com/fwlink?ProdName=Microsoft+SQL+Server&ProdVer=10.00.1600&EvtSrc=MSSQLServer&EvtID=916&LinkId=20476
--

There is a knowledge base article on this issue here: KB956179

There is some additional information here: http://www.orcsweb.com/blog/post/The-server-principal-A-is-not-able-to-access-the-database-B-under-the-current-security-context-(Microsoft-SQL-Server-Error916).aspx

This seems like a relatively easy bug to hit and it took me too long to google up the answer, so I'm hoping this blog post helps other people find an answer more quickly.

6/15/2009 3:35:31 AM (Central Standard Time, UTC-06:00)  #    Disclaimer  |  Comments [0]  |