T4 Templates are really nice given that the tools are included free in Visual Studio (the Express Editions of Visual Studio appear to have some under-documented limitations when it comes to T4 support).

The primary way that T4 templates are processed is from within Visual Studio.  That works pretty well.

What if you want to process a T4 template outside of Visual Studio (as part of an automated build process, for example)?

The easiest option for running a T4 template outside of Visual Studio is through the TextTransform.exe (custom T4 host) command line tool.

The main difficulty I've encountered using the TextTransform.exe command line tool is error handling.

Here's a sample error (written to StandardError):
--
MyT4Template.tt(0,0) : error : Running transformation: System.InvalidCastException: Unable to cast object of type 'Microsoft.VisualStudio.TextTemplating.CommandLine.CommandLineHost' to type 'System.IServiceProvider'.
   at Microsoft.VisualStudio.TextTemplatingc7fe0d0277b54f7c95c25936373918ff.GeneratedTextTransformation.TransformText()
--

Notice the (0,0).  That is where the line and column numbers would normally go for where the error occurred.  Also notice the stack trace contains no useful information (the error message, at least, in this case *is* useful).

The other downsides of using TextTransform.exe are similar to situations where you are calling out to an EXE file from code as opposed to working against an API.

The main positive I've encountered using the TextTransform.exe command line tool is if the template processes okay in Visual Studio, it will likely process okay using TextTransform.exe.  The above error was generated by using a template that tried to reference EnvDTE.DTE which works fine from within Visual Studio, but doesn’t necessarily make sense to do outside of Visual Studio.

As an alternative, you might attempt to use the Visual Studio T4 engine from .NET code like so:
--
ITextTemplatingEngineHost host = new Microsoft.VisualStudio.TextTemplating.VSHost.TextTemplatingService();
ITextTemplatingEngine engine = new Microsoft.VisualStudio.TextTemplating.Engine();
string outputCode = engine.ProcessTemplate(inputContent, host);
--

But that won't work.  You will get these compile time errors:
--
The type 'Microsoft.VisualStudio.TextTemplating.VSHost.TextTemplatingService' has no constructors defined
'Microsoft.VisualStudio.TextTemplating.VSHost.TextTemplatingService' is inaccessible due to its protection level
--

The hardest option for running a T4 template outside of Visual Studio is by implementing the ITextTemplatingEngineHost interface.  There is an example of how to do that here:

Walkthrough: Creating a Custom Text Template Host

The problem with that example is that it fails for multiple distinct reasons for templates that run just fine in Visual Studio and from TextTransform.exe.  It doesn’t appear to be even close to a full featured ITextTemplatingEngineHost implementation (it’s quite a "teaser" of a sample, enough to show it has potential, but not enough to show you how far from reality you are).

Mono provides an alternative implementation of the ITextTemplatingEngineHost interface here:

http://anonsvn.mono-project.com/viewvc/trunk/monodevelop/main/src/addins/TextTemplating/Mono.TextTemplating/Mono.TextTemplating/TemplateGenerator.cs

And while that fails less often on templates that run just fine in Visual Studio, it still fails in multiple ways.  I appreciate what the Mono folks have done, but for this use case (trying to run T4 templates outside of Visual Studio), there isn’t much value add.

The advantage of using a custom ITextTemplatingEngineHost host implementation is you have significantly more power and control than you have with TextTransform.exe.  TextTransform.exe has a limited input/output/error mechanism and its internals are not very extensible/customizable.

The disadvantage of using a custom ITextTemplatingEngineHost host implementation (as opposed to TextTransform.exe) is that you possibly have to write a decent amount of code against an undocumented system before your T4 template that processes fine in Visual Studio will process equivalently from .NET code (this will depend greatly on the contents of your T4 template).

If anyone is aware of other (free or commercial) custom implementations of the ITextTemplatingEngineHost interface that are usable from .NET code, please let me know!

Update March 7, 2010: Apparently this will be much easier in Visual Studio 2010:

Generating Text Files at Run Time by Using Preprocessed Text Templates

After a number of small releases focused on application infrastructure, this release is a semi-major user focused feature release.

The biggest new feature is treating function parameters as obfuscatable variables.

A user requested that function/variable names not grow in size as part of obfuscation, so function/variable names now stay the same size.  This may continue to be refined over time depending on what people think.

Since this obfuscator was originally written for my own use, it made some assumptions about coding style, especially in regards to whitespace.  A number of those types of assumptions have now been removed based on testing against code submitted by users.

While some substring type issues still remain, it is now possible to have local variables which are substrings of one another like "counter" and "counter1" without the obfuscator missing a beat.

It used to be possible to have a variable named "funct" (which is a substring of the lua reserved word "function"), but this is no longer allowed.  More work is needed in this area (not mangling lua reserved words) going forward.

The latest version of the Capprime Lua Obfuscator is available here:

http://www.capprime.com/CapprimeLuaObfuscator/CapprimeLuaObfuscator.aspx

This release increases the number of characters that can be obfuscated at one time from 1000 to 3000.

The daily character obfuscation limit has also been increased from 3000 to 6000.

These limits may change again in the future depending on usage and feedback.

The Capprime Lua Obfuscator is available here:

http://www.capprime.com/CapprimeLuaObfuscator/CapprimeLuaObfuscator.aspx

This release includes fixes for two uncommon server errors:

1) Error occurred when Lua code contained apparent HTML tags.
2) Error occurred when Lua variable name was missing.

These two scenarios are now handled properly without errors being generated.

Since the Capprime Lua Obfuscator was first publically released two months ago, the web application has been visited by approximately 70 unique (non-bot) IP addresses.

Feedback is welcome and encouraged!

There are a number of very significant drawbacks in the O/R Designer that have no reasonable work around.

• The Visual Studio 2008 LinqToSql O/R Designer is fine for a database with ~5 tables, no views, no stored procedures, and no functions, but it doesn't scale well much beyond that.
  • If the database changes, it's very difficult to manually update the O/R Designer properly.  As database complexity increases, this becomes near impossible.
  • When you need to remove an existing object from the O/R Designer to force it to update, it can be difficult to find that object.
• It is overly difficult to diff the generated files from one version to another since the files are rewritten in a different order than they were read in when changes are made.  Even if the files can be sorted in a reasonable way, diffing the files is still not an effective solution for managing changes.
• There are bugs with how the O/R Designer generates code.  In general, it is difficult to evaluate if the O/R Designer is working properly since it is difficult to diff the generated files before and after generation.  There are cases where the O/R Designer output is different than the SQL Metal output and in some of those cases, it could be a bug in either or both of the tools.
• If two or more developers are working under an edit/merge/commit style of source control, it is difficult to resolve conflicts in the LinqToSql O/R Designer files during the merge phase.
• It is not effective to try to diff O/R Designer generated files against files generated by SQL Metal as they don't use the same underlying code generation techniques and file sorting.  The two tools also have meaningfully different feature sets which negatively impacts the ability to diff the underlying files.

Once you start down the path of the O/R Designer, it gets increasingly harder to migrate away from it the farther you go.  This is a significant hidden risk as many of the scalability issues with the tool don’t present themselves right away.

It's not clear if any of these problems will be addressed in Visual Studio 2010.

This is not to say that the O/R Designer doesn't have very interesting and useful features.  It does have interesting and useful features.  They just aren't packaged in a way to make them at all usable for anything but very tiny databases.

Are there alternative LinqToSql code generation tools?

The main alternative tool is SQL Metal, which has quite a few drawbacks of its own.  While SQL Metal can scale up to much larger databases than the O/R Designer, it still can't scale effectively past a certain point.

Another free alternative is Damien Guard's LinqToSQL T4 Template.  Since the T4 template is not an official product of Microsoft and was built by a developer in their spare time, you'll have to judge for yourself whether it meets your criteria for "production worthy".

I may write a future blog entry with additional details on SQL Metal and the T4 templates.

There are non-free alternatives to the O/R Designer as well, but in many cases it is just not possible to get software tool purchases approved as part of the software development project budget.  It would be surprising if LinqToSql adoption hinged on the success of commercial third party tools.

Updated 2009/09/16 - Related blog posts:

The drawbacks of adopting Linq To Sql
.NET and ORM - Decisions, decisions

More and more software products are including a feature that allows using the Lua programming language to author addons that add additional features to the base software product.

One of the most notable of those products is World of Warcraft.

World of Warcraft players can write Lua addons to modify and enhance the user interface in many useful ways.

As a player of World of Warcraft for three years, I've enjoyed using many of these addons and even started to create my own.

The addon I developed seemed useful enough that I thought other players may be willing to purchase the addon for real money, even though the large majority of World of Warcraft addons at the time were free.

Long story short, the company that created World of Warcraft turned on their addon developers and effectively outlawed addon developers from making a living from addon development.

In the process of trying to commercialize my lua addon, I wrote a code obfuscator for lua.  That obfuscator doesn't have much use to me at the moment, so I put a web front end on it and I'm publishing it to see if other people can get any meaningful use out of it.

This version (v1.0.0.4) is free to use with limitations on how many text characters can be processed per day and per IP Address.

Click here to: Obfuscate Lua

There is a feedback page as part of the lua obfuscator application that allows you to submit feedback of any kind.  Please let me know what you think!

Scott Hanselman is great at evaluating all the little tools developers should be using.  Yesterday, he brought me to PureText.

While not the bane of my existence, the "Paste Special" feature is something I use *constantly*.  Unlike hotkey maven Scott, I was doing it with mouse clicks, which is even more painful.  Windows-V to "paste special" with one click is *exactly* the most important feature I've wanted added to all Microsoft software.  Thank you Steve Miller!!!