I have been studying the new PLINQ and Parallel Task Library in .Net 4 looking for various ways to do things that we can’t do in .Net 2.0.  PLINQ is huge, and there are a lot of new ways to do multi threaded programming using .Net 4.  In this article I want to cover a particular problem I have had many times over the years.  How do you speed up multithreaded apps that are bound by blocking functions, or long running I/O operations?

I started looking at this method to speed up some long running file I/O routines deep in the VistaDB engine.  Most of time we are blocked in reads from disk before we can continue working, but usually we have part of the blocks we need.  So we could start working, and then continue when the rest of the blocks are loaded.  Adding that logic is complex and prone to error with traditional threading code.  Fortunately PLINQ has a way to make some of these types of operations very simple. 

Reading multiple websites

For this example I am going to read the first page of 8 websites and then act on that information afterwards.  This is the type of very simple parallel operation that splits up really well.  But these types of long running reads are very similar to what happens in many applications.

Side Note on C# 4.0 In a Nutsell Book

I actually adopted this example from one given in Joseph Albahari’s book C# 4.0 In a Nutshell (he is also the author of the excellent LinqPAD).  Weighing in a 1000 pages is not exactly a Nutshell, but it is a fantastic book for developers who already know C# and just want to go through C# and CLR 4.  The concepts in the book cover older versions of .Net as well, but the juicy parts for me were all the new changes.

More...

In this article I will explain how to build an application that uses VistaDB and Microsoft’s SQL Server, without being hard bound against their providers directly.

Many companies desire to offer the choice from several database providers in a single product or API.  In many cases if you were directly bound, but the provider not installed, you would get a dll not found exception at startup. 

There are two ways to handle this scenario.  The most common method is to use ADO.Net Provider Factories and load the provider dynamically at runtime.  We have covered this in other blog posts.  This works great if you don’t need to use any DDA code, or any other provider specific functions.  But it is not really possible if you want to use provider level abilities.

The alternative is to put the provider specific code in an external dll, and load the code as a part of a plugin model.  Plugins can then be loaded individually and removed from consideration if the assembly fails to load (like when the provider is not actually present).  This allows for your code to continue to run, within needing the main executable to be bound against the provider.

More...

This topic comes up frequently when users are writing CLR Procs and Triggers.  How do I debug the code that is being hosted by the database?

There is a very easy way to do it through the System.Diagnostics.Debugger namespace to launch the just in time debugger for Visual Studio.

Prompting the user for the debugger

You can prompt the user to attach a debugger at runtime using the System.Diagnostics.Debugger. You would NEVER want to do this at runtime in a production environment! Only for use on your development machines. I usually surround the block with an #IF DEBUG to ensure it gets compiled out at release time.

Notice the screen shot above, the user is prompted for which debugger to use.  The CLRProcSample is the correct project in this case because that is what I want to debug.

Notice that it shows all the editions of Visual Studio on my machine (2005, 2008, 2010) and all open projects.  This makes it really handy for debugging CLR Procs, Triggers, just about anything that might be hosted external to a process you control.

More...

In this article I will explain how to build a dynamic LINQ to Entities compiler for any database provider that supports the Ado.net Entity Framework. Due to the wide range of technologies used this article, it will be broken up into two parts as listed below.

We are working on a dynamic linq query mechanism for the next major release of VistaDB.  Our goal is to provide a LinqPad type of environment in Data Builder for users to write LINQ queries against the database without having to first build an EF model.

Blog Article Sections

• Part I. How to use edmgen command line tool to generate an EF model.
• Part II. How to use CodeDom to dynamically compile a LINQ query. See blog post

Technologies Used

• Ado.net Entity Framework (EF) – EF is an Object Relational Mapping (ORM) technology from Microsoft that is built into the .Net framework 3.5 SP1 and higher.
• VistaDB 4 – Commercial embedded SQL database that supports EF
• edmgen Tool (.Net Framework) – Included in the .Net framework, this tool is used to generate the models used by the EF runtime.
• CodeDom.Compiler (.Net Framework) – CodeDom is also built into the .Net Framework and provides the way to dynamically compile code
• LINQ to Entities (.Net Framework) – This is the query mechanism against the EF runtime, it is how you ask questions of the EF model.

Part I. How to use the edmgen command line tool

There are several steps needed in the process of dynamically testing a LINQ to entities query, first of which being the EDMX model itself. Visual Studio has a great set of wizards built in to handle generating an ADO.NET data model.  These wizards handle creating the necessary files for the EF model, and adding the connection strings to the app.config.

These wizards are not available at runtime, and the model generation becomes slightly more complex. There is no API available to generate an EDMX but Microsoft does include a command line tool called edmgen which can be used to generate an EDMX from any database provider that supports Entity Framework. You can find the edmgen tool under the 3.5 and 4.0 .net framework folders (C:\Windows\Microsoft.NET\Framework\).

More...

Have you tried to use LINQ to query a database using Visual Studio?  It can be a frustrating experience of things that compile fail at runtime, and that edit / compile / test cycle can quickly lead to hours of lost time trying to get a single complex query to work correctly.

LINQPad is an editor for LINQ?

LINQPad is sort of like a Notepad, you can write and edit .linq files using it.  But that is where the Notepad similarity stops. You can execute your LINQ queries and see the results without having to run your application.  It is truly something you have to watch in order to believe how much more productive it can make your writing of LINQ.

How I found LINQPad

When I was first working on the VistaDB product store and account manager writing LINQ queries against my Entity Framework objects was incredibly frustrating.  Most of the documentation and samples I found was for Linq2Sql, which is similar syntax… But not the same. And worse, most of the syntax compiles fine, but blows up at runtime with cryptic error messages.

My typical dev and test cycle was around 5 minutes to compile the DAL / Site, login to the account, navigate to the correct page, and visit the yellow screen of death from asp.net.  It was not fun, so I started working in a stand alone tester app I built just for this purpose.  Write the query, compile, debug, step, step, read exception and try to decipher it.

I was reading a post on Stack Overflow about one of those cryptic errors when someone suggested to the poster they use LINQPad to test their queries first before putting them into their apps.  Wow, what a great idea!  Where was this tool?  (You can download it for free from Linqpad.net )

LINQPad to the rescue

LINQPad allows you to execute single LINQ commands against an existing EF model, or even to write dot net code in the editor and execute it like a little dynamic dot net environment.  The main application is free, but there is an auto-complete feature to the editor that you must pay in order to activate. Believe me it is worth it to pay for the license, you also support the author and show him the application is worth money.  The license is very inexpensive and well worth the price in order to get intellisense like behavior on your LINQ queries.

More...

NOTE: This article was originally published on the VistaDB blog, but has been moved to this location permanently.

Each of these items could be a blog post unto themselves, but I am going to try really hard to not be too verbose and just cover the core of the concept and why you need to do it.

Data driven applications rely on their databases

Everyone knows that any app driven by data is much more than just the app.  In most cases the app without a database doesn’t even function, or fails to function properly.  If a database is an integral part of your application, then shouldn’t you be doing all you can to ensure it stays healthy and prepare for the worst case events of corruption or dead drives?

This week we have been contacted by 5 long time users who suddenly lost or corrupted their data.  Two of these were end user data that is going to cost a huge amount of labor to reproduce.  In all of these cases the following simple procedures would have prevented the situation entirely.

More...

NOTE: This article was originally part of the VistaDB.Net blog and has been moved here.

ADO.NET is a set of libraries included in the .Net framework to facilitate the communication of applications with various data storage mechanisms.  These libraries form the basis for all third parties to provide data access services to users of .Net applications.

Visual Studio 2005 and 2008 did not change the data access model.  In fact ADO.NET 2 is the longest running Microsoft data access technology without a major revision.  I don’t know if this is going to change in .Net 4, but the stability of ADO.NET is a major reason for its adoption in VistaDB.

In this article I going to give a high level overview of the ADO.NET object model and how VistaDB supports that model.  I will then follow up with more articles discussing specifics for this model.

More...