This is a summary of the email to all VistaDB 4 users.

Hard decisions have to be made

This is a very painful note for me to write. I have poured literally everything I had into VistaDB; time, energy, and money. But there are some realities that I have to face up to. If you cannot charge what it costs you to build something, then you shouldn’t be building it.

Our costs as a business have climbed quite rapidly, but income has gone down. In the past three years the following have happened: Health Insurance for employees is up 500%, corporate taxes are up 22% due to new laws in the US, unemployment insurance is 160% higher now, credit card merchant fees are double; and have much higher rates for international sales, business insurance is now totally out of our reach, server hosting is almost double, the list goes on and on. How much more are we making? Actually, I am making less today than three years ago.

Closing the Company

I cannot afford to work on VistaDB full time anymore, and I am in negotiations with a third party to acquire the product.

The office will be closing August 1, 2010.

Not the product or website yet, but read on for more details.

There are several different scenarios that may play out as a part of this, and I want to try to explain them. I had hoped some of the options listed below would have already completed their cycle by now, but they are taking much longer in negotiations than I originally expected. 

Option – VistaDB Acquired

If this third party acquires VistaDB then anything could happen. They could pull it from the market, and make it an internal tool. They could change the license to be based upon royalties. They could drastically change anything they want, it would be their product. I hope this happens, only because it would mean more resources into the product as a whole.

I am seriously considering an offer from this company to go work for them full time. While I would not be working on VistaDB full time, I may still retain some measure of influence on the product direction as an advisor.  The company seems quite sincere in their offer, but the devil is always in the details.

Option - No One Acquires VistaDB

If I can’t come to terms with anyone, then I may hold on to VistaDB, but it will be relegated to a nights and weekends type of activity. There will be no more full time work on VistaDB from me.  There will be little maintenance on the current product, as I am planning to spend my free time on a more advanced engine. Items that don’t interest me, or are too expensive to support will be dropped like a hot rock (Medium Trust for example). I still have a ton of ideas for the engine, and know I could improve performance probably 10x over what it is today, but not without massive design changes. If this is a hobby / research project then I will make those changes. I will be no longer worried about backward compatibility, or all the crazy upgrade paths.  I actually have a complete engine designed and protyped that is actually faster than SQL CE, but without a way to make money on it there is little point in developing it as a commercial product.

If I hold on to VistaDB, I would try to keep the websites and forums up as long as I can, but the server costs are not cheap. I would probably continue to sell the product, but as a company only type of sale (including source) without official support.  There would of course be community support.

Some business friends suggested I keep the product selling as is, but outsource everything and just let it sell until I recoup my costs. That is not my style. The product is complex, and I seriously doubt anyone could provide support who is not a programmer.

Option – Terminate the Product

This could happen if no one acquires the product, and I accept full time employment somewhere that forbids me from working on things in my free time. Many companies have intellectual property and non compete clauses in employment contracts these days.   This could happen, as I am actively discussing going to work for two large companies within this industry.  Neither would probably take lightly to me continuing research on a product that could eventually compete with something they sell.

Option – Open Source the Product

No, Not really an option at all. Who would work on it? Sure lots of people love to consume open source projects, but very few people contribute to them. And I have put a LOT of money into this product, I am not going to just give it away until I can at least break even. I have to put my kids through college, hopefully reclaim part of the money I have put into the company, etc.  And lets face it donation type projects never, ever make money.  Advertising on the site, etc are all pointless wastes of time.

New User Options

I am going to be changing the SKUs to a source only license in a few days. That will give everyone a chance to buy the 4.1 product and source at a reasonable price. It will ensure you can continue to run the 4.1 product as long you want / need it. No matter what happens to VistaDB.

What about activation and Visual Studio plugins?

I am going to release VistaDB 4.1 without licensing built in. This will allow all source users to continue to install and run in the event that everything is shut down.

It will not be a free upgrade unless you own the source; there will be some fee for it. User who don’t purchase the upgrade will not have access to it.

It will not have everything in the 4.1 I had hoped to release, but because there are breaking interface changes and the license system is different (gone), I have to bump the minor version number.

There will be no Sync Provider, for example. The Sync Provider has had a ton of time poured into it, but it is not production ready (not even close).   There will be no model first in Entity Framework.  Some EF extension methods will not be implemented in the current 4.x product (Skip / Take).

We do have some other new features that did get into 4.1, but that is a different post. There are also some changes in requirements, Data Builder now requires .Net 3.5 SP1 present on the developer machine.

The VistaDB 4.1 runtime is still .Net 2.0 SP1. This will be the last release for the 4.x line in all likelihood.

My recommendation

I seriously recommend that everyone purchase the source. It is a cheap insurance policy against whatever the future may hold. Existing customers will see the source in the Upgrades section of their account.  New users can purchase the 4.1 product with the source and continue using it no matter what happens.

I will probably keep the Infinite Codex site as a personal blog (again if the company I go to will let me).

It has been a fantastic ride, but one with a lot of regrets on my part.  Ah, to look back things always look so clear. 

FAQ

What about CornerstoneDB?  CornerstoneDB.com will continue as a stand alone entity.  Matthew McDonald will take that over and run it as his own.

LINQ is fantastic for the ability to write queries that express intent much more clearly than the same SQL, or structured code.  One problem that I have run into though is handling NULL database values that are part of a group by statement. 

Grouping by ProductSKU

Grouping in LINQ allows you to return sets of data from a collection for a given key value.  The group by clause is what the key ends up being in the result set.  Lets take a grouping of the Products by the SKU.

from p in Products
    group p by p.ProductSKU

Enumerable IGrouping collection

This results from the group by are enumerable groups ( IGrouping<String, Product> ) with the String being the Key for the groups (the ProductSKU field from the table).  The typical way you walk through this result is a nested for loop.

var groups = from p in Products
    group p by p.ProductSKU;
    
foreach( var groupentry in groups )
{
    Console.WriteLine( "Group: {0}", groupentry.Key );
    
    foreach( var groupitem in groupentry )
    {
        Console.WriteLine("Product: {0}", groupitem.ProductSKU);
    }
}
    

I end up with a list that looks something like this:

Group: VDB4DBA
Product: VDB4DBA
Group: VDB4DMW
Product: VDB4DMW
Group: VDB4PARTNER
Product: VDB4PARTNER

This works, but not what I really wanted.  In this case the first four characters of the SKU are the same per product family (VDB4 for all VistaDB 4 SKUs).  I would like to be able to group by only those first four characters instead of the complete ProductSKU.  You can do this with the following code.

from p in Products
    group p by p.ProductSKU.Substring(0, 4)

What if there are NULL entries?

But what happens if there is a NULL entry in the ProductSKU?  You get a ConstraintException: The property cannot be set to a null value.

More...

ClickOnce applications have many benefits including ease of deployment, optional automatic updates and framework requirement checks.The only real complicated issue with ClickOnce is how to deploy the database. I can’t imagine trying to deploy Sql Server as a part of your application! VistaDB is a perfect fit to be embedded in a ClickOnce application due to the ease of XCopy deployment. Our 100% managed engine means you don’t need any permissions on the client side, no installs or registry permissions are required.  VistaDB can also deploy one dll to either a 32 or 64 it machine, there is no need to target a specific CPU type in advance of the deployment.

This article explains how to get a simple Windows Forms application (databound using Entity Framework) up and running with VistaDB and ClickOnce deployment.  There are a few manual steps, but almost all of these will apply to any client side xcopy deployable database. They obviously do not apply to SQL Server!

Databound WinForms Project

The Windows Forms project used in this example is very simple and consists of the following:

• A VistaDB 4 Database containing one table named Employees with the columns and data shown in the grid to the right.
• An ADO.NET Entity Model (EF model) of the VistaDB database, built using the Visual Studio wizards.
• One Form (shown on the right) that contains a grid and a button that populates the GridView with a simple linq query against the EF model.

More...

I have been studying the new PLINQ and Parallel Task Library in .Net 4 looking for various ways to do things that we can’t do in .Net 2.0.  PLINQ is huge, and there are a lot of new ways to do multi threaded programming using .Net 4.  In this article I want to cover a particular problem I have had many times over the years.  How do you speed up multithreaded apps that are bound by blocking functions, or long running I/O operations?

I started looking at this method to speed up some long running file I/O routines deep in the VistaDB engine.  Most of time we are blocked in reads from disk before we can continue working, but usually we have part of the blocks we need.  So we could start working, and then continue when the rest of the blocks are loaded.  Adding that logic is complex and prone to error with traditional threading code.  Fortunately PLINQ has a way to make some of these types of operations very simple. 

Reading multiple websites

For this example I am going to read the first page of 8 websites and then act on that information afterwards.  This is the type of very simple parallel operation that splits up really well.  But these types of long running reads are very similar to what happens in many applications.

Side Note on C# 4.0 In a Nutsell Book

I actually adopted this example from one given in Joseph Albahari’s book C# 4.0 In a Nutshell (he is also the author of the excellent LinqPAD).  Weighing in a 1000 pages is not exactly a Nutshell, but it is a fantastic book for developers who already know C# and just want to go through C# and CLR 4.  The concepts in the book cover older versions of .Net as well, but the juicy parts for me were all the new changes.

More...

Plugins are used in data applications to support more than one provider from a single codebase.  The Provider Factories in ADO.Net is one such example, but you cannot work with the provider specific features when you use these factories since they only implement generic ADO.Net functions.  If you need to be able to use provider specific functions (like VistaDB’s DDA Pack routines), you have to load the provider somehow.  In most cases you put that logic and bindings into a separate assembly and load it when that ability is needed.  By taking this approach a little further and building interfaces you can abstract your logic to support more than one database provider using this model.

In this article I will explain more of how to code works to use VistaDB in a plug-in model previously explained in building a plug in model for VistaDB - Part 1. The application used to demonstrate the plug-in model is a simple windows form that allows users to deposit, withdrawal or check the balance of an account that is stored in either a VistaDB or Sql Server database.

We will implement the VistaDB custom plugins in this example.  You could implement other plugins for any other database you wish to support through the same interface.  In each case these plugins are hard bound against the database provider, but the actual application has NO binding to the database directly.  This is to allow it to run on machines where the database provider has not been installed.

Factory Classes in Main Application

The following classes are all contained within the applications main assembly.

Plug-in Factory

The plug-in factory is an abstract class that each custom plugin will need to inherit from to load from the singleton factory.

public abstract class PluginFactory
{
    #region public members
    public abstract BankModel.Provider ProviderName { get; }
    #endregion

    #region public methods
    public abstract BankModel GetModel(string connectionString);
    #endregion
}

ProviderName will return the name of BankModel provider, VistaDB or SqlServer. The GetModel method takes a database connection string and returns the abstract BankModel for that provider.

More...

In this article I will explain how to build an application that uses VistaDB and Microsoft’s SQL Server, without being hard bound against their providers directly.

Many companies desire to offer the choice from several database providers in a single product or API.  In many cases if you were directly bound, but the provider not installed, you would get a dll not found exception at startup. 

There are two ways to handle this scenario.  The most common method is to use ADO.Net Provider Factories and load the provider dynamically at runtime.  We have covered this in other blog posts.  This works great if you don’t need to use any DDA code, or any other provider specific functions.  But it is not really possible if you want to use provider level abilities.

The alternative is to put the provider specific code in an external dll, and load the code as a part of a plugin model.  Plugins can then be loaded individually and removed from consideration if the assembly fails to load (like when the provider is not actually present).  This allows for your code to continue to run, within needing the main executable to be bound against the provider.

More...

This topic comes up frequently when users are writing CLR Procs and Triggers.  How do I debug the code that is being hosted by the database?

There is a very easy way to do it through the System.Diagnostics.Debugger namespace to launch the just in time debugger for Visual Studio.

Prompting the user for the debugger

You can prompt the user to attach a debugger at runtime using the System.Diagnostics.Debugger. You would NEVER want to do this at runtime in a production environment! Only for use on your development machines. I usually surround the block with an #IF DEBUG to ensure it gets compiled out at release time.

Notice the screen shot above, the user is prompted for which debugger to use.  The CLRProcSample is the correct project in this case because that is what I want to debug.

Notice that it shows all the editions of Visual Studio on my machine (2005, 2008, 2010) and all open projects.  This makes it really handy for debugging CLR Procs, Triggers, just about anything that might be hosted external to a process you control.

More...

After reading Scott Hansleman’s article on exposing OData for Stack Overflow I thought it would be nice to update the previous post I did on Ado.net data services to include the new WCF Data Services.  WCF Data Services (formerly called Ado.net Data Services, and “Astoria”) can expose OData to callers through a very simple interface. LINQPad was not available to query the interface at the time, so I will also discuss how to use LINQPad to write queries against a Data Service.

For my example I am going to expose a VistaDB test database that shows SQL Commands, and examples of their syntax.  It is a very simple model, but provides interesting data to query against (other than Northwind!).  You can use any Entity Framework provider to perform these steps, they are not specific to VistaDB.

Being able to consume data across the web in a rest-ful manner is part of the power of OData, lots of applications that are powered by .Net are going to be able to consume OData services very easily.  But the OData protocol is not just for .Net, PHP, Java, Javascript and others also have the ability to consume the data.

What is OData?

The Open Data Protocol (OData) is an open web protocol started by Microsoft to expose data using existing web technologies.  HTTP, AtomPub (similar to RSS), and JSON are all supported.  The protocol matches very closely the way web technologies work, and the URL is the primary operator on the data query.  The HTTP verbs match very closely their CRUD operations.  The URL has a very descriptive syntax that makes it easy to build queries by hand, or with any programming language.  OData is not unique to .Net, although .Net sure makes it easy to expose and consume OData through WCF.

WCF Data Service

To expose OData we will build a WCF Data Service and expose our VistaDB EF model.  I am using Visual Studio 2010 and .Net 4 for this example.  The WCF Data Service item template in Visual Studio makes it very easy to expose an Entity Framework model over a service based interface.  You don’t have to use Entity Framework, but doing so makes it really easy to build and deploy.  I believe you could expose a custom collection through the data service as well, but I have not tried this yet.

More...

In this article I will continue to explain how to build a dynamic LINQ to Entities compiler for any database provider that supports the Ado.net Entity Framework.  See part 1 of building a dynamic linq to entities compiler for background information. This part of the series will cover using the .Net CodeDom Compiler to dynamically execute LINQ queries against an EF model.

We are working on a dynamic LINQ query mechanism for the next major release of VistaDB.  Our goal is to provide a dynamic LINQ execution panel (like LinqPad does for Linq to Sql) in Data Builder.  Users will be able to write LINQ to Entities queries against the database without having to first build an EF model.  We include a default data context object that can be used to write the queries the same way they will appear in your code.

Allow users to write a LINQ query

The first step needed in the process of compiling the query is to allow the user to supply me with the database and query they wish to execute. In this example the user must return their result set to a var named query and use the name “context” for the EF model context.

More...

In this article I will explain how to build a dynamic LINQ to Entities compiler for any database provider that supports the Ado.net Entity Framework. Due to the wide range of technologies used this article, it will be broken up into two parts as listed below.

We are working on a dynamic linq query mechanism for the next major release of VistaDB.  Our goal is to provide a LinqPad type of environment in Data Builder for users to write LINQ queries against the database without having to first build an EF model.

Blog Article Sections

• Part I. How to use edmgen command line tool to generate an EF model.
• Part II. How to use CodeDom to dynamically compile a LINQ query. See blog post

Technologies Used

• Ado.net Entity Framework (EF) – EF is an Object Relational Mapping (ORM) technology from Microsoft that is built into the .Net framework 3.5 SP1 and higher.
• VistaDB 4 – Commercial embedded SQL database that supports EF
• edmgen Tool (.Net Framework) – Included in the .Net framework, this tool is used to generate the models used by the EF runtime.
• CodeDom.Compiler (.Net Framework) – CodeDom is also built into the .Net Framework and provides the way to dynamically compile code
• LINQ to Entities (.Net Framework) – This is the query mechanism against the EF runtime, it is how you ask questions of the EF model.

Part I. How to use the edmgen command line tool

There are several steps needed in the process of dynamically testing a LINQ to entities query, first of which being the EDMX model itself. Visual Studio has a great set of wizards built in to handle generating an ADO.NET data model.  These wizards handle creating the necessary files for the EF model, and adding the connection strings to the app.config.

These wizards are not available at runtime, and the model generation becomes slightly more complex. There is no API available to generate an EDMX but Microsoft does include a command line tool called edmgen which can be used to generate an EDMX from any database provider that supports Entity Framework. You can find the edmgen tool under the 3.5 and 4.0 .net framework folders (C:\Windows\Microsoft.NET\Framework\).

More...