About Jason Short

Jason Short photoIf you want to know more about Infinite Codex, Inc. you might want to visit the About the Company page. This page is mostly about me, who I am, why I love programming, and how I got to be here today.

Who is this guy?

I get posts to the blog and forums every now and then asking "who are you?". It is easy to forget that not everyone has been involved over the past several years with the company and knows the back story.

My name is Jason Short and I acquired the VistaDB product technology in April 2007 after having used the previous VistaDB Engine to build some major systems at my previous company. The (then) current owner was looking to move on to other opportunities and I was looking for a new project to work on as well. 

VistaDB filled a personal need

VistaDB 3 was well into the development cycle and I had been a beta tester from early on. There were a lot of design decisions that I didn’t agree with, but it wasn’t my company. I was using the early betas for an internal project that needed a lightweight SQL database we could XCopy deploy to 63 machines . The machines then all aggregated their data back to a central server at night for indexing, collation, and stats tracking. We could have used a single big central SQL Server, but these client machines were running FreeBSD and Mono.  There was no SQL Client on Mono at the time.  And I didn’t want to write a complete client to talk to one of the other databases.  Instead we used SOAP to communicate between the nodes and the hub of the system.  It added a layer that allowed the central point to fail and the machines running were not affected.
 
The applications themselves were a distributed crawling system for classifying content on the Internet. All of the machines were not even in the same physical location, but I needed a common transport mechanism across Windows and FreeBSD to ensure we had a single file format. Adding the database locally was a fantastic time saver for us. The custom flat file index routines we had written were taking 8+ hours to index content per day. By integrating them into a database with indexes we took a small hit in crawling speed (most of your limitations are in network bandwidth and file IO anyway), and the data was always ready with an up to date index.  We could then run reports at any given period during the day instead of having to wait until the nightly batch had run to do the integration.

VistaDB bugged me

During these 2.x and early 3.x days I was always bugged by the lack of quality gates in the product. Frequent regressions in quality and bugs that would appear and then mysteriously go away without explanation.  That drove me nuts. It bothered me so much that we built our own internal NUnit tests to certify a VistaDB build before we would push it live to our systems. That was one of the first things I incorporated when I took over the company; unit testing. It has saved us so many times that I am honestly amazed that any company with an API can ship a stable product without automated regression testing.
 
When the opportunity to acquire the company presented itself I jumped at it and have never looked back. I sold my previous company almost right away, and took on the challenge of data manipulation. It is a fantastic mental challenge to build something so large and complicated, and I really enjoy working with fellow programmers.   Almost all of the really interesting parts of what we did for the crawling and indexes were really low level databases and custom structures to manage them.

Is VistaDB perfect?

Is VistaDB where I want it to be now (writing this in Feb 2010)? No, it is not. There are still a number of design decisions that need to be corrected for performance in Dot Net, and we have a ways to go for documentation and training material.  Technical debt is one of those things that cannot easily be corrected, but we are trying.  Early design decisions in a product often narrow your focus to what you can do in other areas of the code.  We are continuing to rebuild some parts of VistaDB at the very lowest levels, but we also can’t break any of the top level API calls.  It is a delicate dance between adding new internal abilities, but not breaking existing ones. It has been a lot of work, but very rewarding at the same time. I enjoy meeting users who have built fantastic systems with VistaDB, and getting to work with talented developers from all over the world everyday.

VistaDB 4 has made a ton of changes that I always wanted to make, but we still have a ways to go to meet my ideal engine architecture.  Performance continues to improve, but a lot of the bottlenecks are things that are just side effects of 100% managed code.  We do have a plan to address some of those limits by doing things differently to be more Dot Net friendly, more on that later.

Building an API is different

One thing that has surprised me quite a bit is how much harder it is to standardize your own API, and then stick to it. Breaking other people’s code is a big deal to me. I always hate it when a vendor does it to me, so I don’t want to do it to others.
 
But it is so easy when writing internal systems to just change a parameter type, or strongly type something when you see benefit. The public API does force you to focus on implementation versus interface though, and to spend a lot of time on algorithms that have to work in some very broad circumstances. The sort algorithm that worked great at 1 million entries fails horribly at a 100 million, or when low on memory.

We find lots of different users who have put VistaDB into amazing applications.  Finding a way to make all of them work and stay consistent in our performance is a major challenge.  We have begun looking at ways to dynamically swap out certain algorithms based upon the runtime machine abilities, database schema, and the actual content present.  There are times when some operations could be run across multiple cores, but we don't right now. 

Compact Framework probably going away anyway

The Compact Framework was another limiter in our designs as the same abilities had to always be present on mobile machines. 4.x does not include CF support, this lets us take advantage of more of the features of the .Net framework for performance and scale. 
 
And I think the writing on the wall is pretty clear at this point, .Net 4 has no mobile or Compact Framework version.  The newly announced Windows Phone 7 has no Compact Framework install for it either.  I think Microsoft is going to push Silverlight onto the mobile developer as their way to build once and run in more than one location.  You couldn’t take your CF app and run it on a desktop, or tablet device.  Silverlight is actually a better cross platform app than Compact Framework ever was.  But there is no ado.net provider model in Silverlight…

Some kind of Expert?

I do have a PhD in Computer Science, but I do not consider myself an expert in everything computer science related. I am constantly learning and changing. Every year I look back and think about how much more I have learned, and wonder where my coding and design will be in another year. You will often hear me repeat the phrase that I don’t know the answer, but can find out.
 
People sometimes think because I work in the engine I know every possible SQL trick in the book; not so. I know algorithms, low level code, file IO, b-trees, merging, indexing, etc.  All of those are the things that make up a SQL engine. SQL is really a public interface on what we do, but not how we actually do it.  Nothing in the engine runs on SQL.
 
Quite a few people were shocked when I recently wrote that I didn’t consider any of the team to be experts in LINQ or Entity Framework, yet we wrote an Entity Framework provider. You don’t have to be an expert in every technology to solve a need. You have to be flexible, and willing to change with technology as it changes. We studied the EF spec, looked at the samples and then wrote one. Is it perfect? No, we learned a lot as a result of building the first one and have quite a few ideas for how to improve the next one as a result. That is one of the importance of standards like ADO.Net.  We can implement to a specification and be ensured that our back end will work.  There may be specific implementation problems, sure.  But that is to be expected, adapted to as they appear, and overcome.

Written much code?

I started programming when I was around 7 years old on a Commodore VIC-20 my mother bought for her college classes at the time. I used to type in BASIC programs from Compute’s Gazette and spend hours trying to figure out why they didn’t work. At age 11 I was published in a magazine for a small game about catching Bumble Bees (it was a much simpler era for games too!). During my High School years when most kids were working summer jobs cutting grass, I wrote a BBS system for the Amiga and sold it as shareware. I made a lot of money (for a teenager) selling that BBS software and multi user BBS games I wrote to amuse my friends.
 
I have been programming professionally for the past 18 years (Windows, embedded systems, win16, win32, Alpha, Mips, etc). I thoroughly enjoy the programming process, but not always the constant rate of change. Some of the systems I have written include a huge database of names spoken in real-time for a theme park ride, a distributed rendering system that ran as a screen saver, distributed crawling systems, spam filtration, web filters, and huge databases containing content from billions of pages on the Internet.

Data is a constant theme

I think all of the things I have built over my career had one thing in common – data storage needs that were always growing.  It was this constant data growth that originally drove me to want to build my own data storage engine. We had started building a special purpose data indexing and storage system at my previous company.  But at the time I didn’t want to spend the R&D on building something that would just be an internal system.  I acquired VistaDB instead!

Company Information Changed

Against the advice of several good friends I originally called the company VistaDB Software to match the product name.  Now that we are branching out to offer other data centric applications it was confusing for users.  I decided to change the company name to better reflect who we are as a company. Not all of our software products today require VistaDB in order to operate.  Some, for example, work with VistaDB and SQL Server.  Others are stand alone specialized data structures that don’t need a complete database.
 
Rest assured that I have not changed.  I am, as always, dedicated to the pursuit of building excellent applications that programmers can use to build bigger and better projects.

CornerstoneDB is a new brand I created to specifically deal with tools that .net data developers need.  Some of them are tools you could get from the Visual Studio Ultimate edition, but even then you can't build a product based on them or ship them to end users.  I have always been really big on providing API access to things that I build, and CornerstoneDB tools are all driven by a powerful core API that works across multiple database vendors products.

Current Information

I am currently approaching 40 (quite quickly), am happily married to my wife of 14+ years, and have two fantastic girls. I live in Mount Dora Florida (just north of Orlando). 
 
You can find me on our forums, LinkedIn .Net groups, Stack Overflow, Twitter and just about anywhere dot net programming, or data structures can be found. 
 
 
 
   

Jason with his Kids

Picture of me with my girls