I am sure that most people who follow Windows Phone 7 already know all about Mango.  But if you are just sort of casually following what Microsoft is doing around the new phone platform you will have missed a lot of what is in this video.  Joe Belfiore (Corporate Vice President) does a demo of some of the highlights for Mango.

Go watch it directly on YouTube, it is a good video.

More...

I have gotten asked quite a few times why my Visual Studio looks different, or has some additional feature than another developers. There is a large community of plugins for Visual Studio (quite a few are free). I just wanted to link to two of the most popular that I send to people.

More...

I listened to a really good HansleMinutes yesterday about WCF. I was actually going to skip it, but the title said “This is not your fathers’ WCF”.

I am glad I listened because the conversation was mostly around new initiatives in WCF, specifically the WCF Web API. Glenn Block has an entire blog post on the topic WCF Web APIs, HTTP your way. That post does a great job giving the high level overview.

Visit the WCF Codeplex site for the complete code and early release bits.

What’s the Big Idea?

To me the big idea was that you can expose a WCF service endpoint over HTTP with only HTTP messages (very lightweight).

Imagine you have a heavy service with lots of contracts for your “serious” API work, but you need to expose something trivial as well (think a simple time service to ensure the service endpoint is up, and our clocks are close enough together). Normally you would probably switch to using asmx or a simple SOAP service for this service. But you don’t have to do that, you could use WCF.

The problem with WCF has always been the abstractions away from the transport (HTTP in most cases). This is fine for well designed systems with lots of layers. But when you want something quick, it can be a pain to build and deploy. And then what if you want to evolve it? Good luck with versioning. Using a simple HTTP service makes that story much easier.

Links

Visit the WCF Codeplex site for the latest details

I appear to have lost about 5 months of blogs.    That’s ok, most of them were about shutting down the company and my move here to the Northwest.  Probably better off gone anyway.

New Year – New Blog

Took the family up to Whistler Canada for New Years.  First time my kids have see FEET of snow in depth.  It was a lot of fun.

I am slowing getting my feet into things at Microsoft. I have a new improvement that is already ready to go into ship mode.  Good stuff.

Will try to find out what happened to all these images and get everything updated.  I may remove some of the older posts though.

A friend of mine sent me a link to a Zed Shaw blog post “There are No Famous Programmers”.

I agree with most of what he says, programmers are not famous

Not all code that is sloppy or messy is technical debt.  I tend to agree with Martin Fowler in his assessment of technical debt.  When your team is making conscious design decisions for valid reasons (shipping a product, pushing off heavy work for next release, etc) then you are incurring debt.   When you just have a mess of code, it is not technical debt.

Pay it down fast

Technical debt is like a short term high interest loan, you should prioritize paying it off quickly, or the interest quickly overcomes the original principal of the loan.

Remember that whatever you ship to incur that technical debt will still be out there, and have to be supported.  You can’t simply tell all your users to upgrade to 2.0 because 1.0 was full of debt.  In this way technical debt is a long term bond, you can’t simple buy them back.  They take time to mature and be retired.

Learning from your code

Programming is always a journey about learning.  Often after completion of a project I think of better ways it could have been designed, built, etc.  The project could still have been a success, but there were things I would have done differently with the full insight of the completed project. 

This usually happens to me when there was simply not enough time to think through everything up front, or the requirements changed during the course of the development.

Make conscious choices

Bad things happen on projects all the time.  Just make sure you are making the decision to incur technical debt consciously.  Don’t end up in that accidental debt category.

I have been studying the new PLINQ and Parallel Task Library in .NET 4 looking for various ways to do things that we can’t do in .NET 2.0. PLINQ is huge, and there are a lot of new ways to do multi threaded programming using .NET 4. In this article, I want to cover a particular problem I have had many times over the years.  How do you speed up multithreaded apps that are bound by blocking functions, or long running I/O operations?

I started looking at this method to speed up some long running file I/O routines deep in the VistaDB engine. Most of the time, we are blocked in reads from disk before we can continue working, but usually we have part of the blocks we need.  So we could start working, and then continue when the rest of the blocks are loaded. Adding that logic is complex and prone to error with traditional threading code. Fortunately PLINQ has a way to make some of these types of operations very simple. 

Reading Multiple Websites

For this example, I am going to read the first page of 8 websites and then act on that information afterwards. This is the type of very simple parallel operation that splits up really well. But these types of long running reads are very similar to what happens in many applications.

Side Note on C# 4.0 In a Nutsell Book

I actually adopted this example from one given in Joseph Albahari’s book C# 4.0 In a Nutshell (he is also the author of the excellent LinqPAD). Weighing in a 1000 pages is not exactly a Nutshell, but it is a fantastic book for developers who already know C# and just want to go through C# and CLR 4. The concepts in the book cover older versions of .NET as well, but the juicy parts for me were all the new changes.

LINQ Expression

Ok, this expression will go to 8 websites in this list and get the first page of each.  The content length of the page and the content type are then stored in a variable to be used outside of the parallel computation later.

Collapse | Copy Code

static void Main(string[] args)
{
    Stopwatch sw = new Stopwatch();
    sw.Start();

    var results = from site in new[]
    {
        "http://infinitecodex.com",
        "http://www.vistadb.net",
        "http://stackoverflow.com",
        "http://cornerstonedb.com",
        "http://www.bing.com/",
        "http://www.linqpad.net",
        "http://www.cnn.com",
        "http://www.microsoft.com"
     }
     let p = WebRequest.Create( new Uri(site)).GetResponse()
         select new
         {
             site,
             Length = p.ContentLength,
             ContentType = p.ContentType
         };

     foreach (var result in results)
     {
         Console.WriteLine("{0}:{1}:{2}", 
             result.site, result.Length, result.ContentType);
     }

     sw.Stop();

     Console.WriteLine("Total Time: {0}ms", sw.ElapsedMilliseconds);          
}

The initial runs were done with no Parallel extensions being used. Just go through each site and get the first page, storing the ContentLength and the ContentType in the temp variable p. Afterwards, I foreach over the results to output them to a command line. If you take this step out, nothing actually happens because of deferred execution in LINQ (you have to do something with the collection before it is really run). I wrapped all of this in a Stopwatch so I would know how long it took. The graph at the top of this article are the 3 fastest times I received after running each method 10 times.

Three fastest times normal execution (ms):  1916, 2103, 1992.

Adding Parallel (PLINQ)

Now, let's make this use PLINQ and see if it runs faster.

The only change we have to make is to add a single line of code above the let statement like this.

}
.AsParallel()
let p = WebRequest.Create( new Uri(site)).GetResponse()

That’s it, and the entire LINQ query will now run parallel.  It is faster, but not as fast as we can get it.

Three fastest times with AsParallel() (ms): 745, 790, 814.

What PLINQ is doing under the hood is creating a thread pool and spinning up 4 threads on my 4 core machine. But what it doesn’t know is that each of these operations are blocking waiting on I/O from the website. PLINQ assumes that each thread will have a moderate amount of CPU work to do, so it prevents spinning up a lot of threads that would just overwhelm the CPU.

How can we tell the .NET framework that each of these parallel operations are not CPU intensive?

WithDegreeOfParallelism

From the MSDN help:  WithDegreeOfParallelism<TSource> - Degree of parallelism is the maximum number of concurrently executing tasks that will be used to process the query.

Now that doesn’t exactly explain in plain English that you can use this to tell the framework the task is not CPU intensive. Technically you are overriding the default behavior of PLINQ and telling it you know how many of these should be allowed to run concurrently. 

In this case, I am going to set 8 because I know that two of these objects per CPU core is not going to tax my system at all. The maximum you can set is 64. Now each of these thread pools will attempt to run more than 1 thread at a time. Why can we do this without incurring a lot of task switching overhead? Because the objects are all blocked in I/O. The OS will put them to sleep and release the CPU for other tasks to run anyway, we are just going to give each of those tasks more work to keep them a little busier.

Again, a single line change to the first query is all that is needed:

}
.AsParallel().WithDegreeOfParallelism(8) // HERE
let p = WebRequest.Create( new Uri(site)).GetResponse()

Three fastest times with 8 Parallelism set (ms): 543, 578, 589.

That is 3.5 times faster than the original query with one line code changed!

Summary

PLINQ and the .NET 4 framework give you a lot of power to speed up parallel operations very easily. In my page manager application, I was able to get a 4.5x improvement in the page cache manager through the techniques listed in this article. And through changing my queue mechanisms over to the new Concurrent classes, I was able to eliminate a lot of dead time wasted on locking and gained even more performance, but that is another blog post at some point in the future.