I'm using a wiki document library on Share Point as a central repository for design and implementation documentation on a project that I am working on (The selection of the wiki as the documentation repository is the topic for another post). At some point in the future, I know I will be asked to provide a hard copy of my documentation, and I didn't have a really easy way to print the wiki.

My first thought, was that there must be a program out there, that lets you specify a set of URL's, and then it will go print them for you. Maybe I wasn't searching on the correct terms, but I couldn't find anything. I remembered there was a print linked documents function in IE, so I went back to investigate that.

The print links option, is in the options tab of the IE 7 print dialog box. If selected, it will print all 1st level linked documents of the page you are currently on. So, all you really need to do is create a .html file someplace on your computer, and add URL's to it, open in IE, and print it with the print linked documents option. You can create several different .html files each with a different set of URLs. I wish you could specify to print n levels deep, as that way you could build up a hierarchy of .html files. Perhaps this would be worthy of an IE plug-in?

Getting back to the Share Point wiki, I wanted a quick way to get the URLs for the pages I wanted to print. There are the default list views, which list all content in the wiki, which almost work, but there are too many extra hyper links. I decided to create a new view, that only contained the wiki page name (as a hyperlink to the document), as well as various non-linked text fields, like date created, modified and version. I could then copy this to a .html file and print.

So why not just print from that view? Well, there are all the other URLs on the page, such as the side bar, and admin menus, that would also get printed. However, if the pages you want to print are displayed in a grouping (and with the way I name and categorize my wiki pages, this is pretty easy), you can just highlight the rows, and choose the print selection option in the print dialog box (and print linked documents), and then you don't even have to maintain a list of .html files. This meets my needs for the foreseeable future.

I came up with the following list, which outlines what I'd like to see in a standalone application, or IE plug in.

• Site spider to gather URLs (with options to filter specific links using regex)
• Drag Drop URLs to organize them into logical groupings for printing
• Define lists which build form other lists
• More formatting options for printing
• Ability to add meta data to the pages referenced by the URL
• Print a table of all links found in the documents
• Actually this is an option already in IE, but I'd like to see it print a single list, instead of 1 per page, so there are no duplicate URLs. Then add a footnote to each link on the actual document which can be used to reference the table of links.

While searching for information and ideas on configuration stores, I came across an article on better WMI scripting. Defiantly something I will need to review in the future as I get into automated BizTalk management.

It seems as if there is someone at Microsoft, who's sole purpose in life is to figure out how I might want to use something (WCF), and then make sure I can't use it in the way I want to. Today's issue, programmatically configuring a WCF client application.

Background

I have created a central repository store in the form of a SQL database, that put simply, contains key/value pairs that I can query. All of my applications (or each machine.config) only need to specify a single database connection string to this configuration database.  All other configuration settings are retrieved from the database (and cached), making it easy to keep track of configuration, view all current configuration settings, and deploy the applications on new servers.

Expanding upon my simple definition, there is a second "key", which corresponds to what environment (production, staging, development) the application is deployed. So my single configuration database controls the configuration of all applications in all environments. For those settings which are the same across all environments, no deployment key is specified.

So far this has worked great, and I wanted to add configuration information about a WCF client service.

WCF

WCF needs two properties set, a binding, and an endpoint, and each of these 2 properties have sub properties which may or may not have values that need to be set. Out of the box, WCF is most easily configured using application or web configuration files. All of the tools support the configuration files and it's easy to see what's going on (well sort of).

I basically wanted to take the XML sections that define the binding and endpoint, and place it into my configuration database. For some reason, I got it in my head that this would be really easy to do, but found out that it's not. Searching thru Google yielded no direct solution. While you can programmatically set the Binding and Endpoint properties of a proxy class client object, I would either have to have 1 setting per property (and sub property) in my configuration database, or come up with an XML schema, parse the schema and set the values. This was the path I was pursing, but instead of making my own schema, I just used the XML configuration schema provided out of the box, in hopes that one day I could just pass those sections to binding or endpoint objects and have it parse for me.

After creating classes to parse a single binding (WsHttpBinding), I decided there must be a better way.  The code to parse the configuration files must exist somewhere in System.ServiceModel, so I decided to go spelunking using reflector to see what was going on, and determine if I could hack something together.

Implementation

Overview

After spending a couple of hours in reflector, I realized the main roadblock was the use of the System.Configuration namespace to handle the configuration data for System.ServiceModel. System.Configuration only supports the file system, and provides no extension methods at this time (something I'd like to see from Microsoft in the future).

This left me with one option, which was to create temporary XML file with my configuration data from my database, and then read that configuration file using the proper classes/methods in System.Configuration. It took me awhile to accept this option, as writing a temporary file seemed messy, but then I realized, temporary files are a fact of life in .Net, programming, operating systems, etc.

Configuration Store

My configuration store was already defined as I mentioned previously. The 2 XML samples below are stored as separate entries in my configuration database. Below is the XML I stored for handling the bindings:

<bindings>
 <wsHttpBinding>
  <binding name="WSHttpBinding_ITwoWayAsyncVoid" closeTimeout="00:01:00"
          openTimeout="00:00:30" receiveTimeout="00:02:00" sendTimeout="00:00:15"
          bypassProxyOnLocal="false" transactionFlow="false" hostNameComparisonMode="StrongWildcard"
          maxBufferPoolSize="524288" maxReceivedMessageSize="65536" messageEncoding="Text"
          textEncoding="utf-8" useDefaultWebProxy="true" allowCookies="false">
          <readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"
            maxBytesPerRead="4096" maxNameTableCharCount="16384" />
          <reliableSession ordered="true" inactivityTimeout="00:10:00"
            enabled="false" />
          <security mode="Message">
            <transport clientCredentialType="Windows" proxyCredentialType="None"
              realm="" />
            <message clientCredentialType="Windows" negotiateServiceCredential="true"
              algorithmSuite="Default" establishSecurityContext="true" />
          </security>
  </binding>
 </wsHttpBinding>
</bindings>
 

I am able to store multiple bindings, and binding configurations in a single entry in my configuration database (see below). In this example, I only have a single binding and binding configuration. I was originally planning to have 1 entry for each binding, but I don't think that will be necessary, if anything I will have a single entry for all necessary bindings per environment, and even that might be overkill.

This is the XML I stored for the end point:

<bindings>
 <wsHttpBinding>
  <binding name="WSHttpBinding_ITwoWayAsyncVoid" />
 </wsHttpBinding>
</bindings>
<client>
 <endpoint address="http://tersodemodev1/TsiCbsEsbReciever/WcfService_TersoSolutions_CBS_Fusion_BizTalk_InitialProcess.svc"
        binding="wsHttpBinding" bindingConfiguration="WSHttpBinding_ITwoWayAsyncVoid"
        contract="TsiCbsEsbReceiver.WcfService_TersoSolutions_CBS_Fusion_BizTalk_InitialProcess"
        name="WSHttpBinding_ITwoWayAsyncVoid">
        <identity>
          <userPrincipalName value="TERSODEMODEV1\BizTalkWebServices" />
        </identity>
 </endpoint>
</client>

Notice the repeat of the <bindings> section. This was required if I wanted to use the the classes in System.ServiceModel for parsing the endpoint, I needed to define a skeleton for the bindings.

I separated the endpoints from the bindings, as that is what is going to change between environments the most for me. The endpoint specifies which server (test, production, staging), and identity (for single server test scenarios, the account will be local, for multi server production enviroments, the identity will be a domain account).

Code

I created 2 classes under a namespace called ConfigSystem. The first class, called ConfigMgr contains code that allows me to go from my configuration database to a System.Configuration.Configuration object by way of a temporary file. The second class called Wcf, contains code specific to my Wcf configuration implementation, and contains code inferred from reflector.

All of the code in ConfigMgr uses public classes and is pretty simple.

public static class ConfigMgr
 {
  #region MemberVars
  private const string configFileHeader = "<?xml version=\"1.0\"?><configuration>";
  private const string configFileFooter = "</configuration>";
  
  #endregion

  #region Methods - Public

  public static ConfigurationSection GetSection(string configKey,
   string xmlConfigFileHeaderExtra, string xmlConfigFileFooterExtra, string sectionName)
  {
   //Get an xml document from the configuration database to store as a tempory file.
   System.Xml.XmlDocument doc = PrepareXmlDocument(configKey, xmlConfigFileHeaderExtra, xmlConfigFileFooterExtra);

   //Now write out to the temp file.
   string path = System.IO.Path.GetTempFileName();
   using (System.Xml.XmlWriter writer = System.Xml.XmlWriter.Create(path))
   {
    doc.WriteTo(writer);
    writer.Close();
   }
   
   //Setup configuration file map so that we can use standard file based config file
   ExeConfigurationFileMap map = new ExeConfigurationFileMap();
   map.ExeConfigFilename = path;

   System.Configuration.Configuration config = ConfigurationManager.OpenMappedExeConfiguration(map, ConfigurationUserLevel.None);
   
   //Now return the specified section
   return config.GetSection(sectionName);
  }

  private static System.Xml.XmlDocument PrepareXmlDocument(string configKey, string xmlConfigFileHeaderExtra, string xmlConfigFileFooterExtra)
  {
   System.Text.StringBuilder sb = new System.Text.StringBuilder();
   
   //Append header
   sb.Append(configFileHeader);
   sb.Append(xmlConfigFileHeaderExtra);

   sb.Append(ConfigHandler.GetValue(configKey));

   //Append Footer
   sb.Append(xmlConfigFileFooterExtra);
   sb.Append(configFileFooter);

   System.Xml.XmlDocument doc = new System.Xml.XmlDocument();
   doc.LoadXml(sb.ToString());
   return doc;
  }

  #endregion
 }

The code in the Wcf class uses some methods I copied via reflector, and because I don't know the legal ramifications of that, I am only providing an outline of what I did.

GetBinding:

• Calls ConfigMgr.Get section and returns the <bindings> section. This is cast to a BindingsSection object.
• Code taken from System.ServiceModel.Configuration creates a BindingCollectionElement which is the correct type associated with the binding stored in the XML config.
• A new binding object is created using Activator.CreateInstance(BindingCollectionElement.BindingType)
• Additional code taken from System.ServiceModel.Configuration loops thru BindingCollectionElement.ConfiguredBindings and initializes the binding object previously created. A check is made to make sure multiple bindings of the same type are not created.
• Finally, the binding object is returned to the caller.

GetChannelEndpointElement: This class uses only public classes and methods so I am including it. EnpointElement contains the configuration data for an endpoint, including the URI, identity, binding, and  binding configuration. The binding and binding configuration data is actually passed to my GetBinding method to return the correct binding with the correct binding configuration.

public static ChannelEndpointElement GetChannelEndpointElement(string configKey)
  {
   ClientSection section = (ClientSection)ConfigMgr.GetSection(configKey, system_ServiceModelAsOpenXml, system_ServiceModelAsCloseXml, clientSection);

   if (section == null)
    throw new ArgumentException(string.Format("Client section returned from config db for key {0} was null", configKey));

   if (section.Endpoints.Count != 1)
    throw new ArgumentException(string.Format("There must be exactly 1 endpoint returned from the configuration. {0} endpoints were returned for key {1}", section.Endpoints.Count, configKey));

   return section.Endpoints[0];
   
  }

GetEndpointAddress: Another method I created which almost uses all public methods. It takes a ChannelEnpointElement and returns a EndpointAddress which can be assigned to a wcf proxy class.

public static EndpointAddress GetEndpointAddress(ChannelEndpointElement element)
  {
   //Create a new builder, as the endpoint address is an immutable class
   EndpointAddressBuilder builder = new EndpointAddressBuilder();

   builder.Identity = LoadIdentity(element.Identity);
   builder.Uri = element.Address;

   return builder.ToEndpointAddress();
  }

LoadIdentity: This final method was taken from System.ServiceModel.Description.Configloader. This method takes a IdentityElement (a property of ChannelEnpointElement) and returns a EndpointIdentity which is passed to the EndpointAddressBuilder in my GetEndpointAddress method. The method basically checks the the existence of certain properties, and if one exists, it returns the appropriate identity using public factory methods.

• EndpointIdentity.CreateUpnIdentity
• EndpointIdentity.CreateSpnIdentity
• EndpointIdentity.CreateDnsIdentity
• EndpointIdentity.CreateRsaIdentity
• EndpointIdentity.CreateX509CertificateIdentity

There is a identity type of certificate reference which I was unable to add support for, because it referenced additional internal classes which I didn't want to duplicate at this time since I had no need for that identity type.

Issues

Since this is completely unsupported by Microsoft, and uses code that was never meant for external use, in a future version of the framework, something could break. I'm also not an legal expert when it comes with what someone can do with MSIL code discovered thru reflector, so for now, the actual code I wrote is not available publicly. Hopefully I provided enough information to allow someone to reproduce my steps and implement something on their own.

Another somewhat disappointing realization I came to, was that this might make it hard to support write operations to my configuration database. Currently the configuration database is managed thru scripts and direct editing. If write support to the config database became a requirement, I'd probably try to do the reverse and use the built in file save functionality in System.Configuration to save a temporary xml file, read it and then store it into my config database.

Conclusion

This was my first attempt at doing something like this, digging into framework code to see how it really works and trying to come up with a solution. I'm going to go forward using the code I wrote outlined in this post, as I have full read support for WCF XML configuration. The amount of code I had to write and test is significantly less then if I would have wanted to get the same amount of functionality (configuration support for all bindings and endpoint configurations).

I finally got nMap working under Vista. I don't know if it was the latest version of nMap I installed (4.23RC3) or winPCap (4.02), or some other change to Vista (update). You do need to run it as an administrator to get access to the network card at the low level required by the tool.

I don't know when they added this, but in the 4.23RC3 the GUI is included and works well. The command line is still there in the background available for the power users. One nice thing about the GUI is that as you make changes, you see the command line that is going to be execute.

In a recent email from MSDN Flash, I first learned of the new Parallel Extensions to the .Net Framework 3.5. This is an out of band release that is currently available as a CTP on Microsoft Downloads.

The description in the MSDN Flash newsletter has me pretty excited about what this offers.

[Parallel Extensions] is a managed programming model for data parallelism, task parallelism, and coordination on parallel hardware unified by a common work scheduler.

...

Parallel Extensions provides library-based support for introducing concurrency into applications written in with any .NET language, including but not limited to C# and Visual Basic.

It is specifically designed to take advantage of multi-core processors (among other things), which is important due to the recent (last 2 years) shift in the industry from raw clock speed to multi-core.

The computational power of multi-core processors, new programming models and platforms, and advanced research in usability all promise to change the way people interact with computers.

While excited, I do have some reservations. Encapsulating the lower level knowledge needed to take advantage of multiple cores will speed up development, but it will reduce the number of programmers who know how that lower level stuff needs to be written. The concepts used to take advantage of multi-cores, should be very similar to those required to take advantage of multi-threading, which is more "widely available" on the compact framework (although I am seeing more and more embedded computers with Core 2 Duo's). If this framework is only available for the full framework, the pool of skilled workers available for the compact framework could diminish (even more so).

I actually have more of a need to take advantage of multi-threading, and parallel processing in compact framework applications. As far as asp.net applications, I would like to see if this is something that would be recommended for them. Long running processes that would take advantage of this new framework, are usually best left outside the asp.net application. Speaking of long running processes, that is the 3rd area I work in, and in a traditional environment, would love to take advantage of this. However, the applications I am writing are all running on VMWare ESX, and IT has it set so that each VM only has a single CPU, and puts the burden of scheduling the 8 physical cores on ESX itself.

This will probably end up being something I try to work into personal projects. Unfortunately, I do not have the time to play with the CTP at this moment, but it is something I defiantly want to look into in the future.

Additional Information:

• MSDN Developer Center

From Dave Northey's blog, comes a link to an Active Directory tool previously only available to MS Premier Support. The tool discovers information about your Active Directory and Exchange infrastructure and exports it to Visio. The tool is available free of charge from Microsoft downloads.

Avoiding Development Disasters is the tile of an article I came across today and talks about how and why software projects fail. Here's what I got out of the article.

FBI Virtual Case File

The article opens up talking about the monumental failure that is/was the VCF and touches 6 factors which lead to the projects failure.

• Lack of an Enterprise Architecture - Unfortunately the article doesn't go into what they did have, have not.
• Poor management of developers, including a lack of management or micro-management
• Unqualified persons were placed in critical roles
• Constantly changing requirements
• Scope Creep
• Throwing more developers at the project in a last ditch effort to meet deadlines.

I am personally experiencing 3 of the listed items on a current development project, although it's far from a $100 million dollar system, I bet there are some striking similarities. I would imagine that every project at every company experiences 1 or more of the things in that list to some extent or another.

What is Success?

The VCF was eventually scrapped, but the author claims, that had it gone to production, it would have been deemed a success, even with all it's flaws. The author goes on to say that this is the general practice in the software industry (at least enterprise applications). As much as I like Microsoft, I think Vista is a shining example of this (although perhaps it is deemed less of a success inside of Microsoft).

I have to agree, and would be willing to say it's common in software projects in general, from small to large. How many times is a game released, and the day of, a patch is released? Even Epic Games, creators of the Unreal platform, and who coined the phrase "When it's done", still manage to release a product with known issues.

I'm not trying to criticize any one company or developer, other then myself, as I have created some less then magnificent code over my short life as a developer. I think part of the problem is that there are no points for creating good code. At the end of the day, "Well it works doesn't it?" pays the bills.

Why is it so hard to write code, almost 30-40 years since the first programs were written? We have better tools, faster computers, and years of other people's failures to learn from, and here we are, still producing less then our potential.

The Code to Ruin

The diagram "The Code to Ruin" presented in the article is so true, it's scary. You pretty much know what's going to happen before you start the project, but you still can't avoid it. That's depressing at best.

Maintainability

The article spends the last half, talking about the maintainability of code. Without code that is maintainable, while you launch may be a success, you next point release is probably going to be a failure. The article states that enterprise software should have a 15 year life span, that's longer then I have been coding.

I think ideas such as software as a service might help us reach that goal, and have more maintainable code overall. I'm not taking about providing software as a service, I mean, that the internal make up of your application is constructed from (loosely coupled) services. Breaking stuff down into more manageable pieces seems to be the way to go. We already do it with proper OOD design, we should also be applying it to the system in general. Of course there is a trade off, the most notable to me being one of performance, but that's what these faster computers are for ;)

All in all, it was a good article, and made me really think about software projects, both past and current that I am working on. I have to imagine that there are people out there that would be the perfect compliment to a talented programmer such as myself (well at least I think I'm talented). Or, does it mean, that I need to spend less time with technology and programming, and learn more skills like project management, documentation, business analysis, etc.? To specialize, or generalize, that is the question?

Sometimes a file gets locked in TFS, and for whatever reason, you need to unlock it. You can use the tf command line utility to accomplish this.

tf lock /lock:none $/Project/AnyFile.extension /workspace:ComputerName;User /s:http://TfsServer.Com:8080