Posted by Kyle Heon
Fri, 18 Sep 2009 00:03:00 GMT
Today was a day of troubleshooting and fixes. We had a site that was getting ready to go live, hosted on a 64-bit version of Windows Server 2008 (IIS7). I won’t bore you with all the details of all the issues we worked through today but we had one nasty issue that, once it happened it brought the server down; 100% CPU usage by the w3wp.exe process and memory usage steadily climbing. Our error logger was catching mostly one xml argument exception (but with a few variations).
A partial stack trace looked like this:
System.Web.HttpUnhandledException: Exception of type 'System.Web.HttpUnhandledException' was thrown. ---> System.ArgumentException: The node to be removed is not a child of this node.
at System.Xml.XmlNode.RemoveChild(XmlNode oldChild)
at System.Xml.XmlNode.RemoveAll()
at System.Xml.XmlElement.RemoveAll()
This error was at the heart of our navigation framework which makes heavy use of xml and xsl. The process we go through for building a pages navigation looks something like this:
- Check to see if the xml navigation file has been cached, if so get it from cache, otherwise load it and cache it.
- When navigation is ready to be rendered, it is first manipulated to add in an xml node section that stores the current pages selected states
- The modified (in memory) xml is then passed to the desired xsl which handles converting the xml to html and returning the transformed string.
Originally the process of modifying the xml (in memory) looked like this:
protected virtual void AddNavigationSelectedStates()
{
/*
<selected>
<choice level="1">Selected L1</choice>
<choice level="2">Selected L2</choice>
<choice level="3">Selected L3</choice>
</selected>
*/
XmlNode navigationScheme = this._xml.SelectSingleNode("navigationScheme");
XmlNode selected = this._xml.CreateNode(XmlNodeType.Element, "selected", "");
if (null != navigationScheme.SelectSingleNode("selected"))
{
selected = navigationScheme.SelectSingleNode("selected");
selected.RemoveAll();
}
for (int i = 0; i < base.Selected.Count; i++)
{
XmlElement choice = this._xml.CreateElement("choice");
choice.SetAttribute("level", (i + 1).ToString());
choice.InnerText = base.Selected[i].ToString();
selected.AppendChild(choice);
}
navigationScheme.InsertBefore(selected, this._xml.SelectSingleNode("//navigationScheme/mainNav"));
}
During stress testing we identified that the problem code was the if (null != navigationScheme.SelectSingleNode(“selected”)) block. As I was the original developer I struggled to remember why such code was there, given that this node should never exist as the page builds. So, after reproducing the errors locally with the above code I removed the if statement and retested. Not a single error. Quickly browsed the site and didn’t notice any issues (I should have looked harder, there were plenty).
We posted an updated build, stress tested the site, still no errors. Then another developer noticed that the site navigation wasn’t working properly. So, back into the source I went. Digging deep I eventually found that each time the XmlNode navigationScheme = this._xml.SelectSingleNode(“navigationScheme”); was run, the section of the document grew with all previous “choices”.
Through more debugging and some pair programming with a fellow developer (and a lot of discussion about what was really happening) we discovered that, while we were never stuffing the modified xml back into cache, it was indeed being updated. I’m still trying to determine why this is but I think I have read somewhere that XmlDocument objects are in memory structures that are passed by reference (and not by value as I originally thought).
Armed with this information we attempted to clone the document and modify that and while that ultimately worked, it didn’t fit into the system we had for navigation render behaviour. With some changes to how we passed along the modified xml we ended up with this modified method (name changed to better reflect it’s newer intent):
public virtual XmlDocument SelectedStatesXml()
{
/*
<selected>
<choice level="1">Selected L1</choice>
<choice level="2">Selected L2</choice>
<choice level="3">Selected L3</choice>
</selected>
*/
XmlDocument xmlDoc = (XmlDocument)this._xml.CloneNode(true);
XmlNode navigationScheme = xmlDoc.SelectSingleNode("navigationScheme");
XmlNode selected = xmlDoc.CreateNode(XmlNodeType.Element, "selected", "");
for (int i = 0; i < base.Selected.Count; i++)
{
XmlElement choice = xmlDoc.CreateElement("choice");
choice.SetAttribute("level", (i + 1).ToString());
choice.InnerText = base.Selected[i].ToString();
selected.AppendChild(choice);
}
navigationScheme.InsertBefore(selected, xmlDoc.SelectSingleNode("//navigationScheme/mainNav"));
return xmlDoc;
}
Be sure to note in the code above, the call to this._xml.CloneNode(true); as this is important. We are doing a deep copy of the entire XmlDocument. Manipulation is done strictly to the clone and never the master xml document. So now when we call out to render navigation, it looks like this:
return base.OutputNavigation(
new XslTransformBehaviour(
((XmlNavigation)this.Navigation).SelectedStatesXml(),
XslHelper.GetTransformer(path),
((XmlNavigation)this.Navigation).NavigationArguments
)
);
I hope this helps someone someday. I don’t think this behaviour is something I’m going to forget any time soon.
Posted in Development | Tags asp.net, caching, xml | 1 comment | no trackbacks
Posted by Kyle Heon
Fri, 21 Aug 2009 00:26:00 GMT
Nearly two years ago I blogged about an upgraded version of TeamAgile’s DataRollback attribute. Well now I’m doing it again, but this time to say that I’ve upgraded the source for VS.NET 2008, .NET 3.5 and NUnit 2.5.
Refer to the previous blog post for details on how to use this. Grab the binaries or source below:
Posted in Development | Tags agile, database, tdd, teamagile | no comments | no trackbacks
Posted by Kyle Heon
Wed, 08 Jul 2009 22:26:00 GMT
About 4 or 5 years ago I used CodeSmith to automate the creation of our data access layers at PixelMEDIA, Inc. which have served us well up until now. Recently we revisited these templates and made dramatic changes to bring our data access layer current with current development trends.
Even still, we are always on the lookout for better ways to generate the standard “CRUD” code. Today I came across PLINQO which really looks to improve many of the short-comings of Linq2Sql. I hope to get a chance to play around with it a bit in the near future.
Posted in Development | no comments | no trackbacks
Posted by Kyle Heon
Tue, 20 May 2008 23:56:00 GMT
Early in 2008 I decided to build a new, super beefy system to consolidate my old desktop PC and a small development Fedora Core server. My intention was to go 64-bit from the start so that I could load the system up with 8GB of RAM.
Due to a strange memory issue I was not able to run Windows Server setup with all 8GB of RAM in place, it was weeks before I finally discovered the problem. This system currently has 2×2GB chips in and things work, but if I add in the additional two chips, Server 2003 doesn’t get past the “Starting Windows” screen. Weird.
For the past week or so I have been playing around VMWare Server just to get a feel for how things work. I’ve created virtual appliances running Windows XP Professional, Ubuntu Server, FreeNAS, OpenFiler and also OpenSolaris 2008.05.
From what I have seen so far, VMWare Server is simply one of the coolest technologies I’ve played with in a long time. I can’t believe how simple it is to create new appliances, install the OS into it and start it up.
One of my goals with VMWare Server here is to create a really streamlined WinXP Pro install that I can save a snapshot of but start up slices for my children to play in and install whatever they want. If things go south, I just delete the appliance and start over with the fresh WinXP Pro.
My final goal with this system was to setup a redundant storage solution for my growing library of MP3’s and all my RAW photos. For this I went with a motherboard that had RAID5 onboard and bought 3 500GB 7200RPM Western Digital drives, giving me about 1TB of RAID5 storage.
I want this storage array to be as simple to use as possible and will act as the storage device for all of the appliances running plus the aforementioned music and photos.
Thus the reason I have setup appliances with FreeNAS, OpenFiler and OpenSolaris. None of which has worked out quite like I hoped.
- It looks like FreeNAS is really designed to run from a USB drive and as such I was not able to really get it working.
- OpenFiler looks really promising but the interface is very difficult to navigate and while I have figured out how to get “shares” setup through the interface I can’t figure out how to access them from another system.
- And finally, I looked at OpenSolaris specifically to leverage the incredibly awesome ZFS. The problem is, OpenSolaris is wicked slow, at least when run through VMWare. I’m hoping it’s because I’m doing something so I’ll keep exploring.
- If all else fails, I guess I’ll format the array with Server 2003 and create shares as needed. I’d imagine that will mean I’ll need to use the FAT32 file system so that non-Windows nodes can access it.
All of this exploration has been a lot of fun. The issues I ran into getting things built and installed was painful but now I know more then I did before.
Posted in Development | no comments | no trackbacks
Posted by Kyle Heon
Mon, 17 Dec 2007 01:22:00 GMT
I’m currently knee deep with a complete rewrite for one of our large e-commerce sites in .NET 2.0. Part of this rewrite has us finally getting unit tests into the solution.
For the past week I have been working on trying to get a framework in place that allows us to load suitable test data into the testing db for proper unit testing. For this system it is necessary due to all the intricate relationships between products, styles, skus, orders, order line items, etc.
I have been working on a framework similar to what Rails TDD offers, where you specify the fixtures required but continue to run into a strange issue that causes the code to run multiple times for each fixture, even though there is only one test in the fixture.
In an effort to get something in place for this week I began exploring what Team Agile (found via/from Roy Osherove’s ISerializable blog). This has been coined XtUnit.
Even though I met the min requirements of having COM+ 1.5 support (Windows XP SP2 OR Windows 2003) I was not able to get things to work. At first it turned out that I wasn’t sub-classing ExtensibleFixture but then it didn’t like the version of NUnit I had.
As it turns out the source was provided using VS.2003, compiled against NUnit 2.2.1 (2.2.x for sure, believe it was 2.2.1). I’m using NUnit 2.2.8 at this point so I not only upgraded the solution to VS.2005 but also recompiled using NUnit 2.2.8 and so far I think things are working quite well. The DataRollback attribute is working perfectly.
This will work for now in that it at least ensures that tests are atomic, not affecting other tests. It does not however help me ensure that we have appropriate test data in the system. That currently is handled via a nant script that clones the db structure from our dev db into a new local test db and then a data.sql that is run to load initial lookup data.
To make it easier for anyone struggling with the same issues I am providing the upgraded source as well as recompiled binaries for download. You can get them below.
If anyone can explain to me why when adding an attribute at the class level would cause this code to be executed multiple times (with just one test in the test fixture) I would love to hear about it in the comments.
UPDATE: 8/20/2009 – A newer build of this is available here.
Posted in Development | Tags agile, database, related, tdd, teamagile | 2 comments | no trackbacks
Posted by Kyle Heon
Sun, 23 Jul 2006 20:30:00 GMT
So the Typo team has released the long overdue Typo 4.0 upgrade. This latest version includes countless bug fixes and feature additions. Read about it here if you are interested.
Also with this upgrade I’ve rolled out a new focus area on the article detail page. I was inspired by the awesome Hemingway theme but wasn’t able to pull off exactly what I wanted with Typo (as easily as I’d have liked). What I would like to be able to do is easily flag sidebars per page type (list, archives, detail, etc). Speaking of the Hemingway theme, I’ve it out but couldn’t get it to display quite to my liking, plus with sidebars enabled it threw a Rails Application Error that I haven’t tracked down. I’m new to Ruby and Rails and can’t figure out how the sidebar code works.
Posted in Development | Tags rails, ruby, typo | no comments | no trackbacks
Posted by Kyle Heon
Wed, 19 Jul 2006 20:45:00 GMT
The Typo team has been working hard for the past couple of weeks to get to that, thus far, elusive 4.0 release. With any luck it should be out in the next couple of weeks (maybe sooner). I’m keeping my development copy of this site updated on a pretty regular basis. One of the big things this new release offers is a HUGE memory leak bug fix. That alone will be worth the effort. Another nice feature coming is better comment spam protection. I don’t get many comments on this site and therefore don’t have a comment spam problem but I’m sure there are others that do.
I’ve also been making some small but cool design changes over the past few months which I’ll roll into a 4.0 upgrade release when the time comes.
Posted in Development | Tags typo | no comments | no trackbacks
Posted by Kyle Heon
Thu, 13 Jul 2006 20:31:00 GMT
Being relatively new to Ruby and Rails I’ve been taking it slow as I work through an application that I’m building, mostly for fun but has applicability when finished where I work.
Anyways, I have a series of multiple select boxes that exist on a particular form. In all my research online I wasn’t able to find a helper method that would handle multiple select boxes. Given that I decided to write a helper myself. At the moment this just writes out just the option tag and selects it if the value matches something found in a passed in collection.
Here is the helper code:
def option_tag(value, title, collection)
begin
if (!collection.empty? and collection.find(value))
selected = " selected=\"selected\""
end
rescue
selected = nil
end
"<option value=\"#{value}\"#{selected}>#{title}</option>"
end
Here’s how to use it:
<p><label for="browser_id">Browsers</label><br />
<select id="browser_id" name="browser[id][]" size="5" multiple="multiple">
<% for browser in @browsers %>
<%= option_tag(browser.id, browser.name, @item.browsers) %>
<% end %>
</select>
</p>
I’ll probably look into building something that can handle a multiple select and reuse as much of the Rails helper methods as possible.
If I completely missed something and can actually do what I want using what Rails offers please let me know, I’d rather use what is there then roll my own unless I have a good reason to.
Also, should you see something glaringly wrong or inefficient in my code above please say so, as I said, I’m just learning Ruby and Rails and want to learn how to do things the right way.
Posted in Development | Tags rails, ruby | 1 comment | no trackbacks
Posted by Kyle Heon
Sun, 25 Jun 2006 02:05:00 GMT
It’s been super chaotic here in the Heon household with end of school celebrations, father’s day, and multiple family birthdays (father-in-law, my daughter, and my wife) all falling within the same two-week period. Finally we can come up for some air as we are all done with my daughters sixth birthday party. It’s amazing how loud and crazy things get when you add three girls to the mix. I almost think it’s easier with the boys.
So what else have I been up to? Plenty. We bought a new vehicle about a week ago. We’d outgrown our Ford Focus wagon and decided that the time was right to upgrade to a minivan. Thankfully we had a cousin that sells vehicles at a Dodge dealership in the area so we headed over there and he took care of us. We picked up a fully loaded 2006 Dodge Grand Caravan SXT with 11,700 miles on it. Not new but pretty close. When I say fully loaded I do mean it, all it’s missing is heated leather seats (has cloth seats) and the kitchen sink (smirk). Automatic doors, dvd player, 3.8L V6, stow-n-go seating (really sweet), 6-disc dvd/cd changer, 6 (or 8, not really sure) speaker system. We take an annual trip to the family down south (North Carolina) and this will make it much more comfortable.
Other then that it’s been mostly business as usual. I have been listening avidly on the Radiant CMS mailing list and have been exploring this as a replacement for Typo as it seems to be becoming a bit bloated. Everytime I go to upgrade it I inevitably run into issues, be them migration issues or textile/filter related. Additionally my blog seems to be going down on a weekly basis lately. Not sure if that is TextDrive going up/down or Typo simply consuming too much memory and the process being shutdown. I’m also monitoring Mephisto as a possible replacement. Ideally what I want to do is make it possible to continue to have something that makes it easy to post but also flexible enough to let me create a place to post my photography, something that I’m not really finding easy/possible with the current setup.
Speaking of Rails, I picked up a copy of Rails Recipes (for Father’s Day) and I just ordered a copy of Ruby for Rails which recieved a very good review on one of the blogs I subscribe to. It seems to fit well for me simply because I’m trying to learn both Ruby and Rails at the same time and really struggling to grasp either of them (this is probably due to the fact that I only get to spend a few hours a day, sometimes a week, exploring it).
Posted in Development, Family & Friends, Miscellaneous | no comments | no trackbacks
Posted by Kyle Heon
Thu, 27 Apr 2006 21:28:00 GMT
So today is my 29th birthday and my wife gave me the green light to finally order a laptop. I’ve been researching them for about a week now having a tough time deciding on what I wanted/needed and what I was willing to spend and could afford.
I can’t afford one outright so I had to find a deal where I could finance the purchase. Dell offered 6 months at 0% but that would have made my monthly payments around $500 which is just way too much. For their XPS systems however they were offering 18 months at 0% so I configured one of the entry level XPS M1710 systems, only upgraded to XP Pro and added 2 year hardware and CompleteCare protection, the rest is standard.
Here are the full specs:
- XPS M1710, Intel Core Duo Processor T2400 (1.83GHz/667MHz FSB)
- 17 inch UltraSharp TrueLife Wide-screen WUXGA with Black LCD Cover
- 1GB, DDR2, 667MHz 2 Dimm
- 256MB NVIDIA GeForce Go 7900
- 60GB 7200RPM SATA Hard Drive
- Microsoft Windows XP Professional w/SP2
- Integrated 10/100 Network Cardand Modem
- 8X DVD+/-RW Drive
- Integrated High Definition Audio
- Intel PRO/Wireless 3945 802.11a/g Mini Card (54Mbps)
- 80 WHr 9-cell Lithium Ion Primary Battery
It’s pricey and I hope that I don’t regret this purchase but from the reviews I’ve read, the XPS M1710 doesn’t disappoint.
I haven’t had a system powerful enough to play games in years. This system should handle all of the current gen games with ease which is cool because there are plenty I’ve wanted to try out.
Now I need to pick up some supplies, like a carry case, maybe an optical mouse, some headphones (ear buds), an adapter to power the laptop in the vehicle, etc.
The reason I went with such a powerful system is that I want to be able to make this my primary system and that means I need to be able to develop on it and also run Photoshop for on-site photo manipulation.
Posted in Development, Family & Friends, Miscellaneous | Tags birthday, dell, laptop, xps | no comments | no trackbacks