Thought Flow

Tag: .NET

  • Fixing an IIS deadlock

    Found the below post in my drafts section. I should have released it a year ago but better late than never. Maybe someone with a deadlock problem on IIS and .NET 4.0 will find this piece of information useful.

    At the place where I am contracting at the moment, we recently (April 2013) went live with an intranet web application that needs to handle about 1000 concurrent users. This is handled by 8 web-servers running Internet Information Services (IIS) 7.5 and ASP.NET 4.0 and in the beginning, all was well.

    After running for about a day, we started seeing deadlocks in the IIS worker processes. At its worst, it happened every five minutes. When a worker process deadlocks, it recycles itself and the system could not handle that. A Microsoft support ticket was opened and the problem was eventually solved by adding the following code in a random .cs file in all of the assemblies (i.e. projects) for the application:

    [System.Runtime.InteropServices.StructLayout(System.Runtime.InteropServices.LayoutKind.Explicit)]
    struct WorkaroundStruct
    {
        [System.Runtime.InteropServices.FieldOffset(0)]
        [System.Runtime.InteropServices.MarshalAs(System.Runtime.InteropServices.UnmanagedType.I4)]
        public int myField1;
    } 
    

    This magically solved the deadlocks. It turns out that it is a known bug in the .NET 4.0 framework but Microsoft was not going to make a hotfix for it. Hmm.

    Deadlocks are always bad but in our case, they were extra bad. All data in the application is fetched through a Windows Communication Foundation (WCF) webservice from a different company. Instantiation of a WCF client can be slow and in our case, it could take 7-9 seconds for one or more of the WCF clients. By the way, while instantiating the client, the CPU goes nuts. Even though we cache the client for each session to improve performance, when the IIS enters a deadlock and resets, all 1000 users have to instantiate a new WCF client and the servers could simply not handle the CPU load.

    It is funny what weird issues we run into with technology.

  • Strange Linq-to-SQL performance when using Count

    Yesterday, I was writing a Linq-to-SQL query and noticed a quite remarkable difference in performance between two very similar queries (seconds versus minutes of running time). I thought it would be worth sharing.

    Basically, I have a bunch of orders that are represented by two SQL tables, called Basket and BasketItem. If an order e.g. consists of the purchase of items A, B and C, the corresponding rows in the tables could be:

    Basket:     ID = 1
    BasketItem: ID = 1   BasketID = 1   ItemID = A
    BasketItem: ID = 2   BasketID = 1   ItemID = B
    BasketItem: ID = 3   BasketID = 1   ItemID = C
    

    This is a fairly common way to represent order information in a relational database, establishing a one-to-many relationship between an order and its separate order lines.

    Now, somewhere in the code I am working on, I have an item set (for example { A, B }) and I want to figure out in how many orders these two items occur together. The items are represented in a list:

    // items is given as a parameter
    // but manually initialized here for descriptive purposes
    List<object> items = new List<object>() { "A", "B" };
    

    First version of the solution to the problem looked like this:

    int count = 0;
    foreach (Basket b in Baskets)
    {
      IEnumerable<BasketItem> bItems = b.GetBasketItems();
      if (items.All(item => bItems.Any(bItem => bItem.ItemID.Equals(item))))
        count++;
    }
    return count;
    

    If the BasketItems for a Basket contains all the given items, increment a count. This is obviously not optimal since it requires all BasketItems to be fetched from the database. After a lot of pondering and test, I came up with the following Linq query:

    int minLines = items.Count;
    var count =
      from basket in Baskets
      where
        (from basketItem in BasketItems
         where basketItem.BasketID == basket.BasketID &&
         items.Contains(basketItem.ItemID)
         select basketItem).Count() >= minLines
      select basket;
    return count.Count();
    

    If the query finds items.Count or more BasketItems for a given Basket, the basket is selected and the number of valid baskets are returned in the end. This dropped the running time from ~50 seconds to ~8 seconds for about 7000 calls to the method with the above code (for different items lists), a nice improvement.

    I then thought that the above Linq query could be further improved by using Count() directly instead of using Where clauses. It then rewrites to:

    var count =
      AnteconsBaskets.Count(basket =>
        AnteconsBasketItems.Count(basketItem =>
          basketItem.BasketID == basket.BasketID &&
          items.Contains(basketItem.ItemID)) >= minLines);
    return count;
    

    Using the above code, I had to manually halt the program after waiting more than 10 minutes for execution to complete.

    The question now is: Why did this code perform so poorly compared to the code in listing 2? I cannot offer an explanation of this but I can hint at it. The code in listing 2 translates into an SQL query where COUNT(*) is used in the sub-query for the BasketItem selection. For listing 3, Count() translates into an SQL query that uses a SELECT CASE WHEN query with three cases. Why this is so slow, I do not know but I will definitely not use Count() like this in the future.

  • Sorting an ASP.NET GridView with a custom data source

    The other day, I had to present some data in an ASP.NET GridView control and I had to allow sorting of some of the columns. This is easy if the data source is based on e.g. an SQL data source but not particularly easy when using a custom data source. In my case, I had a simple array of custom objects containing invoice data.

    There are many solutions for this problem on the web but none of them really seemed to do the trick in my case. Many of the solutions I could find rely on converting the data source into a DataTable object but this is not a good idea when sorting needs to be applied to many fields because the DataTable needs to have these fields added manually.

    So without further ado, here is a simple solution for the problem. First, let’s say our custom data source consists of an array of Person objects, defined like this:

    class Person {
      string Name { get; set; }
      DateTime BirthDay { get; set; }
    }
    

    The GridView to bind the persons to is very simple:

    <asp:GridView ID="PersonGridView" runat="server"
      AutoGenerateColumns="false"
      AllowSorting="true"
      OnSorting="personGridView_Sorting">
      <Columns>
        <asp:BoundField DataField="Name" SortExpression="Name" />
        <asp:BoundField DataField="BirthDay" SortExpression="BirthDay" />
      </Columns>
    </asp:GridView>
    

    The SortExpression has to be exactly the same name as the field. The sort direction and sort expression are stored in the ViewState and accessed by two fields in code-behind:

    private SortDirection CurrentSortDirection {
      get {
        if (ViewState["SortDirection"] == null)
          ViewState["SortDirection"] = SortDirection.Ascending;
        return (SortDirection)ViewState["SortDirection"];
      }
      set { ViewState["SortDirection"] = value; }
    }
    
    private string CurrentSortExpression {
      get {
        if (ViewState["SortExpression"] == null)
          ViewState["SortExpression"] = "Name";
        return (string)ViewState["SortExpression"];
      }
      set { ViewState["SortExpression"] = value; }
    }
    

    Notice that I have chosen to have Name sorted in ascending order by default. This is of course not necessary but convenient for later. Next up is the OnSorting event:

    protected void personGridView_Sorting(object sender, GridViewSortEventArgs e) {
      CurrentSortExpression = e.SortExpression;
    
      // Ignore e.SortDirection since it is always Ascending, an apparent bug in .NET
      if (CurrentSortDirection == SortDirection.Ascending)
        CurrentSortDirection = SortDirection.Descending;
      else
        CurrentSortDirection = SortDirection.Ascending;
    
      BindPersonData();
    }
    

    Finally, we can write our binding method.

    private void BindPersonData() {
      Person[] persons = GetPersonArrayFromSomewhere();
    
      // Sort the persons, if sorting is applied.
      if (!string.IsNullOrEmpty(CurrentSortExpression)) {
        System.Reflection.PropertyInfo personInfo = (typeof(Person)).GetProperty(CurrentSortExpression);
        if (CurrentSortDirection == SortDirection.Ascending) {
          persons = persons
            .OrderBy(person => personInfo.GetValue(person, null))
            .ToArray();
        }
        else {
          persons = persons
            .OrderByDescending(person => personInfo.GetValue(person, null))
            .ToArray();
        }
      }
    
      PersonGridView.DataSource = persons;
      PersonGridView.DataBind();
    }
    

    The sorting can also be applied with a different sorting method. I happen to like Linq so that is why I have used it here. The key thing to notice here is how the correct field for sorting is found with the use of the reflection framework:

    System.Reflection.PropertyInfo personInfo = (typeof(Person)).GetProperty(CurrentSortExpression);
    

    That is why it is essential that the SortExpression value in the GridView is the same as the field’s name.

    Oh and by the way, paging with a custom data source is very easy. Add this to your GridView:

    AllowPaging="true"
    OnPageIndexChanging="personGridView_PageIndexChanging"
    

    and this method to the code-behind:

    protected void personGridView_PageIndexChanging(object sender, GridViewPageEventArgs e) {
      PersonGridView.PageIndex = e.NewPageIndex;
      BindPersonData();
    }
    
  • Basic Http authentication from .NET to PHP webservice

    Today, I had a major problem that had a not-so-complex-but-rather-stupid solution so I thought it might be worth sharing since I only found online solutions that were rather old.

    I have remote access to a PHP webservice, written using NuSOAP. I call the methods of this webservice from a .NET application. This is no problem at all and I can easily call most webservice methods from within .NET. But some of the methods require basic authentication and now the trouble begins.

    One would think that it would be enough to just set the credentials for the webservice and make sure that it authenticates on every request. Like this:

    MyPhpWebservice ws = new MyPhpWebservice();
    ws.PreAuthenticate = true;
    CredentialCache cache = new CredentialCache();
    cache.Add(
       new Uri(ws.Url),
       "Basic", 
       new NetworkCredential(wsUsername, wsPassword)
    );
    ws.Credentials = cache;
    ws.CallSomeFunctionThatNeedsAuth();
    

    Alas, this did not work. In many places online, people say that .NET does not actually send the authentication details on the first request for some webservice function. If this was true, the solution would simply be:

    ws.CallSomeFunctionThatNeedsAuth(); // Does not auth
    ws.CallSomeFunctionThatNeedsAuth(); // Should auth now?
    

    This did not work either. Other people in other places online say that one needs to manually add the authentication details to the HttpWebRequest header. But the Visual Studio auto-created MyPhpWebservice class does not expose its WebRequest object so it cannot be directly altered. However, we can override the method GetWebRequest for the webservice and that way manually add the authentication header. This took me a while to figure out but here is a simple solution:

    class MyOverriddenPhpWebservice : MyPhpWebservice
    {
        protected override System.Net.WebRequest GetWebRequest(Uri uri)
        {
            HttpWebRequest request;
            request = (HttpWebRequest)base.GetWebRequest(uri);
    
            if (PreAuthenticate)
            {
                NetworkCredential cred =
                    Credentials.GetCredential(uri, "Basic");
                if (cred != null)
                {
                    byte[] credBuffer = new UTF8Encoding().GetBytes(
                        cred.UserName + ":" + cred.Password);
                    request.Headers["Authorization"] =
                        "Basic " + Convert.ToBase64String(credBuffer);
                }
            }
            return request;
        }
    }
    

    The code that I wrote in Listing 1 is now perfectly valid and all I need to do is exchange MyPhpWebservice with MyOverriddenPhpWebservice.

    And the best part: It works :-)