Castle Windsor, Service Locator and a possible memory leak !!!

In recent days, in one of my projects, we found an innocently looking bug that was a little too overweight – sometimes weighing more than 6GB in size in a 8GB web server. Well, a memory leak that caused the application pool to crash by exhausting the system memory. So I thought that, why not share it here so that if anyone else, facing the same problem, can get some insight. Here goes the plot –

The main goal of the project was to build a modular framework to be used to build robust web applications easily. For that reason the most obvious choice was to use Inversion of Control Pattern.

If you are already here then it is highly normal that you are familiar with the IOC( Inversion of Control) pattern already, but if you are not, then here is a good article for you.

As you all know, castle windsor is a widely used service locator library we used it for our IOC implementation. Therefore, we registered our services like this –

_container.AddComponentLifeStyle<IDocumentService, DocumentService>(LifestyleType.Transient);

If you are new to castle windsor and the life cycle of the items it resolves, grab its details from here.

In short, LifestyleType.Transient means each time you ask castle windsor to resolve or give you an instance of an interface it will create a new one each time. But the default implementation of castle windsor with transient life cycle does the following –

  • It creates a new instance each time ask for
  • Keeps a reference of that instance created, so that it can refer it later
  • Assumes that you will release the instance manually, again remember it, manually.

The first two options is not a problem, but the third one is a possible leak. If in any case you do not release the instance manually its a never-ending memory leak. Because the default garbage collector of CLR will never clear it since castle windsor is holding a reference and castle will never clear it because it thinks you will clear it yourself.  That’s what we exactly did and well it took only 2 hours to consume all the server memory (6GB +) and crash it.

If you are interested about the possible memory management scenarios, see this article here, I realized our problem reading this one.

Well, now comes the big question – Whats the easiest solution?

Well you can release all the instance manually that you resolved using castle windsor or you can grab the solution mention here.

I will save your time. Just in the castle service locator where you are creating the kernel, add the following lines –

_container.Kernel.ReleasePolicy = new NoTrackingReleasePolicy();

Basically what it does is that, it prevents castle windsor from keeping any references of resolved instances and thus when you are done with your code and the object needs releasing the default GC collects it and the memory is freed. Which removed the memory leak problem. And you know what now the memory consumption never goes over 600MB. 🙂

It’s highly usual that you are using NHibernate with castle windsor and you think NHibernate is causing the leak?.. well don’t be harsh on NHibernate ; its castle and your unreleased instances, who is causing the leak. 😀

Advertisements
Posted in C# .Net, Castle, Native and Third Party Frameworks for .NET | Tagged , , , , , | Leave a comment

Managed Extensibility Framework –

Introduction

Before I begin lets discuss about a common development scenario and how we might deal with it –

Suppose we have a requirement as follows (This was a real-time scenario once for me )-

  1. The system will collect table data from flat files,  like – plain text (txt).
  2. The task is to collect column information and show the list of columns in a Grid.

Now, lets consider we have started our development process and also completed it and delivered. Your client is happy and so are you.

Now, comes the tricky part. A few days later your happy client asks –

 could you please add an option so that we can also parse Excel WorkSheets, such as xls, xlsx, xlsm, xlsb files or may be delimited CSV files?

If you never anticipated this would happen, I assume you didn’t keep an option to add a parser later. Now what should you do? You will open the source again and modify it and them again build and deliver it. For a small project like this it is still troublesome, there are other projects with millions of lines of codes. What will you do with them? Open the source again and build?

Oh Boy, ain’t that tiresome!!!

Lets, talk about another approach –

Instead of hard coding the parser in the application could we could have done the following –

  1. Create a Plugin directory and keep a parser as a portable assembly (.dll) which will have a generic interface to expose methods.
  2. When executed the program will search for available plugins in that folder and will see that it has Text (.txt) file parser.
  3. Then the program will issue commands or all methods based on exposed interface of the parser.

Now, in this situation even if my client wants more parser, we can just build a new one implementing the base interface and build it as a class library and finally just put it in the Plugin folder and the parse will work out of the box.

Now, I think you have got a rough idea of what I am talking about. So lets, just mention it in words –

Using Managed Extensibility Framework(a.k.a MEF) we can do the above mentioned thing in minutes. In other words, we can inject business logic right into the application and will not even need rebuild it.

Before we start coding, you might wanna know where to get it. If you are using .net framework 4.0 you already have it and for those who are using 3.5, you can get the source from http://mef.codeplex.com/ and add a reference to the sources to get the feature sets. I haven’t tried with other versions of .net framework but if you still need to know, post me a comment and I will dig a little more deeper to get you the info.

 

The Solution

We should create and organize out solution first.

Here is the basic process that I followed for this session –

  • Open MS Visual Studio ( I am using 2010 ultimate and will use C# + .net framework 4.0 for this project) and create a Class Library project. I would name this project MEFIntro.Lib.
      I use “.” extended project names to keep the names meaningful and  easy to mange. Its not a restriction
  • Remove the default class file “Class1.cs” from the project as we will not be needing this and we will create our classes ourselves.
  • So far we have created the library project. We also need an executable project that we can use to show the output of the library. So lets add another project named MEFIntro.Desktop and this would be a Windows Forms Application. I also renamed the form to “MainForm“.
  • Add a reference from MEFIntro.Lib to MEFIntro.Desktop.
  • This is how your screen should look like –
  • Now we will need one more project, that we will use to create Extensions. Lets name it MEFIntro.Lib.Ex.
  • Add reference for the MEF library (System.ComponentModel.Composition) to both of your projects – MEFIntrol.Lib and MEFIntro.Lib.Ex.
  • Our final solution will look like this. We have all the projects that we need for this tutorial. If more is needed we will add later

 

The Extensible Core

Lets create the core section of our MEF Project.

By the by, I have downloaded and installed MS Visual Studio 11 Beta and loving it very much. So, from now on all other tutorials and blogs will be based on MS Visual Studio 11 Beta. Hope you will like the IDE too, its loaded with cool staffs….. 🙂

The core engine of our project will be responsible for the following tasks –

  1. Load all plugins and extensions those we will create and keep as plug & play components.
  2. Expose methods that our library will support.
  3. Provide Options and Interfaces for future plugin and extension development.

Okay, enough high level blabbering, lets jump into more low-level description. To support all the features mentions above we need to expose some interfaces, so that our extensions can inherit them and build their own logic in it, more like a strategic pattern. And at run-time our core engine will collect them using MEF and execute as needed.

When I said “interface” in above paragraph, I literally meant Interface of C# programming language. So don’t be afraid, we are inside the codes. Literally!!!

next, we need to expose some methods to be implemented in the implemented class. Our requirement is to collect column list i.e. metadata, from the files. So lets call this method “GetMetaData“. So our core engine interface will look exactly like the following –

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace MEFIntro.Lib.Interface
{
	public interface ICoreEngine
	{
		String[] GetMetaData(String filename);
	}
}

So far, we have expose the interface that our extension methods and classes will use to integrate themselves with the engine. But we haven’t told your engine about the extensions. In other words, our engine still does not know that there could be extensions to use. So lets add 2 interfaces for that-

  1. The top-level interface of the extensions, all extensions targeting our engine will have to implement that interface, otherwise we will not consider as a plugin. I named it – IFileExtension. Our plan here is to use one extension class for one file of files. Like one class for .txt files and another extension class for .xls files. These classes will implement this interface.
  2. The second interface is for holding information about the extension classes. At run-time MEF will search with these data and give us the correct class to use. This idea will be much clear when you see the code in action. For now, lets just put a pin in it. I named this interface – IFileData
Now, we summarize what we have with a screenshot –

I prefer using a single file for all the interfaces that are under one namespace for all my projects, this helps me finding them later on. This is not a requirement. You are free to implement it your way as long as they all are available when needed.

So far we only have the abstract structure of our engine. But we need some concrete ones. Lets, look at our concrete implementation of our concrete core engine. I will describe the codes shortly –

Lets see what we have here –

  1. We have a concrete implementation of our core interface named CoreEngine.
  2. We have overridden the GetMetaData method. For now it is kept blank, we will add implementation logic later.
  3. The class also has some interesting fields, namely –
    IEnumerable<lazy<IFileExtension, IFileData>> operations;
    private CompositionContainer_container;
     This is main container of all the plugins. At run-time the framework will automatically collect our plugins and put them in the operations field. All we have to do is call it in the appropriate manner.
  4. And one other interesting method ComposeParts.
    This method does composition an puts together all the parts. you can understand the codes very easily by looking at it.

So, far we have nothing done with our implementation. Lets complete our core engine and call some extension methods. First we need to add at least one extractor class  for this example. Lets add one for text ( .txt) files. The task is simple just open the files first line, split by the delimiter character and return the string array –

Now since we have one extension that we can call to test our implementation, lets finish the core. First we need to extract the file extension, then check whether we have any extractor defined for this extensions and if we do we will call it and return the value. Our final code for the GetMetaData will look like this –

That’s it, we have the core section complete.

Posted in Managed Extensibility Framework (MEF), Native and Third Party Frameworks for .NET | Tagged , , , | 1 Comment

Chapter 1: Introduction to FluentNHibernate

After working over 2 years with FluentNHibernate, I have now decided I should write a step by step learning schedule for Fluent NHibernate. The reason? … Well lets just say it’s better to wait for it. Once you learn FluentNHibernate, I guess, you will never go back to your old SQL query based development, trust me, you won’t.

When I learned FluentNH, it took me a while to find solutions, as help was not that much available all the time.

But I must be glad to those extraordinary people at stackoverflow (http://stackoverflow.com/ ) who helped me find solutions to problems and also helped me learn FluentNH.

This is an approach of mine, to bring all those thoughts and solutions in one place, so that you can find all without going under the hassle that I went through.

Before beginning with FluentNHibernate, I would like to introduce you with it a bit.

For those, who have worked with FluentNH earlier, you can skip the next section.

FluentNH is an ORM in long Object Relation(al) Mapper. Now, the big question is, What is a ORM? well keep reading….

  1. What is a ORM?
    In simple words, I would say ORM is an advanced technique which is actually originated from OOP ( Object Oriented Programming) concept and is created on the sole purpose to provide developers like us, the capability to perform persistent store management, such as managing a database, with a complete OOP approach.Confused? Okay lets see an example of what I meant. Consider  a very simple case -” Suppose you have 1 table in database name “User” with 2 fields in it username and password.Now, lets say we have to create a single user to the database, what will be a our approach? Okay, lets say we are creating a huge ERP solution, and thus we will use the 3-tier architecture. So we have Presentation Layer, Business Logic and Data Access Components. No matter what we use, somewhere in the DataAccess layer we will need to create a SQL connection and generate a query of some thing like –Select * From <Table> Where <Condition> Order By <field>This is just for one single table and running a query from it. Think about the whole database with hundreds of tables. Well, its better not to think about it, you know what I am talking about. This is where a ORM kicks in.A ORM is the abstraction layer, that gives you the opportunity to run queries just like normal codes in C#. For example in one of your codes you might use something like this –Person p = Repository.Get<Person>(“Id”, <someid>);

    Thats right, no more sql queries and no more table adapters and no more data sets. That singe line will do the rest.

    If you use a ORM you don’t have to type all the sql codes by hand. You create a nice simple repository class and let the ORM handle the test. And frankly speaking, until I knew about ADO.Entity Framework (v3), I never thought any library or ORM will be able to compete with FluentNH.

I think, thats enough for you to just give you a very rough idea of what is a ORM. But until you use it in your projects, I guess it won’t be of much help.

Stay tuned for the next articles on FluentNH.

Posted in FluentNHibernate + NHibernate, Learning FluentNHibernate + NHibernate (FluentNH + NH), Native and Third Party Frameworks for .NET | Tagged | Leave a comment

How to enable Automatic Reference Counting (ARC) in XCode 4.2

Hi,

First of all, Automatic Reference Counting or ARC in the new Apple LLVM 3.0 compiler is a very interesting feature and I found it very useful, cause, I don’t have to worry about my pointers anymore. But the problem I faced is that, I have a lot of projects built on earlier version of XCode and definitely didn’t have the ARC enabled. So I have to manually enable the ARC for those projects. To enable ARC for your old projects, do the following –

  • Open the project in XCode 4.2
  • Make sure you have the ‘Apple LLVM 3.0 compiler’ selected in your build settings under the “Build Options” section. To open the project settings page, first go to the project navigator pane by clicking the folder icon on the toolbar, then select your project. You should see the project settings page, to open all settings use “All” sub-tab under the “Build Settings” tab. Here is a screenshot that can help you out –

    Enabling Apple LLVM Compiler 3.0

  • Now we need to enable the ARC. Scroll down more until you can get the section name “Apple LLVM Compiler 3.0 – Language” section. Now go to the 17th item and set “Yes” for its value. Here is a screenshot to help you –

    Enabling ARC

Hope this was helpful for some of you. If it does, posting a ‘thank you’ does not cost too much time, does it? .. 🙂

Posted in IPhone, Mobile Developement | Tagged , , , | 5 Comments

Creating a handsome custom JavaScript library

jQuery, ExtJS, etc are some of the most sophisticated javascript libraries used current days.  But If you are as passionate as I am at some moment of your life you could wish to have your very own independent js library that can be used at any site.

Suppose you have created a custom library named js and want to use it like this ..

js("#id").hide();

You interested?.. keep reading.

Lets go and see how we can accomplish that –

Step 1: Create the foundation

JavaScript is an object oriented language and thus everything we do will be regarding objects. If you are reading this post, then you are already familiar with js and thus know that there are some browser object that javascript uses with DOM. We will need two of them to create our library one is the window and the other is the document object. Every single library or js obejct that you need to used must be registered as a property or field of window  object. So first we need to register our library. I will call my library js and the code that is needed to register the library is like this –

(function() {
   var window = this,
   undefined, _js = window.js;
})();

Lets see what this does. (Between you and me, I got this idea from jQuery) –

  1. The first and last line just defined the closure of the library
  2. The second line just saves the window object ( ofcourse when the library is loaded the this object would be the window itself) for later use
  3. The third line defines the undefined object in case it throws an exception if we use it in codes
  4. The second half says that I have a js names library that is to be used as window.js or simply js. The _js is used just an extra caution incase we need to use the library function inside it.

Step 2: Complete the definition

Uptil now our library js has only been declared, but we can’t use it just yet. We need to add functions and methods inside it so that we can manipulate them as library and also to access our DOM object like a jQuery type syntax we need to have a selector defined. Lets just complete our library as following –

//object initializarion
js = window.js = function(selector) {
    return new jsObject(selector);
};
//the custom javascript object
jsObject = function(selector) {
};

I will explaing what happended here. I just created an object protoype and initialized the js as an object of that type. Therefore js is now a valid javascript object and is ready to be implemented.

Step 3: Add Some features and Hook up with events.

Now before we move forward we have one little thing to care about, our library needs to be called when javascript is initialized. So we have to hook up a load event to the window load event and also bubble any event hooked up by other libraries. You don’t wanna mess up other libraries to create your own one. The code will look like this –

this.jsReady = function(func) {
    var oldonload = window.onload;
    if (typeof window.onload != 'function') {
        window.onload = func;
    } else {
        window.onload = function() {
            if (oldonload) {
                oldonload();
            }
            func();
        };
    }
};

What we did here? We first checked that if there is already a function hooked up to be call when the window loaded, if yes we called it first then ours and if not then just hook our function with it that will be called automatically.

Step 4: Putting it all together.

Thats it! We just created our very own js library. Don’t believe me? Lets have a test. But for that we need some functions that we are going to use to test. Lets just add two more function hide() and show(). I will use jQuery as internal library for easy test and also use jQuery selectors to pass the dom objects to the library. So our library will look like this –

(function() {
    var window = this,
    undefined, _js = window.js;

    //object initializarion
    js = window.js = function(selector) {
        return new jsObject(selector);
    };

    //the custom javascript object
    jsObject = function(selector) {
        this.jsReady = function(func) {
            var oldonload = window.onload;
            if (typeof window.onload != 'function') {
                window.onload = func;
            } else {
                window.onload = function() {
                    if (oldonload) {
                        oldonload();
                    }
                    func();
                };
            }
        };

        //the hide function to test our code
        this.hide = function (){
            $(selector).hide();
        };

        //the showfunction to test our code
        this.show = function(){
            $(selector).show();
        };
    };
})();

Step 5: Using the library.

Its simple to use the library, just make sure you include jQuery before including our library and try to use the codes as this –

js(document).jsReady(function(){
   js($("< your selector id, class or anything>")).hide();
});

Have fun….

Posted in Creative JS, JavaScript, Scripting Language | Tagged , , , , , , | 4 Comments

Uploading files from iPhone to ASP.net web site

Hello All,

As a coder sometimes I have to crush my head on the desk for not finding a feasible solution to a problem. One problem that almost had me finished is uploading image files from iPhone to asp.net website. Both the website and iPhone application was developed by me and thus I had to find both the solution –

  • Uploading file data to the site
  • Then collecting that data and saving to folder
  • Passing additional parameters so that I can run security checks about the user who is uploading the file and also track for which item the file is being uploaded.

After Googling a lot I came to two very important posts at stackoverfow that helped me figure out the process of doing this. It was a long time ago so, I can’t remember the addresses of the post, so I just thought I can summarize the full process so that anyone who is about to break his skull can get some help. Ok, enough talk, lets see the process.

1. Selecting files from iPhone

First you need to able to select files from your iPhone. Anyone who has come this far can easily design a view that can select a file with the UIImagePickerControl. I used a UITextBox that was used to select the image. When user click on the textbox the image picker is shown and user then selects the image. After selecting the image the image is shown in a UIImageView under the textbox for review. Here is the code sample that does it ( I have removed unnecessary codes, so just copying won’t work, you have to merge it).

- (BOOL) textFieldShouldReturn:(UITextField *)textField{
[textField resignFirstResponder];
return TRUE;
}

- (void)textFieldDidBeginEditing:(UITextField *)textField{
if(textField == filename){
[description resignFirstResponder];
[filename resignFirstResponder];
//show file select dialog

UIImagePickerController * imagePicker = [[UIImagePickerController alloc] init];
[imagePicker setDelegate:self];
#ifdef __i386__
[imagePicker setSourceType:UIImagePickerControllerSourceTypeSavedPhotosAlbum];
#else
[imagePicker setSourceType:UIImagePickerControllerSourceTypeCamera];
#endif
[self presentModalViewController:imagePicker animated:YES];
}
}

- (void)imagePickerController:(UIImagePickerController *)picker
didFinishPickingImage:(UIImage *)img
editingInfo:(NSDictionary *)editingInfo{
[picker dismissModalViewControllerAnimated:YES];
[self.image setImage:img];

}

- (void)imagePickerControllerDidCancel:(UIImagePickerController *)picker{
[picker dismissModalViewControllerAnimated:YES];
}

Don’t worry about the #ifdef used here, this is just a check that If the app is running from the simulator then it will not throw an exception. Here textField is the text field used to select the image and self.image is the UIImageView used to preview the image to the user. This also has a hidden use, I also used it as the image data cache when sending the file data to the server.

2. Sending file data to server

At the server, I just used a .aspx file to catch the uploaded file and then write a response to indicate if the file was collected or not. I will discuss about the page later, lets just see how we are to send the data to the server.

Once, the user selects the file, the image data is saved in the UIImageView used to preview the image. I just collected the data from the UIImageView and sent it to the server by manually appending into a HTTP post request.

- (IBAction) Upload{
NSData *imageData = UIImagePNGRepresentation([self.image image]);
NSString *urlString = [NSString stringWithFormat:@"http://%@/iphone/Upload.aspx?wid=%@&sskey=%@",[[SettingsManager instance] nurl],[[SettingsManager instance] wid],[[SettingsManager instance] sskey]];
NSMutableURLRequest *request = [[[NSMutableURLRequest alloc] init] autorelease];
[request setURL:[NSURL URLWithString:urlString]];
[request setHTTPMethod:@"POST"];

NSString *boundary = @"---------------------------14737809831466499882746641449";
NSString *contentType = [NSString stringWithFormat:@"multipart/form-data; boundary=%@",boundary];
[request addValue:contentType forHTTPHeaderField: @"Content-Type"];

NSMutableData *body = [NSMutableData data];
[body appendData:[[NSString stringWithFormat:@"--%@\r\n",boundary] dataUsingEncoding:NSUTF8StringEncoding]];

[body appendData:[[NSString stringWithString:@"Content-Disposition: form-data; name=\"userfile\"; filename=\"ipodfile.jpg\"\r\n"] dataUsingEncoding:NSUTF8StringEncoding]];
[body appendData:[[NSString stringWithString:@"Content-Type: application/octet-stream\r\n\r\n"] dataUsingEncoding:NSUTF8StringEncoding]];

[body appendData:[NSData dataWithData:imageData]];

[body appendData:[[NSString stringWithFormat:@"\r\n--%@--\r\n",boundary] dataUsingEncoding:NSUTF8StringEncoding]];

[request setHTTPBody:body];

NSData *returnData = [NSURLConnection sendSynchronousRequest:request returningResponse:nil error:nil];
NSString *returnString = [[NSString alloc] initWithData:returnData encoding:NSUTF8StringEncoding];

if([returnString isEqualToString:@"Success"]){
UIAlertView * alert = [[UIAlertView alloc] initWithTitle:@"Upload" message:@"File successfully uploaded" delegate:self cancelButtonTitle:@"Ok" otherButtonTitles:nil];
[alert show];
[alert release];
}else{
UIAlertView * alert = [[UIAlertView alloc] initWithTitle:@"Upload" message:@"File upload failed" delegate:self cancelButtonTitle:@"Ok" otherButtonTitles:nil];
[alert show];
[alert release];
}
[returnString release];
}

The task is pretty simple, just grab the image data from the imageView, then append manually with the boundary, set the content type for the HTTP request and post the data. Don’t worry about the values used from SettingsManager class, this is a singleton class that I used to save data cache, nurl is the url to be used for sending data, wid is the id that will be associated with the file and sskey is the session key used to check for security. Just ignore the if part here we will discuss it later. We are now halfway through, now all we need a aspx page to save the file content. The next section will do that.

3. Saving file content

The aspx is very simple page with the following code at the page load to save the file content. I took the wid, and sskey from the url and the file from the httpinput stream and saved as necessary.

public partial class iphone_Upload : System.Web.UI.Page
{
protected void Page_Load(object sender, EventArgs e)
{
String wid = Request.QueryString["wid"];
String sskey = Request.QueryString["sskey"];
try
{

//some check for the security

HttpPostedFile file = Request.Files.Count > 0 ? Request.Files[0] : null;
if (file == null)
{
Response.Output.Write("No file found");
Response.Output.Close();
return;
}
else
{
//Get an entry for file upload
string fileName = Utility.GetID(IDType.File);

String[] names = file.FileName.Split('.');
string src = fileName + "." + names[names.Length - 1];

file.SaveAs(Server.MapPath("~/userhud/uploads/") + src);
Response.Output.Write("Success");
Response.Output.Close();
return;
}
}
catch (Exception)
{
Response.Output.Write("Failure");
Response.Output.Close();
}
}
}

The code is simple. Just get the file and other values and manipulate as necessary. One thing you might notice I outputed ‘Success’ or ‘Failure’ directly to the stream and closed it. In this case that If block that we used in 2, will come in hand to notify user about successful or failed uploads. The Utility.GetID() is a method I used to generate a unique id for each file uploaded to overcome the fact that I might overwrite s file while uploading.

4. Conclusioin

In this post I have tried to summarize the process of uploading image files from iPhone to ASP.Net web sites. Hope that this might be helpful for you. Your suggestions are always welcome.

Posted in IPhone, Mobile Developement | Tagged , , , , , , , , , , , | 18 Comments