Why is Exchange Store.exe so RAM hungry?

This seems to be one of most favorite questions ever: "How come that Exchange Store.exe keeps on grabbing more and more memory, even on the server that is not very busy? Is there a memory leak?". This is many times followed by "I am rebooting my server on the weekly basis to keep store memory consumption in check".

Well, let's see what we have here.

It is absolutely normal for Store.exe to grab as much RAM on the server as it can possibly get - as long as Store thinks it needs it to optimize performance. Store was written to do so. It does this as it wants to do as much stuff in memory as possible, without having to go to the slow page file. It is a common misconception - that Store.exe's increasing memory consumption is a "memory leak". This behavior is expected since Exchange 5.5 days:

182505 XADM: Memory Usage of Store.exe Is Higher in Exchange 5.5
http://support.microsoft.com/?id=182505

That being said:

1. Exchange Store will grab as much RAM as it can if it thinks it needs it, yes. But - we constantly monitor the performance of the system in regards to memory usage and we can use this data to infer when we need more memory and when other applications or the OS needs more memory. We then use this data to act accordingly. This scheme allows the system to act as if there is explicit control when in fact it is actually a few autonomous applications cooperating in a disconnected manner. That means that we should NEVER see a "out of memory" message by any application on the server because of the Store - unless there is a leak on the server, of course... or the page file is too small. If there was a malfunction in this Store mechanism it would cause a lot of paging. That is a big performance problem, but shouldn’t cause actual errors.

2. Store memory utilization can go up to 1.2 GB or sometimes even more when viewing through Task Manager. I have not seen it go over 1.5 GB, but it would typically not take more, even if there is 8 GB of RAM in the machine. So - Store taking 1.2 GB is not an indication of the problem all by itself.

3. Exchange Store is not the only product behaving like this... SQL does something very similar, for example. That is one of reasons why we do not necessarily encourage putting SQL and Exchange on the same server, as they will be fighting over whatever RAM is in the server.

All of the above being said, let's go into what can be manifestations of real problems:

- Are there actual performance problems that might be related to store taking up RAM? For example, other services on the server slow down noticeably when store gets "large", events are logged in the application / system log mentioning lack of memory?

- Does client access to the server slow down when Store.exe grabs a lot of RAM?

- Are there any "out of memory" errors on the server at all? Popups when trying to start applications/services on the server?

If not - there is most likely no problem. Again - Store.exe taking up a lot of RAM is NOT a problem on it's own, as the memory can be returned to the OS when needed by other processes. Store is simply taking advantage of the RAM - as it is in the machine :)

One more note - if Exchange 2003 SP1 is applied to the server and you used to monitor Store memory utilization, you could be seeing an increase in memory consumption unless you change the monitoring settings. This is normal and is covered in more detail here:

867628 Monitoring programs report that the Store.exe process consumes
http://support.microsoft.com/?id=867628

Model–view–controller

MVC was first described in 1979 by Trygve Reenskaug, then working on Smalltalk at Xerox PARC. The original implementation is described in depth in the influential paper "Applications Programming in Smalltalk-80: How to use Model View Controller".

There have been several derivatives of MVC. For example, Model View Presenter is used with the .NET Framework, and the XForms standard uses a "model-view-controller-connector architecture". However, standard MVC remains popular.

Model view controller is both an architectural pattern and a design pattern, depending on where it is used.
As an architectural pattern

It is common to split an application into separate layers that can be analyzed, and sometimes implemented, separately.

MVC is often seen in web applications, where the view is the actual HTML or XHTML page, and the controller is the code that gathers dynamic data and generates the content within the HTML or XHTML. Finally, the model is represented by the actual content, which is often stored in a database or in XML nodes, and the business rules that transform that content based on user actions.

Though MVC comes in different flavors, control flow is generally as follows:

1. The user interacts with the user interface in some way (for example, presses a mouse button).
2. The controller handles the input event from the user interface, often via a registered handler or callback.
3. The controller notifies the model of the user action, possibly resulting in a change in the model's state. (For example, the controller updates the user's shopping cart.)
4. A view uses the model indirectly to generate an appropriate user interface (for example, the view lists the shopping cart's contents). The view gets its own data from the model. The model and controller have no direct knowledge of the view.
5. The user interface waits for further user interactions, which restarts the cycle.

Some implementations such as the W3C XForms also use the concept of a dependency graph to automate the updating of views when data in the model changes.

By decoupling models and views, MVC helps to reduce the complexity in architectural design and to increase flexibility and reuse of code.

As a design pattern

MVC encompasses more of the architecture of an application than is typical for a design pattern. When considered as a design pattern, MVC is semantically similar to the Observer pattern.

Model
Is the domain-specific representation of the data on which the application operates. Domain logic adds meaning to raw data (for example, calculating whether today is the user's birthday, or the totals, taxes, and shipping charges for shopping cart items).
Many applications use a persistent storage mechanism (such as a database) to store data. MVC does not specifically mention the data access layer because it is understood to be underneath or encapsulated by the model.

View
Renders the model into a form suitable for interaction, typically a user interface element. Multiple views can exist for a single model for different purposes.

Controller
Processes and responds to events (typically user actions) and may indirectly invoke changes on the model.

VB.net Class for MD5

So what is it and what does it do, basically you could write a worm pretty easily to verify hashes if you so wanted, none of the information on here is meant to be used for dodgy script kiddies etc, it just illustraites weaknesses and basic programming functions and methods. Here you go….

In cryptography, MD5 (Message-Digest algorithm 5) is a widely used cryptographic hash function with a 128-bit hash value. As an Internet standard (RFC 1321), MD5 has been employed in a wide variety of security applications, and is also commonly used to check the integrity of files. However, it has been shown that MD5 is not collision resistant[1]; as such, MD5 is not suitable for applications like SSL certificates or digital signatures that rely on this property. An MD5 hash is typically expressed as a 32 digit hexadecimal number.

MD5 was designed by Ron Rivest in 1991 to replace an earlier hash function, MD4. In 1996, a flaw was found with the design of MD5. While it was not a clearly fatal weakness, cryptographers began recommending the use of other algorithms, such as SHA-1 (which has since been found vulnerable). In 2004, more serious flaws were discovered, making further use of the algorithm for security purposes questionable.In 2007 a group of researchers including Arjen Lenstra described how to create a pair of files that share the same MD5 checksum.[4] In an attack on MD5 published in December 2008, a group of researchers used this technique to fake SSL certificate validity, and US-CERT of the the U. S. Department of Homeland Security said MD5 “should be considered cryptographically broken and unsuitable for further use.”.



Show me the code i hear you say, heres a simple console app using the functions…..

Imports System
Imports System.Security.Cryptography
Imports System.Text

Module Example

‘ Return a hash
Function getMd5Hash(ByVal input As String) As String
Dim md5Hasher As MD5 = MD5.Create()
Dim data As Byte() = md5Hasher.ComputeHash(Encoding.Default.GetBytes(input))
Dim sBuilder As New StringBuilder()
Dim i As Integer

For i = 0 To data.Length - 1
sBuilder.Append(data(i).ToString(”x2″))
Next i

Return sBuilder.ToString()
End Function
‘ Verify a HASH and return a boolean
Function verifyMd5Hash(ByVal input As String, ByVal hash As String) As Boolean
Dim hashOfInput As String = getMd5Hash(input)
Dim comparer As StringComparer = StringComparer.OrdinalIgnoreCase

If 0 = comparer.Compare(hashOfInput, hash) Then
Return True
Else
Return False
End If

End Function



Sub Main()
Dim source As String = “Hello World!
Dim hash As String = getMd5Hash(source)
Console.WriteLine(”The MD5 hash of ” + source + ” is: ” + hash + “.”)
Console.WriteLine(”Verifying the hash…”)
If verifyMd5Hash(source, hash) Then
Console.WriteLine(”The hashes are the same.”)
Else
Console.WriteLine(”The hashes are not same.”)
End If
End Sub
End Module

I have been asked alot lately, what software do you use to write code etc, Visual Basic 2008 / Visual Studio 2008 or text pad, writing command shell code i use notepad and then encode CScript using Windows Script encoder from a cmd line..