More than 3GB memory is not used even though boot.ini has these switches on /3GB /PAE /AWE.

Yet another AWE memory settings blog post here that I'm posting for the issues we have had recently. Before going into details review about 3GBPAE blurb here and also refer to this KBA274750 to enable more than 2GB memory within your SQL Server Enterprise edition instance.

Coming to the problem on a server having 6GB RAM and having required Windows & SQL Server editions  including enabled /3GB and /PAE switched in boot.ini file doesn't show or grow more than 3 gb. Due to this we are getting performance issues for a larger computation queries as this instance is meant to cater such application. On top of that this is a clustered SQL instance installation.

For your information when you have 8GB or less, then you should use these switches in the boot.ini: /3GB /PAE and you must remember to turn on AWE memory in SQL Server. On such AWE settings refer to this TIPS page for more information. Further from the documentation and previous experience I have found that SQL server will be able to use upto available RAM on that server instance and further reference for 64bit installations:

The Windows policy Lock Pages in Memory option is disabled by default. This privilege must be enabled to configure Address Windowing Extensions (AWE). This policy determines which accounts can use a process to keep data in physical memory, preventing the system from paging the data to virtual memory on disk. On 32-bit operating systems, setting this privilege when not using AWE can significantly impair system performance. Although not required, we recommend locking pages in memory when using 64-bit operating systems.

The resolution to this problem was  hotfix that needs to be updated, which is 899761  FIX: Not all memory is available when AWE is enabled on a computer that is running a 32-bit version of SQL Server 2000 Service Pack 4.  I have found that our live SQL installation library builds are updated until build 2039 and with this experience we have updated upto build 3161 Cumulative update package 3 for SQL Server 2005 Service Pack 2.