task manager is the simplest and quick way to check your memory usage. if the memory usage is more than your physical memory, that means the excess memory usage is spilled to your harddisk swap space and this will slow down your system performance significantly. so, check how many excess and add the physical memory accordingly.
Look at it this way in terms of cars, what is the real difference between a 2.0L and 1.5L toyota. Will it help you? If you drive long distances to Malaysia every week, a 2.0L car might be a better idea while driving in shit traffic inside a city like Singapore really doesn't give much of a difference.