Use memory for Citrix Provisioning Services cache location

With a new generation of server hardware hitting the market I think it’s time to revisit this topic. First some background.

Provisioning Services in conjunction with Citrix XenApp allows us to create non-persistent XenApp worker nodes on demand, all of which are identical because they are streamed from a read-only image. This allows us to use a single operating system (Windows 2008 R2) to provide a desktop or applications to multiple users and at a lower cost than using the 1-to-1 OS-to-User model of VDI. Furthermore these XenApp worker nodes maintain their consistency by rebooting them on a regular basis (daily/weekly) at which time they flush their write cache and upon reboot return to their original gold image state. When using Provisioning Services each XenApp worker node uses a cache location for the write IO that they generate. The location of the write cache when using provisioning services is the key item we’re discussing in this post.

The options are as follows as per Citrix documentation:

The focus of this post is that in servers that are shipping today that easily hold 192GB of RAM (at a reasonable cost) we now have the option to instead of using virtual hard disks attached to each worker node (what I hear people using most often) that we can now use memory as a cache location and therefore offload the write IO to memory instead of disk, thereby decreasing the overall cost of deploying this solution even more and undoubtedly improving responsiveness and performance.

So in a server with 192GB of RAM and 16 physical cores lets see how this breaks down. With 16 cores we’re probably looking at a base configuration of eight 4 vCPU vm’s based on the latest testing from Citrix done here. Let’s take the example of eight 4vCPU vm’s with 16GB of RAM each.

8 VM’s 16GB RAM 128GB RAM Total
8 VM’s 5GB RAM Cache 40GB RAM Total
  168GB RAM Total per host

As you might notice in the Citrix testing that Frank Anderson did was that min memory available was a little more than 50GB, we’re getting tight especially when you factor in the 6% threshold before hypervisors like vSphere will start inflating the balloon driver.

So that’s the math and we’re pretty close to maxing out 192GB of RAM, but with hosts holding even larger capacities this napkin math should hold up even better. Non-persistent Windows 2008 R2 XenApp desktops and apps with no disk IO impact…pretty sweet…so how come nobody I know is doing this?


8 thoughts on “Use memory for Citrix Provisioning Services cache location

  1. I think the reason no one does this is because of the severe impact when the PVS RAM cache fills up: the workload just hangs.

    I’ve been pushing Citrix for a while to provide a seamless fall-back to disc-based cache should the RAM-based cache fill up. You’d then be much more likely to assign a RAM-based cache safe in the knowledge should the workload make more writes than your average calculations predicted you won’t lose all your provisioned guests.

  2. >>> so how come nobody I know is doing this? <<<

    Because the servers will blue-screen as soon as they've written 5GB's worth of unique disk blocks on the provisioned drive(s), which on a busy XA server will happen faster than one might hope. Even if the reboot schedule is aggressive (daily?), you'd still be gambling.


  3. I agree with Neil. There needs to be some kind of fail back or spillover for write cache. Start in RAM, but will that fills up fail back to target device hd, server side cache, etc. I also wish Citrix would add better monitoring and alerting of write cache size in either Desktop Director or EdgeSight along with retries and ha events.

  4. Thought I’d provide some more information to Jim’s comment above as I’ve just been through this with a customer who had come to the same conclusion around using RAM as they had frustratingly slow performance using 8 x local 15K SAS drives while they had lots of memory available in the server.

    With Atlantis ILIO (XenApp Version) we are able to use RAM as storage for the write cache with ILIO de-duplicating the multiple write-caches across a number of servers at a block level, keeping the amount of RAM used to a minimum. In their case we were able to almost double the density of XenApp servers and users that could fit on a host through the use of ILIO. Plus performance was awesome.

    We are also working on some interesting concepts around running the entire image in RAM – more to come on that. it will be interesting to see what Citrix announce at Synergy re: MCS/PVS and XenApp.

    DM either @jimmoyle or @maxwellgill for more information on ILIO for XenApp

    1. Finally the Day has come, Neils wish has become true and with PVS 7.1 we now have the ability to fall back to local disk if the ram cache is full. At the moment I´m testing this feature within a customer PoC. Maybe someone of you has already been testing around with this and would share his knowledge. I will keep you posted as soon as I got my testing results.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s