I’ve got problems but 99.999 (five nines) storage isn’t one of them

October 7, 2014 Leave a comment

I recently have been in front of a few customers discussing various designs for application and desktop virtualization.  Inevitably, or at some point, we discuss storage.  When it comes to storage I often pause and read the room because most people i know on the VAR and customer side have their favorites and have what I would refer to as a Dallas Cowboys team (I’m an Eagles fan, if you are a Dallas fan, just reverse the teams, it’ll work).

I’ve architected (is that a real word?) large deployments involving multiple datacenters, high availability and disaster recovery. My focus isn’t on what is the single best technology and gluing things together, it’s about what works (and hopefully, what works well).  Storage can be a very big issue with VDI, traditional SAN-based storage was not designed for desktop workloads and we’ve been oblivious to faster disk speeds and low latency on drives that hum under our wrists when typing.  Moving these workloads to the data center doesn’t always work and when you add in latency from a server reaching out to a separate SAN, it compounds the problem.

The traditional SAN isn’t usually the best fit for heavy desktops and applications, however, adding flash technology to the mix often deals with the IOPs issue and latency can be minimized.  Is flash necessary?  Nope.  I’ve had designs involving 15K SAS drives local to blades work very well.  The Citrix stream to memory, overflow to disk can perform even better with 10k or 7k drives.  However, I often don’t get to position that solution which brings me back to my first point…everyone has favorites.

I can take almost any storage and find a solution.  Even a traditional SAN, if I can use memory to cache, I can make that work.  Local disk? Easy.  Flash appliances, they are great!  But there is one thing I’m hearing that I don’t need.  The storage providing high availability or five nines.  There is a simple reason I don’t need five nines and I cringe when I hear others use it and lean back.

Your application doesn’t solely rely on storage to be available!

How will five nines prevent downtime when your hypervisor crashes or profile corruption occurs?  What about a failed backup on SQL that just eats up disk space?  What should we do?

We need to embrace failure and assume things fail.  It’s so much cheaper than having the hardware give you a warm fuzzing feeling.  When that business app fails, the business doesn’t care whether it’s storage or a cleaning person tripping over a server cord (I hope that isn’t even possible in most of your environments!). They see IT as the failure, not storage.

I wish I could take credit for this thought process but netflix has pretty much perfected this thought.  If you haven’t heard of the chaos monkey you should learn – http://techblog.netflix.com/2012/07/chaos-monkey-released-into-wild.html .

Spend enough time in IT and you’ll realize that chaos always wins and you burn out quick if you’re fighting it.  However, returning to my original point, the design and architecture can do this also.  When we talk of desktops, many argue persistent versus non-persistent.  Persistent means you keep your desktop, non-persistent means you can roam (which usually means some flavor of roaming profiles).  I’m a big advocate of non-persistent.  Your storage or server fails, you get logged off, you log back in and you’re right where you were (or very close to it).  If the application is database driven and supports mirroring, you can survive storage failures, if setup correctly.

Going back to storage, this means two of whatever I have.  Two local drives, two appliances, two SANs.  I’ll take two 95% up-time appliances over a single 99.999% appliance anytime.  I’d rather save costs with single controllers than try to make a single point of failure not fail (because your application never has a single point of failure, it’s got multiple points of failure).

I’m not arguing five nines doesn’t have a place somewhere.  If you can’t use non-persistent, it might be for you.  However, I’d argue that virtualizing your applications and desktops is not a good move if you need persistence anyways.  Just my two cents, feel free to comment if you agree, disagree or think I’m full of it, I’m always open to suggestions!

PS – This is a first draft to publish, I’m sure there some typos and run-on sentences in there.

Categories: Citrix, microsoft, vmware Tags: ,

InfoBlox and Citrix Issues [RESOLVED]

May 15, 2014 1 comment

I have heard a lot on Infoblox issues with Citrix and had the chance to meet some of the Infoblox team today for lunch and a meeting.  My first question, and Kevin Dralle’s, whom I work with, was about the apparent incompatibility of InfoBlox and Citrix, especially with PVS.  Please comment if you think this doesn’t work or has issues.


Some of the issue have been described elsewhere (I know Jarian Gibson has wrote and tweeted a few things on this also)



With Infoblox there is a CTX but it’s a bit mysterious on details


So what is going on with InfoBlox, anytime we have had customer with InfoBlox on and Citrix we cringe or opt for perhaps a dual NIC, isolated PVS VLAN (using Microsoft DHCP).  In any case here is what happens.

InfoBlox assigns the device a UID based on MAC but also on some of the device characteristics. So when we boot off PVS, we bring up the bin file which acts as the OS at the time of PXE boot.  We have a static MAC but since after the bin file pulls the TFTP image then brings up a windows OS, the UID changes, which infoblox assumes you’ll need another IP address.  Obviously there are use cases for this but for PVS this is an issue as you’ll get two IP addresses.

One fix has been to use reservations but this defeats the whole purpose of using an appliance or solution to manage this all.  Furthermore, when or if you get into automation and orchestration, you’ve got one more component to worry about when increasing the scope.

You do need to be on the 6.6 or higher release for this option but it is worth it if you have this issue or are an InfoBlox shop and want to rollout PVS without trying something uncommon to deployments (using BDM has a lot less collateral out there than PXE boot does).

Below are the two areas from the Grid and Member layers where you can set this (courtesy of InfoBlox!).


Categories: Citrix Tags: , , ,

Citrix SynergyTV – SYN119 – How Atlanta Public Schools delivers virtual desktops to 50,000 students #citrixsynergy

Categories: Citrix, XenDesktop Tags:

Redirecting Folders to Office 365

May 11, 2014 11 comments

I created a script a while ago to map folders at logon to enable saving directly to OneDrive for Business (also known as SharePoint Online or SkyDrive Pro). I plan to explain more on this but it was the script I mentioned in my #SYN119 presentation at Citrix Synergy 2014.

This script is now up and published on codeplex and is under the GPLv3 license (a copyleft license). Feel free to use and modify if you can help. Below is a slight description of what the script does and why it was created.


This project is to enable the use of Office 365 as redirected folders in Microsoft. Specifically, this script and method can be used on a Windows 7 desktop (or higher) with Citrix and roaming profiles (or any persistent profile method). What makes this unique is that no local storage is used (unless you can’t connect to office 365 and then it’s only temporary).

This script was developed by Tyler Bithell and Tom Gamull for a customer implementation. The customer desired a method to eliminate the use of local or shared storage and leverage their Microsoft Office 365 subscription.

You must have a subscription that gives you the SharePoint Online (Groove, SkyDrive Pro or OneDrive for Business). This is NOT the same as SkyDrive or OneDrive (these are just an online storage method, like Dropbox).

We leverage a method of WebDAV drive mapping utilizing NET USE. Although you can utilize OneDrive for Business for Microsoft Office applications directly, non-integrated applications are not able to be used except through navigation to the folder. This is often a problem for task works, students and others used to using My Documents or Downloads. Therefore this script was created to deal with this issue.

The script was announced but not shown at the Citrix Synergy 2014 conference in #SYN119. This script will also be discussed in the Cisco Live session Tom Gamull is presenting on Atlanta Public Schools.

XenDesktop 7.1 SQL Mirroring

March 6, 2014 1 comment

Mirroring in SQL is a great way to protect your XenDesktop infrastructure. In 7.1 deployments this can be a bit challenging since the documentation at Citrix doesn’t reflect an accurate way to accomplish this goal.

First let’s go over some basics. SQL Mirroring is a 3 server setup with a primary SQL server running a database, another secondary SQL server also running the database and a third SQL witness server which does NOT run the database (runs SQL, just no data). If you use local disk, this is an excellent setup. If you have two storage appliances, this is a great setup. If you have one big SAN, this doesn’t make much sense. To make mirroring worthwhile, you need 3 SEPERATE storage locations for each server. If you have two servers on the same storage, mirroring will not provide much value (other than a learning opportunity). I feel this is where people can easily forget why you mirror.

To demonstrate why, let’s say I have a two node management cluster, one host runs my primary SQL server and the other runs my secondary and the witness. I put the primary on the local disk of HOST1 and the secondary and witness (which is lightweight) on the local disk of HOST2. I have an issue. Let’s say HOST1 goes down, HOST2 is up and SQL stays up just fine because we have one of the mirrored servers running PLUS the witness. Let’s say I accidentally shut down the witness. No problem. Let’s say I shutdown HOST2 or do maintenance. Now I’ve got a problem. When the primary SQL server can’t see the witness AND the secondary SQL server, it stops. This is by design, it doesn’t know if it’s orphaned. It assume those two other servers don’t see it but must be serving the data. If I simply use a witness on a third unique HOST and storage area, my mirror is looking great! If I only have 2 hosts or shared storage, this is where a cluster makes sense. Clusters cannot survive storage failures, but mirrors can. However, you are writing to both storage locations at the same time. Often I’ll use two unique SANs and put the witness on local disk. I can now survive a storage appliance failure, however it’s only as good as having three different points of failure instead of one.

SQL Mirror

With that said, my main topic was how to get this done on your XenDesktop 7.1 controllers. This appears to have been posted other places but since I had to do it I thought I’d share also. There is an excellent post at Citrix on this here. here.

I did this manually below but I heavily recommend downloading his script and giving a shot before manually doing this.

I want to add one caveat, you MUST create the machine account logins in SQL on the mirror. So you’ll need to do this after a forced failover to the mirror. In addition, you may also need to delete the machine accounts and add them back if you migrate the SQL database, say from SQL Express to Standard/Enterprise for example. This is what worked for me, hopefully it helps

Now for this next part, I like to do this one controller at a time. If you mess up you can’t really go back and fix things if your controllers get orphaned. This is why during upgrades, you only partially update the farm, in case you need to roll back.

$cs = "Server=YOUR_SQL_SERVER_NAME;Initial Catalog=YOU_SQL_DATABASE_NAME;Integrated Security=True"

Set-LogSite -State Disabled
Set-LogDBConnection -DataStore Logging -DBConnection $null
Set-MonitorDBConnection -DataStore Monitor -DBConnection $null

Set-MonitorDBConnection -DBConnection $null
Set-AcctDBConnection -DBConnection $null
Set-ProvDBConnection -DBConnection $null
Set-BrokerDBConnection -DBConnection $null
Set-EnvTestDBConnection -DBConnection $null
Set-SfDBConnection -DBConnection $null
Set-HypDBConnection -DBConnection $null
Set-ConfigDBConnection -DBConnection $null -force
Set-LogDBConnection -DBConnection $null -force
Set-AdminDBConnection -DBConnection $null -force

Another way to clear the controllers out

$controllers = Get-BrokerController | %{$_.DNSName}

foreach ($controller in $controllers)
Write-Host "Disconnect controller $controller ..."

Set-ConfigDBConnection -DBConnection $null -AdminAddress $Controller
Set-AcctDBConnection -DBConnection $null -AdminAddress $Controller
Set-HypDBConnection -DBConnection $null -AdminAddress $Controller
Set-ProvDBConnection -DBConnection $null -AdminAddress $Controller
Set-BrokerDBConnection -DBConnection $null -AdminAddress $Controller
Set-EnvTestDBConnection -DBConnection $null -AdminAddress $Controller
Set-SfDBConnection -DBConnection $null -AdminAddress $Controller
Set-MonitorDBConnection -Datastore Monitor -DBConnection $null -AdminAddress $Controller
reset-MonitorDataStore -DataStore Monitor
Set-MonitorDBConnection -DBConnection $null -AdminAddress $Controller
Set-LogDBConnection -DataStore Logging -DBConnection $null -AdminAddress $Controller
reset-LogDataStore -DataStore Logging
Set-LogDBConnection -DBConnection $null -AdminAddress $Controller
Set-AdminDBConnection -DBConnection $null -AdminAddress $Controller

If the last two won’t work, try adding -force on the end. If they still don’t do the following (may need to reboot)
Get-Service Citrix* | Stop-Service -Force
Get-Service Citrix* | Start-Service

Ok now it’s time to go mirror, once you’re done setting up the mirror set the database on ONE of the servers only and verify it before moving to the next one(s)

You already set the $cs variable but if you opened a new window or lost it, set it again

$cs = "Server=YOUR_SQL_SERVER_NAME;Initial Catalog=YOU_SQL_DATABASE_NAME;Integrated Security=True"
set-ConfigDBconnection -dbconnection $cs
set-AdminDBconnection -dbconnection $cs
set-LogDBconnection -dbconnection $cs
set-AcctDBconnection -dbconnection $cs
set-BrokerDBconnection -dbconnection $cs
set-EnvTestDBconnection -dbconnection $cs
set-HypDBconnection -dbconnection $cs
set-MonitorDBconnection -dbconnection $cs
set-ProvDBconnection -dbconnection $cs
set-SfDBconnection -dbconnection $cs
Set-LogDbConnection -DataStore logging -DbConnection $cs
Set-MonitorDbConnection -DataStore monitor -DbConnection $cs

Set-LogSite -State Enabled

$testString = Get-BrokerDBConnection
Test-BrokerDBConnection $testString | fl

Now make sure you TEST FAILOVER before declaring success.

Categories: Citrix, SQL, XenDesktop Tags: , ,

Fixing Thomas the Tank wooden train tracks (or Brio)

February 1, 2014 Leave a comment
Categories: Uncategorized

Fixing Thomas the Tank wooden train tracks (or Brio)

February 1, 2014 4 comments

If you have young children like I do (4 kids, aged 5,4,3 and 1) you probably have some Thomas the Tank Engine wooden train tracks.  A few of these pieces have this plastic plug that is the “male” piece, which obviously joins with the female piece.  Try as you might, there is little that can be done if your bundle of joy pulls out the plug and loses it.  You have what I’d call a “neutered” piece.  It’s flat, but neither male nor female.  Unfortunately, some of the best pieces are usually the ones with this plastic plug male piece in them (hills, splitters, animated sections, etc).

Since this toy is not cheap (for what you get) and I supposedly can fix anything, I had to attempt to figure this out.  Searching the internet yielded no results but finally I found a random post on some forum that provided a solution (and the guy who posted it deserves all the credit!)


To help you out, you’ll want to get the right sized eye screws (pic is what I bought at Home Depot), also get the drywall #10 anchors.  You’ll end up cutting the anchors (about 1 or 2 ridges should protrude from the hole).  When you screw in the eye it doesn’t need to be dead center but try to get it somewhat straight.  Electrician’s pliers work very well but honestly any pair of pliers will do fine.

Now I have happy 3 and 5 year old boys.  Perhaps your 3 year old son isn’t obsessed with trains like mine, that’s fine, but if he is, I hope this helps.


What you’ll need


What it should look like


Final Product



Get every new post delivered to your Inbox.

Join 725 other followers