Derek Seaman: VCP5 to VMware VCDX #125 in 180 Days

PART 1

This is Part 1 of my tale of the last 180 days of my life, my adventure from VCP5 to becoming …drum roll… VMware VCDX #125. Yes, I successfully defended my design at VMware PEX 2014 in San Francisco this past week, which was for VCDX5-Datacenter Virtualization (DCV)! It was a fun, educational, and a surprisingly pleasant experience. I’m blogging about this experience because I know if I can become a VCDX, then anyone can do it with enough experience and dedication. Do keep in mind that prior to starting my 180 day adventure, I had approximately 4 years of VMware architecture experience for Government and commercial enterprise systems. Years of architecture-level experience is key, however VMware does not require a specific number of years or job title to apply for the VCDX certification.

This was my first attempt at the VCDX certification, and was a great learning process. Nearly all candidates and those who have passed either work for VMware or a VMware partner. At PEX 2014 Josh Coen (VCDX #129) and I were the only customers that I know of defending out of the ~14 candidates. According to the latest unofficial stats Josh and I doubled the customer VCDX count from two to four out of the 132 VCDXs.

VCAP5-DCD Journey

Back in the spring of 2013 I was talking to my boss about goals for my next six month review period. He mentioned something about a big VMware certification and I said “VCDX”. He was like ya…”Why don’t you go for that?” I looked at him like a third eye appeared on his forehead and thought no way in hell can I do that. I mean, those people are book authors, world renowned bloggers, and levitate at will, right? But after a little thought I settled on trying the VCAP5-DCD (VMware Certified Advanced Professional – Datacenter Design) exam…that seemed reasonable. I love architecture/design, and was fairly proficient with VMware virtualization.

Time passed on, and VMworld 2013 SF was just a couple of weeks away. The review period was nearly over, so I had to take the test, pass or fail. Due to lack of time, I didn’t even look at the blueprint and pretty much went in cold (NOT recommended, BTW). I did a little Googling just to see what others experienced, and learned time, or lack thereof, was a big factor. I put down the $400 exam fee, and sat the exam. By some miracle I passed, on August 23rd, the day before I left for VMworld 2013. Review goal accomplished.

The Turning Point

Happy that I passed the VCAP5-DCD, I eagerly packed my bags for SF for a week of learning and adventure. While I had been a beta tester for vSphere 5.5 thanks to my vExpert status, I was eager to get the scoop on the GA features. I was able to get into VMworld TAM Day (Technical Account Manager) on Sunday, which is basically a preview for specific customers of what VMware would unveil the coming week.

As a quick aside, the prior year VMware had released vSphere 5.1, which we all know had a few rough edges around the SSO service and trusted SSL certificates. I lead the way in trying to figure out how to make that work, and my 15-part vSphere 5.1 install series skyrocketed my daily webpage views. But those were just numbers.

Back to TAM day, I was sitting in the front row of a panel session getting ready to blog about the content. One of the panelists, which was also a top rated blogger, came over to me. He had some very nice comments about my blog, and I was floored that someone like him would be a regular reader. I hardly knew what to say. I also attended a few VMworld parties, where strangers recognized me and came up to introduce themselves. This was a completely foreign experience to me, but felt great that my content was helping people. By the end of the conference several people were calling me “The SSL Guy” due to my vSphere 5.1 SSL work. My blog page view count was now growing real faces.

One of the very last sessions at VMworld that I attended was Software Defined Storage the VCDX Way by Wade Holmes and Rawlinson Rivera. It was a kickass session, and by the end I was thinking…you know I should shoot for VCDX. I can do it! I checked the VCDX defense schedule, and figured I would shoot for Partner Exchange 2014, which was in February. I thought submissions would be due early January 2014, based on last year’s schedule. I would later find out that was a bad assumption, and presented a risk to my success.

VCAP5-DCA Journey

Next up on the VCDX trip was taking and passing the VCAP5-DCA (Datacenter Administration exam). I wrote a blog article about therehere. To summarize, I knew I’d have my bacon cooked if I didn’t study. I was not as adept at vSphere command line or PowerCLI as I wanted to be, and knew that time was also a huge factor. So for the first three weeks in September I spent several hours with Jason Nash’s (VCDX #49) excellent TrainSignal (now Pluralsight) DCA prep series. Due to my vExpert status, TrainSignal gave us one year free of unlimited training. I also read every page of Josh Coen (VCDX #129) 250 page unofficial DCA study guide. I went through numerous exercises in my home lab using my Haswell ESXi hosts. I took the exam on September 20th, and got my results back a couple of business days later. I passed! Huge relief. Little did I know that I’d cross paths with Josh a few weeks later.

Time To Blog

By this time it was September 22, and vSphere 5.5 went GA. Clustering the vCenter SQL server was now officially supported, so of course I wanted to blog about how to setup a SQL 2012 failover cluster on vSphere 5.5. The last week of September I published a 12-part SQL clustering guide that of course used SSL and new vSphere 5.5 Microsoft clustering features. Ok, now we were ready for a vSphere 5.5 install guide. Since my vSphere 5.1 install series was popular, I wanted to outdo myself and really ramp it up for vSphere 5.5. Long story short I knocked out Parts 1 – 15 of the series in October, and wrote the first versions of the my vSphere 5.5 Toolkit script. Somewhere around this time the official VCDX defense schedule was published, and the submission deadline was moved up to just before Christmas. I had a mini heart attack, since I knew this would be a lot of work and would likely need the time over Christmas.

The VCDX Design

Picking your design for a VCDX defense is critical. Too complex, and you open up your attack surface area and may have a higher chance of failure (or need more prep time). Too simple, and you risk rejection. Plus VMware stipulates you must have played the role of an architect in the design (although need not be the sole architect.) Thankfully the new VDI refresh project I was the architect for fit the bill, was complex enough to make it interesting, yet not so complex that I couldn’t fully understand all the areas called out in the VCDX-DCV blueprint. Literally the only documentation that existed for the project was a warmed over vCenter install guide. No architecture guide, operations guide, or test plans existed. I had A LOT of work ahead of me. VCDX submission packages typically run into the hundreds of pages, although size is not what counts and less can actually be better.

Time to Write

Having settled on my design, I started writing day and night and weekends. Since I was documenting a production system, and making tweaks based on additional research, I was able to put time into the project during work hours. During this time I also found out that Josh Coen (@JoshCoen), of DCA Guide fame, and Bobby Stampfle (@BobbyFantast1c) were also working on their VCDX submissions. So we formed a study group (extremely important), and reviewed each others documents. It’s very enlightening to see other approaches, and we each ended up melding in changes and suggestions from each other. Working on this in a vacuum is a recipe for a headache and an uphill battle.

All told, the architecture guide topped nearly 200 pages, 60,000 words, 42 Visio figures, and 137 tables. Yes, it was comprehensive. All twelve major sections in my architecture guide followed a strict taxonomy of conceptual design, logical design, physical design, and design justifications. Customer requirements, constraints, assumptions, and risks guided the whole flow of the document. I based some of the flow and content on Duncan Epping’s (VCDX #007) excellentVMware Cloud Infrastructure Architecture Case Study. The installation guide, operations guide and test plans were all much shorter, but still important.

Time to Review

An extremely important facet of a VCDX design is peer review from your study group and ‘real life’ VCDXs. You should aim to finish your complete architecture guide at least 3-4 week prior to the submission deadline. I’m now a Twitter addict, so I was able to find VCDXs that graciously gave up some of their spare time to provide feedback on my design. The level of detail and different viewpoints is very interesting, and provided a good 360 degree view. Chris Wahl (#104), Romain Decker (#120), Jon Kohler (#113) and Chris McCain (#79) possibly others I’m not recalling, all gave great feedback and I really appreciate their time. I made several changes based on input. Josh and Bobby were also great supporters. We also did webex design reviews, each going over our docs and making suggestions for changes or areas to clarify.

As a side note, by this time I had also published four more vSphere 5.5 installation guide posts (now up to 19) and made major updates to the Toolkit script to support more SSL scenarios and added VCSA/ESXi support as well. Talk about busy…I had no life. I heard one VCDX say that “Behind every VCDX is a spouse that hates VMware.” And I can believe it. All told I probably spent 400 hours on the documentation package, since I had to build nearly all of it from scratch and I’m a bit of a perfectionist. I also like pretty diagrams, which are a big time suck.

VCDX Submission Time

December 20th rolls around, the submission deadline for PEX 2014 in February. All the reviews are done, and I finish up a couple of days early. I did one final proof reading, let it simmer over night, zipped up the contents, and sent it off to VMware for review. A nice little payment of $300 was included for them to review my design. We are now 120 days after from passing the VCAP5-DCD exam, 32+ blog posts and one script later. Whew! Talk about my brain being fried. It was burnt to a crisp. Also during this time I rescheduled a New Zealand trip from early January to late April, in the hopes I would get accepted and need the prep time in January.

Next up in Part 2, will be the final installment where I’ll cover how I prepared for the actual defense, my defense experience, and tips for those wanting to pursue your VCDX. Plus…my new and exciting job.

PART 2

In Part 1 of this two part VMware VCDX journey series I accounted for the first 120 days of my VCP5 to VCDX #125 journey. This post will cover the final 60 days, which were just as important, if not more so, than the first 120. In my last post we left off with my completed design package, application form, and a sacrificial payment of $300 was submitted to the VMware Gods. Would my application be chopped in half and sent back in pieces, or spared to live another day?

Accept or Reject?

On December 23rd my application was ‘administratively accepted’ meaning I passed the basic sanity check of filling out the application form and attaching documents. Given the fried status of my brain, I took a couple of weeks mental break from looking at my design documents. I also didn’t want to jinx my acceptance by starting to work on my presentation slide deck. A birthday also came and went without too much fanfare during the waiting.

Fast forward a couple of weeks to January 10th, 2014, and I got a short email stating the technical review was completed and that I was invited to defend my design at PEX 2014 in San Francisco. Yippee! Major goal accomplished, and another payment of $900 to VMware. Twitter went a little crazy that day, and also got a large spike in LinkedIn profile views. Ok time to update LinkedIn profile, check. A couple of days later I got my timeslot booked. The fateful day would be Monday February 10th, 2014 at 0900. Reality set in, tempered with a little panic and excitement. This is real.

Oil Thy Slide Deck

For those of you not familiar with the VCDX defense format, it is in three parts. The first is the 75 minute defense, where you give a short presentation that covers your design. The panel generally interrupts the presentation, and starts asking you questions. You also have a 30 minute simulated design session, and finally a 15 minute simulated troubleshooting session.

Thanks to Chris McCain (VCDX #79), I got some great pointers on how to create a well oiled PowerPoint presentation with a plethora of internal hotlinks and hot spots. The panelists can ask you questions in any order on any topic, so you must be able to flip between slides in literally just seconds. The clock does not pause for slow slide flippers. I spent the better part of a week, dragging my fried fingers across the keyboard trying to assemble my deck. Concentration was difficult…after four months of living my design I just wanted this to be over…like the 70s Calgon commercials, “Just take me away.” Oh how I wished was in Middle Earth instead of staring at PowerPoint.

My basic presentation was just 12 slides, covering key drivers, compute, network, VMs, BC/DR, and other areas. In my over achieving appendix I had 99 slides, with content all pulled from my architecture guide. All 99 slides were organized by category, each slide with its own hotlink on the appendix page. I also had buttons in the lower right on all slides, for easy navigation to my table of contents, appendix, forward, backward, and last slide. Within a couple of clicks I could jump anywhere in the deck, and back. Apply oil liberally. VCDX candidates need a short class on mastering PowerPoint. Oh yes, I should not fail to mention that Josh Coen and I were working closely bouncing ideas off each other, and help each other formulate our deck. Again, don’t work on your VCDX in a vacuum. Misery loves company.

You Mock Me, I Mock You

Somewhere around this time I also saw on Twitter that this guy named Brad Christian was invited to defend. He appeared to be a VCDX candidate wrangler (and Dallas VMUG Leader), and started corralling all of us anointed to defend at PEX. The clock was now ticking, and we were about three weeks away from our defenses. Due to his strong leadership and immense motivation, he helped us organize nightly mock defenses. One lucky victim, I mean candidate, presented their slides and the group ripped them to shreds (gently) with questions.

Many of us looked like Humpty Dumpty after 75 minutes, and had to put ourselves and slide deck back together again. But it was all for a good cause, right? All of us revised our slides, took note of questions others got asked that we (I) didn’t know the answer to. One of my favorite moments, was when we were really beating up a candidate and someone asked him to describe how Load Based NIC teaming worked. By this time Thelma (name and gender changed to protect the innocent) had really been slammed and was a wee bit agitated. So her snippy response was, “Well let me go grab the source code for that and then I’ll tell you.” Ok now…let’s all settle down a bit.

I bet the WebEx server was getting tired of the nightly beatings. The last couple of days during that three week period I was once again deep fried, and actually ditched the sessions. I just couldn’t take it any more. We are now at February 7th, three days before my defense.

WebEx wasn’t enough..let’s do it in Person Too

As if three weeks of beating on each other virtually was not enough, Brad reached out to the VMware community to see if anyone would sponsor a boxing ring for the weekend prior to our defenses so we could beat each other up in person. Nutanix came through and rented a boxing ring, which suspiciously looked like a conference room at the W hotel. All weekend long nearly all candidates were packed into the room taking turns doing mocks.

Some new faces appeared whom had not been in our WebEx sessions, which were fresh and angelic like, but partially resembled Humpty Dumpty after we got done with them. But it was all for the common good, right? Seated along side the boxing ring were VCDXs like James Charter, Tim Antonowicz, and Mark Gabryjelski giving pointers and wiping up the blood.

Oh yes, and let’s not forget the insanely evil troubleshooting scenarios that [redacted] dreamed up from real life. Or how about the customer design scenario where a service provider had both adult streaming media and conservative non-profit tenants? Workload isolation? If it was *EVER* needed, this was the time! XL vShield to the rescue. Or a VDI scenario using linked clones where the master VM was 62.9TB? Yes, we were ruthless..and perhaps got a little carried away. Let’s not forget James Charter frantically waving “Why?” on a napkin, like he was a stranded survivor trying to flag down a rescue mission, during our troubleshooting scenarios.

Dooms Day

Now that all the prep was done, February 10th at 0900 was just hours away. I got a good night’s sleep, had breakfast at the hotel, and made my way over to the Hilton. As you can see from my selfie to the left, I was very relaxed and carefree. Actually I was pretty relaxed, and was confident that I knew my design and slide deck very well. Promptly at 0900 I was led like a sheep by Mark Brunstad into the defense room. No ‘death squad’ members were present, so I relaxed and started off the 75 minute presentation. A worm hole opened up, and before I knew it the 75 minutes were up. I felt like I did really well in that section. Of my 99 backup slides I used possibly three.

Next up was either the design or troubleshooting session, I don’t recall. Either way, the design session was a bit more rocky than I had wanted…and reverse-wormhole formed and the 30 minutes seemed to take forever. Troubleshooting went OK. Overall I felt good about my performance, and the experience was actually quite pleasant and not scary. The panel is there to help you score higher, not pick you apart or make you feel like a dummy. Mark escorted me out, and said the official results SLA was 10 days.

And the Results are in…

Past performance is not an indication of future performance. That was certainly true in this case, but in a good way. For prior defenses VMware released the results a few days after everyone had completed, which was still much quicker than the stated 10 day SLA. However, Mark pulled a fast one on us and in less than a couple of hours after the last defense on Thursday he sent everyone the results. I was so not expecting them, that I wasn’t looking at my phone every time I got an email. There was just no way results would come in that fast. Impossible!

But I was on my computer, happened to have OWA open and heard the ding. The sender of the email was Mark, and my heart practically stopped mid beat. Tunnel vision set in. The results were in a PDF which I had to open. This may sound easy, but it’s not when your heart has stopped. Tunnel vision gets even narrower. Acrobat takes forever to launch..I had never wanted Acrobat to open so fast in my entire life, and it never seemed so slow.

Upon opening I see a number…no text..just a number. 125. I’ve never been happier to be called a number in my life. I now see the words “congratulations.” Yes, I’m now VCDX #125. Shortly thereafter Twitter practically explodes. Yes, Josh tweets his number, Garrett, Kalen, Hersey, Sean……eight of us tweet three digit numbers. The numbers 125 through 132 have new owners. 175 days after my VCAP5-DCD exam mission is accomplished.

Thank you!

I want to give a huge shout out to Brad Christian, Josh Coen, and other VCDXs that participated in mock defenses like Josh Odgers and Romain Decker. Brad did a stellar job at organizing the mock panels. The full weekend of in-person mock defenses was also critical, and want to thank Nutanix for sponsoring the room. I’ve been told this is the first time in VCDX history where nearly all candidates did mocks for so many weeks, and in person, prior to the defenses. That credit goes to Brad! I also really want to thank all my Twitter followers, friends, family and co-workers that were supportive during the whole six month process. There’s no way I could have done this alone.

Aspiring VCDX Resources

If you are still with me at this point, you get an award. No fancy stickers like Chris Wahl, just a pat on the back. Beyond knowing your design inside and out, here are a few must-have resources that you need to start your VCDX journey.

1. Don’t even think about starting the VCDX without reading every word in the VMware VCDX Boot Camp book. Buy it here from Amazon. Read. Every. Word. Read. Again. Read. Again. This your beacon on the VCDX road.

2. Buy the Storage Implementation in vSphere 5.0 by Mostafa Khalilhere from Amazon. Think you know a lot about storage? You won’t after you get done reading this book. Excellent reference book, and a must-read prior to a defense.

3. Buy the relevant version of Clustering Deepdive by Duncan Epping and Frank Denneman. 5.1 version is here.

4. Buy the VMware vSphere Design book by Scott Lowe (and others) from Amazon here.

5. Attend in-person the VCAP and VCDX workshops. These are normally held at large events like VMworld and PEX, but they are also ramping up at other locations and times as well. I’m pretty sure that everyone that passed this time attended one or more in-person boot camps.

6. Watch the VMware VCDX prep videos with John Arrasjid and Rawlinson Rivera here.

7. Check out the blog post by Brad Christian about his experience, and more tips here.

8. Check out the @vSential VCDX Study Group form here.

9. Although not out yet, Chris Wahl is publishing a vSphere networking book that I know will be killer and a must read.

My #1 tip is to join a study group as soon as you know you want to get on the VCDX bandwagon. Share often, share early! Get on Twitter and find other VCDX candidates, and use the group sign up form link above. Yes, you too can become a VCDX with enough experience, dedication, and a study group.

My new Journey

I was so looking forward to my life returning to ‘normal’ post-VCDX. Every weekend and practically every waking hour for the last 180 days was VCDX. Well, there’s a new normal starting in less than two weeks.

I’m very honored and very excited to announce that starting in March I’ll be joining Nutanix as a Sr. Solutions and Performance engineer. This is the same team that Josh Odgers (VCDX #90) and Michael Webster (VCDX #66) are on. Last year The Register wrote an article about Nutanix assembling an elite squad of ‘crack VMware designers’. Michael was the fourth VCDX to join Nutanix, and I will be the fifth. I’m sure I won’t be the last to join. Nutanix has a great blog post about the value of VCDXs to the VMware ecosystem, which you can read here.

I don’t see obtaining a VCDX as the end of a journey, but rather the start of a whole new adventure. I know I can learn a tremendous amount from Josh, Michael, the other VCDXs and the extraordinary bright Nutanix staff. This will be my first startup, and I’m relocating to the Nutanix HQ in San Jose. Storage has always been a passion of mine, and love a good Fibre Channel SAN or tier-1 array. But enterprises need something that now linearly scales out, vastly easier to use, much higher performance, and more dense. That is Nutanix.

Very important to me is being able to blog about my passions, and Nutanix is the perfect fit. I can continue to write about VMware, Microsoft, Citrix, and expand my knowledge base to other products. I will have a lot more opportunity to blog about new passions, and give back even more to the community and Nutanix customers. I’m excited to start this brand new adventure, so expect my blogging to ramp back up to pre-VCDX levels with even better content.

[Source: DerekSeaman.com]