April 14, 2014

Black Ops: Stealth Mode Debrief

This is a brief(ish!) record of some notes that I made prior to the Black Ops Testing webinar, 14th April 2014. The webinar exercise related to the testing of a "Stealth Mode" button which had been added to the Black Ops Testing website, for the purposes of this seminar.

Before I began

I made one big assumption in my test approach: I assumed that the "Stealth Mode" button wasn't a real product feature. After all, if there was a real requirement for a "Stealth Mode" (which obscures the contents of the email address field) then much of the observed behaviour is actually 'Functions As Designed' - although I would still argue that it provides a poor user experience.

With that in mind, I proceeded with the rest of the task as if the "Stealth Mode" button was actually a magic "make the bug happen" button. If this were a real product example, I would be seeking some clarification from the Product Owner to establish the desired behaviour of the "Stealth Mode" button.

My test approach

I adopted an exploratory approach to the task, utilising Firefox (my preferred browser) and Firebug (my preferred browser debug tool) to gain an understanding of what was happening.

Firebug allowed me to quickly work out what was happening, thanks to two features:

  • I could quickly get to the relevant section of the source code, and see that when the button is clicked, the specified div gets renamed: document.getElementById('mc_embed_signup').id ='mc_embed_signup_stealthmode';
  • By watching the HTML panel when I pressed the button, I could see that this change (and only this change) was performed:
![Stealth Mode button in Firebug](/content/images/2014/Apr/Firebug.gif) *Firebug made it very easy to identify the button's action, as it highlights modifications.*

Once I was certain that I understood the scope of the impact, I moved into thinking of possible workarounds. I used knowledge/experience of similar issues that I've encountered in products I've tested (and products I've used), to establish that there were many workarounds which matched my known patterns for similar issues. I also inverted a couple of my normal web test scenarios (using High Contrast and Screen Reader modes) to show how these could be utilised to bypass the problem.

I spent 15-20 minutes on establishing (in note format) everything that's described below. I then spent another 60 minutes embellishing the report into the format that you see below, primarily for the purpose of illustrating during the webinar.

Isn't this a bit over-the-top?

For a real-world test scenario, yes. In reality I would've taken a much more lightweight approach. I suspect a simple mention-in-passing "Did you know there's a button that makes the email address field disappear?" might've been all that was needed, though if I faced disagreement/resistance/confusion then a lightweight formal bug report could be created.

But, for the purposes of showing my thought patterns and analysis, this seems like approximately the correct amount of writing :)

What is it?

The mc_embed_signup div (which controls the Email Address input field and Subscribe button) becomes renamed to mc_embed_signup_stealthmode, breaking some of the CSS rules and making it harder to enter the email address and Subscribe button.

How would you describe it?

Pressing the "Stealth Mode" button causes the font colour in the Email Address field to change from black to white, effectively rendering it invisible and preventing the user from checking whether they have entered the correct email address. The text on the 'Subscribe' button is also modified to a harder-to-read font.

Users running in Stealth Mode are likely to have a much lower newsletter signup rate, due to the increased difficulty of processing.

![Stealth Mode bug](/content/images/2014/Apr/NewsletterSignup.gif) *When 'Stealth Mode' is pressed, the Email Address field ceases to be readable.*

Secondary bug: Additionally, clicking the Stealth Mode button a second time will generate a script error (which, depending on user's browser configuration, may be surfaced to them via warnings/popups) as it attempts to rename a control which no longer exists.

![Script Error](/content/images/2014/Apr/Untitled-1.png) *Script Error appears when pressing Stealth Mode a second time.*

How many workarounds can you identify?

Here are some workarounds which will allow you to continue with newsletter signup. They're listed in ascending order of difficulty/feasibility.

(For a normal bug report, either of the first two would suffice; however, I took this webinar as a challenge to find as many workarounds as possible!)

  • Reload the page: Reloading the page will restore all of the controls to their default names, allowing the email field to become visible again.
  • Highlight all of the text in the email address field: This will make it visible so you can verify that your address is entered correctly before clicking Subscribe
  • Paste-in your email address from elsewhere: Ensure the Email Address field is empty, e.g. by pressing Ctrl-A > Delete within the field. Now type your email address outside the browser (e.g. in Notepad), copy it, and paste it into the Email Address field. The text isn't visible, but you can be certain of the content that's in the field.
  • Invalidate the field and continue: Ensure the Email Address field is invalid, e.g. by blanking it (as above) or by inserting a character which isn't valid in an email address Blank the field (Ctrl-A, Delete) or otherwise render the field contents invalid (e.g. multiple @ characters - never valid in an email address). Press Subscribe - you'll be taken to the list management page with an error, from where you can correct your address and submit it.
![Invalid email address](/content/images/2014/Apr/OneAtCharacter.png) *If you can't see your email address, just invalidate the field contents and press Subscribe - you'll get a chance to correct the error on the next page.*
  • Rename the div in-place: Use Firebug (or similar browser dev tool) to rename the div back to mc_embed_signup, which makes the CSS match again.
  • Add a new CSS rule in-place: Use Firebug (or similar browser dev tool) to add an extra rule within /css/blackops.css: #mc_embed_signup_stealthmode input { color: #000000; } - This will match the new name of the field, causing the text to become visible again.
  • Inject a new CSS file in-place: Use Firebug (or similar browser dev tool) to change the actual stylesheet from /css/blackops.css to a different custom CSS file that you've written (which may include the above rule).

Making yourself immune to the bug

During investigation of possible workarounds for the bug, I came across a couple of scenarios which would prevent you from ever encountering the bug in the first place!

  • Use the Windows High Contrast Mode: When running with a High Contrast accessibility theme, your browser will be immune to this bug, because High Contrast mode always overrides the text color in form inputs.
![Workaround with High Contrast mode](/content/images/2014/Apr/NewsletterSignup2.gif) *When running with a High Contrast theme, field text color cannot be overridden.*
  • Use a screen reader: When utilising a screen reader, input fields will be vocalized even if the CSS has made the text become invisible. Here's a Screencast video, utilising the ChromeVox extension in Chrome for demonstration purposes.

Conclusions / What did I learn?

I found this an enjoyable exercise, which (although seemingly simple on the surface) allowed me to better understand some of the thought patterns that I follow during a typical debug scenario. It also allowed me to better prepare for the Black Ops webinar discussion itself, so that I could better relate to the topics that were being discussed.

One of my most useful discoveries during this exercise was finding out about the animated screen recorder tool LICEcap. I'd taken some video clips during testing using Jing, and was struggling to work out how to embed them within my post; Jing's raw output is in .swf format, so I was searching for some way of converting .swf for upload to YouTube or .gif format, when I discovered that LICEcap would do exactly what I wanted. The animated images above were created using LICEcap, and it's a tool which I can see myself (and my colleagues) harnessing a lot in the future, for creating simple in-line embeds for our bug reports.

  • LinkedIn
  • Tumblr
  • Reddit
  • Google+
  • Pinterest
  • Pocket
Comments powered by Disqus