About Code: On documentation

In my decades writing software in the public and private sector, one of the things that seems to come up repeatedly is documentation, and the one thing I've learned about it is that it's a big deal. Nearly every engineer has an opinion about it - everywhere on the continuum. Over the years, I've heard arguments that have included strongly worded missives about how counterproductive documentation is and it should never be used to how helpful it is and engineers shouldn't consider their work complete without it. So, with all that in mind, I wanted to put down thoughts about documentation - not only why it's important and what its purpose is, but how to write good documentation.

There's a maxim in software engineering - there will always be at least two engineers on a project: you and the engineer you were six months ago. In fast-paced environments - those where features are churned out in one-week or two-week sprints - engineers may be solving multiple problems in a short period of time. In the past, we used project notebooks to keep notes about tests we ran or solutions we tried so those actions were not repeated. But, those days are gone. Today we require a different kind of collaboration - one in which all project notes are shared by everyone on the team. In environments where the problems are increasingly complex - calculating fraud and risk scores for example - it's even more important to track not only where the project is, but where it has been. Documentation should be, in part, a living history of the code.

On a side note, here, I've heard the argument that we should use source control (e.g., git) for this purpose. I would urge you, in the strongest language I could use, to not do this. Tools built into source control for tracking history are purposefully simple, often telling you only who made a change and when that change was made. It's possible, if the tool is robust enough, that you may be able to trace a change back to its original commit - and assuming that commit hasn't been amended or overwritten you may be able to find out what the change was and assuming the commit message is detailed enough you may be able to find out the why and not just the what...but there are a lot of assumptions in that process (and I have a rule about assumptions).

Documentation is not only important as history, but also for a number of other reasons.

Current practice is to write more small, concise functions than monolithic systems. In this pattern, we've gotten away from one of the purposes of documentation - it takes less time to read the documentation than reading the code. The other piece of that justification was that anyone should be able to read the documentation (which has a lower cognitive load than the code) and be able to understand what the solution to a particular was. One might argue that neither of these arguments apply in the face of advances the software engineering community has seen in the past two decades...and I might be inclined to agree, until I encounter engineers who don't understand the difference between i++, ++i, and i = i + 1. The choices that we make when crafting a solution to a particular problem are important, and the next engineer to come along is not likely to frame the same problem in the same way, but if we explain our choices the next engineer can make informed choices about where to make modifications...like is it worth the two extra bytes in i=i+1 to avoid evaluation bugs that can pop up when using an increment.

It's important for the documentation to be in the code, not somewhere else. Why? It's an accessibility issue. Too often engineers not only build interfaces that are not accessible, but they build them without using accessible practices, creating code that lacks accessibility. Placing the documentation outside of the code introduces a level of complexity that can be a significant problem. As a helpful note, I'll just include here that two simple things you can do to significantly improve the accessibility of your code is (1) use tabs instead of spaces and (2) include comments directly in the code in a common tag format, such as jsdoc.

Additionally, while I won't repeat the argument here, documentation - specifically documentation about authors, creators, and innovators - is important because it can help reduce the rampant bias that is ravaging our industry. You can read more about this particular topic in my blog post Creation, Attribution, and Misogyny.

Good documentation does not repeat the code, but explains it in plain language. If the problem is particularly complex or prone to misunderstanding, it should describe the problem as well as the solution. For example, if our problem is credit card validation, what does that mean? Are we validating the issuer number, the card number against the card type (verifying that if the user says it's a Visa™ card they provide a Visa™ card number), are we validating the number against transposition errors using the check digit, are we validating the expiry date, or are we validating the type against a list of accepted types. Any, or all, of these rules can be called 'validation', and even this plain-language list can be confusing, as rules such as check-digit algorithms can be different depending on the card type. That sort of information is not general information, even among engineers who are familiar with the industry and would be very helpful in documentation.

Good documentation also resolves odd, confusing practices engineers or organizations may have. For example, let's say you're creating a JavaScript library and you want to expose an event every time someone changes the value in an HTML input. There is a native HTML event API and there's an event that fires when a value has been changed - called change - but it only fires after the value has changed and the user has gone on to another input or task. In your library, though, you want this new event to fire when there has been any change to the value in the input, not just when all the changes to the input have been made. In this case, you would need to repurpose a different HTML event - keypress or input for example, but what would you name this new event? If you're like some, you would name this new event change because it fires when a value is changed. This would be a good point in the documentation to explain that this new event uses the same name as an existing event but that it's different.

There are a few good arguments to justify the inclusion of documentation, and no good arguments for its exclusion that I've seen work in the wild. In three decades of writing code alone and as part of teams, I have never seen self-documenting code or a sufficiently verbose source control, and I'm not anywhere near being alone in that experience.

If you are writing code, document it, and even more important, retain as much documentation as you can when modifying the code - perhaps by using the jsdoc tag @since to identify changes - your future self and your teammates will thank you for it.

Happy coding.

One For All and All For One

All for one and one for all, united we stand divided we fall. Alexandre Dumas, The Three Musketeers 

I've been putting off attempting to write what I might call "What I've learned about accessibility in twenty years" for a while - partly because even though it's based on a presentation and therefore already written (in a sense), copying PowerPoint slides as images is not very satisfying. Mostly, though, it's because it's likely to be a really long post and those often tend to not perform well. Considering what I've seen in the industry recently though, it's maybe past the time I should have done it, either way, here it goes.

Before I seriously dive into this topic, I want to share a little information about myself. Over the past (almost) two decades I've worked with accessibility in both the public sector, where I was bound by Section 508 of the Rehabilitation Act (1973), and in the private sector, where I've worked with guidelines that are now published as the Web Content Accessibility Guidelines (or WCAG). Over that time I've not only built a significant amount of what we might call "product knowledge" about accessibility, but have built quite a bit of passion for the work as well. I'm going to attempt to share that passion and attempt to convince you to become what I call an "Acessibility Ally" (A11y*ally, A11y^2, or "Ally Squared") - someone who is actively supportive of a11y, or web accessibility.

What Is This Accessibility Stuff, Anyway?

A lot of discussions about interface accessibility start with impairment. They talk about permanent, temporary, and situational impairment. They talk about visual, auditory, speech, and motor impairment (and sometimes throw in neurological and cognitive impairment as well). They'll give you the numbers...how in the US, across all ages,

• approximately 8 percent of individuals report significant auditory impairment (about 4 percent are "functionally deaf" and about 4 percent are "hard of hearing", meaning they have difficulty hearing with assistance)
• approximately 4 percent of individuals report low or no vision (which are not the only visual impairments)
• approximately 8 percent of men (and nearly no women) report one of several conditions we refer to as "colorblindness"
• nearly 7 percent of individuals experience a severe motor or dexterity difficulty
• nearly 17 percent of individuals experience a neurological disorder
• nearly 20 percent of individuals experience cognitive difficulties

They might even tell you how all those numbers are really First World numbers and when you go into emerging markets where reliable access to resources is more limited the numbers double. They'll talk about how, in general, those numbers are skewed by age and how about 10 percent of people under the age of 65 report impairment while more than 25 percent of people between the ages of 65 and 74 report impairment (and nearly 50 percent of those 75 and older report impairment).

I don't generally like to start there...though I guess I just did. Accessibility is not about the user's impairment - or at least it shouldn't be - it's about the obstacles we - the product managers, content writers, designers, and engineers - place in the path of people trying to live their lives. Talking about impairment in numbers like this also tends to give the impression that impairment is not "normal" when the data clearly shows otherwise. Even accounting for a degree of comorbidity, the numbers indicate that most people experience some sort of impairment in their daily lives.

The other approach that's often taken is diving directly into accessibility and what I call impairment categories and their respective "solution". The major problem here is a risk similar to what engineers typically refer to as "early optimization". The "solutions" for visual and auditory and even motor impairments are relatively easy from an engineering point of view, even though neurological and cognitive difficulties are far more significant in terms of numbers. Rather than focus on which impairment falls into the four categories that define accessibility - Perceivable, Operable, Understandable, and Robust - we have to, as I like to say, see the forest and the trees. While there is benefit in being familiar with the Success Criteria in each of the Guidelines within the WCAG, using that as a focus will miss a large portion of the experience.

One other reason I have chosen this broader understanding of accessibility is that accessibility in interfaces is holistic. Everything in the interface - everything in a web page and every web page in a process - must be accessible in order to meet the definition of accessible. For example, we can't claim a web page that meets visual guidelines but not auditory guidelines "accessible", and if the form on our page is accessible but the navigation is not then the page is not accessible.

Why is Accessibility Important?

When considering accessibility, I often recall an experience in interviewing a candidate for an engineer position and relate that story to those listening. This candidate, when asked about accessibility, responded something along the lines of "do you mean blind people - they can't see web pages anyway". I've also worked with designers and product managers who have complained about the amount of time spent building accessibility interfaces for such a "small" group of users or flat out said accessibility isn't a priority. I've worked with content writers who are convinced their writing is clear enough for their intended audience and anyone confused by it is not in their intended audience - what I call the Abercrombe and Fitch Content Model.

For those who consider accessibility important, there are a few different approaches we might typically take when trying to sway those who tend be less inclined to consider the importance of accessibility. In my experience, the least frequently made argument for the importance of accessibility is one of a moral imperative - making an interface accessible is the "right thing to do". While I agree, I won't argue that point here, simply because it's the least frequently made argument and this is going to be a post that bounds the too-long length as is.

The approach people most frequently take in attempting to convince others accessibility is important is the anti-litigation approach. Making sure their interface is accessible is promoted as a matter of organizational security - a form of self-protection. In this approach, the typical method is a focus on the Success Criteria of the WCAG Recommendation alongside automated testing to verify that they have achieved A or AA level compliance. The "anti-litigation" approach is a pathway to organizational failure.

Make no mistake, the risk of litigation is significant. In the US, litigation in Federal court has increased approximately 400 percent year-over-year between 2015 and 2017, and at the time of this writing appears to be growing at roughly the same rate in 2018. Even more significant, cases have held third parties accountable and have progressed even when remediation was in progress, indicating the court is at least sometimes willing to consider a wider scope than we might typically think of in relation to these cases. To make matters even more precarious, businesses operating internationally face a range of penalties and enforcement patterns. Nearly all countries have some degree of statutory regulation regarding accessibility, even if enforcement and penalties vary. Thankfully, the international landscape is not nearly as varied as it was, as nearly all regulations follow the WCAG or are a derivative of those guidelines.

So, why, when the threat of litigation both domestically and internationally is so significant, do I say focus on the Success Criteria is a pathway to failure? My experience has repeatedly shown that even if all Success Criteria are met, an interface may not be accessible - an issue I'll go into a little further when I talk about building and testing interfaces - and only truly accessible interfaces allow us to succeed.

What happens when your interface is not accessible - aside from the litigation already discussed? First, it's extremely unlikely that you'll know your interface has accessibility issues, because 9 of 10 individuals who experience an accessibility issue don't report it. Your analytics will not identify those failing to convert due to accessibility issues - they'll be mixed in with any others you're tracking. Second, those abandoned transactions will be costly in the extreme. In the UK, those abandoning transactions because of accessibility issues account for roughly £12 billion (GBP) annually - which is roughly 10 percent of the total market. Let me say that again because it deserves to be emphasized - those abandoning because of accessibility issues represent roughly 10 percent of the total market - not 10 percent of your market share - 10 percent of the total market.

Whether your idea of success is moral superiority, ubiquity, or piles of cash, the only sure way to that end is a pathway of accessibility.

How Do We Become an Accessibility Ally?

Hearing "it's the right thing to do" or "this is how we can get into more homes" or, sometimes the £12 billion (GBP) number - one of those often convinces people to become at least a little interested in creating accessible interfaces, even if they're not quite to the point of wanting to become an Accessibility Evangelist. The good news is that even something as simple as making creating accessible interface a priority can make you an Accessibility Ally.

The question then becomes how to we take that first step - how do we create accessible interfaces. The first rule you have to know about creating an accessible interface is that it takes the entire team. Accessibility exists at every level - the complexity of processes (one of the leading causes of abandonment), the content in the interface, the visual design and interactions, and how all of that is put together in code by the engineers - all of it impacts accessibility.

At this point, I should give fair warning that although I'll try to touch on all the layers of an interface, my strengths are in engineering, so the Building and Testing Interfaces section may seem weighted a little heavier, even though it should not be considered more important.

Designing for Accessibility

If we were building a house we wanted to be accessible, we recognize that we would have to start at the beginning, even before blueprints are drawn, making decisions about how many levels it will have, where it will be located, and how people will approach it. Once those high-level decisions are made, we might start drawing blueprints - laying out the rooms and making sure that doorways and passages have sufficient space. We would alter design elements like cabinet and counter height and perhaps flooring surfaces that pose fewer navigation difficulties. To remodel a house to make it accessible, while not impossible, is often very difficult...and the same concepts apply to building interfaces.

Most projects that strive to be fully accessible start with Information Architecture, or IA (you can find out more about IA at https://www.usability.gov/what-and-why/information-architecture.html). This is generally a good place to begin, unless what you're building is an interface for a process - like buying or selling something or signing up for something. In the case of a process interface, you've basically decided you're building a house with multiple levels and you have accessibility issues related to traversing those levels...to continue our simile, you have to decide if you're going to have an elevator or a device that traverses stairs...but your building will still need a foundation. Information Architecture is like the foundation of your building. Can you build a building without a foundation? Sure. A lot of pioneers built log cabins by laying the first course of logs directly on the ground...but - and this is a very big but - those structures did not last. If you decide to go another route than good IA, the work further on will be more difficult, and much of it will have to be reworked, because IA affects a core aspect of the Accessibility Tree - the accessible name - the most critical piece of information assistive technology can have about an element of an interface.

Once your Information Architecture is complete, designing for accessibility is considerably less complex than most people imagine it to be. Sure there are some technical bits that designers have to keep in mind - like luminance contrast and how everything needs a label - but there are loads of good, reliable resources available...probably more so for design than for the engineering side of things. For example, there are several resources available from the Paciello Group and Deque, organizations who work with web accessibility almost exclusively, as well as both public and private organizations who have made accessibility a priority, like Government Digital Service, eBay, PayPal, and even A List Apart.

With the available resources you can succeed as an Accessibility Ally as long as you keep one thought at the fore of your mind - can someone use this interface the way they want rather than the way I want. What if they search a list of all the links on your site - does the text inside the anchor tell them what's on the other side? What if they're experienced users and want to jump past all the stuff you've crammed into the header but they're not using a scrollbar - is there something that tells them how to do that? Keep in mind that as a designer, you're designing the interface for everyone, not just those who can [insert action here] like you can.

Building and Testing Interfaces

When building accessible interfaces, there is a massive amount to learn about the Accessibility Tree and how and when to modify it as well as the different states a component might have. Much has been made of ARIA roles and states, but frankly, ARIA is one of the worst (or perhaps I might say most dangerous) tools an engineer can use.

We're going to briefly pause the technical stuff here for a short story from my childhood (a story I'll tie to ARIA, but not till the end).

When I was a child - about 8 years old - my family and I visited a gift shop while on vacation in Appalachia. In this particular gift shop they sold something that my 8 year old mind thought was the greatest thing a kid could have - a bullwhip. I begged and pleaded, but my parents would not allow me to purchase this wondrous device that smelled of leather, power, and danger. I was very dismayed...until, as we were leaving, I saw a child about my age flicking one and listening to the distinctive crack...until he snapped it behind his back and stood up ramrod straight as a look of intense pain crossed his face.

ARIA roles and states are like that bullwhip. They look really cool. You're pretty sure you would look like a superhero with them coiled on your belt. They smell of power and danger and when other engineers see you use them, you're pretty sure they think you are a superhero. They're very enticing...until they crack across your back.

Luckily, ARIA roles and states are almost unnecessary. Yes, they can take your interface to the next level, but they are not for the inexperienced or those who lack caution. If you're creating interfaces designed for a browser, the best tool you have to build an accessible interface is Semantic HTML. Yes, it's possible to build an interface totally lacking accessibility in native HTML. Yes, it's possible to build an accessible interface in Semantic HTML and then destroy the accessibility with CSS. Yes, it's possible to build an accessible interface with JavaScript or to destroy an accessible interface with JavaScript. None of the languages we use in front-end engineering build accessibility or destroy accessibility on their own - that takes engineers. The languages themselves are strong enough...if you are new to accessibility, start somewhere other than the bullwhip.

The next topic most people jump to from here is how to test an interface to make sure it is accessible. This is another place where things can get tricky, because there are a number of different tools, they all serve a different purpose, and they may not do what they're expected to do. For instance, there are tools that measure things like luminance contrast, whether or not landmarks are present, or if any required elements or attributes are missing - validating according to the Success Criteria in the WCAG. In this realm, I prefer the aXe Chrome plug-in (by Deque). Nearly all these tools are equally good at what they do, but - and here's one of the places where it can go sideways - tools that validate according to the Success Criteria are a bit like spellcheckers - they can tell you if you spelled the word correctly, but they cannot tell you if you've selected the correct word.

Beyond Success Criteria validation, there are other tools available (or soon to be available) to help verify accessibility, the most common of which are screen readers. Of screen readers available, some are free and some are paid - VoiceOver on Mac and JAWS on Windows are the most popular in the US - JAWS is not free, but there is a demo version you can run for about 40 minutes at a time. NVDA (another Windows tool) and ChromeVox are free, but less popular. In addition to screen readers, in version 61 of Firefox the dev tools should include a tool that gives visibility into the Accessibility Tree (version 61 is the planned release, this version is not available at the time of this writing).

One thing to remember with any of these - just because it works one way for you doesn't mean it will work that way for everyone. Accessibility platforms are multiple tools that share an interface. Each tool is built differently - typically according to the senior engineer's interpretation of the specification. While the results are often very similar, they will not always be the same. For example, some platforms that include VoiceOver don't recognize a dynamically modified Accessibility Tree, meaning if you add an element to the DOM it won't be announced, or it may only be announced if certain steps are taken, and the exact same code running in JAWS will announce the content multiple times. Another thing to remember is that there is no way you will ever known all the edge cases - in the case VoiceOver not recognizing dynamically added elements mentioned previously, it took more effort than it should have to demonstrate conclusively to the stakeholders the issue was in a difference in the platform.

Finally, when you're trying to ensure your interface is accessible, you will have to manually test it - there is simply no other way - and it should be tested at least once every development cycle. Granted, not every user story will affect accessibility, but because have that holistic view of accessibility that acknowledges that accessibility exists at every level, we know that most stories will affect accessibility.

As with design, there are resources available, but good resources are more difficult to find because engineers are opinionated and usually feel like they understand specifications, even though what they understand is their interpretation of the specifications. If you want to become an accessibility expert, it can be done, but the process is neither quick nor easy. If you want to become an A11y^2, well that process is quicker and easier and mostly consists of keeping everything said in this section in mind. Understand accessibility holistically. Make "Semantic HTML" and "ARIA is a last resort" your mantras. Check your work with one of the WCAG verification tools (again, I prefer the aXe Chrome plug-in) and at least one screen reader. Check it manually, and check it frequently.

Being an Accessibility Ally

Being an Accessibility Ally is really not complicated. You don't need to be an accessibility expert (though you certainly can be one if you want)...you just need to see accessibility as a priority and the pathway to success. Being an Accessibility Ally means you're actively supportive of accessibility.

To be actively supportive, one needs to understand accessibility in a more holistic way than we've traditionally thought about it and we need to understand that not only does accessibility accumulate, its opposite accumulates as well. In other words, inaccessibility anywhere is a threat to accessibility everywhere.

To be actively supportive, we need to do more than act the part by designing and building things like stair-ramps with ramps too steep to safely navigate with a wheelchair or Norman doors. We need to make building interfaces that are perceivable, operable, understandable, and robust a priority...and we need to make that priority visible to others.

When we're actively supportive and people see our action, only then will we be the ally we all need...and we all need all the allies we can get.

---

For another take on age and web interfaces, you may want to take a look at "The Danger of an Adult-oriented Internet", a post in this blog from 2013 or A11y Squared, a post from 2017. 

The Myth of Analytics

I've been writing web interfaces since the mid 1990s - which in this industry is eons.

When I first started, we compiled analytics directly from the web server, watching primarily how many visitors there were, how long it took to push the page(s) out, and what paths they followed when they came to the site. Looking at the data that showed what visitors were reading was helpful - we were able to see what the most popular pages were and what the most popular path was. For merchants, this gives a window into cross-selling opportunities and more.

As technology grew, we noticed that the longer it took to download pages, the less likely people were to buy stuff. So we started tracking page weight a little more closely...and then, as we realized that a 100KB image is not the same as 100KB of HTML, which is not the same as 100KB of CSS, which is not the same as 100KB of JavaScript, we started looking at the weight of each of the resources and then, ultimately, this mysterious measure we now call 'time to interactive'.

I should take a minute to point out that all of these numbers are important, and if you're not keeping track of all of them, and if your [insert your name for front-end developers here] don't understand the difference between each of these categories and why they are each important, you're setting yourself up for failure.

Here's the thing with this shift, though...the only way to get numbers directly from the user's machine (sometimes 'Real User Measurement', or RUM) is using JavaScript. So, the tool we use to measure our performance has to first be delivered to the browser in order to do the measurement.

So the first problem in this shift from server to client measurement is that the total amount of time cannot be accurately measured - it's not possible.

The closest we could come is to calculate the difference between the time (on the server) the response packet was pushed out and the time the request packet fired by the 'interactive' event was received. However, even this time is unreliable because it has two network traversals in it, and while there is a way to measure the upstream time, there is no way to measure the downstream time (and I have yet to see anything that will measure even upstream time).

The second problem in this shift from server to client measurement is that it relies on JavaScript.

Over the course of my career, I've heard numerous times that this reliance on JavaScript is not an issue, because the majority of users have JavaScript enabled, but again, we come into a number that cannot be accurately measured. In order for a user to be measured...

1. they have to get our response that contains the JavaScript code that sends the request(s) to the appropriate recording mechanism on the server
2. that code has to be loaded and compiled by the browser
3. that code cannot conflict with or be dependent on any other code that may or may not be loaded

...and even if all that can happen, it has to happen before the user gets frustrated enough to click away. It's also not a good idea to rely on all three of those stars aligning all the time - there have been instances where developers have included references to code that isn't loaded which have brought down entire sites. When your site goes down in this manner, it can become very difficult to resolve because you have no idea where the problem lies.

One group measuring interactions of a first-world community with a relatively high degree of trust (a group that would have little to no reason to have JavaScript disabled or otherwise unavailable) found that nearly 3 percent of interactions were untracked by their client-side solution.

If we assume, however, that everything goes well. All of our code is delivered to the end user, and there are no conflicts, and all the request packets are coming in and we're able to measure everything - even the upstream traversal time. All of this data gives us a table we'll call "page views by browser", which contains the following records - "Chrome, 67, 55%; Safari, 29, 24%; Firefox, 19, 15%; Other, 5, 6%".

What does this data tell you? It tells you that 67 visitors used Chrome, 29 visitors used Safari, and 19 visitors used Firefox. That's all. Even if you add the version(s) used and what the operating system was and you were able to tell which were mobile versus desktop users, that information is not typically tied into conversion information...and it doesn't tell you anything about the 5 people who stuck around long enough to be tracked but were small enough numbers that their information is not reported.

Additionally, if we were to combine the 6 percent in the "Other" category with the 3 percent that were untracked, that's 10 percent of customers you have very little information about.

The third problem in this shift from server to client measurement is that it does not measure anyone in the 'click away' category. Those who have everything loading properly but who get bored by the length of time your site is taking to respond.

Of course there are ways to correct some of the deficiencies in this shift from server to client measurement. There are also ways to address some of the issues your improved analytics can identify. You have to be willing to put in the work to identify where your analytics package/process is lacking analysis and improve it...or, alternatively, you can continue believing the myth that your 'analytics' is actually describing your (potential and actual) customers rather than just describing a subset of your customers.

I'd encourage you to review the data you're collecting and see the shortfall(s), from that point on it's just a matter of creative coding to rectify the issues.

Happy coding.

Why you should never code for new browsers

A quick, very short post today.

First, I should preface this by saying that I've been writing human/computer interfaces for a very long time...and for a good part of that time - it's been more than two decades now, I've been writing for these crazy user-agents we call browsers. The Mosaic Wars are a little fuzzy, but I remember the First Browser War vividly.

Every great once in a while, I see a question along the lines of "is x compatible with some-really-old-browser" and there's always a response along the lines of "no one uses that browser anymore, if you're coding for anything older than n - 1, then you're wasting money".

I can say with 100 percent certainty, that if you're coding for specific browsers - be they old or new - then you're wasting money.

Browsers are released very frequently - for Chrome and Firefox, it's typically about every 6 weeks. The specs they claim to support are release much less frequently...for example, a new HTML spec was release every 1 or 2 years between 1995 and 1999 when HTML 4.01 came out, but the next spec - HTML5 - didn't come out for 14 years. CSS first came out in 1996 and was updated about every 2 years until CSS 3 came out (in 1999). 

What's that I hear? A host of voices singing the popular refrain "but we use JavaScript"? JavaScript is finally to the point where it is currently being updated annually - which makes it the fastest changing language of front-end development and still much slower than 6 to 8 times a year.

Add into this mix of rapidly updating browsers the mix of assistive technology (with its own specifications), and we begin to get a picture of extremely volatile user-agents and very stable specifications.

If there was one thing we should have learned from the First Browser War, it's that we should be not be coding according features that are available in one browser that we then try to port to another browser with extra work. Determine what your needs (and wants) are, code to the specification, and let the browsers catch up. Browsers update much faster and are very likely to render your code as you want it to look within the year without additional work from you. Also, this approach is as backwardly-compatible as possible (what in the accessibility world we call being robust)...and if you're not attempting to deliver a product to every end user, regardless of the user-agent they use, then you really are wasting money.

Happy coding.

Hiding in Plain Sight: How to affect what you can't see in your analytics

What if I told you there is a way to improve your Google rank (placement) and significantly increase conversion and that you would not be able to discover how to do it by looking at your analytics? Would you do it? Would it matter what the delta was or would it matter what work was involved?

Several years ago, I worked on a project at PayPal with the sole purpose of improving conversion during checkout. A few "features" were added, but the majority of the project was running A/B tests for (minor) design and content changes for the existing product, so the amount of actual development work was minimal aside from refactoring existing code.

One might think such a project would yield as much as one hundred basis points, but realistically, since these were "minor" changes it should have yielded about half that...but the thing is, it was twice that amount. The most reasonable explanation we might have for this remarkable difference is that we were affecting users that were not even on our radar.

Years before that project, as a side project I took over the website for a non-profit. Before running any A/B tests or modifying design or content in any way, I refactored the code to use semantic HTML and CSS and their rank went from fifth to first overnight. Nearly all the research up to that point indicated that selection of keywords and inbound and outbound links were the primary source for SEO, but here again, something hidden had a significant impact.

In both of these projects, it was the use of solid, time-tested best practices of web development that yielded these results, in particular, Semantic HTML and CSS, unobtrusive JavaScript, and Progressive Enhancement. It was what is ordinarily hidden from users and even those in decision-making roles in technology that had the greatest impact.

Frankly, it seems like an outlandish claim, and had I not seen them firsthand, I would suspect such claims were the usual puffery in which engineers engage - sort of like boosting performance by switching turbochargers. After all, we're often told that the best tools are the newest tools and that time-to-market is the most important factor. Abundant anecdotal evidence demonstrates that neither of those beliefs are based in reality.

Again, here are the four principles behind these transformations.

1. All HTML was Semantic HTML.
2. All CSS was Semantic CSS.
3. All JavaScript was unobtrusive.
4. Sites were built using Progressive Enhancement.

I should point out that it's possible to use any of these principles alone, and using any of them will give you some of the benefits. The goal when using these practices were to create pages that are

• fast,
• flexible, enabling a user to complete the task in any user agent,
• easily maintained

The primary goal in these projects was to get a page loaded in less than 4 seconds anywhere in the world. Because network reliability and speed are always unknowns outside the US, and even some places within the US, that benchmark meant the page had to load on an internal network in considerably less time. (It also meant I spent time writing a module that would allow me to simulate network rates and reliability that exist in the real world outside the US to more accurately gauge speed...but that's another story.)

The secondary goal was to build pages that would run in any user agent well enough to allow the user to accomplish the task in the shortest amount of time, in part because we know that the amount of time a process takes is inversely proportional to the likelihood of the user completing that process.

The simple truths that govern this secondary goal are that semantic HTML is easier for user agents other than browsers, like the bot(s) that rank your site, to understand; semantic CSS enables you to write fewer, better-performing rules; and unobtrusive JavaScript and Progressive Enhancement expand your audience into non-traditional user agents with the same codebase.

The Problem with Analytics

Examining the effect of these four principles is difficult, in part because of the way data collection for web analytics changed in the first decade of the 21st century. To simplify it, I'll use a mostly hypothetical example.

Let's say you're tracking user agents and see that seventy-three percent of your traffic is coming from Chrome, twenty-one percent is coming from Safari, and five percent is coming from Firefox - like the stats for this site report. Already, one percent is not reported, and if, as is the case for most analytics, your analytics package is using JavaScript to track that rather than the actual web server logs, there are even more users not tracked at all. It's unlikely that you will ever get insight into these users, which is why it has become so easy to dismiss the effect of certain principles, Progressive Enhancement being principle among them.

Progressive Enhancement

One percent is a small number, is it really enough to justify the amount of work Progressive Enhancement requires? On one hand, one percent likely is a small number - unless you're a company like PayPal, with more than 110 million active users - and on the other hand, that one percent represents a much larger number because it's the one percent we know exists. How many users fall into that not-the-popular-browsers group that typically hovers around "one percent", but aren't even measured because their browser isn't identified?

A government agency in the UK tracked visitors to government websites using actual web server logs to determine what percentage of visitors weren't being tracked by their JS analytics package and it was about two percent. Two percent. On government websites. Within a first-world country. For those government websites, that "one percent" is really three percent.

Retail websites are categorically different than government websites, so attempting to extrapolate user behavior in a government-assisted activity to private activity is a little like comparing apples and oranges. The user population in each group has different needs, desires, and expectations, and as a result, have different behavior. All we need consider is the desires and expectations around privacy between the two different categories to see there is a very clear difference in behavior. We can, however, easily support the case that the percentage of tracked users interacting with a government website is likely higher than those interacting with non-government websites. As a result, there is no way to determine what percentage of your actual traffic is untracked unless you are analyzing server logs rather than relying directly on analytics served by JavaScript. The principle of Progressive Enhancement, however, means even these untracked users are served, regardless of the user agent or its features.

Performance Matters

Since we are unlikely to gain insight into the untracked users, what does the data we collect about the tracked group tell us? The primary message we get from the data is that, performance is the most significant issue in web development - it's typically even more important than content. This is not something we've just learned - we've known it for decades. It's also not something that's unmeasured. Organizations frequently calculate the cost of each page or step in a process by calculating the abandonment rate and the drop off rate based on page load time or time-to-interactive, what Twitter used to call "time to first tweet". In a recent, rather finely-tuned test by etsy.com, we see a high correlation between the addition of a mere 160 kilobytes of page weight and a twelve percent drop off.

Semantic Code

Semantic CSS, writing CSS in a way that actually describes something about the code, and semantic HTML, writing HTML in a way that describes the content it contains, expands the robustness of the code (one of the measures of accessibility), is more maintainable, and reduces the amount of code delivered. Semantic code also performs better - there are fewer class and ID selectors to process and a cleaner, more compact Accessibility Tree is built.

Also, lighter pages load faster, and that's important because one of the lessons we've learned from gathering data about tracked users shows that nearly forty percent of users will abandon your site if it takes your page longer than three seconds to "load". If your HTML suffers from divitis and other non-semantic features, or if your CSS is bloated with non-semantic rules, it will take longer to load and parse, pushing the load time further out.

Even worse is the effect non-semantic HTML has on the Accessibility Tree. If your HTML is not semantic, you will have to add code to "fix" the Accessibility Tree, to make your code accessible. Unfortunately, as you add code the risk of harming accessibility even further raises more than the chance you will resolve issues. Here the problems become even more pronounced, because we run into a sub-group of "untracked" users within the tracked users - users who combine accessibility tools with their user agent.

Generally, we only identify site visitors using accessibility tools when they report an issue. There is no method available to determine if a user has engaged an accessibility tool. Conservative estimates generalize that for every accessibility issue registered, at least ten users have been blocked and are unable to complete their task. All industry experts agree about this premise - use of semantic HTML is the best way to minimize the accessibility risks.

A Hypothetical Example

Let's assume that you've looked at your target market and found that seventy-three percent use Chrome, twenty-one percent use Safari, and five percent use Firefox. Almost all - ninety-nine percent - of your users are using agents that have JavaScript enabled by default and you're not aware of "untracked" users. Based on this fantastic news, you've developed your site using ReactJs or Angular and you've tested it locally and the render time is pretty good - it only takes one or two seconds from the request until it's interactive from your development server.

Here's how that plays out in the real world. Your (average) page is really about eighty kilobytes of HTML, about ninety kilobytes of CSS, and almost two megabytes (or more) of JavaScript. If you've not used React's styled elements, your CSS is entirely separate, but nearly all of your eighty kilobytes of HTML is still embedded in the JavaScript. That JavaScript, which has to be entirely loaded before it can be parsed (to make sure there are no syntax errors) and rendered (or hydrated if you've used React's "server-side rendering") is on a CDN, so there's very little latency in the network, but on a 4G connection, mobile devices have a peak of about one-tenth the speed of your local network connection, which means at 100 percent reliability and speed, it's going to take about ten times as long to download the file(s) on a mobile device.

If we assume that you've really optimized and compressed and tweaked your code in production all the ways it's not optimized, compressed, and tweaked in development and your development network is not really that great, the difference between time to interactive on your development site for the more than two megabytes of code is about half of what it will be in production.

So your development version loads in about two seconds, which is pretty fast, and you're pretty confident. In production, your JavaScript resources load HTML into your page in four seconds (versus the two seconds in development) instead of two, but you decide that's still good. Of course, we know you've still lost forty percent of users. Unfortunately, what we now know is that's forty percent of tracked users.

Your beautifully designed site, which only renders for user agents with JavaScript enabled (rendering a "white screen of death" for at least three percent of visitors) unfortunately hasn't undergone an accessibility or usability review and was not written using semantic code. As a result of this oversight, your site has several color contrast issues, navigation is not identified, and your data collection form is not properly labeled - and some unknown portion of the roughly fifty percent of visitors who have made it to your page unscathed cannot complete the task. Here is where we see the final indignity - our analytics cannot tell us why that percentage does not convert, even if it tells us that they do not convert.

An Alternative Example

As an alternative, if your site were written with semantic code using Progressive Enhancement, the code would be lighter, loading in less than four seconds. Your analytics may not have changed, but by using Progressive Enhancement, no one gets a white screen of death, even those in the untracked category, which simply become a hidden bonus rather than a missed opportunity, because the technology fails gracefully.

The semantic code is, by its nature, accessible, and eliminating accessibility issues and offering an estimated ten-fold benefit that taps into an otherwise untracked market - a market that Barclays estimated in 2017 as worth twelve billion pounds, which is not a small number for any organization.

Conclusion

We know sites written with semantic code and unobtrusive JavaScript using Progressive Enhancement

• perform faster
• have greater accessibility
• have greater stability
• have greater durability
• are more maintainable

The question is what are these qualities worth to you - do you continue on as usual or do you try to tap into what is hidden in your analytics. We tend to know a lot of information about what we know - it's what we don't know where the greatest promise rests.

Happy coding.

Git 'r Done

Wow, it's already far into the new year, and I've not written anything since before the winter (at least in my hemisphere) holidays. Time marches on, though, so this is a short post that maybe will make things a little easier.

One of the things I've come to rely on is the ubiquity of git. Seems like nearly every company is using it, whether it's github, or some internally hosted version...or even something, like Bitbucket, that uses git as its engine.

There are a lot of arguments about how best to use git - some people (*cough* wrongly *cough*) suggest that all development ought to be done in a fork and merged back into the communal repo and some use a clone/branch model. One argument against the clone/branch model is the number of branches that become stale - and frankly, it can become a mess.

One thing you, as a maintainer, will need to do if you're using a clone/branch model is establish, fairly early, a process for pruning those stale or dead branches. Once you have a process in place, though, what do you do? How do you list the branches and their detail in a convenient way?

One way is the following...

git for-each-ref --sort=authordate --format='%(authordate:iso8601) %(refname) by %(authoremail)' refs/remotes

This will give a list that can be used to either manually delete branches, or you could use this within a script that takes some action, like emailing or deleting, when the ref matches some criteria. Here's some sample data in case you want to build such a tool.

2017-07-27 22:29:31 -0400 refs/remotes/origin/master by <hrobertking@mailinator.com>
2017-10-03 11:31:58 -0700 refs/remotes/origin/lint-config by <hrobertking@mailinator.com>
2017-11-01 11:15:33 -0700 refs/remotes/origin/feature/US123456789 by <james.howlett@mailinator.com>
2018-01-10 14:26:58 -0700 refs/remotes/origin/feature/US234567890 by <james.howlett@mailinator.com>
2018-01-23 15:51:15 -0700 refs/remotes/origin/bugfix/DE123456 by <ororo.munroe@mailinator.com>
2018-01-25 16:51:50 -0700 refs/remotes/origin/bugfix/DE234567 by <remy.lebeau@mailinator.com>
2018-01-30 12:00:58 -0500 refs/remotes/origin/bugfix/DE345678 by <ororo.munroe@mailinator.com>
2018-01-31 17:47:52 -0500 refs/remotes/origin/feature/US345678901 by <hrobertking@mailinator.com>
2018-02-02 14:36:42 -0700 refs/remotes/origin/bugfix/DE456789 by <hrobertkinging@mailinator.com>

If you're looking for other git things to do, feel free to check out my github git repo, where you'll find a sample .gitconfig with aliases that have been tested in the bash terminal on a Mac. Most also run on PuTTY or other clients that can run a bash session, but a few will need to be modified.

Hopefully, this will help you "git 'r done" (after thinking about what you need to do, of course).

Happy coding.

Visibility and Obscurity

Several years ago, when I first started at PayPal, the front-end development environment was still fairly young. As a result, tools that might have existed in other environments were missing.

As a veteran coder, I quickly grew tired of repetitive tasks - I wanted to be writing code - and set about writing scripts that developed into a significant tool suite. I shared that tool suite with both front-end and back-end developers (there were no full-stack developers in those days) and the use of those tools spread throughout the company, across the globe.

Out of that activity, there were two different experiences that bear examination. I'll address the later of the two experiences first.

In later years, as the development environment matured, another engineer - one responsible for establishing a standard development environment - took control of the tool suite (totally understandable) and put his name on my work (not understandable). The tools I had birthed and nurtured through numerous changes in the development environment, and continually promoted so they would be visible to all engineers - were adopted and their new foster father promoted himself as their creator when they became visible to upper management.

This is not an unusual situation. It happens all too often - much more frequently to women, of course - that someone other than the individual who has done the work takes credit, especially as the work becomes more visible.

That experience taught me two lessons. First, how you handle it says volumes to those who see the situation. Second, obscurity can be moments away, behind someone else's shadow, even when you think the visibility you've worked to cultivate over years is secure.

The second experience was much more pleasant. On a regular visit to a development office, I was introduced to an engineer who had recently joined the company. The engineer and I exchanged pleasantries - the normal "nice to meet you" bit - and then the engineer who introduced us told her my username (which was explicitly tied to the aforementioned tool suite)...and her expression and demeanor shifted dramatically. As someone who's never been in the "popular" club (yes, I've been a nerd and geek since before secondary school), that reception was quite an ego boost.

I had no real expectation of receiving such a reception - none of my long-time friends who'd seen me develop the tools reacted in the same manner - and it caught me by surprise. That reception also taught me a lesson - there will be some ways in which you're always more visible than you believe you are.

History is eager to write out of the picture those who have struggled to build great things - whether it's a woman who's made a significant contribution to our community (like Nicole Sullivan, the creator of OOCSS) or a man who is more interested in the work than the credit (like Nikola Tesla).

When you find yourself in these situations - situations of visibility and/or obscurity - how you navigate those shoals says volumes about your ambition, your drive, your values - such as integrity and trust, and what you know to be true about yourself. In those situations, may you have fair winds and running seas.

Happy coding.

Oh, the places you'll go

One of the things that took the most effort building the eBay checkout interface for PayPal was making sure everything appeared correctly no matter in which area you lived. To show how different this is, let's look at a few differences between my two favorite countries - the United States and Ireland.

Let's look at three simple, obvious examples first...like an address

The Lord Mayor of Dublin	The President of the United States
Mansion House Dawson Street Dublin 2 D02 AF30 Ireland	The White House 1600 Pennsylvania Ave NW Washington, DC 20500 United States

If we think of this as a microformat, both have a street-address - Dawson Street or 1600 Pennsylvania Ave NW - and both have a locality (city) - Dublin or Washington - and both have a country...and they even (now) both have a postal-code. The number after the city name in the Lord Mayor's residence works something like a postal code, but it's not - and it's also not the locality, it's a region within the city. Additionally, the Ireland address also has no region...and as with many street addresses in Ireland, there is no building number. Nearly every country has its own addressing scheme - and yes, there are addresses written in a format similar to the US format - but there are often differences and they seldom end with the physical address.

Telephone numbers are displayed differently. For example, the Lord Mayor's telephone number is currently shown on the website as 01 222 6200 (which leaves off the country code, though to be fair I'm sure it's seldom that anyone calling the Lord Mayor from outside Ireland would not know to dial 353 beforehand) and on the website for The White House the switchboard is listed as 202-456-1414 (again leaving off the country code). The minor difference here is that in the US there is nearly always a clear delineation between the parts of a telephone number - e.g., the area code is first and is typically either followed by a hyphen or a space and sometimes it is wrapped in parentheses and sometimes it's missing altogether. The central office portion (that's the 456 bit in the number for The White House) is followed by a hyphen but sometimes a dot - I don't recall the last time I saw it connected directly to the line (e.g., 4561414) or the last time I saw it followed by a space.

Finally, let's look at dates. I don't mean those tasty bits of chewy goodness that grow on tall skinny trees, I mean days of the year. In the US you'll nearly always see them collected and displayed as month followed by a slash, then date followed by a slash, then year - for example, 12/31/1999 for New Year's Eve in 1999. The US is nearly the only country where you'll see this month-day-year format. In most other countries, including Ireland, you're much more likely to see day-month-year (which you'll also see if you're looking at a US government document, but in that case you're likely to see that same date as 31-DEC-99).

Why does all this matter?

First there are the little miscommunications - like when the British Prime Minister was scheduled to visit The White House and several individuals were not admitted because the dates they provided as birth dates didn't match the date security personnel had.

Second, there are bigger issues - for example goods and money can be misrouted, negatively affecting any number of things such as someone's life or livelihood.

For people focused, in even a minor way, on developing software people have to actually use, there are two points of impact. First, having to adjust for myriad tiny differences increases the cognitive load - which is a significant contributor to conversion. Second, the amount of time spent localizing an application can be significant, because not only do interaction points - like date collection inputs - have to be localized, designs have to accommodate longer words without breaking.

My experience in working with both Fortune 500 corporations and small consulting firms has shown me one consistent truth - US firms focus on developing for the US market first. Granted, the US is a big market, but imagine if you were an auto manufacturer and you learned that every car you produced had to be customized for the customer. It's not that every customer wants a different color, it's that one customer wants four doors, another two, the next wants five, and yet another wants one and he wants it to open vertically behind the motor so he can climb over the steering column. You'd likely say that's nuts - we'll build a car with four doors and everyone will use the same car. There's a legend that Henry Ford said something like that - you could have any color Ford motorcar as long as it's black - and he did get away with it, for a time. In these days of being globally aware, your organization is not likely to have the same option. If you want to sell widgets in China or Mexico or Germany, you'll have to appeal to the Chinese, Mexican, or German market - or at least not make them hate buying your widgets until they can find a new source.

If you are going to launch in a global market - any global market - build your software from the perspective that you will be internationalizing it. Build a common interface, but build it in such a way that it reduces cognitive load and has the best experience for the greatest number of people. In the long run, you'll be glad you did...and when the next customer wants zero doors and handlebars in place of a steering wheel, you'll be able to say "here's your motorcylce" in weeks rather than months.

Happy coding.

Free Rum

People have asked me what has helped me debug nasty, not-so-easily-reproduced bugs and nearly always the answer is "exhaustive testing"...but what about when it isn't. Ok, most of the time it really is exhaustive testing. Of course, I make it a habit to write code to handle all different data types even when I'm 99.999% sure it will always only be one data type, because when you have 100M+ users using your code daily you'd be surprised how often 0.001% is.

So, how do we debug issues that seem to only occur 0.001% of the time? Obviously we're not talking about those use cases that fall into within the "normal" range, we're talking about outliers - or what I refer to as the oddball case. Here we're talking about those times when all our unit tests pass and still our code fails in the "real world" where there are a few cases - extreme corner cases, admittedly - when users are experiencing a "sub-par visit". These instances can easily bore a hole into our confidence (and ego...it's a little bit ego), leaving us scratching our head in confusion and frustration. What do we do then?

As with any good pirate, the answer is RUM, and lots of it!

Go ahead, start making your own pirate references and talk like a pirate...I'll wait...and I'm pretty sure I can hear you singing....

"Fifteen men on the dead man's chest —
...Yo-ho-ho, and a bottle of rum!
Drink and the devil had done for the rest —
...Yo-ho-ho, and a bottle of rum!"

Robert Louis Stevenson, Treasure Island

Of course in our case RUM does not refer to the pirate's beverage of choice but refers, instead, to Real-time (or sometimes Real) User Monitoring. The emphasis here, obviously, is on user monitoring, not testing or assuming what might be happening, and it's different than synthetic user monitoring, which is a simulation (and typically done as part of a comprehensive testing plan) because it uses real people in real-world situations.

This last point - that it uses real people in real-world situations - cannot be stressed enough. Why? Because, in the specific situation this post addresses, our software engineering has begun to move away from how people have evolved. People typically solve problems in a linear manner - it's how we've evolved. C came about because A and B happened. We make our selections in a store, go to the checkout, and pay. We don't suddenly jump out of line to go to the bank and apply for a credit card and expect to return to our place in the line with our basket full of our selections. For a very long time, our software matched this model exactly, or very nearly so. Oh, we might have a decision point where we would loop back into a process, but it was still a linear process. Most of the time, this works well, but as anyone who's stood in a queue hoping to order food only to find the customer ahead of them hasn't quite decided what they want knows, sometimes there are problems with synchronous, single-thread experiences.

We tried to resolve the synchronous, single-thread problem by adding threads. This often works alright, for the most part, but as we discovered, multiple blocked threads are not really any more productive. The answer, then, was an asynchronous (non-blocking) approach. Now, here we are, years later often suffering in callback hell and struggling to debug software because our linear, synchronous experience no longer applies to development of asynchronous systems...and as was mentioned earlier, that's why RUM enters our toolkit.

There are, of course, many ways to implement (or pour) RUM. If you're running any one of several traditional web servers, IIS or Apache, for example, Splunk is a good choice. I've written a proxy server for a Splunk app and reviewed a couple Splunk references - Splunk Developer's Guide and Learning Splunk Web Framework - so I'm not without respect; however, the downside of Splunk is that it can become expensive. For this reason alone, even if your operation uses Splunk in the production environment you may choose to forego it as a support for research and development.

As a no-cost alternative, however, you can pour your own RUM if you're using a node.js server. That's right, I said it - no-cost and RUM together - FREE RUM - (part of) every pirate's dream. Since it's relatively easy to do, and as long as you don't do something really boneheaded, you can build it securely, hoist the Jolly Roger, talk like a pirate, and follow the map below. Be warned, I'm not pointing out all the dangers (like how you can expose private or confidential information) - there are a few (fairly obvious) pitfalls, but this should be enough to get you in the general area of the treasure you seek (and I should add that you can find a more complete example/prototype in my rum github repo - https://github.com/hrobertking/rum).

First, you'll need to assemble a crew and put them on a ship. Do that by installing the socket.io module, adding it to your server module, and binding it to your httpServer instance.

Server-side JavaScript (Socket instantiation)

var server = http.createServer(handler),     io = require('socket.io')(server);

You should note that the server need not be the native node.js server, it can be an extension of the node.js httpServer object - e.g., an Express instance.

Now that you have a crew and ship, decide when you'll raise the Jolly Roger - do that by emitting an event through the socket and passing an object into the event, e.g., io.emit(event_type, event_object);.

The event type is a string literal so it can be named (nearly) anything and you can emit different events at different times, as in the example below.

Server-side JavaScript (Event Handlers)

var msg = {   id: unique_id(),   req: request,   res: response };   msg.req.on('end', function() {     msg.action = 'request';     io.emit('message', msg);   });   msg.res.on('finish', function() {     msg.action = 'response';     io.emit('message', msg);       if (msg.res.statusCode !== 200) {       io.emit('http error', msg);     }   });

Now put the spyglass to your eye and scan the horizon. Your spyglass is going to be a static page that uses the socket.io client script (https://cdn.socket.io/socket.io-1.3.5.js) and instantiates a socket. That socket will then be monitored and your event handlers will run when the event comes over the socket.

Your 'spyglass' document

<!DOCTYPE html> <html lang="en">   <head>     <meta http-equiv="Content-Type" content="text/html; charset=utf-8">     <style type="text/css">       .request { background-color:red; }       .response { background-color:yellow; }       .request.response { background-color:green; }     </style>   </head>   <body>     <script src="https://cdn.socket.io/socket.io-1.3.5.js"></script>     <script>       var socket = io();             socket.on('message', function(obj) {           var node = document.getElementById(obj.id), cls;           if (!node) {             node = document.createElement('div');             node.id = obj.id;             document.body.appendChild(node);           }           cls = node.className.split(' ');           cls.push(obj.action);           node.className = cls.join(' ');           node.innerHTML = '<p>' + obj.req.url + '</p>';         });             socket.on('http error', function(obj) {           var node = document.getElementById(obj.id);           node.className += 'http-error';           node.innerHTML += '<p>' + obj.res.statusCode + '</p>';         });     </script>   </body> </html>

Now, as you watch the 'spyglass' document in your browser, when the 'io' events are fired on the node.js httpServer they will be handled in the handlers specified in the spyglass.

Now you have insight into the important pieces of code, not as you test them (because you can get that elsewhere, like the Chrome dev tools), but as others use them, and it's debugging in those real-world situations that takes our development to the next level in our drive for results. Now, go get some pirate booty.

Happy coding.

The Danger of Going Above and Beyond

In a recent interview I was asked to describe the last time I went "above and beyond" for the customer. It seems a fair question, after all, we want people working with us who are willing to "go the extra mile" to serve the customer, right? I want to say both "yes" and "no".

I'll begin this journey by saying that for me, this question triggered a bit of a flashback to my first job not long after university. I was engaged to develop and implement the user training program for a new payment system. I spent hours testing the beta version of the software on all the platforms it was offered and wrote both the user and instructor documentation (including creating the graphics). As the second employee (the first was the person who wrote the software) I also did a few other things - all user-focused - like creating the inventory and supplies management system and some support software our customer service people used. One day as I was having lunch with the COO (Chief Operations Officer) he shared with me two management (related) ideals he had followed since his days with the US armed forces:

1. Even if someone gives 100 percent - and few people give 100 percent to their employer - the most you will get is 80 and the other 20 percent of the time they will not be productive
2. Don't keep an employee after you hear them say "it's not my job"

We all need down time. Sometimes the down time we need is a break during the day after we've spent hours working a particularly complex problem (been there, lots). Sometimes the down time is a couple of days off after working 80 hours in a week (been there too...more than once). As managers and team members we need to recognize that none of us give 100 percent. Are there some team members who are "more productive" than others? Yes, and no. It's much more common that we're all equally productive, just producing different things. We all need to recognize more than our own contributions (or our own kind of contributions - contributions of team members with roles like ours) and we all need to adjust our perception a little so that we're not only recognizing the 20 percent others are "not productive" but our own 20-percent time as well.

That second bit of advice still seems harsh to me, all these years later, but I can see how that construct shaped my perspective after having been introduced at such an early stage in my career. That does not mean that I mistakenly think that anyone can do anything and that all members of a team are interchangeable.

Team members each have their own strengths and they're not interchangeable. To put this in role-playing game terms, all good teams need a rogue (to help the team overcome obstacles and protect the team from danger), a fighter (to take care of the danger when it comes), and a healer (to restore the team after a dangerous encounter). This same general format works whether you're crawling a dungeon or launching a product (more about that another time).

If, in the midst of danger, the rogue said "hey, it's just my job to tell you about the dragon, not fight it" the rogue may find that he is crunchy and good with ketchup. If, in the midst of danger, a healer who says "my job is to heal the bite and claw marks after the battle" may find that she does not survive that long. In the same way, an engineer who tells the dev ops team that a failure was caused by "something in the way the build was deployed, and that's not my job" when there's an issue is not likely to survive long - and rightfully so.

As I said earlier, I stepped out of my role developing and implementing the user training program to work on other, related customer-facing issues at that first job. Was that going "above and beyond"? Some might say yes, but the correct answer is "no, not really". Was it going "above and beyond" when the COO instructed me to act as courier to redistribute the workload for another department in the same company? No. My job is to serve the customer, and in this case, as in many cases, "the customer" was my employer.

One of the things we have forgotten, misplaced, or perhaps discarded is not the mistaken ideal that "the customer is always right" but the very correct ideal that the customer deserves satisfaction. There was a time when companies employed slogans that were some version of "whatever it takes" but now it seems we have descended into a "give us a good review on Yelp/Google/Facebook/<insert your marketing here>" approach and we have lost a portion of what we were.

And here I am, back at my original premise - saying both "yes" and "no" to "going the extra mile" - and so I'll sum up as clearly as I am able. As soon as you define something as "above and beyond" you've categorized yourself and everyone else according to your perception of both productivity and job description. That, my padawan, is a dangerous road. A road visited by those who are eager to steal your reputation, your customers, and your livelihood. Don't travel that road.

Happy coding.

Death by a thousand cuts

The Legionnaire that scoffs at a lasgun has not charged across an open field against a hundred of them.

Maor the Scarred, Siege-Champion of the Scargivers, Warhammer 40,000: Black Crusade

A large part of building a website that gives the users the best experience possible is fending off a death by a thousand cuts - usually introduced through practices and design patterns that are sanctuaries for inefficient code and unnecessary complexity. Today I offer you a simple example that will show (on a small scale) how poorly things can be written using code generated by the Demandbase Ad Code Generator (I've changed the user ID to 'no-names' to protect the user - the number of bytes in "no-names" and the user ID hash are the same, though).

Here is their code:

<!-- Demandbase Advertising Code --><script>(function(d,b,a,s,e){ var t = b.createElement(a), fs = b.getElementsByTagName(a)[0]; t.async=1; t.id=e; t.src=('https:'==document.location.protocol ? 'https://' : 'http://') + s; fs.parentNode.insertBefore(t, fs); }) (window, document, 'script', 'scripts.demandbase.com/no-names.min.js', 'demandbase_js_lib');</script><!-- End of Demandbase Advertising Code -->

Wow, that looks complicated; it must do something pretty cool. Let's prettify it and add line numbers so we can figure out what it's actually doing and be amazed at its compact (403 bytes of) awesomeness. I'm going to be a little careful here, because there's a rather threatening message on the Demandbase Ad Code Generator page that tells us clearly not to "alter this code"...but I'm just explaining it for educational purposes, not running it in production (which I would never do, by the way), so I'm going to risk it under fair use.

1. <!-- Demandbase Advertising Code -->
2. <script>
3.   (function(d,b,a,s,e){
4.      var t = b.createElement(a),
5.        fs = b.getElementsByTagName(a)[0];
6.      t.async=1;
7.      t.id=e;
8.      t.src=('https:'==document.location.protocol ? 'https://' : 'http://') + s;
9.      fs.parentNode.insertBefore(t, fs);
10.   })(window, document, 'script', 'scripts.demandbase.com/no-names.min.js', 'demandbase_js_lib');
11. </script>
12. <!-- End of Demandbase Advertising Code -->

Ok, now that I've prettified it, let's dig into the code.

I'll ignore lines 1 and 12 because those are just comments to let us know that this is Demandbase code, and the only comment I have about line 2 is that this line is one of the good things about this snippet - it doesn't bother to use the language or type attributes. One of the other good things is that the author of this snippet wrapped it in parentheses, making it a self-executing function so at least you don't have to add more code to your page to get it to work.

Line 4 creates a script element and line 5 gets the first script  tag. lines 6 through 8 set various properties on the script element (created in line 4), and line 9 adds the script element to the DOM. Line 10 kicks off all this awesomeness by calling the self-executing function and passing in all the data used to set the properties of the script element in lines 6 through 8.

Just so you won't have to go through and figure out what properties are being set, here's the tag that's generated by this code (assuming it's run on a page using http as the protocol - if the page is using https, the src attribute of the script tag will match the protocol).

<script src="http://scripts.demandbase.com/Hwd8TqVa.min.js" async="1"></script>

That's not so bad, right...after all, async is specified, so the external script will be downloaded and executed asynchronously (except on versions of IE less than 10 and versions of Firefox less than 3.6).

Hey, wait a minute...the tag is only 79 bytes (80 if the protocol is https)...and that's way less than 403. Oops. Of course we can make it even shorter in a way that is protocol agnostic, and we can reduce the page weight even more if we ignore XML rules and use HTML validation rules instead.

<script src="//scripts.demandbase.com/Hwd8TqVa.min.js" async></script>

Now it's only 70 bytes and works for both http and https without a problem. Further, using the tag directly not only means you've reduced your page weight by 333 bytes but also means your page is not blocked while the little Demandbase function - which is supposed to be placed at the top of the page - is trying to run, and it's immensely more friendly to attempts to debug problems with your page.

At this point you may be asking yourself "what is the advantage of using the code generated by the generator" - and that's a good question. There are no advantages to using their code, but there are potential disadvantages, and even worse, the generator could generate better code - code that doesn't have some of these disadvantages - but it doesn't.

Code like the code in today's example is poor writing - and I'll be honest, there is a lot of code out there of the same low quality - and poor writing will, in the end, lead to a slow and torturous execution - a death by a thousand cuts. We only use it because we either don't know better or we don't care (which would say something about our GSD attitude). Raise your game - avoid the unnecessary complexity - keep your writing clear and concise - do this, and you will be much happier in the end.

Happy coding.

The Right Answer

Several years ago – in the late 1980s – I was in a graduate MIS class and the professor posed a question to which he made clear there were multiple answers. After a class discussion the professor brought up the answer he considered to be the right answer. (If you haven't read the problem and answer in the sidebar, do so now.) As I've recalled this event, I've been reminded of a few things, and I've tried to determine what the greater lesson has been over the years.

Problem: You have hired a worker who is to be paid 1 inch of gold each day. You have a single bar of gold that is 6 inches and can only cut it twice. Where do you make the cuts to pay the worker exactly the amount they have earned each day - what size are the three resulting bars?

Answer: Cut the bar at 1 inch from one end and 3 inches from the other end, resulting in 3 bars that are 1 inch, 2 inches, and 3 inches. Pay is as follows: Day 1 - the 1" bar; Day 2 - the 2" bar and receive the 1" bar back; Day 3 - the 3" bar and receive the 2" bar back; Day 4 - the 1" bar; Day 5 - the 2" bar and receive the 1" bar back; Day 6 - the 1" bar.

One of the most important lessons I learned in that experience is that there are typically several answers to a problem and they have varying degrees of difficulty, precision, and accuracy. Sometimes choosing the answer is easy – other times it would seem it is not. For example, if we're choosing which compression algorithm to use, we can relatively easily make the determination based upon whether we want to prioritize speed or the compression ratio; however, if we're trying to balance time-to-market, performance, quality, and user experience the answers are not so easily reached. In real-world scenarios, we may suggest to our hypothetical employer that they fold the gold bar 5 times and make the two cuts at the ends – that answer maintains time-to-market and improves the user experience (the worker has immediate full use of everything they receive), but is that the right answer?

Any time we find ourselves asking if an answer is the right answer, we must look beyond the surface of any potential decisions and ask what we would actually be doing – in language terms, what is the connotation of this conversation rather than just what is the denotation. Phrased another way – what are the patterns we are introducing intentionally and what patterns are we introducing unintentionally? Are we just doing what is good for our organization or are we promoting a greater good or are our customers (or users) seeing the benefit?

In the world of software development, managers and business leaders have been toying with the development triangle for years, trying to squeeze out the best solutions, and just because we're now dealing with web-based applications instead of desktop-based applications makes little difference. Granted, trying to keep the three legs of the development triangle balanced is difficult – and it's harder for some teams than others (but that discussion will have to wait for another post). It's made still more difficult by the fact that none of us like to have the scope of our project constrained by the balance of these three legs. Unfortunately, what many organizations do is prioritize their wants and needs over the needs of their customer. There certainly are times when this is appropriate – the customer simply cannot always be right – but this happens far too often to be valid, especially among Internet companies where development speed – enhanced by bloated code like bootstrap and jQuery or (even worse) user-agent dependent languages like Angularjs – is prioritized over a user experience that people argue simply must have all the bells and whistles. Leadership in these cases is somewhat like the obstetricians who argue that they must have the machine that goes ping, except in the real-world these cases are not intentionally comedic. As a result, many leaders who have prioritized development over user experience or have tried in vain to balance all the legs of the development triangle and not constrict scope, believe the use of various frameworks are a solution they can leverage, and many have attempted just that – going down the Angular.js rabbit hole, for example – but is that the right answer?

No, it's not the right answer. Is there a better answer? Yes. In fact, I would posit that there is a right answer for web-based development – one that prioritizes users over development – but it's one that few developers like or want to admit, because it's not, as we say, cool. The right answer is Progressive Enhancement. Simply put, Progressive Enhancement starts with HTML, and it should be semantic HTML which must include using the appropriate tags as well as ARIA attributes that make it as readable as possible by both humans and machines (I'm looking you, developers who use <i> instead of <em>, <b> instead of <strong>, and never use ARIA states and properties). After the document is created, it is styled using CSS – again paying attention to accessibility (e.g., using clip instead of display or height to hide content) – and finally, layering unobtrusive JavaScript on top of all the rest in a way that pays attention to performance.

Developing in this manner ensures that users will always be able to access your content – be it informational or commercial – regardless of whether or not someone's user-agent supports CSS3 or whether or not the user-agent supports JavaScript or whether or not anyone is doing something like pushing code live without having tested it and introducing an error as a result or blocking JavaScript (either intentionally or unintentionally – and yes, it has happened when a "parental filter" operated by an ISP in the UK blocked access to jQuery via CDN) or whether or not you're running third-party code that is not up to par.

Of course, one of the arguments that I've heard repeated is that there aren't that many users with JavaScript disabled and Progressive Enhancement takes too long, both to write and when rendering (because JavaScript-based rendering is much faster). So, let me just address those arguments – and let me say that this reasoning is not based on some ephemeral justice-based ideology but on solid experience building web pages. (If you're really curious about my work experience, read "A machine-readable resume", where there is both an image of a portion of my CV and a link to the full version.)

While the actual percentage of users browsing the Internet with JavaScript disabled is low (the number is arguably around 1%), that number does not count all the users affected by stuff that is broken by error-infested, untested code, nor does it count users affected by not having access to specific libraries because their ISP decided those libraries were potentially dangerous.

Neither of those groups take into account people who are using accessibility software for whom an otherwise acceptable web page offers a broken user experience. Granted, most accessibility software works acceptably with JavaScript, but without paying special attention to making your JavaScript-enhanced page accessible, e.g., by using aria-live or shifting focus to updated content, users with accessibility issues are still left with a broken experience. Because of the way in which accessibility software works with user-agents, the user-agents are not technically, potentially JavaScript-disabled in general, but the user experience is.

Yes, the number of users with user-agents that recognize a SCRIPT tag is around 1% – and 1% of nearly 3 billion is still a lot – but there is a significant difference between something potentially running JavaScript and something actually running JavaScript – a difference any philosopher should immediately recognize.

As for pages built using Progressive Enhancement not rendering as fast as those built using client-side rendering, I'm going to just come out and say that cannot possibly be true in anything other than a test that is non-analogous to real-life. I suppose client-side rendering might be faster if you were delivering a page with a lot of duplicated content, but in practice delivering a template, a client-side rendering library, and JSON data is not a much smaller payload because most content isn't duplicated. What we've really done with this approach is break up the content – the most important piece – into separate documents. To add insult to injury, there are issues of network latency for each of those three requests (for content), and potentially disastrous re-flow issues, not associated with CSS, as containers are loaded with HTML.

If you coerce a multiple-page application into a single page by delivering partial in-document updates – the way it was done in PayPal checkout products for example – you may see some actual performance improvements and users may perceive some performance improvements – though neither is guaranteed. However, in the case of in-context updates you have to take special care to make sure your page/app is still playing well with accessibility software or you're not only shutting out users you may be violating the law.

As an aside, adding all the accessibility hooks and building out any inadequacies in the various libraries and then testing all the extra code certainly has the potential to destroy any improvement you may gain by avoiding Progressive Enhancement.

Yes, saying you're improving the development experience (or your engineers' lives) is seen as sexy and cool by the engineers building your products and prioritizing time to market is sexy to those who have invested in your business, but as anyone who has pursued event the earliest steps in an Economics program knows, those words don't necessarily what you think they do – it's what your users think, and on the web it's a mistake to not prioritize the user experience over other things.

Our users may not always be right, but when we're trying to solve their problem, more often than not they have the right answer.

Note: if you'd like to read another's thoughts on a similar topic, Nicolas Bevacqua has a few thoughts, and has prompted a few comments as well.