The twist on the experiment

Preface

Having resurrected this blog for the purpose of helping find myself a job (again), some things are going to be different.

The principle thing is a switch in focus, less on the technical and more on the people side, corresponding with how my career has evolved over the past decade. I probably will still dig into some technical topics, both out of interest and because it's easier to find opportunities to solve technical problems in the wilds of the internet, but it's important I also demonstrate, as best as I am able, that I have some understanding of organizational dynamics and human issues.

The secondary thing that's different is nearly a consequence of the first thing. In short, I will try to use this space to promote the efforts of those less well-represented in tech than myself. I will still surely include some of my own opinions, but my experience is that my opinion is much less likely to provide unique value than that of, say, a black woman.

Compared to the last 5-7 years, where in general I've sought to avoid injecting myself into discussions, I am going to use this space to highlight and discuss relevant events. Again, where I can I will populate those discussions by including quotes from and links to others thoughts on the subjects, but when I believe I can provide value by collating and commenting on these subjects, I will.

I will be writing about:

People - Process - Product

When I was working on my resume recently, I shrunk it down about 75%, and I wanted a pithy way to convey where my priorities were at this point. Both to put it out into the world, as an expression of what I was looking for, but also because it represented a clear shift in how I approached "work" over the past decade. While process has always been a significant part of the work I do in the tech world, it's generally been in service of the delivery of a particular product. Yes, I spent plenty of time optimizing processes for people, but that wasn't where the focus was. So a big part of resurrecting this blog is also about representing that shift to a focus on people.

People

Writing about people here is likely to be the most difficult, in some ways. I don't think there's a bug tracker for people problems, where I could just volunteer to try to help them, then write about it. But one of the ways that I think I can incorporate writing about people into my efforts here is through the relatively introspective manner that I tend to approach problems. That is, through documented analysis of my own process and behaviour in approaching problems, I can demonstrate at least some personal understanding of the dynamics involved.

Besides that, I will write about patterns and individual instances of workplace people problems that I encounter, primarily on Twitter. I will do my best to gather evidence demonstrating the harms that are occurring, and highlighting opportunities to mitigate those harms. I expect to lean heavily on the work of others here, and if I allow the focus of this type of writing to be on me, please call me out on it. That said, I will incorporate examples and stories from my own experience, as appropriate.

Process

Writing about process is something I generally enjoy, and while I'm used to doing that in a professional setting where I have long-term plans for the processes in question, I don't think I'll have a problem finding many, many relevant opportunities when writing about people or products.

Product

While definitely the lesser focus overall in how I'm approaching work, I will demonstrate delivery of high quality products throughout here. Both in the written work, and any accompanying projects, completeness and correctness will be held to a high standard. 

Enhancing Google Keep

To that end, my first project here is going to be building an opinionated task management tool on top of Google Keep. It will allow me to write about many of the topics relevant to the kind of work I'm seeking, while also scratching a very old itch for me. :)

The Automator's Manifesto, Part Two

This was written quite a while ago, but I never got around to publishing it. So here you go!

Proactive Transparency

One of the most important things any automator can do is provide confidence to stakeholders in the project. Confidence can only develop if stakeholders understand the project, and see meaningful improvement. As such, it behooves the automator to make it easy to understand the state of the project. This is particularly relevant when it comes to actual test results, as even successful results that have no audience are essentially useless.

The term I came up with for this aspect of an automators role is "proactive transparency", based on the idea that it's not enough to simply run tests and call it a day. It's a good start, but publishing the results of those tests in a way so as to maximize their audience is where real value starts to develop.

Entropy Factors

The most challenging automation task of my career came at Research in Motion, where I put together a device lab to test over the air OS software updates (RIM was actually the first to have this available, as I understand). This was a hugely complex endeavour, covering a range of devices with different capabilities, moving between OS versions that could vary greatly. But even more significant to our testing were the externalities we were dependent on.

For instance, knowing what builds were available for updating required manual scraping of a development version of an application maintained by a different team, with no formal connection. This application, in turn, was loosely connected to the servers which devices would contact to identify what updates might apply to them. We frequently encountered issues where a test was queued, but when the device navigated to the update screen, the desired build wasn't found.

These, and many others, came to be known as our "entropy factors". I'm probably misusing entropy, but here the phrase means things outside of your control that could impact the results of your tests. Understanding what these are for a given automation project is one of the most important planning exercises you can do.


Measuring Everything

At my first GTAC, Patrick Copeland said "measure everything" (he probably said "at Google we measure everything", but that wasn't what I wrote down.) This was an inspiring credo for a then new-to-the-field automator, and I spent many years trying to live up to it. But it's wrong, or at least deceptive.

Assuming you have the resources to make it possible, measuring everything is still going to require implementation time. But what is much more important is that it is going to increase the scale of measuring output, while doing nothing to identify which metrics mean success. It might feel great to have every system metric, API call, user session and what-have-you carefully collected, but it does nothing on it's own to make your project better. That's an oversimplification, as having this all can increase stakeholder confidence initially, but after a couple of scares, that will fade.

You need to commit to measurements that correspond to your desired outcomes for the automation project. That doesn't mean you shouldn't collect a ton of other metrics if your tooling can do it for free, but you shouldn't surface them by default. Keep clear what you are measuring and why, and add cases to that as you grow.

Renewing the Experiment

 I started this blog about a decade ago to help me find work.

Once I found work, it began to atrophy rather quickly.

Now, I need to find work again. :)

So here's a placeholder, saying that this blog is coming back to life. I've got a handful of unpublished drafts from the past several years that I may polish and publish, and I'll write about my job search otherwise.

My fling with iOS, or "I like my gardens unwalled"

Somewhat of a digression from the usual content of this blog (either programming or nothing), and ground that has been well-trod in many other places. But here I am, writing it anyway.

TL;DR

I switched to an iPhone 7 2 months ago. It was really cool, but I don't like it and am switching back.

Origin

I've never owned an iPhone. My wife has had a 4, 5, 6, and 7, and other family members have had many others.

I worked at BlackBerry back when it was called Research in Motion, starting in 2008, around the launch of the Bold 9000. Before then, I didn't even own a cellphone (I had an awesome Sanyo in 2001, but it broke and I never replaced it).

While at RIM I changed phones every 6 months or so. Luckily, I got to more or less avoid the Storm and follow-ups (though I was responsible for implementing touch support in our test automation framework), so I got very used to the BlackBerry keyboards. I still think the keyboard in the original Bold is a work of art.

From when RIM and I parted ways in 2011 until the end of last year, I used almost entirely Androids, except for a while where I enjoyed the LG Quantum on Windows. I actually really liked that phone, and Windows there worked really well. Like the commercials emphasized, you could very quickly get things done and put your phone back down. Part of this was because there was no reason to keep looking at your phone, because it couldn't do much more than essential communication functions, but at the time I found that simplicity quite charming.

But Android...

Well, my first experience with Android was a week with the Galaxy Nexus. It was pretty cool at the time, but I returned it due to the expense.

But when the Nexus 4 came-a-callin, I was smitten. It was a beautiful piece of hardware that ran all kinds of cool things, did very little to get in my way, and cost half of what my wife's phone did. Sure, the camera was mostly crap and the battery wasn't great, but I loved that thing.

The Nexus 4 was followed by the Moto X. I picked mine up on Kijiji, and loved the active display and passable camera.

I followed the Moto X with the Droid Turbo, which was nearly a perfect phone. Swiping my hand over to check notifications is still one of my favourite phone interactions, and the battery would go two days before needing a charge (which only took 90 minutes anyway). The camera, despite the resolution, was bad most of the time, and as a Verizon exclusive that I exported to Canada, I had to put up with useless apps and slooooow updates. Still, it lasted me nearly two years before the screen got damaged (which was supposed to be under warranty, but Verizon and Moto figured they could get away with screwing a Canadian).


Which brings me to my last (and near future) Android, the LG G4. Battery life was a big let down from the Droid, but the camera was very good, and it had some other neat features. I cracked the screen on it in 3 months. I ordered a replacement screen, but while I was waiting for it to arrive, an opportunity to go iPhone arose.

The iPhone

For all that my phones have rarely been top-of-the-line, I read multiple reviews of every flagship, and the pace at which Apple has pushed forward their CPU was amazing. With the 6S I was tempted to jump ship just for that reason, and seeing how far they pushed it further in the 7 made it impossible to resist.

The iPhone 7 is the most attractive piece of hardware to touch, ever. The weight is perfect, it's sleek, and it responds to you in a very physical way.

It is blazing fast at practically everything, and the screen is second-to-none.

The camera is super-fast, and generally very good, though I would argue it trades blows with the G4 in some areas.

Force touch is cool, and makes some things nicer to do.

I like being able to ask Siri to time things for me. Very cool.

I get through a day with 40% left on the battery, easily. But that's not the whole story...

My wife can't get through the day without having to recharge, with the same phone. Because she uses it for Facebook. I generally don't use mine for anything like that. In fact, I use my iPhone the way I used my Windows Phone. When I need it, it's there, and then I put it away.

But it's the only piece of Apple in my ecosystem, and that means a lot of the things I want to do are just a little bit inconvenient, so I don't do them. My life is on Google's cloud, and my work is on Microsoft's. These things are available in a pinch from my iPhone, but my Surface is substantially better for either of them, and not that much harder to tote around.

Specific examples of annoying things:
 - I must use a case, because it's $$$, which detracts substantially from the aesthetic appeal. I've never used a case before
 - Google maps navigation cuts out over bluetooth
 - Notifications are among the worst on any phone I've ever had. Better since I disabled the volume switches affecting them, but some apps just fail to notify me consistently (might be on the apps, but they worked fine on Android)
 - I've got relatively large hands, and I cannot seem to get comfortable with the keyboard. Some of this might be the case, some might be that I've developed a preference for Swype, who knows.
 - Siri is really dumb. I don't know if it's somehow related to my lack of Apple ecosystem, but other than timing things or prank questions, I've given up talking to her
 - No headphone jack is just stupid. Yes I have two dongles in the house, but mostly I just carry two sets of headphones now. Especially stupid if they're going to move to USB-C.
 - Chromecast (the only input to my TVs) control is inconsistent. Some apps I can change volume with buttons, others within the app, others I need to open Google Home to control volume. Probably an app issue, but not a problem I had before.
 - My fitbit stopped pairing with it, and one of my vehicles will only pair for a day or so before needing to removed, bluetooth toggled off and on, and re-added.
 - The UI on some of Apples apps is surprisingly bad. Sometimes when I change settings, I click "back" in the top left, other times I hit "save" in the top right. Some apps require you to do both these things to effect a change.

Summary

Subjectively, the feeling is that I need to do things Apple's way, and I'm not interested in that approach. So all that lovely power never gets called upon. A lot of my issues are little things, many of which there may be fixes for, but my experience is that unlike Android, there's not a lot you can do to fix these little things.

So it's back to the G4 for me, with its fresh screen. I won't use a case, and I'll probably break it again. But it's not getting Nougat anyway, so maybe it will be time to trade up again by then. :)

Software Gardening in practice

Software gardening, as expressed by Hunt/Thomas, Atwood, etc uses the metaphor to deal with how a software project is alive, and is constantly being changed by factors outside of the control of the developers. You can plant a seed, and water it, but you can’t control the sunlight, the bugs, bees, etc. As it grows, you can weed the garden, removing features that are not used but taking up resources, add fertilizer, change the plants to better reflect your needs (why didn’t someone tell me Joe was allergic to cucumbers!?).

I want to expand on this idea, as I believe there are several valuable insights that can be derived from adopting this view. In this post I will explore this concept from three different perspectives: the project itself, a software developer, and a client that wants to get some software developed.

The Unseen Environment

As a particular software environment is like a garden, there are many factors that the garden grows out of. There are obvious factors, like the composition of the soil, and the likely availability of water, and sunlight. But there are also more subtle, “living” factors, that are going to be part of the environment of your garden, yet generally operate independently.

One of the most fascinating things I’ve learned recently is the concept of mycorrhizal networks. These are fungal layers that exist within the soil that serve to break down waste matter and distribute molecules that plants depend on for nutrition. It is generally unseen, but is a huge factor in the ability of plant ecosystems to thrive. Generally much better known, but still somewhat unseen, are the pollinators. Birds, bees, and butterflies allow plants to reproduce, and thus stimulate genetic diversity.

So, above and below, there is an environment that allows plant life to thrive. A similar role is played in software development by the ecosystem that exists around both the software, and the developers.
All developers and software libraries come to a project with a history that is shaped through the absorption of “byproducts” from previous projects. These can take the form of learned code snippets, software structure conventions, and many other things. I would liken this to the way the mycorrhizal network digests the output from various past biological processes and produces raw nutrients to begin new ones.

In a similar way, open/shared source cultures could be compared to pollinators. Moving from project to project with very few barriers, a developer can distribute efficient techniques rapidly, effecting the projects and other developers. This has resulted in the establishment of numerous best practices, to the point where high-profile open source projects are the gold standards for software quality. Another interesting way in which open source diversity is like genetic diversity is in the trend of multiple competing software projects springing up with different attributes tailored to their environments. These can continue for very long periods, sharing “genes” with each other but still continuing as distinct “species”.

In some large organizations, the software ecosystem is distinct enough to effect the software that is produced there, but at this point, almost everyone is dependent on Open Source. It produces specific codebases that underly most every modern project, such as Linux, Apache, OpenSSL, GCC, Boost, Python, etc. But it also establishes the best practices and standards for software development that are used by individual developers, whatever they are working on. Conversely, bug reports come from commercial projects that make Open Source stronger, and developer experiences there continue to inform and refine the state of the art. There are also cases where it is beneficial for commercial development to open source certain pieces of their software, to expose them to that wider diversity, and encourage contributions that will help meet business goals (though like natural evolution, predicting the future of an open source project is far from a science).

Tying that back together, that means that for a developer, every project you work on contributes to this larger community, even when the code itself is closed. Because it changes you, and it changes the conversations you have with others, and how you work to solve problems in the future. As a former user of the Zope Web Application Server, I have watched other users and contributors define the cutting edge of web software (Tarek Ziade, Chris McDonough, Alex Gaynor, and Martijn Faasen, among others).

Garden Development as a Service

As someone who works on software, your value is not the creation and delivery of a product, but rather the cultivation of an environment. Even when you are employed delivering products, there’s no such thing as a final release, just something that is able to meet the needs of the moment. Needs will change, the ecosystem will change, you will change, and new opportunities will arise. How might this change how you approach a single project, and your career?

In terms of approaching a particular project, it means you need to understand the landscape in order to effectively execute on the vision. You need to know what kind of software is already there, and what the state of third party libraries are in that particular domain. You need to plan for the inevitable bugs and maintenance. The actual implementation flows out of these concerns, not the other way around.

From a career standpoint, I should first confess that, well, I’m not a gardener. However the image I have of them is that they don’t get annoyed that all the cool gardens are already cultivated. As long as you have a willingness to work, gardens are going to need cultivation. Likewise, as long as you have a willingness to develop it, there will always be available work developing software. Under this philosophy, nothing ever reaches the point that it cannot be improved or does not require maintenance.

Gardening Agency

When we have a need for some software, and we attempt to consider the effort that will be required, we usually think about the features that we want the software to have. The gardening metaphor suggests this is a very limited way to view this. If we think of the features of a particular garden as “has Roses and Violets, and produces Tomatoes”, we can quickly understand that a good deal more information is required to get a reasonable estimate of the effort that will be necessary.

Is the garden in Hawaii, Alaska, or the Central Park Zoo? What if there’s already a garden there? What's the condition of the soil, or local animal life?

Translating these questions over to software, we can see that the environment a piece of software needs to "live" in can impact the effort required to develop it as much or more than the features it needs to have. For instance, if you are looking to add accounting features for exporting financial data to your partners, the effort could vary tremendously depending on the existence and maturity of the open source software that exists to work with data in the formats they use.

Just as working with a professional landscaper will produce dramatically different results than simply purchasing plants, establishing a relationship with someone who can look at your situation holistically is key to a successful software project. They know the questions that need to be asked, the process for finding the answers, and they know how to build out a development team that will take that reality into account.

It's yer data! - how Google secured its future, and everyone else's

Dear Google,

This is a love letter and a call to action.

I believe we stand at a place where there is a unique opportunity in managing personal data.

There is a limited range of data types in the universe, and practically speaking, the vast majority of software works with a particularly tiny fraction of them.

People, for example. We know things about them.

Names, pictures of, people known, statements made, etc.

Tons of web applications conceive of these objects. Maybe not all, but probably most have some crossover. For many of the most trafficked apps, this personal data represents a very central currency. But unfortunately, up until now we've more or less been content with each app having it's own currency, that is not recognized elsewhere.

You can change that. You can establish a central, independent bank of data, owned by users and lent to applications in exchange for functionality. The format of the data itself will be defined and evolved by an independent agency of some sort.

There are two core things this will accomplish.

1) It will open up a whole new world of application development free from ties to you, Facebook, Twitter, etc.

2) It will give people back ownership of their data. They will be able to establish and evolve an online identity that carries forward as they change what applications they use.

Both of these have a dramatic impact on Google, as they allow you to do what you do best, building applications that work with large datasets, while at the same time freeing from you concerns that you are monopolizing people's data.

A new application world

When developing a new application, you start with an idea, and then you spend a lot of time defining a data model and the logic required to implement that idea on that data model. If you have any success with your application, you will need to invest further in your data model, fleshing it out, and implementing search, caching, and other optimizations.

In this new world, all you would do is include a library and point it at an existing data model. For the small fraction of data that was unique to your application, you could extend the existing model. For example:

from new_world import Model, Field

BaseUser = Model("https://new_world.org/users/1.0")

class OurUser(BaseUser):
    our_field = Field("our_field", type=String)

That's it. No persistence (though you could set args somewhere to define how to synchronize), no search, no caching. Now you can get to actually building what makes your application great.

Conceivably, you can do it all in Javascript, other than identifying the application uniquely to the data store.

And you can be guaranteed data interoperability with Facebook, Google, etc. So if you make a photo editing app, you can edit photos uploaded with any of those, and they can display the photos that are edited.

Securing our future

People have good reason to be suspicious of Google, Facebook, or any other organization that is able to derive value through the "ownership" of their data. Regardless of the intent of the organization today, history has shown that profit is a very powerful motivator for bad behaviour, and these caches of personal data represent a store of potential profit that we all expect will at some point prove too tempting to avoid abusing.

Providing explicit ownership and license of said data via a third-party won't take away the temptation to abuse the data, but will make it more difficult in a number of ways:

• Clear ownership will make unfair use claims much more cut-and-dried
• A common data format will make it much easier to abandon rogue applications
• Reduced application development overhead will increase the competitive pressure, lowering the chance of a single application monopolizing a market and needing to grow through exploitation of its users data

A gooder, more-productive, Google

By putting people's data back in their hands, and merely borrowing it from them for specific applications, the opportunities for evil are dramatically reduced.

But what I think is even more compelling for Google here is that it will make you more productive. Internally, I believe you already operate similar to how I've described here, but you constantly bump up against limitations imposed by trying not to be evil. Without having to worry about the perceptions of how you are using people's data, what could you accomplish?

Conclusion

Google wants to do no evil. Facebook is perhaps less explicit, but from what I know of its culture, I believe it aspires to be competent enough that there's no need to exploit users data. The future will bring new leadership and changes in culture to both companies, but if they act soon, they can secure their moral aspirations and provide a great gift to the world.

(Interesting aside, Amazon's recently announced Cognito appears to be in some ways a relative of this idea, at least as a developer looking to build things. Check it out.)

PyCon 2014

I've now been back from PyCon for a week, and I've got some thoughts to share.

Scope

It was huge.

I usually try to memorize everyone's names, and I have some habits that help me with that. But there were so many people, I think that may have fallen apart. :)

A lot of hero worship, as I met, or at least observed from a distance, many people who helped shape my views on software (+Tres Seaver in particular).

Conversely, I managed to avoid running into those attending from my employers (I'm looking at you, +Kenneth Reitz, Sandy Walsh, and probably someone from RIM/BlackBerry).

Diversity

All the promotion of the diversity was terrific. At the same time that it's great to be part of a movement that is markedly more female-friendly then the tech community at large, Jessica McKellar made it clear that we have so much farther to go. As the father of two girls, it's very important to me that we change the culture around technology to emphasize that there's no particular skillset or aptitude that's required for entry.

Software is our world, and we can empower EVERYONE to play a part in shaping it.

Content Overview

I enjoyed the talks that I went to, but I did skip more than I was intending to. I had trouble letting go of work, and there was a lot of content that was relatively beginner focused, or represented tutorials that I knew had high-quality online counterparts, should I need them. I feel like this was a deficiency of my own, and one I hope I handle better if I come back next year.

Meta programming

I've been flirting with creating my own language for a while now, and if I were to do so, it would probably be on top of Python. Thanks to talks by +Allison Kaptur and +Paul Tagliamonte, I feel much more prepared to do so.

Allison provided a brilliant guide to implementing the import mechanism from scratch. Having read +Brett Cannon's blog when he created importlib, I knew there was a huge amount of work that went into getting it right, so it was an intimidating area. Yet in 20 minutes Allison walked us through getting something functional.

Paul's talk on Hy was not quite so accessible, but perhaps even more inspiring. The relative ease with which Hy and Python can co-exist within the same project is just awesome, though mucking around with ASTs remains a bit of a scary idea.

Sprints

While I was skipping talks, I consoled myself in the thought that I would really engage during the Sprints (I had a day and half scheduled for these). But I didn't, and while I think that had more to do with me (once again, I worked), I'll share what I think could have been done better, in case anyone else felt the same way.

Technically Sprints started Sunday evening, but I get the feeling that no one was actually interested in running them Sunday evening (or maybe my timing was off). There were a handful of people there, but no clear organization or plan about what was to be worked on.

Monday morning, it was certainly better attended, but it still wasn't inviting. There was a central chart of what rooms contained what projects, but within the rooms there was no indication of who was working on what. From my limited time being involved in or running other short coding sessions, I was also surprised not to see much use of flipcharts or whiteboards.

I guess how I would do it, if I ran it next year (I'm not volunteering, yet), is provide signs for each project to put at their table, and encourage each of them to write out a list of goals for the sprint in a place where it can be publicly examined and crossed off. Perhaps also provide special shirts/hats/badges to the "leader" of each sprint. The experience I would like is for someone to be able to wander in, examine what each project is doing without actually bothering anybody, and then if they find something they think could fit them, to know who to ask.

Misc.

• Ansible is what we're using at SFX, and while I've had some experience with it, I have a much more robust appreciation for it, thanks to +Michael DeHaan
• Peep should be included in the standard library. Seriously.
• Asyncio makes things most people do easier. Bravo!
• iPython Notebook is cool, and +Catherine Devlins talk about executable documentation has me itching to try it out.

Conclusion

As someone who has been around the block but doesn't find much time to actually code anymore, I may not be the code audience for PyCon. But I'm still delighted to have finally made it to one, and I'm really tempted to make it a family trip next year.