Software Gardening in practice

Software gardening, as expressed by Hunt/Thomas, Atwood, etc uses the metaphor to deal with how a software project is alive, and is constantly being changed by factors outside of the control of the developers. You can plant a seed, and water it, but you can’t control the sunlight, the bugs, bees, etc. As it grows, you can weed the garden, removing features that are not used but taking up resources, add fertilizer, change the plants to better reflect your needs (why didn’t someone tell me Joe was allergic to cucumbers!?).

I want to expand on this idea, as I believe there are several valuable insights that can be derived from adopting this view. In this post I will explore this concept from three different perspectives: the project itself, a software developer, and a client that wants to get some software developed.

The Unseen Environment

As a particular software environment is like a garden, there are many factors that the garden grows out of. There are obvious factors, like the composition of the soil, and the likely availability of water, and sunlight. But there are also more subtle, “living” factors, that are going to be part of the environment of your garden, yet generally operate independently.

One of the most fascinating things I’ve learned recently is the concept of mycorrhizal networks. These are fungal layers that exist within the soil that serve to break down waste matter and distribute molecules that plants depend on for nutrition. It is generally unseen, but is a huge factor in the ability of plant ecosystems to thrive. Generally much better known, but still somewhat unseen, are the pollinators. Birds, bees, and butterflies allow plants to reproduce, and thus stimulate genetic diversity.

So, above and below, there is an environment that allows plant life to thrive. A similar role is played in software development by the ecosystem that exists around both the software, and the developers.
All developers and software libraries come to a project with a history that is shaped through the absorption of “byproducts” from previous projects. These can take the form of learned code snippets, software structure conventions, and many other things. I would liken this to the way the mycorrhizal network digests the output from various past biological processes and produces raw nutrients to begin new ones.

In a similar way, open/shared source cultures could be compared to pollinators. Moving from project to project with very few barriers, a developer can distribute efficient techniques rapidly, effecting the projects and other developers. This has resulted in the establishment of numerous best practices, to the point where high-profile open source projects are the gold standards for software quality. Another interesting way in which open source diversity is like genetic diversity is in the trend of multiple competing software projects springing up with different attributes tailored to their environments. These can continue for very long periods, sharing “genes” with each other but still continuing as distinct “species”.

In some large organizations, the software ecosystem is distinct enough to effect the software that is produced there, but at this point, almost everyone is dependent on Open Source. It produces specific codebases that underly most every modern project, such as Linux, Apache, OpenSSL, GCC, Boost, Python, etc. But it also establishes the best practices and standards for software development that are used by individual developers, whatever they are working on. Conversely, bug reports come from commercial projects that make Open Source stronger, and developer experiences there continue to inform and refine the state of the art. There are also cases where it is beneficial for commercial development to open source certain pieces of their software, to expose them to that wider diversity, and encourage contributions that will help meet business goals (though like natural evolution, predicting the future of an open source project is far from a science).

Tying that back together, that means that for a developer, every project you work on contributes to this larger community, even when the code itself is closed. Because it changes you, and it changes the conversations you have with others, and how you work to solve problems in the future. As a former user of the Zope Web Application Server, I have watched other users and contributors define the cutting edge of web software (Tarek Ziade, Chris McDonough, Alex Gaynor, and Martijn Faasen, among others).

Garden Development as a Service

As someone who works on software, your value is not the creation and delivery of a product, but rather the cultivation of an environment. Even when you are employed delivering products, there’s no such thing as a final release, just something that is able to meet the needs of the moment. Needs will change, the ecosystem will change, you will change, and new opportunities will arise. How might this change how you approach a single project, and your career?

In terms of approaching a particular project, it means you need to understand the landscape in order to effectively execute on the vision. You need to know what kind of software is already there, and what the state of third party libraries are in that particular domain. You need to plan for the inevitable bugs and maintenance. The actual implementation flows out of these concerns, not the other way around.

From a career standpoint, I should first confess that, well, I’m not a gardener. However the image I have of them is that they don’t get annoyed that all the cool gardens are already cultivated. As long as you have a willingness to work, gardens are going to need cultivation. Likewise, as long as you have a willingness to develop it, there will always be available work developing software. Under this philosophy, nothing ever reaches the point that it cannot be improved or does not require maintenance.

Gardening Agency

When we have a need for some software, and we attempt to consider the effort that will be required, we usually think about the features that we want the software to have. The gardening metaphor suggests this is a very limited way to view this. If we think of the features of a particular garden as “has Roses and Violets, and produces Tomatoes”, we can quickly understand that a good deal more information is required to get a reasonable estimate of the effort that will be necessary.

Is the garden in Hawaii, Alaska, or the Central Park Zoo? What if there’s already a garden there? What's the condition of the soil, or local animal life?

Translating these questions over to software, we can see that the environment a piece of software needs to "live" in can impact the effort required to develop it as much or more than the features it needs to have. For instance, if you are looking to add accounting features for exporting financial data to your partners, the effort could vary tremendously depending on the existence and maturity of the open source software that exists to work with data in the formats they use.

Just as working with a professional landscaper will produce dramatically different results than simply purchasing plants, establishing a relationship with someone who can look at your situation holistically is key to a successful software project. They know the questions that need to be asked, the process for finding the answers, and they know how to build out a development team that will take that reality into account.

The Automator's Manifesto, Part One

In my career in the software field, I've had to teach myself about what quality software was. What the differences were between successful software projects, and unsuccessful ones. It's been a long and often frustrating journey, but I feel like I've come out of it with some principles that could be helpful in guiding people in the field.

My principle strength in software development is pattern recognition, especially behavioural. I've used this over the years to refine my approach to software development, by analyzing my own behaviour and that of those around me. For this topic, I've tried to apply that at a higher level to my experience with test automation. What is it about automation that most consistently delivers value? What consistently doesn't? Are there practices can guide us towards the former and away from the latter?

Certainly, to a degree, everything here is derived from common software best practices, but I believe I am able to tailor them in a way that is uniquely suited to the automator. That's why this part is sub-titled…

Automation is Software

(I use the term "automator" as a generalization of test automation developer, quality engineer, software engineer in test, and a dozen other titles).

 

Apply Best Practices, as best you can

I'm sure we've all heard the statistics about how 50%, 75% or 90% of all software projects fail in some way. If we've been around the block, we may even understand some of the steps to take to improve our chances, commonly called "best practices". Automation projects are almost always directly dependent on some other software project, and when they go down, the automation often goes with them.

So from the start, the odds aren't good with your automation. But then consider that of all software projects, automation seems to be the place where best practices are least… practiced. I'm talking about things like:

• Requirements gathering
• Structured development process
• Documentation
• Unit, etc. tests
• Source control (I've seen this missing several times)

Not to say that every piece of automation needs all of these, just that an automation project is like any other software project, and if we fail to apply common knowledge about software projects to it, we are shooting ourselves in the foot.
What's more, by practicing these things, you become better equipped to ensure they get applied to the project you are developing automation for, primarily because it can help you establish credibility with the developers working on it. Walking the walk, and all that.

Have a Client

A key aspect of most successful software projects is the presence of an engaged client. As in most aspects, Automation is no different. Someone who understands the value that you are supposed to be delivering is in the best position to help you understand what needs to be done to deliver that value. Do not simply write tests because there was a decree "we need tests for X".

Instead, you need to understand where the desire for tests comes from? Is it that developers are hesitant to make changes because they have no safety net? Is it that there are lots of regressions in production? Is it that there's a performance optimization project underway, and metrics are needed? Each case would suggest a different focus, and there are many more cases.

But in order to get to this point where you have identified the need, and make progress against meeting it, you need a partner who will vet requirements and validate progress. You need a client. This can be a team of developers, a particular developer, someone in management, or others. But they need to understand the value you are expected to deliver, and have the time to confirm that you are doing so.

Fight on the best terrain available to you

As I believe is common to anyone who has been in this field for a while, actually automating tests is not always the best way you can improve the quality of the software. As described in "Test is Dead", many companies have user bases that accept being used for beta testing. Sure, it might not always be so, but if that is the situation you find yourself in, attempting to keep bugs from those beta testers will not be an ideal use of your time.

A robust functional test suite for a rapidly changing product might cost more in maintenance then it delivers in the first few years. A development team without a mature development process will gain little by forcing them to write unit tests.  Sometimes there's more value to be gained in doing SysAdmin or DevOps type work, such as creating continuous integration infrastructure. The important thing about being an automator is establishing processes that will make producing higher quality software easier. Whether those are actual automated tests run against software, or merely conventions established to improve communication during development is not important. What is important is that you make producing quality software the easiest thing for the developers.

In order of priority, I would suggest:

• Establish a development process
• Establish a development environment
• Automate tests
• Automate monitoring

Each of these makes the following item easier and more valuable.

Summary

Automation is Software, so:

• Follow best practices, as best you can
• Have a client, who can validate the value you deliver
• Understand the terrain of the project you are automating against, and tailor your focus to that

Join me next time as I dive into the value of Proactive Transparency, the danger of Entropy Factors, and the deceptive bliss of Measuring Everything.

Beyond GTAC

In 2013, in the twilight of my "testing" career, I was able to attend Google Test Automation Conference for the third time. Once again, I was impressed by (and jealous of) many of the solutions presented. But for me at least, I'm not sure I'm the target audience anymore.

About me

Before I get into that, let me expand on what I said above about twilight. The arc of my career is the story of trying to figure out how to use technology to improve people's lives. This began as computer and then network support, making sure people were able to use the tech they already had. From there, I began to write software, for myself and then those who would pay me to do it.

Throughout this process, I have had to constantly analyze and refine my own processes. I made mistakes, researched best practices, and tried again. Everything I do comes from continual experimentation and optimization. My career transitions also come out of this drive for self-improvement.

As I wrote and used more software, and especially assisted others in using it, it became (gradually) clear that the only useful measure of quality was whether it made it easier for users to accomplish their goals.

Most recently, that career has led me to look beyond myself in terms of delivering quality software. I'm now responsible for optimizing the output of a team of fantastic developers. In the majority of situations we currently encounter, writing tests is not the most efficient path towards this.

But enough about me

So, to a certain degree, GTAC may not be for me because I need to take a wider view then many of the participants, even though we share the same goals. But I think there's more to it then that. The people at GTAC are on average likely to be the best test engineers on the planet. Yet with few exceptions, the work they presented would have fit right in with GTAC from 4 or 6 years ago. As a community, we are failing to establish and exploit institutional memory, so we keep having to re-solve the same problems.

As a specific example, Google themselves have an incredible build and release system, as the result of a huge ongoing investment. From what we saw of every non-Google presentation, other companies capabilities for automated testing are hampered by the lack of this kind of infrastructure. Not to imply that Google has any obligation to provide it, but it seems to me that everyone could benefit substantially from more direct collaboration on it. Not to say the tooling in this area doesn't hasn't improved, but it still seems like a problem that every company continues to try to solve in their own way.

Don't get the impression from this that GTAC itself is anything less than fantastic. It has helped me grow so much, and likely raised my expectations for what should be possible. If you have any sort of passion for automated testing, there would be few things more valuable than connecting with this community. Also, despite my gripings, there are some things happening today that simply wouldn't have been possible in the recent past.

About those exceptions

Selenium. Not to be trite about it, but the Selenium project is the biggest change to the test automation community over the past 6 years. It's gone from being a nifty tool for somewhat flaky web automation to a standard (literally) platform upon which we can converse about automation and build other tools. Some examples of this include Appium, which is leveraging it to provide cross-platform mobile app testing, and Marionette, which is integrating Selenium into builds of Firefox OS. It's not the end of the story by a long shot, but having Selenium allows us to elevate our dialogue around test automation.

Misc. Buck, by Facebook, seems like a nifty solution to a problem I almost have (though I'm not sure when +Simon Stewart finds time to stop being funny and start writing software). Almost everyone agrees you should just kill flaky tests, which is good, but I'm not sure it's the endpoint in that conversation. Getting developers to write tests was also a near-unanimous sentiment (and why wouldn't it be in this group?), though unlike Thomas Petersen, I didn't get the impression that it was intended to exclude testers from the development process. Pex4Fun is just cool. The Android UI testing team embedding their automation commands in the build thread seems a bit crazy, but still a really clever way to avoid the whole click-and-sleep cycle that plagues most UI automation.

Organization. Despite having a more limited experience of the Google office, I feel like this was by far the best run conference I have ever been to. Things were on schedule, very smooth, and +Anthony Voellm is one heck of an MC.

Summary

GTAC remains in my mind the place to be if you are interested in doing test automation. I believe there's more that the test automation/software quality community needs to do to grow together, but the fact that this conference isn't all about that is mostly a reflection of my changing priorities, rather than any failings on their part.

Full disclosure, I spent 3 years in automation at RIM BlackBerry, and this was a mobile testing conference, so I'm probably just jaded.