Sunday, 31 July 2016

More on estimates - From upcoming book Agile 101

When we talk about estimating using Agile techniques, some things are difficult to accept for those of us who have been planning and calculating timing and deadlines for a few years.

We can find several techniques for Agile planning: Planning Poker, clothing sizes for task classification (S, M, L, XL, etc.) or Team Estimation Game, but, let’s face it, they’re difficult to integrate in a traditional development environment, and even harder to integrate for the clients who purchase our products (or at least, for the clients’ I’ve been able to work with).

We’re not used to seeing a team “playing cards” for a session of Planning Poker (at least, I am not used to it). I do, however, agree with the philosophy behind those techniques. I believe that they’re becoming more and more popular partly because it is necessary to dispel some myths regarding estimations. Let’s try and do just that:

We do estimates wrong

Can we do them right? To estimate is to predict how long a task will take or what materials we will need for a certain job. When we do an estimate with high uncertainty, as for example when we use the data included in a public-tender document, it’s not so much an estimation as a bet.

We need estimations—that’s obvious. We must try to predict what will happen, so we can make decisions. But unless we’ve gone through that specific task many times before, with the same team and under the same circumstances, we need to be aware that our estimates will very likely be wrong.

The more effort we put into an estimation, the closer we’ll be to reality

Estimations have a cost and that cost is high. When we don’t know every specific detail of every bit of the work we’ll be taking on, does it make sense to spend hours and hours to guess about tasks that we do not know very well? When you finish your next project, do this exercise for me. Review the time spent on every task. You’ll see that you estimated 20 hours on tasks that took you 100, and you’ll find that —with luck— you spent 20 hours on tasks that you thought would take you 100. You’ll see that there were some tasks that no-one marked as done. They simply weren’t necessary. However, hours are spent on tasks that were added later and that no-one considered necessary in the initial estimate.

We’ll estimate better with this or that technique

There are many estimation techniques but very few will be as useful as comparing the time spent in similar projects, the decomposition of the work in smaller tasks, or the conclusions of experts (it would be better if we have several of these). The fact that we’re using complex estimation techniques could give us a false sense of security about our estimate, and prevent us from striving for higher reliability. That could actually cause us higher costs (see above) where we invest time that would have been better spent doing something else (like reducing uncertainty).

You can find texts like this and many other about how to manage agile projects in my book Agile 101: Practical Project Management (available on Amazon).

Translation by Begoña Martínez. You can also find her on her LinkedIn profile. Proofreading by David Nesbitt.

Sunday, 24 July 2016

From Agile 101 book: Real artists ship!

Sometimes, and quite often, we delay delivering our work because we believe it is not ready. We think that we still have things to improve or correct, that we still have to work on it. More often than not, that actually comes out of fear of failure. Fear of our product meeting the market and nobody buying it, because it is not perfect, or it is not good enough.

It can also be fear of our product being criticized, because either the product or the book or our software has some typo or small bug. “I will only publish the latest blog post when there are no possible ifs, ands, or buts.” Other times it’s just perfectionism leading to paralysis.

A pottery teacher in an art school divided his students in two groups and told them what they had to do to get an A+ in his class. He told the first group that they had to do 100 vases in order to get the highest mark. However, he told the second group that to get that A+ they had to make the best vase.

This last group devoted enormous amounts of time and energy to debate and read about the best techniques for creating vases. But the best vases were actually done by the first group. They had devoted their time to practice, practice, practice and made 100 vases. Their technique had improved. It’s the same with your blog, your app or that design you’ve been thinking about. Let them meet the real users and let those users give you a thumbs up.

The world of software is full of products that were developed over years until they had every feature a client could imagine, in the way they were initially designed by their creators. Sadly when the software met the market, the market had its own ideas and needs. Very few people used every feature in those applications. Other features were heavily demanded, but none of the creators had thought about them when those products were first designed.

There is a Spanish proverb: “perfect” is the enemy of “good.” When you have a small but already useful product, ship it, show it to the public, get your first clients. They will tell you what’s missing. Learn about that, launch a second version, then a third. If you can’t get at least a small client base with your first version, maybe it is not the right time or place for your product. At least you’ll learn lots of things you didn’t know yet, and you won’t have wasted months and months of your work.

You can find texts like this and many other about how to manage agile projects in my book Agile 101: Practical Project Management (available on Amazon).

Translation by Begoña Martínez. You can also find her on her LinkedIn profile. Proofreading by David Nesbitt.

Thursday, 21 July 2016

From Agile 101 book: The Ant and the Cicada

One summer a cicada was resting in a field, when she noticed a small ant running around here and there.

- “What are you doing?” asked the Cicada.
- “The Queen has asked me to build a new anthill!” replied the Ant.
- “But they already have one there, just a few meters away” offered the Cicada, puzzled.
- “Yes, but it’s getting a bit small, and in winter, with the rain, the things we have collected get wet

The Cicada, ready to learn from her hard-working friends, decided that it would be a good idea to prepare for the winter. The leaves and sticks she used to build her shelter the previous year were getting dry, and it wouldn’t hurt to have a new home to spend the winter in comfort.

She quickly started to plan the wonderful shelter she would build. Now that she had time she would build it with every possible comfort. She would be the envy of the whole meadow. The stories about her new home would be heard far away, further even than the cypress hedge. It would have spacious windows so the light would flood in, and she would design a clever pulley system that would save her many trips to and from the barn to store what she had collected.

The Ant, however, had a plan. In August she would build the first room, which would be the barn. That way, if the rains started early this season, she and the other ants would have at least some extra space to store their grain. Then she planned the rest of the work according to its importance. If she didn’t have enough time to finish it all by the deadline, she would have built at least the most important parts.

In September she would dig a hole that would later become the hall were the first larvae of the year would live. In October she would build an additional room to increase barn capacity. Lastly, in November she would finish the work by digging a hall where she could grow fungi.

In the meantime, the Cicada was worried about having a dining hall that was big enough, and a structure as solid as necessary to hold everything she had imagined. She would run back and forth looking for the materials she would need to build all the things that she had planned in her head.

In October the first rains came, but the Cicada didn’t even have a basic shelter to take cover in. Also, water ruined some of her first constructions that were halfway built, and she had to start again. Then November came and the weather became really ugly. She had to hurry up and started working from sunrise to sunset.

Those first October rains made the Ant think that something could go wrong, and she decided to test whether the work that was already done would be useful. She started to fill her new barn with grain and soon she noticed that the access hole was too small for big leaves, and that it should be a bit higher if they wanted to avoid water or mud coming through. In order to fix this, they had to stop doing other things.

Winter came early that year, and the Ant realized that she wouldn’t have enough time to build the last room, so in the main anthill they decided that they should open an extra entrance, in case the main one got blocked by a stone or an enemy.

The Cicada, however, was surprised by the bad weather. She didn’t really know what was finished and what wasn’t. Also, what she had considered finished had not been tried out, and now there was no time to improve it. Winter was here and no more work could be done.

In her new shelter, under the leaking roof, the Cicada promised herself that this would never happen again: “Next year I’ll start work even before the Ant,” she said to herself.

You can find texts like this and many other about how to manage agile projects in my book Agile 101: Practical Project Management (available on Amazon).

Translation by Begoña Martínez. You can also find her on her LinkedIn profile. Proofreading by David Nesbitt.

Tuesday, 19 July 2016

How we used Scrum in our software projects

These days I have been preparing and updating a Prezi presentation about how we used Scrum (or Agile development) in our software development projects.

You won't find there a Scrum theory implementation but the process and methods we used to try to become more Agile (whatever that means) and to deliver software regularly and frequently (working software).

Reach the presentation at Prezi: Practical Scrum - How we used it.
or have a look at it here:

Sunday, 17 July 2016

Functional tests with Selenium WebDriver

(Sorry, this is a technical post about software testing, but don't worry, you can find here some of my non-technical blog posts)

A few years ago we used Selenium IDE in a web application project just to check the functionalities developed at that sprint were working as expected but there were some problems with that. I try to sum up them here:
  • Selenium IDE works only with Mozilla Firefox web browser. That was a problem when you did need to test another browser or your web app worked only in MS Internet Explorer.
  • Tests and tests suites used to become red frequently. Firefox updates (on security mainly) made script commands deprecated easily so it was needed to search for a replacement, when available, or to find a workaround (after a few hours in Stackoverflow). To keep them working in order was costly (and time consuming).
  • There was some kind of a mess with script commands: type or typeKeys, click or clickAt, mouseAt or mouseDown. You never knew.
There were also some advantages, of course, like the way the user actions could be recorded by the IDE. You just had to click somewhere, move the mouse or type something that your movements were recorded by the IDE as script commands. It was needed to adapt or fix them after recording it but it used to made things easier. 

These days I have been playing around with Selenium WebDriver. It doesn't need to inject javascrit code into the web browser and you can use your favourite programming language to write the tests.

I have written a couple of tests to check one of my web applications just to show you some of its main features and how Selenium WebDriver can work on Chrome. At the end of this post you can find a video showing how these tests work.

This first test is a kind of smoke test just to check that the application is working properly and shows the main page when it is requested:

Below you can find how to get the driver variable for Chrome. Using it you can have access to all WebDriver commands. It is written inside the jUnit @Before annotation so it is going to be executed before any other test run. At the @After method there will be a driver.quit(); sentence.

This last test actually checks that 10 questions are asked at each attempt (no more, no less), chooses randomly one of the multiple choice options and ensures that there is a results page at the end.

This is the Selenium/JUnit tests video recording:

You can find texts like this and many other about how to manage agile projects in my book Agile 101: Practical Project Management (available on Amazon).


Saturday, 16 July 2016

Agile 101: Scrum’s sprint 0 and the initial design

Sometimes we’re in projects in which we need to prepare a large number of things before starting out. Can we start building without stopping to plan or analyse what we’re about to do?
When we read about Scrum, we get the impression that all texts start directly with development, but…what about environment, team or office preparation tasks? What about design? Shouldn’t we stop to think about the whole design and architecture of what we’re about to build before we start programming?

About preparation or “before” programming. Some people call it “sprint #0”, the sprint where you prepare your tools, install the stuff you need, train the team, etc.. It can be a longer sprint (3, 4 weeks or so). Some detractors criticise this sprint because it is not agile, and because it hides some work that is taken care of in the same way as the rest.

It is true that it can be very comfortable, because we tell the client that we’ll be back in 3 weeks to have the preparation meeting for the first sprint, and by then we have organized it all. I have also used those zero sprints in my first projects, but now I prefer to solve those issues also in an Agile manner.

I try to make every one of those tasks (installing the development server, installing the demo server, creating a project and tasks in Trac or Redmine, installing the Integrated Development Environment or IDE, etc.) part of the first sprint and, if possible, have a set deliverable at the end of each demo.

In that demo, we could show, for example, the project-management tool already installed in the final URL and the task list with the tasks ready to be assigned, or we could launch the IDE in that demo meeting, and check whether the environment has been created correctly and whether Jetty executes a “Hello World” application correctly.

Another important step is the initial architecture design. If we make a huge design with all the application architecture (known as a big up-front design) before we start programming, we will be returning to a waterfall design. Firstly, we analyse everything, three months later we do all the design work during a couple of months, and then we program everything in one go until we finish it six months later. If we finish all work and show it to the clients and they say: “that is not what we discussed a year ago…” then we have a problem.

It is better to design one of the modules of the application, or one of the new features, then draft the specifications, and finally send them to development. While part of the team implements them, analysts can gather requirements for another module or another feature, and go on doing so sprint after sprint.

There will be sprints in which we won’t have new stuff to analyse and others in which the development goes a bit faster than the specifications, but normally there is always some task ready to be carried out while Product Owners sign off the latest requirements, or until those requirements are completely defined.

If we work like that, new modules or features could be regularly deployed to production or preproduction, so users will be able to take advantage of them without waiting until the very last feature of the application has been designed.

You can find texts like this and many other about how to manage agile projects in my book Agile 101: Practical Project Management (available on Amazon).

Translation by Begoña Martínez. You can also find her on her LinkedIn profile. Proofreading by David Nesbitt.

Thursday, 14 July 2016

Agile 101: Scrum for fixed-price projects

One of the most common questions we ask when we’re wondering whether we should adopt a methodology like Scrum is how to face the fixed-price contracts that are so common in public procurement and many other sectors. One of the Agile principles says: “we welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.” This means that using Scrum we will be flexible with features we must develop, and we will admit new ones that were not in the original contract listing. This sounds very good but… the final price of the project is not going to change!

I work in the public-procurement department in a software-development company. That means that in at least 90% of the projects I work on I have a public institution as a client. Contracts are normally defined by a document with specific administrative specifications and another one with technical features. These contain all the services that are being hired, the characteristics of the product to be developed and the maximum price of the tender. They include even the technical features of the software and the profile of the projects’ participants. It is all very narrowly defined. It is the price that the public administration must pay to avoid arbitrariness.

If our company has been lucky enough to win the tender, then both of us, provider and client, have to face the harsh day-to-day reality of projects. Circumstances change. New legislation comes up that affects the project. Things that were deemed correct a few months ago are no longer valid, because needs have changed. What should we do? If we adapt the project to every change that the client needs, then we providers have to bear the brunt of additional cost. If we refuse to change anything for the client, always waving the contract and its specifications in their face, we risk delivering something that might be perfect for last year’s decree, but—for that very reason (or for any other reason)—is no longer of any use.

The way to avoid this, or at least the way that has worked best for me so far, arises from the way of working in Scrum projects. At the beginning of the project I make a list with the features to be developed, and give them a score that the client knows right from the beginning.

I normally give the example of a large jar full of ping-pong balls. Each ball is a feature that must be developed. The jar is full. If the Product Owner wants to include two new features, I ask that person to tell me which other features—or ping-pong balls of a similar size—should we take out of the jar. Normally this vision solves the predicament and the Product Owner does not resist giving up other features, now recognizing that they aren’t important. If it all gets recorded in the meeting’s minutes and, most important of all, if the Product Owner understands what is being given up and what will be gained instead, it is normally a comfortable solution for both sides.

You can find texts like this and many other about how to manage agile projects in my book Agile 101: Practical Project Management (available on Amazon).

Translation by Begoña Martínez. You can also find her on her LinkedIn profile. Proofreading by David Nesbitt.

Tuesday, 12 July 2016

Agile 101: Scrum in a maintenance project

There is a common doubt among those considering using Agile methodologies to manage a project: would it be possible to apply Scrum to projects where, on top of the upcoming and planned tasks, there are frequent interruptions to solve maintenance problems, to correct errors or to resolve issues? Many project managers face these types of problems and use several approaches to tackle them. Let’s see some of those approaches:

Short sprints

With this solution, we will keep the tasks that have already been programmed for that sprint. If our sprints are 1 or 2 weeks long, we will be able to make the unplanned tasks a priority on the to-do list for the next sprint. If the sprint is one week long, we will start urgent tasks after 3 days, on average.
In an ideal world, this would work but it would pose some disadvantages. Not every issue can wait for a few days to be solved. A whole service could depend on it.

Low load factor

If we know that as a general rule we will have unplanned tasks or issues that we must solve quickly, we can lower our load factor during the sprint so we have “room” to solve these problems.
If in each sprint the team has the capacity to solve 10 planned stories, we could promise to deliver just 7, so the team has time to solve urgent issues. That way we won’t fail when it’s time to deliver what we promised, sprint after sprint.

In my opinion this solution can be useful in some projects but it could create another problem. Let me explain. Normally the load factor is 75%. If we lower it so we’re able to devote 30% of the team’s time to urgent tasks, we should apply a load factor ranging from 40 to 50%. On this 40% we’re using Scrum but, how are we managing the rest of the team’s time? What do we know about that pile of tasks that we’re solving sprint after sprint?

A different team for each type of task

This solution consists of having a Scrum team for the identified and planned tasks, and a Kanban team for the urgent issues. With Kanban we can add new tasks to the TO-DO column as they arrive, and we will follow them up until they get shifted to the DONE column. With this the team will be even more agile, reducing the overhead in meetings and planning that we could suffer with Scrum.
We still have a dilemma with unplanned tasks, which are, well, unplanned. There will be times when the Kanban team providing support will be oversized, because there are very few issues. But then a week later the number of issues and their importance could be so high that we will need any help we can get.

To minimise those risks, we could rely on the previous solutions. We could have short sprints, so the Scrum development team can plan the tasks for the next iteration. We could also use a lower load factor so the team can have a bit of room in their planning to give a hand if needed. In the same way, the support team can help with planned sprint tasks when their TO-DO column starts to look empty. To make the most of this, it would be good if all members in all teams rotated, so that everyone knows every aspect of the job.

You can find texts like this and many other about how to manage agile projects in my book Agile 101: Practical Project Management (available on Amazon).

Translation by Begoña Martínez. You can also find her on her LinkedIn profile. Proofreading by David Nesbitt.

Sunday, 10 July 2016

Agile 101: How to write an Agile book

Yes, you can also write a book in an Agile manner. Not only software can be created this way. All these Agile things can be applied to a myriad of fields, not only to technology. In this chapter I will tell you how I used this work philosophy to write the book that you have before you:

Focus on what’s important, delete the non-essentials

Nowadays books—especially technical books—are going through the same phenomenon as CDs did a few years ago. You bought a CD only because you had heard that song on the radio, but, how could a CD have less than 20 songs (and cost less than 20 dollars)?

It had to be filled up with something. Maybe the last few songs did not have the same quality as the two or three first hits in the disc? Well, it was necessary to justify the price, to pad it a little. This could make you hate the author for those last three songs in a duet with Tom Jones or the techno-pop version of their first hit.

The same can happen with books. We feel better if our book is 500 pages long, and we write it for two years but, at what cost to you, the writer? And for the reader who buys it? Probably that reader is only interested in the chapters on Agile quotes or about Scrum’s advantages.

Something of the sort happened to me while I wrote this book. It had very few pages and I was tempted to add some filler chapters. They weren’t really as interesting as the rest and they weren’t as close to the main subject of the book. I ended up taking them out. The first version of the book was about 67 pages long, but I chose to keep it that way instead of having readers think that the book was too long or too boring.

Real artists ship!

This expression, attributed to Steve Jobs refers to the fact that real artists are not constantly re-thinking their work until they have the perfect painting or the perfect sculpture. They ship their works, put them up for sale, and then see what the market is really interested in.

Have you written a tutorial on how to install Pentaho? A Wordpress configuration manual? What are they doing in a drawer? Get a cover (there are hundreds of websites for that), format it, write an attractive description, and put it up for sale at Amazon’s KDP, at iTunes or wherever you want.

It is only 30 pages long? Well, then sell it for one or two dollars. If you sell a few you’ll get a return for the 20 or so hours that it took you. Also, you’ll have learned lots of things about digital marketing, sales, royalties…and also about which kinds of subjects sell books and which don’t.

Perfection is a vertical asymptote

No matter how much you try to write the perfect book, with the perfect cover, with the exact price to maximise sales and no typos… you won’t be able to do it. Perfection is a vertical asymptote (sorry again, Math 101 Calculus).

The better you try to make it, the higher the cost will be for you. When you’ve put an enormous amount of hours into it, putting in another enormous amount won’t get you any closer to getting a bestseller.

When I put the book up for sale, I realized from the first opinions on the book that users thought it would be a Scrum manual. I had to learn from that and change the description to make it clearer. Small changes in the description had very high impact on sales.

In the same way, when I updated the book to its second cover, sales increased by 30%. Sadly it went further down when I changed it to the third cover. It was more expensive, and in my opinion prettier, but apparently Amazon’s users didn’t like it.

Your leads will tell you if they find the content, the cover or the subject interesting or not. You can think it over again and again, but until the book is not in the virtual shelves you won’t know what works and what doesn’t. Avoid the paralysis that sets in with perfectionism and don’t overthink it. Put your book up for sale and let them decide (remember, real artists ship!).

You can find texts like this and many other about how to manage agile projects in my book Agile 101: Practical Project Management (available on Amazon).

Translation by Begoña Martínez. You can also find her on her LinkedIn profile. Proofreading by David Nesbitt.