Robby on Rails

As if delivering projects wasn’t hard enough. Delivering projects on time is even harder. As practitioners, we’re all responsible for measuring up the obstacles in front of us and are accountable to those measurements. At least, we should be.

One of those measurements is time. Time is a funny thing. People have a lot of interesting things to say about time. Some say that it’s one of the most valuable things that we have… but I’ll avoid diving into a philosophical discussion for now.

What I wanted to talk about was project estimation. Specifically, estimates for deliverables. For the past several years, our team has put a lot of effort into becoming more accurate in our time estimating skills. Despite analyzing how often we over and/or underestimate the time each of us believes it’ll take to complete a task, we find ourselves coming back to the drawing board.

A few things that we’ve learned.

• Tasks that we believe will take a few days/week/more to complete are often underestimated
• Tasks that we believe will take less than a few hours are often more accurate or overestimated
• Too many tasks were completed with a bigger budget than was necessary (lower ROI)
• A lot of time was spent working on requirements refining to get better estimates

When we began to step back from this and look for patterns, we found that several of the tasks that we would budget hours for (versus estimate hours for) were proving to be more accurate. This approach is most commonly known as timeboxing. With timeboxing, we can place a dollar value on a specific task and work within that constraint. For example, with our clients, both parties can come to the conclusion that, “we believe that it’s worth up to $800 to implement this new functionality.” With that, we’re able take that dollar amount and figure out how many hours to box ourselves within.

The underlying question to our client with each change/feature request is, “How valuable is this to your business at this point in time?” Whereas, with a typical approach to time estimates, a client comes to you with a list of changes/features and you provide them with time estimates. “We estimate that it’ll take 6 hours at $200/hour for feature X and we’d do it like this…” The client will have to evaluate your estimate and figure out if it’s worth $1200 and make a decision. They can respond with, “no, that’s too expensive, can we do it for less?” The following steps would entail your team trying to find ways to reduce your estimate.

While these two paths might seem very similar, it’s been my experience that the standard approach to estimating takes more time for negotiating the terms of the agreement.

However, with timeboxing, you are asking your client to provide you with an initial budget. This will completely change how you respond to the feature request. When you have a timebox, from the moment that you begin to evaluate the request, your brain will add the necessary constraints to keep things within scope.

Through this process, we’ve revamped our estimating process so that as we’re building our iteration costs for clients. For each deliverable, we break down a series of objectives/tasks and apply timeboxes to each of those while knowing what the budget is for the deliverable as a whole. Usually, the deliverable is directly related to the request that came from our client with a budget. The process is completely transparent and our team is responsible for working within those constraints.

..and as we’ve learned from Ruby on Rails, constraints can be extremely beneficial.

While I don’t have all the answers yet, my goal is to share some of my experiences and lessons on the topic. I’d love to hear about how you’re adopting timeboxing in your project planning and estimating process.

Anyhow, just some thoughts that I wanted to share. More to come…

Read Related Articles

• The Art of Delivery
• The Art of Delivery, part 2

Two years ago, I wrote an article titled, The Art of Delivery. I wanted to post a few updates based on how our process has evolved since then (and continues to).

Over the past few years, we’ve been fortunate enough to work on quite a diverse collection of projects. This has enabled us to work with many different clients and solicit feedback on our process. This has given us an opportunity to evolve a set of best practices that fulfills the long-term project goals/budgets of our client while making sure that we’re able to maintain a design and development process that is agile.

As I’ve mentioned in previous posts, our team typically bills work per-iteration on projects rather than per-hour or a flat-bid per-project. Since iterations are bite-sized pieces of the entire project and limited to 1-2 weeks, our teams estimates are much more accurate and we’re able to keep things rolling and on track.

The basic structure of our project looks like this.

• A Project has many releases
• A Release has many iterations
• An Iteration has many deliverables
• A Deliverable has many tasks

Before we begin working on an iteration, we outline a set of goals that we want to create solutions for. This process comes out of discussions between our client and us until we agree on what is the highest value/most critical to the success of the project, based on our shared understanding of where we are today. These goals translate into Deliverables, which in a typical iteration might require Interaction Design, Interface Design, or Development. We tend to break our process up into stages so that Interaction Design on Module XYZ would be implemented in a following iteration. This is because it’s unrealistic to expect someone to provide an accurate estimate on how long it’ll take to implement something before you know how people will interact with it.

Within any given iteration, our team is spread across several sets of deliverables. As a team, we breakdown these deliverables into smaller sets of tasks. It’s our aim to keep tasks smaller than a full days worth of work as it’s much easier to measure progress across the iteration when we can track tasks at a granular level.

Essentially, tasks are the individual steps needed to achieve these goals. We don’t go out of our way to list each one of those during an estimate process as some tasks take less time than it takes to generate an estimate for them. Each person providing estimates should avoid getting too granular and aim to find a good balance that compliments their workflow.

Like most things… mileage may vary.

Through this process, we can get calculate the estimated costs for each deliverable, which then provides us an cost for the entire iteration. In addition to deliverables, we also budget a set of hours/days so that we can be compensated for handling small requests, bug fixes, and project management. It’s important to factor these things into your process.

In future posts, I’ll discuss how we’re handling this process while working on multiple projects… as that’s where it can chaos can start if you’re not careful. ;-)

How does your team work? As we’re always evolving our process in an effort so that we can be more efficient and speed up our delivery cycle, I’d love to learn from those in the community.

While engaging in another one of our deep philosophical conversations in #caboose, the topic of remembering that great idea that we had before we fell asleep or at any point during the evening.

Manfred Stienstra writes, “I always think of stuff right before I fall asleep and can’t remember them when I wake up.”

This happens to me as well, just not as much as it used to. I keep a small moleskin notebook and a pen on my bed side table and try to get everything in there as it comes up. When I open it, I see notes that don’t make any sense, which were probably after waking up from some bizarre dream. There are many business and development ideas that come up, but it’s often hard to record everything.

I’m sure that there are others who’ve faced this problem. Have you come up with a working solution? If so, would you mind sharing it?

Our team is lucky enough to be in a position where we have both designers AND developers working on the same code base in parallel.

Since Ruby on Rails adopts the Model-View-Control pattern for separating business logic from the presentation layer, we’re able to give our designers a lot of breathing room to to design the interface, whether it’s for interaction or aesthetic reasons. However, sometimes this breathing room has resulted in small bugs slipping into the application interface. In general, nothing disastrous, but each bug that slips into the queue, slows down the project and we want to avoid as much of that as possible.

I’d like to share a few issues that we’ve seen occur on various occasions, and then show you what we’ve done to avoid them happening again.

Scenario #1: The case of the changed div id, (victim: designer)

• Designer adds a few HTML elements to the page, defines an id on a <div> tag and styles it with CSS.
• A few days later, a developer needs to make some changes, tests it in their favorite browser and commits.
• Later, the designer doesn’t understand why the styling is all messed up. “It was working fine.”
• ...minutes, hours… go by where the designer tries to track down the issue. “Oh! Someone renamed the id in this <div> tag. Sigh.”
• Developer apologies, but explains that he needed to do it because he needed to make it work with his new RJS code.

Scenario #2: The case of the changed div id, (victim: developer)

• Developer is implementing this cool new Ajax feature into the web application
  • The code relies on there being one or many HTML elements in the DOM with specific id values defined.

Example: <div id="notification_message">

• A few days later, a designer is making some changes to the layout and needs to restyle some of the view that this <div> tag is defined. Designer decides to change the id to a different value for any variety of reasons. (or perhaps they changed it to use a class instead of styling it by the id). Often times, we don’t know who set the id or class… and many times the developers aren’t savvy enough with HTML and designers end up cleaning things up a bit.
• Later, code is checked in and designer didn’t notice that the Ajax was now breaking as they weren’t focusing on just the layout.
• Day or two later, developer sees bug, “Feature X isn’t working, throwing JavaScript error…”
• Developer is confused, “Hey, that was working! What happened?”
• Developer tracks down the problem, discusses with designer, they figure out a solution. Problem solved.

I could outline a few other examples, but I really wanted to highlight these two types of situations, as our team has seen this happen on several occasions. Luckily, we’ve learned through these experiences and have taken some measures to try and avoid them in the future.

Moving forward (together)

Both of the examples above, were essentially the same problem, but resulted in problems for a different role in the design and development cycle. While, I’ve definitely been the victim of #2 several times myself, I know that I’ve also been the guilty of #1. So, what can we do as designers and developers to work with each other without causing these little problems from occurring? (remember: many little problems can add up to a lot of wasted time spent resolving them)

Several months ago, I had a meeting with Chris (User Interface Designer) and Graeme (Lead Architect/Developer) to discuss this very problem. At the time, we were implementing a lot of Ajax into an application and were occasionally running into Scenario #2. We discussed a few possible ways of communicating that, “yes, this div id should NOT be changed (without talking to a developer first)!”

Idea 1: Comment our “special” HTML elements

We discussed using ERb comments in our views to do something like the following.

  <% # no seriously, please don't change this id, it's needed for some Ajax stuff %>
  <div id="notification_message">
    ...

We all agreed that, while effective, it was going to clutter up our RHTML code more than any of us desired.

Team Response: Meh.

Idea 2: Reserve id’s for developers

Another idea that came up, was to ask that designers only use classes and ids wold be used by the developers when they needed it.

  <div id="developer_terriroty" class="designer_territory">
    ...

Chris pointed out that this wasn’t an ideal solution as there is a distinct case for when to use ids versus classes.. and he is very strict about adhering to the HTML/CSS standards.

Team Response: Not hot about it…

Idea 3: Naming convention for Ajax-dependent elements

The third idea that was discussed, was specifying a naming convention for any elements that were needed by our Ajax code. We played around on the whiteboard with some ideas and settled on the idea that we’d prefix our id’s with something easy to remember for both designers and developers.

We agreed on… x_ (x underscore), which would make an element id look something like this:

  <div id="x_notification_message">
    ...

x == ajax... get it?

While this adds the strain of typing two more characters to much of our RJS code, we don’t run into Scenario #2 very often anymore.

  render :update do |page|
    page[:x_notification_message] = 'Something exciting happened... and this is your notification!'
    page[:x_notification_message].visual_effect :highlight
  end

or in client-side JavaScript (where we also use this)...

  $('x_notification_message').do_something

I find that this helps our team keep a clear distinction between what can and shouldn’t be changed in the views by our designers. Sometimes they have a good reason to do so, but they know that if there is x_, then they should ask one of the developers on the team for assistance in renaming it without causing any problems in the application. It also allows our designers to add classes to these elements, or style the id that we’ve defined.

Team Response: Wow, was that all we needed to agree on? Hooray!

This leads me to some other problems that have/may come up, but I’ll discuss that in my next post on this topic, when I’ll show you how we can use RSpec to avoid these sorts of designer versus developer problems.

If you’re working in a similar environment, how are your designers and developers working, together, in perfect harmony?

Until next time, remember to hug your designer. ...and if you’re still having developer design your applications, consider hiring a designer. ;-)

UPDATE: changed examples after a few comments about using div_for as another solution. (see comments for details)

How many times have you participated in a conversation with someone and realized that you really didn’t understand what they had said. Or… perhaps you’ve been talking and even though the other person is nodding, you’re not confident that they’ve really heard what you’ve been saying. Yet, you might find yourself nodding in agreement when they speak… and walk away… totally clueless about what you just talked about.

Were you really listening? Were they speaking over your head? Were you speaking over their head? Perhaps you were distracted? Whatever the reason… it’s probably worth thinking about. We all do it from time to time.

Even worse, you were only thinking about how they were wrong and you had the right answer already in your head…

In Dialogue, there are rules for participation, which we’ll explore in future writings.

One might wonder if we’ve been trained to work this way. In school, we had classes that taught us to debate one another… further cultivating a society focused on you versus I. But, what about the community? What about the team? What about us? Sadly, most of the teamwork that we saw encouraged was in the form of sports. To be fair… we did have debate teams… but the purpose was to argue for one side of an argument… not to find a way for both sides to work together. One might wonder our society would be like if we encouraged Dialogue in the same way.

Perhaps we need Dialogue teams. ;-)

Dialogue allows teams of people to work together. It’s a process that cultivates learning and discovery. Dialogue is not a process that encourages the passing of judgement or pushing for specific outcomes… the aim is to share understanding. Through empathetic listening and questioning, the seeds of trust are planted.

Dialogue-Driven Development is about building trust.

I came across this great table, which contrasts Dialogue and Debate. It’s worth taking a few moments to review.

Here are a few that caught my attention…

Dialogue	Debate
Dialogue is collaborative: the sides work together.	Debate is a type of fight: two sides oppose each other to prove each other wrong.
In a dialogue the goals are finding common ideas and new ideas.	In a debate the goals is winning with your own ideas.
In a dialogue you contribute your best ideas to be improved upon.	In a debate you contribute your ideas and defend them against challenges.
In a dialogue you listen to each other to understand and build agreement.	In a debate you listen to each other to find flaws and disagree.
In a dialogue you may consider new ideas and even change your mind completely.	In a debate you do not admit you are considering new ideas and you must not change your mind, or you lose.
Dialogue encourages you to evaluate yourself.	Debate encourages you to criticize others.
Dialogue promotes open-mindedness, including an openness to being wrong.	Debate creates a close-minded attitude, a determination to be right.

There is something to be said about the art of Dialogue, which is why we’re so excited about the d3 project.

What do we mean exactly when we say that we want to participate in thoughtful dialogue in a project? What is our intention with this? When I recently came across some essays by David Gurteen and read his definition of dialogue as being “a disciplined form of conversation” it got me thinking about how we often forget that like all skills, practice makes perfect. What make our conversations discilplined in the first place? Based on my experience, dialogue (disciplined conversation) manifests when all participants in a conversation are practicing mindfulness. I don’t believe that most people learn or behave well by being beaten into submission, so we must come to an understanding while we actively involve ourselves in dialogue. Most of us are civil towards one another, which does wonders for allowing us to tolerate each other, but I still can’t help but feel that we’re still missing the mark when it comes to having consistent and thoughtful dialogue.

Over the past several months, our team has been spending quite a bit of time and energy analyzing these problems. What we really starting to uncover is that the real problem seems to exist somewhere outside of our development methodologies. Working under the Agile umbrella provides no silver bullet. The real issues seem to exist much deeper in our human nature.

We’re not all that great at communicating

Humans are not perfect… therefore… our ideas are probably far from perfect as well. Our thoughts aren’t perfect. Our interactions aren’t perfect. We’re consistently inconsistent (heh) and while we can rely on averages to some extent to calculate probabilities, we can’t always explain why somethings still go horribly wrong. The principles outlines in the Agile manifesto stress the importance of focusing on people not processes and responding to change. If we are to put a heavy focus on the people involved in projects, we must acknowledge our strengths and weaknesses and find innovative ways to improve our communication skills.

On a daily basis, we’re faced with complex problems. Hopefully, we’re using a lot of our prior experience to aid us in making rational decisions about how we respond to them. There is a lot that goes through each decision that we make. We can’t automate this process (yet), but we can definitely share our lessons with one another. Our intentions need to be thoughtful and empathetic to the needs of all parties affected by each decision. As humans, we have the opportunity to really listen to the concerns of others and use not only our logical intelligence… but also our emotional intelligence.

Much of this comes down to each of us learning to understand how we make decisions and interact with people. It’s our goal with Dialogue-Driven Development that with your help, we’ll be able to outline patterns of dialogue, which we hope will be of great value to the community. Our team has been analyzing our interaction with clients and discussing what has worked well and what hasn’t. How did our clients respond to approach X versus Y? It’s important that we capture this information and have conversations about the results.

“The meaning is what holds it [dialogue] together. ..Meaning is not static – it is flowing. And if we have the meaning being shared, then it is flowing among us; it holds the group together…in that way we can talk together coherently and think together.” – David Bohm

Doesn’t that sound beautiful? Who wouldn’t want to reach such levels of project enlightenment?

d3 aims to be to communication what BDD is to specification

While at RailsConf Europe, I had the privilege to speak with several Rail developers… several of which are doing contract development for several clients, just like our team. We discussed d3 for a while and I walked away feeling really excited about the whole concept. When I explained that our team didn’t see d3 as a replacement for Agile methodologies like Scrum, XP, etc… but as another tool in our tool belt. Dialogue between developers, clients, and users should be agnostic about particular methodologies. We’re really excited about Behavior-Driven Development as a best practice in our development process and we’re seeing Dialogue-Driven Development as another best practice that we start using from the initial point of contact with a potential client to long after we deliver the working product that we were contracted to develop.

We’ll be posting some fun announcements about the d3 project in the coming week(s). Stay tuned…