Let's face it: Estimating is hard!

In the world of software, “estimating” is the process of quantifying how long it will take to complete a certain task, and it can be one of the most challenging aspects of a developer’s job. Sometimes it can be an entire project from start to finish, while other times it can be as simple as changing the color of a button.

The concept and necessity of estimation is not just found in software development, but in any field where something is being built. From crafting a beautiful piece of art, to road or highway installations that make you sit in traffic and wonder, "Didn't they say this would be done months ago?," it's a struggle that seems universal when it comes to time.

While estimating is inherently uncertain, as the name suggests, it's our responsibility to approach it with honesty and accuracy. Usually, the larger the task, the harder it is to estimate. Until a project or deliverable is complete, we can’t know exactly how long it will take, so we need to do our best. Many people don’t know where to start, but this guide is here to help you make more accurate estimates.

My Story

When I first started estimating, it was a completely new experience. Coming from a background in education as a teacher, I was used to having everything scheduled: teach in this room at this time and make sure to cover all the material before midterms and finals. Outside of grading papers, tracking time wasn’t a major focus, and I haven’t had to build or produce something that required an estimate in advance.

In the beginning, estimating was tough for me. Like many other people in the development profession, I struggled many times with my accuracy, specifically underestimating frequently. This often resulted in two situations I wanted to avoid: Burning myself out to complete what I needed to do or (even worse) burning myself out and still not completing what I said I would.

Both of these scenarios were painful. As someone who prides themselves on being competent and valuing their word, not meeting the expectations I set for myself and my clients was a terrible experience. I struggled to face them after missing deadlines. However, each misstep brought experience and wisdom, and taught me valuable lessons about estimation that I hope to share with you.

Clean Coders Studio Business Model

At Clean Coders Studio, we only get paid when we deliver on the estimates for our tasks.

Here's a great article that explains the tools and methods we use to make estimates in our agile development, particularly Planning Poker. While the article is very detailed, what I want to highlight is the three-point estimate based on Optimistic, Realistic, and Pessimistic estimates. I’ll dive into more detail about them later, but this system allows us to estimate a probability distribution rather than a single number, which is crucial for accuracy in our business.

As explained in this article, when companies charge hourly regardless of completion, there’s less incentive to finish tasks quickly. This can lead to slower progress, with clients paying more and waiting longer for their project to be done. Charging based on task completion is a better approach, it benefits the client by delivering projects faster and motivates the development team to work efficiently.

Some might worry that incentivizing speed could compromise quality. While that’s a valid concern, at Clean Coders Studio, quality always comes first. We ensure that speed never comes at the expense of the final product by not charging for bug fixes. This way, we stay committed to delivering work that’s both efficient and high-quality, without needing additional maintenance later. Accurate estimates help us strike the right balance between getting the job done and maintaining a high standard of work.

That’s why, for us, estimates are vital to good business.

What Goes Into an Estimate?

Healthy estimating requires a balance, and careful consideration of several factors:

Writing Tests

Writing code isn't just about making new features work; it's about making sure they keep working. Writing unit, integration, or any other type of test takes time, but it's essential for maintaining quality. When you're adding a new feature or application that hasn't been tested before, setting up the testing framework can take extra time.

Checking Edge Cases

Writing code with zero bugs is almost impossible, but that doesn't mean it shouldn't be our goal. Bugs can cost some clients their business and, in extreme cases, even their lives. At CCS, fixing bugs doesn’t come with an extra charge for the client, but it does cost the development team time to go back and resolve the issues. This can reduce overall productivity and erode client trust if bugs become frequent. Taking the time to consider edge cases up front can help prevent these issues and lead to better outcomes.

Collaboration

Sometimes you need to meet with your team members, attend meetings, or meet with your client to clarify their wants. Like any good relationship, communication is key, and great communication often takes time and intentional effort.

Completing the Task

It’s easy to focus on this part, since it’s the “meat and potatoes” of what gets done. But a great meal isn’t just about the main dish; there’s also the sauce, the sides, the vegetables, the salad, the dessert, and the drinks. Similarly, completing the task is important, but make sure to account for everything else that goes into delivering a project: testing, collaboration, deployments, and more.

Deploying to Staging and Production

Deployments don’t magically happen, and they aren’t instantaneous. Whether you're migrating data, rolling it back, or even just merging branches, these steps can take longer than expected. There are always potential hiccups along the way, so it’s important to factor them into your estimate.

Client Demos

You also need to demonstrate to your client that the project is working as intended. Time might need to be taken to practice or put together a presentation. There are often many pieces that need to function together for a smooth client-facing experience, so it’s important to prepare thoroughly and ensure you appear confident and organized.

Refactoring

It's easy to let technical debt build up, but refactoring helps keep your code clean and maintainable. The more you refactor, the less debt you accumulate over time.

If you're following Test-Driven Development (TDD), this is already part of the process along with writing tests, but it's still easy to underestimate how long this part of the cycle can take. And if you're not using TDD, it's always a good idea to review your work afterward and see where you can make improvements. You’ll thank yourself later.

Common Estimate Pitfalls to Avoid

Every project is different, and there can be many pitfalls. Don't be like the younger version of myself, or like many other burgeoning developers out there, who usually take one of two routes when estimating:

  1. Presenting the best-case time scenario for completing a task
  2. Taking your best-case estimate and multiplying it by 3 or 4

Scenario 1 is too small of an estimate and is unfair to you and your client. It's unfair to you because you will likely need to work overtime to meet your commitment. It's also unfair to your client because they’ll receive rushed work that, let's be honest, may not be the best version of the software you promised.

Scenario 2 is too large of an estimate, and again is unfair. Padding estimates is dishonest and leads to a situation where you take advantage of your clients. It can also be harmful to you because clients may not stick around if they feel you're overcharging them.

In my experience, most developers lean towards the first situation and underestimate more often than not. This seems to stem from the planning fallacy, where we tend to underestimate the time needed to complete tasks.

Here are six more of the most common estimating pitfalls that you should avoid:

1. Overcommitting to Please the Client

We want our client to be happy, and it can be tempting to overcommit because we want to impress them, especially if they're pressuring us to complete something faster. Stand your ground and be a professional. If it is not possible to complete the task in the time asked by your client, be honest and let them know. Don't sacrifice yourself or your good estimate just to “please" them. Setting realistic expectations builds trust in the long run, while missing deadlines will hurt your credibility.

2. Estimating Quickly

This can happen at the end of a long day or meeting when you're tired, and you just want to move on. Don’t let yourself overlook a task’s potential complexity because you feel you need to get it done and move on. Doing so can be harmful, so it is important to pay attention to each estimate and not rush.

3. Not Asking Enough Questions

Make sure everyone is on the same page. Limit your assumptions; clarify uncertainty so you won't be unpleasantly surprised later. This way you can update the acceptance criteria as needed and give yourself a better understanding of the situation.

4. Forgetting Other Responsibilities

Don't forget you have other work as well. Whether it be meetings, collaborations, or other non-coding tasks you might have, don't forget to acknowledge the time it will take to complete additional duties. Estimate for how much actual coding time you have within your schedule, not for an ideal day with no other tasks.

5. Overlooking Dependencies

Sometimes your product is dependent on someone else completing their work as well. It can be great to split up a workload, but try to limit situations that are out of your control. If you do join forces with someone else, communicate with your team to understand their priorities and to produce a realistic timeline for completion.

6. Underestimating Testing

There are situations where understanding the tests, or even getting them to test properly, takes much longer than initially anticipated. Realize if this deliverable is completely new and hasn’t been done before in your project, you will have to be creative and make a whole new type of test, possibly even complex integration tests as well. All of this takes time. Allow yourself to build up the test base properly.

Steps to Estimating More Accurately

Although there are pitfalls to be aware of when it comes to estimating, there are also strategies to actively pursue that will help to improve your ability and lead to favorable results. Here are six of them:

1. Gain More Experience

This is like when a doctor tells you to drink water and get plenty of sleep. It's so obvious, but still worth mentioning. Gaining experience is crucial. In the beginning, you will be wrong often, but realize that's normal. As your understanding of the testing framework and the source code increases, so will your proficiency. To be a better craftsman, you need to understand your tools better, and like most things, over time we get better. We will still make mistakes, just less often.

2. Separate Your Task Into Manageable Chunks

In the past, I have provided an estimate for a large task, only to later write out the steps necessary for the task's completion and realize that I have a different estimate. Sometimes the new estimate is less than my original estimate, but more often than not it's higher. This is because breaking larger projects down into smaller pieces makes them more realistic and accurate, and minimizes the margin of error in your estimates.

For example, if your goal is to lose weight, simply saying "lose weight" isn't good enough. You need to separate it into steps: sleep better, drink more water, limit unhealthy drinks, exercise more, do things outside that you enjoy, eat more plants, limit processed food, etc. Separating the tasks makes the overall effort more manageable. Smaller tasks are easier to understand, thus leading to greater accuracy.

3. Look at Previous Related Estimates

Sometimes we do work on tasks that are similar to ones we have done before. Look at your previous estimates and reflect on how accurate they were. Were they on the money, or were you way off? From there, adjust accordingly.

4. Three-Point Estimation

I mentioned at the beginning that we would come back to the three-point estimation system: Optimistic, Realistic, and Pessimistic. Each estimate that you give is a thought exercise in itself. Let's examine these three points, adding the question "…how long would it take me to do this?" to the end of each:

This Three-Point Estimation from Planning Poker, helps provide a distribution of possibilities, which can be more accurate than a single number.

5. Collaborate

Any team works best when its members collaborate. While you will often work on tasks by yourself, don't hesitate to reach out to others if you're stuck or facing a challenging problem. Also be willing to help out others when they need it. If someone asks for help, and you have the ability to contribute, DO IT! Many hands make light work.

In my teaching days, I sometimes had the class work in pairs to solve a complex Physics problem. I would stop, ask them what they came up with, and then ask others what they thought. Finally, I demonstrated the process to arrive at the solution and repeated the cycle. This collaboration helped build camaraderie, allowed people who were confused to get help, and reinforced the material, because explaining a concept to someone else helps you understand it better yourself.

6. Spike It

This is for if you're entering foreign territory, you need to write in a new language, or your client wants you to do something you have never done before. How can you accurately give an estimate to something you have never seen or done before? You can't, and that's where the “spike” comes in.

A spike is a short-term and focused effort to gain knowledge, before committing to a particular solution. In this case, it means allocating time to research and experiment with an uncertain area. The end goal is to provide an accurate estimate after gaining a better understanding of the subject.

Conclusion

Estimating is tough, and it's something developers grapple with, whether we're building software, crafting a masterpiece, or overseeing a construction project. However, with the right mindset and tools, we can improve.

Success comes from finding balance: being fair to yourself and your clients, while avoiding overcommitment and inflated estimates. Focus on understanding what truly drives an estimate. Remember to stay away from common pitfalls and apply strategies like breaking tasks into manageable chunks, reviewing past estimates, using the three-point estimation, collaborating with your team, and spiking when faced with the unknown.

As we gain experience and learn from our mistakes, our estimates become more accurate. So, next time you're faced with a new task to estimate, take a deep breath, trust your abilities, and apply these tips. You’ve got this!