Today more than ever we have access to large amounts of information. You've probably heard the term "big data," which in essence is having access to large amounts of data and examining the trends in that data. But many executives want to know how they can leverage this information to solve business problems, like lowering IT costs. One way is to use the data to do a better job of estimating IT projects.

Better estimating helps avoid signing up to schedules and budgets that are unrealistic; it helps avoid overstaffing a project or a portfolio of projects; and it helps calculate how much work can be completed within project constraints. In addition, it improves communication internally across the enterprise and externally between the vendor and the client. You can apply estimation to in-house projects and you can use it to generate better proposals or to do a better job of evaluating proposals.  It can also help you negotiate more effectively.

To do a better job of estimating, you need to make good decisions regarding which metrics to leverage. You might have thousands of data points, but it's important to streamline the focus to the core release level metrics: cost, duration, effort, reliability, and productivity.  Next, you need to find a centralized place to organize and store the data so you can analyze it. There are tools out there that can help you. In the view below, you can see a portfolio of projects stored in a centralized place with the ability to manage the access and security.

In any development methodology, we throw around the word “estimate” freely not really understanding how it’s interpreted by many.  In many cases, an estimate, regardless of its content and process by which it was created, is received implicitly as a pin point number with the accuracy of multiple decimal points.  This presents a problem for all parties involved.

I recently had a discussion with a gentleman who told me that prior to using our SLIM tool, the estimates in his organization were arrived at by casual hallway conversations, often started with, “how much do you think this will cost, and how long will it take?”  A typical response is, “hmmm, I’d say about 6 months and $500K.”  That innocent musing then becomes the information upon which business decisions are based, leadership bonuses may be won or lost, and the credibility of the dev team is on the line.

I’d strongly recommend adhering to some definitions when talking estimates.  These definitions will help mitigate potential misunderstandings around the agreement of what makes an estimate:

In a world trending away from traditional waterfall and toward agile development methodologies, it would be understandable to assume that there is no longer a need for software project estimation. Many agile practitioners feel there’s no value in estimation, since they are already working with smaller increments and sprints and grooming their backlogs.

However, that assumption would be wrong.

In a recent interview, Ken Schwaber and Jeff Sutherland, the founders of Scrum, were asked about the #NoEstimate movement. Schwaber believes a more appropriate term may be #NoMeaningfulCommitments. He feels that people often confuse estimation with commitments and that, in fact, estimates should be used in making commitments. Sutherland mentioned a recent Rally (now CA) survey that asked members of 70,000 scrum teams about the estimation techniques they used and then correlated those techniques with speed of delivery. They found that those that eschewed estimates altogether yielded some of the slowest delivery times, while those that employed scope-based estimation delivered the fastest results.

Larry Putnam, Jr.'s latest article for InfoQ explains why estimation is still a very valuable practice, even in organizations that are dependent upon agile development methodologies. He outlines several best practices that stakeholders can use to get their software estimation processes back on track toward adding value to their organizations. Software estimation does not have to be difficult, onerous, or ineffective. Done right, it can be a highly effective tool that can help project managers provide value to their organizations.

The challenges surrounding software estimation are both well known and well documented.  Most discussions on this topic center around the technical challenges estimators face: tight schedules, unclear scope, evolving requirements, and accounting for dependencies and risk.  But there's a more fundamental challenge we don't hear so much about – educating stakeholders and making the business case for structured, yet practical estimation, and why it is a critical success factor. 

Let's face it: process improvement is rarely cost-free.  Businesses expect a visible return on investments made in estimation tools and training.  The benefits of quality software estimation can be compelling, but moving decision makers from "open to the idea of estimation" to "willing to commit money and resources" can be difficult for busy analysts and managers juggling multiple roles and tasks.

In her latest video, QSM's Laura Zuber explains what makes a good software estimate and how empirically-based estimation helps projects:

This post was originally published on Linkedin. Join the QSM Linkedin Group and Company Page to stay up-to-date with more content like this.

QSM is pleased to announce the release of SLIM-Collaborate 3.0, the web-based, software-as-a-service version of our trusted software estimation, tracking and benchmarking suite. With more advanced workflow capabilities, the updated version of SLIM-Collaborate enables more efficient communication between stakeholders throughout the estimation process. Additionally, the demand resource capabilities added in SLIM-Collaborate 3.0 make it easier for users to identify staffing needs and allocate resources to a software project.

Highlights of the new capabilities in SLIM-Collaborate 3.0 include:

Larry Putnam, Jr. was recently quoted in 'Framework and Standards Are the "Essence" of Agile at Scale,' an article published in SD Times. The article consulted other industry experts such as Ivar Jacobson, Matt Schenck, Dean Leffingwell, and Ken France on best practices for implementing agile at scale. Larry's advice below.

Agile estimation is the key to a successful SAFe implementation
With all of the benefits of SAFe, getting it right is key. Using agile estimation can help.

“For organizations that are implementing SAFe, they’re really trying to coordinate a lot of different stakeholders within the organization and the real benefit they’re looking to get out of it is a much more nimble delivery,” said Larry Putnam Jr., co-CEO of QSM. “To be able to do that, we’ve got all these different stakeholders that we have to coordinate. That becomes really complicated and estimation is kind of the communication vehicle for these different stakeholders.”

Putnam explains that the highest level of an organization is the business and its stakeholders. The stakeholders or senior managers within an organization need to ask in what direction the business needs to go and how software will support that. These needs are usually articulated at a high level, said Putnam.

“Those are going to get apportioned across different value streams, and they’re really looking at the whole portfolio of what’s going on within the enterprise,” Putnam said. “In order to implement all those capabilities the business wants, you’ve got all these different cross-functional teams that are working on different pieces of the system.”

At QSM, we understand the importance of looking forward, but we also strongly believe in sharing the insights and advice we’ve gained from our past experiences. That’s why we gathered our best and brightest team members to provide their top five pieces of advice for successful software development in 2018.

1. Use verified project data as a baseline.

Too many organizations think of “estimation” as really just bottom-up planning—dividing a software project into its component tasks, and then trying to pair each task with plausible time and resource numbers. They’re left with “point estimates,” or single values, that don’t account for the inherent uncertainty surrounding a project’s size, scope and productivity. Point estimates can lead to inaccuracies that can ultimately cause projects to run over time and budget.

Instead, organizations should base their estimates on real, verified project data. These “top down” estimates should be completed long before prohibitive schedules, budgets, and task lists get cemented into place. Organizations that use this approach are able to account for any changes in scope or requirements early on and adjust their quantitative estimates as needed, leading to better and more accurate forecasting.

- Larry Putnam Jr., Co-CEO, QSM

2. Fill in the information gaps in your software estimation process.

In my daily work life of supporting my clients, I’ve seen the IT project estimating function reside in many different areas of organizations.   Some groups have this function situated in a core group, such as an Estimating Center of Excellence, PMO or Delivery Excellence group.  These teams are responsible for either creating estimates for the organization after being fed some goals/assumptions, or vetting existing estimates born elsewhere to ensure they are in line with the business’ goals without compromising, as best they can, budget, schedule or quality.

Conversely, I’ve worked with organizations in which the project estimation function is supported by organizationally dispersed estimators who are busy with their other tasks.  Decentralized estimators typically serve a sole siloed division with not much interaction outside their team.  Like the bigger shops, these folks are funneled project estimates to be sanity checked and render their analysis with recommendations for adjusting the estimate if it seems out of line. 

Both scenarios above share a common thread – the estimates, regardless of their origin, have an impact on, and are influenced by, a wide breadth of people.  We at QSM call these people stakeholders, contributors, influencers etc.   They all have some kind of investment in the estimate that is important to them.  What if we could involve them in the estimating process?  With more communal agreement on inputs, there won’t be as much rework of the estimate later, when otherwise uninvolved people see their interests aren’t represented.  With everyone feeling they have a say in the estimate, smoother seas are at least more possible.