If you were unable to attend our recent webinar, "How to Estimate Reliability for On-Time Software Development," a replay is now available.

Software development is a major investment area for thousands of organizations worldwide. The negotiation and early planning meetings often revolve around major cost and schedule decisions. But one of the most important factors, reliability, often gets left behind in these early discussions. This is unfortunate since early reliability estimates can help ensure that a quality product is delivered and predict if it will finish on-time and within budget. In this webinar, Keith Ciocco shows how to leverage the QSM model-based tools to estimate and track the important reliability numbers along with cost, scope, and schedule.

This presentation includes a lively Q&A session with the audience and covers such topics as:

During a recent consulting engagement, a customer asked if the QSM defect discovery model applied to Agile projects.  Of course, the best (and only) way to determine this was empirically.  From our database we extracted a sample of business IT projects that had completed since 2013 that recorded pre-implementation defects.  81 of these projects were Agile and 354 did not specify Agile as their development methodology.  We created average trend lines for both datasets and they displayed very similar patterns that conformed to the QSM defect discovery model.  This allowed us to answer our customer’s question affirmatively.

Having a large project sample at hand and being curious, we decided to compare these metrics:

• Mean time to defect (which measures the average time a system runs defect-free in the first month after implementation)
• Average development time in months
• Staffing
• Cost/effort

In a nutshell, the Agile and non-Agile projects used very similar staff sizes.  The Agile projects completed sooner and expended slightly less effort.  Quality was where the two project sets differed significantly.  Pre-implementation, Agile projects recorded fewer defects than non-Agile ones.  However, post-implementation the non-Agile projects operated longer between discovering defects in production than did Agile projects.

Usually when I am online making a payment or using social media, I am not thinking about software quality. But lately I feel like I have been encountering more bugs than usual.  From activities like clicking on a link where I should be able to input my payment information, to doing a search and receiving an error message, or being redirected to a completely different page which had nothing to do with the mission I had set out to accomplish.  These bugs are sometimes frustrating and I started to wonder what could have been done to prevent these from being released into production.

Since I spend a lot of time speaking with people that manage software projects, I have noticed that quality is often one of the most overlooked aspects of a software system. People I’ve spoken with have mentioned that quality is often not even discussed during the early planning stages of development projects, but it is usually a deciding factor when the software is ready to be released and should be considered from the beginning of the project.

Using a tool like SLIM early in the planning stages of a project can help us with these issues. Not only can it provide reliable cost and schedule estimates, but it can also help estimate how many defects one can expect to find between system test and actual delivery. It can also estimate the Mean Time to Defect (MTTD), which is the amount of time between errors discovered.

With agile projects, we hear a lot about the planning benefits of having a fixed number of people with a fixed number of sprints.  All great stuff when it comes to finishing on time and within budget. But one of the things we also need to focus on is the quality of the software.  We often hear stories about functionality getting put on hold because of reliability goals not being met.

There are some agile estimation models available to help with this, and they can provide this information at the release level, before the project starts or during those early sprints. They provide this information by leveraging historical data along with time-tested forecasting models that are built to support agile projects. 

In the first view, you can see the estimate for the number of defects remaining. This is a big picture view of the overall release. Product managers and anyone concerned with client satisfaction can use these models to predict when the software will be reliable enough for delivery to the customer.

In the second view, you can see the total MTTD (Mean Time to Defect) and the MTTD by severity level. The MTTD is the amount of time that elapses between discovered defects. Each chart shows the months progressing on the horizontal axis and the MTTD (in days) improve over time on the vertical axis. 

This post was originally published on Linkedin. Join the QSM Linkedin Group and Company Page to stay up-to-date with more content like this.

If you were thinking about purchasing a driverless car, and the salesperson told you that there’s a “slight” chance that the car will fail during transit, would you still feel comfortable laying down your money? Or, if you faced an emergency, would you trust an automated robot to perform open-heart surgery, rather than the hands of a skilled physician?

While these questions might seem like the stuff of a science fiction novel, they’re quickly becoming a part of our normal, everyday world. We’re hearing a great deal about artificial intelligence and how it is replacing tasks that were once done by humans. AI is powered by software, and that software is becoming increasingly vital to our lives. This makes ensuring its reliability more important than ever.

But here’s a sobering thought: right now, IT operations teams are building software that is, on average, 95% reliable out the door. That’s right; today, a 5% unreliability gap is considered “good enough.”

QSM's C. Taylor Putnam-Majarian and Doug Putnam recently published an article, Understanding Quality and Reliability, on InfoQ.

One of the most overlooked but important areas of software estimation, measurement, and assessment, is quality. It often is not considered or even discussed during the early planning stages of all development projects, but it’s almost always the ultimate criteria for when a product is ready to ship or deploy. Therefore, it needs to be part of the expectation-setting conversation from the outset of the project. So, how can we talk about product quality? It can be measured a number of ways, but two in particular give excellent insights into the stability of the product.

Read the full article on InfoQ!

On large software development and acquisition programs, testing phases typically extend over many months. It is important to forecast the quality of the software at that future time when the schedule calls for testing to be complete. In this article, originally published in CrossTalk, Paul Below shows how Walter Shewhart’s Control Charts can be applied to this purpose, in order to detect a signal that indicates a significant change in the state of the software. This signal detection is then used to improve mapping of project progress to forecast curves and thereby improve estimates of project schedule.

With the most recent spurt of inclement weather, there is really no denying that winter is here.  After awaking to about 4 inches of snow accumulation, I begrudgingly bundled myself up in my warmest winter gear and proceeded to dig out my car.  Perhaps the brisk air woke me up faster than usual because as I dug a path to the car, I began to think about software testing, specifically how effective early testing can reduce the chances of schedule slippages and cost overruns.  Allow me to explain.