Vendor management is a popular topic with our clients.  We wanted to spend some time showing ways that Quantitative Software Management can help improve the process. We help our clients on both sides of the aisle – the proposal side as well as the vendor evaluation side. A couple of the biggest challenges we see before we start working with people is a need to improve communication between the vendor and the client. And we often see a lack of data being used in the overall vendor management process. Today I'll be showing some ways that data-driven estimation and project control techniques can help.

First, I want to start by giving a brief background on QSM, Quantitative Software Management, in case you're joining us for the first time. QSM has been in the software estimation and management business for over 46 years. We are very well known for being the developers of the SLIM-Suite of software tools.

SLIM is an acronym for Software Lifecycle Management.  It is adapted to all all types of IT and engineering development types of projects - software as well as digital transformation, and cloud migration. In addition to being a leading product company, we're also a research provider. You're going to see some examples of that during the webinar today. QSM’s main focus is helping our clients plan and negotiate both in-house and vendor project and portfolio delivery. Our unique contribution is the QSM industry database. It's one of the largest in the world of its kind, with over 14,000 completed projects over 46 years of research. It supports vendor management because it provides the most recent information on cost, schedule, scope and quality for completed programs, which identify unrealistic delivery expectations so important early in the procurement process.

It is important to begin with an explanation of our software production equation.  This is the equation in its most simplest form – Size = Effort * Time * Productivity. The Software Production Equation was developed by Larry Putnam, Sr., who founded QSM and is very well known in our industry as a pioneer. In the software estimation industry, we were implementing machine learning types of techniques even before they became popular, so Larry was way ahead of his time. What the scatters plot do is link the delivered system size or functionality to the time and effort that it takes to deliver that work. And in addition to that, we're linking the level of productivity, or development efficiency hat it takes to achieve that delivery. We refer to that measure as a Productivity Index, calculated from the historical data. So, once we know the functionality delivered and the time and the effort that it took to deliver the work, we can calculate what that vendors productivity index is, and then we can use that to both manage and measure the vendor.

This analysis is used during the evaluation bid process as well as during in-flight project delivery. The equation can be rearranged to solve multiple estimation problems. For example, if we only have very little minimal information going into the decision making process, we can leverage the production equation to help us solve multiple solutions. The Productivity Index is a macro measure for productivity and efficiency. It takes into account things like experience level, the people, the maturity of the process that the vendor is working in, the complexity of the work that they're delivering.

The historical data that we recommend you capture as part of the vendor management process is just four core metrics:

• Software size or scope, or value delivered
• Duration of the program
• Efforts expended

This scatter plot is an example showing a blue data point that represents a of a vendor bid. Duration is on the vertical axis and Size in on the horizontal axis. This particular trend line is our most recent agile project trend line. You can see how the vendor bid compares with the industry average to make sure it's competitive to help you can make the right decision. The triangle data points represent the historical data from the vendor, if you are able to collect this data as part of the contract. Now you are able to compare the vendor bid to the trend line as well as the historical data from the vendor. And we can do all this even before any detailed planning takes place, long before you figure out how many hours each person's going to work and roll that up in a spreadsheet. I'm going to show you examples that use the SLIM tools early to get the project targets right to make sure the vendor bids are realistic, and we're also going to be able to compare vendor.

This view shows the Impossible Zone. This same scatter plot of time versus functionality delivered, also available for effort, staffing, and defects, a common relationship. There is a gradual slope in the data – the measures all increase with size. The impossible zone, shown in big bold red letters, defines projects that are overly optimistic. A vendor proposal that positions in this area is very likely to fail.  This is the type of analysis we want to use as part of our vendor management process when we're evaluating multiple vendor bids. We want to make sure that you know one vendor may come in and say, it's going to cost one million dollars and take six months. Another vendor may say two million, and 8 to 9 months. QSM tools help you decide which vendor is more competitive and help you make the right decision.

You do not want to use this analysis as a as a baseball bat. You want to use it to ensure everybody is on the same page - proposal side as well as the as well as the client side. Everything will be better for both sides.

Vendor management challenges that we're going to talk about today:

1. Managing Complex Programs with Multiple Vendors. How many times have you been in a situation where you have multiple projects going on, multiple vendor projects slip and they affect the start date of another project? The more programs that you have and the more complex they are, the more you have to do a better job of capturing some data and doing a better job of estimating and planning. A data-driven approach is going to go a long way to help us with that. Trying to take some of the complexity out of the process, trying to manage that a little bit more effectively is one of the things we're going to focus on today. There are a lot of moving parts to deal with, especially when we're talking about digital transformation programs and cloud migrations and similar initiatives.
2.  Communication and Collaboration Across the Enterprise. Not only do we want to help improve the vendor communication with the client, we also want to improve the communication across the enterprise, across the stakeholders, getting everyone on the same page early. A lot of times when we're managing programs, people think differently. They think in terms of the size, the, the functionality that they deliver. They even define that differently. We want to draw some bridges between vendor and client, so everyone is talking the same language.
3. Inadequate Planning and Estimation Methods. Of course, that hits at the heart of what we're going to talk about today. Many times we work with folks that are doing a very, very detailed level planning, determining many hours each person is going to work and roll that up? It is difficult so, they try to guess. Is this going to be a six month schedule or is this going to be a 12 month schedule? What kind of budget are we actually working with? Do we have the capacity to fulfill the demand?
4. Negotiating a Fair Price and Reliable Schedule. Both sides go in with the best intentions, but a lot of times, vendors tend to be overly optimistic.  They want to win your business and they're trying to be competitive. Sometimes these projects don’t work out right. Maybe a price is quoted too low as scheduled, too aggressive, and then we find ourselves throwing more money at the problem. And then things get worse! We want to be able to negotiate a fair price for both sides early in the process.

These are the benefits of using this data-driven estimation approach. The number one benefit is the ability to see the big picture - see the forest before the trees. Get out of the woods early. See what the targets should be before we commit to anything. Often, we're working with vendors we haven't worked with before, or maybe we don't have experience in a certain type of software development or delivery, so being able to see the big picture is a big benefit. Second is a technology knowledge base to help with the early cost and schedule decisions. That's that industry database that I mentioned. QSM has the large industry database that we can pull typical cost and schedule numbers from to help us make good decisions early in the vendor management process. Third, it improves negotiation and communication capabilities not only for people on the project itself but the stakeholders at higher levels.  Usually senior management is involved early in the procurement process but they're not quite as involved during project execution. We need to have a way to report to everybody what's going such that when decisions have to be made in the middle of the project, we can make adjustments, and everybody can feel comfortable going forward because we want that vendor relationship to last. We want to be able to work with that vendor again and feel confident the next time as well as on the current program. And the fourth benefit is these types of tools and processes help manage uncertainty. There is a lot of risk early in, for example, a digital transformation program that has many moving parts. We want to be able to take that uncertainty into account when we're making the decisions on the plans as well as once the project gets started. SLIM uses Monte Carlo Simulation to compute risk-buffered solutions with a high probability of delivery.

Now, I will show you some real-life examples using the SLIM-Suite of tools. The majority of our customers use SLIM.  We also provide services where our consultants work in the vendor management arena. SLIM is a suite of five software applications. SLIM-Estimate is our flagship product. It's the one you're going to use to perform the bid evaluations, when we make those early decisions about the cost, schedule, risk and trade-offs. You can use industry benchmarks to make sure that an estimate is credible. The real power is the ability to look at multiple scenarios. SLIM-Estimate is used for that early estimate. SLIM-Data manager is the history repository tool. This will provide a vehicle for you to capture a little bit of historical data on your vendor. For example, when you're measuring product size and the other core metrics while the project is underway, you will have the cost, effort, and schedule and calculate the Productivity Index. Now can use the actuals to evaluate the vendor going forward. SLIM-MasterPlan is the tool used to estimate at the portfolio levels. SLIM-MasterPlan lets you estimate those complex programs that I mentioned, where there are a lot of things going on at one time, a lot of moving parts. It produces resource demand plans to help determine is you have the capacity to fulfill the demand and models project dependencies. SLIM-Control performs in-flight project analysis based on plans versus actuals. It's like a GPS for your project delivery. It provides a project map so you can see where you are relative to where you want to be. It will forecast estimates to complete based on any changes you may make so you know when to project will complete. SLIM-Metrics is our statistical analysis application that uses completed project data in SLIM-DataManager to perform benchmark analysis. You want to see how a single vendor or group of vendors performed. Did they follow through on what they told you and can you use them the next time? SLIM-Collaborate as our software as a service. It's our estimation tool for the web.

Before I continue with actual examples, I want to pause for questions. Yes. We have one question, Keith. Where did the trend lines on these charts come from? Great question. The trend lines come from the QSM database. All that data that comes from our client relationships by permission. We update the database every couple of two to three years based on how much information we have coming in. We update the tools with that new information in the form of the updated statistical averages and trend lines. You could also create your own custom trend lines and bring those into the estimation tool as well. But those trend lines actually are from the industry database, so that we can sanity check the cost and schedule commitments to see if they're reasonable.

Another question is can our tools support cloud migration? Yes, absolutely. We support any type of cloud migration, digital transformation, any type of IT or engineering development process.

This is the SLIM-Estimate application. This is where we compare multiple vendor bids and sanity check based on industry data. Now if I go to the tools menu, you can see that. I could access all. The tools in the suite that I mentioned. What you see here, the way this architecture is set up, if I look to my left, I've got a navigator that's got a wealth of analytics that describe a vendor management bid so that we can communicate effectively with the stakeholders. You can scroll down and see all the views that describe an estimate that I've already run. And first thing I'll probably do is I'd like to do is I'd probably load our historical data in. If I'm able to capture some data from the vendor I'm working with as part of the contract, this will help me make those decisions a little bit more reliable. When I hit OK it brings the historical data into the estimate tool. You can see a trend line like the slide I showed in the PowerPoint presentation. You can see the industry average for this type of program and this is for effort. I think the slide I showed was schedule or duration. The triangles that you see are the historical projects that you've completed in the past or projects that your vendor has completed in the past. That's even better. And the trend line is the industry average. The red data points that you see are actual vendor bids. There's two of them. The one higher up on the scatter plot for effort on the vertical axis and size on the horizontal is Implementation Units. That's a sizing measure that we're using in this example, but this could say story points. You can us features, user stories, epics - it's flexible.  And this is basically giving me two vendor bids that I'm comparing on the on the cost and you can see one of the vendor bids is much more in line with our industry database. And it's also much more in line with our historical data, so the one up top is less risky. The lower one that you see is a little bit outside the standard deviation. That's another vendor right, vendor A and vendor B. Vendor B is a little bit more risky. They're bidding a lower price, probably a shorter schedule. That's kind of a little snapshot example from 2 vendor bids that we're comparing on an estimate that I've already run.

Let’s see more data about one of these estimates.  You can see here on the staffing profile, this is how the vendor is going to ramp up on the program. This is a hybrid example – part waterfall and agile. I'm going to show you some agile examples as well. This shows the vendors going to ramp up to about eight people. The summary of the of the bid shows they're bidding. 5.2 months with 5000 person hours of effort. And a half $1,000,000 in costs, that's what they're quoting us with about eight people. That's the riskier estimate. I can go up to this solution log with other estimates that I have loaded and you can see the more risky bid as well. So now I'm highlighting the bid that. Was not quite as comfortable. You can see the shorter schedule and the lower cost right compared to the more reasonable vid, which is the seven months at $1.4 million. So, this vendor buffered in additional time and effort into their bid, and even though the cheaper vendor obviously comes in lower, I might be more inclined to go with the more reasonable, higher bid because it's more in line with our historical data. I'm less likely to lose money in the long run.

How did I arrive at this estimate? Ther are multiple approaches based on the information you have available. I'm going to show what we call the Bid Evaluation example. First, let me take one step back before I start using SLIM. Typically, I would open a template that represents the type of program I'm working on so you can see I've opened a calling a bid evaluation template. It brings in all the information I need to make the right decision on the vendor management bid process. If I'm doing a cloud migration I would open up a template configured for a cloud migration program. In this example, I've opened the bid evaluation template and I'm going to show you how I generated this estimate. I selected the Bid Evaluation Wizard, which steps us through the process and prompts us enter what we know. We know the project scope because the vendor told us what they're going to deliver or we've told the vendor what we want delivered. That's the scope. And what we also know is the time and budget available, because that's what the vendors telling us, they're telling us that it's going to take a certain amount of time and they're going to price it out at a certain budget for us. So that's what we know going into this example. What SLIM-Estimate will calculate is that Productivity Index I mentioned -  it is going to calculate that productivity for the vendor and we're going to be able to validate whether that scope is reasonable and whether that time and budget is competitive and reasonable.

We need to enter name of the project, select the Phases that we want to include in the estimate, and input the start date. You can see here I'm I'm working on an e-commerce system. Then I'm going to choose a trend group. Remember I mentioned that QSM industry database, right, thousands of projects. Trend lines are generated from the QSM database. these are all different types of programs that we support, from engineering to real time to business to hybrid or agile. When we click on the application domain that we're working in, SLIM brings in the typical effort and schedule numbers for that particular type of program. I’m going to be able to use the QSM database to make decisions even if I don't have any vendor history. Now I can enter the scope, either a fixed amount like I stories or features, or if I'm not sure I can use a t-shirt sizing or a rough order of magnitude approach. We can indicate if is this big or small based on what we normally do. And here you can see you can see the dial move the count of implementation units. That's the sizing. We're using the industry database to come up with a size estimate for us. A typical size estimate for the type of project that we're asking the vendor to deliver. Now we’re ready to put in what the vendor has quoting. They're quoting 7.3 months and whatever budget they've decided to quote me or team size or whatever information I have available from the vendor bid, I'm going to go ahead and put it in here. And that's it really. 4 steps. Right. Name of the project application type, size and then the bid effort and schedule.

SLIM has generated the estimate and loaded the data into the system and displays it on the scatter plot. It is 7 months $1.4 million. SLIM provides the ability to look at different trade-offs.  Maybe I want to negotiate now with the vendor. Perhaps seven months is not good - I need it done sooner. The vendor may say, don't worry we can add people. Or maybe we reduce the scope. SLIM lets us look at different trade-offs and the impacts of adding people or reducing functionality, or what if we can work at a higher productivity, things like that? Looking at trade-offs allows us to do a deeper dive on the risk assessment. For example, I may see that nine months has a 99 percent chance of completing on time. But if the vendor is promising something in the six to seven month range only about a 10% shot. Same with cost. I can look at the 99 percent $3,000,000 versus the 10 months with a 20% chance of success down. I can see if I'm getting low balled on a vendor bid. Evaluating the trade-offs helps me make decisions.

SLIM breaks the estimate out into more detail as well. In addition to figuring out the cost and schedule, it will also break the estimate out at the skill level. If I need that level of granularity and all these views toggle to report. The report can be exported to Excel. I can also enter more detail on the sizing. I may  tell the vendor that I need a certain number of screens, reports, and interfaces. In this sizing example, I want 81 screens delivered. Then you see in the center column we have what we call Implementation Units per component. These are called gearing factors that are used to normalization the data to get on the same page as the vendor to make sure that the vendor and the customer are talking the same language as far as sizing. Usually sizing is one of the biggest challenge in vendor management. How many times have you been in a situation where the vendor is talking a different language than you are on the deliverable? QSM has thousands of projects, so we know what the typical gearing factors are for different size measures. We know what the typical number of implementation units is per component or per screen. We know what the typical number of story points per feature is. We know what the typical number of lines of code per function point is. Just because we have all this industry data. This helps us come to an agreement on whatever that deliverable is going to be. You might be thinking, what's an implementation unit? It represents the smallest unit of work, roughly equivalent to a source line of code. You can see that the interfaces are much more complex than the reports and screens. If we have a level of detail like this, we can leverage it.

Let’s look at an agile example, an agile scrum. You can see to my left I've got the navigator that's got all the analytics that I need to evaluate the vendor bids and I might have multiple vendor bids you can see here that we're sanity checking again. This scatter plot displays size versus duration. This blue data point is my bid for schedule or duration and you can see I can feel comfortable with this because it's consistent with the industry average for agile projects in the QSM database. We have the triangles that would be your historical data. I can see that my bid is consistent with the historical data before I make a commitment and choose which vendor I would like. Take a closer look at this view. This vendor is telling us we can deliver this in 10 sprints. 4.7 months at a $1 million cost. I've got a little risk snapshot here - the risk gauge telling me that if wanted the vendor to get done in four months that I've got less than a 5% chance of that happening. Much more likely they're going to be delivering in about 4.7 months. OK. I can negotiate now, right? They're telling me four months, $1 million. Alright, well, what if I want more functionality? You know what's going to happen. How much will the cost go up? And how much longer will it take? These are some of the complexity factors that SLIM helps manage so I can quickly assess the impact. You know what's it going to cost to add in that extra backlog. And how much longer will it take? And when I hit OK, it updates the estimate now within seconds to that bigger scope. We can also negotiate things like the team size - how many resources are on the program. You can quickly find out what it will to cost to add in that extra Scrum team. There's the bigger team. You can see, it's still going to take over 13 sprints, about 6-6 months. And the cost is going to go way up when I add the extra scrum team in. This is another example of a way that we can negotiate. We can look at the data, look a the trade-offs pretty quickly and evaluate the risk.

SLIM helps us manage uncertainty by calculating a risk-buffered solution with appropriate amounts of contingency.  This first estimate that I ran 4.7 months is my 50% solution right out-of-the-box. If I get more data later, I can refine the estimate. But usually we're making very early decisions and we don't have a lot of information. We don't have enough information to do a detailed bottom up plan to know how many hours is each person is working. We're only trying to get a scope to get the targets right. So early on we can come in do a contingency plan to compute the schedule and the budget that gives me a 90% chance of meeting those project targets. The slim model helps us figure out what's reasonable and what's not. Here's a more conservative estimate with a little more time and effort that lets us feel more comfortable going into the into the program. Ther are lots of ways to approach the decision making process, lots of analytics that we can use and we can help you with all this. QSM supports is included along with the training.  We will help you set this up.

Once we've chosen the vendor, we want to conduct in flight analysis while the project is under way. We've chosen the vendor. Now we want to make sure that everything goes well. This our SLIM-Control tool. The architecture is similar to SLIM-Estimate. We've got the navigator to our left that's got all the analytics that help us manage the vendor process. You can see the dashboard. Here we can see the Gantt chart. The blue bar is our plan, the bid initially from the vendor. The red is where the vendor is right now in time. We are tracking features, defects, staffing, and cost. Let's look at features for a second. This thin blue line that you see here was the vendor bid for the delivery of feature. That was the estimate initially and you can see that this vendor is getting in trouble. The red data points are the actuals that we're capturing from the vendor. And you can see the actuals are falling off the plan, and so the yellow lights indicate caution - something's going wrong here. It's not right. We are taking the actuals once a month, and just the totals, not very labor intensive at all, just the total people each month or every two weeks or whatever your cycle is. Total features delivered. Total defects found if we want to track that. And we recommend that you do this. It's not only good enough to get done on time. We want the system to work. We want to be able to manage the defects as a part of the contract. You can collect data on any custom metrics you want to track, like design components. You're seeing the plan versus the actuals for each metric. Defects are skyrocket a lot higher than the plan in this example. The red traffic light is on because they're finding a lot more defects than they had planned. You can see, you know, the vendor should have discovered about 136 defects, but they're finding 220. That's a problem. The good thing is we're seeing it early in the process and we can react.

Now, once we're in into the program, we can run another estimate because we have more data. We're going to run a forecast right from here and what this tool is going to do is run a curve fit through the actuals and run a new projection for us.  It's going to use those machine learning techniques, the historical data, the trend lines. The model forecasts where this vendor is going to be based on all the changes that have occurred throughout the release. And now we can see the new estimate. The blue line is the initial plan from the vendor. That's what we agreed to. Red is where the vendor is and green is the new estimate. And it's validating what we just saw, this vendor is going to be late. You can see on the feature chart they're going to deliver out in May or June. They were supposed to finish in December. Staffing and costs look OK, but it's not going to do us any good if the delivery is not there. Now we'll look to the bottom right. You can see we had planned on 508 features delivered. The vendor has only delivered 394. We had planned on spending $2,000,000. We've already overspent with this vendor. We're already up to about 2.2 million. The program's not even over. What SLIM-Control is doing is it's taking the actual work delivered and the actual effort spent and the actual duration that has elapsed and it's recalculating this Productivity Index.  So even if your vendor doesn't know what a PI is, you're going to know when they give you the time and the effort and the scope. You're going to know what their PI is in the plan and now we can see what level they're working to. They're only achieving a PI of 14.5. We can use this information the next time we work with the vendor. If their bid is at 16, we can remember that last time they only achieved a 14.5, and we can take all this into account and capture it for the next time we work with the vendor and we can also use it to negotiate right now.

Let's look at a couple trade-offs. What are we going to do about the vendor delivering late. One of the things that we can try is adding people - that seems to be the fall back for a lot of programs. The plan staffing is 27 people. The vendor says, hey, don't worry, you know, we'll get it done. We're going to add some extra people. Within a couple of seconds we can see the impact of that. This tradeoff forecast creates a new estimate right away. If the team size is now 37 people, we save about 3 weeks and the cost skyrockets almost $7 million. Probably not a great idea, but at least we know now. When I hit OK you can see graphically now the impact of that. You can see that the vendor is still going to be late. Still going to deliver out in the March, April timeframe and staffing costs go way up. This view right here might save you a ton of money and a ton of heartburn because you're seeing it early.

Perhaps more people is not a good idea. Another trade off might be reducing the scope. Can we live without some features till the next release? We can reduce the scope in another curve fit to run a new estimate within seconds there it is new estimate with reduced scope. In this scenario we're almost on time and people and cost are under control. This is part of the negotiating, right. It's part of the communication. Everybody's on the same page. All these reports are made available to the team vendor and client side stakeholders.

This is at the release level. Remember I mentioned a lot of complexity on some of these programs. Well, we can also deal with that at the portfolio level using SLIM-MasterPlan. It helps us manage all of the vendor programs together in the big picture. This example is a digital transformation example. You can see I've got multiple vendor bids in blue. When I highlight you can see the proposed cost and schedule and now we can deal with situations and explore what if? What if one vendor is late. How does that affect the start time of their next program, or another vendor’s program. We can see if we have the capacity to fulfill the demand. With the resources we have, can we let this slip two or three more months and how will that impact the overall portfolio cost? All these charts toggle to report and you can see the total cost of the portfolio, all the bids, all the vendor bids together – the total cost of 34 million dollars, with the elapsed time almost 14 months. So you can look at the big picture, look at the tradeoffs, see how the slippage of one release affects the cost and effort of another and helps us figure out the decisions are we going to be making to adjust to any changes.

Once the programs are over, we can capture final actual performance data. Here's an example. This is a software as a service program. This particular program or project delivery was 722 stories, took seven months, and spent 6700 person months of effort. Once we get those 3 measures, remember that software production equation, the Putnam model, we can calculate the Productivity Index, or PI. The PI for this particular vendor was a 22.4. Now we're going to have this data the next time we work with the vendor or the next time, even if it's a different vendor, we're going to see what's typical so that we can use it the next time to help manage more effectively going forward. And that concludes this example portion of the webinar. Happy to open it up for any questions.

Question: What are the types of data measures do you need in the historical database?

Answer: We'd like to shoot for is just those core metrics. In fact, Larry Putnam Senior wrote a book called 5 Core Metrics. We need the scope, duration and effort. And then the productivity index is calculated. SLIM-DataManager is very robust. In fact, we use it for our industry database. We could you can go into as much detail in granular as you would like on the completed project or delivery metrics, but.

Question: If you're working with the vendor for the first time, you wouldn't have any history, right? What do you do?

Answer: We recommend is try to make that part of the contract if you can. For a vendor to win your business, they have to provide at least a few of their projects so we can see evidence of how they've done in the past. If you're not able to get that, then what? We recommend leveraging the industry trend lines in the QSM SLIM tools. Remember we showed those examples where you can sanity check - you'll be able to see the typical industry numbers. So if you can't get it from your vendor, then use the industry PI's that are built into the tools so that you can make some good decisions and also take into account the uncertainty that you are using the industry and include that in your estimate. You know include that uncertainty so that when you generate the estimate, you know look at the 90% chance versus the 50% chance to help you negotiate and feel more comfortable.

Question: You did mention the trend lines. What are the application types that we support?

Answer: we support any type of software or IT development process. Now the trend lines that you saw include real time, engineering, and business applications. We have agile, financial, and government. Really any type of software development or IT development application domain. And also cloud migration trends as well. Process control is another one, plus package implementation.

Question: Does QSM offer some consulting in these areas?

Yes. A lot of our clients use the tools with our support and some clients like to hire us to do all the all the leg work. We can do independent estimates for you. We be the independent source to evaluate the bid for you. We can also do the in-flight analysis, so once you choose the vendor we can help you do the oversight work. We can run the tools for you as well.

Thank you, everybody, and thank you, Laura. Appreciate it. We will be making the recording available.