Ep. #973 - Why Test? Exploring the Value of QA

Matt DeCoursey 00:01
And we’re back! Back for another episode of Startup Hustle. Matt DeCoursey here to have another conversation on hoping helps your business grow. So many people are building things, but not enough of them are testing. Why do you test things? So you have a better product; you have a better output. And then, for those of us that have started a software company or anything else, oftentimes, to protect our own sanity. Because I sure feel a whole lot better when we launch something, and it actually works. That’s what we’re going to talk about today. So today’s episode of Startup Hustle is sponsored by Double, and that’s a flexible assistant service for busy executives that matches you with an experienced assistant. And you only pay for the hours you need. You tell Double everything you’re looking for in an assistant, and they match you with an experienced, dedicated, remote assistant. Double handles all the upfront time-consuming items such as interviewing, matching, onboarding, invoicing, and training. You can go withDouble, that’s withdouble.com. Today, you get 50% off your first month when you sign up and use the code HUSTLE22. Once again, withdouble.com. There’s a link for that in the show notes, and use the code HUSTLE22. Check my math on that 50%. Yep, that’s exactly half-off. So go check it out. What is here in full and not just half is my guest today. And today’s guest is Jeff Weiner. Jeff is the CEO and co-founder of Real Quantum, it’s a real estate platform. Now, while you’re going for that with double.com’s link, you’ll find one for realquantum.com. Straight out of my hometown and here, live in the studio, a former guest, and back for more surprises. We got you back, Jeff. So, welcome back to Startup Hustle.

Jeff Weiner 01:49
Thanks, Matt. It’s great to be back.

Matt DeCoursey 01:51
So we were reviewing when you were last here. Your episode aired, and it was January 2021. It was a long time ago. I think we’ve probably had 400 or so episodes. So why don’t we just start over and remind everyone of what your backstory is and what you guys do at Real Quantum as well?

Jeff Weiner 02:10
Absolutely. Matt, I love what you guys are doing here at Startup Hustle, with so many great topics. I haven’t been able to listen to all 1000 of them, but you are absolutely on my rotation every week. So my background, I’m the CEO and co-founder at Real Quantum. It’s a software-as-a-service platform for commercial real estate and valuation. And my background is not in the commercial real estate space. It’s been Tech. I did the fortune 50 Telecom thing for a long time, decided to escape the corporate world, and switch to the startup world. So I did. I worked with another company. And we did some really cool core tech and video streaming. And that was sold off to another local Kansas City private equity firm to combine with another IoT company. And then we switched to doing this. And you know, the fun thing about being in a startup and being at a tech startup, it’s just it’s very difficult in 100 different ways. But the things that I really enjoyed about it, the primary thing is that, you know, when I was at my last company, we used to do a lot of custom Dev, and we would see people with ideas that were business-focused. And they thought, all I have to do is hire some coders in Vietnam, and I’ll be a millionaire, right. And everybody who has tech focuses, all I have to do is build this code, and I’ll sell it, and I’ll be a millionaire. While they’re both wrong, usually. And it was only when I was in the situation where we had a pair of co-founders. And they appreciated the contribution that the other one made. That was successful. And what I found with Real Quantum as we were getting this group together, that was the case, and it’s been the case. And so it’s been a wonderful ride.

Matt DeCoursey 04:05
You know, it’s interesting when I talk to founders that have seen an exit, and then they sign up for it again, you know, my co-founder here at Startup Hustle, Matt Watson, and my business partner Full Scale, you know, he did it. And you know, depending on which day of the week you asked, Matt, that’s a good decision or wasn’t. But I think that so many people that are entrepreneurs just know that’s what we do. And there’s only it’s only so long before you can sit still now, we were talking about the quality, and I mentioned something in the intro about the sanity and that being able to test, and just, I don’t know, like, for me, I’m just real hesitant and real protective of putting my name or my company’s name on something that could potentially be crap. And so much of that is As as you know, when we talk about, like, what we’re talking about, why testing? You know, and unfortunately, so many people that reach out and want to use the services of Full Scale, they’re always asking for developers and not asking for testers. No, and I should. And first, for starters, when you have people that support other roles, it will kind of put that role on steroids. I’m a big fan now that I’ve gained more experience. I’m not older. I’ve just become more experienced, right? That’s how it works. But I’ve learned that it’s best to put people in roles, like keep doing things to keep your developers developing, your testers testing your sales, people selling right, you know, and all that stuff. And you know, when you think about that, and what you’ve built in the past, I mean, what are some of the Where do you want to start?

Jeff Weiner 05:52
Well, it’s a good question. Right? So when you’re talking about cars today, nobody asks why crash testing, which we all appreciate.

Matt DeCoursey 06:02
But why remember? I remember driving around and like the back of a shitty Datsun with my mom. And, like, no seatbelt? No, like, you’re in the backseat, you’re gonna write right no crash does dummies were harmed in the creation that there are children, right?

Jeff Weiner 06:10
Probably a lot was actually right. But when it comes to business and consumer applications and software, you know, people do ask that. And so Matt, you and I were talking before we got started. I actually have fairly strong opinions about that, but that has changed significantly. Right? And so the importance of it is you don’t want to produce a product that’s crap. And you want to be able to have the division of labor. So you developers can develop it yourself and sell the product, etc. Absolutely. The arguments that have been used in the past, and I’ve used them too, is one of the ways that you make sure that the development team doesn’t take risks is you take the guardrails away. If there are no testers, and I’ve talked to my last company, we didn’t have any QA. And I’ve talked to other founders and things that don’t use QA, the developers test extensively, and then the product picks up the slack. And there is merit to the idea that if the developers don’t have guardrails, they’re going to be more careful. But, of course, the complexity of your product only gets to a certain point before that’s just not practical. And like you said before, right, if they’re going to be focused on maximizing their productivity as an engineer, spending another sprint testing the stuff they wrote, The prior sprint is not as efficient. And they’re not professional testers. So if you’re gonna get anywhere on an app that is of any kind of material complexity, certainly user experience complexity, you gotta have somebody besides the developer. But yeah, you’ve probably run into, as you said, you’ve run into a lot of people that ask you for developers who don’t seem to have nearly as much interest.

Matt DeCoursey 08:09
Well, the most common, the most common thing that comes up, and I feel like it’s asked from a position of inexperience is so this is actually kind of a red flag question, say, Well, if the developers doing good, should they be testing their own stuff, and it should be fine, right? That’s not really the way perspective works, you know, and the thing is, you can get so close to something. And I always remember when I was an entrepreneur, and I had just gotten a job after college, and my dad, I didn’t get the promotion that I wanted. And I felt like the whole world had just closed, and my dad said, right now you’ve signed, you feel like a door slammed in your face, and you’re so close to the door, you can’t? How do you know you’re not in a whole entire hallway of open doors? Take a minute and look around and see what’s going on. And it was real and, you know, that’s not an uncommon bit of advice to give someone that didn’t get what they wanted. But perspective really is there. And if you’ve been building something, especially if you’re really in the weeds even doing it, it’s just I think it’s really easy to not consider what could be different or what could be wrong. Right?

Jeff Weiner 09:22
And you’re absolute, yeah, so that’s a good point. So you know, and lots of people, so there are people that need to hire developers because they’re not engineering savvy software engineers. I’ll ask questions like, I get questions. Why do we need marketing? Why do we need the tester? What’s a project manager to do, right? So I don’t mind those things? Because, you know, if everything was all figured out already, they wouldn’t be like us. But um, but, you know, there’s a lot of other people too that have had just as bad experience with QA screen. So like I said, at the top, my background is big Fortune company experience. And unfortunately, I’ve had too much experience with a big fortune 100 company that has a big corporate IT operation that’s building something large and maintaining it internally. And so they contract with another big fortune 100 company for what seems like this giant slew of mediocre testers that they are renting by the ton. Right. And so they have this bad experience, right? It’s x well, with the theory that if you throw enough spaghetti at the wall, eventually you’ll cover it, or whatever, whatever analogy when he. And I kind of, I’ll admit, I had that sort of mindset as I was running software engineering teams, and as a software engineer, then I realized there’s absolutely no way I’m certainly not going to expect my engineers to know the end-to-end system. My division alone was 500 people, 1000s of people billion dollar budget. Nobody in my team is going to do it end to end, and you had to do that. So do your best to find good testing. But you can’t let that say, throw the baby out with the bathwater. But I was in that situation like, is it always really this dysfunctional? This is bad, right? And so I had to leave. And then be in an environment where there was really good testing going on. And it’s amazing. So now I’m so fortunate, right, as I’ll give a plug to Full Scale, right? So, row quantum is a small startup operation. We’re growing fast. And we’ve sourced some QA talent from your company, and we’ve hired in on our own. And there’s, it works well. And one of the ways that I know it works well is the QA members are an extension of our product team and our extension of the customer success team. Right? So our product team gets questions, and often, it’s me directly, right. So how do you want it to work this way or that way? And when everybody’s working in sync, it’s wonderful. The developer will say, Hey, I think what the requirements said was Xi did this. Does that look right? The tester? Yes, wonderful. Great collaboration or a production issue will come through. And the first thing that happens is if we’re worried it’s a bug in our QA lead or if one of the QA people validates it? Requirements, they have great data. No developer has been interrupted from the sprint. Right, as they’ve been charging along. They validated that, yes, it’s an issue because they’ve been in the code. And then, the developer has a pinpoint place. Real data, specific examples. Not a, it’s making a noise. I’m not sure what the problem is with the customer. And so I’m. I’ve absolutely come full circle since I started.

Matt DeCoursey 13:09
And one of the things that I’ve learned from building software for over 10 years now is that any software platform has a very small number of core features or functions, and if any, one of them is broken, or not working, you have no business doing anything else until they’re fixed. Oh, yeah, you’re looking at a Giga book, we’ll just use that as an example. If it’s an appointment booking platform, if you can’t make an appointment, it’s broken. Right? It doesn’t work. And like, Yeah, and so you got a Real Quantum. And once again, if you want to learn more about what Jeff does in this business, there’s a link in the show notes, but Real Quantum.com. So you guys do like real estate valuation if it was put in, so you put an extra zero on something. Right? So that would be a big problem. Because by the way, that turns 100,000 into a million.

Jeff Weiner 14:02
You know, important, right, so I made my little joke about crash test dummies. People don’t have to ask why. That’s great. I mean, my first job out of college was in a billing system for a big telecom. And somebody puts COBOL, right, so this is a little indication about how long ago they put a period in the wrong place when they were doing bug fix? Six minutes cost $1 million. Boom, just like that. So the impacts are a big deal. And you’re right, for Real Quantum. Our calculation engine is probably twice or three times the complexity of Quicken or TurboTax. There’s just a crazy amount of stuff that’s going on. And so that calc engine has to be right. We replace our competitor essentially with Excel. Right, which is the case with a lot of financial press. products out there, our competitors Excel. Well, if the user messed up their formula in Excel, well, they can be mad at themselves. And, and but, you know, if we did, then they’re thinking, well, I should have just stayed with Excel, that kind of thing.

Matt DeCoursey 15:19
So we don’t want to do that’s all it takes to lose a user. It can be all it takes to us to lose users.

Jeff Weiner 15:20
So we test the heck out of our product. And we spend an excessive amount of time, just really being careful about the calc engine. We’re a product, we’re a source of record app rain, which means it’s an app that it’s not like Spotify that can go on in the background or off to the side. They’re using our product, they’re using our product every day. So absolutely, it has to be right. We know little things like you said all kinds of different things have to work together in our test environment this morning, a couple hours before a client had to do a demo and it all zero failed. It was just a token that expired. You read it, and that meant, if you had logged out, you’re not logged back in. So none of the awesomeness that was inside of that test environment was available. So thankfully, in that case, our customer success and our QA team, and the person responsible, the one person responsible the development team worked together, the developer probably had a few minutes of work, and it was back up and running.

Matt DeCoursey 16:26
Yeah. And now that said, developers like that kind of support if they get the right support. And there’s evolution, one of the things that you mentioned. You know, prior to setting a record, I hadn’t really thought about what I I know is that good. QA does kind of evolve into that product, when you get the people that. So if you want to be a tester it takes to do it successfully and be really good at it. It takes a certain personality type. That is not that it is not mine. I mean, you’re talking about people that you know, and testing comes in a couple of different, you know, forms, and if we want to keep it in just two categories, manual or automated, right? You can’t, you can’t and talking about manuals so far. Yeah, and you can’t automate anything unless you first know how to do it manually. Because that’s the progression of that you have to know what you want to test and how you want to test it. And, you know, but with that, you have to have someone that can sit there and do it over and over and over again. And then you mentioned like, like the cheap rental, like the lower quality output or service, these are people that are gonna like this, this button works, this button does not work, right? When you get people that are good at this process and really understand that they’re going to take it to a different level and begin to understand, Okay, why do we why are we doing three or four steps here? Because I’ve done this 10 million times. And for me Rwanda building software, or any feature of it is answering this question. Is this annoying? And you mentioned with Real Quantum, you said, if our users are in there, they’re using it all day, every day, right? So the same way, if something is broken, if something’s annoying, well, we mentioned Excel, remember the paperclip guy like that? Was he on Microsoft Word? Libby? Yeah, he’s like, literally one of the most hated icons of all time, because he was annoying, right? So if anything about it. So you know, QA is also going to help. If you get the right people, they’re going to think critically about the product and begin to come up with ways to even make it better. And, and you look at that repetition. I think that and I’m bringing this up, because this is something you want to look for. This is a healthy sign and a team where your testers are going to work with the product. Now, part of one of the things that we do is get a book that some people might not consider as well. So we would, you know, get the book to have a lot of moving parts. It’s kind of like Real Quantum, there’s like all these little gears and tie ends and connections. And you know, if any one of them breaks, it can kind of mess up a lot of different stuff. So we use intercom, which is like live chat for support. Right. And we were finding that most of the time. So our QA is actually also our live support. Because we actually had some people that did live support. And then we found that 99% of the time they were just sending over to QA. Yeah, so we’re like, you know, we because no one’s going to, no one knows how to work all of it on the whole platform better than the QA. So we kind of actually kind of removed a step out of the process. We opened up some resources, because no one really wanted to do the live support anyway, but 99% of the time, so we used to send it to the developer and the developer was 90% of the time. So this is a user error. Right? And it was and so yeah, so it really evolved into a much much cleaner process and became really efficient so we’re able to identify problems and get them to A developer kinda like you mentioned earlier that bang, bang, repetition just because all your users care about is does it work? It’s a very binary thing for them. And that’s a yes or no, whether that’s a token that doesn’t let you sign in, or a period being in the wrong place, or, or a button clicking and not giving you the result, the all the user thinks about as a zero or one works yes or no.

Jeff Weiner 20:19
And, yeah, because they don’t want to shoot, nor should they need to understand what’s happening behind it. So it also helps, right, there’s a couple of different kinds of testing depending on things. So in the case of Real Quantum, hundreds of screens, 1000s of calculations that occur in the background, 1000s and hundreds of database attributes, but it’s a very screen heavy user experience, there’s a fair amount of inter API interactions on the back end. And so a lot of the testing we think about is user interface testing, just because even though we have API’s and and database, and we want to scale testing, etc, that’s, of course, like you were saying the manual intensive. And the next thing we’ll be doing is what we’re looking forward to, and I’m going to be calling you about this right away, as soon as we’re ready to get some more automated going. One, automated user experience testing. So other apps that are really heavy API work, you can’t, you don’t really spend that much time on the front end, you really spend a ton of time on the back end interactions, it’s in a really different skill set. So our recipe seems to work pretty well, at Real Quantum, and we’re on a journey, we’ve been growing really fast, our teams are a lot bigger than they used to be just a few months ago. So we’re dealing with some of those growing pains. But overall, it’s working is that, you know, the developers do test, they need to test they need to in a continuous integration, automated testing environment, they want to have automated test coverage with as much of their code as possible. And we’re on that journey. So that. And then, of course, we have other things that we do, right, so we do security penetration testing, we have a vendor tool. We have API testing, load and performance testing, and need to get done. And we have a ton of tools, really all of that’s by and large run by the development team. At a certain point, we’re gonna get big enough that we need to have a test automation engineer, and or DevOps. But because of the way that our product has been built, we’re actually a Ruby on Rails stack, which is, you and I’ve talked before, it’s hard to find good engineer should say that it’s hard to find engineers, enough engineers, and that stack just because it’s a little, it’s not as popular. But um, but it served us really well. And so the developers tend to do most of that. And then it really comes down to the QA team, to really understand the connection to the customer’s requirements and what the screens are doing. And it’s, it’s, it’s at a point now, where our QA lead often does the demos of new features at the end of the sprint.

Matt DeCoursey 23:18
Like you were saying, because they get it, they understand more about it because testing is just repetition.

Jeff Weiner 23:26
Sure. And they have, they have all the data at their fingertips because they’ve been testing. Some of it’s not always the prettiest, because it’s designed to check the extremes, but they have all the data and they can do a really good job. And some of the best testers are either have in the past also that I’ve encountered have either been really good trainers, for users. Or in one model that I was with for a while the test the business analyst be at the beginning of the sprint would be the tester to test it. Rotation, the analysts would always test. In our case, back to that scale comment you made at the beginning. Testing team has a lot of test cases to run. So they support care and product, but they’re really just focused.

Matt DeCoursey 24:18
Yeah, and I do want to thank the testers of the world for doing such a better job at that than I would on a daily basis. I want to talk a little bit about the sanity side of why testing is a good thing as a software vendor. But as a reminder, today’s episode Startup Hustle is brought to you by Double the experts and pairing founders with remote executive assistants that you can trust Startup Hustle listeners can go to with Double.com That’s wi the Double.com use the code HUSTLE22 and unlock 50% off your first month or you can even just click the link in the show notes while you’re down there. Go to Real Quantum to learn a little bit more about what they do as a reminder that With me today, I’ve got Jeff Weiner, who is the CEO and co-founder of Real Quantum, go to RealQuantum.com doing fun and interesting things with numbers. What do you do? I do find interesting stuff with numbers. Now. I probably should have led with this. But you know, the definition of quality assurance, which is QA, it’s, it’s the definition suggests a quality assurance refers to the attitudes and processes needed to ensure ready for market software products that they perform as they’ve been envisioned at the design stage. And QA enables the delivery of the highest possible quality and software products. Okay, that’s one definition. But for me, it delivers peace of mind. And peace of mind doesn’t actually have infinite value, because I think that what I learned is that without it, not much else has much flavor. And so you talk about the sanity piece of it. Okay, so if you’re this is what it’s like, if you’re alright, so if the history of Gigabook, we were able to know when it was broken, because our phone would never ring. And then all of a sudden, it was just like, Oh, my God, it was like, Where, right? We know something was broken, because users aren’t sometimes testers, always which, which here’s the thing is, is okay, so on the surface level, it’s validating, because you’re like, wow, we’ve built something really useful. Because look out PIs they are when it doesn’t do what they need to do. And then you’re like, Oh, my God, I gotta hurry up and get this, right. So look, the best way to not have that situation come up is to catch it before you make it right. And the technical terms before you move it to your production environment. Now, I just think about the sanity side of things, both as a business owner, as a founder, as a member of a team, I just feel that like the sanity side, look, what I just described as fun for nobody. It’s not fun for your users, it’s not fun for you, it’s definitely not fun for your team. A lot of times if stuff like that occurs, you may even find that you are waking people up. Hey, fire Hurry up. And you know, so many people have distributed teams, there’s people working all around the world. And that can be a real challenge. And, you know, so for when it comes to your overall peace of mind and organizational sanity, a clean release is the way to do it. And and just but that can be tough to do you talking about things like well, why don’t you automate it, okay, so I want to paint a little picture for you here, if you’re going to set up an automated test, you have to maintain that test as well, which means every little change or anything you make is going to break the test. So the idea of that is to get it to get yourself to a point of stability, where most of the meat and potatoes of the testing is happening and an automated and repetitive way that can launch your updates and everything a lot faster. But once again, you have to understand how to do it on a basic level as advice I want to give once again, the tip that you identify the things and this isn’t like you should know like you know 10 Things at Real Quantum that if they are messed up you have no business doing anything else until they’re fix start with that and like whether you have a QA tester or someone your product or whether it’s you now back to that peace of mind, Nothing drives me crazier than than a team saying, hey, look, this is ready to go and you’re like, Okay, I’ll take a look at it. And the very like, in under one minute you find three things wrong with it, right? Not good for peace of mind. And everything. So you know, that’s, that’s where I think so much of it, it matters. And then, you know, the next component of that is in this is just like, the simple math of business is if a QA tester hypothetically makes less than a developer do you should you pay someone $2 an hour to do what someone else does for $1 An hour or $20 an hour compared to 10 or however whatever that math is, that comes out and then I think back to the fact that for people that have chosen testing seem to like it. Yeah, it’s important it’s like it’s like it’s like it’s almost like sadomasochistic in the way that they are, you know, because I struggle with it. It’s like that I have ADD I need to do different things and different times and different stuff but a good QA really like likes to find the air

Jeff Weiner 29:39
I appreciate you know, Matt, you are coming in for this QA interview, but I don’t think it’s gonna work out.

Matt DeCoursey 29:46
Dude, you know what? I bet on the product side, I actually consider myself very high level on the product side. But that’s by the time you’re talking about that when you’re doing product planning or any of that you’re working under the assumption that it actually works.

Jeff Weiner 30:04
Right. So you know, so Matt, you brought up, you know, it’s a good point, having all the different types of testing worked out based on the kind of application software you have. And that at the same time, we love the idea of delegation, so people can specialize. And also you have the right cost model. But it doesn’t replace collaboration, right? So you still need somebody to go back and step away from the test cases, and the test data and the code and everything else and just look at it. And whether that’s somebody in product, or in your case, Founder, CEO, I love to do that, too. In fact, I’m more heavily involved in the product process than I think I will be next year. So I’m really enjoying it now. Until we get too big, but you have to look at it end to end, because nobody at the micro level will be able to look at it the way that you can, looking at it with fresh eyes. You know, and lots of industries do that. You know, I remember, and I apologize for anybody who’s familiar with the rules of journalism, because I got this from a TV show, but watching it where they have a certain group of individuals in, in the news team that aren’t aware of a story that’s being written. And they’re specifically there, they do that, so that when that story is about ready, they give it to those individuals to read to see if they miss something.

Matt DeCoursey 31:46
You know, and that’s another really good technique, you got to have fresh eyes and something. So it was like Red Team Blue Team.

Jeff Weiner 31:49
And that’s, I think that’s what they call the two. Yeah, so the red team and blue team.

Matt DeCoursey 31:51
All right, let’s say you have 10 people in a room. And when you demand a red, a blue team, red team environment, at least one person has to vehemently take the opposite position of the other team. Yeah. So it’s not necessarily a testing thing. But it could be like, Hey, I think that that, you know, this is the route that we need to take. And then the red team person is forced to, I mean, an ideal kind of Devil’s actively like you can, you’re almost unsayable, that you’re the member of the jury that cannot be convinced immediately to take the route of a battle to your NSF. Yeah, and so but that’s a fresh set of eyes and a lot of ways when you force it. And so that’s that, that diversity of opinion and input. And you know, so much of that. So there’s a I don’t know, if you saw there was a really funny video that was going around Facebook for a while, and it was labeled software developers during QA testing. And it had a lady and she had like, there’s like a toy for kids. And you know, like, you have like, the shape will go through that star shape will go through? Well, a lot of those shapes would go through the square spot, if you just turn them a little differently. So you know, the look on his face was like, Oh, my God, I can’t believe that, that works. That too. And so Oh, my God, and so did that. And that’s back to that perspective. And I think that just contributes to a better product overall. Now, one thing you can come up with, though, as it so the one thing we haven’t talked about here at all, is human nature. So have you ever worked with someone who’s just terrible at taking feedback? Like no matter what is uncoachable? Yeah. And so that can be a problem, though. Because you with QA? Like, I mean, all right. So I’m here to Jeff. This is broken. What do you mean, it’s broken? And this isn’t broken? It works. I’m literally using it. Right. What? But it works for what I built it for. Yeah. But you built it to do that. And it needs to do three things, not just one, which, according to this means if all three aren’t checkboxes, it’s broken. Right? Well, if you man and you know, so you can get some heat and stuff in there. And then, you know, well, on the flip side of that, too, is I’ve had a couple of Q A’s in the past that say, come back, and they’re like, Okay, this, okay, first off, for anyone out there, it’s broken is the worst feedback you can give any. Like, don’t ever come and tell me it’s broken.

Jeff Weiner 34:32
But I have to. I won’t use any names, but I have. It was a really clever slash dastardly technique that a business client would have on that. And whether it was the code, the requirements, whatever, then if it wasn’t working like they wanted it to was a quote, broken experience. Yeah. And I’m thinking, what the heck do I do with this?

Matt DeCoursey 34:58
That’s actually that is a really tricky way to put it because I can make a strong argument so that if perception, broken experience, but what am I supposed to do? Well, then it is like, I don’t know how you I don’t. I’m not even sure the best attorneys could argue out of the opposition because they say not that much different than it makes me feel sad. Well, if someone’s perceived, if they perceive it to be crap, then at least in their world, it is now right now where I was going with this as well as sometimes when it comes to testing, I think you got to, you got to avoid this and look for it as I’ve had some testers will come back and they’re like, Okay, this is this error is occurring. And I’m like, How did you even arrive at this? I mean, well, on the 19th time that I did this, and so you got to also look at some of this stuff and say, Okay, we tried a little too hard to break it. Well, like what’s the use case where someone would actually be editing their appointment and get the book for the 19th? Straight time? And yeah, if that’s the case, I’m gonna go ahead and just let that fish swim away from them. So that’s really alright, so that’s a good thing. Well, you can spend a lot of time chasing the fixing that stuff that really sucks on your resources.

Jeff Weiner 36:10
I love the fact that it’s really good. QA testers and analysts will be able to really dig into all the permutations and find stuff because of course, human nature is that we follow the happy path. Yeah, we don’t really think about all the things that could go wrong.

Matt DeCoursey 36:33
Let’s define that real quick, happy path is the normal user experience path where everything works as it should, meaning all the things that I do that I expect to happen, or you get affirmative answers, right, I’m coming to the Startup Hustle studio.

Jeff Weiner 36:38
And I didn’t get stopped by the road construction on state Avenue, I didn’t get stopped by every single stoplight. So we don’t think about somebody who tries to put bogus data in a date and puts alpha characters in $1, you know, those kinds of things. And so making sure that it’s tested on the thresholds is really good. And I just had one of those that came up yesterday. And obviously, that’s something that we should have caught. So. But the point, if I’m tracking with you, Matt, is that there’s a threshold of risk management that you have to take. You want the tester, the testing organization to be very thorough, and that’s the personality type you want. But there’ll be a point, right? So you as a business owner, or if you’re, if you have functional responsibility for division or something like that, you have to make that. So testing creates peace of mind, to a point some people are afraid to let it go, well, we might need to delay the release another week or another week, I’m not sure that you have to be able to step out and take that risk and your best through good testing, and other things to be able to mitigate that. So you do feel confident going in, but you can’t test forever. We’ll get caught in that. So one of the things probably is very popular as the gating versus non gating issue is, gating is, is it bad enough that we can’t put this in production? Because it certainly was the case with the big corporate projects, you know, with Real Quantum, we’ll run into a handful of little things, cosmetic stuff that it’s not worth to stop the train. But in a really complicated corporate release, there could be hundreds of things, and there needs to be a governance process to decide what’s eating what’s not what needs to what, who’s going to stand in front of the train and tell to stop versus not. And some of that is back to making an educated risk calculation and being bold about it.

Matt DeCoursey 39:09
And then putting in some mitigation plans to and just being careful what rat what hole you’re gonna crawl down. So those holes get deep and like when I’m talking about like, I mean, there’s just some rare combinations of use cases or like so, you know, they can start with just trying to figure out how people use your stuff. And you know, like I said, protecting that 99% Like for example if you can’t log in, okay, now it is broken, right? But don’t say it’s just broken. What’s meant is broken doesn’t work that No, and that’s their given feedback and finding solutions is a whole nother thing. Once again, today’s episode Startup Hustle was sponsored by our friends over at Double Doubles remote executive assistants can help you with everything from email and calendar organization to expense reporting and database management. Find your perfect assistant today. Head over to withdouble.com. And use the code HUSTLE22 and unlock 50% off your first month. Even easier, it’s just you know, if you can’t remember with Double.com, just scroll on down and use that link in the show notes. While you’re there, make sure you hit that subscribe button. So you make sure to help us QA that our episodes come out every day at various times. Because as we were talking before you recorded your last time in people listened to 78 countries, we’ve had listeners and over 190 I quit counting 190. I’ve been told there are 202 countries total. But there’s no real QA on that either. Because people disagree about what some of that is. And yeah, so that’s been a very interesting journey. Thanks for coming back.

Jeff Weiner 40:51
It’s thank you to Matt. Again, love what you guys are dealing with and love talking about this. in no small part to the fact that I’m super proud of professionalism.

Matt DeCoursey 41:05
My company here. Oh, yeah, I couldn’t ask for better and you guys run a great operation. You know, there’s a couple other people we might want to give a shout out to because I think the last time I saw you in person was at the pipeline entrepreneurs. Gala. Yeah, that’s right. Yeah, we’ve been we’ve worked with them and Melissa, and she’s been a guest host. She has she’s done a bunch of episodes. And she’s done real well without and you talk about things and organizations that we’re proud of. That’s a real shining star and our entrepreneur community now if you happen to be located in or around the Midwest, so you don’t even need to be from Kansas City. You got a pipeline, pipeline entrepreneurs. Is that the.com? I believe so. Yeah. Look at our QA on that. Right. Yeah. Right. So with that, that’s, that’s something that, you know, you talk about the quality of what you build. And I think as an entrepreneur, the very first thing you can do is always improve your own product, which is you right? And yeah, that’s, that’s been a very, I hadn’t gone to that Gala. Before, which by the way, I’m the guy that shows up with jeans and a sport coat with a Startup Hustle t shirt to the black-tie Gala. I had gold shoes on. I did and they look good. So you know, we’ve talked about a whole lot of different sets. Did you get that? Did you get that?

Jeff Weiner 42:21
Yeah, it’s a pipeline. Entrepreneurs.com?

Matt DeCoursey 42:23
I thought it was.

Jeff Weiner 42:24
Yeah, I mean, can you add that to the description sounds? But um, yeah, you’re right. You’re right. So one of the first things you need to do is as an entrepreneur, make sure you’re making your best self so that you can be a good leader, it’s a great program for that.

Matt DeCoursey 42:37
I think that’s something you know, I’m not. I’m not at all shy about, you know, 47 laps around the sun. For me, these shows are a regular education for me and it’s at this point that I almost graduated, I dropped out of five colleges. I’m not a pedigree guy. But with that I supplemented I think this is I’m estimating this is about the 700 and 50th episode that I’ve done myself or been involved with, which is always amazing. The things that I learned, I love looking at other people’s perspectives. I think, as we review what we talked about today, perspective is a key part of building great stuff. And I want to encourage everyone listening to exercise that tip, you know, like, you’re gonna let different people use it. I’ve tried a bunch of different rules. I like to create rules, because I find that if I make the rules, then they’re easier to follow. And I’m way more successful with him. But I had at one point, I had what I call the five and 75 rule, which was so it had to do with simplicity, and, and just being intuitive. So you had to whether you were five or 75, you needed and sorry, Dad, you were the 75 part of it, you didn’t like but if a five year old or a 75 year old, equally needed to be able to use the product and not and not be confused by it or just completely blown away. Now that isn’t a practical thing for maybe what you do at Real Quantum because I’m not sure a five year old would really know what I know what my son and daughter had put in. It would be like nein, nein, nein, nein, nein, nein, nein, nein, nein, nein, nein, nein, nein, sicher.

Jeff Weiner 44:21
Yeah, well, the point you make is important. Like get Real Quantum. The customer base is pretty diverse in age. So commercial real estate appraisal, it’s a consulting gig. And people can do it like attorneys or financial advisors, accountants they can, they can taper off as they retire. Average age 63 in the United States, right. So people work long into what in other industries maybe would be retirement years. And it’s a wonderful, vibrant community. But that means we do in our case, we don’t need five year olds, but we need 20 year olds and literally 85 year olds, we have 85 year old users on our application and everybody in between. So it requires a really good user experience that is super and intuitive, and that it’s been through QA.

Matt DeCoursey 45:08
Really, if you’re 85. Then you remembered, like, when the record player came out.

Jeff Weiner 45:13
When it came out, I mean, close is not that far off.

Matt DeCoursey 45:15
I mean, it’s not that far out. I mean, you look at like the just recording technology and a lot of that, you’re like an A, it’s, I mean, hey, I, my wife is a little younger than me, and I was talking to her about I said, you know, when I was a kid, I had a BGS 45 record player, and she was she’s like, how old are like she never even had that she never had cassette tape. So it was like straight into CDs almost, which is now, and we just actually threw away like hundreds of CDs or recycled them, and it felt weird because, you know, we spent so much time collecting and putting all that together and only to realize that we had no need to keep any of it because if you can see all stream it now, they are digital.

Jeff Weiner 46:01
And you can save them, so you know that’s the way it goes, but five and 75.

Matt DeCoursey 46:05
And then, you know, once you have any, out of all the stuff we talked about today, as we wrap up this episode, once again with me today, Jeff Weiner, CEO and co-founder of Real Quantum. Go to RealQuantum.com. There, if you listen to the show regularly, I want to encourage you to always click the links that are associated with our guests. Because I just always find it fun and interesting to see what people do as well as asking them, you know, what stood out in our conversation today? I mean, what were a couple of the key ingredients for you?

Jeff Weiner 46:40
Fight hard, work hard for good software testing talent. And find a way to make sure that it’s a great collaborative experience with all the other members of the team. Customer success product engineering, in whatever mix makes sense for your product. But those, I think, are the two keys to success.

Matt DeCoursey 47:02
Yeah, I think having, in general, and whether it’s with a dedicated QA person or not. I think the thing I want to leave everyone with is, you know, read a few things down. I think we didn’t mention that but read a few things down and make it known. Because if you don’t make it known, what’s a pat? What passes and what isn’t? I mean, it’s the same thing with goals and KPIs. A lot of stuff is transparent.

Jeff Weiner 47:23
Get what you don’t measure. Yep.

Matt DeCoursey 47:25
Well, Jeff, thanks for joining me, man. I guess I’ll hopefully not have to wait for another 500 episodes to get you back.

Jeff Weiner 47:31
Yeah, let’s do it sooner than that. I think it sounds good.