VidCruiter – Why Structured Digital Interviews Win Over AI with Sean Fahey

William 0:34
Ladies and Gentlemen, this is William Tincup and you’re listening to the RecruitingDaily podcast. Today we have on Sean from VidCruiter. And we’re gonna be doing a really, really great topic. It’s a topic I wanted to tackle for a while. And it’s why structured digital interviews win over AI. And so Sean and I have done a podcast before, so we’re fairly conversant with one another. But Sean, do us, do the audience a favor and introduce both yourself and VidCruiter.

Sean 1:04
Perfect. Thanks, William. Sure. So yeah, Sean Fahey here, I’m the CEO and founder of VidCruiter. We started in 2009, and we sell video interview software and most other screening tools as recruitment software all around the world.

William 1:21
And, and VidCruiter, as I know, just give us a little bit more depth on that. Where do y’all play? Size of markets that you care about you synchronous, asynchronous, you know all the, all the industry lingo that you and I’ve talked about before.

Sean 1:38
Good point. So yeah, so we play as the perfect partner for an applicant tracking system, we try to do everything that they don’t do, which could be skill testing, asynchronous video, interviewing or on-demand, or pre-recorded auto-scheduling, video conferencing, which would be the synchronous video, the automated reference checking, and then even some digital in-person tools for structured interviews, being in person and scheduling those and then back into the applicant tracking system after so we try to complement any applicant tracking system as their perfect partner to do all the components that they don’t offer today.

William 2:15
And the folks that the ATS’s that you – you’re already probably deeply embedded with, who are they?

Sean 2:22
Now there’s about a dozen or maybe two partners would be ADP, Workday. We have partnerships with Salesforce, SAP, smart recruiters, iCIMS, Greenhouse, and so on, so on the list goes on.

William 2:39
Yeah, and it’s never is never done, right? But you want to keep adding based on your customers.

Sean 2:45
Exactly.

William 2:46
I love that. Okay, so let’s start off with kind of your definition of structured digital interviews. When you say that to your customers, what do you mean by that?

Sean 2:58
It’s a good question. So for us, a structured interview has a series of meanings to it, we’ve actually expanded on the meaning of what maybe it was traditionally viewed as, generally speaking, a structured interview is where every candidate asked the same question, it’d be sort of like the beginners answer to that question. Then, a more intermediate person who understands this concept and is used it would say not only do we ask everyone the same question, but they’re also being evaluated using the same rating guide, or associated scoring matrix.

And then a more expert person would have that skill scoring matrix have a definition of or outline of what are good responses to look for, in that specific answer. And then we take it a step further now and say, not only are we going to structure, the questions, the evaluations, make sure that whoever is watching the reading the evaluations knows how to evaluate, well, let’s even structure the candidates answers so that they’re structured and identical, which means everyone has the same amount of time to think about their answer, and the same amount of time to answer that question, with the same difficulty level as every other applicant. Not too many other formats other than this asynchronous video interviewing allows for a complete structure where everything is identical for every candidate.

William 4:25
So before we get to the stuff, because the way that you’re structuring questions, the evaluations and the candidate answers and things like that. How does that impact you know, just off-topic, but how does it impact D&I?

Sean 4:41
I think it improves, and the research shows that there is less bias that can occur when a structured interview is used. And that’s research from multiple different studies, we tend to reference a personal psychology book from Julia Levin Cina, but the predictive validity is increased using structured interviews, and the opportunity to ensure that everyone has a fair chance because as I mentioned, everything is identical.

William 5:11
Right? Okay. So now that we have kind of a baseline definition of structured digital interviews, when we say it wins over AI, why do you think we own? So that might be we want to what is the definition of AI as we currently know it? Or are we currently see it in the marketplace? But why do you? Why do you think that structured interviews win over AI?

Sean 5:37
Oh, there’s a couple caveats, that’s structured interviews, when over unstructured interviews, that’s the first thing you have to consider, right. And it’s at least triple the predictive validity of using a structured interview versus not. According to the book that I was referencing before. So those statistics are 22% unstructured. And then 65% predictive validity, when it is structured, the AI component in what we’re talking about, we actually looked in research in 2014. And because all of our competitors are talking about it was a pretty hot topic at the time. And it has been for the last five, six years.

We hired a social scientist, this is a person that studies speech patterns. And this gentleman actually developed software that was being used by the FBI, the Department of DIA, the CIA, all within the US federal government to predict and analyze bomb threats and predict and analyze threats of violence against the US government. And he did this by analyzing what very little information they had, and all they had was the speech or the text of that specific person who is making this specific threat. And we sat down with him and said, okay, we want you to predict what people are saying, with your speech specific to if they should be hired or not based on, you know, a predetermined outcome of what we could say is a good answer based on, you know, the team and, or other factors that we could model off of he, at the beginning of our conversation said, this is going to fail.

And I said, Well, how is it going to fail? How is that possible? He said, Well, first of all, speech to text software is not 100%. And so right off the bat, you’re not going to get the right information. And even today, when you talk to Siri or your phone, it doesn’t always pick up what you’re saying. So you miss one word, that’s key, you could make an unconscious decision. But he said, even if we had our speech to text engines at 100% accuracy, he said, in that just to give you one example of one culture for one sentence, in the Japanese culture, they tend to refer to themselves as we, instead of I, when they speak English, because they are a team, and they have more of a culture of working together as groups.

And he said, In English, when you don’t talk in terms of I, and you use we instead, it’s sometimes seen as a lack of confidence. And so he could predict that the AI engine would be biased against the Japanese culture, who speak English, because they would use the word we, instead of the word I. And he said, that’s just one specific language example that I can pinpoint to you out of hundreds and thousands of others that exists that we don’t even know about, that there’s no way that you’re going to be able to tell the artificial intelligence bot, well, if they’re Japanese, don’t consider this word as biased, as if they’re not confident, and then a lack of confidence doesn’t mean they can’t do the job.

Whereas when if that person was sitting in front of you as a human, and they did talk like that, you could probe and say, oh, when you mean we, was that part of you, or was it just a company? Or you could probe on that question? Well, you can’t do that. So this was just one example. And in that moment, when he started explaining these kinds of things, I knew that that technology was not good enough for us to invest in AI. And we decided to go double down on the structured interview process within our product, and built hundreds of features around a structured interview methodology.

William 9:15
Yeah and it’s deeper, as you already mentioned, it’s deeper than just structured interviews and structured everything.

Sean 9:21
You got it.

William 9:22
So So tell me a little bit about the feedback that you’ve received from your customers as this approach. If you’ve taken into market structured everything, versus AI, have they asked you more about AI? Or are they just care more about the results of what goes through with structuring everything?

Sean 9:43
Over the years more and more people would ask us about AI and we had to keep fighting and trying to explain that it sounded really cool, but it was never going to work. And a lot of people may have not believed us in the past. And then you know, as last year, there was some investigation by the FCC into you know, if that was ethical to do and new laws came into place to ban AI from being used in recruiting in Illinois and other places. So and then now some of our competitors have switched and realized that it has no additive predictive ability to add AI. So we’ve been fortunate enough to not go down that path and, and the benefit, though, however, is we’ve added more structure and more automation and more digitization to our entire product. So with VidCruiter, we can structure in-person interviews on a tablet. And if there’s a video conference where half the people are remote, the other half are in person, you get a complete structured process across all interview methods. And that can automate based on humans scores, instead of the software automating based on applicant answers. So we automate based on tasks, not on applicants.

William 10:50
I love that. Take us, because I think people get especially structured interviews, the questions that we use with candidates structured versus unstructured. So you and I interview the same person, if it’s an unstructured interview situation, you ask him a battery of questions, I asked him a battery of questions, those questions can be completely different. And we can have completely different takes on a candidate structured, you would ask them, you and I would both have some very similar, if not the same questions. But you’ve gone further. And this is where I want, I would love to just get some insight into other parts that you’ve structured. You mentioned, the evaluations and candidates, could you go a little bit deeper there and tell us a little bit about how you’ve helped your customers structure those parts?

Sean 11:35
Sure. So it depends on where they start. As I mentioned, at the beginning, there’s different levels of where that person is in their HR journey of how they structure their processes, and how they structure their interviews. So sometimes we’ll recommend, you know, let’s, let’s, let’s get you interview guides, and let’s get you interview guide response answers, not what the answer is. But what, generally speaking, when evaluating this kind of question, what would be a good indicator of if they have better responses?

And the beauty of this system is that you’re right, when there’s different people asking different questions. But when it’s asynchronous, everyone is asked the same question. And then I can share that with multiple people in my team, so that we have different opinions on how well they answered based on the structured interview guide, to then help remove that bias. And so if one of us has a bias there, you know, and I’m the point of contact for a phone interview, there’s a higher probability that I’m going to make a decision that could be in a certain way. But if there’s four of us that are involved in doing an evaluation, or five, or six, or whatever the number is, the probability of bias goes down dramatically.

And so that’s where the asynchronous product or even recording a live video conference dramatically helps reduce that bias on top of the fact that you have the structured interview guide. Like, I can give you another example.

William 12:58
Yeah, please do those. That’s – Yes.

Sean 13:00
So the other cool part of this is on the asynchronous side, you can go back into we have a comparison, comparison mode in our product. So what you do is you have all everyone’s question 1 answers. And you can play, pause, replay double-check, and really get to understand what they are saying, have a thoughtful response to or, you know, evaluation to what is actually said. The other advantage of the asynchronous tool is that you get time to think about what that other person is saying. And so with your structured interview guide, you can read through, okay, what are the competencies I’m looking for?

What is a good answer here in terms of customer service? Does this person have all the right criteria, and then go back and pause, replay, compare double-check, go back and forth, and really put in a thoughtful response to who is that quality person that I want to move forward with? You never had the power to do that before. And some clients will come back and say, You know what, I was watching this video and I thought of three probing questions that I wouldn’t have thought of, if I was on the, on the spot with that person at that time.

But because I had the time to think about what they were saying, and process compared to other people, then I was able to think of better probing questions to be even more prepared for my in-person interview for the entire pool and really drill into who is that better quality person?

William 14:19
I love that. So you’ve you I want to make sure I understand this. With assessments, um, you’re obviously, you want to standardize those with skills testing, standardize those. Do you also have, if not now, down the road look at reference checking and thinking about standardizing that as well?

Sean 14:39
Our product has standardized reference checking built into our system that’s one of the products we actually sell. And the stats on reference checking is they’re not always as valid as any other method of record. But, you know, for ISO compliance, a lot of companies have to check references and with our product and the automated Reference checking it is now being standardized. So every referee or reference is being asked the same question the same way. And generally what you see is because I have to write it down in a sort of online report that’s going back to the employer, you don’t get any gray areas anymore. It’s typically black or white. Because on the phone, I can tell you anything I want. And there’s no really like, there’s no record of what was said. But now there is. And we generally find it references are more honest with that product now, because of the standardization and the way in which is conducted.

William 15:30
Right. Okay, so let’s go backwards to another, we have a good working definition for folks structured versus unstructured, structured versus AI. If they haven’t been down this path before Sean, where do they start with structured interviews?

Sean 15:48
There’s two things that you need. The first thing is you need a product that can deliver the proper structured interviews for you at the right time, and what you need it in that person’s hand. Now, VidCruiter is one of the companies that does that. We have other people in the market. You know, we pride ourselves on being focusing on structured interviews for the last six years non-stop, as soon as we realized that AI wasn’t going to work. And so we can deliver the guide to that person at that time, whether it’s a video conference, in person, a combo, async, whatever it is, that is being structured, but not only do you need the tools. The other part that we see, oftentimes people don’t have the right interview questions, and they don’t have the interview guides.

Now there are vendors that do that VidCruiter is not in the interview guide building competency. are, you know, that’s not one of the products that we do. But we recognize that SIOP is the – I don’t know if you know who that organization is. Sure, there’s a ton of people that are consultants that would love to help organizations build out the right competency guides for the right job for the right fit for that specific requirement that you need. And I’m amazed every time I look at the interview guides, you know, we hire customer service people internally here. And actually, they’re looking at the interview guides. And it’s like, this tells me exactly what I should be looking for. And you just don’t even realize how much it’s a no-brainer once you actually read one of these guides. And people just tend to not realize how good it can help.

Especially for a manager that does one every – once a year, he or she may not know all the details of what to look for in a good answer, they just sort of have like, Oh, well, this kind of, and once you have this in front of you, it just opens up your eyes to Oh, wow. And it just makes sense. But it’s just that extra little nudge to help that manager help make those right decisions. And so the two parts are: find the software that delivers them for you, and then the second part is finding the right guides that work for your organization.

William 17:42
So another question that folks are gonna ask is okay, how do we get hiring managers and recruiters on the same page as to the structured interview questions? Venus and Mars, right, let’s fix that Sean.

Sean 18:00
Well, I mean, what we tell our HR clients that they have that same challenge, everyone has that same talent is with our product, you’re kind of forced to use it, right. So if you don’t use the structured guide, it doesn’t move you forward, and you can’t not use it. And so that’s step one, is just to make sure that they’re forced to use it, and then you kind of have no choice. But to get an option. That’s the easiest way around that. And the other one is to just, you know, have a training with them and explain or maybe explain the statistics, we oftentimes for our clients in HR, do a demo to all their managers and show them the statistics we have in our slides. This is why it’s better, this is how it works. Look at how this works. And we help sell that into the organization. Because sometimes HR may not have as much, you know, it’s fun to bring in a new tool and have an expert that sort of showcase it and then sell them the methodology at the same time. So that’s a service that we offer. And I think that really helps I think clients really enjoy the fact that we’ll do webinars, whether it’s 50, 100, 1000 managers on and we’ll just promote it from a sort of, you know, cheerleader side here, hey, we’re here to help. This is the best practices, this is what the biggest companies in the world are doing. And they’re getting better quality people, let’s help each other and put this in place. So that’s, that’s what I would do is get a champion like us to help you push that through.

William 19:20
So how do they know – how does an organization know that they’re doing structured interviews to structured digital interviews? Well, like how do they know that they’ve, that they’re actually hitting the mark?

Sean 19:32
Um, I mean, that’s a good question. I’ve seen every kind of level of understanding of that comment. We see companies that come in and they wish they could and then I have mine from upper management, or they understand but they’re not there yet. And it’s like that step-by-step approach that I mentioned, right. But we – you know, luckily on early on, we had one of the major banks in the world, the fortune 50 clients come in and sort of teach us–and they had some of the smartest people in recruiting I’ve ever met. And they taught us from the product point of view, no no, this needs to do this, it needs to have this competency guide, it needs to work when there’s an in-person session where the candidates in this, you know, session with an in-person with this person in a group session here, and we need to competency guide this whole thing all the time. And we learned that was in 2012, how to build it out for it works for every single type of interaction. But I don’t know if I answered your question specifically,

William 20:32
You did fine, you did fine, I mean, it’s, it’s hard to look back, and especially for those that are starting, if they’ve been in an unstructured world and suddenly try they try a structured world, you’ve you know, you said at the very beginning that there’s kind of your beginner level, intermediate level, and advanced level. And through that process, you know, that you’re going to start standardizing other parts other than just the interview and other just questions that you’d ask candidates.

Sean 21:01
To add to that, I would say, let’s, let’s start with one job that is the most common in your organization. Because to have structured interview guides for every job, and that’s a huge undertaking, right? We’ve done that with some of our clients, and they have guides and it’s, you know, it can be overwhelming. And so let’s start with asking everyone the same question, okay, let’s get an interview guide for that job that makes sense through a sign-up social engineer to come in and help make sure that we have the right questions. And let’s just start there. And then you’re gonna see a quality up to, you’re gonna see a reduction in turnover, you’re gonna see a better quality team come into play, and then let’s move into, okay, what’s the second most important job, and then you’re gonna start getting buy-in, and then people are gonna start getting excited. And then you move your organization forward.

And that could take a year, you know, we’ve helped clients take, it takes years to move through those different channels. But the goal is, let’s start with that. Because as we’ve seen, this is the best way to hire people in AI is not going to be ready for the next 10 years. And so we’re still gonna have to figure out how to do it ourselves. So we’re here to help you get through those different transition stages to get to that level. And then eventually, once the AI can determine what culture you’re in, and what your language is, you’re actually trying to say, and make sure that every single word that you said is picked up and make sure that your eye movements, which they tracked to or are actually accurate, you know, all this kind of things. That’s another 10 years. I mean, we keep an eye on that. And that’s 10 years from now, if not more. So. For now, we have to do this structured interview, and it’s gonna help with D&I, like, you mentioned all these other things that that’s the goal. So-

William 22:33
Yeah, it takes the, I mean, interviewing can be a very subjective sport. And if you take something that’s subjective, is also code for bias, right? So strip some of that subjectivity out of it. And you basically say, listen, we’re going to ask every candidate that comes in, we got 100 of them, we’re going to ask him the same questions. And that takes that subjectivity out of which questions that we ask and people asking esoteric questions and things like that we ask, we ask him the same questions, then we can, then we can look at their answers.

And, and again, it makes it a bit more objective, which takes some of the bias out. What have you noticed in terms of roles, or industries, or like anything, where you just know structured structure works everywhere? So we’ll just say like that, but any roles or industries, where you just, it’s, it’s just this is easy, and it’s just something you need to do?

Sean 23:29
But I think you have to do it for every role. There’s no like, Yeah, what we, what we find and our client base is the governments are mandated by law in a lot of cases to do this. And they love our products, because it’s already built for them. But they already have a lot of those insights into why it works, how it’s better how it’s structured, how everyone’s treated the same way. And you know, you got to ask yourself, Well, if they’re doing it by law, a lot of countries, then it’s probably a good idea to start adopting this. And it’s not a law for, you know, private companies.

But the goal is like, what’s the fastest way to get to the best practice possible? And, you know, let’s do it for all job. But let’s start at the ones that we have the biggest problems with, you know, that’s what I often tell clients – come to us with the job that’s the hardest to fill. And let me fill that one for you first, and let’s go from there. And then we’ll work our way towards the ones that are easy to fill and in and move our way through those different challenges.

William 24:31
So I’m assuming data from these structured interviews ultimately ends up in the ATS. Is, is that correct?

Sean 24:40
Some ATSs do have the capability to pull the data back into yeah.

William 24:44
Okay. Is that your preference? Ultimately, would you like for that data to be kind of following the candidate along that journey?

Sean 24:53
I mean, generally speaking, yes. The beauty of our system is that you can have a structured score for the asynchronous. A structured score for your work sample or the system score depending, then there’s a score for your first and second interview. There’s a score for the reference check, you can get a global what we call like a VidCruiter score for all the different stages. You can assign the specific competency evaluation score or weighted average to each one of those stages so that it falls in line and gives you an overall score with all the different components of what we do in our platform. And you can also export that to another system.

But it’s sort of like, well, we want interview #1 to be worth 15% of the overall structured score, we want interview #2 to be worth 25%, this pre-screen 10. And then it builds out this overall score across all evaluators and a complete structured method, and then you can automate between all those stages. I mean, what we’re talking about now is how we’re years ahead of everyone else. No one else is that we know of is even remotely close to this, because we’ve been doing structured interviews, for six years, seven years now, you know. So it’s really that’s the next level. Now all that can go back into the ATS, this our reporting might be a little bit more robust, but you can feed the data back and forth, if that ATS has the API’s, right, for us to push that back into not all of it do.

William 26:07
What, what’s our responsibility? Or how should, how should recruiters communicate with candidates about structured interviews?

Sean 26:15
I think you’ve got to be upfront. A lot of what we recommend for candidate knowledge, because sometimes there’s, well, there’s a lot less now. But there used to be a little bit more bias about video. And I think that’s, that’s gone away with with the last year, but it’s to let them know, listen, the reason that we’re doing this asynchronous video is to give every single person a fair chance, because everyone’s evaluated by multiple managers using a structured process.

And in what we often have for clients is a webpage that they can send candidates to, that explains everything they need to know, here’s all the tips you need about video, here’s all the tips you need to know about our process. And to be open about how the process works. I heard you say that on a previous podcast of yours, let’s tell the candidates where they are in their pipeline. But let’s just start telling them about how it works. I mean, yeah, yeah, you’re not exactly where they are. But let’s say, hey, surprises, yeah, at this stage, this takes this time, on average, to go through these steps.

We have this kind of methodology here because we believe in this kind of interviewing. I mean, that’s awesome. Just to share that with people. And then anytime they have a question, you just redirect them back to the page. Hey, let’s just read, you know,

William 27:25
It’s so commonsensical, but at the same time, it’s not done well – yet. You know, I’m hopeful. But it’s, it’s you allow them like, hey, there’s no surprises in our process. This is how we’re going to look for 100 qualified candidates, we’re going to try and through structured interviews, and you know, other things, we’re going to try and get that down to about 10.

And then we’re going to do this, like, there shouldn’t be any guesswork, like in the end, it also shouldn’t be confusing to the candidate. So what’s your time is our time is near? What’s the last thing you’d say about structured interviews versus AI as it currently is?

Sean 28:07
Well, I think we won that challenge. I mean, one of our major competitors pulled out of AI and went full structured with us. So and I’ve got a feeling that we helped move the needle back into that direction, because we were always promoting structured interviews. And, you know, I’m glad that the industry is realizing what’s better overall, and that we can all be working together towards a common good for every interview, basically.

But overall, I think that structured interviews are here to stay. You know, Google has proven the research, and they’ve had hundreds of data scientists analyze it. And so, you know, we’re super excited to be part of this field. We’re super excited to be sort of leading in the structured interview component of this field now. Um, yeah.

William 28:54
Well, brother, every time we talk, I learn something new. So I appreciate your time. I appreciate you coming on to RecruitingDaily podcast. And until next time.