The Use Case Podcast: Storytelling About CodeSignal with Tigran Sloyan

Music:  00:02

Welcome to RecruitingDaily’s Use Case Podcast. A show dedicated to the storytelling that happens or should happen when practitioners purchase technology. Each episode is designed to inspire new ways and ideas to make your business better. As we speak with the brightest minds in recruitment and HR tech, that’s what we do. Here’s your host, William Tincup.

William:  00:25

Ladies, gentlemen, this is William Tincup. You’re listening to the Use Case Podcast. Today, we have Tigran Sloyan, from CodeSignal. We’re going to be talking about the use case, business case, cost benefit analysis, however you want to phrase it, but the case you make for CodeSignal, the case that his customers make for CodeSignal on a daily basis. So without further ado, Tigran, would you do us a favor, introduce both yourself, please introduce yourself and CodeSignal to those that aren’t aware.

Tigran:  00:57

William, thank you so much for having me here, excited to feature on the podcast to introduce myself and the company. As you said, my name is Tigran Sloyan, and I’m the Co-Founder/CEO of the company. I’m also the founder of the nonprofit GoBeyondResumes movement. The use case for CodeSignal, right? The core idea behind what we do is we believe that traditional methods of identifying talent are broken, such as resume reviews and interviews. And the idea is to reinvent that with something that’s far more predictive, far more reliable, and far more scalable. The way we do this, is by essentially creating flight simulations for different skills and having those flight simulations be the direct measurement of skill and ability, giving companies a chance to actually do evaluation versus relying on resumes and looking at pedigree or the interviews, which tend to be very costly and inconsistent. And the reason-

William:  02:06

No, no. The question I wanted to ask you is, you’re going to get through it obviously. Simulations, it gets those skills and experience, but also this helps the candidate as well to understand the job.

Tigran:  02:18

Absolutely. That’s one of the big value propositions of doing flight simulation style evaluations is that as a candidate, you want your candidates to walk away from an evaluation not saying, What did that have to do with the job where I’m totally qualified for this, but like you got it wrong, but rather like, Hey, that was great. And I probably need to upgrade my skills or I’m just not a fit for this job, because you just gave me the opportunity to do it. And, I can see that I’m not fit for it.

William:  02:47

First of all, I love it. And by the way, I love what you did at the last HRTX event. Your presentation was just really, really well done. So where should recruiters, where should they put this in their workflow? Like where’s the optimal place to put place this in kind of their funnel, if you will, do they place it really far out to filter out the folks that just aren’t going to make the cut or do they push it further back into the process?

Tigran:  03:22

Yeah. Our customers actually place it all throughout and I’ll explain why. Just think about traditional ways you would do evaluation of skills and fit and potential, right? To start from resume review typically at the very top of the funnel, then you go into some sort of a technical evaluation, technical interview. Then you probably have an onsite panel where CodeSignal gets embedded into each one of these. For your top of the funnel evaluations, instead of doing resume reviews, you start doing usually our standardized generic assessments that allow for a more fair assessment of skillset. And, then you’ll end up with a talent pool. Our customers tell me consistently that, we just hired someone that we would’ve never even interviewed because based on their resume, that’s not someone that we would’ve even considered.

Tigran:  04:17

Then you go into middle or late in the funnel. Traditional interviews tend to be highly inconsistent even between interviewers for the same role, even depending on the time of day. There is a famous study about judges who, when you look at their sentencing after lunch and before lunch, it turns out judges are a lot more harsh before lunch for the same exact cases than after. Same thing happens in interviews, right? You cannot really depend on human interviewers being consistent even throughout the day, let alone across, multiple interviewers for the same exact role. So we get introduced all throughout wherever assessments are happening, either bring automation or consistency and reliability to the overall evaluation process.

William:  05:08

And so now that we’ve dealt with workflow from a funnel perspective, let’s do it from a tech perspective. CodeSignal needs to be integrated and play well with I’m assuming ATSs?

Tigran:  05:19

Yes. ATSs are the most popular integration we do. And we are by now integrated with almost any ATS you haven’t heard of. So, and the integrations are fairly standard by now. It’s ability to schedule interviews or send out assessments through the ATS, it’s ability to get those results back without having to toggle between two different platforms and it’s ability to automate some of these workflows, in case you’re doing very high volume hiring and you want to do high volume assessments. You don’t necessarily want to be inviting candidates one by one that rather do a batch flow.

William:  05:57

So, you mentioned the two words fit and potential. When you’re doing skills testing, how do you figure out, because some of it’s finite, right? You either have that particular skill or you don’t, but you also figured out potentiality or maybe transferable skills and things like that. So how does one look at that and also think about the potentiality of their skills?

Tigran:  06:26

Absolutely. That’s a phenomenal question. Because that’s the holy grail, right? How do you figure out the potential and-

William:  06:30

That’s why I was asking you if you [crosstalk 00:06:37] If you know what the fountain of youth is like, I’d actually really like to talk to you.

Tigran:  06:41

Well the idea here is that if you want to make projections with any dataset, really the way to make projections is to see historical trends, right? Because once you have historical trends, you can start making reliable projections and given that we’ve been in this space long enough to have accumulated a really large data set. In many cases, we assess the same candidate multiple times throughout the year or over multiple years. And we can not only see where are they in the moment in time, in terms of their skill set, we can see how that has changed so we can see a rate of growth, not just present state and rate of growth obviously helps you project, what it’s going to look like 6 to 12 months from now, which is really what employers are interested in.

William:  07:31

Love that. Yeah, that’s what I’m hearing too, it’s worth. First of all, there’s just a skill shortage. So some of it’s that supply and demand and also some of it’s just like they want to look at potentiality. So, this might be a dumb question, but I’ll ask it anyhow, what roles because you mentioned high volume just a second ago. And so it threw me off for just a second, because I was thinking of this mostly in technical, but does CodeSignal, where does it best play in terms of either industry or job class or any of those types of places?

Tigran:  08:12

Yeah, it’s actually very focused on technical right now. However, even within the technical realm, you’ve got very high volume and very low volume. If you’re hiring, for example, early talent engineers out of colleges, boot camps, or any other educational programs, even a small company gets thousands of applications every day. And, unless they’re going to resort to the traditional method of, look for MIT, Stanford, Harvard on a resume, which is obviously a terrible way of doing it, they need to have a way to automate this.

Tigran:  08:48

But then again, we have use cases where a customer is hiring, Senior Salesforce Developer with some JavaScript skills that is a very low volume application process. But then again, you still need structure. You still need a better way of simulating this, some kind of an environment where the person can demonstrate their ability to perform the job. Because traditional methods of saying, tell me about a time you did X, don’t necessarily work.

William:  09:19

What I love about skills and how we’ve moved conversationally to skills is, it’s kind of a great [inaudible 00:09:26] right? It doesn’t know if you’re black or white or male or female or this or that, Catholic, Protestant, whatever. Doesn’t know any of those things, it just, do you have the skill? Do you have the potential to grow that skill? Do you have the desire, passion, et cetera, it seems at least we’re early on in this, I’m sure. It seems to be, that we’re reducing by, at the same time that we’re getting probably a better fix on the real skills that we need. We’re also at the same time or simultaneously tackling diversity and inclusion. And some of the other things that we should be focused on, have your customers asked you about that or have they talked to you about diversity and inclusion as it relates to CodeSignal.

Tigran:  10:14

Yeah. You literally hit the nail on the head, right? When we talk about use cases and how our customers talk about this internally or make the pitch internally, we already touched on two things, but not the third, which is at least as important as all the other two, if not more important. So the first two being, saving time, creating consistency, but then promoting diversity really becomes an important piece here because it equalizes the way you assess for talent. And it helps remove in many ways the biases that we have and around many different things. The best example to see is when you think about gender bias, I don’t know if you’ve heard of the orchestra experiment, but in the sixties and seventies, there is this big push to say that, only 6% of violinists in most orchestras over the nation are female.

Tigran:  11:14

And the belief was that the reason this is not changing is you can see the person that’s playing when you are auditioning for an orchestra. And that fact along where you can see that the gender of the person playing, biases your judgment over their ability to play. So what they did is they said, what if we did blind auditions where we put a screen in front of a person playing, because it doesn’t matter what they look like. As long as you can hear it, you can judge, whether they’re fit for this fall or not. And over the next 5 to 10 years where this was adapted, it went from 6% to over 40% of violinists in orchestras being female, which proves that just when you see it, your unconscious biases kick in whatever patterns you’ve identified in your head as like who should be a fit for role X and who shouldn’t be, it clouds your judgment and you can no longer make a fair evaluation.

Tigran:  12:12

And this is where skill assessments come in. They do really spend the true chance of actually promoting and bringing diversity to teams in a very tangible way.

William:  12:24

I love that. So, if not now down in the future, do you see clients adopting this internally? In terms of trying to figure out, if we’re getting that DevOps engineer, what skills are we really looking for when we look at the skills that we have and trying to figure out the DNA of their own folks or do you see an application internally, as opposed to externally?

Tigran:  12:56

Yeah, absolutely. Skill assessments have such a massive role to play throughout the industry from early childhood education to high school, to college admission, to getting your first job, to getting promoted, to promoting pay quality. Think about it, right? There’s a lot of talk about pay quality, but a big component of that is skill set. And without verifiable data around who skilled, who’s not, it becomes a judgment goal and nowhere biases thrive. It’s all about a judgment goal of who’s good, who’s not. So I do see massive application of this. And the interesting thing is that precision of the assessment becomes even more important when you’re doing this for an educational or just an internal promotional use case. While for hiring, it’s more does somebody meet the bar or do they not?

Tigran:  13:52

So I think over time you will notice our evaluations are simulations being more actively applied in other places versus now it’s very focused and very successful in talent acquisition.

William:  14:08

Yeah. Eventually I see it being applied for internal mobility, but also in outplacement. So at the end of the cycle, it’s kind of understanding where someone is with their skills and then giving them on to their next best opportunity. I can see it being used there as well.

Tigran:  14:32

The person taking the assessment should be in control of the data and in control of when it’s shared and who it’s shared with, kind of think of your credit score, right? It’s not like, just because it exists, it’s not like anybody can just go look it up. You’re in control of when you decide to share it and who you decide to share it with. So I think putting the privacy and accessibility of this data front and center is fundamental because the last thing that we would want is assessments being used as a method to really not give you the chance to grow versus giving you the [inaudible 00:15:11] to your own skill set and what you need to do to get better.

William:  15:13

Yeah, that helps the candidates fill in again. You can work on your strengths and get stronger. You can fill in on some of your weaknesses, you can make all kinds of choices, but it’s yours. It’s just like your, I love it, the credit score analogy, because it really fits. Take us into a simulation. So let’s go deep now that we understand at least from another perspective, we kind of get it. Take us into any one of your favorite ones. Just take us into a simulation and explain to the audience what the candidate goes through, but also from a recruiter and hiring manager, what they see as well.

Tigran:  15:46

Yeah, absolutely. So let’s say you are hiring a front end engineer, to build something like slack or a competitor for slack, right? What the simulation would look like. It would literally give you an interface, a coding environment, I guess when you think of a cockpit for a pilot, the Integrated Development Environment, which is called IDE for short for software engineers is where they build applications. So we will give the entire infrastructure already set up in your browser, right? So as a candidate, when you get in, it gives you the IDE, it gives you the coding environment. It gives you all the APIs you need. It does some of the basic setup work. So you don’t spend an hour just figuring out how the tools are set up. And then you literally build a chat application or you add a feature on top of it.

Tigran:  16:40

So it can be built from scratch depending on kind of role that you’re being hired for. If it’s a really startup company that needs to build products from scratch, it could be, build this from round up and get you an initial MVP. If it’s more of a later stage company, it could be, Hey, there is a bug with this chat application we’ve already built, go fix it. And this simulation can happen in a sort of you do it on your own. And then it gets auto integrated, or it can happen in a collaborative mode, like a co-pilot mode where you’ve got an person on the other side, who’s observing you doing it, who’s collaborating with you to understand not only your core skillset to perform the job, but also your collaboration ability, your ability to work with others in a semi-stressful situations.

William:  17:27

Real quick, who creates the simulation? Do you already have though, a library of simulations or do they need to be edited and personalized to the job or the company or any of that type of stuff?

Tigran:  17:38

Yeah. While our customers do have the ability to create and customize, we have a massive library called pre-designed assessments. And this is actually very, very important because assessment design is difficult. Assessment design is a discipline of it’s own. And even though a lot of engineers believe that they would be great at designing assessments. Every time we go to display something, that’s already there. We look at it and everything from lack of under standing of what creates a great candidate experience to lack of understanding of regulations cause assessments and interviews are regulated. And especially, if you’re a larger company and you’re not paying enough attention to how your assessments have been designed that they contain any sort of bias and from the get-go, because even the language itself in which the question is phrased or the context can be biased. And, if you’re not trained in assessment design, it becomes really difficult to create something that actually produces results.

William:  18:40

Yeah. It’s the unintended consequences, right? Again, this is where IO psychologists, they can help you get through some of that stuff. So, I knew you had a library. I was just wondering if folks use the library primarily or if they want to go off the reservation a bit and build their own or modify it in some way.

Tigran:  19:06

It’s even more advanced than a library. We call it framework based assessment design, where we design entire frameworks that are around specific job categories. So we’ve got a front-end engineer, we’ve got a front-end assessment framework that specifies, how are we going to measure candidates in this discipline? How many questions are we going to ask? What difficulty, what skills we’re going to measure, what skills we’re not going to measure. And that framework also allows us to introduce variations to our assessments, because without variations questions get leak and there’s no way to prevent. This happens with interviews, traditional interviews all day long. You just go to Glassdoor before your interview and you have massive leg up on any nobody else interviewing without that. And the only way to prevent this and you still stay consistent, you’ve got to build your evaluations from frameworks. So you can have some variety in there while still maintaining consistency.

William:  20:06

All right. So this is one of my favorite questions is, when people first see CodeSignal for the first time and your folks are showing them, your have sales folks, your BDR, your marketing, your trade show, somebody sees it for the first time, what do they fall in love with?

Tigran:  20:23

That’s a great question. It actually depends on who at the company. So there’s two core, I guess, call them buyers call them stakeholders that we work with. So on the one hand, you’ve got the talent acquisition stakeholder. On the other hand, you’ve got the business stakeholder in the form of engineering leadership. Engineering leadership, always falls in love with the depth of the simulation and how realistic it is, how comfortable it is for candidates and how you can actually watch back what they did during the evaluation on a speed up. Because think about it, in traditional evaluations, even if it was an interview, you just get notes from some other interviewer. Those notes don’t do justice to what really happened, right? What if you could play it back on, I don’t know, 60 next speed. And within five minutes, not just see the end result, but what exactly happened during the evaluation?

Tigran:  21:22

It’s really insightful and Engineering Managers, Hiring Managers love it. Now when it comes to talent acquisition side, it’s interesting, because talent acquisition, it’s so used to relying on their business partners and engineering leaders to evaluate candidates that they don’t even consider the possibility that what if you have limitless resources to evaluate candidates, right? Because, talent acquisition is very similar to sales and in sales just like account executives or recruiters, SDRs are like sourcers, but what sales people call opportunities or deals, I often ask sales teams, what would your life feel like if every time you had to qualify a deal, you would have to go ask your engineering team, to spend time with your buyers and to tell you if that deal is qualified or not, it would drive you crazy because this introduces such a headache, right? Like you become resource constraint to how many of those can you actually qualify it, drags out how long it takes you to actually get through the process. And it becomes a block box where you were like, I have no idea why this was a flawed idea, why this wasn’t.

Tigran:  22:36

So when it comes to talent acquisition, what they get shocked by is that what if you lived in a world in which you were in control of and in understanding of how your candidates are getting evaluated, what’s the true criteria and you knew that it was consistent and you knew that it was scalable. And you knew that you don’t need to start rejecting 90% of your candidate pool just based on keywords on their resume. But you can actually dig in, give equal opportunity and find people who deserve a chance who can totally hire while also in many ways, beating the rest of the market. Because, the late adopters are still using those traditional methods. And while innovating, you can actually find candidates that the rest of the market is missing.

William:  23:23

I love that. So two more questions, one of them, and it’s going to be hard for you, but one of them is your favorite customer story without naming names of course, keep all that stuff, but it can be an old story or last week doesn’t really make a difference. But, just something that just sticks with you as one of your favorite ways that your customers have used CodeSignal to help them get to where they need to be.

Tigran:  23:53

Sure. My favorite story is always, always center around the candidate stories suite, sometimes internally called them talent stories and there’s countless number of them, right? And one of my favorite ones was about a candidate who, they were an immigrant at this country like many others, and all of their educational background and experience was based in non-US institutions. So when US recruiters looked at their resume, nothing made sense, right? From the university they went to from companies they worked at, but this is a Senior female engineer, a type of talent that’s very much in-demand all across the country and their skillset is phenomenal. But what was happening with this person is anywhere they would apply for a job, they would not even get to the first interview, like [inaudible 00:24:48] fighting the interview biases. They would just never get an interview to the point where the person had to settle for being a translator instead of utilizing their highly in-demand skills.

Tigran:  25:02

They ended up applying to one of our customers and this customer obviously instead relying on their resume, gives them an assessment to start with. And she literally destroyed this assessment and got one of the best scores they had ever seen with disbelief. Obviously after the resume, after the assessment results, you still start looking at the resume. You’re like, wow, this person did really well. Hopefully, there is nothing fishy going on. So they brought her in and in the same way, she absolutely destroyed the interview did really, really well. And she got hired and became one of their top performing employees. And anytime I meet with hiring managers or the TA team, they always say, it’s just so crazy to realize that someone like this, wasn’t getting an interview. I think you know this is happening, but you don’t appreciate it as much until you see it on a real life story.

William:  26:01

Yeah. And again, that uncovers a lot of that bias, right? If I’m American, let’s say, and I don’t recognize the University in India. It might be the MIT of India, but I just don’t recognize it. And again, skills and this type of simulation, it just cuts through all of that. Whether or not, it, or you don’t, doesn’t matter. It’s just can they do, do they have the skills?

William:  26:28

The last thing is just kind of simple. It’s philosophical more than anything else. I don’t want to get into dollars and sense of pricing, but just give the audience a sense of how y’all go about pricing.

Tigran:  26:38

Yeah. Again, as you said, very important, right? The way we do pricing is we align it with our mission of helping the world go beyond resumes. So we’ve never priced based on usage, because while usage based pricing would make a lot of sense. Because, the unit of value delivered is an evaluation. We don’t price based on that, because guess what? [inaudible 00:27:00] As pricing based on usage, we discourage that usage. Because every time I use it, I’m like, oh wow, I’m paying money to assess this person. But I feel like they’re never going to make it. To discourage that behavior, we primarily price based on the size of the organization. And then we model to understand, what kind of usage do similar organizations have. But then from there we give unlimited access and usage to all of our customers, which really empowers them to start embracing the idea of going beyond resumes versus counting dollars every time they assess somebody.

William:  27:37

It’s no clock watching. I love that. Drops mic walks off stage. Tigran, thank you so much for coming on the show. It was amazing to get a better understanding of CodeSignal and for people to understand the use case for CodeSignal. So thank you.

Tigran:  27:54

Thank you. I really appreciate your time.

William:  27:57

And thanks for everyone listening to the Use Case Podcast. Until next time.

Music:  28:01

You’ve been listening to RecruitingDaily’s Use Case Podcast. Be sure to subscribe on your favorite platform and hit us up @recruitingdaily.com