An Experiment in Hiring Discrimination via Online Social Networks Alessandro Acquisti and Christina Fong Carnegie Mellon University Heinz Seminars, March 2014 In the U.S., it is risky* for employers to ask interview questions about Family status and plans Religious orientation Political orientation
Sexual orientation However . many candidates nowadays volunteer that information online U.S. employers have started using social media to find information about job applicants 61% googled job candidates 51% reviewed available information on MySpace, Facebook or other social networking sites 57% reviewed available blog entries
34% did not hire the person based on what they found Sources: 2007 Ponemon Institute HR Special Analysis, 2008 Career Builder Study, 2010 Microsoft Survey But the actual frequency of the phenomenon is debated Searching not illegal, per se, but can lead to discrimination (Ponemon 2008) Hence, some organizations are now being advised not to seek online information about prospective candidates
In fact, only a minority of employers may actually be using social media sites for hiring purposes (2012 EmploymentScreenIQ Survey) Professional/Unprofessional traits on social media, according to employers Candidates background supported their qualifications for the job Candidate had great communication skills Candidate was a good fit for the companys culture
Candidates site conveyed a professional image drinking or using drugs Candidate posted provocative or inappropriate photographs or information Candidate had poor communication skills Candidate bad-mouthed their previous company or fellow employee Candidate had great references posted about them by others Candidate posted information about them
Candidate lied about qualifications Candidate used discriminatory remarks Candidate showed a wide range of interests Candidate received awards and accolades related to race, gender, religion, etc. Candidates profile was creative Maybe this is too much
[] Candidates screen name was unprofessional (From 2008 Career Builder Study) [] Research questions 1. Do US employers actually seek information about job candidates online? 2.
If so, are their hiring decisions being affected by information the candidates openly divulge on online social networks (including protected information)? Related work Large body of economic research on (job) discrimination, but relatively fewer field experiments Audit studies Resumes studies (e.g., Bertrand and Mullainathan "Are Emily and Greg More Employable than Lakisha and Jamal? AER 2004) Our studies Study 1: Online experiment Spring 2012, survey experiment with over 1,000 Amazon MTurk subjects
Study 2: Field experiment Spring 2013, resumes sent to over 4,000 U.S. employers Common DV Call-back (i.e. invitation to interview) Approach (common to both studies) Created unique names Designed (identical) resumes associated with each name Designed (identical) profiles on a professional social network associated with each name (LinkedIn)
Designed (manipulated) profiles on a personal social network associated with each name (Facebook) In a nutshell Conditions Our experiments focus on: Religious orientation (Christian vs. Muslim) Sexual orientation (Straight vs. Gay) Conditions Our experiments focus on: Religious orientation (Federal protection) Sexual orientation (State protection) Conceptual model
Employers JDM, heuristics, biases Employer checks profile? Candidate reveals traits on profile? Candidates traits Designing a real fake social media profile Timeline Image Personal
Informatio n Open ended fields (e.g., status updates) Profile Image Name Close ended fields (e.g., Likes) Friends Interpreting a null result
A null result (call-back rates equivalent across conditions) could be due to a number of different reasons: Employers do not search for candidates We control for that using Google AdWords and Premiere accounts Employers search, but dont find our profiles We control for that by choosing and testing high-ranked names Our manipulations dont work We control for that using the online experiment Employers search, but only later on in the hiring Study 1: Online experiment Spring 2012 1,170 subjects recruited via Amazon MTurk
4 conditions between-subject design Links to resumes, LinkedIn, and Facebook profiles Plus, another 4 control conditions only with links to resumes and LinkedIn profiles Study 1 : Questionnaire 1. Attention checks 2. DVs: Imagine you are an HR person Would you call this candidate for an interview? (Binary) Additional Likert questions about employability 3.
Manipulation checks 4. Open-ended questions 5. Demographics Study 1: Results Manipulation checks Call-back ratios Regression analysis
Success of deception Study 1: Manipulation checks Successful Study 1: Call-back ratios, all subjects Self would call for interview [0,1] Christian 94.78% Muslim 92.68% {n.s.} Straight
94.44% Gay 94.06% {n.s.} Study 1: Call-back ratios, only subjects with hiring experience Call-back ratios N N Employability Score (s.d. = 1, mean =0) PANEL A: Religious affiliation manipulation Muslim
Christian Muslim Christian 88.03% 96.85% -0.326 0.21 117 127 117 127 Two-sided Fishers exact Two-sided t-test p-value: 0.012 p-value: 0.004 PANEL B: Sexual orientation manipulation Gay Straight Gay Straight 93.02% 93.80% 0.02 -0.08
129 129 129 129 Two-sided Fishers exact Two-sided t-test p-value: 1.00 p-value: 0.55 Study 1: Regression analysis Aggregated conditions into two groups: Advantaged Straight Christian Disadvantaged Gay
Muslim Note: not necessarily economically disadvantaged, but more likely to be discriminated against (according to Study 1: Regression analysis (OLS; DV: employability score) Disadvantaged Candidate Hiring Experience Disadvantaged Candidate * Hiring Experience Democrat Disadvantaged Candidate * Democrat Independent Disadvantaged Candidate * Independent U.S. born Disadvantaged Candidate * U.S. born Constant Controls included? R-squared N
2. Pooled Sample 0.318 (0.276) 0.025 (0.090) -0.273** (0.123) -0.068 (0.118) 0.319* (0.183) -0.291** (0.125) 0.440** (0.185) 0.394** (0.193) -0.537** (0.238) 0.012 (0.239) Yes
0.035 1,017 Study 1: Open-ended answers Only 0.3% expressed doubts about the candidates existence Open-ended answers [I used Google] to check if a [name of the company] really existed in [city]. [I searched for him] to find photos, as well as less tailored info. I found his facebook page. [The LinkedIn profile] didn't affect my opinion - I think LinkedIn is really generic and not very useful. It helps verify
that the person actually exists, though. I don't think it is fair for the applicant to have his personal information like address and phone number given out like this. If I found out my resume was posted on mTurk I would be very angry. Study 1: No Personal profiles conditions Also tested 4 conditions in which subjects were only provided links to Resume + LinkedIn profile No statistically significant differences Study 2: Field experiment Spring 2013
4,152 employers (found via Indeed.com) 4 conditions between-subject design Several job types (and corresponding resumes) Combination of IT, managerial, and analyst positions Note: In Bertrand and Mullainathan (2004) Caucasian names call-back ratio ~10%; Study 2: Geographical distribution of applications
Study 2: Search results Current information about employers online searches comes from wildly differing estimates in self-report surveys Our (also noisy, but field) data: Google AdWords stats Premiere accounts Lower boundary: 9.92%
Higher boundary: 27.68% Study 2: Call-back ratios Gay Interview 10.65% invitations Applicatio ns 1,071 Straig ht Musli m Christi an 10.63%
10.92% 12.63% 1,025 1,026 1,061 Study 2: Call-back ratios Study 2: Regressions (OLS), Religious orientation conditions Muslim candidate Politically mixed area Democratic area Muslim*Political
ly mixed area Muslim*Democr atic area (1) State -0.150*** (2) County -0.145** (3) State -0.145** (4) County -0.180** (5) State -0.117**
(6) County -0.166** (0.0572) (0.0740) (0.0568) (0.0762) (0.0571) (0.0747) -0.0423 -0.111* -0.0495 -0.117
-0.0228 -0.109 (0.0544) (0.0673) (0.0574) (0.0713) (0.0571) (0.0700) -0.0578 -0.108 -0.0642 -0.124
-0.0290 -0.122 (0.0547) (0.0672) (0.0720) (0.0782) (0.0724) (0.0772) 0.129** 0.130 0.125** 0.168**
0.0997* 0.161** (0.0604) (0.0791) (0.0603) (0.0807) (0.0604) (0.0793) 0.152** 0.148* 0.146** 0.183**
0.115* 0.167** (0.0611) (0.0771) (0.0608) (0.0793) (0.0613) (0.0778) YES YES YES YES YES
YES YES YES 2,039 0.008 1,692 0.042 2,039 0.038 1,692 0.074 State fixed effects Geo controls Job/Firm controls Observations R2 YES
2,087 0.003 1,703 0.031 Study 2: Robustness checks Results robust to: OSL/probit specifications Different categorizations of states/counties by political leaning Presidential elections Gallup 2012 political ideology survey Gallup 2012 political party ID survey Union sets
HQ location vs job location Taking 1 state off regression at a time Conclusions Online experiment provides some self-report evidence of discriminatory biases along traits we manipulated Field experiment suggests a minority of U.S. employers actually search (at least, for the job types we applied to) Hence, overall impact of manipulated traits is small Thank you
CMU RAs National Science Foundation under Grant CNS-1012763 Carnegie Mellon CyLab TRUST (Team for Research in Ubiquitous Secure Technology) IWT SBO Project on Security and Privacy for Online Social Networks (SPION) For more information
Google/Bing: economics privacy Visit: http://www.heinz.cmu.edu/~acquisti/economi cs-privacy.htm Email: [email protected] Manipulat ed Randomiz ed TABLE 1 MANIPULATION CHECKS H0: FREQUENCIES OF THREE BELIEF RESPONSE CATEGORIES DO NOT DIFFER (P-VALUES REPORTED) BELIEFS CONDITION
S Gay/ Straight Muslim/ Christia n Kids/ NoKids Unprofe ssional/ Professi onal Female / Male Attractiv e/ Unattract
ive Married/ Single Caucasi an/ African America n NO FACEBOOK LINK PROVIDED Gay/Straight Muslim/ Christian Has kids/No kids Unprofession al/ Professional 0.515 0.129
0.424 0.878 0.083* 1.000 0.236 0.520 0.524 0.820 0.403 0.465 0.104 0.717
0.297 0.194 0.006*** 0.180 0.560 0.841 0.174 0.001*** 0.054* 0.106 0.995 0.485
0.307 0.460 0.895 0.065* 0.437 0.247 FACEBOOK LINK PROVIDED Gay/Straight Muslim/ Christian Has kids/No kids Unprofession al/ Professional 0.000***
0.000*** 0.434 0.211 0.141 0.217 0.000*** 0.915 0.081* 0.000*** 0.165 0.249 0.725
0.071* 0.029** 0.001*** 0.088* 0.325 0.000*** 0.783 0.642 0.961 0.000*** 0.143 0.002***
0.661 0.000*** 0.000*** 0.928 0.684 0.000*** 0.375
