Definition
Reliabilityisreliability,whichreferstothedegreetowhichtheresultsobtainedareconsistentwhenthesamemethodisusedtorepeatedlymeasurethesameobject.Ontheotherhand,reliabilityreferstothereliabilityofmeasurementdata.
Forexample,forthefirstquestionofthefirstpartofthe"LibraryUtilizationandSatisfactionQuestionnaire",ifyouaskthesamequestion3daysapartfromthesameperson,ifyouansweritforthefirsttime,youwillbeTheinvestigatorchoosesA,thesecondanswerchoosesC,andthethirdanswerchoosesD,whichmeansthatthereliabilityofthesurveyresultsforthisquestionislow,becausethesurveyresultsarequitedifferent.Ifthesameanswerisselectedthreetimesortheanswerwithsmalldifferencesisselected,itmeansthatthereliabilityofthesurveyresultsishighundertheconditionofexcludingsystematicerrors.
Formula
Ifisusedtoindicatethetruevalue,indicatesthatthedeviationisthesystematicerror,representstherandomerrorofthemeasurement,representsthemeasurementresult,thereare:
Intheabovedecompositionformula,isanabstractvariable,apotentialvaluethatneedstobeestimated.Systemerrorisanerrorthatcanbeavoidedorreducedbyadoptingcertainmeans,whilerandomerrorisunavoidable.
Ifthemeasuredresultisconsistentwiththetruevalueorthedifferenceissmall,themeasurementissaidtobe"reliable"or"reliable";Otherwise,themeasurementissaidtobe"unreliable"or"unreliable"tosomeextent.
Becausethesystematicerrorisdifficulttodecompose,andthequestionnairedesignrequirestoavoidsystematicerrorsasmuchaspossible,sousuallyonlyrandomerrorsareconsideredinthereliabilityanalysis.Thatis,onthebasisofthefollowingformula,analyzethereliabilityofthequestionnaire:
Reliabilitycoefficient
Overview
Forrandomerrors,itisgenerallyassumedthattheexpectedvalue(averagevalue)is0andisindependentofthetruevalue.Underthisassumption,thereare:
Sinceandareindependentofeachother,therearethefollowingformulas:
VisiblemeasurementvaluesThevarianceisequaltothesumofthetruevaluevarianceandtheerrorvariance,andandTherelativesizeofcanbeusedtodescribethecredibilityofthesurveyresults.Thelargerthemeasurement,thelargertherandomerrorandthelowerthereliabilityofthemeasurement.Weusethereliabilitycoefficienttoexpressthedegreeofreliability.
Definition
Thereliabilitycoefficientofthequestionnaireisdefinedas:thevarianceofthetruevaluethevarianceofthemeasuredvalue
ordefinedas:
Thelarger,Indicatingthegreaterthecredibilityofthequestionnaire.
Disadvantages
Fromtheabovereliabilitycalculationformula,asitinvolvesthesumofsquaresoferrors,itisnecessarytorepeatmultipletimesforthesamesurveyobject.Measurement,onthebasisofestimatingthevaluesofand,thereliabilityestimatecanbeobtained.However,multipleretestswillbringaboutmemoryeffectandconnectioneffect,andwillcauseobjectiontothesurveyed,soitismoredifficulttoachieveinsurveyresearch.
Estimationofreliability
Beforeestimatingthereliabilityofthequestionnaire,itisnecessarytouseanappropriatescale(suchasLikertscale)toAllkindsofsubjectiveorobjectivealternativeanswersintheareconvertedintodigitalform,andthenthequestionnaireisscoredonthisbasis(includingindividualscores,groupscoresofrelatedtopics,andtotalscores,etc.).
Thecommonlyusedspecificmethodsofreliabilityanalysisincludetest-retestreliability,duplicatereliability,split-halfreliability,andreliabilitycoefficientmethods(thelattertwocanbeclassifiedasInternalconsistencyreliability).
Test-retestreliability
Test-retestreliability(test-retestreliability)isalsocalledtest-retestreliability.Itusesthesamequestionnaireforthesamegroupofrespondents.Thesurveywasconductedtwiceatdifferenttimepoints,andthedegreeofdifferencebetweenthetwosurveyresults.Thetest-retestreliabilityreflectstheinfluenceofrandomerrors.
Thesourceoftheerrorexaminedinthetest-retestreliabilityistherandominfluencebroughtaboutbythechangeoftime.Whenassessingthereliabilityofretests,wemustpayattentiontothetimebetweenretests.Forpersonalitytests,theintervalbetweenretestsisbetweentwoweeksandsixmonths.
Whenevaluatingtest-retestreliability,youshouldalsopayattentiontothefollowingtwoimportantissues:(1)Test-retestreliabilitygenerallyonlyreflectschangescausedbyrandomfactors,anddoesnotreflectlong-termchangesinsubjects’behavior.⑵Differentbehaviorsareaffecteddifferentlybyrandomerrors.
Disadvantage:Retestreliabilityhasadilemma.Shorteningthetimeintervalbetweentwotestsmakesiteasierforthesubjecttorecallthetestquestions;whileextendingthetimeintervalbetweenthetwotests,thesubjectismoresusceptibletochangesinexternalinfluences.
Parallel-formsReliability
Parallel-formsReliability,alsoknownasequivalencecoefficient,isakindofEquivalenceReliability,Referstothedegreeofvariationofthequestionnairesurveyresultscomparedtoanotherverysimilarquestionnairesurveyresult.Itusestwoquestionnaireswithequivalentcontentbutdifferenttopicsforthesamegroupofrespondents,andthencomparesthedegreeofcorrelationbetweenthetwosetsofdata.
Itismoreworkloadthanretestingreliability,becausethesamemeasurementtool(questionnaire,mentalscale,etc.)needstoconstructtwoequivalentcopies,andthetwocopiesshouldcontainthesameamount,Type,content,difficultyofthetopic.Toassessthereliabilityofacopy,twocopiesareusedtotestthesamegroupofsubjects,andthenthecorrelationcoefficientofthemeasurementscoresofthetwocopiesisestimated.Thelargerthecorrelationcoefficient,thesmallerthevariationcausedbythecompositionofthetwocopies.Thisisdifferentfromthevariationcausedbyconsideringtimeinthereliabilityofretestingstability.Thatistosay,thecorrelationcoefficientreflectsthedegreeofequivalenceofthemeasuredscores,sothereplicareliabilityisalsocalledtheequivalencereliability.
Themainadvantagesofreplicareliabilityare:⑴Itcanavoidsomeproblemsofretestreliability,suchasmemoryeffect,practiceeffect,etc.;⑵Suitableforlong-termfollow-upresearchorinvestigationofsomeinterferencevariablesonthetestAchievementimpact;⑶reducethepossibilityofcounselingorcheating.☆Thelimitationsofreplicareliabilityare:(1)Ifthemeasuredbehaviorissusceptibletopractice,thereplicareliabilitycanonlybereducedbutnoteliminated;(2)thenatureofsometestswillchangeduetorepetition;(3)someItisdifficulttofindasuitablecopyofthetest.
Internalconsistencyreliability
Itmainlyreflectstherelationshipbetweentheinternalquestionsofthetest,andexamineswhethereachitemofthetestmeasuresthesamecontentorcharacteristics.Internalconsistencyreliabilityisdividedintohalf-halfreliabilityandhomogeneityreliability.
1,Split-halfreliability(split-halfreliability)referstothedegreeofvariationintheresultsofthetwohalvesofthequestionnaireinasurvey.isthereliabilitycoefficientobtainedbydividingthetestintotwohalvesandcalculatingthecorrelationbetweenthetwohalvesofthetest.Thelongerthetest,thehigherthereliabilitycoefficient.ThecorrectionformulaistheSpearman-Brownformula.Spearman-Brownformulaisanempiricalformulaforcorrectingsplit-halfreliability(rtt=2rhh/(1+rhh)Generallyspeaking,ifthetestisdividedintotwohalves,thesplittingcoefficientrttis0.5)
Itshypothesisisthatthevarianceofthetestscoresinthetwohalvesareequal.Whenthehypothesisisnotvalid,oneoftheFlanaganformulaorthekulonformulacanbeusedtodirectlyobtainthereliabilitycoefficientofthetest.
2,Homogeneityreliabilityreferstotheextenttowhicheachiteminthetestexaminesthesamecontent.Whenthereliabilityofhomogeneityislow,eventhougheachtestitemseemstomeasurethesametrait,thetestisactuallyheterogeneous,thatis,thetestmeasuresmorethanonetrait.Thehomogeneityanalysisissimilartotheinternalconsistencyanalysisintheprojectanalysis.Severalformulasforcalculatinghomogeneityreliability:(1)Kudd-Richardsonformula(2)Cronbachalphacoefficient.Forsomecomplexandheterogeneouspsychologicalvariables,itisnotfeasibletouseasinglehomogeneitytest,soseveralrelativelyheterogeneoussubtestsareoftenused.
Raterreliability
referstotheconsistencyofdifferentratersinevaluatingthesameobject.Thesimplestestimationmethodistorandomlyselectanumberofanswersheets,scorethembytwoindependentraters,andthenfindthecorrelationcoefficientofthetwojudgescoresforeachanswersheet.Thecalculationofthiscorrelationcoefficientcanusetheproduct-differencecorrelationmethod,ortheSpearmanrankcorrelationmethod.
Therelationshipbetweenreliabilityandvalidity
Therearenotonlyobviousdifferencesbetweenthereliabilityandvalidityofthequestionnaire,buttherearealsomutualconnectionsandmutualconstraints.Reliabilitymainlyanswersquestionsaboutconsistency,stabilityandreliabilityofmeasurementresults;validitymainlyanswersquestionsaboutvalidityandcorrectnessofmeasurementresults.
TherelationshipbetweenvalidityandreliabilitycanbeunderstoodbytheformulaO=TSRthatconstitutesthemeasuredvalue.Ifthemeasurementiscompletelyvalid,thatis,0=T,S=0,R=0,themeasurementmustbecompletelycredibleatthistime.Ifthereliabilityofthescaleisinsufficient,itmaynotbecompletelyvalid,becauseO=TR.Ifthescaleiscompletelycredible,itcanbecompletelyeffective,oritmaynotbe,becausetheremaybeerrorsthatmayleadtoerrors.Althoughlackofreliabilitynecessarilylacksvalidity,thedegreeofreliabilitydoesnotreflectthedegreeofvalidity.Reliabilityisanecessaryconditionforvalidity,butnotasufficientcondition.Fromatheoreticalpointofview,thequantityshouldhavesufficientvalidityandreliability;fromapracticalpointofview,agoodscaleshouldalsobepractical.Practicalityreferstotheeconomy,convenienceandinterpretabilityofthescale.
Generallyspeaking,reliabilityisanecessaryconditionforvalidity,thatis,validitymustbeestablishedonthebasisofreliability;butthereisnomeasureofvalidity,evenifitsreliabilityishigh,Suchameasurementisalsomeaningless.Therelationshipbetweenreliabilityandvalidityhasthefollowingtypes:
①Credibleandeffective
Thiskindofquestionnaireaccuratelyreflectsthetrueattitudeofthesurveyedperson.ThequestioninthequestionnaireisCloselyrelatedtothetargetoftheinvestigation.ThissituationisshowninFigure8-8(a).Thesolidpoint(x,y)inthefigurerepresentsthetruesituationofthephenomenontobemeasured,andtheremainingpointsrepresentthemeasurementresultsobtainedafterinvestigation.Ifthesurveyresultscantrulyreflectthesurveyedobjectsandthemeasurementerrorissmall,itmeansthattheresultsofthequestionnairesurveyarecredibleandeffective.
②Crediblebutinvalid
Althoughtheresultsofthiskindofquestionnairecanaccuratelyreflectthetrueattitudesoftherespondents,thequestionnairehasaweakcorrelationwiththerealpurposeofthesurvey,Inconsistentwiththeobjectivesoftheinvestigation.AsshowninFigure8-8(b).Thissituationshowsthatalthoughtheresultsobtainedinthesurveyarecredible,theremaybeerrorsinsomelinks.Forexample,thedesignofthequestionsinthequestionnairecausedallthesurveyedpersonneltohaveadeviationinunderstanding,whichresultedinasystem.Sexualdeviation.
③Unreliableandinvalid
Inthiscase,theresultsofstatisticalsurveysarescattered,anditisdifficulttoobtaineffectiveresultsfromthequestionnaires.Thisisthemeasurementapplication.Typestoavoid.AsshowninFigure8-8(c).