Analytics Lie
Kusarura Kwekuongorora
Mark Twain akataura chimwe chinhu chakafanana, "Kune marudzi matatu enhema: nhema, nhema dzakarambwa uye nhema. analytics. "
Isu tinotora sezvinei kuti analytics inotipa ruzivo runobatsira, rwunoita. Chatisingawanzo ziva ndechekuti kusarerekera kwedu pachedu uye kwevamwe kunopesvedzera sei mhinduro dzatinopihwa nesoftware nehurongwa hwakanyanya. Dzimwe nguva, tinogona kunyengedzwa nekusavimbika, asi, kazhinji, zvingave zvisizvo uye zvisizvo zvekusaziva zvinopinda mukuongorora kwedu. Iko kukurudzira kuseri kwe biased analytics kwakawanda. Dzimwe nguva mhedzisiro isingasaruri yatinotarisira kubva kusainzi inopesvedzerwa ne1) sarudzo dzisiri dzemabatirwo anoitwa data, 2) data risingawirirani kana risiri remumiriri, 3) maitiro eAI akadzidziswa, 4) kusaziva, kusagona kwevanoongorora kana vamwe vanoedza. kutaura nyaya, 5) kuongorora pachayo.
Mharidzo Yakarerekera
Mamwe emanyepo ari nyore kuona pane mamwe. Kana iwe uchiziva zvekutsvaga iwe unogona kuona zviri nyore zvinogona kuitika magirafu nemachati anotsausa.
Kune zvirinani nzira shanu dzekurasisa data: 1) Ratidza shoma data seti, 2). Ratidza hukama husina hukama, 3) Ratidza data zvisizvo, 4) Ratidza data zvisiri izvo, kana 5). Ratidza data yakareruka.
Ratidza data shoma
Kudzikamisa data, kana ruoko kusarudza chikamu chisiri-random che data chinogona kazhinji kutaura nyaya isingaenderane nemufananidzo mukuru. Sampling yakaipa, kana kunhonga cherry, ndipo apo muongorori anoshandisa muenzaniso usiri mumiriri kumiririra boka rakakura.
Muna March 2020, Dhipatimendi reGeorgia reHutano Hwevanhu yakaburitsa chati iyi sechikamu chemushumo wemamiriro ezuva nezuva. Chaizvoizvo inomutsa mibvunzo yakawanda kupfuura yainopindura.
Chimwe chezvinhu chisipo inyaya. Semuenzaniso, zvingave zvinobatsira kuziva kuti chikamu chehuwandu ndechei yezera rega rega. Imwe nyaya ine-inotaridzika-inotaridzika pie chati ndeye mazera asina kuenzana. Iyo 0-17 ine makore gumi nemasere, 18-18 ine 59, 42+ yakashama yakapera, asi ine makore anosvika makumi mana. Mhedziso, yakapihwa iyi chati chete, ndeyekuti nyaya zhinji dziri muboka rezera re60-40. Boka rezera remakore makumi matanhatu+ rinotaridzika kunge risinganyanyi kukanganiswa nenyaya dzeCOVID. Asi iyi haisiriyo nyaya yacho yese.
Kuenzanisa, iyi data yakasiyana-siyana yakaiswa pa CDC webhusaiti machati eCCID kesi neboka rezera rine data rekuwedzera pahuwandu hwehuwandu hweUS huri muzera rega rega.
Izvi zviri nani. Tine mamwe mamiriro. Tinogona kuona kuti mapoka ezera 18-29, 30-39, 40-49 vose vane chikamu chepamusoro chezviitiko pane chikamu chezera muhuwandu. Pachine mamwe mapoka ezera asina kuenzana. Sei 16-17 iri boka rezera rakasiyana? Zvakadaro iyi haisiriyo nyaya yese, asi pundits vakanyora makoramu, vakaita fungidziro uye nemirairo pane zvishoma pane izvi. Zviripachena, neCOVID, kune akawanda akasiyana mukuwedzera kuzera anokanganisa kuverengerwa seyakanaka nyaya: mamiriro ekudzivirira, kuwanikwa kwebvunzo, nhamba yenguva dzakaedzwa, comorbidities, uye zvimwe zvakawanda. Nhamba yezviitiko, pachayo, inopa mufananidzo usina kukwana. Nyanzvi zhinji dzinotarisawo Huwandu hwevakafa, kana zvikamu zverufu pa100,000 vanhu, kana kesi-zvinofa kutarisa kuti COVID inobata sei zera rega rega.
Ratidza hukama husina hukama
Zviripachena, pane a kuwirirana kwakasimba pakati pekushandisa kweUS pasainzi, nzvimbo, uye tekinoroji uye huwandu hweKuzviuraya nekurembera, kudzipwa uye kudzipwa. Iyo Correlation ndeye 99.79%, ingangoita mutambo wakakwana.
Ndiani, kunyanguvezvo, angapa mhaka yokuti izvi zvine hukama neimwe nzira, kana kuti chimwe chinokonzera chimwe? Kune mimwe mienzaniso isinganyanyi kunyanyisa, asi isiri yemanyepo. Pane kuwirirana kwakasimba kwakafanana pakati peTsamba muKuhwina Shoko reScripps National Spelling Bee uye Nhamba Yevanhu Vakaurayiwa neVenomous Spider. Sangana? Iwe sarudza.
Imwe nzira yekuchata iyi data inogona kunge isiri kurasisa ingave yekubatanidza zero pane ese maY-axes.
Ratidza data zvisiri izvo
From Maonero Akashata Data, Nyika yeGeorgia yeUS yakapa Madunhu mashanu epamusoro ane Huwandu hukuru hweVakasimbiswa COVID-5 Nyaya.
Zvinotaridzika zviri pamutemo, handiti? Pane zvakajeka maitiro ekudzika kweakasimbiswa COVID-19 kesi. Unogona kuverenga X-axis? X-axis inomiririra nguva. Kazhinji, mazuva anozowedzera kubva kuruboshwe kuenda kurudyi. Pano, tinoona nguva shoma yekufamba pane X-axis:
4/28/2020
4/27/2020
4/29/2020
5/1/2020
4/30/2020
5/4/2020
5/6/2020
5/5/2020
5/2/22020 ...
Wait? Chii? X-axis haina kurongeka zvichitevedzana. Saka, sezvakanaka semaitiro angataridzika, isu hatigone kutora chero mhedziso. Kana mazuva akarongedzerwa, mabhawa ehuwandu hwemakesi anoratidza akawanda eiyo sawtooth pateni pane chero mhando yemaitiro.
Kugadzirisa kuri nyore apa ndiko kuronga misi semaitiro anoita karenda.
Ratidza data zvisina kujairika
Tese takabatikana. Uropi hwedu hwakatidzidzisa kuita kukurumidza kutonga kwakavakirwa pafungidziro dzave dzichienderana munyika yedu. Semuyenzaniso, girafu rega rega randati ndaona rinoratidza max- uye y- axes kusangana pazero, kana kukosha kwakaderera. Uchitarisa ichi chati muchidimbu, ndedzipi mhedziso dzaungatora nezve mhedzisiro yeFlorida “Mirai mutemo wenyu.”? Ndinonyara kuzvibvuma, asi iyi girafu yakandinyengedza pakutanga. Ziso rako rinokweverwa zviri nyore kune zvinyorwa uye museve pakati peiyo graphic. Pasi riri kumusoro mugirafu iri. Inogona kunge isiri nhema - iyo data iri ipapo. Asi, ndinofanira kufunga kuti zvinoreva kunyengera. Kana usati wazviona, zero pa y-axis iri pamusoro. Saka, sezvo data richidzika pasi, zvinoreva kufa kwakawanda. Chati ichi chinoratidza kuti huwandu hwemhondi dzinoshandisa pfuti kuwedzera mushure me2005, zvichiratidzwa nemaitiro ari kuenda pasi.
Ratidza data yakareruka
Mumwe muenzaniso wekuwedzeresa-kurerutsa data unogona kuoneka kana vaongorori vakatora mukana weSimpson's Paradox. Ichi chiitiko chinoitika kana data rakaunganidzwa richiratidza mhedziso yakasiyana pane painopatsanurwa kuita zvikamu zviduku. Musungo uyu uri nyore kuwira mauri kana uchitarisa pachikamu chepamusoro-soro. Imwe yemifananidzo yakajeka yeSimpson's Paradox kubasa ine hukama ne kurova maavhareji.
Pano tinoona kuti Derek Jeter ane avhareji yekurova yakakwira kupfuura David Justice yemwaka wa1995 na1996. Iyo gangaidzo inouya patinoona kuti Justice akakunda Jeter mukurova avhareji yemakore ese iwayo. Kana iwe ukanyatsotarisa, zvine musoro paunoona kuti Jeter anga aine 4x akawanda at-bats (the denominator) muna 1996 paavhareji yakaderera ye .007 muna 1996. Asi, Justice yakanga iine 10x nhamba ye-bats chete . 003 yepamusoro avhareji muna 1995.
Mharidzo yacho inotaridzika yakatwasuka, asi Simpson's Paradox, nekuziva, kana nekusaziva, yatungamira kune zvisiri izvo. Munguva pfupi yapfuura, kwave nemienzaniso yeSimpson's Paradox munhau uye pasocial media ine chekuita nekudzivirira uye kufa kweCOVID. Poshi chati inoratidza mutsara wegirafu inofananidza kufa pakati pevakabaiwa nevasina kubayiwa kune vanhu vane makore 10-59 okuberekwa. Chati inoratidza kuti vasina kubayiwa vanogara vaine mwero wakaderera wekufa. Chii chiri kuitika pano?
Nyaya yacho yakafanana neyatinoona ine maavhareji ekurova. Denominator munyaya iyi nhamba yevanhu vari muzera rega rega. Girafu inosanganisa mapoka ane mhedzisiro dzakasiyana. Kana tikatarisa boka rezera rekare, 50-59, zvakasiyana, tinoona kuti vaccine inofamba zviri nani. Saizvozvo, kana tikatarisa pa10-49, tinoona zvakare kuti nhomba inofamba zviri nani. Zvinoshamisa, kana uchitarisa pane zvakasanganiswa seti, isina kubayiwa inoratidzika kunge ine mhedzisiro yakaipa. Nenzira iyi, iwe unokwanisa kuita nyaya yekupokana kwakapokana uchishandisa iyo data.
Iyo Data iri Bias
Data haigoni kuvimbwa nguva dzose. Kunyange munharaunda yesayenzi, vanopfuura chikamu chimwe muzvitatu chevatsvakurudzi vakaongororwa vakabvuma kudaro "maitiro ekutsvaga asina chokwadi." Another tsvakurudzo yekubiridzira mutikitivha anoti, "Panogona kunge paine hutsotsi hwakanyanya mudata - matafura, magirafu emutsara, kutevedzana kwedata [- kupfuura zvatiri kuwana]. Chero ani akagara patafura yavo yekubikira anogona kuisa dzimwe nhamba muspredishiti uye kugadzira girafu remutsara rinotaridzika kuva rinogutsa. "
Ichi chokutanga muenzaniso zvinoita kunge mumwe munhu akaita izvozvo. Handisi kuti uhu hutsotsi, asi sewongororo, haingobudise chero data rinobatsira kuita sarudzo ine ruzivo. Zvinotaridza senge ongororo yakabvunza vakapindura nezve maonero avo ekofi yekutepfenyura gasi, kana chimwe chiitiko chazvino chakakodzera.
- Murovedzi
- Hukuru
- Kugona chose
Ini ndakadzvanya iyo Twitter positi kuti ndibvise mareferensi kune ane mhosva, asi iyi ndiyo chaiyo chaiyo chati yemhedzisiro yeongororo. Ongororo dzakadai hadzina kujairika. Zviripachena, chero chati yakagadzirwa kubva kune data inokonzerwa nemhinduro icharatidza kofi iri mubvunzo haifanirwe kupotsa.
Dambudziko nderekuti dai wakapihwa ongororo iyi ukasawana mhinduro inoenderana nemafungiro ako, waidarika ongororo iyi. Uyu unogona kunge uri muenzaniso wakanyanyisa wekuti data risingavimbike rinogona kugadzirwa sei. Magadzirirwo asina kunaka eongororo, zvisinei, anogona kutungamira kune mashoma mhinduro uye avo vanopindura vane pfungwa imwe chete, ingori nyaya yedhigirii. Iyo data yakarerekera.
Uyu wechipiri muenzaniso wekurerekera kwedata unobva kumafaira e "Yakaipisisa COVID 19 Inotsausa Girafu. "
Zvakare, izvi zvinyengeri uye hazvisi pachena zvachose. Iyo bar graph inoratidza yakatsetseka - yakatonyanya kutsetseka - kuderera muhuwandu hweakanaka COVID-19 makesi nekufamba kwenguva kune County muFlorida. Unogona kugumisa zviri nyore kuti nyaya dziri kudzikira. Izvo zvakanaka, iyo yekuona inomiririra chaiyo data. Dambudziko riri mu data. Saka, inyaya inonyengera nekuti hauzvione. Yakabikwa mune data. Mibvunzo yaunoda kubvunza, inosanganisira, ndiani ari kuyedzwa? Nemamwe manzwi, chii chinonzi denominator, kana huwandu hwevanhu hwatiri kutarisa kubva muzana. Fungidziro ndeyokuti ihuwandu hwevanhu vose, kana kuti, mumiririri wemuenzaniso.
Nekudaro, panguva iyi, mudunhu iri, bvunzo dzaingopihwa kune vashoma vanhu. Vaifanira kuve neCCIDID-senge zviratidzo, kana kuti vaive vafamba nguva pfupi yadarika kuenda kune imwe nyika pane rondedzero yenzvimbo dzinopisa. Pamusoro pezvo zvinovhiringa zvabuda inyaya yekuti bvunzo yega yega yakaverengerwa uye yega yega bvunzo yakaverengerwa. Kazhinji, kana munhu aonekwa kuti ane hutachiona, vaiyedza zvakare kana hutachiona hwapera uye vaizoonekwa kuti hauna. Saka, neimwe nzira, kune yega yega yakanaka kesi, pane negative test kesi inodzima. Ruzhinji rwebvunzo harwunasi uye ongororo dzemunhu wega wega dzakaverengwa. Iwe unogona kuona kuti iyo data yakarerekera sei uye isinganyanyi kubatsira pakuita sarudzo.
AI Kupinza uye Kudzidzira kwakarerekera
Pane dzinenge nzira mbiri idzo AI inogona kutungamira kune yakarerekera mhedzisiro: kutanga neyakarerekera data, kana kushandisa yakarerekera algorithms kugadzirisa data rakavimbika.
Biased Input
Vazhinji vedu tiri pasi pefungidziro yekuti AI inogona kuvimbwa nayo kudzvanya manhamba, kushandisa maalgorithms ayo, uye kupfira ongororo yakavimbika yedata. Artificial Intelligence inogona chete kuve yakangwara sekudzidziswa kwayakadzidziswa. Kana iyo data iyo yakadzidziswa isina kukwana, mhedzisiro kana mhedziso hazvigone kuvimbwa, kana. Zvakafanana nenyaya iri pamusoro yekusarudzika kweongororo, kune akati wandei nzira dzekuti data rive zvakashata mukudzidza muchina:.
- Sample bias - dhata rekudzidzisa harimiriri ruzhinji rwevanhu.
- Kusarerekera - dzimwe nguva izvo zvinoita kunge zvekunze zvinonyatso shanda, kana, kwatinodhirowa mutsara pane zvekusanganisira (zip codes, misi, nezvimwewo).
- Measurement bias - gungano ndeyekugara uchiyera kubva pakati uye pasi pe meniscus, semuenzaniso, pakuyera zvinwiwa mu volumetric flasks kana test tubes (kunze kwe mercury.)
- Rangarira rusaruro - kana tsvakiridzo inoenderana nekurangarira kwevatori vechikamu.
- Kusarura - masayendisiti, sevanhu vese, vanofarira kuona zvavanotarisira kuona.
- Rusaruro rwepabonde uye rusaruraganda - bonde kana rudzi runogona kunge rwakanyanya- kana kumiririrwa zvishoma.
- Kusarurana - iyo data inosimbisa stereotypes
Kuti AI idzose mhedzisiro yakavimbika, data rayo rekudzidzisa rinoda kumiririra nyika chaiyo. Sezvatakakurukura mune yakapfuura blog chinyorwa, kugadzirira kwedata kwakakosha uye senge chero imwe data data. Data isingavimbike inogona kudzidzisa muchina kudzidza masisitimu chidzidzo chisiri icho uye zvinozoguma nemhedzisiro isiriyo. Izvo zvakati, "Data rese rakarerekera. Iyi haisi paranoia. Ichi ndicho chokwadi. " - Dr. Sanjiv M. Narayan, Stanford University Chikoro cheMishonga.
Kushandisa data yakarerekera pakudzidziswa kwakonzera kuwanda kwekutadza kweAI. (Mienzaniso pano uye pano, research pano..)
Biased algorithms
An algorithm seti yemitemo inogamuchira yekupinza uye inogadzira inobuda kupindura bhizinesi dambudziko. Ivo kazhinji vanonyatsotsanangurwa sarudzo miti. Algorithms inonzwa kunge mabhokisi matema. Hapana ane chokwadi chekuti vanoshanda sei, ehe, kunyangwe ivo makambani anoashandisa. Oo, uye ivo kazhinji ndevemuridzi. Chimiro chavo chisinganzwisisike uye chakaomarara ndicho chimwe chezvikonzero nei biased algorithms iri hunyengeri. .
Funga nezveAI algorithms mumushonga, HR kana mari inotora nhangemutange mukutarisisa. Kana rudzi rwuri chinhu, iyo algorithm haigone kuve mapofu erudzi. Izvi hazvisi zvekufungidzira. Matambudziko akaita seawa akawanikwa munyika chaiyo uchishandisa AI mukati mubayiro, ride-share, chikwereti chikumbiros, uye itsvo transplants.
Chinokosha ndechekuti kana data rako kana maalgorithms akaipa, akaipa kupfuura asina maturo, anogona kunge ane ngozi. Pane chinhu chinonzi "algorithmic kuongororwa.” Chinangwa ndechekubatsira masangano kuona njodzi dzinogona kuitika dzine chekuita neiyo algorithm sezvo ine chekuita nekururamisira, kurerekera uye kusarura. Kumwewo, Facebook iri kushandisa AI kurwisa kusarerekera muAI.
Vanhu vakarerekera divi rimwe chete
Tine vanhu kumativi maviri equation. Vanhu vari kugadzirira ongororo uye vanhu vari kugamuchira ruzivo. Kune vaongorori uye kune vaverengi. Mune chero kutaurirana, panogona kuve nematambudziko mukutapurirana kana kugamuchira.
Somuenzaniso, tora mamiriro okunze. “Mukana wemvura” unorevei? Chekutanga, vanorevei nezvemamiriro ekunze kana vachiti kune mukana wekunaya kwemvura? Maererano nehurumende yeUS National Weather Service, mukana wekunaya kwemvura, kana kuti yavanodaidza kuti Probability of Precipitation (PoP), chimwe chezvinhu zvisinganzwisisike muchimiro chemamiriro ekunze. Iyo ine tsananguro yakajairwa: "Mukana wekunaya kwemvura ingori nhamba yehuwandu hwe 0.01" inch [sic] ye [sic] yakawanda yekunaya panzvimbo yakapihwa munzvimbo yakapihwa yekufanotaura munguva yakatarwa." "Nzvimbo yakapihwa" inzvimbo yekufanotaura, kana broadcast area. Izvi zvinoreva kuti iyo yepamutemo Mukana weKunaya kunoenderana nekuvimba kuti kunaya kune imwe nzvimbo munzvimbo uye muzana yenzvimbo inonaya. Mune mamwe mazwi, kana meteorologist ane chivimbo chekuti kuchanaya munzvimbo yekufanotaura (Kuvimba = 100%), ipapo PoP inomiririra chikamu chenzvimbo ichawana mvura.
Paris Street; Zuva Rokunaya,Gustave Caillebotte (1848-1894) Chicago Art Institute Public Domain
Mukana wekunaya kwemvura unoenderana nezvose kuvimba uye nzvimbo. Handina kuzviziva. Ndinofungidzira kuti vamwe vanhu havazvizivewo. Vanosvika zvikamu makumi manomwe neshanu kubva muzana (75%) yevanhu havanyatsonzwisisa kuti PoP inoverengwa sei, kana kuti inomiririrei. Saka, tiri kunyengedzwa here, kana, iri idambudziko rekuona. Ngatitiidze kunaya kwekuona. Isu tinopomera muongorori wemamiriro ekunze here? Kutaura chokwadi, pane zvimwe kuvhiringidzika pakati pevanofanotaura nezvemamiriro ekunze, zvakare. Mune imwe tsvakurudzo, 43% yevanoongorora mamiriro ekunze vakaongororwa vakati pane kuenderana kushoma mutsanangudzo yePoP.
Kuongorora Kwacho Kwakarerekera
Pakati pezvinhu zvishanu zvinofurira, kuongorora pachayo kungave kwakanyanya kushamisa. Mukutsvagisa kwesainzi kunokonzeresa kuti bepa rakadzokororwa ribudiswe, kazhinji dzidziso inofungidzirwa, nzira dzinotsanangurwa kuyedza iyo hypothesis, data inounganidzwa, uye data inoongororwa. Rudzi rwekuongorora kunoitwa uye kuti rinoitwa sei harikosheswi mumabatiro azvinoita mhedziso. Mune a bepa rakaburitswa kutanga kwegore rino (Ndira 2022), muInternational Journal yeCancer, vanyori vakaongorora kana mhedzisiro yezviyedzo zvakadzorwa zvakadzorwa uye zvidzidzo zvekuona zvekare. Zvavakawana zvakazoti,
Nekusiyanisa sarudzo dzekuongorora mukuenzanisa nekuita tsvakiridzo, takagadzira zvinopesana. Zvigumisiro zvedu zvinoratidza kuti zvimwe zvidzidzo zvekuongorora zvekare zvinogona kuwana kurapwa kunovandudza migumisiro yevarwere, asi imwe tsvakurudzo yakafanana inogona kuiwana haina, inongobva pane zvisarudzo zvekuongorora.
Munguva yakapfuura, kana uchiverenga chinyorwa chejenari resainzi, kana wakafanana neni, unogona kunge wakafunga kuti mhedzisiro kana mhedziso zvese nezve data. Iye zvino, zvinoratidzika kuti migumisiro, kana kuti yekutanga hypothesis yakasimbiswa kana kurambwa inogonawo kuvimba nenzira yekuongorora.
Another kudzidza akawana mibairo yakafanana. The article, Vazhinji Vaongorori, Imwe Dhata Seti: Kuita Kujeka Sei Kusiyana kweSarudzo dzeAnalytics Kunobata Mhedzisiro, inotsanangura kuti vakapa sei data rakafanana seti kune 29 zvikwata zvakasiyana kuti zviongorore. Kuongororwa kwedata kunowanzoonekwa seyakasimba, yakanyatsotsanangurwa nzira inotungamira kune imwe mhedziso.
Zvisinei nekutsiura kwevemethothodologist, zviri nyore kufuratira chokwadi chekuti mhedzisiro inogona kutsamira pane yakasarudzwa yekuongorora nzira, iyo pachayo yakazadzwa nedzidziso, fungidziro, uye sarudzo dzesarudzo. Muzviitiko zvakawanda, kune akawanda anonzwisisika (uye akawanda asina musoro) maitiro ekuongorora data rine chekuita nemubvunzo wekutsvagisa.
Vatsvakurudzi vakaungana-vakatsvaga kuongororwa kwedata uye vakasvika kumhedziso yokuti tsvakurudzo yose inosanganisira zvisarudzo zvepfungwa - kusanganisira rudzi rwekuongorora rwekushandisa - runogona kukanganisa mhedzisiro yechidzidzo.
Kurudziro yeumwe muongorori uyo akaongorora chidzidzo chiri pamusoro apa kuve akangwarira pakushandisa bepa rimwechete pakuita sarudzo kana kuita mhedziso.
Kugadzirisa Bias muAnalytics
Izvi zvinongorehwa kuti ive yambiro ngano. Ruzivo runogona kutidzivirira kuti tisabatwa nechitsotsi. Kunyanya kuziva nzira dzingashandiswa nescanner kutinyengedza, kashoma kuti tipinde mukati, toti, ne, toti, kukanganisa kwemuhomwe, kana kutaura kwakanaka kwemutambo wePonzi. Ndozvazvakaita nekunzwisisa uye nekuziva zvinogona kurerekera zvinokanganisa analytics yedu. Kana tichiziva zvingangoitika, tingakwanisa kupa nyaya yacho zvirinani tozoita sarudzo dziri nani.
