Idatha enkulu, isoftware yamahhala nomthombo ovulekile: Izicelo Ezitholakalayo

Idatha enkulu ngumqondo wobuchwepheshe ohlobene nokuphathwa kwemininingwane emikhulu yedatha, ehlelekile futhi engahlelekile, njengamanje ephethwe imikhakha emikhulu yamabhizinisi, ezobuchwepheshe, ezesayensi kanye nohulumeni.

Yize lapho kukhulunywa ngakho Idatha enkulu, akuyona ngempela inani ledatha elibalulekile, kodwa ukuthi yiziphi izinhlangano ezenzayo ngedatha. Njengoba iBig Data, ubuchwepheshe obuhambisana nayo, ingawahlaziya ukuze ithole imibono eholela ekuthathweni kwezinqumo okungcono, ukunyakaza kanye namasu. Futhi kulesi sici, I-Free Software (SL) ne-Open Source (CA) zinikele kakhulu kulobu buchwepheshe, ngoba izinhlelo eziningi ezithuthukisiwe zenziwe ngale ndlela yokuthuthukisa.

Idatha enkulu ne-Free Software

Kulabo abanamakhono kwezobuciko, sekuvele kwaziwa kahle lokho Isoftware yamahhala, imodeli yayo yokuthuthuka, ifilosofi yayo, isuselwa ekwakheni ubuchwepheshe, ikakhulukazi imikhiqizo yesoftware, nayo engasetshenziswa, iguqulwe futhi isatshalaliswe ngokukhululeka. Futhi lowo Mthombo Ovulekile uyinto ebalulekile ekwakhiweni kwesoftware yamahhala, ngoba igxila ezinzuzweni ezibonakalayo zalokhu kuthuthukiswa okunamandla ngaphezu kwezimiso zenkululeko yomkhiqizo nezakhamizi.

Ngakho-ke, ngenkathi I-SL / CA inikela ngezindlela zokwenza i-Big Data, IBig Data igcwalisa lokhu ngokungaqondile, hhayi nje kuphela ukuze kuzuze ukwanda okusheshayo kwentuthuko yezobuchwepheshe, kodwa futhi nenkululeko yokufinyelela kulwazi iBig Data eletha nalo.

Yini idatha enkulu?

concepto

Kwenye yama-greats weSoftware nentuthuko yezobuchwepheshe, IBM, Idatha enkulu yile:

«... ubuchwepheshe obuvule iminyango yendlela entsha yokuqonda nokuthatha izinqumo, esetshenziselwa ukuchaza inani elikhulu lemininingwane (ehlelekile, engahlelekile kanye nehleleke kancane) engathatha isikhathi esiningi futhi ebiza kakhulu layisha ku-database yobudlelwano yokuhlaziywa.

Inhloso

IBig Data, ubuchwepheshe bayo, yazalwa ngenhloso yokuhlanganisa yonke imininingwane yokuhlaziywa kwedatha, okungukuthi, ukumboza kokubili okukhona nokuxazululwa ngobuchwepheshe bamanje nobwehlukile, kanye nalokho okungaxazululwa ubuchwepheshe obukhona, njengokuthi ukugcinwa nokuphathwa kwemininingwane emikhulu yedatha ezinezici ezithile.

Idatha

Idatha ye-Bid iphatha amavolumu wedatha avame ukuchazwa ngezici ezilandelayo:

• Ivolumu: Usayizi wedatha kusuka emithonjeni eminingi.
• Ijubane: Ijubane lapho idatha evela emithonjeni eminingi ifika futhi iphethwe.
• Okunhlobonhlobo: Ifomethi yedatha ehlaziyiweyo evela emithonjeni eminingi.

Ngisho, Imiqulu yedatha evame ukwakheka ngedatha eyakhiwe, eyakhiwe kancane, futhi engahleliwe, futhi iphathwe ngobuningi obukhulu obuvame ukuchazwa ngeziqalo zamanani aphezulu, njenge: Tera, Peta noma i-Exa, phakathi kwabanye.

Futhi kuzo zonke izinhlobo zemithombo, njenge-Intanethi (Amanethiwekhi omphakathi, i-Digital Media, amawebhusayithi nemininingwane), Amaqembu (Omakhalekhukhwini, abadlali beMultimedia, amaPositioning system, ama-Civil and industrial sensors, phakathi kwabanye) kanye Izinhlangano (Ezizimele nezikaHulumeni, ezezoHwebo, uHulumeni kanye noMphakathi).

Ukubaluleka

Yini eyenza iBig Data ibe ubuchwepheshe obusebenziseka kangako ezinhlanganweni (Ezizimele Nezomphakathi, Ezentengiselwano, Uhulumeni Nomphakathi), iqiniso lokuthi linikeza imininingwane ebalulekile lokho kuvame ukusebenza njengempendulo enembile nethembekile emibuzweni engakaze ibuzwe ngezimo ezithile noma izinkinga. Lokho wukuthi, ukuba wusizo kwalo kuvame ukubonakala kuzici ezivame ukuvela kulwazi olufanayo oluqoqiwe futhi lwaphathwa.

Ukucutshungulwa kwemininingwane emikhulu kwenza kube lula ukuthi idatha esetshenzisiwe yakhiwe noma ihlolwe ngendlela efanelekile. noma icacisa, lokho kuthathwa njengokufanele ngumlawuli wayo. Lokhu kuvumela izinhlangano ezisebenzisa i-Big Data ukuthi zikwazi ukubona izinkinga ngendlela eqondakala kakhudlwana.

Ukuqoqwa kwemininingwane emikhulu yedatha kanye nokuhlaziywa kwayo okulandelayo ukusesha izitayela ngaphakathi kwazo kuvumela Izinhlangano ukuthi zisebenze kahle futhi zisebenze kahle, ngokuhamba ngokushesha okukhulu, ngokushelela nangesikhathi esifanele phezu kwazo. Ngaphezu kwalokho, ibavumela ukuthi basuse izindawo ezinenkinga ngaphambi kokuba izinkinga zibathole, zibenze balahlekelwe yizinzuzo, idumela noma ukwesekwa.

Izinzuzo

Idatha Enkulu isiza Izinhlangano ukuphatha idatha yazo kangcono kakhulu, lokhu kuphumela ekuhlonzweni kwamathuba amasha amahle noma akhiqizayo kumalungu azo (amaklayenti noma izakhamizi). Futhi lokhu nakho kuholela ezenzweni ezihlakaniphile nezisebenza kahle, ukonga amahora / umsebenzi nemali, okuvame ukuhumusha kube injabulo kubo bonke abathintekayo. Lapho kusetshenziswa i-Big Data, inani livame ukungezwa emisebenzini eyenziwa ngalezi zindlela ezilandelayo:

• Ukunciphisa izindleko: Ekugcinweni nasekuphathweni kwemininingwane emikhulu yedatha.
• Ukunciphisa isikhathi: Ukusebenza kahle nokusebenza ngempumelelo ekuthathweni kwezinqumo.
• Imikhiqizo nezinsizakalo ezintsha: Nekhono lokukala nokulindela izidingo nezinkinga zabasebenzisi (amaklayenti kanye / noma izakhamizi), ukwaneliseka kwabo kuyakhula.

Izinzuzo

Idatha enkulu esetshenziswe kahle imvamisa iyakwazi ukunquma izimbangela zokwehluleka, izinkinga nokukhubazeka cishe ngesikhathi sangempela. Noma kunjalo, kucatshangelwa lokho Ubuchwepheshe beBig Data abuyona ipanacea uqobo. Ngakho-ke icaphuna obunye ubuchwepheshe obukhulu njenge Oracle, kungangezwa ukuthi:

«Ukuhlonza inani ledatha enkulu akusho nje ukuyicubungula (osekuvele kuyinzuzo uqobo). Kuyinkqubo yonke yokutholwa edinga abahlaziyi, abasebenzisi bebhizinisi nabaphathi ukuba babuze imibuzo efanele, bakhombe amaphethini, benze izinqumo ezinolwazi futhi babikezele izindlela zokuziphatha.

Izicelo ze-SL / CA zeDatha Enkulu

Phakathi kwezinhlelo zeSoftware Emahhala kanye Nomthombo Ovulekile okufanele ukushiwo ngocwaningo, ukuhlolwa, nokuqaliswa kukhona:

Okuhlobene

• I-Apache Hadoop: Ipulatifomu yomthombo ovulekile eyenziwe ngeHadoop Distributed File System (HDFS), iHadoop MapReduce, neHadoop Common.
• I-Avro: Iphrojekthi ye-Apache enikezela ngezinsizakalo ze-serialization.
• UCassandra: Kusatshalaliswe i-database engahlobene nobudlelwano ngokuya ngemodeli yokugcina ye , yathuthukiswa eJava.
• Chukwa: Isoftware eyenzelwe ukuqoqwa okukhulu nokuhlaziywa kwamalogi omcimbi.
• I-Flume: Isoftware enomsebenzi wayo omkhulu ukuqondisa idatha kusuka komunye umthombo iye kwenye indawo.
• I-HBase: I-database yekholomu (i-database egxile kwikholomu) esebenza ku-HDFS.
• Isidleke: Ingqalasizinda ye- "Data Warehouse" esiza ukuphathwa kwemininingwane emikhulu egcinwe endaweni esatshalalisiwe.
• UJaql: Ulimi olusebenzayo nolwehlisayo oluvumela ukuxhashazwa kwedatha ngefomethi ye-JSON eyenzelwe ukucubungula imiqulu emikhulu yolwazi.
• ULucene: Isoftware enikezela ngemitapo yolwazi yokukhomba nokusesha kumbhalo.
• I-Oozie: Iphrojekthi yomthombo ovulekile eyenza ukuhamba komsebenzi kuhambisane kahle phakathi kwenqubo ngayinye.
• Ingulube: Isoftware evumela abasebenzisi be-Hadoop ukuthi bagxile kakhulu ekuhlaziyeni wonke amasethi wedatha futhi basebenzise isikhathi esincane ukwakha izinhlelo zeMepReduce.
• I-ZooKeeper: Ingqalasizinda emaphakathi kanye nezinsizakalo ezingasetshenziswa izinhlelo zokusebenza ukuqinisekisa ukuthi izinqubo zeqoqo ngalinye zenziwa nge-serial noma zavumelaniswa.

Ezimele

Ezinye ziyaziwa, kepha azihlobene nepulatifomu yomthombo ovulekile iHadoop yilezi:

• I-Elasticsearch: Injini yokusesha nokuhlaziya okusekelwe ngokuphelele embhalweni.
• I-MongoDB: Idatha ye-NoSQL ngokuya ngemodeli yedatha yedokhumenti.
• UCassandra: Iphrojekthi yomthombo ovulekile eyenzelwe ukuphathwa kwedatha ye-NoSQL.
• I-CouchDB: Idatha evulekile yomthombo we-NoSQL ngokususelwa kuzindinganiso ezivamile zokutholakala kalula nokuhambisana kwewebhu ngokuhlukahluka.
• I-Solr: Injini yokusesha evulekile ngokususelwa kulabhulali yephrojekthi yeLucene Java.
  Amanye amathuluzi we-RDBMS: I-MySQL Cluster neVoltDB.

Isiphetho

Isikhathi sethu samanje (futhi esiseduze ngokulandelayo) sicwiliswa noma sicwiliswe kunqwaba yedatha ekhulayo ekhulayo, enokuningi kokukusho kukonke, kunokukodwa. Ngakho-ke, ukusetshenziswa kobuchwepheshe beBig Data njengamanje nakusasa oluzosiza umphakathi, sonke isintu, ukuthola ubuningi bezinto (izehlakalo noma izinto ezisunguliwe), obekungathatha iminyaka eminingi ukuzitholela zona, ngaphandle kokusebenzisa kwalokhu.

Kusukela Idatha enkulu namathuluzi ayo ahlinzeka ngesivinini esanele sokuhlaziya hlaziya umphumela otholwe ngokushesha uphinde uwusebenzise kaningi ngangokunokwenzeka, ngesikhathi esifushane, ukuthola inani leqiniso noma eliseduze kakhulu ozama ukufinyelela kulo. Uma uthole isihloko se-Big Data sithakazelisa, unganweba isihloko kancane ngokufunda lo Mbiko BBVA.