I-Apache Storm yiphrojekthi ekuvumela ukuthi uhlele ukucubungula kuqinisekisiwe yemicimbi eyahlukahlukene ngesikhathi sangempela. Isibonelo, iSiphepho ingasetshenziselwa ukuhlaziya ukusakazwa kwedatha ngesikhathi sangempela, yenza imisebenzi yokufunda komshini, ukuhlela ukubala okuqhubekayo, ukusebenzisa i-RPC, i-ETL, njll.
Uhlelo isekela ukuhlanganiswa, lukwakha ukucushwa okubekezelela amaphutha, imodi yokucubungula idatha eqinisekisiwe, futhi inokwenyuka okuphezulu ngokwanele ukucubungula izicelo ezingaphezu kwesigidi ngomzuzwana endaweni eyodwa yeqoqo.
Ukuhlanganiswa kwe-Apache Storm nezinhlelo ezahlukahlukene zokucubungula ulayini kanye nobuchwepheshe be-database.
Ukwakhiwa kwesiphepho kufaka phakathi ukuthola nokucubungula imifudlana yedatha engahlelekile futhi ivuselelwa njalo kusetshenziswa izilawuli eziyinkimbinkimbi ezinamandla okuhlukanisa phakathi kwezigaba zokubala ezihlukile.
Mayelana ne-Apache Storm
Le phrojekthi yadluliselwa emphakathini wama-Apache ngemuva kokutholwa kweTwitter ngabakwaBackType, okuyinkampani eyasungula uhlaka ekuqaleni.
Ngokwenziwayo, iSiphepho sasetshenziswa ku-BackType ukuhlaziya ukubonakaliswa kwemicimbi kuma-microblogs, ngokuqhathanisa ama-tweets amasha endizeni nasezixhumanisweni ezazisetshenziswa kuzo (ngokwesibonelo, zahlolwa njengezixhumanisi zangaphandle noma izikhangiso ze-Twitter zazisakazwa ngabanye ababambiqhaza).
Ukusebenza kwesiphepho kuqhathanisa nesikhulumi se-Hadoop, futhi umehluko omkhulu ukuthi idatha ayifakwa endaweni yokugcina izinto, kepha itholwa ivela ngaphandle futhi icutshungulwe ngesikhathi sangempela.
KuStorm, asikho isendlalelo sesitoreji esakhelwe ngaphakathi futhi umbuzo wokuhlaziya uqala ukusebenza kudatha engenayo ize ikhanselwe (uma iHadoop isebenzisa umsebenzi weMephuReduce othatha isikhathi esilinganiselwe, lapho iSiphepho sisebenzisa umqondo wokusebenzisa " topologies "ngokuqhubekayo.
Ukwenziwa kwabaphathi kungasatshalaliswa kumaseva amaningi: iSiphepho sifanisa umsebenzi ngokuzenzekelayo nemicu ezindaweni ezihlukene zeqoqo.
Amacala wokusebenzisa amakhulu anganikezwa i-Apache Storm
Ukuqhubekiswa kwemifudlana emisha yedatha noma izibuyekezo zedatha ngesikhathi sangempela
Izibalo eziqhubekayo: Isiphepho singenza izicelo eziqhubekayo futhi sicubungule ukuhamba okuqhubekayo, idlulisela imiphumela yokucubungula kwiklayenti ngesikhathi sangempela.
Kusatshalaliswa inqubo yenqubo yesilawuli kude (RPC): Isiphepho singasetshenziswa ukuhlinzeka imali ehambisanayo ekwenzeni imibuzo edinga izinsizakusebenza.
Umsebenzi ("topology") eSiphepho umsebenzi osatshalaliswayo phakathi kwama-node olinde imilayezo engenayo ukuthi icutshungulwe.
Ngemuva kokuthola umlayezo, umsebenzi uwucubungula kumongo wendawo bese ubuyisa umphumela. Isibonelo sokusebenzisa i-RPC esatshalalisiwe kungaba ukucubungula okufanayo kwemibuzo yosesho noma ukwenza imisebenzi kusethi enkulu yamasethi.
Izici Ezintsha Eziyinhloko ze-Apache Storm 2.0
Isisekelo se-Apache wethule izinhlelo zokudlulisa iSiphepho ku-kernel entsha ebhalwe eJava, imiphumela yayo ehlongozwayo kunguqulo ye-Apache Storm 2.0.
Zonke izingxenye eziyisisekelo zepulatifomu zibhalwa kabusha eJava. Ukusekelwa kokubhala abaphathi eClojure kuyagcinwa, kepha manje sekuhlinzekwa ngendlela yokuxhumana. I-Java 8 iyadingeka ukuze iSporm 2.0.0 sisebenze.
Imodeli yokucubungula enamakhompiyutha yenziwe kabusha ngokuphelele, okuholele ekwandeni kokusebenza okuphawulekayo (kokunye ubungcweti, ama-latency ancishiswe ngama-50-80%).
Enguqulweni entsha Kuphakanyiswe i-API entsha ye-Streams API, ekuvumela ukuthi umise abaphathi usebenzisa imisebenzi ngesitayela sokusebenza esisebenzayo.
I-API entsha isetshenziswa ngesisekelo se-API ejwayelekile futhi isekela ukuhlangana okuzenzakalelayo kwemisebenzi yokwandisa ukucubungula kwayo. I-Window API yokusebenza kwewindi ingeza ukusekelwa kokonga nokubuyisa isimo ngemuva.
Ngakolunye uhlangothi isilawuli ukuqala izinsizakusebenza ezingeziwe lapho kwenziwa izinqumo ezingagcini ku-CPU nememori, njengamapharamitha wenethiwekhi ne-GPU, ingezwe kusihleli se-boot.
Izithuthukisi eziningi ezihlobene nokuqinisekisa ukuhlanganiswa neplatifomu yeKafka.
Uhlelo lokulawulwa kokufinyelela selukhulisiwe, lapho kuvela khona ithuba lokwakha amaqembu abaphathi kanye nethimba lamathokheni.
Kungezwe ukuthuthuka okuhlobene nokusekelwa kwe-SQL namamethrikhi. Isixhumi esibonakalayo somlawuli sinemiyalo emisha yokulungisa iphutha lesimo seqoqo.