I-Apache Pinot, indawo yokugcina idatha ye-OLAP yomthombo ovulekile

I-Apache Pinot iyisisombululo sesitoreji se-OLAP esisatshalaliswa ngesikhathi sangempela esenzelwe ukuletha ukuhlaziywa kwesikhathi sangempela okungalinganiswa, kokubambezeleka okuphansi.

Ingangenisa idatha evela emithonjeni yedatha yeqembu (njenge-HDFS, i-S3, i-Azure Data Lake, i-Google Cloud Storage) kanye nemithombo yokusakaza (njenge-Kafka). I-Pinot yakhelwe ukukala ngokuvundlile, ukuze ikwazi ukukala ibe amasethi edatha amakhulu kanye namazinga aphezulu emibuzo njengoba kudingeka.

Mayelana ne-Apache Pinot

Iphrojekthi yePinot yasungulwa ekuqaleni yi-LinkedIn kwathi ngo-2015 yadluliselwa kwi-Apache Foundation ukuze ithuthukiswe ngokubambisana. Isitoreji senzelwe ukusebenza endaweni lapho idatha entsha ifakwa khona njalo futhi sakhiwe ukuze sinikeze ukubambezeleka okuncane nokubikezelwayo, okuvumela ukusetshenziswa kwayo ekucutshungulweni kwemibuzo ngesikhathi sangempela.

Njengezinye izindawo zokugcina idatha eziningi kanye nezixazululo zokugcina idatha ze-OLAP, i-Pinot isekela ulimi lombuzo olufana ne-SQL olusekela ukukhetha, ukuhlanganisa, ukuhlunga, ukuqoqa, ukuhlunga, kanye neminye imibuzo yedatha.

I-Apache Pinot inikeza ukukhuliswa okuvundlile kanye nendlela yokufeza ukubekezelela amaphutha kanye nokusinda ekuhlulekeni kwesofthiwe nehadiwe. Izinqubo zokuphindaphinda kanye nokusekela zihlanganiswe ngqo emjikelezweni wokucubungula idatha yedatha engezwe esitolo sedatha. Ngakolunye uhlangothi, le ndlela yenza kube lula kakhulu ukwakheka, kodwa ngakolunye uhlangothi, ifaka ukubambezeleka phakathi kokufakwa kwedatha kanye nokutholakala kwayo ukuze kuphenywe.

Idatha igcinwa kumathebula ngaphakathi kwesizindalwazi esiqondiswe kukholomu, futhi izinhlelo ezahlukene zokucindezela kanye nekhono lokubeka amanani amaningi ensimini ziyasekelwa. I-Pinot inikeza uhlelo lokukhomba oluxhumekayo olungasebenzisa ubuchwepheshe obuhlukahlukene bokukhomba (inkomba e-odawe, inkomba ye-bitmap, inkomba eguquliwe, inkomba ye-StarTree, isihlungi se-Bloom, inkomba yebanga, inkomba yokusesha umbhalo (i-Lucence/FST), inkomba ye-JSON, inkomba ye-geospatial).

Ezicini ezivelele ku-Apache Pinot:

• Ikholamu iqondiswe- Isizindalwazi esigxile kukholomu enamasu ahlukene okucindezela afana nobude bokugijima kanye nobude bebhithi obumisiwe.
• Inkomba exhumekayo: Ubuchwepheshe bezinkomba obuxhumekayo, Inkomba Ehleliwe, Inkomba ye-Bitmap, Inkomba Ehlanekezelwe.
• Ukusebenza kombuzo- Amandla okuthuthukisa umbuzo/uhlelo lokuqalisa ngokusekelwe embuzweni nakumethadatha yengxenye.
• Ukungeniswa kwemifudlana namaqoqo: Eduze kokungenisa kwesikhathi sangempela kokusakaza kwe-Hadoop nokungeniswa kwenqwaba.
• Ukubonisana: Injini yokusebenzisa imibuzo esekelwe ku-SQL.
• Ukucasuka ngesikhathi sokungenisa ngesikhathi sangempela: buyekeza idatha esikalini ngokuvumelana
• Izinkambu yenani eziningi: ukwesekwa kwezinkambu zamanani amaningi, okukuvumela ukuthi ubuze izinkambu njengamanani ahlukaniswe ngokhefana.
• Umdabu wamafu ku-Kubernetes: Ishadi le-Helm lihlinzeka ngokusetshenziswa okuhlanganisiwe okukalwa ngokuvundlile, okubekezelela amaphutha okulula ukukuphatha nge-Kubernetes.

Inguqulo entsha ye-Apache Pinot

Kuhle ukusho ukuthi inguqulo 1.0 ye-Apache Pinot isanda kukhishwa , okufingqa umsebenzi omningi wokuqinisa i-codebase nokucabangela izifiso zomphakathi (kucatshangelwe imibono engaphezu kuka-300).

Ngaphezu kwalokho, kubalulekile ukuqaphela ukuthi i -Multi-Stage Query Engine entsha isifinyelele amandla ayo aphelele , okwenza kube nokusekelwa kokuhlanganiswa kwetafula (ama-JOIN). Injini esetshenziswe ekuqaleni isebenze kahle kakhulu ngemisebenzi elula yokuhlunga nokuhlanganisa, kodwa ukuqinisekisa izikhathi zokwenziwa kwemibuzo ezingabikezelwa, ayizange isekele imisebenzi yokujoyina itafula.

Injini entsha ihlanganisa izigaba eziphakathi zokucubungula imibuzo eyinkimbinkimbi, futhi i-SQL semantics yayo iseduze ne-ANSI SQL . Ngaphezu kwalokho, inguqulo entsha inikeza ukwesekwa kwemvelo kokucubungula idatha ye-JSON, inikeza ukwesekwa kwenani le-"NULL", ihlangana ne-Apache Spark 3.x, futhi ithuthukisa ukusetshenziswa kwamathebula kumodi ye-Upsert (engeza ukucindezelwa kwesigaba kanye nokunikeza ukwesekwa kwemisebenzi yokususa).

Okokugcina, uma unesithakazelo sokufunda okwengeziwe , kufanele wazi ukuthi ikhodi yephrojekthi ibhalwe nge-Java futhi isatshalaliswa ngaphansi kwelayisensi ye-Apache. Ungathola imininingwane mayelana nenguqulo entsha kusixhumanisi esilandelayo.